General Discussions

 View Only
  • 1.  Success rates

    SUPERSTAR CONTRIBUTOR
    Posted 06-02-2022 16:22
    Hi intrepid ones!

    I was wondering how your institution determines proposal success rates?  We happen to be mainly NIH-centric, and NIH has their own metric (which basically counts an original submission and a resubmission as a single entity (whether awarded or not), but usually we prefer to track it one-one (per submission, so yes, that unapproved extra submission may "bring the rate down", but often they'll fall in separate fiscal years so it's more reflective of our actual effort on proposals as well as PI productivity.  And over time, if you use the same metric year-over-year, you still get a feel for trends.  Thoughts?

    Related:  NIH is notoriously slow in providing "final" results on proposals (we just received final Decision Not to Fund on applications submitted in Q4 of calendar 2019!).  How much time do you allow before you mark an application as Not Approved due to lack of agency feedback (obviously "not discussed" is an immediate NA)?  Do you rank the potential viability of pending-yet-reviewed proposals based on score or percentile (what's your crystal ball)?

    Best,

    ------------------------------
    Lisa Churchill
    Director, Grants Administration
    The Salk Institute for Biological Studies
    ------------------------------


  • 2.  RE: Success rates

    Posted 06-03-2022 04:47
    Hi Lisa,
    Thanks for sharing the NIH data nuance.
    I wasn't aware of that.

    Now to success rate formula...
    We use the NIH formula, but with Cayuse data:
    Funded / (Funded + Not Funded)

    The data used is proposals...and only those that
    have a sponsor decision - either Funded or Not Funded. 
    This means the rate is always a year behind or so.

    For NIH proposals: we determine which NIH units we are 
    most successful with by using the sponsor award number 
    which has the acronym for the NIH unit within it.

    I'm also interested in what others do.

    Hope this helps,

    Terri

     

    The only way to make sense out of change

    is to plunge into it, move with it,

    and join the dance.  ~ Alan Watts

    Terri Hall

    Director, Research Business Intelligence

    Notre Dame Research Administration

    University of Notre Dame

    thall2@nd.edu

    574.631.7378






  • 3.  RE: Success rates

    SUPERSTAR CONTRIBUTOR
    Posted 06-06-2022 13:27
    FWIW, I want a much more robust definition of success - but I don't have one to offer, so that's not much help.  We do track more than hit-rate, though.  But you asked about PROPOSAL success, and we do pretty much what you do.

    To answer your question about assuming a proposal won't be funded after a certain amount of time - we have no fixed rule.  DOE is notorious for taking multiple years to fund something.  NIH and NSF are easier to track.  If NIH Council Review has past and we don't have any indication that there will be an award, I may not wait until the official notice comes to flip the status - because you're right, they take forever.

    Our other metrics of success include things like number of faculty submitting their first proposal or receiving their first award (using Cayuse flags).  We are looking to grow the percentage of research-active, funded faculty - so first award, first proposal is one aspect of that.  We track as one of our performance indicators the number of people serving as either PI or Co-PI, using an unduplicated count.  To get this number, we use the Award side of Cayuse SP to give us that info.  We also track the number of graduate students involved on grants.  When I started here, that number was surprisingly low and having active labs with grad students enhances the climate of research, so we watch for growth in that number - but I can't remember how we get at it.  That one has to come from Fund Manager, or that's the only way I know how to do it anyway.

    ------------------------------
    Andrea Buford
    Director, Office of Sponsored Programs
    Oakland University
    ------------------------------



  • 4.  RE: Success rates

    STAR CONTRIBUTOR
    Posted 06-14-2022 11:49
    Hi Lisa!
    We've tried to nail this one down for a while and it's just not clear that whatever formula we use is actually giving us a good understanding of activity. As we all well know there is nuance! A contract with a local sponsor to do a community engagement project is a different beast than an NSF proposal, and doesn't go through the same level of development (usually). So far no one is asking us to produce this metric, being more interested in level of activity (# of proposals & awards) and expenditures.

    My 'general rule' for marking things not funded is 18 months. NIH records are a little different because of that mega delay they have in actually notifying applicants, but that's a small number of records annually that are fine to leave outstanding for a bit longer. I don't know if you have this experience at Salk, but our PIs will get a notification and then never tell us, so I try to do an annual cleanup of our 'Submitted to Sponsor' bucket - very easy to ID what's pending thanks to Cayuse.

    ------------------------------
    Augusta Isley
    Senior Proposal Manager
    Ball State University
    amwray@bsu.edu
    ------------------------------



  • 5.  RE: Success rates

    SUPERSTAR CONTRIBUTOR
    Posted 06-15-2022 08:32
    Thanks, Augusta!

    You speak the truth of our lives, as usual.

    Perhaps it’s because we’re a smaller place (and/or our Executive Team wants to stay on top of what’s “really” pending and what’s not), we generally use a six-month timeframe. If we haven’t heard (by way of preparing JIT for another proposal, for instance, and get, “oh, THAT old thing? That was denied ages ago!”) within six months of start, or if/when we find out that a proposal was “not discussed”, we mark it as unfunded in SP—which automatically sends an email to the Lead PI informing them that either we were told it was not funded or six months since the requested start date has elapsed and we assumed it so. It also asks them to let us know if this was done in error, of course!

    Best,
    -Lisa