General Discussions

 View Only
Expand all | Collapse all

Metrics and Cayuse

  • 1.  Metrics and Cayuse

    Posted 06-16-2021 15:03
    Is anyone using Cayuse to determine success rate of your institution's proposals?  What metrics are you collecting?  We do number of proposals, number of awards, by department, by faculty member and come up with some basic pie charts or whatever.  But I'm wanting to look for more ways to evaluate what we do.  Is anyone using Cayuse for that?  Are there other ways you're evaluating your office?

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------


  • 2.  RE: Metrics and Cayuse

    Posted 06-18-2021 10:11
    Hi Audrey,
    At Notre Dame, we use the proposal information from Cayuse to determine success rates. The formula we use is:
    Proposals Funded divided by the sum of Proposals Funded and Proposals Not Funded

    In other words, the data set includes only those proposals on which we've received a decision from the sponsor. Anything pending is not included.  This means our success rates will always run 12-18 months behind due to the time it takes sponsors to make a decision. Our leadership can live with that.

    For success by fiscal year, we must wait until a decision is received on ALL proposals submitted... or if the data shows sponsor decisions through a certain month then we sometimes go that route being sure to note on the report that these success rates are only through the month of X in Fiscal Year A.

    We do not use award data because many of them are multi-year, and the funding is dispersed over time, some obligated, some anticipated. It can get complicated quickly. We decided instead to keep it simple and not mess with dollars. We use proposals and their status.

    This method will also allow you to calculate success rates by dept, by faculty, by sponsor "in pie charts or whatever." As an aside, while I generally hate pie charts, they will work in this case to easily show a split between funded or not funded.  :-)
    Hope this helps,
    Terri

    ------------------------------
    Terri Hall
    Director, Research Business Intelligence
    University of Notre Dame
    ------------------------------



  • 3.  RE: Metrics and Cayuse

    RISING STAR CONTRIBUTOR
    Posted 06-18-2021 12:05
    I'm glad to hear what others are doing to evaluate proposal success.  I like how you are approaching it Terri and agree that there needs to be understanding that there is  leeway in the "current" status of proposals.  We would like to start tracking more in depth information as well such as which institutes at NIH we have the most success and to which ones we are not applying.  Does anyone capture that detailed information to measure as well?

    ------------------------------
    Jennifer Mills
    Senior Grants and Contract Specialist/ERA Administrator
    University of Alabama - Tuscaloosa
    ------------------------------



  • 4.  RE: Metrics and Cayuse

    RISING STAR CONTRIBUTOR
    Posted 06-21-2021 08:53
    Hi Jennifer,

    We recently reset our Proposal Instrument Types and Subtypes to capture the various funding mechanisms on federal grants (NIH: R01, R03, ... NSF: RUI, HSI, MRI, ... etc.) so that we can easily identify these proposals and track our success rates. Happy to share more details with anyone who may be interested in doing the same.

    ------------------------------
    Sherrie Hixon
    Director, Strategic Research Initiatives & Innovation
    California State University, Northridge (CSUN)
    ------------------------------



  • 5.  RE: Metrics and Cayuse

    Posted 06-21-2021 09:12
    For proposals, I like your idea, Sherrie.

    For NIH awards, we often use the sponsor award number in Cayuse to determing funding mechanism and institute.

    For example... 1R01CA255064-05
      The 1 tells me it's a new award
      R01 tells me the program
      CA signifies the National Cancer Institute, etc.

    Here is a great explanation from NIH on deciphering their award numbers:
    https://era.nih.gov/files/Deciphering_NIH_Application.pdf

    ------------------------------
    Terri Hall
    Director, Research Business Intelligence
    University of Notre Dame
    ------------------------------



  • 6.  RE: Metrics and Cayuse

    Posted 06-21-2021 12:28
    Terri, this is very helpful!  Thank you.

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 7.  RE: Metrics and Cayuse

    Posted 06-21-2021 12:27
    Thank you for your input!  Wow...this is something to consider.

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 8.  RE: Metrics and Cayuse

    Posted 06-21-2021 12:26
    Thanks, Jennifer!  I'd be interested in knowing if folks are capturing this type of info as well and if they're using Cayuse to do it.

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 9.  RE: Metrics and Cayuse

    Posted 06-21-2021 12:25
    Thank you so much for this feedback.  I appreciate it!

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 10.  RE: Metrics and Cayuse

    STAR CONTRIBUTOR
    Posted 06-18-2021 11:52
    Audrey,

    We are not currently using Cayuse for this purpose,  but I do like what Terri is doing with it at ND.

    We are a smaller volume institution (PUI), so we could probably also calculate almost as quickly by hand, but I have not had anyone ask about success rates since I have been here.  I also don't think they would be of much value in our instance, since so many of our awards are contracts that we didn't submit an actual proposal for, so our numbers would be somewhat skewed.  It would look we have great success rates, but the reality is that we are "submitting" for things we already know are being funded.

    ------------------------------
    Pamela Vargas
    Director, Research & Grant Development
    Southeast Missouri State University
    ------------------------------



  • 11.  RE: Metrics and Cayuse

    Posted 06-21-2021 12:32
    Pamela, thanks for your perspective.  Much appreciated!

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 12.  RE: Metrics and Cayuse

    RISING STAR CONTRIBUTOR
    Posted 06-18-2021 12:00
    Here at UC Irvine, we currently do not use Cayuse for that analysis. The data from Cayuse is brought into our Data Warehouse where we run reports manually. There is discussion of using Tableau in the future to help with visualizations and reporting that would be outward facing to the campus.

    ------------------------------
    Jonathan Lew
    ERA Officer
    University of California, Irvine
    ------------------------------



  • 13.  RE: Metrics and Cayuse

    SUPERSTAR CONTRIBUTOR
    Posted 06-21-2021 10:44
    I use Cayuse for all kinds of metrics.  In another life I was a quantitative researcher.  We like to count things, and make it look like we worked hard ;)  Cost transfers, time to fund #/start of spending, all kinds of things.

    But to your actual question, yes.  That's the first place I look for hit rates.  We have flags in SP for "first proposal and "first award" among other things.  That helps us track faculty newly-engaged in external funding.  I keep a little eye on the # of failed proposals/PI, in a row.  I don't do anything formal with that information - certainly I don't report it out.  But the Research Development Officers and I might talk about it and make sure a person's not headed for fundamental discouragement.  At least one person struggles on the other side, and submits too MANY proposals in a year - but we haven't figured out how to get her to slow down and pay meaningful attention to many fewer proposals.  We track percentage of research-active faculty/department (measured by grant awards, which is a flawed metric but it's easy to count).  This allows us to get past the "Physics always has the most awards and we'll never come close" conversations, with PIs, Chairs, and Deans.

    There's probably more, but that's all I can think of right now.

    A


    ​​

    ------------------------------
    Andrea Buford
    Director, Office of Sponsored Programs
    Oakland University
    ------------------------------



  • 14.  RE: Metrics and Cayuse

    Posted 06-21-2021 13:46
    Andrea, thanks so much!  I'll definitely be turning to Cayuse to help us take a closer look.

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 15.  RE: Metrics and Cayuse

    Posted 06-21-2021 12:36
    Jonathan, thanks for your response.  Is that the name of the platform, Data Warehouse, or is that a generic term?  If it's generic do you mind sharing the name of the platform since it's compatible with Cayuse?

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------



  • 16.  RE: Metrics and Cayuse

    RISING STAR CONTRIBUTOR
    Posted 06-21-2021 13:04
    @Audrey Wineglass Foster Pretty sure it's a generic term... go for it! ​

    ------------------------------
    Jonathan Lew
    ERA Officer
    University of California, Irvine
    ------------------------------



  • 17.  RE: Metrics and Cayuse

    Posted 06-23-2021 10:47
    Hi Audry,

    At American University we have reported on success rate based on the funded status but we have taken a break as we had many pending proposals awaiting a funding decision. This turns out partly because, PIs do not always inform us if a proposal decision has been provided, in particular, if it is not funded. I do not like to admit it but we do have a few PIs who tries to avoid routing proposals and only route proposals which require an institutional endorsement/submission or already funded, thus the success rate we reported was screwed.



    ------------------------------
    Vibeke Svensson
    Director, Office of Sponsored Programs
    American University
    ------------------------------



  • 18.  RE: Metrics and Cayuse

    SUPERSTAR CONTRIBUTOR
    Posted 06-23-2021 10:52
    NO!   Faculty members who try to work around Sponsored Programs????  Well, I never.  *Andrea fans herself, in shock*

    Kidding.  Sigh.  I totally commiserate.

    A

    ------------------------------
    Andrea Buford
    Director, Office of Sponsored Programs
    Oakland University
    ------------------------------



  • 19.  RE: Metrics and Cayuse

    Posted 06-23-2021 11:17
    Vibeke, thanks for your input!  I've never experienced such a thing...imagine going around the sponsored programs office!  It happens but we do try to meet with faculty periodically to remind them that we're here to facilitate and "to keep them out of orange jumpsuits and off the front page of the Washington Post!"  You make a very good point about that delay and, yes, particularly if it's not funded.  Thanks again!

    ------------------------------
    Audrey Wineglass Foster
    Assistant Dean for OSP + Research Svcs
    Gallaudet University
    ------------------------------