Over the past 20 years I have fortunately been involved with numerous federally funded projects through the National Science Foundation (NSF). A Mathematics and Science Partnership project brought me to Arizona State University as a research assistant and I have participated on an NSF grant in some capacity every year since. The foci of these grants include (a) improving the practice of and collaboration between mathematics and science teachers, (b) designing an innovative quantitative reasoning course for pre-service mathematics teachers, (c) exploring ways to support generalization in 6-14 mathematics classrooms, and (d) developing user-friendly data representations to help citizens remain informed about critical issues including their health.
All of the grants were service oriented to some extent, meaning they worked toward making a transformative and positive impact on students, teachers, or citizens. All of the grants were research minded to some extent, meaning they involved an intensive plan for producing foundational discipline knowledge. Grants often have both service and research purposes, and they can range significantly in how much they emphasize one or the other. Basic research grants might focus extensively on research with few (or no) obvious applications, while scale-up grants might focus extensively impacting large populations with only a cursory research component.
To ensure a proposed grant addresses a balance of research and service, NSF requires that all proposals address Intellectual Merit and Broader Impact criteria.
A Principle Investigator (PI)—the project leader on a grant proposal—typically summarizes how their grant addresses each criterium using a concise summary section in their grant proposal. On my CAREER grant, which is an award given to pre-tenure faculty, these looked something like:
Short narratives fail to capture the complex and holistic work of a proposed project. This is by design. The purpose is to summarize to the reviewer, program offices, and public how the proposed project is worthy of spending federal funds. They are essentially a written abstract for the worthiness of a project.
Worthiness…for the last 20-years, I have been both honored and fortunate to work a job dedicated to the betterment of individuals’ lives and society. I have also been lucky to have a job where such work can be done without spotlight or typical industry demands. Each day I wake up and decide what questions are worth pursuing. Each day I wake up and decide how to pursue those worthy questions. This feature of the academy enables work to unfold in ways that are nimble, flexible, need-responsive, risky, and, most importantly, independent of external influences. It is a job where I can put my head down and do work without a boss’s, politician’s, or media’s hand exerting pressure or direction.
In the past few weeks, the federal support of university research—the major public R&D arm of the United State of America—has been inserted front-and-center into media coverage and politician speak. To some individuals, it has been deemed no longer worthy of funding. It has been deemed a political initiative of “wokeness”. It has been deemed inefficient beyond saving. It has been deemed a fixed cost worth cutting.
I am not motivated to write this post as a direct response to these points. Plenty of individuals are doing this better than I ever could, particularly in identifying the resource capital—monies/GDP, intellectual property, labor expertise, labor recruitment, education, etc.—generated by the academy as a public good. I am also sympathetic to the efficiencies argument, as the academy (as with any large corporation) undoubtedly has bloat. More on that later.
I am motivated to write this post to provide a few insights into how the federal funding process works for NSF grants. I believe knowledge dissemination is an important action. I am surprised at how few individuals have reached out to ask how the funding process works. My belief and surprise collectively fuel this post.
Each of the three points below are written from my point-of-view as an NSF proposer, reviewer, and PI. I also write them based on my relationship with several NSF Program Officers (POs) who work tirelessly to ensure NSF receives, funds, and monitors excellent proposals. These points do not paint an exhaustive picture, nor do they paint a nuanced picture. I hope they paint a picture that helps people have a baseline understanding of some features of the NSF grant process.
1) What is the process?
The process is rather straight-forward. NSF publishes Requests for Proposals (RFPs) that define particular areas of funding. In my world, RFPs are typically large in grain-size. They might specify grade bands (e.g., K-12 or undergraduate), disciplines (e.g., STEM or programming), or structure (e.g., basic research or institutional partnerships), to name a few. Within an RFP, there can be further levels of specificity. These are often tied to funding size. For instance, within an RFP, a basic research project might be set at a different maximum grant amount and timeframe than a project geared toward scaling up a basic research project to a broader societal audience.
As a researcher, we monitor RFPs for those that loosely or directly align with our work. With an RFP identified, we write a proposal and submit it to NSF by the RFP deadline. Proposal writing is an intensive process, the details of which I’ll leave for another post. Each proposed submission is then assigned to a review panel with a subset of other proposals for that RFP. The review panel spends several days reviewing the proposals assigned to their panel. The panel gives each proposal a rating and loosely situates it with respect to the other proposals reviewed by that panel. The number of review panels needed for any particular RFP depends on the number of submissions to that RFP.
With each proposal rated, the PO overseeing the review panel’s work takes any proposal above a particular rating forward to discuss potential awardees with other POs. At the panel level, the PO does not influence the reviewers’ assessment. But, at the stage of moving proposals forward out of panel, the POs collectively decide which proposals are worthy of funding. Their decision is based on their collective expertises, their read of the proposals, and the review panels’ assessments. The POs can, and often do, reach out to the proposers for an additional round of questions and clarifications to ensure a grant is worthy of funding. This additional round of interaction with proposers can result in modified budgets and updated work plans in order to ensure the grant will be successful and within appropriate resource limits. In the end, the POs decide which proposals to push forward for funding.
2) Who is involved?
Somewhat obvious, the proposer (or project PI) is typically university faculty or units. It is less obviously who the reviewers and POs are.
They are us.
An intentional feature of NSF is that the reviewers and POs are discipline experts. One reason for this is to maintain the independence of the agency. As an independent agency, NSF’s funding decisions are meant to be kept separate from direct political influence. Not only are politicians not to make funding decisions, they are to be kept from influencing funding decisions.
With respect to review panels, POs work hard to identify, solicit, and organize a group of discipline experts. They primarily draw from the faculty pool around the nation. As a faculty member, you receive an email asking for your participation on a review panel. If you accept, you are given your assigned proposals and scheduled dates for the review panel. Participation is done on a volunteer basis, but you are financially compensated for your time as a reviewer. If a diligent reviewer, this amount probably equates to $20-50 per hour; we choose to review as an important service to the field and not as a way to earn money. Each review panel ultimately consists of (in my experience) around 8-12 individuals with expertise related to the RFP and proposals.
With respect to POs, they are part faculty “on loan” from their university positions and part former faculty that have become permanent POs. For several different reasons, a faculty member might desire taking a turn at being a PO. The role of PO is typically a 1-3 year temporary appointment, although my understanding is that it can move into a more formal advising or administrative position at NSF if desired. My sense is this is rare, both because a majority of faculty enjoy their faculty positions and because there are not many positions available at NSF. When a PO finishes their temporary position, they return to their university faculty position. Compensation is in line with their faculty position (ignoring the potential increased cost of living associated with the DC area). Much like being a reviewer, we choose to be a PO as an important service to the field.
3) What happens after a proposal is approved for funding?
You, the PI and project team, get to work. The team begins the project plan, and they do their best to respond to ongoing, unpredictable project needs and directions while remaining faithful to the original proposal and promised work.
A grant is an award, not a contract.
The goal is to embrace the cavalier and independent nature of the faculty job. The project team, led by the PI (or co-PIs), execute the grant in the way the PI sees fit. In basic research, it is often unclear how the project will unfold until the project is unfolding. In scale -up research, it is often unclear what issues will inhibit scale efforts until they are experienced. Regardless, the PI relies on their own expertise as well as that of the project team to carry out the work in a way faithful to NSF and US taxpayers.
Some monitoring does occur over the course of a funded project. A PO is assigned to each project. The PO reviews a submitted annual report each year. The annual report details all project activities, spending, and decisions of the prior year. This includes any delays, modifications, or new directions that occurred. The PO interacts with the PI and project team during this time period to ensure they are appropriately using grant funds and that any departures are justified and viable. A PO can make site visits in order to get an on-the-ground picture of project activities and expenditures.
Another form of monitoring is through an Advisory Board or Evaluation Team (or both). The Advisory Board is a collection of experts (typically 3-6 individuals) in areas related to the grant. The PI and project team is able to rely on the board for mentorship and external critique throughout their project. At times, the project team will convene the entire board for a multi-day meeting during which the project team presents their progress, outcomes, decisions, and anything else they would like the Advisory Board to consider. The Advisory Board offers challenges, advice, considerations, etc. to the project team. Similarly, grant proposals include an evaluation component. This is typically in the form of some team of evaluation experts that provides a formal assessment of grant activities related to the proposed work and goals. As captured by their names, an Advisory Board’s purpose is to provide critical advice and guidance and an Evaluation Team’s purpose is to evaluate and assess the project relative to its goals.
Beyond the Advisory Board, Evaluation Team, PO, and any informal colleague mentorship that occurs, no other formal monitoring or advisement typically occurs with a grant. Also, and this is critical, none of the aforementioned parties can tell or force a project team into doing something. The work and decision making is always left up to the project team lead (the PI and co-Pis). A PO can choose to not approve an annual report, thus delaying the release of the subsequent year’s funding, if a project team goes too rogue. But that is rare, as a PI is typically trusted in their expertise. To date I have never been told what to research or how to research it. I’ve been mentored in wonderful ways by my doctoral committee, major advisor, advisory boards, evaluation teams, POs, and colleagues. Most critically, such mentorship always left decisions in my hands.
Worthiness…and making decisions. I hope the above points make clear that the grant process has been designed to leave the decision making process to discipline experts. The grant process is built on the premise that those worthy of making funding decisions are those worthy of carrying out the work. Unfortunately, recent political actions have chipped away at the independence of funding agencies. Recent political actions have conveyed that we are no longer worthy of making funding decisions including the direction of research within our own field. I consider this rather unfortunate, especially when we consider the potential influence of private or political interests. These interests are now positioned to determine the research that is done (or not done) for public good and consumption, which will be undoubtedly influenced by their political, corporate, and private desires.
A quick aside on efficiencies: As the public R&D arm of the USA, so-called inefficiencies are necessary to take chances and risk in research. For every risky project that turned out innovative and transformative, there are numerous failed projects that did not meet their aspirations. This is a feature, not a bug of research. Much like venture capitalism, success and efficiency is not determined by the gambles that do not provide return. It is measured by the ceiling of the few successful investments in comparison to those that fail. By that metric, the USA public R&D arm in the form of university research is a net positive.