I recently had the dubious distinction to participate in and serve on an National Institutes of Health grant review called and NIH study section. I’ve done this before and have reviewed lots of grants over the years, but there have been a lot of changes recently and it is a whole new experience that is still very hard to swallow.
For those of you not familiar with the NIH peer review system, don’t tune out. Let me explain it as only a truly entrenched academic geek such as myself can.
First let me say that the US government’s NIH granting and reviewing system strives to be fair and transparent. But human peculiarities do indeed come into play as do politics and a little bit of luck. Did I mention semantics, bureaucracy, administration, economy, public perception and salesmanship are involved too? They are.
With the Rise of AI, What IP Disputes in Healthcare Are Likely to Emerge?
Munck Wilson Mandala Partner Greg Howison shared his perspective on some of the legal ramifications around AI, IP, connected devices and the data they generate, in response to emailed questions.
Let me begin with some facts. The NIH peer review is a system where experts in a given field read and review grant proposals that have been written and submitted by experts in the field.
A grant proposal typically has a research section, budget, resources and personnel sections. The length of the sections varies according to the type of grant proposal, but can be 100 pages long.
The study section can review large numbers of grants sometimes over 100. my study section reviewed about 25 in one day.
Each grant is assigned to 3 or 4 reviewers. I received 6 grants to review or about ¼ the total. The reviewers are expected to read these grants thoroughly cover to cover. I read mine 2 or 3 times. Prior to the meeting, 1 week, all grants are given numerical scores by the referees.
A Personalized Approach to Medication Nonadherence
At the Abarca Forward conference earlier this year, George Van Antwerp, managing director at Deloitte, discussed how social determinants of health and a personalized member experience can improve medication adherence and health outcomes.
The reviewers are expected to also read all the grants in a study section. Let me tell you this takes an incredible amount of time and makes for one bleary eyed group of referees at these study sections. It is a lot of work and everyone takes it very seriously.
More facts about the review. There is a chair for the meeting and a couple of support personnel. The lead support person, called the grants management specialist (GMS) is a highly trained scientist and administrator. The chair is a respected and experienced scientist who defers to the GMS concerning procedures and admin issues. The chair also has substitute chairs in the case of there being a conflict or if the grant the chair reviewed is being discussed. Conflicts are most often a grant comes from that person’s institution. So I am not allowed to review grants coming from Cincinnati.
I apologize for this reading like a text book. Lets get to some interesting human dynamics. The primary referee as labeled by the GMS summarizes the science of the grant and gives an impression of the strengths and weaknesses. The other referees then add or reinforce details. These are supposed to be objective and dispassionate bits of information. It is so easy, however, to tell if a person is pushing for or against a grant. Plus the referees are quick to try to convince the others how smart they are. I had to suppress a chuckle on multiple occasions because of inane posturing by people.
Here are some excerpts I thought I would share. Note, I am redacting details to protect the people as well as the grants being discussed.
“I am likely the only person in the room capable of reviewing this grant.”
“In my study section we do it {this way}.” With the implication being others are wrong.
“it is a fatal error to call a neonatal rodent a model for pre-natal humans. These are my patients and they are not rodents.” This was silly because an MD seemed to want to dismiss all animal models here.
“It is simply wrong for them to call these {xyx} cells.” The response: “The applicant calls them {xyx} – like cells.”
“in my lab we do these experiments and we do it this way.” Again an example of different is wrong.
“This would be a great application if they studied {ABC}.” {ABC} is what the referee studies. The implication is that they do important research and the application does not.
Here is a quote from my notebook. “Shouting and karate chopping the desk does not make you right.”
What gave me tremendous concern however is the catch 22 that exists. There needs to be innovation in a grant but innovation often means little evidence from others. If you do new stuff, that by definition means others have not done it. Without evidence from others there is concern re performance ability and that leads to difficulty funding. So why be innovative? Do what others have done; others tend to be the experts making decisions anyway.
On another review that was highly innovative and the type of study that would make the national news if it succeeded, a particular person, who tended to karate chop to make a point, complained that the research group did not control for environmental variables. I had read the study as did the person making the complaint and I said that I thought they controlled for those variables by using geographic restrictions. The reviewer obviously missed that point and instead of admitting that this is a control mechanism, this colleague dismissed my point and the grant by saying that is not how everyone else, their lab, does it.
That is just sad.
This grant was novel, innovative and would have a huge impact. But because a big ego, attached to a loud voice, dismissed it due to their own missing facts the grant sank in the scoring and may never be done.
On that sad note, I’ll end with one more fact. There were about 25 grants reviewed that day and I think 7 or 8 should have been funded, but there were only sufficient funds to fund 3.
Dr. Joseph Clark is a professor of neurology at the University of Cincinnati and the author of My Ambulance Education. He writes regularly at Josephfclark.com.
