1 00:00:06,160 --> 00:00:07,400 >>Welcome, everybody. 2 00:00:07,400 --> 00:00:13,600 This is the March 2022 meeting of the CSR advisory 3 00:00:13,600 --> 00:00:14,200 council. 4 00:00:14,200 --> 00:00:15,400 I'm Bruce Reed. 5 00:00:15,400 --> 00:00:21,080 I'm the deputy director at CSR. 6 00:00:21,080 --> 00:00:24,480 I'll serve as the executive secretary, so I guess 7 00:00:24,480 --> 00:00:27,840 that's the master of ceremonies and timekeeper is what 8 00:00:27,840 --> 00:00:29,000 I'll be doing. 9 00:00:29,000 --> 00:00:32,200 Before I go to introductions, I just want to remind 10 00:00:32,200 --> 00:00:33,840 people of a few things. 11 00:00:33,840 --> 00:00:37,360 So first of all, this is a public meeting. 12 00:00:37,360 --> 00:00:42,680 The only people that you'll see on the screen are 13 00:00:42,680 --> 00:00:46,440 yourself, the council members, plus the presenters, 14 00:00:46,440 --> 00:00:50,160 Noni and myself and then the interpreters. 15 00:00:50,160 --> 00:00:52,880 Since it's a small group, we would certainly like you 16 00:00:52,880 --> 00:00:54,240 to keep your screens on. 17 00:00:54,240 --> 00:00:57,840 I recognize that this goes from 10:00 in the morning 18 00:00:57,840 --> 00:01:01,080 until 2:00 in the afternoon for some of you. 19 00:01:01,080 --> 00:01:04,400 And we did not give you a decent break. 20 00:01:04,400 --> 00:01:09,080 So if you've got to turn off the screen to, you know, 21 00:01:09,080 --> 00:01:13,720 catch a snack, a short lunch, feel free to do so, but 22 00:01:13,720 --> 00:01:16,440 then come back and join us visually. 23 00:01:16,440 --> 00:01:18,120 The chat is shut off. 24 00:01:18,120 --> 00:01:21,040 The Q&A is shut off. 25 00:01:21,040 --> 00:01:24,520 So as I said the other day, just raise your physical 26 00:01:24,520 --> 00:01:27,400 hand, raise your zoom hand or just speak up. 27 00:01:27,400 --> 00:01:28,880 It's a small group. 28 00:01:28,880 --> 00:01:32,600 So don't be shy about just interrupting during the 29 00:01:32,600 --> 00:01:36,720 presentation if you'd like to -- if you have a question 30 00:01:36,720 --> 00:01:37,760 or comment. 31 00:01:37,760 --> 00:01:38,160 Okay. 32 00:01:38,160 --> 00:01:42,680 So with that, let's go ahead and let's go through the 33 00:01:42,680 --> 00:01:46,400 introductions. I will -- I'll just walk you through 34 00:01:46,400 --> 00:01:46,960 this. 35 00:01:46,960 --> 00:01:49,840 So I'll start off with a name. 36 00:01:49,840 --> 00:01:53,720 If you could tell us, just introduce yourself, where 37 00:01:53,720 --> 00:01:57,160 you're from, a couple words about your science. 38 00:01:57,160 --> 00:02:02,920 So Dr. Cabassa. 39 00:02:02,920 --> 00:02:06,520 >> DR. LEOPOLDO CABASSA: Hello, everyone. I am a 40 00:02:06,520 --> 00:02:09,440 professor at the George Warren Brown School of Social 41 00:02:09,440 --> 00:02:09,760 Work. 42 00:02:09,760 --> 00:02:13,680 My areas of expertise are health disparities research 43 00:02:13,680 --> 00:02:15,160 and implementation science. 44 00:02:15,160 --> 00:02:17,320 Glad to be here today. 45 00:02:17,320 --> 00:02:18,480 >> DR. BRUCE REED: Thanks. 46 00:02:18,480 --> 00:02:20,200 Dr. Carpenter . 47 00:02:20,200 --> 00:02:22,360 >> DR. MATTHEW CARPENTER: Hello, everyone. 48 00:02:22,360 --> 00:02:27,200 Matt carpenter with the medical University of South 49 00:02:27,200 --> 00:02:32,040 Carolina, my personal area of research is tobacco 50 00:02:32,040 --> 00:02:36,280 control, smoking cessation, clinical trials and the 51 00:02:36,280 --> 00:02:36,880 like. 52 00:02:36,880 --> 00:02:41,680 >> DR. BRUCE REED: Dr. Gao >> DR. JINMING GAO: Good 53 00:02:41,680 --> 00:02:42,400 afternoon, everybody. 54 00:02:42,400 --> 00:02:47,640 This is Jim Ming Gao. 55 00:02:47,640 --> 00:02:51,800 I'm a professor of pharmacology and oncology. 56 00:02:51,800 --> 00:03:01,680 My research area is in cancer center medicine and nano 57 00:03:01,680 --> 00:03:03,680 immuno oncology. 58 00:03:03,680 --> 00:03:08,880 >> DR. CHRISTINE HENDON: My name is Christine Hendon. 59 00:03:08,880 --> 00:03:15,160 My research is in optical tenography with applications 60 00:03:15,160 --> 00:03:17,520 in surgical guidance. 61 00:03:17,520 --> 00:03:19,000 >> DR. BRUCE REED: Thanks. 62 00:03:19,000 --> 00:03:23,160 Dr. Janelsins. 63 00:03:23,160 --> 00:03:24,400 >> DR. MICHELLE JANELSINS: Hi, everybody. 64 00:03:24,400 --> 00:03:26,760 My name is Michelle Janelsins . 65 00:03:26,760 --> 00:03:31,000 I'm a tenured associate professor at the university of 66 00:03:31,000 --> 00:03:35,200 Rochester in Rochester, New York, at the department of 67 00:03:35,200 --> 00:03:36,640 surgery and neuroscience. 68 00:03:36,640 --> 00:03:39,320 I'm chief of our division in supportive care in cancer. 69 00:03:39,320 --> 00:03:44,520 I'm a member at Wilmont research institute and my 70 00:03:44,520 --> 00:03:48,600 research focuses on cognitive decline and cancer 71 00:03:48,600 --> 00:03:49,160 survivorship. 72 00:03:49,160 --> 00:03:51,480 >> DR. BRUCE REED: Thanks, Michelle. 73 00:03:51,480 --> 00:03:54,200 Dr. Palermo. 74 00:03:54,200 --> 00:03:58,560 >> DR. TONYA PALERMO: Hi. 75 00:03:58,560 --> 00:04:01,120 I'm Tonya Palermo. 76 00:04:01,120 --> 00:04:04,720 I run a lab focused on a couple of clinical and 77 00:04:04,720 --> 00:04:07,040 behavior research on management and prevention of 78 00:04:07,040 --> 00:04:09,000 chronic pain in children and adolescents. 79 00:04:09,000 --> 00:04:12,600 >> DR. BRUCE REED: Dr. Peifer. 80 00:04:12,600 --> 00:04:13,840 >> DR. MARK PEIFER: Hi there. 81 00:04:13,840 --> 00:04:16,000 I'm mark Peifer. 82 00:04:16,000 --> 00:04:22,280 I'm a professor of biology at UNC Chapel hill. 83 00:04:22,280 --> 00:04:27,480 My lab focuses on the cell biology of cell adhesion, 84 00:04:27,480 --> 00:04:30,080 the cytoskeleton and cell signaling. 85 00:04:30,080 --> 00:04:33,120 >> DR. BRUCE REED: Great. 86 00:04:33,120 --> 00:04:36,480 Dr. Rajaram. 87 00:04:36,480 --> 00:04:38,160 >> DR. NARASIMHAN RAJARAM: I was muted. 88 00:04:38,160 --> 00:04:42,600 My name is Narasimhan Rajaram. 89 00:04:42,600 --> 00:04:47,680 I'm at the University of Arkansas in the Ozark 90 00:04:47,680 --> 00:04:50,520 mountains, social professor of bioengineering. 91 00:04:50,520 --> 00:04:54,040 My lab focuses on methods for looking at treatment 92 00:04:54,040 --> 00:04:55,200 response in cancer. 93 00:04:55,200 --> 00:04:58,720 >> DR. BRUCE REED: And Dr. Villa. 94 00:04:58,720 --> 00:05:03,200 >> DR. ELIZABETH VILLA: My name is Elizabeth Villa. I 95 00:05:03,200 --> 00:05:06,560 am an associate professor of molecular biology at the 96 00:05:06,560 --> 00:05:08,400 university of California San Diego. 97 00:05:08,400 --> 00:05:15,840 My lab focuses on developing technologies for using 98 00:05:15,840 --> 00:05:18,640 cryo-electron microscopy . 99 00:05:18,640 --> 00:05:21,520 >> DR. BRUCE REED: And we have four people joining us 100 00:05:21,520 --> 00:05:23,240 as ad hoc members for this meeting. 101 00:05:23,240 --> 00:05:25,040 Let me have them introduce themselves. 102 00:05:25,040 --> 00:05:26,720 Dr. Anderson. 103 00:05:26,720 --> 00:05:35,520 >> Dr. Karen Anderson: I am a member of the Mayo 104 00:05:35,520 --> 00:05:37,880 comprehensive cancer center. 105 00:05:37,880 --> 00:05:42,720 My research focuses on tumor immunology and molecular 106 00:05:42,720 --> 00:05:43,320 diagnostics. 107 00:05:43,320 --> 00:05:46,000 >> DR. BRUCE REED: Dr. Davidson: 108 00:05:46,000 --> 00:05:49,240 >> Dr. Sean Davidson: From the department of pathology 109 00:05:49,240 --> 00:05:51,400 at the university of since sings. 110 00:05:51,400 --> 00:05:55,120 I study lack of protein metabolism. 111 00:05:55,120 --> 00:05:57,000 >> DR. BRUCE REED: Thank you. 112 00:05:57,000 --> 00:06:00,440 Dr. Spiekerkoetter 113 00:06:00,440 --> 00:06:05,400 >> Dr. SPIEKERKOETTER: I am an associate professor in 114 00:06:05,400 --> 00:06:09,840 medicine in the pulmonary critical care division at 115 00:06:09,840 --> 00:06:10,960 Stanford university. 116 00:06:10,960 --> 00:06:16,600 And my research is basic translational science in 117 00:06:16,600 --> 00:06:20,840 pulmonary vascular disease including malformations and 118 00:06:20,840 --> 00:06:22,960 right heart failure. 119 00:06:22,960 --> 00:06:24,520 >> DR. BRUCE REED: Thanks. 120 00:06:24,520 --> 00:06:27,640 And Dr. Yee. 121 00:06:27,640 --> 00:06:32,880 >> Dr. Lynn yee: I'm an internal medicine in maternal 122 00:06:32,880 --> 00:06:37,040 fetal science and my research focuses on health 123 00:06:37,040 --> 00:06:40,720 services and health equity focused interventions for 124 00:06:40,720 --> 00:06:41,760 pregnant people. 125 00:06:41,760 --> 00:06:47,920 >> DR. BRUCE REED: And by no means least, but -- Noni. 126 00:06:48,560 --> 00:06:52,920 >> DR. NONI BYRNES: Noni Byrnes, director and Center 127 00:06:52,920 --> 00:06:54,200 for Scientific Review. 128 00:06:54,200 --> 00:06:56,480 >> DR. BRUCE REED: Thanks, everybody. 129 00:06:56,480 --> 00:07:02,520 So with that, we will turn to our agenda. 130 00:07:02,520 --> 00:07:13,480 And Noni is up first with update on CSR. 131 00:07:13,480 --> 00:07:14,040 Yep. 132 00:07:14,040 --> 00:07:14,840 See the slides. 133 00:07:14,840 --> 00:07:17,040 >> DR. NONI BYRNES: Excellent. 134 00:07:17,040 --> 00:07:17,600 Good. 135 00:07:17,600 --> 00:07:18,680 And you can hear me. 136 00:07:18,680 --> 00:07:20,760 So this is our new setup. 137 00:07:20,760 --> 00:07:23,120 The last couple of years, I have joined you from my 138 00:07:23,120 --> 00:07:25,080 living room and here I am in my office. 139 00:07:25,080 --> 00:07:29,600 I think it's our first day back for a subset of our 140 00:07:29,600 --> 00:07:33,360 staff and we're starting to all come back, starting to 141 00:07:33,360 --> 00:07:34,480 feel normal again. 142 00:07:34,480 --> 00:07:36,400 So welcome, everyone. 143 00:07:36,400 --> 00:07:40,080 Let me start off by welcoming our advisory council 144 00:07:40,080 --> 00:07:40,480 members. 145 00:07:40,480 --> 00:07:43,600 Welcome back to all of our continuing members. 146 00:07:43,600 --> 00:07:47,080 And I want to introduce our newest members for whom 147 00:07:47,080 --> 00:07:50,960 this is their first meeting as a full member. They've 148 00:07:50,960 --> 00:07:53,400 been with us before as ad hocs. 149 00:07:53,400 --> 00:07:57,600 Leopoldo Cabassa, Matthew Carpenter and 150 00:07:58,440 --> 00:08:00,120 Christine Hendon. 151 00:08:00,120 --> 00:08:02,480 So welcome to all three of them. 152 00:08:02,480 --> 00:08:08,800 I also want to welcome our ad hocs for this meeting, 153 00:08:08,800 --> 00:08:12,840 Karen Anderson, Sean Davidson, Edda Spiekerkoetter and 154 00:08:12,840 --> 00:08:14,000 Lynn Yee. 155 00:08:14,000 --> 00:08:16,680 Great to have you join us and this is a pretty dynamic 156 00:08:16,680 --> 00:08:16,880 group. 157 00:08:16,880 --> 00:08:20,080 And I know that you will participate fully in all of 158 00:08:20,080 --> 00:08:21,240 our discussions here today. 159 00:08:21,240 --> 00:08:25,880 So it's a full agenda. 160 00:08:25,880 --> 00:08:29,120 As I do every time, I just want to get started with 161 00:08:29,120 --> 00:08:31,040 what it is we do at CSR. 162 00:08:31,040 --> 00:08:36,280 Talk about a strategic framework and our scope. 163 00:08:36,280 --> 00:08:39,280 Just before every meeting, I think it's important for 164 00:08:39,280 --> 00:08:41,280 us to be graded in that. 165 00:08:41,280 --> 00:08:46,240 CSR is one of the 27 institutes and centers at the NIH 166 00:08:46,240 --> 00:08:49,560 and we review applications for all 24 funding 167 00:08:49,560 --> 00:08:54,120 institutes at the NIH as well as the office of the 168 00:08:54,120 --> 00:08:54,520 director. 169 00:08:54,520 --> 00:08:58,440 Our mission is singular and focused entirely on 170 00:08:58,440 --> 00:09:01,880 ensuring that NIH applications get fair, independent 171 00:09:01,880 --> 00:09:05,800 expert and timely reviews that are free from 172 00:09:05,800 --> 00:09:09,720 inappropriate influences to allow the NIH to identify 173 00:09:09,720 --> 00:09:14,120 and fund the most high impact and promising research. 174 00:09:14,120 --> 00:09:16,640 It's an important mission. 175 00:09:16,640 --> 00:09:19,960 I've talked about our strategic framework in the past. 176 00:09:19,960 --> 00:09:24,600 So all of the members and even those who have ad hocked 177 00:09:24,600 --> 00:09:28,040 before are familiar with this, but just a quick 178 00:09:28,040 --> 00:09:30,720 overview, our framework, obviously, as our singular 179 00:09:30,720 --> 00:09:34,200 mission is entirely focused on the quality of care 180 00:09:34,200 --> 00:09:36,520 review and it's in three domains. 181 00:09:36,520 --> 00:09:41,000 Study sections, whether the scope of the study sections 182 00:09:41,000 --> 00:09:44,960 is appropriate to identify the best science, reviewers, 183 00:09:44,960 --> 00:09:47,960 any aspects or qualifications and training. 184 00:09:47,960 --> 00:09:50,160 And sort of inherent parts of our process. 185 00:09:50,160 --> 00:09:53,240 All of these three domains very much impact what comes 186 00:09:53,240 --> 00:09:55,080 out of the peer review process. 187 00:09:55,080 --> 00:09:59,520 So it's an important part of our mission. 188 00:09:59,520 --> 00:10:03,160 And we do this with the basis of our operating 189 00:10:03,160 --> 00:10:03,520 principles. 190 00:10:03,520 --> 00:10:08,560 And I hope that, you know, those of you who have been 191 00:10:08,560 --> 00:10:12,760 here before and as you'll see today, that you'll see 192 00:10:12,760 --> 00:10:16,960 that we're doing things with a basic core principles of 193 00:10:16,960 --> 00:10:20,720 transparency with the public, using data to make our 194 00:10:20,720 --> 00:10:22,840 decisions, engaging with stakeholders, especially 195 00:10:22,840 --> 00:10:25,360 council, but all stakeholders and communications. 196 00:10:25,360 --> 00:10:30,960 You've heard in the past different topics of discussion 197 00:10:30,960 --> 00:10:35,280 related to different parts of this framework. 198 00:10:35,280 --> 00:10:38,800 ENQUIRE is an ongoing process that you've heard about 199 00:10:38,800 --> 00:10:39,200 before. 200 00:10:39,200 --> 00:10:42,200 For reviewers, I've talked in the past couple of 201 00:10:42,200 --> 00:10:45,520 councils, I had -- you know, I laid out strategies for 202 00:10:45,520 --> 00:10:47,840 diversifying and broadening the reviewer pool and 203 00:10:47,840 --> 00:10:50,040 shared some data related to that. 204 00:10:50,040 --> 00:10:54,360 I've talked in the past about integrity and we've 205 00:10:54,360 --> 00:10:58,680 rolled out integrity training modules as well as other 206 00:10:58,680 --> 00:11:00,080 fairness bias mitigation. 207 00:11:00,080 --> 00:11:04,400 So today's agenda will include, again, two 208 00:11:04,400 --> 00:11:09,320 presentations on ENQUIRE by two of our division 209 00:11:09,320 --> 00:11:14,840 directors, Ray Jacobson and Ross Shonat. In addition, 210 00:11:14,840 --> 00:11:20,960 you will hear from Hope Cummings who will talk about 211 00:11:20,960 --> 00:11:26,480 the survey analysis of the bias training that we 212 00:11:26,480 --> 00:11:28,320 launched last fall. 213 00:11:28,320 --> 00:11:33,200 And then Kristin Kramer, who directs our communications 214 00:11:33,200 --> 00:11:39,880 and outreach effort is going to talk to you about her 215 00:11:39,880 --> 00:11:45,920 office's scope and get council advice on some of our 216 00:11:45,920 --> 00:11:47,160 outreach efforts. 217 00:11:47,160 --> 00:11:52,560 Then, of course, again, last but not least, you are 218 00:11:52,560 --> 00:11:58,000 going to hear from Elizabeth villa and Bruce Reed about 219 00:11:58,000 --> 00:12:02,840 an interim report of the fellowship working group that 220 00:12:02,840 --> 00:12:07,720 is looking at criteria and how those impact outcomes. 221 00:12:07,720 --> 00:12:11,840 We have, if you recall last time, one of our division 222 00:12:11,840 --> 00:12:14,800 directors discussed with you our draft strategic plan. 223 00:12:14,800 --> 00:12:19,640 That plan was issued and -- as a draft. We are now in 224 00:12:19,640 --> 00:12:20,760 a comment period. 225 00:12:21,000 --> 00:12:24,400 I've actually, you know, really been impressed by all 226 00:12:24,400 --> 00:12:27,440 of the comments that we've received from individuals, 227 00:12:27,440 --> 00:12:31,200 from societies. They are really well thought out and 228 00:12:31,200 --> 00:12:35,000 helpful to us. We have extended the comment period 229 00:12:35,000 --> 00:12:37,640 until the end of April, April 30th. 230 00:12:37,640 --> 00:12:42,160 And what are we going to do moving forward once we have 231 00:12:42,160 --> 00:12:46,320 all the comments? We will do a content analysis, look 232 00:12:46,320 --> 00:12:50,480 at all the feedback. We will adjust aspects of the 233 00:12:50,480 --> 00:12:54,640 plan that we can and that we feel are necessary based 234 00:12:54,640 --> 00:12:59,160 on the feedback. We will publish a summary of the main 235 00:12:59,160 --> 00:13:03,680 feedback we got and then we hope, if all goes well, to 236 00:13:03,680 --> 00:13:07,480 issue the final plan sometime in late spring of this 237 00:13:07,480 --> 00:13:07,840 year. 238 00:13:07,840 --> 00:13:11,960 Another slide that I like to show every time just to 239 00:13:11,960 --> 00:13:15,320 remind people about the scope of CSR's operations and 240 00:13:15,320 --> 00:13:18,680 just when people talk about changing things in peer 241 00:13:18,680 --> 00:13:22,800 review, it's good to know how broad that is, how broad 242 00:13:22,800 --> 00:13:24,280 implementing change can be. 243 00:13:24,280 --> 00:13:27,280 It covers all of NIH science. 244 00:13:27,280 --> 00:13:35,400 So NIH in fiscal year '21 received 88,000 applications. 245 00:13:35,400 --> 00:13:40,120 Of those, 75% were handled by CSR, so that's around 246 00:13:40,120 --> 00:13:42,960 66,000 applications in just one year. 247 00:13:42,960 --> 00:13:48,480 That includes a majority of the RO1s and fellowships 248 00:13:48,480 --> 00:13:52,200 that CSR is very familiar for. 249 00:13:52,200 --> 00:13:53,880 We didn't do this alone, obviously. 250 00:13:53,880 --> 00:14:00,480 This is with the invaluable assistance of greater than 251 00:14:00,480 --> 00:14:05,640 20,000 reviewers, individual reviewers with whom we 252 00:14:05,640 --> 00:14:12,960 engage and who did this entirely voluntarily. $200 is 253 00:14:12,960 --> 00:14:16,640 pretty much a volunteer gig. 254 00:14:16,640 --> 00:14:20,360 They did this to identify the best science in our areas 255 00:14:20,360 --> 00:14:23,040 and we did this in around 1,300 meetings. 256 00:14:23,040 --> 00:14:28,440 In addition to RO1s and SVIRs and fellowships, CSR in 257 00:14:28,440 --> 00:14:32,760 the last fiscal year reviewed 182 special initiatives 258 00:14:32,760 --> 00:14:34,360 of the NIH. 259 00:14:34,360 --> 00:14:36,720 Many of these you will recognize. 260 00:14:36,720 --> 00:14:42,760 These are just a few examples of some of the high 261 00:14:42,760 --> 00:14:46,080 profile initiatives that our staff oversaw. 262 00:14:46,080 --> 00:14:50,960 This is obviously not complete. We have many special 263 00:14:50,960 --> 00:14:55,400 reviews that we handle whenever something is sort of 264 00:14:55,400 --> 00:14:59,320 high profile, trans-NIH, it is CSR's responsibility to 265 00:14:59,320 --> 00:15:00,800 handle those reviews. 266 00:15:00,800 --> 00:15:05,080 This is a plot that I've shown the last few times 267 00:15:05,080 --> 00:15:09,400 because I think a question came up maybe a few councils 268 00:15:09,400 --> 00:15:12,880 ago about whether the application rates from women -- I 269 00:15:12,880 --> 00:15:16,000 think the question was specifically related to women -- 270 00:15:16,000 --> 00:15:17,560 had fallen post pandemic. 271 00:15:17,920 --> 00:15:21,200 They haven't for R01 submissions. 272 00:15:21,200 --> 00:15:26,600 They've stay steady. 273 00:15:26,600 --> 00:15:29,800 The overall submissions of women and by 274 00:15:29,800 --> 00:15:30,720 underrepresented minorities. 275 00:15:30,720 --> 00:15:36,200 This green line represents the pandemic. 276 00:15:36,200 --> 00:15:41,640 So we're continuing examine this. 277 00:15:41,640 --> 00:15:44,880 Last time I think I showed how scoring outcomes occur 278 00:15:44,880 --> 00:15:46,840 for women post pandemic, as well. 279 00:15:46,840 --> 00:15:48,440 I'm happy to share that data again. 280 00:15:48,440 --> 00:15:51,560 But there wasn't that much of an impact in terms of the 281 00:15:51,560 --> 00:15:52,320 outcomes of review. 282 00:15:52,320 --> 00:15:55,120 And I think the question originally raised was about 283 00:15:55,120 --> 00:15:57,600 success rates, which happens about a year later. 284 00:15:57,600 --> 00:16:04,040 And I know that OER has published some information 285 00:16:04,040 --> 00:16:06,880 recently on success rates. 286 00:16:06,880 --> 00:16:10,680 Let me turn to CSR news, especially personnel news. 287 00:16:10,680 --> 00:16:13,880 Then I'll update you on a couple of items that you had 288 00:16:13,880 --> 00:16:16,040 asked for, initiatives that the council had been 289 00:16:16,040 --> 00:16:18,440 involved in and what is the update on those. 290 00:16:18,440 --> 00:16:19,880 So I'll go to those next. 291 00:16:19,880 --> 00:16:23,640 So let me start with personnel. 292 00:16:23,640 --> 00:16:26,680 With, you know, several leadership and management 293 00:16:26,680 --> 00:16:31,040 transition that's we've had at the -- at CSR since last 294 00:16:31,040 --> 00:16:33,240 time we met in September. 295 00:16:33,520 --> 00:16:38,760 So the first item is a sadder one. 296 00:16:38,760 --> 00:16:43,880 Not necessarily a promotion or a new person, but we 297 00:16:43,880 --> 00:16:48,000 have our chief of basic neuroscience, Carole Jelsema, 298 00:16:48,000 --> 00:16:53,120 is retiring at the end of next month, I believe. 299 00:16:53,120 --> 00:16:55,520 And I wanted to acknowledge Carole's service. 300 00:16:55,520 --> 00:17:01,720 She has been an integral part of the CSR community for 301 00:17:01,720 --> 00:17:03,440 almost three decades. 302 00:17:03,440 --> 00:17:06,200 She's been involved in neuroscience reviews throughout 303 00:17:06,200 --> 00:17:10,200 our career and she's led a lot of high profile 304 00:17:10,200 --> 00:17:13,800 initiatives and trained an entire generation of SROs. 305 00:17:13,800 --> 00:17:18,560 So she will really be missed and I want to wish her 306 00:17:18,560 --> 00:17:22,160 happy retirement out here in public with the council 307 00:17:22,160 --> 00:17:22,560 here. 308 00:17:22,560 --> 00:17:26,080 We created a new position called senior SROs. 309 00:17:26,080 --> 00:17:34,440 These are -- this was specifically targeted to create a 310 00:17:34,440 --> 00:17:41,120 cohort of SROs who are sciencely flexible. 311 00:17:41,120 --> 00:17:47,080 Suddenly initiatives come up and they're high priority 312 00:17:47,080 --> 00:17:50,080 and they're fast track. 313 00:17:50,080 --> 00:17:52,040 This allows us some flexible. 314 00:17:52,040 --> 00:17:55,720 They cover everything from basic science to 315 00:17:55,720 --> 00:17:57,840 epidemiology, social science, bioengineering, 316 00:17:57,840 --> 00:17:58,360 everything. 317 00:17:58,360 --> 00:18:06,880 So it's a really nice group that we have and I want to 318 00:18:06,880 --> 00:18:10,840 acknowledge John Arias, Jessica bellinger, Michael 319 00:18:10,840 --> 00:18:14,120 Bloom, Heidi Friedman, James Li. 320 00:18:14,120 --> 00:18:17,280 They're located in the office of the director. 321 00:18:17,280 --> 00:18:19,760 We have three new branch chiefs. 322 00:18:19,760 --> 00:18:25,840 They oversee SROs and study sections in given 323 00:18:25,840 --> 00:18:27,360 scientific areas. 324 00:18:27,360 --> 00:18:31,200 So Tom Beres is the new chief of the health services 325 00:18:31,200 --> 00:18:32,280 and systems branch. 326 00:18:32,280 --> 00:18:39,240 Emily Foley is the new chief of disease control and 327 00:18:39,240 --> 00:18:45,480 applied immunology and elia Ortenberg is head of the 328 00:18:45,480 --> 00:18:50,360 social and community influences across the lifecourse. 329 00:18:50,360 --> 00:18:53,920 In addition, Dipak Bhattacharyya who was our acting 330 00:18:53,920 --> 00:18:58,800 director is now the new director of a new division of 331 00:18:58,800 --> 00:19:01,920 planning analysis and information management. Because 332 00:19:01,920 --> 00:19:05,480 of our focus on data-driven decision making -- and 333 00:19:05,480 --> 00:19:10,800 you've seen a lot of our analysis and data come out of 334 00:19:10,800 --> 00:19:15,680 CSR, this has been an integral division for us and it 335 00:19:15,680 --> 00:19:19,680 was necessary to create a broader structure to manage 336 00:19:19,680 --> 00:19:23,800 all the work going on in analysis and planning. 337 00:19:23,800 --> 00:19:29,920 So please welcome Dipak to the leadership team, CSR. 338 00:19:29,920 --> 00:19:34,640 And then if you recall, last time I talked about the 339 00:19:34,640 --> 00:19:38,120 establishment of an office of training and development 340 00:19:38,120 --> 00:19:42,440 for SRO and reviewer training within the office of the 341 00:19:42,440 --> 00:19:46,760 director. And I had announced that Vanessa Boyce is 342 00:19:46,760 --> 00:19:48,920 handling the new SRO training. 343 00:19:48,920 --> 00:19:52,160 So these coordinator positions -- Vanessa is one of 344 00:19:52,160 --> 00:19:54,160 them -- are full time SROs. 345 00:19:55,440 --> 00:19:57,480 And they have one element of their job. 346 00:19:57,480 --> 00:20:02,560 So part-time they assist in areas of their expertise. 347 00:20:02,560 --> 00:20:12,120 So Vanessa oversees the training of our new SROs. 348 00:20:12,120 --> 00:20:19,360 I am happy to announce we have completely staffed up in 349 00:20:19,360 --> 00:20:23,960 our office starting with Miriam Mintzer, director. 350 00:20:24,600 --> 00:20:31,560 And we have three new coordinators this time. Tanya 351 00:20:31,560 --> 00:20:36,440 Cohen, overseeing reviewer training, Ben Shapero is 352 00:20:36,440 --> 00:20:41,280 overseeing the handbook and policy coordination and 353 00:20:41,280 --> 00:20:45,480 Natalia Komissarova is overseeing continuing education 354 00:20:45,480 --> 00:20:49,640 for all of on our SROs. 355 00:20:49,640 --> 00:20:53,520 So I'm happy to have this established and have a 356 00:20:53,520 --> 00:20:56,280 rigorous training program under way at CSR. 357 00:20:56,280 --> 00:21:04,400 You had wanted, I know, to hear about an update on the 358 00:21:04,400 --> 00:21:07,080 early career reviewer program. 359 00:21:07,080 --> 00:21:11,160 Just a reminder for those who were not here when this 360 00:21:11,160 --> 00:21:14,080 started, back in 2019, an advisory council working 361 00:21:14,080 --> 00:21:18,160 group looked at the ECR program and rehauled it. You 362 00:21:18,160 --> 00:21:21,840 know, and part of the main change that was made -- 363 00:21:21,840 --> 00:21:25,160 there were many and multiple changes made -- but the 364 00:21:25,160 --> 00:21:29,200 main change made was to double the number of ECRs that 365 00:21:29,200 --> 00:21:31,800 we were using in each study section. 366 00:21:32,120 --> 00:21:35,320 They also developed new criteria to really target the 367 00:21:35,320 --> 00:21:38,520 program more towards where it may have the biggest 368 00:21:38,520 --> 00:21:41,400 impact, which is career investigators who don't have 369 00:21:41,400 --> 00:21:45,320 review experience or an R01. So it positions them to 370 00:21:45,320 --> 00:21:49,600 experience the new process at a time when it can have a 371 00:21:49,600 --> 00:21:50,320 big impact. 372 00:21:50,320 --> 00:21:56,480 And instead of having individual SROs select ECRs and 373 00:21:56,480 --> 00:22:02,640 evaluate whether any CR is qualified or not, which 374 00:22:02,640 --> 00:22:08,800 leads to some discrepancy -- of course, we have 250 375 00:22:08,800 --> 00:22:09,480 SROs. 376 00:22:09,880 --> 00:22:13,360 So they made this a central process. We have an SRO 377 00:22:13,360 --> 00:22:16,280 committee that looks at them and looks at the criteria 378 00:22:16,280 --> 00:22:17,720 for acceptance into the program. 379 00:22:17,720 --> 00:22:21,600 And we had a new database because the past database did 380 00:22:21,600 --> 00:22:23,720 not allow easy tracking or evaluation. 381 00:22:23,720 --> 00:22:26,640 So going forward, we are able to track. 382 00:22:26,640 --> 00:22:29,080 So those were the changes made. 383 00:22:29,080 --> 00:22:34,040 Let me give you a quick update on what our early career 384 00:22:34,040 --> 00:22:37,720 reviewers sort of the demographics of the early career 385 00:22:37,720 --> 00:22:38,960 reviewers looked like. 386 00:22:38,960 --> 00:22:43,560 We just compared the February/March meetings since the 387 00:22:43,560 --> 00:22:49,320 program was rehauled. You can look that females, among 388 00:22:49,320 --> 00:22:51,600 the cohort were increased. 389 00:22:51,600 --> 00:22:55,480 That's from 49 to 55% almost. 390 00:22:55,480 --> 00:22:58,800 Under represented minority, it was already kind of 391 00:22:58,800 --> 00:23:03,000 enriched and went up significantly in the next year and 392 00:23:03,000 --> 00:23:05,960 then kind of fell again this year. 393 00:23:05,960 --> 00:23:08,280 Talk a little bit about why that might be. 394 00:23:08,280 --> 00:23:12,880 And the same trend is true for black African-American 395 00:23:12,880 --> 00:23:16,960 reviewers and to some extent for Hispanic reviewers. 396 00:23:16,960 --> 00:23:18,000 So we can speculate. 397 00:23:18,000 --> 00:23:22,040 We haven't done a deeper analysis yet to see what 398 00:23:22,040 --> 00:23:24,440 happened and why that bump happened. 399 00:23:24,440 --> 00:23:28,720 I suspect -- and Kristin can fill this in when she 400 00:23:28,720 --> 00:23:32,200 gives her talk later today -- about communications is 401 00:23:32,200 --> 00:23:36,080 that we did a pretty successful webinar outreach to 402 00:23:36,080 --> 00:23:37,800 recruit early career reviewers. 403 00:23:38,160 --> 00:23:42,880 And I believe that that bump occurred soon after that. 404 00:23:42,880 --> 00:23:45,880 So it's possible that we need to do more of that kind 405 00:23:45,880 --> 00:23:48,680 of outreach, but we have to do a lot more analysis 406 00:23:48,680 --> 00:23:48,920 first. 407 00:23:48,920 --> 00:23:52,640 The other thing that we noticed is that different 408 00:23:52,640 --> 00:23:55,560 scientific areas are represented differentially in ECR. 409 00:23:55,560 --> 00:23:59,920 So some have more ECRs and are oversubscribed and some 410 00:23:59,920 --> 00:24:01,640 have a lot fewer. 411 00:24:01,640 --> 00:24:04,400 So we are targeting our efforts towards scientific 412 00:24:04,400 --> 00:24:07,840 areas where we don't have as many ECRs signing up. 413 00:24:07,840 --> 00:24:11,760 And then one of the questions that comes up is the 414 00:24:11,760 --> 00:24:12,840 outcomes for ECR. 415 00:24:12,840 --> 00:24:17,320 So this is sort of program outcomes starting in 2012 416 00:24:17,320 --> 00:24:19,960 when the program was first initiated. 417 00:24:19,960 --> 00:24:25,720 And, you know, 46% or so have competed for an R01 418 00:24:25,720 --> 00:24:27,800 successfully, which is good. 419 00:24:27,800 --> 00:24:31,680 And around 14% have served as standing members. 420 00:24:31,680 --> 00:24:34,760 These data are little -- there's no real control for 421 00:24:34,760 --> 00:24:35,680 these, right? 422 00:24:35,680 --> 00:24:39,720 So we don't know if they would have done it despite or, 423 00:24:39,720 --> 00:24:42,760 you know, whether the ECR program made a difference. 424 00:24:42,760 --> 00:24:46,480 It's hard to know that and this is not a good -- we 425 00:24:46,480 --> 00:24:49,600 haven't quite gotten our head wrapped around how to do 426 00:24:49,600 --> 00:24:53,000 this analysis in a way where we can see the effect 427 00:24:53,000 --> 00:24:54,240 specifically of the program. 428 00:24:54,520 --> 00:24:59,520 So we plan to continue targeted outreach efforts both 429 00:24:59,520 --> 00:25:04,480 broadly as well as targeted areas of science and 430 00:25:04,480 --> 00:25:06,680 further analysis, as well. 431 00:25:06,680 --> 00:25:10,360 The other update is related to the simplifying review 432 00:25:10,360 --> 00:25:14,440 criteria effort. Several of you were involved in the 433 00:25:14,440 --> 00:25:18,560 working group. There was a non clinical trial working 434 00:25:18,560 --> 00:25:21,400 group that was convened in early 2020. 435 00:25:21,400 --> 00:25:24,680 And then there was a clinical trial working group with 436 00:25:24,680 --> 00:25:26,320 several of the same people. 437 00:25:26,320 --> 00:25:29,320 Tonya was the co-chair in both cases. 438 00:25:29,320 --> 00:25:33,400 That presented its final recommendations to this 439 00:25:33,400 --> 00:25:38,120 advisory council and got approval for those, after 440 00:25:38,120 --> 00:25:42,800 which it moved to NIH for further development. 441 00:25:42,800 --> 00:25:49,880 So just an update, in July of 2021, this was -- this 442 00:25:49,880 --> 00:25:55,680 concept was cleared and approved by a senior leadership 443 00:25:55,680 --> 00:26:00,800 committee at NIH which looks at everything extramural. 444 00:26:01,200 --> 00:26:04,640 They recommended we form a working group to see which 445 00:26:04,640 --> 00:26:06,320 ones are feasible and practical. 446 00:26:06,320 --> 00:26:11,120 The working group met. I was a co-chair of the working 447 00:26:11,120 --> 00:26:13,120 group along with John Lorsh. 448 00:26:13,120 --> 00:26:16,000 We did the same thing that your working group did, 449 00:26:16,000 --> 00:26:18,560 which was break it up into non clinical trials. 450 00:26:18,560 --> 00:26:23,840 Clinical trials is a little less than 10% of the 451 00:26:23,840 --> 00:26:27,520 applications and it's ten times more complicated. 452 00:26:27,520 --> 00:26:29,960 So we thought we would do this first. 453 00:26:29,960 --> 00:26:33,960 So we developed recommendations and presented those 454 00:26:33,960 --> 00:26:39,120 back to the extramural working group where it got 455 00:26:39,120 --> 00:26:39,680 approval. 456 00:26:39,680 --> 00:26:44,400 And the next up is next month we will be presenting 457 00:26:44,400 --> 00:26:47,400 these recommendations to the NIH steering committee. 458 00:26:47,400 --> 00:26:52,920 So just stay tuned. It is moving along through the NIH. 459 00:26:52,920 --> 00:26:56,800 The other thing is not really an update, but just a 460 00:26:56,800 --> 00:26:57,160 reminder. 461 00:26:57,160 --> 00:27:03,960 Gabriel Fosu is our associate director Of Diversity and 462 00:27:03,960 --> 00:27:05,480 Workforce Development. 463 00:27:05,480 --> 00:27:08,960 Last month, CSR established a reporting avenue, a 464 00:27:08,960 --> 00:27:12,400 direct way in which investigators and reviewers can 465 00:27:12,400 --> 00:27:15,440 report by us and review to CSR. 466 00:27:15,440 --> 00:27:18,720 We include it in our signature as we tried to publicize 467 00:27:18,720 --> 00:27:19,000 it. 468 00:27:19,000 --> 00:27:21,640 It's gotten a little bit of traction, but not as much 469 00:27:21,640 --> 00:27:24,240 as I thought it would. We have some Emails coming 470 00:27:24,240 --> 00:27:25,440 through, but not a lot. 471 00:27:25,440 --> 00:27:29,240 So we welcome your assistance in getting the word out 472 00:27:29,240 --> 00:27:31,920 that there is a way to report. 473 00:27:31,920 --> 00:27:36,880 Every allegation is investigated by CSR senior 474 00:27:36,880 --> 00:27:37,600 management. 475 00:27:37,600 --> 00:27:40,920 Both Gabriel and the scientific division director 476 00:27:40,920 --> 00:27:43,800 together look at every allegation carefully. 477 00:27:43,800 --> 00:27:47,880 If we agree that this was a bias review, we review it 478 00:27:47,880 --> 00:27:50,640 because we don't want to disadvantage the applicant. 479 00:27:50,640 --> 00:27:55,520 If we don't agree, we guide them to go through the 480 00:27:55,520 --> 00:27:58,640 normal appeals process that is always available. 481 00:27:58,640 --> 00:28:01,640 One point that I wanted to make, which is something we 482 00:28:01,640 --> 00:28:04,640 didn't used to do before. We have done rereviews in 483 00:28:04,640 --> 00:28:05,720 past years, as well. 484 00:28:05,720 --> 00:28:09,000 But we never look back to the reviewer who caused the 485 00:28:09,000 --> 00:28:12,000 problem, right, with the statement that was bias. We 486 00:28:12,000 --> 00:28:13,520 are doing that regularly now. 487 00:28:13,520 --> 00:28:16,200 So the division director calls the reviewer, tells him 488 00:28:16,200 --> 00:28:20,040 or her, you know, this is what you did and this is the 489 00:28:20,040 --> 00:28:21,240 problem that it caused. 490 00:28:21,240 --> 00:28:23,600 It caused us to rereview. 491 00:28:23,600 --> 00:28:26,080 I think that's important for reviewer education, for 492 00:28:26,080 --> 00:28:29,200 culture change, for them to know what caused an issue. 493 00:28:29,200 --> 00:28:31,240 And then we watch to see what kind of reaction. 494 00:28:31,240 --> 00:28:34,680 If the reaction is oh, wow, I've messed up, I've 495 00:28:34,680 --> 00:28:36,720 learned something, that's a good thing. 496 00:28:36,720 --> 00:28:40,920 If the reaction is that is not a bias statement, that 497 00:28:40,920 --> 00:28:44,320 gives us more information about the reviewer and we 498 00:28:44,320 --> 00:28:46,240 take further action as necessary. 499 00:28:46,240 --> 00:28:46,680 All right. 500 00:28:46,680 --> 00:28:54,480 And then finally, let's talk about the future of CSR 501 00:28:54,480 --> 00:28:56,840 peer review meetings. 502 00:28:56,840 --> 00:28:58,360 You've seen this before. 503 00:28:58,360 --> 00:29:02,440 I'll go over there very quickly. We have the reports 504 00:29:02,440 --> 00:29:06,120 there, but clearly, one of the main areas of concern 505 00:29:06,120 --> 00:29:08,720 with Zoom meetings -- overall, we're doing fine. 506 00:29:09,000 --> 00:29:12,160 Overall, quality seems okay. 507 00:29:12,160 --> 00:29:15,160 There is many types of evaluations that were done and 508 00:29:15,160 --> 00:29:18,120 overall it seems like the outcomes of review are okay. 509 00:29:18,120 --> 00:29:22,520 They're working fine in a virtual setting. 510 00:29:22,520 --> 00:29:24,920 Reviewer engagement suffers. 511 00:29:24,920 --> 00:29:27,440 I think that is not surprising. 512 00:29:27,440 --> 00:29:30,960 It's true for anything that is happening through Zoom. 513 00:29:30,960 --> 00:29:34,480 Because of that, you see also a similar worsening in 514 00:29:34,480 --> 00:29:35,520 the productive discussion. 515 00:29:35,520 --> 00:29:39,560 This is reviewer impressions of the meeting. 516 00:29:39,560 --> 00:29:41,480 And it's pretty consistent. 517 00:29:41,480 --> 00:29:45,280 We did this for two years in 2020 and 2021. 518 00:29:45,280 --> 00:29:49,200 I showed this last time. Which is kind of still 519 00:29:49,200 --> 00:29:51,000 interesting, which is the preference. 520 00:29:51,000 --> 00:29:58,880 So most reviewers prefer in-person meetings. 521 00:29:58,880 --> 00:30:02,560 Some prefer Zoom meetings and some have no preference. 522 00:30:02,560 --> 00:30:06,640 When you break it down by gender, both men and women 523 00:30:06,640 --> 00:30:09,600 prefer in-person, but the difference is greater for 524 00:30:09,600 --> 00:30:09,960 men. 525 00:30:09,960 --> 00:30:14,640 So that is an interesting thought and we can come up 526 00:30:14,640 --> 00:30:18,040 with reasons as to why that might be. 527 00:30:18,040 --> 00:30:23,040 We did this for career stage and we also went back and 528 00:30:23,040 --> 00:30:28,000 did it for chairs, which kind of maps on to the full 529 00:30:28,000 --> 00:30:31,720 professor outcome. Which is, again, all career stages 530 00:30:31,720 --> 00:30:35,440 prefer in-person, but the margin is larger for full 531 00:30:35,440 --> 00:30:39,600 professors for more senior reviewers. We can talk a 532 00:30:39,600 --> 00:30:42,920 little bit why that might be, as well. 533 00:30:42,920 --> 00:30:44,680 I've shown this before. 534 00:30:44,680 --> 00:30:48,240 We've done a pretty comprehensive analysis of score 535 00:30:48,240 --> 00:30:50,880 distributions and out of range scoring. 536 00:30:50,880 --> 00:30:55,360 So actual outcomes, not just reviewer impressions of 537 00:30:55,360 --> 00:30:55,920 meetings. 538 00:30:55,920 --> 00:31:00,920 There is not a real difference in the score 539 00:31:00,920 --> 00:31:04,800 distributions from in-person versus Zoom taking the 540 00:31:04,800 --> 00:31:06,480 same study sections. 541 00:31:06,480 --> 00:31:10,720 So in the end, we've talked about this before, both 542 00:31:10,720 --> 00:31:13,680 formats have their own strengths and weaknesses. 543 00:31:13,680 --> 00:31:17,360 For in-person meetings, we know more reviewers like 544 00:31:17,360 --> 00:31:17,800 those. 545 00:31:17,800 --> 00:31:20,640 They're more engaging. You're more attentive. 546 00:31:20,640 --> 00:31:23,520 There's a chance for the group to come together and 547 00:31:23,520 --> 00:31:25,000 form some ties and bonds. 548 00:31:25,000 --> 00:31:37,840 It's really important for recurring panels. 549 00:31:37,840 --> 00:31:40,800 And people want to come because they get to talk to 550 00:31:40,800 --> 00:31:41,320 other people. 551 00:31:41,320 --> 00:31:47,040 That is lost during zoom. 552 00:31:47,040 --> 00:31:51,320 Negative, of course, is all the time that is spent 553 00:31:51,320 --> 00:31:54,760 going back and forth in planes and Ubers. 554 00:31:54,760 --> 00:31:58,400 There are other aspects that might make it more 555 00:31:58,400 --> 00:32:01,240 difficult for others to participate, child care, 556 00:32:01,240 --> 00:32:03,240 clinical, our types of duties. 557 00:32:03,240 --> 00:32:06,400 For us, this is important because it might hurt 558 00:32:06,400 --> 00:32:08,480 recruitment efforts and broader participation that 559 00:32:08,480 --> 00:32:09,880 we're interested in seeing. 560 00:32:09,880 --> 00:32:16,320 And then, of course, the environmental and fiscal 561 00:32:16,320 --> 00:32:17,120 impact. 562 00:32:17,120 --> 00:32:21,840 For virtual meetings, some reviewers, a good 30% or so 563 00:32:21,840 --> 00:32:23,240 prefer virtual meetings. 564 00:32:23,240 --> 00:32:25,360 The time saved is hard to compete with. 565 00:32:25,360 --> 00:32:29,120 It's a lot of time saved and still participate and 566 00:32:29,120 --> 00:32:29,480 review. 567 00:32:29,480 --> 00:32:33,560 It's easier for people with other duties. 568 00:32:33,560 --> 00:32:38,600 And, of course, there's no environmental impact and 569 00:32:38,600 --> 00:32:43,640 fiscally it comes out really well for NIH. 570 00:32:43,640 --> 00:32:46,560 But the negative, again, what we've heard, reviewer 571 00:32:46,560 --> 00:32:49,840 engagement, attention span, no chance for the group to 572 00:32:49,840 --> 00:32:53,080 come together and then you don't have the networking. 573 00:32:53,080 --> 00:32:56,760 So there is a balance. 574 00:32:56,760 --> 00:33:03,640 And so in fall 2022, we will start to hold some 575 00:33:03,640 --> 00:33:09,320 in-person meetings beginning in the fall. There are 576 00:33:09,320 --> 00:33:14,960 people -- because all the mask mandates, even in the 577 00:33:14,960 --> 00:33:18,240 really blue virulent are gone. 578 00:33:18,240 --> 00:33:20,640 Many people are saying why don't you have meetings this 579 00:33:20,640 --> 00:33:21,360 summer in person? 580 00:33:21,360 --> 00:33:25,280 This is when you have to look back at the scope slide I 581 00:33:25,280 --> 00:33:27,400 showed you. 1300 meetings, 20,000 reviewers. 582 00:33:27,400 --> 00:33:29,080 It's not that easy to do quickly. 583 00:33:29,080 --> 00:33:32,320 We need five to six-month lead time to do all the 584 00:33:32,320 --> 00:33:33,800 contracts and all the travel. 585 00:33:33,800 --> 00:33:41,040 So what we're going to do is provided everything stays 586 00:33:41,040 --> 00:33:47,560 stable in terms of COVID, all standing panels, those 587 00:33:47,560 --> 00:33:52,640 that have recurring membership or chartered study 588 00:33:52,640 --> 00:33:58,440 sections, SVIR and fellowships will hold one in-person 589 00:33:58,440 --> 00:34:00,600 meetings a year. 590 00:34:00,600 --> 00:34:04,720 And about a third of our panels will meet in person 591 00:34:04,720 --> 00:34:07,720 this fall, ideally, and the remaining two-thirds will 592 00:34:07,720 --> 00:34:11,120 hold in-person meetings coming up in the next two 593 00:34:11,120 --> 00:34:11,480 rounds. 594 00:34:11,480 --> 00:34:15,240 This will all depend on reviewer preference, especially 595 00:34:15,240 --> 00:34:16,160 member preference. 596 00:34:16,160 --> 00:34:19,200 So if we lose half the members because they don't want 597 00:34:19,200 --> 00:34:21,680 to travel yet, they might be more comfortable next 598 00:34:21,680 --> 00:34:21,960 June. 599 00:34:21,960 --> 00:34:24,760 Those are probably the ones we'll delay a bit. 600 00:34:24,760 --> 00:34:26,120 We're going to go slowly. 601 00:34:26,120 --> 00:34:29,760 We want to make sure that we survey our reviewers and 602 00:34:29,760 --> 00:34:32,760 SROs to see what they're comfortable doing this fall 603 00:34:32,760 --> 00:34:35,400 versus in the spring and then the summer. 604 00:34:35,400 --> 00:34:40,320 And for me, it's really important that we're looking at 605 00:34:40,320 --> 00:34:44,760 recruitment success, are we able to recruit as broadly, 606 00:34:44,760 --> 00:34:50,160 are we able to get women and clinicians to support in 607 00:34:50,160 --> 00:34:51,160 in-person meetings. 608 00:34:51,160 --> 00:34:54,960 We have made a decision to not hold hybrid meetings 609 00:34:54,960 --> 00:34:55,760 this fall. 610 00:34:55,760 --> 00:34:59,680 As we go out of the gate, we're not going to do hybrid. 611 00:34:59,680 --> 00:35:01,960 Hybrid means some people participate virtually and 612 00:35:01,960 --> 00:35:03,600 others are in the room. 613 00:35:03,600 --> 00:35:06,600 Part of it is technological. 614 00:35:06,600 --> 00:35:10,120 We don't have as much technological capacity. Hotels 615 00:35:10,120 --> 00:35:12,840 and all of that, it's more difficult. 616 00:35:12,840 --> 00:35:17,000 But, really, it's more about ensuring that our staff 617 00:35:17,000 --> 00:35:21,600 and all of us understand the issues related to inequity 618 00:35:21,600 --> 00:35:25,280 of participation and experience with the virtual people 619 00:35:25,280 --> 00:35:28,040 and learn how to manage that. 620 00:35:28,040 --> 00:35:31,200 That my change in the future so we're open to that. 621 00:35:31,200 --> 00:35:36,440 But just this first go round, we're not going to have 622 00:35:36,440 --> 00:35:37,400 hybrid meetings. 623 00:35:37,400 --> 00:35:42,680 That said, the September 2022 meeting of CSR advisory 624 00:35:42,680 --> 00:35:46,200 council will be held in person. 625 00:35:46,200 --> 00:35:50,280 The date has slightly changed because the first -- the 626 00:35:50,280 --> 00:35:53,920 lasts Monday, I believe, is a Jewish holiday. 627 00:35:54,160 --> 00:35:57,680 So this is the last Monday before the Jewish holiday. 628 00:35:57,680 --> 00:36:02,040 So mark that down, all of the members, and we hope to 629 00:36:02,040 --> 00:36:03,840 see you here in Bethesda. 630 00:36:03,840 --> 00:36:07,000 Now, I want to end with an acknowledgement. 631 00:36:07,000 --> 00:36:11,480 March 2020 is when we all went to virtual meetings and 632 00:36:11,480 --> 00:36:12,320 remote work. 633 00:36:12,320 --> 00:36:16,240 And I want to acknowledge our staff because for the 634 00:36:16,240 --> 00:36:20,200 last two years, you know, the really important work of 635 00:36:20,200 --> 00:36:23,760 scientific peer review, that is so critical to NIH 636 00:36:23,760 --> 00:36:24,960 funded research continuing. 637 00:36:24,960 --> 00:36:29,200 It has continued uninterrupted thanks to our staff, our 638 00:36:29,200 --> 00:36:31,600 scientific support administrative and technological 639 00:36:31,600 --> 00:36:36,800 staff have all worked together, and at a time when the 640 00:36:36,800 --> 00:36:41,080 pandemic was disrupting their drives just as much as 641 00:36:41,080 --> 00:36:44,880 it's disrupting all of the scientific community's life 642 00:36:44,880 --> 00:36:45,840 out there. 643 00:36:45,840 --> 00:36:49,480 This is what we've been doing for the last couple of 644 00:36:49,480 --> 00:36:53,160 years, but we're hoping that we can get back to that 645 00:36:53,160 --> 00:36:56,800 building on the left as where Bruce and I are sitting 646 00:36:56,800 --> 00:36:59,800 right now. We have a couple of buildings. 647 00:36:59,800 --> 00:37:04,360 And then we have some remote staff across the U.S., 648 00:37:04,360 --> 00:37:06,600 which is interesting and fun. 649 00:37:06,600 --> 00:37:13,720 So thank you to CSR staff. 650 00:37:13,720 --> 00:37:15,120 That's it for me. 651 00:37:15,120 --> 00:37:16,520 >> DR. BRUCE REED: Thanks, Noni. 652 00:37:16,520 --> 00:37:18,920 So it's open for discussion. 653 00:37:18,920 --> 00:37:28,360 Anyone? 654 00:37:28,360 --> 00:37:33,840 Jinming. 655 00:37:33,840 --> 00:37:38,680 >> DR. JINMING GAO: Thank you for the update on the 656 00:37:38,680 --> 00:37:41,920 review progress related to consolidation. I'm thinking 657 00:37:41,920 --> 00:37:45,960 that it seems like it's moving along, I guess, nicely 658 00:37:45,960 --> 00:37:50,400 and with the council review. I was thinking about with 659 00:37:50,400 --> 00:37:54,000 the in-person meeting coming up, how would that change 660 00:37:54,000 --> 00:37:56,440 with the review process being implemented? 661 00:37:56,440 --> 00:38:01,760 Would that -- where a timeline first and then the 662 00:38:01,760 --> 00:38:07,120 implementation, I guess, if it is approved by the 663 00:38:07,120 --> 00:38:07,720 council? 664 00:38:07,960 --> 00:38:12,240 >> DR. NONI BYRNES: So I guess I'm not clear, Jinming, 665 00:38:12,240 --> 00:38:16,160 that it would make a difference. I think the review 666 00:38:16,160 --> 00:38:18,840 criteria -- that's what you're talking about, right? 667 00:38:18,840 --> 00:38:19,600 >> DR. JINMING GAO: Right. 668 00:38:19,600 --> 00:38:22,600 >> DR. NONI BYRNES: If that were to move forward, and 669 00:38:22,600 --> 00:38:25,840 I can't say that it will. We have to take it through 670 00:38:25,840 --> 00:38:27,120 the steering committee and beyond. 671 00:38:27,120 --> 00:38:29,240 But I suspect something will move forward. 672 00:38:29,240 --> 00:38:31,240 It's not going to happen this year. 673 00:38:31,240 --> 00:38:35,120 Like I said, it's going to take another year or so just 674 00:38:35,120 --> 00:38:38,680 to get our systems, just ERA itself takes a long time. 675 00:38:38,680 --> 00:38:41,600 And I hope we will be very familiar with handling 676 00:38:41,600 --> 00:38:44,280 virtual and in-person and hybrid meetings by the time 677 00:38:44,280 --> 00:38:45,160 that comes around. 678 00:38:45,160 --> 00:38:45,960 >> DR. JINMING GAO: Yeah. 679 00:38:45,960 --> 00:38:48,240 That's where I was coming from. I was worried about 680 00:38:48,240 --> 00:38:49,480 too many things going on and -- 681 00:38:49,480 --> 00:38:51,400 >> DR. NONI BYRNES: Yeah, no, I understand. 682 00:38:51,400 --> 00:38:53,800 >> DR. JINMING GAO: Yeah. 683 00:38:53,800 --> 00:38:58,440 >> DR. BRUCE REED: We wish, but it won't change that 684 00:38:58,440 --> 00:38:58,840 fast. 685 00:38:58,840 --> 00:39:00,800 Lynn. 686 00:39:00,800 --> 00:39:05,240 >> Lynn Yee: Thank you, Noni. That was a great 687 00:39:05,240 --> 00:39:05,640 overview. 688 00:39:05,640 --> 00:39:08,280 My question is about the ECR reviewers. 689 00:39:08,280 --> 00:39:11,600 I really -- you know, as one of the more junior people 690 00:39:11,600 --> 00:39:14,640 on this panel, maybe one of the few who participated 691 00:39:14,640 --> 00:39:18,240 and I did the program four or five years ago and found 692 00:39:18,240 --> 00:39:19,320 it incredibly informative. 693 00:39:19,320 --> 00:39:22,480 What I was wondering about after seeing the statistics 694 00:39:22,480 --> 00:39:25,600 on who participates is about the statistics on people 695 00:39:25,600 --> 00:39:28,760 who apply that are never selected and how -- you 696 00:39:28,760 --> 00:39:31,920 know, what their characteristics might be and if they 697 00:39:31,920 --> 00:39:35,080 might be a good comparison group for determining the 698 00:39:35,080 --> 00:39:37,000 outcomes of the ECR program. 699 00:39:37,000 --> 00:39:41,960 That is the people who apply, but are not chosen, but 700 00:39:41,960 --> 00:39:46,440 still, you know, what proportion of them both apply for 701 00:39:46,440 --> 00:39:49,120 an R01 and receive an R01 702 00:39:49,120 --> 00:39:53,600 >> DR. NONI BYRNES: That is a good point. 703 00:39:53,600 --> 00:39:57,120 Haven't -- one of the things we have yet to do is go 704 00:39:57,120 --> 00:40:00,320 back to our database and do a rigorous analysis of who 705 00:40:00,320 --> 00:40:02,960 applied and who got in and how they're doing, 706 00:40:02,960 --> 00:40:04,560 especially with areas of science. 707 00:40:04,560 --> 00:40:07,560 So we plan to present a future update to council, maybe 708 00:40:07,560 --> 00:40:10,840 in a year or so, a couple of more councils and have 709 00:40:10,840 --> 00:40:13,800 more information. That would be good to see, you know, 710 00:40:13,800 --> 00:40:14,880 who we didn't select. 711 00:40:14,880 --> 00:40:17,360 But the criteria is so clean, right? 712 00:40:17,360 --> 00:40:21,440 I mean, you know that -- this group developed the 713 00:40:21,440 --> 00:40:24,640 criteria and they're very clean and clear. 714 00:40:24,880 --> 00:40:28,480 >> Lynn Yee: I agree, there are. I know of people who 715 00:40:28,480 --> 00:40:30,720 said I applied and I never heard back. 716 00:40:30,720 --> 00:40:33,160 And I wonder -- I think sometimes early career 717 00:40:33,160 --> 00:40:35,640 investigators might like feedback on why they aren't 718 00:40:35,640 --> 00:40:39,000 selected. I know that would be a large undertaking for 719 00:40:39,000 --> 00:40:40,520 CSR to provide that feedback. 720 00:40:40,920 --> 00:40:42,880 >> DR. NONI BYRNES: Is this recent experience? 721 00:40:42,880 --> 00:40:49,800 >> Lynn Yee: No. 722 00:40:49,800 --> 00:40:51,520 It was a few years. 723 00:40:51,520 --> 00:40:54,880 >> DR. NONI BYRNES: That's one of the reasons we did 724 00:40:54,880 --> 00:40:55,440 this review. 725 00:40:55,440 --> 00:40:57,240 We're trying to do that more in the last couple of 726 00:40:57,240 --> 00:40:57,400 years. 727 00:40:57,400 --> 00:41:00,680 And I think they do hear back if they don't get in. 728 00:41:00,680 --> 00:41:04,560 >> DR. BRUCE REED: They certainly hear if they 729 00:41:04,560 --> 00:41:06,840 didn't -- if they didn't get in. 730 00:41:07,160 --> 00:41:11,320 Turnover is pretty brisk in that program. I don't 731 00:41:11,320 --> 00:41:15,880 know -- I'm sure there are some who stay in the pool 732 00:41:15,880 --> 00:41:20,000 and not selected, but my concern has been more along 733 00:41:20,000 --> 00:41:24,160 the lines of keeping enough people coming in to meet 734 00:41:24,160 --> 00:41:26,440 the need and study sections. 735 00:41:26,440 --> 00:41:30,800 We use about 380, about 380 people around and we 736 00:41:30,800 --> 00:41:34,320 have -- I don't know, Kristin, we've got somewhere 737 00:41:34,320 --> 00:41:37,520 maybe around 1,000 people in the pool. 738 00:41:37,520 --> 00:41:37,760 So -- 739 00:41:37,760 --> 00:41:38,440 >> DR. KRISTIN KRAMER yeah. 740 00:41:38,440 --> 00:41:40,360 So let me expand a bit on that. 741 00:41:40,360 --> 00:41:42,680 So we have about 1200 people in our database. 742 00:41:42,680 --> 00:41:45,040 And it's kind of hovering around 1200. 743 00:41:45,040 --> 00:41:48,560 And in terms of the wait time, it's much reduced from 744 00:41:48,560 --> 00:41:52,080 what it had been before the working group took this on 745 00:41:52,080 --> 00:41:54,000 and established new criteria for participation. 746 00:41:54,000 --> 00:41:56,800 Right now, people who applied and were accepted into 747 00:41:56,800 --> 00:42:00,240 the program in the last two years, 75% of them have 748 00:42:00,240 --> 00:42:04,000 served on a panel. So the vast majority of people who 749 00:42:04,000 --> 00:42:06,200 are applying are serving on a panel. 750 00:42:06,200 --> 00:42:08,440 The wait time is relatively little. 751 00:42:08,440 --> 00:42:12,160 But we have that 25, almost 25% who haven't served yet. 752 00:42:12,160 --> 00:42:15,440 And I think that's largely due to the fact that there 753 00:42:15,440 --> 00:42:17,800 is a very large oversubscription of early career 754 00:42:17,800 --> 00:42:18,960 reviewers in certain fields. 755 00:42:18,960 --> 00:42:23,240 So, for example, there are some areas such as in social 756 00:42:23,240 --> 00:42:26,760 sciences, population sciences, where we have as many as 757 00:42:26,760 --> 00:42:30,280 300 early career reviewers in our database for those 758 00:42:30,280 --> 00:42:34,160 study sections in that area. When we're only recrewing 759 00:42:34,160 --> 00:42:37,280 two early career reviewers per standing panel, there's 760 00:42:37,280 --> 00:42:41,560 a lot of people who are waiting and who probably will 761 00:42:41,560 --> 00:42:45,440 not get the opportunity to serve as are reviewer before 762 00:42:45,440 --> 00:42:46,240 they're tenured. 763 00:42:46,240 --> 00:42:49,280 So I think that's where the large wait time is. 764 00:42:49,280 --> 00:42:53,400 It refers to oversubscription in certain fields. 765 00:42:53,400 --> 00:42:56,120 >> DR. BRUCE REED: Thanks. 766 00:42:56,120 --> 00:42:57,320 Mark. 767 00:42:57,320 --> 00:42:59,840 >> DR. MARK PEIFER: Thanks for that update on the ECR 768 00:42:59,840 --> 00:43:00,040 program. 769 00:43:00,040 --> 00:43:02,920 You guys have made remarkable progress on that. 770 00:43:02,920 --> 00:43:06,400 And as somebody who played a small role in that, that's 771 00:43:06,400 --> 00:43:07,360 really -- that's nice. 772 00:43:07,960 --> 00:43:11,600 And just another kudos, right, for continuing to follow 773 00:43:11,600 --> 00:43:13,600 the impact of the pandemic. 774 00:43:13,600 --> 00:43:17,480 And I'm confident you will continue to do that and 775 00:43:17,480 --> 00:43:19,800 continue to report back to us. 776 00:43:19,800 --> 00:43:23,280 Again, I'm surprised but pleased to see the application 777 00:43:23,280 --> 00:43:27,880 and thus far taken a big hit but, again, I think it 778 00:43:27,880 --> 00:43:31,720 will be important to follow that out for several years. 779 00:43:31,720 --> 00:43:32,960 But thank you for doing that. 780 00:43:32,960 --> 00:43:38,320 >> DR. BRUCE REED: Thanks, Mark. 781 00:43:38,320 --> 00:43:43,240 Edda. 782 00:43:43,240 --> 00:43:47,320 >> Dr. SPIEKERKOETTER: I had a question reporting bias 783 00:43:47,320 --> 00:43:53,600 in the CSR. Do you have any more data on how many PIs 784 00:43:53,600 --> 00:43:57,640 form kind of this -- or, like, contact Gabriel, how 785 00:43:57,640 --> 00:44:02,120 many other reviewers, who is the outcome? Because I 786 00:44:02,120 --> 00:44:06,640 feel often people feel they have applied and that maybe 787 00:44:06,640 --> 00:44:11,560 found some kind of bias feel a little bit helpless that 788 00:44:11,560 --> 00:44:16,080 they don't really know where to turn to and whether 789 00:44:16,080 --> 00:44:20,560 they will be successful and whether they are hurt and 790 00:44:20,560 --> 00:44:21,440 all this. 791 00:44:21,960 --> 00:44:24,080 So do you have a little bit more information on that? 792 00:44:24,080 --> 00:44:28,240 >> DR. NONI BYRNES: Yeah. Unfortunately, we haven't 793 00:44:28,240 --> 00:44:32,760 had a lot of traffic into that mailbox since we -- we 794 00:44:32,760 --> 00:44:37,720 established it a year ago in March of 2021. We have, 795 00:44:37,720 --> 00:44:41,040 however, gotten more bias reporting or other concerns 796 00:44:41,040 --> 00:44:44,760 raised through the SRO or through our chiefs or 797 00:44:44,760 --> 00:44:47,240 management, not through the box, necessarily. 798 00:44:47,560 --> 00:44:49,840 We are getting a few. 799 00:44:49,840 --> 00:44:53,360 I don't have the numbers yet. I would simply -- I 800 00:44:53,360 --> 00:44:56,880 don't know if anyone -- I don't know if Gabriel is on, 801 00:44:56,880 --> 00:45:01,360 but it's very small. I would say a dozen to 12 to 15 802 00:45:01,360 --> 00:45:03,920 that we've actually done anything or gotten any 803 00:45:03,920 --> 00:45:04,880 significant action on. 804 00:45:05,160 --> 00:45:08,320 I want to be -- that's why I'm trying to publicize it 805 00:45:08,320 --> 00:45:11,440 more. We will do it through social media and others, 806 00:45:11,440 --> 00:45:14,320 but we would appreciate your help. We have taken 807 00:45:14,320 --> 00:45:17,160 action and when I have some more numbers, hopefully in 808 00:45:17,160 --> 00:45:19,440 a future council meeting, I'll report the specific 809 00:45:19,440 --> 00:45:20,320 types of actions. 810 00:45:20,680 --> 00:45:24,840 We have other division directors here who can tell you 811 00:45:24,840 --> 00:45:27,800 anecdotally, but we don't have numbers yet. 812 00:45:27,800 --> 00:45:30,480 >> DR. BRUCE REED: Thanks. 813 00:45:30,480 --> 00:45:32,000 Matt. 814 00:45:32,000 --> 00:45:34,160 >> DR. MATTHEW CARPENTER: Thank you so much. 815 00:45:34,160 --> 00:45:35,400 Wonderful presentation. 816 00:45:35,400 --> 00:45:38,840 I think I saw or heard the statistic that there are 817 00:45:38,840 --> 00:45:39,440 20,000 reviewers. 818 00:45:39,440 --> 00:45:43,280 And I was wondering what kind of data you keep on the 819 00:45:43,280 --> 00:45:46,760 quality of these reviewers. I know when I served and 820 00:45:46,760 --> 00:45:49,320 chaired study sections, the vast majority of reviewers 821 00:45:49,320 --> 00:45:52,480 were outstanding. There was a small minority who were 822 00:45:52,480 --> 00:45:52,800 not. 823 00:45:52,800 --> 00:45:56,840 And I'm wondering how you keep track of us. 824 00:45:56,840 --> 00:45:58,960 >> DR. NONI BYRNES: Okay. 825 00:45:58,960 --> 00:46:02,560 I'll invite anybody to jump in, but let me just tell 826 00:46:02,560 --> 00:46:06,200 you that we don't have a central way of keeping track 827 00:46:06,200 --> 00:46:07,160 of 20,000 reviewers. 828 00:46:07,160 --> 00:46:10,840 We have SROs who are very active and will not invite 829 00:46:10,840 --> 00:46:14,840 back people who are not strong. We are -- I do think 830 00:46:14,840 --> 00:46:18,520 as we bring -- as we broaden the pool of reviewers that 831 00:46:18,520 --> 00:46:21,520 we're actively doing and bringing new people into the 832 00:46:21,520 --> 00:46:24,200 system, it's really important to have strong reviewer 833 00:46:24,200 --> 00:46:25,880 training and reviewer feedback loop. 834 00:46:26,160 --> 00:46:29,760 And I think that's what our office of training is 835 00:46:29,760 --> 00:46:33,680 targeting -- is trying to target to do that in a much 836 00:46:33,680 --> 00:46:34,920 more systemic way. 837 00:46:34,920 --> 00:46:36,080 >> DR. BRUCE REED: Yeah. 838 00:46:36,080 --> 00:46:42,080 A couple of thoughts. You know, so you can 839 00:46:42,080 --> 00:46:46,880 differentiate between reviewers who are probably not so 840 00:46:46,880 --> 00:46:52,840 good and those who are, shall we say, like integrity 841 00:46:52,840 --> 00:46:57,640 challenged in that the integrity challenged ones, we 842 00:46:57,640 --> 00:47:02,440 act vigorously to block from further review service. 843 00:47:02,440 --> 00:47:07,560 But the other question, as Noni said, we've really just 844 00:47:07,560 --> 00:47:10,640 begun to think about this systemically. 845 00:47:10,640 --> 00:47:16,080 So how could we really evaluate reviewers at scale? 846 00:47:16,080 --> 00:47:19,920 As you say, 20,000 people say how do you do this in a 847 00:47:19,920 --> 00:47:22,280 way that would provide us with useful feedback. 848 00:47:22,280 --> 00:47:26,640 And it's not simply a matter of trying to screen out 849 00:47:26,640 --> 00:47:28,640 good ones and bad ones. 850 00:47:28,640 --> 00:47:32,680 It's try to link it to reviewer training and trying to 851 00:47:32,680 --> 00:47:35,240 identify, for example, scoring behavior that might 852 00:47:35,240 --> 00:47:38,160 benefit from additional training or those kind of 853 00:47:38,160 --> 00:47:38,520 things. 854 00:47:38,520 --> 00:47:43,360 So we don't have it worked out, but it is on our agenda 855 00:47:43,360 --> 00:47:47,080 to give that much more careful thought about how we 856 00:47:47,080 --> 00:47:50,440 should be, you know, as you say, tracking, but 857 00:47:50,440 --> 00:47:53,040 evaluating reviewers with an eye wards improving 858 00:47:53,040 --> 00:47:53,760 performance overall. 859 00:47:53,760 --> 00:47:57,560 >> DR. NONI BYRNES: And let me follow up really 860 00:47:57,560 --> 00:48:00,280 quickly, Matthew, is that the strategic framework that 861 00:48:00,280 --> 00:48:03,720 I showed, reviewer evaluation is very much a part of 862 00:48:03,720 --> 00:48:04,080 that. 863 00:48:04,080 --> 00:48:08,280 But you didn't see that highlighted and that's because 864 00:48:08,280 --> 00:48:12,920 as Bruce said, we're just starting those efforts and it 865 00:48:12,920 --> 00:48:16,640 is parts of our strategic plan, as well. 866 00:48:16,640 --> 00:48:18,560 >> DR. BRUCE REED: Tonya. 867 00:48:18,560 --> 00:48:21,480 >> DR. TONYA PALERMO: Thanks sort so much for the 868 00:48:21,480 --> 00:48:21,720 update. 869 00:48:21,720 --> 00:48:25,160 I appreciate the efforts contributed over the last few 870 00:48:25,160 --> 00:48:25,520 years. 871 00:48:25,520 --> 00:48:30,400 I had a question about the bias sort of reporting and 872 00:48:30,400 --> 00:48:33,920 wondered if you have gotten feedback from the 873 00:48:33,920 --> 00:48:37,040 community and from applicants specifically about what 874 00:48:37,040 --> 00:48:41,000 they think warrants, you know, an email to Gabriel. 875 00:48:41,000 --> 00:48:44,640 Are they conceptualizing only the most egregious forms 876 00:48:44,640 --> 00:48:49,640 of biases fitting that or do they see the spectrum and 877 00:48:49,640 --> 00:48:52,800 understand that that is available to them. 878 00:48:52,800 --> 00:48:56,320 My gut is they think it's for egregious forms of bias 879 00:48:56,320 --> 00:48:58,880 and that they're probably not understanding the full 880 00:48:58,880 --> 00:49:01,120 spectrum of what we're hoping for there. 881 00:49:01,120 --> 00:49:03,960 But I'm curious if you've gotten some feedback from the 882 00:49:03,960 --> 00:49:04,840 community about that. 883 00:49:04,840 --> 00:49:06,840 >> DR. NONI BYRNES: We have not. 884 00:49:06,840 --> 00:49:12,240 I do know that what we have gotten, that we've taken 885 00:49:12,240 --> 00:49:16,680 action on has been pretty egregious and pretty obvious. 886 00:49:16,680 --> 00:49:21,120 You know, I'm sure others can weigh in. 887 00:49:21,120 --> 00:49:24,280 I want to invite any of the division directors to 888 00:49:24,280 --> 00:49:27,400 unmute and join in because they're the ones who really 889 00:49:27,400 --> 00:49:28,680 handle this with Gabriel. 890 00:49:28,680 --> 00:49:32,680 But, yeah, it's been pretty egregious. I don't think 891 00:49:32,680 --> 00:49:36,320 we've had a lot of subtle bias things submitted. 892 00:49:36,320 --> 00:49:39,840 >> DR. BRUCE REED: One thing I want to make sure 893 00:49:39,840 --> 00:49:42,800 people are clear on, anybody who has a concern about 894 00:49:42,800 --> 00:49:45,160 bias, there's multiple avenues through which they can 895 00:49:45,160 --> 00:49:48,720 report. You know, so the bias mailbox is one, but we 896 00:49:48,720 --> 00:49:51,960 actually get a lot of reports directly to the SROs or 897 00:49:51,960 --> 00:49:54,320 to the chief or to the division director. 898 00:49:54,320 --> 00:49:56,600 Those are perfectly fine. 899 00:49:56,600 --> 00:49:59,720 We absolutely act on them that way, too. 900 00:49:59,720 --> 00:50:02,000 So we get them through a variety of channels. 901 00:50:02,000 --> 00:50:04,400 >> DR. NONI BYRNES: And we're actually tracking all of 902 00:50:04,400 --> 00:50:05,280 that centrally, as well. 903 00:50:05,280 --> 00:50:09,360 So that in the future, I can report sort of the scope 904 00:50:09,360 --> 00:50:13,440 and scale of what we got and the types of reports we 905 00:50:13,440 --> 00:50:13,800 got. 906 00:50:13,800 --> 00:50:22,120 >> DR. BRUCE REED: Michelle. 907 00:50:22,120 --> 00:50:23,520 >> DR. MICHELLE JANELSINS: Hi, Noni. 908 00:50:23,520 --> 00:50:24,840 Great updates all around. 909 00:50:24,840 --> 00:50:28,160 I have a question about the logistics for the 910 00:50:28,160 --> 00:50:31,440 in-person, going back to in-person meetings and Seth up 911 00:50:31,440 --> 00:50:35,480 to the one in-person meeting a year. I was wondering 912 00:50:35,480 --> 00:50:38,800 if you're going to give flexibility to study sections 913 00:50:38,800 --> 00:50:42,480 or if they have a meeting in September in-person, will 914 00:50:42,480 --> 00:50:45,760 it be anticipated next September they would also meet 915 00:50:45,760 --> 00:50:49,440 in-person or is there flexibility given or will it be 916 00:50:49,440 --> 00:50:49,800 random? 917 00:50:49,800 --> 00:50:53,800 So sometimes will study section might meet in the 918 00:50:53,800 --> 00:50:58,680 winter one year or it might meet in the spring or 919 00:50:58,680 --> 00:51:00,480 summer the following year. 920 00:51:00,480 --> 00:51:02,760 I'm wondering if you can comment on that. 921 00:51:02,760 --> 00:51:06,240 Also if you've heard kind of any thoughts about people 922 00:51:06,240 --> 00:51:09,040 avoiding winter months or any context like that. 923 00:51:09,040 --> 00:51:09,320 Thanks. 924 00:51:09,320 --> 00:51:13,360 >> DR. NONI BYRNES: So thanks, Michelle. I will say 925 00:51:13,360 --> 00:51:16,040 yes, we've heard thoughts about avoiding the winter 926 00:51:16,040 --> 00:51:16,720 months, absolutely. 927 00:51:16,720 --> 00:51:19,120 So I'll answer that. 928 00:51:19,120 --> 00:51:21,920 And then I'll turn it to Bruce because he's been -- 929 00:51:21,920 --> 00:51:24,440 we're just starting to grapple with this and figuring 930 00:51:24,440 --> 00:51:26,880 out how we're going to deal with this. 931 00:51:26,880 --> 00:51:31,520 And I believe we're going to start with surveys with a 932 00:51:31,520 --> 00:51:32,800 lot of flexibility. 933 00:51:32,800 --> 00:51:34,120 Bruce. 934 00:51:34,120 --> 00:51:39,520 >> DR. BRUCE REED: CRS will be surveying both 935 00:51:39,520 --> 00:51:43,280 reviewers and SROs looking at their preferences. 936 00:51:43,280 --> 00:51:45,760 And I have to say, Michelle, your question is well, 937 00:51:45,760 --> 00:51:46,760 you're a step ahead. 938 00:51:46,760 --> 00:51:49,280 We're just trying to get some meetings into place here 939 00:51:49,280 --> 00:51:51,800 and trying to look at a year at a time. 940 00:51:51,800 --> 00:51:54,600 Ultimately, we do have to figure out those questions 941 00:51:54,600 --> 00:51:55,520 that you asked. 942 00:51:55,520 --> 00:51:59,600 We have heard -- it's a fairly common concern about the 943 00:51:59,600 --> 00:52:02,880 winter months. You know, weather problems aren't 944 00:52:02,880 --> 00:52:07,760 unique to the winter, but we run into more of them. 945 00:52:07,760 --> 00:52:11,440 Also you can think public health issues might be 946 00:52:11,440 --> 00:52:14,520 greater in the winter than other seasons. 947 00:52:14,520 --> 00:52:19,800 At this point, we're trying to space out this first 948 00:52:19,800 --> 00:52:25,600 batch and this is very much a learning as we go 949 00:52:25,600 --> 00:52:26,160 process. 950 00:52:26,160 --> 00:52:28,160 But thanks. 951 00:52:28,160 --> 00:52:28,440 Christine. 952 00:52:28,440 --> 00:52:33,000 >> DR. CHRISTINE HENDON: Hi, Noni. Thank you for the 953 00:52:33,000 --> 00:52:33,360 overview. 954 00:52:33,360 --> 00:52:37,000 I had a follow-up question regarding logistics for 955 00:52:37,000 --> 00:52:40,160 in-person meetings. Will the in-person meetings 956 00:52:40,160 --> 00:52:44,240 primarily be in the D.C./Maryland area or will they 957 00:52:44,240 --> 00:52:46,040 rotate around the country? 958 00:52:46,040 --> 00:52:51,560 And then the other was prepandemic, we've always had 959 00:52:51,560 --> 00:52:55,840 some reviewers participate, you know, via phone. 960 00:52:55,840 --> 00:53:00,640 Will that continue? 961 00:53:00,640 --> 00:53:04,120 >> DR. NONI BYRNES: So I think we haven't made any 962 00:53:04,120 --> 00:53:07,000 decision about only he holding these in the D.C. area. 963 00:53:07,000 --> 00:53:09,600 I suspect that we will allow meetings elsewhere. 964 00:53:09,600 --> 00:53:12,600 We haven't quite worked out a hotel contracting and all 965 00:53:12,600 --> 00:53:14,360 of that is now 3-year-old information. 966 00:53:14,360 --> 00:53:17,160 Not even sure those contracts are in place. 967 00:53:17,160 --> 00:53:20,800 And then participation by phone, yeah, we will allow 968 00:53:20,800 --> 00:53:22,800 that as we always have. 969 00:53:22,800 --> 00:53:25,760 And, you know, the hybrid option is available in the 970 00:53:25,760 --> 00:53:28,680 future. We know how to do Zoom pretty well. 971 00:53:28,680 --> 00:53:32,920 We just need to work out how to management equity of 972 00:53:32,920 --> 00:53:33,280 participation. 973 00:53:33,280 --> 00:53:38,440 Just this first round is when we're not doing it. 974 00:53:38,440 --> 00:53:40,960 >> DR. BRUCE REED: And Elizabeth. 975 00:53:40,960 --> 00:53:44,120 >> DR. ELIZABETH VILLA: Hi. Noni, this is a little 976 00:53:44,120 --> 00:53:46,960 bit out of left field because it's not related to this. 977 00:53:46,960 --> 00:53:50,320 But thank you for following up on the women. I am 978 00:53:50,320 --> 00:53:52,840 still not surprised, but very happy that women continue 979 00:53:52,840 --> 00:53:53,960 to do well regardless. 980 00:53:53,960 --> 00:53:57,680 That doesn't mean that it wasn't awful. 981 00:53:57,680 --> 00:54:01,200 But I also want to ask about -- we didn't talk about 982 00:54:01,200 --> 00:54:04,680 this and I didn't ask you before, but what is the 983 00:54:04,680 --> 00:54:07,880 status for ESIs with the pandemic? Because it's not 984 00:54:07,880 --> 00:54:08,360 done. 985 00:54:08,360 --> 00:54:13,000 And so how will their status might be extended upon 986 00:54:13,000 --> 00:54:17,640 request or anything if they can -- you know, if they 987 00:54:17,640 --> 00:54:22,240 ask for an extension or something due to care taking 988 00:54:22,240 --> 00:54:25,480 responsibilities or the like during the pandemic? 989 00:54:25,720 --> 00:54:27,040 >> DR. NONI BYRNES: So I don't know. 990 00:54:27,040 --> 00:54:29,320 I'll have Bruce and Kristin if they have heard anything 991 00:54:29,320 --> 00:54:30,000 to the contrary. 992 00:54:30,000 --> 00:54:33,480 This is outside of my lane. 993 00:54:33,480 --> 00:54:34,640 >> DR. ELIZABETH VILLA: I realize. 994 00:54:34,640 --> 00:54:36,320 >> DR. NONI BYRNES: I'm just not aware. 995 00:54:36,320 --> 00:54:39,600 But I'm happy to touch back with OER and follow up with 996 00:54:39,600 --> 00:54:40,120 this group. 997 00:54:40,120 --> 00:54:40,760 >> DR. ELIZABETH VILLA: Thanks. 998 00:54:40,760 --> 00:54:44,560 >> DR. NONI BYRNES: And to just ask him to publicize 999 00:54:44,560 --> 00:54:45,520 whatever they're planning -- 1000 00:54:45,520 --> 00:54:50,040 >> DR. RAY JACOBSON: Also, I know this is not CSR 1001 00:54:50,040 --> 00:54:53,440 business, but since this is an official channel, I 1002 00:54:53,440 --> 00:54:56,840 think in general the CSI status might need some 1003 00:54:56,840 --> 00:55:00,200 revision from NIH based on when they started their 1004 00:55:00,200 --> 00:55:03,240 position rather than when they finished or terminal 1005 00:55:03,240 --> 00:55:04,720 degree or other things. 1006 00:55:04,720 --> 00:55:05,960 >> DR. NONI BYRNES: Noted. 1007 00:55:05,960 --> 00:55:08,800 >> DR. ELIZABETH VILLA: I know it's not CSR 1008 00:55:08,800 --> 00:55:09,680 >> DR. 1009 00:55:09,680 --> 00:55:11,680 NONI BYRNES: I'm happy to pass along. 1010 00:55:11,680 --> 00:55:13,160 We've done that before for this group. 1011 00:55:13,160 --> 00:55:15,720 >> DR. ELIZABETH VILLA: Thank you. 1012 00:55:15,720 --> 00:55:21,280 >> DR. BRUCE REED: I just got a note from -- to the 1013 00:55:21,280 --> 00:55:25,600 effect saying ES extensions are still in place. I 1014 00:55:25,600 --> 00:55:30,280 can't tell you what the details are on that, but I 1015 00:55:30,280 --> 00:55:33,720 think OER is aware of the issue, anyway. 1016 00:55:34,000 --> 00:55:36,720 And we will try to get you more information. 1017 00:55:36,720 --> 00:55:40,360 >> DR. ELIZABETH VILLA: I found some interesting where 1018 00:55:40,360 --> 00:55:44,440 it says you can't get both an extension for COVID and 1019 00:55:44,440 --> 00:55:47,760 having a child, you know, because young parents won't 1020 00:55:47,760 --> 00:55:49,960 get affected by the pandemic somehow. 1021 00:55:49,960 --> 00:55:53,520 But, yeah, some interesting discussions there. 1022 00:55:53,520 --> 00:55:58,440 It would be very nice to pass along to whoever needs to 1023 00:55:58,440 --> 00:56:00,480 hear it. Thank you. 1024 00:56:00,480 --> 00:56:03,840 >> DR. NONI BYRNES: I will. 1025 00:56:03,840 --> 00:56:08,240 >> Lynn Yee: One thing to add to that topic, the 1026 00:56:08,240 --> 00:56:11,440 physician science have a different experience in the 1027 00:56:11,440 --> 00:56:14,240 pandemic, researchers who were pulled away from 1028 00:56:14,240 --> 00:56:17,840 research to provide care in the hospital might be 1029 00:56:17,840 --> 00:56:21,440 another reason to consider how to modify the ESI 1030 00:56:21,440 --> 00:56:21,840 status. 1031 00:56:21,840 --> 00:56:29,440 >> DR. ELIZABETH VILLA: Absolutely. 1032 00:56:29,440 --> 00:56:31,480 >> DR. BRUCE REED: That's a very good point. 1033 00:56:31,480 --> 00:56:36,240 I'm going to take advantage to move to the next item on 1034 00:56:36,240 --> 00:56:39,440 the agenda because we're right on time here. 1035 00:56:39,440 --> 00:56:42,680 So thanks, everybody, for that discussion. 1036 00:56:42,680 --> 00:56:49,120 The next item up here is introduction to the CSR 1037 00:56:49,120 --> 00:56:50,440 communications office. 1038 00:56:50,440 --> 00:56:54,200 This is going to be a presentation by Kristin Kramer. 1039 00:56:54,200 --> 00:56:57,080 So I will turn this over to Kristin. 1040 00:56:57,080 --> 00:57:02,560 >> DR. KRISTIN KRAMER: All right. Thank you very much. 1041 00:57:02,560 --> 00:57:05,000 I appreciate it, Bruce. 1042 00:57:05,000 --> 00:57:14,640 Let me share my slides. 1043 00:57:14,640 --> 00:57:27,920 >> DR. NONI BYRNES: Yep. I see them. 1044 00:57:27,920 --> 00:57:33,520 El >> DR. KRISTIN KRAMER: So today I want to give you 1045 00:57:33,520 --> 00:57:36,960 an update on what the communications and outreach 1046 00:57:36,960 --> 00:57:38,680 initiatives office is doing. 1047 00:57:38,680 --> 00:57:41,360 Our broad goals are to increase transparency and 1048 00:57:41,360 --> 00:57:44,400 facilitate stakeholder input. There are a number of 1049 00:57:44,400 --> 00:57:47,080 things we hope to gain by doing this. 1050 00:57:47,080 --> 00:57:50,280 First, we hope to level the playing field by equal 1051 00:57:50,280 --> 00:57:51,240 access to information. 1052 00:57:51,240 --> 00:57:55,400 When I say this, I'm thinking about the fact that some 1053 00:57:55,400 --> 00:57:58,040 investigators have access essentially to an informal 1054 00:57:58,040 --> 00:58:01,040 network like colleagues in their department, in their 1055 00:58:01,040 --> 00:58:04,080 hallway who have NIH funding and experience surveying 1056 00:58:04,080 --> 00:58:05,560 on NIH review panels. 1057 00:58:05,560 --> 00:58:07,880 But not everyone has that network to draw upon. 1058 00:58:07,880 --> 00:58:11,200 So by increasing outreach to ensure that investigators 1059 00:58:11,200 --> 00:58:14,080 understand the review process, we can potentially 1060 00:58:14,080 --> 00:58:17,840 decrease the importance of coming center from a network 1061 00:58:17,840 --> 00:58:20,320 or institution that has NIH experience. 1062 00:58:20,320 --> 00:58:23,080 By increasing transparency in terms of around our 1063 00:58:23,080 --> 00:58:26,240 policies and procedures, we can increase trust in the 1064 00:58:26,240 --> 00:58:30,040 process and in CSR and NIH. We can capitalize on 1065 00:58:30,040 --> 00:58:33,160 stakeholder input. We can gain from their perspective 1066 00:58:33,160 --> 00:58:36,640 and experiences to make improvements, be it in the peer 1067 00:58:36,640 --> 00:58:39,760 review process or even in our own workplace culture. 1068 00:58:39,760 --> 00:58:43,120 So our audiences are Congressional staff, external 1069 00:58:43,120 --> 00:58:46,920 scientific community, those within NIH but outside of 1070 00:58:46,920 --> 00:58:50,720 CSR and our own staff inside of CSR. 1071 00:58:50,720 --> 00:59:00,520 Each of these audiences can be further subdivided. 1072 00:59:00,520 --> 00:59:02,640 Example, the external scientific community, we might 1073 00:59:02,640 --> 00:59:05,040 think about them in terms of investigators, reviewers 1074 00:59:05,040 --> 00:59:06,520 and office of sponsored research. 1075 00:59:06,520 --> 00:59:11,760 So today I'm going to focus our work on trying to reach 1076 00:59:11,760 --> 00:59:13,480 the external scientific community. 1077 00:59:13,480 --> 00:59:18,600 We use a range of communication tools to reach these 1078 00:59:18,600 --> 00:59:19,080 audiences. 1079 00:59:19,080 --> 00:59:27,840 Social media, Twitter and LinkedIn. 1080 00:59:27,840 --> 00:59:35,280 We give presentation that's are aimed at large 1081 00:59:35,280 --> 00:59:36,200 audiences. 1082 00:59:36,200 --> 00:59:39,040 In the past, we have handled facilitated discussions 1083 00:59:39,040 --> 00:59:40,440 with the external community. 1084 00:59:40,440 --> 00:59:43,760 Then this last one, direct email campaigns and direct 1085 00:59:43,760 --> 00:59:47,120 replies, this is simple and low tech and time 1086 00:59:47,120 --> 00:59:50,800 consuming, but I think this is one of the most 1087 00:59:50,800 --> 00:59:52,680 important things my office does. 1088 00:59:52,680 --> 00:59:55,480 We respond to every email we get and I think that 1089 00:59:55,480 --> 00:59:57,280 one-on-one interaction is important for building trust. 1090 00:59:57,280 --> 01:00:02,000 It gives us a lot of detailed information we can use in 1091 01:00:02,000 --> 01:00:04,360 terms of understanding the audience's needs. 1092 01:00:04,360 --> 01:00:07,880 So first I'm going to talk a little bit about review 1093 01:00:07,880 --> 01:00:08,200 matters. 1094 01:00:08,200 --> 01:00:09,320 This is our blog. 1095 01:00:09,320 --> 01:00:13,800 These three posts are our top posts in terms of page 1096 01:00:13,800 --> 01:00:14,200 views. 1097 01:00:14,200 --> 01:00:18,960 Raise and peer review was posted by Noni Byrnes in 1098 01:00:18,960 --> 01:00:19,440 2020. 1099 01:00:19,440 --> 01:00:21,320 Should we keep meeting this way. 1100 01:00:21,320 --> 01:00:25,040 And then our current post that is up calls for comment 1101 01:00:25,040 --> 01:00:26,400 on our strategic plan. 1102 01:00:26,400 --> 01:00:30,560 So you can get a sense hereof on our typical page 1103 01:00:30,560 --> 01:00:30,960 views. 1104 01:00:30,960 --> 01:00:33,560 The top one has about 9,000 page views. 1105 01:00:33,560 --> 01:00:36,960 Since the beginning of March. 1106 01:00:36,960 --> 01:00:40,200 That's the cumulative page views. 1107 01:00:40,200 --> 01:00:45,400 When we look at the analytics, sometimes the office 1108 01:00:45,400 --> 01:00:51,200 cross posts our blog so we see another increase and 1109 01:00:51,200 --> 01:00:56,440 then a gradual increase in additional page views over 1110 01:00:56,440 --> 01:00:57,000 time. 1111 01:00:57,000 --> 01:01:00,680 The blog is valuable for allowing us to get comments 1112 01:01:00,680 --> 01:01:02,880 and feedback from the external community. 1113 01:01:02,880 --> 01:01:06,400 So we typically publish comments that are directly 1114 01:01:06,400 --> 01:01:11,240 relevant to the post that we made, but we receive many 1115 01:01:11,240 --> 01:01:13,000 comments that are tangental. 1116 01:01:13,000 --> 01:01:18,040 If we don't publish those tangental comments, we read 1117 01:01:18,600 --> 01:01:19,160 them. 1118 01:01:19,160 --> 01:01:23,120 We're fortunate is to have a great team that helps us 1119 01:01:23,120 --> 01:01:26,360 collect those comments and synthesize them in a way 1120 01:01:26,360 --> 01:01:28,880 that we can take action on them. 1121 01:01:28,880 --> 01:01:32,120 This blog has about 121,000 subscribers. 1122 01:01:32,120 --> 01:01:34,640 That clearly does not translate into page views. 1123 01:01:34,640 --> 01:01:36,200 So there is room for improvement. 1124 01:01:36,200 --> 01:01:40,520 We probably need to kind to have clean up our 1125 01:01:40,520 --> 01:01:44,440 subscription list and think about a different way to 1126 01:01:44,440 --> 01:01:46,160 develop a subscriber base. 1127 01:01:46,160 --> 01:01:48,840 The comments often suggest we're reaching a different 1128 01:01:48,840 --> 01:01:51,880 audience with the blog versus the audience we reach 1129 01:01:51,880 --> 01:01:54,880 through Twitter. For example, many of the comments 1130 01:01:54,880 --> 01:01:57,240 refer to a long history with NIH. 1131 01:01:57,240 --> 01:02:01,320 To me, that suggests the importance of using multiple 1132 01:02:01,320 --> 01:02:04,480 means of communications and multiple social media 1133 01:02:04,480 --> 01:02:08,080 platforms. Because even though I subdivided our 1134 01:02:08,080 --> 01:02:10,800 external community audience into investigators and 1135 01:02:10,800 --> 01:02:14,400 reviewers and offices of responsive research, some of 1136 01:02:14,400 --> 01:02:17,120 those offices can be further subdivided. 1137 01:02:17,120 --> 01:02:21,240 So we might think more about how we use social media 1138 01:02:21,240 --> 01:02:24,960 and diversify so we can reach all of our audiences. 1139 01:02:24,960 --> 01:02:28,480 We need to use analytics to better understanding who 1140 01:02:28,480 --> 01:02:32,720 our blog is reaching and to increase our reach of this 1141 01:02:32,720 --> 01:02:33,120 blog. 1142 01:02:33,120 --> 01:02:35,880 So Twitter can be used for a number of things. 1143 01:02:35,880 --> 01:02:38,680 It's obviously useful for sharing information. So the 1144 01:02:38,680 --> 01:02:42,440 tweet on the right is just a tweet about our new chair 1145 01:02:42,440 --> 01:02:44,320 orientation that we do every year. 1146 01:02:44,320 --> 01:02:49,240 So when we train our chairs, we publish a video of that 1147 01:02:49,240 --> 01:02:52,520 training to be transparent so that the external 1148 01:02:52,520 --> 01:02:56,240 committee knows what advice we're giving to our chairs 1149 01:02:56,240 --> 01:02:59,520 and how we respect meetings to be run. 1150 01:02:59,520 --> 01:03:02,320 The tweet on the left seems minor, but it happens to 1151 01:03:02,320 --> 01:03:04,880 away tweet that had a lot of eyes on it. 1152 01:03:04,880 --> 01:03:07,960 And this came about just because I saw on Twitter that 1153 01:03:07,960 --> 01:03:11,040 there was a lot of concern about having a figure such 1154 01:03:11,040 --> 01:03:13,000 as a flow chart and specific aims. 1155 01:03:13,000 --> 01:03:16,920 Investigators had found that they were getting warnings 1156 01:03:16,920 --> 01:03:21,360 when they submitted grants with a figure on specific 1157 01:03:21,360 --> 01:03:21,840 aims. 1158 01:03:21,840 --> 01:03:25,360 Even though ERA make public all the history around 1159 01:03:25,360 --> 01:03:28,480 warnings and errors and what triggers them, it's 1160 01:03:28,480 --> 01:03:31,600 difficult for the external community to find that 1161 01:03:31,600 --> 01:03:31,960 information. 1162 01:03:31,960 --> 01:03:36,720 So I was able to find that end tweet and provide some 1163 01:03:36,720 --> 01:03:39,520 assurance to investigators that their grants would 1164 01:03:39,520 --> 01:03:43,880 still move forward with review even if it had a figure 1165 01:03:43,880 --> 01:03:45,440 in the specific aims. 1166 01:03:45,440 --> 01:03:49,680 And I think we can work on this so that we understand 1167 01:03:49,680 --> 01:03:53,160 what the pain points are each round, what questions are 1168 01:03:53,160 --> 01:03:57,040 they getting repeatedly and we can kind of head it off 1169 01:03:57,040 --> 01:03:59,840 and start communications in advance of grant deadlines 1170 01:03:59,840 --> 01:04:01,960 to reduce some of those problems. 1171 01:04:01,960 --> 01:04:04,680 We can use the input that we get through Twitter to 1172 01:04:04,680 --> 01:04:05,400 improve our processes. 1173 01:04:05,400 --> 01:04:08,880 So this tweet is from someone who really likes the Zoom 1174 01:04:08,880 --> 01:04:11,720 format, but pointed out that the meetings he had 1175 01:04:11,720 --> 01:04:15,200 participated in had gone really long. 14 to 16 hours. 1176 01:04:15,200 --> 01:04:18,680 And we heard this from reviewers through other means, 1177 01:04:18,680 --> 01:04:19,480 as well. 1178 01:04:19,480 --> 01:04:23,360 So in response to this, we issued guidance to SROs to 1179 01:04:23,360 --> 01:04:26,880 remind them to keep the review meetings to a normal 1180 01:04:26,880 --> 01:04:30,440 length of day instead of trying to take advantage of 1181 01:04:30,440 --> 01:04:33,240 the fact that people didn't necessarily need travel 1182 01:04:33,240 --> 01:04:33,600 time. 1183 01:04:33,600 --> 01:04:36,440 I think we can use Twitter to instill trust. 1184 01:04:36,440 --> 01:04:41,600 So this tweet came about because an investigator 1185 01:04:41,600 --> 01:04:46,120 tweeted about a grant that is withdrawn. 1186 01:04:46,120 --> 01:04:47,520 She felt we made a mistake. 1187 01:04:47,520 --> 01:04:51,160 So we reached out to her directly, we looked at the 1188 01:04:51,160 --> 01:04:54,840 details and we realized that we had, in fact, made an 1189 01:04:54,840 --> 01:04:55,200 error. 1190 01:04:55,200 --> 01:04:58,440 We felt that it was important to tweet in response to 1191 01:04:58,440 --> 01:05:01,360 her tweet and make sure that people in the community 1192 01:05:01,360 --> 01:05:04,000 understood that we know we're not infallible, that we 1193 01:05:04,000 --> 01:05:07,520 do make mistakes and when we find out that we made a 1194 01:05:07,520 --> 01:05:10,760 mistake, we do everything we can to address it in a 1195 01:05:10,760 --> 01:05:11,360 timely fashion. 1196 01:05:11,360 --> 01:05:14,960 We also wanted to make sure the external community knew 1197 01:05:14,960 --> 01:05:18,560 there was an appeals process and they could contact the 1198 01:05:18,560 --> 01:05:21,440 division of receipt and referral directly, that they 1199 01:05:21,440 --> 01:05:25,040 didn't have to rely on making me see something on 1200 01:05:25,040 --> 01:05:25,400 Twitter. 1201 01:05:25,400 --> 01:05:29,480 So this tweet came about because I saw a conversation 1202 01:05:29,480 --> 01:05:33,600 on Twitter about bias and NIH funding and bias and 1203 01:05:33,600 --> 01:05:34,000 review. 1204 01:05:34,000 --> 01:05:38,200 Dr. Watson wasn't necessarily aiming his tweet at CSR 1205 01:05:38,200 --> 01:05:43,320 or asking for action, but I took the time to read 1206 01:05:43,320 --> 01:05:45,200 through his summary statements. 1207 01:05:45,200 --> 01:05:48,320 I admit that I didn't fully understand what the issues 1208 01:05:48,320 --> 01:05:52,040 were. So I contacted him via email and we had a 1209 01:05:52,040 --> 01:05:52,680 lengthy exchange. 1210 01:05:52,680 --> 01:05:56,320 I think it was really helpful to basically sharpen my 1211 01:05:56,320 --> 01:05:57,440 eye for bias. 1212 01:05:57,440 --> 01:06:00,240 And I think it was helpful in terms of thinking about 1213 01:06:00,240 --> 01:06:02,000 the content and the bias awareness training. 1214 01:06:02,000 --> 01:06:05,120 One of the things that was great about this, this is 1215 01:06:05,120 --> 01:06:07,400 before we had a reporting avenue for bias. 1216 01:06:07,400 --> 01:06:12,080 He tweeted that, a, we were receptive, so that is 1217 01:06:12,080 --> 01:06:12,520 great. 1218 01:06:12,520 --> 01:06:16,000 He also pointed out people in the community could reach 1219 01:06:16,000 --> 01:06:19,840 out and talk to their SRO if they were concerned about 1220 01:06:19,840 --> 01:06:20,880 bias and review. 1221 01:06:20,880 --> 01:06:24,720 So our websites also provide a great way to give 1222 01:06:24,720 --> 01:06:26,640 detailed information to our communities. 1223 01:06:26,640 --> 01:06:29,760 Our current website is set up with the applicant 1224 01:06:29,760 --> 01:06:31,480 community in mind for reviewers. 1225 01:06:31,480 --> 01:06:34,800 And then there's other high level menu items for news 1226 01:06:34,800 --> 01:06:37,480 and policy and study sections and the like. 1227 01:06:37,480 --> 01:06:42,520 So each of these connections has more content developed 1228 01:06:42,520 --> 01:06:45,840 specifically for that audience in mind. 1229 01:06:45,840 --> 01:06:50,640 So applicant's channel has information on the 1230 01:06:50,640 --> 01:06:54,760 application process, planning and writing, submission 1231 01:06:54,760 --> 01:06:58,200 and assignment and so forth. 1232 01:06:58,200 --> 01:07:02,320 When we look at the analytics, even though we invest a 1233 01:07:02,320 --> 01:07:06,400 lot of time in those areas, that's not where people go. 1234 01:07:06,400 --> 01:07:09,880 So I want to call your attention to the top bar in 1235 01:07:09,880 --> 01:07:10,480 darker blue. 1236 01:07:10,480 --> 01:07:13,360 These are our page views for the month of February. 1237 01:07:13,360 --> 01:07:17,640 All of our CSR pages combined had a total of about 210 1238 01:07:17,640 --> 01:07:22,280 page views. When we look at the rest of the content on 1239 01:07:22,280 --> 01:07:25,840 the site, what we see is that study sections -- these 1240 01:07:25,840 --> 01:07:29,080 are the description pages for each study section -- has 1241 01:07:29,080 --> 01:07:31,560 far and away the most page views. 1242 01:07:32,000 --> 01:07:36,360 So study section description pages account for about 1243 01:07:36,360 --> 01:07:39,640 120,000 of those 210,000 page views. 1244 01:07:39,640 --> 01:07:42,440 And the other thing we can see from analytics is that 1245 01:07:42,440 --> 01:07:45,000 people are not going to our landing page and browsing 1246 01:07:45,000 --> 01:07:46,760 to get to the study section pages. 1247 01:07:46,760 --> 01:07:51,800 So we spend time thinking about the content that is on 1248 01:07:51,800 --> 01:07:55,440 that landing page, but actually relatively few people 1249 01:07:55,440 --> 01:07:56,360 see it. 1250 01:07:56,360 --> 01:07:57,640 People use Google. 1251 01:07:57,640 --> 01:08:02,440 They Google an acronym for a study section and they go 1252 01:08:02,440 --> 01:08:05,040 directly to that study section page. 1253 01:08:05,040 --> 01:08:11,680 So what that means to us is study section description 1254 01:08:11,680 --> 01:08:14,320 landing pages are cues. 1255 01:08:14,320 --> 01:08:17,400 We need to use those study section pages thoughtfully. 1256 01:08:17,400 --> 01:08:21,160 So we have long had a reporting avenue for breaches in 1257 01:08:21,160 --> 01:08:21,480 integrity. 1258 01:08:21,480 --> 01:08:24,720 But as Noni mentioned, in the last year, we established 1259 01:08:24,720 --> 01:08:27,280 another reporting avenue for concerns about an unfair 1260 01:08:27,280 --> 01:08:28,240 or bias review. 1261 01:08:28,240 --> 01:08:33,240 When we did this, we added a bar on every study section 1262 01:08:33,240 --> 01:08:37,040 page so that people had ready access to information 1263 01:08:37,040 --> 01:08:39,520 about how to report these concerns. 1264 01:08:39,520 --> 01:08:43,920 We can take this a step further and we can think about 1265 01:08:43,920 --> 01:08:47,560 how to thoughtfully drive traffic to other parts of our 1266 01:08:47,560 --> 01:08:47,920 website. 1267 01:08:47,920 --> 01:08:53,600 So this is a screen shot from an SRO add. 1268 01:08:53,600 --> 01:08:57,240 In the month of February, we had about 3,000 page 1269 01:08:57,240 --> 01:08:57,600 views. 1270 01:08:57,600 --> 01:09:00,680 So in thinking about the value of study section 1271 01:09:00,680 --> 01:09:03,760 description pages, we added banners to all the study 1272 01:09:03,760 --> 01:09:06,520 section description pages that would point to this 1273 01:09:06,520 --> 01:09:10,280 advertisement. When we look at the data, we see that 1274 01:09:10,280 --> 01:09:13,680 almost 1300 page views of that add were referred from 1275 01:09:13,680 --> 01:09:17,440 study section pages. In other words, people were on a 1276 01:09:17,440 --> 01:09:20,520 study section page and they went from that study 1277 01:09:20,520 --> 01:09:22,240 section page to our advertisement. 1278 01:09:22,240 --> 01:09:25,880 We can also use analytics to see the impact of 1279 01:09:25,880 --> 01:09:27,360 different forms of outreach. 1280 01:09:27,360 --> 01:09:31,600 So these -- this is a tweet and a post on LinkedIn, 1281 01:09:31,600 --> 01:09:33,640 again, about an SRO advertisement. 1282 01:09:33,640 --> 01:09:36,640 And on the right, you can see the number of page views 1283 01:09:36,640 --> 01:09:37,120 over time. 1284 01:09:37,120 --> 01:09:41,160 And each of those spikes correspond to some effort in 1285 01:09:41,160 --> 01:09:41,560 outreach. 1286 01:09:41,560 --> 01:09:46,320 So we can do a little -- what we can do in the future 1287 01:09:46,320 --> 01:09:51,040 is do better analysis. We can try to reach out via one 1288 01:09:51,040 --> 01:09:55,080 social media stream and see what the impact is and take 1289 01:09:55,080 --> 01:09:59,120 a little bit of time, space it, put some time between 1290 01:09:59,120 --> 01:10:03,120 that tweet, for example, and a post on LinkedIn and see 1291 01:10:03,120 --> 01:10:04,960 how the page views change. 1292 01:10:05,200 --> 01:10:07,960 We've also done webinars and panel discussions. 1293 01:10:07,960 --> 01:10:13,360 This is an Early Career Reviewer webinar that we did 1294 01:10:13,360 --> 01:10:15,480 about three years ago. 1295 01:10:15,480 --> 01:10:19,520 This was a suggestion, we were discussing how we should 1296 01:10:19,520 --> 01:10:22,320 go about increasing the number of underrepresented 1297 01:10:22,320 --> 01:10:25,120 minorities participating in the early reviewer program. 1298 01:10:25,120 --> 01:10:30,200 Her comment was generally it's best to find a way to 1299 01:10:30,200 --> 01:10:33,880 let minority scientists see themselves in that role. 1300 01:10:33,880 --> 01:10:36,600 So we decided to run a panel discussion. 1301 01:10:36,600 --> 01:10:44,400 We publicized it through a range of means. 1302 01:10:44,400 --> 01:10:49,880 Dr. Gabriel Fosu sent information about this webinar 1303 01:10:49,880 --> 01:10:53,320 through networks that he's developed. 1304 01:10:53,320 --> 01:10:57,480 And I also reached out to offices Of Sponsored Research 1305 01:10:57,480 --> 01:10:59,960 and department chairs and minority-serving institutions 1306 01:10:59,960 --> 01:11:03,640 and asked them to share this information with their 1307 01:11:03,640 --> 01:11:04,080 faculty. 1308 01:11:04,080 --> 01:11:06,560 Finally, we used Twitter, as well. 1309 01:11:06,560 --> 01:11:09,840 What we saw was really interesting. 1310 01:11:09,840 --> 01:11:12,640 We did each of these outreach efforts at different 1311 01:11:12,640 --> 01:11:16,320 points in time. We could look at when we did the 1312 01:11:16,320 --> 01:11:16,640 outreach. 1313 01:11:16,640 --> 01:11:21,600 So, for example, when an email was sent to those 1,000 1314 01:11:21,600 --> 01:11:25,640 ESIs and compared it to registrations to participate in 1315 01:11:25,640 --> 01:11:27,880 the early career reviewer program. 1316 01:11:27,880 --> 01:11:32,000 What we found from looking at those data is that 1317 01:11:32,000 --> 01:11:35,720 Twitter was actually the most successful means to drive 1318 01:11:35,720 --> 01:11:38,640 traffic to the early career reviewer program. 1319 01:11:38,640 --> 01:11:42,800 So our webinar results were great. We had over 700 1320 01:11:42,800 --> 01:11:44,280 attendees, over 150 questions. 1321 01:11:44,280 --> 01:11:47,640 We saw an increase in registration in the early career 1322 01:11:47,640 --> 01:11:50,000 reviewer program, often without registering for the 1323 01:11:50,000 --> 01:11:50,680 webinar, itself. 1324 01:11:50,680 --> 01:11:56,440 And we saw an increase in early career reviewers who 1325 01:11:56,440 --> 01:12:00,440 were from groups underrepresented by medical devices. 1326 01:12:00,440 --> 01:12:05,000 So the data on the right side are from may of 2021 when 1327 01:12:05,000 --> 01:12:06,720 we were publicizing this webinar. 1328 01:12:06,720 --> 01:12:11,160 And the data in the left column are from the same time 1329 01:12:11,160 --> 01:12:12,600 span a year prior. 1330 01:12:12,600 --> 01:12:17,560 What you can see is that we had a large increase in the 1331 01:12:17,560 --> 01:12:20,960 number of people accepted to participate in the early 1332 01:12:20,960 --> 01:12:24,360 career reviewer program as well as the number of 1333 01:12:24,360 --> 01:12:26,240 underrepresented minorities who submitted applications 1334 01:12:26,240 --> 01:12:27,400 and were accepted. 1335 01:12:27,400 --> 01:12:32,880 So I'm going to briefly touch on some of the work we're 1336 01:12:32,880 --> 01:12:35,160 doing in communications within NIH. 1337 01:12:35,160 --> 01:12:39,000 Our broad objective is to ensure there's a better 1338 01:12:39,000 --> 01:12:40,720 understanding of peer review. 1339 01:12:40,720 --> 01:12:45,920 I'm thinking about what works well in peer review and 1340 01:12:45,920 --> 01:12:50,560 what doesn't and how we can ensure useful review 1341 01:12:50,560 --> 01:12:51,080 outcomes. 1342 01:12:51,080 --> 01:12:55,240 What CSR has done in that space is established review 1343 01:12:55,240 --> 01:12:58,960 liaisons to ensure useful review outcomes. We also 1344 01:12:58,960 --> 01:13:03,120 know, of course, that program officers spend a lot of 1345 01:13:03,120 --> 01:13:04,760 time talking to investigators. 1346 01:13:04,760 --> 01:13:10,120 And we want to make sure people within NIH are aware of 1347 01:13:10,120 --> 01:13:13,680 CSR initiatives and what our policies and procedures 1348 01:13:13,680 --> 01:13:16,320 are so they can convey that. 1349 01:13:16,320 --> 01:13:20,880 To that end, we've developed a new letter for program 1350 01:13:20,880 --> 01:13:25,400 staff and develop a web page so program staff can 1351 01:13:25,400 --> 01:13:26,800 communicate that investigators. 1352 01:13:26,800 --> 01:13:31,360 We also want to partner with ICs in the future so we 1353 01:13:31,360 --> 01:13:33,240 can collaborate on outreach efforts. 1354 01:13:33,240 --> 01:13:35,920 In many cases, funding institutes are developing 1355 01:13:35,920 --> 01:13:38,640 workshops at meetings of scientific societies and 1356 01:13:38,640 --> 01:13:42,480 instead of CSR going it alone and having our own 1357 01:13:42,480 --> 01:13:45,960 separate talk, it would be probably have a greater 1358 01:13:45,960 --> 01:13:49,040 impact if we participated alongside funding ICs and 1359 01:13:49,040 --> 01:13:51,760 added information on the peer review process. 1360 01:13:51,760 --> 01:13:55,160 And then another objective is to continue increasing 1361 01:13:55,160 --> 01:13:58,960 the diversity of our review panels and to collaborate 1362 01:13:58,960 --> 01:14:01,960 with those inside NIH to do that. 1363 01:14:01,960 --> 01:14:07,240 So one tool that we've developed to that end is 1364 01:14:07,240 --> 01:14:10,400 reviewer recommendation tool through which program 1365 01:14:10,400 --> 01:14:14,640 staff can recommend strong scientists to serve as 1366 01:14:14,640 --> 01:14:15,160 reviewers. 1367 01:14:15,160 --> 01:14:18,240 So generally I think this is an area we need to spend a 1368 01:14:18,240 --> 01:14:19,160 lot more time on. 1369 01:14:19,160 --> 01:14:24,960 Within CSR, in terms of our communications with our own 1370 01:14:24,960 --> 01:14:30,800 staff, I think it's important to remember that our 250 1371 01:14:30,800 --> 01:14:36,000 SROs spend a lot of time talking to reviewers. 1372 01:14:36,000 --> 01:14:39,880 So I think we need to expand in terms of providing 1373 01:14:39,880 --> 01:14:41,640 resources and materials to SROs. 1374 01:14:41,640 --> 01:14:45,560 In terms of engaging with our own staff, I think it's 1375 01:14:45,560 --> 01:14:49,120 important to connect all of our staff to CSR's mission 1376 01:14:49,120 --> 01:14:52,720 and to engage our staff in terms of getting input 1377 01:14:52,720 --> 01:14:54,480 related to policy and practice. 1378 01:14:54,480 --> 01:14:59,840 Also, I think we have an important role to play in 1379 01:14:59,840 --> 01:15:03,720 terms of improving workplace culture in terms of 1380 01:15:03,720 --> 01:15:06,120 diversity, equity, inclusion, accessibility effort, 1381 01:15:06,120 --> 01:15:09,040 hiring transparency, career development and staff 1382 01:15:09,040 --> 01:15:12,920 recognition. So I'll touch on that first bullet. 1383 01:15:13,400 --> 01:15:17,200 This is an example of a resource that we developed for 1384 01:15:17,200 --> 01:15:17,520 SROs. 1385 01:15:17,520 --> 01:15:21,440 So pretty early on in the pandemic, we heard from 1386 01:15:21,440 --> 01:15:25,000 reviewers that there was some concern that SROs maybe 1387 01:15:25,000 --> 01:15:28,520 weren't being empathetic to the level of stress that 1388 01:15:28,520 --> 01:15:29,720 reviewers were feeling. 1389 01:15:29,720 --> 01:15:33,920 So we issued guidance to SROs, which included things 1390 01:15:33,920 --> 01:15:38,640 such as inviting reviewers to let them know if they 1391 01:15:38,640 --> 01:15:40,080 needed reasonable accommodations. 1392 01:15:40,080 --> 01:15:45,800 We asked SROs to explain their thinking to reviewers in 1393 01:15:45,800 --> 01:15:49,240 terms of setting the meeting date. 1394 01:15:49,240 --> 01:15:56,680 We asked, SROs to review the meetings nodes. SROs have 1395 01:15:56,680 --> 01:16:03,400 a welcome letter that they send out to reviewers and 1396 01:16:03,400 --> 01:16:08,760 communications they send out in advance of critique 1397 01:16:08,760 --> 01:16:15,520 deadlines. In a normal world, we recycle dates and 1398 01:16:15,520 --> 01:16:16,160 deadlines. 1399 01:16:16,160 --> 01:16:21,040 I think in the pandemic world, we were continuing to do 1400 01:16:21,040 --> 01:16:21,480 that. 1401 01:16:21,480 --> 01:16:24,200 So those communications really needed to be 1402 01:16:24,200 --> 01:16:27,760 re-evaluated to make sure that they -- the tone was 1403 01:16:27,760 --> 01:16:29,320 appropriate for the time. 1404 01:16:30,040 --> 01:16:34,000 So our plans for the future really are to expand all of 1405 01:16:34,000 --> 01:16:34,680 this work. 1406 01:16:34,680 --> 01:16:39,800 So we need to use -- make broader use of all of our 1407 01:16:39,800 --> 01:16:40,240 platforms. 1408 01:16:40,640 --> 01:16:43,760 I think we need to make better use of data and 1409 01:16:43,760 --> 01:16:46,320 analytics so that we can assess a given campaign. 1410 01:16:46,320 --> 01:16:50,680 And modify it to get a better long-term impact. 1411 01:16:50,680 --> 01:16:58,280 And I think we also need to do better targeting of 1412 01:16:58,280 --> 01:17:03,160 outreach to institutions and institutions not highly 1413 01:17:03,160 --> 01:17:05,200 funded by NIH. 1414 01:17:05,200 --> 01:17:08,480 So these are the principles that Noni mentioned that 1415 01:17:08,480 --> 01:17:09,560 CSR operates with. 1416 01:17:09,560 --> 01:17:12,360 Our research is always specific, intentional and done 1417 01:17:12,360 --> 01:17:14,080 with these principles in mind. 1418 01:17:14,080 --> 01:17:17,040 This third one in particular in terms of multi 1419 01:17:17,040 --> 01:17:19,320 directional communication, that is something that my 1420 01:17:19,320 --> 01:17:20,640 office takes to heart. 1421 01:17:20,640 --> 01:17:22,800 It's the core of our work. 1422 01:17:22,800 --> 01:17:25,000 It's in our responses via email. 1423 01:17:25,000 --> 01:17:28,880 It's in our responses on Twitter and responses to 1424 01:17:28,880 --> 01:17:30,160 people tweeting directly. 1425 01:17:30,160 --> 01:17:34,120 It's really the core of everything we do and it's what 1426 01:17:34,120 --> 01:17:37,720 makes CSR better and it helps make review better, as 1427 01:17:37,720 --> 01:17:38,080 well. 1428 01:17:38,080 --> 01:17:43,320 So before I end, I want to acknowledge those who helped 1429 01:17:43,320 --> 01:17:48,560 carry out this work. Within my office, I have two 1430 01:17:48,560 --> 01:17:51,880 associate directors, Chuck Dunn and Lamont Williams. 1431 01:17:51,880 --> 01:17:53,800 Ann GraHamlin-Hamlin . 1432 01:17:55,080 --> 01:18:04,320 Lynndy-Ann Thai supports all of on our efforts. 1433 01:18:04,320 --> 01:18:07,120 Software development, video production, they do many 1434 01:18:07,120 --> 01:18:07,520 things. 1435 01:18:07,520 --> 01:18:10,800 And then the CSR outreach committee comprises SROs who 1436 01:18:10,800 --> 01:18:14,800 have an interesting in outreach and spend a lot of time 1437 01:18:14,800 --> 01:18:16,640 doing outreach on our behalf. 1438 01:18:16,640 --> 01:18:18,800 That's chaired by Jessica Smith. 1439 01:18:18,800 --> 01:18:24,640 And, of course, Dr. Gabriel Faso who helps connect with 1440 01:18:24,640 --> 01:18:25,800 diverse audiences. 1441 01:18:25,800 --> 01:18:29,880 With that, I'll stop sharing my slides and I'd love to 1442 01:18:29,880 --> 01:18:33,920 have your input on areas I think are missing and ways 1443 01:18:33,920 --> 01:18:35,040 we can expand. 1444 01:18:35,040 --> 01:18:37,720 >> DR. BRUCE REED: Thanks very much, Kristin. 1445 01:18:37,720 --> 01:18:43,480 It occurred to me too late that I should have given 1446 01:18:43,480 --> 01:18:46,080 some context for your talk. 1447 01:18:46,080 --> 01:18:48,360 >> DR. KRISTIN KRAMER: I'm sorry. 1448 01:18:48,360 --> 01:18:49,320 >> DR. BRUCE REED: That's okay. 1449 01:18:49,320 --> 01:18:50,120 I'll do it now. 1450 01:18:50,120 --> 01:18:53,160 For people who are new to council that don't appreciate 1451 01:18:53,160 --> 01:18:57,080 how much of a change this is for CSR, this has been an 1452 01:18:57,080 --> 01:19:00,080 area where we've invested a lot of effort and resources 1453 01:19:00,080 --> 01:19:01,920 in the last couple of years. 1454 01:19:01,920 --> 01:19:06,320 So one of the -- you know, so yes, we like to -- we 1455 01:19:06,320 --> 01:19:09,280 wanted to introduce you to the communications office, 1456 01:19:09,280 --> 01:19:13,320 but as I say, there is a direction that is very 1457 01:19:13,320 --> 01:19:17,400 important to CSR and we wanted to get some of your 1458 01:19:17,400 --> 01:19:21,080 feedback on the efforts of that office and where it's 1459 01:19:21,080 --> 01:19:24,760 headed, as Kristin said. Does it make sense where 1460 01:19:24,760 --> 01:19:25,480 we're headed? 1461 01:19:26,160 --> 01:19:27,360 Can we do better? 1462 01:19:27,360 --> 01:19:28,600 Those kind of things. 1463 01:19:28,600 --> 01:19:30,600 So thanks. 1464 01:19:30,600 --> 01:19:34,640 So Mark. 1465 01:19:34,640 --> 01:19:36,200 >> DR. MARK PEIFER: Mostly just a thank you. 1466 01:19:36,200 --> 01:19:40,360 I really think you guys have made a remarkable amount 1467 01:19:40,360 --> 01:19:42,440 of progress in this area. 1468 01:19:42,440 --> 01:19:46,280 And, you know, the idea of using all these different 1469 01:19:46,280 --> 01:19:49,760 ways of communicating with people I think is actually 1470 01:19:49,760 --> 01:19:52,840 critical, right? Because each person is seeing 1471 01:19:52,840 --> 01:19:56,320 information in different ways, right? So the blog 1472 01:19:56,320 --> 01:20:00,600 might be great for an old person like me, but Twitter 1473 01:20:00,600 --> 01:20:04,440 might be the place that a brand new investigator sees 1474 01:20:04,440 --> 01:20:04,840 things. 1475 01:20:04,840 --> 01:20:08,000 And just a kudo, because I think of all the of the 1476 01:20:08,000 --> 01:20:10,640 branches of NIH, you all are doing this -- you're way 1477 01:20:10,640 --> 01:20:12,480 at the time front of the line. 1478 01:20:12,720 --> 01:20:17,960 That includes any favorite institute, NIGMS. 1479 01:20:17,960 --> 01:20:21,720 They do okay, but it's mostly sort of boring. 1480 01:20:21,720 --> 01:20:23,400 Really boring. 1481 01:20:23,400 --> 01:20:26,920 I really think you guys are really pioneering something 1482 01:20:26,920 --> 01:20:31,200 that's critical. You are the face of NIH for most 1483 01:20:31,200 --> 01:20:31,600 people. 1484 01:20:31,600 --> 01:20:36,440 And even when I, who is on this council, right, 1485 01:20:36,440 --> 01:20:40,760 realizes I don't have really understand what is your 1486 01:20:40,760 --> 01:20:46,080 job and what is Mike Lauer's job, that's way true for 1487 01:20:46,080 --> 01:20:47,040 most people. 1488 01:20:47,040 --> 01:20:50,160 So I think you guys are doing a great job. 1489 01:20:50,160 --> 01:20:53,160 >> DR. BRUCE REED: Thank you, Mark. 1490 01:20:53,160 --> 01:20:54,680 Tonya. 1491 01:20:54,680 --> 01:20:58,480 >> DR. TONYA PALERMO: I completely echo Mark's 1492 01:20:58,480 --> 01:20:58,880 comments. 1493 01:20:58,880 --> 01:21:01,920 It's amazing the changes over the past several years. 1494 01:21:01,920 --> 01:21:06,120 I do a lot of teaching of grant writing and I pull up 1495 01:21:06,120 --> 01:21:08,080 the CSR website all the time. 1496 01:21:08,080 --> 01:21:11,160 Over the past ten years, just the evolution of that 1497 01:21:11,160 --> 01:21:12,360 website is truly amazing. 1498 01:21:12,360 --> 01:21:16,880 But I did wonder, I kind of feel like there's so much 1499 01:21:16,880 --> 01:21:20,640 good stuff tucked in there, how do you drive new 1500 01:21:20,640 --> 01:21:21,400 applicants there? 1501 01:21:21,400 --> 01:21:24,080 I haven't looked enough at -- the NIH website is 1502 01:21:24,080 --> 01:21:24,880 completely overwhelming. 1503 01:21:24,880 --> 01:21:28,440 But I don't know if there's strategic places to drive 1504 01:21:28,440 --> 01:21:31,680 people from there to CSR, especially around, again, all 1505 01:21:31,680 --> 01:21:34,520 of that, how to actually prepare an application 1506 01:21:34,520 --> 01:21:35,960 material that's so wonderful. 1507 01:21:35,960 --> 01:21:38,800 >> DR. KRISTIN KRAMER: So actually revising the 1508 01:21:38,800 --> 01:21:41,360 website and the information hierarchy on it is 1509 01:21:41,360 --> 01:21:43,880 something that I hope to accomplish in 2022. 1510 01:21:43,880 --> 01:21:46,480 Not so much thinking about applicants resources, but 1511 01:21:46,480 --> 01:21:49,440 more thinking about all of the data and evaluations 1512 01:21:49,440 --> 01:21:50,400 that CSR does. 1513 01:21:50,400 --> 01:21:55,400 Those things are really tucked away and is are very 1514 01:21:55,400 --> 01:21:55,920 visible. 1515 01:21:55,920 --> 01:21:59,880 So my goal is to pull that up to be a more top level 1516 01:21:59,880 --> 01:22:00,440 content item. 1517 01:22:00,440 --> 01:22:04,200 And in terms of applicant resources, I think the way he 1518 01:22:04,200 --> 01:22:07,960 we might expand would be doing -- we have plans to film 1519 01:22:07,960 --> 01:22:11,880 a new mock study section, a video that we can post. 1520 01:22:11,880 --> 01:22:15,920 But I think in a lot of cases, we have pointed to OERs 1521 01:22:15,920 --> 01:22:17,760 material on how to prepare grant. 1522 01:22:17,760 --> 01:22:22,720 So I think we're going to do more pointing to OER for 1523 01:22:22,720 --> 01:22:26,000 some of that material rather than duplicating their 1524 01:22:26,000 --> 01:22:26,440 efforts. 1525 01:22:26,440 --> 01:22:32,640 >> DR. BRUCE REED: Michelle. 1526 01:22:32,640 --> 01:22:36,200 >> DR. MICHELLE JANELSINS: Great presentation. I was 1527 01:22:36,200 --> 01:22:39,760 also kind of in awe about the efforts on communication. 1528 01:22:39,760 --> 01:22:42,520 And I think especially, you know, during the pandemic 1529 01:22:42,520 --> 01:22:45,560 when a lot of people have been feeling isolated, you 1530 01:22:45,560 --> 01:22:48,920 know, a little bit less so now, but, you know, it's 1531 01:22:48,920 --> 01:22:51,360 still, you know -- we're still going through tough 1532 01:22:51,360 --> 01:22:51,640 times. 1533 01:22:51,880 --> 01:22:55,800 And so I was actually thinking and it occurred to me 1534 01:22:55,800 --> 01:22:59,720 when you said the sentence that we read and respond to 1535 01:22:59,720 --> 01:23:03,280 every email or message. That could be your banner. 1536 01:23:03,280 --> 01:23:06,440 I think a lot of times, you know, people see blogs and 1537 01:23:06,440 --> 01:23:09,880 they think, oh, if I make a comment, no one is going to 1538 01:23:09,880 --> 01:23:10,640 listen to me. 1539 01:23:10,640 --> 01:23:14,520 But you're really digging deep and, like, reaching 1540 01:23:14,520 --> 01:23:16,960 people on their individual concerns. 1541 01:23:16,960 --> 01:23:21,920 So, you know, thanks for that and I do think that could 1542 01:23:21,920 --> 01:23:26,080 be, like, a possible banner or one of your rotating 1543 01:23:26,080 --> 01:23:29,800 banners. Because I was struck by that comment. 1544 01:23:29,800 --> 01:23:30,680 So thanks. 1545 01:23:30,680 --> 01:23:36,280 And I guess just for a question, you know, do you ask 1546 01:23:36,280 --> 01:23:40,480 for upcoming topics that people might, you know, want, 1547 01:23:40,480 --> 01:23:45,120 you know, to hear from CSR about -- or anything like 1548 01:23:45,120 --> 01:23:45,600 that? 1549 01:23:45,840 --> 01:23:49,360 >> DR. KRISTIN KRAMER: I think that's a good idea for 1550 01:23:49,360 --> 01:23:52,600 the future, but I'll be honest, we have a backlog of 1551 01:23:52,600 --> 01:23:53,800 blogs that we're drafting. 1552 01:23:53,800 --> 01:23:56,280 Right now we have a lot of content to get out. 1553 01:23:56,280 --> 01:23:59,560 But I appreciate your point about being responsive. I 1554 01:23:59,560 --> 01:24:02,800 feel like that is terribly important to do to respond 1555 01:24:02,800 --> 01:24:03,480 to people. 1556 01:24:03,480 --> 01:24:09,960 NIH is such a behemoth and it's difficult to navigate. 1557 01:24:09,960 --> 01:24:13,400 A lot of times, I just -- I want to make sure that we 1558 01:24:13,400 --> 01:24:14,320 respond to everyone. 1559 01:24:14,320 --> 01:24:17,520 And we might not be the right person to answer the 1560 01:24:17,520 --> 01:24:17,800 question. 1561 01:24:17,800 --> 01:24:20,960 We might be referring them to OER or some other group 1562 01:24:20,960 --> 01:24:24,080 at NIH, but it's really hard to figure out who to 1563 01:24:24,080 --> 01:24:24,360 contact. 1564 01:24:24,360 --> 01:24:27,240 So I really try to respond. 1565 01:24:27,240 --> 01:24:29,800 >> DR. BRUCE REED: Leo. 1566 01:24:29,800 --> 01:24:30,760 >> DR. LEOPOLDO CABASSA:. 1567 01:24:30,760 --> 01:24:34,360 I wanted to -- Michelle got my comment, the response is 1568 01:24:34,360 --> 01:24:38,320 fascinating to see and that I am so glad that your 1569 01:24:38,320 --> 01:24:41,960 office and your group is doing that and being very 1570 01:24:41,960 --> 01:24:43,040 thoughtful about that. 1571 01:24:43,280 --> 01:24:46,800 So I would highly recommend to continue to do that best 1572 01:24:46,800 --> 01:24:47,760 as you can. 1573 01:24:47,760 --> 01:24:50,920 The only other component I saw that was fascinated and 1574 01:24:50,920 --> 01:24:53,480 I'm glad that you're using data analytics here. 1575 01:24:53,480 --> 01:24:57,000 So where people are landing on the page and really 1576 01:24:57,000 --> 01:24:59,840 targeting information where people are most going. 1577 01:24:59,840 --> 01:25:03,360 Because I think that is a really important piece for 1578 01:25:03,360 --> 01:25:06,880 all these NIH websites which are enormous to navigate. 1579 01:25:06,880 --> 01:25:08,640 So I really appreciate that. 1580 01:25:08,640 --> 01:25:11,360 And excellent work on doing that. 1581 01:25:11,360 --> 01:25:14,960 Looking forward to see how this evolves and how you're 1582 01:25:14,960 --> 01:25:17,440 using those analytics to continue to improve. 1583 01:25:17,440 --> 01:25:19,600 So excellent job. 1584 01:25:19,600 --> 01:25:24,600 >> DR. BRUCE REED: Thanks, Leo. 1585 01:25:24,600 --> 01:25:27,040 Jinming. 1586 01:25:27,040 --> 01:25:30,160 >> DR. JINMING GAO: Kristin, I want to echo the 1587 01:25:30,160 --> 01:25:32,160 positive sentiments from all the council members. 1588 01:25:32,160 --> 01:25:37,200 And I really want to thank part of this process is 1589 01:25:37,200 --> 01:25:40,880 communicating what study section do SRO, you know, 1590 01:25:40,880 --> 01:25:44,120 responsibility to the outside community and really 1591 01:25:44,120 --> 01:25:48,720 appreciate the inside CSR so that we can see how 1592 01:25:48,720 --> 01:25:53,320 thoughtful you are in terms of thinking about the tones 1593 01:25:53,320 --> 01:25:57,000 and transparency and then explain the thinking process. 1594 01:25:57,000 --> 01:26:00,480 And I think it's easy for the council member to 1595 01:26:00,480 --> 01:26:03,240 appreciate, you know, how much thought and effort 1596 01:26:03,240 --> 01:26:04,280 you've gone through. 1597 01:26:04,280 --> 01:26:08,480 But at the same time, I think SROs, you know, those are 1598 01:26:08,480 --> 01:26:11,240 the counterparts of the reviewers inside of CSR. 1599 01:26:11,240 --> 01:26:15,960 How do we get the -- I guess, you know, words out so 1600 01:26:15,960 --> 01:26:19,080 that the outside reviewers, you know, the scientists 1601 01:26:19,080 --> 01:26:23,000 appreciates how much of the SROs actually are doing all 1602 01:26:23,000 --> 01:26:24,680 this behind the scene. 1603 01:26:24,680 --> 01:26:28,080 I think this is probably very important to -- I think, 1604 01:26:28,080 --> 01:26:32,480 you know, for many of us who want to pick up a phone 1605 01:26:32,480 --> 01:26:35,880 and complain, there's so much work behind the scene and 1606 01:26:35,880 --> 01:26:39,280 to appreciate all the efforts and the thought process I 1607 01:26:39,280 --> 01:26:43,000 think is good to let the outside world know what, you 1608 01:26:43,000 --> 01:26:44,360 know, you are doing. 1609 01:26:44,640 --> 01:26:45,720 >> DR. KRISTIN KRAMER: Yeah. 1610 01:26:45,720 --> 01:26:52,440 We recently had an article on the role of an SRO and 1611 01:26:52,440 --> 01:26:53,000 biospace. 1612 01:26:53,000 --> 01:26:56,520 I think there's a place to do more of that sort of 1613 01:26:56,520 --> 01:26:59,720 outreach to let people know what the role is of SROs 1614 01:26:59,720 --> 01:27:01,760 and to make sure that they're approachable. 1615 01:27:01,760 --> 01:27:05,080 We've tried to do some things in the past to humanize 1616 01:27:05,080 --> 01:27:08,360 CSRs such as with one of our revamps of our website. 1617 01:27:08,360 --> 01:27:11,480 We made photos of SROs a much larger and prominent 1618 01:27:11,480 --> 01:27:14,280 position on the study section description pages. That 1619 01:27:14,280 --> 01:27:17,400 was done with the goal of really humanizing, you know, 1620 01:27:17,400 --> 01:27:21,120 CSR so that is there is a person you can contact and 1621 01:27:21,120 --> 01:27:21,760 it's clear. 1622 01:27:21,760 --> 01:27:24,720 But I think we have more work to do in that space. 1623 01:27:24,720 --> 01:27:28,840 >> DR. JINMING GAO: Thank you. 1624 01:27:28,840 --> 01:27:30,760 >> DR. BRUCE REED: Thanks. 1625 01:27:30,760 --> 01:27:33,480 Narasimhan. 1626 01:27:33,480 --> 01:27:36,400 >> DR. NARASIMHAN RAJARAM: I just wanted to echo what 1627 01:27:36,400 --> 01:27:38,040 Mark and the others have said. 1628 01:27:38,040 --> 01:27:40,360 Great job on a lot of this stuff that you presented. 1629 01:27:40,360 --> 01:27:45,440 I am fairly active on Twitter and I see -- it's always 1630 01:27:45,440 --> 01:27:50,120 nice to see CSR responding to people who have comments 1631 01:27:50,120 --> 01:27:51,040 or comments. 1632 01:27:51,400 --> 01:27:54,360 It humanizes the agency and let's people know that 1633 01:27:54,360 --> 01:27:57,320 someone is reading their tweets and they're just not 1634 01:27:57,320 --> 01:27:59,320 sending this off into thin air. 1635 01:27:59,320 --> 01:28:02,880 I really like the data driven approach where you 1636 01:28:02,880 --> 01:28:06,480 figured out that putting stuff on the -- directly on 1637 01:28:06,480 --> 01:28:10,080 the study section pages was more impactful than having 1638 01:28:10,080 --> 01:28:14,440 it on the -- I realized that I never have visited the 1639 01:28:14,440 --> 01:28:16,040 front page of CSR. 1640 01:28:16,320 --> 01:28:20,120 I do exactly what you said, study section, name, put 1641 01:28:20,120 --> 01:28:22,760 that into Google and go with that. 1642 01:28:22,760 --> 01:28:26,920 >> DR. KRISTIN KRAMER: I can see it in the analytics. 1643 01:28:26,920 --> 01:28:30,600 >> DR. NARASIMHAN RAJARAM: I think that the -- given 1644 01:28:30,600 --> 01:28:34,280 how seamless, I was going to say this after Noni 1645 01:28:34,280 --> 01:28:37,240 finished her talk, just given how seamless the 1646 01:28:37,240 --> 01:28:40,160 transition was from in-person to virtual during the 1647 01:28:40,160 --> 01:28:43,840 pandemic, I don't know if there is -- maybe there is 1648 01:28:43,840 --> 01:28:47,520 here, but I don't know if there is enough appreciation 1649 01:28:47,520 --> 01:28:50,840 for how monumental the task was and especially given 1650 01:28:50,840 --> 01:28:54,520 that in some other countries that the review process is 1651 01:28:54,520 --> 01:28:55,600 completely shut down. 1652 01:28:56,080 --> 01:28:57,880 >> DR. KRISTIN KRAMER: Yeah. 1653 01:28:57,880 --> 01:29:02,320 >> DR. NARASIMHAN RAJARAM: So I know Mark posted on 1654 01:29:02,320 --> 01:29:05,120 Twitter, but reiterating that fact, how many 1655 01:29:05,120 --> 01:29:08,760 applications were reviewed and the fact that we didn't 1656 01:29:08,760 --> 01:29:10,400 drop a single cycle. 1657 01:29:10,400 --> 01:29:15,320 Not in sort of beating our own trumpet, but mostly as a 1658 01:29:15,320 --> 01:29:15,720 fact. 1659 01:29:15,720 --> 01:29:16,440 >> DR. KRISTIN KRAMER: Yeah. 1660 01:29:16,440 --> 01:29:18,800 I agree. 1661 01:29:18,800 --> 01:29:24,840 >> DR. BRUCE REED: Thanks, Matt. 1662 01:29:24,840 --> 01:29:26,440 >> DR. MATTHEW CARPENTER: Wonderful presentation. 1663 01:29:26,440 --> 01:29:28,600 I applaud your interactivity. 1664 01:29:28,600 --> 01:29:29,760 So thank you very much. 1665 01:29:29,760 --> 01:29:34,600 My question is about how to assist investigators from 1666 01:29:34,600 --> 01:29:39,960 finding the right study section. You know, the example 1667 01:29:39,960 --> 01:29:44,240 has been given about Googling one specific thing. 1668 01:29:44,240 --> 01:29:47,280 I'm guilty of that, as well. When you do that, you 1669 01:29:47,280 --> 01:29:48,560 don't know what you're missing. 1670 01:29:48,560 --> 01:29:49,320 >> DR. KRISTIN KRAMER: Yeah. 1671 01:29:49,320 --> 01:29:53,840 >> DR. MATTHEW CARPENTER: I know that my research 1672 01:29:53,840 --> 01:29:58,800 posted one of two study sections. I don't know what 1673 01:29:58,800 --> 01:30:02,400 else is -- my options, one you're alphabetical, I'm 1674 01:30:02,400 --> 01:30:07,800 wondering if there is a way we can help key words, that 1675 01:30:07,800 --> 01:30:09,160 kind of thing. 1676 01:30:09,400 --> 01:30:11,640 >> DR. KRISTIN KRAMER: It's just a referral tool. 1677 01:30:11,640 --> 01:30:12,760 And that's the name for it. 1678 01:30:12,760 --> 01:30:16,320 It's not -- I think a very intuitive name. 1679 01:30:16,920 --> 01:30:21,520 We tried calls it the guided study section selector and 1680 01:30:21,520 --> 01:30:26,120 that didn't take off because our own staff already knew 1681 01:30:26,120 --> 01:30:27,480 it as ART. 1682 01:30:27,480 --> 01:30:30,640 But, yeah, on the landing page is a place where you can 1683 01:30:30,640 --> 01:30:33,560 search. You can put in your specific names and title 1684 01:30:33,560 --> 01:30:35,920 and abstract. It will tell you recommended study 1685 01:30:35,920 --> 01:30:38,280 sections based on the last three rounds of referral 1686 01:30:38,280 --> 01:30:38,560 data. 1687 01:30:38,560 --> 01:30:41,040 >> DR. MATTHEW CARPENTER: That's interesting that you 1688 01:30:41,040 --> 01:30:42,160 can enter your abstract. 1689 01:30:42,160 --> 01:30:43,600 That's fascinating. 1690 01:30:43,600 --> 01:30:46,640 I think I just found what you're talking about. In 1691 01:30:46,640 --> 01:30:49,920 fact, maybe it could be a little more clear. Because I 1692 01:30:49,920 --> 01:30:52,960 typed in one word not knowing I can type in an 1693 01:30:52,960 --> 01:30:53,240 abstract. 1694 01:30:53,240 --> 01:30:54,280 Well done. 1695 01:30:54,280 --> 01:30:56,560 >> DR. KRISTIN KRAMER: So we need to publicize that 1696 01:30:56,560 --> 01:30:56,760 more. 1697 01:30:56,760 --> 01:30:59,680 That's again where we can probably use Twitter better 1698 01:30:59,680 --> 01:31:02,600 in advance of grant deadlines, sending out more tweets 1699 01:31:02,600 --> 01:31:05,840 saying hey, if you're trying to figure out what you 1700 01:31:05,840 --> 01:31:09,080 should put in, your assignment request form, go to this 1701 01:31:09,080 --> 01:31:12,680 tool or we can provide guidance on how to use NIH 1702 01:31:12,680 --> 01:31:13,640 reporter and matchmaker. 1703 01:31:13,640 --> 01:31:16,920 That is another great way to identify study sections. 1704 01:31:16,920 --> 01:31:17,920 >> DR. BRUCE REED: All right. 1705 01:31:17,920 --> 01:31:21,640 I'm going to take Christine and then Lynn and then 1706 01:31:21,640 --> 01:31:26,120 we're going to close out and move on to the next topic. 1707 01:31:26,120 --> 01:31:26,440 Christine. 1708 01:31:26,440 --> 01:31:29,160 >> DR. CHRISTINE HENDON: Thank you again, Kristin, to 1709 01:31:29,160 --> 01:31:30,520 you and your wonderful team. 1710 01:31:30,520 --> 01:31:34,600 I had a question regarding outreach to investigators, 1711 01:31:34,600 --> 01:31:37,680 primarily undergraduate institutions such as liberal 1712 01:31:37,680 --> 01:31:40,240 arts colleges or women's colleges. 1713 01:31:40,240 --> 01:31:42,560 So could you comment on that? 1714 01:31:42,560 --> 01:31:46,760 >> DR. KRISTIN KRAMER: We haven't done a lot of that, 1715 01:31:46,760 --> 01:31:49,880 I'll say. Most of our targeted outreach to 1716 01:31:49,880 --> 01:31:52,000 universities has been looking at minority-serving 1717 01:31:52,000 --> 01:31:54,440 institutions or institutions that are basically R-15 1718 01:31:54,440 --> 01:31:56,520 eligible and reaching out to those. 1719 01:31:56,520 --> 01:32:00,720 So I think that the way we've been finding contacts 1720 01:32:00,720 --> 01:32:02,800 would have excluded primarily undergraduate 1721 01:32:02,800 --> 01:32:06,560 institutions because the way I've started out is by 1722 01:32:06,560 --> 01:32:09,920 looking at institutions that have had some interactions 1723 01:32:09,920 --> 01:32:14,080 with NIH, but not a lot of interaction with NIH. 1724 01:32:14,080 --> 01:32:14,960 >> DR. CHRISTINE HENDON: I see. 1725 01:32:14,960 --> 01:32:20,800 >> DR. KRISTIN KRAMER: So I'm not picking up a lot of 1726 01:32:20,800 --> 01:32:23,480 schools that haven't submitted fellowships or 1727 01:32:23,480 --> 01:32:25,280 applications or research grants. 1728 01:32:25,280 --> 01:32:28,040 So that's an area that we could potentially expand to. 1729 01:32:28,040 --> 01:32:29,640 >> DR. CHRISTINE HENDON: Thank you. 1730 01:32:29,640 --> 01:32:31,800 >> DR. BRUCE REED: And Lynn. 1731 01:32:31,800 --> 01:32:40,240 >> LYNN YEE: I can the shared interest and overlap 1732 01:32:40,240 --> 01:32:49,520 section is incredibly helpful. I had no idea about the 1733 01:32:49,520 --> 01:32:52,040 ART tool, either. 1734 01:32:52,040 --> 01:32:56,080 And so it's kind of cool to see this. And I wonder if 1735 01:32:56,080 --> 01:32:58,960 the shared interest in overlap page could be also sort 1736 01:32:58,960 --> 01:33:02,120 of advertised or -- I mean could be a place to link 1737 01:33:02,120 --> 01:33:02,880 that tool. 1738 01:33:02,880 --> 01:33:08,120 >> DR. KRISTIN KRAMER: That's a good idea, yep. We 1739 01:33:08,120 --> 01:33:09,440 can do that. 1740 01:33:09,440 --> 01:33:10,480 >> DR. BRUCE REED: All right. 1741 01:33:10,480 --> 01:33:12,120 Thanks, everyone. 1742 01:33:12,120 --> 01:33:15,840 Thanks, Kristin and thanks for the discussion, 1743 01:33:15,840 --> 01:33:16,360 everybody. 1744 01:33:16,360 --> 01:33:16,880 Okay. 1745 01:33:16,880 --> 01:33:22,160 So looking ahead at the agenda, the next two items are 1746 01:33:22,160 --> 01:33:24,080 reports of ENQUIRE reviews. 1747 01:33:24,080 --> 01:33:28,720 I'm going to have Ross Shonat present ENQUIRE cluster 1748 01:33:28,720 --> 01:33:34,880 14 and then we're going to take a break around 3:00 and 1749 01:33:34,880 --> 01:33:39,520 we'll come back and Ray Jacobson will present ENQUIRE 1750 01:33:39,520 --> 01:33:40,520 Cluster 10. 1751 01:33:40,520 --> 01:33:44,640 So just to make sure that everyone has a basic 1752 01:33:44,640 --> 01:33:48,280 understanding of what we're talking about when we say 1753 01:33:48,280 --> 01:33:52,800 ENQUIRE, let me show one slide and give you a very 1754 01:33:52,800 --> 01:33:56,040 brief background on what the ENQUIRE process is. 1755 01:33:56,040 --> 01:33:59,080 So ENQUIRE is an acronym. 1756 01:33:59,080 --> 01:34:03,240 And I couldn't even begin to give you its derivation 1757 01:34:03,240 --> 01:34:03,680 now. 1758 01:34:03,680 --> 01:34:09,120 But think of it as a label for the process that CSR 1759 01:34:09,120 --> 01:34:13,200 uses to address the basic challenge of keeping our 1760 01:34:13,200 --> 01:34:16,840 study sections aligned with the progress of science. 1761 01:34:16,840 --> 01:34:21,800 So as scientific ideas and methods evolve as topics get 1762 01:34:21,800 --> 01:34:26,800 more and less popular, we have to change our study 1763 01:34:26,800 --> 01:34:27,320 sections. 1764 01:34:27,320 --> 01:34:32,160 So that they're manageable and so that they provide a 1765 01:34:32,160 --> 01:34:35,040 good review context for every application. 1766 01:34:35,040 --> 01:34:42,040 So the way we do this is that we take our hundred -- 1767 01:34:42,040 --> 01:34:47,280 170 or standing study sections and we cluster them. 1768 01:34:47,560 --> 01:34:54,560 We put them into clusters of scientifically related 1769 01:34:54,560 --> 01:34:56,280 study sections. 1770 01:34:56,280 --> 01:35:00,560 Roughly about a dozen in each, but can go bigger than 1771 01:35:00,560 --> 01:35:00,960 that. 1772 01:35:00,960 --> 01:35:03,440 We put them into clusters. 1773 01:35:03,440 --> 01:35:06,160 We run each of these clusters through the ENQUIRE 1774 01:35:06,160 --> 01:35:06,440 process. 1775 01:35:06,440 --> 01:35:09,440 This is a fairly time and resource-intensive process. 1776 01:35:09,440 --> 01:35:12,280 We do about four of them a year. 1777 01:35:12,280 --> 01:35:15,560 And over the course of five years, that means we will 1778 01:35:15,560 --> 01:35:18,840 get through all of our study sections and then we will 1779 01:35:18,840 --> 01:35:20,640 go back and do it again. 1780 01:35:20,640 --> 01:35:24,800 So the process works in this way. 1781 01:35:24,800 --> 01:35:33,960 It starts by -- it starts with -- come on. 1782 01:35:34,520 --> 01:35:36,040 There we go. 1783 01:35:36,040 --> 01:35:39,560 So we submit the cluster to an external scientific 1784 01:35:39,560 --> 01:35:40,360 working group. 1785 01:35:40,360 --> 01:35:44,920 These are -- this is a relatively small group of 1786 01:35:44,920 --> 01:35:46,440 scientifically broad scientists. 1787 01:35:46,720 --> 01:35:50,600 They're not representatives of individual study 1788 01:35:50,600 --> 01:35:51,240 sections. 1789 01:35:51,240 --> 01:35:54,280 They're more broad thinkers in the field. 1790 01:35:54,280 --> 01:35:59,640 We asked them to consider the scientific that is 1791 01:35:59,640 --> 01:36:05,600 reviewed in that cluster of study sections and to give 1792 01:36:05,600 --> 01:36:11,600 us the best set of study sections that should cover 1793 01:36:11,600 --> 01:36:13,960 that area of science. 1794 01:36:13,960 --> 01:36:16,480 So they work out a list of study sections and 1795 01:36:16,480 --> 01:36:17,000 preliminary descriptions. 1796 01:36:17,000 --> 01:36:20,920 We give that to on internal trans-NIH working group. 1797 01:36:20,920 --> 01:36:25,880 These are senior-level IC and OD stakeholders. 1798 01:36:25,880 --> 01:36:29,680 It's also senior CSR leadership. 1799 01:36:29,680 --> 01:36:33,600 And we ask this group to basically start with the 1800 01:36:33,600 --> 01:36:37,120 recommendations of the external group and see if they 1801 01:36:37,120 --> 01:36:41,040 concur or if they have concerns about how this would 1802 01:36:41,040 --> 01:36:42,600 play out in review. 1803 01:36:42,600 --> 01:36:48,120 Based on the input of those two groups, we make a 1804 01:36:48,120 --> 01:36:52,120 preliminary list of study sections that should handle 1805 01:36:52,120 --> 01:36:53,120 that cluster. 1806 01:36:53,120 --> 01:36:55,080 And that is where we are today. 1807 01:36:55,080 --> 01:36:58,520 We bring this to CSR council. 1808 01:36:58,520 --> 01:37:02,600 We ask you to look at it and we want your high level 1809 01:37:02,600 --> 01:37:03,520 review of these. 1810 01:37:03,520 --> 01:37:08,080 Assuming that we get approval, we then take that set of 1811 01:37:08,080 --> 01:37:09,320 study sections forward. 1812 01:37:09,320 --> 01:37:11,440 We do some more intense work on them. 1813 01:37:11,440 --> 01:37:16,040 We do mock-sorts to look at the numbers very carefully 1814 01:37:16,040 --> 01:37:20,160 and we refine the descriptions of the study sections. 1815 01:37:20,160 --> 01:37:24,560 Eventually heading for implementation about six months 1816 01:37:24,560 --> 01:37:27,680 down the road from there. 1817 01:37:27,680 --> 01:37:34,200 So that is where we are today. 1818 01:37:34,200 --> 01:37:40,760 And so with that, I am going to turn this over to Ross, 1819 01:37:40,760 --> 01:37:47,360 who is going to give you -- who is going to talk to you 1820 01:37:47,360 --> 01:37:49,080 about Cluster 14. 1821 01:37:49,080 --> 01:37:50,440 So Ross. 1822 01:37:50,440 --> 01:37:53,280 >> DR. ROSS SHONAT: Thank you, Bruce. 1823 01:37:53,280 --> 01:37:59,080 So if you can see my slides, I'm going to be presenting 1824 01:37:59,080 --> 01:38:02,920 Cluster 14, a series of study sections addressing 1825 01:38:02,920 --> 01:38:05,840 microbiology and infectious diseases For background, 1826 01:38:05,840 --> 01:38:09,680 there are ten study sections in this cluster. 1827 01:38:09,680 --> 01:38:14,600 Four of focused on a bacterial pathogens, two in 1828 01:38:14,600 --> 01:38:19,520 virology, two largely in the area of tropical medicine 1829 01:38:19,520 --> 01:38:23,920 and some pathogen agnostic study sections related to 1830 01:38:23,920 --> 01:38:27,720 post events and vaccinations on microbial diseases. 1831 01:38:27,720 --> 01:38:31,360 You'll see in parentheses, approximately the number of 1832 01:38:31,360 --> 01:38:35,000 applications that that study section has been seeing 1833 01:38:35,000 --> 01:38:38,160 over the last three or four rounds. 1834 01:38:38,160 --> 01:38:41,200 So this is the external workgroup members. 1835 01:38:41,200 --> 01:38:44,200 It was chaired by Avery August. 1836 01:38:44,200 --> 01:38:48,360 Michelle was our CSR observer and I'm sure would be 1837 01:38:48,360 --> 01:38:51,680 happy to provide some context about what happened 1838 01:38:51,680 --> 01:38:52,920 during this meeting. 1839 01:38:52,920 --> 01:38:56,160 So here are the study section working group was -- 1840 01:38:56,160 --> 01:38:59,400 looked at these ten study sections and began by 1841 01:38:59,400 --> 01:39:01,560 thinking about some of the overarching 1842 01:39:01,560 --> 01:39:03,920 considerations that they wanted to address. 1843 01:39:03,920 --> 01:39:06,560 I'll list a few of them here. 1844 01:39:06,560 --> 01:39:10,960 They largely were asked the question about how study 1845 01:39:10,960 --> 01:39:12,920 sections should be organized. 1846 01:39:12,920 --> 01:39:17,480 And so a lot of them are focused narrowly on a 1847 01:39:17,480 --> 01:39:18,320 particular pathogen. 1848 01:39:18,320 --> 01:39:24,160 But science has become much more multidisciplinary. 1849 01:39:24,160 --> 01:39:27,680 Some sections are become overcrowded, particularly 1850 01:39:27,680 --> 01:39:30,640 those focused on viral research. 1851 01:39:30,640 --> 01:39:38,600 They felt the need to better focus tropical, and noted 1852 01:39:38,600 --> 01:39:45,800 there were no obvious places to investigate microbe or 1853 01:39:45,800 --> 01:39:48,160 poly microbe interactions. 1854 01:39:48,160 --> 01:39:50,640 With that, I want to go through their recommendations 1855 01:39:50,640 --> 01:39:51,200 by group. 1856 01:39:51,200 --> 01:39:55,080 And I'll start with what they thought about the 1857 01:39:55,080 --> 01:39:56,360 bacterial pathogen group. 1858 01:39:56,360 --> 01:40:01,040 So as you see, there were four study sections in this 1859 01:40:01,040 --> 01:40:02,760 group. PCMB, BACP. 1860 01:40:02,760 --> 01:40:05,840 The acronyms probably have little meaning, but what 1861 01:40:05,840 --> 01:40:08,960 they effectively did was reconfigured some of these, 1862 01:40:08,960 --> 01:40:12,440 move some of the guidelines around and I'll explain 1863 01:40:12,440 --> 01:40:12,840 that. 1864 01:40:12,840 --> 01:40:18,880 So they came up with prokaryotic biology, which was 1865 01:40:18,880 --> 01:40:24,240 looking at the bacterial pathogen from a pathogen 1866 01:40:24,240 --> 01:40:30,280 perspective and BV was middle ground, BHI was bacterial 1867 01:40:30,280 --> 01:40:31,600 host interactions. 1868 01:40:31,600 --> 01:40:37,000 And then they created a brand new study called IMII, 1869 01:40:37,000 --> 01:40:39,720 interspecies microbial interactions and infections. 1870 01:40:39,720 --> 01:40:41,920 I'll have more to say about that in a moment. 1871 01:40:41,920 --> 01:40:47,240 And you may have noticed that we were running a large 1872 01:40:47,240 --> 01:40:50,640 overflow, bacterial SEP, and those applications were 1873 01:40:50,640 --> 01:40:53,520 effectively dissolved and redistributed around these 1874 01:40:53,520 --> 01:40:55,000 new study sections. 1875 01:40:55,000 --> 01:40:59,400 In the area of -- in the viral pathogen group, we were 1876 01:40:59,400 --> 01:41:03,440 running two study sections. You can see they are 1877 01:41:03,440 --> 01:41:03,840 overloaded. 1878 01:41:04,200 --> 01:41:07,960 Our typical size for a study section is around 60 to 1879 01:41:07,960 --> 01:41:11,000 80. So the external working group effectively crafted 1880 01:41:11,000 --> 01:41:13,040 three study sections out of two. 1881 01:41:13,040 --> 01:41:18,160 Again, along this broad distribution across from 1882 01:41:18,160 --> 01:41:25,480 molecular all the way up to host and human level. 1883 01:41:25,480 --> 01:41:30,080 Same broad themes as that defined for bacteria. 1884 01:41:30,080 --> 01:41:35,320 I'm going to jump ahead a little bit and talk about the 1885 01:41:35,320 --> 01:41:39,680 host-focused vaccine groups. There was IHD and VMD. 1886 01:41:39,680 --> 01:41:41,840 Both of these are oversubscribed. 1887 01:41:41,840 --> 01:41:46,720 And so largely what they did, they liked these study 1888 01:41:46,720 --> 01:41:47,200 sections. 1889 01:41:47,200 --> 01:41:52,760 They liked how they worked, but they proposed some ways 1890 01:41:52,760 --> 01:41:57,760 to reduce their scope, moving some of the pathogen, 1891 01:41:57,760 --> 01:42:02,200 more pathogen-focused applications out and to keep it 1892 01:42:02,200 --> 01:42:04,440 more -- IHD more immune-focused. 1893 01:42:05,080 --> 01:42:10,640 And with VMD, thinking about where this line was 1894 01:42:10,640 --> 01:42:14,360 between vaccine developed and virus focused. 1895 01:42:14,360 --> 01:42:15,680 All right. 1896 01:42:15,680 --> 01:42:20,080 Now I would spend a little bit more time on what they 1897 01:42:20,080 --> 01:42:21,560 thought about tropical diseases. 1898 01:42:21,560 --> 01:42:25,560 There were two study sections in this cluster in this 1899 01:42:25,560 --> 01:42:27,120 section to start with. 1900 01:42:27,120 --> 01:42:29,440 One was called PHET. 1901 01:42:29,440 --> 01:42:36,160 That's a eukaryotic section. 1902 01:42:36,160 --> 01:42:40,680 Another was vector biology. From those, they created 1903 01:42:40,680 --> 01:42:42,680 three new study seconds. 1904 01:42:42,680 --> 01:42:48,680 One was called medically important fungi and medical 1905 01:42:48,680 --> 01:42:49,400 mycoses. 1906 01:42:49,400 --> 01:42:54,400 They chose to take the mycology applications out of 1907 01:42:54,400 --> 01:42:58,800 PTHE and created a section focused exclusively on 1908 01:42:58,800 --> 01:42:59,360 fungi. 1909 01:42:59,360 --> 01:43:05,120 And then they created PZVA and PZVB which stands for 1910 01:43:05,120 --> 01:43:08,000 parasitic, zoonotic and vector-born diseases. 1911 01:43:08,000 --> 01:43:15,160 They imagined two of those study sections. 1912 01:43:15,160 --> 01:43:20,200 The overarching theme for PZVA and B is an emphasis on 1913 01:43:20,200 --> 01:43:23,400 their complex live cycles including the environmental 1914 01:43:23,400 --> 01:43:25,680 drivers and One Health Concept. 1915 01:43:25,680 --> 01:43:32,240 So PTHE got split into MFMM and the others went into 1916 01:43:32,240 --> 01:43:34,040 PZVA and PZVB. 1917 01:43:34,040 --> 01:43:40,320 And then I also wanted to note that PZVA and B are not 1918 01:43:40,320 --> 01:43:44,160 just parasites, but they would include any pathogen, 1919 01:43:44,160 --> 01:43:48,480 viruses, bacteria where the emphasis is on this complex 1920 01:43:48,480 --> 01:43:49,000 lifecycle. 1921 01:43:49,000 --> 01:43:53,320 Since they spent a lot of time reimagining this 1922 01:43:53,320 --> 01:43:57,160 interspecies and complex lifecycle group, let me spend 1923 01:43:57,160 --> 01:44:01,040 a few seconds outlining what their interests were. 1924 01:44:01,040 --> 01:44:06,240 So these three study sections, IMII, PZVA and B are 1925 01:44:06,240 --> 01:44:10,400 not focused on a single pathogen, but investigate 1926 01:44:10,400 --> 01:44:14,040 interspecies and complex interactions. IMII handles 1927 01:44:14,040 --> 01:44:17,680 things like biofilms, microbiomes, those type of 1928 01:44:17,680 --> 01:44:22,880 situations. PZVA and back seat are complex live cycle 1929 01:44:22,880 --> 01:44:27,600 focused with A focused at the genome and cellular 1930 01:44:27,600 --> 01:44:31,760 levels and B focused on community and population 1931 01:44:31,760 --> 01:44:32,280 levels. 1932 01:44:32,280 --> 01:44:35,640 So here is what the group came up with in the end. 1933 01:44:35,640 --> 01:44:39,440 They took the ten applications that you see in yellow 1934 01:44:39,440 --> 01:44:43,600 on the left and they imagined 12 study sections on the 1935 01:44:43,600 --> 01:44:43,960 right. 1936 01:44:43,960 --> 01:44:48,200 And I wanted to just kind of quickly go through more 1937 01:44:48,200 --> 01:44:50,920 visually kind of what their recommendations were. 1938 01:44:50,920 --> 01:44:55,040 So for the bacterial group, you'll see largely a lot of 1939 01:44:55,040 --> 01:44:58,400 the bacteria remained in this grouping of three study 1940 01:44:58,400 --> 01:45:02,160 sections. They would -- the SEP was divided across the 1941 01:45:02,160 --> 01:45:03,280 three study sections. 1942 01:45:04,040 --> 01:45:08,400 And then there would be some that would be peeled off 1943 01:45:08,400 --> 01:45:11,600 into these interspecies and complex lifecycle group if 1944 01:45:11,600 --> 01:45:14,800 that was an appropriate driver of that application. 1945 01:45:14,800 --> 01:45:18,200 In the viral group, they did more or less the same 1946 01:45:18,200 --> 01:45:18,520 thing. 1947 01:45:18,520 --> 01:45:25,120 2 became 3 plus pulling out the ones that are more 1948 01:45:25,120 --> 01:45:26,320 complex interspecies. 1949 01:45:26,320 --> 01:45:30,760 On the host focus, what I mentioned was the idea was to 1950 01:45:30,760 --> 01:45:31,840 reduce scope slightly. 1951 01:45:31,840 --> 01:45:35,760 And they did this by imagining and thinking about the 1952 01:45:35,760 --> 01:45:39,640 way we could redefine the guidelines such that it was 1953 01:45:39,640 --> 01:45:43,920 more pathogen focused, that it would go back to one of 1954 01:45:43,920 --> 01:45:47,040 these pathogen focused study sections and that would 1955 01:45:47,040 --> 01:45:48,560 bring those sizes down. 1956 01:45:48,560 --> 01:45:55,840 And then I want to just touch a little bit more on kind 1957 01:45:55,840 --> 01:46:00,880 of the big picture stuff for the eukaryotic vector 1958 01:46:00,880 --> 01:46:02,000 biology group. 1959 01:46:02,000 --> 01:46:10,560 So they proposed the TPHE would more or less be split 1960 01:46:10,560 --> 01:46:17,560 into two, the fungi and then PZVA and PZVB. 1961 01:46:17,560 --> 01:46:21,360 And they imagined and spent a great deal of time 1962 01:46:21,360 --> 01:46:25,120 discussing the concept that if you did this, you would 1963 01:46:25,120 --> 01:46:26,600 grow these study sections. 1964 01:46:26,600 --> 01:46:30,880 So I have indicated that here, that they would imagine 1965 01:46:30,880 --> 01:46:34,720 a lot of applications coming from outside the cluster, 1966 01:46:34,720 --> 01:46:38,120 new applications, people would be wanting these types 1967 01:46:38,120 --> 01:46:42,360 of environments and so that would help grow those two 1968 01:46:42,360 --> 01:46:43,240 study sections. 1969 01:46:43,240 --> 01:46:47,720 So the internal panel -- this was stretch two in 1970 01:46:47,720 --> 01:46:48,720 Bruce's outline. 1971 01:46:49,040 --> 01:46:54,240 They met in February and these were largely program 1972 01:46:54,240 --> 01:47:00,040 staff and directors at mostly NIAAD and NMS which is 1973 01:47:00,040 --> 01:47:03,520 largely where they study sections reside. 1974 01:47:03,520 --> 01:47:06,400 Let me go through quickly what they agreed with us with 1975 01:47:06,400 --> 01:47:07,200 the external panel. 1976 01:47:07,200 --> 01:47:10,360 So they were supportive of the recommendations to 1977 01:47:10,360 --> 01:47:13,520 restructure bacterial viral study sections. They had 1978 01:47:13,520 --> 01:47:17,080 some minor suggestions to clarify scope and overlaps. 1979 01:47:17,080 --> 01:47:20,240 They were supportive of recommendations changes to IHD 1980 01:47:20,240 --> 01:47:21,000 and VMD. 1981 01:47:21,000 --> 01:47:24,360 They thought we should rename them because we're 1982 01:47:24,360 --> 01:47:26,000 changing the scope slightly. 1983 01:47:26,000 --> 01:47:30,600 That makes perfect sense. 1984 01:47:30,600 --> 01:47:34,640 They welcomed the creation of a microbial-microbial 1985 01:47:34,640 --> 01:47:39,280 interactions study section, and finally with regard to 1986 01:47:39,280 --> 01:47:43,920 PZVA and PZVB, felt chat including these environmental 1987 01:47:43,920 --> 01:47:47,400 and population drivers would be welcomed. 1988 01:47:47,400 --> 01:47:51,840 So they also had some concerns, and I'll add the three 1989 01:47:51,840 --> 01:47:53,080 large ones here. 1990 01:47:53,080 --> 01:47:58,160 One was the mycology study section MFMM would not have 1991 01:47:58,160 --> 01:48:02,760 enough applications and that the science would be too 1992 01:48:02,760 --> 01:48:03,240 narrow. 1993 01:48:03,240 --> 01:48:06,840 They were concerned that all the parasites previously 1994 01:48:06,840 --> 01:48:11,320 housed in the PTHE would be reviewed alongside the big 1995 01:48:11,320 --> 01:48:14,880 ones, the viruses and vectors and maybe disadvantaged 1996 01:48:14,880 --> 01:48:19,360 in the new broader study sections. They were concerned 1997 01:48:19,360 --> 01:48:23,400 that pair parasite biology would no longer have a 1998 01:48:23,400 --> 01:48:24,280 review home. 1999 01:48:24,280 --> 01:48:29,320 As an alternative, they thought maybe if we were going 2000 01:48:29,320 --> 01:48:34,880 to move forward with PZVA and PZVB, that PZVA could be 2001 01:48:34,880 --> 01:48:38,920 the parasitic focus while PZVB would have a 2002 01:48:38,920 --> 01:48:39,920 bacterial/viral focus. 2003 01:48:39,920 --> 01:48:43,720 We spent a little bit of time doing some analysis to 2004 01:48:43,720 --> 01:48:46,880 see could we create, would there be enough applications 2005 01:48:46,880 --> 01:48:48,960 to create an MFMM study section. 2006 01:48:48,960 --> 01:48:52,840 What you see is the distribution of applications in the 2007 01:48:52,840 --> 01:48:57,080 fungal area based on where they are assigned to the IC. 2008 01:48:57,080 --> 01:49:01,200 As you can see imagine, NIID received the vast majority 2009 01:49:01,200 --> 01:49:02,040 of them. 2010 01:49:02,040 --> 01:49:09,080 But study section, you can see PTHE received most of 2011 01:49:09,080 --> 01:49:14,000 the fungal applications, but they are distributed 2012 01:49:14,000 --> 01:49:21,040 across a number of study sessions. ODCS, for example, 2013 01:49:21,040 --> 01:49:27,360 has a dental aspect and includes oral conditions that 2014 01:49:27,360 --> 01:49:29,440 might include fungi. 2015 01:49:29,440 --> 01:49:33,960 However, when we added up all the numbers, we currently 2016 01:49:33,960 --> 01:49:37,560 do not believe that there are enough fungal 2017 01:49:37,560 --> 01:49:41,160 applications across all CSR study sections to support 2018 01:49:41,160 --> 01:49:43,000 the creation of MFMM. 2019 01:49:43,000 --> 01:49:47,480 So I come to you with two possible scenarios, and we 2020 01:49:47,480 --> 01:49:51,520 can discuss as many scenarios that you think might be 2021 01:49:51,520 --> 01:49:52,360 available here. 2022 01:49:52,360 --> 01:49:57,240 But scenario 1 would be to go with what the external 2023 01:49:57,240 --> 01:50:00,800 panel recommended. They had some creative ideas. 2024 01:50:00,800 --> 01:50:05,680 They really rallied around these ideas and thought it 2025 01:50:05,680 --> 01:50:10,040 would be wonderful for the communities, particularly in 2026 01:50:10,040 --> 01:50:12,760 the area of tropical medicine. 2027 01:50:12,760 --> 01:50:16,600 Proposal number one would be to create MFMM as proposed 2028 01:50:16,600 --> 01:50:20,080 with the expectation of growth, with the study section 2029 01:50:20,080 --> 01:50:23,160 grow to be large enough down the road. 2030 01:50:23,160 --> 01:50:27,880 Scenario number two I offer up as a possibility would 2031 01:50:27,880 --> 01:50:33,040 be that we would leave PTHE alone, that is fungal and 2032 01:50:33,040 --> 01:50:36,320 parasitic applications would remain in the study 2033 01:50:36,320 --> 01:50:39,600 section. MFMM would not be created. 2034 01:50:39,600 --> 01:50:45,360 The P in PZVA and PZVB would disappear and that we 2035 01:50:45,360 --> 01:50:50,080 would need to do some additional analyses and mock 2036 01:50:50,080 --> 01:50:54,280 sorts to determine whether we would have enough 2037 01:50:54,280 --> 01:50:58,480 applications to support two study sections given that 2038 01:50:58,480 --> 01:51:02,680 the parasitic biology would be back in PTHE. 2039 01:51:02,680 --> 01:51:08,760 With that, I will pause for any discussion, comments, 2040 01:51:08,760 --> 01:51:09,400 feedback 2041 01:51:09,400 --> 01:51:12,120 >> DR. BRUCE REED: Thanks, Ross. 2042 01:51:12,120 --> 01:51:14,920 So reactions from council? 2043 01:51:14,920 --> 01:51:19,960 This is an area where we would, at the end of the 2044 01:51:19,960 --> 01:51:22,080 discussion, like a council action. 2045 01:51:22,080 --> 01:51:23,560 Michelle. 2046 01:51:23,560 --> 01:51:27,800 >> DR. MICHELLE JANELSINS: So I'll make a couple of 2047 01:51:27,800 --> 01:51:32,040 comments as an observer. I was able to observe the 2048 01:51:32,040 --> 01:51:35,880 meeting and I thought that, you know, it was very 2049 01:51:35,880 --> 01:51:38,600 engaging group, definitely the expertise needed was 2050 01:51:38,600 --> 01:51:40,160 there at the table. 2051 01:51:40,160 --> 01:51:44,880 I think that the group did really consider the size of 2052 01:51:44,880 --> 01:51:48,760 the -- like the applications within the study section a 2053 01:51:48,760 --> 01:51:53,080 lot and they discussed that a lot and how to 2054 01:51:53,080 --> 01:51:53,720 reorganize. 2055 01:51:53,720 --> 01:51:56,480 Then also I thought the group was very forward thinking 2056 01:51:56,480 --> 01:51:58,680 about, you know, where is the science going? 2057 01:51:58,680 --> 01:52:04,960 Where is the future going? There are definitely, you 2058 01:52:04,960 --> 01:52:09,960 know, interesting scenarios that could be happening I 2059 01:52:09,960 --> 01:52:12,440 think in tropical medicine. 2060 01:52:12,440 --> 01:52:16,480 Considering dynamic and interspecies dynamics and 2061 01:52:16,480 --> 01:52:20,520 really across all the different pathogens. 2062 01:52:20,520 --> 01:52:27,240 So I thought they did a really good job coming up with 2063 01:52:27,240 --> 01:52:29,480 these proposed study sections. 2064 01:52:29,480 --> 01:52:32,600 >> DR. BRUCE REED: Thanks, Michelle. 2065 01:52:32,600 --> 01:52:32,880 Mark. 2066 01:52:32,880 --> 01:52:35,600 >> DR. MARK PEIFER: I'm coming from well outside this 2067 01:52:35,600 --> 01:52:37,560 area. Just a couple of thoughts, right? 2068 01:52:37,560 --> 01:52:41,320 One is in general, I love what they did. 2069 01:52:41,320 --> 01:52:44,840 Really sort of thinking more big picture, not so 2070 01:52:44,840 --> 01:52:47,160 organism focused, but really question focused. 2071 01:52:47,160 --> 01:52:51,720 And I think that is present across almost all of the 2072 01:52:51,720 --> 01:52:53,800 changes that they initially proposed. 2073 01:52:53,800 --> 01:52:59,320 The one striking counterexample is trying to put all 2074 01:52:59,320 --> 01:53:03,640 the fungal applications in one study section. 2075 01:53:03,640 --> 01:53:08,400 So that seems counter to the overall broadening of the 2076 01:53:08,400 --> 01:53:08,880 biology. 2077 01:53:08,880 --> 01:53:12,600 The second thing -- and, again, you guys are probably 2078 01:53:12,600 --> 01:53:16,680 thinking about this, but I think it's important that we 2079 01:53:16,680 --> 01:53:21,200 realize that diseases that might be at the top of our 2080 01:53:21,200 --> 01:53:24,880 mind in the U.S. aren't necessarily the same diseases 2081 01:53:24,880 --> 01:53:27,480 that are the most important worldwide. 2082 01:53:27,480 --> 01:53:31,400 Again, this whole area of, you know, parasites and 2083 01:53:31,400 --> 01:53:34,880 probably fungal infections, too, things that have a 2084 01:53:34,880 --> 01:53:39,680 major, major impact in other parts of the world and not 2085 01:53:39,680 --> 01:53:45,360 so much of an impact in the U.S., I do hope we keep 2086 01:53:45,360 --> 01:53:49,280 homes for them, right? Because that is really 2087 01:53:49,280 --> 01:53:50,160 important work. 2088 01:53:50,160 --> 01:53:51,760 >> DR. ROSS SHONAT: Yeah. 2089 01:53:51,760 --> 01:53:55,720 So you noticed in one of my first slides, I said the 2090 01:53:55,720 --> 01:53:58,680 group tried to think about the organization of science 2091 01:53:58,680 --> 01:54:00,040 that wasn't pathogen focused. 2092 01:54:00,040 --> 01:54:01,920 But these pathogens are very different. 2093 01:54:01,920 --> 01:54:05,440 And you'll notice we do have pathogen-focused study 2094 01:54:05,440 --> 01:54:09,400 sections, the bacteria and the viruses happen to be 2095 01:54:09,400 --> 01:54:13,800 large components and they live within two or three or 2096 01:54:13,800 --> 01:54:15,560 four different study sections. 2097 01:54:15,560 --> 01:54:18,760 The interesting thing here is the two pathogens that 2098 01:54:18,760 --> 01:54:21,240 can't sustain themselves as the stand-alone study 2099 01:54:21,240 --> 01:54:23,720 sections are the parasites and the fungi. 2100 01:54:23,720 --> 01:54:30,160 So I think if there were enough applications, I think 2101 01:54:30,160 --> 01:54:35,960 the -- that the existence of a mycoses study section 2102 01:54:35,960 --> 01:54:42,400 would be different than what is being done with the 2103 01:54:42,400 --> 01:54:44,960 bacteria and the viruses. 2104 01:54:45,640 --> 01:54:50,440 >> DR. BRUCE REED: Sean. 2105 01:54:50,440 --> 01:54:54,920 >> Dr. Sean Davidson: I'm impressed with the rigger you 2106 01:54:54,920 --> 01:54:56,720 have put into this. 2107 01:54:56,720 --> 01:54:58,960 I think science is constantly changing and requires a 2108 01:54:58,960 --> 01:54:59,720 process like this. 2109 01:54:59,720 --> 01:55:03,400 So kudos to you guys, CSR in terms of whoever came up 2110 01:55:03,400 --> 01:55:06,440 with this idea and the people that are implementing it. 2111 01:55:06,440 --> 01:55:10,880 As a more specific comment, I have a fungal pathologist 2112 01:55:10,880 --> 01:55:16,200 who has an office just a couple doors down from me. 2113 01:55:16,200 --> 01:55:20,600 I've been hearing him for years talk about the conflict 2114 01:55:20,600 --> 01:55:23,720 between the fungal people and the parasitologists. 2115 01:55:23,720 --> 01:55:28,120 So I have a general knowledge of what this was going to 2116 01:55:28,120 --> 01:55:32,920 be based on the materials you sent me. And I sort of 2117 01:55:32,920 --> 01:55:36,600 explained to him that there is a thought about breaking 2118 01:55:36,600 --> 01:55:39,920 the fungal people off into separate sections with the 2119 01:55:39,920 --> 01:55:40,280 parasiteologists. 2120 01:55:40,280 --> 01:55:43,280 He likes that idea because he felt there was a 2121 01:55:43,280 --> 01:55:45,680 disconnect between how the fungal people think about 2122 01:55:45,680 --> 01:55:48,040 these oral nimps and how the parasitologists do. 2123 01:55:48,040 --> 01:55:52,120 And they don't have a good way of bridging across those 2124 01:55:52,120 --> 01:55:52,840 different organisms. 2125 01:55:52,840 --> 01:55:58,400 So I asked him if he thought this was a big enough 2126 01:55:58,400 --> 01:56:03,040 growth area to support his own study section down the 2127 01:56:03,040 --> 01:56:08,600 line. And he felt that there was because there is an 2128 01:56:08,600 --> 01:56:11,840 increasing issues with increasing immune result issues 2129 01:56:11,840 --> 01:56:15,080 that can result with these fungal problems. 2130 01:56:15,080 --> 01:56:21,400 That's an N equals one, but he said if that change was 2131 01:56:21,400 --> 01:56:25,600 implemented, it would be very popular, he thinks, 2132 01:56:25,600 --> 01:56:29,320 between the parasitologists and the fungal people. 2133 01:56:29,320 --> 01:56:31,480 I thought I would throw that out there. 2134 01:56:31,480 --> 01:56:33,840 >> DR. ROSS SHONAT: That resinated with the external 2135 01:56:33,840 --> 01:56:34,520 panel, as well. 2136 01:56:34,520 --> 01:56:36,760 They recognize the conflict among those two 2137 01:56:36,760 --> 01:56:37,080 communities. 2138 01:56:37,080 --> 01:56:44,280 And I think what we're ultimately -- we're receptive to 2139 01:56:44,280 --> 01:56:53,080 the idea, and I think that is what resulted in the 2140 01:56:53,080 --> 01:56:57,080 creation of this study section. 2141 01:56:57,960 --> 01:56:58,160 For us, though, we need to have study sections 2142 01:56:58,160 --> 01:56:58,280 appropriately sized with good competition, etcetera. 2143 01:56:58,280 --> 01:57:03,280 We don't have that so it largely becomes a management 2144 01:57:03,280 --> 01:57:04,800 issue for us. 2145 01:57:04,800 --> 01:57:07,600 What was pointed out to me, those organisms live 2146 01:57:08,200 --> 01:57:11,320 in different kingdoms and the only thick they share is 2147 01:57:11,320 --> 01:57:15,040 a nucleus. So it becomes then the SRO has to engage 2148 01:57:15,040 --> 01:57:17,840 fully with these two communities to make sure that 2149 01:57:17,840 --> 01:57:19,120 they're playing well together. 2150 01:57:19,120 --> 01:57:23,520 That is something that we can probably continue to do 2151 01:57:23,520 --> 01:57:27,880 and work harder at until perhaps the day comes when 2152 01:57:27,880 --> 01:57:31,840 there are enough fungal applications for it to become 2153 01:57:31,840 --> 01:57:34,480 its own healthy standing study section. 2154 01:57:34,480 --> 01:57:41,440 >> DR. BRUCE REED: Karen. 2155 01:57:41,440 --> 01:57:45,320 >> Karen Anderson: Thank you. This was well informed. 2156 01:57:45,320 --> 01:57:46,760 I have two comments. 2157 01:57:46,760 --> 01:57:52,960 One is when it comes to this, the recognition of the 2158 01:57:52,960 --> 01:57:57,480 ecology that influences the disease development and not 2159 01:57:57,480 --> 01:58:02,000 just a single pathogen, but certainly the microbial 2160 01:58:02,000 --> 01:58:06,520 microenvironment, for example, in which it lives and 2161 01:58:06,520 --> 01:58:12,160 the evolution of that process is really central to a 2162 01:58:12,160 --> 01:58:14,440 lot of that pathogenesis. 2163 01:58:14,440 --> 01:58:21,000 And one of the challenges when we break out to disease 2164 01:58:21,000 --> 01:58:24,560 focused approaches or organism-focused approaches is 2165 01:58:24,560 --> 01:58:29,920 that that ecology, that genome gets lost within the 2166 01:58:29,920 --> 01:58:34,680 context of how that -- of how they develop. 2167 01:58:35,160 --> 01:58:39,160 And just as a thought as the field shifts. 2168 01:58:39,160 --> 01:58:43,680 And my second thought on this is all microbial 2169 01:58:43,680 --> 01:58:47,680 pathogenesis happens in the context of immune systems, 2170 01:58:47,680 --> 01:58:50,720 functional or not, diverse or not. 2171 01:58:50,720 --> 01:58:54,920 And that that intersection between immunity and 2172 01:58:54,920 --> 01:58:59,120 pathogenesis and how that fits, where does 2173 01:58:59,120 --> 01:59:02,720 immunologist perhaps studies across multiple organism 2174 01:59:02,720 --> 01:59:08,760 types fit within the context of perhaps of these study 2175 01:59:08,760 --> 01:59:09,360 sections. 2176 01:59:09,360 --> 01:59:15,560 And these are comments not requiring answers, but just 2177 01:59:15,560 --> 01:59:16,960 as thoughts. 2178 01:59:16,960 --> 01:59:21,160 >> DR. ROSS SHONAT: What you said resinated strongly 2179 01:59:21,160 --> 01:59:25,800 with the group in that a lot of their innovative ideas 2180 01:59:25,800 --> 01:59:29,160 here were related to this interspecies /complex live 2181 01:59:29,160 --> 01:59:32,960 cycle interactions and they felt strongly this is where 2182 01:59:32,960 --> 01:59:37,160 the future was going, that this was how they could 2183 01:59:37,160 --> 01:59:40,520 create study sections that weren't so pathogen specific 2184 01:59:40,520 --> 01:59:42,200 and perhaps too insular. 2185 01:59:42,200 --> 01:59:47,440 >> DR. BRUCE REED: Jinming. 2186 01:59:47,440 --> 01:59:51,600 >> DR. JINMING GAO: I'm not an expert in infectious 2187 01:59:51,600 --> 01:59:51,960 disease. 2188 01:59:51,960 --> 01:59:56,000 When I was the chair of the gene and drug delivery 2189 01:59:56,000 --> 02:00:00,080 section, so we do have a diverse group of viral people 2190 02:00:00,080 --> 02:00:03,400 for doing gene delivery and then the polymer synthetic 2191 02:00:03,400 --> 02:00:05,960 folks doing a non viral gene delivery. 2192 02:00:05,960 --> 02:00:09,200 I think since the information is productive. 2193 02:00:09,200 --> 02:00:14,600 Where maybe the mNRA vaccine is the merging of the two 2194 02:00:14,600 --> 02:00:19,080 worlds in terms of complexity and the practical impact. 2195 02:00:19,080 --> 02:00:24,320 I wonder this sort of a discussion on the parasitology 2196 02:00:24,320 --> 02:00:29,040 and then the fungi, without knowing the field, whether 2197 02:00:29,040 --> 02:00:34,280 that could be beneficial for having a joint, you know, 2198 02:00:34,280 --> 02:00:38,480 study section to cross pollinate some of that 2199 02:00:38,480 --> 02:00:39,000 information. 2200 02:00:39,000 --> 02:00:42,400 And Sean's comment seems to support that. 2201 02:00:42,400 --> 02:00:47,480 And then the other thought I have is having this sort 2202 02:00:47,480 --> 02:00:51,200 of a broad umbrella interspecies and a complex 2203 02:00:51,200 --> 02:00:55,360 lifecycle group seems like it's probably a good buffer 2204 02:00:55,360 --> 02:00:59,040 idea to invite, you know, across disciplinary different 2205 02:00:59,040 --> 02:01:04,120 applications, maybe let it play out so that we can have 2206 02:01:04,120 --> 02:01:08,760 the A and a B for further division if needed. 2207 02:01:08,760 --> 02:01:17,960 But I am more for maybe using option number two from 2208 02:01:17,960 --> 02:01:20,440 the possible scenarios. 2209 02:01:20,440 --> 02:01:21,720 >> DR. ROSS SHONAT: Yeah. 2210 02:01:21,720 --> 02:01:25,280 So I think I already mentioned, so when the fungal 2211 02:01:25,280 --> 02:01:27,760 group -- obviously, the fungal's applicants would like 2212 02:01:27,760 --> 02:01:29,880 to have a fungal study section. 2213 02:01:30,160 --> 02:01:34,560 That led to the parasitologists saying we would like 2214 02:01:34,560 --> 02:01:37,520 to have our own study section. 2215 02:01:37,520 --> 02:01:40,720 So you have to balance tout needs of these communities 2216 02:01:40,720 --> 02:01:43,920 to what is the best for the peer review process. 2217 02:01:43,920 --> 02:01:51,800 In terms of PZA or PZVB, or whatever those ultimately 2218 02:01:51,800 --> 02:01:57,320 become, we have to do additional analysis. 2219 02:01:57,320 --> 02:02:01,000 But the external panel did come one some clear 2220 02:02:01,000 --> 02:02:04,280 distinguishing features between those two, one at a 2221 02:02:04,280 --> 02:02:07,120 cellular/molecular level and one at the population 2222 02:02:07,120 --> 02:02:10,800 level and it's something called the One Health 2223 02:02:10,800 --> 02:02:14,440 Concept, which it has to do with the relationship 2224 02:02:14,440 --> 02:02:16,920 between humans, animals and the environment. 2225 02:02:16,920 --> 02:02:38,240 They saw great growth there for NIH. 2226 02:02:38,240 --> 02:02:43,080 >> DR. BRUCE REED: A quick comment and I'll get to you. 2227 02:02:43,080 --> 02:02:46,560 thinking about growth areas, CSR has not had much luck 2228 02:02:46,560 --> 02:02:50,360 with the if you build it, they would come concept of 2229 02:02:50,360 --> 02:02:53,800 study sections. That has just -- let's say we can 2230 02:02:53,800 --> 02:02:57,640 think of no examples where that has proved to be true. 2231 02:02:57,880 --> 02:03:02,000 Another way of saying this is that current numbers have 2232 02:03:02,000 --> 02:03:06,160 proved to be a better guide to some study section 2233 02:03:06,160 --> 02:03:08,640 viability than sort of future estimates. 2234 02:03:08,640 --> 02:03:15,640 Obviously, we adjust things all the time based on what 2235 02:03:15,640 --> 02:03:21,920 happens, but it's proven very difficult to make these 2236 02:03:21,920 --> 02:03:26,840 sort of guesses about, you know, applications 2237 02:03:26,840 --> 02:03:33,120 increasing if you structure a study section a certain 2238 02:03:33,120 --> 02:03:33,840 way. 2239 02:03:33,840 --> 02:03:35,560 Edda. 2240 02:03:35,560 --> 02:03:42,560 >> Dr. SPIEKERKOETTER: What is really limiting the 2241 02:03:42,560 --> 02:03:51,360 number of applications from the fungal group? Is it 2242 02:03:51,360 --> 02:04:00,160 that they are in this one study section with the 2243 02:04:00,160 --> 02:04:07,160 parasitologists or is there a different reason why 2244 02:04:07,160 --> 02:04:11,560 they're not so much applications? 2245 02:04:11,560 --> 02:04:13,320 And then do you have any data on whether there are less 2246 02:04:13,320 --> 02:04:14,480 applications being funded because they are in this 2247 02:04:14,480 --> 02:04:15,800 joint group because then the -- maybe there are more 2248 02:04:15,800 --> 02:04:16,840 reviewers from the parasitology group that disfavor 2249 02:04:16,840 --> 02:04:17,280 the other applications? 2250 02:04:17,280 --> 02:04:22,200 Do we know any of those data that would require maybe a 2251 02:04:22,200 --> 02:04:25,440 separate group to protect the fungal, you know, 2252 02:04:25,440 --> 02:04:27,080 researchers, something like this? 2253 02:04:27,080 --> 02:04:28,400 >> DR. ROSS SHONAT: Yeah, yeah. 2254 02:04:28,400 --> 02:04:32,360 So I think we do need to do some more analysis of where 2255 02:04:32,360 --> 02:04:35,080 the applications are coming from, where they would be 2256 02:04:35,080 --> 02:04:36,880 coming from if they stood alone. 2257 02:04:36,880 --> 02:04:41,840 So that was one of the arguments that was made by the 2258 02:04:41,840 --> 02:04:45,560 external panel was that the mycologists need their own 2259 02:04:45,560 --> 02:04:49,720 study sections and if you build it, they will come. 2260 02:04:49,720 --> 02:04:53,240 We don't know whether that is true. Are the 2261 02:04:53,240 --> 02:04:55,360 parasitologists overwhelming the fungal community? 2262 02:04:55,360 --> 02:04:58,160 There are some arguments that that was true. 2263 02:04:58,160 --> 02:05:01,240 What is the solution to that? 2264 02:05:01,240 --> 02:05:05,360 As I mentioned, I think we need to make sure that those 2265 02:05:05,360 --> 02:05:08,120 two communities are working together to do review 2266 02:05:08,120 --> 02:05:10,520 fairly and are not siloing their reviews. 2267 02:05:10,520 --> 02:05:14,640 We have lots of study sections where -- not a lot. We 2268 02:05:14,640 --> 02:05:17,720 have a number of study sections where two communities 2269 02:05:17,720 --> 02:05:21,120 that don't get along are put together simply based on 2270 02:05:21,120 --> 02:05:22,360 size and need. 2271 02:05:22,360 --> 02:05:27,120 And which then becomes a management challenge for us to 2272 02:05:27,120 --> 02:05:30,920 ensure that we recruit appropriately across the sign 2273 02:05:30,920 --> 02:05:32,840 appropriately and manage expectations. 2274 02:05:32,840 --> 02:05:37,000 So I don't know where the answer is, but over the 2275 02:05:37,000 --> 02:05:40,760 course of the last six months that I've been involved 2276 02:05:40,760 --> 02:05:43,760 in this particular inquiry, the fungal and parasitic 2277 02:05:43,760 --> 02:05:46,400 divide has become very clear to me. 2278 02:05:46,400 --> 02:05:50,440 >> DR. NONI BYRNES: Let me jump in really briefly, 2279 02:05:50,440 --> 02:05:50,800 Ross. 2280 02:05:50,800 --> 02:05:55,240 I think we're past time here and we do need a vote on 2281 02:05:55,240 --> 02:05:59,000 options. If you decided to vote on option two, which 2282 02:05:59,000 --> 02:06:02,400 is not forming the mycology study section, you have to 2283 02:06:02,400 --> 02:06:05,480 rest assured that we will take a very careful 2284 02:06:05,480 --> 02:06:08,200 management look at the current existing study section, 2285 02:06:08,200 --> 02:06:11,280 which has both parasitology and mycology. We have 2286 02:06:11,600 --> 02:06:15,720 the data. It is not difficult to do the data scoring 2287 02:06:15,720 --> 02:06:19,480 analysis. We will have that in hand and we will 2288 02:06:19,480 --> 02:06:23,560 proceed with that in mind to make sure that no group is 2289 02:06:23,560 --> 02:06:24,560 being unfairly disadvantaged. 2290 02:06:24,560 --> 02:06:30,000 Historical issues aside, moving forward. 2291 02:06:30,000 --> 02:06:34,280 >> DR. BRUCE REED: So my sense of this discussion is 2292 02:06:34,280 --> 02:06:37,840 that there is a lot of enthusiasm for the general 2293 02:06:37,840 --> 02:06:41,040 framework and most of the study sections that the -- 2294 02:06:41,040 --> 02:06:44,040 that were put forth by the external group. 2295 02:06:44,040 --> 02:06:49,080 The question comes down to -- Ross put the options on, 2296 02:06:49,080 --> 02:06:51,080 option one, option two. 2297 02:06:52,440 --> 02:06:56,880 I've heard a couple of comments from council sort of in 2298 02:06:56,880 --> 02:07:01,720 the favor of option two. Is that a sense of the 2299 02:07:01,720 --> 02:07:02,120 council? 2300 02:07:02,120 --> 02:07:07,120 Two, yes, I'm seeing that. 2301 02:07:07,120 --> 02:07:14,480 Would -- could somebody put a motion forth, then? 2302 02:07:14,960 --> 02:07:15,280 Michelle. 2303 02:07:15,280 --> 02:07:17,800 >> DR. JINMING GAO: I put a motion for option two. 2304 02:07:17,800 --> 02:07:19,000 >> DR. BRUCE REED: Okay. 2305 02:07:19,000 --> 02:07:23,840 Jinming. 2306 02:07:23,840 --> 02:07:25,560 And Michelle, you had a comment? 2307 02:07:25,560 --> 02:07:29,880 >> DR. MICHELLE JANELSINS: I had a question about if 2308 02:07:29,880 --> 02:07:34,200 we went with option two and if it came with the 2309 02:07:34,200 --> 02:07:37,360 opportunity for a fungal SEP to be formed. 2310 02:07:37,360 --> 02:07:40,200 Like if there became an abundance of applications, 2311 02:07:40,200 --> 02:07:44,120 would that be considered or would that be one of the 2312 02:07:44,120 --> 02:07:45,160 management approaches ? 2313 02:07:45,160 --> 02:07:47,720 I'm just wondering if someone can respond to that. 2314 02:07:47,720 --> 02:07:49,960 >> DR. NONI BYRNES: Let me answer that right away. 2315 02:07:49,960 --> 02:07:54,040 If suddenly there is a big growth of applications in 2316 02:07:54,040 --> 02:07:57,760 one area, we're trying to approach our overall panel 2317 02:07:57,760 --> 02:08:01,440 sizes to have better review outcomes and better review 2318 02:08:01,440 --> 02:08:05,120 management. We will create SEPs as necessary, yes. 2319 02:08:05,120 --> 02:08:05,960 >> DR. BRUCE REED: All right. 2320 02:08:05,960 --> 02:08:11,920 So the motion is to accept the recommendations for the 2321 02:08:11,920 --> 02:08:17,240 external panel with option 2, with the addition of 2322 02:08:17,240 --> 02:08:18,440 option 2. 2323 02:08:18,440 --> 02:08:24,040 I think we know what we're talking about here. 2324 02:08:24,040 --> 02:08:30,000 So let's just have a vote. Could I see a show of hands 2325 02:08:30,000 --> 02:08:33,400 of people -- council members in favor of that. 2326 02:08:33,640 --> 02:08:34,840 All right. 2327 02:08:34,840 --> 02:08:39,240 Do I see any opposed? 2328 02:08:39,240 --> 02:08:43,040 Not seeing any opposed. 2329 02:08:43,040 --> 02:08:47,400 Michelle, is your -- is that a vote or -- 2330 02:08:47,400 --> 02:08:49,760 >> DR. MICHELLE JANELSINS: Yes, I raised my hand. 2331 02:08:49,760 --> 02:08:51,040 Yes, I agree with option two. 2332 02:08:51,040 --> 02:08:53,320 >> DR. BRUCE REED: You agree with option two. Okay. 2333 02:08:53,320 --> 02:08:58,320 Great. Thank you. 2334 02:08:58,320 --> 02:09:01,680 So well, we're a little over time, but we all need a 2335 02:09:01,680 --> 02:09:01,960 break. 2336 02:09:01,960 --> 02:09:04,160 So we're going to take a break. 2337 02:09:04,160 --> 02:09:07,040 We'll take ten minutes. 2338 02:09:07,040 --> 02:09:11,920 It's 3:10 by my clock right now. Let's start promptly 2339 02:09:11,920 --> 02:09:12,800 at 3:20. 2340 02:09:12,800 --> 02:09:14,080 All right? 2341 02:09:14,080 --> 02:09:21,680 So I'll see you back shortly. 2342 02:09:21,680 --> 02:09:26,640 Next up on the agenda is the report from ENQUIRE 2343 02:09:26,640 --> 02:09:31,640 Cluster 10. This will be presented by Ray Jacobson. 2344 02:09:31,640 --> 02:09:34,760 Ray, you're up. 2345 02:09:34,760 --> 02:09:36,120 Thanks, Bruce. 2346 02:09:36,120 --> 02:09:40,280 I'm going to take the next 15 minutes and hopefully 2347 02:09:40,280 --> 02:09:44,880 only the next 15 minutes to provide the council with an 2348 02:09:44,880 --> 02:09:48,240 update about a second ENQUIRE process that's been 2349 02:09:48,240 --> 02:09:51,160 ongoing at CSR since late last year. 2350 02:09:51,160 --> 02:09:55,840 This is one I've been managing that has to do with the 2351 02:09:55,840 --> 02:09:58,960 drug discovery process, which is ENQUIRE Cluster 10. 2352 02:09:58,960 --> 02:10:03,200 It's a little broader than just drug discovery. 2353 02:10:03,200 --> 02:10:08,160 It involves drug and small molecule synthesis, 2354 02:10:08,160 --> 02:10:12,400 environmental toxins and xenobiotics, as well. 2355 02:10:12,400 --> 02:10:17,080 So I'm going to start off by showing you the study 2356 02:10:17,080 --> 02:10:19,200 sections involved in Cluster 10. 2357 02:10:19,200 --> 02:10:22,680 So this slide shows the set of the panels that were 2358 02:10:22,680 --> 02:10:23,000 included. 2359 02:10:23,000 --> 02:10:25,040 There's 11 chartered panels here. 2360 02:10:25,040 --> 02:10:33,560 And 2 recurring steps at the bottom of the slide. 2361 02:10:33,560 --> 02:10:35,880 BST(55) and AIDC(82). 2362 02:10:35,880 --> 02:10:41,680 So topics range from basic science to synthetic 2363 02:10:41,680 --> 02:10:47,440 chemistry All the way to preclinical pre-IND studies 2364 02:10:47,440 --> 02:10:49,600 for oncology therapeutics. 2365 02:10:49,600 --> 02:10:56,280 So when you look at this slide, you can see that the 2366 02:10:56,280 --> 02:11:01,840 study sections range in size from very small. BST(55) 2367 02:11:01,840 --> 02:11:06,320 has 39 applications. XNDA is at 56. 2368 02:11:06,320 --> 02:11:09,080 And some of them are quite large. 2369 02:11:09,080 --> 02:11:14,400 So DDR, drug delivery and mechanisms of antimicrobial 2370 02:11:15,040 --> 02:11:17,040 resistance is 81. 2371 02:11:17,040 --> 02:11:20,680 This is historical data for the applications of these 2372 02:11:20,680 --> 02:11:24,280 panels over the last few years, last three years. 2373 02:11:24,280 --> 02:11:29,000 So there's been about 250 applications or so coming in. 2374 02:11:29,000 --> 02:11:35,120 So they've been split off to keep the panel size 2375 02:11:35,120 --> 02:11:36,960 manageable for DDR. 2376 02:11:36,960 --> 02:11:42,520 So if you total up the numbers of applications that are 2377 02:11:42,520 --> 02:11:46,560 coming in, there's roughly less than -- about 1100, 2378 02:11:46,560 --> 02:11:49,080 slightly less than 1100 applications. 2379 02:11:49,640 --> 02:11:55,000 For an ideal study section size, that would be enough 2380 02:11:55,000 --> 02:12:01,440 for 14 to 15 panels and at the moment we have 11 2381 02:12:01,440 --> 02:12:05,720 chartered panels and a couple sets of SEPs. 2382 02:12:05,720 --> 02:12:07,480 So I want to tell you a little bit. 2383 02:12:07,480 --> 02:12:10,840 This is probably familiar to most of you to talk about 2384 02:12:10,840 --> 02:12:12,960 what we asked the workgroup to do. 2385 02:12:12,960 --> 02:12:17,760 We brought in 12 members who had experience as either 2386 02:12:17,760 --> 02:12:21,120 review or applicant across multiple study sections 2387 02:12:21,120 --> 02:12:23,520 within this group of panels. 2388 02:12:23,520 --> 02:12:27,200 The members were given representative sets of 2389 02:12:27,200 --> 02:12:30,880 applications titles and abstracts from each section, 2390 02:12:30,880 --> 02:12:34,520 study section guidelines and workload information over 2391 02:12:34,520 --> 02:12:35,040 time. 2392 02:12:35,040 --> 02:12:37,520 The external workgroup was charged with examining 2393 02:12:37,520 --> 02:12:39,600 existing study section organization and recommending 2394 02:12:39,600 --> 02:12:42,400 changes to optimize the study section size and 2395 02:12:42,400 --> 02:12:42,760 function. 2396 02:12:42,760 --> 02:12:48,720 And those changes that we were going to ask the 2397 02:12:48,720 --> 02:12:51,720 workgroup's recommendations for could include 2398 02:12:51,720 --> 02:12:55,320 modification of referral guidelines and boundaries, 2399 02:12:55,320 --> 02:12:59,520 adding emerging fields, creating new study sections, 2400 02:12:59,520 --> 02:13:03,720 disbanding study sections or merging and redistributing 2401 02:13:03,720 --> 02:13:04,320 topics. 2402 02:13:04,320 --> 02:13:08,040 So we assembled an external workgroup and they met 2403 02:13:08,040 --> 02:13:10,480 December 13th and 14th last year. 2404 02:13:10,480 --> 02:13:14,440 The workgroup members were drawn from a wide range of 2405 02:13:14,440 --> 02:13:17,600 institutions and had expertise spanning the topics that 2406 02:13:17,600 --> 02:13:22,360 were included in this. Kip guy, the dean of the school 2407 02:13:22,360 --> 02:13:26,280 of pharmacy at the University of Kentucky acts as the 2408 02:13:26,280 --> 02:13:29,840 chairperson and we were fortunate to have tone ya 2409 02:13:29,840 --> 02:13:33,000 Palermo serve as the observer from the advisory 2410 02:13:33,000 --> 02:13:33,400 council. 2411 02:13:33,400 --> 02:13:38,480 The individuals covers basic science from synthetic 2412 02:13:38,480 --> 02:13:45,000 chemistry all the way to drug delivery and advanced 2413 02:13:45,000 --> 02:13:45,720 therapeutics. 2414 02:13:45,720 --> 02:13:49,720 So the first thing we did was we asked them to give us 2415 02:13:49,720 --> 02:13:52,760 their thoughts based on the examination of the data we 2416 02:13:52,760 --> 02:13:55,560 sent them and their experiences from serving on these 2417 02:13:55,560 --> 02:13:58,000 panels or as having their applications reviewed there. 2418 02:13:58,000 --> 02:14:02,200 And we asked them to come up with their observations to 2419 02:14:02,200 --> 02:14:06,400 help drive the discussion in terms of what we should do 2420 02:14:06,400 --> 02:14:08,320 to put together new panels. 2421 02:14:08,320 --> 02:14:11,840 So I will list what the sort of high level ideas that 2422 02:14:11,840 --> 02:14:14,160 were identified at the beginning of the meeting. 2423 02:14:14,160 --> 02:14:18,360 The panel noted that there's often a tension between 2424 02:14:18,360 --> 02:14:22,080 the basic science applications and more applied in 2425 02:14:22,080 --> 02:14:25,800 translational topics, both in the chemistry panels and 2426 02:14:25,800 --> 02:14:27,680 the drug discovery panels. 2427 02:14:27,680 --> 02:14:33,000 This is a common feature in a lot of our study 2428 02:14:33,000 --> 02:14:33,480 sections. 2429 02:14:33,480 --> 02:14:36,600 But they noted that was prevalent here. 2430 02:14:36,600 --> 02:14:41,840 Very early screens to identify potential drug leads 2431 02:14:41,840 --> 02:14:47,720 would be better served in the chemistry panels rather 2432 02:14:47,720 --> 02:14:54,240 than in the later stage drug discovery channels, DDR or 2433 02:14:54,240 --> 02:14:54,920 DMP. 2434 02:14:54,920 --> 02:14:58,120 At the present time, if you were doing drug discovery 2435 02:14:58,120 --> 02:15:01,000 in a panel in a different biological system, there 2436 02:15:01,000 --> 02:15:04,240 isn't a place for those applications to go, even though 2437 02:15:04,240 --> 02:15:07,120 the methodologies they use for drug discovery are all 2438 02:15:07,120 --> 02:15:07,440 similar. 2439 02:15:07,440 --> 02:15:12,320 And then the panel also noted that an aspect of the 2440 02:15:12,320 --> 02:15:15,400 drug discovery and resistance study section currently 2441 02:15:15,400 --> 02:15:18,520 looks at anti-infected resistance and that's an 2442 02:15:18,520 --> 02:15:22,480 important aspect of drug efficacy that needed to be 2443 02:15:22,480 --> 02:15:26,920 considered when they looked at the new study section in 2444 02:15:26,920 --> 02:15:27,800 the space. 2445 02:15:27,800 --> 02:15:32,080 In terms of later stage drug discovery types of 2446 02:15:32,080 --> 02:15:35,880 applications and delivery, there was a feeling that 2447 02:15:35,880 --> 02:15:40,160 there needed to be more support for late-stage drug 2448 02:15:40,160 --> 02:15:43,520 development pre-IND work, that there wasn't enough 2449 02:15:43,520 --> 02:15:44,440 capacity there. 2450 02:15:44,440 --> 02:15:50,040 They also mentioned that biologics, not nucleic 2451 02:15:50,040 --> 02:15:55,600 acids, perhaps monocolonal antibodies, it was pointed 2452 02:15:55,600 --> 02:16:02,760 out that biologics are a growing area, but they 2453 02:16:02,760 --> 02:16:09,920 often are forgotten about There was an expectation that 2454 02:16:09,920 --> 02:16:17,880 there would be growth in the area of mRNA based 2455 02:16:17,880 --> 02:16:26,640 therapeutics and this was likely to be a place where we 2456 02:16:26,640 --> 02:16:33,040 might see more applications come in the future. 2457 02:16:33,040 --> 02:16:37,680 And then the last element in this area was that there 2458 02:16:37,680 --> 02:16:41,080 also isn't a place for advanced therapeutic studies 2459 02:16:41,080 --> 02:16:42,760 that don't involve cancer. 2460 02:16:42,760 --> 02:16:46,960 So there is a developmental therapeutics panel that 2461 02:16:46,960 --> 02:16:51,680 looks at later stage drug development types of studies, 2462 02:16:51,680 --> 02:16:55,320 but that only reviews applications related to 2463 02:16:55,320 --> 02:16:56,360 cancer-based topics. 2464 02:16:56,360 --> 02:17:01,680 And then there was one additional comment in the area 2465 02:17:01,680 --> 02:17:05,920 of environmental toxins that the workgroup noted that 2466 02:17:05,920 --> 02:17:09,640 currently environmental neurotoxins are reviewed in a 2467 02:17:09,640 --> 02:17:13,880 neurotoxicity and alcohol study section, but there's a 2468 02:17:14,400 --> 02:17:20,240 tension in that panel and they felt it would be worth 2469 02:17:20,240 --> 02:17:26,600 us having a look at some point in terms of whether the 2470 02:17:26,600 --> 02:17:30,840 environmental toxins might be better reviewed in the 2471 02:17:30,840 --> 02:17:34,560 context of other environmental non neuro-based systems. 2472 02:17:34,560 --> 02:17:40,840 So the panel was asked to identify what the main ideas 2473 02:17:40,840 --> 02:17:44,840 were in the applications we sent them. 2474 02:17:44,840 --> 02:17:48,480 So on a whiteboard, we put up what they felt to be the 2475 02:17:48,480 --> 02:17:49,040 main topics. 2476 02:17:49,040 --> 02:17:56,240 They came up with 21 broad different topic areas. 2477 02:17:56,240 --> 02:18:02,360 Over the couple-day period, we could assesed those into 2478 02:18:02,360 --> 02:18:08,480 areas with what they were proposing as new study 2479 02:18:08,480 --> 02:18:09,160 sections. 2480 02:18:09,160 --> 02:18:13,440 They clustered these 14 new proposed panels into 2481 02:18:13,440 --> 02:18:16,680 conceptual framework involving a drug discovery 2482 02:18:16,680 --> 02:18:20,400 pipeline with three phases: Enabling discovery, drug 2483 02:18:20,400 --> 02:18:24,680 discovery, and drug development, which is the later 2484 02:18:24,680 --> 02:18:25,240 stage. 2485 02:18:25,240 --> 02:18:29,640 So in the next few slides, I'm going to go through very 2486 02:18:29,640 --> 02:18:32,600 quickly the different panels that they envisioned in 2487 02:18:32,600 --> 02:18:34,080 each one of these. 2488 02:18:34,080 --> 02:18:37,120 The council has the external workgroup's report that 2489 02:18:37,120 --> 02:18:40,960 has all the of the detailed guidelines for these study 2490 02:18:40,960 --> 02:18:45,160 sections so you can look at those in more detail at 2491 02:18:45,160 --> 02:18:46,320 your own pace. 2492 02:18:46,320 --> 02:18:50,120 So the enabling discovery phase is the basic science 2493 02:18:50,120 --> 02:18:53,960 part of the drug discovery pipeline that they proposed. 2494 02:18:53,960 --> 02:18:57,320 They suggested redistributing the topics that are 2495 02:18:57,320 --> 02:19:01,200 currently reviewed in the two chemistry panels so 2496 02:19:01,200 --> 02:19:06,040 create one panel that is more very basic chemistry that 2497 02:19:06,040 --> 02:19:08,440 involved synthetic and biosynthetic methods. 2498 02:19:08,440 --> 02:19:12,680 And a separate panel that would be directed towards 2499 02:19:12,680 --> 02:19:16,000 more applied type of chemistry, including chemical 2500 02:19:16,000 --> 02:19:17,880 biology and medicinal chemistry. 2501 02:19:17,880 --> 02:19:24,600 Those were called chemical synthesis and biosyntheses 2502 02:19:24,600 --> 02:19:25,560 . 2503 02:19:25,560 --> 02:19:29,320 They suggested redistributing the drug delivery and 2504 02:19:29,320 --> 02:19:34,720 nano science topics from the GDP and nano panels into 2505 02:19:34,720 --> 02:19:35,240 three. 2506 02:19:35,240 --> 02:19:37,000 Those panels are a little bit big at the moment. 2507 02:19:37,000 --> 02:19:40,400 And they pulled out the very basic nanotechnology 2508 02:19:40,400 --> 02:19:43,800 aspect material sciences and development of new nano 2509 02:19:43,800 --> 02:19:47,600 particles into a panel called innovations and non know 2510 02:19:47,600 --> 02:19:51,440 systems and nanotechnology that would look at the basic 2511 02:19:51,440 --> 02:19:53,120 science aspects of nano. 2512 02:19:53,120 --> 02:19:55,840 And they separated out delivery aspects that are going 2513 02:19:55,840 --> 02:19:58,560 to show up in one of the later phases. 2514 02:19:58,560 --> 02:20:03,360 And then they suggested expanding the topics covered by 2515 02:20:03,360 --> 02:20:08,120 the existing panel SIEE, which is systemic injury from 2516 02:20:08,120 --> 02:20:11,880 environmental exposure to more broadly study etiology 2517 02:20:11,880 --> 02:20:15,040 outcomes and mechanisms of environmentally induced 2518 02:20:15,040 --> 02:20:15,600 disease. 2519 02:20:15,600 --> 02:20:18,280 They called that panel environmental determinants of 2520 02:20:18,280 --> 02:20:18,680 disease. 2521 02:20:18,680 --> 02:20:24,160 The drug discovery phase, which they described as being 2522 02:20:24,160 --> 02:20:29,640 studies where early panics can be tested which are 2523 02:20:29,640 --> 02:20:35,120 currently reviewed in DDR, DMP and the recurring panel 2524 02:20:35,120 --> 02:20:40,000 SEP A, DC82, they suggested merging those applications 2525 02:20:40,000 --> 02:20:45,480 for those areas together to form four drug discovery 2526 02:20:45,480 --> 02:20:50,960 panels that each have very similar guidelines, but a 2527 02:20:50,960 --> 02:20:52,800 different biological system. 2528 02:20:52,800 --> 02:20:57,200 So DMPA, win focused on anti-bacterial, anti-fungal and 2529 02:20:57,200 --> 02:21:01,040 anti-parasitic drug discovery, DMPB which focused on 2530 02:21:01,600 --> 02:21:06,000 antiviral drug discovery, DMPC, which would be cancer 2531 02:21:06,000 --> 02:21:10,960 drug discovery and DMPD would be neuroscience and other 2532 02:21:10,960 --> 02:21:14,800 assorted biological systems that people were interested 2533 02:21:14,800 --> 02:21:17,560 in developing therapeutic compounds for. 2534 02:21:17,560 --> 02:21:25,840 And they also suggested creating a panel that would 2535 02:21:25,840 --> 02:21:32,320 focus discovery and drug efficacy area called 2536 02:21:32,320 --> 02:21:36,000 anti-infective resistance and targets. 2537 02:21:36,000 --> 02:21:42,880 The last thing they recommended was to absorb the small 2538 02:21:42,880 --> 02:21:47,000 BST(55) SEP which involves high throughput screening 2539 02:21:47,000 --> 02:21:51,840 applications into the relevant drug discovery panels, 2540 02:21:51,840 --> 02:21:53,880 DMPA through D. 2541 02:21:53,880 --> 02:21:55,920 The last phase was the drug development phase. 2542 02:21:55,920 --> 02:21:59,440 The remaining topics from existing drug delivery and 2543 02:21:59,440 --> 02:22:02,520 nanoscience panels, they included splitting those to 2544 02:22:03,360 --> 02:22:07,320 a panel that was focused on the nucleic acid 2545 02:22:07,320 --> 02:22:11,280 therapeutic delivery and a second that would focus on 2546 02:22:11,280 --> 02:22:14,320 small molecule and buy logic therapeutic delivery. 2547 02:22:14,320 --> 02:22:21,720 Topics currently reviewed in the XMDA panel, which is 2548 02:22:21,720 --> 02:22:27,440 basically limited to sympatic systems, they suggested 2549 02:22:27,440 --> 02:22:34,000 any cell type, organ system or anatomical 2550 02:22:34,000 --> 02:22:41,360 compartment, that would be called drug and buy logic 2551 02:22:41,360 --> 02:22:43,840 disposition and toxicology. 2552 02:22:43,840 --> 02:22:49,560 They suggested the broadly involving development of new 2553 02:22:49,560 --> 02:22:53,840 therapeutic strategies Forney row plastic diseases, 2554 02:22:53,840 --> 02:22:59,560 solid tumors and leukemias, were mapped to advancing 2555 02:22:59,560 --> 02:23:01,000 therapeutics A. 2556 02:23:01,000 --> 02:23:04,280 Therapeutics strategies would map to the therapeutics B 2557 02:23:04,280 --> 02:23:04,680 panel. 2558 02:23:04,680 --> 02:23:10,720 And then the last study section included in this grew, 2559 02:23:10,720 --> 02:23:15,560 cancer etiology which has mitogens that lead to 2560 02:23:15,560 --> 02:23:21,000 oncogenesis, they didn't see that as being released to 2561 02:23:21,000 --> 02:23:23,440 the drug discovery pipeline. 2562 02:23:23,440 --> 02:23:28,760 It's related to SINDE but it's not part of the pipeline 2563 02:23:28,760 --> 02:23:30,200 that they envisioned. 2564 02:23:30,200 --> 02:23:33,560 But the panel members recommended that the topics 2565 02:23:33,560 --> 02:23:37,360 currently reviewed in CE were basic cancer biology that 2566 02:23:37,360 --> 02:23:41,600 very well could be reviewed in the context of other 2567 02:23:41,600 --> 02:23:43,720 basic cancer biology study sections. 2568 02:23:43,720 --> 02:23:47,040 Those are currently being organized via ENQUIRE from 2569 02:23:47,040 --> 02:23:51,160 cluster 5, which you heard about at the last CSR 2570 02:23:51,160 --> 02:23:52,000 advisory council. 2571 02:23:52,000 --> 02:23:58,800 So the plan is to absorb the bullet points from the 2572 02:23:58,800 --> 02:24:01,280 cancer etiology study section. 2573 02:24:01,280 --> 02:24:04,720 Those topics will be added to the new sections 2574 02:24:04,720 --> 02:24:05,880 currently being formed. 2575 02:24:05,880 --> 02:24:10,160 And those will be implemented for the January 2023 2576 02:24:10,160 --> 02:24:11,080 council round. 2577 02:24:11,080 --> 02:24:15,040 We assembled a internal workgroup meeting that met 2578 02:24:15,040 --> 02:24:16,520 February and March. 2579 02:24:16,520 --> 02:24:19,840 The internal panels includes the CSR division 2580 02:24:19,840 --> 02:24:23,160 directors, IRG chiefs, senior level individuals from 2581 02:24:23,160 --> 02:24:25,520 the involves ICs listed there. 2582 02:24:25,520 --> 02:24:27,960 The internal panel was asked to look at the 2583 02:24:27,960 --> 02:24:29,560 recommendations from the external workgroup for 2584 02:24:29,560 --> 02:24:32,000 potential issues that would arise if we were to 2585 02:24:32,000 --> 02:24:32,560 implement them. 2586 02:24:32,560 --> 02:24:35,760 They looked at survey information from POs and 2587 02:24:35,760 --> 02:24:39,400 reviewers about the existing panels as well as notes 2588 02:24:39,400 --> 02:24:42,640 from observers who visited the sites, the study 2589 02:24:42,640 --> 02:24:43,040 sections. 2590 02:24:43,040 --> 02:24:46,040 The internal panel recommended, they expressed 2591 02:24:46,040 --> 02:24:49,560 concurrence with the general recommendations from the 2592 02:24:49,560 --> 02:24:50,560 external workgroup. 2593 02:24:50,560 --> 02:24:53,960 They really didn't have any issues that they raised. 2594 02:24:53,960 --> 02:24:56,680 There were a few minor modifications that they 2595 02:24:56,680 --> 02:24:57,000 suggestion. 2596 02:24:57,000 --> 02:25:00,800 They recommended instead of using the initial names, 2597 02:25:00,800 --> 02:25:05,560 which I didn't show on these slides, which were more 2598 02:25:05,560 --> 02:25:09,360 specific and included the biological systems in the 2599 02:25:09,360 --> 02:25:13,640 titles, that we replace those with general names, drug 2600 02:25:13,640 --> 02:25:17,440 discoveries and molecular pharmacology A through D and 2601 02:25:17,440 --> 02:25:21,280 advancing therapeutics A and B instead of biological 2602 02:25:21,280 --> 02:25:25,560 named panels for the reason it's very difficult to 2603 02:25:25,560 --> 02:25:29,840 include all the subdisciplines we might have going to 2604 02:25:29,840 --> 02:25:33,640 those study sections and these general names don't 2605 02:25:33,640 --> 02:25:35,520 exclude any particular community. 2606 02:25:35,520 --> 02:25:39,280 Then they said some minor adjustments including change 2607 02:25:39,280 --> 02:25:43,040 and review loci for some orphaned environmental studies 2608 02:25:43,040 --> 02:25:46,800 that were reviewed elsewhere and provided input about 2609 02:25:46,800 --> 02:25:50,040 referral overlap boundaries where the boundary is 2610 02:25:50,040 --> 02:25:53,800 between the synthetic chemistry panel and the drug 2611 02:25:53,800 --> 02:25:54,760 discovery panels. 2612 02:25:54,760 --> 02:25:58,200 In the end, there were 14 study sections proposed. 2613 02:25:58,200 --> 02:26:03,480 These would be implemented in May of 2023 if the 2614 02:26:03,480 --> 02:26:04,520 council agrees. 2615 02:26:04,520 --> 02:26:08,440 And these are listed here and I've gone through them, 2616 02:26:08,440 --> 02:26:11,560 so I won't read them again for you. 2617 02:26:11,560 --> 02:26:16,920 And then the last slide I want to show you would be the 2618 02:26:16,920 --> 02:26:20,600 proposed next steps assuming the council concurs. We 2619 02:26:20,600 --> 02:26:23,920 would refine if the draft guidelines and overlap 2620 02:26:23,920 --> 02:26:28,040 statements next month and through May. We would look 2621 02:26:28,040 --> 02:26:31,320 at mock-sock data using historical application data to 2622 02:26:31,320 --> 02:26:35,440 see what the study section sizes looked like and make 2623 02:26:35,440 --> 02:26:39,520 adjustments. We would post new guidelines to the CSR 2624 02:26:39,520 --> 02:26:44,080 website in August this year been which would give us a 2625 02:26:44,080 --> 02:26:47,360 couple of months before the first applications to 2626 02:26:47,360 --> 02:26:51,040 arrive to the first receipt dates where the study 2627 02:26:51,040 --> 02:26:55,160 sections would go live would be for the October and 2628 02:26:55,160 --> 02:26:56,400 November receipt dates. 2629 02:26:56,400 --> 02:26:58,600 Our current study section members would be transferred 2630 02:26:58,600 --> 02:27:01,320 to the new panels late this year, December of 2022. 2631 02:27:01,320 --> 02:27:04,680 And then the first meeting of these new study sections 2632 02:27:04,680 --> 02:27:07,360 would take place January and February of 2023. 2633 02:27:07,360 --> 02:27:11,560 And those applications that were reviewed during those 2634 02:27:11,560 --> 02:27:16,800 meetings would go to advisory council for May and June 2635 02:27:16,800 --> 02:27:17,840 of 2023. 2636 02:27:17,840 --> 02:27:18,880 Okay. 2637 02:27:18,880 --> 02:27:22,920 So I'm going to stop there and take questions or 2638 02:27:22,920 --> 02:27:27,400 comments and give Tonya a chance to give any input that 2639 02:27:27,400 --> 02:27:29,040 she would like to. 2640 02:27:29,040 --> 02:27:33,760 And we'll come back and I'll ask whether the council 2641 02:27:33,760 --> 02:27:34,240 concurs. 2642 02:27:34,240 --> 02:27:39,280 So I'm going to stop sharing my slides. 2643 02:27:39,280 --> 02:27:40,200 >> DR. BRUCE REED: All right. 2644 02:27:40,200 --> 02:27:42,320 So thanks, Ray. 2645 02:27:42,320 --> 02:27:45,360 Tonya, do you have comments? 2646 02:27:45,360 --> 02:27:47,640 >> DR. TONYA PALERMO: Quick comments as the observer. 2647 02:27:47,640 --> 02:27:51,120 This topic is pretty far outside my own area of 2648 02:27:51,120 --> 02:27:54,240 science, so definitely was in an observer only role. 2649 02:27:54,240 --> 02:27:58,040 As Ray mentioned, I thought this group did a neat job 2650 02:27:58,040 --> 02:28:01,160 of pulling in the conceptualization of how they wanted 2651 02:28:01,160 --> 02:28:02,200 to approach this. 2652 02:28:02,200 --> 02:28:05,440 Once they did that, they were able to fit in all of 2653 02:28:05,440 --> 02:28:06,760 these topic areas quite nicely. 2654 02:28:06,760 --> 02:28:12,200 The biggest job for the them was reducing the number of 2655 02:28:12,200 --> 02:28:14,200 topics that they had. 2656 02:28:14,200 --> 02:28:16,200 So, yeah, kudos to the group. 2657 02:28:16,200 --> 02:28:19,520 I thought they did an awesome job. 2658 02:28:19,520 --> 02:28:25,480 >> DR. BRUCE REED: Thanks, Tonya. 2659 02:28:25,480 --> 02:28:31,880 >> DR. RAY JACOBSON: I think Mark had his hand up. 2660 02:28:31,880 --> 02:28:33,080 >> DR. BRUCE REED: No. 2661 02:28:33,080 --> 02:28:36,480 Sean did. 2662 02:28:36,480 --> 02:28:40,400 >> DR. SEAN DAVIDSON: If you haven't done the mock 2663 02:28:40,400 --> 02:28:43,240 application sorts, how often do those change the 2664 02:28:43,240 --> 02:28:46,080 outcomes of what the internal people have decided? 2665 02:28:46,080 --> 02:28:51,760 And does it have to come back to this committee if 2666 02:28:51,760 --> 02:28:57,400 there's a big change or is that kind ooh a performer 2667 02:28:57,400 --> 02:28:59,960 type thing that gets done? 2668 02:28:59,960 --> 02:29:03,640 >> DR. RAY JACOBSON: The mock-sort data can affect the 2669 02:29:03,640 --> 02:29:06,960 outcome. Unless there was a large change to the 2670 02:29:06,960 --> 02:29:09,960 framework that was being proposed, we would not bring 2671 02:29:09,960 --> 02:29:11,640 it back to the committee. 2672 02:29:11,640 --> 02:29:17,920 So, for example, let's say one of the panels, DMPA 2673 02:29:17,920 --> 02:29:23,600 through D was too small because the numbers have 2674 02:29:23,600 --> 02:29:29,920 drifted since we looked at the historical data. We 2675 02:29:29,920 --> 02:29:34,960 would likely compress those through into three panels, 2676 02:29:34,960 --> 02:29:36,200 for example. 2677 02:29:36,200 --> 02:29:40,720 That has happened in the past and we have done that 2678 02:29:40,720 --> 02:29:44,360 because we ultimately need to have study sections that 2679 02:29:44,360 --> 02:29:46,000 are a functional size. 2680 02:29:46,000 --> 02:29:49,040 Karen. 2681 02:29:49,040 --> 02:29:51,280 >> DR. KAREN ANDERSON: Thank you very much. 2682 02:29:51,280 --> 02:29:55,040 It's obviously an impressive portfolio. 2683 02:29:55,040 --> 02:30:00,440 There is obviously a clear distinction between the 2684 02:30:00,440 --> 02:30:05,840 basic science and the development of the biology 2685 02:30:05,840 --> 02:30:09,880 development, cancer biology, microbiology and the 2686 02:30:09,880 --> 02:30:11,240 therapeutics development. 2687 02:30:11,240 --> 02:30:16,720 Is there any mechanism or effort to at least foster 2688 02:30:16,720 --> 02:30:20,560 collaborations between these or across thinking? 2689 02:30:20,560 --> 02:30:25,480 Because now if you're developing basic science of a 2690 02:30:25,480 --> 02:30:29,840 microbial pathogen and you're studying that, you might 2691 02:30:29,840 --> 02:30:34,200 not be thinking about the issues involved in 2692 02:30:34,200 --> 02:30:38,040 translation to therapeutics until five years after 2693 02:30:38,040 --> 02:30:40,240 you're completing your project. 2694 02:30:40,240 --> 02:30:44,320 And those can have, you know, clear delays in terms of 2695 02:30:44,320 --> 02:30:45,800 the development into therapeutics. 2696 02:30:45,800 --> 02:30:48,080 How do you balance that? 2697 02:30:48,080 --> 02:30:51,880 >> DR. RAY JACOBSON: That is a great point. And it 2698 02:30:51,880 --> 02:30:54,520 was brought up in the internal panel discussions, as 2699 02:30:54,520 --> 02:30:57,760 well, that you need some input of -- in items of input 2700 02:30:57,760 --> 02:31:00,080 from the translational perspective when your doing the 2701 02:31:00,080 --> 02:31:01,400 basic science, as well. 2702 02:31:01,400 --> 02:31:06,760 I think what saved things a little bit is that the 2703 02:31:06,760 --> 02:31:11,160 typically these people don't fall neatly into one box, 2704 02:31:11,160 --> 02:31:11,640 right? 2705 02:31:11,640 --> 02:31:15,920 So if you look at the chemists who observed, many of 2706 02:31:15,920 --> 02:31:19,400 the people who served, served on the basic science 2707 02:31:19,400 --> 02:31:22,520 chemistry panels, but they're pulled right now to 2708 02:31:22,520 --> 02:31:26,760 review in the sort of mid or later stage drug discovery 2709 02:31:26,760 --> 02:31:27,160 panels. 2710 02:31:27,160 --> 02:31:30,360 So people -- even though we have boxes where we put 2711 02:31:30,360 --> 02:31:33,600 things in terms of how we review things, I think 2712 02:31:33,600 --> 02:31:36,480 expertise of reviewers is often broader than our study 2713 02:31:36,480 --> 02:31:37,320 section guidelines. 2714 02:31:37,320 --> 02:31:40,000 It doesn't completely help and it's something that the 2715 02:31:40,000 --> 02:31:42,680 management perspective we have to keep an eye on. 2716 02:31:42,680 --> 02:31:49,880 And that is something in terms of how we look at 2717 02:31:49,880 --> 02:31:55,760 significance when we're asking to reviewers to do that. 2718 02:31:55,760 --> 02:31:58,880 That is something for the chair to make sure that input 2719 02:31:58,880 --> 02:31:59,720 is there, too. 2720 02:31:59,720 --> 02:32:04,680 >> DR. BRUCE REED: Let me jump in at this point. We 2721 02:32:04,680 --> 02:32:08,600 have a fairly busy agenda ahead and this is an area -- 2722 02:32:08,600 --> 02:32:12,480 this was a review where the external panel came up with 2723 02:32:12,480 --> 02:32:16,400 a nice conceptual and very clear layout. And it seemed 2724 02:32:16,400 --> 02:32:19,960 like there was a high degree of concurrence from the 2725 02:32:19,960 --> 02:32:20,720 internal panel. 2726 02:32:20,720 --> 02:32:26,200 So I'd like to hear if there are any objections or 2727 02:32:26,200 --> 02:32:30,160 concerns from council regarding what the panels had 2728 02:32:30,160 --> 02:32:30,680 suggested. 2729 02:32:30,680 --> 02:32:34,760 And if we don't have concerns, then I'd like to move 2730 02:32:34,760 --> 02:32:37,360 ahead and get concurrence with those recommendations. 2731 02:32:37,360 --> 02:32:43,280 But I certainly do want to leave time if there are 2732 02:32:43,280 --> 02:32:45,960 concerns that anybody has got. 2733 02:32:45,960 --> 02:32:49,600 Jinming, you had your hand up. 2734 02:32:49,600 --> 02:32:52,800 >> DR. JINMING GAO: It is not a serious concern, but a 2735 02:32:52,800 --> 02:32:54,040 minor concern about the labelling. 2736 02:32:54,040 --> 02:32:58,120 The labelling discovery and the drug developments. 2737 02:32:58,120 --> 02:33:02,200 I would start from the drug development because I am 2738 02:33:02,200 --> 02:33:05,480 getting into helping translate the technology from the 2739 02:33:05,480 --> 02:33:10,400 lab into the clinic. I feel like then the typical drug 2740 02:33:10,400 --> 02:33:14,080 development, the definition, there are a lot of CMC, 2741 02:33:14,080 --> 02:33:17,760 you know, the chemistry manufacturing, a lot of the 2742 02:33:17,760 --> 02:33:20,600 practical end of the analytical thoughts development, 2743 02:33:20,600 --> 02:33:24,680 the QAQCs, I wonder whether that's a good name to 2744 02:33:24,680 --> 02:33:25,080 characterize. 2745 02:33:25,080 --> 02:33:31,000 To me, all of these different topics are still on drug 2746 02:33:31,000 --> 02:33:34,760 delivery technology, maybe aren't, you know, nucleic 2747 02:33:34,760 --> 02:33:35,800 acid delivery. 2748 02:33:35,800 --> 02:33:39,560 So it's not quite striking me as a drug development. 2749 02:33:39,560 --> 02:33:42,240 Maybe as an early early stage, but I just -- I don't 2750 02:33:42,240 --> 02:33:43,800 have a good name for it. 2751 02:33:43,800 --> 02:33:47,760 If I think of one, I can probably just email that. 2752 02:33:47,760 --> 02:33:51,440 And then the clarification on enabling discovery, I 2753 02:33:51,440 --> 02:33:56,000 wonder whether that is just focusing on all the effort 2754 02:33:56,000 --> 02:34:00,600 on the prediscovery side or is it more the technology 2755 02:34:00,600 --> 02:34:04,280 piece that, you know, instead of just enabling 2756 02:34:04,280 --> 02:34:07,960 discovery doesn't seem to come across as clear. 2757 02:34:07,960 --> 02:34:12,040 There are a lot of things enabling discovery, but I 2758 02:34:12,040 --> 02:34:15,280 wonder whether that is a -- the innovation and 2759 02:34:15,280 --> 02:34:18,160 technology part that's, you know, enabling the 2760 02:34:18,160 --> 02:34:22,240 discovery is a goal, but it's the focus is developing, 2761 02:34:22,240 --> 02:34:25,080 you know, innovative probes or, you know, 2762 02:34:25,080 --> 02:34:26,000 nanotechnology, nanosystems. 2763 02:34:26,000 --> 02:34:29,560 And then, you know, leading the discovery. 2764 02:34:29,560 --> 02:34:35,280 So also thinking that discovery might be better 2765 02:34:35,280 --> 02:34:36,000 phrased. 2766 02:34:36,000 --> 02:34:39,120 So two minor, I guess, suggestion webs not really major 2767 02:34:39,120 --> 02:34:39,400 concerns. 2768 02:34:39,400 --> 02:34:43,040 >> DR. RAY JACOBSON: So are you asking about the broad 2769 02:34:43,040 --> 02:34:45,440 category titles or the individual study section titles? 2770 02:34:45,440 --> 02:34:48,800 >> DR. JINMING GAO: Really the broad categorical -- 2771 02:34:48,800 --> 02:34:51,640 >> DR. RAY JACOBSON: So no one will ever see those 2772 02:34:51,640 --> 02:34:51,880 again. 2773 02:34:51,880 --> 02:34:54,920 Those are basically a framework for us to think about 2774 02:34:54,920 --> 02:34:58,920 how to put them together. When you go on to the site, 2775 02:34:58,920 --> 02:35:01,360 it will be the individual study section names. 2776 02:35:01,360 --> 02:35:05,960 But your points are -- you know, it's hard to get a 2777 02:35:05,960 --> 02:35:09,080 name that captures everything at this point. 2778 02:35:09,080 --> 02:35:10,680 That's what we suffered from, probably. 2779 02:35:10,680 --> 02:35:11,600 >> DR. BRUCE REED: Thanks. 2780 02:35:11,600 --> 02:35:14,640 Okay. Can I get a motion from council on this? 2781 02:35:14,640 --> 02:35:21,640 I think this is -- I think I'm looking for a motion to 2782 02:35:21,640 --> 02:35:26,320 concur with the recommendations of the external panel. 2783 02:35:26,560 --> 02:35:27,920 External and internal, I guess. 2784 02:35:27,920 --> 02:35:29,200 >> DR. MARK PEIFER: So moved. 2785 02:35:29,200 --> 02:35:32,880 >> DR. BRUCE REED: Mark moved. 2786 02:35:32,880 --> 02:35:34,600 >> DR. NONI BYRNES: Second. 2787 02:35:34,600 --> 02:35:37,840 >> DR. BRUCE REED: Can I see anybody who is opposed. 2788 02:35:37,840 --> 02:35:38,360 . 2789 02:35:38,360 --> 02:35:41,800 Not seeing any, so that carries unanimously. 2790 02:35:41,800 --> 02:35:43,080 Thank you, everybody. 2791 02:35:43,080 --> 02:35:48,200 Okay. 2792 02:35:48,200 --> 02:35:50,560 Hope Cummings has been waiting patiently. 2793 02:35:50,560 --> 02:35:53,560 She has some interesting data to provide you. 2794 02:35:53,560 --> 02:35:59,600 Just for context, I think all of you know this summer 2795 02:35:59,600 --> 02:36:04,560 CSR pushed out training for reviewers on bias awareness 2796 02:36:04,560 --> 02:36:08,960 and bias mitigation and a peer reviewed context. 2797 02:36:08,960 --> 02:36:14,200 This is a very important effort to us. We are 2798 02:36:14,200 --> 02:36:17,560 interested in assessing how effective it is. 2799 02:36:17,560 --> 02:36:20,960 Part of that is getting feedback from the reviewers 2800 02:36:20,960 --> 02:36:23,600 themselves about their experience with this training. 2801 02:36:23,600 --> 02:36:32,200 Hope conducted a survey on this and she's here this 2802 02:36:32,200 --> 02:36:39,960 afternoon to talk to us about what she's learned. 2803 02:36:39,960 --> 02:36:45,120 >> DR. HOPE CUMMINGS: Thank you, Bruce. 2804 02:36:45,120 --> 02:36:50,040 The training was developed by a diverse working group 2805 02:36:50,040 --> 02:36:54,960 that included extramural sciences as well as staff and 2806 02:36:54,960 --> 02:36:59,320 CSR, the office of scientific workforce diversity and 2807 02:36:59,320 --> 02:37:03,720 the institute of general medical sciences. Several 2808 02:37:03,720 --> 02:37:08,640 members had background in expertise and bias and bias 2809 02:37:08,640 --> 02:37:09,160 training. 2810 02:37:09,160 --> 02:37:13,840 So to help reviewers develop ways to mitigate bias. 2811 02:37:13,840 --> 02:37:19,280 The training's heart is the most common bias in review 2812 02:37:19,280 --> 02:37:24,200 and was not explicitly designed to be implicit bias 2813 02:37:24,200 --> 02:37:24,720 training. 2814 02:37:24,720 --> 02:37:28,640 The overall focus is on bias related to the principal 2815 02:37:28,640 --> 02:37:31,400 investigator and institutional reputation and on the 2816 02:37:31,400 --> 02:37:34,920 importance of adhering to the review criteria in the 2817 02:37:34,920 --> 02:37:36,840 areas of investigator and environment. 2818 02:37:36,840 --> 02:37:39,320 The training is approximately 30 minutes long. 2819 02:37:39,320 --> 02:37:42,760 It begins with testimonials from real NIH grant 2820 02:37:42,760 --> 02:37:46,200 applicants who describe instances of bias that they 2821 02:37:46,200 --> 02:37:49,240 have experienced in review of their applications. 2822 02:37:49,240 --> 02:37:55,160 They move on and engages reviewers in a short activity 2823 02:37:55,160 --> 02:37:59,920 that illustrates the implicit biases that language can 2824 02:37:59,920 --> 02:38:00,520 elicit. 2825 02:38:00,520 --> 02:38:04,880 The next activity focusing on reviewing a statement and 2826 02:38:04,880 --> 02:38:05,360 critique. 2827 02:38:05,360 --> 02:38:11,160 Last is a study section that presents two instances of 2828 02:38:11,160 --> 02:38:14,640 a review of a flawed application. 2829 02:38:14,640 --> 02:38:21,200 The first demonstrates two versions in which viewers 2830 02:38:21,200 --> 02:38:26,120 intervene to mitigate the bias remarks. 2831 02:38:26,120 --> 02:38:28,960 So for the January 2022 council round, almost 10,000 2832 02:38:28,960 --> 02:38:31,800 reviewers who came from 412 study sections were asked 2833 02:38:31,800 --> 02:38:34,640 to take the training four weeks before their fall 2834 02:38:34,640 --> 02:38:37,800 meeting. They were sent a reminder email two weeks 2835 02:38:37,800 --> 02:38:38,720 before the meeting. 2836 02:38:38,720 --> 02:38:43,120 By the end of January, around 6,000 reviewers took the 2837 02:38:43,120 --> 02:38:45,760 training for a 60% response rate. 2838 02:38:45,760 --> 02:38:50,280 At the end of the training, reviewers were asked if 2839 02:38:50,280 --> 02:38:54,360 they wanted to complete a survey of the vacation 2840 02:38:54,360 --> 02:38:57,520 purchase we had a 53% response rate. 2841 02:38:57,520 --> 02:39:03,360 These reviewers represented 389 of the study sections. 2842 02:39:03,360 --> 02:39:07,000 An additional 5,300 reviewers have taken the training 2843 02:39:07,440 --> 02:39:10,200 for the May 22 council round. 2844 02:39:10,200 --> 02:39:13,680 The evaluation consisted of nine questions that measure 2845 02:39:13,680 --> 02:39:16,760 reviewer's general satisfaction with the training and 2846 02:39:16,760 --> 02:39:19,800 whether the training helped them become more 2847 02:39:19,800 --> 02:39:22,880 knowledgeable and aware of bias peer review. 2848 02:39:22,880 --> 02:39:25,640 If the training helped them feel more comfortable as 2849 02:39:25,640 --> 02:39:29,040 well as how they might intervene in the future and on 2850 02:39:29,040 --> 02:39:31,480 the usefulness of some of the specific training 2851 02:39:31,480 --> 02:39:33,040 activities that I described earlier. 2852 02:39:33,040 --> 02:39:37,360 There were a couple of other measures not directly 2853 02:39:37,360 --> 02:39:41,640 related to the training, but were included to help 2854 02:39:41,640 --> 02:39:44,520 gather reviewers' behaviors and perceptions regarding 2855 02:39:44,520 --> 02:39:45,960 peer review bias. 2856 02:39:45,960 --> 02:39:49,000 These measures included the presence of bias, their 2857 02:39:49,000 --> 02:39:52,000 ability to correctly identify bias and their frequently 2858 02:39:52,000 --> 02:39:53,120 intervening on bias. 2859 02:39:53,120 --> 02:39:56,560 The survey concluded with an open-ended field where 2860 02:39:56,560 --> 02:40:00,040 reviewers could share their overall comments to review 2861 02:40:00,040 --> 02:40:02,600 and recommendations for improving the training. 2862 02:40:02,600 --> 02:40:07,880 As I move to the results, I'm going to start with the 2863 02:40:07,880 --> 02:40:11,840 findings on reviewers' perceptions on bias and the year 2864 02:40:11,840 --> 02:40:15,800 prior to the training just to provide some background 2865 02:40:15,800 --> 02:40:16,240 context. 2866 02:40:16,240 --> 02:40:21,360 So if you look at the first graph, you can see the 2867 02:40:21,360 --> 02:40:25,160 great majority of reviewers believe NIH has a problem 2868 02:40:25,160 --> 02:40:29,400 with bias and review with 45% of reviewers believe bias 2869 02:40:29,400 --> 02:40:34,080 is a moderate problem and 42% think there's a big or 2870 02:40:34,080 --> 02:40:35,360 very big problem. 2871 02:40:35,360 --> 02:40:39,440 In the last year, they read a critique or attended a 2872 02:40:39,440 --> 02:40:42,440 review meeting where they felt bias was present. 2873 02:40:42,440 --> 02:40:46,560 55% of reviewers reported they sometimes did and 39% 2874 02:40:46,560 --> 02:40:51,600 said they never or rarely attend a critique or read a 2875 02:40:51,600 --> 02:40:55,280 review meeting where they felt bias was present. 2876 02:40:55,280 --> 02:40:58,720 In regard to reviewers ability to correctly identified 2877 02:40:58,720 --> 02:41:02,600 bias in the last year, about half were moderately 2878 02:41:02,600 --> 02:41:04,720 confident they could do so. 2879 02:41:04,720 --> 02:41:08,520 Approximately 30% were extremely confident in their 2880 02:41:08,520 --> 02:41:12,880 ability to identify bias in the last year. 2881 02:41:12,880 --> 02:41:16,080 The results show most reviewers, or 53%, never or 2882 02:41:16,080 --> 02:41:19,320 rarely intervened when they thought bias was present. 2883 02:41:19,320 --> 02:41:22,520 Only 4% of reviewers reported that they always enter 2884 02:41:22,520 --> 02:41:25,000 vined on bias in the last year. 2885 02:41:25,000 --> 02:41:25,360 Okay. 2886 02:41:25,360 --> 02:41:31,160 So now I'm turning to the results of the training 2887 02:41:31,160 --> 02:41:31,720 itself. 2888 02:41:31,720 --> 02:41:34,960 Overall, reviewers were highly satisfied with the 2889 02:41:34,960 --> 02:41:39,080 training. Around 90% agreed or strongly agreed that 2890 02:41:39,080 --> 02:41:42,800 the training met their needs and expectations, the 2891 02:41:42,800 --> 02:41:46,480 testimonials were believing and that the training was 2892 02:41:46,480 --> 02:41:46,920 engaging. 2893 02:41:46,920 --> 02:41:50,040 About assessing whether the training helped reviewers 2894 02:41:50,040 --> 02:41:54,040 many more knowledgeable, the results show around 60% of 2895 02:41:54,040 --> 02:41:58,080 reviewers thought the training helped them to a large 2896 02:41:58,080 --> 02:41:58,520 extent. 2897 02:41:58,520 --> 02:42:01,360 One, identify potential biases in review, two, 2898 02:42:01,360 --> 02:42:04,640 understand the importance of using the review criteria 2899 02:42:04,640 --> 02:42:07,480 to assess and avoid bias and review. 2900 02:42:07,480 --> 02:42:10,840 Three, write an unbiased critique or present an unbias 2901 02:42:10,840 --> 02:42:14,160 argument during a review meeting and last that the 2902 02:42:14,160 --> 02:42:17,480 training helped them become more aware of the personal 2903 02:42:17,480 --> 02:42:19,360 review biases that they hold. 2904 02:42:19,360 --> 02:42:24,560 Most reviewers, 60% say they helped to a large extent 2905 02:42:24,560 --> 02:42:27,200 across all of these items. 2906 02:42:27,200 --> 02:42:30,880 To follow, approximately 30% of reviewers report the 2907 02:42:30,880 --> 02:42:34,120 training helped them to a moderate extent. 2908 02:42:34,120 --> 02:42:35,720 This is across all four items. 2909 02:42:35,720 --> 02:42:39,840 So between these two categories, around 90% of 2910 02:42:39,840 --> 02:42:43,440 reviewers believe the trainings really increased their 2911 02:42:43,440 --> 02:42:47,000 knowledge around awareness and bias and review. 2912 02:42:47,000 --> 02:42:50,640 The results showed that reviewers felt the training 2913 02:42:50,640 --> 02:42:54,280 helped to feel more comfortable intervening on bias 2914 02:42:54,280 --> 02:42:58,840 with around 70% believe it helped them to a large 2915 02:42:58,840 --> 02:43:00,680 extent feel more comfortable. 2916 02:43:00,680 --> 02:43:04,720 Only a very small percentage of reviewers, 7% to be 2917 02:43:04,720 --> 02:43:08,360 exact, reported that the training had little or no 2918 02:43:08,360 --> 02:43:11,600 effect in helping them to feel more comfortable 2919 02:43:11,600 --> 02:43:12,800 intervening on bias. 2920 02:43:12,800 --> 02:43:15,920 Now beyond reviewers' feeling more comfortable with 2921 02:43:15,920 --> 02:43:19,080 intervening, the results showed reviewers were highly 2922 02:43:19,080 --> 02:43:23,560 likely to take actions in the future around bias. 2923 02:43:23,560 --> 02:43:27,120 Around 82% stated that they would definitely or 2924 02:43:27,120 --> 02:43:30,720 probably ask for clarification or justification from a 2925 02:43:30,720 --> 02:43:34,760 reviewer if they felt the reviewer made a bias 2926 02:43:34,760 --> 02:43:36,560 statement during the meeting. 2927 02:43:36,560 --> 02:43:40,880 Approximately 60% of reviewers would definitely or 2928 02:43:40,880 --> 02:43:46,440 probably contact the SRO or Chair if they perceived 2929 02:43:46,440 --> 02:43:51,400 bias in a critique or during a meeting. 2930 02:43:51,400 --> 02:43:57,360 In regard to the specific training activities, around 2931 02:43:57,360 --> 02:44:03,360 90% of reviewers thought the training activities 2932 02:44:03,360 --> 02:44:05,600 were very useful. 2933 02:44:05,600 --> 02:44:10,240 Quickly moving on to the qualitative results, here 2934 02:44:10,240 --> 02:44:14,880 reviewers were given the opportunity to share any 2935 02:44:14,880 --> 02:44:19,520 general comments about bias in NIH review or 2936 02:44:19,520 --> 02:44:22,440 recommendations for improving the training. 2937 02:44:22,440 --> 02:44:25,960 Given this it's an open-ended question, it tends to 2938 02:44:25,960 --> 02:44:29,520 require more thought and effort. 46% of reviewers 2939 02:44:29,520 --> 02:44:31,880 responded and provided around 1,500 comments. 2940 02:44:32,280 --> 02:44:34,680 The qualitative analysis identified 8 common and 2941 02:44:34,680 --> 02:44:37,080 salient themes present in interviewers's comments and 2942 02:44:37,080 --> 02:44:41,520 I'll be brief as I go through these. You can see more 2943 02:44:41,520 --> 02:44:45,320 details in the ran report. so the first theme 2944 02:44:45,320 --> 02:44:47,720 general satisfaction and value of the training. 2945 02:44:47,720 --> 02:44:50,800 So here the great majority of reviewer comments were 2946 02:44:50,800 --> 02:44:53,520 those of simple thanks and appreciation for the 2947 02:44:53,520 --> 02:44:53,880 training. 2948 02:44:53,880 --> 02:44:57,280 Reviewers applauding NIH for tackling bias in review 2949 02:44:57,280 --> 02:45:01,120 and were pleased with the high quality of training. 2950 02:45:01,120 --> 02:45:03,880 They stated they learned a lot from the training and 2951 02:45:03,880 --> 02:45:06,600 believed they could have a real impact in reducing bias 2952 02:45:06,600 --> 02:45:07,120 and review. 2953 02:45:07,120 --> 02:45:10,320 They felt that the training activities were relating 2954 02:45:10,320 --> 02:45:13,880 and were very impressed by the study section scenario. 2955 02:45:13,880 --> 02:45:17,720 Reviewers overwhelmingly repressed more scenarios that 2956 02:45:17,720 --> 02:45:24,120 they could watch and learn from and many thought it 2957 02:45:24,120 --> 02:45:29,840 should be required every year or in every review 2958 02:45:29,840 --> 02:45:30,480 meeting. 2959 02:45:30,480 --> 02:45:33,360 Many reviewers were motivated and empowered by the 2960 02:45:33,360 --> 02:45:33,720 training. 2961 02:45:33,720 --> 02:45:36,280 They stated the training helped them become more 2962 02:45:36,280 --> 02:45:38,840 confident in speaking up and out against bias. 2963 02:45:38,840 --> 02:45:42,600 The training helped reviewers recognize their own 2964 02:45:42,600 --> 02:45:47,440 biases and many made vows to correct their bias 2965 02:45:47,440 --> 02:45:49,600 attitudes for future reviews. 2966 02:45:49,600 --> 02:45:53,200 Requests for more direct approaches to tackle bias. 2967 02:45:53,200 --> 02:45:56,120 Reviewers found the bias mitigation strategies and the 2968 02:45:56,480 --> 02:45:59,040 training extremely valuable and they wanted more 2969 02:45:59,040 --> 02:46:01,960 resources and examples on how to intervene in 2970 02:46:01,960 --> 02:46:04,880 situations where different types of biases are present, 2971 02:46:04,880 --> 02:46:08,520 such as scientific bias and when bias is present in 2972 02:46:08,520 --> 02:46:11,080 different context such as with non discussed 2973 02:46:11,080 --> 02:46:11,440 applications. 2974 02:46:11,440 --> 02:46:15,640 Reviewers wanted resources and strategies to help 2975 02:46:15,640 --> 02:46:19,840 themselves themselves from expressing bias in review. 2976 02:46:19,840 --> 02:46:25,240 They suggested examples such as clarifying questions to 2977 02:46:25,240 --> 02:46:30,000 ask when challenging statements, different avenues to 2978 02:46:30,000 --> 02:46:35,400 raise bias concerns during the meeting, templates to 2979 02:46:35,400 --> 02:46:41,480 write unbias critiques, and a cheat sheet of common 2980 02:46:41,480 --> 02:46:44,840 reviewed language that was bias. 2981 02:46:44,840 --> 02:46:47,440 Reviewers noted the challenges and potential 2982 02:46:47,440 --> 02:46:50,920 consequences that many reviewers faced that could deter 2983 02:46:50,920 --> 02:46:53,960 them from intervening bias, primarily the power 2984 02:46:53,960 --> 02:46:56,560 dynamics associated with reviewers social and 2985 02:46:56,560 --> 02:46:57,400 demographic status. 2986 02:46:57,400 --> 02:47:00,240 Reviewers felt that young, less experienced reviewers 2987 02:47:00,240 --> 02:47:03,120 and reviewers from marginalized and minority groups 2988 02:47:03,120 --> 02:47:07,600 would be less likely to speak out against bias out of 2989 02:47:07,600 --> 02:47:11,640 fear of being ignored or undermined or because of the 2990 02:47:11,640 --> 02:47:14,480 potential for negative repercussions as an applicant 2991 02:47:14,480 --> 02:47:14,920 reviewer. 2992 02:47:14,920 --> 02:47:18,600 Reviewers recommended the training include more 2993 02:47:18,600 --> 02:47:22,320 acknowledgement and intervention strategies that took 2994 02:47:22,320 --> 02:47:25,400 these power dynamics into account. 2995 02:47:25,400 --> 02:47:30,680 If the need for chairs and SROs to intervene more and 2996 02:47:30,680 --> 02:47:32,640 receive more bias training. 2997 02:47:32,640 --> 02:47:36,440 Reviewers believe SROs and chairs hold the most power 2998 02:47:36,440 --> 02:47:40,680 in the room and setting the tone for acceptable and 2999 02:47:40,680 --> 02:47:41,520 unacceptable behavior. 3000 02:47:41,520 --> 02:47:48,520 As such was they believe SROs should take the lead and 3001 02:47:48,520 --> 02:47:54,240 changing review culture to one that does not tolerate 3002 02:47:54,240 --> 02:47:54,880 bias. 3003 02:47:54,880 --> 02:47:58,080 Many reviewers noted that they would feel more 3004 02:47:58,080 --> 02:48:01,320 comfortable and confident intervening if they saw the 3005 02:48:01,320 --> 02:48:05,320 chair and SRO play a more active role in intervening. 3006 02:48:05,320 --> 02:48:09,360 They thought the chairs and SROs could benefit from 3007 02:48:09,360 --> 02:48:12,480 additional training on bias given their unique 3008 02:48:12,480 --> 02:48:12,920 position. 3009 02:48:12,920 --> 02:48:16,160 Experiences with bias as a reviewer and applicant. 3010 02:48:16,160 --> 02:48:19,280 Reviewers stated that they could relate to the scenario 3011 02:48:19,280 --> 02:48:22,720 and the bias training with many stating that they have 3012 02:48:22,720 --> 02:48:25,520 been in meetings where these exact scenarios have 3013 02:48:25,520 --> 02:48:28,960 occurred. They also shared other accounts of bias that 3014 02:48:28,960 --> 02:48:31,400 they experienced or observed as a reviewer. 3015 02:48:31,400 --> 02:48:35,720 Reviewers said that the training highlighted many of 3016 02:48:35,720 --> 02:48:40,600 the biases that they have faced as grant applicants 3017 02:48:40,600 --> 02:48:44,360 mainly from reviewer comments on their applications. 3018 02:48:44,360 --> 02:48:48,320 Reviewer recommendations to reduce bias in review. 3019 02:48:48,320 --> 02:48:53,880 Reviewers gave their own recommendations on how NIH can 3020 02:48:53,880 --> 02:48:56,320 reduce bias on review. 3021 02:48:56,320 --> 02:49:00,280 Many looked at incorporating blind reviews and adopting 3022 02:49:00,280 --> 02:49:02,240 a stage review process. 3023 02:49:02,240 --> 02:49:05,240 Reviewers noted that increasing the diversity on panels 3024 02:49:05,240 --> 02:49:06,720 could help reduce bias. 3025 02:49:06,720 --> 02:49:10,360 Many reviewers suggested removing the environment and 3026 02:49:10,360 --> 02:49:13,480 investigative criteria since they were heavily 3027 02:49:13,480 --> 02:49:15,040 susceptible to bias. 3028 02:49:15,040 --> 02:49:18,280 Last, recommendations to improve training. 3029 02:49:18,280 --> 02:49:21,520 Aside from the overwhelming request to add more 3030 02:49:21,520 --> 02:49:24,720 scenarios, reviewers stated that the training could be 3031 02:49:24,720 --> 02:49:28,360 improved by covering more types of bias and by 3032 02:49:28,360 --> 02:49:31,560 addressing review biases that are more implicit in 3033 02:49:31,560 --> 02:49:31,960 nature. 3034 02:49:31,960 --> 02:49:35,280 Many reviewers wanted to learn more about scientific 3035 02:49:35,280 --> 02:49:38,600 bias and bias associated with the socio demographic 3036 02:49:38,600 --> 02:49:40,640 characteristics of reviewers and applicants. 3037 02:49:40,640 --> 02:49:44,280 Bias associated with critique score mismatch and non 3038 02:49:44,280 --> 02:49:46,520 discuss applications were of concern. 3039 02:49:46,520 --> 02:49:50,240 Finally, reviewers noted that the biases covered in the 3040 02:49:50,240 --> 02:49:53,160 training were explicit in nature and easily 3041 02:49:53,160 --> 02:49:56,480 identifiable and more likely to be addressed and 3042 02:49:56,480 --> 02:50:00,200 corrected. They wanted the training to cover more 3043 02:50:00,200 --> 02:50:03,960 implicit or acceptable form of biases that arise during 3044 02:50:03,960 --> 02:50:04,360 review. 3045 02:50:04,360 --> 02:50:07,320 So in short, reviewers applauded NIH for tackling bias 3046 02:50:07,320 --> 02:50:10,280 in review. They were highly satisfied with the 3047 02:50:10,280 --> 02:50:12,560 training and that it was very useful. 3048 02:50:12,880 --> 02:50:16,200 The training was very successful in increasing 3049 02:50:16,200 --> 02:50:19,960 reviewers' knowledge and awareness of bias in peer 3050 02:50:19,960 --> 02:50:23,760 review and it significantly helped reviewers feel more 3051 02:50:23,760 --> 02:50:27,520 comfortable in intervening on bias and it empowered 3052 02:50:27,520 --> 02:50:31,320 reviewers to actively take action in the future. 3053 02:50:31,320 --> 02:50:36,640 So our next steps include taking measures to increase 3054 02:50:36,640 --> 02:50:39,640 reviewer participation in the training. 3055 02:50:39,640 --> 02:50:42,720 Identifying opportunities to keep reviewers engaged 3056 02:50:42,720 --> 02:50:47,880 about bias mitigation and to continue to help chairs is 3057 02:50:47,880 --> 02:50:52,520 SROs play a more active role in bias intervention. 3058 02:50:52,520 --> 02:50:58,000 And last, for the next module, we need to determine the 3059 02:50:58,000 --> 02:51:03,960 next type of bias to cover and how to address biases in 3060 02:51:03,960 --> 02:51:05,440 more implicit ways. 3061 02:51:05,440 --> 02:51:06,160 Any questions? 3062 02:51:06,160 --> 02:51:09,360 >> DR. BRUCE REED: Thanks very much, hope. 3063 02:51:09,360 --> 02:51:12,600 Maybe you want to take down your slides and we'll open 3064 02:51:12,600 --> 02:51:13,760 it up to discussion. 3065 02:51:13,760 --> 02:51:16,840 >> DR. HOPE CUMMINGS: Sure. 3066 02:51:16,840 --> 02:51:18,080 >> DR. BRUCE REED: Let's see. 3067 02:51:18,080 --> 02:51:21,320 Tonya. 3068 02:51:21,320 --> 02:51:25,320 >> DR. TONYA PALERMO: Thanks so much, Hope. That was 3069 02:51:25,320 --> 02:51:26,320 a wonderful overview. 3070 02:51:26,320 --> 02:51:31,560 I just wanted to share I have heard wonderful comments 3071 02:51:31,560 --> 02:51:36,280 from my own colleagues who have gone through the 3072 02:51:36,280 --> 02:51:36,800 training. 3073 02:51:36,800 --> 02:51:38,480 I just wanted to know if there's thoughts now about 3074 02:51:38,480 --> 02:51:39,800 making this mandatory for reviewers and thoughts about 3075 02:51:39,800 --> 02:51:40,480 how that might look? 3076 02:51:40,480 --> 02:51:42,640 >> DR. BRUCE REED: Yeah. Noni, do you want to -- 3077 02:51:42,640 --> 02:51:43,720 >> DR. NONI BYRNES: Yeah. 3078 02:51:43,720 --> 02:51:44,400 Let me take that. 3079 02:51:44,400 --> 02:51:50,160 Yes, I think that ultimately this has to become 3080 02:51:50,160 --> 02:51:57,160 mandatory. This was our first attempt out of the door. 3081 02:51:57,160 --> 02:51:59,280 We went sure how it would be received. 3082 02:51:59,280 --> 02:52:02,480 It looks like it's well received. We have places to 3083 02:52:02,480 --> 02:52:02,760 adjust. 3084 02:52:02,760 --> 02:52:05,960 And yes, that is planned for the future. 3085 02:52:05,960 --> 02:52:07,880 Not next round, but soon. 3086 02:52:07,880 --> 02:52:10,320 >> DR. BRUCE REED: Sean. 3087 02:52:10,320 --> 02:52:12,920 >> DR. SEAN DAVIDSON: I want to reiterate that I think 3088 02:52:12,920 --> 02:52:14,240 this is a great training program. 3089 02:52:14,240 --> 02:52:17,480 And I also reiterate that I think this needs to be made 3090 02:52:17,480 --> 02:52:17,760 mandatory. 3091 02:52:17,760 --> 02:52:19,280 It's an enjoyable training. 3092 02:52:19,280 --> 02:52:21,760 I think people, once they realize it's going on be well 3093 02:52:21,760 --> 02:52:24,200 taken to for everybody and there has to be a mechanism 3094 02:52:24,200 --> 02:52:25,560 to get people to do that. 3095 02:52:25,560 --> 02:52:31,760 The other thing I would suggest you do is think about 3096 02:52:31,760 --> 02:52:37,960 something like this for the spiel that the SROs give to 3097 02:52:37,960 --> 02:52:44,160 all their section members about how to be a good study 3098 02:52:44,160 --> 02:52:48,680 section reviewer and things like that with this 3099 02:52:48,680 --> 02:52:53,760 experiential type of a paradigm which I would hope would 3100 02:52:53,760 --> 02:52:56,000 help homogenize the experience. 3101 02:52:56,000 --> 02:52:58,480 >> DR. NONI BYRNES: Let me respond to that. 3102 02:52:58,480 --> 02:53:01,760 Our office of training has a number of innovative ideas 3103 02:53:01,760 --> 02:53:04,440 that are already sort of halfway in progress. 3104 02:53:04,440 --> 02:53:11,600 And I'm hoping that if not in September, then hopefully 3105 02:53:11,600 --> 02:53:20,240 by next March she would be able to give you an update 3106 02:53:20,240 --> 02:53:27,400 on what wear doing to make consistent the training for 3107 02:53:27,400 --> 02:53:32,440 SROs and reviewers, including updating this training 3108 02:53:32,440 --> 02:53:39,600 for the next iteration which I'm hoping we can make 3109 02:53:39,600 --> 02:53:40,320 mandatory. 3110 02:53:40,320 --> 02:53:40,880 Frankly, I think if we're going to make something 3111 02:53:40,880 --> 02:53:41,400 mandatory, it's simpler to go through the ERA systems 3112 02:53:41,400 --> 02:53:42,080 and have it be something people have to do before their 3113 02:53:42,080 --> 02:53:42,280 submit their application. 3114 02:53:42,280 --> 02:53:46,360 That is manage we need to the with OER. 3115 02:53:46,360 --> 02:53:48,360 >> DR. LEOPOLDO CABASSA: This is very helpful. 3116 02:53:48,360 --> 02:53:51,400 I think it highlights the work that has been done for 3117 02:53:51,400 --> 02:53:53,880 this and the follow through of understanding how this 3118 02:53:53,880 --> 02:53:55,800 works, what's working and what is not. 3119 02:53:55,800 --> 02:54:00,440 I definitely think that making this mandatory would be 3120 02:54:00,440 --> 02:54:04,560 helpful, but continue to create materials in small 3121 02:54:04,560 --> 02:54:10,240 handouts of the -- not only of the bias, but of the 3122 02:54:10,240 --> 02:54:14,880 activities and the actions that reviewers, SROs and the 3123 02:54:14,880 --> 02:54:16,600 chairs can do. 3124 02:54:16,600 --> 02:54:19,360 When you're going to review, we get all these handouts 3125 02:54:19,360 --> 02:54:21,840 on the scoring system and all of that stuff. 3126 02:54:21,840 --> 02:54:23,640 This should be part of that package, as well. 3127 02:54:23,640 --> 02:54:27,200 So it's continuously being reminded and highlighted. 3128 02:54:27,200 --> 02:54:31,800 And then I think the other component here, as we are 3129 02:54:31,800 --> 02:54:35,160 expanding the diversity of our review committees and 3130 02:54:35,160 --> 02:54:37,680 bringing younger investigators or early career 3131 02:54:37,680 --> 02:54:41,040 investigators, this is an important area of training. 3132 02:54:41,040 --> 02:54:44,480 That we -- not only diversity, but we need to empower 3133 02:54:44,480 --> 02:54:46,920 the younger investigators and the underrepresented in 3134 02:54:46,920 --> 02:54:51,080 order to what are the actions to take to be able to 3135 02:54:51,080 --> 02:54:52,280 address this bias. 3136 02:54:52,280 --> 02:54:54,320 Here are some examples and training around that. 3137 02:54:54,320 --> 02:54:58,280 So follow through on that will be really important. 3138 02:54:58,280 --> 02:55:01,720 Lastly, I think figuring out what the metrics are for 3139 02:55:01,720 --> 02:55:04,160 evaluating the impact that this is having. 3140 02:55:04,160 --> 02:55:07,480 So, for example, are there more reports or bias right 3141 02:55:07,480 --> 02:55:09,120 now coming after this training? 3142 02:55:09,120 --> 02:55:14,680 What other metrics you can think of of figuring out 3143 02:55:14,680 --> 02:55:20,200 whether this is working to make adjustments as this is 3144 02:55:20,200 --> 02:55:21,320 rolled out. 3145 02:55:21,320 --> 02:55:24,040 I commend you for doing this and I think it's going in 3146 02:55:24,040 --> 02:55:25,640 the right direction agency we're moving forward. 3147 02:55:25,640 --> 02:55:27,240 Thanks. 3148 02:55:27,240 --> 02:55:29,760 >> DR. BRUCE REED: Thanks, Leo. 3149 02:55:29,760 --> 02:55:32,600 Karen. 3150 02:55:32,600 --> 02:55:33,840 >> DR. KAREN ANDERSON: Thank you so much. 3151 02:55:33,840 --> 02:55:34,920 This training is terrific. 3152 02:55:34,920 --> 02:55:39,240 I think the concepts of the training is not a single 3153 02:55:39,240 --> 02:55:42,760 time point, but it is an ongoing, continuous education 3154 02:55:42,760 --> 02:55:46,680 process on behalf of all scientists and it is important 3155 02:55:46,680 --> 02:55:51,000 that the training be not just for viewers but for grant 3156 02:55:51,000 --> 02:55:54,560 recipients as they are the mentors of the next 3157 02:55:54,560 --> 02:55:57,680 generation that is going to be doing this. 3158 02:55:57,680 --> 02:56:03,200 And I think the opportunities here are terrific. 3159 02:56:03,200 --> 02:56:09,240 And my second thought is to echo the comment about 3160 02:56:09,240 --> 02:56:14,040 environment in particular, being an at-risk area for 3161 02:56:14,040 --> 02:56:18,280 overt implicit bias and thinking through those 3162 02:56:18,280 --> 02:56:23,680 criterion and how that might be reimagined sometime in 3163 02:56:23,680 --> 02:56:26,680 the future. Thank you. 3164 02:56:26,680 --> 02:56:30,560 >> DR. BRUCE REED: Thanks, Karen. 3165 02:56:30,560 --> 02:56:30,920 Lynn. 3166 02:56:30,920 --> 02:56:34,400 >> DR. LYNN YEE: I want to express gratitude for this 3167 02:56:34,400 --> 02:56:35,000 training program. 3168 02:56:35,000 --> 02:56:37,400 I think it's outstanding and it's great to hear that 3169 02:56:37,400 --> 02:56:38,080 others did, too. 3170 02:56:38,080 --> 02:56:42,440 I think that one area that could be a focus of future 3171 02:56:42,440 --> 02:56:46,040 trainings is as others have echoed already is a little 3172 02:56:46,040 --> 02:56:49,280 bit about the culture of power inside this study 3173 02:56:49,280 --> 02:56:52,160 section and who -- whether, for example, the junior 3174 02:56:52,160 --> 02:56:55,400 members of the study section, whether ad hoc or 3175 02:56:55,400 --> 02:56:58,640 standing members defer to people who have been standing 3176 02:56:58,640 --> 02:57:02,240 members for longer or are people they respect in their 3177 02:57:02,240 --> 02:57:02,800 fields. 3178 02:57:02,800 --> 02:57:06,840 And whether that deference and power, which is probably 3179 02:57:06,840 --> 02:57:11,280 true in many fields, but especially in my field, for 3180 02:57:11,280 --> 02:57:14,440 example, a surgical subspeciality are hierarchies are 3181 02:57:14,440 --> 02:57:17,120 intrinsic to the way we practice. 3182 02:57:17,120 --> 02:57:20,240 I think that power dynamic can influence the direction 3183 02:57:20,240 --> 02:57:24,080 of study section. And it would be interesting to call 3184 02:57:24,080 --> 02:57:27,560 that outline in future training else and not just try 3185 02:57:27,560 --> 02:57:31,080 to convince people to speak up, but to try to 3186 02:57:31,080 --> 02:57:33,880 acknowledge that power differentials exist and that we 3187 02:57:33,880 --> 02:57:35,960 need to work to overcome them. 3188 02:57:35,960 --> 02:57:40,800 >> DR. BRUCE REED: Thanks. 3189 02:57:40,800 --> 02:57:44,760 >> DR. NONI BYRNES: That is a really good point. I 3190 02:57:44,760 --> 02:57:46,000 see Hope taking notes. 3191 02:57:46,000 --> 02:57:49,080 I will say we are trying to incorporate this training 3192 02:57:49,080 --> 02:57:51,880 and more discussions around the power dynamic in our 3193 02:57:51,880 --> 02:57:54,360 chair orientations that we do once a year. 3194 02:57:54,360 --> 02:57:58,920 And I think the chair plays a critical role in 3195 02:57:58,920 --> 02:58:02,560 ensuring, you know, sort of mitigating that power 3196 02:58:02,560 --> 02:58:03,000 dynamic. 3197 02:58:03,000 --> 02:58:03,920 >> DR. BRUCE REED: Yeah. 3198 02:58:03,920 --> 02:58:09,600 And I will just say briefly add on to that, I'm not 3199 02:58:09,600 --> 02:58:13,880 sure it's always appreciated that we didn't simply roll 3200 02:58:13,880 --> 02:58:19,560 this out to reviewers, but we made a lot of effort to 3201 02:58:19,560 --> 02:58:23,840 help SROs get more comfortable with the concepts of 3202 02:58:23,840 --> 02:58:28,560 bias, to work with this training, to work with this 3203 02:58:28,560 --> 02:58:32,840 training with their reviewers because as many of the 3204 02:58:32,840 --> 02:58:37,080 peoples commented, the SROs play a critical role in 3205 02:58:37,080 --> 02:58:41,360 this, both in training reviewers and in the meeting 3206 02:58:41,360 --> 02:58:41,840 itself. 3207 02:58:41,840 --> 02:58:57,240 So we are paying attention to that aspect, as well. 3208 02:58:57,240 --> 02:59:01,680 Okay. Let's move on. Thank you, Hope. 3209 02:59:01,680 --> 02:59:06,040 We'll move on to the last section of the agenda, which 3210 02:59:06,040 --> 02:59:08,840 I think will be a lively discussion. 3211 02:59:08,840 --> 02:59:10,880 I hope it will be. 3212 02:59:10,880 --> 02:59:14,360 This is the report from the working group on review of 3213 02:59:14,360 --> 02:59:15,000 fellowship applications. 3214 02:59:15,000 --> 02:59:27,880 And let me just bring up my screen. 3215 02:59:27,880 --> 02:59:28,440 Okay. 3216 02:59:28,440 --> 02:59:40,640 All right. Hopefully now you see my screen. 3217 02:59:40,640 --> 02:59:44,520 Elizabeth and I are going to divide this presentations. 3218 02:59:44,520 --> 02:59:47,840 This report is an interim report. 3219 02:59:47,840 --> 02:59:50,360 And the working group is looking forward to feedback 3220 02:59:50,360 --> 02:59:52,880 from council about the ideas that they have developed 3221 02:59:52,880 --> 02:59:53,720 to this point. 3222 02:59:53,720 --> 02:59:54,840 So we are dividing this. 3223 02:59:54,840 --> 02:59:58,840 I'm going to go over some of the background that sort 3224 02:59:58,840 --> 03:00:02,120 of informs the group's thinking and Elizabeth will give 3225 03:00:02,120 --> 03:00:05,720 you some of their thoughts about how things might be 3226 03:00:05,720 --> 03:00:06,080 changed. 3227 03:00:06,080 --> 03:00:10,560 So just in terms of background, I think, I hope, one of 3228 03:00:10,560 --> 03:00:14,240 the themes of the meeting today is that CSR is 3229 03:00:14,240 --> 03:00:16,840 committed to improving the peer review process. 3230 03:00:16,840 --> 03:00:23,600 That is goal 4 of our strategic plan if 3231 03:00:25,080 --> 03:00:31,840 anybody is paying attention to that. This fellowship 3232 03:00:31,840 --> 03:00:38,600 effort extends beyond that. We have heard persistent 3233 03:00:38,600 --> 03:00:43,840 concerns that fellowship reviews may disadvantage some 3234 03:00:43,840 --> 03:00:46,080 highly qualified applicants. 3235 03:00:46,080 --> 03:00:50,800 These concerns led CSR late last summer to convene a 3236 03:00:50,800 --> 03:00:55,080 working group on fellowship review with the charge to 3237 03:00:55,080 --> 03:00:58,840 evaluate the fellowship review process and to make 3238 03:00:58,840 --> 03:01:02,640 recommendations to make that process as fair and 3239 03:01:02,640 --> 03:01:05,000 effective as possible for all. 3240 03:01:05,000 --> 03:01:09,240 This is the group. 3241 03:01:09,240 --> 03:01:14,240 It consists of some council members in case of Scott 3242 03:01:14,240 --> 03:01:18,760 Miller former council member, members of the NIH staff, 3243 03:01:18,760 --> 03:01:23,800 both from within CSR and across NIH, and then external 3244 03:01:23,800 --> 03:01:24,320 scientists. 3245 03:01:24,320 --> 03:01:28,640 The external scientists come from a variety of 3246 03:01:28,640 --> 03:01:31,920 institutions and represent different career stages. 3247 03:01:31,920 --> 03:01:36,240 They all had personal experience with writing 3248 03:01:36,240 --> 03:01:40,600 fellowship applications and with the review of 3249 03:01:40,600 --> 03:01:41,840 fellowship applications. 3250 03:01:41,840 --> 03:01:45,880 They have been a very active engaged group. 3251 03:01:45,880 --> 03:01:53,720 What I want to cover here is some of the external 3252 03:01:53,720 --> 03:02:00,120 input, some of the information that they considered in 3253 03:02:00,120 --> 03:02:06,520 thinking about where the issues are and how things 3254 03:02:06,520 --> 03:02:08,640 might be changed. 3255 03:02:08,640 --> 03:02:12,520 So one of the inputs that we got were comments from the 3256 03:02:12,520 --> 03:02:12,840 community. 3257 03:02:12,840 --> 03:02:16,640 We wrote a blog and did a content analysis of those. 3258 03:02:16,640 --> 03:02:19,840 That is what I'd like to start with because it gives 3259 03:02:19,840 --> 03:02:22,480 you a flavor for what these consistent concerns are. 3260 03:02:22,480 --> 03:02:26,640 So highlighting frequent comments, we heard ultimate 3261 03:02:26,640 --> 03:02:31,400 many times concerns that there's bias against junior 3262 03:02:31,400 --> 03:02:36,120 faculty members, women and those from under represented 3263 03:02:36,120 --> 03:02:40,880 groups, concerns that the current process is bias 3264 03:02:40,880 --> 03:02:45,600 towards senior mentors with extremely well funded labs, 3265 03:02:45,600 --> 03:02:49,760 concerns that institutional bias tilts towards large 3266 03:02:49,760 --> 03:02:55,120 universities with ample resources and the idea that the 3267 03:02:55,120 --> 03:02:56,880 rich get richer. 3268 03:02:56,880 --> 03:03:08,880 We also heard multiple reminders that junior faculty 3269 03:03:08,880 --> 03:03:14,880 can provide great mentorship. 3270 03:03:14,880 --> 03:03:20,240 In terms of people suggesting we could change this, the 3271 03:03:20,240 --> 03:03:23,960 request that we eliminate the requirement for 3272 03:03:23,960 --> 03:03:25,040 undergraduate grades. 3273 03:03:25,040 --> 03:03:30,760 Other comments we got a number of comments on relating 3274 03:03:30,760 --> 03:03:36,520 to how the science of fellowship reviews ought to be 3275 03:03:36,520 --> 03:03:37,080 evaluated. 3276 03:03:37,080 --> 03:03:42,360 We got pleas to educate reviewers to provide 3277 03:03:42,360 --> 03:03:48,960 constructive feedback and not to be overly harsh in the 3278 03:03:48,960 --> 03:03:53,560 comments with these very early career applicants. 3279 03:03:53,560 --> 03:03:57,520 And we got comments regarding the bias awareness 3280 03:03:57,520 --> 03:04:00,960 training that we were just talking about. 3281 03:04:00,960 --> 03:04:06,360 The idea that the bias awareness training ought to be 3282 03:04:06,360 --> 03:04:10,120 mandated for fellowship reviewers and the suggestion 3283 03:04:10,120 --> 03:04:15,000 that we think about creating a separate bias awareness 3284 03:04:15,000 --> 03:04:19,320 training module specifically aimed at fill low ships. 3285 03:04:19,320 --> 03:04:23,400 The concerns raised in these public comments sort of -- 3286 03:04:23,400 --> 03:04:27,040 certainly resinated with the concerns that the working 3287 03:04:27,040 --> 03:04:28,840 group members had themselves. 3288 03:04:29,360 --> 03:04:36,360 They asked us to develop data to see if these were 3289 03:04:36,360 --> 03:04:40,760 borne out in review outcomes of fellowship 3290 03:04:40,760 --> 03:04:41,400 applications. 3291 03:04:41,400 --> 03:04:42,320 So we did. 3292 03:04:42,320 --> 03:04:50,600 I'm just going to highlight a few finding else from the 3293 03:04:50,600 --> 03:04:51,360 data. 3294 03:04:51,360 --> 03:04:54,920 So when you look, you do, indeed, see that applications 3295 03:04:54,920 --> 03:04:57,760 are highly concentrated in a small number of 3296 03:04:57,760 --> 03:04:58,120 institutions. 3297 03:04:58,120 --> 03:05:02,480 These data came from the 2021 council round. 3298 03:05:02,480 --> 03:05:11,400 We looked at all 6,676 applications at NIH reviewed 3299 03:05:11,400 --> 03:05:13,360 that year. 3300 03:05:13,360 --> 03:05:16,080 The figure shows you this. 3301 03:05:16,080 --> 03:05:19,600 Here you have bends that indicate how many applications 3302 03:05:19,600 --> 03:05:21,960 and institutions submitted during the year. 3303 03:05:21,960 --> 03:05:25,400 And the count is the number of institutions that fall 3304 03:05:25,400 --> 03:05:26,120 in that. 3305 03:05:26,120 --> 03:05:32,680 So if you look out here, starting at the right end of 3306 03:05:32,680 --> 03:05:38,200 this figure, you see that there are 15 schools that 3307 03:05:38,200 --> 03:05:42,040 each submitted over 100 applications, over 100 3308 03:05:42,040 --> 03:05:47,520 fellowship applications. If you look at the other end 3309 03:05:47,520 --> 03:05:53,000 of the distribution, you see there are over 100 schools 3310 03:05:53,000 --> 03:05:58,480 that submitted just 1 or 2 applications. There are 3311 03:05:58,480 --> 03:06:03,440 almost another 100 that submitted between 3 and 5 3312 03:06:03,440 --> 03:06:03,960 applications. 3313 03:06:03,960 --> 03:06:05,680 So it is highly skewed. 3314 03:06:05,680 --> 03:06:11,120 You might wonder whether the applications of the 3315 03:06:11,120 --> 03:06:17,240 schools that submit not so much applications are more 3316 03:06:17,240 --> 03:06:17,920 successful. 3317 03:06:17,920 --> 03:06:25,360 Yes, it's not black and white, but you do see that the 3318 03:06:25,360 --> 03:06:30,320 schools that submit the most applications do distinctly 3319 03:06:30,320 --> 03:06:35,280 better than applications that come from the schools 3320 03:06:35,280 --> 03:06:37,120 that submit fewer. 3321 03:06:37,120 --> 03:06:40,880 So down here on the bottom, the schools that submit 3322 03:06:40,880 --> 03:06:44,640 over 100 applications a year, they ended up scoring in 3323 03:06:44,640 --> 03:06:46,160 the high impact range. 3324 03:06:46,160 --> 03:06:51,680 The gray is the percent non discussed. You can see as 3325 03:06:51,680 --> 03:06:55,320 the number of applications submitted each your goes 3326 03:06:55,320 --> 03:06:59,920 down, so does the proportion that are highly scored and 3327 03:06:59,920 --> 03:07:03,600 the proportion that are non discussed goes up. 3328 03:07:03,600 --> 03:07:09,080 What about academic rank of the sponsor? 3329 03:07:09,080 --> 03:07:15,880 You see that review outcomes steadily improve as the 3330 03:07:15,880 --> 03:07:20,400 academic rank of the sponsor rises. 3331 03:07:20,400 --> 03:07:23,440 So let me direct your attention here to the top part of 3332 03:07:23,440 --> 03:07:23,960 this figure. 3333 03:07:23,960 --> 03:07:31,000 Look at these top four bars. If you start here at the 3334 03:07:31,000 --> 03:07:35,320 top, these are applications that came from assistant 3335 03:07:35,320 --> 03:07:39,120 professor sponsors, then associates, then professor and 3336 03:07:39,120 --> 03:07:40,200 then other. 3337 03:07:40,200 --> 03:07:42,920 In this grant, this figure, the applications that end 3338 03:07:42,920 --> 03:07:46,280 up scoring in the high impact range are shaded blue and 3339 03:07:46,280 --> 03:07:49,040 then here in gray, again, are the non discussed. 3340 03:07:49,040 --> 03:07:54,600 And you see that as you go from assistant to associate 3341 03:07:54,600 --> 03:07:59,120 to professor, the percents scoring in this high impact 3342 03:07:59,120 --> 03:08:00,160 range rises. 3343 03:08:00,160 --> 03:08:03,880 The percent discussed -- non discussed goes down. 3344 03:08:04,320 --> 03:08:07,080 This group, this other group does particularly well and 3345 03:08:07,080 --> 03:08:08,600 people wonder who that is. 3346 03:08:08,600 --> 03:08:12,320 These are deans, directors, investigators. 3347 03:08:12,320 --> 03:08:16,400 I think those must be like HMMI investigators. 3348 03:08:16,400 --> 03:08:17,920 These are senior folks. 3349 03:08:17,920 --> 03:08:20,360 And you can see that they do especially well. 3350 03:08:20,360 --> 03:08:24,680 Briefly, when you look at multiple sponsor 3351 03:08:24,680 --> 03:08:31,480 applications, you see the same sort of pattern. It is 3352 03:08:31,480 --> 03:08:32,720 somewhat accentuated. 3353 03:08:32,720 --> 03:08:33,560 Okay. 3354 03:08:33,560 --> 03:08:38,120 Then I just wanted to say briefly something about where 3355 03:08:38,120 --> 03:08:41,760 these review -- where the NIH review criteria come 3356 03:08:41,760 --> 03:08:42,200 from. 3357 03:08:42,560 --> 03:08:49,000 And what the limitations are in changing them. 3358 03:08:49,000 --> 03:08:54,840 So review criteria start in law. So the fellowship 3359 03:08:54,840 --> 03:09:00,080 criteria derive from this section of the code of 3360 03:09:00,080 --> 03:09:03,600 federal regulations, which says -- I'll just 3361 03:09:03,600 --> 03:09:08,240 paraphrase, but within the limits of funds available, 3362 03:09:08,240 --> 03:09:12,320 the Secretary shall may awards taking into 3363 03:09:12,320 --> 03:09:15,840 consideration, among other pertinent factors, these 3364 03:09:15,840 --> 03:09:17,160 four things. 3365 03:09:17,160 --> 03:09:22,160 So these are the four considerations on which we are to 3366 03:09:22,160 --> 03:09:23,080 evaluate applications. 3367 03:09:23,080 --> 03:09:27,080 One, the scientific, technical or educational merit of 3368 03:09:27,080 --> 03:09:28,560 the particular proposal. 3369 03:09:28,560 --> 03:09:32,960 Two, the availability of resources and facilities to 3370 03:09:32,960 --> 03:09:34,600 carry it out. 3371 03:09:34,600 --> 03:09:37,480 Three, if qualifications and experience of the 3372 03:09:37,480 --> 03:09:37,920 applicant. 3373 03:09:37,920 --> 03:09:43,160 And, four, the need for personnel in the subject area 3374 03:09:43,160 --> 03:09:46,320 of the proposed research or training. 3375 03:09:46,320 --> 03:09:50,320 So that is the law. 3376 03:09:50,320 --> 03:09:54,200 NIH has the authority and obligation to interpret the 3377 03:09:54,200 --> 03:09:56,800 law to turn it into criteria. 3378 03:09:56,800 --> 03:09:58,560 So that is what happens. 3379 03:09:58,560 --> 03:10:04,280 NIH takes the law, interprets it and this slide shows 3380 03:10:04,280 --> 03:10:08,880 you the current NIH criteria that represent NIH's 3381 03:10:08,880 --> 03:10:11,200 interpretation of the statute. 3382 03:10:11,200 --> 03:10:17,640 You can see that it is, indeed, an interpretation. 3383 03:10:17,640 --> 03:10:22,400 So the criteria fellowship applicant flows pretty 3384 03:10:22,400 --> 03:10:25,760 directly from this consideration three. 3385 03:10:25,760 --> 03:10:31,880 But if you look at the rest of the criteria, they're 3386 03:10:31,880 --> 03:10:33,520 clearly an interpretation. 3387 03:10:33,520 --> 03:10:37,280 They're not specifically mentioned in the 3388 03:10:37,280 --> 03:10:39,760 considerations of the law. 3389 03:10:39,760 --> 03:10:41,800 But they certainly flow from it. 3390 03:10:41,800 --> 03:10:45,600 So laws are really hard to change. 3391 03:10:45,600 --> 03:10:47,680 NIH can change its interpretation. 3392 03:10:47,680 --> 03:10:54,040 That's not necessarily easy, but it can be done. 3393 03:10:54,040 --> 03:11:00,240 And what the office of the general council says is that 3394 03:11:00,240 --> 03:11:05,280 the new -- any new interpretation must reflect the law. 3395 03:11:05,280 --> 03:11:11,480 So it has to derive from the Code of Federal 3396 03:11:11,480 --> 03:11:15,400 Regulations and it can be otherwise illegal. 3397 03:11:16,280 --> 03:11:19,760 So as long as you connect on the legislation and you're 3398 03:11:19,760 --> 03:11:21,040 not illegal, you're okay. 3399 03:11:21,040 --> 03:11:25,360 With that, I'm going to turn this over to you, 3400 03:11:25,360 --> 03:11:29,240 Elizabeth, and you can talk about the changes that the 3401 03:11:29,240 --> 03:11:30,560 group has contemplated. 3402 03:11:30,560 --> 03:11:38,880 So, Elizabeth, do you want to run your slides? 3403 03:11:38,880 --> 03:11:39,760 >> DR. ELIZABETH VILLA: Yes. 3404 03:11:39,760 --> 03:11:43,280 >> DR. BRUCE REED: So let me give up sharing here and 3405 03:11:43,280 --> 03:11:44,640 hopefully you can take over. 3406 03:11:44,640 --> 03:11:45,000 >> DR. ELIZABETH VILLA: 3407 03:11:45,000 --> 03:11:45,520 >> DR. HOPE CUMMINGS:? 3408 03:11:45,520 --> 03:11:46,840 >> DR. BRUCE REED: Yes. 3409 03:11:46,840 --> 03:11:47,880 >> DR. ELIZABETH VILLA: Here we go. 3410 03:11:47,880 --> 03:11:52,840 So the working group convened a few times. 3411 03:11:52,840 --> 03:11:57,760 And I think that the goals that we sort of set out for 3412 03:11:57,760 --> 03:12:01,120 ourselves is to say there are wonderful applicants out 3413 03:12:01,120 --> 03:12:05,640 there that may have, you know -- may not be one of the 3414 03:12:05,640 --> 03:12:09,040 15 or other expected institutions or don't have the 3415 03:12:09,040 --> 03:12:12,040 typical background or don't have enough access to 3416 03:12:12,040 --> 03:12:14,280 information to write those wonderful applications. 3417 03:12:14,560 --> 03:12:19,880 And so the idea is can we encourage them to apply and 3418 03:12:19,880 --> 03:12:23,840 can we revise the review criteria and other processes 3419 03:12:23,840 --> 03:12:28,720 to make sure they get a fair chance to have a 3420 03:12:28,720 --> 03:12:29,600 successful application. 3421 03:12:29,600 --> 03:12:32,400 So there's sort of five things that I'm going to talk 3422 03:12:32,400 --> 03:12:33,440 to you about today. 3423 03:12:33,440 --> 03:12:36,760 They kind of go in the order that they happen. 3424 03:12:36,760 --> 03:12:39,200 So first proposed changed to application materials. 3425 03:12:39,200 --> 03:12:44,360 And then the review criteria. A little bit about the 3426 03:12:44,360 --> 03:12:49,080 review process and then finally I'll tell you a little 3427 03:12:49,080 --> 03:12:53,320 bit of suggestions we have on those outreach and 3428 03:12:53,320 --> 03:12:54,240 reviewer training. 3429 03:12:54,240 --> 03:12:58,120 So the first are application materials. 3430 03:12:58,120 --> 03:13:03,240 Basically, the goal of this is to try to give a chance 3431 03:13:03,240 --> 03:13:07,080 for the applicants to present this to the reviewer. 3432 03:13:07,080 --> 03:13:12,920 We thought one of the most efficient ways was to change 3433 03:13:12,920 --> 03:13:18,240 the sketch and to be able to align the proposed 3434 03:13:18,240 --> 03:13:20,400 criteria to evaluate them. 3435 03:13:20,400 --> 03:13:23,720 So there are a few things incorporated into the 3436 03:13:23,720 --> 03:13:24,080 biosketch. 3437 03:13:24,080 --> 03:13:26,800 The first one is a personal statement and a personal 3438 03:13:26,800 --> 03:13:28,120 statement will have three parts. 3439 03:13:28,120 --> 03:13:30,760 The first one is their scientific interest. 3440 03:13:30,760 --> 03:13:34,200 Really, to answer why is this field of science 3441 03:13:34,200 --> 03:13:38,000 important and why are you selected it for your project? 3442 03:13:38,000 --> 03:13:40,800 So any sort of personal relationship between their 3443 03:13:40,800 --> 03:13:42,560 scientific interest and the project. 3444 03:13:42,560 --> 03:13:47,080 In goals -- and this is a really big one -- is what are 3445 03:13:47,080 --> 03:13:50,520 your career goals and how do those fellowship goals 3446 03:13:50,520 --> 03:13:52,400 relate to your career goals. 3447 03:13:52,880 --> 03:13:58,760 And here we want to make a point that there is nowhere 3448 03:13:58,760 --> 03:14:03,680 in the definition of an F application that states that 3449 03:14:03,680 --> 03:14:07,600 these successful applicants need to go to become 3450 03:14:07,600 --> 03:14:10,560 professors or scientists at certain institutions. 3451 03:14:10,560 --> 03:14:17,560 So we want to make sure that this is a place where they 3452 03:14:17,560 --> 03:14:22,440 can discuss other career goals and that the reviewers 3453 03:14:22,440 --> 03:14:27,280 will evaluate the entire proposal based on their career 3454 03:14:27,280 --> 03:14:27,840 goals. 3455 03:14:27,840 --> 03:14:31,960 Here they also have a chance to specify what difference 3456 03:14:31,960 --> 03:14:35,280 will the fellowship make in advancing your career. 3457 03:14:35,280 --> 03:14:39,200 So we're going to talk about this concept of the delta 3458 03:14:39,200 --> 03:14:42,400 today. We would like to encourage reviewers to 3459 03:14:42,400 --> 03:14:45,600 consider if getting this award will help someone take 3460 03:14:45,600 --> 03:14:48,080 off and have a wonderful scientific career. 3461 03:14:48,080 --> 03:14:51,840 And how that, you -- you know, what impact will the 3462 03:14:51,840 --> 03:14:53,720 award have on the fellow. 3463 03:14:53,960 --> 03:14:56,640 And last but not least, here is your fellowship 3464 03:14:56,920 --> 03:14:57,800 vision or goals. 3465 03:14:57,800 --> 03:15:01,160 That is what do you hope to achieve with this 3466 03:15:01,160 --> 03:15:03,840 fellowship and what knowledge, skills or expertise do 3467 03:15:03,840 --> 03:15:05,160 you hope to acquire. 3468 03:15:05,160 --> 03:15:09,320 And then what will this specific mentor and mentee team 3469 03:15:09,320 --> 03:15:11,840 help you to approach this goal. 3470 03:15:11,840 --> 03:15:15,440 So this is somewhere where they can bring in the 3471 03:15:15,440 --> 03:15:15,800 mentor. 3472 03:15:15,800 --> 03:15:18,000 I'll talk about this in the future. 3473 03:15:18,000 --> 03:15:23,040 But the idea is that we want the fellow to introduce 3474 03:15:23,040 --> 03:15:28,040 themselves to you and then in the second one will be 3475 03:15:28,040 --> 03:15:29,440 contributions to science. 3476 03:15:29,440 --> 03:15:33,120 Here in contributions to science, obviously if they 3477 03:15:33,120 --> 03:15:38,200 have papers, they can explain them, but we want to give 3478 03:15:38,200 --> 03:15:42,800 an open forum to say other things that they've done 3479 03:15:42,800 --> 03:15:45,560 that they consider contributions to science. 3480 03:15:45,560 --> 03:15:49,480 And then we have fellowship qualifications. We are 3481 03:15:49,480 --> 03:15:52,920 hoping that everybody will agree that grades for 3482 03:15:52,920 --> 03:15:57,280 undergrad are not needed, not for grad students and not 3483 03:15:57,280 --> 03:15:58,600 for post docs. 3484 03:15:58,600 --> 03:16:02,160 This is not a place to list their courses and grades, 3485 03:16:02,160 --> 03:16:04,440 also not the institutions they went to. 3486 03:16:04,440 --> 03:16:05,240 This is elsewhere. 3487 03:16:05,240 --> 03:16:08,480 This is really an opportunity to document experiences 3488 03:16:08,480 --> 03:16:12,080 that they have that they think will contribute to 3489 03:16:12,080 --> 03:16:16,120 qualify them for this proposal. This can be lab 3490 03:16:16,120 --> 03:16:18,160 rotations, conferences, internships, other appointments 3491 03:16:18,160 --> 03:16:22,200 or any life experience that I can contribute to their 3492 03:16:22,200 --> 03:16:25,040 professional development and will make them qualified 3493 03:16:25,040 --> 03:16:26,240 for these grants. 3494 03:16:26,240 --> 03:16:30,400 We are encouraging team to include outreach for 3495 03:16:30,400 --> 03:16:34,560 science, engagement or advocacy and other things that 3496 03:16:34,560 --> 03:16:38,720 might demonstrate their passion and commitment to a 3497 03:16:38,720 --> 03:16:40,280 successful scientific career. 3498 03:16:40,280 --> 03:16:42,440 And honors, which is pretty straightforward. 3499 03:16:42,440 --> 03:16:46,440 And last but not least, an optional part for special 3500 03:16:46,440 --> 03:16:49,640 circumstances. This can be pandemic effect, having 3501 03:16:49,640 --> 03:16:53,240 experienced harassment or any other kind of personal or 3502 03:16:53,240 --> 03:16:54,440 family difficulty etcetera. 3503 03:16:54,440 --> 03:16:58,440 Importantly, we want to allow that this last part, if 3504 03:16:58,440 --> 03:17:01,640 they don't want to include in their biosketch, 3505 03:17:01,640 --> 03:17:05,240 sometimes it might be more appropriate to allow the 3506 03:17:05,240 --> 03:17:09,200 institution a chair or someone else to add a letter 3507 03:17:09,200 --> 03:17:12,000 that might explain what these extenuating circumstances 3508 03:17:12,000 --> 03:17:12,800 have been. 3509 03:17:12,800 --> 03:17:18,080 So this is the application materials. 3510 03:17:18,080 --> 03:17:21,200 The second one is a pretty -- it seems drastic, but 3511 03:17:21,200 --> 03:17:24,360 it's actually a pretty good revision of the criteria. 3512 03:17:24,360 --> 03:17:28,120 So the current cry tear ya in the nutshell is on the 3513 03:17:28,120 --> 03:17:28,400 left. 3514 03:17:29,280 --> 03:17:32,440 What we want to do on the right is to take a away a -- 3515 03:17:32,440 --> 03:17:34,920 this is a training fellowship. So the science needs to 3516 03:17:34,920 --> 03:17:37,200 be evaluated and the training needs to be evaluated and 3517 03:17:37,200 --> 03:17:39,200 the sponsor and the institutions are only important in 3518 03:17:39,200 --> 03:17:40,520 the context of these goals. 3519 03:17:40,520 --> 03:17:44,760 So we decided to take out the sponsor and the 3520 03:17:44,760 --> 03:17:48,120 institution as stand alone criteria. Because at 3521 03:17:48,120 --> 03:17:52,800 least -- I think for most of us, we are trained to 3522 03:17:52,800 --> 03:17:55,320 evaluate scientific achievements and environment in -- 3523 03:17:55,320 --> 03:17:57,880 not in the concept of training. 3524 03:17:58,560 --> 03:18:03,520 So we -- and many of the people that gave input found 3525 03:18:03,520 --> 03:18:08,520 that a person who is a fantastic scientist might not be 3526 03:18:08,520 --> 03:18:13,040 a very good mentor, but not necessarily get low scores 3527 03:18:13,040 --> 03:18:14,360 when evaluated independently. 3528 03:18:14,880 --> 03:18:18,960 So we weaved all of these and ended up with three 3529 03:18:18,960 --> 03:18:19,680 proposed criteria. 3530 03:18:19,680 --> 03:18:23,720 The first one is focus on the applicant, the second on 3531 03:18:23,720 --> 03:18:27,760 the science and the third one on the training of the 3532 03:18:27,760 --> 03:18:28,120 proposal. 3533 03:18:28,120 --> 03:18:35,160 So to the applicant, this is the idea of casting a 3534 03:18:35,160 --> 03:18:40,960 wider net to find the commons in the rough. 3535 03:18:40,960 --> 03:18:46,560 So the idea for this will basically be first score the 3536 03:18:46,560 --> 03:18:49,080 research interests, professional goals and 3537 03:18:49,080 --> 03:18:51,120 qualifications of the applicant. 3538 03:18:51,120 --> 03:18:54,000 So basically the information including the biosketch. 3539 03:18:54,000 --> 03:18:58,560 In the second one, we will ask reviewers to try to not 3540 03:18:58,560 --> 03:19:01,960 focus only on their comments so far, but also 3541 03:19:02,320 --> 03:19:06,120 how likely it is that they will benefit from the 3542 03:19:06,120 --> 03:19:06,880 proposed fellowship. 3543 03:19:06,880 --> 03:19:12,440 Pay a lot of attention on what they talk about, their 3544 03:19:12,440 --> 03:19:14,960 determination, persistence, creativity and otherwise 3545 03:19:14,960 --> 03:19:17,520 potential to do incredibly well. 3546 03:19:17,520 --> 03:19:20,920 For science and scientific resources, the goal is to 3547 03:19:20,920 --> 03:19:23,600 eliminate sponsor and environment as stand alone 3548 03:19:23,600 --> 03:19:27,000 categories and weave them into the importance of both 3549 03:19:27,000 --> 03:19:28,920 the science and the training. 3550 03:19:28,920 --> 03:19:33,080 For the scientific part, we would like reviewers to 3551 03:19:33,080 --> 03:19:37,680 score the quality of the proposed science based on the 3552 03:19:37,680 --> 03:19:40,920 scientific rigger and feasibility of the proposed 3553 03:19:40,920 --> 03:19:41,400 project. 3554 03:19:41,400 --> 03:19:45,200 But we would like to introduce, again, like this delta, 3555 03:19:45,200 --> 03:19:48,680 which is how is the project serves the applicant 3556 03:19:48,680 --> 03:19:50,560 training goals and training needs. 3557 03:19:50,560 --> 03:19:55,760 So, again, we ask reviewers to think of the goals of 3558 03:19:55,760 --> 03:20:00,440 the scientist and try to figure out how this scientific 3559 03:20:00,440 --> 03:20:03,720 percentaging that they choose helps them advance 3560 03:20:03,720 --> 03:20:05,120 towards those goals. 3561 03:20:05,120 --> 03:20:08,400 The extent to which they have scientific resources 3562 03:20:08,400 --> 03:20:11,680 available to them realistically, not just that they 3563 03:20:11,680 --> 03:20:15,800 exist in the institution or that they have a fantastic 3564 03:20:15,800 --> 03:20:20,320 team of mentors, but that they are engaged in making it 3565 03:20:20,320 --> 03:20:21,960 into a successful application. 3566 03:20:21,960 --> 03:20:26,560 And then last but most least is to describe in detail 3567 03:20:26,560 --> 03:20:29,920 how the sponsored training team will provide this 3568 03:20:29,920 --> 03:20:30,360 expertise. 3569 03:20:30,360 --> 03:20:33,880 So both the resources available to them and sort of 3570 03:20:33,880 --> 03:20:35,280 like a human resources. 3571 03:20:35,280 --> 03:20:39,520 For the training plan it's a similar thing, but in this 3572 03:20:39,520 --> 03:20:40,640 instance, it's separate. 3573 03:20:40,640 --> 03:20:43,400 It's not only about the scientific project, but really 3574 03:20:43,400 --> 03:20:45,560 about the training program for the scientist. 3575 03:20:45,560 --> 03:20:49,000 And, again, evaluate a few considerations and the first 3576 03:20:49,000 --> 03:20:52,880 one is is the training program tailored to advance the 3577 03:20:52,880 --> 03:20:55,960 goals for the career that this person particularly 3578 03:20:55,960 --> 03:21:00,200 wants. How much of a difference this proposal does, so 3579 03:21:00,200 --> 03:21:02,120 this is the so-called delta. 3580 03:21:02,120 --> 03:21:05,880 If they have the proposal, do they have a larger 3581 03:21:05,880 --> 03:21:07,720 ability to accomplish these goals? 3582 03:21:07,720 --> 03:21:10,160 The extent to which the institutional training 3583 03:21:10,160 --> 03:21:12,920 resources are available to this candidate, not just 3584 03:21:12,920 --> 03:21:15,680 exist, so specifically saying, you know, they will 3585 03:21:15,680 --> 03:21:18,800 attend this workshops, they will learn these things a 3586 03:21:18,800 --> 03:21:21,920 little bit more specifically and, again, the plan for 3587 03:21:21,920 --> 03:21:25,040 how the sponsor and training team will provide the 3588 03:21:25,040 --> 03:21:27,800 teaching and mentorship that the applicant needs to 3589 03:21:27,800 --> 03:21:30,920 advance towards their career goals and to -- and the 3590 03:21:30,920 --> 03:21:33,160 goals of the proposal, of course. 3591 03:21:33,160 --> 03:21:40,120 So here is an opportunity to say if you're a junior 3592 03:21:40,120 --> 03:21:45,800 person, for example, what is your training plan, what 3593 03:21:45,800 --> 03:21:47,680 is your mentorship? 3594 03:21:47,680 --> 03:21:50,800 Here is your opportunity to describe how you're going 3595 03:21:50,800 --> 03:21:54,600 to be a great mentor for the student and how you'll 3596 03:21:54,600 --> 03:21:56,360 receive wonderful training and etcetera. 3597 03:21:56,360 --> 03:21:59,440 Last but not least, this was a common thing. 3598 03:21:59,440 --> 03:22:02,840 People say that if I don't have a number one, I'll 3599 03:22:02,840 --> 03:22:05,280 never get a fellowship or things like that. 3600 03:22:05,280 --> 03:22:08,640 This is an opportunity to say the need to demonstrate 3601 03:22:08,640 --> 03:22:11,320 there's sufficient resources to carry out the science 3602 03:22:11,320 --> 03:22:13,680 proposed, obviously e obviously, we know fellowships 3603 03:22:13,680 --> 03:22:16,320 only go towards the salary of the fellow. 3604 03:22:16,320 --> 03:22:20,720 They don't allow a lot of money -- a little money, 3605 03:22:21,160 --> 03:22:26,000 but not a lot of money to carry out this process. 3606 03:22:26,280 --> 03:22:29,400 Here is an opportunity for a junior person to say I 3607 03:22:29,400 --> 03:22:32,200 have enough starter funds to carry out this project or 3608 03:22:32,200 --> 03:22:34,760 there's money from different agencies or there's not a 3609 03:22:34,760 --> 03:22:37,560 lot of money required for these goals that target the 3610 03:22:37,560 --> 03:22:40,400 career goals of the applicant and the money is there 3611 03:22:40,400 --> 03:22:41,800 and then that is enough. 3612 03:22:41,800 --> 03:22:44,840 And it should not be -- you know, a good proposal does 3613 03:22:44,840 --> 03:22:47,600 not need to come from an incredibly largely funded lab. 3614 03:22:47,880 --> 03:22:51,760 Just, you know, sufficient and necessary for the 3615 03:22:51,760 --> 03:22:52,720 particular project. 3616 03:22:52,720 --> 03:22:53,080 All right. 3617 03:22:53,080 --> 03:22:55,960 So that is reviewing criteria. 3618 03:22:55,960 --> 03:23:00,480 We think there are other things that might help in the 3619 03:23:00,480 --> 03:23:03,720 review process, also held sort of evaluate all 3620 03:23:03,720 --> 03:23:05,760 candidates in the same way. 3621 03:23:05,760 --> 03:23:09,120 One of the things we learn training grounds already do 3622 03:23:09,120 --> 03:23:13,120 and are already sort of done in a way of the review 3623 03:23:13,120 --> 03:23:16,120 process is to cluster the applications and discuss them 3624 03:23:16,120 --> 03:23:16,480 separately. 3625 03:23:16,480 --> 03:23:20,760 So the minds are preparing like proposals and find the 3626 03:23:20,760 --> 03:23:22,440 best within that group. 3627 03:23:22,440 --> 03:23:25,320 There are two things that kept coming up. 3628 03:23:25,320 --> 03:23:28,840 One is early career sponsors feels like they don't have 3629 03:23:28,840 --> 03:23:30,280 a very good chance. 3630 03:23:30,280 --> 03:23:33,160 And institutions that are not research intensive 3631 03:23:33,160 --> 03:23:33,560 institutions. 3632 03:23:33,560 --> 03:23:38,160 One idea was to maybe separate research active versus 3633 03:23:38,160 --> 03:23:41,760 research intensive institutions and also, for example, 3634 03:23:41,760 --> 03:23:45,320 junior sponsor versus more senior sponsors being 3635 03:23:45,320 --> 03:23:46,320 discussed separately. 3636 03:23:46,320 --> 03:23:49,560 This is an ongoing discussion in the workgroup and we 3637 03:23:49,560 --> 03:23:51,480 would love your input in this. 3638 03:23:51,480 --> 03:23:53,920 Another suggestion change is to change the order in 3639 03:23:53,920 --> 03:23:55,280 which the information is presented. 3640 03:23:55,280 --> 03:23:57,800 Again, it's something that is being talked about in 3641 03:23:57,800 --> 03:24:00,600 general in the review process, which is that in the 3642 03:24:00,600 --> 03:24:03,400 biosketch, the first thing that you see is not the 3643 03:24:03,400 --> 03:24:05,920 institution sort of like the pedigree of the applicant. 3644 03:24:05,920 --> 03:24:09,720 It's something you see later in the application, but 3645 03:24:09,720 --> 03:24:13,560 maybe doesn't immediately make you think about sort of 3646 03:24:13,560 --> 03:24:13,960 that. 3647 03:24:13,960 --> 03:24:19,960 Last but very much not least, we thought of ideas to 3648 03:24:19,960 --> 03:24:22,640 improve outreach and reviewer training. 3649 03:24:22,640 --> 03:24:25,520 So I think you heard from Kristin the wonderful changes 3650 03:24:25,520 --> 03:24:28,680 her office is making and we would love to piggyback on 3651 03:24:28,680 --> 03:24:28,960 this. 3652 03:24:28,960 --> 03:24:34,200 The first to try to outreach schools that have fewer 3653 03:24:34,200 --> 03:24:34,720 resources. 3654 03:24:34,720 --> 03:24:38,520 In order-serving institutions or many institutions that 3655 03:24:38,520 --> 03:24:43,920 have a low frequency or no frequency of submitting F. 3656 03:24:43,920 --> 03:24:48,800 Applications, this would be good for partners with ICs. 3657 03:24:48,800 --> 03:24:52,160 Typically these kinds of things are done at scientific 3658 03:24:52,160 --> 03:24:54,000 conferences and things like that. 3659 03:24:54,000 --> 03:24:57,840 Some of us in particular think this is not good enough 3660 03:24:57,840 --> 03:25:00,600 because some people from some institutions don't have 3661 03:25:00,600 --> 03:25:03,760 access to travel and go to the scientific conferences. 3662 03:25:03,760 --> 03:25:07,120 So because of the online ability to serve multiple 3663 03:25:07,120 --> 03:25:10,480 institutions at the same time, this might be something 3664 03:25:10,480 --> 03:25:14,600 that CSR may be able to do, sort of offer online 3665 03:25:14,600 --> 03:25:18,320 workshops for a broad group of institutions to do this. 3666 03:25:18,320 --> 03:25:22,440 Obviously, the second one would be to have outreach 3667 03:25:22,440 --> 03:25:25,600 material that is accessible online to anyone. 3668 03:25:25,600 --> 03:25:29,360 And not necessarily through a workshop, but to try to 3669 03:25:29,360 --> 03:25:32,680 mimic or to say overcome the advantage that large 3670 03:25:32,680 --> 03:25:35,320 institutions have to provide, for example, successful 3671 03:25:35,320 --> 03:25:38,280 examples of previous applications or access to material 3672 03:25:38,280 --> 03:25:39,760 or things like that. 3673 03:25:39,760 --> 03:25:43,720 So this idea would be maybe having a website that are 3674 03:25:43,720 --> 03:25:46,600 almost like myth busters or frequently asked questions 3675 03:25:46,600 --> 03:25:50,200 that if you're concerned about this, these are ways to 3676 03:25:50,200 --> 03:25:50,920 address it. 3677 03:25:50,920 --> 03:25:56,680 These are successful outcomes, this is how the review 3678 03:25:56,680 --> 03:25:58,600 criteria would work. 3679 03:25:58,600 --> 03:26:02,400 If we change the review criteria, these might take some 3680 03:26:02,400 --> 03:26:05,040 time to do, but provide successful applications. 3681 03:26:05,040 --> 03:26:08,040 Show people that people from non conventional 3682 03:26:08,040 --> 03:26:11,520 backgrounds are successful and show them how they 3683 03:26:11,520 --> 03:26:15,000 became successful in doing these kinds of proposals 3684 03:26:15,000 --> 03:26:19,320 would be something that would be very nice to the 3685 03:26:19,320 --> 03:26:19,760 outreach. 3686 03:26:19,760 --> 03:26:22,720 There were other nice ideas about having people that 3687 03:26:22,720 --> 03:26:26,040 served on panels or chairs or things like that gives 3688 03:26:26,040 --> 03:26:29,000 sort of mentoring office hours or things like that 3689 03:26:29,000 --> 03:26:32,320 that might be able to review proposals and help people 3690 03:26:32,320 --> 03:26:35,960 develop them that might not have a direct ability to do 3691 03:26:35,960 --> 03:26:37,280 this within their institution. 3692 03:26:37,280 --> 03:26:41,480 And then for reviewer training, I think as you saw, a 3693 03:26:41,480 --> 03:26:44,160 lot of people thought that specialized reviewer 3694 03:26:44,160 --> 03:26:47,960 training for fellowships would be a very nice thing to 3695 03:26:47,960 --> 03:26:48,360 develop. 3696 03:26:48,360 --> 03:26:52,400 Again, evaluate the application as training merits, not 3697 03:26:52,400 --> 03:26:53,920 on sponsor reputation. 3698 03:26:53,920 --> 03:26:56,720 It's something that came over very much. 3699 03:26:56,720 --> 03:27:00,880 Be very careful of hurtful and discouraging language. 3700 03:27:00,880 --> 03:27:04,760 This is something that I think a lot of us have 3701 03:27:04,760 --> 03:27:07,240 experienced, that person who got their fellowship 3702 03:27:07,240 --> 03:27:10,440 reviews and they were not constructive and it really 3703 03:27:10,440 --> 03:27:13,960 kind of makes a dent in their enthusiasm for science 3704 03:27:13,960 --> 03:27:17,160 that I think just doesn't need to be there. 3705 03:27:17,160 --> 03:27:21,640 So I think it seems like it is a good thing to remind 3706 03:27:21,640 --> 03:27:22,680 us of that. 3707 03:27:22,680 --> 03:27:26,000 Make sure that reviewers understand that if someone 3708 03:27:26,000 --> 03:27:31,000 says that they don't want to become a PI in the image 3709 03:27:31,000 --> 03:27:33,040 of their sponsor is okay. 3710 03:27:33,040 --> 03:27:38,120 There are a lot of wonderful scientists needed in all 3711 03:27:38,120 --> 03:27:42,640 sorts of life and particularly these fellow ships are 3712 03:27:42,640 --> 03:27:47,200 for those that have a future career in biomedic 3713 03:27:47,200 --> 03:27:47,680 research. 3714 03:27:47,680 --> 03:27:51,080 There's a lot of flavors. 3715 03:27:51,080 --> 03:27:51,360 Sorry. 3716 03:27:51,360 --> 03:27:53,880 I thought I had one more. Apparently not. 3717 03:27:53,880 --> 03:27:58,240 Last, I want to open the discussion to council. We 3718 03:27:58,240 --> 03:28:01,040 would like your discussion and reactions, especially 3719 03:28:01,040 --> 03:28:02,240 regarding these ideas. 3720 03:28:02,240 --> 03:28:05,200 Eliminate the sponsor, collaborators and institutional 3721 03:28:05,200 --> 03:28:08,680 requirement in the training criteria, explicitly allow 3722 03:28:08,680 --> 03:28:11,680 a wider range of career paths. 3723 03:28:11,680 --> 03:28:15,800 Have reviewers consider "the delta," how much of a 3724 03:28:15,800 --> 03:28:19,880 difference this would make in the fellowship of their 3725 03:28:19,880 --> 03:28:20,360 career. 3726 03:28:20,360 --> 03:28:25,000 Eliminate grades as indicators of qualifications, 3727 03:28:25,000 --> 03:28:30,360 encourage a statement of applicant qualifications that 3728 03:28:30,360 --> 03:28:36,560 extends beyond course, grades and publications. And 3729 03:28:36,560 --> 03:28:41,960 then allow an optional statement of special 3730 03:28:41,960 --> 03:28:49,640 circumstances and optionally have this to come if it is 3731 03:28:49,640 --> 03:28:50,440 school. 3732 03:28:50,440 --> 03:28:51,560 That is it. Okay. I'll put that back so you can see 3733 03:28:51,560 --> 03:28:56,600 If you want to unshare, we'll open it up for discussion 3734 03:28:56,600 --> 03:28:58,000 at this point. 3735 03:28:58,000 --> 03:28:58,600 Mark. 3736 03:28:58,600 --> 03:29:02,520 >> DR. MARK PEIFER: Could she lead the slide up while 3737 03:29:02,520 --> 03:29:04,760 we have a summary of that last -- 3738 03:29:06,880 --> 03:29:09,600 >> DR. BRUCE REED: Sure. 3739 03:29:09,600 --> 03:29:17,720 Sorry. 3740 03:29:17,720 --> 03:29:19,760 >> DR. ELIZABETH VILLA: There you go. 3741 03:29:19,760 --> 03:29:22,480 >> DR. MARK PEIFER: I think everyone will agree with 3742 03:29:22,480 --> 03:29:24,200 some of these things like eliminating grades. 3743 03:29:24,200 --> 03:29:26,720 But I want to focus on the bigger picture thing. 3744 03:29:26,720 --> 03:29:28,280 I want super excited about this. 3745 03:29:28,280 --> 03:29:32,640 I think this potentially will have as much or more 3746 03:29:32,640 --> 03:29:36,560 impact than the discussion that we had of review 3747 03:29:36,560 --> 03:29:37,440 criteria overall. 3748 03:29:37,440 --> 03:29:43,040 And, you know, the big picture, I think we all realize, 3749 03:29:43,040 --> 03:29:47,640 right, that the fellowship programs, to a large extent, 3750 03:29:47,640 --> 03:29:52,240 have become an opportunity for people who have had 3751 03:29:52,240 --> 03:29:57,840 privilege to get more. That would fit my own career, 3752 03:29:57,840 --> 03:29:58,360 right? 3753 03:29:58,360 --> 03:30:04,560 I had a -- and NSF graduate fellowship. 3754 03:30:05,320 --> 03:30:09,440 Once you get on that ladder and you're at the right 3755 03:30:09,440 --> 03:30:12,840 institution, working for the right person, then all of 3756 03:30:12,840 --> 03:30:16,200 these things become much easier for you to get. 3757 03:30:16,200 --> 03:30:20,320 Again, not to diminish the -- you know, the work of all 3758 03:30:20,320 --> 03:30:24,480 those people who are in labs like that who write a 3759 03:30:24,480 --> 03:30:25,400 fantastic proposal. 3760 03:30:25,400 --> 03:30:32,040 So for me, the most important things I see here are -- 3761 03:30:32,040 --> 03:30:38,080 and, again, completely in concert with what we did with 3762 03:30:38,080 --> 03:30:42,920 respect to overall review criteria, greatly reduce the 3763 03:30:42,920 --> 03:30:47,760 constitutional environment piece except as key parts of 3764 03:30:47,760 --> 03:30:49,760 the training plan. 3765 03:30:49,760 --> 03:30:53,920 Someone might be a Nobel laureate, but if they never 3766 03:30:53,920 --> 03:30:57,600 see their graduate student or post doc, they're not 3767 03:30:57,600 --> 03:31:00,520 providing a training environment to somebody doing 3768 03:31:00,520 --> 03:31:01,760 their very best. 3769 03:31:01,760 --> 03:31:09,080 And I really like your description of "the delta," the 3770 03:31:09,080 --> 03:31:15,680 idea that we're -- you know, the idea that taking 3771 03:31:15,680 --> 03:31:22,280 somebody with emotional and moving them up, I think 3772 03:31:22,280 --> 03:31:29,640 that's a place where these kinds of programs can have 3773 03:31:29,640 --> 03:31:34,040 the -- by far, the most impact. 3774 03:31:36,680 --> 03:31:39,000 I'm very excited about it. 3775 03:31:39,000 --> 03:31:45,680 >> DR. KAREN ANDERSON: I just cannot say how much this 3776 03:31:45,680 --> 03:31:50,120 could impact the diversity in S.T.E.M. moving forward 3777 03:31:50,120 --> 03:31:55,680 and this just really touches on so many areas that 3778 03:31:55,680 --> 03:32:00,120 might students have had, these comments, these exact 3779 03:32:00,120 --> 03:32:03,440 comments on multiple different trainee grants. 3780 03:32:03,440 --> 03:32:07,880 And in many of press approaches, we're in a Hispanic 3781 03:32:07,880 --> 03:32:10,520 serving institution, this culture shift in 3782 03:32:10,520 --> 03:32:14,040 transformative change, I think, can really impact just 3783 03:32:14,040 --> 03:32:18,040 a tremendous amount of students across the country and 3784 03:32:18,040 --> 03:32:22,880 I'm really looking forward to this. I have no specific 3785 03:32:22,880 --> 03:32:28,200 comments on any of this, other than I just want to say 3786 03:32:28,200 --> 03:32:33,040 from the bottom of my heart, thank you to the working 3787 03:32:33,040 --> 03:32:36,600 group and to all -- to the incredibly thoughtful 3788 03:32:36,600 --> 03:32:39,240 approach that you've taken to this. 3789 03:32:39,520 --> 03:32:42,440 >> DR. BRUCE REED: Thanks, Karen. 3790 03:32:42,440 --> 03:32:43,120 Tonya. 3791 03:32:43,120 --> 03:32:47,040 >> DR. TONYA PALERMO: Great job, Bruce and Elizabeth 3792 03:32:47,040 --> 03:32:48,960 and the workgroup, as well. 3793 03:32:48,960 --> 03:32:52,600 I wanted to ask you a little bit more about the 3794 03:32:52,600 --> 03:32:54,920 interesting component that you added around evaluating 3795 03:32:54,920 --> 03:32:57,560 this delta or the difference the fellowship makes. 3796 03:32:57,560 --> 03:33:01,320 I'm not sure how the applicant will write about this in 3797 03:33:01,320 --> 03:33:04,040 the fellowship and how that would be reviewed. 3798 03:33:04,040 --> 03:33:08,600 I'm trying to understand how that could be 3799 03:33:08,600 --> 03:33:13,160 evaluated, I guess, with less judgment because it seems 3800 03:33:13,160 --> 03:33:16,720 like a potentially difficult thing to vault. 3801 03:33:16,720 --> 03:33:18,880 >> DR. ELIZABETH VILLA: Bruce, do you want to try or -- 3802 03:33:18,880 --> 03:33:21,200 >> DR. BRUCE REED: Why don't you start. 3803 03:33:21,200 --> 03:33:22,120 >> DR. ELIZABETH VILLA: Okay. 3804 03:33:22,120 --> 03:33:23,520 That sounds good. 3805 03:33:23,520 --> 03:33:27,880 So, first of all, I forgot to say that it was an 3806 03:33:27,880 --> 03:33:30,400 absolute pleasure to work with the workgroup. 3807 03:33:30,400 --> 03:33:34,360 I think they were very sort of like strongly 3808 03:33:34,360 --> 03:33:37,840 opinionated group which was great and finding consensus 3809 03:33:37,840 --> 03:33:39,160 was not easy. 3810 03:33:39,160 --> 03:33:42,440 Tonya, to your point, I completely agree. 3811 03:33:42,440 --> 03:33:46,720 I think that it is not trivial to establish and it's 3812 03:33:46,720 --> 03:33:50,640 probably on the fellow and the -- and the -- or the 3813 03:33:50,640 --> 03:33:54,160 applicant and the sponsor to try to explain these 3814 03:33:54,160 --> 03:33:54,560 things. 3815 03:33:55,400 --> 03:34:04,640 Like if I get this, putting it bluntly, it gets me on 3816 03:34:04,640 --> 03:34:09,280 the train that Mark just described. 3817 03:34:09,280 --> 03:34:14,440 We came to this and I go to the same place, because 3818 03:34:14,440 --> 03:34:18,320 ideally we would hope -- again, this is a program 3819 03:34:18,320 --> 03:34:21,760 decision -- that there would be different pools of 3820 03:34:21,760 --> 03:34:25,640 applicants where you could -- you know, people that are 3821 03:34:25,640 --> 03:34:28,200 extraordinary will write flawless proposals would 3822 03:34:28,200 --> 03:34:30,360 presumably still get these fellowships. 3823 03:34:31,040 --> 03:34:33,320 They still deserve it and they're great. 3824 03:34:33,320 --> 03:34:37,680 And then to give an opportunity to expand that pool to 3825 03:34:37,680 --> 03:34:41,600 help people get on the right track to have a 3826 03:34:41,600 --> 03:34:44,760 presentation in future generations is what we were 3827 03:34:44,760 --> 03:34:49,880 trying to do. It is not a trivial thing to include at 3828 03:34:49,880 --> 03:34:53,440 the reviewer level, but this is what we're trying. 3829 03:34:53,440 --> 03:34:57,400 So, yeah, I think mentioning if I had this proposal, I 3830 03:34:57,400 --> 03:35:00,640 would have access to other trainings that I couldn't 3831 03:35:00,640 --> 03:35:03,880 otherwise because my adviser might use those funds to 3832 03:35:03,880 --> 03:35:06,040 do this or to do that. 3833 03:35:06,040 --> 03:35:09,360 It's not -- yeah, so I think that the collective brain 3834 03:35:09,360 --> 03:35:12,680 of council and others that might be listening to help 3835 03:35:12,680 --> 03:35:14,680 us materialize this would be great. 3836 03:35:14,920 --> 03:35:19,920 As I said, one thing that would be great is to try to, 3837 03:35:19,920 --> 03:35:23,760 in the future, sort of try to encourage program to 3838 03:35:23,760 --> 03:35:27,600 have -- you know, there are instances in which you have 3839 03:35:27,600 --> 03:35:31,800 the same RFA, but different sort of pools of money that 3840 03:35:31,800 --> 03:35:35,280 allow you to not only cluster the applications and 3841 03:35:35,280 --> 03:35:37,560 discussion, but actually assign them 3842 03:35:37,560 --> 03:35:38,080 differently. 3843 03:35:38,080 --> 03:35:48,160 This is not something that would happen here, but maybe 3844 03:35:48,160 --> 03:35:53,160 in the future it could. 3845 03:35:53,160 --> 03:35:59,160 >> DR. BRUCE REED: I would say evaluating these 3846 03:35:59,160 --> 03:36:00,960 candidates are challenging. 3847 03:36:00,960 --> 03:36:04,360 I think the delta came from both directions. Sometimes 3848 03:36:04,360 --> 03:36:07,080 you'll see an application and it's a fabulous 3849 03:36:07,080 --> 03:36:10,520 application. And you end up thinking even if the 3850 03:36:10,520 --> 03:36:13,600 person doesn't get this fellowship, they're going to do 3851 03:36:13,600 --> 03:36:17,320 great. They are in a fabulous environment. They are 3852 03:36:17,320 --> 03:36:21,080 going to do great. You see other people for whom 3853 03:36:21,080 --> 03:36:23,840 you're not so sure that that is true. 3854 03:36:23,840 --> 03:36:25,720 They seem well qualified. 3855 03:36:25,720 --> 03:36:28,520 They seem to have great potential. 3856 03:36:28,520 --> 03:36:33,280 But they're not maybe in as well a resourced place. 3857 03:36:33,280 --> 03:36:38,600 They don't maybe have the same connections. 3858 03:36:38,600 --> 03:36:42,520 And you think if they don't get this, yeah, it probably 3859 03:36:42,520 --> 03:36:46,840 is going to make a difference one way or the other, a 3860 03:36:46,840 --> 03:36:48,960 lot of difference in their career. 3861 03:36:48,960 --> 03:36:53,680 So part of this is sort of a -- I guess a philosophical 3862 03:36:53,680 --> 03:36:57,560 thing of where NIH should put its resources and that's 3863 03:36:57,560 --> 03:37:00,320 clearly not just a with you decision. 3864 03:37:00,600 --> 03:37:04,840 And then, of course, there is a challenging decision of 3865 03:37:04,840 --> 03:37:08,680 how do you evaluate on who those people are. 3866 03:37:08,680 --> 03:37:09,400 Michelle. 3867 03:37:09,400 --> 03:37:13,800 >> DR. MICHELLE JANELSINS: Great presentation, 3868 03:37:13,800 --> 03:37:15,680 Elizabeth and Bruce. 3869 03:37:15,680 --> 03:37:17,200 Really liked a lot of it. 3870 03:37:17,200 --> 03:37:21,040 Definitely support the getting rid of the grade 3871 03:37:21,040 --> 03:37:25,360 requirement. You know, somebody got into grad school, 3872 03:37:25,360 --> 03:37:30,160 then that's great and they can say whatever they would 3873 03:37:30,160 --> 03:37:34,960 like about their grades if they want in their personal 3874 03:37:34,960 --> 03:37:35,440 statement. 3875 03:37:35,440 --> 03:37:39,600 But that is one I strongly feel needs to go. 3876 03:37:39,600 --> 03:37:44,040 And then I guess, you know, my other thought -- I have 3877 03:37:44,040 --> 03:37:46,840 just one comment, but also a question. 3878 03:37:47,080 --> 03:37:51,920 One comment would be -- and I really like this idea of 3879 03:37:51,920 --> 03:37:55,440 evening the playing field and creating diversity of 3880 03:37:55,440 --> 03:37:57,360 scientists in chosen profession. 3881 03:37:57,360 --> 03:38:03,160 And so then it's kind of like a little bit like a 3882 03:38:03,160 --> 03:38:04,120 marketing thing. 3883 03:38:04,120 --> 03:38:10,320 How can we encourage and get the word out to those in 3884 03:38:10,320 --> 03:38:14,960 all different types of colleges, you know, like through 3885 03:38:14,960 --> 03:38:15,480 organizations. 3886 03:38:15,480 --> 03:38:18,040 I certainly didn't know necessarily about an F 3887 03:38:18,040 --> 03:38:21,560 application -- I mean, I had an F award, but I didn't 3888 03:38:21,560 --> 03:38:25,440 know about it until I was in that school, in that grad 3889 03:38:25,440 --> 03:38:25,920 school. 3890 03:38:25,920 --> 03:38:30,560 So just getting the word out as a possibility just to 3891 03:38:30,560 --> 03:38:32,240 something to think about. 3892 03:38:32,240 --> 03:38:38,600 And then my question, I do think that it's a really 3893 03:38:38,600 --> 03:38:43,800 intriguing idea to think about similar, like, doing ESI 3894 03:38:43,800 --> 03:38:49,000 for R01 review together and then the more experienced 3895 03:38:49,000 --> 03:38:54,200 seasoned types of applications together or ESI and non 3896 03:38:54,200 --> 03:38:59,400 ESI, I guess. Would you review the assistant 3897 03:38:59,400 --> 03:39:04,600 professors together and everybody else or would you do 3898 03:39:04,600 --> 03:39:06,320 tenured, non tenured? 3899 03:39:06,320 --> 03:39:10,640 What was the thought about how things might be split if 3900 03:39:10,640 --> 03:39:14,160 there was a dedicated discussion area to an earlier 3901 03:39:14,160 --> 03:39:15,760 career mentor, for example? 3902 03:39:15,760 --> 03:39:17,520 >> DR. ELIZABETH VILLA: So there was a lot of 3903 03:39:17,520 --> 03:39:18,000 discussion about it. 3904 03:39:18,000 --> 03:39:21,760 We did not reach a consensus partially because some 3905 03:39:21,760 --> 03:39:25,520 people thought, well, you know, maybe you're lucky and 3906 03:39:25,520 --> 03:39:29,680 you're not ESI any more, but it's your first fellowship 3907 03:39:29,680 --> 03:39:33,440 application or, you know, where to draw the line, 3908 03:39:33,440 --> 03:39:34,680 assistant versus non. 3909 03:39:34,680 --> 03:39:38,640 So building a new category, apparently, is not a 3910 03:39:38,640 --> 03:39:41,680 trivial thing to do within the databases. 3911 03:39:41,680 --> 03:39:45,720 A discussion was made of, like, only if it's the first 3912 03:39:45,720 --> 03:39:49,800 time you -- like if you've never had an F fellow in 3913 03:39:49,800 --> 03:39:54,240 your lab, you would be a newbie and this would allow to 3914 03:39:54,240 --> 03:39:57,920 have people from other areas, if their funding is not 3915 03:39:57,920 --> 03:40:00,240 NIH, their considered newbie only once. 3916 03:40:00,240 --> 03:40:05,480 So they can't populate their labs with 40 fellows 3917 03:40:05,480 --> 03:40:09,520 because they're only senior not typically NIH. 3918 03:40:09,520 --> 03:40:14,760 But I think some people prefer to have only really 3919 03:40:14,760 --> 03:40:15,840 newbie sponsors. 3920 03:40:15,840 --> 03:40:20,880 So we haven't tuned in on one and that it would be 3921 03:40:20,880 --> 03:40:23,800 something left to get input from you. 3922 03:40:23,800 --> 03:40:27,520 It is the same thing with research active versus 3923 03:40:27,520 --> 03:40:30,000 research intensive institutions when people thought 3924 03:40:30,000 --> 03:40:34,920 idea would be one thing but a lot of people say there's 3925 03:40:34,920 --> 03:40:37,800 people that are a non idea stage. 3926 03:40:37,800 --> 03:40:40,400 There's plenty of institutions in California or 3927 03:40:40,400 --> 03:40:42,960 Massachusetts that are not research intensive that 3928 03:40:42,960 --> 03:40:45,560 would also benefit from being discussed separately. 3929 03:40:45,560 --> 03:40:50,120 So these are things that we're still maybe opening 3930 03:40:50,120 --> 03:40:54,680 discussion and to feedback from you and everyone else 3931 03:40:54,680 --> 03:41:00,240 and how to implement that. We would like the concept 3932 03:41:00,240 --> 03:41:04,840 and we're still in discussions about how to implement 3933 03:41:04,840 --> 03:41:05,320 it. 3934 03:41:05,320 --> 03:41:06,840 >> DR. NONI BYRNES: There's people in line. 3935 03:41:06,840 --> 03:41:11,880 I do want to say a couple of comments at some point, 3936 03:41:11,880 --> 03:41:16,080 but please go ahead and get the people in line first. 3937 03:41:16,080 --> 03:41:24,120 >> DR. BRUCE REED: Let's go Leo, Jinming and then I'll 3938 03:41:24,120 --> 03:41:26,760 go to you, Noni. 3939 03:41:26,760 --> 03:41:27,160 Leo. 3940 03:41:27,160 --> 03:41:30,000 >> DR. LEOPOLDO CABASSA: I really appreciate the 3941 03:41:30,000 --> 03:41:31,240 discussion on the work. 3942 03:41:31,240 --> 03:41:35,160 One I guess more thinking about the focus of the 3943 03:41:35,160 --> 03:41:39,440 sponsor, and it's not -- I get the -- I totally get the 3944 03:41:39,440 --> 03:41:42,960 analysis that you have done here, which is very 3945 03:41:42,960 --> 03:41:43,360 telling. 3946 03:41:43,840 --> 03:41:45,440 But I've seen it both ways. 3947 03:41:45,440 --> 03:41:51,080 I've seen it where too senior is great and too junior 3948 03:41:51,080 --> 03:41:55,680 is also great and there's a catch-22 that happens 3949 03:41:55,680 --> 03:41:56,200 sometimes. 3950 03:41:56,200 --> 03:41:58,960 It's not the elimination of the sponsor, but maybe 3951 03:41:58,960 --> 03:42:01,720 around the delta that you're talking about about what 3952 03:42:01,720 --> 03:42:04,160 is the training plan, what is the engagement. 3953 03:42:04,160 --> 03:42:05,400 >> DR. ELIZABETH VILLA: Right. 3954 03:42:05,400 --> 03:42:09,960 >> DR. LEOPOLDO CABASSA: So I think some guidance 3955 03:42:09,960 --> 03:42:14,480 around that would be very important and access to that. 3956 03:42:14,480 --> 03:42:19,880 And then I do want to say as a directive of T-32, this 3957 03:42:19,880 --> 03:42:20,720 is great. 3958 03:42:20,720 --> 03:42:27,160 I appreciate the changes in the F-31. 3959 03:42:27,160 --> 03:42:30,200 But there are unintended consequences here that could 3960 03:42:30,200 --> 03:42:30,600 happen. 3961 03:42:30,600 --> 03:42:35,640 So it's thinking through how this will play out of the 3962 03:42:35,640 --> 03:42:39,320 training grounds because, you know, T-32s want F-31s 3963 03:42:39,320 --> 03:42:41,640 and they want KO awards. 3964 03:42:41,640 --> 03:42:44,800 So I think a discussion needs to be how this influences 3965 03:42:44,800 --> 03:42:46,560 the ecology of training within NIH. 3966 03:42:46,560 --> 03:42:52,040 >> DR. ELIZABETH VILLA: So we had -- we had people 3967 03:42:52,040 --> 03:42:56,560 from program specializing in T-32s and people that are 3968 03:42:56,560 --> 03:42:58,040 directors of T-32s. 3969 03:42:58,320 --> 03:43:02,640 I mean, everybody is worried every time you change a 3970 03:43:02,640 --> 03:43:04,800 criteria there are unintended consequences. 3971 03:43:04,800 --> 03:43:08,960 That's why we keep revising criteria forever and ever. 3972 03:43:08,960 --> 03:43:12,760 But we engaged -- besides at least me personally, but I 3973 03:43:12,760 --> 03:43:16,560 think a lot of people engage people that are interested 3974 03:43:16,560 --> 03:43:19,960 in these outcomes and everybody seemed to think that 3975 03:43:19,960 --> 03:43:23,040 this would be pretty -- a pretty reasonably good 3976 03:43:23,040 --> 03:43:23,400 outcome. 3977 03:43:23,960 --> 03:43:27,240 Again, ideally maybe there would be, you know, separate 3978 03:43:27,240 --> 03:43:29,440 and clarify and things like that. 3979 03:43:29,440 --> 03:43:34,360 But, you know, separate pools for the delta and non 3980 03:43:34,360 --> 03:43:34,880 delta. 3981 03:43:34,880 --> 03:43:39,880 But, yeah, I think if you have any specific thoughts of 3982 03:43:39,880 --> 03:43:43,080 what outcomes -- unintentional outcomes in the future, 3983 03:43:43,080 --> 03:43:44,440 just, yeah, please -- 3984 03:43:44,800 --> 03:43:46,280 >> DR. LEOPOLDO CABASSA: And I think this is great. 3985 03:43:46,280 --> 03:43:49,680 What I was saying is this is an opportunity to think 3986 03:43:49,680 --> 03:43:52,480 through the applicants of T-32 because this opens the 3987 03:43:52,480 --> 03:43:55,880 pool to a whole other applicant that as T.-32 you might 3988 03:43:55,880 --> 03:43:59,280 be like I don't want to bring that person in because 3989 03:43:59,280 --> 03:44:02,680 two or three years down the road the Fs will be 3990 03:44:02,680 --> 03:44:05,120 difficult because that person doesn't have the right 3991 03:44:05,120 --> 03:44:05,440 training. 3992 03:44:05,440 --> 03:44:09,000 So this opens that pool for providing opportunities. 3993 03:44:09,000 --> 03:44:16,720 >> DR. ELIZABETH VILLA: That is great. 3994 03:44:16,720 --> 03:44:20,880 >> DR. LEOPOLDO CABASSA: So I think that's great on 3995 03:44:20,880 --> 03:44:24,680 thing that through >> DR. ELIZABETH VILLA: And it's 3996 03:44:24,680 --> 03:44:27,720 making sure that everybody has the opportunity to 3997 03:44:27,720 --> 03:44:31,520 describe why they would be a good sponsor and just 3998 03:44:31,520 --> 03:44:34,960 being a fantastic institution and being a fantastic and 3999 03:44:34,960 --> 03:44:38,000 accomplished scientist does not immediately make you a 4000 03:44:38,000 --> 03:44:38,760 good sponsor. 4001 03:44:38,760 --> 03:44:41,960 I think it's what we are trying to say, that everybody 4002 03:44:41,960 --> 03:44:44,280 needs to -- everybody has, you know, whatever metrics 4003 03:44:44,280 --> 03:44:46,920 they want to demonstrate that makes a great sponsors, 4004 03:44:46,920 --> 03:44:50,400 they have a chance to do it, but it's not -- yeah, I 4005 03:44:50,400 --> 03:44:53,040 think that was -- that's what I'm -- that's what we're 4006 03:44:53,040 --> 03:44:54,200 trying to accomplish, anyway. 4007 03:44:55,240 --> 03:44:58,880 And I think by weaving it as opposed to having it 4008 03:44:58,880 --> 03:45:02,160 separate, it's easier for the reviewers not to do the 4009 03:45:02,160 --> 03:45:05,800 same thing which is in the review train, oh, it's Dr. 4010 03:45:05,800 --> 03:45:06,120 Fauci. 4011 03:45:06,120 --> 03:45:10,840 Hundreds of people have come out of that lab and become 4012 03:45:10,840 --> 03:45:11,280 successful. 4013 03:45:11,280 --> 03:45:14,800 That is not necessarily a given and that's what we're 4014 03:45:14,800 --> 03:45:15,840 trying to capture. 4015 03:45:15,840 --> 03:45:20,920 >> DR. BRUCE REED: So I'm going to go the Jinming. 4016 03:45:20,920 --> 03:45:22,480 And then to Noni. 4017 03:45:22,480 --> 03:45:25,240 We're tight on time. 4018 03:45:25,240 --> 03:45:26,200 I can let this run. 4019 03:45:26,200 --> 03:45:27,720 I acknowledge that it's 5:00. 4020 03:45:27,720 --> 03:45:31,120 This is a scheduled end so if you have a hard stop and 4021 03:45:31,120 --> 03:45:33,200 have to, that's unfortunate but I understand that. 4022 03:45:33,200 --> 03:45:38,520 But I will let the discussion run for a bit here. 4023 03:45:38,520 --> 03:45:41,200 Jinming. 4024 03:45:41,200 --> 03:45:43,640 >> DR. JINMING GAO: Thank you, Bruce. Thank you, 4025 03:45:43,640 --> 03:45:44,720 Elizabeth were for doing this. 4026 03:45:44,720 --> 03:45:47,240 I can imagine the heated probably discussion in the 4027 03:45:47,240 --> 03:45:47,520 panel. 4028 03:45:47,520 --> 03:45:54,640 I guess we are all not for giving free passes and how 4029 03:45:54,640 --> 03:45:59,400 do we provide equal opportunities to the candidates. 4030 03:45:59,400 --> 03:46:04,120 But I think we have to set up the guiding principles so 4031 03:46:04,120 --> 03:46:07,240 they don't lose equality or meritocracy in the 4032 03:46:07,240 --> 03:46:08,040 evaluation process. 4033 03:46:08,040 --> 03:46:12,080 And then there are sort of a system-based correction. 4034 03:46:12,080 --> 03:46:15,680 I wonder that for all of us working am deem ya, it's 4035 03:46:15,680 --> 03:46:17,760 not necessarily the most attractive career path. 4036 03:46:17,760 --> 03:46:22,200 I'm looking at my two boys. I don't think either of 4037 03:46:22,200 --> 03:46:25,520 them are serious think thinking about being a professor 4038 03:46:25,520 --> 03:46:26,960 when they grow up. 4039 03:46:26,960 --> 03:46:32,480 So I think this is sort of a threat to our profession 4040 03:46:32,480 --> 03:46:37,120 and systemically how do we, you know, raise the level 4041 03:46:37,120 --> 03:46:41,720 of the pay line so maybe diversifying the career path 4042 03:46:41,720 --> 03:46:46,320 and making it more enticing for the young people to 4043 03:46:46,320 --> 03:46:50,000 think about science and academic careers as something 4044 03:46:50,000 --> 03:46:50,440 exciting. 4045 03:46:50,440 --> 03:46:54,160 I wonder whether from the systems standpoint is 4046 03:46:54,160 --> 03:46:58,320 something helpful to get more talents into the field. 4047 03:46:58,320 --> 03:47:01,160 >> DR. ELIZABETH VILLA: I think you are absolutely 4048 03:47:01,160 --> 03:47:04,000 right that increasing the pie is always a good solution 4049 03:47:04,000 --> 03:47:05,440 if you can do it. 4050 03:47:05,440 --> 03:47:08,200 I'm not entirely sure it's feasible. 4051 03:47:08,200 --> 03:47:11,320 But I do want to say one thing about what you just 4052 03:47:11,320 --> 03:47:11,560 said. 4053 03:47:11,560 --> 03:47:15,600 And that is that there's actually a paper that shows of 4054 03:47:15,600 --> 03:47:19,600 F applicants, only a small fraction go to get a narrow 4055 03:47:19,600 --> 03:47:21,800 ride. Which is preoccupying itself. 4056 03:47:21,800 --> 03:47:25,040 And also we need fantastic scientists everywhere. 4057 03:47:25,040 --> 03:47:28,120 If you don't think that there's amazingly good careers 4058 03:47:28,120 --> 03:47:31,560 in policy, junior editor, you know, working in the lab 4059 03:47:31,560 --> 03:47:35,360 and all sorts of other things, I think that, you know, 4060 03:47:35,360 --> 03:47:38,480 we need to think about how we're presenting other 4061 03:47:38,480 --> 03:47:40,880 career goals because I think they're fantastic. 4062 03:47:40,880 --> 03:47:46,760 I don't know that my daughter is going to have a 4063 03:47:46,760 --> 03:47:51,600 professor, but I think there's ways to be scientists 4064 03:47:51,600 --> 03:47:56,960 that are very varied. There's a small sliver of 4065 03:47:56,960 --> 03:47:59,080 trainees that become professors. 4066 03:47:59,080 --> 03:48:00,160 In any meaningful way. 4067 03:48:00,160 --> 03:48:01,360 Respectfully. 4068 03:48:01,360 --> 03:48:05,080 I think we need to also think about maybe preparing 4069 03:48:05,080 --> 03:48:08,440 them for amazing careers that are alternative to ours 4070 03:48:08,440 --> 03:48:12,160 and, you know, because it's -- this is what the job 4071 03:48:12,160 --> 03:48:13,800 market is like, anyway. 4072 03:48:13,800 --> 03:48:18,400 So it is important to be supportive of other career 4073 03:48:18,400 --> 03:48:19,760 paths, I think. 4074 03:48:19,760 --> 03:48:24,320 >> DR. BRUCE REED: Noni and then I'll come back to 4075 03:48:24,320 --> 03:48:27,320 you, Mark. I see your hand up. 4076 03:48:27,320 --> 03:48:29,760 >> DR. NONI BYRNES: First, I want to say thank you so 4077 03:48:29,760 --> 03:48:29,960 much. 4078 03:48:29,960 --> 03:48:31,440 This is a tremendous amount of work. 4079 03:48:31,440 --> 03:48:34,280 It's very thoughtful and I really appreciate how you 4080 03:48:34,280 --> 03:48:37,800 took all of the information that we got and came up 4081 03:48:37,800 --> 03:48:40,320 with some really fantastic directions to go in. 4082 03:48:40,320 --> 03:48:44,080 This is an interim report so I just want to provide a 4083 03:48:44,080 --> 03:48:47,240 little bit of guidance moving forward before we get to 4084 03:48:47,240 --> 03:48:47,840 the final. 4085 03:48:47,840 --> 03:48:58,320 One is in -- you saw the scope of applications. 4086 03:48:58,760 --> 03:49:02,600 The more effective they are, the more complex. Within 4087 03:49:02,600 --> 03:49:06,080 the NIH, there's a tendency to make things complex 4088 03:49:06,080 --> 03:49:09,520 because you want to address everything that you think. 4089 03:49:09,520 --> 03:49:12,200 And I think that doesn't come out effective at the end 4090 03:49:12,200 --> 03:49:13,160 of the review process. 4091 03:49:13,160 --> 03:49:16,720 Just the sheer number of reviewers we have to train, 4092 03:49:16,720 --> 03:49:20,960 they have to behave the way you want them to behave in 4093 03:49:20,960 --> 03:49:22,000 an ideal world. 4094 03:49:22,000 --> 03:49:22,800 They don't. 4095 03:49:22,800 --> 03:49:25,800 So the simpler you make, the less complicated the 4096 03:49:25,800 --> 03:49:29,800 process, the easier it is. I would suggest you look t 4097 03:49:29,800 --> 03:49:32,120 different directions you're proposing and separate out 4098 03:49:32,120 --> 03:49:35,480 what is something that is very much something we can 4099 03:49:35,480 --> 03:49:38,480 actually tackle and review and what is something that 4100 03:49:38,480 --> 03:49:41,800 is kind of a broad -- really important, but broader NIH 4101 03:49:41,800 --> 03:49:45,440 direction. Which we can capture in the report, but we 4102 03:49:45,440 --> 03:49:47,440 both -- outreach, for example, the fellowship 4103 03:49:47,440 --> 03:49:49,440 applicants, especially in certain scientific areas. 4104 03:49:50,000 --> 03:49:53,800 There's some scientific areas that spent a lot of 4105 03:49:53,800 --> 03:49:56,720 fellowships and others that submit very few. 4106 03:49:56,720 --> 03:50:03,920 The ones that submit very few have a lot of URMs in 4107 03:50:03,920 --> 03:50:05,120 those fields. 4108 03:50:05,120 --> 03:50:08,560 I think thinking about that and developing those 4109 03:50:08,560 --> 03:50:11,600 recommendations separately would be helpful to us. 4110 03:50:11,600 --> 03:50:16,520 So I wanted to put that out there as a direction moving 4111 03:50:16,520 --> 03:50:16,920 forward. 4112 03:50:16,920 --> 03:50:17,640 So thank you very much. 4113 03:50:17,640 --> 03:50:18,520 >> DR. ELIZABETH VILLA: Thank you. 4114 03:50:18,520 --> 03:50:20,240 And thanks, everybody, for your input. 4115 03:50:20,240 --> 03:50:25,040 >> DR. BRUCE REED: And Mark. 4116 03:50:25,040 --> 03:50:28,800 >> DR. MARK PEIFER: How would you measure the delta? 4117 03:50:28,800 --> 03:50:36,800 Having reviewed and advised folks for the HHMI Gilliam 4118 03:50:36,800 --> 03:50:43,920 program, they have developed ways to measure that. 4119 03:50:43,920 --> 03:50:46,960 The way they set up their applications involves people 4120 03:50:46,960 --> 03:50:50,000 telling their story and that gives you an opportunity 4121 03:50:50,000 --> 03:50:53,040 to see already the delta that somebody has come 4122 03:50:53,040 --> 03:50:56,760 through, where have they come from to get to where they 4123 03:50:56,760 --> 03:50:57,440 are today? 4124 03:50:57,440 --> 03:51:01,720 And that kind of information gave me, when I was 4125 03:51:01,720 --> 03:51:04,320 reviewing those applications, some confidence in 4126 03:51:04,320 --> 03:51:08,200 predicting if I give this person an opportunity, where 4127 03:51:08,200 --> 03:51:11,200 are they going to go with it? 4128 03:51:11,200 --> 03:51:15,760 That kind of resilience and ability to succeed, I 4129 03:51:15,760 --> 03:51:21,320 think, is a scorable criterion. And I do have to 4130 03:51:21,320 --> 03:51:24,360 pushback on something I just heard. 4131 03:51:24,360 --> 03:51:27,640 We just came back to the word meritocracy. 4132 03:51:27,640 --> 03:51:32,040 I'm sorry, but our business is not a meritocracy. We 4133 03:51:32,040 --> 03:51:33,600 can pretend it is. 4134 03:51:33,600 --> 03:51:37,440 We can pretend everyone has an equal opportunity, but 4135 03:51:37,440 --> 03:51:42,160 we just saw the data, you know? 15 institutions with 4136 03:51:42,160 --> 03:51:46,440 30% of all the applications and an even larger for 4137 03:51:46,440 --> 03:51:48,560 action of the successful applications. 4138 03:51:48,560 --> 03:51:53,920 If we try to convince ourselves that by changing the 4139 03:51:53,920 --> 03:51:58,760 criteria we're going to damage the meritocracy, well, I 4140 03:51:58,760 --> 03:52:00,360 find that disturbing. 4141 03:52:00,360 --> 03:52:05,680 I'm sorry, but I just had to say that. 4142 03:52:05,680 --> 03:52:09,360 >> DR. BRUCE REED: Jinming. 4143 03:52:09,360 --> 03:52:14,040 >> DR. JINMING GAO: If I may quickly respond, I feel 4144 03:52:14,040 --> 03:52:17,520 we're trying to correct the bias towards the big 4145 03:52:17,520 --> 03:52:17,920 institutions. 4146 03:52:17,920 --> 03:52:20,760 I think giving people the free right. 4147 03:52:20,760 --> 03:52:25,720 But I think we do have the set up the metrics, though, 4148 03:52:25,720 --> 03:52:29,440 to really not penalize, I feel, the candidates because 4149 03:52:29,440 --> 03:52:31,920 they get to the bigger institutions. 4150 03:52:31,920 --> 03:52:32,720 >> DR. ELIZABETH VILLA: Yeah. 4151 03:52:32,720 --> 03:52:37,880 But I specifically said that, right? Anybody equally 4152 03:52:37,880 --> 03:52:43,600 can just -- you know, there is no penalization, there is 4153 03:52:43,600 --> 03:52:46,440 no punishing for being successful. 4154 03:52:46,760 --> 03:52:51,480 >> DR. JINMING GAO: No, just the comments from Mark is 4155 03:52:51,480 --> 03:52:53,800 where I guess I'm coming from. 4156 03:52:53,800 --> 03:52:56,560 >> DR. BRUCE REED: Yeah. 4157 03:52:56,560 --> 03:53:00,240 I want to echo what Elizabeth said, which is that, 4158 03:53:00,600 --> 03:53:04,320 you know, I think the fundamental focus of the working 4159 03:53:04,320 --> 03:53:07,640 group was on identifying strong candidates, how do you 4160 03:53:07,640 --> 03:53:08,360 do this? 4161 03:53:08,360 --> 03:53:10,120 How do you do it fairly? 4162 03:53:10,120 --> 03:53:14,920 How do you really give opportunity across the board and 4163 03:53:14,920 --> 03:53:20,680 open it up to people who are in places where there is 4164 03:53:20,680 --> 03:53:24,040 less cumulative knowledge, where there are less 4165 03:53:24,040 --> 03:53:26,920 resources, where there's less experience, but 4166 03:53:26,920 --> 03:53:29,800 nonetheless there may be fabulous potential. 4167 03:53:29,800 --> 03:53:37,600 And that has been what the group is struggling with. 4168 03:53:37,600 --> 03:53:41,440 Okay. Thank you, everybody. 4169 03:53:41,440 --> 03:53:44,640 It's 5:05. 4170 03:53:44,640 --> 03:53:47,840 This has been a really -- a very interesting 4171 03:53:47,840 --> 03:53:51,120 discussion, an interesting meeting. Thank you all. 4172 03:53:51,120 --> 03:53:55,840 I'll just say on behalf of the working group, if you 4173 03:53:55,840 --> 03:54:00,120 have other ideas, feel free to shoot me or Elizabeth, 4174 03:54:00,120 --> 03:54:02,680 anybody in the group an email. 4175 03:54:02,680 --> 03:54:06,760 We really do want -- would like to continue the 4176 03:54:06,760 --> 03:54:11,360 discussion and, of course, we will be coming back to 4177 03:54:11,360 --> 03:54:16,120 council at another time with more refined set of ideas. 4178 03:54:16,120 --> 03:54:20,360 So council, I'll sorry I left you no time for open 4179 03:54:20,360 --> 03:54:20,720 discussion. 4180 03:54:20,720 --> 03:54:26,560 >> DR. NONI BYRNES: We've had lots of open discussions 4181 03:54:26,560 --> 03:54:27,080 across. 4182 03:54:27,080 --> 03:54:30,520 >> DR. BRUCE REED: So thanks for everything. 4183 03:54:30,520 --> 03:54:34,800 And with that, we will adjourn for today. 4184 03:54:34,800 --> 03:54:37,680 >> DR. NONI BYRNES: Thank you, folks. 4185 03:54:37,680 --> 03:54:38,000 >> DR. ELIZABETH VILLA: Thanks, everybody. 4186 03:54:38,000 --> 00:00:00,000 >> Thank you.