1 00:00:12,412 --> 00:00:15,148 So, welcome to the third 2 00:00:15,148 --> 00:00:17,508 session of ethical and Regulatory 3 00:00:17,508 --> 00:00:19,653 Aspects of clinical Research. 4 00:00:20,220 --> 00:00:22,089 I'm Holly. 5 00:00:22,089 --> 00:00:25,092 You might recognize me from the first two sessions. 6 00:00:25,592 --> 00:00:29,853 Today I'm going to start with a relatively brief lecture on 7 00:00:29,853 --> 00:00:30,864 study design. 8 00:00:30,864 --> 00:00:33,658 And then Dave Wendler from our department 9 00:00:33,658 --> 00:00:35,702 is going to join us in person 10 00:00:35,702 --> 00:00:39,358 and then see who's at, northwestern is going to join 11 00:00:39,358 --> 00:00:39,640 us. 12 00:00:40,507 --> 00:00:43,510 By zoom, you'll recognize my disclaimer. 13 00:00:43,844 --> 00:00:47,414 The bottom line, here is that good study design 14 00:00:47,414 --> 00:00:51,184 is key to conducting ethical research. 15 00:00:52,119 --> 00:00:53,487 This is from. Oops. 16 00:00:53,487 --> 00:00:53,954 Yeah. 17 00:00:53,954 --> 00:00:56,853 Document from now almost, you know, quite a long 18 00:00:56,853 --> 00:00:57,457 time ago. 19 00:00:57,457 --> 00:00:59,993 But it remains true. 20 00:00:59,993 --> 00:01:03,797 Researchers us have a fundamental obligation 21 00:01:03,797 --> 00:01:06,425 to plan, design and conduct studies 22 00:01:06,425 --> 00:01:09,503 with honesty, truthfulness and integrity 23 00:01:09,903 --> 00:01:12,296 values demonstrated by how researchers 24 00:01:12,296 --> 00:01:14,942 observe, record and interpret their work. 25 00:01:16,610 --> 00:01:19,546 So I'm going to, spend a little time 26 00:01:19,546 --> 00:01:21,748 on the first two of these circled 27 00:01:21,748 --> 00:01:23,617 ethical requirements today. 28 00:01:24,151 --> 00:01:26,464 We already covered independent review, 29 00:01:26,464 --> 00:01:28,655 but that's a sort of key component, 30 00:01:29,122 --> 00:01:32,125 of the review of the study design. 31 00:01:32,492 --> 00:01:35,862 So I'm just going to give us, some basics. 32 00:01:35,862 --> 00:01:38,432 So we're really all on the same page. 33 00:01:38,432 --> 00:01:43,537 So there are basically two types of research. 34 00:01:43,537 --> 00:01:45,608 And this is a little oversimplifying, 35 00:01:45,608 --> 00:01:46,840 but you'll you'll see 36 00:01:46,840 --> 00:01:48,608 why I'm making these distinctions. 37 00:01:48,608 --> 00:01:52,012 So there's observational research which is 38 00:01:52,746 --> 00:01:55,983 merely though importantly descriptive right. 39 00:01:55,983 --> 00:01:58,952 You are observing something and collecting data. 40 00:01:59,219 --> 00:02:01,488 You can do that retrospectively. 41 00:02:01,488 --> 00:02:04,372 Looking backwards or prospectively looking 42 00:02:04,372 --> 00:02:05,058 forwards. 43 00:02:05,292 --> 00:02:07,070 Retrospectively, it may be that you're 44 00:02:07,070 --> 00:02:08,895 going to go and look at medical charts 45 00:02:08,895 --> 00:02:11,322 and abstract information from the past 46 00:02:11,322 --> 00:02:14,067 five years of medical charts, for example. 47 00:02:14,701 --> 00:02:17,504 Experimental research includes 48 00:02:17,504 --> 00:02:21,143 some sort of intervention, and it's almost always 49 00:02:21,143 --> 00:02:22,109 perspective. 50 00:02:22,109 --> 00:02:25,180 It would be pretty hard to do a retrospective 51 00:02:25,180 --> 00:02:26,613 interventional trial 52 00:02:28,482 --> 00:02:31,551 now with in that experimental design. 53 00:02:31,551 --> 00:02:34,087 There are a couple of things that often happen. 54 00:02:34,087 --> 00:02:37,157 First is there's some sort of manipulate 55 00:02:37,858 --> 00:02:40,560 changing artificially, right. 56 00:02:40,560 --> 00:02:43,293 We're not just observing or manipulating 57 00:02:43,293 --> 00:02:44,865 at least one variable. 58 00:02:45,365 --> 00:02:48,368 Sometimes there is control rule, 59 00:02:48,368 --> 00:02:51,680 and you do that in order to reduce the 60 00:02:51,680 --> 00:02:52,639 likelihood 61 00:02:52,639 --> 00:02:55,709 that external factors will influence the study 62 00:02:55,709 --> 00:02:56,309 outcome. 63 00:02:56,676 --> 00:03:00,881 And then randomization to hopefully ensure 64 00:03:00,881 --> 00:03:04,051 that you have an unbiased selection 65 00:03:04,051 --> 00:03:06,453 among those who get the intervention 66 00:03:06,453 --> 00:03:07,854 and those who don't. 67 00:03:08,688 --> 00:03:11,313 Now there's a sometimes it's impossible 68 00:03:11,313 --> 00:03:12,726 to randomize, right? 69 00:03:12,726 --> 00:03:15,762 You may want to compare two interventions, but 70 00:03:16,063 --> 00:03:19,066 one intervention, requires 71 00:03:19,599 --> 00:03:22,769 an injection, the other requires a pill. 72 00:03:22,769 --> 00:03:25,772 You can't really. I'm sorry. 73 00:03:25,972 --> 00:03:27,607 You can't. 74 00:03:27,607 --> 00:03:31,344 Really randomize in a way that is meaningful to the person 75 00:03:31,344 --> 00:03:34,514 because they're going to know which one they get. 76 00:03:34,514 --> 00:03:38,385 You could randomize the, assignment, 77 00:03:38,385 --> 00:03:40,538 but you can't blind the individual 78 00:03:40,538 --> 00:03:42,122 to what they're getting. 79 00:03:42,756 --> 00:03:45,331 And so a quasi experimental design 80 00:03:45,331 --> 00:03:47,527 is when there's manipulation 81 00:03:47,527 --> 00:03:50,630 and some sort of control, but no randomization. 82 00:03:52,299 --> 00:03:53,767 So the first question we're 83 00:03:53,767 --> 00:03:57,950 going to ask about any study is whether or not it has social 84 00:03:57,950 --> 00:03:58,438 value. 85 00:03:58,438 --> 00:04:00,640 And Dave Wendler, who's going to follow 86 00:04:00,640 --> 00:04:04,611 me is going to talk a bit about social value. 87 00:04:04,611 --> 00:04:07,747 And so I'm not going to say much about it now. 88 00:04:07,747 --> 00:04:10,951 But for the sake of argument, we're going to say 89 00:04:10,951 --> 00:04:12,786 that a project shouldn't go forward 90 00:04:12,786 --> 00:04:14,621 unless it has social value, right? 91 00:04:14,621 --> 00:04:17,847 That there's some benefit, direct benefit 92 00:04:17,847 --> 00:04:19,893 to patients or some value 93 00:04:19,893 --> 00:04:23,001 to future patients for gaining knowledge 94 00:04:23,001 --> 00:04:25,332 that will lead you to benefit 95 00:04:25,332 --> 00:04:28,935 for future patients and or for, individual patients. 96 00:04:29,503 --> 00:04:32,606 And then scientific validity is a component of this. 97 00:04:32,606 --> 00:04:35,089 And an important question from the study design 98 00:04:35,089 --> 00:04:35,775 perspective, 99 00:04:35,775 --> 00:04:38,778 is this study design valid and, 100 00:04:39,646 --> 00:04:41,821 easy way to think about this concept 101 00:04:41,821 --> 00:04:44,117 is to think about sample size, right? 102 00:04:45,252 --> 00:04:46,353 When you're designing 103 00:04:46,353 --> 00:04:49,078 your study, you need to have enough 104 00:04:49,078 --> 00:04:50,090 participants 105 00:04:50,090 --> 00:04:52,410 in order to answer your scientific 106 00:04:52,410 --> 00:04:53,093 question. 107 00:04:53,426 --> 00:04:57,130 If you don't have enough, you can't answer your question. 108 00:04:57,130 --> 00:05:02,482 And you have, let's say you you enroll 100, you really need 109 00:05:02,482 --> 00:05:02,936 200. 110 00:05:03,236 --> 00:05:06,273 You have wasted the resources 111 00:05:06,273 --> 00:05:09,943 that it took you to do that half of the trial. 112 00:05:10,844 --> 00:05:13,880 And you're not going to get, an answer. 113 00:05:13,880 --> 00:05:15,845 So that would be a bad study design 114 00:05:15,845 --> 00:05:17,417 because you have the sample 115 00:05:17,417 --> 00:05:20,420 size is too small or it's too large. 116 00:05:20,420 --> 00:05:21,254 Right. 117 00:05:21,254 --> 00:05:24,624 Let's imagine that you only need 100 people to be exposed 118 00:05:24,624 --> 00:05:28,261 to the intervention or intervention and control. 119 00:05:28,528 --> 00:05:31,631 But you and you know, you've enrolled 200. 120 00:05:31,631 --> 00:05:35,702 That means you've exposed 100 people to more risk 121 00:05:35,702 --> 00:05:39,739 than they needed to be, and also wasted some resources. 122 00:05:41,274 --> 00:05:42,375 Now, an important 123 00:05:42,375 --> 00:05:46,112 question about study design and scientific validity is 124 00:05:46,546 --> 00:05:49,250 is there uncertainty about the potential 125 00:05:49,250 --> 00:05:51,818 benefit of the proposed intervention? 126 00:05:52,352 --> 00:05:57,057 If there is no uncertainty, then we can't go forward, right? 127 00:05:57,057 --> 00:06:00,927 If we already know the answer, we can't ethically 128 00:06:00,927 --> 00:06:03,930 or, you know, scientifically move forward. 129 00:06:04,431 --> 00:06:08,034 If yes, if there is uncertainty, then 130 00:06:08,034 --> 00:06:11,571 a randomized control trial might be acceptable. 131 00:06:11,972 --> 00:06:14,365 And a follow up question would be 132 00:06:14,365 --> 00:06:17,410 is randomization to a placebo acceptable? 133 00:06:18,245 --> 00:06:21,310 So when we talk about a randomized control 134 00:06:21,310 --> 00:06:21,748 trial 135 00:06:22,148 --> 00:06:25,819 there is the there is uncertainty right. 136 00:06:25,819 --> 00:06:27,487 There's an indifference 137 00:06:27,487 --> 00:06:31,224 among a community of clinicians about which treatment is 138 00:06:31,224 --> 00:06:31,891 superior. 139 00:06:32,192 --> 00:06:34,416 This is often referred to as equipoise 140 00:06:34,416 --> 00:06:35,528 in the literature. 141 00:06:36,396 --> 00:06:39,633 But it really means indifference 142 00:06:39,633 --> 00:06:42,969 among usually a community of experts. Right. 143 00:06:43,270 --> 00:06:46,770 If you're randomly if you're if you're doing a study about 144 00:06:46,770 --> 00:06:47,374 diabetes, 145 00:06:47,741 --> 00:06:50,076 it would be really important for those physicians 146 00:06:50,076 --> 00:06:53,079 who are experts in diabetes or providing care and diabetes. 147 00:06:53,380 --> 00:06:56,688 There's indifference among them, not among all 148 00:06:56,688 --> 00:06:57,550 clinicians. 149 00:06:57,751 --> 00:06:58,518 For example, 150 00:07:00,020 --> 00:07:00,787 as I mentioned 151 00:07:00,787 --> 00:07:03,877 before, there's this concept of blinding 152 00:07:03,877 --> 00:07:04,958 where there's 153 00:07:04,958 --> 00:07:07,345 you may want to blind the individuals 154 00:07:07,345 --> 00:07:08,828 from their assignment. 155 00:07:09,062 --> 00:07:12,732 You may also want to, blind the investigators 156 00:07:12,932 --> 00:07:15,935 so you can do a single blind or a double blind 157 00:07:16,670 --> 00:07:20,173 where the investigator and the subjects are blinded. 158 00:07:21,508 --> 00:07:23,310 So what about whether 159 00:07:23,310 --> 00:07:26,313 we can use a placebo or an active control? 160 00:07:27,647 --> 00:07:30,650 So, just as a reminder, 161 00:07:30,884 --> 00:07:33,753 a placebo, right, is generally. 162 00:07:33,753 --> 00:07:36,790 Let's take the classic example of a pill. 163 00:07:36,790 --> 00:07:39,025 It looks it's a, it's a pill that looks 164 00:07:39,025 --> 00:07:41,478 just like the drug but doesn't contain any 165 00:07:41,478 --> 00:07:42,062 medicine. 166 00:07:42,062 --> 00:07:45,441 And, you know, we do this and it's important that we share 167 00:07:45,441 --> 00:07:45,732 this 168 00:07:45,732 --> 00:07:47,434 with potential participants. 169 00:07:47,434 --> 00:07:51,638 We do this in order to find out whether the real medicine, 170 00:07:51,638 --> 00:07:54,574 in this case, the drug works like we hope it 171 00:07:54,574 --> 00:07:54,841 to. 172 00:07:55,141 --> 00:07:58,445 And we need to compare people who get the medicine 173 00:07:58,712 --> 00:08:01,715 to those who don't to see if the medicine actually works. 174 00:08:02,315 --> 00:08:06,653 And so the next question is when is it ethically appropriate 175 00:08:06,653 --> 00:08:08,621 to include a placebo? 176 00:08:08,621 --> 00:08:12,826 And the bottom line is that, or the most common reason 177 00:08:12,826 --> 00:08:15,642 to use a placebo is when there is no good 178 00:08:15,642 --> 00:08:17,497 standard treatment, right? 179 00:08:17,497 --> 00:08:21,668 So imagine it's 1988. 180 00:08:22,135 --> 00:08:25,105 We know that HIV causes Aids. 181 00:08:26,072 --> 00:08:27,841 But we have no treatment. 182 00:08:27,841 --> 00:08:32,268 So the first trials of AZT, the first antiretroviral that 183 00:08:32,268 --> 00:08:32,579 was 184 00:08:33,079 --> 00:08:37,617 tested, was tested in a trial where half of the participants 185 00:08:37,617 --> 00:08:41,554 got a placebo, the other half got AZT, 186 00:08:41,921 --> 00:08:44,652 and that was ethically appropriate 187 00:08:44,652 --> 00:08:46,659 or ethically, acceptable 188 00:08:46,659 --> 00:08:50,397 because there was no good standard treatment. 189 00:08:50,830 --> 00:08:54,134 When a standard treatment exists, 190 00:08:54,134 --> 00:08:56,536 there is an expectation 191 00:08:56,536 --> 00:09:01,147 that the study will be have an active control, meaning 192 00:09:01,147 --> 00:09:01,574 that 193 00:09:01,875 --> 00:09:04,252 the old medicine is going to be tested 194 00:09:04,252 --> 00:09:05,879 against the new medicine. 195 00:09:07,046 --> 00:09:08,882 There are a couple of other, 196 00:09:08,882 --> 00:09:11,993 examples that when a standard treatment 197 00:09:11,993 --> 00:09:12,552 exist, 198 00:09:12,552 --> 00:09:16,189 it is ethically acceptable to use a placebo. 199 00:09:16,189 --> 00:09:21,073 So one is maybe obvious when the placebos are additive, 200 00:09:21,073 --> 00:09:21,694 right. 201 00:09:21,694 --> 00:09:24,115 So there's a standard of care it's 202 00:09:24,115 --> 00:09:24,898 acceptable 203 00:09:24,898 --> 00:09:28,635 to randomize individuals to the standard of care. 204 00:09:28,635 --> 00:09:32,739 Plus the new thing and standard of care. 205 00:09:32,739 --> 00:09:35,175 Right. It's it's merely an a placebo. 206 00:09:35,175 --> 00:09:36,409 It's merely additive. 207 00:09:37,710 --> 00:09:40,680 Another example when the medical question, 208 00:09:40,680 --> 00:09:43,016 medical condition in question is not serious. 209 00:09:43,016 --> 00:09:46,781 So, let's say a drug company wants to develop a new pain 210 00:09:46,781 --> 00:09:47,454 reliever, 211 00:09:47,454 --> 00:09:50,694 and they want to enroll people who are likely to have a 212 00:09:50,694 --> 00:09:51,224 headache 213 00:09:51,224 --> 00:09:53,226 within the next month. 214 00:09:53,226 --> 00:09:55,228 Just making this up. 215 00:09:55,228 --> 00:09:57,249 You know, for we're talking about 216 00:09:57,249 --> 00:09:59,699 a regular old headache, not a migraine. 217 00:10:00,033 --> 00:10:03,002 It would be acceptable to ask people 218 00:10:03,002 --> 00:10:05,656 to be willing to not take a pain reliever 219 00:10:05,656 --> 00:10:07,273 for that period of time. 220 00:10:07,474 --> 00:10:09,776 So it would be okay to randomize. 221 00:10:09,776 --> 00:10:12,501 And then another example is when being off the standard 222 00:10:12,501 --> 00:10:13,046 medication 223 00:10:13,046 --> 00:10:16,124 for a length of time won't cause serious, 224 00:10:16,124 --> 00:10:17,550 irreversible harm. 225 00:10:17,550 --> 00:10:20,553 So, this is. 226 00:10:22,989 --> 00:10:24,891 You have to be careful, right? 227 00:10:24,891 --> 00:10:26,917 Because there are some medications 228 00:10:26,917 --> 00:10:28,228 that people take that 229 00:10:28,228 --> 00:10:31,291 if you take them off of them, they could experience 230 00:10:31,291 --> 00:10:32,432 irreversible harm. 231 00:10:32,732 --> 00:10:35,374 But if we're talking about a medication 232 00:10:35,374 --> 00:10:37,203 that won't result in that, 233 00:10:37,403 --> 00:10:39,724 it might be okay to ask participants 234 00:10:39,724 --> 00:10:42,108 to stop taking the drug that they're 235 00:10:42,108 --> 00:10:45,311 taking, be randomized to a new drug 236 00:10:45,311 --> 00:10:47,953 versus a placebo and be followed over 237 00:10:47,953 --> 00:10:48,381 time. 238 00:10:48,381 --> 00:10:50,691 But this right, this will all depend 239 00:10:50,691 --> 00:10:53,386 on the context, the justifications, etc.. 240 00:10:53,853 --> 00:10:57,290 Oh, so this is another interesting thing. 241 00:10:57,724 --> 00:10:59,292 Piece of placebo, right? 242 00:11:00,493 --> 00:11:03,363 Why not just have a no intervention arm. 243 00:11:03,363 --> 00:11:06,132 Right. So one example. 244 00:11:06,132 --> 00:11:09,135 So let's say you're trying a new, treatment. 245 00:11:09,769 --> 00:11:11,804 Why not just have people 246 00:11:11,804 --> 00:11:14,876 get no intervention and the intervention 247 00:11:14,876 --> 00:11:17,410 and meaning that half the people 248 00:11:17,410 --> 00:11:20,118 would get the pain patch, the other half 249 00:11:20,118 --> 00:11:22,081 wouldn't get the pain patch. 250 00:11:22,649 --> 00:11:28,154 So the concern here is that if you just give half the people 251 00:11:28,154 --> 00:11:30,529 the pain patch and the other group 252 00:11:30,529 --> 00:11:31,157 nothing. 253 00:11:31,958 --> 00:11:34,178 When the outcome can be subjective, 254 00:11:34,178 --> 00:11:36,462 like pain or emotional or endurance 255 00:11:36,462 --> 00:11:39,371 or something like that, the individuals 256 00:11:39,371 --> 00:11:41,534 who get the pain patch might 257 00:11:42,902 --> 00:11:44,871 report information in a way 258 00:11:44,871 --> 00:11:46,968 that is biased by the fact that they have 259 00:11:46,968 --> 00:11:48,708 the pain patch on, or vice versa. 260 00:11:48,708 --> 00:11:50,737 So you'd want to have individuals 261 00:11:50,737 --> 00:11:53,012 have a pain patch that has an active 262 00:11:53,012 --> 00:11:55,434 ingredient, and a pain patch, or a patch 263 00:11:55,434 --> 00:11:57,917 that doesn't have the active ingredient. 264 00:11:59,085 --> 00:12:00,753 Another reason why you, 265 00:12:00,753 --> 00:12:04,157 you want to have something in terms of, 266 00:12:04,524 --> 00:12:06,617 a comparison is when risk behaviors 267 00:12:06,617 --> 00:12:09,128 might be affected by thinking one receive 268 00:12:09,128 --> 00:12:12,899 the experimental treatment so we can think back to Kovit, 269 00:12:12,899 --> 00:12:16,196 for example, that if you were in a, in a 270 00:12:16,196 --> 00:12:17,103 trial and, 271 00:12:17,804 --> 00:12:20,740 you knew that because you got a shot, 272 00:12:20,740 --> 00:12:22,307 you got the vaccine, if you didn't 273 00:12:22,307 --> 00:12:24,243 get the shot, you didn't get the vaccine. 274 00:12:24,644 --> 00:12:27,382 One possibility is that if you didn't get the 275 00:12:27,382 --> 00:12:27,747 shot, 276 00:12:28,014 --> 00:12:31,384 you would be hyper vigilant about avoiding Covid. 277 00:12:31,651 --> 00:12:35,355 And when comparing you to those who got the vaccine, 278 00:12:35,355 --> 00:12:37,390 we might not be able to tell the difference. 279 00:12:37,390 --> 00:12:40,627 So it would be important for everybody to get the shot. 280 00:12:42,962 --> 00:12:46,196 So I want to transition a little and bring your attention to a 281 00:12:46,196 --> 00:12:46,666 document 282 00:12:46,666 --> 00:12:48,789 that was just published last month, 283 00:12:48,789 --> 00:12:50,670 the World Health Organization. 284 00:12:51,137 --> 00:12:54,321 Published guidance for best practices in clinical 285 00:12:54,321 --> 00:12:54,841 trials. 286 00:12:55,141 --> 00:12:58,311 I have the link for those of you who are 287 00:12:58,845 --> 00:13:01,423 actively engaged in or interested in clinical 288 00:13:01,423 --> 00:13:01,881 trials. 289 00:13:02,148 --> 00:13:05,100 This is a really nice document in hitting 290 00:13:05,100 --> 00:13:07,620 some of the sort of most important 291 00:13:07,620 --> 00:13:11,757 key aspects of thinking about clinical research and or 292 00:13:11,757 --> 00:13:13,059 clinical trials. 293 00:13:13,059 --> 00:13:16,049 And I've included this little, diagram 294 00:13:16,049 --> 00:13:18,331 because they're also talking 295 00:13:18,331 --> 00:13:23,709 about the research ecosystem and the important parts of that 296 00:13:23,709 --> 00:13:24,337 system 297 00:13:24,337 --> 00:13:27,573 that support all the research that's being done. 298 00:13:27,573 --> 00:13:31,084 So I recommend that those of you who are interested, take a look 299 00:13:31,084 --> 00:13:31,577 at that. 300 00:13:31,944 --> 00:13:33,846 And I just pulled this one, 301 00:13:34,847 --> 00:13:35,782 piece of the 302 00:13:35,782 --> 00:13:39,919 puzzle out of the document where they talk about 303 00:13:40,119 --> 00:13:43,462 the key scientific and ethical considerations for clinical 304 00:13:43,462 --> 00:13:43,923 trials. 305 00:13:44,357 --> 00:13:47,326 And you won't be surprised 306 00:13:47,326 --> 00:13:51,545 by any of these, pronouncements that it should be designed to 307 00:13:51,545 --> 00:13:52,098 produce 308 00:13:52,098 --> 00:13:54,399 scientifically sound answers, respects 309 00:13:54,399 --> 00:13:57,003 the rights and well-being of participants. 310 00:13:57,437 --> 00:14:01,074 And then I, I think these three are sort of, 311 00:14:01,607 --> 00:14:04,899 not new, but I like that they put these 312 00:14:04,899 --> 00:14:05,912 in the list 313 00:14:06,245 --> 00:14:08,938 that the research needs to be collaborative and 314 00:14:08,938 --> 00:14:09,682 transparent, 315 00:14:10,116 --> 00:14:13,286 be designed to be feasible for their context, 316 00:14:13,619 --> 00:14:16,989 and manage trial quality effectively and efficiently. 317 00:14:16,989 --> 00:14:20,193 So the last three are ethical, 318 00:14:20,727 --> 00:14:23,730 but also important practically 319 00:14:24,297 --> 00:14:27,033 when thinking about design. 320 00:14:27,033 --> 00:14:30,136 And then, another article that I just wanted 321 00:14:30,136 --> 00:14:32,673 to bring your attention to or another 322 00:14:32,673 --> 00:14:35,141 set of principles is, again, things 323 00:14:35,141 --> 00:14:37,562 you wouldn't be surprised to hear, 324 00:14:37,562 --> 00:14:39,912 but they add the last item here. 325 00:14:39,912 --> 00:14:42,448 Study findings must be reported completely 326 00:14:42,448 --> 00:14:46,052 and promptly during the trial and after the trial. 327 00:14:46,385 --> 00:14:49,092 And I would say in the last ten years 328 00:14:49,092 --> 00:14:52,091 or so, there's been more, maybe even 20. 329 00:14:52,091 --> 00:14:55,027 There's been more emphasis on making sure that study 330 00:14:55,027 --> 00:14:57,923 findings go back to the study population 331 00:14:57,923 --> 00:14:59,298 that participated. 332 00:14:59,298 --> 00:15:03,720 So back in the day, the expectation was you did your 333 00:15:03,720 --> 00:15:04,570 research, 334 00:15:04,570 --> 00:15:07,495 you did your clinical trials, and then you published it and 335 00:15:07,495 --> 00:15:08,040 that's it. 336 00:15:08,274 --> 00:15:10,943 You have conveyed your findings. 337 00:15:10,943 --> 00:15:13,456 You've conveyed them clearly in this case, 338 00:15:13,456 --> 00:15:15,848 right, to the scientific community. But 339 00:15:17,283 --> 00:15:19,952 maybe you also have an obligation 340 00:15:19,952 --> 00:15:23,223 to share those findings with those who participated in 341 00:15:23,223 --> 00:15:23,890 the trial. 342 00:15:24,123 --> 00:15:27,789 And it doesn't mean you give them like here are your 343 00:15:27,789 --> 00:15:28,494 findings, 344 00:15:28,728 --> 00:15:31,453 but provide them with the aggregate 345 00:15:31,453 --> 00:15:32,231 findings. 346 00:15:32,231 --> 00:15:35,189 The primary findings in and lay language 347 00:15:35,189 --> 00:15:37,703 in a way that they can understand 348 00:15:38,070 --> 00:15:41,073 that it's important for 349 00:15:42,041 --> 00:15:44,210 individuals to appreciate that 350 00:15:44,210 --> 00:15:48,147 their contribution led to an outcome. 351 00:15:48,347 --> 00:15:50,978 And I would say that's true whether it's positive or 352 00:15:50,978 --> 00:15:51,484 negative, 353 00:15:51,484 --> 00:15:54,620 that their contribution 354 00:15:55,822 --> 00:15:58,291 led to something that will advance 355 00:15:58,291 --> 00:16:01,238 science again, regardless of whether it's positive or 356 00:16:01,238 --> 00:16:01,794 negative. 357 00:16:01,794 --> 00:16:03,663 In terms of whether they answered, 358 00:16:03,663 --> 00:16:05,645 we're able to answer the question 359 00:16:05,645 --> 00:16:06,666 they set out to. 360 00:16:07,733 --> 00:16:12,705 So a couple of more comments about really about terminology. 361 00:16:12,705 --> 00:16:15,708 When we think about trials and you, 362 00:16:15,975 --> 00:16:18,572 I'm sure that we have all mentioned these, 363 00:16:18,572 --> 00:16:20,613 in the first couple of sessions. 364 00:16:20,880 --> 00:16:23,583 But just want to clarify. So there are 365 00:16:24,784 --> 00:16:26,419 again, 366 00:16:26,419 --> 00:16:29,546 probably I think it was in the 1970s, 367 00:16:29,546 --> 00:16:30,223 the FDA 368 00:16:30,823 --> 00:16:34,293 formally defined trials 369 00:16:34,293 --> 00:16:37,223 as phase one, phase two, phase three, and phase 370 00:16:37,223 --> 00:16:37,597 four. 371 00:16:37,997 --> 00:16:41,572 And this was sort of, I know, doctor letter presented, 372 00:16:41,572 --> 00:16:42,235 you know, 373 00:16:42,235 --> 00:16:44,711 sort of the early history of the FDA 374 00:16:44,711 --> 00:16:46,706 and how they came to develop 375 00:16:47,340 --> 00:16:50,676 even guidelines about how you can prepare 376 00:16:50,676 --> 00:16:53,679 a particular, 377 00:16:54,146 --> 00:16:56,582 solution, how it can be labeled, 378 00:16:56,582 --> 00:16:58,374 whether you've found it to be safe, 379 00:16:58,374 --> 00:17:00,319 whether you found it to be effective. 380 00:17:01,921 --> 00:17:05,892 In all of that momentum, there was also, an interest 381 00:17:05,892 --> 00:17:08,748 in defining these trials, specifically 382 00:17:08,748 --> 00:17:11,831 as investigators, pharma companies, etc. 383 00:17:11,831 --> 00:17:14,800 moved forward in their trial development. 384 00:17:14,800 --> 00:17:17,117 It was important for there to be, sort 385 00:17:17,117 --> 00:17:19,739 of expectation of how things move forward. 386 00:17:20,106 --> 00:17:23,109 So when we talk about phase one studies, 387 00:17:23,643 --> 00:17:25,755 those are the ones that are first 388 00:17:25,755 --> 00:17:26,779 in human right. 389 00:17:26,779 --> 00:17:30,021 There's in most cases there's some sort of animal 390 00:17:30,021 --> 00:17:30,616 research 391 00:17:30,950 --> 00:17:33,859 and you now are going to see what happens 392 00:17:33,859 --> 00:17:35,988 when you put it into a human. 393 00:17:36,355 --> 00:17:37,690 And generally, 394 00:17:37,690 --> 00:17:38,224 we're going to talk 395 00:17:38,224 --> 00:17:41,047 a little more in detail, but generally those are for 396 00:17:41,047 --> 00:17:41,861 safety, right? 397 00:17:41,861 --> 00:17:44,864 Is it okay for a human to receive it? 398 00:17:45,064 --> 00:17:47,833 And a relatively small number of people 399 00:17:47,833 --> 00:17:50,770 from those 100 trials, about 70 400 00:17:50,770 --> 00:17:55,308 go to phase two about, you know, a third of those go 401 00:17:55,308 --> 00:17:59,347 to phase three and 6 or 7, make it to phase four or to 402 00:17:59,347 --> 00:17:59,946 market. 403 00:18:00,179 --> 00:18:02,286 So again, just to give you a sense 404 00:18:02,286 --> 00:18:04,083 of how much research happens 405 00:18:04,083 --> 00:18:07,086 at one end to get relatively few 406 00:18:07,753 --> 00:18:10,189 effective outcomes in the end. 407 00:18:10,189 --> 00:18:13,859 And, just again, to give you a little context, 408 00:18:13,859 --> 00:18:16,829 the NIH Clinical Center is unique 409 00:18:16,829 --> 00:18:20,673 in the sense that 95% of the trials conducted 410 00:18:20,673 --> 00:18:21,100 here 411 00:18:21,334 --> 00:18:25,237 are early phase trials, phase one and phase two. 412 00:18:26,238 --> 00:18:27,640 That's not the 413 00:18:27,640 --> 00:18:30,676 case in most academic medical centers. 414 00:18:31,510 --> 00:18:35,548 Just to say that we are at the sort of exploratory 415 00:18:35,548 --> 00:18:38,551 end of the pipeline 416 00:18:39,251 --> 00:18:41,765 in 2023, which is the most recent data 417 00:18:41,765 --> 00:18:43,022 that I could find. 418 00:18:44,056 --> 00:18:45,948 You know, almost half were phase two, 419 00:18:45,948 --> 00:18:47,226 a little less than that. 420 00:18:47,226 --> 00:18:49,795 Phase one very few phase three. 421 00:18:49,795 --> 00:18:52,141 Phase three would be a large 422 00:18:52,141 --> 00:18:53,733 comparative trial. 423 00:18:54,900 --> 00:18:57,103 So let's talk a little more about each one. 424 00:18:57,103 --> 00:18:59,757 As I said, the phase one is usually 425 00:18:59,757 --> 00:19:02,108 a relatively small population. 426 00:19:02,108 --> 00:19:05,344 First in human safety, sometimes 427 00:19:05,344 --> 00:19:08,377 dosage for the maximum tolerated dose, 428 00:19:08,377 --> 00:19:09,415 meaning that 429 00:19:10,383 --> 00:19:13,318 you give the individuals in the trial 430 00:19:13,318 --> 00:19:13,953 perhaps 431 00:19:14,887 --> 00:19:17,957 larger and larger doses in order to find 432 00:19:17,957 --> 00:19:21,060 the one that is too much and back off one 433 00:19:21,260 --> 00:19:25,197 and you use that dose to go into phase two. 434 00:19:25,664 --> 00:19:29,201 So it's one lower than that maximum tolerated dose. 435 00:19:30,369 --> 00:19:31,737 In phase two, 436 00:19:31,737 --> 00:19:34,540 you start to look at the biologic effect 437 00:19:34,540 --> 00:19:35,941 and look at adverse 438 00:19:35,941 --> 00:19:39,845 events in in phase one they can be relatively short. 439 00:19:40,179 --> 00:19:42,592 Phase two, they might start to get 440 00:19:42,592 --> 00:19:45,217 a little longer in order to document 441 00:19:45,217 --> 00:19:47,926 some possible biologic effects in adverse 442 00:19:47,926 --> 00:19:48,454 events. 443 00:19:49,055 --> 00:19:51,733 Phase three we're now getting larger up 444 00:19:51,733 --> 00:19:53,793 to, you know, more than 3000. 445 00:19:53,793 --> 00:19:56,060 As you can recall from the Covid trials, 446 00:19:56,060 --> 00:19:58,497 tens of thousands of people in some cases, 447 00:19:59,698 --> 00:20:02,568 the intervention is the intervention 448 00:20:02,568 --> 00:20:05,728 that's going to be moved forward is based on the results in phase 449 00:20:05,728 --> 00:20:05,971 two. 450 00:20:06,272 --> 00:20:08,475 And this is where you start looking at 451 00:20:08,475 --> 00:20:10,910 effectiveness, continue to look at safety 452 00:20:11,143 --> 00:20:14,346 and then risk benefit in terms of adopting 453 00:20:14,346 --> 00:20:17,349 into clinical practice. 454 00:20:17,550 --> 00:20:18,984 Phase four are 455 00:20:18,984 --> 00:20:21,667 what are often referred to as post-marketing 456 00:20:21,667 --> 00:20:22,154 trials. 457 00:20:22,455 --> 00:20:26,659 And in those we're looking at long term safety. 458 00:20:27,693 --> 00:20:29,495 Not every 459 00:20:29,495 --> 00:20:32,380 intervention goes into a post-marketing 460 00:20:32,380 --> 00:20:32,898 trial. 461 00:20:34,366 --> 00:20:36,902 In those cases where there is maybe 462 00:20:36,902 --> 00:20:39,754 a higher expectation of risk or serious 463 00:20:39,754 --> 00:20:40,339 events, 464 00:20:40,339 --> 00:20:43,295 there's an interest in following the long term 465 00:20:43,295 --> 00:20:43,809 safety. 466 00:20:44,477 --> 00:20:48,614 So in terms of oversight, we spent, a whole lecture 467 00:20:48,614 --> 00:20:50,950 on institutional review boards. 468 00:20:50,950 --> 00:20:53,953 Those are the generally local groups 469 00:20:53,953 --> 00:20:59,058 affiliated with the institution that the investigator is at, 470 00:20:59,391 --> 00:21:02,417 and they're doing the initial review 471 00:21:02,417 --> 00:21:03,929 of their research 472 00:21:03,929 --> 00:21:07,290 to see whether or not it is appropriate to move 473 00:21:07,290 --> 00:21:07,933 forward. 474 00:21:08,567 --> 00:21:12,735 Another thing that is often in place is a data monitoring 475 00:21:12,735 --> 00:21:13,539 committee. 476 00:21:13,973 --> 00:21:18,644 So in general, a data monitoring committee there, 477 00:21:19,512 --> 00:21:22,515 there are at least two primary types. 478 00:21:22,915 --> 00:21:25,251 One type of data monitoring committee 479 00:21:25,251 --> 00:21:28,587 is assigned to a particular study. 480 00:21:28,888 --> 00:21:32,191 So again let's use Covid as an example. 481 00:21:32,191 --> 00:21:35,227 The Covid vaccine trials had 482 00:21:35,227 --> 00:21:38,497 each one had a data monitoring committee. 483 00:21:39,064 --> 00:21:42,635 The other type is a data monitoring committee 484 00:21:42,635 --> 00:21:46,527 that is a hub for a range of trials 485 00:21:46,527 --> 00:21:49,308 or a group of trials or, 486 00:21:50,409 --> 00:21:51,277 you know, 487 00:21:51,277 --> 00:21:54,079 the trials don't necessarily have to be related, 488 00:21:54,079 --> 00:21:56,146 but they're all looking at a particular 489 00:21:56,146 --> 00:21:57,683 condition and interventions. 490 00:21:57,983 --> 00:22:03,270 For example, I sat on the Nhlbi National Heart, lung and Blood 491 00:22:03,270 --> 00:22:04,123 Institute 492 00:22:04,590 --> 00:22:09,528 Pediatric Heart Network, Dsm-v, or DMC, 493 00:22:09,862 --> 00:22:12,698 and we reviewed all the trials 494 00:22:12,698 --> 00:22:16,502 that were conducted under that umbrella. 495 00:22:17,269 --> 00:22:19,905 So the the goal 496 00:22:19,905 --> 00:22:22,908 or the role of the data monitoring committee 497 00:22:23,709 --> 00:22:26,719 is to monitor the trial once it's in the 498 00:22:26,719 --> 00:22:27,246 field. 499 00:22:27,580 --> 00:22:30,583 So if you remember, the IRB 500 00:22:30,783 --> 00:22:34,854 does their review and approves or not the protocol, 501 00:22:35,521 --> 00:22:38,039 they generally don't have much to do 502 00:22:38,039 --> 00:22:39,158 with the trial. 503 00:22:39,158 --> 00:22:42,033 Once it's launched, the investigators 504 00:22:42,033 --> 00:22:45,064 are asked to provide an annual review, 505 00:22:45,431 --> 00:22:48,434 but the IRB is not looking 506 00:22:48,434 --> 00:22:51,604 at the data coming out from the trial, 507 00:22:51,604 --> 00:22:55,841 except for adverse events, which they may be reported to. 508 00:22:57,476 --> 00:22:59,111 But the data monitoring committee 509 00:22:59,111 --> 00:23:02,114 picks up once the trial, starts. 510 00:23:02,982 --> 00:23:07,653 And one really important, component of that monitoring 511 00:23:07,653 --> 00:23:11,402 is to protect subjects from previously unknown adverse 512 00:23:11,402 --> 00:23:11,957 events. 513 00:23:12,358 --> 00:23:15,914 So imagine there's been some animal 514 00:23:15,914 --> 00:23:16,829 studies. 515 00:23:16,829 --> 00:23:19,164 There's been some phase one studies. 516 00:23:19,164 --> 00:23:21,834 There's been some phase two studies. 517 00:23:21,834 --> 00:23:23,936 Not a lot of time has passed. 518 00:23:23,936 --> 00:23:27,091 So maybe there's going to be an adverse 519 00:23:27,091 --> 00:23:29,275 event that comes up a year 520 00:23:29,275 --> 00:23:32,678 into the trial that it would be really important to, to measure. 521 00:23:32,678 --> 00:23:36,782 So for example, with the Covid vaccine trials, 522 00:23:36,782 --> 00:23:41,053 there was this unlikely but potential adverse 523 00:23:41,053 --> 00:23:44,056 event that those so 524 00:23:44,523 --> 00:23:49,028 this is known from other vaccine products 525 00:23:49,628 --> 00:23:54,646 that if you get the vaccine and are then exposed to the 526 00:23:54,646 --> 00:23:55,467 disease, 527 00:23:55,467 --> 00:23:59,672 you might have an even worse response to the virus. 528 00:23:59,672 --> 00:24:01,640 Again, highly unlikely. 529 00:24:01,640 --> 00:24:05,716 It wasn't expected, nor did it happen in the Covid vaccine 530 00:24:05,716 --> 00:24:06,278 trials. 531 00:24:06,679 --> 00:24:09,181 But one of the jobs of the data monitoring committees 532 00:24:09,181 --> 00:24:11,990 was to monitor the data to make sure that 533 00:24:11,990 --> 00:24:13,018 didn't happen. 534 00:24:13,619 --> 00:24:16,055 Now, if that happens, 535 00:24:16,055 --> 00:24:19,091 it's their job to decide whether or not 536 00:24:19,591 --> 00:24:23,429 that signal means that the study has to stop. 537 00:24:23,862 --> 00:24:27,066 So one reason it might have to stop 538 00:24:27,066 --> 00:24:31,303 is because the adverse event is so severe 539 00:24:31,303 --> 00:24:34,373 and so unanticipated that 540 00:24:34,373 --> 00:24:36,757 you need to stop the trial for a while 541 00:24:36,757 --> 00:24:38,577 and look into what happened. 542 00:24:38,844 --> 00:24:42,414 Maybe it had nothing to do with the intervention 543 00:24:42,414 --> 00:24:44,692 that the person received, but you don't 544 00:24:44,692 --> 00:24:46,852 want to move forward with the trial. 545 00:24:46,852 --> 00:24:49,410 If there's a possibility that others 546 00:24:49,410 --> 00:24:50,689 who are receiving 547 00:24:50,689 --> 00:24:53,359 the treatment are going to have that same adverse event. 548 00:24:54,493 --> 00:24:57,963 So they're monitoring for adverse events, 549 00:24:58,297 --> 00:25:02,034 they're also looking at how is the trial doing. 550 00:25:02,534 --> 00:25:05,971 So again, let's imagine you're doing your trial. 551 00:25:05,971 --> 00:25:09,541 And you need in order to answer your question, 552 00:25:10,009 --> 00:25:13,846 a sample of 200 in order to find out 553 00:25:13,846 --> 00:25:16,479 whether the new treatment is better than the old 554 00:25:16,479 --> 00:25:17,082 treatment. 555 00:25:17,750 --> 00:25:20,486 And you're planning to do that over a year. 556 00:25:20,486 --> 00:25:23,601 The data monitoring committee is also looking at your 557 00:25:23,601 --> 00:25:24,189 progress. 558 00:25:24,189 --> 00:25:27,426 So if halfway through, 559 00:25:27,426 --> 00:25:31,584 you only have 50 participants and your anticipated enrollment 560 00:25:31,584 --> 00:25:32,197 was 100, 561 00:25:32,598 --> 00:25:35,754 the data monitoring committee might say, okay, what's going 562 00:25:35,754 --> 00:25:35,968 on? 563 00:25:36,869 --> 00:25:40,139 Why aren't you at where you want to be? 564 00:25:40,139 --> 00:25:42,374 And what are you doing to address that? 565 00:25:42,374 --> 00:25:46,478 Because it's important for a trial to move forward, 566 00:25:46,779 --> 00:25:49,748 and it's funded for a certain period. 567 00:25:49,748 --> 00:25:52,862 That's important for the data monitoring committee to be 568 00:25:52,862 --> 00:25:53,252 aware. 569 00:25:53,986 --> 00:25:57,184 We've got what happens if we can't fully enroll the 570 00:25:57,184 --> 00:25:57,623 trial. 571 00:25:58,557 --> 00:26:01,799 So they might say, tell us what you're doing to increase 572 00:26:01,799 --> 00:26:02,494 enrollment. 573 00:26:02,494 --> 00:26:05,764 So they might add a new site, or they might, 574 00:26:06,532 --> 00:26:09,454 develop relationships with local providers or 575 00:26:09,454 --> 00:26:10,169 something. 576 00:26:10,169 --> 00:26:13,626 But it's, again, just a way to remind the 577 00:26:13,626 --> 00:26:14,807 investigators 578 00:26:15,174 --> 00:26:17,736 from an independent external body 579 00:26:17,736 --> 00:26:19,445 how things are going. 580 00:26:19,445 --> 00:26:21,550 And I should have made that more clear 581 00:26:21,550 --> 00:26:22,548 at the beginning. 582 00:26:22,548 --> 00:26:25,146 The data monitoring committee is made up 583 00:26:25,146 --> 00:26:27,419 of individuals who are independent 584 00:26:27,419 --> 00:26:30,422 from the study, generally independent 585 00:26:30,422 --> 00:26:33,425 from the PiS local institution. 586 00:26:33,959 --> 00:26:35,093 If, for example, 587 00:26:36,361 --> 00:26:37,296 let's say you have a data 588 00:26:37,296 --> 00:26:40,955 monitoring committee and you add a new site, and that 589 00:26:40,955 --> 00:26:41,300 site 590 00:26:41,300 --> 00:26:46,104 PII is, you know, works with the head of the DMC, 591 00:26:46,338 --> 00:26:49,660 the the chair might have to recuse him 592 00:26:49,660 --> 00:26:50,709 or herself, 593 00:26:51,076 --> 00:26:53,487 but it's independent in the sense 594 00:26:53,487 --> 00:26:55,314 that they don't have any 595 00:26:57,015 --> 00:26:59,251 expectation, 596 00:26:59,251 --> 00:27:03,422 that they're decisions, 597 00:27:04,590 --> 00:27:08,393 are somehow influenced by outside, 598 00:27:10,496 --> 00:27:12,931 concerns or bias. 599 00:27:12,931 --> 00:27:17,703 So the data monitoring committee also has 600 00:27:18,437 --> 00:27:21,874 what are referred to as in the end, 601 00:27:21,874 --> 00:27:25,110 stop being rules or interim analysis. 602 00:27:25,110 --> 00:27:28,113 The last so 603 00:27:28,413 --> 00:27:30,849 one thing that you want to make sure is true 604 00:27:30,849 --> 00:27:33,986 is that you are not exposing subjects 605 00:27:33,986 --> 00:27:37,823 to something that is overly risky, 606 00:27:38,156 --> 00:27:41,093 or withholding something of benefit 607 00:27:41,093 --> 00:27:44,630 from a group who could benefit so 608 00:27:46,031 --> 00:27:49,134 interim analysis is the data 609 00:27:49,134 --> 00:27:52,137 monitoring committee receives data 610 00:27:52,538 --> 00:27:55,173 that is sometimes blinded in the sense 611 00:27:55,173 --> 00:27:57,656 that it doesn't say the name of the drug A 612 00:27:57,656 --> 00:27:59,311 and the name of the drug B. 613 00:27:59,311 --> 00:28:04,048 It literally says A versus B, and they're looking to see 614 00:28:04,048 --> 00:28:05,317 whether or not 615 00:28:05,551 --> 00:28:11,823 it looks like the new drug drug B or sorry, 616 00:28:12,090 --> 00:28:15,072 one of the drugs is doing much better 617 00:28:15,072 --> 00:28:16,361 than the other. 618 00:28:16,929 --> 00:28:19,855 And in terms of an interim analysis, 619 00:28:19,855 --> 00:28:21,400 it looks like it's 620 00:28:21,400 --> 00:28:26,434 statistically doing better than the A in this 621 00:28:26,434 --> 00:28:27,105 case. 622 00:28:27,506 --> 00:28:31,776 And so they could decide, according to the interim 623 00:28:31,776 --> 00:28:32,544 analysis 624 00:28:32,544 --> 00:28:34,801 and the stopping rules that we have 625 00:28:34,801 --> 00:28:37,316 a statistically significant difference 626 00:28:37,516 --> 00:28:40,652 at this point earlier that we thought 627 00:28:40,652 --> 00:28:43,522 and we think the trials should be stopped 628 00:28:43,522 --> 00:28:46,525 and the intervention is unblinded. 629 00:28:46,792 --> 00:28:51,122 Now when you unblind them, it could be either that people 630 00:28:51,122 --> 00:28:51,730 getting 631 00:28:51,730 --> 00:28:54,305 the new drug are better or the people 632 00:28:54,305 --> 00:28:56,602 getting the old drug are better. 633 00:28:56,602 --> 00:28:59,521 You don't know, but you know that something's 634 00:28:59,521 --> 00:29:00,105 going on 635 00:29:00,105 --> 00:29:02,407 and there's a statistically significant difference. 636 00:29:03,575 --> 00:29:04,576 And so that would be an 637 00:29:04,576 --> 00:29:07,646 example of shortening a trial 638 00:29:07,646 --> 00:29:11,016 because it's doing even better than you imagined. 639 00:29:11,717 --> 00:29:14,886 And in this case, it could be that it's the new or the old. 640 00:29:14,886 --> 00:29:16,888 But it's a reason to stop 641 00:29:16,888 --> 00:29:19,391 because withholding the new, 642 00:29:19,391 --> 00:29:22,447 the treatment B puts the participants 643 00:29:22,447 --> 00:29:24,429 in a worse off position 644 00:29:24,696 --> 00:29:28,967 and not giving the B to those who are getting A, 645 00:29:29,401 --> 00:29:31,555 you know, they should all now get the 646 00:29:31,555 --> 00:29:32,371 intervention. 647 00:29:33,705 --> 00:29:37,509 And the other thing to think about is that 648 00:29:39,444 --> 00:29:41,780 bad things could happen as well. 649 00:29:41,780 --> 00:29:44,950 So stopping rules are in place. 650 00:29:44,950 --> 00:29:48,453 Sometimes stopping rules refer to like 651 00:29:48,720 --> 00:29:51,509 we know more than one death on the end, 652 00:29:51,509 --> 00:29:52,224 you know. 653 00:29:52,224 --> 00:29:55,899 So stopping rules are laid out in advance 654 00:29:55,899 --> 00:29:56,795 that say, 655 00:29:57,062 --> 00:30:00,032 if this happens, we are going to stop the trial 656 00:30:00,599 --> 00:30:04,269 and you may reopen the trial 657 00:30:04,469 --> 00:30:07,773 or stop and close the trial. So, 658 00:30:08,907 --> 00:30:10,742 that's all to say that 659 00:30:10,742 --> 00:30:13,862 especially for phase three trials 660 00:30:13,862 --> 00:30:15,847 and sometimes trials 661 00:30:15,847 --> 00:30:18,896 that are sort of high profile or there's 662 00:30:18,896 --> 00:30:22,020 a, concerns about severe adverse events. 663 00:30:22,020 --> 00:30:25,791 Phase two studies have data monitoring committees 664 00:30:25,791 --> 00:30:29,413 in order to make sure that the data 665 00:30:29,413 --> 00:30:30,862 is coming out 666 00:30:30,862 --> 00:30:35,567 in a reasonable, timely fashion and that nothing especially bad 667 00:30:35,567 --> 00:30:40,238 or especially good is happening in order to stop it as needed. 668 00:30:40,872 --> 00:30:45,310 So, with that, I am going to just remind you 669 00:30:45,310 --> 00:30:49,347 that good study design is key to conduct of an ethical study.