1 00:00:09,342 --> 00:00:11,678 So, what are our types of randomized studies. 2 00:00:11,678 --> 00:00:15,482 Parallel group is our classic study. I've mentioned that already a few times. 3 00:00:15,482 --> 00:00:18,418 We have sequential trials. A whole list of other ones. 4 00:00:18,418 --> 00:00:21,054 I'm going to go through all of these. 5 00:00:21,054 --> 00:00:25,725 So, in a parallel group design, the idea is that I'm going to randomized patients 6 00:00:25,725 --> 00:00:27,193 to one of x treatments. 7 00:00:27,193 --> 00:00:30,397 So one of two treatments, one of four treatments, however many. 8 00:00:30,930 --> 00:00:33,600 And I'm going to look for some response. 9 00:00:33,600 --> 00:00:38,271 I'm going to measure them at the end of the study and just compare, 10 00:00:38,271 --> 00:00:41,941 you know, how is everybody at the end of one year. 11 00:00:41,941 --> 00:00:45,945 I might look at a change or a percent change from baseline. 12 00:00:45,945 --> 00:00:49,282 So how do they change between baseline and one year. 13 00:00:49,282 --> 00:00:51,284 I may look at repeated measures. 14 00:00:51,284 --> 00:00:56,256 Maybe I actually take their blood pressure every four weeks, and I'm going to look 15 00:00:56,256 --> 00:01:00,827 at the change in systolic blood pressure over time, and I want to look 16 00:01:00,827 --> 00:01:03,930 at a curve of that information. That's called repeated measures. 17 00:01:03,930 --> 00:01:07,033 Or I may look at a function of multiple measures. 18 00:01:07,033 --> 00:01:09,536 If you think about it, body mass index, 19 00:01:09,536 --> 00:01:13,573 BMI, it is in fact a function of your height and your weight. 20 00:01:14,207 --> 00:01:17,644 So there are a lot of variations on parallel group designs. 21 00:01:17,644 --> 00:01:21,381 We sometimes, but not always, do dosed titration with multiple study arms. 22 00:01:21,381 --> 00:01:25,718 This is becoming more popular, especially if it's not a first in human product. 23 00:01:26,352 --> 00:01:30,790 The idea is you want to titrate to the maximum tolerated dose 24 00:01:30,790 --> 00:01:32,258 within a given subject. 25 00:01:32,258 --> 00:01:36,663 Dose escalation studies with a control arm that you're simultaneously randomizing to. 26 00:01:36,663 --> 00:01:40,366 People underestimate the importance of controls in really early research. 27 00:01:40,366 --> 00:01:46,239 It used to be, you know, the old way is you only put everybody on intervention. 28 00:01:46,239 --> 00:01:50,677 But especially when you have subjects that a lot of bad things 29 00:01:50,677 --> 00:01:55,448 might happen to them, you may say, oh, one of my subjects died. 30 00:01:55,448 --> 00:02:00,220 Well, if your subjects have a 50 percent mortality rate, it's kind of hard to tell 31 00:02:00,220 --> 00:02:03,823 was it the treatment or the disease that caused them to die? 32 00:02:03,823 --> 00:02:09,195 If you have a control arm, even in those very early studies, you can start to tease out 33 00:02:09,195 --> 00:02:11,564 what are the differences in the adverse events? 34 00:02:11,564 --> 00:02:14,567 What are the differences in the death rates, et cetera? 35 00:02:15,468 --> 00:02:18,638 I know some of you all are making like ugly faces. 36 00:02:18,638 --> 00:02:22,375 I'll be honest with you, like, this is real life in clinical research. 37 00:02:22,375 --> 00:02:26,379 If you do interventional research, there's a very good chance you will kill people. 38 00:02:26,379 --> 00:02:29,816 Not that you mean to, but you may in fact cause harm. 39 00:02:29,816 --> 00:02:34,988 If we knew the answer, if we knew that people were or were not going to be harmed, 40 00:02:34,988 --> 00:02:38,725 if we knew that something worked, we wouldn't need to do the research. 41 00:02:39,926 --> 00:02:43,029 So, this is something that you have to kind of in 42 00:02:43,029 --> 00:02:46,432 your gut make a decision if you're willing to do or not. 43 00:02:46,432 --> 00:02:50,970 If you have thoughts about that, Steven Strauss' chapter in the book is a good one 44 00:02:50,970 --> 00:02:51,838 to read. Dr. 45 00:02:51,838 --> 00:02:53,239 Strauss died several years ago, 46 00:02:53,239 --> 00:02:58,344 but he talks about a very personal journey that he had with one of the studies he did 47 00:02:58,344 --> 00:03:02,882 where they found out very late in the process that in fact, they were causing harm. 48 00:03:03,850 --> 00:03:08,321 And some of the people that worked on that study decided they had to leave research. 49 00:03:08,321 --> 00:03:09,989 They had to leave medicine completely 50 00:03:09,989 --> 00:03:12,492 because it was something that they just couldn't handle. 51 00:03:12,492 --> 00:03:15,428 It's not an easy thing, but it's something to consider. 52 00:03:15,428 --> 00:03:16,963 Now back to the design. 53 00:03:16,963 --> 00:03:20,500 I mean, the whole goal of this course is to try to make 54 00:03:20,700 --> 00:03:24,504 that you hopefully figure this out before you ever touch a human being, right? 55 00:03:24,837 --> 00:03:28,007 We do not want to cause harm. 56 00:03:28,007 --> 00:03:30,376 Not all dose escalation, dose 57 00:03:30,376 --> 00:03:35,949 titration studies are randomized, but some of them are, more of them are now. 58 00:03:35,949 --> 00:03:37,150 In sequential trials. 59 00:03:37,150 --> 00:03:39,919 So sequential trials happen more in engineering, 60 00:03:39,919 --> 00:03:44,290 but if you're doing device manufacturing, you may also do this. 61 00:03:44,290 --> 00:03:50,263 You don't necessarily have a fixed sampled size or period that you're running the study. 62 00:03:50,496 --> 00:03:53,233 This, of course, makes funders kind of scary. 63 00:03:53,466 --> 00:03:56,569 IRBs go, "What?" So those are your institutional review 64 00:03:56,569 --> 00:03:59,405 boards, the groups that approve human subjects research. 65 00:03:59,405 --> 00:04:04,043 The idea with the sequential trial is that ends when one treatment shows 66 00:04:04,043 --> 00:04:09,015 clear superiority or is highly unlikely any important difference is going to be seen. 67 00:04:09,015 --> 00:04:14,354 So it kind of makes sense but you'll see this, like, computers, capacitors, et cetera, 68 00:04:14,354 --> 00:04:18,992 but very special statistical design methods are needed when you do these trials. 69 00:04:20,827 --> 00:04:25,198 One that you do see commonly in clinical research is group sequential trials. 70 00:04:25,198 --> 00:04:31,237 So here kind of what we're going to talk about is type 1 error can be easily computed. 71 00:04:31,237 --> 00:04:35,241 And you can't do that easily in the straight up sequential trials. 72 00:04:35,241 --> 00:04:39,946 So these are very popular because these group sequential trials, you analyze your data 73 00:04:39,946 --> 00:04:41,281 after a certain proportion 74 00:04:41,281 --> 00:04:44,951 of the results, or the information from the trial is available. 75 00:04:46,619 --> 00:04:49,756 There's early stopping, depending on how you set this up, 76 00:04:49,756 --> 00:04:53,826 if one treatment arm is clearly superior, if it looks like there's futility, 77 00:04:53,826 --> 00:04:58,765 so if you got to the end of the trial, you're still not going to have 78 00:04:58,965 --> 00:05:01,401 significant results, you may as well stop now. 79 00:05:01,401 --> 00:05:03,903 You may also still stop for adverse events. 80 00:05:03,903 --> 00:05:08,308 So all trials should be monitored to see if they need to be stopped. 81 00:05:08,341 --> 00:05:12,478 We'll talk about that in the second part of the course. 82 00:05:12,478 --> 00:05:17,016 This takes a lot of really careful planning and statistical design work. 83 00:05:17,016 --> 00:05:22,288 It's going to impact your sample size, so you have to roll the fact 84 00:05:22,288 --> 00:05:25,692 that you're going to be analyzing your data before 85 00:05:25,692 --> 00:05:30,229 the study is done into the planning of the study initially. 86 00:05:30,229 --> 00:05:36,169 But this is an example that we had from a trial from NIAID, it's 87 00:05:36,169 --> 00:05:40,907 actually a very old example, that was one of the first studies of 88 00:05:40,907 --> 00:05:43,843 zidovudine, and it was done in pregnant women. 89 00:05:43,843 --> 00:05:48,214 And so at the first interim analysis where one-third of the projected 90 00:05:48,214 --> 00:05:52,251 infant infections happened, the data safety monitoring board saw this picture. 91 00:05:52,251 --> 00:05:56,989 We're going to come back to this picture in the survival analysis lecture. 92 00:05:56,989 --> 00:06:01,828 But this was a randomized trial, they randomized the mothers to either take 93 00:06:01,828 --> 00:06:02,962 zidovudine or placebo, 94 00:06:02,962 --> 00:06:08,234 and then they looked at the probability of transmission of HIV to the infants. 95 00:06:08,234 --> 00:06:12,739 And you can see how kind of the study arms worked out. 96 00:06:12,739 --> 00:06:17,276 So this is a Kaplan Meier curves that are on this screen 97 00:06:17,276 --> 00:06:20,313 and then there's a p-value associated with it. 98 00:06:20,313 --> 00:06:26,319 But a lot of work went in to trying to decide what should the interventions be. 99 00:06:26,319 --> 00:06:29,021 What should the population be? 100 00:06:29,021 --> 00:06:34,627 We knew zidovudine at the time was able to slow the progression of HIV in adults 101 00:06:34,627 --> 00:06:37,430 with advanced disease, but this AIDS Clinical Trials 102 00:06:37,430 --> 00:06:42,001 Group Protocol 076 was looking at safety and efficacy of zidovudine in preventing 103 00:06:42,001 --> 00:06:46,572 transmission of HIV from infected but not necessarily advanced women to their babies. 104 00:06:46,572 --> 00:06:51,110 So now I've got to figure out -- I've got a different population. 105 00:06:51,110 --> 00:06:53,012 They're not necessarily advanced. 106 00:06:53,012 --> 00:06:55,882 I've got to worry about not only mom. 107 00:06:55,882 --> 00:06:58,017 I've got to worry about infant. 108 00:06:58,017 --> 00:07:02,688 Because maybe they don't get HIV, but they have some other horrible problem 109 00:07:02,688 --> 00:07:04,123 that happens to them. 110 00:07:04,123 --> 00:07:07,226 You've got to think about what are the ramifications 111 00:07:07,460 --> 00:07:11,631 of giving a drug to somebody, especially if it's a pregnant woman, 112 00:07:11,631 --> 00:07:14,867 or if it's a male who might impregnant somebody. 113 00:07:16,402 --> 00:07:20,640 Yeah, that's a little trick they forget to tell you about. 114 00:07:20,640 --> 00:07:22,942 Everyone thinks about lactation and pregnancy, 115 00:07:22,942 --> 00:07:27,780 and they forget that the guy plays a role in that pregnancy. 116 00:07:27,780 --> 00:07:32,985 Anyway, so preventing HIV transmission, they had to power this study to detect 117 00:07:32,985 --> 00:07:36,055 a 33 percent reduction in the transmission rate. 118 00:07:36,055 --> 00:07:41,461 So the placebo rate, or the normal, natural history, I should say, rate of 119 00:07:41,461 --> 00:07:46,499 transmission was 30 percent, and they wanted to drop it to 20 percent. 120 00:07:46,499 --> 00:07:51,871 So this study they had planned was going to do accrual over five years. 121 00:07:51,871 --> 00:07:53,439 They expected 15 percent dropouts. 122 00:07:53,439 --> 00:07:57,743 Again, you expect that some of these folks you're not going to be able 123 00:07:57,743 --> 00:08:00,847 to follow the entire time, for a variety of reasons. 124 00:08:00,847 --> 00:08:03,316 Infants die for a lot of different reasons. 125 00:08:03,316 --> 00:08:06,719 Sometimes also, you know, moms and infants they go somewhere else. 126 00:08:06,719 --> 00:08:07,954 You can't track them. 127 00:08:07,954 --> 00:08:12,291 You have to think about all this when you're trying to design your trial. 128 00:08:13,793 --> 00:08:17,930 Also at that time, HIV testing -- not all that great. 129 00:08:17,930 --> 00:08:22,101 So they had to figure out, like, what was in fact 130 00:08:22,101 --> 00:08:27,006 a positive test in order to decide that someone had an event. 131 00:08:27,006 --> 00:08:30,042 Another type of study's a crossover study. 132 00:08:30,042 --> 00:08:34,213 So in a crossover study, let's say I have two treatments. 133 00:08:34,213 --> 00:08:36,849 I will then have a two-period crossover. 134 00:08:36,849 --> 00:08:39,485 Each patient acts as their own control. 135 00:08:39,485 --> 00:08:43,289 The trick here is you need to eliminate crossover effects. 136 00:08:43,923 --> 00:08:49,061 So if I'm going to have you all use an asthma inhaler 137 00:08:49,061 --> 00:08:54,200 that then changes your lung structure in some way, changes your cells, 138 00:08:54,200 --> 00:09:00,606 I probably am not going to wash that out, at least not for a while. 139 00:09:00,606 --> 00:09:05,177 If I teach you meditation, I can't unteach it from you. 140 00:09:05,177 --> 00:09:10,016 Somethings you cannot do in a crossover trial, but the idea 141 00:09:10,016 --> 00:09:13,452 is that a lot of things you can. 142 00:09:14,086 --> 00:09:18,257 So the women's alcohol study, we did a three eight-week dietary periods. 143 00:09:18,257 --> 00:09:22,428 Each woman was randomized to the order in which they took in 144 00:09:22,428 --> 00:09:23,829 different doses of alcohol. 145 00:09:23,829 --> 00:09:27,300 So they either had 30 grams of alcohol a day, 146 00:09:27,300 --> 00:09:32,171 which is the equivalent of about two drinks, 15 grams of alcohol a day, 147 00:09:32,171 --> 00:09:37,777 so one drink, zero grams of alcohol a day, so they got an alcohol-free beverage. 148 00:09:37,777 --> 00:09:40,680 Basically, they got orange juice and Everclear. 149 00:09:40,680 --> 00:09:45,618 The order of the assignment of the three alcohol levels was random. 150 00:09:45,618 --> 00:09:48,521 So that's the part that got randomized. 151 00:09:48,521 --> 00:09:52,658 Each woman had each one of these three doses. Why? 152 00:09:52,658 --> 00:09:56,796 Because we each have different set of hormones, different cardiovascular 153 00:09:56,796 --> 00:10:02,568 risk factors, and we were trying to look at a lot of cancer risks. 154 00:10:03,603 --> 00:10:04,270 Well, in 155 00:10:04,270 --> 00:10:09,675 that case, kind of, it's better kind of do it inside a person, and their own diet, 156 00:10:09,675 --> 00:10:14,447 and all of their own stuff that they are bringing outside of the alcohol. 157 00:10:14,447 --> 00:10:16,048 We had washout periods. 158 00:10:16,048 --> 00:10:20,519 And because this was such a long study, we actually varied the washout period. 159 00:10:20,519 --> 00:10:25,591 So we had one group of people, the people literally packing the lunch at the USDA 160 00:10:25,591 --> 00:10:26,559 and the dinners. 161 00:10:26,559 --> 00:10:31,063 So every night they had a snack, and that snack included this beverage, 162 00:10:31,063 --> 00:10:35,901 and they were told take it at the end, before you go to sleep, 163 00:10:35,901 --> 00:10:37,970 do not drive, blah, blah, blah. 164 00:10:37,970 --> 00:10:40,373 But, you know, differing washout periods -- 165 00:10:40,373 --> 00:10:45,211 and sometimes we always had the same washout period, but at least with alcohol, 166 00:10:45,211 --> 00:10:50,049 we knew how long physiologically it took to get it out of the system. 167 00:10:51,117 --> 00:10:52,885 And this was a double-blind study. 168 00:10:52,885 --> 00:10:55,521 The investigators who were, you know, drawing their blood, 169 00:10:55,521 --> 00:10:58,791 and checking their blood pressure three downings [spelled phonetically] a week. 170 00:10:58,791 --> 00:11:01,427 They did not know what this person was on. 171 00:11:01,427 --> 00:11:04,096 The women did not know what they were on. 172 00:11:04,096 --> 00:11:08,801 Some of them said they thought that they knew that they were getting drunk at night. 173 00:11:08,801 --> 00:11:12,338 And this was actually in a Washington Post article on this study. 174 00:11:12,772 --> 00:11:15,074 And so after the study was over with, 175 00:11:15,074 --> 00:11:19,945 the PI said can someone look back to see if she had been taking alcohol or not. 176 00:11:19,945 --> 00:11:24,817 It turns out she'd only been in the placebo part of it at that point in time. 177 00:11:24,817 --> 00:11:26,552 That's actually another trick I've learned. 178 00:11:26,552 --> 00:11:29,989 Like, people who think like they're having all these huge side effects, 179 00:11:29,989 --> 00:11:34,627 they think they know what study they're on, a lot of times you don't. 180 00:11:34,627 --> 00:11:39,331 But anyway, same thing goes for the clinicians who are trying to 181 00:11:39,331 --> 00:11:42,034 guess what study arm somebody's on. 182 00:11:42,034 --> 00:11:45,171 So then you have these factorial designs. 183 00:11:45,171 --> 00:11:46,338 So factorial designs, 184 00:11:46,338 --> 00:11:51,811 each level of the factor, or treatment, or condition occurs with every other factor. 185 00:11:51,811 --> 00:11:56,082 So this was a study that my NCI colleagues worked on, 186 00:11:56,082 --> 00:11:59,218 it was in gastroenterology, where they randomized people. 187 00:11:59,552 --> 00:12:04,557 You either got Selenium placebo, and Celecoxib placebo, or some combination. 188 00:12:04,557 --> 00:12:10,496 So what you'll notice is kind of this bottom box is Celecoxib only. 189 00:12:10,496 --> 00:12:14,166 And this top far box is Selenium only. 190 00:12:14,166 --> 00:12:19,638 Down here in the bottom corner they're getting both Selenium and Celecoxib. 191 00:12:19,638 --> 00:12:21,941 Now how does that work? 192 00:12:21,941 --> 00:12:26,045 Well, it works if you don't think that Selenium 193 00:12:26,045 --> 00:12:31,984 and Celecoxib interact with each other in particular with respect to your outcome. 194 00:12:33,385 --> 00:12:34,186 So you'd 195 00:12:34,186 --> 00:12:39,125 break up these factors and when you do the analysis, I compare 196 00:12:39,125 --> 00:12:41,994 everybody in kind of this Selenium placebo 197 00:12:41,994 --> 00:12:46,532 arm to everybody getting Selenium real, ignoring what Celecoxib they got. 198 00:12:46,532 --> 00:12:51,437 And then I compare all the Celecoxib folks and the placebo Celecoxib 199 00:12:51,437 --> 00:12:53,906 to Celecoxib real, ignoring their Selenium. 200 00:12:53,906 --> 00:13:01,313 The problem is, a lot of times I do these, and then the investigators come back and say, 201 00:13:01,313 --> 00:13:06,719 "So can you tell me if there is an interaction?" All right, 202 00:13:06,719 --> 00:13:12,158 if you care about that interaction, if you expect it might exist, 203 00:13:12,158 --> 00:13:17,129 you need to do a four arm study, not a -- 204 00:13:17,129 --> 00:13:24,403 so you can design -- it can look like this, but when you power this study, 205 00:13:24,403 --> 00:13:28,941 you cannot power the study assuming that these two interventions 206 00:13:28,941 --> 00:13:31,644 are independent of each other. 207 00:13:31,644 --> 00:13:35,281 So you've got to make a decision. 208 00:13:35,281 --> 00:13:38,450 Two two-arm studies or a four-arm study. 209 00:13:38,450 --> 00:13:41,620 The MsFLASH also used a factorial design. 210 00:13:41,620 --> 00:13:43,823 I liked this one because, okay, 211 00:13:43,823 --> 00:13:48,627 there are a lot of things I don't like about the MsFLASH study. 212 00:13:48,627 --> 00:13:49,728 I'll be honest. 213 00:13:49,728 --> 00:13:53,032 But here you'll see that it's not evenly randomized 214 00:13:53,032 --> 00:13:56,702 because what they did is they went a step farther. 215 00:13:56,702 --> 00:14:01,140 They said we're not even going to compare yoga to aerobic exercise. 216 00:14:01,140 --> 00:14:03,709 We're comparing aerobic exercise to usual activity. 217 00:14:05,144 --> 00:14:07,246 We're comparing yoga to usual activity. 218 00:14:07,246 --> 00:14:08,314 We're not going 219 00:14:08,314 --> 00:14:12,885 to compare all three of these arms, but they have an unequal randomization 220 00:14:12,885 --> 00:14:18,190 between the study groups in order to achieve the statistical power that they needed. 221 00:14:18,190 --> 00:14:22,761 Then you run into thinks like incomplete, partial, or fractional factorial trials. 222 00:14:22,761 --> 00:14:26,966 Depending on where you train, it is labeled one of these three. 223 00:14:26,966 --> 00:14:29,435 Nutritional intervention trial's an example of this. 224 00:14:29,935 --> 00:14:34,206 They had four different types of micronutrients that they were looking at. 225 00:14:34,206 --> 00:14:39,245 And what they did is they didn't want to look at all possible interactions. 226 00:14:39,245 --> 00:14:41,747 In fact, this study in the end, 227 00:14:41,747 --> 00:14:46,385 I want to say had maybe like 30,000, 20,000 plus people in it. 228 00:14:46,385 --> 00:14:49,255 And that was only looking at certain combinations. 229 00:14:49,255 --> 00:14:54,260 But you will see groups that will do this, that will choose certain combinations 230 00:14:54,260 --> 00:14:58,731 to look at, and you have to also make sure that you have 231 00:14:58,731 --> 00:15:03,235 the ones in there that you need in order to do the analysis you care about. 232 00:15:03,235 --> 00:15:07,740 The problem is, you know, in the end people want you to look at certain interactions 233 00:15:07,740 --> 00:15:10,843 that you don't have in there, certain combinations you don't have. 234 00:15:10,843 --> 00:15:15,614 So you do have to think pretty hard in advanced about what you want to leave out. 235 00:15:17,383 --> 00:15:17,883 I'm going 236 00:15:17,883 --> 00:15:22,621 to spend several minutes on adaptive designs because they're gaining a lot of popularity. 237 00:15:22,621 --> 00:15:25,658 Here you have maybe two to eight different arms. 238 00:15:25,658 --> 00:15:27,693 Sometimes it's dose ranging, sometimes not. 239 00:15:27,693 --> 00:15:32,064 People think you have, and you'll see it in all these clinical journals, 240 00:15:32,064 --> 00:15:36,135 if you do adaptive designs, you'll have a smaller overall sample size. 241 00:15:36,135 --> 00:15:36,802 Yeah, sometimes. 242 00:15:36,802 --> 00:15:41,206 Sometimes you have a larger overall sample size but at least, you know, 243 00:15:41,206 --> 00:15:43,909 you're able to do it in one trial. 244 00:15:45,711 --> 00:15:49,515 A lot of these have kind of a run-in period 245 00:15:49,515 --> 00:15:53,719 and then you start analyzing data continuously or at fixed points. 246 00:15:53,719 --> 00:15:58,324 But for any adaptive design, and there're like 30 plus different versions 247 00:15:58,324 --> 00:16:01,393 of adaptive designs, you need to be clear. 248 00:16:01,393 --> 00:16:02,895 What is being adapted? 249 00:16:02,895 --> 00:16:06,365 So the number of people in each study arm? 250 00:16:06,365 --> 00:16:10,569 Is it something about the randomization, like the characteristics of people? 251 00:16:10,569 --> 00:16:12,471 Is it the invention themselves? 252 00:16:12,471 --> 00:16:14,907 What is being adapted? 253 00:16:14,907 --> 00:16:17,910 When are you going to adapt it? 254 00:16:17,910 --> 00:16:22,247 And based on what evidence does this adaptation take place? 255 00:16:22,247 --> 00:16:24,850 Who decides an adaptation is needed? 256 00:16:24,850 --> 00:16:27,019 And how is it implemented? 257 00:16:29,154 --> 00:16:32,891 So this is a slide from one of Paul Wakim's lectures 258 00:16:32,891 --> 00:16:37,329 that he actually got from Paul Gallo, who's in the pharma working group. 259 00:16:37,329 --> 00:16:42,468 Basically, the idea with adaptive designs is this is a clinical study design that uses 260 00:16:42,468 --> 00:16:43,135 accumulating data 261 00:16:43,135 --> 00:16:48,607 from your trial to decide how to modify aspects of that same trial as it continues. 262 00:16:48,607 --> 00:16:52,711 But the trick is, you've got to do this in a way 263 00:16:52,711 --> 00:16:55,781 that doesn't undermine the validity, integrity of the trial. 264 00:16:57,583 --> 00:17:03,689 Now, if you look at my employer's work on this, we'll also say an adaptive 265 00:17:03,689 --> 00:17:08,160 design is defined as a study that includes prospectively planned opportunities 266 00:17:08,160 --> 00:17:13,465 for modification of one or more aspects of the study design and hypotheses 267 00:17:13,465 --> 00:17:19,171 based on analysis of data, usually interim data, from subjects in the study. 268 00:17:19,171 --> 00:17:24,043 So one of my studies, they wanted to do adaptive study. 269 00:17:24,043 --> 00:17:29,348 We thought it was a good idea except then we looked and found out 270 00:17:29,348 --> 00:17:31,617 they basically had to follow patients 271 00:17:31,617 --> 00:17:36,155 for three years before they had any good information on their outcome. 272 00:17:36,155 --> 00:17:39,558 They were going to enroll over a four-year period. 273 00:17:39,558 --> 00:17:43,328 So the question was, what types of adaptations made sense? 274 00:17:43,328 --> 00:17:49,001 Like maybe we could look early to decide in fact patients shouldn't finish the trial, 275 00:17:49,001 --> 00:17:50,903 but we had to actually 276 00:17:50,903 --> 00:17:55,441 get some of that long-term data in order to make that determination. 277 00:17:56,408 --> 00:17:59,711 So you have to think about what adaptations make sense. 278 00:17:59,711 --> 00:18:02,714 It doesn't always make sense to adapt your randomization. 279 00:18:02,714 --> 00:18:05,184 It might make sense to stop early. 280 00:18:05,184 --> 00:18:06,685 So you have adapted 281 00:18:06,685 --> 00:18:10,989 randomizations, adapted dose findings where we may turn on and off different doses 282 00:18:10,989 --> 00:18:13,625 based on the adverse events or other characteristics 283 00:18:13,625 --> 00:18:16,929 that we're seeing, drop the loser or pick the winner. 284 00:18:16,929 --> 00:18:21,900 Again, you have to be careful there's some really bad examples where, you know, 285 00:18:21,900 --> 00:18:28,574 they made a decision to drop a study arm, but that arm only had one patient in it, 286 00:18:28,574 --> 00:18:32,644 and it was the patient that was the sickest of everybody. 287 00:18:32,644 --> 00:18:34,513 That's kind of a problem. 288 00:18:34,513 --> 00:18:38,784 We also do these adaptive seamless phase II/phase III trials. 289 00:18:38,784 --> 00:18:40,085 Biomarker adaptive trials. 290 00:18:40,085 --> 00:18:42,287 So sometimes based on your biomarker, 291 00:18:42,287 --> 00:18:47,860 we may put you in different study arms, or we may change your study arm. 292 00:18:47,860 --> 00:18:49,728 We have group sequential methods. 293 00:18:49,728 --> 00:18:51,797 So realistically, that group sequential 294 00:18:51,797 --> 00:18:57,169 -- parallel design that I talked about first, is basically an adaptive trial. 295 00:18:57,169 --> 00:19:00,038 We also do these sample size recalculations. 296 00:19:00,038 --> 00:19:03,775 We'll talk about the variance and issues like that 297 00:19:03,775 --> 00:19:09,114 and how you can use that to try to re-estimate sample size. 298 00:19:09,114 --> 00:19:15,320 So a lot of people like adaptive trials, but this is not willy-nilly folks. 299 00:19:16,188 --> 00:19:19,391 The rules have to be pre-specified in the protocol. 300 00:19:19,391 --> 00:19:23,262 The changes are made by design. This is not ad hoc. 301 00:19:23,262 --> 00:19:28,600 This is not because you see something and you want to make a little change. 302 00:19:28,600 --> 00:19:32,504 This is not a way to fix a badly designed trial. 303 00:19:32,504 --> 00:19:37,109 Stuffs going down the tubes, now you want to try to fix it, 304 00:19:37,109 --> 00:19:40,979 you know, that's a salvage operation. It is not an adaptive design. 305 00:19:43,081 --> 00:19:46,084 Adaptive designs require a lot of understanding. 306 00:19:46,084 --> 00:19:51,156 They are hard to do for investigators, reviewers, DSMB members, journal editors. 307 00:19:51,156 --> 00:19:55,427 Not all statisticians know how to do all of them. 308 00:19:55,427 --> 00:19:58,830 There are a lot of advantages and disadvantages. 309 00:19:58,830 --> 00:20:03,936 You actually -- while you have flexibility, it comes at a price. 310 00:20:03,936 --> 00:20:07,739 You need a lot more quantification of statistical risk. 311 00:20:07,739 --> 00:20:13,278 You have to understand a lot more information to actually plan these adaptations. 312 00:20:13,278 --> 00:20:17,516 A lot of them happen and they happen based on statistical rules. 313 00:20:17,516 --> 00:20:21,420 It's going to be following the data and make a change. 314 00:20:21,420 --> 00:20:25,657 You don't actually make a decision that it should make a change. 315 00:20:25,657 --> 00:20:29,561 That means you have to know well enough what might happen. 316 00:20:29,561 --> 00:20:31,663 You also have these covariate imbalances. 317 00:20:31,663 --> 00:20:32,731 So I mentioned 318 00:20:32,731 --> 00:20:36,969 how the confounders aren't a problem unless you have an adaptive randomization. 319 00:20:37,336 --> 00:20:39,438 This is part of your problem. 320 00:20:39,438 --> 00:20:41,540 It's a lot more work upfront. 321 00:20:41,540 --> 00:20:45,377 But they can be very useful if you have the information. 322 00:20:45,377 --> 00:20:50,282 Your big negative for any trial, though, is that whenever you make a decision 323 00:20:50,282 --> 00:20:53,418 to continue or to make a change, that information 324 00:20:53,418 --> 00:20:57,256 about the study may be provided to investigators, the public, investigators. 325 00:20:57,256 --> 00:21:02,160 When you have a data safety monitoring meeting, and decide to continue a trial, 326 00:21:02,527 --> 00:21:04,429 it can change stock prices. 327 00:21:04,429 --> 00:21:10,702 That is very sad, but it is very true, and a problem that we have today. 328 00:21:10,702 --> 00:21:12,404 So enriched enrollment designs. 329 00:21:12,404 --> 00:21:16,942 This is kind of a variant of your crossover or n-of-1 studies. 330 00:21:16,942 --> 00:21:22,281 N-of-1 is when I take a patient, and I kind of do randomly assign 331 00:21:22,281 --> 00:21:27,219 when I'm going to assign them, so it's like an expanded crossover study, 332 00:21:27,219 --> 00:21:30,622 but within any given patient. In enrolled enrichment designs, 333 00:21:30,622 --> 00:21:34,026 I try to identify potential responders to the treatment. 334 00:21:34,359 --> 00:21:35,727 I enter those responders 335 00:21:35,727 --> 00:21:39,798 into a second prospective comparison study, and people think this is great. 336 00:21:39,798 --> 00:21:44,236 I have a better chance of a win except this is not generalizable 337 00:21:44,236 --> 00:21:45,937 to your general patient population. 338 00:21:45,937 --> 00:21:49,341 Sometimes, though, clinicians will tell me, well, it actually is. 339 00:21:49,341 --> 00:21:51,043 I actually try my patients. 340 00:21:51,043 --> 00:21:54,780 If they seem to be responding, we stay on the drug, 341 00:21:54,780 --> 00:21:58,884 and if they don't seem to be responding, I switch their drug. 342 00:21:59,618 --> 00:22:04,423 Well, again, you need to think about every clinical trial within large situations. 343 00:22:04,423 --> 00:22:07,392 How are you going actually implement this therapy? 344 00:22:07,392 --> 00:22:12,197 And how can you work that implementation process into your actual trial structure? 345 00:22:12,197 --> 00:22:14,433 Results tend to not be generalizable 346 00:22:14,433 --> 00:22:18,503 and you get this thing called regression to the mean. 347 00:22:18,503 --> 00:22:24,810 So the problem when you try to enroll, let's say, I have a hot flash study, 348 00:22:24,810 --> 00:22:29,614 and I want to enroll people that are having hot flashes in order 349 00:22:29,614 --> 00:22:34,019 to see if I can decrease their number of hot flashes, right. 350 00:22:34,019 --> 00:22:38,890 Problem is, like, I will get -- they'll be having 10 hot flashes a week. 351 00:22:38,890 --> 00:22:42,160 I put them on the main study, I randomize them, 352 00:22:42,160 --> 00:22:46,398 and like in my control group I'm seeing two hot flashes a week. 353 00:22:46,398 --> 00:22:50,969 That's because a lot of times we enter trials when we are fairly sick. 354 00:22:50,969 --> 00:22:56,208 And it's kind of a natural ebb and flow for a lot of parts of disease. 355 00:22:56,942 --> 00:23:01,279 And so then we naturally go back to kind of a normal low 356 00:23:01,279 --> 00:23:05,584 and now my people that are trying to analyze the data say, "Oh, 357 00:23:05,584 --> 00:23:10,222 we don't have enough events." I also saw this happen in like, you know, 358 00:23:10,222 --> 00:23:11,223 infectious disease studies. 359 00:23:11,223 --> 00:23:14,793 Like, you see this happens in a lot of strange studies. 360 00:23:14,793 --> 00:23:18,864 Herpes studies, they're having all these outbreaks, and now they have none. 361 00:23:18,897 --> 00:23:22,667 Good for the patients, not good for my study investigator. 362 00:23:22,667 --> 00:23:24,636 Group or cluster randomized studies. 363 00:23:24,636 --> 00:23:28,240 We've got a unit of randomization that's not the individuals. 364 00:23:28,240 --> 00:23:32,577 So normally when we randomize, we randomize the individuals in the study. 365 00:23:32,577 --> 00:23:38,683 But if I want to randomize an entire school and give all the kids in that school 366 00:23:38,683 --> 00:23:41,920 an intervention, if I'm going to randomize a community, 367 00:23:41,920 --> 00:23:45,490 I'm going to vaccinate everybody in a community, for example. 368 00:23:45,490 --> 00:23:48,493 If I'm going to change practice within a clinic 369 00:23:48,493 --> 00:23:51,196 and then observe what happens to the individuals 370 00:23:51,196 --> 00:23:55,233 who are patients in that clinic, or the providers in the clinic. 371 00:23:55,233 --> 00:24:00,272 Then my unit of randomization is the school, or the community, or the clinic, it's 372 00:24:00,272 --> 00:24:02,274 not the individuals inside of it. 373 00:24:02,274 --> 00:24:04,276 Now this could be really important 374 00:24:04,276 --> 00:24:09,314 because sometimes you are trying to make a change where, you know, I can't give 375 00:24:09,314 --> 00:24:13,685 pamphlets of information in the waiting room to one person versus the other. 376 00:24:14,085 --> 00:24:18,623 If they're all in the same waiting room, they can all pick up the pamphlets. 377 00:24:18,623 --> 00:24:22,260 Sometimes you'll see providers saying, you know, it's very hard for me 378 00:24:22,260 --> 00:24:27,098 to kind of change my treatment across different people when this is an open study. 379 00:24:27,098 --> 00:24:31,369 So other times, also, like we were looking at charges for bed nets, 380 00:24:31,369 --> 00:24:35,907 this was a group out of MIT, and they wanted to look at the impact 381 00:24:35,907 --> 00:24:37,142 on infant malarial cases. 382 00:24:37,142 --> 00:24:40,812 So everyone had said you need to charge for bed nets 383 00:24:40,812 --> 00:24:43,448 so that people are feeling like they're empowered. 384 00:24:43,448 --> 00:24:47,118 They have spent their money. They will use the bed nets. 385 00:24:47,118 --> 00:24:51,790 And this couple of female economists were sitting around looking at what was going 386 00:24:51,790 --> 00:24:53,124 and they're like, no. 387 00:24:53,124 --> 00:24:57,796 I can't remember their names, but they gave a really great talk on this. 388 00:24:58,430 --> 00:25:03,502 And so they actually randomized different clinics to have different pricing structures. 389 00:25:03,502 --> 00:25:06,471 Some charged none, some charged different prices. 390 00:25:06,471 --> 00:25:12,844 And then they looked to see how many bed nets got picked up or sold, 391 00:25:12,844 --> 00:25:18,350 then how many got used, then how many got used appropriately, but overall 392 00:25:18,350 --> 00:25:22,988 they said all the other economic analyses look at all those, 393 00:25:22,988 --> 00:25:26,791 but what we care about is infant malarial cases. 394 00:25:27,125 --> 00:25:31,296 So they looked to see what the infant malarial case count was. 395 00:25:31,296 --> 00:25:36,167 In fact, people who buy a bed net are more likely to use it. 396 00:25:36,167 --> 00:25:40,338 But so many more people picked up the free bed nets, that 397 00:25:40,338 --> 00:25:43,808 that in fact was what lowered the infant malarial cases. 398 00:25:44,509 --> 00:25:49,047 What they also did as they went and visited these different houses, is 399 00:25:49,047 --> 00:25:51,116 they noticed like these bed nets 400 00:25:51,116 --> 00:25:54,953 basically take up the entire structure and people were decorating them. 401 00:25:55,554 --> 00:25:59,591 They also heard that one of the main reasons people didn't put them together is 402 00:25:59,591 --> 00:26:01,192 they still didn't understand the instructions. 403 00:26:01,192 --> 00:26:04,162 And to their credit, those investigators sat there with the instructions, 404 00:26:04,162 --> 00:26:07,899 and tried to put together the bed net, and they couldn't figure it out. 405 00:26:07,899 --> 00:26:11,670 So they came up with the equivalent -- if you know the store IKEA, 406 00:26:11,670 --> 00:26:14,639 they kind of came up with these like IKEA picture instructions. 407 00:26:14,940 --> 00:26:19,878 They tested the instructions with some folks, figured out how to help them understand it. 408 00:26:19,878 --> 00:26:21,880 They also make prettier bed nets. 409 00:26:21,880 --> 00:26:26,851 Because if this is your central fixture in the household, it helps to make it 410 00:26:26,851 --> 00:26:31,156 pretty because people said they were more likely to use it that way. 411 00:26:31,156 --> 00:26:33,124 So then they ran another study. 412 00:26:33,124 --> 00:26:36,761 Pretty bed nets for free versus standard bed nets for free. 413 00:26:36,962 --> 00:26:40,432 People like pretty bed nets. It lowers infant malarial cases. 414 00:26:40,432 --> 00:26:45,236 So, you can test pretty much anything, you just have to find the place 415 00:26:45,236 --> 00:26:48,673 to do it, and make sure it's a worthwhile question.