1 00:00:09,442 --> 00:00:10,110 So, we 2 00:00:10,110 --> 00:00:14,414 have some ideas of DSMBs, I just want to say that they are 3 00:00:14,714 --> 00:00:17,217 extremely helpful and certainly people blind to trial, 4 00:00:17,217 --> 00:00:19,586 phase three trial, they're compulsory, I think. 5 00:00:19,586 --> 00:00:21,287 An NIH policy requires it. 6 00:00:21,287 --> 00:00:24,324 And I think many other institutions would as well. 7 00:00:24,324 --> 00:00:26,693 But not all trials need the DSMB. 8 00:00:26,693 --> 00:00:29,763 There are many other ways to protect the patients. 9 00:00:30,830 --> 00:00:34,234 And you can read about these policies from various institutes. 10 00:00:34,234 --> 00:00:35,969 NIH, just to give you 11 00:00:35,969 --> 00:00:40,073 some general guidelines for when you do need one of these bodies, 12 00:00:40,073 --> 00:00:44,511 all trials need the safety monitoring plan and plan for how they're going 13 00:00:44,511 --> 00:00:48,615 to make sure the data and the trial are conducted as intended 14 00:00:48,615 --> 00:00:51,684 and have integrity and also to protect the safety. 15 00:00:52,685 --> 00:00:57,424 But as soon as you have multicentered phase three trials that an independent 16 00:00:57,424 --> 00:01:00,727 DSMB has been required since 1979, it's also required 17 00:01:00,727 --> 00:01:03,997 that the IRBs are notified of all DSMB recommendations. 18 00:01:03,997 --> 00:01:08,368 And I'd say, beyond that, the policies start to vary between institutes. 19 00:01:08,368 --> 00:01:12,005 I used to work at NIAID, and I know that 20 00:01:12,005 --> 00:01:16,743 as soon as a trial was blinded and randomized, it had a DSMB. 21 00:01:17,744 --> 00:01:20,747 But that is not true across all institutes. 22 00:01:20,747 --> 00:01:24,284 So, it would be even phase two at NIAID. 23 00:01:24,284 --> 00:01:30,090 And so, here are some links that you go to, to read about these guidelines. 24 00:01:30,090 --> 00:01:33,827 The FDA, I actually -- because of very informative guidance 25 00:01:33,827 --> 00:01:37,197 and would recommend reading if you wanted as well. 26 00:01:37,197 --> 00:01:39,432 It's pretty comprehensive if you're interested 27 00:01:39,432 --> 00:01:43,002 in thinking about the best practices informing these committees. 28 00:01:43,002 --> 00:01:47,273 I would say that there is no policy of the FDA 29 00:01:47,273 --> 00:01:52,979 on what you should do or what -- there's no requirements of what is done 30 00:01:52,979 --> 00:01:55,815 but simply recommendations, because they're not governing, say, 31 00:01:55,815 --> 00:02:00,787 pharmaceutical firms or other -- but in terms of how they do these DSMBs, 32 00:02:00,787 --> 00:02:03,990 but they will make recommendations for the best practice. 33 00:02:03,990 --> 00:02:07,160 Same thing for how to do a statistical analysis, 34 00:02:07,160 --> 00:02:11,798 how to monitor your trial, the FDA has a lot of guidance documents. 35 00:02:11,798 --> 00:02:15,168 If you haven't found those, look online, very useful. 36 00:02:15,168 --> 00:02:19,939 I did want to point out also for those who are at NIH, 37 00:02:19,973 --> 00:02:24,777 it was interesting that there's, for quite a while, sort of understanding 38 00:02:24,777 --> 00:02:29,983 that the centers and institutes differ in terms of how they do DSMBs. 39 00:02:29,983 --> 00:02:36,389 And some of them are a lot more open to the idea of the study sponsor 40 00:02:36,389 --> 00:02:40,793 being a heavy participant in the DSMB and the executive session. 41 00:02:40,793 --> 00:02:42,795 And some complaints have lodged 42 00:02:42,795 --> 00:02:46,599 some investigators about an interference or conflict of interest. 43 00:02:46,599 --> 00:02:51,204 And so, the Health and Human Services Office of the Inspector 44 00:02:51,204 --> 00:02:56,409 General took a couple years, I think in 2009 or 2010, and reviewed 45 00:02:56,409 --> 00:03:00,413 all the DSMBs across institutes and centers and everything, investigators, 46 00:03:00,413 --> 00:03:04,817 DSMB members, the sponsor and just asked some set of questions 47 00:03:04,817 --> 00:03:08,321 to try to evaluate whether these DSMBs were conducted 48 00:03:08,321 --> 00:03:12,926 in a way that met NIH's own policy and if there any challenges 49 00:03:12,926 --> 00:03:18,097 to the way things are currently being done in terms of the DSMB effectiveness. 50 00:03:18,097 --> 00:03:20,767 And this makes for a good reading. 51 00:03:20,767 --> 00:03:24,671 It's a brief report, but you can see there are recommendations, 52 00:03:24,671 --> 00:03:28,608 but there's sort of a back and forth between NIH officials 53 00:03:28,608 --> 00:03:32,512 and the Health and Human Services in terms of what recommendations 54 00:03:32,512 --> 00:03:37,116 or assessments of what was needed and not needed, particularly on the issue 55 00:03:37,116 --> 00:03:40,320 of that independence, actually, is the real sticking point, 56 00:03:40,320 --> 00:03:44,958 I think, between some of the institutes and those outside of the NIH. 57 00:03:46,259 --> 00:03:49,829 But the main findings were that the NIH's DSMBs 58 00:03:49,829 --> 00:03:53,833 didn't meet the guidelines overwhelmingly across all those and few 59 00:03:53,833 --> 00:03:57,003 -- you know, there's a few dissenting opinions, 60 00:03:57,003 --> 00:04:00,173 but quite the number of that said across 61 00:04:00,173 --> 00:04:04,177 investigators through the members that the policies were being met. 62 00:04:04,177 --> 00:04:07,347 But there were complaints particularly from certain investigators 63 00:04:07,347 --> 00:04:10,516 about participation of study staff in the close 64 00:04:10,516 --> 00:04:14,487 meetings was not, in all circumstances, seemed to be helpful. 65 00:04:14,487 --> 00:04:18,891 But in other circumstances, it was pointed that it was helpful 66 00:04:18,891 --> 00:04:22,061 because they were experts in certain rare diseases. 67 00:04:22,061 --> 00:04:27,233 They were the study staff that think about these issues, day and night. 68 00:04:27,233 --> 00:04:31,204 So, I think maybe one size does not fit all, 69 00:04:31,204 --> 00:04:36,576 but that is still room for debate in how we do these things. 70 00:04:36,576 --> 00:04:41,547 I think another issue is that we do these reports as masked, 71 00:04:41,547 --> 00:04:43,950 and there wasn't really good understanding 72 00:04:43,950 --> 00:04:47,920 amongst all members of the DSMB themselves that were interviewed 73 00:04:47,920 --> 00:04:52,325 that they had the right to ask for the unblinding codes. 74 00:04:52,325 --> 00:04:57,096 And so, the -- there was as recommendation to avoid that confusion 75 00:04:57,096 --> 00:05:01,067 and certainly to make sure this was an accepted policy 76 00:05:01,067 --> 00:05:04,671 that have stayed in the policies in all centers 77 00:05:04,671 --> 00:05:08,641 that the DSMB has the right to see unmasked data, 78 00:05:08,641 --> 00:05:13,613 to make sure that's absolutely clear that was historically seen as a challenge. 79 00:05:13,613 --> 00:05:15,748 Some DSMB members had no idea. 80 00:05:15,748 --> 00:05:19,152 And then it ultimately -- what's really interesting about this 81 00:05:19,152 --> 00:05:22,722 field is it's incredibly complex, so when you see someone's resume 82 00:05:22,722 --> 00:05:27,593 that they were the one on the cancer trial and you're doing a cardiovascular trial, 83 00:05:27,593 --> 00:05:30,830 you're going to want the same people on your DSMB. 84 00:05:30,830 --> 00:05:34,734 And so, you have the same experts serving on the HIV trials, 85 00:05:34,734 --> 00:05:35,702 the cardiovascular trials. 86 00:05:35,702 --> 00:05:38,371 And it's an Asian population of experts. 87 00:05:38,371 --> 00:05:42,575 When you see them talk and there's a lot of concern 88 00:05:42,575 --> 00:05:47,513 that this expertise is going to disappear and that we don't have enough 89 00:05:47,513 --> 00:05:51,351 people who are new that are engaging in this activity, 90 00:05:51,351 --> 00:05:55,922 and so there's active efforts across NIH and other sponsored trial settings 91 00:05:55,922 --> 00:06:01,828 to get new people into this endeavor, which as you can imagine is quite complex. 92 00:06:01,828 --> 00:06:06,566 So, at NIH, it was recommended they should direct the ICs, 93 00:06:06,566 --> 00:06:10,370 the Institutes and Centers to at least articulate the circumstances 94 00:06:10,370 --> 00:06:14,941 in which the Institutes and Centers staff can participate in the DSMB. 95 00:06:15,141 --> 00:06:19,779 So, at least everyone knows what to expect and at least, in those circumstances, 96 00:06:19,779 --> 00:06:24,083 understand if a particular IC member is not abiding by their own restrictions. 97 00:06:24,083 --> 00:06:26,219 So, that was seen as agreed. 98 00:06:26,219 --> 00:06:30,022 Also, it was -- you know, a lot of them agreed 99 00:06:30,022 --> 00:06:34,994 that they would make it more clear that all DSMBs have -- that the policies 100 00:06:34,994 --> 00:06:38,297 specifically say that the DSMBs have access to unmasked data 101 00:06:38,297 --> 00:06:41,901 that was an easy fix and that they're all 102 00:06:41,901 --> 00:06:45,138 working on ways to identify in recruiting members. 103 00:06:45,138 --> 00:06:50,143 So, with this sort of negotiating set of recommendations, the NIH agreed. 104 00:06:50,143 --> 00:06:53,579 So, talking about earlier that you -- it's 105 00:06:53,579 --> 00:06:56,783 not always needed to have an independent body. 106 00:06:56,783 --> 00:07:00,019 And you can imagine it's quite a complex 107 00:07:00,019 --> 00:07:04,424 and expensive endeavor to get a group of experts like this 108 00:07:04,424 --> 00:07:08,461 and fly them around the world to have these meetings. 109 00:07:09,962 --> 00:07:12,999 They are -- so universally except for this. 110 00:07:12,999 --> 00:07:14,167 Important and needed 111 00:07:14,167 --> 00:07:18,371 when you have large randomized trials with mortality or major morbidity. 112 00:07:18,371 --> 00:07:21,808 Certainly, if the trial, for whatever reasons, has implications 113 00:07:21,808 --> 00:07:25,645 for public health, you're going to want an independent body 114 00:07:25,645 --> 00:07:29,248 to help you through the conducting of this trial. 115 00:07:29,248 --> 00:07:33,653 If there's a serious toxicity, that's thought to be a risk 116 00:07:33,653 --> 00:07:37,490 or potentially, for whatever reasons, other kinds of high-risk treatments, 117 00:07:37,490 --> 00:07:40,560 then people generally, even if it's not blinded, 118 00:07:41,761 --> 00:07:46,866 might think about having a DSMB has an independent advisory board 119 00:07:46,866 --> 00:07:52,405 to help make decisions about what might be an ethically complex situation. 120 00:07:52,405 --> 00:07:55,675 And generally, if there's a vulnerable population, 121 00:07:55,675 --> 00:08:01,214 whether be children or pregnant women or anybody you might find vulnerable, 122 00:08:01,214 --> 00:08:04,450 you know, folks with Alzheimer's or dementia, 123 00:08:04,450 --> 00:08:08,621 these are the populations that would need extra protections. 124 00:08:08,621 --> 00:08:10,456 When they're not needed, 125 00:08:11,791 --> 00:08:12,592 typically, if 126 00:08:12,592 --> 00:08:17,597 you don't have any of those ethical tricky or sinuous and difficult situations, 127 00:08:17,597 --> 00:08:21,834 but maybe run-of-the-mill, you know, allergy medicines but in their -- 128 00:08:21,834 --> 00:08:26,072 so not anything unusual in terms of risk being taken, single-arm 129 00:08:26,072 --> 00:08:27,406 trials not blinded. 130 00:08:27,406 --> 00:08:31,844 There's really no reason to have an independent body if everyone 131 00:08:31,844 --> 00:08:36,649 can see they have totality of evidence as people accrue short-term trials. 132 00:08:36,649 --> 00:08:40,052 It's almost impossible to get a DSMB together 133 00:08:40,052 --> 00:08:45,992 if a trial's going to be quite quick in terms of accrual at endpoints. 134 00:08:45,992 --> 00:08:47,693 This is your point. 135 00:08:47,693 --> 00:08:49,395 You can't stop early. 136 00:08:49,395 --> 00:08:52,565 So, what is DSMB going to do? 137 00:08:52,565 --> 00:08:58,704 And certainly, I think, when you have that unblinded data with no ethical reason 138 00:08:58,704 --> 00:09:02,942 to be extra concerned, there's really no need for DSMB. 139 00:09:02,942 --> 00:09:04,644 And there are disadvantages. 140 00:09:04,644 --> 00:09:08,247 So, a lot of people will try to be better safe than sorry, 141 00:09:08,247 --> 00:09:10,750 but it increases the complexity of the trial management. 142 00:09:10,750 --> 00:09:14,654 Once you say you're going to do these DSMB meetings, they have to happen 143 00:09:14,654 --> 00:09:15,621 and increases costs. 144 00:09:15,621 --> 00:09:18,824 And so, I think, the thought is that you should -- 145 00:09:18,824 --> 00:09:19,659 and to think 146 00:09:19,659 --> 00:09:23,829 we have sort of a small set of individuals who can serve on these boards 147 00:09:23,829 --> 00:09:27,433 that have expertise in your area, don't use them up in a situation 148 00:09:27,433 --> 00:09:29,669 where they're not really needed and keep them 149 00:09:29,669 --> 00:09:32,705 fresh and available for situations where you really do need them. 150 00:09:32,705 --> 00:09:34,840 So, I think that's another important 151 00:09:34,840 --> 00:09:38,711 reason, especially in smaller fields, to think about whether you need 152 00:09:38,711 --> 00:09:43,282 -- you don't have one if you don't think you need one. 153 00:09:43,282 --> 00:09:46,419 Because you have many other forms of monitoring, 154 00:09:46,419 --> 00:09:51,357 you can have your own local safety monitors, perhaps some kind of partial -- 155 00:09:51,357 --> 00:09:54,860 you know, if you're not going to be monitoring efficacy, 156 00:09:54,860 --> 00:09:57,697 but you want extra experts to monitor safety, 157 00:09:57,697 --> 00:10:00,866 then you can recruit those specific safety monitoring board 158 00:10:00,866 --> 00:10:04,904 without all the other extra goals that the DSMB typically has. 159 00:10:04,904 --> 00:10:08,174 You have the IRB, the Institutional Review Board, that's 160 00:10:08,174 --> 00:10:11,844 going to look at the human subjects at your institution. 161 00:10:11,844 --> 00:10:15,147 And the protocol team themselves should be constantly reviewing 162 00:10:15,147 --> 00:10:20,486 how their study is going and the safety of the patients in their study. 163 00:10:20,486 --> 00:10:25,057 So, as promised, I just want to go from those big concepts 164 00:10:25,057 --> 00:10:28,361 to some just not some balls of example tables 165 00:10:28,361 --> 00:10:32,031 or shells or things that I've seen in monitoring reports. 166 00:10:32,031 --> 00:10:33,132 Many of you, 167 00:10:33,132 --> 00:10:37,169 if you're familiar with clinical search, know about the consort diagram. 168 00:10:37,169 --> 00:10:41,774 I think it's useful to see it in the DSMB report, 169 00:10:41,774 --> 00:10:45,978 because then you see how people are being screened, approached, 170 00:10:45,978 --> 00:10:51,017 and how many of those are accepting or being -- or accepting 171 00:10:51,017 --> 00:10:55,221 and then failing the eligibility, continuing later for other reasons. 172 00:10:55,221 --> 00:10:58,157 It gives you an idea of feasibility 173 00:10:58,157 --> 00:11:01,927 in representation and then also a great concise way 174 00:11:01,927 --> 00:11:07,400 to look at things like of those that should've been analyzed, what percent 175 00:11:07,400 --> 00:11:11,604 were withdrawn, lost to follow-up, got discontinued early, or completed. 176 00:11:11,604 --> 00:11:14,940 All those things, I think, are very useful 177 00:11:14,940 --> 00:11:18,310 and quite concisely summarized in a consult diagram. 178 00:11:20,012 --> 00:11:21,213 Going back to validity, you 179 00:11:21,213 --> 00:11:23,382 know, are you doing the protocol that you intended, 180 00:11:23,382 --> 00:11:25,084 are you going to have interoperable results? 181 00:11:25,084 --> 00:11:27,019 You definitely want to look at missed visits, 182 00:11:27,019 --> 00:11:30,623 you want to look at them by visits, you want to look at the baseline. 183 00:11:30,623 --> 00:11:33,526 Certainly, if they're not even getting through the baseline, that's a concern. 184 00:11:33,526 --> 00:11:36,429 And you want to see how many are completing the last visit. 185 00:11:37,329 --> 00:11:40,966 Overdue forms, they might give you an idea of whether or not 186 00:11:40,966 --> 00:11:44,603 your protocols are a little too difficult in terms of the schedule. 187 00:11:44,603 --> 00:11:48,240 People are willing to do it but can't quite make the timelines. 188 00:11:48,240 --> 00:11:50,976 It's useful information, it might also be very fixable. 189 00:11:50,976 --> 00:11:52,278 Baseline characteristics, are these 190 00:11:52,278 --> 00:11:55,514 the people you intended to enroll, are they balanced by arm? 191 00:11:57,116 --> 00:11:58,184 If randomization worked, 192 00:11:58,184 --> 00:12:02,421 these things should be balanced by arm, but randomization doesn't always work 193 00:12:02,421 --> 00:12:06,492 and sometimes it's been discovered in meetings like this by DSMB. 194 00:12:06,492 --> 00:12:07,727 Serious adverse events. 195 00:12:07,727 --> 00:12:09,829 In many ways, that's a no-brainer. 196 00:12:09,829 --> 00:12:12,665 Obviously, DSMB is going to look at that. 197 00:12:12,665 --> 00:12:17,269 But I think what's important is how you present it to the DSMB. 198 00:12:17,269 --> 00:12:20,806 You know, if you have 4,000 people in the trial 199 00:12:20,806 --> 00:12:24,343 and you're going to present lineless things of every sniffle 200 00:12:24,343 --> 00:12:27,513 and headache, we'll never find the issues of concern. 201 00:12:28,380 --> 00:12:30,015 And that is a concern. 202 00:12:30,015 --> 00:12:33,152 And so, you want to be complete, maybe have appendices 203 00:12:33,152 --> 00:12:37,289 of all the events in case somebody asks a question about a particular one. 204 00:12:37,289 --> 00:12:40,559 But you're going to want to think how to summarize this 205 00:12:40,559 --> 00:12:43,796 so the necessary trends can become apparent if there are any. 206 00:12:43,796 --> 00:12:47,366 And then you want appropriate detail, so at the fingertips of start 207 00:12:47,366 --> 00:12:51,804 and stop dates, how long did it take for these AEs to resolve, did they resolve. 208 00:12:51,804 --> 00:12:55,608 People often ask what is the relationship do they think of this adverse event. 209 00:12:55,608 --> 00:12:57,810 Something happened to the person on the trial, 210 00:12:57,810 --> 00:13:01,881 they fell down the stairs, you know, whatever it is, it has to be reported 211 00:13:01,881 --> 00:13:04,617 if there's an injury to a patient in your trial. 212 00:13:04,950 --> 00:13:08,020 And you know, sometimes it's quite obvious that that maybe perhaps 213 00:13:08,020 --> 00:13:11,957 -- maybe it is obvious, maybe it's not, maybe they fell down the stairs 214 00:13:11,957 --> 00:13:15,394 because they were dizzy, and it had to do with the treatment. 215 00:13:15,394 --> 00:13:18,130 So, essentially, you can ask for the expert opinion 216 00:13:18,130 --> 00:13:19,532 the investigators often will list 217 00:13:19,532 --> 00:13:21,767 what they think is whether it's probably related, 218 00:13:21,767 --> 00:13:23,869 possibly related, definitely related, or unrelated. 219 00:13:23,869 --> 00:13:25,171 But ultimately, you want 220 00:13:25,171 --> 00:13:29,074 it presented to the DSMB, so they can make their own opinions. 221 00:13:29,074 --> 00:13:31,010 But the opinion of the investigator 222 00:13:31,010 --> 00:13:35,214 who was an expert is very -- you know, it's useful to have 223 00:13:35,214 --> 00:13:39,618 but it should also just be considered or not thought of as absolute. 224 00:13:39,618 --> 00:13:43,656 But you'll -- so, you also then want to start to summarize 225 00:13:43,656 --> 00:13:47,960 these maybe by body system if these can happen multiple times, 226 00:13:47,960 --> 00:13:52,731 perhaps you count the number of individuals who've had at least one event, 227 00:13:52,731 --> 00:13:57,102 the timing of the first event, the seriousness of the worst event. 228 00:13:57,102 --> 00:13:58,370 Things like that. 229 00:13:58,370 --> 00:14:02,608 The fundamental concept here is that the summary should be able 230 00:14:02,608 --> 00:14:06,979 to easily present -- allow trends to just arise to the top. 231 00:14:06,979 --> 00:14:09,915 So, something that's visually easy to look at. 232 00:14:09,915 --> 00:14:13,919 And in this particular one, there's not a lot of interesting 233 00:14:13,919 --> 00:14:17,590 data here, but you might think about as a percent 234 00:14:17,590 --> 00:14:22,695 of the total number who were treated, how many had any infection or sinusitis. 235 00:14:22,695 --> 00:14:27,466 So, you might start with the sum category and then the individual categories 236 00:14:27,466 --> 00:14:30,569 of those with at least to 10 percent rate, 237 00:14:30,569 --> 00:14:34,740 and then list them both as overall and then by worst grade. 238 00:14:34,740 --> 00:14:36,275 And often times, DSMB, 239 00:14:36,275 --> 00:14:40,913 a failing system is something that they want to see last for it. 240 00:14:40,913 --> 00:14:45,417 And so, in addition, of course, if this is part of the plan, 241 00:14:45,417 --> 00:14:49,889 you would be looking at efficacy generally for long trials or risky trials 242 00:14:49,889 --> 00:14:54,059 or trials for people who might stand to have a huge benefit 243 00:14:54,693 --> 00:14:56,295 you might think about, you know, monitoring. 244 00:14:56,295 --> 00:14:57,096 And so, you 245 00:14:57,096 --> 00:15:01,033 might look at the 30 percent of treatment success in the area of the curve, 246 00:15:01,033 --> 00:15:02,101 the Kaplan Meier curve, 247 00:15:02,101 --> 00:15:04,203 and then you would have your official boundaries 248 00:15:04,203 --> 00:15:06,305 if you are doing efficacy or futility monitoring. 249 00:15:06,305 --> 00:15:08,674 And you would just simply look at the boundary 250 00:15:08,674 --> 00:15:11,844 in where you were relative to the boundary and probably the whole history, 251 00:15:12,444 --> 00:15:15,114 so you can kind of say how the data are trending. 252 00:15:15,114 --> 00:15:18,250 The basic concepts are you want to make it as easy as possible 253 00:15:18,250 --> 00:15:19,852 for the DSMB to do their job. 254 00:15:19,852 --> 00:15:21,954 So, the report should be provided to the DSMB 255 00:15:21,954 --> 00:15:24,490 in enough time for their view to happen -- you know, 256 00:15:24,490 --> 00:15:27,960 they should be able to look at it before the meeting and think about it. 257 00:15:28,260 --> 00:15:30,596 Don't email it the night before. 258 00:15:30,596 --> 00:15:34,800 The report should be including concise summary of the trial itself. 259 00:15:34,800 --> 00:15:38,671 Who knows how many boards these DSMB members are own? 260 00:15:38,671 --> 00:15:42,908 Typically, they serve multiple because of the expertise that they have. 261 00:15:42,908 --> 00:15:47,880 So, include that schema, that one-page summary that you have in your protocol, 262 00:15:47,880 --> 00:15:50,582 telling you the basic endpoints, the number 263 00:15:50,582 --> 00:15:54,053 to enroll, the primary objective that stores analysis plan. 264 00:15:54,053 --> 00:15:58,290 You should have an executive summary that processes the main points 265 00:15:58,290 --> 00:16:01,727 of what the -- say, for the open report 266 00:16:01,727 --> 00:16:04,830 in any case where you are in accrual 267 00:16:04,830 --> 00:16:08,267 and how many new AEs since the last meeting. 268 00:16:08,267 --> 00:16:09,435 Things like that. 269 00:16:09,435 --> 00:16:12,604 And the tables and figures should be concise, 270 00:16:12,604 --> 00:16:15,774 easy to understand, and of course, accurate. 271 00:16:15,774 --> 00:16:17,676 So, you know, hopefully, 272 00:16:17,676 --> 00:16:21,880 I've given you some information today that you can appreciate. 273 00:16:21,880 --> 00:16:25,684 The DSMBs, they do have a very difficult job 274 00:16:25,684 --> 00:16:29,054 ahead of them and a very important job. 275 00:16:29,054 --> 00:16:33,292 They have to balance individual ethics and collective ethics, risk/benefits. 276 00:16:33,292 --> 00:16:37,496 They have to think about statistically technical things in terms 277 00:16:37,496 --> 00:16:41,467 of being able to interpret and/or monitoring efficacy boundaries and 278 00:16:41,967 --> 00:16:43,402 inflations of false-positive rates. 279 00:16:43,402 --> 00:16:47,206 Sometimes, they need to consider futility and the balance 280 00:16:47,206 --> 00:16:51,810 of what can be gained if you continue versus not continuing. 281 00:16:52,978 --> 00:16:55,981 And most importantly, something that sometimes gets lost, 282 00:16:55,981 --> 00:16:59,018 which is the decisions are not purely statistically 283 00:16:59,018 --> 00:17:02,054 and in generally are always taking into account 284 00:17:02,054 --> 00:17:06,592 much more than a simple boundary or one statistic and are often 285 00:17:06,592 --> 00:17:08,093 very complex in terms 286 00:17:08,093 --> 00:17:12,664 of how many different aspects you might consider in terms of overall 287 00:17:12,664 --> 00:17:16,802 -- if a trial should continue as planned or some alteration. 288 00:17:17,403 --> 00:17:21,306 And finally, I always like to emphasize -- because sometimes investigators, 289 00:17:21,306 --> 00:17:25,244 I think, just want this weight to have decision making on 290 00:17:25,244 --> 00:17:26,311 somebody else's shoulders. 291 00:17:26,311 --> 00:17:29,882 But the DSMB is there to offer advice and guidance. 292 00:17:29,882 --> 00:17:33,085 But it is the investigator and the sponsor ultimately 293 00:17:33,085 --> 00:17:36,121 who bear the responsibility of stopping or going. 294 00:17:36,121 --> 00:17:38,057 Here are some future readings. 295 00:17:38,057 --> 00:17:42,694 Say, if like those that I mentioned would be like suspense novels, read 296 00:17:42,694 --> 00:17:46,265 the Data Monitoring and Clinical Trials: A Case Studies Approach. 297 00:17:46,665 --> 00:17:50,035 First author, Dave DeMets, who is a great storyteller 298 00:17:50,035 --> 00:17:53,806 and who is involved in a lot of landmark trials 299 00:17:53,806 --> 00:17:57,943 and interesting DSMBs with just unexpected swings and curveballs and does 300 00:17:57,943 --> 00:18:02,848 a good job explaining lessons learned as well as telling these good stories. 301 00:18:02,848 --> 00:18:06,985 A really informative book as well as the one by Susan 302 00:18:06,985 --> 00:18:10,722 Ellenberg in this overall preference by Friedman, Furberg, and DeMets. 303 00:18:10,722 --> 00:18:14,860 And the other two here at the end are statistical references. 304 00:18:14,860 --> 00:18:18,764 If there are any statisticians out there who want to know 305 00:18:18,931 --> 00:18:20,999 about the nuts and bolts of these boundaries 306 00:18:20,999 --> 00:18:23,802 and other statistical issues, those are both very good references. 307 00:18:23,802 --> 00:18:25,270 So, thank you very much. 308 00:18:25,270 --> 00:18:28,774 Let's just stop the time for a few questions if you have any. 309 00:18:28,774 --> 00:18:31,977 Yes, and I will try to remember myself to repeat the questions. 310 00:18:31,977 --> 00:18:36,248 I know no one else other than the 20 people in this room can hear you. 311 00:18:36,248 --> 00:18:37,082 So, go ahead. 312 00:18:38,183 --> 00:18:40,319 Do you ever have 313 00:18:40,319 --> 00:18:44,590 situations where the DSMB can't reach a consensus 314 00:18:44,590 --> 00:18:48,861 if any member, experts have disagreements among themselves? 315 00:18:49,194 --> 00:18:49,495 Yeah. 316 00:18:49,495 --> 00:18:54,366 So, the question was, is there ever -- if I was sort of aware of situations 317 00:18:54,366 --> 00:18:55,267 where the DSMB 318 00:18:55,267 --> 00:18:59,204 themselves can't come to a consensus and -- yeah, that, absolutely, can happen. 319 00:18:59,204 --> 00:19:01,039 Obviously, you think about, you know, 320 00:19:01,039 --> 00:19:04,977 if you can have a home jury, you can have a home DSMB. 321 00:19:04,977 --> 00:19:10,449 And you know, honestly, I think about -- a lot of times in the charter, there is -- 322 00:19:10,449 --> 00:19:15,621 there are rules such as -- you know, first of all, if you want to think about 323 00:19:15,621 --> 00:19:19,558 how many members is the reasonable amount for it to be a quorum, 324 00:19:19,558 --> 00:19:23,195 because it's impossible to make sure that everyone gets to every meetings. 325 00:19:23,195 --> 00:19:24,429 So, the first challenge 326 00:19:24,429 --> 00:19:28,333 is to make sure you have enough people at that meeting to vote. 327 00:19:28,333 --> 00:19:32,271 And others then will say the decision -- there might be a majority/minority 328 00:19:32,271 --> 00:19:35,307 and they simply -- they will report to the sponsor, 329 00:19:35,307 --> 00:19:38,010 the outcome of the vote that it wasn't unanimous. 330 00:19:38,010 --> 00:19:42,114 So, you have -- like the Supreme Court, you have the dissenting opinion. 331 00:19:42,114 --> 00:19:43,448 Because this is advice-giving, 332 00:19:43,448 --> 00:19:47,085 in some sense, there's no absolute reason other than it would be 333 00:19:47,085 --> 00:19:50,656 nice for the sponsor and the investigator if the DSMB could come 334 00:19:50,889 --> 00:19:52,124 with a clear recommendation. 335 00:19:52,124 --> 00:19:55,093 But you know, the answer might not be clear. 336 00:19:55,093 --> 00:19:58,430 And so, the importance is to present the two opinions. 337 00:19:58,430 --> 00:20:03,068 Sometimes, you can have a member -- and I've certainly heard -- you know, 338 00:20:03,068 --> 00:20:07,406 it's a bit like social engineering, like, you people need to get along. 339 00:20:07,406 --> 00:20:10,709 You need to think about, I think, the personalities involved 340 00:20:10,709 --> 00:20:13,712 and can they work together in a difficult situation. 341 00:20:13,745 --> 00:20:14,713 And it's not that 342 00:20:14,713 --> 00:20:17,849 they're -- it's not -- it's difficult because it's a challenging job ahead. 343 00:20:17,849 --> 00:20:19,985 And so, if you have a very opinionated person 344 00:20:19,985 --> 00:20:22,621 who is not every going to listen to anyone else's opinion, 345 00:20:22,621 --> 00:20:25,023 that's not a great person to have on the DSMB. 346 00:20:25,023 --> 00:20:27,192 So, there have been times when people have been 347 00:20:27,192 --> 00:20:30,329 -- essentially been asked to leave, or it's been uncomfortable enough for them 348 00:20:30,329 --> 00:20:31,263 that people just resign. 349 00:20:31,263 --> 00:20:34,700 They just, like, know it's not going to work out and then replaced. 350 00:20:34,700 --> 00:20:37,202 It's because they can't function well as a body. 351 00:20:37,202 --> 00:20:40,205 So, you know, a lot of times, I know at NIAID, 352 00:20:40,205 --> 00:20:44,409 there was one person whose job it was to think about how to make the DSMB. 353 00:20:44,409 --> 00:20:48,080 You know, she was in the division -- essentially in the division of AIDS, 354 00:20:48,113 --> 00:20:52,117 so anything related to infectious disease, you know, there was -- at NIAID, 355 00:20:52,117 --> 00:20:53,652 they have separate divisions, say, 356 00:20:53,652 --> 00:20:57,689 one for transplant or allergies, someone's job was to focus on infectious disease. 357 00:20:57,689 --> 00:21:02,027 And she spent -- she networks, she knew people who serve in these bodies, 358 00:21:02,027 --> 00:21:06,031 and she would kind of create these bodies and think about, "That person's 359 00:21:06,031 --> 00:21:10,969 not going to get along with that person," which is kind of stupid in some sense, 360 00:21:10,969 --> 00:21:14,339 you know, we're scientists, but at the same time, 361 00:21:14,339 --> 00:21:18,076 it's like you guys have to work -- that group 362 00:21:18,076 --> 00:21:22,214 has to work together and personalities inevitably will come into play. 363 00:21:22,214 --> 00:21:22,948 Yeah? 364 00:21:23,115 --> 00:21:26,952 Is there a waiver or a challenge process 365 00:21:26,952 --> 00:21:30,289 for somebody, you know, go back -- 366 00:21:30,289 --> 00:21:35,560 the investigator can go back and perhaps challenge what they said? 367 00:21:36,295 --> 00:21:36,628 Yeah. 368 00:21:36,628 --> 00:21:41,066 So, the question was, "Is there some kind of waiver or challenge process 369 00:21:41,066 --> 00:21:42,901 that investigator can essentially challenge 370 00:21:42,901 --> 00:21:45,871 the recommendations of the DSMB?" And I can say, 371 00:21:45,871 --> 00:21:47,939 I definitely was in a situation. 372 00:21:47,939 --> 00:21:49,975 I can't remember all the specifics. 373 00:21:49,975 --> 00:21:53,045 I was on a study team, not the DSMB, 374 00:21:53,045 --> 00:21:56,148 and the DSMB mad a recommendation about the treatment. 375 00:21:56,148 --> 00:22:00,519 I think it had to do with uncomfortable number of AEs. 376 00:22:00,519 --> 00:22:02,087 But the investigator presented 377 00:22:02,087 --> 00:22:06,058 an opposing opinion, "You know, these AE is very treatable, 378 00:22:06,058 --> 00:22:09,995 this disease was very dire," they came up with something 379 00:22:09,995 --> 00:22:13,965 that gave the DSMB more information to process the data. 380 00:22:13,965 --> 00:22:19,471 And so, there was a series of letters that were brought back and forth. 381 00:22:19,504 --> 00:22:23,308 And then ultimately DSMB should've stayed when they were more comfortable with 382 00:22:23,308 --> 00:22:27,412 -- I think the DSMB might even recommended stopping and then they didn't. 383 00:22:27,412 --> 00:22:31,850 And they gave reasons why, and then everyone was comfortable and then went on. 384 00:22:31,850 --> 00:22:34,386 And that study was positive in the end. 385 00:22:34,386 --> 00:22:36,755 They were on the New England journal. 386 00:22:36,755 --> 00:22:39,124 So, you know, the DSMB offers advice. 387 00:22:39,458 --> 00:22:45,430 I think if there was a DSMB that felt very strongly and the WHI -- I don't know. 388 00:22:45,430 --> 00:22:48,400 To think about that itself needs a whole lecture. 389 00:22:48,400 --> 00:22:51,036 It's hard to just dabble into that example. 390 00:22:51,036 --> 00:22:54,473 But do know that the first DSMB recommended stopping and then 391 00:22:54,873 --> 00:22:58,977 the sponsors, for whatever the reasons, they didn't want to, and they continued. 392 00:22:58,977 --> 00:23:00,612 And they got a whole 393 00:23:00,612 --> 00:23:04,583 another body and it's -- so that's the sponsor's prerogative, you know. 394 00:23:04,583 --> 00:23:06,251 But they have public opinion. 395 00:23:06,251 --> 00:23:11,056 They have the scientific community that in the end will judge all of these actions. 396 00:23:11,056 --> 00:23:12,991 And the sponsor's only responsibility ultimately 397 00:23:12,991 --> 00:23:16,995 is to listen to the DSMB, but they are the decision makers. 398 00:23:16,995 --> 00:23:21,299 So, the idea of challenging the -- you know, I think the investigators 399 00:23:21,299 --> 00:23:23,535 want to make sure the DSMB understands, 400 00:23:23,535 --> 00:23:27,372 that they don't want to feel like they're ignoring the DSMB's advice. 401 00:23:27,372 --> 00:23:30,575 They want to continue together to continue on the trial. 402 00:23:30,876 --> 00:23:34,646 So, a lot of times, if you're not going to stop as recommended, 403 00:23:34,646 --> 00:23:38,016 the investigators like to explain why and get the DSMB on board 404 00:23:38,216 --> 00:23:41,019 so they can continue in a congenial and agreeable manner. 405 00:23:42,487 --> 00:23:45,157 How's the DSMB financed in related to their expenses? 406 00:23:45,257 --> 00:23:49,327 So, the question is about how are these things financed? 407 00:23:49,327 --> 00:23:52,998 Yes, the DSMB, I believe there is an honorarium 408 00:23:52,998 --> 00:23:55,834 for the meetings that might require travel. 409 00:23:55,834 --> 00:23:57,903 Not always, I don't think. 410 00:23:57,903 --> 00:24:00,739 So, that is a little bit unclear. 411 00:24:00,739 --> 00:24:04,009 I've definitely been in situations where I received 412 00:24:04,009 --> 00:24:06,445 honorariums and situations where I haven't. 413 00:24:06,445 --> 00:24:09,281 So, there might be a small honorarium. 414 00:24:09,281 --> 00:24:14,186 And certainly, all their expenses in terms of travel, most people prefer, 415 00:24:14,186 --> 00:24:18,256 especially in maybe ethically challenging or complex trials in-person meetings, 416 00:24:18,256 --> 00:24:22,260 so all that travel expenses is all part of the grant of the study. 417 00:24:22,260 --> 00:24:24,429 The study sponsor covers all of that. 418 00:24:24,429 --> 00:24:28,867 And if the DSMB says we need an extra meeting and there's 12 of them 419 00:24:28,867 --> 00:24:33,171 coming from, you know, around the country, the sponsor has to cough up that money. 420 00:24:33,171 --> 00:24:38,043 So, they have to have, like, reserve of funds to be able to cover a DSMB -- 421 00:24:38,910 --> 00:24:43,048 and so, you make a guestimate like you do, say, 422 00:24:43,048 --> 00:24:46,818 if it's grant-funded study and potentially, if it's unexpected 423 00:24:46,818 --> 00:24:51,389 burden, perhaps a study team could go back to their sponsor, 424 00:24:51,389 --> 00:24:55,560 whether it's the NIH or if it's an industry sponsor 425 00:24:55,560 --> 00:25:00,131 and figure out a way to get supplemental funds if necessary. 426 00:25:00,131 --> 00:25:02,834 Patient safety is the primary responsibility. 427 00:25:02,834 --> 00:25:05,103 Are there any other question? 428 00:25:07,005 --> 00:25:07,639 All right. 429 00:25:07,639 --> 00:25:10,175 Well, if not, thank you for your attention. 430 00:25:10,175 --> 00:25:14,880 And I'll be here for extra minutes if anyone else wanted to come up offline 431 00:25:14,880 --> 00:25:15,814 and ask questions. 432 00:25:15,814 --> 00:25:18,016 Thank you very much for your time.