1 00:00:09,042 --> 00:00:10,777 So, I, next, wanted to 2 00:00:10,777 --> 00:00:14,914 talk a little bit about the actual study agent or the intervention. 3 00:00:14,914 --> 00:00:18,385 So, because this was written -- this particular protocol template 4 00:00:18,385 --> 00:00:22,555 is for either drugs or devices, they talk about the study agent. 5 00:00:22,555 --> 00:00:26,359 Other protocols certainly can be done and developed for behavioral interventions 6 00:00:26,359 --> 00:00:29,796 or for procedures like surgeries or other types of interventions. 7 00:00:29,796 --> 00:00:30,497 And certainly, 8 00:00:30,497 --> 00:00:33,600 you can have a protocol for observational studies too 9 00:00:33,600 --> 00:00:37,771 that talk about how you're going to collect the data over time, 10 00:00:37,771 --> 00:00:42,275 and you may not have an intervention at all in that particular protocol. 11 00:00:42,275 --> 00:00:46,746 In which case, these particular sections of the template would just be removed 12 00:00:46,746 --> 00:00:49,883 and stated as not applicable for that particular trial. 13 00:00:52,185 --> 00:00:54,454 So, you want to 14 00:00:54,454 --> 00:00:59,559 as much as possible define the treatment very clearly. 15 00:00:59,559 --> 00:01:05,799 So, with NIH, we've seen both, at NIH, the recent publications 16 00:01:05,799 --> 00:01:12,038 around enhancing rigor and reproducibility, and part of that is knowing 17 00:01:12,038 --> 00:01:16,576 what was the treatment that was actually delivered. 18 00:01:16,576 --> 00:01:21,681 And we've also seen, certainly in the psychology literature, 19 00:01:21,681 --> 00:01:26,786 the push for open science and transparency and publishing 20 00:01:26,786 --> 00:01:32,459 a priori hypotheses and protocols and other information like that. 21 00:01:32,759 --> 00:01:37,964 So, this is a trend that we're starting to see so that someone else could attempt 22 00:01:37,964 --> 00:01:42,836 -- if you find amazing results on your study that they could take your protocol 23 00:01:42,836 --> 00:01:44,771 and be able to replicate it. 24 00:01:44,771 --> 00:01:49,676 And if we're able to see the same results by a different group of investigators, 25 00:01:49,676 --> 00:01:53,246 recruiting the same types of patients and delivering the same intervention, 26 00:01:53,246 --> 00:01:56,182 then we have reproducible findings, which is really great, 27 00:01:56,182 --> 00:02:00,887 that means that any provider then could think that they would likely find 28 00:02:00,887 --> 00:02:05,391 the same results if they applied the same intervention in their offices. 29 00:02:05,391 --> 00:02:10,663 So, this is why it's so important to really specify and describe your treatment 30 00:02:10,663 --> 00:02:14,634 and how it's going to be delivered in your protocol itself. 31 00:02:14,634 --> 00:02:17,170 And it's a tradeoff between the generalizability 32 00:02:17,170 --> 00:02:20,440 of how people really deliver it versus the specificity 33 00:02:20,440 --> 00:02:24,410 of how it's going to be delivered in this specific trial. 34 00:02:25,278 --> 00:02:30,817 And so, you want to very clearly lay out all of these different elements 35 00:02:31,084 --> 00:02:36,222 including when can someone -- when would someone be withdrawn from the study, 36 00:02:36,222 --> 00:02:42,295 that it's not safe for them to continue or they need to get treatment elsewhere? 37 00:02:42,295 --> 00:02:48,401 When and how do you track deviations from your protocol itself in terms of them 38 00:02:48,401 --> 00:02:52,839 not completing the treatment as it was recommended in the study? 39 00:02:53,473 --> 00:02:57,877 And then making sure you track all concomitant medications, procedures, 40 00:02:57,877 --> 00:03:04,717 other kinds of interventions that people may have had over the course of the study. 41 00:03:04,717 --> 00:03:09,322 So, the interventions themselves, you want to give enough information 42 00:03:09,322 --> 00:03:15,495 that you -- like I said, someone else could repeat and deliver this intervention. 43 00:03:15,495 --> 00:03:22,535 So, from our perspective at NCCIH, we -- as I mentioned, we often used natural products. 44 00:03:22,936 --> 00:03:26,606 So, say, for example, a study is going to be 45 00:03:26,606 --> 00:03:30,843 looking at one of the commonly used products like fish oil. 46 00:03:30,843 --> 00:03:35,848 So, fish oil is one of the top-selling natural products in the US, 47 00:03:35,848 --> 00:03:39,152 but pretty much every single formulation that's available over 48 00:03:39,152 --> 00:03:43,957 the counter is different from the one next to it on the shelf. 49 00:03:43,957 --> 00:03:49,462 It -- they have differing amounts of the two primary fatty acids, EPA and DHA. 50 00:03:49,462 --> 00:03:52,799 They may have other oils in them as well. 51 00:03:53,733 --> 00:03:57,503 And so, a very -- if you're doing a study of fish oil, 52 00:03:57,503 --> 00:04:02,108 you want very clear specification as to what's in the product and what is your control? 53 00:04:02,108 --> 00:04:05,278 What kind of oil do you use as a control capsule? 54 00:04:05,278 --> 00:04:07,013 How much are you giving people? 55 00:04:07,013 --> 00:04:09,315 How many capsules and what's the total milligrams 56 00:04:09,315 --> 00:04:13,920 of each of the different types of EPA or DHA in the case of fish oil? 57 00:04:14,487 --> 00:04:16,522 What's the quality check on that? 58 00:04:16,522 --> 00:04:21,427 Is there an independent group that's done an assessment to assure that it actually has 59 00:04:21,427 --> 00:04:24,597 what is labelled in the -- on the bottle itself? 60 00:04:24,597 --> 00:04:28,701 Is there any way to assure that it's actually being taken by participants? 61 00:04:28,701 --> 00:04:34,707 Do we have a way to a blood test to see if their fatty acid levels shift over time? 62 00:04:34,974 --> 00:04:39,746 Do we have a memory cap on top of their bottle to see when they opened it 63 00:04:39,746 --> 00:04:42,815 and when they closed it so that we can actually track 64 00:04:42,815 --> 00:04:46,452 when they did that, and that they didn't bring us the empty bottle 65 00:04:46,452 --> 00:04:50,390 because they threw them all in the trash on their way up the stairs 66 00:04:50,390 --> 00:04:53,760 so that their pill count is zero in terms of remaining pills? 67 00:04:55,395 --> 00:04:56,162 So, there's 68 00:04:56,162 --> 00:05:01,401 a lot of elements to this in terms of really detailing all of these different pieces. 69 00:05:01,401 --> 00:05:04,737 And when your intervention is something like cognitive behavioral therapy, 70 00:05:04,737 --> 00:05:07,740 this gets even a little bit more complicated, right? 71 00:05:07,740 --> 00:05:08,408 Because now 72 00:05:08,408 --> 00:05:12,412 you have someone providing an intervention that's loosely -- that's hopefully manualized 73 00:05:12,412 --> 00:05:15,748 and that they're delivering the same way for all participants. 74 00:05:16,316 --> 00:05:19,485 How much variation do you allow in your study? 75 00:05:19,485 --> 00:05:24,457 And specifying that is really important so that somebody else can repeat the results 76 00:05:24,457 --> 00:05:27,293 if they -- if you find positive findings. 77 00:05:27,994 --> 00:05:31,197 So, we struggle with the word "dose" a lot. 78 00:05:31,197 --> 00:05:36,502 I think it's very easy, or I should say it's easier in studies of drugs 79 00:05:36,502 --> 00:05:40,406 where you have a single agent and you count the milligrams 80 00:05:40,406 --> 00:05:44,644 that you give to an individual, you may have a frequency question. 81 00:05:44,644 --> 00:05:48,915 "Does it need to be given once a day, twice a day, 82 00:05:48,915 --> 00:05:53,152 three times a day?" You may have questions about, "Is it delivered orally? 83 00:05:53,152 --> 00:05:56,122 Is it injected? Is it to be put in the eyes? 84 00:05:56,122 --> 00:06:00,760 Is it to be put in the ears?" There's a lot of different ways that you can 85 00:06:00,760 --> 00:06:02,795 deliver drugs depending on what they're for. 86 00:06:02,795 --> 00:06:07,033 But when it comes to behavioral interventions, like I said, it gets a little bit 87 00:06:07,033 --> 00:06:07,567 more complicated. 88 00:06:07,567 --> 00:06:08,668 So, how many sessions, 89 00:06:08,668 --> 00:06:13,039 how many times a week does someone go in if you're doing a study on yoga? 90 00:06:13,039 --> 00:06:14,941 Do they come in twice a week? 91 00:06:14,941 --> 00:06:19,545 How long is the session, is it 30 minutes, is it 60 minutes, is it 90 minutes? 92 00:06:19,545 --> 00:06:24,450 Are you asking them to practice at home on their own with the video that you gave them? 93 00:06:24,450 --> 00:06:25,818 So, what is the dose? 94 00:06:25,818 --> 00:06:28,020 Is it how much they practice per week? 95 00:06:28,020 --> 00:06:30,923 Is it how much they attended the classes per week? 96 00:06:30,923 --> 00:06:34,727 All of those things need to be specified so that you can determine 97 00:06:34,727 --> 00:06:36,763 when everyone completes, well, who was adherent? 98 00:06:36,763 --> 00:06:39,098 Who did what we asked them do to? 99 00:06:39,098 --> 00:06:43,469 And so, we want to have those sorts of definitions in our protocols as well. 100 00:06:46,939 --> 00:06:48,408 And, you know, there's 101 00:06:48,408 --> 00:06:53,446 many different ways to kind of slice and dice this, but I think it often 102 00:06:53,446 --> 00:06:57,550 comes down to the number of sessions, the frequency of the sessions 103 00:06:57,550 --> 00:07:01,320 and the durations of those sessions, and then the practice element 104 00:07:01,320 --> 00:07:06,659 is the other piece that comes into play with a lot of the behavioral interventions. 105 00:07:06,659 --> 00:07:10,930 So, the other elements that happen in studies that often get forgotten 106 00:07:10,930 --> 00:07:15,034 about are some of the common confounders that we see in study. 107 00:07:15,034 --> 00:07:21,474 And time is the most common of these, so we -- potential participants often call 108 00:07:21,474 --> 00:07:24,043 and have the impetuous to call 109 00:07:24,043 --> 00:07:28,781 to participate in a research study when things are really bad. 110 00:07:28,781 --> 00:07:34,787 It's finally gotten to a point that, "I need to do something about this. 111 00:07:34,787 --> 00:07:39,926 My eczema is really bad, it's really inflamed, it's really bothering me. 112 00:07:39,926 --> 00:07:42,929 I've got to do something about this. 113 00:07:42,962 --> 00:07:45,998 Or you know what, I read about a study. 114 00:07:45,998 --> 00:07:50,736 I'm going to give them a call and participate in that." Well, eczema 115 00:07:50,736 --> 00:07:53,439 is something that tends to wax and wane. 116 00:07:53,439 --> 00:07:59,545 It tends -- there tend to be flares and people can get better and the lesions can reduce. 117 00:07:59,545 --> 00:08:01,914 And that can just happen over time. 118 00:08:01,914 --> 00:08:06,652 So, if you have an uncontrolled study where you only have an evaluation pre-opposed 119 00:08:06,652 --> 00:08:11,057 of someone getting an intervention, can we reduce stress in individuals with eczema 120 00:08:11,290 --> 00:08:13,259 by having them meditate, for example. 121 00:08:13,259 --> 00:08:17,563 And everyone gets better over time and we don't have a comparison group 122 00:08:17,563 --> 00:08:22,502 that just came in and had a social support where they chatted with other people. 123 00:08:22,502 --> 00:08:27,106 And instead of doing meditation, they maybe discussed a book or they read -- 124 00:08:27,106 --> 00:08:30,209 you know, discussed current events or something like that. 125 00:08:30,209 --> 00:08:33,346 Might the social gathering also be a stress reliever? 126 00:08:33,613 --> 00:08:37,817 Might that also have an improvement in their symptoms over time? 127 00:08:37,817 --> 00:08:43,155 And might patients just get better with time itself compared to when things were 128 00:08:43,155 --> 00:08:48,127 at their potentially worse when they decided to actually call about the study? 129 00:08:48,127 --> 00:08:53,833 So, for conditions that tend to wax and wane like this, get better and worse, 130 00:08:53,833 --> 00:08:58,404 that's when comparison groups are particularly very important to include in studies. 131 00:08:58,404 --> 00:09:01,474 And it's -- randomization is our favorite friend 132 00:09:01,474 --> 00:09:05,645 in clinical trials because it balances the groups at the beginning. 133 00:09:06,045 --> 00:09:10,516 And so, randomizing to some sort of comparison group is often really important 134 00:09:10,516 --> 00:09:14,620 in studies of things like depression where symptoms get better and worse, 135 00:09:14,620 --> 00:09:16,355 and certainly things like eczema. 136 00:09:16,355 --> 00:09:17,723 And there's many conditions 137 00:09:17,723 --> 00:09:22,194 that you can think of that would fall into that sort of category. 138 00:09:25,164 --> 00:09:25,998 And one 139 00:09:25,998 --> 00:09:30,202 of the things that you also have to think about 140 00:09:30,202 --> 00:09:34,840 is whether your comparison group might actually be an active intervention. 141 00:09:34,840 --> 00:09:41,180 And so, sometimes we'll see in our studies that people will want to use exercise 142 00:09:41,180 --> 00:09:45,785 as a comparison for yoga, or for other forms of interventions. 143 00:09:45,785 --> 00:09:50,423 And the hypothesis that you're testing then is whether yoga, as 144 00:09:50,423 --> 00:09:55,061 a form of exercise, has added benefit over exercise by itself. 145 00:09:55,928 --> 00:09:59,899 And so, it's not, "Does yoga help compared to time and attention, 146 00:09:59,899 --> 00:10:05,171 or compared to usual care?" It's really, "Is it over and above what we would see 147 00:10:05,171 --> 00:10:09,442 with something like exercise itself?" So, the comparison group is really important 148 00:10:09,442 --> 00:10:13,412 as you're planning your studies and working on your study design aspects. 149 00:10:15,748 --> 00:10:20,553 And this is kind of -- I led into my next slide here. 150 00:10:20,553 --> 00:10:22,788 And what is the comparison group? 151 00:10:22,788 --> 00:10:26,092 And I often get this question from investigators, "Okay. 152 00:10:26,092 --> 00:10:29,061 I want to do a study of massage. 153 00:10:29,061 --> 00:10:30,930 What should be my control 154 00:10:30,930 --> 00:10:35,735 group?" And I say, "Well, what's the hypothesis that you want to test? 155 00:10:35,735 --> 00:10:37,603 Do you want to know 156 00:10:37,603 --> 00:10:42,408 whether a specific form of massage is better than another form of massage? 157 00:10:42,775 --> 00:10:47,046 Do you want to know if massage is better than -- 158 00:10:47,046 --> 00:10:51,751 for example, for knee, osteoarthritis, for knee pain due to joint degeneration? 159 00:10:51,751 --> 00:10:54,854 Is massage better than taking an over-the-counter NSAID 160 00:10:54,854 --> 00:10:59,725 to relieve pain?" Well, then that needs to be your comparison group. 161 00:10:59,725 --> 00:11:05,765 So, really focusing on what your hypothesis is will tremendously help you in defining what 162 00:11:05,765 --> 00:11:10,836 your comparison group is, and how you need to describe that comparison group. 163 00:11:12,505 --> 00:11:13,339 There are 164 00:11:13,339 --> 00:11:14,607 many different options 165 00:11:14,607 --> 00:11:20,046 and each of them change the hypothesis that's actually tested in the study 166 00:11:20,046 --> 00:11:25,084 as you shift the control group that's used in a particular trial. 167 00:11:25,084 --> 00:11:29,722 I'll make a note about placebos are often easier to mask 168 00:11:29,722 --> 00:11:33,092 and create for medication trials, for drug trials. 169 00:11:33,092 --> 00:11:37,897 They're much more difficult to create sham procedures or sham interventions. 170 00:11:37,897 --> 00:11:41,901 And so, we often see health education comparison groups 171 00:11:41,901 --> 00:11:47,373 for a lot of behavioral interventions, but some of them are quite active. 172 00:11:47,373 --> 00:11:50,209 And so, again, determining whether it's an active comparison 173 00:11:50,209 --> 00:11:54,980 or whether it's a true control intervention where you don't expect to see much benefit 174 00:11:54,980 --> 00:11:57,516 is really important in your statistical analysis plan. 175 00:11:57,516 --> 00:12:00,386 And we'll get to that in just a moment. 176 00:12:03,723 --> 00:12:06,892 Another issue that you need to deal with 177 00:12:06,892 --> 00:12:12,064 and describe very clearly in your protocol is, what other treatments are allowed. 178 00:12:12,064 --> 00:12:18,437 And if you allow a lot of other treatments in your study, how will you know 179 00:12:18,437 --> 00:12:24,009 if it's the intervention you delivered that produced the benefit that you may see, 180 00:12:24,009 --> 00:12:28,781 or if it was all the other interventions that people were doing? 181 00:12:29,715 --> 00:12:34,620 And so, in your consent form, your -- you guys, I think, have this talk already. 182 00:12:34,620 --> 00:12:37,957 They're told about other treatment options that are available to them. 183 00:12:37,957 --> 00:12:42,561 And so, if you then allow them to do all of those other treatment options 184 00:12:42,561 --> 00:12:45,297 and they get better, and they've started them all 185 00:12:45,297 --> 00:12:49,268 since they started your study, what drove the difference? What drove the improvement? 186 00:12:50,936 --> 00:12:52,138 Now, denying people 187 00:12:52,138 --> 00:12:56,909 access to readily available and effective treatments has also other ethical concerns. 188 00:12:56,909 --> 00:13:03,682 And so, you need to kind of think through what will you allow, what won't you allow. 189 00:13:03,682 --> 00:13:06,852 Are there any elements in terms of medications 190 00:13:06,852 --> 00:13:12,024 that may interfere with the primary outcomes that you want to look at? 191 00:13:12,024 --> 00:13:16,428 If you're interested in the immunological effects of a given botanical 192 00:13:16,428 --> 00:13:22,401 on the immune system in humans, but you allow patients who -- to take steroids, 193 00:13:23,002 --> 00:13:27,206 well I can tell you that the steroid is going to have such an impact on the cytokine 194 00:13:27,206 --> 00:13:30,209 function that you're going to see that whatever effect of the botanical you're 195 00:13:30,209 --> 00:13:33,579 giving is going to have zero impact compared to the effect of the steroid. 196 00:13:33,579 --> 00:13:36,982 So, you got to think through what's going to interfere with your primary outcomes. 197 00:13:36,982 --> 00:13:39,084 Are there any things that you need to exclude? 198 00:13:39,084 --> 00:13:41,153 Are you doing imaging as your primary outcome measure? 199 00:13:41,153 --> 00:13:43,589 Are you going to do an MRI? 200 00:13:43,589 --> 00:13:46,792 Are you going to exclude people who have implanted metal? 201 00:13:46,792 --> 00:13:51,263 It's a pretty important safety concern to think about and to exclude those individuals. 202 00:13:51,263 --> 00:13:55,434 So, those are the kinds of things that you want to think about. 203 00:13:55,434 --> 00:13:58,938 You know, would you allow patients to get braces put in? 204 00:13:58,938 --> 00:14:04,076 What kind of braces would it need to be if they're going to continue to participate? 205 00:14:04,310 --> 00:14:09,014 And there are reasons that people just have to drop out of studies, or not do 206 00:14:09,014 --> 00:14:13,385 certain elements because it's no longer safe for them in terms of the data collection. 207 00:14:13,385 --> 00:14:17,223 And those things are certainly allowed, but you note them as protocol deviations 208 00:14:17,223 --> 00:14:18,958 because they didn't follow the step-by-step 209 00:14:18,958 --> 00:14:22,795 of what's supposed to be collected according to the protocol. All right. 210 00:14:22,862 --> 00:14:26,065 You guys are hanging in and doing great. 211 00:14:26,065 --> 00:14:30,069 We've just got a few more sections to talk about. 212 00:14:30,069 --> 00:14:33,239 So -- and I know you've had quite 213 00:14:33,239 --> 00:14:38,043 a bit of information around statistics and study design and sample sizes. 214 00:14:38,043 --> 00:14:41,647 And so, there are a lot of different elements. 215 00:14:41,647 --> 00:14:44,416 So, again, I'm just showing you screenshots 216 00:14:44,416 --> 00:14:49,221 of how detailed the different elements that go into a protocol are. 217 00:14:49,588 --> 00:14:52,491 And for more information, certainly look to the instructions 218 00:14:52,491 --> 00:14:56,996 for the template that tell you what goes into each of these different sections. 219 00:14:59,999 --> 00:15:01,200 So, the other 220 00:15:01,200 --> 00:15:07,573 element that -- when I think about a protocol, it's something that I could hand to 221 00:15:07,840 --> 00:15:12,578 if I was the principal investigator and I had a research coordinator 222 00:15:12,578 --> 00:15:20,686 who was on vacation and I needed someone else to collect the -- I have a -- we weren't able 223 00:15:20,686 --> 00:15:25,591 to reschedule this participant, they need to come in at this date. 224 00:15:25,591 --> 00:15:26,992 It's their scheduled study visit. 225 00:15:26,992 --> 00:15:30,930 We can't reschedule it to when my staff member is going to be back. 226 00:15:30,930 --> 00:15:34,033 And I now have to have someone cover, or heaven forbid, 227 00:15:34,033 --> 00:15:38,537 I actually have to do it, and collect all of the data and draw the blood 228 00:15:38,537 --> 00:15:42,074 and do all that stuff myself if I'm trained to do so. 229 00:15:42,074 --> 00:15:47,313 I should be able to go to the protocol and know what it is I'm supposed to do. 230 00:15:47,613 --> 00:15:52,284 It's study visit six and this is the intervention that this person is on. 231 00:15:52,284 --> 00:15:56,322 What forms do I need to have them fill out, what samples 232 00:15:56,322 --> 00:16:00,693 do I need to collect, anyone should be able to pick that up. 233 00:16:00,693 --> 00:16:01,360 And that's 234 00:16:01,360 --> 00:16:06,065 what's so wonderful about having a detailed enough protocol, is that you don't miss 235 00:16:06,065 --> 00:16:11,103 half of the data you were supposed to collect because somebody got sick one day, 236 00:16:11,103 --> 00:16:15,808 or wasn't available to do the study visit, the person who normally does it. 237 00:16:15,808 --> 00:16:18,143 And, you know, with budgets being tight 238 00:16:18,143 --> 00:16:20,646 in clinical trials, they are often very short-staffed. 239 00:16:20,646 --> 00:16:23,449 And so, this is a real issue that happens. 240 00:16:23,449 --> 00:16:26,885 There's also a lot of staff turnover that occurs in trials. 241 00:16:26,885 --> 00:16:31,890 And so, making sure that you have everything very well-detailed, so anybody can pick it up, 242 00:16:31,890 --> 00:16:37,363 and once trained in the protocol, can go ahead and collect all the data that you need. 243 00:16:37,363 --> 00:16:42,201 So, one of the elements that you include is this kind of schedule of evaluation. 244 00:16:42,201 --> 00:16:46,171 So, how many study visits are needed, what happens at each of those visits 245 00:16:46,171 --> 00:16:49,008 from screening all the way through the final study visit. 246 00:16:49,008 --> 00:16:49,842 And it's often 247 00:16:49,842 --> 00:16:54,380 really helpful to have an overall timeline as to when do we need to start recruiting. 248 00:16:54,380 --> 00:16:59,218 How many participants do we need to recruit on a monthly basis or on a weekly basis 249 00:16:59,218 --> 00:17:02,888 in order to finish recruitment in time to have time to follow up 250 00:17:02,888 --> 00:17:06,725 all of our participants and collect all of the data that we need? 251 00:17:06,725 --> 00:17:09,128 And so, there's a couple of different elements. 252 00:17:09,128 --> 00:17:12,264 There's the overall study timeline as well as the individual timeline. 253 00:17:12,631 --> 00:17:16,301 And this is often most easily just provided to individual -- 254 00:17:16,301 --> 00:17:21,640 or provided in the protocol as a table, or as some sort of graphic like this. 255 00:17:21,640 --> 00:17:24,676 There's additional specific information about, well, what's a DXA? 256 00:17:24,676 --> 00:17:29,348 That's a DEXA scan and where do you go to get the DEXA scan, 257 00:17:29,348 --> 00:17:32,351 and who needs to schedule it ahead of time 258 00:17:32,351 --> 00:17:36,388 and make sure that you can get somebody in to do that. 259 00:17:36,989 --> 00:17:41,660 And then for this, what you can see here is that they put a little X 260 00:17:41,660 --> 00:17:44,596 as to -- at each visit what needs to happen. 261 00:17:44,596 --> 00:17:48,734 And so, this is where in your protocol, you might have something like this. 262 00:17:48,734 --> 00:17:52,237 In your Manual of Operations and Procedures, you may have individual checklists 263 00:17:52,237 --> 00:17:56,942 that you create for each study visit that allows your staff to walk down the list 264 00:17:56,942 --> 00:18:00,446 and check off exactly what they're supposed to do at each visit. 265 00:18:00,446 --> 00:18:05,751 Because if they have to go find the protocol and pull it off the shelf, and pull open 266 00:18:05,751 --> 00:18:06,618 the thing, it 267 00:18:06,618 --> 00:18:11,023 -- then often their -- the time pressures make it difficult for that to happen. 268 00:18:11,223 --> 00:18:12,191 And so, "Oops. 269 00:18:12,191 --> 00:18:16,695 We forgot at week five we were supposed to -- at treatment visit five, 270 00:18:16,695 --> 00:18:21,200 we were supposed to get blood and we thought it was treatment visit four. 271 00:18:21,200 --> 00:18:25,704 And so, we didn't collect blood." And now we don't have that midpoint data 272 00:18:25,704 --> 00:18:26,672 for this participant. 273 00:18:26,672 --> 00:18:29,541 And when -- there's often -- something that happens 274 00:18:29,541 --> 00:18:33,078 that when you forget it once, you start to forget it 275 00:18:33,078 --> 00:18:38,250 for all of the next patients that you see too because it becomes out of habit. 276 00:18:38,484 --> 00:18:40,652 And so, having just a really clear checklist, "Okay. 277 00:18:40,652 --> 00:18:41,587 This patient is treatment 278 00:18:41,587 --> 00:18:46,291 visit four, this is what I need to collect," and that you can go through and check it off. 279 00:18:46,291 --> 00:18:48,794 Those details don't necessarily -- they're here in the protocol, 280 00:18:48,794 --> 00:18:49,995 but how you implement it 281 00:18:49,995 --> 00:18:53,832 in terms of the procedure to implement it is what you would put in the MOP. 282 00:18:54,233 --> 00:18:55,734 And so, that's one 283 00:18:55,734 --> 00:19:00,606 example of kind of the difference between the protocol and the MOP there. 284 00:19:00,606 --> 00:19:04,376 Notice that adverse events are collected at all study visits? 285 00:19:04,376 --> 00:19:06,645 That's something that we typically see 286 00:19:06,645 --> 00:19:11,717 and encourage people to do to make sure that you're soliciting for those. 287 00:19:11,717 --> 00:19:14,887 And concomitant treatments or medications is another aspect 288 00:19:14,887 --> 00:19:19,791 that's collected routinely to find out, "Oh, yeah, I'm being treated for eczema. 289 00:19:19,791 --> 00:19:20,826 But you know what, 290 00:19:20,826 --> 00:19:25,397 I started a brand new -- I went to my doctor and he said, 'That's looking pretty bad. 291 00:19:25,397 --> 00:19:27,966 Here, try this new medication.' And I forgot to mention 292 00:19:27,966 --> 00:19:32,838 that I was in a research study for eczema." And we now need to track that for the study 293 00:19:32,838 --> 00:19:35,908 and indicate that the person has been started on something like that.