So, I, next, wanted to
talk a little bit about the actual study agent or the intervention.
So, because this was written -- this particular protocol template
is for either drugs or devices, they talk about the study agent.
Other protocols certainly can be done and developed for behavioral interventions
or for procedures like surgeries or other types of interventions.
And certainly,
you can have a protocol for observational studies too
that talk about how you're going to collect the data over time,
and you may not have an intervention at all in that particular protocol.
In which case, these particular sections of the template would just be removed
and stated as not applicable for that particular trial.
So, you want to
as much as possible define the treatment very clearly.
So, with NIH, we've seen both, at NIH, the recent publications
around enhancing rigor and reproducibility, and part of that is knowing
what was the treatment that was actually delivered.
And we've also seen, certainly in the psychology literature,
the push for open science and transparency and publishing
a priori hypotheses and protocols and other information like that.
So, this is a trend that we're starting to see so that someone else could attempt
-- if you find amazing results on your study that they could take your protocol
and be able to replicate it.
And if we're able to see the same results by a different group of investigators,
recruiting the same types of patients and delivering the same intervention,
then we have reproducible findings, which is really great,
that means that any provider then could think that they would likely find
the same results if they applied the same intervention in their offices.
So, this is why it's so important to really specify and describe your treatment
and how it's going to be delivered in your protocol itself.
And it's a tradeoff between the generalizability
of how people really deliver it versus the specificity
of how it's going to be delivered in this specific trial.
And so, you want to very clearly lay out all of these different elements
including when can someone -- when would someone be withdrawn from the study,
that it's not safe for them to continue or they need to get treatment elsewhere?
When and how do you track deviations from your protocol itself in terms of them
not completing the treatment as it was recommended in the study?
And then making sure you track all concomitant medications, procedures,
other kinds of interventions that people may have had over the course of the study.
So, the interventions themselves, you want to give enough information
that you -- like I said, someone else could repeat and deliver this intervention.
So, from our perspective at NCCIH, we -- as I mentioned, we often used natural products.
So, say, for example, a study is going to be
looking at one of the commonly used products like fish oil.
So, fish oil is one of the top-selling natural products in the US,
but pretty much every single formulation that's available over
the counter is different from the one next to it on the shelf.
It -- they have differing amounts of the two primary fatty acids, EPA and DHA.
They may have other oils in them as well.
And so, a very -- if you're doing a study of fish oil,
you want very clear specification as to what's in the product and what is your control?
What kind of oil do you use as a control capsule?
How much are you giving people?
How many capsules and what's the total milligrams
of each of the different types of EPA or DHA in the case of fish oil?
What's the quality check on that?
Is there an independent group that's done an assessment to assure that it actually has
what is labelled in the -- on the bottle itself?
Is there any way to assure that it's actually being taken by participants?
Do we have a way to a blood test to see if their fatty acid levels shift over time?
Do we have a memory cap on top of their bottle to see when they opened it
and when they closed it so that we can actually track
when they did that, and that they didn't bring us the empty bottle
because they threw them all in the trash on their way up the stairs
so that their pill count is zero in terms of remaining pills?
So, there's
a lot of elements to this in terms of really detailing all of these different pieces.
And when your intervention is something like cognitive behavioral therapy,
this gets even a little bit more complicated, right?
Because now
you have someone providing an intervention that's loosely -- that's hopefully manualized
and that they're delivering the same way for all participants.
How much variation do you allow in your study?
And specifying that is really important so that somebody else can repeat the results
if they -- if you find positive findings.
So, we struggle with the word "dose" a lot.
I think it's very easy, or I should say it's easier in studies of drugs
where you have a single agent and you count the milligrams
that you give to an individual, you may have a frequency question.
"Does it need to be given once a day, twice a day,
three times a day?" You may have questions about, "Is it delivered orally?
Is it injected? Is it to be put in the eyes?
Is it to be put in the ears?" There's a lot of different ways that you can
deliver drugs depending on what they're for.
But when it comes to behavioral interventions, like I said, it gets a little bit
more complicated.
So, how many sessions,
how many times a week does someone go in if you're doing a study on yoga?
Do they come in twice a week?
How long is the session, is it 30 minutes, is it 60 minutes, is it 90 minutes?
Are you asking them to practice at home on their own with the video that you gave them?
So, what is the dose?
Is it how much they practice per week?
Is it how much they attended the classes per week?
All of those things need to be specified so that you can determine
when everyone completes, well, who was adherent?
Who did what we asked them do to?
And so, we want to have those sorts of definitions in our protocols as well.
And, you know, there's
many different ways to kind of slice and dice this, but I think it often
comes down to the number of sessions, the frequency of the sessions
and the durations of those sessions, and then the practice element
is the other piece that comes into play with a lot of the behavioral interventions.
So, the other elements that happen in studies that often get forgotten
about are some of the common confounders that we see in study.
And time is the most common of these, so we -- potential participants often call
and have the impetuous to call
to participate in a research study when things are really bad.
It's finally gotten to a point that, "I need to do something about this.
My eczema is really bad, it's really inflamed, it's really bothering me.
I've got to do something about this.
Or you know what, I read about a study.
I'm going to give them a call and participate in that." Well, eczema
is something that tends to wax and wane.
It tends -- there tend to be flares and people can get better and the lesions can reduce.
And that can just happen over time.
So, if you have an uncontrolled study where you only have an evaluation pre-opposed
of someone getting an intervention, can we reduce stress in individuals with eczema
by having them meditate, for example.
And everyone gets better over time and we don't have a comparison group
that just came in and had a social support where they chatted with other people.
And instead of doing meditation, they maybe discussed a book or they read --
you know, discussed current events or something like that.
Might the social gathering also be a stress reliever?
Might that also have an improvement in their symptoms over time?
And might patients just get better with time itself compared to when things were
at their potentially worse when they decided to actually call about the study?
So, for conditions that tend to wax and wane like this, get better and worse,
that's when comparison groups are particularly very important to include in studies.
And it's -- randomization is our favorite friend
in clinical trials because it balances the groups at the beginning.
And so, randomizing to some sort of comparison group is often really important
in studies of things like depression where symptoms get better and worse,
and certainly things like eczema.
And there's many conditions
that you can think of that would fall into that sort of category.
And one
of the things that you also have to think about
is whether your comparison group might actually be an active intervention.
And so, sometimes we'll see in our studies that people will want to use exercise
as a comparison for yoga, or for other forms of interventions.
And the hypothesis that you're testing then is whether yoga, as
a form of exercise, has added benefit over exercise by itself.
And so, it's not, "Does yoga help compared to time and attention,
or compared to usual care?" It's really, "Is it over and above what we would see
with something like exercise itself?" So, the comparison group is really important
as you're planning your studies and working on your study design aspects.
And this is kind of -- I led into my next slide here.
And what is the comparison group?
And I often get this question from investigators, "Okay.
I want to do a study of massage.
What should be my control
group?" And I say, "Well, what's the hypothesis that you want to test?
Do you want to know
whether a specific form of massage is better than another form of massage?
Do you want to know if massage is better than --
for example, for knee, osteoarthritis, for knee pain due to joint degeneration?
Is massage better than taking an over-the-counter NSAID
to relieve pain?" Well, then that needs to be your comparison group.
So, really focusing on what your hypothesis is will tremendously help you in defining what
your comparison group is, and how you need to describe that comparison group.
There are
many different options
and each of them change the hypothesis that's actually tested in the study
as you shift the control group that's used in a particular trial.
I'll make a note about placebos are often easier to mask
and create for medication trials, for drug trials.
They're much more difficult to create sham procedures or sham interventions.
And so, we often see health education comparison groups
for a lot of behavioral interventions, but some of them are quite active.
And so, again, determining whether it's an active comparison
or whether it's a true control intervention where you don't expect to see much benefit
is really important in your statistical analysis plan.
And we'll get to that in just a moment.
Another issue that you need to deal with
and describe very clearly in your protocol is, what other treatments are allowed.
And if you allow a lot of other treatments in your study, how will you know
if it's the intervention you delivered that produced the benefit that you may see,
or if it was all the other interventions that people were doing?
And so, in your consent form, your -- you guys, I think, have this talk already.
They're told about other treatment options that are available to them.
And so, if you then allow them to do all of those other treatment options
and they get better, and they've started them all
since they started your study, what drove the difference? What drove the improvement?
Now, denying people
access to readily available and effective treatments has also other ethical concerns.
And so, you need to kind of think through what will you allow, what won't you allow.
Are there any elements in terms of medications
that may interfere with the primary outcomes that you want to look at?
If you're interested in the immunological effects of a given botanical
on the immune system in humans, but you allow patients who -- to take steroids,
well I can tell you that the steroid is going to have such an impact on the cytokine
function that you're going to see that whatever effect of the botanical you're
giving is going to have zero impact compared to the effect of the steroid.
So, you got to think through what's going to interfere with your primary outcomes.
Are there any things that you need to exclude?
Are you doing imaging as your primary outcome measure?
Are you going to do an MRI?
Are you going to exclude people who have implanted metal?
It's a pretty important safety concern to think about and to exclude those individuals.
So, those are the kinds of things that you want to think about.
You know, would you allow patients to get braces put in?
What kind of braces would it need to be if they're going to continue to participate?
And there are reasons that people just have to drop out of studies, or not do
certain elements because it's no longer safe for them in terms of the data collection.
And those things are certainly allowed, but you note them as protocol deviations
because they didn't follow the step-by-step
of what's supposed to be collected according to the protocol. All right.
You guys are hanging in and doing great.
We've just got a few more sections to talk about.
So -- and I know you've had quite
a bit of information around statistics and study design and sample sizes.
And so, there are a lot of different elements.
So, again, I'm just showing you screenshots
of how detailed the different elements that go into a protocol are.
And for more information, certainly look to the instructions
for the template that tell you what goes into each of these different sections.
So, the other
element that -- when I think about a protocol, it's something that I could hand to
if I was the principal investigator and I had a research coordinator
who was on vacation and I needed someone else to collect the -- I have a -- we weren't able
to reschedule this participant, they need to come in at this date.
It's their scheduled study visit.
We can't reschedule it to when my staff member is going to be back.
And I now have to have someone cover, or heaven forbid,
I actually have to do it, and collect all of the data and draw the blood
and do all that stuff myself if I'm trained to do so.
I should be able to go to the protocol and know what it is I'm supposed to do.
It's study visit six and this is the intervention that this person is on.
What forms do I need to have them fill out, what samples
do I need to collect, anyone should be able to pick that up.
And that's
what's so wonderful about having a detailed enough protocol, is that you don't miss
half of the data you were supposed to collect because somebody got sick one day,
or wasn't available to do the study visit, the person who normally does it.
And, you know, with budgets being tight
in clinical trials, they are often very short-staffed.
And so, this is a real issue that happens.
There's also a lot of staff turnover that occurs in trials.
And so, making sure that you have everything very well-detailed, so anybody can pick it up,
and once trained in the protocol, can go ahead and collect all the data that you need.
So, one of the elements that you include is this kind of schedule of evaluation.
So, how many study visits are needed, what happens at each of those visits
from screening all the way through the final study visit.
And it's often
really helpful to have an overall timeline as to when do we need to start recruiting.
How many participants do we need to recruit on a monthly basis or on a weekly basis
in order to finish recruitment in time to have time to follow up
all of our participants and collect all of the data that we need?
And so, there's a couple of different elements.
There's the overall study timeline as well as the individual timeline.
And this is often most easily just provided to individual --
or provided in the protocol as a table, or as some sort of graphic like this.
There's additional specific information about, well, what's a DXA?
That's a DEXA scan and where do you go to get the DEXA scan,
and who needs to schedule it ahead of time
and make sure that you can get somebody in to do that.
And then for this, what you can see here is that they put a little X
as to -- at each visit what needs to happen.
And so, this is where in your protocol, you might have something like this.
In your Manual of Operations and Procedures, you may have individual checklists
that you create for each study visit that allows your staff to walk down the list
and check off exactly what they're supposed to do at each visit.
Because if they have to go find the protocol and pull it off the shelf, and pull open
the thing, it
-- then often their -- the time pressures make it difficult for that to happen.
And so, "Oops.
We forgot at week five we were supposed to -- at treatment visit five,
we were supposed to get blood and we thought it was treatment visit four.
And so, we didn't collect blood." And now we don't have that midpoint data
for this participant.
And when -- there's often -- something that happens
that when you forget it once, you start to forget it
for all of the next patients that you see too because it becomes out of habit.
And so, having just a really clear checklist, "Okay.
This patient is treatment
visit four, this is what I need to collect," and that you can go through and check it off.
Those details don't necessarily -- they're here in the protocol,
but how you implement it
in terms of the procedure to implement it is what you would put in the MOP.
And so, that's one
example of kind of the difference between the protocol and the MOP there.
Notice that adverse events are collected at all study visits?
That's something that we typically see
and encourage people to do to make sure that you're soliciting for those.
And concomitant treatments or medications is another aspect
that's collected routinely to find out, "Oh, yeah, I'm being treated for eczema.
But you know what,
I started a brand new -- I went to my doctor and he said, 'That's looking pretty bad.
Here, try this new medication.' And I forgot to mention
that I was in a research study for eczema." And we now need to track that for the study
and indicate that the person has been started on something like that.