1
00:00:09,075 --> 00:00:11,778
So, a few more things about study designs.

2
00:00:11,778 --> 00:00:15,815
Your number one question is
how good is your primary research question?

3
00:00:15,815 --> 00:00:20,420
At the end of the day,
when your research is done, and your data

4
00:00:20,653 --> 00:00:25,592
analyzed, will the answer -- regardless
of what the answer is to the primary research

5
00:00:25,592 --> 00:00:27,927
question, advance scientific knowledge
or clinical practice?

6
00:00:27,927 --> 00:00:33,333
If it does not advance scientific knowledge
or clinical practice, if it does not at least

7
00:00:33,333 --> 00:00:37,370
lay the necessary foundation that's missing
to advance it, you have failed,

8
00:00:37,370 --> 00:00:40,407
and you don't have a good primary research
question.

9
00:00:42,008 --> 00:00:44,644
I'm not saying that you answered

10
00:00:44,644 --> 00:00:49,015
the question
the way you hoped it would be answered.

11
00:00:49,015 --> 00:00:52,485
But you know, does a failed -- like,

12
00:00:52,485 --> 00:00:57,724
if you get the exact opposite answer,
does that still advance science?

13
00:00:57,724 --> 00:00:59,492
It's an important thing.

14
00:00:59,492 --> 00:01:02,095
If you get a negative result,

15
00:01:02,095 --> 00:01:06,900
not that positive result you're hoping for,
that's still important.

16
00:01:06,900 --> 00:01:11,704
The second most important
element is a good primary outcome

17
00:01:11,704 --> 00:01:14,340
measure
that's clinically meaningful and simple.

18
00:01:15,575 --> 00:01:19,312
I now have to write and turn these endpoints
for studies

19
00:01:19,312 --> 00:01:24,084
into labels that the American public can read
and that the prescribers can understand.

20
00:01:24,084 --> 00:01:27,487
It is really hard to interpret
some of these endpoints.

21
00:01:27,487 --> 00:01:28,488
It makes sense

22
00:01:28,488 --> 00:01:33,259
when you're trying to cover all your ground
to combine all this stuff together,

23
00:01:33,259 --> 00:01:38,031
but as you get into the discussions
of measures and endpoints in this course,

24
00:01:38,031 --> 00:01:42,769
you'll realize it can be very hard
to actually express it and explain it

25
00:01:42,769 --> 00:01:43,803
to somebody.

26
00:01:43,803 --> 00:01:49,576
But I want to give a little bit
more information about how you want to start

27
00:01:49,576 --> 00:01:52,979
designing one of these lovely studies
we're just talking about.

28
00:01:54,047 --> 00:01:58,318
You've got study aims, your background,
your rational, then you're going to talk

29
00:01:58,318 --> 00:02:00,954
about the endpoints,
the outcome variables, any assessment

30
00:02:00,954 --> 00:02:04,557
you're going to take on the people
in your clinical trial,

31
00:02:04,557 --> 00:02:07,861
or the animals in your trial,
whatever's in the trial.

32
00:02:07,861 --> 00:02:09,829
What are your assessments? Be specific.

33
00:02:09,829 --> 00:02:12,132
Don't tell me you're measuring sleep, folks.

34
00:02:12,132 --> 00:02:14,734
Tell me exactly how you are measuring it.

35
00:02:15,835 --> 00:02:18,438
Talk about -- think about the specific
elements.

36
00:02:18,438 --> 00:02:20,039
Is it sensitive to change?

37
00:02:20,039 --> 00:02:23,943
Do not take a wooden ruler
and try to measure my waistline.

38
00:02:23,943 --> 00:02:27,180
That is not a good way
to take that measurement.

39
00:02:27,180 --> 00:02:30,416
You want something that's reliable.
You want something that's valid.

40
00:02:30,416 --> 00:02:34,921
You need to know all this information
about those measures you're going to use

41
00:02:34,921 --> 00:02:37,190
because the measures are what you're going

42
00:02:37,190 --> 00:02:40,760
to use in some combination
to get that actual final endpoint.

43
00:02:41,794 --> 00:02:43,163
Think about your inclusion/exclusion
criteria.

44
00:02:43,163 --> 00:02:46,733
Can you measure all of these things
on the people in your study?

45
00:02:46,733 --> 00:02:51,404
We'll talk a lot about this, and Wendy
Weber will talk a little bit more about it

46
00:02:51,404 --> 00:02:53,039
when she talks about protocols, too.

47
00:02:53,039 --> 00:02:56,876
So you have to make sure people are legal
to be in your study.

48
00:02:56,876 --> 00:02:59,078
But sometimes I see
so many exclusion criteria,

49
00:02:59,078 --> 00:03:02,882
I'm like there's nobody left
in this world that can be in your study.

50
00:03:03,983 --> 00:03:07,053
So you've
got to balance and think about safety.

51
00:03:07,053 --> 00:03:10,823
Like, you know,
can they not ethically be in your trial?

52
00:03:10,823 --> 00:03:14,794
And after that,
you might want to let everybody else in.

53
00:03:14,794 --> 00:03:18,331
How do you start designing?
Think about that accrual plan.

54
00:03:18,331 --> 00:03:20,033
Think about the preparatory test.

55
00:03:20,033 --> 00:03:24,137
What's the timeline for your overall study
and for the individual participants?

56
00:03:24,137 --> 00:03:26,873
Do they have to come between 9:00 a.m.

57
00:03:26,873 --> 00:03:27,907
and 4:00 p.m.

58
00:03:27,907 --> 00:03:30,643
on days that they are likely to work?

59
00:03:31,344 --> 00:03:33,613
That is going to make a big problem.

60
00:03:33,613 --> 00:03:38,151
Yeah, someone's like raising their head and
I'm like, yeah, that's right, it's a problem.

61
00:03:38,151 --> 00:03:40,153
Treatments.
What are the participant implications?

62
00:03:40,153 --> 00:03:43,556
So I was in a trial,
and I couldn't take certain antibiotics.

63
00:03:43,556 --> 00:03:46,960
So then, of course, I get sick
and I need an antibiotic.

64
00:03:46,960 --> 00:03:50,964
So there was this back and forth
about what drug I could actually take.

65
00:03:52,065 --> 00:03:56,069
But you have to think about the implications
on your participants.

66
00:03:56,069 --> 00:03:57,870
What is the exact product?

67
00:03:57,870 --> 00:04:01,140
What is the exact dose?
What about the quality?

68
00:04:01,140 --> 00:04:04,444
How is it administered?
Can you reproduce the intervention?

69
00:04:04,444 --> 00:04:10,250
Whether it is a drug or whether it is yoga,
can you actually reproduce that intervention?

70
00:04:12,919 --> 00:04:17,223
So if I say yoga, exactly
what type of yoga is it?

71
00:04:17,223 --> 00:04:19,759
Am I doing it once a week?

72
00:04:19,759 --> 00:04:22,262
Six times a week? What's going on?

73
00:04:22,262 --> 00:04:24,430
Does it interfere with patient management?

74
00:04:24,430 --> 00:04:25,865
Again, kind of my

75
00:04:25,865 --> 00:04:31,271
being in that trial interfered with my doctor
trying to give me a medication.

76
00:04:31,271 --> 00:04:34,841
Generalizability
is often lost in this quest for specificity.

77
00:04:34,841 --> 00:04:37,377
So you have to decide and balance.

78
00:04:37,377 --> 00:04:42,815
And at different parts of your scientific
knowledge, you may decide to have more

79
00:04:42,815 --> 00:04:43,950
or less generalizability.

80
00:04:43,950 --> 00:04:44,350
Specify

81
00:04:44,350 --> 00:04:49,322
the criteria for withdrawal from studies
or a deviation from the protocol definitions.

82
00:04:49,322 --> 00:04:54,294
If they do not show up at exactly day
72, does it matter?

83
00:04:54,294 --> 00:04:58,131
Or is it day 72 plus or minus seven days.

84
00:04:58,131 --> 00:05:01,200
Like,
what are the windows for every assessment?

85
00:05:01,200 --> 00:05:07,340
Sometimes you have to be super precise
and sometimes you can be a little bit looser.

86
00:05:08,841 --> 00:05:10,343
Sometimes someone may need

87
00:05:10,343 --> 00:05:15,181
to go off intervention,
but they do not need to leave your study.

88
00:05:15,181 --> 00:05:18,518
You can still follow them
and take the assessments.

89
00:05:18,518 --> 00:05:22,989
You want to have a list of all
the concurrent medications, procedures,

90
00:05:22,989 --> 00:05:27,827
et cetera, that are prohibited, permitted,
and how you're going to record them.

91
00:05:27,827 --> 00:05:30,430
Do remember, it's not just medicine, folks.

92
00:05:30,430 --> 00:05:32,298
These people are drinking teas,

93
00:05:32,298 --> 00:05:35,668
they are taking supplements,
they are taking over-the-counter medications.

94
00:05:36,102 --> 00:05:41,741
Do you want them to still do Tai chi or go to
spinning class if you are studying yoga?

95
00:05:41,741 --> 00:05:45,611
You need to think about
all the different things they could be doing

96
00:05:45,611 --> 00:05:47,680
that might interfere
with what you're studying.

97
00:05:47,680 --> 00:05:50,650
But also, you have to think about,
you know what,

98
00:05:50,650 --> 00:05:54,787
sometimes people are going to get headaches
and what are they going to take?

99
00:05:54,821 --> 00:05:57,724
Like,
don't tell them to disavow or not do anything

100
00:05:57,724 --> 00:06:02,829
because you'll get nobody in your study, or
they'll do it anyway and just not tell you.

101
00:06:02,829 --> 00:06:04,430
So what is the dose?

102
00:06:04,430 --> 00:06:07,633
It might be the number of sessions,
or pills, or treatments.

103
00:06:07,633 --> 00:06:09,369
It could be social media attempts.

104
00:06:09,369 --> 00:06:14,107
So if some of these studies are in fact
trying to actually tell you, like,

105
00:06:14,107 --> 00:06:17,810
all right, we want to give you information
about your cigarette smoking.

106
00:06:17,810 --> 00:06:20,880
Well, do I give you one text message a day?

107
00:06:20,880 --> 00:06:23,649
So I send you six text messages a day

108
00:06:23,649 --> 00:06:28,855
to try to keep you so that you are still --
you have not started smoking again?

109
00:06:28,855 --> 00:06:30,256
What is the amount?

110
00:06:30,256 --> 00:06:30,690
Frequency.

111
00:06:30,690 --> 00:06:33,459
Do I go to the chiropractor once a week?

112
00:06:33,459 --> 00:06:34,927
Seven days a week?

113
00:06:34,927 --> 00:06:37,797
How frequently do I go?
And for how many weeks?

114
00:06:37,797 --> 00:06:39,232
And for how much time?

115
00:06:39,232 --> 00:06:42,402
There was one good massage study
that they looked at 30-minute,

116
00:06:42,402 --> 00:06:45,838
60 minute, or 90-minute massage. Once, twice,
or three times a week.

117
00:06:45,838 --> 00:06:48,408
Now they did take out something
because they're like,

118
00:06:48,408 --> 00:06:51,844
well, you're not going to study
three times a week, 90-minute massage.

119
00:06:51,911 --> 00:06:54,180
People will not go to that.

120
00:06:54,180 --> 00:06:56,416
But there've been these very interesting

121
00:06:56,416 --> 00:07:00,953
dosing studies, not just of drugs,
but a lot of other interventions.

122
00:07:00,953 --> 00:07:05,091
Same thing if you're talking about
a lot of psychological interventions.

123
00:07:05,091 --> 00:07:06,225
How much practice?

124
00:07:06,225 --> 00:07:09,962
What do people need
to do outside of a classroom?

125
00:07:09,962 --> 00:07:11,497
Who is the leader?

126
00:07:11,497 --> 00:07:14,500
Who is the surgeon? Who is that person?

127
00:07:14,500 --> 00:07:18,070
How much contact is there
and how well trained are they?

128
00:07:18,070 --> 00:07:19,705
There're a lot of combinations.

129
00:07:19,705 --> 00:07:21,674
You're only going to be able

130
00:07:21,674 --> 00:07:26,379
to test some of these possible doses,
but it's important to look at it.

131
00:07:26,379 --> 00:07:31,417
As I mentioned at the very end, you've got to
think about your practitioner impact.

132
00:07:31,417 --> 00:07:34,353
There could be a lot of false/negatives
and false/positives

133
00:07:34,353 --> 00:07:39,225
that come up here because we have some people
that are just -- they're enigmatic.

134
00:07:39,225 --> 00:07:41,494
People like them and good things happen.

135
00:07:42,628 --> 00:07:46,966
But sometimes you say, listen, I'm
doing a proof of principle study.

136
00:07:46,966 --> 00:07:51,671
If the best massage therapist, who's really
well-trained, and trains others, if that

137
00:07:51,671 --> 00:07:57,076
person cannot give a massage that improves
low back pain, probably nobody can do it.

138
00:07:57,076 --> 00:08:02,515
If my best surgeon cannot fix this,
maybe nobody can be trained to do that.

139
00:08:02,515 --> 00:08:05,384
So sometimes we actually go
for really well-trained

140
00:08:05,384 --> 00:08:09,722
folks in a proof of concept model
and then broaden that out.

141
00:08:10,022 --> 00:08:12,525
But again,
you probably want to choose techniques

142
00:08:12,525 --> 00:08:15,628
that can be generalized
if you're going to do that.

143
00:08:15,628 --> 00:08:19,732
But other times, such as the Perlman studies
that were looking at massage

144
00:08:19,732 --> 00:08:23,803
actually for osteoarthritis of the knee,
he chose Swedish massage because he said

145
00:08:23,803 --> 00:08:28,474
that's what everybody is trained in
in most of the countries he was looking at.

146
00:08:28,474 --> 00:08:33,713
He was like, that's the foundation
that they have, we want to build on that.

147
00:08:33,713 --> 00:08:35,748
So the study analysis population.

148
00:08:35,748 --> 00:08:38,150
You have this mechanistic proof of concept.

149
00:08:38,150 --> 00:08:40,887
That's your throw in the best at it.

150
00:08:40,887 --> 00:08:41,888
We sometimes call

151
00:08:41,888 --> 00:08:47,360
these also the protocol analyses that we do
because we only use the patients that behave,

152
00:08:47,360 --> 00:08:51,797
the study subjects who do everything we tell
them, we'll analyze them here.

153
00:08:51,797 --> 00:08:56,602
That's along the idea of, like,
in the perfect world what might we expect?

154
00:08:57,770 --> 00:09:00,706
But general
use is like these intent-to-treat analyses.

155
00:09:00,706 --> 00:09:01,440
That's everybody.

156
00:09:01,440 --> 00:09:06,212
You tell a patient to take a drug,
they don't necessarily take it.

157
00:09:06,212 --> 00:09:10,616
You randomize a patient to a study arm;
they don't necessarily comply

158
00:09:10,616 --> 00:09:11,717
with your intervention.

159
00:09:11,717 --> 00:09:15,755
So intent to treat versus
kind of these different completers analyses,

160
00:09:15,755 --> 00:09:19,425
this gets down to
what is your data analyses population.

161
00:09:20,092 --> 00:09:24,730
So we could say ITT, or intent
to treat, once randomized, always analyzed.

162
00:09:24,730 --> 00:09:26,165
And then observational trial,

163
00:09:26,165 --> 00:09:31,170
we're like, once you're in the trial,
we follow you in the trial.

164
00:09:31,170 --> 00:09:35,808
You assume that all study participants
are adhering to your study regiment

165
00:09:35,808 --> 00:09:37,944
and that they complete the study.

166
00:09:37,944 --> 00:09:42,949
So they -- you assume they behaved perfectly,
regardless of what they actually do.

167
00:09:44,550 --> 00:09:48,721
Most of your regulatory
agencies are going to say, "We

168
00:09:48,721 --> 00:09:52,858
expect you to do an intent-to-treat
analysis." Most high-quality research

169
00:09:52,858 --> 00:09:57,396
regulated or not, we assume
you should be doing intent-to-treat analysis.

170
00:09:57,396 --> 00:10:01,534
But then people kind of
want to skirt the edges.

171
00:10:01,534 --> 00:10:05,271
They do something called a modified
or MITT analysis.

172
00:10:05,271 --> 00:10:06,939
Modified intent to treat.

173
00:10:06,939 --> 00:10:11,911
They may only include patients who start
the intervention they're assigned to.

174
00:10:11,911 --> 00:10:16,048
So they include the study subjects
who start the intervention.

175
00:10:17,149 --> 00:10:17,483
Well,

176
00:10:17,483 --> 00:10:22,855
if I randomized you to go to psychotherapy
versus not, you decide you don't like what

177
00:10:22,855 --> 00:10:27,560
your randomized to, and you don't start,
you know, that's still telling me something.

178
00:10:27,560 --> 00:10:31,263
So should I really throw those people
out of my analyses?

179
00:10:31,263 --> 00:10:34,266
It depends on the question
you're trying to answer.

180
00:10:34,266 --> 00:10:39,305
Sometimes people say if they don't make it
to like the first or the second

181
00:10:39,305 --> 00:10:41,641
post-baseline assessment,
then I don't count them.

182
00:10:41,707 --> 00:10:46,512
I'm like, well, again, you can't really
compare both groups of patients then.

183
00:10:46,512 --> 00:10:50,182
You know, your study
subjects may no longer be comparable.

184
00:10:50,182 --> 00:10:52,018
You have undermined the randomization.

185
00:10:52,018 --> 00:10:55,688
So that's the problem with modified
intent to treat analyses.

186
00:10:55,688 --> 00:10:58,658
You've got to be very careful with those.

187
00:10:58,658 --> 00:11:02,695
But sometimes we do these completers
or adheres analyses that, again,

188
00:11:02,695 --> 00:11:06,732
you're only dealing with your well-behaved
and that's a problem.

189
00:11:06,732 --> 00:11:12,271
So as John described last night, we have
this kind of superiority in equivalence.

190
00:11:12,638 --> 00:11:16,042
So I wanted to put this in a picture for you.

191
00:11:16,042 --> 00:11:20,079
Equivalence or no difference, the confidence
in a role that we'll talk about

192
00:11:20,079 --> 00:11:22,248
in a hypothesis testing lecture,
everything's inside

193
00:11:22,248 --> 00:11:25,051
this little, tiny, round,
kind of in the middle.

194
00:11:25,051 --> 00:11:27,186
It's like where there is no difference.

195
00:11:27,186 --> 00:11:29,355
So this is a normal distribution curve.

196
00:11:29,355 --> 00:11:32,792
And this line in the middle
is kind of the zero.

197
00:11:33,693 --> 00:11:37,463
It's the middle of the bell curve
or the Gaussian curve.

198
00:11:37,463 --> 00:11:42,234
So if you're in a little tight area here,
there's no difference between your study

199
00:11:42,868 --> 00:11:42,902
arms.

200
00:11:42,902 --> 00:11:46,972
If you're in these orange areas
in either end, you have superiority.

201
00:11:46,972 --> 00:11:52,445
So when I test two arms,
I can come up with a value along this curve.

202
00:11:52,445 --> 00:11:56,849
And if I'm far enough out,
that's saying that basically these two groups

203
00:11:56,849 --> 00:11:58,250
are statistically different,

204
00:11:58,250 --> 00:12:01,420
And it's probably not just a random error
that I'm saying that that

205
00:12:02,521 --> 00:12:05,091
noninferiority

206
00:12:05,091 --> 00:12:08,160
is when I say, well,
they may be a little bit inferior

207
00:12:08,794 --> 00:12:09,595
or they could and it's probably not just
a random error that I'm [unintelligible].

208
00:12:09,595 --> 00:12:09,895
Non-inferiority is when I say, well,
they may be a little bit inferior,

209
00:12:09,895 --> 00:12:10,196
or they could even be superior, so
it kind of gives me this huge area to be in.

210
00:12:10,196 --> 00:12:10,429
So what are the comparison groups
I might use to try to get there?

211
00:12:10,429 --> 00:12:10,529
Well, I've got experimental interventions
versus control.

212
00:12:10,529 --> 00:12:10,663
Sometimes I'm in my epidemiology land.
be superior.

213
00:12:10,663 --> 00:12:13,365
So kind of gives me this huge area to be

214
00:12:15,901 --> 00:12:19,271
So what are the comparison groups
I might use to try to get there?

215
00:12:20,072 --> 00:12:23,676
Well, I've got experimental interventions
versus control

216
00:12:24,376 --> 00:12:27,213
sometimes I'm in my epidemiology land.

217
00:12:27,213 --> 00:12:29,348
I also have cases versus control.

218
00:12:29,348 --> 00:12:31,450
Just because it's a controlled group

219
00:12:31,450 --> 00:12:34,653
doesn't mean you actually have
a randomized control arm.

220
00:12:34,653 --> 00:12:37,456
Sometimes I'm comparing
the exposed to the unexposed.

221
00:12:37,456 --> 00:12:42,394
So at the World Trade Center sites,
they actually took people who were doing

222
00:12:42,394 --> 00:12:46,632
cleanup there who had been exposed
to that work and compared them

223
00:12:46,632 --> 00:12:51,570
to a group of people not doing cleanup,
who did not have that exposure.

224
00:12:52,505 --> 00:12:57,810
So that is a study that the Center
for Disease Control and Prevention

225
00:12:57,810 --> 00:12:59,044
has been running.

226
00:12:59,044 --> 00:13:04,150
We may have various levels of exposure
that we want to compare.

227
00:13:04,150 --> 00:13:05,551
Men versus women.

228
00:13:05,551 --> 00:13:09,221
Common comparison
you see. The old versus the young.

229
00:13:09,221 --> 00:13:12,892
Maybe BMI that's over 25 to 25 and under.

230
00:13:12,892 --> 00:13:16,962
You have the usual
or standard of care or practice.

231
00:13:16,962 --> 00:13:20,699
Standard of care, not always
that standardized, by the way, though.

232
00:13:20,699 --> 00:13:23,769
And sometimes we're doing this history.
Sometimes I do pre-post.

233
00:13:23,769 --> 00:13:26,539
I take somebody's baseline measures,
I intervene on them,

234
00:13:26,539 --> 00:13:28,374
and I look at them afterwards.

235
00:13:28,374 --> 00:13:30,543
Or maybe it's even just natural history.

236
00:13:30,543 --> 00:13:35,147
I look at them when I diagnose them,
and I see how they've changed over

237
00:13:35,147 --> 00:13:37,316
the next 12 months or two years.

238
00:13:39,819 --> 00:13:42,154
When you talk about placebos, standard

239
00:13:42,154 --> 00:13:46,759
of care, attention controls, you might have
some type of experimental treatment

240
00:13:46,759 --> 00:13:51,397
that you might offer, say, supportive care
or some other current treatment.

241
00:13:51,397 --> 00:13:54,099
So supportive care is not no care.

242
00:13:54,099 --> 00:13:56,001
It's not a true placebo.

243
00:13:56,001 --> 00:14:01,040
But if you have a yoga intervention,
what should your control group be?

244
00:14:01,040 --> 00:14:02,942
Maybe it's exercise or stretching.

245
00:14:02,942 --> 00:14:07,980
Maybe it's cooking class
because you kind of want people in a group

246
00:14:07,980 --> 00:14:10,683
but not really interacting with each other.

247
00:14:11,383 --> 00:14:15,287
Maybe it's a book club
because you want them to

248
00:14:15,287 --> 00:14:18,791
actually go someplace
and be someplace for 90 minutes.

249
00:14:18,791 --> 00:14:20,326
Maybe you do nothing.

250
00:14:20,326 --> 00:14:24,230
Maybe you say, in fact, again,
proof of concept trial.

251
00:14:24,230 --> 00:14:27,333
Does anything change if they go to yoga?

252
00:14:27,333 --> 00:14:28,901
Sometimes we'll do something

253
00:14:28,901 --> 00:14:33,973
called a weightless control
where we say, "We want you to do nothing

254
00:14:33,973 --> 00:14:39,011
new for the next 12 weeks."
And then you can do the yoga.

255
00:14:39,812 --> 00:14:42,181
The controls cost money.

256
00:14:42,181 --> 00:14:47,453
You will see in the sample size lecture,
you have much higher sample sizes

257
00:14:47,453 --> 00:14:49,321
when we have control arms.

258
00:14:49,321 --> 00:14:51,190
You want to control everything

259
00:14:51,190 --> 00:14:55,294
except the smallest element
of the intervention you want to test.

260
00:14:55,294 --> 00:14:59,031
Be careful,
however, it's not too small of a difference.

261
00:14:59,031 --> 00:15:03,903
So I have this one meditation study,
and they really wanted to test

262
00:15:03,903 --> 00:15:04,670
the mindfulness

263
00:15:04,670 --> 00:15:08,774
aspect of meditation, and control
for everything else about the meditation.

264
00:15:09,875 --> 00:15:14,179
This is a very small difference
and very hard to do.

265
00:15:14,179 --> 00:15:19,652
Like, why don't you just test like
the meditation as a whole versus something.

266
00:15:19,652 --> 00:15:23,155
But they're consequences
when you have more control imposed.

267
00:15:23,155 --> 00:15:25,491
You can have larger sample sizes.

268
00:15:25,491 --> 00:15:27,459
You may miss a difference.

269
00:15:27,459 --> 00:15:33,699
And if you have a less sensitive outcome
measure, you may not pick that difference up.

270
00:15:34,800 --> 00:15:36,368
So plan accordingly.

271
00:15:36,368 --> 00:15:38,304
What are the differences?

272
00:15:38,304 --> 00:15:43,342
Again, you've got to think about every
single difference between your study arms

273
00:15:43,342 --> 00:15:45,277
is basically defining your intervention.

274
00:15:45,277 --> 00:15:49,548
If somebody is spending
one hour a week with your study

275
00:15:49,548 --> 00:15:54,186
participants versus three hours a week,
that is something that is different.

276
00:15:54,186 --> 00:15:59,992
If you tell people to spend 15 minutes
at home working on that meditation versus

277
00:15:59,992 --> 00:16:05,864
60 minutes a day at home working on it,
again, this is defining your intervention.

278
00:16:05,864 --> 00:16:11,070
And so when you say to me, if you just have
two study arms against each other,

279
00:16:11,070 --> 00:16:14,773
well I need to figure out
if it's the participant contact time

280
00:16:14,773 --> 00:16:19,712
or the time they spend at home, and say,
well, one had one hour of contact

281
00:16:19,712 --> 00:16:21,246
and 15 minutes at home

282
00:16:21,246 --> 00:16:25,551
and the other one had three hours of contact
and 60 minutes at home.

283
00:16:25,751 --> 00:16:27,686
I can't tease it apart, folks.

284
00:16:27,686 --> 00:16:33,192
You have to plan all of our interventions
to figure out if you can tease that out.

285
00:16:35,661 --> 00:16:36,428
So your

286
00:16:36,428 --> 00:16:40,699
control group might be placebo, most widely
accepted treatments, standard treatments.

287
00:16:40,699 --> 00:16:44,203
Always make sure that these two bullets
are well-defined.

288
00:16:44,203 --> 00:16:48,073
You're going to have to record it
during your study.

289
00:16:48,073 --> 00:16:49,641
Most accepted prevention intervention.

290
00:16:49,641 --> 00:16:55,047
Are you going to do an HIV prevention
intervention and not talk about condoms?

291
00:16:55,047 --> 00:16:58,150
What is it that you're going to do?

292
00:16:58,150 --> 00:17:01,653
Again, usual care,
not all that usual across sites.

293
00:17:01,653 --> 00:17:04,490
You've got to record what it is.

294
00:17:04,490 --> 00:17:07,326
What are the accepted means of detection?

295
00:17:07,326 --> 00:17:09,228
What is that diagnostic test?

296
00:17:09,228 --> 00:17:15,634
If I did that pap smear for cervical cancer,
am I going to get the same results

297
00:17:15,634 --> 00:17:19,805
as if I swipe with vinegar
and then do the test?

298
00:17:19,805 --> 00:17:20,539
Non-diseased population.

299
00:17:20,539 --> 00:17:25,844
Sometimes we compare disease to a non-disease
population, especially to try to figure out

300
00:17:25,844 --> 00:17:26,979
special differences.

301
00:17:26,979 --> 00:17:28,480
All control groups,

302
00:17:28,480 --> 00:17:32,251
especially when you're doing
interventional studies, need to be ethical.

303
00:17:32,985 --> 00:17:37,189
If you're going to assign anybody to a group,
anyone meeting the study criterion

304
00:17:37,189 --> 00:17:40,192
has to be able to be in any study group.

305
00:17:40,192 --> 00:17:41,660
If that's not the case,

306
00:17:41,660 --> 00:17:46,165
you need to make sure that your randomization
can figure that out, and they know

307
00:17:46,165 --> 00:17:50,669
not to randomize people to study arms
that will easily be known to kill them.

308
00:17:50,669 --> 00:17:53,005
I set up an algorithm, folks.

309
00:17:53,005 --> 00:17:58,911
I only know as much medicine as you have
imparted to me in setting that up.

310
00:17:58,911 --> 00:18:00,012
Your inclusion/exclusion criteria

311
00:18:00,012 --> 00:18:05,551
should keep people from being in study arms
that you know are wrong for them.

312
00:18:05,551 --> 00:18:10,355
If you have questions about that, Chuck
Natanson actually has a whole lecture

313
00:18:10,355 --> 00:18:14,793
that he talks about mistakes
that have been made in randomization.

314
00:18:14,793 --> 00:18:16,762
In standard of care.

315
00:18:16,762 --> 00:18:20,132
Is it really standard?
Good controls cannot always be masked.

316
00:18:20,132 --> 00:18:24,203
Somebody asks me this every year,
it is now on the slide.

317
00:18:24,203 --> 00:18:27,606
Try but you may not be able to do it.

318
00:18:27,606 --> 00:18:32,678
People do tend to be better after receiving
any type of therapy placebo or not.

319
00:18:32,678 --> 00:18:33,345
Care matters.

320
00:18:33,345 --> 00:18:37,082
Do not underestimate
that when you are planning your trials.

321
00:18:37,082 --> 00:18:38,784
Comparing population incidence rates

322
00:18:38,784 --> 00:18:43,522
to the beginning of programs
does not take into account a lot of factors.

323
00:18:43,522 --> 00:18:47,226
So, I had someone,
they're like, oh, well just do pre-post.

324
00:18:48,660 --> 00:18:52,264
Well, if you had taken into account
the control

325
00:18:52,264 --> 00:18:57,035
group, in fact, while they're pre-post,
the post looked worse than baseline.

326
00:18:57,035 --> 00:19:00,239
Everybody else looked a lot worse than that.

327
00:19:00,239 --> 00:19:05,010
So, if you hadn't had that control arm,
you wouldn't have known

328
00:19:05,010 --> 00:19:07,813
that they actually basically salvaged
the slides.

329
00:19:07,813 --> 00:19:11,416
So, everybody else is kind of going
like this.

330
00:19:11,416 --> 00:19:12,985
They're all going downhill.

331
00:19:12,985 --> 00:19:18,190
And they kind of -- they went down
but not by as much.

332
00:19:18,190 --> 00:19:20,359
It's important to understand this.

333
00:19:20,359 --> 00:19:24,496
No control group.
You've got a lot of problems.

334
00:19:24,496 --> 00:19:25,898
Researchers and participants

335
00:19:25,898 --> 00:19:30,002
tend to interpret findings
in favor of new treatments.

336
00:19:30,002 --> 00:19:34,573
There's investigator and participant bias
when you don't have randomization.

337
00:19:34,573 --> 00:19:37,309
You cannot distinguish effect and time.

338
00:19:37,309 --> 00:19:42,814
If I wait 10-14 days,
we will all get over our colds.

339
00:19:42,814 --> 00:19:45,551
So what's the right control group?

340
00:19:45,551 --> 00:19:48,287
There is no right control group.

341
00:19:48,287 --> 00:19:51,056
You just have to choose one.

342
00:19:51,056 --> 00:19:54,259
Also again, remember control groups
may happen in nonrandomized studies.

343
00:19:54,259 --> 00:19:56,662
We'll talk more about this next week.

344
00:19:56,662 --> 00:19:59,398
But you have to consider all your effects.

345
00:19:59,398 --> 00:20:01,967
Positive, negative.
Are your effects going to plateau?

346
00:20:01,967 --> 00:20:05,504
When do I measure things?
Are you looking at long-term differences?

347
00:20:05,504 --> 00:20:07,406
Is it going to be that,

348
00:20:07,406 --> 00:20:12,878
you know, you're going to see a change
and then it's going to kind of trickle away?

349
00:20:12,878 --> 00:20:13,512
Delayed response.

350
00:20:13,512 --> 00:20:17,683
Do I have to wait six months
before I actually see a change?

351
00:20:18,650 --> 00:20:23,155
You got to consider all this stuff
when you're planning a trial.

352
00:20:23,155 --> 00:20:26,592
So time is my favorite confounder
in uncontrolled studies.

353
00:20:26,592 --> 00:20:30,062
You've got differential dropout
between the study arms. Different seasons.

354
00:20:30,062 --> 00:20:34,199
So if you are trying to test
and see different histamine levels,

355
00:20:34,199 --> 00:20:39,037
that is going to vary depending on the season
that you are measuring people.

356
00:20:39,037 --> 00:20:41,106
Bone density changes throughout the year.

357
00:20:41,106 --> 00:20:47,412
You've got to make sure you measure them
on the 12-month mark, not nine months.

358
00:20:47,412 --> 00:20:49,781
Social support has an issue.

359
00:20:49,781 --> 00:20:53,318
Just empathy and talking to people,
really matters. Exercise

360
00:20:53,318 --> 00:20:58,056
we know controls stress, cardiovascular
risk factors, a lot of other issues.

361
00:20:58,056 --> 00:21:01,193
You see immune responses with all of these.

362
00:21:01,193 --> 00:21:03,161
So what's your study about?

363
00:21:03,161 --> 00:21:06,698
What do you need to control and not control?

364
00:21:07,499 --> 00:21:10,235
Masking can help,
especially masking outcome assessors.

365
00:21:10,235 --> 00:21:16,141
When in doubt, when you can mask nobody else,
mask the people collecting your data.

366
00:21:16,141 --> 00:21:19,311
Mask them to the hypotheses of the study.

367
00:21:20,612 --> 00:21:21,580
But you need

368
00:21:21,580 --> 00:21:25,684
to specify in your protocols
who is masked, why, how, and to what.

369
00:21:25,684 --> 00:21:29,821
If I break my ankle,
and I'm in a medical products study, actually

370
00:21:29,821 --> 00:21:34,559
the PI of the study
may not need to know what study arm I'm in.

371
00:21:34,559 --> 00:21:36,762
The safety officer might need to know.

372
00:21:36,762 --> 00:21:37,729
The person doing

373
00:21:37,729 --> 00:21:42,467
my surgery might need to know
to see if they have to adjust my anesthesia.

374
00:21:43,201 --> 00:21:46,038
But realistically, I don't need to know.

375
00:21:46,038 --> 00:21:49,308
A lot of people don't need to know.

376
00:21:49,308 --> 00:21:54,546
Everything about blinding is this idea
-- it's like playing like secret spy.

377
00:21:54,546 --> 00:21:56,982
It's need to know information.

378
00:21:56,982 --> 00:22:01,053
All studies should be reproducible,
regardless of your study

379
00:22:01,053 --> 00:22:03,088
design, regardless of your study.

380
00:22:03,088 --> 00:22:06,725
You need a well-defined study population.
Well-defined inclusion/exclusion criteria.

381
00:22:06,725 --> 00:22:12,798
If I had seven people in this room,
you should all decide the same thing

382
00:22:12,964 --> 00:22:16,268
if subject
A should be in or out of the study.

383
00:22:16,268 --> 00:22:18,337
It needs to be well understood.

384
00:22:18,337 --> 00:22:21,606
The study conduct
-- well, someone's having fun out back.

385
00:22:21,606 --> 00:22:23,408
Study conduct needs to be well-described.

386
00:22:23,408 --> 00:22:25,777
How are you going to do your study?

387
00:22:25,777 --> 00:22:28,780
And if somebody is injured
and has to leave quickly,

388
00:22:28,780 --> 00:22:34,119
someone else's got to walk in, are they going
to walk in and do it the same way?

389
00:22:34,119 --> 00:22:35,220
And that's everyone,

390
00:22:35,220 --> 00:22:38,323
from your nurse coordinators
to your statisticians to your data managers.

391
00:22:38,323 --> 00:22:41,760
You have to know that the labs
are going to be processed

392
00:22:41,760 --> 00:22:46,565
the same way, that you're going to go
and try to get people from the same place,

393
00:22:46,565 --> 00:22:50,836
and then every little step of that study
is going to follow the same way.

394
00:22:50,836 --> 00:22:53,038
Can I reproduce the outcome measures?

395
00:22:53,038 --> 00:22:56,141
How are you collecting those?
How are you doing it?

396
00:22:56,141 --> 00:23:00,178
We'll talk more about that later
and also that data and analyses.

397
00:23:00,178 --> 00:23:00,812
There're

398
00:23:00,812 --> 00:23:05,784
a lot of potential biases in clinical trials,
and there are a lot of potential remedies.

399
00:23:05,784 --> 00:23:10,422
So think about these if somebody says, "Oh,
here's a problem." Well, try to find

400
00:23:10,422 --> 00:23:11,056
your solution.

401
00:23:13,492 --> 00:23:14,793
And you also have

402
00:23:14,793 --> 00:23:19,431
to think we all have the bias of who's
actually in the study, anyway.

403
00:23:19,431 --> 00:23:23,402
You are trying to generalize
to this big, lighter box over here.

404
00:23:23,402 --> 00:23:26,071
But realistically, you've got
those that are interested

405
00:23:26,071 --> 00:23:29,374
in participating, meet the criteria, consent,
and then get randomized.

406
00:23:29,374 --> 00:23:35,313
It's a much smaller group and it may not be
as representative as you'd like it to be.

407
00:23:35,313 --> 00:23:39,618
But that's part of why it's really hard
work to be a trialist.

408
00:23:40,952 --> 00:23:46,558
All of our studies aren't gold,
but we can sure try for it.

409
00:23:46,558 --> 00:23:49,561
So some conclusions
in the next minute-and-a-half.

410
00:23:49,561 --> 00:23:51,263
What is the question?

411
00:23:51,263 --> 00:23:53,832
You've got population or disease, P.

412
00:23:53,832 --> 00:23:56,401
So some people call it PICO.

413
00:23:56,401 --> 00:23:58,136
Some call it PICO(T).

414
00:23:58,136 --> 00:24:03,708
The intervention or variable of interest,
the comparison group, your outcome, and time.

415
00:24:03,708 --> 00:24:06,711
And you want to write this sentence.

416
00:24:07,612 --> 00:24:12,918
This is an example, like, in this population,
how does this intervention or variable

417
00:24:12,918 --> 00:24:17,856
of interest compare to this control influence
the outcome during this time period.

418
00:24:17,856 --> 00:24:20,525
Phrase every single study question like this.

419
00:24:20,525 --> 00:24:22,427
This is your study summary.

420
00:24:22,427 --> 00:24:28,500
When I ask you for a one sentence
summary of your study, write it like this.

421
00:24:28,500 --> 00:24:33,038
And if you cannot fill in those
letters, you have a problem.

422
00:24:34,306 --> 00:24:35,273
The other part

423
00:24:35,273 --> 00:24:39,778
of your study question is who cares
about your study question, other than you?

424
00:24:39,778 --> 00:24:44,616
Your question is always going to come first,
but you've got to consider the questions

425
00:24:44,616 --> 00:24:48,153
you want to ask, those hypotheses
that you're trying to test,

426
00:24:48,153 --> 00:24:51,356
because I've got to turn your question
into something testable.

427
00:24:51,356 --> 00:24:54,259
What are the key factors
you've got to control?

428
00:24:54,259 --> 00:24:57,662
Those ethical issues and constraints?
What can be said?

429
00:24:57,662 --> 00:25:02,200
Maybe you need multiple control groups
in order to actually answer your

430
00:25:02,200 --> 00:25:03,301
questions of interest.