1
00:00:05,305 --> 00:00:05,506
I have a

2
00:00:05,506 --> 00:00:09,042
question about the threshold
for certainty. Yes.

3
00:00:09,243 --> 00:00:10,944
Or uncertainty?

4
00:00:10,944 --> 00:00:13,844
So in particular,
I guess I was wondering, let's

5
00:00:13,844 --> 00:00:14,448
say that,

6
00:00:15,649 --> 00:00:17,518
scientists just want to, like,
see

7
00:00:17,518 --> 00:00:20,521
if a study is is replicable.

8
00:00:20,721 --> 00:00:23,056
It seems like they'd have
a certain degree of certainty,

9
00:00:23,056 --> 00:00:23,223
but

10
00:00:23,223 --> 00:00:25,325
it's still valuable to, to do it
again.

11
00:00:25,325 --> 00:00:26,393
So. Yeah.

12
00:00:26,393 --> 00:00:27,227
Great point.

13
00:00:27,227 --> 00:00:30,230
And so.

14
00:00:30,631 --> 00:00:33,511
Equipoise
for having this community

15
00:00:33,511 --> 00:00:34,334
agreement

16
00:00:34,334 --> 00:00:39,373
about uncertainty is about the
first time

17
00:00:39,373 --> 00:00:42,376
you're doing a study right now

18
00:00:42,943 --> 00:00:47,047
in terms of replication
to a couple important things.

19
00:00:47,414 --> 00:00:51,318
One is replication is always
good, right?

20
00:00:51,318 --> 00:00:54,454
Like if you find an outcome
that is positive,

21
00:00:54,855 --> 00:00:57,126
it's good to replicate that
finding,

22
00:00:57,126 --> 00:00:58,892
at least in one more trial.

23
00:00:58,892 --> 00:01:01,447
It's really the exception to the
rule

24
00:01:01,447 --> 00:01:01,862
that,

25
00:01:02,296 --> 00:01:04,965
one trial changes clinical
practice.

26
00:01:07,200 --> 00:01:08,201
We could talk more about

27
00:01:08,201 --> 00:01:10,700
examples of that, but often it's
the case

28
00:01:10,700 --> 00:01:12,773
that this group is doing a
trial.

29
00:01:12,773 --> 00:01:15,309
They find this
another group is doing a trial.

30
00:01:15,309 --> 00:01:19,179
They find this a group of people
look at that data and say, okay,

31
00:01:19,179 --> 00:01:22,182
we have enough now
to change clinical practice.

32
00:01:22,482 --> 00:01:25,185
So there's value in that
replication, but

33
00:01:25,185 --> 00:01:28,555
it's a different expectation
of uncertainty.

34
00:01:28,555 --> 00:01:31,491
Once you've done the initial
trial,

35
00:01:31,491 --> 00:01:34,643
the other important thing though
is that

36
00:01:34,643 --> 00:01:38,031
as I'm sure you all know, not
every study,

37
00:01:38,031 --> 00:01:40,474
especially those with negative
results,

38
00:01:40,474 --> 00:01:42,102
appear in the literature.

39
00:01:42,602 --> 00:01:46,106
So it's another reason
why it's really important

40
00:01:46,106 --> 00:01:48,842
to have negative results
in the literature.

41
00:01:48,842 --> 00:01:52,012
And I don't mean negative by,
you know, bad results.

42
00:01:52,012 --> 00:01:53,246
What I mean is

43
00:01:53,246 --> 00:01:55,151
let's say you test A versus B
and

44
00:01:55,151 --> 00:01:57,517
you find out that they are no
different.

45
00:01:57,884 --> 00:02:00,502
Well, maybe the company that
sponsored

46
00:02:00,502 --> 00:02:03,256
you isn't interested in moving
forward.

47
00:02:03,256 --> 00:02:05,425
Maybe the journal doesn't really
care.

48
00:02:05,425 --> 00:02:06,994
Like you didn't find anything.

49
00:02:06,994 --> 00:02:08,762
You just found that they're the
same.

50
00:02:08,762 --> 00:02:11,269
So if that doesn't go into the
literature,

51
00:02:11,269 --> 00:02:13,000
then someone else says, hey,

52
00:02:13,000 --> 00:02:14,201
I want to ask this question.

53
00:02:14,201 --> 00:02:15,669
I wonder if anybody has.

54
00:02:15,669 --> 00:02:18,105
They look it up. No. Right. So,

55
00:02:19,473 --> 00:02:21,708
sometimes that community

56
00:02:21,708 --> 00:02:24,209
level of agreement has to be
more

57
00:02:24,209 --> 00:02:27,014
than just looking at the
literature.

58
00:02:27,214 --> 00:02:30,283
Sometimes there's, like a state
of the art

59
00:02:30,283 --> 00:02:32,919
NIH will often sponsor things
like

60
00:02:32,919 --> 00:02:35,653
state of the art treatment for
hepatitis

61
00:02:35,653 --> 00:02:38,592
C, and they get all the experts
in a room,

62
00:02:38,592 --> 00:02:42,396
and they develop what they think
might be a research

63
00:02:42,396 --> 00:02:46,933
agenda that ought to influence
how the field goes forward.

64
00:02:47,134 --> 00:02:50,470
So that's a more formal,
you know, community

65
00:02:50,470 --> 00:02:53,974
engagement of finding,
you know, what to do next.

66
00:02:53,974 --> 00:02:56,193
But, often
it's just relies on the

67
00:02:56,193 --> 00:02:56,977
literature.

68
00:02:57,844 --> 00:02:59,646
Yeah. Yeah.

69
00:02:59,646 --> 00:03:01,014
This is, super interesting.

70
00:03:01,014 --> 00:03:03,036
My question is regarding the
data

71
00:03:03,036 --> 00:03:04,751
monitoring, community. Yes.

72
00:03:05,218 --> 00:03:08,221
So for an IRB, it's usually
like,

73
00:03:08,221 --> 00:03:10,027
there's like some clinician
scientists

74
00:03:10,027 --> 00:03:11,024
and other scientists

75
00:03:11,024 --> 00:03:12,626
and then some non-scientists.

76
00:03:12,626 --> 00:03:14,661
But I'm wondering for the,

77
00:03:14,661 --> 00:03:17,950
the monitoring communities,
are those usually just comprised

78
00:03:17,950 --> 00:03:18,498
of people

79
00:03:18,498 --> 00:03:20,672
that are able, that are going to
be able

80
00:03:20,672 --> 00:03:22,302
to, like, interpret raw data?

81
00:03:22,302 --> 00:03:24,204
Or do they also have
non-scientists?

82
00:03:24,204 --> 00:03:27,328
And does it
like if it's, I don't know, like

83
00:03:27,328 --> 00:03:27,541
a,

84
00:03:29,176 --> 00:03:31,278
a trial related,

85
00:03:31,278 --> 00:03:34,084
to, cancer or is it going to be
mostly

86
00:03:34,084 --> 00:03:35,782
like ecologists, like.

87
00:03:35,782 --> 00:03:36,817
Great question.

88
00:03:36,817 --> 00:03:40,654
So and I'm sorry
I didn't mention this earlier.

89
00:03:40,654 --> 00:03:42,489
So thank you so much for the
question.

90
00:03:42,489 --> 00:03:46,326
So in general, they are, as I
said,

91
00:03:46,326 --> 00:03:50,197
either assigned to a trial
or assigned to a set of related

92
00:03:50,197 --> 00:03:50,730
trials.

93
00:03:51,098 --> 00:03:53,867
And so there will always be

94
00:03:53,867 --> 00:03:56,442
experts in that field on the
data

95
00:03:56,442 --> 00:03:58,939
monitoring committee or whoever

96
00:03:58,939 --> 00:04:01,181
is creating the data monitoring
committee,

97
00:04:01,181 --> 00:04:03,210
whether it's the a steering
committee

98
00:04:03,210 --> 00:04:05,654
for the study or a sponsor is
saying

99
00:04:05,654 --> 00:04:07,080
you must have a DMC.

100
00:04:07,581 --> 00:04:10,037
There's always going to be
expertise

101
00:04:10,037 --> 00:04:12,152
about the particular condition

102
00:04:12,152 --> 00:04:15,122
on the DMC.

103
00:04:15,455 --> 00:04:17,958
In addition to that, in my
experience,

104
00:04:17,958 --> 00:04:20,527
there's also always a
biostatistician.

105
00:04:20,794 --> 00:04:22,028
And so.

106
00:04:22,028 --> 00:04:25,481
Right, I mean, I know enough
bio stats to sort of get me to

107
00:04:25,481 --> 00:04:25,832
hear,

108
00:04:25,832 --> 00:04:29,803
but you really need
someone who knows a lot

109
00:04:29,803 --> 00:04:32,038
about biostatistics
with some of these studies

110
00:04:32,038 --> 00:04:34,474
that are very complicated
in terms of their analysis.

111
00:04:35,442 --> 00:04:37,801
And in my experience,
there's an ethicist at the

112
00:04:37,801 --> 00:04:38,145
table.

113
00:04:38,145 --> 00:04:39,646
So I don't know.

114
00:04:39,646 --> 00:04:44,057
That is not every group
is going to have content

115
00:04:44,057 --> 00:04:44,885
experts,

116
00:04:44,885 --> 00:04:47,331
a biostatistician and an
ethicist,

117
00:04:47,331 --> 00:04:49,489
but a lot of some of them do.

118
00:04:50,857 --> 00:04:54,060
Now, the other thing that is
true

119
00:04:54,060 --> 00:04:59,432
is that there's
usually someone on the study

120
00:04:59,633 --> 00:05:03,861
who's a biostatistician
who is not blinded to the

121
00:05:03,861 --> 00:05:04,638
outcome,

122
00:05:05,238 --> 00:05:08,241
and his or her job is to prepare

123
00:05:08,241 --> 00:05:10,700
a report
for the data monitoring

124
00:05:10,700 --> 00:05:11,545
committee.

125
00:05:11,811 --> 00:05:15,816
And part of the data monitoring
committee is going to be closed.

126
00:05:15,816 --> 00:05:17,678
That is, that only the data
monitoring

127
00:05:17,678 --> 00:05:18,952
committee members are on,

128
00:05:19,286 --> 00:05:22,138
because they're going to be
given data

129
00:05:22,138 --> 00:05:24,691
that if the Pi saw, it might
bias

130
00:05:24,691 --> 00:05:27,083
his or her, you know, choices or
decisions

131
00:05:27,083 --> 00:05:28,962
or engagement with participants.

132
00:05:29,229 --> 00:05:32,399
So there's a inside man or
inside woman

133
00:05:32,399 --> 00:05:36,292
who's preparing that data,
and they're an expert in their

134
00:05:36,292 --> 00:05:36,770
field.

135
00:05:36,770 --> 00:05:40,022
And then the biostatistician
can also look at that and give

136
00:05:40,022 --> 00:05:40,574
feedback.

137
00:05:40,807 --> 00:05:42,501
And there might in my
experience,

138
00:05:42,501 --> 00:05:44,144
I know Dave has experience too.

139
00:05:44,144 --> 00:05:45,478
They're relatively small.

140
00:05:46,413 --> 00:05:49,130
You know, there are 3 or 4
content

141
00:05:49,130 --> 00:05:49,850
experts.

142
00:05:50,483 --> 00:05:53,420
If it's a network trial,
there might be a couple more

143
00:05:53,420 --> 00:05:56,020
because you need a little more
expertise

144
00:05:56,020 --> 00:05:58,425
to cover the waterfront, as it
were.

145
00:05:59,125 --> 00:06:02,128
But that's it's different in
the.

146
00:06:02,329 --> 00:06:05,232
Not every IRB has the specific

147
00:06:05,232 --> 00:06:07,759
expertise,
like a data monitoring

148
00:06:07,759 --> 00:06:08,602
committee.

149
00:06:08,602 --> 00:06:10,070
What,

150
00:06:10,070 --> 00:06:11,796
Maxine asks, could you give
example

151
00:06:11,796 --> 00:06:13,473
of the most common reason
studies

152
00:06:13,707 --> 00:06:17,010
fail in phase three or phase
four?

153
00:06:17,210 --> 00:06:20,013
I'm trying to understand
why so many interventions do

154
00:06:20,013 --> 00:06:22,983
well through phase two
and then crash and burn.

155
00:06:22,983 --> 00:06:28,134
So basically,
one of the reasons why it goes

156
00:06:28,134 --> 00:06:29,422
down from,

157
00:06:29,422 --> 00:06:32,659
you know, 100 in phase one, down
to 70 in

158
00:06:32,659 --> 00:06:35,996
phase two, down to 38 ish in
phase three.

159
00:06:35,996 --> 00:06:40,655
Is is time right
that those early studies are

160
00:06:40,655 --> 00:06:42,002
about safety

161
00:06:42,269 --> 00:06:45,505
and biological metabolism

162
00:06:45,505 --> 00:06:49,743
for looking at whether or not
in a phase two, whether or not

163
00:06:49,743 --> 00:06:52,137
the particular intervention has
an effect

164
00:06:52,137 --> 00:06:53,947
on a particular type of tumor.

165
00:06:53,947 --> 00:06:58,166
In oncology, for example,
it isn't until the phase three

166
00:06:58,166 --> 00:06:58,618
trial

167
00:06:58,618 --> 00:07:01,888
where you are now giving the
intervention

168
00:07:01,888 --> 00:07:05,925
to a large group of regular old
people

169
00:07:06,493 --> 00:07:08,801
in the sense
that, you know, you've only

170
00:07:08,801 --> 00:07:09,262
exposed

171
00:07:09,262 --> 00:07:12,722
a relatively small number
in those phase one and phase two

172
00:07:12,722 --> 00:07:13,199
trials.

173
00:07:13,566 --> 00:07:16,703
And so it's in the phase three
where you're actually

174
00:07:16,703 --> 00:07:20,507
doing this sort of head
to head testing of compound

175
00:07:20,507 --> 00:07:24,110
A versus compound
B, or placebo versus compound A,

176
00:07:24,544 --> 00:07:27,213
and you're not going to find

177
00:07:27,213 --> 00:07:30,216
that the new compound is
effective.

178
00:07:30,216 --> 00:07:32,715
Right. You're doing the trial
in order to answer that

179
00:07:32,715 --> 00:07:33,186
question.

180
00:07:33,186 --> 00:07:35,789
You answer the question.
The question is no.

181
00:07:35,789 --> 00:07:38,058
It doesn't move forward to
market.

182
00:07:38,058 --> 00:07:40,405
Now, there could be some other,
you know,

183
00:07:40,405 --> 00:07:42,696
commercial market forces that
influence

184
00:07:42,696 --> 00:07:44,641
whether a particular
intervention

185
00:07:44,641 --> 00:07:47,000
goes to phase, you know, out to
market.

186
00:07:47,233 --> 00:07:49,919
It could be
that the difference is really

187
00:07:49,919 --> 00:07:50,770
small or not

188
00:07:50,770 --> 00:07:53,922
large enough to say
we should therefore change

189
00:07:53,922 --> 00:07:54,607
practice.

190
00:07:54,607 --> 00:07:59,012
So but generally it's
because the trial, the,

191
00:07:59,913 --> 00:08:03,183
the question is answered,
is it more effective?

192
00:08:03,183 --> 00:08:04,384
And the answer is no.