1
00:00:08,675 --> 00:00:09,442
Moving into

2
00:00:09,442 --> 00:00:14,014
Phase II, this is where we see
our screening of new therapies.

3
00:00:14,014 --> 00:00:17,050
So, Phase II, they've gotten through Phase I.

4
00:00:17,050 --> 00:00:19,719
Not killing that many people, that's good.

5
00:00:19,719 --> 00:00:20,453
Phase II,

6
00:00:20,453 --> 00:00:26,926
I'm trying to still learn a lot about safety
but learn a lot more about my doses.

7
00:00:26,926 --> 00:00:30,330
I'm not going to probably
look at final efficacy,

8
00:00:30,330 --> 00:00:35,268
but I might have some type of biomarker
or surrogate outcome that I'm

9
00:00:35,268 --> 00:00:40,206
going to determine kind of a pseudo-efficacy
that I'm going to power on.

10
00:00:40,206 --> 00:00:44,310
Our general idea in Phase
II is, is it safe enough?

11
00:00:44,310 --> 00:00:49,482
And is there sufficient activity enough
to bother to move this to Phase III?

12
00:00:49,482 --> 00:00:52,052
Move these still at large pivotal trials?

13
00:00:52,052 --> 00:00:56,122
Again, if you look at the old definitions,
they'll say, "Well,

14
00:00:56,122 --> 00:01:00,560
should we bother to move it
to a randomized trial?" These days,

15
00:01:00,560 --> 00:01:04,264
almost every Phase
II trial I work on is randomized.

16
00:01:04,264 --> 00:01:07,567
It is a rare exception
that they're not randomized.

17
00:01:09,369 --> 00:01:14,040
Your issues of safety
are still going to be really important here.

18
00:01:14,040 --> 00:01:19,112
And you have a fairly small
number of patients, might be several hundred,

19
00:01:19,112 --> 00:01:23,383
but it still depends --
like I don't see that many

20
00:01:23,383 --> 00:01:28,254
that are more than 100 people per arm,
but sometimes that happens.

21
00:01:28,254 --> 00:01:31,157
Your problems in Phase II is that

22
00:01:31,157 --> 00:01:36,229
you are more likely to run into the unblinded
or single blinded studies

23
00:01:36,229 --> 00:01:42,102
here, have all the problems that we talked
about at the beginning of the course.

24
00:01:42,102 --> 00:01:46,840
You have issues with placebo effect,
investigator bias, regression to the mean.

25
00:01:46,840 --> 00:01:51,911
If you're running a single arm study,
you're not really sure what caused

26
00:01:51,911 --> 00:01:53,480
those changes to happen.

27
00:01:53,480 --> 00:01:57,016
That's the reason that you see this big push.

28
00:01:57,016 --> 00:02:02,122
So, if you work in industry,
you're one of these pharmaceutical or biotech

29
00:02:02,122 --> 00:02:05,258
companies, you're running non-randomized
trials, you're running unblinded

30
00:02:05,258 --> 00:02:09,963
trials, you tend to move into Phase
III, and you get burned.

31
00:02:10,997 --> 00:02:11,698
Everything that

32
00:02:11,698 --> 00:02:15,969
looked fabulous in Phase
II does not pan out in Phase III.

33
00:02:15,969 --> 00:02:20,974
So, now I've just wasted sometimes billions
of dollars running those Phase III trials.

34
00:02:20,974 --> 00:02:26,679
That's the reason that people are getting
a lot tighter in Phase I and Phase II.

35
00:02:26,679 --> 00:02:30,950
Because better to stop the product early,
if it's not producing results.

36
00:02:30,950 --> 00:02:33,086
Same thing in academic medicine, folks.

37
00:02:33,887 --> 00:02:37,190
You're better to fail small, but be wise

38
00:02:37,190 --> 00:02:39,692
about your failures.

39
00:02:41,461 --> 00:02:42,595
And with that,

40
00:02:42,595 --> 00:02:46,833
then I'm going to talk to you
about a non-randomized design.

41
00:02:46,833 --> 00:02:50,703
But actually, randomized designs
have been built out of this.

42
00:02:50,703 --> 00:02:54,941
So, for Phase II, there's something called
a two-stage optimal design.

43
00:02:54,941 --> 00:03:00,713
This was developed out of NCI back
I think in 1980s, if I remember right.

44
00:03:00,713 --> 00:03:06,119
And the idea is the premise
for a lot of current Phase II designs.

45
00:03:06,119 --> 00:03:10,757
You want to seek to rule out
an undesirably low response probability.

46
00:03:10,757 --> 00:03:14,994
So, let's say only 20
percent of people respond to treatment.

47
00:03:14,994 --> 00:03:20,400
I want to rule that out in favor
of something that shows useful activity.

48
00:03:20,400 --> 00:03:25,939
And let's say I've defined useful activity
to be that 40

49
00:03:25,939 --> 00:03:31,211
percent of my patients are stable
or have improved outcomes.

50
00:03:31,211 --> 00:03:36,983
So, in oncology,
they might run a single arm two-stage design

51
00:03:36,983 --> 00:03:41,521
using an optimal design
and a predefined response definition.

52
00:03:41,521 --> 00:03:49,596
It is hard to define a response because then
people want to tweak those numbers, right?

53
00:03:49,596 --> 00:03:54,601
But, again,
if we want to rule out a probability

54
00:03:54,601 --> 00:04:01,674
of 20 percent in favor of 40 percent,
let's say, I set my Type

55
00:04:01,674 --> 00:04:07,146
I error at 0.1 and my Type II error at 0.1,
all right?

56
00:04:07,146 --> 00:04:10,416
So, I'm willing to make -- I'm going to --

57
00:04:10,416 --> 00:04:14,954
this is one of those cases
where I set them equal to each other.

58
00:04:14,954 --> 00:04:18,524
The idea being,
I'm willing to tolerate a Type I error.

59
00:04:18,524 --> 00:04:22,128
And really, if there's something
there, though, I want high power.

60
00:04:22,128 --> 00:04:24,230
I really want to see it.

61
00:04:24,230 --> 00:04:26,666
So, roughly, in this kind of more

62
00:04:26,666 --> 00:04:29,269
probability-based setting,
I've got a 10 percent probability

63
00:04:29,269 --> 00:04:34,140
of accepting a poor agent and a 10 percent
probability of rejecting a good agent.

64
00:04:34,874 --> 00:04:39,979
So, if you'll look in Simon's paper --
and if it's not posted,

65
00:04:39,979 --> 00:04:44,684
I will ask Daniel to post it
up in your class segment.

66
00:04:44,684 --> 00:04:48,988
He has this paper
with all of these different kind of

67
00:04:48,988 --> 00:04:53,326
tradeoffs of these different probabilities
and varying alpha and beta values.

68
00:04:53,326 --> 00:04:58,431
This is something that Steinberger invented
and then built upon in the early

69
00:04:58,431 --> 00:04:59,999
2000s for randomized trials.

70
00:05:01,801 --> 00:05:04,971
So, we're going to blow this up in a second.

71
00:05:04,971 --> 00:05:07,507
This table is Table 1 from Simon's paper.

72
00:05:07,507 --> 00:05:10,677
We have the difference in the probabilities
being 20 percent.

73
00:05:10,677 --> 00:05:11,944
And we've got that

74
00:05:11,944 --> 00:05:16,049
we're going to reject the drug
if the response rate falls through here.

75
00:05:16,049 --> 00:05:20,820
There is something called the optimal design,
which is what Simon was trying to promote,

76
00:05:20,820 --> 00:05:25,258
and one that it was already out
in the literature called the minimax design.

77
00:05:26,592 --> 00:05:29,062
So, let's look up here.

78
00:05:29,062 --> 00:05:33,466
Our null is a response rate of 20 percent.

79
00:05:33,466 --> 00:05:37,370
We want to actually see
40 percent responders.

80
00:05:37,370 --> 00:05:40,306
So what do I do here?

81
00:05:40,306 --> 00:05:47,180
What I want to do is, initially,
I am going to enroll 17 patients.

82
00:05:47,180 --> 00:05:55,021
So, if you'd looked at the tiny footnotes,
we're going to be in kind of this

83
00:05:55,021 --> 00:06:00,393
top row here
-- or the bottom row, the bottom chunk.

84
00:06:01,661 --> 00:06:04,630
So, Stage 1, I enrolled 17 patients.

85
00:06:04,630 --> 00:06:09,469
If zero to three of them
have a clinical response, I stop accrual.

86
00:06:09,469 --> 00:06:12,071
I assume it's not an effective agent.

87
00:06:12,071 --> 00:06:19,512
So, what I'm saying is there is no way I'm
going to be able to rule out this 20 percent.

88
00:06:19,512 --> 00:06:21,381
So, I'm just stopping early.

89
00:06:21,381 --> 00:06:28,087
But if four to 17 patients have a positive
response, we can move on to the second stage.

90
00:06:28,087 --> 00:06:30,690
I'm going to accrue up to 37.

91
00:06:30,690 --> 00:06:33,960
So, I'm going to accrue 20 more patients.

92
00:06:33,960 --> 00:06:40,099
And if 10 of the 37 have a positive response
-- or sorry, if 11

93
00:06:40,099 --> 00:06:46,639
or more of the 37 have a positive response,
then I think it's an active agent.

94
00:06:46,639 --> 00:06:53,579
So, basically, I'm rejecting if I have three
or fewer, then if I have 10 or fewer.

95
00:06:53,579 --> 00:06:56,449
What these other numbers were, are --

96
00:06:56,449 --> 00:07:01,454
you know, here
-- I have a chance on stopping early, right?

97
00:07:01,454 --> 00:07:04,957
It might be I stopped at 17.

98
00:07:04,957 --> 00:07:10,496
I feel like if 17 out of 17,
I'm done realistically.

99
00:07:10,496 --> 00:07:12,999
But let's look at this.

100
00:07:12,999 --> 00:07:17,003
So, if I do the standard sample size,

101
00:07:17,003 --> 00:07:22,008
under the null hypothesis,
I have to enroll 26 patients.

102
00:07:22,008 --> 00:07:27,547
Probability, I'm going to end under the null
that's 55 percent.

103
00:07:27,547 --> 00:07:29,282
It's pretty good.

104
00:07:29,282 --> 00:07:35,555
So, here it is in words for you
what I just said.

105
00:07:35,555 --> 00:07:40,092
Under the design,
if the null hypothesis is true,

106
00:07:40,092 --> 00:07:44,096
there's a 55 percent
probability of early termination.

107
00:07:44,096 --> 00:07:48,601
This is one reason
we like phase designs, right?

108
00:07:49,368 --> 00:07:51,237
We get out early.

109
00:07:51,237 --> 00:07:57,343
It's not 100 percent sure,
but 55 percent is better than 0 percent.

110
00:07:57,343 --> 00:08:03,916
On average in that two-stage optional design
under the null, I enrolled 26 patients.

111
00:08:03,916 --> 00:08:10,056
If I use the one sample test of proportions,
I need 34 patients.

112
00:08:10,056 --> 00:08:17,096
So, 34 is fewer than 37,
but I don't have that chance to stop early.

113
00:08:17,096 --> 00:08:21,300
And that's
what I'm gaining in this two-stage design.

114
00:08:22,201 --> 00:08:27,440
If I use a two sample randomized test
of proportions, I have 86 patients per group.

115
00:08:27,440 --> 00:08:32,311
And that's the reason this --
you have this lore of a single arm study.

116
00:08:32,311 --> 00:08:37,550
Why do I want to have two arms and randomized
when I can get some information?

117
00:08:37,550 --> 00:08:39,819
Maybe there's a problem that I have

118
00:08:39,819 --> 00:08:43,723
so many fewer patients,
it takes me less time, and less money.

119
00:08:44,624 --> 00:08:48,661
This switch
you have to balance as an investigator.

120
00:08:48,661 --> 00:08:52,665
Again,
there are a lot of newer methods available.

121
00:08:52,665 --> 00:08:55,201
They all end up citing Simon.

122
00:08:55,201 --> 00:08:57,970
So, that's why Simon is here.

123
00:08:57,970 --> 00:09:01,140
So, let's talk about historical controls now.

124
00:09:01,140 --> 00:09:05,378
And historical controls,
let's say I've got a rare disease.

125
00:09:05,378 --> 00:09:07,914
I want to double their survival.

126
00:09:07,914 --> 00:09:12,151
Survival rate now is 15.7 months,
and I want to

127
00:09:12,151 --> 00:09:16,822
with my new intervention
double that median survival to 31 months.

128
00:09:18,624 --> 00:09:22,962
I'd say I'm going to have a Type
I error of 0.05.

129
00:09:22,962 --> 00:09:25,131
I want a one tailed test.

130
00:09:25,131 --> 00:09:28,367
And I want my power to be 80 percent.

131
00:09:28,367 --> 00:09:29,802
So, why one tailed?

132
00:09:29,802 --> 00:09:33,406
Basically, if they're dying,
then this is going nowhere, right?

133
00:09:33,406 --> 00:09:36,676
So, if my survival is less, no, that's it.

134
00:09:36,676 --> 00:09:42,448
So, they -- and also, the reason
that they went ahead and did a one tailed

135
00:09:42,448 --> 00:09:49,288
test is they said, "This is a rare disease,
and we have to do everything we can to try

136
00:09:49,288 --> 00:09:51,090
to have a reasonable test."

137
00:09:51,090 --> 00:09:53,192
And because they're like, "It's
going nowhere,"

138
00:09:53,192 --> 00:09:56,128
and they swore they're going to publish it
either way,

139
00:09:56,128 --> 00:10:00,266
they're like, "Yeah, it's interesting,"
and it will be clear that this is wrong.

140
00:10:00,266 --> 00:10:04,103
So, they better went to a test.
You need 60 patients for this.

141
00:10:04,103 --> 00:10:06,772
And we'll talk about how you figure this out.

142
00:10:06,772 --> 00:10:09,442
About 30 in each of the two study arms.

143
00:10:09,442 --> 00:10:11,811
They come back and they say, "Okay, Dr.

144
00:10:11,811 --> 00:10:16,682
Johnson, I can accrue one per month."
It was actually a different doctor

145
00:10:16,682 --> 00:10:22,321
they were talking to you,
but he let me share the example.

146
00:10:22,321 --> 00:10:28,427
So, if you can get 60 patients,
one per month, that's five years

147
00:10:28,427 --> 00:10:35,001
to just enroll them in your study
let alone follow them in the study.

148
00:10:35,001 --> 00:10:37,803
They needed 36 months of follow-up.

149
00:10:37,803 --> 00:10:40,606
So, that's an additional three years.

150
00:10:40,606 --> 00:10:46,712
So, now it's going to take me, and best case
scenario, eight years

151
00:10:46,712 --> 00:10:52,818
from the time I start enrolling patients
to when I stop collecting data.

152
00:10:52,985 --> 00:10:55,955
Yeah,
that's a long time to wait for an answer.

153
00:10:55,955 --> 00:10:56,522
Plus, realize

154
00:10:56,522 --> 00:11:01,227
it probably took you a couple of years
to get this study up and running, it's

155
00:11:01,227 --> 00:11:05,665
going to take you a little bit of time
to analyze and publish your data.

156
00:11:05,665 --> 00:11:08,601
So, they said, "Well,
we want to use historical controls."

157
00:11:08,601 --> 00:11:12,004
We have a bunch of patients,
they were controls in other studies,

158
00:11:12,004 --> 00:11:15,374
and we want to use their data
and put all patients

159
00:11:15,374 --> 00:11:19,478
that we have on the new therapy
and compare them to the old patients.

160
00:11:21,480 --> 00:11:24,250
So, there was this old dataset.

161
00:11:24,250 --> 00:11:26,118
We had 35 patients.

162
00:11:26,118 --> 00:11:30,790
They've been treated in National Cancer
Institute with this disease.

163
00:11:30,790 --> 00:11:35,428
Problem is they were initially treated
from 1980 to 1999.

164
00:11:35,428 --> 00:11:40,099
Of those 35 patients
at the time they were looking

165
00:11:40,099 --> 00:11:44,270
at the records,
three of them are still alive.

166
00:11:44,270 --> 00:11:47,540
Median survival
time for the historical patients,

167
00:11:47,540 --> 00:11:52,211
so where they'd gotten the value
from, was 15.7 months.

168
00:11:52,211 --> 00:11:57,316
And this is basically kind of like
this observational study, right?

169
00:11:58,718 --> 00:12:01,854
But, you know, 1980, it's 2015 now.

170
00:12:01,854 --> 00:12:05,858
There's a lot different
when we diagnose them, how

171
00:12:05,858 --> 00:12:10,730
we diagnose them, what measurements
you're taking, how you're recording it,

172
00:12:10,730 --> 00:12:16,502
how you decide that people are progressing,
the treatments that we're giving them.

173
00:12:16,502 --> 00:12:20,506
Not just who treat their disease,
but everything else,

174
00:12:20,506 --> 00:12:25,411
you know, to help their nausea
and to help their anything.

175
00:12:25,411 --> 00:12:26,746
It's all different.

176
00:12:28,180 --> 00:12:32,184
So, you have to decide, do you use this data?

177
00:12:32,184 --> 00:12:36,589
Is it really controlling
for everything you need to control for?

178
00:12:36,589 --> 00:12:38,591
Does anybody else have data?

179
00:12:38,591 --> 00:12:43,362
Can you work with people around the world
to do this protocol?

180
00:12:43,362 --> 00:12:47,366
How do you do it?

181
00:12:47,666 --> 00:12:52,238
So, these are the problems
that come up in Phase II studies.

182
00:12:52,238 --> 00:12:54,507
You've got to constantly be weighing

183
00:12:54,507 --> 00:12:57,543
the advantages
and disadvantages of your study designs.

184
00:12:57,543 --> 00:13:02,114
You may have a single arm study
that has a small sample

185
00:13:02,114 --> 00:13:06,685
size, that's good in some ways,
but you've got no control information.

186
00:13:06,685 --> 00:13:12,024
You may have a two-stage design, and again,
you might stop early with a

187
00:13:12,024 --> 00:13:17,730
small sample size, but if it's a single arm
two-stage design, you've got no control.

188
00:13:18,864 --> 00:13:23,536
You might have the correct responder,
non-responder rules, or you may not.

189
00:13:23,536 --> 00:13:28,207
Are you sure that that difference
is really what you care about?

190
00:13:28,207 --> 00:13:32,912
Can you make a binary distinction
to really even define a responder?

191
00:13:32,912 --> 00:13:33,679
Historical controls.

192
00:13:33,679 --> 00:13:38,350
And again, it's great
because you can use a small sample size,

193
00:13:38,350 --> 00:13:42,655
but you don't know how accurate
that control group really is.

194
00:13:43,923 --> 00:13:47,726
You might decide to run multiple arms,
sometimes randomized, sometimes not.

195
00:13:47,726 --> 00:13:49,128
Sometimes I see people

196
00:13:49,128 --> 00:13:53,666
run like eight plus arm studies,
but then they have no control arm.

197
00:13:53,666 --> 00:13:57,136
They're just comparing
all the different doses to each other.

198
00:13:57,136 --> 00:13:58,537
I'm like, "That's great.

199
00:13:58,537 --> 00:14:03,042
But what if none of your doses work,
or all your doses work

200
00:14:03,042 --> 00:14:06,512
the same?" Like, "You
don't have a way to compare."

201
00:14:06,512 --> 00:14:09,715
So, if you're going to be running
multiple arms, please

202
00:14:09,715 --> 00:14:14,153
have something with some control in it,
because it could be that every dose

203
00:14:14,153 --> 00:14:17,022
does something
but it all does the same thing.

204
00:14:19,592 --> 00:14:23,062
The downside is it takes a lot more people.

205
00:14:23,062 --> 00:14:28,467
That said, I see more and more Phase
I and Phase II studies enrolling

206
00:14:28,467 --> 00:14:33,505
hundreds of people, because they want
to make sure what they're moving forward

207
00:14:33,505 --> 00:14:38,944
into these final large pivotal trials
is what they've got the best shot at.

208
00:14:38,944 --> 00:14:43,215
And that they'd better understand
a lot of the safety information,

209
00:14:43,215 --> 00:14:44,750
the pharmacokinetics, and pharmacodynamics,

210
00:14:44,750 --> 00:14:49,021
and a lot of other information
about potential efficacy in trials,

211
00:14:49,021 --> 00:14:53,659
especially in multiple different populations,
because that's where we do our trials.