1 00:00:08,375 --> 00:00:12,746 So, we can go a little bit through this Phase III example. 2 00:00:12,746 --> 00:00:15,648 And here, this is actually a survival example. 3 00:00:15,648 --> 00:00:18,551 So, our primary objective here was to determine 4 00:00:18,551 --> 00:00:21,821 if patients with metastatic melanoma, who underwent this procedure 5 00:00:21,821 --> 00:00:26,192 had a different overall survival compared with patients receiving standard of care. 6 00:00:26,192 --> 00:00:29,796 So, remember, we have to then define standard of care. 7 00:00:29,796 --> 00:00:34,034 But they were trying to do all this and the idea was 8 00:00:34,034 --> 00:00:38,304 that we're going to have a two-arm randomized Phase III single institution trial. 9 00:00:38,304 --> 00:00:44,411 We don't see many Phase III trials done at one site, but this one was going to be. 10 00:00:44,411 --> 00:00:48,214 So, they came to us and said, "How many patients do 11 00:00:48,214 --> 00:00:52,819 we need to enroll?" We want to have a 1-to-1 ratio between the arms. 12 00:00:52,819 --> 00:00:57,090 So, we're going to equally randomize to the two study arms. 13 00:00:57,090 --> 00:00:59,059 We want 80 percent power 14 00:00:59,059 --> 00:01:03,730 to detect a difference between eight-month median survival and 16-month median survival. 15 00:01:03,730 --> 00:01:09,202 Again, then you need to look and say, "Okay, what's currently on the market? 16 00:01:09,202 --> 00:01:10,370 What's going on? 17 00:01:10,370 --> 00:01:16,209 Do these values still make sense?" They added two-tailed 0.05 test they wanted to run. 18 00:01:16,209 --> 00:01:22,449 And they wanted to do 24 months of follow-up after the last patient had been enrolled. 19 00:01:22,449 --> 00:01:26,553 And they were planning on doing 36 months of accrual. 20 00:01:26,553 --> 00:01:29,089 These lovely things are called nomograms. 21 00:01:29,089 --> 00:01:33,359 So, I have plenty of books with lots of formulas. 22 00:01:33,359 --> 00:01:38,031 And I have to say, actually, of all sample size calculations, 23 00:01:38,031 --> 00:01:41,434 probably I like survival analysis ones the most 24 00:01:41,434 --> 00:01:45,705 because you get to move a lot of different things. 25 00:01:45,705 --> 00:01:51,244 But some of the best statisticians I work with keep photocopying this picture. 26 00:01:51,244 --> 00:01:54,647 There's this paper that basically has different figures 27 00:01:54,647 --> 00:01:58,051 and the number of patients per treatment group. 28 00:01:58,051 --> 00:02:03,690 So, you set -- they have different pictures for alpha and beta combinations. 29 00:02:03,690 --> 00:02:10,396 The dashed lines are for two-sided tests and the solid lines are for one-sided tests. 30 00:02:10,396 --> 00:02:12,765 So, what do you do? 31 00:02:12,765 --> 00:02:14,501 And I said, "Okay. 32 00:02:14,501 --> 00:02:20,974 First, my follow-up period is two years." So, you draw a line that is perpendicular 33 00:02:20,974 --> 00:02:27,447 to this down to intersect the line for your accrual period, which was 36 months. 34 00:02:27,447 --> 00:02:30,016 So, you draw that line out. 35 00:02:30,016 --> 00:02:34,721 And then where line one and line two intersect, you draw 36 00:02:34,721 --> 00:02:39,425 straight up parallel to this y-axis with the number of patients. 37 00:02:39,425 --> 00:02:42,829 You draw straight up. You say, "All right. 38 00:02:42,829 --> 00:02:49,235 R is the ratio between the two median survival values." They said that they wanted 39 00:02:49,235 --> 00:02:55,708 to basically double median survival from eight to 16 months, they had a two-sided 40 00:02:55,708 --> 00:02:56,109 test. 41 00:02:56,109 --> 00:03:00,580 So, we'd go from the dash line for R equals 2.0. 42 00:03:00,580 --> 00:03:05,285 We draw a line parallel to this x-axis, boom, 37 patients. 43 00:03:05,285 --> 00:03:07,921 So, they have all these pictures. 44 00:03:07,921 --> 00:03:13,226 And no joke, you will see the statisticians like drawing with their lines. 45 00:03:13,226 --> 00:03:15,662 You can do it online now. 46 00:03:15,662 --> 00:03:17,697 They have a little app. 47 00:03:17,697 --> 00:03:20,533 Schoenfeld and Richter made these pictures back 48 00:03:20,533 --> 00:03:24,604 before we had computers that easily calculated those for us. 49 00:03:25,171 --> 00:03:29,108 Now, most statisticians will honestly run a bunch of simulations and simulate 50 00:03:29,108 --> 00:03:33,379 your sample size for you taking into account a lot of different issues 51 00:03:33,379 --> 00:03:36,349 about your study design. But it's a nice check. 52 00:03:36,349 --> 00:03:40,286 I think for folks, it's a low-tech check that you can use. 53 00:03:40,286 --> 00:03:42,922 You can use formulas, you can use simulations. 54 00:03:42,922 --> 00:03:47,860 Almost everybody can draw straight lines, as long as you have a ruler or something 55 00:03:47,860 --> 00:03:49,195 to draw them against. 56 00:03:50,163 --> 00:03:51,030 Oh, I'm sorry. 57 00:03:51,030 --> 00:03:53,933 It wasn't 37. It's 38 per arm that we needed. 58 00:03:53,933 --> 00:03:59,172 So, the decision was to try to recruit 42 people per arm for a total of 84 studies, 59 00:03:59,172 --> 00:04:00,640 because they assumed, you know, 60 00:04:00,640 --> 00:04:04,978 that they were going to have censoring, people were going to drop out, et cetera. 61 00:04:04,978 --> 00:04:07,313 They said that they anticipated approximately 30 patients 62 00:04:07,313 --> 00:04:12,051 a year entering the trial, we said, "Yay, that works." Because sometimes they come back 63 00:04:12,051 --> 00:04:15,955 and you're like, "Oh, you had an accrual period of 36 months. 64 00:04:15,955 --> 00:04:20,860 Well, maybe that's not going to work anymore, because of the number that you think 65 00:04:20,860 --> 00:04:22,495 you can get per year. 66 00:04:22,495 --> 00:04:27,100 We're going to have to jigger and do this estimation back and forth." So, 67 00:04:27,100 --> 00:04:28,401 30 patients a year. 68 00:04:28,401 --> 00:04:30,036 You think in three years 69 00:04:30,036 --> 00:04:34,274 I'll get 90 patients, I should hopefully clear my 84 that I need. 70 00:04:35,642 --> 00:04:37,076 So, again, nomograms. 71 00:04:37,076 --> 00:04:43,182 I showed this picture because you're not always -- sometimes you're up here. 72 00:04:43,182 --> 00:04:49,756 You're not always down here. So, let's think of non-survival simple sample size. 73 00:04:49,756 --> 00:04:54,460 Let's start off with one arm or one sample study. 74 00:04:54,460 --> 00:05:00,566 Then we're going to move into a two-arm sample or a two-arm study. 75 00:05:00,566 --> 00:05:01,501 Cheat trick. 76 00:05:01,501 --> 00:05:08,541 If you have three plus arms, calculate the per arm sample size for two-arm study. 77 00:05:08,541 --> 00:05:10,243 Use that per arm. 78 00:05:10,243 --> 00:05:14,480 It doesn't always work, but it's typically a reasonable estimate 79 00:05:14,480 --> 00:05:18,518 of how many people you need per study arm. 80 00:05:18,518 --> 00:05:22,555 So, let's go back to the single sample test. 81 00:05:22,555 --> 00:05:25,091 I've got a new sleep aid. 82 00:05:25,091 --> 00:05:31,464 I want to look at baseline sleep time after taking the medication for a week. 83 00:05:31,464 --> 00:05:37,136 I'm going to run a two-sided test, alpha equals 0.05, power of 90 percent. 84 00:05:37,136 --> 00:05:39,238 The difference is, on average, 1. 85 00:05:39,238 --> 00:05:44,877 So, I think on average, people are going to have four hours of sleep at baseline. 86 00:05:44,877 --> 00:05:50,850 And I'm going to move them to five hours of sleep after my intervention for one week. 87 00:05:50,850 --> 00:05:52,251 Standard deviation, 2 hours. 88 00:05:52,251 --> 00:05:54,020 These are not real numbers. 89 00:05:54,020 --> 00:05:56,823 They are numbers to make this thing move. 90 00:05:58,458 --> 00:06:00,426 So, now, I've got my equations. 91 00:06:00,426 --> 00:06:02,428 I've got this one sample test. 92 00:06:02,428 --> 00:06:05,732 You'll notice that there's no 4 in front of this. 93 00:06:05,732 --> 00:06:10,670 That's because the 2 and the 2 before were because we had a two-sample test. 94 00:06:10,670 --> 00:06:15,274 So, 2 times 2 equals 4, which is why that 4 is there usually. 95 00:06:15,274 --> 00:06:19,579 But I have kind of that critical value for my Type I error. 96 00:06:19,579 --> 00:06:23,549 It's a two-sided test that was divided by two. 97 00:06:23,549 --> 00:06:28,388 Critical value for the power, variance, and the difference of interest. 98 00:06:28,388 --> 00:06:34,127 So, for an alpha equals 0.05 two-sided test, my Z value is 1.960. 99 00:06:34,127 --> 00:06:38,531 For the 80 percent power, my Z value is 1.282. 100 00:06:38,531 --> 00:06:41,200 Add those together, and square it. 101 00:06:41,200 --> 00:06:46,038 My standard deviation was 2, so I'm going to square that. 102 00:06:46,038 --> 00:06:49,142 So, I have 4 for my variance. 103 00:06:49,609 --> 00:06:53,312 The difference of interest is 1 squared is 1. 104 00:06:53,312 --> 00:06:57,417 If I calculate this, it says I need 42.04 people 105 00:06:57,417 --> 00:07:02,121 in my sleep study, which means I need at least 43. 106 00:07:02,121 --> 00:07:06,426 Let's say now I want a two-hour difference in sleep. 107 00:07:06,426 --> 00:07:12,999 So, I want to measure and then go from 4 to 6, not 4 to 5. 108 00:07:12,999 --> 00:07:15,902 My sample size shifts down to 11. 109 00:07:17,103 --> 00:07:19,005 How did I get 11? 110 00:07:19,005 --> 00:07:23,209 Well, remember what I said, it's easier to see big differences. 111 00:07:23,209 --> 00:07:28,548 It's easy to see if people have on average two hours more of sleep. 112 00:07:28,548 --> 00:07:31,584 It's easier to see that than one hour. 113 00:07:31,584 --> 00:07:36,155 It might be harder in reality to make that change happen, however. 114 00:07:36,155 --> 00:07:39,192 But I took this value in the denominator. 115 00:07:39,192 --> 00:07:42,628 Remember, it's a value squared on top of that. 116 00:07:43,529 --> 00:07:48,301 So, you can -- if you're more math algebra-oriented, remember the formula. 117 00:07:48,301 --> 00:07:53,072 If you're not, just remember, if you increase the difference of interest, 118 00:07:53,072 --> 00:07:56,843 you decrease your sample size for the same power. 119 00:07:56,843 --> 00:08:04,183 But the sample size of 11 leads us back to one of those little tricks in the front. 120 00:08:04,183 --> 00:08:07,386 I found this using the Z critical values. 121 00:08:07,386 --> 00:08:09,355 But it's a small number. 122 00:08:09,355 --> 00:08:13,326 So, I need to do this iteration with the T's. 123 00:08:14,527 --> 00:08:18,264 If you are not a math-y person, do not stress. 124 00:08:18,264 --> 00:08:21,267 Your computer will typically do it for you. 125 00:08:21,267 --> 00:08:26,873 If you are trying to plug numbers in, you need to stress and remember this. 126 00:08:26,873 --> 00:08:31,010 So, use -- you first calculate it using your regular formula. 127 00:08:31,010 --> 00:08:33,246 Then you take that sample size, 128 00:08:33,246 --> 00:08:38,117 and you're going to have N minus 1 degrees of freedom, down here. 129 00:08:38,117 --> 00:08:41,854 For a simple one sample test, you're going to find 130 00:08:41,854 --> 00:08:46,692 the critical value of T for 10 degrees of freedom in this case. 131 00:08:46,692 --> 00:08:49,228 Recalculate the sample size using the T's, 132 00:08:49,228 --> 00:08:53,966 instead of the Z's, you're going to find a sample size of 13. 133 00:08:53,966 --> 00:08:55,801 You repeat this whole thing. 134 00:08:55,801 --> 00:09:00,540 You iterate it down until you get a single sample size that sticks. 135 00:09:00,540 --> 00:09:03,442 Sometimes you have to iterate this a lot. 136 00:09:03,442 --> 00:09:04,911 Again, only for those 137 00:09:04,911 --> 00:09:10,016 that bother to do this essentially by hand are going to program their own. 138 00:09:10,016 --> 00:09:11,284 Do you care about this? 139 00:09:13,986 --> 00:09:16,622 Although, you should check, you know, this auto formula. 140 00:09:16,622 --> 00:09:19,892 Is it using the T or is it using the Z? 141 00:09:19,892 --> 00:09:21,961 If you use this T's, you're good. 142 00:09:21,961 --> 00:09:26,966 If you use the Z for the critical values, you may want to use a different formula. 143 00:09:26,966 --> 00:09:31,103 So, let's say my power -- before, I wanted a power of 90 percent. 144 00:09:31,103 --> 00:09:32,605 Now, I want 80 percent. 145 00:09:32,605 --> 00:09:35,541 Oops, I'm sorry, I had the wrong Z there before. 146 00:09:35,541 --> 00:09:37,910 My sample size goes from 11 to eight. 147 00:09:38,911 --> 00:09:42,882 Again, I need rethinking the T distribution on Z distribution. 148 00:09:42,882 --> 00:09:48,421 But this value in the numerator just got smaller because my power got smaller. 149 00:09:48,421 --> 00:09:54,393 If your power you're entrusted in is lowered, your sample size as necessary as lowered. 150 00:09:54,393 --> 00:09:59,532 Standard deviation -- oh, and I should say here, you do not need 151 00:09:59,532 --> 00:10:01,934 an actual sample size of eight. 152 00:10:01,934 --> 00:10:07,873 You need something larger because your math for all your regressions are going to fail. 153 00:10:08,341 --> 00:10:12,011 But this is kind of your starting point. 154 00:10:12,011 --> 00:10:13,646 It's the eight. 155 00:10:13,646 --> 00:10:17,116 Standard deviation goes from 2 to 3. 156 00:10:17,116 --> 00:10:24,957 My sample size has now moved from eight to 13 -- I'm sorry, eight to 18. Why? 157 00:10:24,957 --> 00:10:29,128 Because I squared that 3. It's in my numerator. 158 00:10:29,128 --> 00:10:33,532 Increased variance means you need an increased sample size. 159 00:10:33,532 --> 00:10:36,535 Let's say you remembered that Dr. 160 00:10:36,535 --> 00:10:43,075 Johnson told you to do a randomized two-arm design, don't do these single arm studies. 161 00:10:43,075 --> 00:10:49,749 You can if you need to, but you're like, "I'm going to do better." If you take 162 00:10:49,749 --> 00:10:55,254 the original design -- remember, we needed 43 people in the one sample design. 163 00:10:55,254 --> 00:10:58,824 We have -- we stuck with our original numbers. 164 00:10:58,824 --> 00:11:04,330 Two-sample study needs twice of that in each group in order to account for 165 00:11:04,330 --> 00:11:09,068 the variance, because now you have variance in two different study arms. 166 00:11:09,068 --> 00:11:11,037 You have even more variance. 167 00:11:12,071 --> 00:11:14,840 So, because of that, I need 168 00:11:14,840 --> 00:11:19,412 85 people in each study arm or 170 people total. 169 00:11:19,412 --> 00:11:22,615 And then you go "Well, didn't Dr. 170 00:11:22,615 --> 00:11:27,219 Johnson say sometimes single arm studies were okay?" So, again, 171 00:11:27,219 --> 00:11:30,423 you're going back and forth on this. 172 00:11:30,423 --> 00:11:32,258 Here are your 2s. 173 00:11:32,258 --> 00:11:36,595 Those 2s are because you have a two-sample study. 174 00:11:36,595 --> 00:11:43,269 Again, five-arm study, calculate that sample size per study arm, multiply it by 5. 175 00:11:43,269 --> 00:11:46,472 It doesn't consider all the multiple comparisons. 176 00:11:46,472 --> 00:11:50,142 It depends on the analysis techniques you're using. 177 00:11:50,142 --> 00:11:54,947 But this is your rough estimate to get you started. 178 00:11:54,947 --> 00:11:57,483 So, think about it again. 179 00:11:57,516 --> 00:12:02,488 You change the difference of interest from one hour to two hours, 180 00:12:02,488 --> 00:12:05,791 that 170-person study goes to 44 people total. 181 00:12:05,791 --> 00:12:07,059 Sounds pretty sweet. 182 00:12:07,059 --> 00:12:10,763 Changing my power from 90 percent to 80 percent, 183 00:12:10,763 --> 00:12:14,100 the sample size goes down to some more. 184 00:12:14,100 --> 00:12:18,637 My standard deviation maybe needed to be a little bit higher, 185 00:12:18,637 --> 00:12:21,540 my sample size has gone back up. 186 00:12:21,540 --> 00:12:26,078 So, in conclusion, a lot of things drive your sample size. 187 00:12:27,379 --> 00:12:31,450 You can either remember the directions, or you can remember the formula 188 00:12:31,450 --> 00:12:34,520 and that will help you figure out the directions. 189 00:12:34,520 --> 00:12:39,125 But just remember, basically everything about your study is driving that sample size. 190 00:12:39,125 --> 00:12:43,028 So, are there other designs? Sure, there are tons of designs. 191 00:12:43,028 --> 00:12:48,134 If you do a matched pair design, essentially this looks like the one sample formula. 192 00:12:48,134 --> 00:12:52,905 If you're doing means, you're going to do the formula for a paired t-test. 193 00:12:53,372 --> 00:12:58,043 It's going to look at the mean difference from the paired data 194 00:12:58,043 --> 00:13:00,412 and the variance of the differences. 195 00:13:00,412 --> 00:13:01,580 And proportions, proportions 196 00:13:01,580 --> 00:13:06,085 are actually based on discordant pairs for your sample size calculation. 197 00:13:06,085 --> 00:13:10,990 The textbook has various examples with paired designs, two and one sample 198 00:13:10,990 --> 00:13:12,525 means, the various proportions. 199 00:13:12,525 --> 00:13:19,799 And we also talk a little bit about how you take pilot data and design your next study. 200 00:13:19,799 --> 00:13:25,070 I mentioned I don't like Cohen's effect sizes, but it is very popular 201 00:13:25,070 --> 00:13:30,543 to use them in sample size calculations when you don't have any other information. 202 00:13:30,543 --> 00:13:35,648 And Cohen himself who has a very commonly used textbooks in the psychological 203 00:13:35,648 --> 00:13:39,952 literature says, "Don't use them unless you don't have a choice." 204 00:13:40,953 --> 00:13:42,154 A large effect 205 00:13:42,154 --> 00:13:46,225 size is typically 0.8, medium is 0.5, small is 0.2. 206 00:13:46,225 --> 00:13:51,897 Medium -- again, you'll use the same sample size regardless of what you're measuring. 207 00:13:51,897 --> 00:13:55,534 That's the reason Cohen says, "Please don't do this."