So, what are some changes?
If you have unequal numbers in each group, the ratio of cases to controls,
you want to use this if you have, say, a certain lambda, a
set of patients randomized to the treatment arm
for every patient randomized to the placebo arm.
You can also use this in a case-control setting.
So, the idea here is that you're going to have different variances.
You're going to kind of use this ratio, calculate the sample size for one group,
and then use that to calculate the sample size for the other group.
The shortcut way of doing this,
let's say I'm going to use equal variance sample size formula.
So, I assume both groups have the same variance.
I want to look at the total sample size, calculate that, then increase it
by this factor k.
So, let's say I'm doing 2-to-1 randomization.
I calculate the total sample size.
If I had two equally sized study arms, it's 26, so it's going to have 13 per arm.
I'd say, okay, 26 times 2 plus
1, because that's my ratio, squared, divided by 4 times the 2.
And I find out that I need 30 people in my study instead.
So, I would have 20 in one group, 10 in the other.
Why might I do this?
Well, sometimes I have a device.
Let's say, I have a new artificial knee.
And I want to know how many knees I need to make.
And I want to do an unequal randomization.
This might be a case where I'll use that.
Sometimes I have a fixed number of cases.
I've already made the knees.
I'd only have so many of them to use.
So, if my sample size calculation said I needed 13 per arm,
but I only have 11 of the knees, I want the same precision that I'm going to calculate
how many people will be in my control arm and back calculate that using this formula.
When you're doing survival analysis and other types of analyses,
you may have a cohort of say those exposed and unexposed people that we talked about.
You have this relative risk that you're interested in.
I know the prevalence and the unexposed population.
I need to figure out the number of events.
So, the risk of an event in the exposed group divided by the risk of the event
in the unexposed group, and back
calculate the number of events I think I'm going to be seeing.
So, I have the number of events in my unexposed group, calculate
the number of events in the exposed group, then calculate how many people
I need to see the events. Remember stability issues.
In logistic regression, there's this big discussion,
and a whole bunch of simulations were done that you need at least 10 subjects
for every variable investigated.
You look at principal components analysis and a lot of other methods that get used.
You see everything from 10 to 100.
So, all sorts of crazy.
Some people don't say you have to use balanced designs.
Other people say you have to use them.
I'm just going to say, if you have balanced designs,
you have the same number of people in each study
arm, it's easier to figure out your power and your sample size.
It is easier to
handle, but you know, I have a job to work on complicated designs.
So, you can do simulations.
You can do pretty much anything you want as long as you can
do the work to make sure it's going to be rigorous and valid.
So, what about multiple comparisons?
If you have four groups, and you want to do all two-way
comparisons of means, you have six different tests.
But sometimes you'd say, "Well, I have four groups.
I have one placebo arm and three different doses.
And I actually just want to compare each dose to the placebo.
I'm not going to compare the doses to each other." So, you have to figure out
exactly how many tests you really are going to run.
The simple way people do it is this Bonferroni test, right?
They divide their alpha by the number of tests. It's common.
There's a long literature, but it's super conservative.
And in some of your high-throughput laboratory tests, it can be worked out.
Ed Corin, I think it was, worked this out, and maybe Lisa
or somebody else in that branch.
It said, like, there were cases that it was physically impossible for them
to have any statistically significant results
after they accounted for multiple comparisons.
That's not an experiment worth running.
This comes up a lot and it's like microarray, proteomics, and other omics
types of experiments.
Many times people say, "Just
set your alpha and your beta stricter." I tend to agree with this.
You know, just set a really strict Type
I error level that you're going to adhere to and test everything against that.
If that's what you want to think about for false positives, false negatives,
there is also like familywise error rates, and all these other things.
But I think just set strict upfront and plan for that.
So, what are some of these rejected statements?
Again, this St.
George's Hospital -- you know,
I'm looking at this link, I think this is the correct link.
But I don't remember
if I double checked it this morning because they keep moving some stuff around.
Anyway, this entire -- you can put the name into your favorite search engine and find it.
St. George's Hospital Medical School has one of the best statistics
guys for research grant applicants that I've ever seen.
Like many times, I was like, "Can I just copy and reference it and be done?"
I think it's very applicable across any country that you'd go to.
So, one of my favorite parts, because I see it way too often.
It's people who go, "Me, too." It's like, "No, you must justify your sample size
in every single application." So, this is an example, an exact quote, "A
previous study in this area recruited 150 subjects
and found highly significant results, p value equals 0.014, and therefore similar
sample size should be sufficient here." Well, they could have been lucky.
It could have been random sampling variation.
It could have been a lot of other things.
You need to give an estimate of what you're expecting for the variance,
for the difference, what your Type I error is going to be, et cetera.
You have to give all those little elements.
It's good to know that some study who had similar elements found something significant.
But honestly, if a lot of studies have already found this, why are you doing your study?
No prior information.
Now, this is always trouble because sometimes you do have no prior information.
And what you're trying to do is run the little baby first pilot.
But the quote in the application -- for substantial sample size application,
I should say sample sizes aren't provided because there's no prior information
on which to base them. All right.
We have to know how many people,
how many animals, whatever you are enrolling in this study.
Find something that's published previously, and try to extrapolate from that.
Conduct a small pre-study.
But if this is your application to conduct the small pre-study,
then you probably, in fact, don't need sample size calculations.
Don't tell me the sample sizes aren't provided.
Tell me like this is where you are, and that you're going
to use this amount of information to then calculate appropriate sample sizes.
That this is what you're gathering in order to do it.
But if
I can easily find a bunch of information that could have informed your sample sizes,
I'm probably not going to have a very happy review for you.
Variance? No prior information on standard deviations?
Well, again, this might be where you talk
about detecting the size of difference in terms of the standard deviations.
This is where those effect sizes that Cohen and others
and their different effect size values for different regression models.
But this is where that can be useful.
Again, it's your starting step,
but it's not going to be what you use for a pivotal or final study.
Now, the number of available patients,
Wendy Weber and others are going to talk a little bit more about this,
but you have to make sure you have patients to be in the study.
Few things.
You will not get all patients to be in your study, number one.
In fact, you're lucky if like 5 percent agreed to be in your study in most cases.
Number two, feasible is important.
It does not tell me about your power.
So, this is, again, a direct quote that they got at St.
George's, "The clinic sees around 50 patients
a year, of whom 10 percent may refuse to take part in the study."
I don't think that's
realistic that only 10 percent will refuse, but we keep moving on.
"Therefore over the two years of the study, the sample size will be 90
patients." Let's say a miracle occurs, and they get their 90 patients.
I have no idea what the power is for this study.
I know nothing about the differences that they want to detect.
I don't even know what they want to measure.
I don't know about the variance.
I know nothing.
It's okay to give this information and then say, "And with 90 patients,
our power will be X based on the following assumptions."
That would have been okay, but that sentence never appeared in the application.
So, you do have to balance feasibility
and power, but sample size isn't decided just on available patient numbers.
So, what are some of our resources and conclusions?
What impacts your sample size?
The differences, the standard deviations
or the variance, your Type I error that you're willing to tolerate, your Type II
error that you're willing to tolerate, number of arms or samples, one or two-sided test.
Are you randomizing?
If you are, what type of randomization are you using?
What are the analysis plans?
If you don't have an estimate of the variance, make sample size or power
tables, make graphs.
You should in fact always -- even when you think you have good estimates,
they may not be that good, you're always going to want to make tables and graphs.
Show a wide variety of possible standard deviations, of possible differences.
And protect yourself with a high sample size, if it's at all possible.
So, when I was reviewing grants
for the NIH, the top 10 statistics questions I had to ask back to investigators.
The exact mechanism they were going to use to randomize their patients.
Don't tell me you're just going to randomize them.
Dr. Wakim went through a whole bunch of different ways you can randomize people.
Tell me more information.
Tell me when, et cetera.
They would say they're going to stratify.
I'd be like, "Why are you going to stratify?
What are you going to stratify on?" The EMA, the European Medicines Agency,
actually provides different information on something called dynamic allocation.
So, if you're going to use different types of allocation
schemes, they may or may not be acceptable to different regulatory bodies worldwide.
Or they may be acceptable in pediatrics, but not acceptable in adults,
acceptable on oncology, not acceptable in, you know, cardiovascular disease.
Blinded or masked personnel.
Who is doing your endpoint assessment?
Who is masked?
To what are they masked?
Are you compromising your blinded study with something else that you were doing?
Many times, I had to ask people to list every single hypothesis.
The hypothesis test,
the specific analysis,
and the specific sample size or power justification for those hypothesis tests.
That should be very clear in your grant application.
Somebody shouldn't have to ask you for this.
It should also be clear in the statistical analysis plans that go with it.
And it needs to match everything else in your protocols.
I would need to ask how or if they were adjusting for multiple comparisons.
Some fields get very, very picky about multiple comparisons.
Others say, "Well, if you're honest and you report everything,
you can say I'm not adjusting for multiple comparisons."
Typically, most of your regulators and pivotal trials will make you do
what's called a statistical hierarchy or some way to control the Type I error.
So, it may or may not be technically adjusting for multiple comparisons,
but you are kind of doing a conservation, they call it, of the Type I error.
I would ask if they're testing for effect modification.
Are you doing any interim analyses?
And if so, what exactly are you analyzing? When?
What's the error spending model?
And what are the stopping rules?
So, these error spending models are, again, to conserve your Type I error.
And I would ask if this was accounted for in the sample size calculations.
Why all of those ways that you do interim analyses impact your sample size calculation?
What is your expected dropout?
You know, they would tell me about their sample size,
but not the expected dropout, and did they plus up in order to account for this.
How do you handle the dropouts and missing data in your analyses?
Again, those methods may also impact your sample size and your power.
And then I always had to ask about repeated measures and longitudinal data,
almost always. Are you using a linear mixed model?
Are you trying to use repeated measures ANOVA?
Repeated measures ANOVA is an awful method.
It's very antiquated. It has a lot of assumptions.
The newer methods have a lot fewer assumptions, they were a lot more robust.
Maybe they're going to use generalized estimating equations.
But again, what is your actual method?
How are you collecting the data?
What's the study design?
What are the analyses in order to actually test that data?
All of that goes into figuring out your sample size and the analyses and conclusions
that you can draw.