So, I want to
end with an example of a study that kind of highlights
some of the points that I've tried to lay out for you.
And this is a study called ENRICHD.
It was done a couple of years ago.
It's over now.
Called Enhancing Recovery in Coronary Heart Disease Patients.
And the basic premise of ENRICHD comes from data like these.
This is not the ENRICHD data, these are other data from an earlier study.
And it's a phenomenon that we know very well.
And the phenomenon is this, that after somebody has a heart attack,
if they're depressed, they are much more likely to die
than if they're not depressed.
So, what you can see,
this is a man's post-MI, post heart attack.
And within two months post-MI, as you can see here, we see a divergence
of participants who become depressed versus those who do not become depressed.
And you can see the mortality rates just go up.
It's a phenomenon that we're -- it's a very real phenomenon.
So, the ENRICHD trial said, "We need to do something about this.
Let's see what happens if we treat the depression.
Let's try that." So, that's what the ENRICHD trial was.
And they also knew that depression was very strongly linked to social support.
So, they actually had an intervention, a dual intervention,
to target low social support and high depression.
And so, the
hypothesis was that they wanted to test whether treating depression
and low social support right after a heart attack
was going to reduce death rates and non-fatal recurrent heart attacks.
Simple study, right?
It was not simple.
So, ultimately, they enrolled almost 2,500 post-heart attack participants
that either presented with depression or low
social support or both, very commonly both.
And they randomized into a group that got usual care versus one
that got a psychosocial intervention to treat the social support and depression.
And their endpoints were cardiac mortality and a recurrent heart attack.
And they followed them for almost three and a half years.
And they were totally masked in terms of their primary outcomes
and they did an intent to treat analysis.
So, here is what I want you -- no, this is not what I want you to see.
Their inclusionary criteria were that
they were going to enroll people within 28 days of having a heart attack.
Now, they wanted to enroll them quicker.
They wanted to get them more quickly than that.
Because you saw on that graph that the divergence happened at two months,
so they wanted to enroll them as quickly as possible but they knew that
it would not be feasible to enroll them much more quickly than this.
Why is that?
Well, you have a patient who just had a heart attack,
they're in a hospital, there are families involved,
they have other things to think about besides your study.
So, they settled on recruiting them within about a month after a heart attack,
and they had a bunch of criteria
-- they had some inclusionary criteria that could substantiate
the fact that they had a heart attack.
They looked at enzymatic increases.
And then they identified whether they were depressed
and/or had low social support and there are good measures to do that.
Here's what I want to show you.
This is the CONSORT diagram.
They screened almost 34,000 people to get 2,481.
So, they screened this number.
Some didn't meet criteria for having had
a heart attack, right, but not that many.
Well, assume they had a heart attack, so they met that criteria.
And then some of them, fair number, 7,000 of them almost,
weren't depressed or had social support, so they didn't meet those criteria.
But look at this, 23,000 didn't meet their medical eligibility.
Given what we've just talked about, what does that mean?
Did they
do something wrong with their inclusionary or their exclusionary criteria?
Yeah. You can't say it's wrong.
I mean, these are some of the best researchers
around who were doing this study.
It's not wrong.
But ultimately, when you step back
and look at it, this is where they had the problem.
This is where it happened.
So, it wasn't that it was wrong, but they had to screen so many people
in order to get, you know, 2,500 people because of these eligibility criteria.
Scientifically, it was the right thing to do,
but it made for a very difficult study.
And to operationalize this a little bit more clearly for you,
what this means is that for every 100 participants that they screened
-- these weren't people that called up and said, "I
want to participate." These are 100 participants that they screened,
only 7 were actually enrolled and randomized into the trial.
That's a lot of work.
Another way of thinking about
it, in order to get 1, they had to screen 14.
And of course, they had a lot of sites, and not all
the sites were really able to adequately enroll participants.
And that's something that when you have a multi-site trial
you will find that sites recruit at different rates.
There may be reasons to keep sites that have low recruitment
because they're recruiting, you know, particular
characteristics of the patient population that you're interested in.
But it is a phenomenon that occurs fairly frequently.
So, you know, there's been a lot of talk among the investigators
about why there were these differences, why they have these problems
with recruitment and retention.
And some of them had to do with access.
Some of them had to do with the fact that they were recruiting
at very research-heavy sites, so there was a lot of research going on.
And you have to think about that, is there a lot of stuff
going on in this area where you're going to be recruiting?
They didn't have that quite collaborative teams,
and so that reduced their ability to recruit.
And their burden was high, so these participants had to come in
and participate in a psychosocial intervention, which
always is a little bit more burdensome than taking a pharmacologic pill.
And they had a lot of assessments
and the study went on for three and a half years, right?
So, this was a very burdensome study.
They -- but the real key is here, they had really restrictive eligibility
criteria, as you can see from the exclusion,
the high number of patients that were excluded.
So, let me walk through a couple of overall conclusions.
Know your literature.
Know the history of your literature, know where things sit, be able
to identify contextual factors,
know where your question lies on that research continuum.
That's really critical.
Make sure you're designing the right study for the right time for that literature.
Make sure you have clarity on your question.
And this sounds kind of silly, like, "Of course I have clarity on my questions.
I want to know if this does this." But understanding the complexity
of your question really means that you have to dive deep
into some of the issues that we're talking about and more, as well,
to understand what your -- you know, all of the complexities of your question.
Have a really good, very specific targeted understanding
of the patient characteristics that you want to target.
Make sure the participants match the outcomes, your primary outcomes,
make sure it's a good match, it's a feasible match.
And make sure that you're asking the right question at the right time.
Going back to internal versus external validity, figure out
in the context of your question, what's most important; or in other words,
think about what you worry about the most, false negatives or false positives?