So, we
have some ideas of DSMBs, I just want to say that they are
extremely helpful and certainly people blind to trial,
phase three trial, they're compulsory, I think.
An NIH policy requires it.
And I think many other institutions would as well.
But not all trials need the DSMB.
There are many other ways to protect the patients.
And you can read about these policies from various institutes.
NIH, just to give you
some general guidelines for when you do need one of these bodies,
all trials need the safety monitoring plan and plan for how they're going
to make sure the data and the trial are conducted as intended
and have integrity and also to protect the safety.
But as soon as you have multicentered phase three trials that an independent
DSMB has been required since 1979, it's also required
that the IRBs are notified of all DSMB recommendations.
And I'd say, beyond that, the policies start to vary between institutes.
I used to work at NIAID, and I know that
as soon as a trial was blinded and randomized, it had a DSMB.
But that is not true across all institutes.
So, it would be even phase two at NIAID.
And so, here are some links that you go to, to read about these guidelines.
The FDA, I actually -- because of very informative guidance
and would recommend reading if you wanted as well.
It's pretty comprehensive if you're interested
in thinking about the best practices informing these committees.
I would say that there is no policy of the FDA
on what you should do or what -- there's no requirements of what is done
but simply recommendations, because they're not governing, say,
pharmaceutical firms or other -- but in terms of how they do these DSMBs,
but they will make recommendations for the best practice.
Same thing for how to do a statistical analysis,
how to monitor your trial, the FDA has a lot of guidance documents.
If you haven't found those, look online, very useful.
I did want to point out also for those who are at NIH,
it was interesting that there's, for quite a while, sort of understanding
that the centers and institutes differ in terms of how they do DSMBs.
And some of them are a lot more open to the idea of the study sponsor
being a heavy participant in the DSMB and the executive session.
And some complaints have lodged
some investigators about an interference or conflict of interest.
And so, the Health and Human Services Office of the Inspector
General took a couple years, I think in 2009 or 2010, and reviewed
all the DSMBs across institutes and centers and everything, investigators,
DSMB members, the sponsor and just asked some set of questions
to try to evaluate whether these DSMBs were conducted
in a way that met NIH's own policy and if there any challenges
to the way things are currently being done in terms of the DSMB effectiveness.
And this makes for a good reading.
It's a brief report, but you can see there are recommendations,
but there's sort of a back and forth between NIH officials
and the Health and Human Services in terms of what recommendations
or assessments of what was needed and not needed, particularly on the issue
of that independence, actually, is the real sticking point,
I think, between some of the institutes and those outside of the NIH.
But the main findings were that the NIH's DSMBs
didn't meet the guidelines overwhelmingly across all those and few
-- you know, there's a few dissenting opinions,
but quite the number of that said across
investigators through the members that the policies were being met.
But there were complaints particularly from certain investigators
about participation of study staff in the close
meetings was not, in all circumstances, seemed to be helpful.
But in other circumstances, it was pointed that it was helpful
because they were experts in certain rare diseases.
They were the study staff that think about these issues, day and night.
So, I think maybe one size does not fit all,
but that is still room for debate in how we do these things.
I think another issue is that we do these reports as masked,
and there wasn't really good understanding
amongst all members of the DSMB themselves that were interviewed
that they had the right to ask for the unblinding codes.
And so, the -- there was as recommendation to avoid that confusion
and certainly to make sure this was an accepted policy
that have stayed in the policies in all centers
that the DSMB has the right to see unmasked data,
to make sure that's absolutely clear that was historically seen as a challenge.
Some DSMB members had no idea.
And then it ultimately -- what's really interesting about this
field is it's incredibly complex, so when you see someone's resume
that they were the one on the cancer trial and you're doing a cardiovascular trial,
you're going to want the same people on your DSMB.
And so, you have the same experts serving on the HIV trials,
the cardiovascular trials.
And it's an Asian population of experts.
When you see them talk and there's a lot of concern
that this expertise is going to disappear and that we don't have enough
people who are new that are engaging in this activity,
and so there's active efforts across NIH and other sponsored trial settings
to get new people into this endeavor, which as you can imagine is quite complex.
So, at NIH, it was recommended they should direct the ICs,
the Institutes and Centers to at least articulate the circumstances
in which the Institutes and Centers staff can participate in the DSMB.
So, at least everyone knows what to expect and at least, in those circumstances,
understand if a particular IC member is not abiding by their own restrictions.
So, that was seen as agreed.
Also, it was -- you know, a lot of them agreed
that they would make it more clear that all DSMBs have -- that the policies
specifically say that the DSMBs have access to unmasked data
that was an easy fix and that they're all
working on ways to identify in recruiting members.
So, with this sort of negotiating set of recommendations, the NIH agreed.
So, talking about earlier that you -- it's
not always needed to have an independent body.
And you can imagine it's quite a complex
and expensive endeavor to get a group of experts like this
and fly them around the world to have these meetings.
They are -- so universally except for this.
Important and needed
when you have large randomized trials with mortality or major morbidity.
Certainly, if the trial, for whatever reasons, has implications
for public health, you're going to want an independent body
to help you through the conducting of this trial.
If there's a serious toxicity, that's thought to be a risk
or potentially, for whatever reasons, other kinds of high-risk treatments,
then people generally, even if it's not blinded,
might think about having a DSMB has an independent advisory board
to help make decisions about what might be an ethically complex situation.
And generally, if there's a vulnerable population,
whether be children or pregnant women or anybody you might find vulnerable,
you know, folks with Alzheimer's or dementia,
these are the populations that would need extra protections.
When they're not needed,
typically, if
you don't have any of those ethical tricky or sinuous and difficult situations,
but maybe run-of-the-mill, you know, allergy medicines but in their --
so not anything unusual in terms of risk being taken, single-arm
trials not blinded.
There's really no reason to have an independent body if everyone
can see they have totality of evidence as people accrue short-term trials.
It's almost impossible to get a DSMB together
if a trial's going to be quite quick in terms of accrual at endpoints.
This is your point.
You can't stop early.
So, what is DSMB going to do?
And certainly, I think, when you have that unblinded data with no ethical reason
to be extra concerned, there's really no need for DSMB.
And there are disadvantages.
So, a lot of people will try to be better safe than sorry,
but it increases the complexity of the trial management.
Once you say you're going to do these DSMB meetings, they have to happen
and increases costs.
And so, I think, the thought is that you should --
and to think
we have sort of a small set of individuals who can serve on these boards
that have expertise in your area, don't use them up in a situation
where they're not really needed and keep them
fresh and available for situations where you really do need them.
So, I think that's another important
reason, especially in smaller fields, to think about whether you need
-- you don't have one if you don't think you need one.
Because you have many other forms of monitoring,
you can have your own local safety monitors, perhaps some kind of partial --
you know, if you're not going to be monitoring efficacy,
but you want extra experts to monitor safety,
then you can recruit those specific safety monitoring board
without all the other extra goals that the DSMB typically has.
You have the IRB, the Institutional Review Board, that's
going to look at the human subjects at your institution.
And the protocol team themselves should be constantly reviewing
how their study is going and the safety of the patients in their study.
So, as promised, I just want to go from those big concepts
to some just not some balls of example tables
or shells or things that I've seen in monitoring reports.
Many of you,
if you're familiar with clinical search, know about the consort diagram.
I think it's useful to see it in the DSMB report,
because then you see how people are being screened, approached,
and how many of those are accepting or being -- or accepting
and then failing the eligibility, continuing later for other reasons.
It gives you an idea of feasibility
in representation and then also a great concise way
to look at things like of those that should've been analyzed, what percent
were withdrawn, lost to follow-up, got discontinued early, or completed.
All those things, I think, are very useful
and quite concisely summarized in a consult diagram.
Going back to validity, you
know, are you doing the protocol that you intended,
are you going to have interoperable results?
You definitely want to look at missed visits,
you want to look at them by visits, you want to look at the baseline.
Certainly, if they're not even getting through the baseline, that's a concern.
And you want to see how many are completing the last visit.
Overdue forms, they might give you an idea of whether or not
your protocols are a little too difficult in terms of the schedule.
People are willing to do it but can't quite make the timelines.
It's useful information, it might also be very fixable.
Baseline characteristics, are these
the people you intended to enroll, are they balanced by arm?
If randomization worked,
these things should be balanced by arm, but randomization doesn't always work
and sometimes it's been discovered in meetings like this by DSMB.
Serious adverse events.
In many ways, that's a no-brainer.
Obviously, DSMB is going to look at that.
But I think what's important is how you present it to the DSMB.
You know, if you have 4,000 people in the trial
and you're going to present lineless things of every sniffle
and headache, we'll never find the issues of concern.
And that is a concern.
And so, you want to be complete, maybe have appendices
of all the events in case somebody asks a question about a particular one.
But you're going to want to think how to summarize this
so the necessary trends can become apparent if there are any.
And then you want appropriate detail, so at the fingertips of start
and stop dates, how long did it take for these AEs to resolve, did they resolve.
People often ask what is the relationship do they think of this adverse event.
Something happened to the person on the trial,
they fell down the stairs, you know, whatever it is, it has to be reported
if there's an injury to a patient in your trial.
And you know, sometimes it's quite obvious that that maybe perhaps
-- maybe it is obvious, maybe it's not, maybe they fell down the stairs
because they were dizzy, and it had to do with the treatment.
So, essentially, you can ask for the expert opinion
the investigators often will list
what they think is whether it's probably related,
possibly related, definitely related, or unrelated.
But ultimately, you want
it presented to the DSMB, so they can make their own opinions.
But the opinion of the investigator
who was an expert is very -- you know, it's useful to have
but it should also just be considered or not thought of as absolute.
But you'll -- so, you also then want to start to summarize
these maybe by body system if these can happen multiple times,
perhaps you count the number of individuals who've had at least one event,
the timing of the first event, the seriousness of the worst event.
Things like that.
The fundamental concept here is that the summary should be able
to easily present -- allow trends to just arise to the top.
So, something that's visually easy to look at.
And in this particular one, there's not a lot of interesting
data here, but you might think about as a percent
of the total number who were treated, how many had any infection or sinusitis.
So, you might start with the sum category and then the individual categories
of those with at least to 10 percent rate,
and then list them both as overall and then by worst grade.
And often times, DSMB,
a failing system is something that they want to see last for it.
And so, in addition, of course, if this is part of the plan,
you would be looking at efficacy generally for long trials or risky trials
or trials for people who might stand to have a huge benefit
you might think about, you know, monitoring.
And so, you
might look at the 30 percent of treatment success in the area of the curve,
the Kaplan Meier curve,
and then you would have your official boundaries
if you are doing efficacy or futility monitoring.
And you would just simply look at the boundary
in where you were relative to the boundary and probably the whole history,
so you can kind of say how the data are trending.
The basic concepts are you want to make it as easy as possible
for the DSMB to do their job.
So, the report should be provided to the DSMB
in enough time for their view to happen -- you know,
they should be able to look at it before the meeting and think about it.
Don't email it the night before.
The report should be including concise summary of the trial itself.
Who knows how many boards these DSMB members are own?
Typically, they serve multiple because of the expertise that they have.
So, include that schema, that one-page summary that you have in your protocol,
telling you the basic endpoints, the number
to enroll, the primary objective that stores analysis plan.
You should have an executive summary that processes the main points
of what the -- say, for the open report
in any case where you are in accrual
and how many new AEs since the last meeting.
Things like that.
And the tables and figures should be concise,
easy to understand, and of course, accurate.
So, you know, hopefully,
I've given you some information today that you can appreciate.
The DSMBs, they do have a very difficult job
ahead of them and a very important job.
They have to balance individual ethics and collective ethics, risk/benefits.
They have to think about statistically technical things in terms
of being able to interpret and/or monitoring efficacy boundaries and
inflations of false-positive rates.
Sometimes, they need to consider futility and the balance
of what can be gained if you continue versus not continuing.
And most importantly, something that sometimes gets lost,
which is the decisions are not purely statistically
and in generally are always taking into account
much more than a simple boundary or one statistic and are often
very complex in terms
of how many different aspects you might consider in terms of overall
-- if a trial should continue as planned or some alteration.
And finally, I always like to emphasize -- because sometimes investigators,
I think, just want this weight to have decision making on
somebody else's shoulders.
But the DSMB is there to offer advice and guidance.
But it is the investigator and the sponsor ultimately
who bear the responsibility of stopping or going.
Here are some future readings.
Say, if like those that I mentioned would be like suspense novels, read
the Data Monitoring and Clinical Trials: A Case Studies Approach.
First author, Dave DeMets, who is a great storyteller
and who is involved in a lot of landmark trials
and interesting DSMBs with just unexpected swings and curveballs and does
a good job explaining lessons learned as well as telling these good stories.
A really informative book as well as the one by Susan
Ellenberg in this overall preference by Friedman, Furberg, and DeMets.
And the other two here at the end are statistical references.
If there are any statisticians out there who want to know
about the nuts and bolts of these boundaries
and other statistical issues, those are both very good references.
So, thank you very much.
Let's just stop the time for a few questions if you have any.
Yes, and I will try to remember myself to repeat the questions.
I know no one else other than the 20 people in this room can hear you.
So, go ahead.
Do you ever have
situations where the DSMB can't reach a consensus
if any member, experts have disagreements among themselves?
Yeah.
So, the question was, is there ever -- if I was sort of aware of situations
where the DSMB
themselves can't come to a consensus and -- yeah, that, absolutely, can happen.
Obviously, you think about, you know,
if you can have a home jury, you can have a home DSMB.
And you know, honestly, I think about -- a lot of times in the charter, there is --
there are rules such as -- you know, first of all, if you want to think about
how many members is the reasonable amount for it to be a quorum,
because it's impossible to make sure that everyone gets to every meetings.
So, the first challenge
is to make sure you have enough people at that meeting to vote.
And others then will say the decision -- there might be a majority/minority
and they simply -- they will report to the sponsor,
the outcome of the vote that it wasn't unanimous.
So, you have -- like the Supreme Court, you have the dissenting opinion.
Because this is advice-giving,
in some sense, there's no absolute reason other than it would be
nice for the sponsor and the investigator if the DSMB could come
with a clear recommendation.
But you know, the answer might not be clear.
And so, the importance is to present the two opinions.
Sometimes, you can have a member -- and I've certainly heard -- you know,
it's a bit like social engineering, like, you people need to get along.
You need to think about, I think, the personalities involved
and can they work together in a difficult situation.
And it's not that
they're -- it's not -- it's difficult because it's a challenging job ahead.
And so, if you have a very opinionated person
who is not every going to listen to anyone else's opinion,
that's not a great person to have on the DSMB.
So, there have been times when people have been
-- essentially been asked to leave, or it's been uncomfortable enough for them
that people just resign.
They just, like, know it's not going to work out and then replaced.
It's because they can't function well as a body.
So, you know, a lot of times, I know at NIAID,
there was one person whose job it was to think about how to make the DSMB.
You know, she was in the division -- essentially in the division of AIDS,
so anything related to infectious disease, you know, there was -- at NIAID,
they have separate divisions, say,
one for transplant or allergies, someone's job was to focus on infectious disease.
And she spent -- she networks, she knew people who serve in these bodies,
and she would kind of create these bodies and think about, "That person's
not going to get along with that person," which is kind of stupid in some sense,
you know, we're scientists, but at the same time,
it's like you guys have to work -- that group
has to work together and personalities inevitably will come into play.
Yeah?
Is there a waiver or a challenge process
for somebody, you know, go back --
the investigator can go back and perhaps challenge what they said?
Yeah.
So, the question was, "Is there some kind of waiver or challenge process
that investigator can essentially challenge
the recommendations of the DSMB?" And I can say,
I definitely was in a situation.
I can't remember all the specifics.
I was on a study team, not the DSMB,
and the DSMB mad a recommendation about the treatment.
I think it had to do with uncomfortable number of AEs.
But the investigator presented
an opposing opinion, "You know, these AE is very treatable,
this disease was very dire," they came up with something
that gave the DSMB more information to process the data.
And so, there was a series of letters that were brought back and forth.
And then ultimately DSMB should've stayed when they were more comfortable with
-- I think the DSMB might even recommended stopping and then they didn't.
And they gave reasons why, and then everyone was comfortable and then went on.
And that study was positive in the end.
They were on the New England journal.
So, you know, the DSMB offers advice.
I think if there was a DSMB that felt very strongly and the WHI -- I don't know.
To think about that itself needs a whole lecture.
It's hard to just dabble into that example.
But do know that the first DSMB recommended stopping and then
the sponsors, for whatever the reasons, they didn't want to, and they continued.
And they got a whole
another body and it's -- so that's the sponsor's prerogative, you know.
But they have public opinion.
They have the scientific community that in the end will judge all of these actions.
And the sponsor's only responsibility ultimately
is to listen to the DSMB, but they are the decision makers.
So, the idea of challenging the -- you know, I think the investigators
want to make sure the DSMB understands,
that they don't want to feel like they're ignoring the DSMB's advice.
They want to continue together to continue on the trial.
So, a lot of times, if you're not going to stop as recommended,
the investigators like to explain why and get the DSMB on board
so they can continue in a congenial and agreeable manner.
How's the DSMB financed in related to their expenses?
So, the question is about how are these things financed?
Yes, the DSMB, I believe there is an honorarium
for the meetings that might require travel.
Not always, I don't think.
So, that is a little bit unclear.
I've definitely been in situations where I received
honorariums and situations where I haven't.
So, there might be a small honorarium.
And certainly, all their expenses in terms of travel, most people prefer,
especially in maybe ethically challenging or complex trials in-person meetings,
so all that travel expenses is all part of the grant of the study.
The study sponsor covers all of that.
And if the DSMB says we need an extra meeting and there's 12 of them
coming from, you know, around the country, the sponsor has to cough up that money.
So, they have to have, like, reserve of funds to be able to cover a DSMB --
and so, you make a guestimate like you do, say,
if it's grant-funded study and potentially, if it's unexpected
burden, perhaps a study team could go back to their sponsor,
whether it's the NIH or if it's an industry sponsor
and figure out a way to get supplemental funds if necessary.
Patient safety is the primary responsibility.
Are there any other question?
All right.
Well, if not, thank you for your attention.
And I'll be here for extra minutes if anyone else wanted to come up offline
and ask questions.
Thank you very much for your time.