THANK YOU FOR JOINING US FOR DAY
TWO OF THE NIH PRAGMATIC TRIALS
COLLABORATORY 2022 ANNUAL
WORKSHOP.
WE ARE EXCITED TO CONTINUE OUR
DISCUSSIONS TODAY TALKING ABOUT
THE CRITICAL QUESTIONS FOR
PRAGMATIC CLINICAL TRIALISTS AND
GIVE YOU INSIGHT THE NIH
PRAGMATIC TRIAL HEARSAY
COLLABORATORIES FIRST DECADE.
WE KICKED OFF YESTERDAY AND
TALKED ABOUT A VARIETY OF
TOPICS.
WE HAD A WONDERFUL KEYNOTE
PRESENTATION FROM THE DIRECTOR
OF NINR THE NURSING RESEARCH
INSTITUTE WITH DR. SHANNON ZENK
STARTING US OFF.
IT WAS A WONDERFUL KEYNOTE
TALKING ABOUT HOW NURSING
SCIENCE FITS INTO AREA OF DO
TRIALS IN HEALTHCARE SYSTEMS
WHEN WE EMBED THESE
INTERVENTIONS INTO FAIR
DELIVERY.
SO THAT WAS A WONDERFUL WAY TO
START THE DAY.
THEN OUR FIRST PANEL TALKED
ABOUT WHAT WE LEARNED ABOUT
ELECTRONIC HEALTH RECORD DATA
AND HOW TO USE ELECTRONIC HEALTH
RECORD DATA IN DIFFERENT STUDIES
AND IN SOME WAYS SOME OF THE
ADVANCES AND INCREASING
CAPABILITIES WE HAVE FOR GETTING
INFORMATION EITHER DIRECTLY TO
PATIENTS OR THROUGH CARE
PATHWAYS OR WORK FLOWS INTO THE
ELECTRONIC HEALTH RECORD BUT
ALSO COLLECTING BETTER OUTCOMES
MAKING THOSE OUTCOMES LESS LIKE
RESEARCH SURVEYS AND
FUNDAMENTALLY DESIGNED AND MUCH
MORE USER FRIENDLY FOR THE
PATIENTS WHO ARE ANSWERING THOSE
QUESTIONS.
A WONDERFUL MEETING YESTERDAY.
WE THEN MOVED TO A WONDERFUL
PANEL DISCUSSION AROUND THE
DESIGN OPTIONS THINKING
DIFFERENT STUDY DESIGNS AND WHAT
WE LEARNED ABOUT DESIGNING
STUDIES FOR TRIALS CONDUCTED
WITHIN THESE HEALTHCARE SYSTEMS.
IF YOU MISSED US YESTERDAYS IT
IS AN ARCHIVED PRESENTATION SO
BY NEXT WEEK THAT WILL BE UP AND
POSTED AND YOU CAN FIND IT ON
THE NIH VIDEOCAST WEBSITE OF
PAST PRESENTATIONS.
WE WILL ALSO PUT LINKS TO THE
WORKSHOP AND TO THOSE RECORDINGS
ON THE HEALTHCARE SYSTEMS
RESEARCH COLLABORATORY WE ALL
KNOW IS NOW REBRANDED AS THE
PRAGMATIC TRIALS COLLABORATORY
WEBSITE.
WHICH IS RETHINKING CLINICAL
TRIALS.ORG.
SO WE LOOK FORWARD TO YOU BEING
ABLE TO GET THOSE PRESENTATIONS
EVEN IF YOU WEREN'T ABLE TO JOIN
US YESTERDAY.
TODAY I GET TO START MODERATING
OUR FIRST PANEL, TODAY WE HAVE
TWO DIFFERENT PANEL SESSIONS
PLANNED.
FIRST LOOKING AT THE USEFULNESS
OF THE PLANNING YEAR AND HOW WE
CAN USE THAT PLANNING YEAR OR
START UP TIME TO ENSURE SUCCESS
OF THE TRIAL ONCE LAUNCHED.
ALL THE TRIALS THAT HAVE BEEN
LAUNCHED THROUGH THE PRAGMATIC
TRIALS COLLABORATORY HAD A ONE
YEAR PLANNING PERIOD WHICH IS
REALLY CLOSER TO NINE OR TEN
MONTHS, YOU DON'T GET THE FULL
YEAR BECAUSE WE NEED SOME TIME
TO TRANSITION AND APPROVE THOSE
REQUESTS TO GO TO THE SECOND
PHASE.
SO A LOT OF THINGS HAPPEN IN
THAT STARTUP TIME AND WE WILL
HEAR FROM FOUR INVESTIGATORS
ABOUT THE KEY THINGS THEY FOUND
HELPFUL DURING THAT PLANNING
PERIOD AND ADVICE THEY HAVE FOR
INVESTIGATORS THAT ARE THINKING
ABOUT DOING THESE TRIALS AND HOW
TO BEST UTILIZE THAT TIME.
OUR SECOND PANEL OF PRESENTER
WILL TALK ABOUT THE
IMPLEMENTATION CHALLENGES OF
DOING STUDIES IN HEALTHCARE
SYSTEMS WHAT WE LEARNED AND
BRINGING FROM OUR COLLEAGUES IN
THE FIELD OF IMPLEMENTATION
SCIENCE ABOUT WHAT METHODS CAN
BE USED TO ENSURE INTERVENTIONS
ARE DELIVERED IN THESE
HEALTHCARE SETTINGS.
IT IS NOT ALWAYS EASY TO DO AND
EVERY SITE IS DIFFERENT SO HOW
DO YOU TAKE ON THOSE CHALLENGES
AND WE WILL HEAR FROM A NUMBER
OF OUR SPEAKERS THERE.
DO I HAVE LO GIST UKS SLIDE
NEXT?
A LITTLE COUPLE OF NOTES ABOUT
HOUSEKEEPING.
FOR THOSE PARTICIPATING VIA
VIDEOCAST WE ARE MONITORING THIS
EMAIL BOX AND WE WILL PUT IT UP
AGAIN AT THE Q&A PERIOD BUT IF
YOU WANT TO JOT IT DOWN, PLEASE
SEND YOUR QUESTIONS IN WE WILL
GET TO AS MANY QUESTIONS AS WE
CAN WITH OUR VIDEOCAST AUDIENCE.
IF WE DON'T GET BACK TO YOUR
QUESTION, WE WILL RESPOND TO YOU
VIA EMAIL TO YOUR GET ANSWER TO
YOUR QUESTION.
PLEASE DON'T HESITATE TO SEND
QUESTIONS THROUGHOUT THE
WORKSHOP THIS AFTERNOON.
NOTE FOR THOSE PANELISTS AND OUR
PRESENTERS TODAY WE HAVE AN
OPTION TO USE OUR CHAT BUT KEEP
IN MIND OUR VIDEOCAST AUDIENCE
CANNOT SEE THAT.
SO PLEASE FEEL FREE TO PUT YOUR
QUESTIONS THERE AND WE WILL ADD
THOSE TO THE MIX AS WE ARE
GETTING TO THE DISCUSSION
SESSION AS WELL AS FEEL FREE
WHEN WE GET TO DISCUSSION
SESSION IF YOU WANT TO ASK THE
QUESTION TO RAISE YOUR HAND.
YOU ARE CERTAINLY WELCOME TO DO
THAT AS WELL.
I DID SEE DR. LANGEVIN POP IN,
ANY WELL COMING REMARKS BEFORE
WE START THIS AFTERNOON?
>> SURE, HELLO, NICE TO BE BACK.
I WAS POPPING IN AND OUT
YESTERDAY AND I WAS REALLY BLOWN
AWAY BY WHAT I HEARD.
SOME JUST VERY SUBSTANTIVE
DISCUSSIONS GETTING TO THE
HEARTS OF THE MATTER SO I LOOK
FORWARD TO US HEARING MORE.
>> WONDERFUL, THANK YOU FOR JOIN
US TODAY AND LOOKING FORWARD TO
OUR SESSIONS TODAY.
I THOUGHT IT WAS A GREAT SET OF
DISCUSSIONS YESTERDAY.
REALLY INTERESTING.
SO WONDERFUL.
WHY DON'T I GO AHEAD AND MOVE
ON, I AM THE MODERATOR FOR OUR
FIRST PANEL.
WHAT WE WILL DO FOR EACH SPEAKER
IS YOU CAN TURN CAMERAS ON WHEN
PRESENTING AND WE GET TO
DISCUSSION TIME POINTS, WE
ENCOURAGE YOU TO TURN OFF YOUR
CAMERA WHEN YOU ARE NOT SPEAKING
AND SOMEONE ELSE IS IN THE WAY
IT PRESENTS ON THE VIDEOCAST
MAKES IT CLEANER AND AS I
MENTIONED, THE VIDEOCAST IS
ARCHIVED AND AVAILABLE FOREVER
SO YOU KEEP YOUR CAMERA UP, YOU
WILL BE ON THAT FOR EVERYONE TO
SEE IN THE FUTURE IF NOT
SPEAKING SO TRY TO REMEMBER TO
DO THAT MYSELF.
I'M WENDY WEBER, SNATCHOR
NATIONAL CENTER FOR
COMPLIMENTARY AND INTEGRATIVE
HEALTH.
THE DECLARE CLINICAL TRIALS
COLLABORATORY PROGRAM HAS BEEN
COMMON FUND PROGRAM FOR THE
LASQUE DADE AND DR. HELENE
LANGEVIN, THE NATIONAL CENTER
FOR COMPLIMENTARY AND
INTEGRATIVE HEALTH AS WELL AS
DR. RICHARD HODES, THE DIRECTOR
OF THE NATIONAL INSTITUTE ON
AGING, HAVE BEEN CO-CHAIRS OF
THE THAT COMMON FUND PROGRAM AND
WE ARE NOW TRANSITIONING THE
COMMON FUND SUPPORT TO ALL THE
INSTITUTES AND CENTERS ACROSS
NIH TO CONTINUE THIS PROGRAM OF
DOING CLINICAL TRIALS MR. THEY
ARE PRAGMATIC TRIALS OR
IMPLEMENTATION TRIALS, EMBEDDED
IN HEALTHCARE SYSTEMS ACROSS THE
PRAGMATIC TRIALS COLLABORATORY.
I HAVE BEEN WORKING WITH THE
PROGRAM OFFICER FOR THE
COORDINATING CENTER SINCE THE
PROGRAM STARTED BACK IN 2012.
NEXT SLIDE.
OUR OBJECTIVE IN THIS SESSION
WHERE WE ARE TALKING HOW TO BEST
UTILIZE THAT PLANNING YEAR IS
WHAT IS IT THAT YOU CAN DO
DURING THAT TIME TO REALLY
INCREASE THE SUCCESSFUL
COMPLETION OF THE PRAGMATIC
TRIAL.
WE WILL GIVE EXAMPLES, FROM
SEVERAL OF OUR INVESTIGATORS,
AND PROVIDE AT THE END A FEW
RESOURCES THAT MIGHT HELP YOU
AND DEVELOPING PLANNING BOTH THE
PLANNING OR STARTUP PERIOD BUT
ALSO YOUR TRIALS OVERALL IF YOU
ARE THINKING ABOUT SUBMITTING AN
APPLICATION TO NIH OR TO OTHER
FUNDERS WHO SUPPORT THIS TYPE OF
RESEARCH.
A FEW KEY CONSIDERATIONS YOU ARE
GOING TO HEAR FROM OUR SPEAKERS
IS IN THAT PLANNING PERIOD
FOCUSING ON CHALLENGING OR
DIFFICULT THINGS YOU KNOW WILL
BE TOUGH TO DO YOUR TRIAL.
SO FOCUS ON THOSE PAIN POINTS
AND WORKING OUT DETAILS DURING
THAT STARTUP OR PLANNING PHASE.
FIGURING OUT ALL OUR STUDIES
WORKED ACROSS THREE HEALTHCARE
SYSTEMS SO HOW TO AMERICAN AND
AGGREGATE THAT DATA, CAN YOU DO
THAT FIGURING THAT OUT IN YOUR
PLANNING PHASE WILL HELP YOU
TREMENDOUSLY BEFORE YOU GET TO
THE END.
ANOTHER LESSON LEARNED IS FIGURE
WHAT DOESN'T WORK QUICKLY.
FAIL FAST AND TRY DIFFERENT
THINGS TO SOLVE PROBLEMS AND
BETTER TO TRY SOMETHING, IF IT
DOESN'T WORK GO TO THE NEXT
THING.
THAT IS SUCH A SHORT TIME TO
START UP AND PLAN STUDIES THAT
FAILING FAST CAN BE REALLY
IMPORTANT.
WE CERTAINLY WANT OUR TRIALS
HAVE GENERALIZABLE POPULATION SO
DURING THAT PLANNING TIME CAN
YOU ACTUALLY MAXIMIZE AND
EVALUATE ARE YOU GETTING A
DIVERSE SET OF POPULATIONS, ARE
YOU ACTUALLY EQUITABLY OFFERING
THE INTERVENTION TO EVERYONE
WHOSE WOULD BE ELIGIBLE FOR THE
STUDY AND NOT CREATING
UNFORESEEN BARRIERS FOR THE
IMPLEMENTATION OF THE
INTERVENTION TO BE STUDIED
ACROSS YOUR DIFFERENT HEALTHCARE
SYSTEMS.
ANOTHER THING WE LEARNED THOUGH
HARD TO DO IS FALLS IN THE FAIL
FAST OR FOCUS ON PAIN POINTS,
START WITH DIFFICULT SITES FIRST
AND FIGURE HOW TO MAKE IT HAPPEN
THERE.
THAT MAY MAKE IT EASIER TO DO
THINGS IN ALL OF YOUR OTHER
SITES.
THE BIGGEST QUESTION IS WHAT DO
YOU NEED TO FIGURE OUT BEFORE
YOU START YOUR TRIAL AND THAT IS
WHAT YOU WANT TO FOCUS ON IN
YOUR PLANNING PHASE.
NEXT SLIDE.
SO AS I MENTIONED WE HAVE FOUR
AMAZING INVESTIGATORS FROM OUR
DIFFERENT TRIALS ACROSS THE
COLLABORATORY PROGRAM, AND THEY
ARE GOING TO TALK ABOUT THEIR
EXPERIENCE WHAT THEY LEARNED
DURING THEIR PLANNING PHASES AND
WHAT ADVICE THEY HAVE FOR ALL OF
YOU.
WE ARE GOING TO START OFF WITH
BRIEF PRESENTATIONS FROM EACH SO
THEY CAN GIVE YOU THE CONTEXT OF
WHAT THEIR TRIAL IS AS WELL AS
KEY THINGS THEY LOOK AT IN THEIR
PLANNING PHASE.
WE HAVE A SET OF QUESTIONS WE
WILL DISCUSS AS A GROUP, THAT I
WILL POSE TO THE GROUP AND WE
WILL ANSWER MOST KEY ELEMENTS
YOU SAW IN THOSE KEY TAKE AWAYS,
WE WILL PROVIDE EXAMPLES OF EACH
OF THOSE.
THEN WE WILL OPEN UP FOR THE
AUDIENCE FOR QUESTIONS FROM THE
AUDIENCE AND THEN WRAP UP WITH A
FEW RESOURCES THAT MIGHT BE
HELPFUL FOR EVERYONE AS THINKING
ABOUT PLANNING AND CONDUCTING
THESE TYPES OF STUDIES.
BY INTRODUCTION WE HAVE DR. GAIL
D'ONOFRIO WHO HAS JOINED FROM
THE ALBERT E KENT PROFESSOR
EMERGENCY MEDICINE AT THE YALE
SCHOOL OF MEDICINE AND PROFESSOR
OF MEDICINE AND PUBLIC HEALTH
AND SHE WILL TALK TO US ABOUT
THE EMBED STUDY.
WE HAVE DR. JAMES TULSKY, CHAIR
DEPARTMENT OF PSYCHOSOCIAL
ONCOLOGY AND PALLIATIVE CARE AT
DANA-FARBER CANCER INSTITUTE AND
PROFESSOR MEDICINE AT HARVARD
MEDICAL SCHOOL AND HE WILL TALK
ABOUT THE ECPP STUDY, WE HAVE
DR. DOUG ZATZICK, A REPEAT FROM
YESTERDAY WE HAD TO BRING HIM
BACK BECAUSE WE HAVE GOOD
STORIES TO TELL, HE IS PROFESSOR
IN THE DEPARTMENT OF PSYCHIATRY
AND BEHAVIORAL SCIENCES AT THE
UNIVERSITY OF WASHINGTON SCHOOL
OF MEDICINE AND HE WILL TALK TO
US ABOUT THE T STUDY.
FINALLY WE HAVE DR. MICHAEL HO
WHO IS GOING TO TALK TO US ABOUT
THE NUDGE STUDY AND HE IS A
PROFESSOR OF MEDICINE AT THE
UNIVERSITY OF COLORADO.
SO WE ARE REALLY EXCITED TO HAVE
ALL OF OUR SPEAKERS HERE.
AND WE WILL GET START AND I
THINK GAIL, YOU ARE OUR FIRST
SPEAKER SO I WILL TURN THINGS
OVER TO YOU.
>> I AM TRYING TO START THE
VIDEO, DID YOU PUT ME ON THE
HOST HAS SOPPED IT, IT SAID MY
VIDEO.
>> MARY JOE CAN YOU REQUEST FOR
HER TO TURN IT BACK ON?
>> I CAN START ANYWAY.
>> I THINK WE SHOULD BE ABLE TO
GET THAT UP.
FOR TAMMY, YOUR CO-HOST AS WELL,
DO YOU HAVE THE ABILITY TO DO
THAT?
>> SHE JUST ASKED ME.
>> PERFECT.
>> I'M GOING TO TALK TO YOU
ABOUT EMBED, I WANT TO REALLY
TELL YOU THAT THIS WAS A REALLY
GROUP EFFORT, DR. MALNAC IS PI
ON THIS WITH ME AND A HUGE GROUP
OF PEOPLE INTERESTED IN THE MOST
IMPORTANT THING IS THAT DR.
MONAC EXPERTISE IN INFORMATICS
MAKES THIS WORK SO ON TO THE
NEXT SLIDE.
SO EMBED IS A PRAGMATIC TRIAL
USER CENTERED CLINICAL DECISION
FOR EMERGENCY DEPARTMENT
INITIATED BUPRENORPHINE FOR
OPIATE USE DISORDER.
RIGHT A WAY IT IS A PRAGMATIC
TRIAL WHICH IS ITSELF USING
USUAL CONDITIONS AND WHEN
APPLIED GENERALLY THAT USUAL
CARE MAKES IT MORE DIFFICULT, IT
IS A FIVE YEAR PROJECT.
PLANNING PHASE ET CETERA AND
WHAT WE DID WAS TAKE WHAT WAS A
VERY STATIC INITIATED
BUPRENORPHINE ALGORITHM AND MAKE
IT DYNAMIC TO AUTOMATED
APPLICATION, AS YOU CAN SEE ON
THE RIGHT HAND SIDE, AND TO OUR
INTERVENTION SITES IT WAS AN 18
MONTH TRIAL.
THERE WERE 21 ED SITES OVER FIVE
HEALTH SYSTEMS, YOU CAN SEE THEM
THERE AND THAT INTERVENTION WAS
USING THAT CLINICAL DECISION
SUPPORT AND IT WAS RANDOMIZED
ONE TO ONE RATIO INTERVENTION
VERSUS USUAL CARE ARM WITH OUR
PRIMARY OUTCOME BEING INITIATION
OF IN ED PATIENTS AT THE PATIENT
LEVEL.
THAT WAS FROM THE CLINICIAN
USING IT.
SO LET'S GO ON.
WE HAD PLANNING PHASE AND THIS
IS WHAT OUR MILESTONES WERE.
WE ARE GOING TO DESIGN DEVELOP
OUR PROTOTYPE, WE DID THE IT
BUILD WITH LOCAL EHR INTEGRATION
AND TESTED THAT ON SITE.
AND WE DID THE TRIAL PLANNING
PHASE.
YOU CAN SEE WE REALLY DID HAVE
SOME DIFFICULTIES AND CHALLENGES
AND THEY WERE IN ALL THEIRSIAS
HERE.
AND WE CAN MOVE FORWARD.
THIS WAS OUR ORIGIN MALL DESIGN
HOW WE CHANGED IT AND SO YOU CAN
SEE REALLY VERY QUICKLY THE BLUE
REPRESENTED WHAT WE PLANNED ON
DOING IN OUR ORIGINAL PLAN, AND
GREEN WE ACTUALLY DID SO THAT
YOU CAN SEE THAT THINGS CHANGED
THROUGHOUT, ONE HAS TO BE FAIRLY
NIMBLE AND RESILIENT.
SO THE NEXT SLIDE IS SHOWING WE
HAD THE FIRST BIG HURDLE WAS
THAT WE HAD A CHANGE IN ONE OF
OUR PIs AT MAYO, MIGRATED TO
UNIVERSITY OF ALABAMA HEALTH
SYSTEM SO WE ADDED THEM TO THE
EMBED SITE.
ALSO IN THE ORIGINAL TRIAL WE
WERE DOING A STEP WEDGE, SHOW
THE NEXT SLIDE.
YOU CAN SEE WE WERE ORIGINALLY
PLANNING A STEP WEDGE DESIGN BUT
THE NIH STEERING COMMITTEE THAT
WENT TO 2018 TOLD US WE WOULD BE
BETTER OFF IF WE CHANGED THAT
BECAUSE OF THE GROWING PRICES
AND LOT OF TEMPORAL CHANGES AND
WE NEEDED TO FIND A SOLUTION TO
THAT.
SO WE DID THE CLUSTER DESIGN.
SO THAT HAPPENED RIGHT THERE AND
YOU CAN SEE WHAT HAPPENED.
WE SUBMITTED A SUPPLEMENT
APPLICATION TOO WHICH THEN ALLOW
US TO ADD TWO ADDITIONAL HEALTH
SYSTEMS TO ACHIEVE OUR TARGET
POWER WHICH WERE THE UNIVERSITY
OF COLORADO OR AURORA AND WE HAD
MORE RESOURCES TO IMD BY
DISSEMINATION SO WE HAD TIME
TOPHI FISH, ANALYZE RESULTS AND
DO DISSEMINATION.
WE WILL NOT GO OVER THIS NOW, WE
WILL ANSWER MANY QUESTIONS BUT
THOUGHT GIVE OUT THE SLIDES IT
IS NICE FOR THE PEOPLE WHO ARE
ON TO HAVE IT TALKS REALLY
ISSUES BARRIERS AND SOLUTIONS WE
WILL WILL TALK ABOUT THAT AS WE
GO THROUGH.
FINALLY AS YOU CAN SEE HERE
THERE WERE DATA COLLECTION
CHALLENGES, MANY THINGS THAT
WERE ISSUES HERE, WE HAD TO GO
THROUGH EACH ONE OF THEM AND
THESE ARE IT THINGS THAT NEED TO
BE DRESSED DURING THE TIME.
HAVING THE MASTER DICTIONARY
READY ENSURING THE DATA WAS
CONSISTENT, THE MOST REALLY THE
MOST IMPORTANT THING IS WE DID
MONTHLY POLLS DURING THE TRIAL
TO MAKE SURE THIS DATA WAS
COMPLETE, THE -- THEY WERE
CORRECTLY MAPPED TO EHR FIELDS
THAT CORRESPONDED TO CLINICAL
INTENT OF THE VARIABLE AFTER
ACCOUNTING FOR DOCK MEN TAKES
PLAQUE AT THIS AND WORK FLOW AT
EACH SITE.
WE HAD TO HAVE QUALITY ASSURANCE
SO THAT WAS VERY DIFFICULT, IT
WAS VIGOROUS AND WE HAD A LOT OF
CHALLENGES WE DID MULTIPLE
ROUNDS OF ITERATIONS AND SAMPLE
DATA VALIDATION BY THE -- OUR
DATA COORDINATING CENTER HERE AT
YALE.
SO MANY THINGS WE CAN TALK ABOUT
FOR THE NEXT HOUR.
RUNNING A CHECKLIST TELLING
EVERYBODY THAT WAS ESSENTIAL TO
GET EVERYBODY UP TO SPEED AND
KNOW WORKS THOSE SITES THAT NEED
MORE WORK WITH AND THOSE THAT
DIDN'T AND MAKE SURE THAT ALL OF
YOUR MILESTONES WERE REACHED FOR
LOTS OF THINGS, WE HAD TO GO TO
MOVE FOR.
GO AHEAD.
THESE ARE BUNCH OF PUBLICATION,
YOU HAVE TO READ WHICH WENT
THROUGH EACH OF THESE DIFFERENT
AREAS AND ISSUES THAT WE HAD AND
HOW WE REALLY MET SOME OF THOSE
BARRIERS AND FOUND THOSE
SOLUTIONS YOU CAN FIND THEM ALL
IN THESE PAPERS.
SO THAT IS IT, WENDY.
>> GREAT, THAW SO MUCH FOR THE
INTRODUCTION -- THANK YOU SO
MUCH, I HOPE THAT GIVING
EVERYBODY A GOOD IDEA AND WE
WILL GET INTO THE DETAILS OF
SPECIFIC ISSUES AND CHALLENGES
YOU FACE AS WE GET INTO
DISCUSSION.
WE GO TO THE NEXT SLIDE.
SO IT LOOKS LIKE WE HAVE A
LITTLE DELAY WITH DR. HO JOINING
US SO WE WILL SKIP AHEAD IF THAT
OKAY, MARY JOE, JAMESLY ASK YOU
TO COME UP DR. TULSKY AND GO
NEXT FOR US AND WE WILL COME
BACK TO DR. HO'S SLIDES WHEN HE
IS ABLE TO JOIN US, HE HAS A
DELAY IN JOINING US, IF YOU CAN
BRING UP YOUR VIDEO WHICH YOU
MIGHT HAVE ALREADY.
I WILL TURN THINGS OVER TO YOU.
>> THANK YOU SO MUCH WENDY.
HAPPY TO SPEAK ABOUT OUR STUDY
WHICH WAS ACPP WHICH STANDS FOR
PLANNING PROMOTE EFFECTIVE AND
RELIANT COMMUNICATION MANY THE
ELDERLY.
AND I WOULD BE REMISS IF I DID
NOT ACKNOWLEDGE MY CO-PI MASS
GENERAL HOSPITAL WITHOUT NONE OF
THIS WOULD BE POSSIBLE.
WONDERFUL TEAM LOTS OF YOUNG
PEOPLE.
NEXT SLIDE PLEASE.
SO WHAT ACPP IS IS A PRAGMATIC
STEP WEDGE CLUSTER RANDOMIZE
CONTROL RILE WHICH IS TESTING
WHETHER CLINICIAN COMMUNICATION
SKILLS TRAINING ON THE ONE HAND
PLUS PATIENT VIDEO DECISION AIDS
ON THE OTHER WILL INCREASE
ADVANCE CARE PLANNING OUTCOMES
FOUR OUTCOMES WE LOOK AT IN THE
STUDY ARE ADVANCE CARE PLANNING,
COMPLETION, RESUSCITATION ORDERS
IN THE CHART, PALLIATIVE CARE
CONSULTATION AND HOSPICE USE,
ALL THIS IS ADVANCE CANCER
PATIENT WHO ARE OVER THE AGE OF
65.
IF YOU CAN JUST ADVANCE ONCE.
SO THE TWO DIFFERENT
INTERVENTIONS, ONE IS VITAL TALK
WHICH IS A STANDARD GOLD
STANDARD CLINICIAN COMMUNICATION
SKILLS TRAINING PROGRAM OUT
THERE FOR SERIOUS ILLNESS
COMMUNICATION.
AND IT IS COMBINED TOGETHER WITH
CARE BINDING DECISIONS WHICH IS
THE VIDEO LIBRARY, ALWAYS USED
TO EDUCATE PATIENTS HELP THEM
MAKE DECISIONS.
AND WE WERE DOING THIS, WE ARE
DOING THIS IN THREE HEALTH
SYSTEMS YOU CAN ADVANCE MORE,
AND 36 ONCOLOGY CLINICS WHICH
REPRESENTS ONCOLOGY CLINIC BUT
THIS IS A DISEASE FOCUS ONCOLOGY
CLINIC SO FOR EXAMPLE THORACIC
GI ONCOLOGY OR LEUKEMIA OR
WHATEVER AND THERE ARE 12 IN
EACH OF THESE SITES, PROBABLY IN
A SENSE THREE DIFFERENT KINDS OF
PLACES, DUKE IS CLASSIC ACADEMIC
MEDICAL CENTER, MAYO IS AS WELL,
CLINIC ENVIRONMENT AND SERVE AS
BIT MORE COMMUNITY POPULATION
THOUGH THEY HAVE ACADEMIC ARMS
AS WELL.
THE NUMBER OF PATIENTS THAT WE
WERE -- WE CALL ENROLLING BUT
NOT REALLY ENROLLING BECAUSE IT
IS LOOKING AT THEIR DATA.
THEY END UP BEING 30,000, IF YOU
EVER DO GRAND ROUNDS OR STUDY
BEFORE THAT NUMBER WAS 4500,
TURNS OUT WE WERE OFF, WAY MORE
PEOPLE THAN WE ANTICIPATED, WE
NEED TO MAKE ADJUSTMENTS,
UNUSUALLY GOOD PROBLEM TO HAVE.
NEXT SLIDE PLEASE.
ADVANCE.
IN THE ORIGINAL DESIGN OUR PLAN
HAD BEEN WE WERE GOING TO HAVE
SIX STEPS THAT WERE GOING TO
INVOLVE TOTAL OF 12 CLUSTERS AND
EACH OF THESE CLUSTERS WAS GOING
TO BE COMMUNITY CLINICS.
SO THEN WE WERE GOING TO HAVE A
BASELINE PHASE, GOING TO
BASICALLY HAVE ONE END AND
COLLECT THE DATA FROM THE
ELECTRONIC ALL THE DATA COMES
OUT OF -- COMES OUT OF
ELECTRONIC HEALTH RECORDS, WE
WERE THEN GOING TO DO EACH OF
THE STEPS AND IN THE CLASSIC
FORMAT.
HOWEVER YOU CAN ADVANCE ONCE.
THAT IS ONE MORE TIME THAT
STARTED.
WE DID DO THE FIRST STEP.
EVERYTHING WENT GREAT.
WE GOT HIT BY COVID.
SO THIS OBVIOUSLY REQUIRED
CHANGES I DON'T WANT TO GET INTO
IT BUT BLEW UP EVERYTHING WE
WERE DOING SO WE REVISED THE
STUDY.
NEXT SLIDE.
SO WHEN WE DID THE REVISED
DESIGN, WE -- THERE WERE NUMBER
OF ISSUES, FIRST WE HAD A WHOLE
PERIOD OF TIME TO SKIP SO WE
(INAUDIBLE) RESEARCH.
THE OTHER THING WAS THAT
REALIZED THAT BECAUSE OF COVID
IT CHANGED A LOT OF APPROACHES
TO ADVANCE CARE PLANNING IN
THESE INSTITUTIONS.
PATIENTS WERE LOOKING AT IT
DIFFERENTLY, A LOT NEW CHANGES
BUT BASICALLY HUGE SECULAR TREND
TO ACCOUNT FOR.
SO WE DECIDED HOW MUCH COULD WE
LEARN FROM THIS SO WE DECIDED
THAT WHAT WE WERE GOING TO DO IS
WE WERE GOING TO USE THE DATA WE
HAVE STEPS 1 AND 2 AND COLLECT
THE ADVANCE CARE PLANNING RATE
BEFORE AND AFTER INTERVENTION,
THIS WOULD BE CLEAN, JUST UNDER
POWERED BECAUSE IT IS ONLY ONE
STEP.
ONLY ONE STEP AND OBVIOUSLY
WASN'T ENTIRELY RANDOM.
NEXT NOW WHAT WE WILL DO IS TAKE
STEPS 3 THROUGH 12.
LOOK AT THE INTERVENTION EFFECT
POST COVID-19.
BASICALLY AT THIS POINT WE HAVE
A NEW BASELINE AND WE WERE GOING
TO WORK OFF THAT NEW BASELINE
AND STUDY THE -- IN FACT WE
COULDN'T STUDY -- WE HAD TO KEEP
IT SEPARATE FROM STEPS 1 AND 2.
OBVIOUSLY THAT WASN'T GOING TO
WORK.
WE CREATED FOUR STEPS WHERE
THERE SHOULD BE FIVE AND AS YOU
CAN SEE WE INCLUDED MORE CLINIC
CLUSTERS IN THE LAST TWO STEPS
IN ORDER TO GET EVERYBODY IN.
ONE MORE AFTER THAT.
SORRY.
ONE MORE YET.
GREAT.
SO THEN THE OTHER THING WE WERE
GOING -- WE ARE DOING IS
MEASURING THE COVID-19 EFFECT.
WE WERE GOING TO ESTIMATE THE
PRE-COVID ADVANCE CARE PLAN RATE
FROM THE ORIGINAL BASE PIPELINE
PLUS STEP 1.
THE POST COVID RATE FROM STEP 2
DATA AND WE CAN EXAMINE TRENDS
OVER TIME SO TURNS OUT WE ARE
NOT ONLY GOING TO DO AND BE ABLE
TO SUCCESSFULLY COMPLETE THE
STUDY WE ORIGINALLY PLANNED WE
ALSO ARE ALSO GOING TO HOPEFULLY
LEARN SOMETHING ABOUT THE EFFECT
COVID HAD ON ADVANCE CARE
PLANNING AT LEAST IN THE SET OF
ONCOLOGY.
THIS IS MY LAST SLIDE.
I JUST WANTED TO BRIEFLY SPEAK
ABOUT SOME DATA CHALLENGES WE
RAN INTO AND AGAIN THESE ARE --
THIS IS SOMETHING YOU MIGHT BE
TALKING ABOUT MORE DISCUSSION
BUT WE WENT INTO THIS WITH THE
PLAN THAT WE WERE GOING TO USE
STRUCTURED DATA ELEMENTS FROM
ELECTRONIC HEALTH RECORDS TWO
WHICH WERE -- AND ONE WHICH WAS.
WE WERE GOING TO STRUCTURE DATA
ELEMENTS MEANING THERE IS A
PLACE IN THE RECORD WHERE
ADVANCE CARE PLAN DISCUSSION
DOCUMENT, WHERE THERE IS A MOST
-- DNR OR HEALTHCARE.
WE PULL THESE OUT AND THAT WAS
THE OUTCOME.
WE DID A CHART REVIEW OF THE
DATA ELEMENTS THAT WE POLLED
FROM THE SITES IN OUR INITIAL
PERIOD WHEN WE FIRST PLAN THE
STUDY.
WHAT WE FOUND IS VERY DIFFERENT
SO KEY YOUR EYES TO THE BOLDFACE
HERE AND IF YOU LOOK AT THE TOP
WHERE IT SAYS TOTAL CORRECT
OCCUPANTS, YOU CAN SEE ACROSS
ALL THREE SITES UNDER 60%
ACTUALLY HAD THE CORRECT
DOCUMENTS WE THOUGHT WE WERE
POLLING FROM THOSE SITES.
IF YOU LOOK DOWN AT THE BOTTOM
YOU CAN SEE THERE WERE 24%
INCORRECT DOCUMENTS, WHERE
SUPPOSEDLY DOCUMENT EXISTED OR
ANOTHER 17% OF DIGITAL DOCUMENTS
SO WE LEARN LEARNED IN THAT PROS
THAT DATA ELEMENTS WHICH APPEAR
TO HAVE BEEN FULL PROOF WAY OF
GETTING DATA, WAS GOING TO BE
ACCURATE.
SO WE WERE GOING TO DO LOTS OF
CHART REVIEW, IMPRACTICAL
PARTICULARLY WHEN NUMBERS GO TO
30,000 OR WE HAD TO THINK OF
SOMETHING ELSE, THIS IS WHERE WE
HAD ADVANTAGE OF HAVING A
LABORATORY HERE THAT WORKS ON
NATURAL LANGUAGE PROCESSING AND
WAS ABLE TO APPLY NATURAL
LANGUAGE PROCESSING ALLOT THES
AT DANA-FARBER TO THE SITES AN
EXTRACT THE SAME DATA WITH --
NOT SHOWING YOU THE OUTCOMES
HERE BUT WITH FAR MORE ACCURACY
LIKE EXCEEDINGLY FAR MORE
ACCURACY THAN WE GOT
CONSTRUCTION DATA ELEMENTS.
SO THAT SOMETHING WE LEARNED
EARLY ON THAT COMPLETELY CHANGED
THE STUDY LATER ON.
SO I WILL STOP THERE AND HAND IT
BACK TO WENDY.
>> GREAT, THANK YOU SO MUCH,
JAMES.
WONDERFUL INTRODUCTION TO YOUR
STUDY AS WELL.
I WILL TURN THINGS OVER, THE
NEXT SLIDE WILL BRING UP DR.
ZATZICK'S STUDY.
DOUGLY TURN TO YOU ABOUT THE
TSOS TRIAL.
>> THANKS WENDY.
A BRIEF OVERVIEW TO GET INTO
DISCUSSION, NEXT SLIDE PLEASE.
OUTCOMES SUPPORT WAS A STEP
WEDGE CLUSTER DESIGNED TRIAL
THAT LOOKED AT INNOVATIVE
COLLABORATIVE CARE INTERVENTION
FOR PATIENTS WHO WERE ADMITTED
TO LEVEL ONE TRAUMA CENTERS
NATIONALLY.
635 PATIENTS RANDOMIZED INTO THE
TRIAL COLLABORATIVE CARE VERSUS
USUAL CARE CONTROL, AND BASELINE
SURGERY ADMISSION WAS BEGAN WITH
PTSD SCREEN TO IDENTIFY PATIENTS
AT LEVEL FOR HIGH SYMPTOM
LEVELS, THE MAIN OUTCOME WAS
SYMPTOMS ASSESSED BY INTERVIEW
AT 3, 6, 12 MONTHS POST INJURY,
THE MAIN TRIAL PAPER WAS
PUBLISHED LAST YEAR IN JAMA
SURGERY AND SHOWED SIGNIFICANT
CLINICALLY STATISTICALLY
SIGNIFICANT REDUCTION IN PTSD
SYMPTOMS SIX MONTHS BUT NOT 12
MONTHS AFTER THE INJURY.
SO THE PLANNING PHASE WAS
COMPLEX FOR TSOS THAT WE WERE
INTERESTED IN GETTING 24, 25
SITES FROM THE POPULATION OF
LEVEL 1 TRAUMA CENTERS
NATIONALLY.
SO ON THE LEFT YOU SEE A GRAPHIC
OF ALL TRAUMA CENTERS IN THE
UNITED STATES, OVER A THOUSAND
AND ON THE RIGHT YOU SEE THE
GREEN DOTS, SMALL GREEN DOTS, 25
SITES THAT ACTUALLY RECRUITED
INTO THE TSOS TRIAL.
YOU SEE IN SEATTLE, AND OUR TEAM
IS A BIG DOT, WE COORDINATED THE
STUDY FROM SEATTLE AND DID OUR
PILOTING AT TWO LEVEL 2 TRAUMA
CENTERS SO MANY THE PLANNING
PHASE WE RECRUIT THESE SITES AND
AT THE SAME TIME DOING A SMALL
PILOT OF THE PROCEDURES.
AND WE HAD A VERY SORT OF
IMPLEMENTATION SCIENCE INFORM
CONCEPTUAL FRAMEWORK USING REAIM
TO THINK ABOUT THIS POPULATION
OF LEVEL 1 TRAUMA CENTERS AND
SORT OF SOME OF THE POINTS WENDY
WAS ARTICULATING WE WERE
INTERESTED IN A REPRESENTATIVE
SAMPLE OF LEVEL 1 TRAUMA
CENTERS, WE DIDN'T WANT ACADEMIC
SITES WE WANTED RURAL SITES, SO
THIS IS IS THE TABLE ONE OR
TABLE 5 DESIGN PAPER SHOWING THE
CHARACTERISTICS WE RECRUITED IN
THE STUDY, ORGANIZAL
CHARACTERISTICS, ADMISSION
VOLUME, RESIDENCY TRAIN STATUS,
REGIONAL LOCATION RURAL VERSUS
URBAN AND WE DID WELL BALANCING
THE ORGANIZATIONAL
CHARACTERISTICS OF THE SITES IN
THE STUDY WITH THE 200 TRAUMA
CENTERS ELIGIBLE FOR THE STUDY.
SO VERY DIFFERENT FROM REAIM
IMPLEMENTATION SCIENCE FRAMEWORK
PRAGMATIC ISSUES THAT NEEDED TO
BE TESTED BEFORE THE UH 3 ROLL
OUT IN THAT UH 2 ONE YEAR
PERIOD.
SO THERE WAS TREMENDOUS
VARIABILITY AT SITES AT BASELINE
AND THEN OVER TIME, MANY OF
THESE BASELINE ISSUES WE DIDN'T
GET GOOD ENOUGH CHANCE WHILE
RECRUITING WE DIDN'T HAVE A
CHANCE TO PILOT ISSUES THAT
CAUSE DIFFICULTY OVER TIME SUCH
AS ELECTRONIC HEALTH RECORD
CAPACITY, THE RESEARCH
EXPERIENCE OF TRAUMA CENTER
SITES AND PTSD INTERVENTION
EXPERIENCE.
HONING IN ON EHR CAPACITY, WE
HAD A TEN DOMAIN ELECTRONIC
HEALTH RECORD PTSD SCREEN
DESIGNED TO BE AUTOMATED, YOU
COME IN AND YOU DO AUTOMATED
SWATH OF TRAUMA ADMISSION SINCE
THE LAST TIME YOU RECRUITED,
THAT WOULD SAVE TIME SO YOU CAN
MOVE TO RECRUITMENT AND
INTERVENTION.
TURNED OUT, WE DIDN'T PILOT THAT
SUFFICIENTLY AND WE DIDN'T
CAPTURE THE VARIABILITY ACROSS
THE 25 SITES, MANY SITES THAT
WERE UNARABLE TO AUTOMATE OR
MANUALLY GO THROUGH IN A HYBRID
FASHION TEN DOMAIN MAYBE 5 AND
THAT ENDED UP SLOWING THINGS
DOWN IN TERMS OF OUR ABILITY TO
RECRUIT AND OVER TIME SAW SITES
THAT HAD TO SPEND MORE TIME
MANUALLY RATHER THAN
ELECTRONICALLY DOING PRELIMINARY
SCREEN.
SO LEGS SONS LEARNED WE ALREADY
HEARD, MAXIMIZING THE DIVERSITY
OF SITE CHARACTERISTIC IN THE
PLANNING AND ESPECIALLY THE
PILOTING, AND THEN TRYING DATURE
AS MUCH INFORMATION AS POSSIBLE
ACROSS KEY DOMAINS WHICH ARE
CHALLENGING TO IDENTIFY IN THAT
START UP.
THAT IS WHY WE ALSO THINK
FURTHER EVALUATION MAYBE TOO
STRONG A WORD BECAUSE YOU DON'T
WANT TO COLLECT REAMS OF
QUALITATIVE DATA YOU NEED NIMBLE
DATA COLLECTION THAT ALLOWS THE
STUDY LEADERSHIP TEAM TO KNOW
WHAT IS HAPPENING ON THE GROUND.
WITHOUT EXTENSIVE ANALYSIS YOU
WANT TO KNOW WHAT IS HAPPENING
IN THE ROLL OUT YEAR AND WE IN
TSOS A LOT KEPT HAPPENING
INTERCURRENT EVENTS PATRICK
HAGGERTY CALLED YESTERDAY, JUST
THINGS HAPPEN IN PRAGMATIC
TRIALS SO YOU WANT THAT CAPACITY
IN PLANNING PHASE FORM
EVALUATION CAPACITY AND ONGOING
AND I WILL LEAVE IT AT THAT.
>> GREAT.
THANKS SO MUCH, DOUG.
I THINK WE ARE GOING TO GO MARY
JOE, IF YOU CAN GO BACK WE WILL
GO TO DR. MICHAEL HO WHO WILL
TALK TO US ABOUT THE NUDGE
STUDY.
I WILL TURN OVER TO YOU IF YOU
WANT BRING YOUR CAMERA UP AND
TURN YOUR AUDIO ON.
>> THANK YOU.
SO MY NAME IS MICHAEL HO, I'M
GOING TO TALK ABOUT THE
PERSONALIZED PATIENT DATA AND
BEHAVIORAL NUDGES TO IMPROVE
ADHERENCE TO CHRONIC
CARDIOVASCULAR MEDS.
WE HAVE BEEN CALLING IT THE
NUDGE STUDY FOR SHORT.
I WILL PROVIDE A QUICK OVERVIEW
AND HIGHLIGHT KEY CHALLENGES OR
LESSONS LEARNED THAT FIRST YEAR.
THIS WAS A PATIENT LEVEL
RANDOMIZED STUDY ACROSS THE
HEALTH SYSTEMS WITH PRIMARY GOAL
OF IMPROVING ADHERENCE TO
CHRONIC CARDIOVASCULAR
MEDICATIONS.
AND THE PRIMARY OUTCOMES PRIMARY
OUTCOME WILL BE MEDICATION
ADHERENCE AS DEFINED BY PHARMACY
REFILL DATA.
AND THEN THE SECONDARY OUTCOMES
ARE SOME CLINICAL MEASURES,
CLINICAL EVENTS AND HEALTHCARE
UTILIZATION.
WE ARE CONDUCTING THIS TRIAL AT
THREE HEALTH SYSTEMS IN COLORADO
THE FIRST ONE IS FDA, SECOND IS
DENVER HEALTH WHICH IS THE
SAFETY NET HOSPITAL, HOSPITAL
SYSTEM FOR DENVER COUNTY, THE
THIRD IS UNIVERSITY OF COLORADO
HEALTH WHICH HAS 11 HOSPITALS
THROUGHOUT COLORADO.
SO IN TERMS OF PATIENTS INCLUDED
WE INCLUDE PATIENTS WHO ARE
DIAGNOSED WITH ONE OR MORE OF
THE CARDIOVASCULAR CONDITIONS
THAT ARE LISTED IN THE TABLE AS
WELL AS BEING PRESCRIBED AT
LEAST ONE MEDICATION CLASS OF
INTEREST.
THEY HAVE TO HAVE HYPERTENSION,
HYPERLIPIDEMIA DIABETES CORONARY
ARTERY DISEASE OR ATRIAL
FIBRILLATION AND HAVE TO BE ON
MEDICATION TO TREAT THAT
MEDICATION.
NEXT SLIDE.
FOR OUR STUDY WE USED A KNOCKED
OUT DESIGN AND REALLY THE
PROCESS WAS THAT WE IDENTIFY
PATIENTS USING THE EHR EACH OF
THE HEALTH SYSTEMS, BASED ON
DIAGNOSIS, PRESCRIBED
MEDICATION, ONCE THEY WERE
IDENTIFIED AS HAVING THESE
CONDITIONS AND BEING PRESCRIBED
MEDICATIONS, WE SENT THEM A
LETTER ABOUT THE STUDY AS WELL
AS AN OPT OUT LETTER THAT THEY
COULD MAIL BACK TO US IF THEY
WANTED TO OPT OUT OF THE STUDY.
FOR THOSE PATIENTS WHO DIDN'T
SEND US THE OPT OUT LETTER BACK
WE MONITOR THEM OVER TIME TO SEE
IF THEY WOULD HAVE A GAP IN
THEIR MEDICATION REFILLS IN
TERMS OF WE LOOK AND ANTICIPATED
REFILL DATE AND IF THEY DIDN'T
FILL AFTER SEVEN DAYS THEY WERE
ELIGIBLE.
THESE ARE FOUR ARMS OF THE
STUDY, PATIENTS ARE RANDOMIZED
TO USUAL CARE GENERIC TEXT
MESSAGES OPTIMIZE TEXT MESSAGES
MESSAGES THAT HAD THESE
BEHAVIORAL NUDGES, AND THEN
FOURTH ARM WHERE THEY CAN BE
RANDOMIZED TO BEHAVIORAL NUDGES
AS WELL AS AI CHAT BOT THE CHAT
BOT WHICH TRY TO ENGAGE THE
PATIENTS IN TERMS OF TRYING TO
IDENTIFY WHY THEY WERE DELAYING
REFILLING THEIR MEDICATIONS AND
PROVIDE POTENTIAL SOLUTIONS.
TO HELP THEM WITH REFILLING
MEDICATIONS.
THESE WERE OUR YEAR ONE
MILESTONES, WE BROKE OUR
MILESTONES FOR THAT YEAR INTO
DIFFERENT CORES.
BASICALLY WHAT WE DID WAS WE DID
A PILOT STUDY OF THE LARGER
TRIAL WHERE WE EHR PATIENT
IDENTIFICATION PROCESS WE SENT
THE OPT OUT LETTERS AND
RANDOMIZE PATIENTS TO EACH OF
THE INTERVENTION ARMS AND
PILOTED TEXT MESSAGING
INTERVENTION TERMS OF SENDING
OUT THE TEXT MESSAGES SEEING IF
PATIENTS WE RESPOND TO THOSE
MESSAGES ASSESS WHETHER WE COULD
COLLECT OUTCOMES WHICH WERE
MEDICATION REFILL DATA AT EACH
OF THE HEALTH SYSTEMS OF THE
STUDY SO THAT CONSISTED OF
MILESTONES FOR THAT YEAR 1.
IN TERMS OF LESSONS LEARNED FROM
THAT INITIAL YEAR IS THAT THE
TIME FRAME IS PRETTY SHORT.
WHILE IT IS STATED THAT YOU HAVE
12 MONTHS TO DO THE PILOT, YOU
REALLY ONLY HAVE ABOUT TEN
MONTHS IN TERMS OF FINISHING THE
PILOT AND WRITING THE RESULTS OF
THE PILOT UP IN PREPARATION WITH
THE UH 3 PHASE OF THE STUDY.
A LOT OF OUR ADMINISTRATIVE
BARRIERS WERE RELATED TO
CONTRACTING AS PART OF OUR
INTERVENTION WE NEEDED PHARMACY
DATA.
AND AT TWO OF OUR HEALTH SYSTEMS
WE HAD TO WORK THROUGH GETTING
SURE SCRIPS PHARMACY DATA AND A
MORE FREQUENT -- MORE FREQUENT
INTERVAL THAN WHAT THE HEALTH
SYSTEM WAS GETTING AT THAT TIME.
SO THAT TOOK A LOT OF WORK IN
TERMS OF WORKING WITH HEALTH
SYSTEM WHO THEN HAD TO WORK WITH
SURE SCRIPS IN ORDER TO PLAN FOR
GETTING THAT PHARMACY DATA.
I THINK IN TERMS OF WHAT WE
LEARNED AND WHAT WAS HELPFUL FOR
US THROUGHOUT THE STUDY AND
RIGHT NOW WE ARE IN THE
FOLLOW-UP PHASE OF THE STUDY IS
FORMING COLLABORATIVE
PARTNERSHIPS AT EACH O OUR
SITES.
AND OUR HEALTH SYSTEMS AND IT
HAS PREVIOUSLY ALLUDED TO COVID
HIT IN THE MIDST OF OUR
INTERVENTION SO WE HAD TO REALLY
WORK WITH OUR PARTNERS TO COME
UP WITH INTERIM SOLUTIONS AS THE
PANDEMIC HIT AND THEN FOR US AS
A TEAM IT WAS HELPFUL TO HAVE A
PROJECT MANAGER THAT HAD DONE
THIS BEFORE IN TERMS OF
ORCHESTRATING OR MANAGING A
MULTI-SITE STUDY WHERE THERE WAS
MULTIPLE DIFFERENT CORES WORKING
DURING THAT PILOT YEAR TO GET
ALL MILESTONES COMPLETED.
WITH ANY GRANT WE DID PRE-WORK
IN TERMS OF GETTING IRB ON
BOARD, MAKING SURE THAT THEY
WOULD BE OKAY WITH THE OPT OUT
STRATEGY THAT WE THOUGHT WAS
CRITICAL FOR ENTERVENGES.
SO THEY WERE ON BOARD BY THE
TIME THAT THE GRANT WAS APPROVE
AND FUND AND WE SUBMITTED OUR
IRB APPLICATION.
SO GETTING SOME OF THAT PRE-WORK
DONE I THINK IS CRITICAL TO HIT
THE GROUND RUNNING.
THAT IS ALL I HAD, LOOKING
FORWARD TO THE DISCUSSION.
>> THANK YOU SO MUCH, MICHAEL.
SO WE CAN GO TO THE FIRST
QUESTION AND THEN INVITE THE
PANELISTS TO TURN ON YOUR VIDEO,
ON VIDEOCAST THEY WILL SEE THE
SLIDE WITH QUESTION AND ALL FOUR
OR FIVE OUR OUR FACES SO THEY
WILL BE ABLE TO SEE US ON VIDEO.
SO TO START OFF A DISCUSSION I
WANTED TO ASK THE GROUP, THINK
WE WILL START OFF WITH THIS
QUESTION TO YOU JAMES AND THEN
OPEN UP TO EVERYBODY ELSE TO ADD
COMMENTS.
WHAT IS THE ONE THING YOU DID
YOU THINK DURING THE PLANNING
PHASE THAT REALLY SET YOUR TRIAL
UP FOR SUCCESS?
>> FIRST I SHOULD SAY I
APPRECIATE THAT QUESTION AND I
WILL ANSWER IT BUT THERE'S SO
MANY THINGS WE DIDN'T DO.
DIDN'T SET UP FOR SUCCESS.
THAT WE LEARNED OVER TIME.
I THINK THAT WHAT I WOULD POINT
TO IS WHAT I MENTIONED ABOUT THE
DATA.
MY SHORT RECITATION ABOUT THE
DATA.
WE KNEW WE WERE GOING TO BE
USING ELECTRONIC HEALTH RECORD
WE WERE GOING TO EXTRACT DATA
ELEMENTS.
IF YOU THINK LET'S SAY WE WERE
STUDYING DIABETES AND GOING TO
BE PULLING HEMOGLOBIN A 1C, MOST
WOULDN'T HAVE THOUGHT, WE WOULD
HAVE THOUGHT THE DATA THAT COMES
OUT ABOUT HEMOGLOBIN A 1C FROM
PATIENTS WAS ACCURATE.
AND IF YOU HAVE THE DATA FOR
THOSE PATIENTS MAYBE SOME PEOPLE
WILL BE OBSESSIVE TO ACTUALLY
VALIDATE THAT BUT MANY WOULDN'T
BE.
ONLY BECAUSE WE GOT SOME HINTS
AND HAD CONCERNS RAISE SOD WE
DECIDED TO GO AHEAD AND DO A
CHART REVIEW.
OF DATA ELEMENTS FOR ADVANCE
CARE PLANNING PIECES.
FOUND OUT AS I SHOWED YOU IT WAS
WILDLY INACCURATE.
THAT COULD HAVE BEEN A KILLER
FOR US.
LUCKILY WE WERE ABLE TO HAVE --
WE HAD ANOTHER ALTERNATIVE
METHOD TO GO.
AND USE BUT IN RETROSPECT WE
WOULD HAVE COLLECTED ALL THOSE
DATA SOMEWHERE ALONG THE WAY
ALERTED TO THE FACT THIS DATA
WERE NOT SO ACCURATE AND HAVE I
DON'T KNOW RESEARCH ASSISTANCE
MEDICAL STUDENTS POURING THROUGH
CHART REVIEWS TRYING TO CLEAN
THE DATA THAT WAS REALLY MESSY.
SO I GUESS THE POINT OF LEARNING
AND MADE THIS WORK WOULD BE TO
MAKE SURE VERY EARLY ON THE DATA
WE ARE COLLECTING THE IS DATA
YOU THINK YOU ARE COLLECTING.
AND IF THERE IS ANY REASON
WHATSOEVER TO THINK OTHERWISE,
WE ALL KNOW MANY OF US ARE
PULLING ADMINISTRATIVE DATA FROM
CHARTS.
IF YOU WORRY ABOUT BEING
OTHERWISE THE VALIDATE IN SOME
WAY.
HONESTLY IDEALLY WE HAVE BEEN
DOING WHEN YOU WRITE A GRANT OR
PRE-PILOT PHASE BUT A LOT OF US
DON'T (INAUDIBLE).
>> THANK YOU FOR SHARING THAT.
SO I WILL OPEN UP TO THE REST OF
THE PANEL.
ARE THERE -- IS THERE A
PARTICULAR THING YOU THINK THAT
MADE YOUR TRIAL SUCCESSFUL BY
DOING IT PLANNING PHASE?
GAIL.
>> I THINK THERE MANY THINGS,
JUST LIKE PREVIOUS PERSON SAID
THAT WERE IMPORTANT.
I THINK THE MOST IMPORTANT WERE
TO THINK ABOUT ALL THE ISSUES
THAT CAN HAPPEN WITH EHR FOLDS
WHETHER FROM DIFFERENT VENDORS,
WE HAD (INAUDIBLE) EPIC AND
REALLY MAKE SURE THAT DATA
DICTIONARIES ARE PERFECT, PEOPLE
ARE SENDING YOU THE DATA THAT
YOU CAN LOOK AT IT WE WERE
CONSTANTLY DOING THAT EVERY
MONTH.
DON'T WAIT TO THE END TO SEE
WHAT HAPPENS.
AND THEN SAY WE WILL PULL IT
BECAUSE THERE ARE MILLION
DIFFERENT REITERATIONS AND
SAMPLE VALIDATIONS THAT NEED TO
BE DONE AND HAVE SOME REALLY
GREAT IT PEOPLE WITH YOU HAVE
DONE OTHER STUDIES TO THAT
REQUIRE HR AT EACH STUDY, NEED
TO FUND PEOPLE WELL THAT ARE THE
IT PEOPLE, SO THEY WILL BE
ENGAGED IN YOUR STUDY AND THEY
WILL WANT TO MAKE SURE IT IS
CORRECT AND GET YOU INFORMATION
ON TIME.
>> GREAT.
THANK YOU.
DOUG, DID I SEE YOU?
THEN MICHAEL.
>> MICHAEL GO I A HEA GO AHEAD.
>> SO THANKS.
I DEFINITELY AGREE WITH GAIL'S
COMMENT ABOUT FUNDING, THE RIGHT
AMOUNT OF IT TIME AS FAR AS OUR
STUDY WE USE PHARMACY DATA TO
IDENTIFY PATIENTS AND TO SEND
OUT TEXT MESSAGES.
DURING THE COURSE OF THIS STUDY
THIS HAPPENED A COUPLE OF TIMES
WHERE THE HEALTH SYSTEM HAS
CHANGED THEIR CODES FROM
MEDICATIONS AND SO THAT RESULTED
IN PATIENTS NOT GETTING TEXT
MESSAGES OR GETTING TEXT
MESSAGES THAT THEY WEREN'T
SUPPOSED TO AND WE WOULDN'T HAVE
FOUND OUT IF OUR LOCAL IT STAFF
HADN'T ALERTED US TO THOSE
CHANGES.
SO I THINK IT IS CRITICAL TO
TEST THE PROCESS DURING THAT
YEAR ONE BUT ALSO TO FUND THEM
THROUGHOUT THE STUDY SO THAT YOU
CAN GET THAT SUPPORT WHEN THINGS
CHANGE AT THE HEALTH SYSTEM.
>> DOUG.
>> HAVING HEARD JAMES MICHAEL
AND GAIL, THE ONE THING TSOS DID
NOT WELL PLANNED BUT STUDY
FOLLOW-UP AND FINISH BEFORE
COVID HIT UNLIKE THE OTHER LEE
STUDIES, IT WAS A STEP WEDGE
DESIGN, ASIDE FROM THAT WE HAVE
A WE RECRUIT CHAMPIONS AT EACH
SITE.
WE HAD AD CHAMPION WHO
ESTABLISHED RELATIONSHIPS WITH
CHAMPIONS AT EACH OF SITES.
WHAT HAPPENED THROUGHOUT THE
TRIAL IF THINGS GOT TOUGH JERRY
WOULD STEP IN AND USE HIS
RELATIONSHIP WITH PEOPLE THE
SURGICAL CHAMPIONS TO TRY TO
RECTIFY WHATEVER WAS GOING ON AT
THE SITE WE NEEDED TO ADDRESS.
>> GREAT.
WHY NOT GO TO NEXT SLIDE.
I HAVE ANOTHER QUESTION, I WILL
START WITH YOU DOUG ON THIS ONE
AND THEN OPEN UP FOR EVERYBODY
ELSE.
IF YOU HAD IT TO DO OVER, ARE
ARE THERE ACTIVITIES YOU WOULD
HAVE ADDED TO YOUR PLANNING
PHASE, IF YOU PILOTED CERTAIN
PARTS WHEN YOU CHANGE WHAT YOUR
HIGHLIGHTED DURING THAT TIME.
>> THERE WAS INTENSIVE DATA
ACQUISITION AND PILOTING GOING
ON.
WE HAD A TRAUMA SYSTEM AND
TRAUMA SURGEON, WE HAD PTSD
INTERVENTION CHAMPION AND IT
CHAMPION BUT DIDN'T DIG DEEPER,
WE JUST REQUIRED CHECKLIST SITE
HAD AN IT CHAMPION AND WE DID
NOT ACTUALLY DIVE INTO THE WEEDS
ON WHETHER OR NOT OR NOT THE
SITES AUTOMATE THE PTSD SCREEN
OR OTHER IT FUNCTIONS IN EHR
CAPACITY DIDN'T DO DRILLING
ENOUGH DRILLING DOWN, TO SLOW
DOWN RECRUITMENT AND NUMBER OF
OTHER CHALLENGES THROUGHOUT THE
TRIAL.
>> OTHERS WANT TO CONTRIBUTE
WHAT YOU WISH YOU WOULD HAVE
DONE OR DONE DIFFERENTLY IF YOU
DO IT ALL OVER?
THIS IS GAIL.
I WOULD SAY THAT WE SHOULD HAVE
PROBABLY DONE MORE ROBUST
TRAINING ON THE IMPLEMENTATION
WITH THE IT INTERVENTION AND IT
WAS TOUGH TO DO THIS, AT PILOT
SITE BUT WE COULD HAVE ADDED
OTHER SITES AND LOOKED MORE
CAREFULLY.
>> I THINK THAT IS THE TWO
THINGS DIFFERENTLY, ONE IS TRY
TO HAVE GOTTEN BETTER BUY IN
FROM CLINICS AND DIFFERENT
CLINIC LEADERS, AND TURNED OUT
IN ONE SITE WE WERE ABLE THE GET
THE BUY IN AND ALMOST
INVENTORIALLY HAD CANCER CENTER
DIRECTOR WHO ATTENDED OUR PILOT
TRAINING, THAT MADE ALL THE
DIFFERENCE, WHEREAS ELSEWHERE WE
DIDN'T HAVE QUITE THAT LEVEL AND
WE PROBABLY SHOULD HAVE.
WORKED A LITTLE BIT HARDER TO DO
THAT.
ALSO THE VIDEO IMPLEMENTATION
WHAT WE CAME TO UNDERSTAND OVER
TIME WAS THAT EACH CLINIC WAS
GOING TO WANT TO DO THIS
DIFFERENTLY, NOT ONLY EACH SITE
WHICH IS WHAT WE MIGHT HAVE
EXPECTED, MAYOR NORTH WELL AND
DUKE, BUT WITHIN MAYO CLINIC
THORACIC ONCOLOGY GROUP WANTED
ONE WAY, THE BREAST ONCOLOGY
GROUP A DIFFERENT WAY.
WE ARE NOT REALLY NECESSARILY
PREPARED FOR DIFFERENT
PERMUTATION AND WHAT THAT MEANT,
SHOULD THE PATIENTS GET EMAIL OR
VIDEO -- CODES THAT WAY,
PHYSICIANS OR BY CARE
COORDINATORS OR WHOEVER.
SO I'M NOT SURE WE COULD HAVE
COME TO FACT ALL THE SITES HAD
DIFFERENT APPROACHES TO DOING
THIS.
I THINK WE COULD HAVE BEEN A
BETTER PREPARED FOR (INAUDIBLE)
EARLIER.
>> ABSOLUTELY.
AND ONE THING MAY COME UP IN THE
NEXT PANEL TOO IS SOMETIMES WITH
THOSE IMPLEMENTATION CHALLENGES
FIGURING WHAT IS CORE AND WHAT
YOU DON'T WANT THEM TO CHANGE
VERSUS WHAT IS VARIABLE AND WHAT
YOU WOULD ALLOW THEM TO CHANGE
CAN BE REALLY HELPFUL TOO IN
THOSE SORT OF PLANNING THIS OUT
HOW YOU DO IT ACROSS SITES.
MICHAEL, DO YOU WANT TO ADD
ANYTHING OR WANT ME TO GO THE
NEXT QUESTION?
>> I THINK I WOULD JUST AGREE
WITH JAMES IN TERMS OF BEING
ABLE TO ENGAGE THE SITES AND
FINDING A GOOD PARTNER THERE
ALTHOUGH WITH US I FOUND AT
LEAST FOR ONE OF OUR SITES, OUR
PARTNERS KEPT CHANGING SO IT WAS
REALLY HARD TO ESTABLISH A
COLLABORATIVE RELATIONSHIP IN
TERMS OF TRYING TO MOVE THE
STUDY FORWARD.
SO I THINK PROBABLY TRYING TO
FIND MORE THAN ONE PERSON AT
ASYME, SEEMS LIKE AT A LOT OF
HOSPITALS THE CLINICAL STAFF
CHANGE.
YOU DON'T WANT TO DEPEND ON ONE
PERSON AND WHEN THEY LEAVE HAVE
TO BE -- RESTART THESE
RELATIONSHIPS.
>> ABSOLUTELY.
MARY JOE IF WE CAN GO TO THE
NEXT SLIDE.
I THINK THIS ONE, I THINK SORT
OF FOLLOWS ON WHAT YOU WERE JUST
SAYING MICHAEL, GOING TO BRING
THIS QUESTION TO YOU.
SO WHAT OTHER ADVICE DO YOU HAVE
IN WORKING WITH THE HEALTHCARE
SYSTEM PARTNERS DURING THAT
PLANNING PHASE.
IN THINKING THROUGH THAT, AS WE
ARE THINKING MORE ABOUT
GENERALIZABILITY OF THE STUDIES,
MAKING SURE WE GET
REPRESENTATIVE PATIENT
POPULATIONS, ARE THE THINGS YOU
CAN DO DURING THAT TIME TO
ENHANCE DIVERSITY OF
PARTICIPANTS OR EVEN THINKING
ABOUT OTHER SITES AND WOULD IT
BE POSSIBLE TO BRING ON OTHER
SITES DURING THE PLANNING PHASE.
>> I THINK IN TERMS OF WORKING
WITH THE HEALTH SYSTEM PARTNERS,
I THINK PART OF IT IS REALLY
TRYING TO AS A RESEARCHER TRYING
TO REFRAME THE GOALS OF THE
STUDY SO THAT THEY SEE THAT THE
INTERVENTION COULD BE
POTENTIALLY BENEFICIAL TO THEM
AND THEIR WORK BECAUSE I THINK A
LOT OF TIMES IT FEELS LIKE AT
LEAST TO THEM WE ARE
RESEARCHERS, WE KNOW HOW TO DO
THIS STUDY, BUT THEIR -- ON THE
OTHER HAND THE HEALTH SYSTEM
PARTNERS ARE TASSENINGED WITH
TRYING TO IMPROVE -- FATSKED
WITH TRYING TO -- TASKED WITH
TRYING TO IMPROVE CLINICAL CARE
AND EFFICIENCY AND OFTENTIMES
RESEARCH IS SEEN AS SOMETHING
THAT IS GOING TO SLOW THE
PROCESS DOWN.
IF WE CAN FRAME OUR RESEARCH AS
A WIN WIN FOR OUR PARTNERS THAT
MIGHT GO A LONG WAYS IN TERMS OF
HELPING TO ENGAGE THEM WITH
STUDY AND HAVING THEM HELP
FACILITATE THE PROCESS.
IN TERMS OF TRYING TO ENHANCE
DIVERSITY N OUR STUDY WE USE EHR
TO IDENTIFY PATIENTS, SO
HOPEFULLY THERE IS LESS
SELECTION BIAS IN TERMS OF
PARTICIPANTS WHO ARE APPROACHED
FOR THE STUDY BECAUSE ALL
PATIENTS WHO ARE ELIGIBLE WOULD
BE APPROACHED.
I THINK IT WOULD BE HARD IN THAT
ONE YEAR PLANNING PHASE TO
ENGAGE NEW SITES AND I THINK
GOING IN IT WOULD BE NICE TO
HAVE THOSE SITES IDENTIFIED
BECAUSE SOME OF THESE
RELATIONSHIP BUILDING AND THEN
ALL THE TECHNICAL ISSUES MAY
TAKE MORE THAN THAT ONE YEAR TO
KIND OF DEVELOP RELATIONSHIPS
AND EXECUTE THAT PILOT WORK TO
MAKE SURE THAT THE DATA IS WHAT
YOU THINK IT IS.
>> GREAT.
GAIL.
>> I WAS GOING TO SAY THIS IS
WHY THIS IS SUCH A GRANT THAT IS
FRAUGHT WITH DIFFICULTIES AND
ONE HAS TO TRY TO ANTICIPATE
THEM.
SO WHEN YOU DESIGNING GRANT,
SUBMITTING A GRANT YOU WANT TO
ENGAGE DIFFERENT HEALTHCARE
SYSTEMS BUT YOU NEVER KNOW ABOUT
THEIR LEVEL OF INTEREST ALWAYS
FROM THE BEGINNING SO YOU HAVE
TO SPEND A LOT OF TIME ON
THINKING ABOUT THAT BEFORE YOU
GET THE GRANT.
SO LOT OF RELATIONSHIP BUILDING
BEFORE THAT TO MAKE SURE IT IS
GOING TO GO WELL BECAUSE MANY
THAT TEN MONTHS YOU DON'T HAVE A
LOT OF TIME.
WE WERE STRESSED BY THE FACT
THAT WE NEEDED TO MOVE SITES AND
WE NEED TO ADD SITES AND WE WERE
VERY LUCKY TO GET A SUPPLEMENT,
WE WOULD HAVE BEEN ABLE TO DO
THAT WITHOUT HEALING COMMUNITY
SUPPLEMENT, SO THAT WAS GREAT.
BUT IT REALLY STARTS PUTTING THE
-- THE APPLICATION IN, NOT AT
THE TIME YOU ARE AWARDED THE
APPLICATION, AND GETTING THOSE
PARTNERS AND IN PLACE AND AS
SOMEONE SAID MORE THAN ONE
PARTNER, BECAUSE THAT PERSON CAN
LEAD SO YOU WANT TO MAKE SURE
YOU HAVE MULTIPLE PARTNER THERE
AND IN THE PLANNING STAGE
LOOKING AT THE DIVERSITY OF
PARTICIPANTS WOULD BE RIGHT OFF
THE BAT, WHERE DEMOGRAPHICS ARE
IN THE PARTICULAR SITE THAT YOU
ARE GOING TO USE BECAUSE THERE
IS NOT MUCH YOU CAN DO ONCE YOU
PICK YOUR SITE.
>> YEAH, I ECHO WHAT MIKE AND
GAIL ARE SAYING.
IN THE TSOS STUDY WE STARTED, IT
IS THE OPPOSITE OF GAIL WE START
WITH PARALLEL GROUP CLUSTER
RANDOMIZE DESIGN AND SWITCHED TO
THE STEP WEDGE SIMULTANEOUSLY
LOOKING AT THE BIOSTATS SCORE,
WE HAD 20 SITES, WE HAD A A
PREVIOUS PRAGMATIC TRIAL LOOKING
AT ALCOHOL PRIOR TO PROPOSING
THE TSOS TRIAL SO WE HAD A
PRACTICE NETWORK, 20 SITES WE
WORKED WITH THAT HAVE RESEARCH
EXMEANS DATA EXPERIENCE IRB
EXPERIENCE.
BUT WE DIDN'T WANT TO USE THOSE
20 SITES AND IT WASN'T UNTIL WE
WERE ASKED TO GO FROM 20 TO 24
SITES WE WENT BACK WITH NUMBER
OF THOSE PREVIOUSLY WORKED WITH
SITES AND SAID HEY WE HAVE THE
SECOND TRIAL, YOU HAVE DONE
RESEARCH WITH US BEFORE, CAN YOU
SIGN ON AND THAT WAS DURING THE
PLANNING PHASE AND SORT OF
RUSHED BUT WE WORKED WITH THOSE
SITES BEFORE.
OTHERWISE 21 SITES WE HADN'T
WORKED WITH BEFORE.
>> I WOULD AGREE WITH ALL THAT I
HEARD BEFORE AND I THINK WITH
REGARD TO DIVERSITY WHILE THERE
MAY BE SOME THINGS THAT CAN BE
DONE WITHIN THE CONTEXT OF
EXISTING SITES I BELIEVE THE
SITES DICTATE THE PERSON MORE
THAN ANYTHING ELSE, THAT IS
WHERE THAT DECISION HAS TO BE
MADE.
THIS WHOLE ISSUE OF BRINGING IN
SITES HOW YOU GET SITES TO WORK
AT MULTI-SITE STUDY, IS ONE THAT
IS ALWAYS FAR FOR ME BECAUSE THE
TRUTH IS, YOU NEED TO GO IN WITH
A GRANT, WITH AT LEAST SOME
COMMITMENT AND SOME SENSE OF
WHERE YOU ARE GOING TO BE DOING
IT AND PRE-EXISTING
RELATIONSHIPS TO KNOW THAT YOU
CAN DO IT BECAUSE IT WILL SAVE
YOU TIME IN THE PLANNING PHASE
OR JUMPING INTO THE WORK, BUT
ALSO GRANTSMANSHIP PERSPECTIVE
REVIEWERS ARE KEPT CAN BELIEVE
IF THEY DON'T FEEL LIKE IT IS
REALLY THERE.
ON THE OTHER HAND THERE IS SO
MUCH WORK YOU CAN DO, UNLESS YOU
HAVE A LONG STANDING
RELATIONSHIP WITH THE SITE AND
DONE OTHER PROJECTS WITH THEM,
THERE IS SO MUCH WORK YOU CAN DO
IN A PRE-SUBMISSION STATE.
WHERE YOU HAVE NO FUNDS TO BE
DOING THIS.
TO REALLY FIND OUT WHAT IT IS
GOING TO LOOK LIKE WHEN YOU ARE
ACTUALLY DOING THE PROJECT.
SOME PROJECTS I FOUND THESE
THINGS YOU DISCOVER, DISCOVERY
THINGS, WHAT IS DOABLE AND NOT
DOABLE.
IF I HAD I DON'T KNOW HOW
PRACTICAL THIS IS.
I DON'T KNOW IF ANYONE HAS DONE
THIS BEFORE BUT IN AN IDEAL
WORLD WHICH ISN'T NICE I MIGHT
GO IN, IF I NEED THREE SITES I
MIGHT GO INTO FIVE AND SEE WHO
WOE UNDERSTANDING OF COMPETITION
TO SEE WHO IS GOING TO SUCCEEDS.
AND THEN GO FORWARD BECAUSE THE
PROBLEM OF GOING WITH THREE AND
THEN NEEDING FIVE ADDITIONAL IS
DO YOU HAVE THE BUDGET EXTENDED
ACCOUNTED FOR YOUR BUDGET, I
GUESS GAIL GOT A SETTLEMENT BUT
THAT IS SOMETIMES HARD TO DO.
SO IT WOULD BE GREAT TO BE ABLE
TO GO INTO THIS KIND OF A PHASE
AND HAVE THE BEST ONES SURVIVE
ALTHOUGH IT IS BRUTAL AND
PROBABLY NOT GREAT FOR
RELATIONSHIPS IF YOU WANT TO
CONTINUE WITH OTHERS SO NOT
NECESSARILY FROM THAT BUT
(INAUDIBLE) PERSPECTIVE GETTING
IT DONE.
>> WE HAVE IN IF OTHER PROGRAMS
SEEN PEOPLE WHO HAVE A SET OF
POTENTIAL SITES BUT THEY THEN
NEGOTIATE WITH OVER TIME BUT I
THINK YOU ARE RIGHT, JAMES, THIS
IDEA OF IT IS HARD TO FIX A
DIVERSITY ISSUE LATER.
IF YOU DON'T HAVE COME IN WITH
SITES THAT REACH WHETHER
UNDERSERVED POPULATIONS OR
DIVERSE COMMUNITIES OR WHOEVER
IT IS, SO IN YOUR APPLICATION
PROCESS REALLY NEEDING CAREFULLY
TO LOOK AT THOSE ISSUES, MAKING
SURE THAT YOU ARE GOING TO GET
THE POPULATION YOU WANT WITH THE
PARTNERS THAT YOU COME IN WITH,
I THINK IS A REALLY INTERESTING
ELEMENT.
AND MICHAEL, YOUR THOUGHT THERE
ABOUT THE WAY YOU REACHED OUT TO
FOLKS, YOU WEREN'T WAIT FOR A
REFERRAL FROM SOMEBODY, YOU WERE
DIRECTLY GOING TO THE EHR AND
REACHING OUT OF EVERYBODY WHO
WAS ELIGIBLE INSTEAD OF HAVING
POTENTIAL REFERRAL BIAS IN THE
TYPES OF PATIENTS THAT WERE
REFERRED FOR STUDIES.
>> I WAS JUST GOING TO SAY I
TOTALLY AGREE WITH JAMES, SEEMS
LIKE THE GRANT PROCESS DOESN'T
ALLOW FOR TRYING TO INCREASE
DIVERSITY IF YOU DON'T ALREADY
HAVE IT GOING IN JUST BECAUSE IT
IS HARD WHEN YOU ARE WRITING THE
GRANT TO ASK SITES TO DO A LOT
OF THIS EXTRA WORK WHEN YOU KNOW
CHANCES OF GETTING THE GRANT ARE
SMALL, ASKING FOR ALL THIS DATA
AND PILOT LIEU THIS PROCESS.
ONCE YOU GET THE GRANT YOU
WANTED TO EXECUTE IT AND IT IS
VERY HARD TO THEN GET NEW SITES
ON BOARD JUST BECAUSE YOU ARE
STARTING FROM GROUNDS ZERO IN
TERMS OF TRYING TO DEVELOP THESE
RELATIONSHIPS THAT ARE CRITICAL
BECAUSE IT IS NOT JUST A ONE
YEAR PROCESS, YOU ARE TRYING TO
DEVELOP THIS RELATIONSHIP FOR
OVER A FOUR TO FIVE YEAR PERIOD.
>> IT IS INTERESTING THIS IDEA
YOU WERE ABLE TO GET A
SUPPLEMENT TO BRING ON
ADDITIONAL SITES AND MAYBE THAT
SOMETHING FROM THE FUNDING SIDE
WE COULD THINK ABOUT.
THE OTHER QUESTION WE SOMETIMES
HAVE FROM THE FUNDING SITE WE
ALWAYS USE THE SAME HEALTHCARE
SYSTEMS, SEE THE SAME HEALTHCARE
SYSTEMS COME THROUGH THE SYSTEM
WHO WERE GREAT AT DOING THIS
WORK.
HOW DO WE DO RESEARCH IN GROUPS
AND POPULATIONS THAT DON'T
TYPICALLY PARTICIPATE OR
HEALTHCARE SYSTEMS THAT REACH
PEOPLE THAT DON'T OFTEN
PARTICIPATE IN RESEARCH AND
MAYBE DON'T KNOW HOW THESE
INTERVENTIONS WORK IN THESE
DIFFERENT POPULATIONS.
WHAT WAS YOUR EXPERIENCE IN
GETTING SUPPLEMENT TO GET TO DO
THAT WORK AND ADD ON SITES?
>> WE ARE EXTREMELY LUCKY
BECAUSE WE ARE WORKING MANY THE
SPACE, USE DISORDER SO IN THAT
SPACE THERE IS MONEY AVAILABLE
THROUGH THE HEALING COMMUNITIES
INITIATIVE.
SO WE WERE FORTUNATE.
WOULDN'T BE ABLE TO DO THAT
OBVIOUSLY.
THAT WOULD HAVE BEEN HORRIBLE
FOR US IF WE WEREN'T ABLE TO DO
THAT.
SO THAT WAS JUST WE WERE LUCKY
TO BE ABLE TO.
I WOULD SAY THAT THIS --
EVERYTHING COSTS A LOT OF MONEY
SO IT IS A GIVE AND TAKE OF WHAT
YOU CAN DO.
AND WHAT YOU CAN'T DO AND THE
OTHER REALLY DIFFICULT THING
WITH USING THIS STUDY AND OTHER
STUDIES THAT I HAVE DONE WITH
COMMUNITY SITES, IS THAT THE NIH
IS NOT THAT NIMBLE ABOUT GIVING
WHEN THE NOAs COME OUT AND HOW
MONEY IS GIVEN SO YOU CAN OFTEN
-- YOU CAN OFTEN DO AT RISK
ACCOUNTS IN MULTIPLE YEARS, YOU
DO ACCOUNTS FOR ACADEMIC SITES
AND THAT HAPPENS ALL THE TIME,
WE SAY OKAY, BUT YOU CAN'T
REALLY TELL A COMMUNITY PLACE
THAT'S GOT ONE IT -- ONE IT
PERSON OR SOMETHING OH BY WAY,
YOU NEED TO DO THIS EXTRA AMOUNT
OF WORK BECAUSE THEY CAN HIRE A
WHOLE NEW IT PERSON, IT IS
REALLY PUTTING IT ON ONE PERSON
AND BY THE WAY, YOUR MONEY IS
COMING, I PROMISE YOU NOT QUITE
SURE WHEN.
SO THAT WITH IS WHAT MAKES IT
DIFFICULT TO INCLUDE COMMUNITY
SMALLER WORLD WHICH WE NEED TO
INCLUDE BUT WE HAVE TO GIVE THEM
ALMOST EXTRA MONEY IN A WAY AND
MAKING SURE THAT WE ARE PAYING
THEM ON TIME.
>> HELPFUL FEEDBACK.
DEFINITELY FOR US AS FUNDERS TO
THINK ABOUT IN MY COLLEAGUES
WHETHER NIH AND OTHER FUNDING
AGENCIES IMPORTANT TO THINK HOW
WE MIGHT BE ABLE TO DO THAT.
BECAUSE I THINK THERE IS
DEFINITELY INTEREST THERE.
WHY DON'T WE GO TO THE NEXT
QUESTION.
THIS IS THE LAST STANDARD
QUESTION THAT I HAVE AND I WILL
ENCOURAGE THE AUDIENCE TO SEND
IN QUESTIONS VIA EMAIL AS WELL
AS OUR PANELISTS THAT ARE HERE
ON THE ZOOM TO SUBMIT QUESTIONS
TO CHAT BECAUSE WE WILL HAVE
TIME FOR QUESTIONS BUT THIS LAST
QUESTION I WILL FIRST POSE TO
YOU GAIL WHAT ACTIVITIES IN
PLANNING PHASE TO DETERMINE
QUALITY OF DATA AND HOW YOU END
UP PUTTING THE DATA ACROSS YOUR
DIFFERENT HEALTHCARE SYSTEMS?
>> I THINK I MENTIONED EARLY ON
WE NEEDED TO I GUESS DURING THE
PLANNING PHASE TEST SOME OF
THIS.
SO WE HAVE TO REALLY LOOK AT ALL
OF THOSE HR DATA ELEMENTS AND
HOW THEY WOULD BE INTERPRETED AT
EACH SITE, HOW TO DO THAT AND
FACILITATE ON LARGE SCALE
COLLECTION.
WE HAD TO ENSURE THAT DATA
COLLECTION WAS CONSISTENT
BETWEEN EACH SITE AND WE HAD TO
VALIDATE IT AT EACH SITE SO THIS
WHOLE THING, THIS WHOLE MAPPING
AND QUALITY ASSURANCE WAS A HUGE
LIFT.
WE HAD TO DO THAT EARLY ON AND
MAKE SURE IT HAPPENS SO WE KEPT
ASKING THEM FOR SOME DATA
INITIALLY AND THEN SEND TO OUR
DATA COORDINATING SESSION WE DID
MULTIPLE ROUND AND SAMPLED DATA
BY OUR TEAM WHICH WAS GREAT HERE
ARE DATA COORDINATING CENTER.
WE HAD TO DO THAT WITH BOTH
SYSTEMS, TURNER AN EPIC SYSTEMS
SO THAT BECAME MORE COMPLICATED
BUT BY MONTHLY DATA COLLECTION
FROM SITES SENT HERE WE WERE
ABLE TO MONITOR THAT.
ALL DATA WAS DEIDENTIFIED SO WE
WERE ABLE TO SHARE IT WITH
INVESTIGATIVE TEAM IN OUR STUDY.
SO THAT WAS GOOD.
>> OTHER COMMENTS ON IN QUESTION
FROM THE PANELISTS?
>> FROM OUR PERSPECTIVE IT WAS
ALL ABOUT GETTING PEOPLE -- IT
WAS ALL ABOUT GETTING GOOD
ACCESS O THE SITE AND MADE ALL
THE DIFFERENCE AND WHERE WE HAD
THAT, THE DATA WAS SO MUCH
BETTER.
WHERE WE DIDN'T SEND TO DO THIS
THEN THE QUESTION WAS PAYMENT,
SO FORTH.
THAT WAS THE ISSUE.
>> DOUG OR MICHAEL.
>> I THINK FOR US IT WAS
CREATING A DATA DICTIONARY
AGREED UPON DATA DICTIONARY THAT
WE CAN MAP OUT USE IT TO MAP OUT
ANOTHER EACH SITE, EACH OF THE
SITES HAD DIFFERENT WAYS OF
IDENTIFYING THE CLINICAL
PHENOTYPE THAT WE WERE
INTERESTED.
FOR EXAMPLE HYPERTENSION
PATIENTS WITH HYPERTENSION THE
CODES TO IDENTIFY THESE PATIENTS
WERE DIFFERENT AT EACH OF THE
SITES AND WE NEEDED THE
EXPERTISE OF EACH OF OUR DATA
PERSON AND IT PEOPLE AT EACH
SITE FOR US THESE ARE THE CODES
THAT WOULD HELP YOU IDENTIFY
PATIENTS WITH HYPERTENSION.
I THINK HAVING EXPLICIT CRITERIA
AND HAVING THE SITES TELL US
WHAT WOULD BE THE BEST CODES WAS
HELPFUL.
>> I WOULD ADD THAT TOO, WE DID
PEEN TYPE OF OUD AND REALLY CHAD
AND HIS GROUP DEVELOPED WHAT
THOSE WOULD LOOK LIKE AND TESTED
THEM.
>> DOUG.
>> THE ELECTRONIC HEALTH RECORD
PILOTING OF THE PTSD SCREEN WAS
ONE THING WE DIDN'T DO WELL
ENOUGH IN THE PILOT PHASE.
WE DO CENTERS REQUIRE TO KEEP
REGISTRIES COMPLETE CENSUS OF
ALL PATIENTS WITH MULTIPLE
VARIABLES WITH REGARD TO CD
SUBSTANCE USE.
WE HAVE DONE A PRIOR PRAGMATIC
TRIAL SO WE KNEW WE COULD GET
THAT DATA ALREADY NICELY
CATEGORIZED, THAT WAS ONE OF THE
DESIGN FEATURES THAT WE NEW
GOING IN EVERYBODY WOULD HAVE A
TRAUMA REGISTER, THEY HAD TO AND
WE COULD MANIPULATE THAT DATA AT
THENT OF THE STUDY ONCE WE GOT
IT FROM THEM.
>> THANKS, DOUG.
THERE IS A QUESTION THAT CAME IN
VIA THE CHAT WHICH IS A COMMON
-- BUT WONDERING DOUG I WILL
READ IT OUT BUT I THINK ONE OF
THE THINGS YOU MENTIONED SORT OF
MADE ME THINK OF YOU TO MAYBE
TALK ABOUT THIS.
THE COMMENT IS THAT THE
SELECTION OF SITES AND PICKING
SITES THAT HAVE A TRACK RECORD
SEEMS TO BE REALLY IMPORTANT
CRITICAL CONSIDERATION BECAUSE
IF YOU PICK SITES THAT REALLY
DON'T KNOW HOW TO DO THIS, IT
COULD INTRODUCE A LOT OF
PROBLEMS ALONG THE WAY.
I REMEMBER HEARING YOU SAY THE
SITES YOU CAME IN WITH WERE ONES
THAT HADN'T WORKED IN YOUR
PREVIOUS STUDY OR HADN'T WORKED
WITHIN YOUR PREVIOUS TRIAL SO
WHAT IS YOUR THOUGHT ABOUT
WORKING WITH SITES THAT ARE
MAYBE NEW TO RESEARCH OR NEW TO
MULTI-SITE KINDS OF TRIALS.
>> TOTALLY AGREE THERE IS THIS
TENSION.
FOR US WE WANT A REPRESENTATIVE
SAMPLE OF SITES BECAUSE WE ARE
GOING TO TAKE OUR DATA AND MAKE
CASES FOR ALCOHOL SCREENING OR
BRIEF INTERVENTION PTSD BRIEF
INTERVENTION AMERICAN COLLEGE OF
SURGEONS.
THERE IS NOTHING WORSE THAN
PICKING 15 -- 24 UNIVERSITY
SITES AND THEN HAVING COMMUNITY
HOSPITALS THAT HAPPEN TO HAVE
LEVEL ONE TRAUMA CENTERS HAVE TO
SORT OF DO SCREENING AN
INTERVENTION PROCEDURES THAT
DERIVE FROM UNIVERSITY SITES SO
WE REALLY TRY TO NOT JUST GO
WITH UNIVERSITY ACADEMIC SITES
THIS IS ATTENTION TO PRACTICE
RESEARCH NETS WORK.
THERE ARE ACUTE CARE PRACTICE
RESEARCH NETWORKS A LOT OF KEY
STUDIES ARE DONE.
WHEN IT COMES TO WIDESPREAD
SCALE UP AND IMPLEMENTATION,
THAT DOESN'T WORK, YOU WANT TO
HAVE THE PROBLEMS YOU HAVE AT
THE SITES THAT ARE MORE
REPRESENTATIVE OF THE REAL
WORLDERS THERE'S ALSO DIVERSITY
OF PARTICIPANTS SO IT IS A REAL
TENSION.
AND WE A LOT OF REGULATORY
PROBLEMS, THE OTHER THING, THE
SITE HASN'T DONE A LOT OF
RESEARCH YOU CAN RAN RUN INTO
MAJOR REGULATORY PROBLEMS AN
TSOS RAN INTO THAT.
AND THAT IS ONE OF THE PROBLEMS
WITH DIVERSITY OF SCIENCE THAT
DON'T HAVE THE FULL EXPERIENCE.
>> I WILL SECOND THAT, THAT IS
REALLY IMPORTANT.
EFFICACY STUDIES ARE DONE BEST
IN LARGE COMPLICATED HEALTH
SYSTEMS WHO CAN DO THINGS.
PRAGMATIC TRIALS OR
IMPLEMENTATION TRIALS, IF WE
DON'T PUSH THEM OUT TO MORE
REPRESENTATIVE SITES AROUND THE
COUNTRY, THERE'S ONLY A FEW
SITES COMPARED TO ALL OF THE
COMMUNITY RURAL SITES, IF WE
DON'T PUSH THE ENVELOPE TRYING
TO INCLUDE THOSE IN THESE
PRAGMATIC IMPLEMENTATION TRIALS
WE REALLY AREN'T DOING ANYONE A
FAVOR.
IF WE HAVE OUR NIH PARTNERS
HERE, THAT INVOLVES A LOT OF
MONEY.
GETTING THOSE SITES REALLY
ALMOST FTEs WORTH OF PEOPLE TO
BE ABLE TO BRING THEM IN.
AND TRY IT.
WE HAVE TO HAVE GOOD FAITH THAT
WE ARE GOING TO SEE IT THROUGH
IF THEY HIRE PEOPLE.
IT IS A PROBLEM WE ARE NOT GOING
TO SOLVE TODAY BUT IT IS
IMPORTANT WE SPREAD THESE OUT
FOR IMPLEMENTATION TRIALS.
NOT EFFICACY TRIALS BUT
IMPLEMENTATION TRIALS IS
ESSENTIAL.
>> I SEE DR. TURN NEAR, DID YOU
WANT TO GET IN ON THIS
CONVERSATION?
>> THANK YOU.
QUICKLY I WANT TO SAY I THINK
THIS CONFERENCE IS FANTASTIC, I
HAVE BEEN LISTENING FOR THE LAST
TWO DAYS MY COMMENT DOUG AND
DALE, I THINK THAT WHAT I AM
LEARNING FROM OUR EXPERIENCE AND
SOME -- HEARING SOME OF THIS
GROUP, WE NEED TO FIGURE OUT OR
DEVELOP STRATEGIES HOW TO GET
OUTSIDE GROUPS WE ARE ALL
REACHING OUT TO RYE TO BE GET
DIVERSITY, COMMUNITY PEOPLE
ENGAGED SO IT IS REAL WORLD.
IT IS TOUGH ENOUGH IN YOUR OWN
ACADEMIC HEALTH CENTER TO GET
YOUR HEALTH SYSTEM TO APPLY AND
AGREE AND DELIVER POINT OF CARE,
WHATEVER.
INTERVENTION IS THAT YOU ARE
TRYING TO DO.
LET ALONE SOMEBODY WHO YOU MAY
NOT HAVE WORKED WITH FOR TEN,
10, 20 YEARS.
I DON'T KNOW IF THERE MANY OF
YOU, MANY OF YOU ARE
CO-INVESTIGATORS, I HAVE NEVER
BEEN INVOLVED WITH PCORI, MAYBE
THERE'S LESSONS, I DON'T KNOW.
BUT THIS GROUP IS PROBABLY
LEADING EDGE IN THIS COUNTRY FOR
THESE KINDS OF TRIALS.
IT IS SOMETHING -- A TOUGH NUT
TO CRACK BUT WE HAVE TO COME UP
WITH STRATEGIES AND FIGURE HOW
TO DO THAT.
I DON'T HAVE THE ANSWER.
JUST RAISING THIS BECAUSE IT IS
PEEKING NOW.
YOU CAN'T JUST GO INTO THE
COMMUNITY AS PEOPLE HAVE SAID,
YOU CAN'T JUST GO INTO COMMUNITY
AND SAY WE WANT TO PARTNER AND
GET THIS GRANT AND THEY SAY
YEAH, WE WANT TO PARTICIPATE AND
WITH RECOGNIZED BUT DOWN THE
ROAD THEY GET SOLD OFF.
YOU HAVE A NEW SHERIFF IN TOWN.
NOTHING YOU CAN DO ABOUT THAT
PER SE.
PROACTIVELY WHAT I AM THINKING
NEXT TIME I WILL WRITE PRAGMATIC
CLINICAL TRIAL I WILL DO MY
DARNEST TO IDENTIFY WHEREVER
THEY ARE, PLACES THAT HAVE A
TRACK RECORD DOING THIS, ALREADY
HAD SOME EXPERIENCES EVEN THOUGH
MAY NOT BE PERFECT AND TRY TO
GET THEM ENGAGED.
MAYBE Y'ALL ALREADY KNOW ALL
THIS.
BUT I THINK THIS IS GROUP IS THE
LEADERS HELP MOVE THIS FIELD
FORWARD AND DEFINITELY NEEDS TO
MOVE FORWARD.
>> THANK YOU SO MUCH.
GREAT TO HAVE YOUR INPUT.
ANY OTHER REACTION FROM THE
PANELISTS?
IN TERMS OR THOUGHTS OF HOW BECK
DO THIS?
>> ANOTHER THING WITH RFA YOU
HAVE POT OF MONEY PUT ASIDE FOR
UNFORESEEN CHALLENGES THAT COME
UP WITH A GRANT LIKE THIS.
AND THAT MONEY IS APPLIED FOR
FOR THE SITES THAT YOU HAVE
CHOSEN.
YOU CAN ACTUALLY -- WE ARE
EMERGENCY PHYSICIAN SO YOU HAVE
TO PREPARE FOR EVERYTHING AND
WHEN YOU PREPARE FOR SOMETHING
THERE IS NO EMERGENCY.
IF YOU WERE TO PREPARE UP FRONT
THERE IS GOING TO BE TROUBLE
THERE WILL BE CHALLENGES AND
THAT THERE IS THIS POT LEFT AND
THAT POT CAN BE DIVIDED OR GIVEN
TO SYSTEMS THAT RUN INTO
DIFFICULTIES IT MIGHT BE BETTER
TO DO THAT THAN HAVE FIVE OR SIX
SITES STRUGGLE AND NOT GET THE
OUTCOMES THEY NEED.
>> I WOULD QUESTION WHETHER OR
NOT IT IS A MONEY ISSUE.
IN OTHER WORDS HAVING MORE MONEY
MAY OR MAY NOT SOLVE RECRUITMENT
OR INTERVENTION HETEROGENEITY
PROBLEM YOU FACE EVEN IF YOU
THROW MORE DOLLARS, NOT SAYING I
DON'T WANT MORE MONEY, EPAULES
DO.
JUST SAYING IF YOU HAVE THAT
MORE MONEY YOU HAVE TO HAVE A
DARN GOOD REASON FOR HAVING IT
AND THEN SPENDING IT YOU HAVE TO
HAVE OUTCOME YOU EXPECT TO
ACHIEVE BY SPENDING.
>> WE ARE WORKING INDUCTIVELY
FROM A FEW TRIALS.
ONE THAT COMES TO MIND THAT
ADDRESSES THE ISSUE IN A NOVEL
WAY IS THE COLLABORATION BETWEEN
SUSAN WONG AND BAIT AND REDUCE
PRAGMATIC TRIALS WHERE THEY HOE
I FORGET WHAT HIS ORGANIZATIONAL
TITLE WAS BUT HE BASICALLY
OVERSAW HOSPITAL CORPORATION OF
AMERICA, THEY HAD A BACTERIAL
ANTIBACTERIAL BATHING PROTOCOL.
AND BASICALLY THE DEAL WAS IF IT
WORKED THERE WOULD BE UPTAKE.
THEY HAD OVER 50 HOSPITALS
DIVERSE GROUP OF HOSPITALS ALL
ONE CORPORATION SO THAT IS STAKE
HOLDER BUY IN AT THE GET GO CAN
BE REALLY IMPORTANT.
IN TERMS OF SORT OF THINKING
DIFFERENTLY, IN TERMS OF
COMPARATIVE HEALTHCARE SYSTEMS,
SO OFTEN WE ARE TRYING TO HELP
IT HAPPEN BUT BEING ABLE TO
DECREE THIS WORK THAT IS SORT OF
A MAKE IT HAPPEN SYSTEM.
THAT IS ONE WAY AROUND THIS
ISSUE.
>> THAT MEANS YOU HAVE GOT A
COMMITTED LEADERSHIP.
YOU HAVE PEOPLE COMMITTED AT
THAT HIGH LEVEL TO GET THAT
DONE.
AGREE WITH THAT.
THAT IS A STRATEGY WE ARE
WORKING WITH AT OUR INSTITUTION.
NOT PARTICULARLY IMPACT TRAUMA
BUT OTHER AREAS THAT INVOLVE
HEALTH SYSTEM NECESSARILY.
>> THE OTHER PROGRAM FOR LAST
YEAR'S WORKSHOP WE DID A GRAND
ROUNDS SERIES.
JULIE FRITZSCHE PRESENTED WITH
HER CAN'T REMEMBER NAME OFF TOP
OF MY HEAD, HER IMMUNITY PARTNER
WORKING WITH FEDERALLY QUALIFIED
HEALTH CENTERS ACROSS THE STATE
OF UTAH FOR A NUMBER OF YEARS.
AND THEY HAVE BUILT THIS LONG
STANDING PARTNERSHIP WITH THE
COMMUNITY CLINICS ACROSS THE
STATE.
THEY CAME TO HER WITH A QUESTION
SAYING HOW DO WE ADDRESS PAIN
WITH OUR FACILITIES AND SHE WAS
THE PAIN RESEARCHER WHO THEN
BROUGHT IN HOW DO WE DO THIS TO
HELP THE COMMUNITY CLINICS SO
THEY HAVE LOOK WHERE THE
COMMUNITY CLINICS HAVE BEEN
IDENTIFYING WHAT THE QUESTIONS
ARE, HE GOES AND FIND RESEARCHER
TO ADDRESS THE QUESTIONS AT THE
COMMUNITY HEALTH CENTERS HAVE
WHICH IS A REALLY INTERESTING
DIFFERENT WAY THAT I HAVEN'T
SEEN GROUPS DO WHAT WOULD BE
GREAT MARY JO I WILL COME AHEAD
A MOMENT AND THEN GO TO VICTOR.
IF YOU CAN GO TO THE QUESTION
SLIDE, SKIP THROUGH -- PERFECT.
ENCOURAGE OUR AUDIENCE, IF YOU
HAVE QUESTIONS, PLEASE SEND THEM
IN.
BUT DR. SOLAS DID YOU WANT TO
JUMP INTO THE CONVERSATION AS
WELL?
>> I WANTED TO CLARIFY THAT THAT
ORGANIZATION NAME IS OUCH.
ASSOCIATION FOR UTAH COMMUNITY
HEALTH GROUP.
WHAT YOU SAID WAS ACCURATE.
JUST SPEAKING ON BEHALF OF DR.
FRITZSCHE.
>> THANK YOU SO MUCH.
OTHER QUESTIONS FROM OUR GROUP
HERE?
I WILL TAKE A MOMENT AND SEE IF
THERE'S ANYTHING FROM THE
AUDIENCE.
ONE THING WE TALKED ABOUT IN
PLANNING FOR THIS MEETING AND
THE SESSION WAS SORT OF IS THERE
ANYTHING YOU CAN DO PLANNING
PHASE TO ANTICIPATE THINGS THAT
MIGHT COME UP DURING THE TRIAL?
DOUG YOUR TALKED SECULAR TRENDS
AND CHANGES THAT HAPPENED
NATURALLY OVER TIME AND WE HEARD
FROM A FEW PANELISTS, HAVING
MULTIPLE CHAMPIONS AT EACH SITE
BECAUSE INEVITABLY PEOPLE MOVE
ON OR MORE LIKELY THEY PROMOTE
IT OR HIRED OUT BY SOMEBODY EL
ELSE.
ANYTHING YOU RECOMMEND IN TERMS
HOW YOU ANTICIPATE THOSE SECULAR
TRENDS?
>> THIS IS NOT SECULAR TRENDS,
THIS IS ABOUT OTHER -- BUT OTHER
THINGS SPECIFIC SECULAR TRENDS
ARE HARD TO PREDICT IF YOU --
YOU ADDRESS CREATE SKILL.
BUT I THINK YOU MENTIONED THIS
IN YOUR INTRODUCTION AND NONE
TALKED ABOUT IT.
THIS ISSUE WITH HARD SITES, I
WANT TO EMPHASIZE THAT.
BECAUSE IT IS NATURAL DURING
PILOT PHASE OR INITIATION PHASE
TO GO TO THE PLACE WHERE YOU
KNOW YOU CAN GET THE THING DONE
YOU HAVE YOUR BEST WORKING
PARTNERS, YOU CAN FEEL LIKE YOU
CAN TRY THINGS OUT.
I HAVE FOUND NOT JUST
(INAUDIBLE) BUT OTHER STUDIES I
HAVE BEEN INVOLVED WITH, THAT IT
CAN LEAD ONE TO FALSE CONFLICTS
BECAUSE YOU CONSCIOUSLY OR
UNCONSCIOUSLY WE BIAS FORWARD
STARTING WITH THE GROUP WHATEVER
THAT MIGHT BE THAT IS MOST --
THAT IS EASIEST OR GOING TO BE
MOST AMENABLE.
AND THAT IS A NATURAL
INCLINATION.
IN FACT YOU THEN THE NEXT SIDE
TURNED OUT VERY DIFFERENCE AND
HARD.
SO I THINK IT IS AS CRAZY AS IT
SOUNDS IDEALLY YOU CAN DO TWO
PLACES ONE WHERE YOU CAN GET
PROCESSES EASILY PUT INTO PLACE
AND MAKE SURE YOU HAVE
COOPERATION.
MAYBE A SITE THAT YOU ARE
WORRIED ABOUT.
AND GET A SENSE OF WHAT ARE THE
CONCERNS, BECAUSE WHERE IS THE
PUSH BACK COMING FROM.
WHAT ARE THE RESISTANCE.
TO RESEARCH OR GETTING THE DATA
OUT.
THAT IS ANY MODULE FOR ME GOING
FORWARD START WITH SOME SITES
THAT ARE NOT GOING TO BE AS
EASY.
>> GOOD ADVICE.
OTHERS?
JAMES THIS TRANSITIONS MADE TO
THIS LAST QUESTION I WAS GOING
ASK YOU AND TIME FOR ALL OF YOU
TO ADDRESS THIS, THIS SORT OF --
WHAT IS YOUR ADVICE TO THOSE
PUTTING THESE APPLICATIONS
TOGETHER OR THINKING ABOUT
DURING PRAGMATIC TRIALS COUPLE
OF THINGS YOU ENCOURAGE THEM TO
DO DURING THEIR PLANNING PHASE.
YOU THINK KEY ELEMENTS SET UP
FOR SUCCESS.
>> I LOVER THE OUCH ACRONYM.
I HAVE BEEN SAYING THIS QUITE A
WHILE BUT THE MORE YOU CAN ALIGN
YOUR TRIAL, THESE TRIALS ARE SO
HARD I SURVIVED COVID BUT ALMOST
DIDN'T SURVIVE (INAUDIBLE)
YESTERDAY.
HAVING SOME UMF AT THE COMMUNITY
LEVEL AND POLICY LEVEL, IT IS
IMPORTANT FOR THE TRIAL AND
MOREALE OF THE TEAM BUT ALSO A
REASON TO DO THE TRIAL.
I WAS UNAWARE OF THE OUCH GROUP.
THAT IS FANTASTIC BECAUSE THAT
GETS YOU THROUGH -- IF YOU CAN
TAKE AN ISSUE REALLY IMPORTANT
AND YOU HAVE A POLICY GROUP OR
COMMUNITY GROUP WILLING TO TAKE
ACTION ON THE RESULTS, AND YOU
LINE THAT UP TO BEGIN WITH, THAT
CAN KEEP THE VIGOR AND THE LIFE
FORCE GOING.
TO WHAT IS REALLY A PAST PREDICT
FUTURE THESE ARE HARD TO DO.
>> OTHER ADVICE TO THOSE WHO ARE
GOING TO PURSUE THE SAME PATH
YOU HAVE?
>> I FEEL I SAID THINGS I WOULD
RECOMMEND BUT DOUG'S COMMENT
THINGS I HEARD HIGHLIGHT THIS
QUESTION IS ARE SITES PARTNERS
OR SCOTI SUBJECTS?
WHEN WE TREAT THEM LIKE SUBJECTS
WHEN THEY ARE SUBJECTS
DROPPENING TRYING TO DO
SOMETHING BECAUSE THEY HAVE WHAT
WE NEED, WE STRUGGLE.
IF THEY ARE TRUE PARTNERS, I
LOVE THIS IDEA THE SITES
GENERATING IDEAS RESEARCH ON, IF
THEY ARE TRUE PARTNERS THEN THEY
ARE TRUE PARTNERS, THEN IT WILL
BE DIFFERENT, THEY ARE MUCH MORE
INVESTED.
WHAT THAT MEANS IS USUALLY
HAVING LONGITUDINAL
RELATIONSHIPS OVER VERY LONG
PERIOD OF TIME.
WITH PLACES, TO THE EXTENT ONE
DEVELOP (INAUDIBLE).
>> GAIL OR MICHAEL.
>> I REALLY LIKE WHAT JAMES SAID
IN TERMS OF HAVING PARTNERS.
FORMING COLLABORATIVE
RELATIONSHIPS WITH YOUR SITES,
SO I THINK IT RESONATES WELL
WITH ME.
I THINK YOU REALLY -- IT HAS
BEEN HELPFUL TO HAVE SITES AS
PARTNERS AND COLLABORATORS
RATHER THAN WHERE RESEARCHERS,
YOU DON'T HAVE ANY SAY, I THINK
THE CONVERSATIONS AND THE
COLLABORATION HAS BEEN GREAT AND
REALLY HAS MADE RESEARCH BETTER.
I THINK IT ALSO HELPED WHEN
COVID HIT IN TERMS OF BEING ABLE
TO COME UP WITH SOLUTIONS
LOCALLY.
>> I WOULD ADD YOU WANT TO
CONTRIBUTE TO SOMETHING EVEN IF
ONE YOUR CONTROL SITES OR
WHATEVER.
TO KEEP EVERYBODY ON BOARD TO BE
PASSIONATE ABOUT WHAT YOU ARE
DOING, WHATEVER THE RESULTS ARE
MOVING FORWARD SOME OF THAT WE
ARE LUCKY EMBED THAT EPIC IS
GOING TO EMBED THIS IN THEIR NEW
ITERATION SO IT IS GIVEN TO
EVERY SIGHT NO MATTER WHAT THEY
WERE CONTROL SITE OR NOT.
EVEN IF NOT THE CASE TO FIGURE
OUT WHAT YOU CAN DO TO MAKE
THINGS BETTER FOR EACH SITES
THAT ARE WORKING WITH YOU AND
THESE COLLABORATORS.
IT IS HARD TO SAY TO A GROUP
WHAT DO YOU WANT BEFORE YOU GET
THE GRANT BECAUSE I HAVE BEEN IN
THAT POSITION TOO WORKED WITH A
SYSTEM AND DIDN'T GET IT.
LET'S TRY SOMETHING ELSE, ONLY
SO MUCH YOU CAN DO TO GET THAT
FUNDED.
SO IT IS A CATCH 22 BUT IF YOU
HAVE THE STUDY OF COURSE IT IS
GREAT TO GO OUT AND SEE WHAT
MOTIVATES THEM, WHAT ARE THE
REAL ISSUES AND HOW HELP THEM IN
ANY WAY POSSIBLE DURING THE
PROCESS.
>> GOOD ADVISE.
ANY PANELISTS OR QUESTIONS YOU
HAVE FOR ONE ANOTHER OR ITEMS,
WE HAVE A FEW MORE MINUTES
BEFORE I WRAP UP WITH RESOURCES
FOR THE AUDIENCE.
GREAT DISCUSSION.
I THINK WE TALKED ABOUT
RESOURCES, WE TALKED SOME ABOUT
CHALLENGES.
THIS IDEA OF REMINDS ME THE
QUOTE WE BRING UP A LOT FROM
GREG SIMON OF THE TAIL WAGGING
THE DOG AND THIS IDEA OF
RESEARCH IS REALLY THE TAIL AND
DOG IS THE HEALTHCARE SYSTEM AND
GOT TO REMEMBER WHO IS IN CHARGE
KIND OF THING.
HOW WE BUILD PARTNERSHIPS AND
TIME IT TAKES TO BUILD
PARTNERSHIPS AND COME UP WITH
QUESTIONS IMPORTANT ABOUT AND
DOUG, YOU DIDN'T TALK TODAY OR
YESTERDAY ABOUT RELATIONSHIP YOU
HAVE AND REALLY PUTTING THIS
TOGETHER WITH ACCREDITATION
AGENCY AND YOUR PLANNING
PROCESS.
THAT SOMETHING THAT IS UNIQUE,
CAN YOU TELL US A LITTLE BIT
ABOUT SORT OF HOW YOU REALLY
HAVE FROM THE TIME OF DOING THE
TRIAL YOU HAVE THIS PLAN HOW IT
WILL BE IMPLEMENTED TO THE VERY
END.
>> YOU SAID AT THE INTRO I TELL
GOOD STORIES SO I STARTED OUT
WITH STORY, I BELIEVE I WON THE
COLLABORATORY OPRAH WINFREY
AWARD SINGING THE PRAISE OF THE
TSOS TRIAL O I MANAGED NOT THE
TALK ABOUT THAT.
BUT I WILL PUT ON MY OPRAH HAD.
IT HAS TO DO WITH PLANNING
PHASE, THE REASON WE ARE
INTERESTED IN THAT
REPRESENTATIVE POPULATION OF
SITES IS BECAUSE IF WE KNEW
ONGOING DIALOGUE WITH AMERICAN
COLLEGE OF SURGEONS' COMMITTEE
ON TRAUMA WHICH HAS CAPACITY TO
REQUIRE ALCOHOL SCREENING AND
INTERVENTION, AFTER OUR PAST
TRIAL THEY REQUIRED ALL LEVEL 1
AND 2 TRAUMA CENTERS TO SCREEN
INTERVENE FOR ALCOHOL SO WE WERE
LOOKING FOR SOMETHING SIMILAR
FROM RESULTS OF TSOS FOR PTSD
FROM THE COLLEGE.
WITH THAT SURGERY PUBLICATION
THERE IS AN EDITORIAL BY LEADING
TRAUMA SURGEON, LETS'S JUST DO
THIS, LET'S IMPLEMENT
COLLABORATIVE CARE.
BECAUSE OF THE RESULTS OF THE
TRIAL, HELD BACK, WE DIDN'T
RECOMMEND INTERVENTION, WE
THOUGHT SCREENING AND REFERRAL
WOULD BE APPROPRIATE AND NOW
THERE IS THIS YEAR COLLEGE
REQUIREMENT NATIONWIDE THAT
LEVEL 1 AND 2 TRAUMA CENTERS
SCREEN AND REFER FOR PTSD.
DATA FROM OUR TRIAL ACTUALLY WE
SAW THAT -- WE SAW -- WE TALKED
A BUNCH OF SITES THAT DIDN'T DO
A GOOD JOB WITH THE
INTERVENTION.
THAT WORRIED US IF THERE COULD
BE REQUIREMENT WITHOUT NATIONAL
QUALITY RUBRIC WE HAVE SITES
DOING THIS BUT NOT DOING IT
WELL.
SO THE TRIAL DID INFORM POLICY
AND WHILE WE LOVE TO HAVE SEEN A
NATIONWIDE REQUIREMENT FOR PTSD
INTERVENTION WE DIDN'T FEEL THE
DATA WAS THERE.
IT HAS BEEN REALLY INTERESTING
PROCESS AND IT IS A LOT OF
WEIGHT ON A TRIAL ESPECIALLY IF
YOU MAKE AS GAIL SAID COMMUNITY
HOSPITALS THAT AREN'T UNIVERSITY
SITES DO SOMETHING LIKE THIS SO
THAT IS WHERE YOU WANT TO BE
CERTAIN YOU KICK THE TIRES AND
WENT TO SITES THAT HAVEN'T DONE
RESEARCH BUT HAVE TO ROLE THIS
OUT AS PART OF COLLEGE MANDATE
OR REQUIREMENT.
I WILL TAKE OFF THE OPRAH HAT.
I WAS WITH THE STEERING
COMMITTEE LIKE TO KNOW WHAT SOME
OF THE OTHER AWARDS WERE, AND --
I WILL STOP.
>> WE WILL GET THE LIST FOR YOU
DOUG.
GAIL DID YOU WANT TO COMMENT?
>> I WAS GOING TO SAY WE -- SO
REAL PROBLEM WITH UNFUNDED
MANDATES.
TO COMMUNITY SMALLER HOSPITALS
AND EVEN LARGER HOSPITALS THAT
CAN'T WIN OVER LEADERSHIP ENTETE
SO THAT IS A REAL PROBLEM.
AGAIN WHEN DOING THIS, DOUG WHEN
YOU HAVE DONE PHENOMENAL WORK IT
NEEDS TO GO TO CMS AND CMS HAS
TO TAKE IT OVER, ONE THING THAT
IS REQUIRED AND THEN GETS DONE.
WITHOUT THAT, IT IS AN UNFUNDED
MANDATE AND IT DOESN'T WORK.
JUST LIKE REFERRAL DOESN'T WORK
IN MOST PLACES NO ONE TO SEND
THEM TO.
BUT IF CMS GET IN LINE THEN
THINGS WORK.
>> GAIL, AS YOU KNOW, TRAUMA
SURGERY WORK BEFORE WENT TO SEE
GALVANIZING TRAUMA SURGEON, GOT
THE CPT CODES FOR ALCOHOL
SCREENING AND BRIEF INTERVENTION
AS PARTS OF THE LAST TRIAL.
IT IS A LOT -- THERE IS A LOT OF
WORK ON THAT END AS WELL.
>> LOT TO DO.
LOT TO DO.
>> IT IS AMAZING THESE TRIALS
CAN INFORM THOSE KINDS OF POLICY
DECISIONS, AND INFORM THOSE
PAYER DECISIONS, ABOUT WHEN DOES
IT MAKE SENSE TO HAVE A
SCREENING PIECE VERSUS AN
INTERVENTION PIECE AND THE
NUANCES, REALLY EXCITING AND I
THINK CERTAINLY AS A -- WHO
TALKS TO INVESTIGATORS I WANT TO
DO A PRAGMATIC TRIAL, WHAT IS
YOUR RESEARCH QUESTION?
AND FIGURE OUT IF PRAGMATIC
TRIAL IS THE BEST WAY TO DO
THAT.
AS YOU HAVE SHARED, THESE ARE
TOUGH STUDIES TO DO, THEY ANSWER
DIFFERENT QUESTIONS AND REALLY
IMPORTANT QUESTIONS BUT THEY ARE
NOT NECESSARILY THE EASIEST
KINDS OF STUDIES TO DO.
WITH THAT I WANT TO THANK OUR
AMAZING INVESTIGATORS FOR
SHARING ALL OF YOUR CHALLENGES
AND HOW YOU OVERCAME THEM DURING
PLANNING PHASE.
I WILL GO MARY JO BACK A COUPLE
OF SLIDES, ON THOSE RESOURCES
SLIDES, I DID WANT TO SHARE WITH
EVERYONE SOME INFORMATION THAT
IS AVAILABLE ON THE PRAGMATIC
TRIALS COLLABORATORY WEBSITE.
ONE IS THIS LIVING TEXTBOOK
WHICH IS WHERE YOU WILL END UP
WHEN YOU CLICK OR DO THE SEARCH
FOR THIS PARTICULAR SITE.
THAT HAS A WEALTH OF INFORMATION
AND RESOURCES THAT HAVE BEEN PUT
TOGETHER BY THE COORDINATING
CENTER AND MANY OF THE LESSONS
LEARNED ACROSS THE DIFFERENT
TRIALS.
THERE IS A WHOLE SERIES OF
CHAPTERS HERE AROUND DESIGN,
AROUND DATA TOOLS AND CONDUCT OF
PRAGMATIC TRIALS AS WELL AS
DISSEMINEIGH AND YOU LEARN A LOT
OF INFORMATION.
IF YOU GO TO THE NEXT SLIDE I
THINK I PULLED UP HERE IN THE
TOOL SECTION YOU CAN FIND SERIES
OF DIFFERENT QUICK START GUIDES
WHICH AS YOU THINK ABOUT
DEVELOPING OR PUTTING TOGETHER A
TRIAL DONE IN THE A HEALTHCARE
SYSTEM REALLY PROVIDES STEP BY
STEP RECOMMENDATIONS FOR WHAT
CHAPTER TO START WITH THAT HAS
INFORMATION ABOUT WHAT HER FIRST
STEP MIGHT BE, BUILDING
PARTNERSHIPS WITH HEALTHCARE
SYSTEMS OR SELECTING THE RIGHT
STUDY DESIGN LIKE WHAT WE TALKED
ABOUT YESTERDAY AS WELL AS SORT
OF IMPORTANT THINGS YOU NEED TO
KNOW.
THESE ARE JUST ALL REALLY
WONDERFUL RESOURCES THAT YOU CAN
TAKE A LOOK AT AND IF YOU GO TO
THE NEXT SLIDE THIS IS WHAT WILL
COME UP AS YOU CLICK ON THE
QUICK START GUIDE FOR
INVESTIGATORS.
LEARNING WHAT MAKES THESE
PRAGMATIC TRIALS DIFFERENT,
BUILDING THE PARTNERSHIPS TO
ENSURE SUCCESSFUL TRIAL,
PLANNING FOR SUSTAINABILITY FROM
THE VERY BEGINNING, SUCH THAT IF
THE INTERVENTION WORKS HOW DO
YOU MAKE SURE THAT IT CAN BE
MAINTAINED IN THAT HEALTHCARE
SYSTEM.
ALL THESE DIFFERENT THINGS
IMPORTANT TO KNOW AND ALL THE
LINKS WILL TAKE YOU TO DIFFERENT
RESOURCES WITHIN THE WEBSITE
HERE AND INFORMATION AND USEFUL
TOOLS AND RESOURCES.
NEXT SLIDE.
ANOTHER HELPFUL QUICK START
GUIDE THERE IS A READINESS
CHECKLIST, THE THINGS YOU WANT
TO DO.
SET OF MILESTONES YOU MIGHT WANT
TO THINK ABOUT ACCOMPLISHING
DURING YOUR PLANNING PHASE IN
THE SLIDE YOU SAW FROM EACH OF
OUR PRESENTERS, THEY TALKED
ABOUT THE DIFFERENT MILESTONES
THEY WANTED TO ACCOMPLISH DURING
THEIR PLANNING PHASE.
WE HAVE BROKEN THESE DOWN INTO
RECRUITMENT ISSUE, HOW YOU ARE
GOING TO DO THAT, HAVE YOU
TACKLED ALL THE ETHICAL AND
REGULATORY ASPECT AROUND CONSENT
OR WAIVERS OR THAT TYPE OF
THING.
IS INTERVENTION READY TO GO, DO
YOU HAVE TRAINING MATERIALS
READY TO GO, ARE YOU READY TO
ROLL OUT ACROSS YOUR SITE AND
THEN I THINK WHAT YOU HEARD FROM
OUR INVESTIGATORS TODAY IS THIS
DATA COLLECTION ELEMENT AND
BEING ABLE TO GET THE DATA YOU
NEED FROM THE RECORDS, AND AS
WELL AS BEING ABLE TO MAKE SURE
THERE IS GOOD QUALITY OF THAT
DATA AND I THINK WE HEARD SOME
GOOD INFORMATION AND TIPS ABOUT
THE BUDGET NOT FORGETTING THE
IMPORTANCE OF THE IT EXPERTISE
THAT YOU ARE GOING TO NEED AND
MAKING SURE THAT YOU BUDGET FOR
THAT ACROSS YOUR DIFFERENT
SITES.
NEXT SLIDE.
WE ARE READY FOR OUR BREAK.
SO WE ARE GOING TO START BACK AT
3 O'CLOCK.
WITH OUR NEXT SESSION WHICH IS
HOW DO WE IMPLEMENT
INTERVENTIONS AND COMPLEX
HEALTHCARE SYSTEMS AND OUR
MODERATOR WILL BE DR. STEVEN
GEORGE.
THANK YOU AGAIN TO ALL OF THE
GREAT INVESTIGATORS WHO JOINED
ME ON THE FIRST PANEL.
WE WILL SEE YOU ALL BACK AT 3
O'CLOCK.
ENCOURAGE EVERYBODY ON THE ZOOM
TO GO AHEAD AND TURN OFF YOUR
CAMERAS AND SEE YOU BACK AT 3
O'CLOCK.
THANK YOU SO MUCH EVERYONE.
WE ARE GOING TO GET STARTED I
THINK RIGHT ON TIME.
WITH THE FOURTH AND FINAL PANEL.
AS YOU CAN SEE THIS ONE IS
CENTERED ON CRITICAL QUESTIONS
RELATED TO HOW TO IMPLEMENT
INTERVENTIONS IN COMPLEX
HEALTHCARE SYSTEMS.
A VERY BRIEF INTRODUCTION, OF
THE PANEL.
MY NAME IS STEVEN GEORGE, I'M
SERVING AS MODERATOR, I'M AT
DUKE UNIVERSITY AND THEN OUR
ESTEEMED PANEL INCLUDES GLORIA
CORONADO AS YOU CAN SEE FROM
KEISER PERMANENTE, MIRIAM
EZENWA, UNIVERSITY OF FLORIDA.
MARGARET KUKLINSKI UNIVERSITY OF
WASHINGTON AND MIGUEL VASQUEZ
UNIVERSITY OF TEXAS SOUTH
WESTERN MEDICAL CENTER.
SO WE ARE VERY MUCH LOOKING
FORWARD TO DISCUSSING THIS WITH
YOU.
THE PATH WE TOOK WAS TO HAVE A
VERY BRIEF INTRODUCTION WHICH
I'M GOING TO DO.
THAT INTRODUCTION WILL ALSO
PANELISTS EXPLAIN A LITTLE BIT
ABOUT EACH TRIAL.
JUST ENOUGH TO PROVIDE CONTEXT
AND IF THERE ARE QUESTIONS OR
MORE DETAILS AROUND A PARTICULAR
QUESTION THEY WILL ADD TO THAT.
THEN WE HAVE FIVE KEY QUESTIONS.
IT DID SEEM THERE WAS ACTIVITY
IN THE CHAT OR THIRD PANEL THE
FIRST PANEL FOR TODAY.
SO WE WILL SEE IF WE CAN PLUCK A
FEW QUESTIONS FROM THE CHAT AND
CERTAINLY ENCOURAGE PEOPLE TO DO
THAT.
GO TO THE NEXT SLIDE.
THIS IS WELL KNOWN, THIS IS A
VISUAL FROM THE LIVING TEXTBOOK
WENDY MENTIONED THAT EMBEDDED
TRIALS CLINICAL CARE RESEARCH.
THERE'S MANY WAYS THEY DO THAT.
OUR SESSION IS REALLY GOING TO
FOCUS ON THAT REDDISH BROWN
COLOR CIRCLE THERE ABOUT
INTERVENTIONS THAT ARE
INCORPORATED IN ROUTINE CLINICAL
WORK FLOW.
THAT DOESN'T HAPPEN IN
ISOLATION, THAT WILL BE THE
EMPHASIS OF OUR SESSION.
NEXT SLIDE.
UNDERESTIMATING THE COMPLEXITY
OF DOING THAT IS A COMMON
CHALLENGE IN EMBEDDED TRIALS, WE
ARE HOPEFULLY GOING TO EXAMINE
HOW THIS SETOFF TRIALISTS
THOUGHT ABOUT THIS DURING TRIAL
PLANNING AND CONDUCT AND IT
BOILS DOWN TO THINKING ABOUT THE
COMPLEXITY OF THE INTERVENTION
ITSELF AND ALSO INTERVENTION OF
THE SYSTEM OR THE COMPLEXITY OF
THE SYSTEM THAT WILL BE
DELIVERED AND THEN OF COURSE THE
INTERSECTION OF THOSE TWO.
WE HAVE BEEN CHARGED TO
HIGHLIGHT IMPLEMENTATION SCIENCE
STRATEGIES THAT BOLSTERED
SUCCESS FOR THE TRIAL.
SO THIS PANEL WILL FOCUS VERY
MUCH IMPLEMENTATION SCIENCE
STORIES.
FIRST I THOUGHT WE WOULD TAKE A
BRIEF TANGENT, TO THE NEXT
SLIDE.
ONE OF THE REASONS COMPLEXITY
MAYBE UNDERESTIMATED IS THAT IT
IS A HARD THING TO MEASURE, WE
TALK ABOUT IT A LOT.
IT IS ACKNOWLEDGED.
BUT IT IS ARD HARD THING TO
MEASURE.
SO ONE OF THE THINGS NIH
PRAGMATIC TRIALS COLLABORATORY
IS WORKING ON, I HAVE BEEN LUCKY
ENOUGH TO BE INVOLVED IS A TOOL
TO ADDRESS THIS ISSUE.
SO I THOUGHT WITH THE NEXT SLIDE
I WILL SHOW SOME OF THE
COMPONENTS.
THIS TOOL HAS GONE THROUGH
DERIVATIONS AND ITERATIONS AND
TALKING WITH TRIALISTS IN THE
COLLABORATORY, AND REALLY HAVE
BOILED IT DOWN TO SOME OF THE
KEY COMPONENTS WE THINK MAY
DRIVE COMPLEXITY.
THIS IS MORE COMPLEXITY OF
INTERVENTION.
I'M SHOWING THESE BECAUSE I
THINK YOU WILL HEAR THESE POP UP
IN DIFFERENT DISCUSSION AND
DIFFERENT AREAS BECAUSE A LOT OF
THEM DO HAVE A DIRECT LINKAGE TO
THE IMPLEMENTATION STRATEGY THAT
IS GOING TO USE SO YOU CAN SEE
HOW DISRUPTIVE AN INTERVENTION
IS TO EXISTING WORK FLOW OR
TASKS.
CERTAINLY CONTRIBUTES TO THE
COMPLEXITY, THE TRAINING BURDEN,
THE SHEER VOLUME OF CLINICS AN
HEALTH SYSTEMS BECAUSE OF THE
VARIABILITY WE KNOW THAT EXISTS
ACROSS EACH OF THOSE.
HOW DEPENDENT THE SETTING OR
SPECIFIC THE SETTING IS.
FOR THE INTERVENTION BOTH
QUALITY AND QUANTITY OF THE
INTERVENTION COMPONENTS NUMBER
OF STEPS BETWEEN THAT
INTERVENTION AND THE OUTCOME
BEING MEASURED.
SO STAY TUNED THIS TOOL IS
GETTING CLOSE TO THE FACT WHERE
WE ARE TRYING TO DISSEMINATE IT
BUT I THOUGHT LOOKING AT THE
COMPONENTS WE WOULD PRIME THE
AUDIENCE FOR THEMES BACK UP.
SO THANK YOU FOR THAT SOMEWHAT
SHAMELESS PLUG THERE FOR TOOL.
FROM THE COLLABORATORY AND NOW
WE WILL GET BACK TO OUR
REGULARLY SCHEDULED PROGRAMMING.
WITH THAT, LET'S GO TO THE NEXT
SLIDE.
NOW WE HAVE THE TRIALISTS HERE
BETTER TO HEAR FROM THEM ON
THEIR VERY OWN TRIALS SO
MARGARET, YOU WANT TO KICK IT
OFF.
>> SURE.
THANK YOU, STEVE.
I'M MARGARET KUKLINSKI, ONE PI
ALONG WITH STACY STERLING ON THE
GUIDING GOOD CHOICES FOR HEALTH
ORGGC 4H STUDY.
OURS IS A CLUSTER RANDOMIZED
TRIAL INVOLVING ADOLESCENTS IN
PRIMARY CARE AND THEIR FAMILIES.
IN THE TRIAL WE ARE FOCUSED ON
DELIVERING GUIDING GOOD CHOICES
WHICH IS A PREVENTION AND HEALTH
PROMOTION PROGRAM FOR PARENTS
YOUNGER ADOLESCENTS.
THE GOAL IS TO PREVENT PRIMARY
OUTCOME TO REDUCE SUBSTANCE USE
INDICATED BY ALCOHOL CANNABIS
eCIGARETTE AND CIGARETTE USE,
FOLLOWING AD LESSENS OVER COUPLE
OF YEARS BECAUSE PREVENTION
OUTCOMES TEND TO TAKE TIME TO
REALIZE IN THIS POPULATION.
T BECAUSE INITIATION RATES
INCREASE DRAMATICALLY.
WITH AGE.
SO THAT IS OUR TRIAL IN A
NUTSHELL.
>> THANK YOU.
AS I MENTIONED, IF THERE IS
INTEREST IN MORE DETAILS
ESPECIALLY WHEN WE GETS TO
IMPLEMENTATION FEEL FEE TO PING
ME AND I WILL TRY TO GET THAT OR
ASK DIRECTLY.
SO THIS WON'T BE THE LAST TIME
YOU HEAR ABOUT THE TRIALS.
MIRIAM, YOU ARE UP.
>> OUR TRIAL IS HYBRID 1
EFFECTIVE IMPLEMENTATION TRIAL
FOR RELAXATION AND ACUPUNCTURE
IN CHRONIC PAIN AND PATIENTS
WITH SICKLE CELL DISEASE.
PRIMARY OUTCOME CONTROL ONE TO
ONE TO ONE (INAUDIBLE) VISUAL
CARE AND INTERVENTION ABOUT FIVE
TO SIX WEEKS AND THEN PATIENTS
ARE RERANDOMIZED SO IF THEY ARE
NOT -- ACUPUNCTURE OR ADAPTATION
AND IN THAT INTERVENTION FOR SIX
WEEKS AGAIN THEN THEY ARE
FOLLOWED -- FOLLOW-UP AT SIX, 12
WEEKS AND 24 WEEKS TO SEE IF
INFORMATION IS (INAUDIBLE)
REDUCING CHRONIC PAIN AN SICKLE
CELL DISEASE.
SO THAT OUR TRIAL.
>> THANK YOU, MIRIAM.
GO TO THE NEXT SLIDE.
GLORIA.
>> I WILL STEP IN.
SO I CO-LED WITH DR. GREEN FROM
KEISER PERMANENTE WASHINGTON.
OURS WAS A CLUSTER RANDOMIZED
TRIAL THAT INCLUDED 26 FEDERALLY
QUALIFIED HEALTH CENTERS AND WE
IDENTIFIED OVER 40,000 PATIENTS
WHO WERE DUE FOR COLORECTAL
CANCER SCREENING, OUR
INTERVENTION WAS A MALE OUTREACH
PROGRAM BUT PROVIDED REAL TIME
ELECTRONIC HEALTH RECORD TOOLS
AND TRAINING TO DELIVER THE
PROGRAM IN OUR PRIMARY OUTCOME
WAS FIT COMPLETION WITHIN 12
MONTHS PATIENT IDENTIFICATION.
>> GREAT.
MIGUEL HAS THE LAST TRIAL THAT
WILL BE FEATURED YOU ARE UP.
ON MUTE MIGUEL.
SORRY.
>> I'M MIGUEL VASQUEZ, SPEAKING
ON BEHALF OF (INAUDIBLE) WITH MY
CO-INVESTIGATOR.
WE ARE A GROUP IMPROVING CHRONIC
DISEASE MANAGEMENT WITH
PRAGMATIC TRIALS TO ADDRESS
MULTIPLE CHRONIC CONDITIONS.
IN OUR CLUSTER RANDOMIZE TRIAL
WE STUDIED PATIENTS WHO HAD
CHRONIC KIDNEY DISEASE
(INAUDIBLE) WE ENROLLED 11,000
PATIENTS PLUS FOUR LARGE HEALTH
SYSTEMS IN THE MAIN INTERVEN
INTERVENTIONS WERE (INAUDIBLE)
FACILITATING THE (INAUDIBLE) FOR
INTERVENTIONS PROVEN TO BE
EFFECTIVE FOR KIDNEY DISEASE,
DIABETES AND HYPERTENSION.
THAT PRIMARY OUTCOME WAS A ONE
YEAR HOSPITALIZATION RATE, WE
ALSO LOOK AT OTHER IMPORTANT
OUTCOMES LIKE (INAUDIBLE)
CARDIOVASCULAR EVENTS.
(INAUDIBLE) TRIAL.
>> THANK YOU.
WE WILL GO ON TO THE NEXT SLIDE
AND I THINK OUR PANELISTS ARE
GOING TO REMAIN ON SO WE DID
MEET BEFORE THE SESSION AND
IDENTIFIED SOME QUESTIONS THAT
WE THOUGHT WOULD SHOWCASE THE
THEMES OF INTEREST SO WE WILL
DIVE INTO THOSE AND HAVE
DISCUSSION AROUND THAT.
THE FIRST QUESTION IS IN
PLANNING YOUR TRIAL, THINK ABOUT
INTERVENTION AN HEALTH SYSTEM
COMPLEXITY AND HOW DID THIS
CHANGE DURING YOUR TRIAL
CONDUCT.
WE WILL GO TO MARGARET FIRST FOR
THIS QUESTION.
>> THANK YOU.
WE REALLY DID THINK ABOUT THIS
QUESTION A LOT AND WE THOUGHT
ABOUT IT A LOT THROUGHOUT OUR
TRIAL.
AND DEPENDING ON POINT IN THE
TRIAL WE WERE AT.
JUST AS REMINDER INTERVENTION
WAS COMPLEX, IT IS A SIX SESSION
PARENTING PROGRAM DELIVERED AS A
RESULT OF COVID VIRTUALLY BUT
OVER SIX WEEKS.
SO RESOURCE INTENSIVE.
SO COMBINE THAT WITH THE
COMPLEXITY THAT OUR HEALTHCARE
SYSTEMS AGAIN PARTICULARLY
DURING COVID, THERE IS A LOT TO
NAVIGATE.
IN RESPONSE TO THIS QUESTION IS
BRIEFLY HIGHLIGHT THREE OF THE
WAYS IN WHICH WE ADDRESS THIS
COMPLEXITY THAT I THINK MIGHT
YIELD GENERALIZABLE LESSONS OR
POINTS FOR OTHERS.
CONSIDERING YOU ARE IN THE
MIDDLE OF PRAGMATIC TRIALS.
WE KNEW FROM THE BEGINNING THAT
IT WAS GOING TO BE ESSENTIAL TO
WORK WITH RESEARCH TEAMS THAT
HAD EXPERIENCE THING EMBEDDED
RESEARCH IN HEALTHCARE SYSTEMS.
WE ARE WORKING WITH KEISER
PERMANENTE COLORADO, NORTHERN
CALIFORNIA AND HENRY FORD HEALTH
SYSTEM IN THIS TRIAL AND EACH OF
THOSE SITES HAS EMBEDDED
RESEARCH TEAMS WITH
PROFESSIONALS IN A VARIETY OF
ROLES FROM PIs TO CLINICAL
STAFF TO INFORMATICISTS TO IRB
EXPERTS.
AS A RESULT OF HAVING LOTS OF
EXPERIENCE DOING MANY TRIALS
OVER MANY YEARS, THEY KNOW HOW
TO WORK WITH SYSTEMS, THEY KNOW
WHAT WILL FLY AND WHAT WON'T.
THEY HAVE GOT THAN
RELATIONSHIPS, THE EXPERIENCE,
THE RIGHT MIND SET.
THAT ACTUALLY MADE OUR ENTRY
INTO THIS SYSTEM AND OUR WORK
WITH CLINICS RELATIVERY STRAIGHT
FORWARD AT THE BEGINNING AND SO
I JUST THINK THAT WHAT THEY KNEW
ABOUT HEALTHCARE SYSTEMS AND HOW
TO DO TRIALS IS CRITICAL TO
SUCCESS OF ABILITY TO GET OFF
THE GROUND.
SECOND THING TO HOY LIGHT IS HOW
WE NAVIGATE THAT COMPLEXITY IS A
MONITOR REALLY WHEN WE WERE
COMING OUR PROTOCOLS, WAS THAT
WE HAD TO DO THINGS IN A WAY
THAT FIT WITHIN NORMAL BUSINESS
OPERATION AND REALLY BUSY WORLD
DAY TO DAY PEDIATRICS PRACTICE.
SO IT IS THAT RUST COLORED
CIRCLE THAT STEVE HIGHLIGHTED
THAT WAS A CENTER OF OUR
THINKING BOTH AT THE PLANNING
PHASE AND THROUGHOUT THE
PROJECT.
COUPLE OF EXAMPLES OF THE WAY
THIS CAME UP WAS AGAIN SIX
SESSION INTERVENTION, WE WERE
UNDER NO ILLUSIONS THAT THE
DOCTORS WERE GOING TO DELIVER
THE INTERVENTION BUT THEIR ROLE
IN RECOMMENDING PARENTS TO AWE
PROGRAM WE FELT WAS CRITICAL.
WE HAD SOME EVIDENCE FROM PRIOR
WORK THAT PARENTS CARE ABOUT
WHAT DOCTORS HAVE TO SAY AND
THAT DOCTORS RECOMMENDATIONS TO
PROGRAM LIKE OURS COULD BE
PERSUASIVE.
WE NEED TO ASK DOCTORS TO FILL
THIS ROLE IN A WAY THAT WAS LOW
BURDEN.
SO WE GAVE SCRIPTS TO MAKE THAT
RECOMMENDATION, GAVE A NUMBER OF
DIFFERENT SCRIPS TO FIT
DIFFERENT STYLES.
AND LET THEM USE THEM TO
WHATEVER DEGREE THEY WANT TO, WE
DIDN'T TRY TO OVERCONTROL.
BUT I HOPE THEY WOULD MAKE THAT
RECOMMENDATION.
WE ALSO OFFER SOME PRESCRIPTION
PADS QUOTE UNQUOTE THEY CAN USE
TO -- GIVE TO PARENTS WITH
INFORMATION ABOUT OUR
INTERVENTION AND WAYS TO CONTACT
US.
AND WE PROVIDED THEM WITH
REMINDERS TO THE HR WHEN
ELIGIBLE PATIENTS WERE GOING TO
COME IN AND THEY COULD ACTUALLY
BE IN A POSITION TO MAKE
REFERRAL TO THOSE FAMILIES.
SO WE WERE -- I THINK THAT WAS
PRETTY SUCCESSFUL AND ABLE TO
KEEP BURDEN ON PEDIATRICIANS LOW
AND THE OTHER THING WE DID IN
OUR STUDY WAS TO HAVE THE
INTERVENTION ROLE THOUGH CARRIED
OUT BY MEMBER OF THESE EMBEDDED
TEAMS DIFFERENT MEMBERS THAT
ROLE MIMICKED THE ROLE OF HEALTH
EDUCATOR, REAL ROLE WITHIN
SYSTEMS SO THINKING THROUGHOUT
THE TRIAL HOW TO STRUCTURE WHAT
WE ARE DOING IN WAY THAT COULD
BE SUSTAINABLE.
THE LAST POINTLY MAKE BRIEFLY,
WE TRY TO ADOPT A MIND SET OF
PATIENTS WITH THE UNEXPECTED.
COVID OF COURSE I THINK WAS THE
ULTIMATE UNEXPECTED FOR MANY OF
US BUT OTHER THINGS CAME UP
ALONG THE WAY IN OUR TRIAL LIKE
PEDIATRICS BEING REORGANIZED IN
ONE OUR SYSTEMS WHICH THREATENED
RANDOMIZATION.
THIS HAPPENED EARLY AND WE CAN
WAIT SO WHICH DEALT WITH THAT IN
THAT WAY.
I WOULD ALSO SAY BE PREPARED FOR
UNEXPECTED AND TRY TO JUST DEAL
WITH AS PATIENTLY AND WELL AS
YOU CAN BECAUSE IT IS GOING TO
HAPPEN DURING THE COURSE OF OUR
TRIAL.
>> MIGUEL YOU WANTED TO JUMP ON
THIS ONE.
>> FOR US IT WAS HAVING
INTERVENTION THAT HAVE SOME
DEGREE OF COMPLEXITY.
WE WERE EXPRESSING THREE
DIFFERENCE CONDITIONS THAT
CO-EXISTED IN PATIENT AND
COMPLEXITIES ON THE HEALTH
SYSTEMS FIRST INTERVENTION WAS
MULTI-LEVEL MULTI-COMPONENT AND
REPEATED OVER TIME.
BUT SOMETHING TO SUSTAIN DURING
THE ITERATION OF THE STUDY AND
PUT THAT INTO THE CONTEXT OF
FOUR DIFFERENT HEALTH SYSTEMS TO
REMIND EVERYONE WE HAD A ONE FOR
EXAMPLE LEVEL OF INTEGRATION, WE
HAD QUITE DIVERSITY ON THE
HEALTH SYSTEMS, WE HAD PUBLIC
SAFETY NET HOSPITAL VERY
INTEGRATED.
SAME AS VA HOSPITAL INTEGRATED.
WE HAD LESS INTEGRATION LEVEL OF
NETWORKING IN EXTHE TEAKS.
THEY HAD LOOSE INTEGRATION WITH
ACL CONNECTOR SO DIFFERENT
LEVELS OF INTEGRATION WHICH
BROUGHT DIFFERENCES IN TERMS OF
HOW TO PLAN IS A FOR
IMPLEMENTATION.
THE PATIENTS SERVED OR DIFFERENT
FROM UNDERSERVED POPULATIONS TO
VETERANS TO PATIENTS IN NORTH
TEXAS BY PRIVATE SYSTEM VERSUS
-- THAT IS VERY DIFFERENT EHRs
THAT I MENTIONED YESTERDAY
INCLUDE EPIC OR SCRIPS, CBRS, SO
IT REQUIRES TAKING ALL THESE
THINGS INTO CONSIDERATION.
WE HAD TO PUT INTERVENTIONS
INCLUDING MANAGING DIABETES AND
HYPERTENSION ON THIS.
,WE HAVE TO BE FLEXIBLE BECAUSE
INITIAL PLANS VERY WELL DEFINE
THERAPY PLANS WERE NOT INITIALLY
WELL RECEIVED BY SOME OF THE
PROVIDERS SO WE HAD VERY EARLY
STAGE PLANNING PHASE REVIEWED
EARLIER TO MAKE ADJUSTMENTS TO
ADDRESS.
ACADEMIC AND NON-ACADEMIC
SETTINGS AND THIS WAS ALSO A
REFERENCE IN THE PRIOR SESSION
DOUG AND WENDY TALK ABOUT
DIFFERENCES.
AND IT WAS EASIER TO ACADEMIC
SETTINGS BUT REAL WORLD ALSO
INVOLVES NON-ACADEMIC SETTINGS
SO WE THOUGHT IT WAS IMPORTANT
AS FAR AS EXPERIENCE IN RESEARCH
MARGARET ALLUDED TO HOW HELPFUL
TO HAVE PARTNERS WHO KNOW HOW TO
DO RESEARCH IN OUR CASE YES
COLLABORATOR AT DONE RESEARCH.
WE HAVE TO SAY THAT PRAGMATIC
RESEARCH IS NOT COMMONLY A
FAMILIAR TO COLLABORATORS SO IT
WAS A UPLIFT IN AREA IN TERMS OF
HOW TO SET THESE COLLABORATIONS
WITH OUR HEALTH SYSTEMS HOWEVER
RECEPTIVE AND WILLING TO HELP US
WITH THIS.
AS FAR AS SETTING THIS UP THEN
TO IMPLEMENT WE HAD TO ADDRESS
HEALTH SYSTEMS DIFFERENT.
ENT LEARN COMPLEXITY, MORE
INTEGRATED HEALTH SYSTEM,
STARTED WITH CONVERSATIONS WITH
LEADERS AND IT WAS SOMETHING
THAT PERMEATED DOWN TO THE
INDIVIDUAL PRACTICES.
MOSTLY AGGRAVATED RECORD
INTEGRATED HELP SYSTEMS, IT WAS
REALLY VISITING EACH INDIVIDUAL
PRACTICE IN WORKING WITH ONE OR
TWO INDIVIDUALS TO BUY IN TO A
WHAT WE WERE DOING TOGETHER.
AS FAR AS HOW TO PLAN TO DELIVER
THIS IMPLEMENT ALSO HAD TO BE
FLEXIBLE.
EVERYONE REBS FROM THE PRESSES
THAT FLEXIBILITY ON THE DELIVERY
OF INTERVENTION.
THAT SOMETHING WE WERE AWARE
FROM THE BEGINNING.
AND LOT OF EMPHASIS IN THIS.
FOR EXAMPLE IN TWO HEALTH
SYSTEMS WE USE A PHARMACIST TO
HELP US DELIVER INTERVENTION.
THE OTHER TWO HEALTH SYSTEMS IT
WAS DONE BY POPULATION NURSES.
HEALTH SYSTEMS LOOKED INTO THIS
IN DIFFERENT -- AND PUT HIGHER
VALUE ONE WAY OR ANOTHER THEY
FELT MORE COMFORTABLE WITHIN
THEIR SYSTEM.
OBVIOUSLY THIS INTRODUCES
HETEROGENEITY BUT WE HOPE THAT
THAT WILL ALLOW TO BE MORE
GENERALIZABLE.
FOR VARIOUS HEALTH SYSTEMS.
WHEN WE GET DOWN TO INDIVIDUAL
INTERVENTION, WE WERE ALSO
FLEXIBLE FOR EXAMPLE, SPARSE
EDUCATION -- AS FAR AS EDUCATION
SOME HEALTH SYSTEMS WANT TO USE
BROCHURES.
AND OTHER HEALTH SYSTEM WANTED
TO USE VIDEOS.
OTHER INFORMATIONAL.
SO AGAIN WE HAVE VARIETY, THERE
WAS SOMETHING THAT INITIALLY
BROUGHT ADDITIONAL WORK TO
SETTLE THIS ACROSS FOUR
DIFFERENT HEALTH SYSTEMS BUT AT
THE END WE BELIEVE IT ALLOWS US
TO HAVE A WIDER PERSPECTIVE OF
WHAT WORKS AND WHAT IS BEST IN
ONE SYSTEM OR ANOTHER AND BE
FLEXIBLE AND READY TO ADAPT.
THANK YOU.
>> THANK YOU, QUICK FOLLOW-UP
BUT BEFORE I ASK MARGARET OR
MIGUEL OR ANYONE ELSE ON THE
PANEL BUT WONDER IF GLORIA OR
MIRIAM WANTED TO ADD ANYTHING
THAT POPPED INTO THEIR MIND
LISTENING TO THESE RESPONSES.
>> THIS IS GLORIA.
I CAN ADD A LITTLE EMPHASIS ON
SOMETHING THAT MIGUEL SAID WHICH
WAS THAT HIS SYSTEMS WEREN'T
THAT FAMILIAR WITH PRAGMATIC
RESEARCH.
IN OUR CASE STEPS YOU SEE WE
DEFINITELY ENCOUNTERED THAT WITH
OUR (INAUDIBLE) AS WELL.
SOMETIMES WHEN WE PITCH THE IDEA
TO CLINIC THEY SAID OKAY WELL,
WHEN IS THE STUDY GOING TO BE
DONE?
WE SAID WELL, WE ARE HOPING YOU
MIGHT INCORPORATE THIS AS PART
OF USUAL CARE.
REALLY THE CONCEPT OF LIKE DOING
IMPROVEMENT SUSTAINING THAT
IMPROVEMENT OVER TIME WAS
UNFAMILIAR TO THEM THE OTHER
CONCEPT, EVEN DURING THE TRIAL
SOME CLINICS FELT LIKE OKAY
THOSE CLINICS ARE RESEARCH
CLINICS SO WE ARE NOT GOING TO
TOUCH THEM.
SO IF WE WANT TO DO ANY
IMPROVEMENT WE ARE GOING TO DO
THE IMPROVEMENT IN THE USUAL
CARE CLINICS.
SO I THINK THAT IN SOME CASES,
THERE WAS SOME CONCERNS THAT WE
HAS ABOUT CONFOUNDING SO FORTH
BECAUSE OF THEIR PERCEPTION THAT
DON'T TOUCH CLINICS THAT ARE
PART OF THE RESEARCH WHEN IN
FACT WE WANTED THEM TO ENGAGE IN
WORK AND THAT THEY HAD IDEAS FOR
HOW TO FURTHER ADAPT THE PROGRAM
OR MAKE IMPROVEMENTS WE WANTED
THEM TO WORK WITH US TO DO THAT.
IT WAS REALLY INTERESTING JUST
TO KIND OF TRY TO OVERCOME MORE
TRADITIONAL IDEA THEY HAD ABOUT
RESEARCH AND WHAT OUR ROLE WAS.
THERE WERE STRATEGIES WE USE
THAT ARE HELPFUL AND I MIGHT
TALK ABOUT THESE LATER BUT WE
DID HAVE CLINICS DO PLAN STUDY
CYCLES SO THIS WAS A COMMON
QUALITY IMPROVEMENT STRATEGY
THEY WERE FAMILIAR WITH, SO IT
WAS EASY TO DO AND THAT HELPED
ENGAGE THEM IN THE WORK.
I WILL STOPPLE THERE.
FAMILIARITY WITH PRAGMATIC
RESEARCH, IT IS INTERESTING
ESPECIALLY AT TRIALS WHETHER
THAT IS SOMETIME GOING TO BRING
CONTEMPT OR GOING TO BE
SOMETHING THAT IS HELPFUL
BECAUSE I THINK THAT PRESSURE TO
CONTINUE IT OR AT LEAST INTEREST
IN CONTINUING IT SOMETIMES
PLACES ARE HAPPY TO DO RESEARCH
AND BE DONE WHEREAS THIS IS A
LITTLE DIFFERENT PATH.
BEFORE WE MOVE ON I DID WANT TO
MAYBE THROW THIS TO MARGARET
BECAUSE I APPRECIATED THE THREE
BIG THEMES.
IS THERE A THEME YOU CAN LINK TO
CHANGING HOW YOUR THOUGHTS
CHANGE?
OR DO YOU THINK IT IS KEY TO BE
FLEXIBLE OR DO YOU THINK THERE
IS WAYS THAT GENERALIZATION
ABOUT HOW THAT THINKING HAS TO
CHANGE FROM PRESET NOTIONS TO
HOW IT ENDS UP OR IS IT JUST
THAT F WORD WE HAVE BEEN HEARING
FLEXIBILITY?
>> I THINK IT IS THAT F WORD.
I THINK THAT IS BOTH AROUND WHAT
IS UNEXPECTED WHAT WE KNOW WE
WILL LEARN AND LEARN WHAT WORKS
BETTER AROUND INTERVENTION
DELIVERY AND OTHER ASPECTS OVER
TIME.
SO I THINK IT IS THE F WORD AND
N WORD, THE NIMBLE WORD THAT TO
MY MIND FEEL MOST IMPORTANT.
>> I APPRECIATE THAT WILLINGNESS
TO LEARN BECAUSE THAT
DIFFERENTIATES THE DESIGN WHERE
WE -- WHEREAS YOU DON'T COME IN
THIS IS THE WORK BOOK AND MANUAL
AND SEE YOU LATER.
IT REALLY IS, THERE HAS TO BE
THAT WILLINGNESS TO WORK WITH
THE LOCAL TEAM.
AND LEARN FROM THEM AND REALIZE
THEY WILL PROBABLY ANTICIPATE
SOME THINGS BETTER THAN WE COULD
HAVE WHEN WE HAD IT ON THE WHITE
BOARD.
THANK YOU, SORRY TO PUT YOU ON
THE SPOT.
MARY JO, WE WILL GO TO NEXT
SLIDE.
THIS QUESTION IS HOW TO
INCORPORATE WHAT IS KNOWN ABOUT
IMPLEMENTATION SCIENCE
STRATEGIES IN YOUR TRIAL AND
GLORIA WE WILL GO BACK.
>> SO I ALREADY TALKED ABOUT THE
FACT THAT WE USE PLAN STUDY
CYCLES WHICH WERE REALLY HELPFUL
FOR COUPLE OF REASONS.
ONE IS THAT REALLY HELPED OUR
CLINIC BECOME ENGAGE MISDEMEANOR
THE WORK AND NOT TO THINK ABOUT
US DOING RESEARCH ON THEIR
CLINICS BUT AS PARTNERS IN
REALLY IMPLEMENTING THE PROGRAM.
SECONDLY, IT HELPED US
UNDERSTAND WHAT THE
IMPLEMENTATION BARRIERS WERE.
SO GOING IN IN TO THE PLAN STUDY
ADD CYCLES IN COLLABORATION WITH
OUR PRIMARY CARE ORGANIZATIONS
LOCALLY TO RETRAIN CLINICS HOW
TO DO PSAs AND SET BENCHMARKS
WHEN THEY NEED TO SUBMIT
DOCUMENTS TO US.
OUR EXPECTATION WAS THAT EACH OF
THESE CLINICS WOULD TEST OUT A
REMINDER TO THE PROGRAM.
OUR STANDARD INTERVENTION
INCLUDED AN INTERLETTER MAIL A
REMINDER BUT THOUGHT THEY MIGHT
TEST OUT TELEPHONE CALLS OR
AUTOMATED APPROACHES AND TO OUR
SURPRISE, ONLY A FEW CLINICS
DECIDED TO DO THAT.
BUT SOME OTHER CLINICS SAID WE
WILL WORK ON WORK FLOW OR WE ARE
GOING TO FIGURE OUT THIS ISSUE
WE ARE HAVING WITH OUR LAB AND
THE FACT PATIENTS AREN'T PUTTING
DATES ON THE KITS WHEN THEY
COLLECT THEM SO WE -- DONE HAVE
COLLECTION DATES, SO WE
IDENTIFIED CHALLENGES WE MAY NOT
HAVE KNOWN EXISTED HAD WE NOT
DONE THE PSA CYCLES SO THAT WAS
ONE EXAMPLE OF REALLY MERGING
OUR PROGRAM INTO A STANDARD
QUALITY IMPROVEMENT FRAMEWORK,
CLINICS WERE INTERESTED IN.
IN ADDITION TO THAT, WE WERE
REALLY INTERESTED IN ASSESSING
DIFFERENT CHARACTERISTICS AT THE
LEVEL OF OUR CLINICS.
THE ORGANIZATIONAL LEVEL
CHARACTERISTICS.
THE INFLUENCE THAT THAT HAD ON
OUR IMPLEMENTATION SUCCESS.
WE BEGAN COLLABORATION WITH
EDWARD MIC FROM THE VA IN
INDIANA WHO AGREED TO SERVE AS
CONSULTANT TO OUR STUDY TO DO
SOME CONFIGURATIONAL COMPARATIVE
METHODS TO LOOK AT THE
ORGANIZATION, ROLE OF THE
ORGANIZATION LEVEL
CHARACTERISTICS IN THE SUCCESS
OF IMPLEMENTATION OF OUR
PROGRAM.
SO THAT IS ANOTHER EXAMPLE HOW
WE INCORPORATED THOSE PRINCIPLES
THROUGH OUR STUDY.
>> THANK YOU.
MIRIAM.
>> (INAUDIBLE) EFFECTIVENESS
PATIENT TRIAL FOCUSING MORE ON
THE EFFECTIVENESS LOOKING AT
ACUPUNCTURE TESTING USING SICKLE
CELL DISEASE PAIN, WHILE AT THE
SAME TIME OBSERVING UNDOCUMENTED
SOME OF THE BARRIERS, AS WELL AS
SOLUTIONS, TO THESE BARRIERS SO
HAVING SAID THAT WE ARE USING
(INAUDIBLE) IMPLEMENTATION
RESEARCH THAT WE DO UNDERSTAND
THEY IMPLEMENT TRIAL WITH FOCUS,
SO (INDISCERNIBLE) LEVEL THAT WE
ALSO FOCUSED ON POINT BLANK --
WE ARE FOCUSING ON POLICY LEVEL
AS WELL AS (INAUDIBLE) LEVEL SO
TOGETHER TO IMPLEMENT OUR STUDY
SUCCESSFULLY EFFECTIVENESS, OUR
TEAM INCLUDES A
MULTIDISCIPLINARY TEAM THAT
INCLUDES CLINICIANS AN LOW POLL
GISTS, NURSING, MEDICINE, AS
WELL AS PEOPLE WITH EXPERTISE IN
BOARD SO HELP UNDERSTAND
PERSPECTIVE ON (INAUDIBLE) IN
THIS POPULATION.
SO HAVING SAID THAT, WE ALSO
ENGAGED MULTIPLE STAKEHOLDERS I
MENTIONED EARLIER, WE USE
SEMISTRUCTURED INTERVIEW TO
(INAUDIBLE) ABOUT SOME OF THESE
FACILITATORS BARRIERS AND AS
WELL AS SOLUTIONS TO THESE
PROBLEMS.
THE OTHER THING THAT WE HAVE
BEEN ALSO TO -- WE ARE WE ARE
DEVELOPING LOGO, WE JUST DIDN'T
WANT TO CHOOSE ANY LOGO WE
WANTED SO WE CONVERTED WITH
PATIENTS DOING INTERVIEW AND
PROVIDED WHAT YOU THINK ABOUT
THIS LEVEL, VERY CRITICAL SICKLE
CELL DISEASE OR (INAUDIBLE) LENS
SO THAT CULTURALLY SENSITIVE AS
WELL AS HAVE SOME RED BLOOD
CELLS TO SHOW WHAT IS HAPPENING
IN THE RED BLOOD CELL OF THESE
PATIENT WITH SICKLE CELL DISEASE
SO THAT WAS VERY, VERY GOOD FOR
OUR TEAM BECAUSE WE WERE ABLE TO
ENGAGE ABOUT THAT LEVEL AND ABLE
TO GET CRITICAL INFORMATION THAT
WAS LET US MOVE FORWARD.
WE ALSO HAVE PATIENTS SOME
QUESTIONS ABOUT INTERVENTIONS
AND SOME OF THE ISSUES THAT CAME
UP WAS (INAUDIBLE) PATIENT LIKE
GET INTO (INAUDIBLE) ALL THE
TIME BLOOD DRAWN, MONTHLY
SOMETIMES MORE SO THEY HAVE A
REAL VERSION (INAUDIBLE) WHAT
ARE WE GOING TO DO TO HELP YOU
UNDERSTANDNA WE POSSIBLE --
LEVEL OF BLOOD DRAWN SO SOME OF
THE ADVICE THEY GIVE US TO
CORRECT -- TO HELP SHOW WHAT
ACUPUNCTURE IS LIKE IN THE
MIDDLE, YOU CAN USE -- THAT WE
USE FOR ACUPUNCTURE TO HELP THEM
TRULY UNDERSTAND AND THEY ARE
GOING TO PARTICIPATE IN THE
TRIAL.
THEN THE INFORMATION WE GATHERED
WE DID A BLUEPRINT
IMPLEMENTATION BLUEPRINT THAT
GUIDED THE PHASE AS WELL AS NOW,
ONE BEAUTY ABOUT THE BLUEPRINT,
IT IS A BLUEPRINT, THAT IS WE
ARE ABLE TO REVISE IT AS STUDIES
ONGOING, WHAT WE LEARN IT IS ONE
WHAT WE ARE LEARNING NOW IN THE
TRIAL PART OF THE STUDY.
SO THESE ARE SOME OF THE WAYS
ABLE TO INCORPORATE WHAT IS
KNOWN ABOUT IMPLEMENTATION
SCIENCE IN CLINICAL TRIALS.
T SO I'M GOING TO STOP AND ALLOW
OTHERS TO CHIME IN AND ANSWER
QUESTIONS THAT COME UP WITH THIS
TRIAL.
>> THANKS.
WE HAVE MARGARET IS NEXT ON THE
SIGN UP LIST.
GO AHEAD MARGARET.
THANK YOU MIRIAM.
>> SURE IT IS SOME OVERLAP WHAT
I'M GOING TO SAY AND POINTS
GLORIA AND MIRIAM MADE, I WOULD
SAY IMPLEMENTATION SCIENCE AND
RELATED STRATEGIES WERE REALLY
IMPORTANT IN OUR TRIAL, I THINK
WE ARE VERY FORTUNATE TO HAVE
TWO LEADERS INCLUDING CO-PI
STACY STERLING INCORPORATED
SCIENCE.
WE INCORPORATED EXPLICITLY, I
WANT TO HIGH LIE WAYS WE
INCORPORATED IT EXPLICITLY.
WE USE THE PRISM AND FRAME
NETWORKS TO GUIDE MEASUREMENT OF
CONSTRUCTS IN OUR TRIAL SO IT
WAS A HUGE ANCHOR FOR STUDY
DESIGN AND MEASUREMENT.
WITH AD LESS SENT CLINIC LEADER
AND DOCTORS AND HEALTH TRAINING
WITH STAFF TO TRY TO GET
EVERYBODY THAT MIGHT BE AFFECTED
IN OUR CLINICS BY THE
INTERVENTION ON BOARD FAMILIAR
WITH WHAT IS GOING ON.
SO THAT WAS IMPORTANT AT THE
BEGINNING THEN DURING THE TRIAL
WHEN COVID HIT WE ENGAGED
PARENTS WE WANTED TO KNOW IF OUR
GROUPS WOULD ESSENTIALLY BE IN
DEMAND, GIVEN THE STREAMS THAT
THE PANDEMIC MEANT FOR --
STRAINS PANDEMIC MEANT FOR
FAMILIES OF AD LESSENS AND KIDS
OF -- AD LESS SEASONS AND ALL
AGE KIDS REALLY.
SO WE WANTED TO SEE OFFER AS A
WAY OF CONNECTING WITH OTHER
PARENTS AND ENGAGE AROUND
PARTICULAR ISSUES IN
ADOLESCENTS, MIGHT BE SOMETHING
THEY WANT TO DO EVEN DURING THIS
DIFFICULT TIME IN THE LIFE OF A
PANDEMIC.
WHILE THE PARENTS DIFFERED WE
HAVE THE SENSE THEY DID WANT US
TO OFFER THE PROGRAM AND WE
WANTED TO DO IT AND THAT ENGA
ENGAGEMENT IN PARENTS IS REALLY
IMPORTANT TO US GOING FORWARD
AND HOW WE THOUGHT ABOUT THE
INTERVENTION AND WHAT IT MIGHT
DO TO SUPPORT PARENTS.
WE ALSO MADE SURE TO MAINTAIN
CONNECTIONS WITH PEDIATRICS
PEDIATRICIANS AND THE CLINICS IN
WHICH WE WERE WORKING.
TO TELL THEM HOW THE TRAVEL IS
GOING TO SHARE QUOTE FROM
PARENTS, TO DO OTHER THINGS TO
SUPPORT THEIR ONGOING ENGAGEMENT
AND SUPPORT FOR WHAT WE WERE
DOING.
THINK THOSE ARE REALLY IMPORTANT
AND THE FINAL THING THAT I WANT
TO HIGHLIGHT AS ELIMINATION
SCIENCE STRATEGY H IS IMPORTANT
TO YIELD SUCH RICH INFORMATION
IS A QUALITATIVE INTERVIEW STUDY
THAT STACY IS LEADING THAT IS
FRAMED IN THE PRISM FRAMEWORK
PRINCIPLES.
AND THAT STUDY IS INVOLVING
STAFF AT EACH OF OUR SITES
INTERVIEW SEMISTRUCTURED
QUALITATIVE INTERVIEWS WITH
VARIOUS PEDIATRIC STAFF
LEADERSHIP AND NON-PEDIATRICS,
OTHER PARTS OF THE HEALTHCARE
SYSTEM AS WELL.
PEDIATRICS LEADERS SOMETIMES
FINANCE BUSINESS PEOPLE,
SOMETIMES PEDIATRICIANS TO
REALLY UNDERSTAND MULTIPLE
PERSPECTIVES SUPPORT FOR
PREVENTIVE INTERVENTIONS.
LIKE GGC BUT NOT LIMITED.
AND BETTER UNDERSTAND CONDITIONS
NECESSARY TO SUPPORT INVESTMENTS
IN EFFECTIVE PREVENTION APPROACH
FOR PEDIATRICS POPULATIONS.
I JUST THINK THE WORK THAT STACY
IS LEADING AND OTHERS ARE
PARTICIPATING IN, ARE GOING TO
PROVIDE YET ANOTHER SET OF
PERSPECTIVES NOW THAT WE ARE IN
A DIFFERENT POINT IN THE TRIAL
THAT WILL HELP BRIDGE
IMPLEMENTATION TO
SUSTAINABILITY.
THOSE ARE A FEW DIFFERENT WAYS
IN WHICH IMPLEMENTATION SCIENCE
CONSTRUCTS THAT ARE EMPHASIZED
FRAME WORKS, HAVE BEEN REALLY
CENTRAL IN OUR WORK ALL ALONG.
>> THANKS.
I'M A PRAGMATIC TRIALS RUBE BUT
DID HAVE SOMETHING THAT POPPED
IN MY HEAD THAT I WAS INTERESTED
IN PERSPECTIVE, SO MUCH TALK AND
TIME INVESTED IN IMPLEMENTATION
STRATEGIES ACROSS ALL THREE
TRIALS.
ONE WAS PRE-DETERMINED HYBRID
EFFECTIVENESS IMPLEMENTATION
TRIAL, CLUSTER RANDOMIZATION
TRIALS.
MOVING FORWARD YOU THINK THAT
LINE WILL BLUR OR DO YOU THINK
THERE IS SOME ADVANTAGE OF
DECLARING THAT HYBRID DESIGN?
CURIOUS BECAUSE I'M SITTING HERE
THINKING IF I DIDN'T SEE HOW YOU
DID THE TRIAL DESIGN, I WOULD
HAVE HEARD A LOT ABOUT
IMPLEMENTATION.
SO CURIOUS ABOUT WHAT YOUR
THOUGHTS ARE MOVING FORWARD, IS
THAT LINE BLURRIER OR DO YOU
THINK THERE IS SOME
DISTINCTIONTIVENESS THERE THAT
WILL CHARACTERIZE THE
DIFFERENCES IN THE TRIALS?
SORRY THAT POPPED IN MY HEAD
CURIOUS WHAT YOU THREE MIGHT
THINK ABOUT THAT.
THIS IS THE DANGER OF GONE OFF
MENU.
ANY THOUGHT?
>> I WILL JUMP IN HOPE THAT IS
OKAY WITH GLORIA AND MIRIAM AND
MIGUEL.
IMPLEMENTATION SCIENCE IN THE
WAY IN WHICH IMPLEMENTATION
CONSTRUCTS MODERATE IMPACT IS
JUST GOING TO GAIN MORE
ATTENTION, WE ARE GOING TO
REALLY CONTINUE TO HAVE AND GROW
OUR APPRECIATION FOR THE SYSTEMS
LEVEL FACTORS AND OTHER THINGS
THAT INFLUENCE WHAT WE DO.
I THINK THE FRAMEWORK LIKE MIR
GRAM IS USING I THINK ARE REALLY
IMPORTANT TO HELPING US WHAT WE
ARE TRYING TO CONTROL AND WHAT
WERE NOT.
I THINK THEY WILL CONTINUE TO BE
IMPORTANT EVEN IF WE GET MORE
PRAGMATIC PERHAPS AS WE GIVE
STRONGER CONSIDERATION TO THE
ROLE OF IMPLEMENTATION.
I DON'T WANT THEM AS FRAMEWORK
TO GO AWAY OR LINES TO BLUR TOO
MUCH BECAUSE I THINK THEY ARE
HELPFUL.
>> THIS IS GREAT.
I WILL ADD COUPLE OF THINGS.
I THINK THE EMPHASIS OF YOUR
TRIAL NEEDS TO DEPEND UPON THE
LITERATURE AND WHAT HAS BEEN
DONE TO DATE.
SO WHETHER YOU ARE CHOOSING TYPE
1, 2 OR 3 HYBRID EFFECTIVENESS
DESIGN DEPENDS ON WHAT IS THAT
FOUNDATION OF RESEARCH TELLING
YOU BUT I WILL ADD TO THAT I
THINK THERE CONTINUES TO BE A
BIAS ON STUDY SECTIONS FOR THE
RANDOMIZED TRIAL.
SO I THINK THAT WE NEED TO
CONTINUE TO EMPHASIZE THE
IMPORTANCE OF IMPLEMENTATION AND
FIGURING OUT IMPLEMENTATION
BARRIERS MOVING THOSE MODELS
REFINE DEVELOP CONCEPTUAL MODELS
THAT ARE INFORMING OUR WORK.
>> I WOULD LIKE TO ADD IN MY --
GIVEN WHAT WE KNOW FROM GRACE
THAT TRIALS TYPE 1 BECAUSE
EFFECTIVENESS PART OF THIS STUDY
COLLECTING DATA OR DOCUMENTING
THE FACILITATORS AND BARRIERS
AND SOLUTIONS TO SOME OF THOSE
BARRIERS SO BAD, EFFECTIVENESS
OUT OF THE TRIAL IS SUCCESSFUL
INTERVENTIONS ARE EFFECTIVE IN
WHATEVER OUTCOME YOU LOOK AT.
YOU HAVE SOME PROBLEMS AHEAD OF
TIME SOME OF O THOSE BARRIERS
ABOUT VISIBLE STRATEGIES TO
IMPLEMENT SO THAT WILL HELP
INTERVENTION IN THE HEALTHCARE
SYSTEM.
SO THAT IS ONE THING THINKING
ABOUT GIVEN OUR STUDY SOME OF
THE BARRIERS WE ARE FACING AND
EFFECTIVE IMPLEMENTATION WE ARE
AHEAD OF TIME OUTCOMES OF THIS
PROBLEM, WE ARE STARTING SOME OF
THEM NOW.
THEY ARE EACH GOING TO INCLUDE
POLICY MAKERS.
HOW TO MANAGE HELPERS WITH
BIGGER ISSUES WE CAN OBSERVE SO
HAVE THAT VISION FOR THE FUTURE.
>> THANK YOU.
THANK YOU ALL THREE FOR FIELDING
THAT QUESTION FROM THE ROOM OF
THE PANEL.
LET'S MOVE ON THE NEXT ONE IS A
LITTLE BIT OF A QUICK HITTER BUT
IT IS A GOOD FOLLOW-UP TO THE
IMPORTANCE IMPLEMENTATION
SCIENCE IS EMPHASIZED.
WHERE DID YOU FIND THIS
EXPERTISE?
GLORIA, DO YOU WANT TO LEAD OFF
WITH THAT ONE?
>> I THINK THAT WE WERE LUCKY
BECAUSE MY CO-P I PARTICIPATED
IN THE TITLER PROGRAM THAT
TRAINS PEOPLE IN IMPLEMENTATION
SCIENCE SO THAT IS REALLY
HELPFUL AND I HEARD THAT
MENTIONED BY MARGARET AND OTHERS
THAT HAVING THAT EXPERTISE ON
YOUR LEADERSHIP TEAM MAKES A
DIFFERENCE.
THEN ALSO MENTION WE REACHED OUT
TO THE PRIMARY CARE ASSOCIATION
TO HELP US WITH PLAN STUDY
CYCLES TO MAKE SURE THAT WHAT WE
WERE DOING WAS REALLY CONSISTENT
WITH HOW THE CLINICS ARE BEING
TAUGHT THAT QUALITY IMPROVEMENT
STRATEGY.
THERE COULD BE LOCAL RESOURCES
THAT CAN BE ACCESSED WE DID FIND
PARTNERS TO DO MORE
SOPHISTICATED CONFIGURATIONAL
COMPARATIVE METHODS TYPE WORK
AGAIN ONCE STUDY FUNDED WESTERN
ABLE TO MAKE THOSE PARTNERSHIPS
AND COLLABORATIONS HAPPEN SO
THAT WE CAN DO SOME MORE STATE
OF THE ART CUTTING EDGE ANALY
ANALYSIS.
WE BENEFITED FROM THOSE
DIFFERENT RESOURCES.
>> QUICK HITTER BUT GIVE ANYONE
ELSE IN THE PANEL TO ADD TO THAT
ONE.
GO AHEAD MARY.
>> I USED TO BE UIC PRIMARY
INSTITUTION FOR THIS STUDY.
SO WE HAVE DEVELOPED
COLLABORATION RELATIONSHIP WITH
PROVIDERS OF OTHER RESEARCHERS
IMPLEMENTATION SCIENTIST DATA SO
CHICAGO TO FLORIDA WE ALSO
DEVELOP EXPERTISE HERE, WHEN UIC
REACHED TO US ESTABLISHED IN
FLORIDA COLLEAGUES IN CHICAGO I
THINK THE OUTCOMES THAT WE DO,
WITH THOSE WE HAVE PEOPLE WORK
WITH THEM IN THE PAST AND THEY
HAVE A -- CRITICAL FOR ALL SO WE
GET TOGETHER AS A TEAM WITH A
NAME WITH EXPERTISE
ANTHROPOLOGY, CHINESE MEDICINE,
CLINICIANS, SUCH BEAUTIFUL HOW
OUR TEAM IS CONFIGURED HOW
STIMULATION TO EVERYONE
FUNCTION, HOW OUR (INAUDIBLE)
DEVELOP --
>> I WILL ADD ONE MORE THING.
IN THE FIRST YEAR TWO STOP CRC
WE BENEFITED FROM OUR OFFICERS
BEING (INAUDIBLE) AS MANY PEOPLE
KNOW IS THE FOUNDER OF THE REAIM
FRAME WORK SO THAT ALSO REALLY
BOLLED OUR EMPHASIS ON
IMPLEMENTATION SCIENCE.
AND JUST TO PUT IN ANOTHER PLUG,
I MENTION THAT MY CO-PI ATTENDED
A TRAINING PROGRAM I AM SERVING
AS A MENTOR AND TRAINING PROGRAM
CALLED IMPLEMENTATION SCIENCE
SCHOLARS RUN OUT OF WASHINGTON
UNIVERSITY.
IT IS REALLY GREAT PROGRAM FOR
EARLY STAGE INVESTIGATORS, SO IF
ANYBODY ON YOUR TEAM IS NEEDING
TO GET TRAINED IN IMPLEMENTATION
SCIENCE I WOULD HIGHLY RECOMMEND
THAT PROGRAM.
THERE MIGHT BE OTHERS OUT THERE
THAT ARE REALLY GOOD.
>> FOR US WE WERE FORTUNATE IN
ONE OF OUR CO-INVESTIGATORS CHAD
FOX HAD ALREADY GONE QUITE A LOT
OF THIS WORK IMPLEMENTATION
INTERVENTIONS, WITHIN OUR
SETTINGS.
AND THEN A ONE OF OUR NIH
OFFICERS IN EXPERIENCE THIS WITH
INDIAN HEALTH SERVICES WHICH
SOME VERY IMPORTANT -- SAW VERY
IMPORTANT WORK AS FAR AS
IMPLEMENTING INTERVENTIONS IN
THOSE SPECIFIC POPULATIONS SO WE
WERE FORTUNATE WE WERE ABLE TO
BRING THAT EXPERTISE.
IN OUR STUDY.
>> I THINK JUST GOING TO TAKE A
ADVANTAGE OF BEING MODERATOR TO
TALK BRIEFLY REFLECT ON MY OWN
EXPERIENCE BUT I WAS INVOLVED
WITH A TRIAL WHERE WE TRAINED
PROVIDERS ON INTERVENTIONS
ACROSS FIVE DIFFERENT HEALTH
SYSTEMS AND IT WAS REALLY
FASCINATING/FRUSTRATING FOR US
AS THE TRIAL TEAMS TO WATCH THE
HEALTH SYSTEMS THAT HAVE THIS IN
THEIR CULTURE TAKE WHAT WE DID
AND RUN WITH IT IT WAS
INCREDIBLY FSTRATING TO
REALIZE WE SPENT LOT OF TIME
TRAINING AND IN SOME OF THE
OTHER SYSTEMS THAT DIDN'T HAVE
THIS CULTURE TO THAT IT WAS
GOING TO BASICALLY BEGIN AND END
WITH OUR TRAINING, IT WAS AN EYE
OPENING EXPERIENCE FROM SOMEONE
NEW TO THIS AREA.
MAYBE THE MOST IMPORTANT THING
IS MAKE SURE IF YOU DON'T HAVE
IT YOU DO FIND IT OUT OF ALL
THIS BUT SOUNDS LIKE THAT IS
JUST BECOMING MORE AND MORE,
THERE'S MORE TRAINING
OPPORTUNITIES AND CERTAINLY THE
PANEL HAS HIGHLIGHTED THAT.
SO I THINK THANK YOU FOR THE
INPUT ON THAT ONE.
WHY DON'T WE GO TO NEXT SLIDE.
MIGUEL STICKING WITH YOU TO TAKE
THE FIRST SHOT AT QUESTION FOUR.
MARY JO NEXT SLIDE.
WHAT IS VALUE OF MAPPING OUT
WORK FLOW AND HOW WERE THESE
MAPS USED DURING THE TRIAL?
>> HAPPY TO START WITH THIS ONE.
MAPPING WAS EXTREMELY IMPORTANT
FOR US, HOW DIFFERENCE OUR
HEALTH SYSTEMS WERE.
I THINK ONE THING WE HAD NOT
REALIZED PLANNING WAS IMPORTANCE
OF PLANNING TOGETHER INDIVIDUAL
OF EACH HEALTH SYSTEM, THE EARLY
SESSIONS WITH HEALTH SYSTEMS
TOGETHER, SOMEONE SAY OKAY WE
UNDERSTAND WHAT YOU WANT TO
ACCOMPLISH AND WHAT TOOLS YOU
GOING TO BE MADE AVAILABLE?
CAN WE COME UP WITH THE
DIFFERENT SWING RATES HERE TO
GET THERE.
THIS IS ONE OF THE MORE
INTEGRATED HEALTH SYSTEMS WITH
MANY RESOURCES.
WE DON'T HAVE TO MAP IN SWIM
LANES WE ONLY HAVE ONE SWIMMER
SO NO REASON TO MAP ANYTHING
HERE FOR DIFFERENT SWIMMERS.
EVERYTHING FOR ONE.
IN ONE.
THE MAPPING IS VERY IMPORTANT
BUT IT HAS TO BE DIFFERENT,
ACCORDING TO WHAT HEALTH SYSTEM
YOU HAVE.
KNOWING WHAT TO DO AND WHEN.
ALSO ALEAVE YATES CONCERNS THERE
WAS INITIALLY SOME -- I GUESS
ANXIETY, SOME OF THE HEALTH IN
THE PRACTICES, THIS IS GOING TO
BE MORE WORK, ARE WE GOING MORE
RESEARCH INSTEAD OF WORKING, AS
I SAID BEFORE WHAT WE ARE DOING
IS DELIVERING INTERVENTION
SYSTEM BUT WE WANT TO MAKE CLEAR
WE DO THE RESEARCH FOR THE
HEALTH SYSTEM IMPLEMENT THESE
PRACTICES.
HAVING THIS VERY CLEARLY DEFINED
AND CREATED THIS WORK FLOWS BY
STANDING DOWN NOT ONLY
INVESTIGATORS BUT THE PROVIDERS,
MANAGERS LEADERSHIP FROM THE
HEALTH SYSTEMS DO CLEAR WHAT
POINT EACH PERSON WAS DOING
WHAT, IN P WHAT HEALTH SYSTEM.
IT ALLOWS THEM TO PROVIDE EARLY
REPORTS TO THE HEALTH SYSTEM, WE
SAID THAT A WITH SOME OF THE
QUALITY GROUPS WITH SOME
PERFORMANCE IMPROVEMENT GROUPS
IN QA, THEY WERE SAYING THIS IS
GREAT, WORKING FOR SOME OF YOUR
SYSTEMS WE TELL THEM WE ARE
HAVING SOME CHALLENGES HERE, CAN
YOU HELP US TO ADVANCE THIS
WITHIN YOUR HEALTH SYSTEM.
FROM THAT PERSPECTIVE IT WAS
VERY HELPFUL.
WE ALSO WERE ABLE TO IDENTIFY IN
ADVANCE BEFORE WITH SOME OF THE
LEADERSHIP WHAT PRACTICES WERE
READY FOR THIS AT DIFFERENT
POINTS.
WE DID NOT ACTIVATE ALL SITES AT
THE SAME TIME, WE WERE JUST IN A
SEQUENTIAL FASHION IN THE FIRST
FEW MONTHS OF THE STUDY.
AND REALLY HELP TO ENSURE IF
SOMEBODY WAS GETTING RANDOMIZED,
THEY WOULD BE CAPABLE, THEY
CONTROL TO ACTUALLY DO WHAT WAS
NEEDED.
NOVEL PRACTICES READY AT ALL
TIMES AS FAR AS RESOURCES
WILLINGNESS AND CULTURE TO CARRY
OUT THIS TYPE OF WORK.
SO IT WAS HELPFUL TO HAVE WORK
FLOWS SO IT CAN BE IN PLACE AND
RETHE I TO GO IN INDIAN HEALTH
SYSTEM WHEN THE TIME CAME FOR
THEM.
THEN WE COULD GO BACK AS FAR AS
HOW THEY WERE USED IN THE TRIAL,
A DIFFERENT TIME DIFFERENT
HEALTH SYSTEMS DIFFERENT CLINICS
WE HAVE SOME HICCUPS THERE WAS
TURN OVER OF PERSONNEL ACROSS
ALL SITES IN HAVING CLEAR WORK
FLOWS WAS JUST A SO HELPFUL
BECAUSE NO MATTER HOW WELL
SOMEONE WAS PERFORMING THERE WAS
NO GUARANTEE THAT THAT
INDIVIDUAL OR THOSE INDIVIDUALS
ARE CHAMPIONS, GOING TO BE THERE
FOR THE DURATION OF THE STUDY.
SO BESIDES HAVING NOTES AND
ASSIGNING TO ETCH OOH OTHER
HAVING THE WORK FLOWS THERE,
HAVING SOMETHING THAT CAN PROVE
IT IS READY TO GO NO MATTER WHO
IS COMING TO BE THE NEXT
CHAMPION WAS EXTREMELY HELPFUL.
TURN OVER IS A MAJOR REALITY FOR
THESE STUDIES IN THE WORK FLOWS
ARE TO KEEP THE INTEGRITY OF
STUDY.
>> THANK YOU.
MIRIAM.
YOU WANT TO ADD ON?
>> YES, I DO.
JUST LIKE MIGUEL SAID WE FOUND
THE HEALTHCARE SYSTEMS SLIGHTLY
DIFFERENT.
DIFFERENT STRUCTURE AND CULTURE
WHEN WE DID THE MAP FOUND SOME
SIMILARITIES ONE DIFFERENCE WE
FOUND WAS THAT AT DUKE PATIENT
TRIED TO MAP OUT WHAT HAPPENS
WHEN PATIENT (INAUDIBLE) AT WHAT
POINT WE CAN VENTURE OUT AND
INCLUDE IT IN STUDY.
IN DOING THAT WE FOUND THAT THE
ONLY DIFFERENT AMONG (INAUDIBLE)
IS TIMING OF THE BLOOD DRAW SO
AT DUKE THEY COME GET THEIR
BLOOD DRAWN, BUT FOR -- BLOOD
DRAW HAS ANY TIME OF VISIT SO
KNOWING THAT FOCUS CLINICIAN,
HOW DO WE RECRUIT THE ONE IN A
LITTLE AMOUNT OF TIME, XYZ GET
TAKING CLINICIAN SEE THEM COME
OUT DO BLOOD TEST AND THEN
LEAVE.
WE WERE ADVISED TO (INAUDIBLE)
(INAUDIBLE) LOCATION BUT PATIENT
COME OUT OF CLINIC, TO HAVE
OPPORTUNITY TO ENCOUNTER US SO
WE HAVE STUDIES, SO VERY BRIGHT
COLORS SO THEY CAN STOP AND SAY
WHAT IS GOING ON CAN BE BOTH,
BECAUSE TELL ME MORE.
(INDISCERNIBLE) THINGS
CONDUCTING THEN ABOUT MAP
CLINICIANS IMPLEMENT PLAN PHASE
SO WE CAN'T REALLY BY CLINIC
STAFF (INAUDIBLE) CLINICIAN SO
THEY HAVE THE OPTION TO THEY
HAVE THE TIME OR INTRODUCE STUDY
TO THE PATIENT BY BRINGING THEM
AND ASK THEM TO (INAUDIBLE)
OPPORTUNITY WHICH IS IMPORTANT
FOR INTO THE CLINIC WHEN THEY
ARE DONE THEY HAVE OPPORTUNITY
BECAUSE WE FOUND WAS WAS THAT
WHEN PATIENT GOT DONE WITH THAT
CLINIC VISIT, THERE IS
(INAUDIBLE) BECAUSE
TRANSPORTATION SO JUST A FEW
MINUTES TO STOP AND SAY OKAY,
CAN YOU PLEASE CALL ME, SO
(INAUDIBLE) CAN YOU CALL ME
LATER AND THEY GIVE -- WHICH IS
-- HAVE THAT INTRODUCTION SO
THAT WE CAN (INAUDIBLE) DO STUDY
PROCESS SO THAT IS HOW THE
CLINIC IS REALLY HELPFUL FOR OUR
STUDY.
>> THANK YOU.
THANKS MIRIAM.
I THINK GLORIA, YOU ARE NEXT.
>> I CAN JUST EMPHASIZE WHAT
MIGUEL AND MIRIAM SAID THAT THE
WORK FLOW MAPPING WAS HELPFUL
FOR OUR STUDY AS WELL.
I THINK IN ADDITION TO WORK FLOW
MAPPING WE ALSO DID SOME DATA
MAPPING.
REASON THIS WAS HELPFUL IS
BECAUSE WE -- OUR CLINICS USED
EPIC BUT USED IT IN DIFFERENT
WAYS.
THEY HADN'T PREVIOUSLY USED
REPORTING WORKBENCH WHICH WAS
THE TOOL THAT WE WERE PLANNING
TO USE AND THAT WE DID USE FOR
OUR STUDY.
SO WE NEEDED TO MAKE SURE WHEN
WE WERE CREATING REPORTS
REPORTING WORKBENCH THEY WERE
ACCURATELY POOLING THOSE
PATIENTS THAT WERE DOING
CO-COLORECTAL CANCER SCREENING
INTO THOSE REPORTS.
SO WE MET WITH ALL THE IT SITE
SPECIALISTS, REALLY MAPPED OUT
WHERE THE DATA -- HOW IS THE
DATA BEING ENTERED WHO WAS
INTERRING IT TO MAKE SURE AND
CAPTURE THE HETEROGENEITY AND
HOW THE EHR TOOLS ARE BEING USED
TO MAKE SURE THEY WOULD SERVE
THE PURPOSE OF OUR STUDY.
IN ADDITION TO THE SWIM LANES
AND WHO IS DOING WHAT WE NEEDED
TO GET VERIFICATION THAT WHO IS
DOING WHAT FROM PERSPECTIVE OF
THE DATA TO ENSURE TOOLS WORK
PROPERLY.
>> THAT IS A GOOD POINT
ESPECIALLY WITH THE HR CONCERNS
OR QUESTIONS.
SORRY MARGARET GO AHEAD.
>> I WAS GOING TO SAY WHAT YOU
SAID, GLORIA'S COMMENT MADE ME
REALIZE THAT WE ACTUALLY DID A
LOT OF THAT DATA MAPPING AS WELL
WAS REALLY IMPORTANT TO KNOW IF
CONSTRUCTS MEASURED SIMILARLY
ACROSS SYSTEMS WHICH WERE
AVAILABLE IN THE HR AND I WANT
TO EMPHASIZE GLORIA'S POINT AND
SAY I HAVE ENCOURAGED THAT FOR
ALL THE REASONS.
>> THANK YOU.
THERE IS A QUESTION FROM THE
AUDIENCE THAT IS RELEVANT TO
THIS AND MAYBE WE CAN TOUCH ON
IT, IT MAY COME OUT IN OUR
SUMMARY THEMES TOO BUT WHILE WE
ARE HERE WITH MAPPING OUT BEFORE
WE GET TO THE NEXT QUESTION
WHICH I THIS HI IS A REALLY
IMPORTANT ONE I WANT TO LEAVE
TIME FOR IT OBVIOUSLY BUT THERE
IS A QUESTION ABOUT BALANCE OF
STAYING FLEXIBLE AND I'M
PARAPHRASING IT.
WHAT IS THE -- TALKING ABOUT
STAYING FLEXIBLE WHEN DOES IT
BECOME -- WHEN DOES
IMPLEMENTATION STRATEGY BECOME
ADAPTATION AND ACKNOWLEDGING
THAT YOU NEED TO BE FLEXIBLE
WHEN YOU NEED TO TRAP THESE
ADAPTATIONS, I BRING THIS
QUESTION UP BECAUSE WONDER IF
WORK FLOW MAPPING HELPS WITH
THAT OR NOT, IF THAT SOMETHING
MAPPED OUT BEFOREHAND DO YOU
THINK THAT HIMSELF IN PROVIDING
SOME OF THE SWIM LANES FOR WHAT
IS TOO FLEXIBLE AND ALSO
ADAPTATIONS THAT CAN BE TRACKED
SO HOPEFULLY THAT IS NOT TOO
MUCH A SQUEEZE BUT I SAW THAT
QUESTION, I THINK IT MAY COME UP
AGAIN, BUT SEEMS RELEVANT TO
THIS WORK FLOW MAPPING.
WONDER IF ANY PANELISTS HAD A
THOUGHT ON THAT.
IT IS IN THE CHAT FOR THOSE OF
YOU THAT ARE ON THE ZOOM IF YOU
WANT TO SEE HOW BAD A JOB I DID
MANY IN PARAPHRASING IT.
>> I CAN TIE IN A LITTLE BIT.
THERE ARE PATIENT POPULATION
WITH SICKLE CELL DISEASE,
COMPLICATIONS OF DISEASE IS
(INAUDIBLE) CHRONIC PAIN, THEY
HAVE RECURRENT ONGOING DISABLING
PAIN.
SO WE HAVE WORKING WITH THIS
POPULATION FOR OVER TEN YEARS,
SCHEDULE TODAY OR SCHEDULE FOR
VISIT, THEY WAKE UP IN THE
MORNING THEY HAVE SO MUCH PAIN,
THEY CANNOT COME TO THE VISIT
HAVE TO STOP, HAVE TO FINISH
THIS NOW SO WE SOMETIMES
(INAUDIBLE) SEVERAL TIMES TO
EVEN GET SCHEDULE, SOMETIME SO
(INAUDIBLE) DISEASE PROCESSES
AND DIRECTION THEY OBTAIN AND
MORE THAN FLEXIBLE AND STANDARD
THAT THIS IS THE WAY IT IS WE
ARE GOING TO BE (INAUDIBLE)
THOSE PATIENTS AND ANOTHER
EXAMPLE WE USE ACUPUNCTURE FOR
THE TREATMENT ARMS, MANY OF
THESE (INAUDIBLE) TRIAL TOO FAR
AWAY, AND THEN THE QUESTION IS
WHY GAS PRICE SO HIGH WHAT DO
YOU DO TO HELP GET THEM TO VISIT
SO YOU HAVE TO BE FLEXIBLE
THINKING OUTSIDE THE BOX,
THINKING HOW TO HELP THEM TO BE
ABLE TO DO INTERVENTION BECAUSE
WE CAN HAVE ALL THESE STUDIES
AND PATIENT HAVE THOSE STUDIES.
SO THAT IS MY RESPONSE TO THAT
QUESTION, FLEXIBILITY CONDUCT
PATIENTS SO JUST TRYING TO
ACCOMMODATE PATIENTS BECAUSE
PRAGMATIC TRIAL REAL WORLD
SITUATION FOR THIS PATIENT WHICH
IS PAIN MEASURE EACH DELIVERY AS
PART OF THAT DISEASE
>> THANKS MIRIAM.
OTHER THOUGHTS ON THE VALUE OF
THE MAP FOR FLEXIBILITY VERSUS
ADAPTATIONS AND IF AND WHEN,
LIKE I SAID THIS MAY COME UP
AGAIN IN THE GRAND SUMMARY BUT
ANY THOUGHTS ON THAT NOW?
>> I CAN ADD A FEW THINGS.
THIS IS GLORIA.
IN OUR TRIAL WE MADE SURE THAT
WE WERE REALLY CLEAR ON WHAT THE
CORE COMPONENTS WERE AND WHAT
THINGS COULD BE ADAPTED, IN OUR
CASE THE CORE COMPONENT WAS A
MALE (INAUDIBLE) ANYTHING ELSE
CLINICS DID WHETHER THEY SENT
REMINDERS OR DID TELEPHONE CALLS
WHATEVER, THAT WAS CONSIDERED
ADAPTATION.
IN SUBSEQUENT WORK WE HAVE CODED
ADAPTATIONS ACCORDING TO
(INAUDIBLE) INSTRUMENTS
FRAMEWORK SO WE LOOK AT THEM
ACROSS DIFFERENT DOMAINS AND I
THINK IT HAS BEEN REALLY USEFUL
EXERCISE, WE HAVE USED PERIODIC
REFLECTIONS TO REALLY CAPTURE
WHAT ARE THEY ADAPTATIONS
HAPPENING OVER TIME.
I THINK THERE ARE CHALLENGES TO
DOING THAT WHEN CLINIC STAFF
TURN OVER AS MIGUEL MENTIONED,
CLINICAL TRIALS TRIAL TOO
SOMETIMES IT IS CAPTURING THE
ADAPTATIONS THAT ARE OCCURRING
IF YOU DON'T KNOW PEOPLE MIGHT
BE LEAVING THEIR POSITION, BUT I
FEEL LIKE WE GOT A LOT OF GOOD
INFORMATION FROM USING PERIODIC
REFLECTIONS AND PUTTING THEM
ACCORDING TO SHANNON'S
FRAMEWORK.
>> I WILL ADD WE TOO PAID
ATTENTION TO FIDELITY AND CORE
COMPONENTS VERSUS THE NEED TO
CONSIDER WHAT WE WERE HEARING
FROM PARENTS AND CONSIDER HOW TO
ADAPT.
THE WAY WE WORK THAT OUT IS
THROUGH THE FACT WE HAD REGULAR
MEETINGS OF INTERVENTION TEAM
MASTER TRAINER WHO KNOWS THOSE
CORE COMET POINTS OF OUR
INTERVENTION EXTREMELY WELL.
IT WAS IN THE DIALOGUE THAT
OCCURRED IN THOSE MEETINGS THAT
WE FIGURED OUT I THINK HOW TO
FIND THE RIGHT BALANCE OF SAYING
TRUE TO THOSE CORE COMPONENTS
WHILE MAKING SURE WE WERE
RESPONSIVE TO THE OTHER THINGS
PARENTS WERE BRINGING THEIR WAY
BECAUSE IF YOU ARE NOT
RESPONSIVE TO PARENTS YOUR
UPTAKE WILL SUFFER AND YOU WON'T
GET THE IMPACT YOU HOPE FOR.
SO I THINK IT IS ALL A DELICATE
BALANCE, IT IS I GUESS THAT WAS
OUR VERSION OF REFLECTION IN A
WAY.
THAT ALLOWED US TO COME TO A
POINT WE WERE COMFORTABLE WITH.
AND WE ALSO HAD A MEMBER OF OUR
TEAM TRACKING, DEPACKTURES WE
PLANNED SO I DON'T KNOW IF I
CALL THIS ADAPTATIONS NOT
ADAPTATIONS OF THE INTERVENTION
BUT JUST THINGS THAT DIDN'T GO
THE WAY WE INITIALLY PLANNED.
I THINK THAT THAT IS GOING TO BE
HELPFUL AS WE EXAMINE WHAT WE
ARE FINDENING OUR OUTCOMES DAY
AND TRY TO INTERPRET IT.
WHAT WE EXPECTED TO FIND.
>> I CAN'T GET TRIALS FOR US
ADAPTING FLEXIBILITIES IN THE
AREAS WAS KEY TO SURVIVAL
CHANGES NOT ONLY LEVEL OF HEALTH
SYSTEMS BUT PRACTICES.
I MENTION THE EVIDENCE
GUIDELINES INTERVENTION, THOSE
CHANGED FOR SOME OF OUR MAIN
CONDITIONS FOR -- SO WHAT WE FOR
CHANGES IN TERMS OF WHAT WAS
DONE IN HOW OR NOT IN TERMS OF
THE GOAL OR THE OVERALL PLAN OF
WHAT WAS A DESIRED INTERVENTION
BUT HOW WAS ALWAYS SOME
CONVERSATION ABOUT HOW IT COULD
BE DONE WITH SOME DIFFERENCES.
>> GREAT.
FANTASTIC.
RESPONSE, THANKS FOR DIGGING
DEEP AND RESPONDING TO THAT.
LET'S MOVE ON TO QUESTION FIVE
HOW CAN PRAGMATIC TRIALS BE
CONDUCTED TO REDUCE HEALTH
DISPARITIES.
MIRIAM YOU OKAY FIRST ON THIS
ONE?
YOUR TURN TO LEAD OFF.
>>
>> OKAY.
FINE WITH ME.
THINKING ABOUT THIS QUESTION
WHAT CAME TO MY MIND WAS THAT
WHAT PATIENTS ARE, RATHER THAN
WHERE HEALTHCARE SYSTEM IS FOR
GIVE BUT WE DO KNOW HOW
(INAUDIBLE) TO THE PATIENT AND
TRANSPORTATION WHICH IS KNOWN AS
SOCIAL DETERMINANTS OF HEALTH,
IMPACTED (INAUDIBLE).
SO WE TO OUT WE WOULD SOLVE THIS
PROBLEM, GETTING TO THIS
PROBLEM, AND FOR ACUPUNCTURE TO
COME TO ICV FOR FIVE WEEKS.
MANY PATIENTS SOME OF THEM SAY
(INAUDIBLE) I DON'T HAVE THE
TRANSPORTATION, I DON'T HAVE
(INAUDIBLE) TO BE ABLE TO COME.
SO WE HAVE TO THINK ABOUT OUR
STUDY ADDRESSING HEALTH
DISPARITIES WE HAVE TO ACCESS
TRANSPORTATION.
FOR GENERAL PART OF THE STUDY WE
ARE IN ISSUE OF PATIENT MIGHT BE
-- RUN OUT OF THE PLAN TO BE
ABLE THE GO TO OUR STUDY OF FIVE
TO SIX WEEKS, STUDIES WITH SMART
DEVICE OR CELL PHONE IS ONLY
ACCEPTABLE DEVICE, MIGHT BE TIME
FOR THEM (INAUDIBLE) STUDY.
SO IN THINKING ABOUT THIS, --
SIMILAR INFORMATION SO THE
PROBLEM RANDOMIZE PATIENT
ACUPUNCTURE, TREATMENT ARM WHO
HAVE ACCESS TRANSPORTATION
INFORMATION TO HAVE CELL PHONE
DATA OR QUALITY DATA OR
(INAUDIBLE) HAVE WE (INAUDIBLE)
HEALTH DISPARITIES, BECAUSE WHO
MIGHT HAVE BENEFITED FROM
INTERVENTION SO THIS (INAUDIBLE)
SOME TRIALS JOINT HEALTH
DISPARITIES ARE MAGNIFIED.
THINKING ABOUT HOW TO USE THIS
PROBLEM, OUR TEAM TALKING ABOUT
THESE PATIENTS DEVELOP
ADDITIONAL FOR THOSE IN THE
ACUPUNCTURE TREATMENT ARM TO BE
ABLE TO COME BUT LARGE
(INAUDIBLE) COME TWICE A WEEK.
THE PROBLEM IS CONFOUNDED BY THE
HIGH GAS PRICE.
MY FINAL THOUGHT (INAUDIBLE) AS
WELL AS SEE IF YOU CAN HAVE
AQUEUE PUNCTURE IN THE COMMUNITY
AS OPPOSED TO HAVING VISUAL
CENTERS HAVE MORE ACUPUNCTURE IN
THE COMMUNITY WE HAVE TO LOOK AT
SCHEDULING BECAUSE THIS PATIENT
SOME OF THE WORK THEY HAVE CHILD
CARE ISSUES, SO HOW DO WE THEN
TAKE CARE OF THESE SOCIAL
DETERMINANTS OF HEALTH TO THEN
USE INFORMATION.
FOR THOSE (INAUDIBLE) THINKING
ABOUT DEVICE TO THEM AS WELL AS
PROVIDE SOME INCENTIVE TO HELP
THEM BE ABLE TO (INAUDIBLE) --
THIS PROBLEM WE HAVE SELECT
GROUP IN OUR -- PRAGMATIC TRIAL.
>> THANK YOU.
MIGUEL, DO YOU WANT TO GO NEXT?
>> I THINK THIS IS A REALLY
IMPORTANT QUESTION AT THIS TIME.
HOW CAN THIS TRIAL BE CONDUCTED
TO HEALTH DISPARITIES AND I
THINK THAT WE HAVE TO GO TO THE
VERY BEGINNING ABOUT CHOOSING
YOUR PARTNERS.
THAT AS INVESTIGATORS WE HAVE
THE ABILITY TO CHOOSE PARTNERS,
HOPEFULLY THE PARTNERS WANT TO
WORK WITH US.
VERY EARLY IN THE PROCESS.
I THINK WE HAVE TO ANSWER TWO
KEY QUESTIONS.
WHERE CAN I DO THIS TRIAL.
WHERE SHOULD I DO THIS TRIAL.
FIRST ARE VERY DIFFERENT WHETHER
WE SHOULD BE DOING THE TRIAL,
SOMETHING THAT HASOR HAS TO
CHANGE, WE NEED TO FIND NEW
PARTNER OR PARTNERS TO DO THIS
STUDY OR THE QUESTION MAYBE
DIFFERENCE.
IN OUR CASE WE WANTED TO STUDY
SOME CONDITIONS THAT HAVE
DISPROPORTIONATE BURDENS IN SOME
POPULATIONS.
I HAVE OVERREPRESENTED IN SOME
OF OUR SAFETY NET HOSPITALS SO
WE IMMEDIATELY HAVE SYNERGY OF
INTEREST HERE WITH HEALTH SYSTEM
TAKING CARE OF OUR INDIGENT
PATIENTS IN DALLAS COUNTY.
IMMEDIATELY QUESTION ABOUT HOW
PROOF OF CARE FOR THESE CHRONIC
CONDITIONS AND MAY BE MORE
LIKELY TO GET THEM BINDING FROM
LEADERSHIP AND FROM PROVIDERS IN
THE CLINICAL STEP.
ABOUT HOW TO MAKE IT HAPPEN.
FROM THE PERSPECTIVE OF
PATIENTS, IT IS ALSO MUCH BETTER
FIT WHEN THE HEALTH SYSTEM TAKES
CARE OF THESE PATIENTS IN THE
HEALTH SYSTEM IS WITH US
INVESTIGATORS, BECAUSE THEN THE
PATIENT SEE THAT THIS IS STUDIES
IN CARE THROUGH HEALTH SYSTEM
WHERE THEY ARE COMFORTABLE,
FAMILIAR, KNOW THE CARE SYSTEM
IS INTERESTED IN THEIR WELFARE.
THEY ARE FAMILIAR WITH THEIR
PROVIDERS AN CLINICS AND WE ARE
NOT ASKING FOR ADDITIONAL VISITS
OR THING THAT CAN BE PART OF
SOME OTHER STUDIES BUT PRAGMATIC
CLINICAL TRIALS BUT WE HAVE
GREATER OPPORTUNITY HERE IN
PRAGMATIC TRIALS TO ADDRESS
HEALTH DISPARITIES, ESPECIALLY
IF WE CAN CHOOSE SYSTEMS AND
THOSE HEMAL HEALTH SYSTEMS WANT
TO WORK WITH US.
>> THANKS, MARGARET DO YOU WANT
TO GO NEXT?
>> SURE.
HAVING A CHANCE TO TALK ABOUT
THIS QUESTION WHICH FOR ME I
SUSPECT LIKE MANY OF US IS
MOVING LARGER OVER TIME GIVEN
TIMES WE LIVE IN AND FOR
POPULATIONS WE ARE WORKING WITH
AND THE CHOICER TO HEALTH STUDY
DISPARITIES AMONG YOUNG PEOPLE
AND THEIR FAMILIES ARE
INCREASING WITH COVID.
SO MY THINKING OF THIS IS
INFLUENCED BY COUPLE OF PAPERS
AND IMPLEMENTATION SCIENCE THAT
FRAME THE ISSUE OF EQUITY AND
HEALTH DISPARITIES AND
CENTRALLY, BY RACHEL SHELL TON
AND BOWMAN PUBLISHED THE LAST
COUPLE OF YEARS.
THERE CENTRAL POINT THAT I'M
TRYING TO HANG ON TO IN MY WORK
IS IF WE WANTS TO REDUCE HEALTH
DISPARITIES IN THE OUTCOMES WE
ARE CARING ABOUT, WE NEED TO
CENTER EACH STEP OF THE WAY,
ECHOING MIGUEL'S POINT WE NEED
TO DO THAT FROM THE BEGINNING
THROUGHOUT THE EXPLORATION
PLANNING AND IMPLEMENTATION AN
SUSTAINABILITY PHASES.
THIS DOES PLAY OUT IN THE
PERTINENT SITES WE CHOOSE TO DO
INTERVENTIONS IN.
IN OUR STUDY WE BROUGHT TOGETHER
KEISER PERMANENTE COLORADO,
NORTHERN CALIFORNIA AND HENRY
FORD HEALTH DETROIT AND TOGETHER
WE HAD A DIVERSE SET OF
FAMILIES.
OUR EXPLICIT GOAL WASN'T TO
REDUCE HEALTH DISPARITIES BUT
WAS TO SHOW WE HAVE UPTAKE BY
FAMILIES FROM REGIONALLY DIVERSE
LOCATIONS AND SOCIO DEMOGRAP
DEMOGRAPHICALLY DEVOIRS
BACKGROUNDS SHOWED INTEREST IN
GOOD CHOICES.
IF YOU WANT TO REDUCE HEALTH
DISPARITIES MAYBE THAT NEEDS TO
BE A PRIMARY END POINT, IT IS
NOT JUST THE INITIATION OF
SUBSTANCE USE WE ARE AIMING FOR
EXAMPLE BUT REDUCTIONS IN
INITIATION AMONG THIS IN THAT
POPULATION MIGHT BE THE WAY WE
CENTER STUDY IF WE WANT TO
REDUCE HEALTH DISPARITIES.
I THINK THEN WE NEED TO REALLY
THINK HARD ABOUT HOW WE ARE
STAFFING OUR INTERVENTIONS.
TO BE MODIFIED MORE CERTAIN WAYS
TO BE ACCEPTABLE AND SATISFYING
TO CERTAIN POPULATIONS TO FIT
BETTER HOW DO WE KNOW WHAT TO
DO, WHO ARE WE LISTENING TO AS
WE DECIDE HOW TO TRY TO INCREASE
THE FIT AND ATTRACTIVENESS OF
THE INTERVENTION TO POPULATIONS
THAN WE REALLY WANT TO REACH.
ANOTHER THINK MIGUEL AND MARY
TALKED ABOUT IS BARRIERS TO
GETTING TOWARD INTERVENTIONS WE
NEED TO THINK ABOUT HOW TO
REDUCE THOSE BECAUSE THEY DON'T
FALL EVENLY, THEY FALL IN LINE
WITH DISPAIR IT IS QUITE OFTEN.
-- DISPAIR IT IS QUITE OFTEN.
-- DISPARITIES QUITE OFTEN.
WHEN WE THINK ABOUT SETTINGS
LIKE CMHCs, SOMETIMES BE THE
MOST WELL-POSITIONED TO REACH
THE POPULATIONS WE WANT TO WORK
WITH, THEY MIGHT ALSO BE THE
LEAST RESOURCED TO REACH
POPULATIONS SO WE NEED TO THINK
ABOUT HOW WE ARE GOING TO
PROVIDE RESOURCES TO SUPPORT
IMPLEMENTATION.
IF WE WANT TO REACH POPULATIONS
SUCH THAT DISPARITIES ARE
REDUCED.
THERE'S THINGS ON ANALYSIS SIDE
ALSO BUT WANT TO STOP THERE AND
MAKE SURE THERE'S TIME FOR
OTHERS TO SPEAK.
>> THANK YOU.
GLORIA.
>> I WILL OFFER A FEW --
>> ON THIS QUESTION.
>> I WILL OFFER A FEW ADDITIONAL
WORDS.
APPRECIATE MY COLLEAGUES TALKING
ABOUT THE IMPORTANCE OF BRINGING
SERVICES TO THE COMMUNITY AND
BEING FLEXIBLE AND OPERATING
INCENTIVES TO PARTICIPANTS AND
THINKING ABOUT THE PARTNERS AS
WELL AS STAFFING AND CONSIDERING
EQUITY ACROSS ALL THE ACTIVITIES
OF THE PROJECT.
I WOULD ECHO ALL THIS AND SAY I
ENCOURAGE TO SEE NEW
IMPLEMENTATION SCIENCE MODELS
THAT INVITE US NOT SIMPLY TO
LOOK AT DISPARITIES WITH REGARD
TO THE EFFECTIVENESS OF OUR
PROGRAM BUT ALSO IMPLEMENTATION
REACH AND OTHER COMPONENTS ALONG
THE ENTIRE CONCEPTUAL MODEL SO I
FEEL LIKE WE ARE CALLED TO THINK
ABOUT THIS HOLISTICALLY WHICH IS
ECHOING WHAT MARGARET SAID.
I SEE IN OUR TRIAL TO GIVE A
COUPLE OF EXAMPLES, IN OUR TRIAL
WE DECIDED SPECIFICALLY
INTENTIONALLY TO WORK WITH OUR
MOST DIVERSE CLINIC AS OUR PILOT
SO IN DOING SO WE GOT TO
UNDERSTAND UNIQUE ISSUES AND HOW
TO ENGAGE LATINO PATIENTS IN
PARTICULAR AND AS A RESULT OF
WORKING WITH THISICALLY MCWE
DEVELOPED WORD LIST INSTRUCTIONS
FOR COMPLETING A FIT TEST FOR
COLORECTAL CANCER SCREENING,
THESE -- THIS WAS DONE ON THE
SUGGESTION OF OUR PROJECT
OFFICER, AND IT REALLY HELPED US
BECAUSE IT IS CHALLENGING TO GET
THE RIGHT LANGUAGE INSTRUCTION
TO THE RIGHT PATIENT IF YOU ARE
INCLUDING TWO DIFFERENT WORDED
INSTRUCTIONS.
WE DEVELOPED WORD LIST
INSTRUCTIONS USED BY ALL OUR
SYSTEMS AND THEY SINCE HAVE BEEN
ADOPTED BY KEISER PERM
PERMANENTE REGIONS AND ONCE OR
TWICE A MONTH WE REACH OUT FROM
A HEALTH SYSTEM THAT WANTS TO
USE THOSE INSTRUCTIONS SO IF YOU
HAVEN'T SEEN THEM, MAILFIT.ORG
THEY AVAILABLE ON OUR SITE.
THAT IS ONE OF THE GREAT
SUCCESSES IS THAT BECAUSE WE
STARTED WITH THAT CLINIC WE
DEVELOPED TOOLS LATER USED BY
ALL OF OUR SYSTEMS THAT WERE
MORE COULDLY APPROPRIATE AND
RELEVANT.
I WOULD ALSO SAY IT IS IMPORTANT
TO THINK ABOUT IN OUR CASE THE
CONTINUUM OF CARE.
SO WE CHOSE AN INTERVENTION THAT
IN EARLY DATA SHOWED MORE
EFFECTIVE IN LATINO PATIENTS
WHICH NATURALLY HAVE LOWEST
RATES OF COLORECTAL CANCER
SCREENING, IN OUR DATA CONFIRMED
OUR INTERVENTION HAD HIGHER
EFFECTIVEN 'T IN THE LATIN
COMMUNITY BECAUSE IT WAS
ENCOURAGING TO ADDRESS THAT
INEQUITY.
IT WAS IMPORTANT FOR US TO LOOK
AT FOLLOW-UP COLONOSCOPY RATES
AND I THINK THAT AT THE END OF
THE DAY IS OUR INTERVENTION
COLORECTAL SAVING LIVES, WE DID
SEE MANY HEALTH CENTERS THE
RATES OF FOLLOW-UP WERE
DISAPPOINTINGLY LOW.
THOUGH OUR INTERVENTION DIDN'T
FOCUS ON THAT, IT IS REALLY
IMPORTANT TO CONSIDER IN OUR
CASE CANCER CONTINUUM BUT OTHER
CASES MIGHT BE FOLLOW-UP CARE
PATIENT NEEDS IF THEY HAVE
ABNORMAL TEST RESULT.
>> THANK YOU WE ARE COMING UP ON
OUR TIME HERE.
NOW THAT WE HEARD EVERYONE ON
THE PANEL DOES ANYONE WANT TO
ASK SOMETHING RELATED TO THIS
HEALTH DISPARITIES TO KEEP
ECHOING?
I HAVE ANOTHER QUESTION FROM THE
CHAT THAT I CAN USE THEN
PLANNING TO TRANSITION TO
SUMMARY FINAL THOUGHTS AND
GIVING YOU A MINUTE OR TWO TO DO
THAT STARTING AT 4:25 DO YOU
WANT KNOW THROW THIS QUESTION
OUT AND SEE WHAT PEOPLE THINK OR
CONTINUE THIS CONVERSATION?
>> THIS IS A QUESTION ABOUT
ADVANTAGES AND DISADVANTAGES OF
THE HYBRID DESIGN BOILS DOWN TO
WHAT -- WHEN YOU DECIDE WHAT IS
THE DEFINITION OF WELL
ESTABLISHED INTERVENTION
ADVANTAGES DISADVANTAGES OF THE
IMPLEMENTATION STUDY APPROACHES
WHEN EFFECTIVENESS OF
INTERVENTION IS NOT WELL
ESTABLISHED AND WHAT IS THE
DEFINITION OF WELL ESTABLISHED
INTERVENTION.
IS THAT IS A LITTLE BIT MAYBE
LONGER BUT MAYBE IF ONE OF YOU
ONE OR TWO BRIEFLY COMMENT ON
EITHER THAT IS THE DECISION ONLY
TO USE WELL ESTABLISHED
INTERVENTIONS?
IF SO, WHAT MEETS THAT
DEFINITION IN YOUR MIND.
THIS IS WHY I DON'T WORK IN
RADIO, DEAD AIR DOESN'T BOTHER
ME A BIT.
WE CAN LET PEOPLE THINK ABOUT IT
OR WE CAN MOVE ON.
>> I CAN JUMP IN.
I THINK THERE IS GRAY AREA
AROUND THIS.
IT IS GOING TO -- I THINK ONE
CAN MAKE ARGUMENTS TO SUPPORT
WHERE THEY WANT TO GO, CLEARLY
EVIDENCE FROM EFFICACY TRIALS
GIVES SENSE INTERVENTION IS
ESTABLISHED BUT THERE ARE LOTS
OF CAVEATS LIKE WHAT POPULATION
ESTABLISH AND WHERE ARE YOU MOW
TRYING TO IMPLEMENT AND THE ONUS
IS ON US TO THINK CAREFULLY
ABOUT HOW WHAT DATA IMPACT DATA
ARE IN EXISTENCE RELATE TO WHAT
WE ARE TRYING TO DO.
THERE ARE TRADE OFFS COMPARED TO
DOING IMPLEMENTATION STUDY.
SO YOU HAVE TO WEIGH PROS AND
CONS OF THOSE CHOICES AND TRY TO
MAKE THE BEST YOU CAN FOR
ADVANCING THE SCIENCE HOW YOU
THINK IT NEEDS TO BE DONE.
IN LIGHT OF THE BODY OF EVIDENCE
AGAINST WHAT YOU ARE TRYING TO
DO.
SO I'M SPEAKING GENERALLY BUT I
DON'T THINK THAT THERE IS JUST A
SINGLE ANSWER HERE.
>> THANK YOU, MARGARET.
ONE THING THAT POPPED UP WHEN
YOU SAID THAT IS GLORIA
MENTIONED DIFFERENTIAL EFFECTS
LATINO POPULATION BENEFITING
MORE FROM THE SCREENING, I THINK
THE MORE YOU THINK ABOUT THESE
PRAGMATIC TRIALS WONDER IF
TREATMENT MODERATORS THINGS LIKE
THAT WILL BECOME REALLY
IMPORTANT BECAUSE EVERYONE
THINKS OF THIS CLASSIC AS A
DEMONSTRATED EFFICACY BUT LOOK
AT POPULATION SOMETIMES THOSE
ARE WHICH THOSE ARE TESTED AND
HOW TESTED, I WONDER IF
TREATMENT MODERATORS ARE GOING
TO BE REALLY HELPFUL IN THIS
DECISION TO INSTEAD OF JUST
HAVING THAT DEMONSTRATION
BECAUSE WE COULD BE MISSING
SOMETHING ET CETERA SPECIALLY
FOR POPULATIONS LIKE WE SAID
PRAGMATIC TRIALS MIGHT BE THE
ANSWER TO SOME OF THESE, WE NEED
TO THINK ABOUT THAT BAR A LITTLE
BIT AND THAT MAKES IT GRAY IROF
COURSE BUT GLAD YOU CHARACTERIZE
IT AS A GRAY ZONE BECAUSE I
THINK UNFORTUNATELY IS ONE OF
THE GRAYEST ZONES.
ANY OTHER THOUGHTS ON THAT
QUESTION?
I DON'T KNOW IF MIGUEL WANTED TO
-- I THOUGHT I SAW HIM GO OFF
MUTE MAYBE NOT.
>>PROFESSOR: I THINK THAT
DEPENDS ALSO ON QUESTION FOR IN
OUR TRIAL WE WENT WITH EVIDENCE
BASED GUIDELINES BUT THERE WAS
(INAUDIBLE) RAPIDLY PROBABLY
WOULD BE A DIFFERENCE QUESTION.
A DIFFERENT QUESTION.
>> THANK YOU.
ALL RIGHT LET'S GO TO NEXT
SLIDE.
THIS IS WHERE WE GIVE EACH
PANELIST A VERY BRIEF FINAL
THOUGHT.
AND THEY CAN STICK TO ONE OF
THESE THEMES OR THEY CAN GO ON
THEIR OWN.
THIS IS WHAT WE USE TO SUMMARIZE
PRE-MEETING DISCUSSION.
WE SPENT MORE TIME IN DEVELOPING
DIFFERENT DIRECTION SO WHO WOULD
LIKE TO GO FIRST?
OR DO I GET TO PICK?
REMEMBER THIS IS BRIEF, THIS IS
THE LIGHTNING ROUND.
>> I CAN START THIS IS GLORIA.
I THINK THAT ONE OF THE -- THE
QUESTION THAT RESONATED WITH ME
WAS THE LAST ONE.
IS WHAT WE DID FOR OUR PRAGMATIC
TRIAL IS IT GOING TO BE
SUSTAINABLE AFTER THE TRIAL?
AND I THINK THIS IS SUCH A
CRITICAL QUESTION.
IN OUR WORK WHEN WE BROUGHT
TOGETHER ADVISORY BOARD WHICH
CONSISTED OF MANY OF THE MEDICAL
DIRECTORS FROM OUR FEDERALLY
QUALIFIED HEALTH CENTER CLINICS
THEY TALKED ABOUT AN IMPORTANT
BARRIER TO DOING FIT TESTING.
AND THAT IS THAT PATIENTS WITH
ABNORMAL TEST RESULT MIGHT FACE
CO-PAYS WHEN THEY NEED THAT
COLONS SO PI PROCEDURE.
SO WE TALKED ABOUT THIS ISSUE,
BECAUSE THEY WERE CONCERNED IT
PREVENTED THEM FROM REALLY
STRONGLY ENDORSING FECAL
TESTING.
ON OUR ADVISE BOARD WE HAD MITCH
GREENLICH WHO WAS LEGISLATOR AND
HE WAS ACTUALLY CO-FOUNDER OF MY
RESEARCH INSTITUTE.
AND WE DECIDED TO TAKE THIS
ISSUE TO HIM AND CHANGE STATE
POLICY TO NO LONGER BE BARRIER
FOR PATIENTS.
AND SO I THINK WE WERE AGE TO
BREAK DOWN SOME OF THE POLICY
ISSUES BECAUSE I THINK THAT
OFTEN TIMES AS RESEARCHERS WE
STAY AWAY FROM POLICY BUT
SOMETIMES YOU HAVE TO TACKLE
POLICY IF IT IS GETTING IN THE
WAY OF YOUR PROGRAM BEING
SUCCESSFUL.
SO IN ADDITION THE THAT, I JOIN
THE SCORING COMMITTEE FOR
MEDICATE HEALTH PLANS AND
INCORPORATED COLORECTAL CANCER
SCREENING, HELPS MAKES CLINICS
MORE INTERESTED BEING PART OF
OUR STUDY SO THOSE ARE EXAMPLES
OF THINGS WE DID TO KIND OF
REALLY CREATE MORE CONTEXT FOR
SUSTAINABILITY OVER THE LONG
TERM.
SORRY TO TAKE SO MUCH TIME.
>> THAT IS ALL RIGHT.
>> I WOULD ASK THE SAME QUESTION
BECAUSE WE HAVE BEEN THINKING
ABOUT THIS A LOTTEN OUR TEAM.
WE GOT RELAXATION ACUPUNCTURE,
GIVEN THE BARRIER THEY HAVE
TRANSPORTATION, ABILITY TO PAY
BECAUSE PAYING FOR (INAUDIBLE)
TO DO THIS SESSION SO IF THE
TRIAL IS OVER PATIENT WANT TO
CONTINUE THEY HAVE ACCESS TO
CARE, I SHOULD SAY, SO ONE OF
THE BARRIER FOR PAIN FOR THIS
(INAUDIBLE) WE HAVE INSURANCE
COMPANIES PAY FOR OUR THE
INFORMATION AND FOR THEM TO PAY
PROVIDE ACTUALLY PRESCRIBE
BETTER RELAXATION, PRESCRIBE
ACUPUNCTURE FOR YOUR PAIN AND
HAVE A CHANCE TO SEPARATE.
SO THIS IS THE WAY I SEE WITH
OUR INFORMATION (INAUDIBLE)
AFTER THE TRIAL SO FAR.
AND POLICY LEVEL AT LEAST.
>> THANK YOU, MIRIAM.
MARGARET OR MIGUEL.
>> I THINK REGARDING
SUSTAINABILITY I THINK WHAT WE
LEARNED IS IS SUSTAINABLE AS FAR
AS PRINCIPLES HOW TO ADDRESS
THESE TYPES OF QUESTIONS.
SPECIFICS ARE LESS IMPORTANT
BECAUSE AGAIN THE FIELDS CHANGES
SO FAST THE QUESTIONS WE ASK
WILL BE (INAUDIBLE) ARE NOT EVEN
THE SAME QUESTIONS WE WOULD ASK
TODAY BUT CERTAINLY THE APPROACH
IN THE OVERALL WAY TO CARRY THIS
TYPE OF RESEARCH IS SUSTAINABLE
NO QUESTION.
>> THANK YOU, MARGARET, YOU GET
TO WRAP THIS UP.
>> MY COMMENTS ARE GOING
DIFFERENT DIRECTION.
I RESONATE WITH THE FIRST BULLET
POINT I THINK ALL PCTs RUN
THROUGH IMPLEMENTATION SCIENCE
AND OUR COMMENTS IN THIS PANEL I
THINK SUPPORT THAT.
I WANT TO SHARE SOMETHING MY
COLLEAGUE PRESENTED, JIM BOGS
SAID TO ME, WHEN INTERVENTIONS
WORK IN HEALTHCARE SETTINGS AND
WE KNOW SETTINGS ARE COMPLEX.
HEALTHCARE SYSTEMS HAVE CAPACITY
TO DISSEMINATE THEM AND REACH SO
MANY CHILDREN AND FAMILIES.
AND WHATEVER POPULATION WE ARE
WORKING WITH.
SO WILL IS THE POTENTIAL FOR
IMPACT WE CAN CRAFT THAT
COMPLEXITY IMPLEMENTATION
BARRIERS.
I WANTED TO ADD I WAS IMPRESSED
WHEN I WAS AT HEALTHCARE SYSTEMS
TEN YEAR ANNIVERSARY PETE
MEETING A COUPLE OF MONTHS AGO
MADE ME AWARE OF MANY RESOURCES
DEVELOPED OVER THE COURSE OF
THOSE TEN YEARS TO SUPPORT
TRIALS LIKE OURS IN BEING
SUCCESSFUL AND ABLE TO MANAGE
THAT COMPLEXITY AND GET TO THE
OUTCOMES THAT WE ARE HOPING TO.
>> THANK YOU.
THANK YOU TO ALL PANELISTS FOR A
GREAT DISCUSSION.
I REALLY APPRECIATED WORKING
WITH YOU ON THIS.
AND HOPEFULLY THIS WAS -- WE MET
OUR CHARGE WHICH I THINK WENDY
HAS GIVEN US A PRELIMINARY
INDICATION THAT WE HAVE THERE IN
THE CHAT ROOM.
I THINK THATS IS MY QUEUE TO END
IT BACK OVER TO HER.
THANK YOU ALL.
REALLY ENJOYED IT.
>> YES.
LET ME ADD MY THANKS TO YOU
STEVEN AND THE ENTIRE PANEL FOR
REALLY AN EXCELLENT SESSION AS
WELL AS TO YOU WENDY FOR THE
EXCELLENT SESSION YOU LED.
WENDY WILL PROBABLY COME BACK ON
CAMERA AND THE PANELISTS FROM
THIS LAST SESSION CAN TURN THEIR
VIDEOS OFF AND MAYBE IF STEVE
YOU CAN JOIN US ON CAMERA WE
WILL DO A QUICK WRAP UP WITH
TODAY.
FIRST I THOUGHT I MIGHT JUST
REFLECT ON SOME OF WHAT WE HEARD
YESTERDAY AND PULL TOGETHER
OVERARCHING THEMES WHICH
THEREUPON A FEW THAT REALLY WE
HEARD TIME AND AGAIN.
WHEN KEITH MARSOLO AND THE FIRST
PANEL TALKED ABOUT WHAT ABOUT
THE LEK TROPIC HEALTH RECORD CAN
SQUEEZE MORE VALUE FROM THAT, A
TAKE AWAY IS FLEXIBILITY AND
AGILITY, GIVEN HOW DYNAMIC THE
SYSTEMS ARE AND HOW DYNAMIC THE
ORGANIZATIONS ARE AND REAL NEED
TO PAY ATTENTION TO POTENTIAL
IMPACT OF EVEN SMALL CHANGES.
I GUESS FROM MY PERSPECTIVE, IFF
THERE IS A OVERALL TAKE AWAY
FROM BOTH DAYS, IT IS NOT DON'T
SWEAT THE SMALL STUFF, IT IS PAY
ATTENTION TO THE SMALL STUFF.
THAT IS IMPORTANT.
THEN PATRICK AND THE TEAM OR
PANEL THAT ASKED OR ANSWERED THE
QUESTION ABOUT WHETHER WE SHOULD
CONSIDER DIFFERENT STUDY DESIGN,
REALLY UNDERSCORED THE
IMPORTANCE OF MONITORING AT
EVERY STEP ALONG THE WAY,
INCLUDING THE QUALITY OF THE
DATA AND ALSO THE IMPORTANCE OF
MULTI-DISCIPLINARY STUDY TEAM
AND DATA SAFETY MONITORING
BOARD.
I FELT LIKE THOSE THEMES WE
HEARD AGAIN TODAY AS WELL, NOT
JUST ABOUT FLEXIBILITY AND
PAYING ATTENTION BUT THE NEED
FOR TRUE MULTI-DISCIPLINARY TEAM
TO MANAGE THE COMPLEXITY AND
MANY FACETS OF PRAGMATIC TRIALS.
WENDY, I WONDER AS YOU REFLECT
ON THE PANEL THAT YOU MODERATED,
WHAT REALLY STOOD OUT FOR YOU AS
KEY TAKE AWAYS?
>> THERE'S SO MUCH EXPERIENCE
THESE INVESTIGATORS HAVE.
I THINK IS THE BIGGEST THING.
I FEEL LIKE EVERY TIME I TALK TO
TEAM AND INVESTIGATORS, I LEARN
A LITTLE BIT MORE, I THINK I
SAID THAT YESTERDAY.
LEARN NEW NUANCE OR NEW THING
THAT WE DIDN'T KNOW THEY DID.
AND WAS A KEY ELEMENT FOR
SUCCESS FOR THOSE STUDIES.
IT WAS INTERESTING HOW EACH OF
THE PROJECTS REALLY USED THEIR
PLANNING PHASES DIFFERENTLY TO
TACKLE WHAT THEY THOUGHT WERE
GOING TO BE THE TOUGHEST PIECES.
THAT IS A REALLY INTERESTING
NUANCE OF PLANNING PERIODS,
THERE IS NOT NECESSARILY A
STANDARD SET OF THINGS YOU NEED
TO DO.
THERE ARE IN SOME WAYS YOU GOT
TO FIGURE HOW YOU RECRUIT AND
HOW TO GET OUTCOMES BUT HOW
CHALLENGES FOR EACH INDIVIDUAL
TRIAL FOR DIFFERENT PARTS OF
THOSE ELEMENTS IS UNIQUE AND HOW
DO YOU PRIORITIZE EVERYTHING YOU
COULD DO IN THE PLANNING PHASE
WITH WHAT YOU GOT TO FIGURE OUT,
THERE'S JUST NO WAY YOU ARE ABLE
TO DO THE TRIAL.
THAT WAS THE TAKE AWAY FOR ME,
HOW MUCH VARIABILITY ACROSS THE
DIFFERENT STUDIES.
AND HOW EACH TEAM TOOK ON THEIR
APPROACH DIFFERENTLY DEPENDING
WHAT IT WAS TO BE THE MOST
CHALLENGING THING TO DO.
>> THAT CAME ACROSS LOUDLY.
AND CLEARLY.
IT REALLY DID.
STEVE, WHAT ABOUT YOU AND THESE
REALLY HUGE CHALLENGE
IMPLEMENTING INTERVENTIONS THAT
ARE OFTEN COMPLEX OR CAN BECOME
COMPLEX IN DYNAMIC COMPLEX
HEALTHCARE SYSTEM?
>> IN THE THREE AND A HALF
MINUTES I HAD TO REFLECT,
LESLIE.
I THINK YOUR COMMENT ABOUT
SWEATING THE SMALL STUFF, ONE OF
THE THINGS IMPLEMENTATION
SCIENCE DOES IS MAYBE GIVE YOU A
FRAMEWORK TO SWEAT -- TO --
BECAUSE OTHERWISE YOU RUN INTO A
CASE WHERE YOU ARE WORRYING
ABOUT EVERYTHING OR IT IS
CONTROLLING YOU BUT IF YOU HAVE
THE IMPLEMENTATION EXPERTISE YOU
ARE ABLE TO HANDLE IT AS IT
COMES TO YOU IN WAVES SO I
HADN'T THOUGHT OF IT THAT WAY
UNTIL YOU SAID -- OBVIOUSLY YOU
ARE NOT OFTEN ENCOURAGED TO
SWEAT THE SMALL STUFF BUT IN
PRAGMATIC TRIALS YOU HAVE TO.
BUT YOU NEED A FRAMEWORK TO DO
THAT OR IT WILL OVERWHELM YOU.
SO EACH OF THE TRIALISTS ON OUR
PANEL HAD CLEAR ROLES AND
ADVOCATED FOR INVOLVEMENT OF
IMPLEMENTATION SCIENCE.
THE TENSION OF BEING FLEXIBLE
ENOUGH TO WORK BUT MAINTAINING
THE CORE COMPONENTS IS THAT IS
GOING TO CONTINUE TO BE THERE
AND THERE IS JUST SOME PEOPLE
THAT ARE PURE CLINICAL TRIALISTS
THAT IS A VERY UNSATISFYING WAY
BUT THE MORE YOU WORK IN HEALTH
SYSTEMS THE MORE IT IS VERY
HELPFUL TO KNOW WHAT THOSE CORE
COMPONENTS ARE.
IN MY EXPERIENCE WE DECIDE THAT
AHEAD OF TIME BECAUSE WHAT YOU
FIND IN MY EXPERIENCE IS YOU
WANT TO WORK WITH PEOPLE BUT YOU
HAVE TO HAVE THAT SENOR CORE SO
IF SOMEONE DOES HAVE A
SUGGESTION THAT IS A REALLY GOOD
IDEA AND WOULD WORK CLINICALLY
BUT DOES PULL YOU TOO FAR AFIELD
YOU HAVE IT THERE SO EVERYONE
CAN MAKE THAT DECISION AND YOU
ARE NOT MAKING IT BASED ON THE
FACT THAT OH WE NEED SIX MORE
SITES TO GO TO UH 3 AND THIS ONE
MIGHT WORK.
SO I THINK THAT FLEXIBILITY
ADAPTATION AND HAVING SOME OF
THAT STANDARD IS REALLY
IMPORTANT.
AND THEN I THINK THE THIRD PART
THAT CAME UP AT THE END IS JUST
THERE IS AN INCREDIBLE
OPPORTUNITY TO ADDRESS THE
DISPARITIES BECAUSE IF YOU DO
THE TRIALS, THE WAY THAT A LOT
OF THEM ARE DONE, THE CONSENT
BAR MIGHT BE LOWER THAN -- THERE
ARE SON THINGS THAT REALLY IF WE
GET GOOD AT LEVERAGING IN THE
SPACE, I AGREE MIGUEL BROUGHT UP
FIRST BUT EVERYONE REINFORCED IT
PARTNERING WITH NEW PLACES, THAT
HAVEN'T DONE RESEARCH AS MUCH
BECAUSE I THINK THOSE PLACE --
NOT THAT THERE IS ANYTHING CRONK
WITH PLACES THAT DO RESEARCH A
LOT BUT TENT TO NOT HAVE THE
POPULATIONS WE LIKE TO EXPAND
TO.
AND THEN TO GO BACK TO
IMPLEMENTATION PART OF THAT
LESLIE, WE HEARD SOME OF IT WITH
GLORIA'S EXAMPLE OF THE WORD
LIST INSTRUCTIONS BUT
IMPLEMENTATION TEAM HAS TO BE
READY TO TAKE ON THAT CHALLENGE.
BECAUSE IT CAN'T BE SAME
APPROACH AS MOVING INTO
DIFFERENT POPULATIONS.
>> EXCELLENT.
>> THAT'S THE STICKS WITH ME.
>> EXCELLENT.
EXCELLENT TAKE AWAYS.
THANK YOU, STEVE.
WENDY I WILL HAND OFF TO YOU FOR
FINAL COMMENTS.
I GUESS ONE OTHER SORT OF REAL
-- NOT REALIZATION BUT ONE OTHER
ASPECT THAT STRUCK ME I THINK
TEN YEARS AGO WHEN THE
COLLABORATORY WAS JUST
LAUNCHING, I WOULD SAY THERE WAS
IN SOME AREAS AN ASSUMPTION
MAYBE NOT AMONG US BUT AMONG
OTHERS THAT PRAGMATIC TRIALS
WERE THE EASY ONES TO DO, THE
FLEXIBILITY MADE THEM EASIER AND
WHAT WE HAVE LEARNED AND WHAT WE
HAVE HEARD REPEATEDLY TODAY IS
THAT THE FLEXIBILITY BRINGS SUCH
CHALLENGE AND OPPORTUNITY BUT
REAL CHALLENGE.
WITH THAT I WILL HAND IT OFF TO
YOU FOR ANY FINAL THOUGHTS OR
REMARKS YOU HAVE.
>> THANK YOU SO MUCH.
I JUST WANT TO THANK ALL OF OUR
SPEAKERS AND OUR MODERATORS AND
ALL OF OUR PARTNERS ACROSS NIH
AND ALL THE OTHER ORGANIZATIONS
THAT WE WORK WITH HERE AS PART
OF THE PRAGMATIC TRIALS
COLLABORATORY, IT HAS BEEN AN
AMAZING TWO DAYS, THANK YOU FOR
STICKING THROUGH TO THE END, I
KNOW EVERYONE HAS BUSY SCHEDULES
AND REALLY APPRECIATE EVERYONE'S
INPUT, WE HAVE FOB LAWS
QUESTIONS COME IN FROM THE
AUDIENCE AND IF WE DIDN'T GET TO
YOUR QUESTION, WE WILL GET BACK
TO YOU VIA EMAIL.
I THINK ONE OF MY BIG TAKE AWAYS
IS SORT OF REFLECTING BACK TO
THE OPENING KEYNOTE HONESTLY.
IN THINKING BACK TEN YEARS AGO
TAKING THIS ADVENTURE TO DOING
TRIALS AND HEALTHCARE SYSTEMS
AND NOT KNOWING HOW MANY
INSTITUTES AND CENTERS THOUGHT
WE COULD DO THIS AND CAN YOU DO
RANDOMIZE STUDIES OR SO MUCH
EASIER TO DO OBSERVATIONAL WORK
WITH ELECTRONIC HEALTH RECORDS
AND PUSHING ALONG AND REALLY
WANTING TO SEE IF WE -- LET'S DO
TRIALS.
SEE IF WE CAN TAKE ADVANTAGE OF
RANDOMIZATION TO REDUCE BIAS AND
LOOK AT ALL THESE DIFFERENT
STUDY DESIGNS AND WE HAVE
LEARNED SO MUCH OVER THE LAST
DECADE.
I THINK THE KEYNOTE FROM DR.
SHANNON ZENK FROM NINR REALLY
OPENED YOU GUYS' EYES AND MR. 'S
PLACES ACROSS THE NIH AS WELL AS
OUTSIDE THE NIH THAT ARE FUNDING
THIS KIND OF WORK AND THERE ARE
OPPORTUNITIES TO DO TRIALS IN
HEALTHCARE SYSTEM, COMMUNITY
BASED CLINICS AND OTHER
FACILITIES AND THINGS LIKE THIS,
THAT CAN ANSWER QUESTIONS THAT
WILL REALLY ADDRESS QUESTIONS TO
HELP IMPROVE CARE FOR PATIENTS.
THIS IDEA OF ADDRESSING HEALTH
DISPARITIES AND INEQUITIES IS
AMAZING OPPORTUNITY AND FINDING
THE RIGHT PARTNERS TO ANSWER
QUESTIONSNA WILL HELP THEM
DELIVER BETTER CARE TO PATIENTS.
THOSE STUDIES MAY INFORM NOT
ONLY PATIENTS AND PROVIDERS BUT
ALSO PAYERS AND ORGANIZATIONS
THAT DELIVER HEALTHCARE.
SO WE HAVE AMAZING SETS OF
PARTNERS THAT WE CAN WORK WITH
FOR THIS RESEARCH.
I'M EXCITED I THINK THERE IS A
BRIGHT FUTURE AND I THINK WE
HAVE LEARNED A LOT AND SURE
THERE IS MORE TO LEARN.
AND CERTAINLY THERE IS KNOW SYBO
TON WHEN IT COMES TO PRAGMATIC
TRIALS BUT IT IS A DIFFERENT WAY
TO ANSWER QUESTIONS ABOUT HOW DO
THESE INTERVENTIONS WORK ONCE
YOU PUT INTO THE HEALTHCARE
SYSTEMS.
AND IMPORTANTLY CAN WE GET THEM
INTO HEALTHCARE SYSTEMS AND
ACTUALLY IMPLEMENT THEM IN A WAY
THAT IS POTENTIALLY SUSTAINABLE
IN THE LONG TERM.
TO INFLUENCE PATIENT CARE.
I WILL CLOSE THERE AND THANK
EVERYONE BEHIND THE SCENES
INCLUDING OUR AMAZING STAFF AT
NCCIH AND COMMUNICATIONS OFFICE
AND IT GROUPS AS WELL AS
FABULOUS TEAM AT THE
COORDINATING CENTER MARY JH
TAMMY AND OTHERS THAT HELPED
MAKE THIS HAPPEN AS WELL AS OUR
NIH VIDEOCAST COLLEAGUES MAKING
SURE WE ARE ABLE TO BROADCAST
LIVE AND ARCHIVE TO MAKE
AVAILABLE FOR THOSE WATCHING
AFTER THE FACT IF NOT ABLE TO
TUNE IN LIVE.
WITH THAT I WILL CLOSE US FOR
THE AFTERNOON AND THANK YOU ALL
AGAIN FOR PARTICIPATING WE LOOK
FORWARD TO SEEING YOU FOR FUTURE
WORKSHOPS WE HOLD AND SEE YOU
THEN.
THANKS SO MUCH.