>> ALL RIGHT, EVERYBODY, LET'S
GO AHEAD AND GET STARTED.
THANKS FOR YOUR ATTENDANCE TODAY
IN PERSON AND ON VIDEOCAST FOR
OUR LATEST GME GRAND ROUNDS
SESSION FOR JULY.
SO TODAY IS OUR CLINICAL CENTER
GME GRAND ROUNDS, OF COURSE, AND
FOR CME PURPOSES THE CODE FOR
TODAY'S GRAND ROUNDS IS 50566.
SO PLEASE TEXT THIS CODE TO THE
HOPKINS CME PHONE NUMBER ON THE
SLIDE TO RECEIVE CME CREDIT.
WE KINDLY INVITE YOU ALSO TO
PROVIDE FEEDBACK ON TODAY'S
LECTURE BY SCANNING THE QR CODE
SHOWN ON THE CME SLIDE, AND FOR
THOSE APPLYING FOR CME, YOU WILL
RECEIVE THE FEEDBACK SURVEY LINK
VIA EMAIL.
THE SURVEY WILL BE USED, OF
COURSE, TO PROVIDE US WITH
IMPORTANT FEEDBACK ABOUT THIS
PRESENTATION AND ALLOWS YOU TO
SUBMIT ANY SUGGESTIONS FOR
FUTURE GRAND ROUNDS TOPICS,
WHICH WE ALWAYS APPRECIATE.
SO FOLLOWING THE PRESENTATION
TODAY, WE'LL HAVE THE
OPPORTUNITY FOR QUESTIONS FOR
THE SPEAKER.
I'LL BE TAKING IN-PERSON FOR THE
MICROPHONES IN THE AISLES.
ALSO THE VIDEOCAST VIEWERS MAY
SUBMIT QUESTIONS AT ANY TIME
DURING THE LECTURE BY SCROLLING
DOWN AND CLICKING THE LIVE
FEEDBACK BUTTON LOCATED ON THE
VIDEOCAST WEBSITE.
SO QUESTIONS WILL BE TAKEN AS
TIME PERMIT AT THE CONCLUSION OF
THE PRESENTATION AND DR. CHUNG
WILL FACILITATE THAT.
SO I'M REALLY EXCITED TODAY TO
INTRODUCE OUR SPEAKER TODAY, DR.
DR. DESAI IS AN ACCOMPLISHED
CLINICIAN RESEARCHER, EDUCATOR
AND ADMINISTRATOR, AND A FRIEND
OF MINE.
HE HAS APPOINTMENTS IN
PULMONARY, CRITICAL CAR AND
GENERAL INTERNAL MEDICINE.
HE'S WIDELY RECOGNIZED FOR HIS
ACCOMPLISHMENTS IN HEALTH
SYSTEMS ADMINISTRATION.
HIS RESEARCH EFFORTS FOCUS ON
GENERATING HIGH QUALITY DATA TO
INFORM POLICY AND PRACTICE IN
MEDICAL EDUCATION.
HE HAS AUTHORED OVER 50 PEER
REVIEWED MANUSCRIPTS INCLUDING A
SERIES OF ARTICLES IN THE NEW
ENGLAND JOURNAL OF MEDICINE.
DR. DESAI GRADUATED CUM LAUDE
FROM THE UNIVERSITY OF
PENNSYLVANIA AND EARNED A
BACHELOR OF SCIENCE IN
BIOENGINEERING AND A BACHELOR OF
ECONOMICS FROM THE WHARTON
SCHOOL OF BUSINESS IN HEALTHCARE
MANAGEMENT.
HE EARNED HIS MEDICAL DEGREE
FROM THE HARVARD MEDICAL SCHOOL
AND COMPLETED HIS RESIDENCY AS
WELL AS CHIEF RESIDENCY AT THE
JOHNS HOPKINS HOSPITAL, AS WELL
AS A FELLOWSHIP IN PULMONARY AND
CRITICAL CARE MEDICINE.
HE SERVED AS DIRECTOR OF THE
MEDICAL ICU AT MEDSTAR
WASHINGTON HOSPITAL, THEN
RECRUITED BACK TO HOPKINS AS THE
MEYER PROFESSOR OF MEDICINE, THE
DIRECTOR OF THE OCER MEDICAL
RESIDENCY AND VICE CHAIR IN THE
DEPARTMENT OF MEDICINE.
DURING HIS TIME ON UNIVERSITY
FACULTY, DR. DESAI WAS AWARDED
THE PRESTIGIOUS PROFESSOR'S
AWARD FOR EXCELLENCE IN TEACHING
FROM THE JOHNS HOPKINS
UNIVERSITY SCHOOL OF MEDICINE.
IN 2021, DR. DESAI JOINED THE
AMERICAN MEDICAL ASSOCIATION AS
CHIEF ACADEMIC OFFICER.
AND NOW HE OVERSEES MEDICAL
EDUCATION ACTIVITIES FOR THE
ORGANIZATION.
THAT SAME YEAR HE ALSO JOINED
MCKENZIE AS COMPANY AS SENIOR
ADVISOR WHERE HE HAS CONSULTED
FOR A VARIETY OF INDUSTRIES BOTH
IN THE U.S. AND IN SOUTHEAST
ASIA.
AND IN JUNE, DR. DESAI CONCLUDED
A FIVE-YEAR, $1.8 MILLION
RESEARCH EFFORT FOR THE AMA IN
PARTNERSHIP WITH STANFORD
UNIVERSITY AND THE UNIVERSITY OF
ALABAMA BIRMINGHAM ENTITLED
REIMAGINING RESIDENCY TRAINING,
OVERSEEING GRADUATE MEDICAL
LABORATORY TO MEASURE RELEVANT
VARIABLES IN RESIDENCY AND
ASSOCIATE THEM WITH IMPORTANT
DOMAINS SUCH AS BURNOUT,
CLINICAL SKILLS AND OTHER KEY
OUTCOMES.
SO REALLY EXCITED TO HAVE
SANJAY.
HIS TALK TODAY IS PRECISION
EDUCATION, THE FUTURE OF
LIFELONG LEARNING.
PLEASE JOIN ME IN WELCOMING
DR. SANJAY DESAI.
[APPLAUSE]
>> THANKS.
I WANT TO THANK EVERYONE THAT
PARTICIPATED IN INVITING ME.
IT'S AN HONOR TO BE HERE.
I'VE COME HERE A COUPLE OF TIMES
FOR THE CRITICAL CARE TEAM AND
IT'S REALLY NICE TO BE PART OF
THIS WIDER GROUP AS WELL.
WHAT I'M HOPING TO DO IS TALK
MAYBE FOR 30, 35 MINUTES JUST ON
THE CONTENT THAT I HAVE BUT
HOPEFULLY STIMULATE SOME
DISCUSSION WITH ALL OF YOU HERE
IN THE AUDITORIUM AS WELL AS
ANYBODY ONLINE THAT'S INTERESTED
IN THIS, AND THE GOAL HOPEFULLY
IS TO SHARE WORK THAT WE'VE BEEN
THINKING ABOUT, WORK THAT WE'RE
STARTING TO DEPLOY GRANTS IN AS
WELL BUT REALLY THE OPPORTUNITY
THAT WE'RE TRYING TO SHAPE IS
THINKING ABOUT THE FUTURE OF
MEDICAL EDUCATION AND FOR US,
WE'RE TITLING THIS PRECISION
EDUCATION.
SO I'LL GIVE A DEFINITION OF
THAT AS WELL AS THE CONCEPTUAL
MODEL THAT WE'VE THOUGHT
THROUGH.
MY DISCLOSURES ARE ONLY THAT
THESE OPINIONS ARE MY OWN, THE
ONES THAT I'LL SHARE, AND WE
HAVE MANY, MANY ACKNOWLEDGMENTS.
THIS WORK HAS BEEN DONE BY A
LARGE TEAM ACROSS THE KUN TRU
AND I'LL TRY AND BRING IN THEIR
NAMES AS I GO THROUGH THE
EXAMPLES WE HAVE.
THE OBJECTIVES ARE TO IDENTIFY
THE BARRIERS OF LIFELONG
LEARNING.
I THINK FOR ALL THE PHYSICIANS
IN THE AUDITORIUM AS WELL AS
ONLINE, I THINK THESE WILL NOT
BE SURPRISING TO YOU AND
HOPEFULLY WON'T CAUSE ANY PTSD
FOR YOU EITHER.
WE'LL SHARE THE CONCEPTUAL MODEL
WE'VE DEVELOPED IN PRECISION
EDUCATION AND I THINK THE MORE
IMPORTANT PART AND REALLY THE
SECTION I THINK THAT BRINGS THIS
TO LIFE ARE THE EXAMPLES.
SO REALLY DESCRIBING HOW THIS IS
ALREADY BEING USED ACROSS THE
COUNTRY, WHICH HOPEFULLY WILL
ILLUSTRATE THE POWER OF
PRECISION EDUCATION IN MEDICINE.
SO THIS IS THE MODEL FOR MEDICAL
EDUCATION IN THIS COUNTRY.
SO JUST AS BACKDROP, WHY DO WE
HAVE MEDICAL EDUCATION IN THE
COUNTRY, AND I THINK WE FORGET
THIS SOMETIMES OR AT LEAST WE
DON'T THINK ABOUT IT EVERY DAY
AS IS NOT SURPRISING BUT THE
REASON WE EVEN HAVE A MEDICAL
EDUCATION SYSTEM IS PRODUCE A
PHYSICIAN WORKFORCE.
THE GOAL IS TO PRODUCE A
PHYSICIAN WORKFORCE THAT CAN
CARE FOR OUR PATIENTS, FAMILIES
AND COMMUNITIES IN THIS COUNTRY.
THE WAY THAT WE DO THAT IN THIS
COUNTRY, AND IT'S NOT -- THIS IS
ACTUALLY IN SOME WAYS UNIQUE TO
THIS COUNTRY, IS THE MODEL HERE.
I WON'T GO THROUGH ALL THE
DETAIL WITH YOU, BUT THE GOAL
IDEALLY IS TO HAVE THESE
COMPARTMENTS OF EDUCATION THAT
WE TRAVERSE AND AS WE TRAVERSE
THEM, THERE ARE GATES OBVIOUSLY
BETWEEN ONE STAGE OF LEARNING TO
THE NEXT STAGE OF LEARNING.
THOSE GATES ARE HOPEFULLY MEANT
TO AND DESIGNED TO BRING IN
PEOPLE TO RECRUIT PEOPLE TO
DEVELOP PEOPLE AND TO PROMOTE
PEOPLE EQUITABLY THROUGH THIS
PROCESS.
IF WE'RE ABLE TO DO THAT AND
BRING THEM INTO LEARNING
ENVIRONMENTS THAT ARE ABLE TO
CONTINUE TO DEVELOP THEM AS THEY
GO FORWARD THEN THIS WOULD BE
THE OUTCOME.
THE OUTCOME IS THAT WE HAVE A
CONTINUOUS INCREASE IN THE
COMPETENCY OF THE PHYSICIANS
THAT ARE TRAVERSING THIS AND THE
CPD STANDS FOR CONTINUOUS
PROFESSIONAL DEVELOPMENT SO THIS
IS -- THAT'S THE LONG ARC OF
WHERE WE ARE, AND THE GOAL IS
THAT WE HAVE LIFELONG LEARNING
IN WHICH WE CONTINUE TO DEVELOP
COMPETENCY.
WE RECOGNIZE, HOWEVER, THIS
IDEALIZED MODEL IS NOT WHAT WE
EXPERIENCE IN THE COUNTRY AND SO
THERE'S LOTS OF DEVIATIONS FROM
THAT IDEALIZED MODEL.
THE FIRST IS THAT WE HAVE AN
INEFFICIENT PROCESS.
SO WE DON'T FOLLOW THIS LINEAR
TRAJECTORY.
INSTEAD WE HAVE THIS UNDULATING
TRAJECTORY.
THERE ARE MOMENTS WHERE WE
ACCELERATE, THERE ARE MOMENT
THAT WE DECELERATE OR STALL IN
COMPETENCY DEVELOPMENT.
IT'S EASY TO THINK THROUGH SOME
EXAMPLES OF THAT AND THAT'S
BECAUSE WE ARE ALL TAUGHT IN A
ONE -- GENERALLY SPEAKING IN A
ONE SIZE FITS ALL MODEL.
SO YOU GO TO MEDICAL SCHOOL, ONE
PERSON HAS A DEGREE IN
BIOCHEMISTRY, ANOTHER PERSON
GOES TO SCHOOL -- IT'S PRETTY
EASY TO UNDERSTAND ONE PERSON
WILL GET MORE OUT OF THAT COURSE
WORK THAN THE OTHER PERSON SO
AGAIN WE'RE NOT DEVELOPING THEM
EVERY DAY THAT WE'RE THERE,
WHICH IS A PROBLEM.
THE OTHER IS THE RANDOMNESS THAT
OCCURS.
THIS IS EASIEST TO EXPLAIN IN
RESIDENCY OR FELLOWSHIP PROGRAMS
WHICH MANY OF YOU ARE EITHER
LEADING OR PARTICIPATING IN,
WHERE YOU HAVE ROTATIONS BASED
ON CONVENIENCE OR WHEN WE HAVE
VACATION PREFERENCE, NOT BASED
ON WHAT IS BEST FOR YOU TO LEARN
TOMORROW.
SO THERE'S INEFFICIENCY AS PART
OF THE CYST TE. THE SECOND THING
THAT HAPPENS ARE THERE ARE THESE
PLATEAUS THAT OCCUR THAT
PREDOMINATE AT THE TRANSITIONS
BETWEEN ONE COMPARTMENT OF
LEARNING TO THE NEXT.
THAT'S BECAUSE WE DO NOT SHARE
INFORMATION ABOUT SOMEONE FROM
ONE COMPARTMENT TO THE NEXT IN A
WAY THAT IS MOST INFORMATIVE.
WHEN THAT ENTERS YOU ENTER
WITHOUT KNOWING WHAT IS BEST FOR
THIS PERSON TO LEARN AND THAT
LEADS TO A PLATEAU IN THEIR
DEVELOPMENT AS THEY GO FORWARD.
THE BIGGER PROBLEM THAT WE
OBSERVE, HOWEVER, IS NOT JUST
THE INEFFICIENCY AND THE
PLATEAUS THAT OCCUR BUT THE
VARIABILITY AND JOURNEYS THAT WE
EACH HAVE.
SO I'M GOING TO SHARE JUST A FEW
ARCHETYPES OF JOURNEYS THAT
PEOPLE HAVE TRAVERSED.
I THINK MANY OF THESE WILL BE
FAMILIAR FOR YOU.
SO THIS IS THE COMMON ONE.
THIS IS WHAT WE CALL THE
PREDOMINANT PATH.
THIS IS THE UNDULATION AS YOU
SEE, THE PLATEAUS THAT OCCUR,
BUT THANKFULLY, AT THE END OF
THIS JOURNEY, THIS PERSON HAS
REACHED THE LEVEL OF COMPETENCY
THAT'S EXPECTED OF THEM FROM THE
COUNTRY.
SO AGAIN THE PATH MAY NOT BE
IDEAL BUT THEY REACH A LEVEL OF
COMPETENCY TO DELIVER CARE IN A
WAY WE WOULD EXPECT THEM TO
DELIVER BUT THIS IS NOT EVE
EVERYONE.
ANOTHER PATH IS WHAT WE CALL THE
CLASSROOM ACHIEVER.
THIS IS THE PERSON THAT EXCELS
IN STRUCTURED LEARNING
ENVIRONMENTS ON THE STANDARDIZED
OR THE TYPICAL WAYS THAT WE DO
ASSESSMENTS IN THIS COUNTRY,
EXCELS IN ALL OF THOSE.
THEY HOWEVER GET TO THE CLINICAL
ENVIRONMENT AND THEY'RE NOT
EXCELLING AT THE SAME PACE.
AND THEY CONTINUE TO EXCEL BUT,
AGAIN, NOT AT THE SAME PACE, AND
PROBABLY NOT PREDICTABLE BASED
ON THE PERFORMANCE THAT THEY HAD
PREVIOUSLY AND THE INVERSE OF
THIS ALSO OCCURS, WHICH WE'VE
CALLED THE LATE BLOOMER.
THIS IS THE PERSON THAT DOESN'T
EXCEL IN THE STANDARDIZED
TRADITIONAL WAYS OF ASSESSMENT
HOWEVER THEY HIT THE BEDSIDE AND
IT IS MAGICAL.
I WOULD SAY JUST MYSELF HAVING
BEEN A M PRA DIRECTOR FOR OVER
10 YEAST, THIS IS THE MORE
COMMON ARCHETYPE THAT I'VE SEEN
AND THE ONE THAT WE ARE AGAIN
OVERJOYED TO SEE.
BUT REALLY IT BRINGS TO LIFE THE
PROBLEM THAT WE HAVE, THAT WE
HAVEN'T BEEN ABLE TO PREDICT WHO
THIS PERSON IS.
THERE'S ALSO THE PERSON THAT
CONTINUES TO GET PUSHED THROUGH,
THE STAGNATING LEARNER, THE
PERSON THAT HASN'T QUITE MET THE
LEVEL OF EXPECTED COMPETENCY
THAT WE WOULD WANT THEM TO HAVE,
HOWEVER, IT'S A LOT OF WORK TO
CONTINUE TO DEVELOP THEM.
AND OUR STRUCTURED ENVIRONMENTS
ARE NOT OFTEN DESIGNED OR
RESOURCED TO BE ABLE TO BRING
EVERYBODY UP TO THAT LEVEL OF
EXPECTATION THAT WE WOULD HAVE.
AND THE EASIER PATH IS TO PUSH
THEM FORWARD.
THIS IS VERY COMMON IN K THROUGH
12, CONTINUE TO PUSH PEOPLE
THROUGH BECAUSE IT'S THE PATH OF
LEAST RESISTANCE.
THE PROBLEM IS AGAIN AS YOU GET
FURTHER AND FURTHER ALONG, THE
MORE DISTANT YOU BECOME FROM THE
LEVEL OF EXPECTATION THAT'S BEEN
DESIGNED.
THEN OF COURSE THERE'S THE
PERSON THAT WE CALL UNSUPPORTED
IN THE LEARNING ENVIRONMENT.
THESE ARE THE INDIVIDUALS THAT
FIND THEMSELVES IN AN
ENVIRONMENT THAT UNFORTUNATELY
IS NOT DESIGNED TO SUPPORT THEM.
NOT DESIGNED TO DEVELOP THEM.
NOT DESIGNED TO PROMOTE THEM AS
THEY MOVE FORWARD.
IF IT HAPPENS, CERTAINLY THEY
STAGNATE, THEY PLATEAU, BUT IN
THE WORST-CASE SCENARIO AS SHOWN
HERE, THEY DROP OUT OF THE
SYSTEM ALL TOGETHER AND THAT CAN
HAPPEN AT ANY STAGE, IT CAN
HAPPEN IN MEDICAL SCHOOL, IT CAN
HAPPEN IN RESIDENCY, AND WE KNOW
THIS HITS MINORITIES MORE THAN
IT HAPPENS TO OTHER INDIVIDUALS
AS THEY TRAVERSE THIS JOURNEY.
AND FINALLY, THE LAST ONE THAT
I'LL DESCRIBE IS THE PERSON THAT
GOES THROUGH IT AND ACTUALLY
FOLLOWS THE PATH THAT IS THE
PREDOMINANT PATH BUT THEN FINDS
THEMSELVES AFTER LEAVING THESE
STRUCTURED ENVIRONMENTS OF
LEARNING IN A PRACTICE
ENVIRONMENT THAT NO LONGER
FACILITATES LIFELON LEARNING.
THAT IS, I THINK, ONE OF THE
MOST DANGEROUS EXAMPLES OF ARC
TYPES THAT WE HAVE BECAUSE NOW
THEY'RE NOT UNDER SUPERVISION,
AND YOU GET FURTHER AND FURTHER
AWAY FROM THE LEVEL OF
COMPETENCY THAT'S EXPECTED
BECAUSE THE FIELD OBVIOUSLY
CONTINUES TO EVOLVE AND DEVELOP.
SO THESE ARE JUST A FEW
ARCHETYPES, I THINK IF WE GIVE
YOU A FEW MINUTES YOU COULD COME
UP WITH A HUNDRED MORE
ARCHETYPES THAT YOU COULD
PROBABLY THINK THROUGH OF HOW WE
ARE DIFFERENT THAN THE IDEALIZED
STATE.
THE PROBLEM ISN'T JUST THE
VARIABILITY OF JOURNEY.
THE PROBLEM IS THE VARIABILITY
IN OUTCOMES.
SO WE LAND WITH -- WE END WITH A
PHYSICIAN WORKFORCE IN WHICH
THERE ARE A SUBSET OF PHYSICIANS
THAT HAVE NOT MET THE LEVEL OF
COMPETENCY THAT AGAIN WE HAVE
EXPECTED AND THE SYSTEM IS
DESIGNED TO ACHIEVE.
JUST TO BE VERY CLEAR ABOUT IT,
WE WANT VARIABILITY IN
COMPETENCY IN TERMS OF WHAT
PEOPLE MASTER, WHAT PEOPLE DO,
BUT WE WANT A MINIMUM LEVEL OF
COMPETENCY IN TERMS OF
DELIVERING CARE FOR OUR PATIENTS
AND OUR COMMUNITIES, AND THAT'S
THE VARIABILITY THAT WE'RE
TRYING TO AVOID.
SO THIS NOISE IN THE SYSTEM AND
BIAS IN THE SYSTEM HAS TO BE
ADDRESSED.
THERE ARE FOUR WAYS THAT WE'VE
PRIORITIZED THINKING ABOUT IN
TERMS OF ADDRESSING THIS AND
MAKING THIS A BETTER SYSTEM THAT
CAN ACHIEVE THESE OUTCOMES.
THE FIRST IS EXPANSION OF
COMPETENCY BASED MEDICAL
EDUCATION.
THIS HAS BEEN A GOAL, AN
ASPIRATION FOR MEDICAL EDUCATION
AND EDUCATION IN GENERAL
GLOBALLY FOR DECADES.
AND IN FACT, OTHER COUNTRIES ARE
FURTHER AHEAD THAN US IN TRYING
TO PURSUE COMPETENCY-BASED
MEDICAL EDUCATION. WE HAVE A
TIME-BASED NOT A
COMPETENCY-BASED MODEL AND I
THINK IF YOU ASK ANYBODY THAT
YOU CARE FOR WHAT'S MORE
IMPORTANT, THE NUMBER OF YEARS
YOU GO TO SCHOOL OR THE ABILITY
TO CARE FOR SOMEBODY CAPABLY, I
THINK THAT ANSWER IS OBVIOUS, WE
DO NOT HAVE A SYSTEM THAT
DEPENDS ON THIS.
BUT IF WE CAN EXPAND COMPETENCY
BASED MEDICAL EDUCATION SYSTEMS
THEN WE CAN START TO MOVE THESE
GRAPHS IN THE WAY THAT'S
DESCRIBED HERE.
WE CAN ELEVATE THE SLOPES OF
THESE CURVES SO EACH LEVEL, EACH
STAGE OF LEARNING AND WITHIN
EACH COMPARTMENT OF LEARNING,
WE'RE ABLE TO ENSURE SOMEBODY
REACHES THE COMPETENCY LEVEL
BEFORE THEY MOVE FORWARD AND WE
CAN REDUCE VARIABILITY THAT WAY.
THE SECOND IS IMPROVING
TRANSITION.
SO WE HAVE THESE TRANSITIONS
BETWEEN EACH COMPARTMENT, THE
ONE WE DESCRIBE MOST OFTEN IS
BETWEEN MEDICAL SCHOOL AND TO
RESIDENCY.
AND INFORMATION IS NOT SHARED.
AGAIN, WHEN INFORMATION ISN'T
SHARED BECAUSE THERE'S MISTRUST
AND WE DON'T HAVE SYSTEMS THAT
PROVIDE INFORMATION WITH
FIDELITY, THEN LEARNING IS LOST
AND DEVELOPMENT IS LOST AND THIS
PLATEAUS OCCUR, SO IF WE CAN
IMPROVE THE FIDELITY OF
INFORMATION TRANSFER AND MAKE
THAT INFORMATION REPRESENTATIVE
OF THE PERSON FOR THE
COMPETENCIES THAT MATTER, THEN
WE CAN START TO, AGAIN, INCREASE
THE SLOPES OF THESE CURVES AND
REDUCE VARIABILITY.
EMBEDDING EQUITY IN THE LEARNING
ENVIRONMENT IS INCREDIBLY
IMPORTANT SO WE DON'T FIND
PEOPLE IN ENVIRONMENTS THAT ARE
NOT DESIGNED TO SUPPORT THEM, TO
DEVELOP THEM AND TO PROMOTE
THEM.
AND IF WE'RE ABLE TO DO THAT, WE
CAN STOP THE PLATEAUS, BUT
IDEALLY IN THE WORST CASE, WE
STOP THESE DROPOFFS THAT ARE
CRITICAL FOR US DEVELOPING THE
WORKFORCE THAT WE NEED IN THE
COUNTRY.
WE ALREADY FIXED THE PIPES HERE,
THE IDEA WITH WATER IN THE
HOUSE, IF YOU HAVE A LEAK IN
YOUR HOUSE IN THE PIPES, IT'S A
PROBLEM WITH THE -- THE
INCORRECT THING TO DO IS TRY TO
ADDRESS THE WATER, IT'S NOT THE
PROBLEM THAT'S THE PROBLEM, IT'S
THE PIPES THROUGH WHICH THE
WATER FLOWS, THAT'S THE PROBLEM,
THE DURABLE SOLUTION WE HAVE TO
ALL BE THINKING ABOUT.
FINALLY, BUILDING SYSTEMS OF
PRECISION EDUCATION WILL BE THE
TALK THAT I GIVE TODAY, AND THE
IDEA OF PRECISION EDUCATION IS
THE RIGHT LEARNING FOR THE RIGHT
PERSON AT THE RIGHT TIME AND
LEVERAGING DATA AND TECHNOLOGY
TO DO THIS.
OF IF WE'RE ABLE TO
INDIVIDUALIZE LEARNING,
ELIMINATE THE RANDOMNESS,
ELIMINATE THE ONE SIZE FITS ALL
PARADIGM THAT WE'D HAVE, REALLY
GENERALLY THROUGHOUT ALL OF OUR
COMPARTMENTS OF LEARNING, THEN
WE CAN REDUCE THESE UNDULATIONS
THAT OCCUR.
AND TO BE, AGAIN, VERY CLEAR
ABOUT IT, WE KNOW THAT LEARNING
IS FRICTION, THERE'S FRICTION IN
LEARNING.
THAT'S HOW YOU LEARN, BUT
THERE'S UNNECESSARY FRICTION.
THERE'S THIS PLATEAU, THERE'S
DECELERATIONS.
THOSE ARE THE MOMENTS THAT WE
CAN ELIMINATE IF WE'RE ABLE TO
PERSONALIZE LEARNING WITH
PRECISION EDUCATION.
AND THAT WOULD AGAIN ELEVATE
THE -- AND SMOOTH THE PATH.
SO IF WE'RE ABLE TO DO THIS
EFFECTIVELY, THEN WE CAN TAKE
THIS OUTCOME WE HAVE NOW WITH
VARIABILITY IN THE PHYSICIAN
WORKFORCE THAT WE PRODUCE AND
IDEALLY PRODUCE A PHYSICIAN
WORKFORCE WITH LESS VARIABILITY
AND OUTCOMES RELATED TO
COMPETENCY AND ONE THAT'S
RELIABLE AND ENABLES BETTER
PATIENT CARE.
SO THAT'S THE GOAL THAT WE HAVE,
AND THE FOCUS AGAIN FOR US WITH
THE WAY THAT WE BELIEVE THE
FUTURE SHOULD BE DESIGNED TO
PROMOTE THIS AND TO ESTABLISH IT
IS THROUGH PRECISION EDUCATION.
SO WHY PRECISION EDUCATION?
WE'LL GET INTO THE CONCEPTUAL
MODEL IN JUST A MOMENT, BUT WE
WOULD PROPOSE TO EVERYBODY THAT
PRECISION EDUCATION ELEVATES ALL
OF THESE DIFFERENT INTERVENTIONS
THAT LEAD TO THIS REDUCED
VARIABILITY IN OUTCOME.
SO PRECISION EDUCATION, IF
YOU'RE ABLE TO UNDERSTAND THE
PROFILE OF COMPETENCY, OF
LEARNING, THE GAPS, THE NEEDS OF
THE INDIVIDUAL THEN YOU CAN
EXPAND -- YOU CAN'T DO THAT
WITHOUT ASSESSMENT, WITHOUT
UNDERSTANDING PRECISELY WHAT
THIS INDIVIDUAL NEEDS.
IF YOU UNDERSTAND PRECISION --
IF YOU EXPAND AND INVEST IN
PRECISION EDUCATION, THEN YOU
GET DATA THAT THEN CAN BE
SHARED.
THOSE DATA THAT WHEN THEY'RE
SHARED CAN HELP ELIMINATE THE
PROBLEMS THAT WE HAVE WITH
TRANSITIONS.
IF WE UNDERSTAND THE ACTUAL
OBJECTIVE OBJECTIVELY, MAKE
THESE ASSESSMENTS EQUITABLE,
MAKE THEM OBJECTIVE, ELIMINATE
BIAS, AND UNDERSTAND EACH
INDIVIDUAL IN TERMS OF WHAT
THEY'RE ABLE TO CONTRIBUTE AND
AS WELL AS WHAT THEY'RE ABLE TO
DEVELOP, AND CREATE A MODEL
WHERE WE'RE THERE TO SUPPORT
EVERYONE RATHER THAN A DEFICIT
MODEL WHERE WE'RE TRYING TO SAY
YOU'RE NOT GOOD ENOUGH, THEN WE
CAN CHANGE THE CULTURE AND THE
ENVIRONMENT OF ASSESSMENT, OF
PROCESS AND OF DEVELOPMENT SUCH
THAT EQUITY IS LIFTED AND AGAIN
LIFTING EQUITY, EMBEDDING EQUITY
LIFTS ALL THOSE.
SO WE SEE PRECISION EDUCATION AS
A TOOL, THESE SYSTEMS AS A TOOL
THAT ENABLES US TO ACHIEVE
BETTER COMPETENCY-BASED MEDICAL
EDUCATION, A TOOL THAT CAN HELP
US EMBED EQUITY MORE EQUITABLY
INTO THE MEDICAL EDUCATION
STRUCTURES THROUGHOUT THE
CONTINUUM.
SO WHAT IS IT?
WE TALKED ABOUT IT BEING THE
RIGHT LEARNING FOR THE RIGHT
PERSON AT THE RIGHT TIME USING
TECHNOLOGY.
PRECISION EDUCATION HAD BEEN
DEVELOPED OR AT LEAST THOUGHT
ABOUT AND DISCUSSED IN OTHER
DISCIPLINES IN EDUCATION, NOT
MEDICINE, BEFORE IT CAME TO
MEDICINE.
AND SO WE TOOK TIME TO TALK TO A
LOT OF DIFFERENT TYPES OF
PEOPLE.
WE STARTED OUR JOURNEY IN
PRECISION EDUCATION BACK IN
2022, WHEN WE HAD ALL THE SUMMIT
ASKING ABOUT THIS.
THIS ENERGY THAT WE HAD FOR
PRECISION EDUCATION WAS TIMED
REALLY CONVENIENTLY BECAUSE AS
ALL OF YOU KNOW, THE BOOM IN
A.I. AND IN GENERATIVE A.I. IN
PARTICULAR HAS REALLY COINCIDED,
WE WERE ABLE TO SORT OF CAPTURE
SOME OF THE WAVE THAT CAME WITH
THAT, BUT WE TOOK TIME TO -- AND
I THINK THIS IS A LITTLE BIT
AUTOMATED, BUT WE TOOK TIME TO
TALK NOT ONLY TO MEDICAL
EDUCATION EXPERTS BUT TO TALK TO
INDUSTRY EXPERTS, TO TALK TO
TECHNOLOGY EXPERTS AND TALK TO
EDUCATION EXPERT OUTSIDE OF
MEDICINE AS WELL AND THIS IS US
SPEAKING WITH SAL KHAN, WHO I
HOPE MANY OF YOU KNOW SOME OF
HIS WORK, HE'S RECENTLY WRITTEN
A NEW BOOK ON A.I. AND
REVOLUTION OF A.I. RELATED TO
EDUCATION GENERALLY.
HE IS, IN MY VIEW, PROBABLY THE
FURTHEST AHEAD IN TERMS OF
INNOVATION AND THINKING
CREATIVELY BL PRECISION
EDUCATION AND HE'S APPLIED THAT
CERTAINLY TO K THROUGH 12 IN
EFFECTIVE MODELS THAT HAVE BEEN
SCALED GLOBALLY.
SO THIS IS THE DEFINITION THAT
WE DEVELOPED WITH PRECISION
EDUCATION.
THESE ARE SYSTEMS IN MEDICINE
THAT USE DATA AND TECHNOLOGY TO
TRANSFORM LIFELONG LEARNING.
THEY DO THREE THINGS, IMPROVE
PERSONALIZATION OF LEARNING,
THEY IMPROVE THE EFFICIENCY, WE
DESCRIBED HOW THE CURRENT MODELS
ARE INEFFICIENT, AND REALLY
IMPORTANTLY, THEY TRANSFER
AGENCY OF LEARNING TO THE
LEARNER.
SO THE IDEA HERE AS THE QUICK
EXAMPLE WOULD BE A DASHBOARD, SO
THAT YOU AS THE LEARNER
UNDERSTAND AND CAN DRIVE THE
LEARNING THAT YOU HAVE, YOU HAVE
OWNERSHIP OVER IT THAT YOU DON'T
HAVE TODAY.
BECAUSE TODAY, LARGELY IT'S
DETERMINED BY THE DIRECTORS OF
THAT EDUCATIONAL PROGRAM OR
CURRICULUM.
WE ALSO IMPORTANTLY DESCRIBE
THIS IN TERMS OF INDIVIDUALS,
PROGRAMS AND ORGANIZATIONS.
SO THIS CAN HAPPEN AT DIFFERENT
LEVELS THROUGHOUT THE
EDUCATIONAL STRUCTURES.
IF WE DO THIS WELL, WE CAN
IMPROVE PATIENT OUTCOMES.
SO THIS IS THE CONCEPTUAL MODEL
GRAPHICALLY, SO THIS CIRCLE WILL
LOOK FAMILIAR TO PROBABLY ALL OF
YOU.
IT'S BUILT OFF OF A PDA CYCLE,
MASTER ADAPTIVE LEARNER CYCLE.
IT'S ADAPTED IN A COUPLE OF
DIFFERENT WAYS.
THE FIRST I IS THE TOP ARROW,
ANALYTICS.
WE GATHER DATA ABOUT
INDIVIDUALS, PROGRAMS OR
ORGANIZATIONS AND ANALYTICS ARE
PERFORMED ON THOSE DATA.
THOSE ANALYTICS REVEAL INSIGHTS.
UPON THOSE INSIGHTS WE DEVELOP
INTERVENTIONS THROUGH A PLANNING
PROCESS, INTERVENTIONS ARE
IMPLEMENTED, LEARNING OCCURS, WE
ASSESS BASED ON THOSE THINGS.
OUTCOMES ARE GENERATED AND THEN
YOU ADJUST AND YOU CONTINUE.
THE CORE OF THIS IS THE MIDDLE,
WHICH IS THE DATA.
THE DATA ARE THE ENGINE FOR
PRECISION EDUCATION SYSTEMS.
WE DESCRIBE DATA REALLY AS ALL
TYPES OF DATA THAT COULD BE
POSSIBLY COLLECTED ABOUT AN
INDIVIDUAL.
AS YOU PROBABLY ARE AWARE, THERE
ARE DATA COLLECTED ABOUT YOU
EVERY DAY.
CONTINUOUSLY REALLY.
THROUGH THE PHONES THAT YOU USE,
THROUGH THE EMR THAT YOU USE,
THROUGH LEARNING MANAGEMENT
SYSTEM, THROUGH THE FORMS AND
INFORMATION THAT YOU SUBMIT TO
THE GOVERNMENT OR TO PAYORS.
SO THERE ARE ABUNDANT AMOUNT OF
DATA THAT ARE COLLECTED.
THE GOAL HERE WOULD BE TO
LEVERAGE THAT DATA IN A USEFUL
WAY TO DO ASSESSMENT AS WELL AS
DEVELOPMENT.
WE PUT AROUND THAT DATA, AND
AGAIN, THE REASON -- THE
INDIVIDUAL, THE PROGRAM AND THE
ORGANIZATION, THERE'S A YELLOW
CIRCLE IN HERE THAT WE DESCRIBE
WE ARE ENTITIES AND PRODUCTS,
AND THE GOAL OF THIS LAYER, THE
REASON THAT LAYER IS THERE, IS
THIS IS THE LAYER THROUGH WHICH
DATA ARE CONSUMED.
SO THE DATA ARE COLLECTED IN A
VARIETY OF DIFFERENT AREAS,
REALLY FRAGMENTED IN THIS
COUNTRY, THE DIFFERENT PLACES
THAT DATA ARE COLLECTED AND
HELD, BUT THEY'RE CONSUMED
THROUGH ENTITIES AND PRODUCTS.
SO ENTITIES WOULD BE THE NIH
CLINICAL CENTER AS AN EXAMPLE.
A PRODUCT WOULD BE THE LEARNING
MANAGEMENT SYSTEM OR THE
ELECTRONIC MEDICAL RECORD THAT
YOU HAVE.
THAT'S HOW DATA ARE ACTUALLY
COLLECTED, CONSUMED AND SHARED
WITH OTHERS.
SO THE ENTIRE PRECISION
EDUCATION SYSTEM RELIES ON ALL
OF THIS.
THE GOAL WOULD BE TO LEVERAGE
ALL OF THIS, AGAIN, DEPLOY TO
DEVELOPMENT AND ASSESSMENT OF
LEARNERS.
THERE'S A LOT OF PRINCIPLES AND
STANDARDS WE NEED TO ESTABLISH
IN THIS WORK TO ENSURE EQUITY,
TO ENSURE THAT LEARNERS DO HAVE
AGENCY AND STANDARDS TO MAKE
SURE THAT IT'S SAFE FOR
EVERYBODY TO USE, AND YOU'VE ALL
I'M SURE READ ABOUT THIS IN
RELATION TO A.I.
IT'S SIMILAR IN TERMS OF THE
THINGS THAT WE THINK ABOUT IN
PRECISION EDUCATION.
IF YOU TURN THAT ON ITS SLIDE,
THE LAST SLIDE BEFORE I GIVE YOU
EXAMPLES, ONE IMPORTANT CONCEPT
WE THINK ABOUT WITH PRECISION
EDUCATION IS IT'S NOT STATIC,
IT'S NOT ONE POINT IN TIME, BUT
REALLY FOR US TO ACHIEVE
EFFECTIVE LIFELONG LEARNING,
THIS IS ACTUALLY A LONGITUDAL
CONTINUOUS PROCESS.
THE IDEA HERE IS THAT YOU
TRAVERSE FROM UME TO GME TO
CONTINUOUS PRACTICE AND DATA ARE
SHARED BIDIRECTIONALLY SO THE
MEDICAL SCHOOL FROM WHICH YOU'VE
GRADUATED WILL LEARN ABOUT THE
OUTCOMES THAT YOU GENERATE AS A
PRACTICING PHYSICIAN.
OR THE RESIDENCY PROGRAM THAT
YOU JOIN.
THOSE DATA THEN CAN HELP DRIVE
CONTINUOUS PROCESS IMPROVEMENT
AND CHANGE THE WAY THAT THEY
DELIVER EDUCATION FOR THE
STUDENTS THAT CONTINUE TO COME
THROUGH.
AND EVEN MORE IMPORTANTLY, MOVE
UPSTREAM FROM THERE, SO THEY CAN
GENERATE INFORMATION ABOUT HOW
TO SELECT FOR PEOPLE IN A WAY
THAT YOU THINK ARE MORE
EFFECTIVE FOR THEIR MISSION OR
FOR THE PHYSICIAN WORKFORCE THAT
THEY WANT TO HELP DEVELOP.
SO AGAIN THIS BIDIRECTIONAL
LONGITUDAL ASPECT OF PRECISION
EDUCATION, WE FEEL IS VERY
IMPORTANT.
SO NOW I'M GOING TO TALK A
LITTLE BIT ABOUT HOW.
THIS HAS REALLY TAKEN OFF, I
WOULD SAY, THERE ARE SILOS, THIS
IS A HANDFUL, I KNOW THESE
ACRONYMS WON'T MAKE SENSE TO
YOU, BUT THE GOAL OF PUTTING
THIS ON THE SLIDE AND SHARING IT
IS TO SHOW YOU THAT THERE IS
ACTIVITY IN THIS SPACE.
SO ACROSS THE TOP IS THE
COMPARTMENTS OF EDUCATION, UME,
GME AND CONTINUOUS PRACTICE,
ORGANIZATIONS, PROGRAMS AND
INDIVIDUALS AT THE BOTTOM.
YOU CAN SEE EACH OF THESE BULLET
POINTS ARE INITIATIVES IN
PRECISION EDUCATION THAT WE ARE
HELPING SUPPORT.
I'M GOING TO HIGHLIGHT THE ONES
THAT ARE IN GREY, AND THIS IS A
SUBSET OF WHAT'S OCCURING IN THE
COUNTRY.
ONE THING I WANT TO HIGHLIGHT
HERE IS YOU CAN SEE THERE'S A
LOT OF ACTIVITY BUT THESE
ACTIVITY ARE OCCURRING AND BEING
DEVELOPED IN, NOT SU SURPRISING,
HIGH RESOURCED INSTITUTION IN
THE COUNTRY.
SO ONE OF THE GOALS WE HAVE AT
THE AMA IN HELPING TO NURTURE
THIS IS TO ENSURE AT THE OUTSET
THAT WE DEVELOP PRINCIPLES AND
STANDARDS FOR THIS WORK SO THAT
WHAT IS CREATED CAN BE A PUBLIC
GOOD.
THAT THERE'S INTEROPERABILITY,
THAT WHATEVER IS CREATED AS AN
INSTITUTION THAT HAS A LOT OF
RESOURCE CAN BE REPLICATED AND
SCALED IN INSTITUTIONS THAT HAVE
LESS RESOURCE.
ULTIMATELY THIS IS TO DEVELOP A
PHYSICIAN WORKFORCE SO IT'S
IMPORTANT THAT WE DO THAT.
SO I'M GOING TO SHARE WITH YOU,
AGAIN, BECAUSE THIS IS
GME-RELATED AUDIENCE PRIMARILY,
I'M GOING TO FOCUS ON GME.
YOU CAN SEE A LOT OF THE GREY IN
THE GME AND ONE EXAMPLE IN
CONTINUING PRACTICE AS WELL, AND
THEN HOPEFULLY THESE WILL
GENERATE SOME IMAGINATION IN
TERMS OF WHAT'S POSSIBLE WITH
PRECISION EDUCATION.
SO I'VE STATED EACH OF THESE
EXAMPLES AS A PROBLEM THAT THAT
IB OVATINNOVATION IS TRYING TO .
SO THE FIRST IS FIVE, I'M GOING
TO SHARE FIVE WITH YOU, IS
SELECTION INTO RESIDENCY
PROGRAMS.
YOU CAN OBVIOUSLY IMAGINE THAT
SELECTION INTO FELLOWSHIP
PROGRAMS AS WELL.
HOW DO YOU POSSIBLY SCREEN
THOUSANDS OF APPLICATIONS?
THESE NUMBERS ARE JUST
INCREASING EACH AND EVERY YEAR
THAT WE GO THROUGH THIS.
AND THE OPPORTUNITY HERE IS
THROUGH NYU USING A.I. TO
AUGMENT RECRUITMENT, AND I'VE
HIGHLIGHTED HERE DR. JESSI RAFEL
WHO HAS LED THIS WORK.
IDEA HERE, THIS WAS DONE AT THE
NYU INTERNAL RESIDENCY PROGRAM.
THE RES DEN SEEP PROGRAM
RECEIVES THOUSANDS OF
APPLICATIONS AND THERE'S A
PROGRAM DIRECTOR REVIEW, THE
GOAL IS TRYING TO BE HOLISTIC
AND THEY RECOGNIZE AND
UNDERSTOOD THAT THEY WERE
MISSING PEOPLE.
SO HOW MIGHT THEY USE A.I. NOT
TO RELAYS THEM, THEMSELVES, BUT
TO ENHANCE THEIR SYSTEM AND
THEIR PROCESS SUCH THAT THEY
DON'T MISS PEOPLE BECAUSE OF
THEIR MANUAL REVIEW.
SO WHAT THEY DO IS THIS IS AN
IDEA THAT THERE'S A DIAMOND IN
THE RUFF, SO WHAT THEY DID IS
THEY TOOK HISTORICAL DATA ON WHO
THEY SELECTED, SO THIS IS NOT
THE RIGHT APPLICANT OR THE WRONG
APPLICANT, IT'S JUST BASED ON
WHO THEY SELECTED IN THE PATION.
SO THEY USED THAT INFORMATION,
THEY FED THAT INTO AN A.I.
ENGINE AND THEN THEY TOOK THEIR
UPCOMING YEARS OF APPLICANTS AND
THEY FED IT INTO THE A.I. ENGINE
AND THIS IS THE TYPE OF GRAPH
THAT'S DISTRIBUTED.
WHAT THE AI ENGINE PUTS OUT HERE
ARE BASED ON HISTORICAL
INFORMATION AND THE ALGORITHM
IT'S CREATED.
ON THE LEFT ARE THE PEOPLE THE
ALGORITHM WOULD SAY WE SHOULD
NOT INVITE FOR INTERVIEW AND INN
THE RIGHT IN ORANGE ARE THE ONES
THE ALGORITHM SAYS WE SHOULD
INVITE TO INTERVIEW.
ON THE WHITE, IF YOU GUYS CAN
SEE THE WHITE LITTLE BAR IN THE
ORANGE, THOSE ARE AM KANGTS THAT
THE COMMITTEE HAS NOT EXTENDED
AN INTERVIEW TO.
SO THE IDEA IS TO HIGHLIGHT
PEOPLE THAT THE ALGORITHM WOULD
SAY BASED ON HE HAVE BEEN YOU
TYPICALLY INTERVIEW, YOU WOULD
HAVE SELECTED THIS PERSON, BUT
YOU DEPARTMENT.
SO DO YOU WANT TO TAKE A LOOK AT
THAT PERSON?
SO THEN CAN YOU ACTUALLY DOUBLE
CLICK, YOU CAN SEE EXACTLY WHO
THOSE PEOPLE ARE.
THIS PERSON WAS NOT INVITED
ALTHOUGH THE ALGORITHM SAYS THEY
SHOULD HAVE BEEN INVITED BASED
ON HISTORICAL INFORMATION.
THEY'LL TELL YOU WHY THEY SAID
THIS PERSON WOULD HAVE BEEN
INVITED BASED ON HISTORICAL
DATA.
YOU CAN STILL MAKE YOUR OWN
DECISION, MAYBE THERE'S
SOMETHING UNIQUE THAT WAS NOT
CAPTURED, THIS IS SOMEBODY YOU
MAY HAVE MISSED.
THEY THEN ADDED THIS TO -- THEY
ADDED INTO THIS NATURAL LANGUAGE
PROCESSING OF THE PERSONAL
STATEMENTS IN THE APPLICATION.
AND THEY CAN LOOK FOR KEYWORDS
THAT ARE MISSION-FOCUSED OR
MISSION-ALIGNED FOR NYU.
WHEN THEY DO THAT, THEY INCREASE
THE PERFORMANCE OF THIS ENGINE.
SO AGAIN THE IDEA IS YOU CAN
START TO HOPEFULLY IMPROVE
HOLISTIC REVIEW, WHEN IT'S
MECHANICALLY ALMOST IMPOSSIBLE,
AGAIN HAVING GONE THROUGH THIS
FOR A DECADE MYSELF, IT IS
IMPOSSIBLE TO BE ABLE TO DO
THIS.
EVERY YEAR THAT I WAS PROGRAM
DIRECTOR, WE WOULD GET AN EMAIL
FROM AN APPLICANT SAYING CAN YOU
TAKE A LOOK, I'D REALLY LIKE TO
BE CONSIDERED, AND YOU TAKE A
LOOK AND YOU HAVE NO EXPLANATION
FOR WHY THEY WEREN'T
INTERVIEWED, NONE.
SOMETIMES THAT PERSON GOES ON
AND BECOMES THE CHIEF RESIDENT.
SO YOU REALIZE YOU HAVE AN
IMPERFECT FEBT SYSTEM.
SYSTEM SO HOPEFULLY
THAT WILL START TO BOOST THAT
PROCESS.
NUMBER TWO, THE RESIDENCY
PROGRAMS OR THE FELLOWSHIP
PROGRAMS ARE MEANT TO IMPROVE
CLINICAL SKILLS AND IMPROVE
WELL-BEING AS YOU DO IT BUT WE
DON'T KNOW ON THE ACTIVITIES
THAT YOU HAVE IF THEY CORRELATE
TO THAT OR NOT.
WE DESIGN ACTIVITIES BASED ON A
LOT OF DIFFERENT CLINICAL KNEE,
BASED ON SCHEDULES, BASED ON
NEEDS THAT ARE NOT RELATED TO
DEVELOPMENT OF THE PERSON.
AND HOPE THAT THEY DO ACHIEVE
CLINICAL SKILL AND WELL-BEING
IMPROVEMENT BUT WE'RE NOT SURE
ABOUT THAT.
ONE WAY TO DO THAT IS TO DO
REALTIME DATA COLLECTION.
SO THE IDEA HERE IS TO SEE WHERE
PEOPLE ARE IN THE HOSPITAL.
THIS IS A HOSPITAL-BASED
INTERVENTION.
AND BASED ON THEIR LOCATION, YOU
CAN MAKE HYPOTHESES.
FOR EXAMPLE, YOU CAN MAKE THE
HYPOTHESIS THAT IF SOMEBODY IS
WITH PATIENTS MORE OFTEN, THEN
THAT WOULD IMPROVE THEIR
CLINICAL SKILLS.
THAT COULD BE A HYPOTHESIS THAT
SOMEBODY COULD REASONABLY MAKE.
OR IF THEY'RE WITH PATIENTS MORE
OFTEN, THAT MIGHT IMPROVE THEIR
EELK AS OPPOSED TO BEING IN FROT
OF A COMPUTER.
-- YOU HIRE MEDICAL STUDENTS
AND SEND THEM AROUND, WE
ACTUALLY RAN THE LARGEST ONE
THAT'S BEEN PUBLISHED IN THE
COUNTRY, AND THAT HAD 2,000
HOURS OF OBSERVATION, BECAUSE
IT'S REALLY HARD TO TRAIN
MEDICAL STUDENTS AND HAVE THEM
FOLLOW PEOPLE AROUND.
BUT IF YOU DO THIS WITH REALTIME
DATA COLLECTION, THIS WAS DONE
WHICH THEBY THE WAY AT HOPKINS,D
AND UAB, YOU CAN COLLECT
HUNDREDS OF THOUSANDS OF HOURS,
CONTINUOUSLY.
AND SO THIS WAS DONE WITH THE
BADGE THAT PEOPLE WOULD WEAR IN
THE HOSPITAL, THERE ARE THESE
SENSORS THAT MONITOR THESE
BADGES.
THEY'RE DESIGNED FOR HIGH
CAPITAL COST EQUIPMENT OR FOR
ULTRASOUND MACHINES OR FOR OTHER
PEOPLE THAT ARE BEING MONITORED.
THEY DID THIS FOR INTERNS IN
THEIR RESIDENCY PROGRAM.
YOU CAN SEE 43 INTERNS, HUNDREDS
OF THOUSANDS OF HOURS BY NOW,
AND IT PRODUCES GRAPHS LIKE
THIS.
THE IDEA IS, AGAIN, CREATE DATA
THAT IS ABUNDANT, HIGH VOLUME
AND PRECISE THAT CAN BE
ACTIONABLE.
SO I'LL TRY TO DESCRIBE HOW THIS
COULD BE ACTIONABLE.
SO THESE ARE HOW INTERNS SPEND
THEIR TIME.
ACTUALLY 13% WAS AT BEDSIDE.
IF YOU LOOK AT THE TOP ROW ONLY,
YELLOW IS PATIENT ROOM, THAT'S
13% OF THEIR TIME THAT'S SPENT
AT THE BEDSIDE.
AGAIN, MANY PEOPLE WOULD ARGUE
THAT THAT IS SAD, THAT THAT'S
HOW MUCH TIME WE SPEND AT THE
BEDSIDE.
THIS ACTUALLY CORRELATES TO THE
TIME MOTION STUDIES THAT HAVE
BEEN DONE HISTORICALLY BUT THIS
IS NOT NECESSARILY ACTIONABLE.
HOW DO YOU INTERVENE UPON THIS
AND SEE IF IT'S WORKING OR NOT.
SO ONE NEXT LEVEL OF DATA
BECAUSE YOU COLLECT THIS
CONTINUOUSLY IS YOU CAN SEE IF
THERE'S VARIATION IN THIS DATA
AND IN FACT THERE ARE.
EACH OF THESE BARS IS ONE
INTERN.
YOU CAN SEE THAT THERE'S
VARIATION FROM THE PERSON THAT
SPENDS THE LEAST AMOUNT OF TIME
AT THE BEDSIDE, # #%, 9%, TO THE
PERSON THAT SPENDS 18%, DOUBLE.
SO THERE IS OPPORTUNITY WITHIN
THAT STRUCTURE FOR DIFFERENT
AMOUNTS OF TIME AT THE BEDSIDE.
IF EVERYBODY GOES TO 18%, THAT'S
ALMOST 400 HOURS MORE OF TIME AT
THE BEDSIDE THROUGH THE YEAR.
AGAIN -- YOU CAN GET A LITTLE
BIT FURTHER DETAIL, YOU CAN
BREAK IT DOWN BY SERVICE AS
WELL.
SO THIS IS THE GENERAL MEDICINE
WARDS.
CAN YOU SEE HERE THE 13% WHICH
IS THE PATIENT ROOM, THEY SPENT
THIS MUCH TIME IN THE HALLWAY,
THIS YOU MUCH TIME IN THE WORK
ROOM, COMPARE THAT TO THE
SPECIALTY SERVICE WHERE IT MUCH
MORE CONCENTRATED, A LOT LESS
VARIABILITY IN THE SPECIALTY
SERVICES IN TERMS OF WHERE THEY
SPEND THEIR TIME.
AGAIN INTERESTING, THEN THE
HYPOTHESIS OR IDEA HERE WAS HOW
DO WE CHANGE THIS?
WE CAN CHANGE THIS BECAUSE
THERE'S A TIME AND A DAY WHERE
WE CONTROL WHERE PEOPLE ARE, AND
THAT'S ROUNDS.
SO LET'S LOOK AT ROUNDS.
YOU CAN DO THAT BECAUSE YOU CAN
BREAK THESE DATA DOWN ANY WAY
YOU WISH TO, AND THIS IS WHAT
HAPPENS ON ROUNDS.
YOU CAN SEE THAT ON THE GENERAL
MEDICINE WARDS, THERE'S THIS
AMOUNT OF TIME AT THE BEDSIDE, A
LOT OF VARIABILITY AGAIN, SOME
PEOPLE IN THE WARD HALL, SOME
PEOPLE IN THE WORK ROOM.
SPECIALTY SERVICE, THEY'RE
ROUNDING IN THE WORK ROOM
PRIMARILY.
THIS IS AN OPPORTUNITY FOR
INTERVENTION.
SO YOU CAN INTERVENE, YOU CAN
MAKE A DECISION THAT I BELIEVE
ROUNDING AT THE BEDSIDE IS
BETTER, IT'S A BELIEF THAT YOU
CAN HAVE, BUT NOW BECAUSE WE
HAVE CONTINUOUS PRECISION
EDUCATION SYSTEM, CAN YOU
ACTUALLY DO THAT AND A TWO WEEKS
LATER WOULD YOU KNOW IF IT'S
ACTUALLY WORKING OR NOT, ARE
THEY SPENDING MORE TIME AT THE
PATIENT ROOM OR NOT.
THAT WOULD GIVE YOU LOAK, AND
THE GOAL THEN IS TO CORRELATE
THAT TO WELL-BEING.
SO THEY MONITOR -- THIS IS EARLY
SO WE DON'T HAVE THESE DATA YET,
BUT THEY ALSO ARE NOW COLLECTING
CORRELATIONS OF LOCATION TO
CLINICAL SKILL AND CORRELATIONS
OF DATA TO WELL-BEING INDICES,
WHICH THEY MEASURE EVERY TWO
WEEKS.
SO THEN YOU CAN NOW HAVE -- IT'S
REALLY HARD TO TELL HOSPITALS
AND THE HOSPITAL ADMINISTRATION
THAT WE ARE GOING TO SPEND THIS
MUCH TIME ROUNDING OR THIS MUCH
RESOURCE ON ROUNDS DURING THE
MORNING BECAUSE OF THE FORCES TO
TRY TO GET PEOPLE OUT OF THE
HOSPITAL, BUT IF YOU HAVE DATA
TO SHOW THAT IT IMPROVES
CLINICAL CARE OR DATA THAT
IMPROVES WELL-BEING, THOSE ARE
THE TYPE OF INFORMATION I THINK
ARE IMPORTANT TO HAVE IN THOSE
CONVERSATIONS.
AND THE TRANSFERRING AGENCY TO
LEARNERS, SO EACH INTERN GETS
THIS KIND OF REPORT.
WE DON'T KNOW WHAT THE RIGHT
AMOUNT OF TIME IS AT THE BEDSIDE
BECAUSE YOU COULD ARGUE IF I
SPEND TWICE AS MUCH TIME AS
ANYBODY ELSE, MAYBE I'M NOT AS
EFFICIENT, MAYBE I DON'T KNOW
HOW DO THAT WELL.
BUT IT'S PROBABLY IMPORTANT YOU
KNOW YOU SPEND TWO TIMES THE
AMOUNT OF TIME AT THE BEDSIDE
THAN YOUR PEERS DO.
SO THAT'S THE TYPE OF REPORT
THAT THEY GET, THAT SAYS HOW
MUCH TIME THEY SPEND COMPARED TO
THE REST OF THE PROGRAM.
AND YOU CAN USE THAT TO MAKE
INTERVENTIONS.
YOU'D USE A COACH AND PROCESS
THIS INFORMATION.
THE PRELIMINARY DATA WHICH
HASN'T BEEN PUBLISHED YET IS
PERCEIVED TIME AT BEDSIDE DOES
CORRELATE TO WELL-BEING.
SO WE KNOW THERE IS SOME SIGNAL
HERE.
WE NEED TO UNDERSTAND IT BETTER.
AGAIN, FOLLOWING PEOPLE AROUND
IS NOT GOING TO HELP US
UNDERSTAND THIS BETTER.
WELL-BEING IS ONE OF THOSE
REALLY STUBBORN PROBLEMS THAT WE
NEED TO UNDERSTAND THE ROOT
CAUSES FOR AND IF WE CAN FIND
INTERVENTIONS THAT GIVE US
MEANING THEN WE CAN INVEST IN
THEM.
THE THIRD ONE I WILL DESCRIBE IS
CLINICAL COMPETENCY IN A
TEAM-BASED MODEL.
SO ALL OF US WORK IN TEAMS.
IT'S REALLY HARD, PROBABLY ALL
OF YOU SUPERVISE RESIDENTS AND
FELLOWS IN TEAMS AS WELL.
IT'S HARD TO UNDERSTAND THE
COMPETENCY OF AN INDIVIDUAL THE
NEXT DAY WHEN YOU SEE WHAT'S
BEEN HAPPENING WITH THE PATIENTS
THAT THEY'VE CARED FOR BECAUSE
YOU RECOGNIZE THAT THERE ARE
TEAMS THAT HAVE BEEN WORKING ON
THAT CARE.
SO HOW DO YOU CREATE
ATTRIBUTION?
SO THIS IS A PROBLEM AGAIN FOR
DECADES PEOPLE HAVE BEEN
THINKING ABOUT.
AND IT VERY DIFFICULT TO DO.
HOW DO I CREATE ATTRIBUTION FOR
THE CARE THAT'S BEEN PROVIDED
AND ATTRIBUTE THAT TO ONE
PERSON.
SO NOW AGAIN WITH TECHNOLOGY AND
DATA IN PRECISION EDUCATION
SYSTEMS WE CAN START TO DO IS
THAT USING INFORMATICS.
THIS WORK HAS BEEN DONE AT
CINCINNATI, AT NYU AND AT
STANFORD AND THESE ARE SOME OF
THE PRINCIPLE INVESTIGATORS
WORKING ON THIS PROJECT.
AND THIS WORK WAS DONE LOOKING
AT PATIENTS WHO WERE ADMITTED TO
THE HOSPITAL WITH COMPLICATIONS
OF DIABETES.
AND THE IDEA WAS TO TRY TO
UNDERSTAND THE COMPETENCY OF AN
INDIVIDUAL IN THE MANAGEMENT OF
A PATIENT WHO COMES IN WITH A
COMPLICATION OF DIABETES THAT
REQUIRES HOSPITALIZATION.
SO THESE ARE THE TYPES OF
PATIENTS, JUST GRAPHICALLY
THERE'S PATIENTS THAT COME IN
DWIRDIFFERENT VERITIES OF
DECOMPENSATED DIABETES.
THEY GET ADMITTED TO MEDICINE
AND THE WAY THAT THEY SET THIS
UP IS THAT THEY UNDERSTAND THIS
WAS SET UP IN ONE HOSPITAL, THIS
WAS ACTUALLY AT UNIVERSITY OF
CINCINNATI.
AND THEY UNDERSTAND AT
UNIVERSITY OF CINCINNATI WHAT
THE TEAM STRUCTURE IS AND WHAT
THE BEHAVIORS ARE ON THAT TEAM.
WHEN SOMEBODY COMES IN ADMITTED
TO MEDICINE, THEY KNOW WHOEVER
PUTS IN THE ORDERS FOR THE FIRST
12 HOURS OF THAT PERSON'S STAY
IS THE PERSON THAT'S MAKING THE
DECISIONS FOR THAT PARTICULAR
PATIENT.
THAT PERSON IS -- SO THAT'S NOT
TRUE IN EVERY CIRCUMSTANCE.
SOMETIMES SOMEBODY ELSE IS
PUTTING IN ORDERS AND SENIOR
RESIDENT IS MAKING DECISIONS,
BUT THEY KNOW THIS TEAM AND THAT
PERSON THAT'S PUTTING IN THE
ADMISSION ORDERS IS MAKING THE
DECISION.
SO WHEN THIS PERSON COMES IN,
THERE'S AN ADMISSION ORDER
THAT'S CREATED.
WHEN THAT ADMISSION ORDER IS
CREATED THERE'S A TRAINEE THAT'S
ATTRIBUTED TO HAVING MADE
DECISIONS RELATED TO THE CARE.
AND THEY LOOK AT THE FIRST 12
HOURS, THEY SEE WHO'S MADE THE
PLURALITY OF ORDERS AND THAT
PERSON IS NOW ATTRIBUTED AS
MAKING DECISIONS FOR THIS
PATIENT.
THEY THEN COLLECT ALL THE
CLINICAL DATA FOR THE CARE OF
THE PATIENT THAT COMES THROUGH,
THAT'S SENT TO AN INFORMATICS
ENGINE AND BEHIND THAT, THEY
HAVE PROGRAMMED INTO THIS
INFORMATICS ENGINE WHAT IS
EXPECTED FOR THE MANAGEMENT OF A
PATIENT WITH DIABETES, FOR A
PGY1, A PGY2 AND A PGY3.
SO NOW YOU HAVE THE CLINICAL
DATA COME IN, YOU HAVE WHAT'S
EXPECTED, YOU KNOW WHAT LEVEL
YEAR THE ATTRIBUTED TRAINEE IS
AND YOU CROSS THAT AND IDENTIFY
GAPS.
SO FINALLY, YOU SEE EXACTLY
WHERE THEY'VE MET EXPECTATION
AND WHEN THEY FALL AWAY FROM
EXPECTATION.
I THINK IT'S EASY TO SAY, BUT I
THINK FOR ALL OF US WHO HAVE
SEEN AND HELPED COACH AND
DEVELOP TRAINEES, IT'S REALLY
HARD FOR YOU TO COLLECT THAT
TYPE OF INFORMATION AT A
GRANULAR LEVEL, PARTICULARLY ON
A CONTINUOUS LEVEL THAT'S
STANDARDIZED IN ANY PARTICULAR
WAY.
IT'S COMPLETELY RANDOM WHEN THEY
ADMIT SOMEBODY WITH ONE
CONDITION AND WHO WAS GOING TO
BE OBSERVING THEM AND WHAT KIND
OF INFORMATION YOU GET ABOUT
THEM.
SO THAT'S THE POWER.
THE POE WE ARE SYSTEM IS DONE
PASSIVELY, WE CALL THESE
NO-TOUCH ASSESSMENTS.
THIS IS JUST THROUGH THE
CLINICAL CARE THAT THEY DELIVER
FROM A NO-TOUCH ASSESSMENT, YOU
COLLECT THE DATA IN THE
BACKGROUND AND YOU'RE ABLE TO DO
ASSESSMENTS THAT ARE FAR MORE
FREMONT AND INFORMATIVE THAN WE
CAN DO WITH US AS SUPERVISORS.
SO THAT DATA, THE GAPS GO TO A
PERFORMANCE DASHBOARD AND THAT
GOES TO A COACH.
AGAIN THE PRECISION EDUCATION
MODEL WE HAVE RELIES ON
COACHING.
WE BELIEVE THAT YOU HAVE TO HAVE
COACHES TO BE ABLE TO INTERPRET
THESE DATA AND TO USE TO DEVELOP
SOMEBODY EVEN MORE EFFECTIVELY.
SO THIS WORKS AT -- THIS HAS
BEEN DEPLOYED AT THE UNIVERSITY
OF CINCINNATI, I SHOULD SAY.
I DON'T MEAN TO SUGGEST ALL OF
THESE THINGS ARE EASY TO DO.
THIS IS REALLY HARD TO DO.
IN FACT THE THREE UNIVERSITIES
THAT PUT THIS TOGETHER, THEY
COULDN'T REPLICATE IT AT THE
NEXT UNIVERSITY.
THAT'S HOW HARD IT IS TO
ACTUALLY DEVELOP THESE TYPES OF
ALGORITHMS AND DEPLOY THEM IN A
CLINICAL ENVIRONMENT.
THAT WILL GET BETTER, IT'S GOING
TO TAKE ALL OF US TO START TO
INVEST IN THIS EVEN MORE, BUT
THIS WORK NEEDS TO GET STARTED.
BUT THE POWER, AGAIN, IS THE
IDENTIFICATION AND CREATION OF
AGENCY IN A LEARNER, IN A TEAM,
THROUGHOUT THE YEAR.
AND I THINK, AGAIN, LOOKING
BACK, THIS TYPE OF DASHBOARD
WOULD HAVE BEEN INCREDIBLY
USEFUL FOR ME AS I WENT THROUGH
TRAINING.
PROCEDURAL TRAINING.
SO THIS IS ANOTHER REALLY
COMPLICATED PROBLEM.
HOW DO WE GIVE FEEDBACK FOR
SOMEONE THAT'S DOING A PROCED
PROCEDURE?
I WROTE HERE WHAT WE USED TO SAY
WHEN I TRAINED, SEE ONE, DO ONE,
TEACH ONE, I THINK MANY OF YOU
PROBABLY HAVE HEARD THAT.
I THINK MANY PLACES STILL HAVE
THAT CRITERIA THAT IF YOU
PERFORM FIVE CENTRAL VENOUS
CATHETER INSERTIONS, THEN CAN
YOU DO THEM.
MAGICALLY, IF YOU'VE DONE FIVE,
YOU ARE COMPETENT.
WE KNOW THAT'S NOT TRUE.
SO HOW DO WE START TO IMPROVE
USING OUR ASSESSMENT OF
PROCEDURES.
CAN YYO.YOU CAN DO THIS IN A SCE
WAY.
THIS WAS WORK DONE BY DR. PEW,
AND HER IDEA WAS TO TRACK MOTION
USING WEARABLES.
SO WEARABLES, AGAIN, ARE
BECOMING OBVIOUSLY MORE AND MORE
COMMONLY USED IN ASSEM ASSESSMED
IN SIMULATION.
YOU CAN SEE ON THE WHITE COAT
THE INDIVIDUAL IS WEARING AS
WELL AS IN THE GLOVE YOU CAN SEE
THE TRACKING DEVICES AND IT
CREATES THESE TYPES OF GRAPHICS
THAT YOU CAN -- THESE ARE THE
DATA THAT COME OUT OF A MOTION
TRACKING SYSTEM.
SO YOU CAN SEE THE LEFT HAND ON
THE LEFT SIDE, THIS IS CARDIAC
SURGERY THAT'S OCCURRING AND YOU
CAN SEE IN THREE DIMENSIONS
WHERE PEOPLE'S HANDS ARE MOVING.
SO YOU CAN USE THIS TO DO
ASSESSMENT FOR AND COACHING AND
DEVELOPMENT FOR PROCEDURAL
TRAINING.
SO THIS IS AN EXAMPLE THAT --
OUTPUT THAT CAN OCCUR THAT WOULD
HELP INFORM THE TRAINING OF AN
INDIVIDUAL.
SO THIS, AGAIN, A CARDIAC
SURGERY CASE, ON THE TOP IS THE
FELLOW, ON THE BOTTOM IS THE
FACULTY MEMBER, ON THE LEFT IS
THE LEFT HAPPENED, ON THE RIGHT
IS THE RIGHT HAND.
YOU'LL SEE LOTS OF LINES MOVING
AROUND.
SO I'LL POINT OUT TWO THINGS FOR
YOU.
THE FIRST IS THE LINES HAVE
DIFFERENT COLORS.
SO THE COLORS REPRESENT VERY R
VELOCITY.
SO IF YOU LOOK ON THE RIGHT
SIDE, YOU'LL SEE A LOT MORE
GREEN IN THE FELLOW AND THERE
ARE MOMENTS OF BLUE IN THE
FACULTY.
DR. PEW WILL DESCRIBE THIS AS
THE FACULTY MEMBER SLOWING DOWN
AT THE MOST CRITICAL PORTIONS OF
THIS PARTICULAR PROCEDURE.
SHOOL FURSHE'LL FURTHER SAY THET
THE FELLOW'S HAND DOES NOT SLOW
DOWN IS IMPERFECT SEPTEMBERI IM.
AGAIN, IMPERFECT ACCEPT TIBL
WHEN YOU'RE WATCHING THEM, BUT
SOMETHING THAT THEY CAN BE
COACHED ON AND TO DEVELOP TO BE
MORE EFFICIENT WITH THEIR HAND.
FINALLY, ON THE LEFT HAND,
YOU'LL SEE THAT THE FACULTY
MEMBER HAS MORE SPLAY IN THEIR
HAND THAN THE FELLOW.
AND HER INTERPRETATION OF THIS
IS THE FACULTY MEMBER IS USING
THEIR LEFT HAND WITH MORE
AGILITY TO SUPPORT THEIR RIGHT
HAND THAN THE FELLOW IS.
AGAIN, SOMETHING THAT IS
IMPERFSOMETHING YOU CAN HOPEFULE
THEM FEEDBACK ON.
CREATING HIGHLY INDIVIDUALIZED
FEEDBACK FOR SOMEBODY IN A
PRECISION EDUCATION SYSTEM IN
SURGERY.
AS SHE SCALED THIS TO -- SHE
GOES TO SURGICAL CONFERENCES AND
DOES HUNDREDS OF SIMULATED
SURGICAL SCENARIOS FOR HUNDREDS
OF PEOPLE AT A TIME.
WITH THAT, YOU'RE ABLE TO
ESTABLISH FOUNDATIONAL DATA THAT
CAN SET UP A BASE FOR CRITERIA
FOR WHAT ARE THRESHOLDS THAT WE
WANT TO ACHIEVE AS A SYSTEM IN
PARTICULAR PROCEDURES.
BECAUSE IT'S HARD TO DO THAT AS
INDIVIDUALS.
INDIVIDUALLY YOU CAN HELP THEM
GET A LITTLE BETTER BUT HERE YOU
CAN ACTUALLY SET UP STANDARDS
FOR SURGICAL TRAINING AND
CRITERIA FOR WHAT WE THINK IS
ACCEPTABLE FOR A THRESHOLD FOR
COMPETENCY.
NOW SHE'S USING THIS NOT JUST IN
THESE SETTINGS BUT FOR INTERNAL
MEDICINE PROCEDURES, ALSO NOW
WITH A.I. AND WITH SIMULATION
AND ABOUT B.R., STARTING TO IK
INCORPORATE THAT WHICH WILL MAKE
IT EVEN MORE POWERFUL.
FINALLY, THE WILD, WILD WEST OF
CLINICAL PRACTICE, UNSTRUCTURED
ENVIRONMENT WHERE THE RISKS I
THINK ARE THE GREATEST FOR US.
WE HAVE CME OBLIGATIONS, MOC
OBLIGATIONS, BUT MY EXPERIENCE
IN MOC IS THAT IT'S NOT
PATIENT-RELEVANT.
I TYPICALLY -- AND I'M HAPPY TO
REVEAL AND I DON'T THINK I'M
ALONE THAT WHEN I LOOK FOR MOC,
I LOOK FOR THE EASIEST THING FOR
ME TO COMPLETE BECAUSE I JUST
WANT TO GET IT DONE.
IT IS NOT DESIGNED IN ANY WAY
FOR ME TO ACTUALLY LEARN ABOUT
THE PATIENT I'M GOING TO SEE
TOMORROW.
AGAIN THIS IS A.I.-BASED
LEARNING PORTFOLIOS, WORK WE'VE
ACTUALLY GENERATED AT THE AMA.
THIS IS AN EXAMPLE OF HOW YOU
MIGHT THINK ABOUT LEARNING
STRATEGIES FOR CONTINUING
PRACTICE.
THESE ARE DIFFERENT -- YOU CAN
SEE MOC, WHICH IS THE Y AXIS
TIME-CONSUMING TO QUICK,
RELEVANT TO THE FIELD, AND
RELEVANT TO THE PATIENT
THEMSELVES THAT YOU'RE GOING TO
BE SEEING.
SO MOC IS QUITE TIME-CONSUMING,
NOT NECESSARILY THAT
PATIENT-RELEVANT.
YOU CAN SEE RE-CERTIFICATION
EXAMS OVER HERE, IT'S A VERY
TIME-CONSUMING FIELD BUT NOT
NECESSARILY FOR US.
WE USUALLY USE THESE PUSH AND
PULLS WHERE YOU GET EMAIL
ARTICLES OR YOU GO AND LOOK FOR
THEM YOURSELVES WHERE YOU'RE
POOLING THEM.
SO THIS IS THE TYPICAL SCENARIO,
NONE OF THESE ARE PARTICULARLY
PATIENT-RELEVANT.
THIS LEADS TO THIS IDEA OF MORAL
DISTRESS.
THIS IS ACTUALLY A MEASUREMENT
WE HAVE.
WHAT I'LL DESCRIBE, WE PILOTED
IN THE WORK LEADING UP TO THAT
PILOT WE ASKED PEOPLE ABOUT
THEIR CONTINUING ME MEDICAL
EDUCATION.
THERE IS MORAL DISTRESS AROUND
THE NOTION THAT YOU'RE UNABLE TO
LEARN IN A WAY THAT HELPS YOU
TREAT THE PATIENTS YOU'RE GOING
TO SEE TOMORROW EFFECTIVELY.
SO THAT CREATES DISSONANCE FOR
ALL OF US.
THE RECONNECT PILOT, THE ONE WE
GENERATED, IS DESIGNED TO
PROVIDE EDUCATE FOR YOU RELATED
TO THE PATIENTS YOU'RE GOING TO
SEE.
QUICKLY HOW IT WORKS, IT IS AN
A.I. ENGINE LINKED TO THE EMR.
THIS IS AN OUTPATIENT SCENARIO.
IT IS AWARE OF WHAT PATIENTS
YOU'RE GOING TO BE SEEING NEXT
WEEK, SO FOR ONE WEEK -- IT
DIGESTS ALL THAT INFORMATION.
PROBLEM I LIST, MEDICATION LIST,
HISTORY OF PATIENTS YOU'VE SEEN
BEFORE.
IT TAKES THAT INFORMATION AND
POINTS TO PUBMED, RIGHT NOW JUST
TO PUBMED BUT YOU CAN IMAGINE IT
CAN POINT TO A VARIETY OF
DIFFERENT DATABASES AND IT
CURATES FROM PUB MET ARTICLES
THAT IT THINK FOR YOU TO BE
USEFUL BEFORE THIS WEEK'S
PATIENTS YOU'RE GOING TO SEE.
IF IT'S A CONDITION YOU HAVEN'T
SEEN AT ALL OR VERY INFREESHTLY,
IT MIGHT POOL JUST GUIDELINES OR
ARTICLES FOR YOU.
IT SHARES IT WITH YOU IN A
CONSUMABLE WAY, YOU'LL DECIDE
WHETHER YOU WILL READ THEM OR
NOT.
THE IDEA HERE IS THAT IT PUSHES
THIS TO PATIENT RE RELEVANCE.
AND THE GOAL WOULD BE -- SO WE
PILOTED THIS AT ONE HEALTH
SYSTEM.
IN THAT ONE HEALTH SYSTEM, WE
LEARNED THAT PHYSICIANS ONLY
LOOKED AT THIS FOR 10 MINUTES,
BUT IN THOSE 10 MINUTES, THEY
THEN REPORTED THAT ON AVERAGE,
IT INFLUENCED 1 1/2 DECISIONS
THAT THEY MADE FOR THEIR
PATIENTS IN THE UPCOMING WEEK.
SO IT HAS INFLUENCE, AND THIS
WAS A REALLY RAW SYSTEM, VERY
POOR USER INTERFACE, BUT THE
IDEA IS YOU'RE ABLE TO DO THIS,
THEN YOU POINT IT NOT JUST TO
PUBMED WI BUT YOU POINT TO ACP F
YOU'RE INTERNAL MEDICINE, TO
OTHER DATABASES, AND FINALLY THE
AMERICAN BOARD OF MEDICAL
SPECIALTIES IS PARTNERING WITH
US AS WELL.
THE IDEA WOULD BE THAT YOU WOULD
GET MOC FOR THIS IF YOU CONSUME
THIS INFORMATION.
BECAUSE THAT IS WHAT MAINTENANCE
IS MEANT TO BE, CONTINUING
MEDICAL EDUCATION IS MEANT TO BE
RELEVANT INFORMATION AND IF YOU
DO IT YOU SHOULD PASSIVELY
RECEIVE CREDIT FOR DOING THIS SO
HOPEFULLY THESE PRECISION
EDUCATION SI TEMS CAN REPLACE
THE MOC NOW TO ONE THAT IS IN
THE SPIRIT OF LIFELONG LEARNING.
THAT WOULD BE THE GOAL.
SO AGAIN I'LL CONCLUDE THAT WE
HAVE A SYSTEM THAT WE THIN IS
TOO I INEFFICIENT, TOO VARIABLE,
OUR GOAL IS TO REDUCE THIS WITH
PRECISION EDUCATION SYSTEMS TO
MAKE IT EFFECTIVE FOR US TO
ACHIEVE LIFELONG LEARNING WITH
THE IDEA OF PRODUCING A
PHYSICIAN WORKFORCE THAT CAN
CAPABLY CARE FOR OUR PATIENTS,
OUR FAMILIES AND OUR
COMMUNITIES.
I'LL JUST CONCLUDE WITH WHAT
WE'VE DESCRIBED AS IMMEDIATE
NEEDS.
WE NEED TO CULTIVATE THIS MORE,
WE NEED TO THINK MORE ABOUT THE
THEORY AND DEVELOP
SCHOLARSHIPNESS.
WE'VE STARTED TO DO THIS, BUT
MORE IS DEFINITELY NEEDED, IT'S
A NASCENT FIELD.
AS I MENTIONED WE NEED
PRINCIPLES AND STANDARDS BECAUSE
OTHERWISE THE DEFAULT FUTURE
WILL BE THAT THOSE THAT HAVE
RESOURCE WILL HAVE DISRESOURCE
AND WE DON'T WANT THAT TO BE THE
FUTURE.
THE FUTURE NEEDS FOR THIS TO BE
A PUBLIC GOOD.
FINALLY WE WANT TO BUILD, SHARE
AND SCALE SYSTEMS TO ADDRESS
URGENT PROBLEMS AND OUR GOAL IS
TO START INITIATIVES TO DO THAT
AS WELL.
SO I THANK YOU FOR YOUR TIME.
I'M GOING TO PAUSE THERE AND I'M
HAPPY TO TAKE ANY QUESTIONS.
[APPLAUSE]
>> HI.
THANK YOU VERY MUCH FOR YOUR
TALK.
I THOUGHT THAT THE CONCEPT OF
USING DATA TO DRIVE EDUCATION IS
BOTH COMFORTING AS A SCIENTIST
AND IT ALSO BROUGHT UP A
QUESTION FOR ME, BECAUSE I
NOTICED IN YOUR INTRODUCTION,
WHEN YOU'RE TALKING ABOUT THE
ARCHETYPES, A LOT OF THE SOURCES
OF THE PLATEAUS OR THE STOPPAGE
POINTS WERE BECAUSE OF A LACK OF
SUPPORT, A LACK OF COMMUNITY,
AND I'M ALSO LIKE VERY AWARE
THAT MY SHIRT SAYS "IN THIS
TOGETHER," SO IT'S LIKE KIND OF
COINCIDENTAL.
BUT I'M CURIOUS WHAT THE AMA
RECOMMENDS IN ADDITION TO ALL OF
THESE LIKE INCREDIBLE -- THIS
INCREDIBLE PROGRESS IN PRECISION
EDUCATION, WHICH I CAN IMAGINE
CAN BECOME QUITE ISOLATING.
HOW CAN WE PROVIDE A COMMUNITY
OF SUPPORT TO PHYSICIANS BOTH IN
PRACTICE THROUGHOUT BUT ALSO
HERE AT NIH FOR OUR FELLOWS?
>> TERRIFIC.
THANK YOU FOR THAT QUESTION.
HOPEFULLY I'VE CONVEYED THAT
THIS IS A TOOL, PRECISION
EDUCATION SYSTEMS ARE TOOLS THAT
HELP US ENHANCE THE LEARNING
THAT WE PROVIDE.
IT IS NEVER, AT LEAST IN MY
VIEW, MEANT TO REPLACE THE
LEARNING THAT WE HAVE.
SO WHAT I BELIEVE IT DOES, IT
FILLS IN GAPS SO AGAIN IF WE
JUST REFLECT ON THE TRAINING
THAT PROBABLY EACH OF US HAVE
GONE THROUGH, I DESCRIBED PERSON
LEARNING ABOUT THEIR GAPS IN
MANAGING A PERSON WITH DIABETES.
SO THAT DOESN'T CHANGE WHAT YOU
WILL DO CLINICALLY, BUT IT
ENHANCES TREMENDOUSLY THE
INFORMATION YOU GET ABOUT HOW
YOU MIGHT IMPROVE.
THE OTHER WAY THAT I THINK IT
HELPS BUILD COMMUNITY IS BY
RELIANCE ON COACHING.
SO WE BELIEVE THAT YOU NEED TO
HAVE INTERACTION WITH OTHERS.
THIS IS NOT DONE IN ISOLATION
FOR YOU TO IMPROVE.
THE THIRD THING I WOULD MENTION
IS THAT USING DATA AND
PARTICULARLY USING TECHNOLOGY, I
THINK GENERATES COMMUNITY THAT
WE'RE UNABLE TO GENERATE TODAY.
WE KNOW THIS POST PANDEMIC,
WHERE WE'VE BEEN ISOLATED, WHERE
WE'RE NOT GATHERING.
I DON'T KNOW WHAT THE TYPICAL
AUDIENCE IS HERE, THIS IS
ACTUALLY QUITE A NICE AUDIENCE
IN THE PLACES I'VE GONE WHERE
THERE'S USUALLY JUST A HANDFUL
OF PEOPLE POST PANDEMIC.
SO TECHNOLOGY HAS HOPEFULLY IN
THIS NEW WORLD WILL BRING PEOPLE
TOGETHER THAT COULD BE DISTANT
PHYSICALLY, AND HELP GENERATE
COMMUNITY FURTHER.
THE LAST THING I'LL SAY IS, I
THINK THAT THERE'S MOWER POWER A
AND IF WE CAN LEVERAGE THAT DATA
AND THAT POWER TO CHANGE
CULTURE, AN FOR ME, THIS COMES
DOWN TO PRECISION EDUCATION
SYSTEMS, THE REAL CORE OF IT IS
AROUND ASSESSMENT.
SO IF WE CAN CHANGE THE CULTURE
OF ASSESSMENT, BECAUSE WE NOW
HAVE A SYSTEM WHERE WE
UNDERSTAND HOW TO TREAT YOU AS
AN INDIVIDUAL AND HOW WE FRAME
EDUCATION AS AN OPPORTUNITY TO
DEVELOP SOMEBODY, NOT AS AN
OPPORTUNITY TO SAY YOU'RE NOT
GOOD ENOUGH.
AND WE NEED DATA AND WE NEED
EQUITABLE ASSESSMENT FOR US TO
DO THAT WELL.
AND I THINK THAT WILL GENERATE
CULTURE CHANGE WHICH HOPEFULLY,
AGAIN, ENHANCES THE COMMUNITY
THAT YOU'RE DESCRIBING.
BUT FOR ME, AGAIN, IF WE BOIL
THIS DOWN TO ONE FUNDAMENTAL
CONCEPT, IT'S EQUITABLE
ASSESSMENT, WHICH WE DON'T HAVE
RIGHT NOW.
AND IT'S NECESSARY FOR US TO
ACHIEVE ANY OF THESE GOALS THAT
WE HAVE.
>> THANK YOU, SANJAY.
FANTASTIC TALK.
KIND OF BUILD ON THAT A BIT, IN
TERMS OF GETTING THAT EQUITABLE
DATA, I THINK THERE'S A TENSION
WHEN, FOR EXAMPLE, IN THOSE
TRANSITIONS, THOSE VULNERABLE
TIMES BEING ABLE TO
INDIVIDUALIZE KIND OF SOME OF
THE FEEDBACK AS THEY TRANSITION,
BUT HOW HAVE YOU OVERCOME I
THINK THE CONCERN THAT PEOPLE
WILL HAVE IS THAT YOU'RE GIVING
CANDID FEEDBACK, MEDICAL SCHOOL,
SOMEONE IS GROWING, YOU DON'T
WANT THAT TO BE SHARED, THAT
SHOULD BE CONFIDENTIAL AS THEY
GO TO RESIDENCY, THAT TENSION
ABOUT THAT BE USED NEGATIVELY AS
OPPOSED TO SUPPORTING THAT
TRANSITION?
>> YEAH, WE'VE SPENT THE LAST
SIX MONTHS CONVENING GROUPS
AROUND THIS TRANSITION FROM
MEDICAL SCHOOL TO RESIDENCY, AND
ONE OF THE FEEMS THEEMS THAT
COMES OUT VERY STRONGLY IS THIS
MISTRUST, BIDIRECTIONAL MISTR
MISTRUST, FROM LEARNS ABOUT HOW
THAT DATA IS GOING TO BE USED.
IT GETS BACK TO THE COMMENT I
WAS MAKING EARLIER WHERE IT
REVEALS THIS SYSTEM THAT WE HAVE
THAT IS SO CORRUPTED IN TERMS OF
CULTURE BUT THERE'S THIS
INCREDIBLE HIGH STAKES
ENVIRONMENT THAT CREATES A LOT
OF FALSE OR DISINGENUOUS
BEHAVIORS WHI BY MANY DIFFERENT
PEOPLE.
TO GET AWAY FROM THAT, I THINK
DATA IS ONE LEVER THAT WE CAN
OUUSE TO GET AWAY FROM THAT.
I THINK THE YO CULTURE CHANGE WL
LAG BECAUSE THAT MISTRUST IS SO
DEEPLY EMBEDDED IN OU OUR SYSTEM
RIGHT NOW.
THE WAY TO START IN OUR VIEW IS
TO START POST MATCH.
SO CAN WE CREATE INFORMATION
ABOUT AN INDIVIDUAL THAT IS
AUTHENTIC, THAT IS INFORMATIVE,
THAT IS THIS MIND OF -- OR AT
LEAST THE ATTITUDE OF ABUNDANCE
THAT THIS IS WHERE SOMEBODY CAN
STRETCH, AND CAN WE CREATE THAT
IN A WAY THAT'S STANDARDIZED AND
OBJECTIVE TO SHARE THE MOMENT
SOMEBODY MATCHES WITH THEIR
RESIDENCY PROGRAM.
WHERE HOPEFULLY THE STAKES ARE
NOW LOWER, BECAUSE THEY'RE
COMING FOR SURE, AND I KNOW AND
I THINK HERE TOO, YOU DON'T
REALLY LEARN ANYTHING ABOUT
SOMEBODY UNTIL THEY ARRIVE.
YOU MIGHT GET DEMOGRAPHIC
INFORMATION, BUT YOU DON'T LEARN
IMPORTANT THINGS ABOUT THEIR
DEVELOPMENT, IMPORTANT THINGS
ABOUT THEIR BACKGROUND,
IMPORTANT THINGS ABOUT THEIR
EMOTIONAL STATE, THEIR FINANCIAL
STATE.
THERE'S SO MANY THINGS THAT I
THINK COULD INFLUENCE HOW YOU
SUPPORT SOMEBODY AND TRANSITION
THEM INTO YOUR PROGRAM THAT WE
DO NOT COLLECT TODAY.
SO THAT'S AN OPPORTUNITY FOR US
TO SHARE MORE FREELY ACROSS THE
TRANSITION.
EVEN THAT IS HARD, THOUGH,
BECAUSE HOW DO YOU CREATE THOSE
ASSESSMENTS, BUT I THINK THAT'S
SOMETHING THAT WE'RE ACTUALLY
QUITE INTERESTED IN AND FOCUSING
ON, HOW MIGHT WE -- THEN HOW DO
YOU DO THAT FOR INTERNATIONAL
MEDICAL GRADUATES, FOR EXAMPLE?
HOW MIGHT WE USE THESE
TECH-ENABLED ASSESSMENTS TO
CREATE INTERPRETABLE DATA FOR
KEY COMPETENCIES TO SHARE FROM
THE POINT OF MATCH TO RESIDENCY
PROGRAM?
AND IF WE DO THAT, HOPEFULLY YOU
START TO GENERATE TRUST.
AND THEN YOU CAN WORK ON
UPSTREAM FROM THERE, BUT
STARTING UPSTREAM IS PROHIBITIVE
TO ME.
>> THIS SPEAKS TO SORT OF
EVIDENCE-BASED EDUCATION.
WE DO TRIALS AND ALL SORTS OF
RESEARCH HERE AT THE NIH AND
JUST THINKING ABOUT REALLY
APPLYING THOSE PRINCIPLES TO
EDUCATION IS REALLY KIND OF -- I
MEAN, MAKES A LOT OF SENSE.
I WAS WONDERING IF YOU COULD
SPEAK A LITTLE BIT MORE THAN
JUST NAMING THE PIs ON THESE
INITIATIVES ABOUT THE ACTUAL
TECHNOLOGY AND WHAT'S REQUIRED
TO IMPLEMENT THESE SORTS OF
THINGS.
I MEAN RANGING FROM THE EHR,
CLOUD SYSTEMS.
CAN YOU SPEAK THROUGH AN EXAMPLE
AND GIVE ALL THE STEPS THAT ARE
NEEDED TO IMPLEMENT SOMETHING
LIKE THIS?
>> SO I'LL DESCRIBE SORT OF THE
THE -- PROBABLY THE INSTITUTION
THAT'S FARTHEST AHEAD IN MY VIEW
IS NYU.
THEY HAVE DEVOTED MILLIONS OF
DOLLARS FOR THIS.
THIS IS A HIGH RESEARCH
INITIATIVE.
THEY HAVE SET UP -- THEY HAVE 20
PEOPLE IN THAT TEAM THAT ARE
THERE TO DESIGN PRECISION
EDUCATION SYSTEMS.
THEY CREATED A DATA WAREHOUSE
FOR ALL OF THEIR MEDICAL
STUDENTS.
THEY HAVE THE FORTUNATE
CIRCUMSTANCE OF 40% OF THEIR
MEDICAL STUDENTS STAY AT NYU FOR
RESIDENCY.
SO THEY HAVE AN OPPORTUNITY TO
SCALE ACROSS UME TO GME THAT'S
DIFFERENT THAN MOST PLACES BUT
THEY HAVE INFRASTRUCTURE.
THEY HAVE PARTNERED WITH OPEN AI
TO ACTUALLY GET GENERATIVE AI
INTO THEIR EMR.
THEY'VE DEVELOPED TOOLS SO THAT
WHEN A MEDICAL STUDENT ADMITS A
PATIENT ON A CLERKSHIP, THEN
THEY WOULD SEE THE DIAGNOSIS AND
THEN SEND JUST IN TIME TO THAT
MEDICAL STUDENT MODULES FOR THAT
DIAGNOSIS SO THEY CAN CONSUME
THAT BEFORE ROUNDS THE NEXT
MORNING.
SO THERE ARE A LOT OF STEPS, BUT
I THINK THERE IS A TECHNOLOGY
PIECE TO THAT IS PROBABLY THE
HARDEST OR AT LEAST THE HIGHEST
RESOURCE, BUT THEN THERE'S ALSO
THE PHYSICIAN LOADER SHIP FOR
THAT.
WHAT I'VE LEARNED IS THE BIGGEST
NEED IS FOR THE PHYSICIAN
EXECUTIVES TO BE CHAMPIONS FOR
THIS.
BECAUSE IT'S REALLY DIFFICULT TO
IMPLEMENT THIS WITHOUT A
PHYSICIAN CHAMPION THAT'S READY
TO SEE THIS AS THE FUTURE.
AND I THINK THE OBLIGATION IS ON
A LOT OF US IN EDUCATION TO MAKE
THE CASE TO THOSE PHYSICIAN
EXECUTIVES THAT THIS IS -- YES,
THIS IS EDUCATION, BUT THE
DOWNSTREAM BENEFIT OF THIS
EDUCATION IS OUR PATIENT
OUTCOMES.
THAT'S THE LINK THAT WE NEED TO
DO, WE NEED TO MAKE THAT
CONNECTION MORE EFFECTIVELY IN
MY VIEW FOR US TO GENERATE THE
ENTHUSIASM AND THE SUPPORT
BECAUSE THAT WILL COME DOWN TO
FINANCIAL SUPPORT ULTIMATELY.
BUT JUST BACKING OFF OF ALL OF
THAT, THAT'S WHY IT SITS IN HIGH
RESOURCE INSTITUTIONS TODAY.
OUR GOAL -- AND I THINK JUST
LIKE EVERYTHING ELSE THAT
STARTS, THE COST WILL COME DOWN.
SO THAT'S WHY ONE OF THE
URGENCIES WE FEEL IS TO CREATE
PRINCIPLES AND STANDARDS DONE IN
A WAY THAT'S INTEROPERABLE
BECAUSE THE COST WILL COME DOWN.
WHEN THEY COME DOWN, IF THERE'S
A THOUSAND DIFFERENT MECHANISMS
TO DO THIS, THEN THEY CAN'T BE
SHARED.
BUT IF WE'RE ABLE TO CREATE A
LANGUAGE NOW FOR HOW TO DO THIS,
WHEN COSTS DO COME DOWN, THEN WE
CAN REPLICATE AND SCALE THEM FOR
THE COUNTRY.
OUR GOAL, ACTUALLY INITIATIVES
WE'RE GOING TO INVEST IN, WE'RE
GOING TO DELIBERATELY SCALE THEM
IN LOWER RESOURCED INSTITUTIONS,
BECAUSE WE WANT TO PROVE THEY
CAN BE EFFECTIVE THERE.
>> WILL THIS BE EHR-AGNOSTIC
OR --
>> THESE ARE ALL EHR-AGNOSTIC.
>> EPIC RESEARCH OR SOMETHING
LIKE THAT.
>> THEY ALL SIT ON TOP, THEY'VE
WORKED WITH SERNER, EPIC, EVEN
SMALLER EHRs THAN THAT.
BUT IT TAKES AN INFO MA TISH --
IT DOES TAKE EXPERTISE LIKE THAT
TO DO THIS EFFECTIVELY.
>> I THINK THIS IS A GREAT TALK.
QUICK QUESTION ABOUT SORT OF THE
BIAS.
YOU MENTIONED FOR EXAMPLE DATA
AT NYU WHEN THEY'RE LOOKING AT
RESIDENCY APPLICATIONS SO
THEY'VE LOOKED AT WHAT THEY'VE
ALREADY ACCEPTED INTO THEIR
PROGRAM.
AS THE MODEL FOR WHICH THEY CAN
GO BACK AND THEN RE-SCREEN
CURRENT APPLICATIONS TO GET THE
PREVIOUS OUTCOMES IN A SENSE.
HOW DO YOU -- I MEAN, I THINK
ALL OF US, THAT'S SORT OF A
REPLICATION OF BIAS IN THE
ALGORITHM ITSELF.
SO YOU'RE USING YOUR PREVIOUS
DECISIONS TO JUSTIFY FUTURE ONES
IN A SYSTEM THAT YOU ALREADY
ADMIT IS IMPERFECT AND HAS
MISSED MANY POTENTIAL CANDIDATES
PREVIOUSLY.
THIS IS NOT AN ISSUE FOR THE AMA
NECESSARILY TO SOLVE BUT THERE
HAS TO BE SORT OF A LOT OF WORK
THOUGHT AROUND THE BIAS THAT HAS
ECK -- EXISTS AND HAS EXISTED IN
THE PATCH.
A.I. WILL GIVE YOU WHAT YOU'VE
PUT IN.
BUT THE QUESTION IS HOW DO WE
THINK DEEPER ABOUT CREATING NEW
VISIONS FOR WHAT WE WANT A.I. TO
HELP US ACHIEVE.
AND I DON'T KNOW SORT OF HOW TO
DO THAT.
I MEAN, OTHER THAN, OF COURSE,
USING PREVIOUS DATA, BUT I
THINK -- I DON'T KNOW IF NYU
TUNED THIS A LITTLE BIT TO SEE
IF THEY COULD GET AN OUTCOME
THAT THEY WANT AND OF COURSE THE
OUTCOME THEY WANT IS GOING TO BE
CLEARLY BIASED BY WHATEVER YOUR
INSTITUTION VALUES, BUT I THINK
THAT DEFINITELY IS SOMETHING
THAT I'M SURE EVERY PLACE THINKS
ABOUT AS THEY TRY TO IMPLEMENT
THESE SYSTEMS, I JUST DON'T KNOW
HOW THAT'S BEING SORTED OUT NOW.
>> THANK YOU FOR HIGHLIGHTING
THAT.
IF THEY WERE HERE, I THINK
THEY'D BE THE FIRST TO AFFIRM
WHAT YOU JUST DESCRIBED AND
RECOGNIZE THAT THAT BIAS -- THAT
WHAT I SHARED WAS A PROOF OF
CONCEPT TO SHOW THE MODEL COULD
WORK BUT THEY ABSOLUTELY
RECOGNIZE THAT THEY JUST --
AGAIN, PROOF OF CONCEPTS, THEY
RECOGNIZE WHATEVER THEY CREATED
IS GOING TO PROPAGATE THE BIAS
THAT WAS ALREADY INSTILLED
EARLIER.
SO THEY ARE AWARE OF THAT AND
ARE THINKING ABOUT HOW TO
DE-BIAS THE SYSTEM AS WELL.
BUT I THINK THE EQUITY ISSUE AND
THE BIAS ISSUE IS TOP OF MIND
FOR -- AT LEAST I'VE SEEN IN THE
PEOPLE THAT WE'VE WORKED WITH
TOP OF MIND FOR THOSE THAT ARE
INNOVATING IN THIS SPACE.
SO I DON'T HAVE THE SOLUTION FOR
YOU, BUT I THINK IT IS A
SENSITIVITY THAT EVERYONE SEEMS
TO SAY IS A HIGH PRIORITY.
BUT THANK YOU FOR HIGHLIGHTING
IT.
>> YEAH, THAT WAS I THINK REALLY
SO BEAUTIFULLY SAID AND
HIGHLIGHTED A LOT OF MY CONCERNS
SO JUST SORT OF A SIMILAR
FOLLOW-UP IS TO WHAT EXTENT DO
YOU ACTUALLY THINK -- COULD
THERE BE A RISK OF MINIMIZING
VARIABILITY ESPECIALLY WHEN THE
FUTURE ROLE OF US AS DOCTORS IS
SO UNCERTAIN IN THE HEALTHCARES
ISHEALTHCARESYSTEM, COULD YOU AY
SCRUB OUT A TYPE OF ARCHETYPE
THAT ACTUALLY TURNS OUT TO BE
REALLY, REALLY VALUABLE IN 15
YEARS?
>> I THINK ALL OF THAT TO BE
DETERMINED.
I HOPE PERSONALLY HOPE NOT.
I THINK THE POWER OF THIS, TO
ME, IS TO ACTUALLY STEP AWAY
FROM WHAT WE'VE EMPHASIZED
TODAY.
TODAY WE EMPHASIZE MEDICAL
KNOWLEDGE.
IF YOU LOOK AT WHAT DO YOU USE
TO GET IN RE RESIDENCY, CME SCO,
ONE OF THE REASONS IT'S
PASS/FAIL IS BECAUSE OF THE BIAS
IT INTRODUCES INTO THE SYSTEM.
IT'S NOT MEANT FOR THAT UM IN
BER ONE AND IT'S NOT
PREDECORATIVE FOR WHO IS GOING
TO SUCCEED IN RESIDENCY BUT
THOSE THINGS ARE GOING AWAY AND
IT'S LEFT BEHIND A VACUUM, SO
HOW DO WE LEARN ABOUT PEOPLE?
NOW I THINK WE ARE LEARNING THAT
COMMUNICATION, EMPATHY,
PROFESSIONALISM, UNDERSTANDING
SOCIAL DRIVERS, THERE ARE ALL OF
THESE OTHER COMPETENCIES THAT
WE'RE STARTING TO UNDERSTAND
MATTER FOR A PHYSICIAN TO
THRIVE.
SO MY HOPE IS THAT WE ARE ABLE
TO ACTUALLY CREATE OR USE THESE
TOOLS TO ASSESS AGAINST THOSE,
IDEALLY DEVELOP AGAINST THOSE
AND BRING PEOPLE IN LOOKING AT
THOSE OBJECTIVELY WHEREAS TODAY,
WITHOUT THOSE DATA, AGAIN, WITH
THE VACUUM AROUND ASSESSMENT FOR
THOSE PARTICULAR ATTRIBUTES,
IT'S IMPOSSIBLE TO DO THAT, AT
LEAST EFFECTIVELY.
WE TALKED ABOUT HOLISTIC REVIEW,
BUT MY VIEW IS THAT WE'RE NOT
ACHIEVING THAT EFFECTIVELY YET.
SO I THINK IT'S A REALLY
IMPORTANT POINT.
AND I THINK AS WE EVOLVE
FURTHER, WE'LL SEE HOW THIS GOES
BUT MY -- I'M OPTIMISTIC THAT
THIS WILL ELEVATE ATTRIBUTES
THAT WERE NOT -- THAT ARE BEING
NEGLECTED AND THAT WE THINK DO
MATTER.
BECAUSE WE'RE UNABLE TO MEASURE
THEM.
THANK YOU GUYS VERY MUCH.
[APPLAUSE]