IT'S MY PLEASURE TO WELCOME YOU
TO OUR WORKSHOP, THE NIH
WORKSHOP TOWARD AN ETHICAL
FRAMEWORK FOR ARTIFICIAL
INTELLIGENCE IN BIOMEDICAL AND
BEHAVIORAL RESEARCH,
TRANSPARENCY FOR DATA AND MODEL
REUSE, THINKING ABOUT A
FRAMEWORK FOR ETHICAL A.I. AND
TODAY ATTACKING HOW TO BUILD IN
TRANSPARENCY AS A FIRST STEP TO
BUILDING OUT ETHICAL A.I.
WE HAVE COFFEE IN THE CAFETERIA,
PLEASE HELP YOURSELF, THAT
SHOULD BE AVAILABLE THROUGHOUT
THE DAY.
BATHROOMS, GO TO THE LOBBY AND
HEAD TOWARD THAT DOOR, AND
FOLLOW AROUND TO THE RIGHT OR
LEFT DEPENDING WHICH YOU NEED.
BREAK YOITS WILL BE BOTH --
BREAKOUTS WILL BE ON THIS FLOOR
AND SECOND FLOOR, STAY TUNED FOR
ADJUSTMENTS TO THE HANDOUTS
RIGHT BEFORE WE BREAK.
FOR THE SPEAKERS, MY COLLEAGUE
JESSICA IN THE FRONT WILL GIVE
YOU A WARNING IF YOU'RE ABOUT TO
HIT YOUR TIME LIMIT.
IT'S MY PLEASURE TO INTRODUCE
DR. SUSAN GREGURICK, ASSOCIATE
DIRECTOR FOR DATA SCIENCE FOR
NIH, ALSO MY BOSS AND DIRECTOR
OF THE OFFICE OF DATA SCIENCE
STRATEGY, I'M SO PLEASED SHE'S
HERE TO TALK ABOUT HOW THIS
MEETING FITS WITH THE BROADER
PICTURE AND GOALS OF NIH.
SUSAN?
[APPLAUSE]
>> THANK YOU, LAURA.
GOSH, IT'S SO NICE TO BE HERE,
SO NICE TO SEE SO MANY FRIENDS
AND COLLEAGUES I HAVEN'T SEEN IN
YEARS, LOOKING AT YOU, TOM, DAN.
IT'S LIKE SO NICE.
TO WELCOME YOU TO OUR WORKSHOP.
LAURA SAID I'M HER BOSS, I THINK
OF IT AS A COLLEAGUE.
WE HAVE A WONDERFUL SMALL TEAM.
I'LL TELL YOU ABOUT THE WORK
WRIT LARGE AND HOW THE WORK FITS
INTO A LARGER PERSPECTIVE WITH
A.I. EXECUTIVE ORDER, BECAUSE
THAT'S IMPORTANT FOR YOU TO HEAR
AS WELL.
SO LET'S SEE WHAT HAPPENS IF
I -- WHAT DO I DO?
OKAY.
I NEED SOME TECHNICAL HELP.
I'M SUSAN GREGURICK, ASSOCIATE
DIRECTOR OF DATA SCIENCE.
OUR ROLE, A SMALL WONDERFUL
TEAM, TIGER TEAM, TO CATALYZE
DATA SCIENCE AND RELATED
ACTIVITIES WITHIN THE COMMUNITY,
WITHIN THIS COMMUNITY, OUR ROLE
IS ALSO TO COORDINATE DATA
SCIENCE AND ARTIFICIAL
INTELLIGENCE ACTIVITIES ACROSS
THE 27 INSTITUTES, CENTERS,
OFFICES, THAT MAKE UP NIH.
AND OUR ROLE IS ALSO TO PROVIDE
SUPPORT THAT'S FUNDING SUPPORT
TO YOU, THE COMMUNITY, BUT ALSO
TO OUR NIH INSTITUTES, CENTERS,
OFFICES, TO DO THINGS LIKE
DEVELOP INFRASTRUCTURE FOR DATA
SCIENCE, CLOUD COMPUTING, TO
DEVELOP PROGRAMS TO SUPPORT DATA
REPOSITORIES AND KNOWLEDGE
BASES, PROGRAMS TO SUPPORT A.I.
AND NEXT GENERATION OF
RESEARCHERS AND DATA SCIENTISTS.
WE'RE A SMALL TEAM OF I THINK
ABOUT 30 FOLKS, MOST FEDS BUT
NOT ALL, HOPEFULLY YOU'LL MEET A
FEW OF THEM TODAY.
SO, I'LL START WITH THE END,
WHERE DO WE SEE OURSELVES GOING.
I COULD TELL YOU ABOUT ALL THE
THINGS WE'VE DONE IN THE PAST
OVER THE PAST THREE YEARS BUT
WHAT YOU MIGHT FIND MORE
INTERESTING IS TO ASK US WHERE
ARE WE GOING AT NIH.
THIS IS DATA SCIENCE PLAN THAT'S
DEVELOPED IN COLLABORATION WITH
EVERYBODY FROM THE NIH, ALL
INSTITUTES, CENTERS, OFFICES,
AND WITH LEADERSHIP OF DR.
MONICA BERTAGNOLLI.
FIRST GOAL WHAT THE I.C.
DIRECTORS WAS IMPORTANT ON THEIR
MINDS, TO IMPROVE CAPABILITIES
TO SUSTAIN NIH POLICY FOR DATA
MANAGEMENT AND DATA SHARING.
I'LL TELL WHAT YOU SOME
CHALLENGES ARE AND GOALS ARE.
A BIG PRIORITY FOR DR.
BERTAGNOLLI AND MYSELF IS TO
DEVELOP PROGRAMS, THESE ARE
LARGE PROGRAMS, TO ENHANCE USING
HUMAN DERIVED DATA FOR RESEARCH
AND BEYOND.
I'M A COMPUTER PERSON.
MY PASSION IS GOING TO BE IN
SOFTWARE, COMPUTATIONAL METHODS
AND A.I., OF COURSE THE GOAL IS
TO IMPROVE THESE OPPORTUNITIES
IN SOFTWARE MODELING AND
ARTIFICIAL INTELLIGENCE.
WE ALSO WANT TO MAKE IT EASIER
TO FIND DATA, SO FEDERATED
BIOMEDICAL RESEARCH DATA
INFRASTRUCTURE IS THE FOURTH
GOAL.
OF COURSE PROVIDING
OPPORTUNITIES FOR NEXT
GENERATION OF DATA SCIENCE AND
A.I. RESEARCHERS.
I'LL DIG IN A LITTLE BIT ABOUT
SOME FUTURE THINKING GOALS.
YOU PROBABLY REMEMBER LAST
OCTOBER, WELL ACTUALLY LAST
JANUARY 25, I BELIEVE, WE
RELEASED THE NIH FINAL POLICY
FOR DATA MANAGEMENT AND DATA
SHARING.
THIS REQUIRES NIH RESEARCHERS TO
DEVELOP A DATA MANAGEMENT
SHARING PLAN WHEN THEY SUBMIT
THEIR GRANT, CONTRACT, OTHER
TRANSACTIONAL AUTHORITY.
THIS CREATES A NEED FOR
RESEARCHERS TO UNDERSTAND DATA
MANAGEMENT IN WAYS THAT WILL
FOSTER GREATER FAIRSHARING AND
REQUIRES NEED FOR DATA TO GO
SOMEWHERE, FROM NEW STRATEGIES
FOR BIOMEDICAL REPOSITORIES AND
KNOWLEDGE BASES WRIT LARGE.
SO WHAT ARE WE GOING TO DO?
WE NEED TO SUPPORT AND WILL
SUPPORT RESEARCHERS TO CREATE
FAIR DATA, AND WE'VE HAD A
NUMBER OF SUPPLEMENTS, CREATING
ADDITIONAL OPPORTUNITIES FOR
RESEARCHERS TO SHARE DATA NOT
JUST IN THE ONLY COMMUNITY BUT
ALSO IN THE INTRAMURAL RESEARCH
COMMUNITY WITH NEW OPPORTUNITIES
IN ELECTRONIC LAB NOTEBOOKS, FOR
EXAMPLE.
WE NEED TO THINK ABOUT THAT DATA
IN A WAY THAT MAKES IT USABLE
FOR OTHERS SO THAT'S DATA
HARMONIZATION.
THERE'S REALLY GREAT
OPPORTUNITIES THAT HAVE HAPPENED
AND I'M LOOKING AT CHRIS, DATA
HARMONIZATION, N3C, A PHENOMENAL
WORK THAT CHRIS AND HIS
COLLEAGUES DID BUT WE NEED TO
THINK ABOUT THAT AT SCALE, HOW
CAN WE REALLY UTILIZE ONTOLOGY,
COMMON DATA ELEMENTS.
YOU'LL SEE LEVERAGE IN
HARMONIZATION OF COMMON DATA
ELEMENTS IN THE FUTURE, MORE
OPPORTUNITIES LIKE SOME OF THE
WORK CHRIS DID BUT NEED TO THINK
ABOUT THAT.
I WANT TO PUT A PLUG IN FOR A.I.
THERE'S SOME NEAT WORK HAPPENING
IN LARGE LANGUAGE MODELS FOR
DATA CURATION AND HARMONIZATION
THAT WE CAN EXPLORE MORE.
FINALLY WE'RE SUPPORTING THE
NIH-FUNDED DATA REPOSITORIES AND
KNOWLEDGE BASES IN WAYS THAT
MAKE THEM RESOURCES WITH NEW
PROGRAMS AND SPECIAL EMPHASIS
STUDY SECTIONS.
AN IMPORTANT GOAL FOR DR.
BERTAGNOLLI AND MYSELF TO THINK
ABOUT THAT PATIENT-DERIVED DATA,
FROM THE CLINICS AND HEALTH CARE
SYSTEMS AND REAL WORLD DATA.
THERE'S A NEED FOR AT-SCALE
ACQUISITION OF THAT DATA,
PROTECTING THAT DATA, UTILIZING
THAT DATA FROM ELECTRONIC AND
REAL WORLD HEALTH DATA THAT
ENHANCES PATIENT TRUST MODEL,
PATIENT CONFIDENTIALITY,
PRIVACY, AND CONCERNS.
IF YOU WORKED WITH ELECTRONIC
HEALTH CARE DATA AS CHRIS HAS
THERE'S A REAL DATA ISSUE THERE,
DATA QUALITY.
SO WE NEED TO THINK ABOUT THAT
AND REALLY HOW DO WE USE THAT
DATA IN A WAY THAT MAKES SENSE
FOR THE MANY USE CASES THAT
WE'RE GOING TO HAVE.
AND AT THE SAME TIME, THERE'S A
NEED TO LINK THAT DATA, THAT'S
SOMETHING THAT WAS REALLY
INTERESTING WITH N3C AND ALL OF
US, WAYS WE CAN LINK DATA THAT
PRESERVE ETHICAL, LEGAL
IMPLICATIONS, IMPORTANT FOR
UNDERREPRESENTED AND AT-RISK
PEOPLES.
SO WHAT CAN WE DO?
WE WILL IMPROVE THE ABILITY TO
UTILIZE CLINICAL DATA, ELECTRIC
-- ELECTRONIC AND REAL WORLD
DATA.
YOU MAY HAVE SEEN FUNDING
ANNOUNCEMENTS, I SUSPECT YOU
MIGHT BE SEEING A FEW MORE OF
THOSE TYPES OF ACTIVITIES.
WE, MY OFFICE AND OTHERS,
UNDERSTAND THERE'S A WHOLE
HEALTH I.T. SYSTEM OUT THERE
WITH ITS OWN DATA STANDARDS,
FHIR, UNC, WE CAN LEVERAGE
HEALTH AND ADMINISTRATIVE DATA
INTO THE RESEARCH PLATFORMS.
FINALLY, THIS IS REALLY
IMPORTANT, LOOKING AT DEBRA FOR
SURE, WAYS TO THINK ABOUT
ADOPTING SOCIAL AND
ENVIRONMENTAL DETERMINANTS OF
HEALTH.
SHARE HAS BEEN PILOTING AND WE
CAN LOOK AT SCALE FOR NIH.
YOU'LL BE SEEING MORE OF THIS IN
THE FUTURE.
I'M A COMPUTER PERSON.
I LOVE COMPUTATIONAL SCIENCE.
SO THERE'S SO MUCH OPPORTUNITY
AND TRUSTWORTHY A.I. ETHICALLY,
THAT'S WHY WE'RE HERE.
WE NEED HELP.
WE NEED TO KNOW HOW CAN WE
ACTUALLY TAKE THESE IDEAS AND
TAKE THEM AND INSTANTIATE.
NEW OPPORTUNITIES AND A.I., TAKE
ADVANTAGE OF GENERATIVE A.I. AND
FOUNDATIONAL MODELS ON THE
HORIZON, NEW SOFTWARE,
TECHNOLOGIES, OUR OFFICE DOES
SUPPORT SUSTAINABLE SOFTWARE AND
SO YOU'LL SEE TWO FUNDING
ANNOUNCEMENTS COMING OUT SOON TO
SUPPORT ROBUST SOFTWARE
DEVELOPMENT FOR WHAT DAN KATZ
HAS BEEN TALKING ABOUT, BRIDGING
THE GAP, REAL SOFTWARE ENABLING
TECHNOLOGIES AS WELL AS
SUPPORTING RESEARCH SOFTWARE
ENGINEERS.
SO ACTUALLY CAREER SOFTWARE
PEOPLE IN ACADEMIA.
SO LET ME TELL YOU BECAUSE I
THINK I HAVE ENOUGH TIME, ABOUT
WHAT IS HAPPENING IN TRUSTWORTHY
A.I.
AND SO MUCH HAPPENING IN THE
FEDERAL SPACE.
THERE'S THE BILL OF RIGHTS FOR
A.I., THE TRUSTWORTHY A.I. PLAY
BOOK, TONS OF STRATEGIC PLANS
FOR A.I., BUT THE MOST IMPORTANT
THING IS REALLY EXECUTIVE ORDER
ON SAFE SECURE AND TRUSTWORTHY
DEVELOPMENT THAT CAME UP ON
OCTOBER 30 THIS PAST YEAR.
HOW DOES THIS IMPACT OUR WORK
AND HHS?
THERE ARE FOUR MAIN OBJECTIVES
IN THAT EXECUTIVE ORDER HHS WRIT
LARGE, ESTABLISHING POLICIES FOR
HEALTH AND HUMAN SERVICES,
ADVANCING QUALITY AND SAFETY OF
A.I. AND HEALTH, LEVERAGING
GRANTMAKING TO ADVANCE A.I. USE,
THAT SOUNDS LIKE NIH.
AND PROMOTING COMPLIANCE WITH
NON-DISCRIMINATORY AND PRIVACY
LAWS.
HOW DOES THIS MAP INTO WHAT
WE'RE ACTUALLY DOING?
I LOVE THAT SLIDE.
WHAT YOU SEE IS THE OBJECTIVES,
THE ACTIONS THAT ARE IN THE
EXECUTIVE ORDER, AND THEN THE
TIME FRAME THEY ARE SUPPOSED TO
HAPPEN IN.
I WON'T GO THROUGH EACH ONE.
I WANT TO JUST SHOW YOU SO YOU
HAVE IT AND YOU CAN LOOK AT THIS
IN THE FUTURE.
THERE'S A LOT OF ACTIVITIES.
SOME OF THESE ARE -- YEARS LONG
DEVELOPING THE FRAMEWORK, A REAL
LONG ACTIVITY.
OTHERS DUE A DAY AGO,
ESTABLISHING A.I. TASK FORCE.
WE HAVE ESTABLISHED AN A.I. TASK
FORCE WITH LEADERSHIP FROM NIH,
FROM THE OFFICE OF NATIONAL
COORDINATOR, AND FROM THE DEP
SEC OF HHS AND FDA.
WE HAVE EIGHT WORKING GROUPS
TAKING EACH OF THESE ACTIONS AS
WELL AS OVERALL STRATEGY FOR
A.I. AND WORKING THROUGH THOSE
GOALS TO DELIVER A STRATEGIC
PLAN THAT IS REQUIRED AT THE END
OF 365 DAYS AS WELL AS CONCRETE
DELIVERABLES DUE EVEN IN 180
DAYS.
SO YOU'LL SEE A LOT OF ACTIVITY
HAPPENING.
IF YOU'RE AT NIH YOU ARE
INVOLVED IN A LOT OF ACTIVITY
FROM THE HHS WORKING GROUPS.
I DON'T WANT YOU TO THINK THAT
A.I. IS NEW TO NIH.
IN FACT, IT'S NOT.
THE DATA WE'VE COLLECTED GO BACK
MUCH FARTHER THAN 2019, THAT'S
WHEN WE STARTED LABELING AS A.I.
OVER SEVERAL YEARS THE FUNDING
WE HAVE SUPPORTED IN ARTIFICIAL
INTELLIGENCE GRANTS, CONTRACTS,
HAS GONE UP QUITE A BIT.
YOU SEE A NUMBER OF THE
ACTIVITIES HERE.
MOST OF THE FOLKS WHO ARE
LEADING THESE ACTIVITIES ARE
IT'S -- ALSO HERE, YOU CAN
CONNECT WITH THEM.
WE'VE SUPPORTED A $29 MILLION TO
NIH AWARDEES TO BASICALLY MAKE
THEIR EXISTING DATA A.I. READY
TO DEVELOP NEW TRAINING
OPPORTUNITIES, SKILLS AND
COMPETENCIES, TO TAKE THE FIRST
LOOK AT ETHICS, WHEN THINKING
ABOUT A.I. DATA.
THERE'S A FEW OF THE AWARDEE
GRANTS LISTED ON THE FAR RIGHT.
THIS INVOLVED A COLLABORATION
WITH 20 INSTITUTES, CENTERS,
OFFICE, THE FOUNDATION OF THE
WORK WE DID THAT'S NOW GOING TO
BE BUILT UPON BY THIS WORKSHOP
AS WELL AS OTHER ACTIVITIES
HAPPENING IN THE FUTURE.
THIS IS A GENERAL THEME, WE DO A
NUMBER OF SUPPLEMENT, GET WHAT
WORKS AND WHERE GAPS AND
OPPORTUNITIES ARE AND MOVE
FORWARD.
WE'LL BE MOVING FORWARD.
HERE'S ONE OF THE WAYS WE'RE
MOVING FORWARD.
WITH THE NATIONAL A.I. RESEARCH
RESOURCE.
THIS IS LED BY NSF, WITH A
SIGNIFICANT NUMBER OF AGENCY
PARTNERSHIPS, AS WELL AS
INDUSTRY PARTNERSHIPS; TO
PROVIDE -- THINK OF THIS AS A
RESOURCE LIKE A BTRR, YOU COME
WITH A PROPOSAL, IT'S REVIEWED,
ACCESS TO COMPUTING POWER,
DATASETS AND OTHER SOFTWARE
STACKS.
THAT'S REALLY WHAT THIS IS.
WE'RE SUPER THRILLED TO BE PART
OF IT.
WE'RE PART OF BOTH THE OPEN
NAIRR AND SECURE NAIRR, WE
CO-LEAD WITH OUR MAIN NAIRR
SECURE PARTNERS AT THIS MOMENT.
WE'LL HOPEFULLY BE LEVERAGING
CLOUDLAB AND OPPORTUNITIES TO
DEVELOP LARGE NETWORKS FOR THE
NAIRR.
FEDERATION IS A KEY WORD YOU'LL
HEAR FROM MONICA AS SHE'S
SPEAKING IN THE FUTURE HOW TO
BRING TOGETHER ALL THE DATA AND
THE DATA ASSETS, SO RESEARCHERS
CAN USE THEM, SOMETHING WE'RE
WORKING ON WITH HER AND IN
COLLABORATION WITH THE NATIONAL
LIBRARY OF MEDICINE.
FINALLY, IT REALLY TAKES A
VILLAGE TO CREATE A.I.
SO WE WANT TO STRENGTHEN AND
NURTURE THAT TALENT FROM DIVERSE
SCIENTIFIC INTERESTS AND DIVERSE
INSTITUTIONS, SOMETHING WE'LL BE
CONTINUING IN THE FUTURE.
BECAUSE I'M RUNNING OUT OF TIME
THIS IS THE SLIDE I WANT TO
LEAVE YOU WITH.
IT ASKS YOU TO GIVE US INPUT ON
WHAT YOU'VE HEARD.
I REALLY MEAN IT.
I HOPE YOU WILL TAKE THE TIME TO
PROVIDE US INPUT, WHERE ARE WE
HITTING THE MARK, MISSING THE
MARK, WHAT SHOULD BE IN THERE
THAT ISN'T IN THERE.
THAT'S REALLY IMPORTANT.
SO I'M GOING TO LEAVE THIS SLIDE
UP AND I DON'T DON'T KNOW IF
THERE'S TIME FOR QUESTIONS.
THANK YOU FOR COMING AND MAKING
THE TREK TO SOMEWHAT RAINY AND
GRAY D.C.
[APPLAUSE]
>> THANK YOU.
>> OKAY, WONDERFUL.
MY JOB NOW IS TO TALK A LITTLE
BIT WHAT YOU'LL BE DOING OVER
THE NEXT COUPLE DAYS.
AND WHY IT'S IMPORTANT TO US
INSTEAD OF HOW WE CAME TO THIS
POINT, HOPEFULLY DO A LITTLE BIT
OF MIND MELD, THIS IS THE
BEGINNING OF A JOURNEY.
WE HAVE A SERIES OF EXERCISES
MAPPED OUT FOR YOU THAT
HOPEFULLY WILL SORT OF RESULT IN
THE SORT OF INPUT AND DEEP
THINKING THAT WE'RE HOPING YOU
ALL WILL COME ALONG WITH US TO
DO.
SO JUST IN TERMS OF BACKGROUND,
HOW WE GOT HERE, IT'S NOT A
SECRET.
I THINK THERE ARE MANY EXAMPLES
WHERE A.I. HAS UNINTENDED
CONSEQUENCES IN THE BIOMEDICAL
AND HEALTHCARE SETTINGS.
AND THE ROOT CAUSES OF THAT
COULD BE IN THE DATA THAT ARE
USED FOR TRAINING A MODEL, COULD
BE IN THE META DATA, IN THE WAY
THE MODEL WAS TRAINED, OR ANY
COMBINATION OF THESE THINGS.
SO WE REALLY NEED TO UNDERSTAND
MORE ABOUT HOW TO NOT ONLY --
SOURCES OF THOSE IMPACTS BUT
ALSO HOW TO REMEDIATE.
WHAT WE HEARD FROM THE COMMUNITY
AT THE POINT WHEN AN A.I. MODEL
IS APPLIED IN A PARTICULAR
SITUATION, IF IT DOESN'T PERFORM
WELL OR AS EXPECTED IT'S VERY
HARD FOR PEOPLE TO SORT OF SEE
UPSTREAM WHERE THE CAUSES MIGHT
BE.
THERE'S ALSO A LOT OF ATTENTION
AROUND A.I. ASSURANCE, SO
THINKING ABOUT HOW TO MEASURE A
MODELS PERFORMANCE WITH RESPECT
TO ACCURACY BUT ALSO SOME MORE
SOCIAL QUESTIONS, LEGAL
QUESTIONS.
SO WE WANT TO HAVE A BETTER
HANDLE HOW TO DO THAT.
AND FINALLY IN OPEN SCIENCE,
FEDERALLY FUNDED RESEARCH,
PERHAPS IN CONTRAST TO WHAT YOU
MIGHT SEE IN INDUSTRY THERE'S
MUCH MORE REUSE OF RESEARCH
PRODUCTS.
REUSE OF DATA, REUSE OF MODELS,
ALGORITHMS, WORKFLOWS, A
HALLMARK OF THE RESEARCH WE
FUND.
SO IT MAKES IT MORE DIFFICULT TO
FOLLOW THOSE BREAD CRUMBS
UPSTREAM.
AND FINALLY, WE'RE GETTING ASKED
BY OUR RESEARCH COMMUNITY, BY
ALL OF YOU, BY OUR COLLEAGUES
WITHIN NIH WHO ARE FUNDING
RESEARCH, WHAT CAN THE RESEARCH
COMMUNITY DO TO HELP SOLVE SOME
OF THESE PROBLEMS, AND SO ONE OF
THE MAIN THINGS I HOPE WE COME
OUT OF THIS WORKSHOP WITH IS
SOME ANSWER TO THAT QUESTION.
WHAT CAN RESEARCHERS DO TO HELP
MAKE A.I. MORE TRANSPARENT SO
PEOPLE CAN MAKE MORE INFORMED,
RESPONSIBLE, ETHICAL DECISIONS
HOW THEY REUSE DATA AND MODELS.
THIS IS THE WONDERFUL CAST OF
CHARACTERS THAT WE HAVE THAT
HELPED US SHAPE THIS WORKSHOP.
SO OUR THREE CO-CHAIRS, TINA
HERNANDEZ-BOUSSARD, JULIA
STOYANOVICH AND IT'S BEEN
WONDERFUL TO WORK WITH THEM .
THE FIRST GOAL IS TO BEGIN TO
DEVELOP SOME OF THESE
TRANSPARENCY GUIDELINES FOR NIH
AWARDEES.
MAYBE USING, DEVELOPING,
CONTRIBUTING TO A.I.
THINK ABOUT WHAT WE MEAN BY NIH
AWARDEE, IT COULD BE AN R01
P.I., COULD BE SOMEONE WHO IS
MANAGING A REPOSITORY OR
DEVELOPING A NEW SEARCH
CAPABILITY.
WE WANT TO ALSO IDENTIFY SOME OF
THE TOOLS AND CAPABILITY GAPS,
THERE MIGHT BE A VISION OF THE
FUTURE HOW WE WANT A.I.
DEVELOPMENT TO WORK IN A
TRANSPARENT, THERE'S A PART WE
CAN DO NOW, PART MIGHT BE
CONTINGENT ON BUILDING OUT MORE
CAPABILITY.
FINALLY WE WANT TO TAP INTO YOUR
BRAINS AND HAVE YOU TELL US
ABOUT WHERE THE FUTURE OF THIS
FIELD OF TRANSPARENCY IS GOING.
WHAT THE DRIVERS MIGHT BE AND
WHAT NIH SHOULD BE PAYING
ATTENTION TO.
IN TERMS OF WHERE WE MIGHT GO,
OUR THINKING IS WE'LL COME OUT
WITH MAYBE NOT A FULL SKELETON
BUT BONES OF GUIDANCE FOR NIH
RESEARCHES AND WOULD PUT THAT
OUT FOR FURTHER COMMENT FROM THE
RESEARCH COMMUNITY.
SO POSSIBLY THROUGH REQUEST FOR
INFORMATION OR SOME OTHER MEANS,
SORT OF BY DEFINITION THIS ROOM
CAN'T DO IT ALONE AND WE'LL HAVE
BROADER INPUT ONTO THE OUTPUT
FROM THIS WORKSHOP.
AND THEN THE OTHER TWO GOALS
WILL FEED INTO STRATEGY HOW WE
BUILD OUT THIS DATA ECOSYSTEM.
LET ME NOW DO A MIND MELD WITH
YOU.
THIS IS A SCHEMATIC THAT I HOPE
IS HELPFUL IN TERMS OF THINKING
ABOUT WHAT WE'RE DOING TOGETHER
TODAY.
SO, THE BLUE BAR IN THE MIDDLE
IS THE DATA AND A.I. DEVELOPMENT
CYCLE, THE DEVELOPMENT PATH.
THINK OF IT GOING FROM DATA
GENERATION AND COLLECTION ALL
THE WAY UP TO DEPLOYMENT OF A.I.
MODEL IN PERHAPS THE HEALTHCARE
SETTING OR RESEARCH ENVIRONMENT.
THE LITTLE HEADS AROUND THAT ARE
MEANT TO DEPICT STAKEHOLDERS,
AND SOME OF THESE STAKEHOLDERS
ON THE TOP I THINK OF IT THIS
WAY, ON THE TOP PEOPLE WITH
INFORMATION NEEDS.
SO THESE ARE PEOPLE ALONG THAT
CYCLE WHO MIGHT NEED TO KNOW
SOMETHING ABOUT THE DATA THAT
THEY ARE ABOUT TO REUSE OR THE
MODEL THEY ARE ABOUT TO REUSE.
AND ON THE BOTTOM ARE
STAKEHOLDERS THAT HAVE
INFORMATION.
SO THESE ARE MAYBE THE P.I. THAT
CREATED THE DATA, THAT
UNDERSTANDS THE NUANCES OF HOW
THE DATA WERE PROCESSED, WHAT
CLEANING THEY DID, MAYBE SOMEONE
WHO DEVELOPED A MODEL THAT
UNDERSTAND THE DETAILS OF THE
TRAINING MECHANISM.
OUR GOAL IS TO FIGURE OUT HOW
INFORMATION SHOULD FLOW FROM THE
PEOPLE WHO HAVE IT TO THE PEOPLE
WHO NEED IT.
SO FROM THIS DIAGRAM ONE OF THE
THINGS YOU SHOULD TAKE OUT OF
THIS IS OUR THINKING AND OUR
APPROACH TO TRANSPARENCY IS
TRANSPARENCY IS A SORT OF
SYSTEM, CHARACTERRIC OF A
SYSTEM, ROOTING THIS IN
PRACTICAL UNDERSTANDING WHAT THE
STAKEHOLDERS NEEDS ARE.
NOT JUST TRANSPARENCY FOR
TRANSPARENCY SAKE BUT WHAT
STAKEHOLDERS NEED WHAT
INFORMATION AND HOW CAN WE GET
IT TO THEM AND REALLY HELP THEM
MAKE INFORMED AND RESPONSIBLE
DECISIONS.
JUST TO GIVE A SENSE, YOU'LL BE
GOING -- THIS AFTERNOON YOU'LL
BE GOING INTO A STAKEHOLDER
MAPPING ACTIVITY AND SO JUST TO
GIVE A SENSE OF WHAT WE THOUGHT
ABOUT IN TERMS OF QUESTIONS
PEOPLE MIGHT BE ASKING, SO THERE
MIGHT BE SOMEONE WHO IS A
PATIENT, AND THERE'S GOING TO BE
A HEALTH DECISION MADE FROM, YOU
KNOW, USING AN A.I. MODEL, THAT
PATIENT MIGHT BE LET'S SAY FROM
DIVERSE OR MINORITY GROUP,
EITHER ETHNIC, RACIAL, OR
BECAUSE THEY HAVE A RARE DISEASE
OR SOME OTHER REASON, THEY WANT
TO KNOW HOW WELL THE A.I. MODEL
IS GOING TO SERVE THEIR NEEDS,
REFLECT THEIR HEALTH CONDITION
AND PREFERENCE ABOUT CARE.
THEY WILL WANT TO KNOW WHAT SORT
OF FACTORS WENT INTO THE MODEL,
WHAT PERFORMANCE THEY THEY
EXPECT.
THESE WILL BE AVAILABLE ONLINE
SO YOU CAN REFER BACK.
WE WANTED TO GIVE YOU A SENSE OF
THE QUESTIONS AND INFORMATION
NEEDS THAT EXIST AROUND THIS
MODEL.
AND REALLY THINKING ABOUT THE
ENTIRE DATA LANDSCAPE.
IN TERMS OF GUIDELINES FOR NIH
AWARDEES, WE'LL ASK YOU TO THINK
THROUGH THIS STAKEHOLDER MAPPING
AND PIVOT TO, OKAY, WHAT ARE THE
ACTIONS THAT DIFFERENT AWARDEES
TAKE, WHETHER DATA SHARING,
SHARING A MODEL, REUSING A
MODEL, WE'LL THINK THROUGH THAT
SENSE AS WELL.
WHEN YOU -- THE EXERCISES THAT
WILL LEAD YOU THROUGH WILL GUIDE
YOU TO THINKING ABOUT NOW THAT
YOU'VE MAPPED OUT THE
STAKEHOLDER NEEDS HOW DO YOU GET
INFORMATION TO FLOW FROM ONE TO
THE OTHER AND WHAT ARE
CAPABILITY GAPS, SO THAT WILL BE
OUR SESSION 2.
FINALLY WHAT DOES TRANSPARENCY
LOOK LIKE IN THE FUTURE, SO WE
HAVE DONE THIS EXERCISE, YOU
KNOW, IN THE PRESENT, BUT WHAT
ARE THE DRIVERS OUT THERE, MAYBE
TECHNOLOGY DRIVERS, CULTURAL
DRIVERS, THEY MIGHT BE POLICY
DRIVERS, SO WHAT CAN WE EXPECT
FROM THIS FIELD IN THE NEXT FEW
YEARS.
SO, OUR AGENDA HAS THIS TEMPO TO
IT.
BREAKOUT SESSIONS, AND WE'LL
COME BACK INTO THE ROOM FOR
PLENARY READOUT, AND SO THE RED
BOXES HERE ARE FOR THE ONLINE
PARTICIPANTS.
THAT'S WHEN YOU'LL HAVE A CHANCE
TO TALK TO US THROUGH SLIDO.
AND AFTER THE THIRD SESSION
WE'LL COME BACK TOMORROW
AFTERNOON FOR A SORT OF BIG
GROUP DISCUSSION ABOUT WHAT BEEN
CREATED IN BREAKOUTS AND WE'LL
CONTINUE THAT DISCUSSION INTO
THE MORNING OF FRIDAY.
SO IF YOU'RE ONLINE, THAT'S THE
MOST INTERACTIVE SESSIONS THAT
WE'LL HAVE, SO TOMORROW
AFTERNOON AND BEGINNING OF
FRIDAY.
LET ME GO QUICKLY INTO JUST MORE
DETAIL INTO WHAT SORTS OF
ARTIFACTS WE'LL BE CREATING, AND
I'LL SPEND MORE TIME BECAUSE
WE'LL GO STRAIGHT TO THE
BREAKOUTS AFTER THE NEXT FEW
TALKS.
FIRST BREAKOUT SESSION THE GOAL
IS THEMATIC BREAKOUTS, CONSIDER
YOUR THEMES, IDENTIFY THE
RELEVANT STAKEHOLDERS AND THEIR
INFORMATION NEEDS.
SO WHAT WE'RE IMAGINING, YOUR
LEADS WILL GUIDE YOU THROUGH
EXERCISE, YOU'LL COME OUT WITH
SOME IDEAS OF, OKAY, HERE'S A
STAKEHOLDER, HERE IS THEIR
DECISION NEEDS, HERE IS THE
INFORMATION THEY NEED IN ORDER
TO MAKE INFORMED ETHICAL
DECISIONS ABOUT DATA AND MODEL
REUSE.
AND SO YOU'LL COME OUT WITH A
COLLECTION OF THOSE.
THAT WILL HELP THEN GOING INTO
THE NEXT SESSION, NEXT BREAKOUT
SESSION, WHERE YOU'LL LINK THEN
THOSE NEEDS WITH INFORMATION
SOURCES, THINKING ABOUT HOW THE
INFORMATION CAN FLOW AND WHAT
SYSTEM CAPABILITIES ARE NEEDED.
THIRD SESSION LEADS WILL TELL
YOU WE'RE PIVOTING FROM INSTEAD
OF THINKING ABOUT INDIVIDUALS
LIKE A PATIENT OR A RESEARCHER,
YOU'RE THINKING THEN ABOUT THEIR
ACTIVITIES.
SO FOR DATA SHARING, AND THEN
BUILDING OUT SOME OF THE
GUIDANCE THAT YOU WOULD WANT TO
GIVE TO THAT PERSON WHO IS
SHARING DATA.
THIS IS WHAT YOU SHOULD BE
THINKING ABOUT, THESE ARE THE
PRINCIPLES YOU SHOULD BE
FOLLOWING, TO MAKE THIS WHOLE
SYSTEM MORE TRANSPARENT.
AND THAT'S -- AFTER THAT SESSION
WE'LL COME BACK INTO THE PLENARY
ROOM HERE, AND WE'LL HAVE THIS
DISCUSSION SORT OF THINKING
ABOUT, OKAY, DID WE GET THE
COLLECTION OF ACTIVITIES RIGHT?
ARE WILL THINGS MISSING?
ACTIONS WE MIGHT WANT TO PROVIDE
GUIDANCE FOR THAT WE HAVEN'T
THOUGHT OF YET?
FOR EACH OF THOSE, DO WE HAVE
THE RIGHT GUIDANCE FOR EACH OF
THOSE ACTIONS AND WE'LL LEAD YOU
THROUGH THAT DISCUSSION AS WELL.
FINALLY FOR SAKE OF TIME I'LL BE
QUICK HERE, THIS IS WHAT WE'LL
DO FOR THINKING ABOUT THE
FUTURE, JUST THINKING REALLY
OPEN-ENDED CONVERSATION, WE WANT
TO HAVE YOUR BRAINSTORMING ABOUT
WHERE THIS FIELD MAY BE GOING.
SO, A COUPLE OF TOP TIPS, RANDOM
COMMENTS.
THE FIRST IS OUR APPROACH TO
TRANSPARENCY IS BASED ON THIS
PRACTICAL THINKING AROUND
INDIVIDUALS INFORMATION AND
DECISION NEEDS, WHAT DO
DIFFERENT PEOPLE AROUND THIS
SORT OF PIPELINE, DIFFERENT
STAKEHOLDERS, REALLY NEED TO
KNOW.
IN THE COURSE OF YOUR
CONVERSATION, I'M SURE IT WILL
COME UP THINKING ABOUT NOT JUST
HOW TO MAKE THIS SYSTEM
TRANSPARENT BUT ALSO RIGHT FROM
WRONG.
WHAT SHOULD PEOPLE BE DOING,
WHAT SHOULD PEOPLE NOT BE DOING,
AND I WOULD SAY IF YOU HAVE A
CHOICE, ERR ON THE SIDE OF
THINKING ABOUT TRANSPARENCY.
THE PRIMARY GOAL HERE IS NOT TO
DECIDE WHAT'S RIGHT, WHAT'S
WRONG, BUT TO GIVE PEOPLE THE
INFORMATION THEY NEED TO MAKE
THOSE DECISIONS FOR THEMSELVES.
THE THIRD BULLET IS AROUND BIAS
VERSUS CAUSALITY VERSUS
PREJUDICE.
WE'VE HAD CONVERSATIONS THAT GO
AROUND THIS.
A MODEL CAN DISPLAY BIAS EVEN
WHEN PERHAPS WITH RESPECT TO
CERTAIN CATEGORIES, EVEN WHEN
CATEGORIES ARE NOT NECESSARILY
CAUSALLY RELATED TO THE OUTCOME.
AND THAT ALSO DOESN'T MEAN THAT
THE MODEL WAS DEVELOPED IN ANY
KIND OF BIASED OR PREJUDICED
WAY.
THEY ARE JUST THINKING ABOUT
THOSE SORT OF SEPARATELY AND
BEING CAREFUL ABOUT HOW WE USE
OUR LANGUAGE I WOULD SUGGEST
PAYING ATTENTION TO.
THE BREAKOUT THEMES WE'VE GIVEN
YOU AND THAT INSTRUCTIONS I'VE
GONE THROUGH ARE STARTING
POINTS.
THESE ARE NOT MEANT TO BE
RESTRICTIVE OR CONSTRAINTS.
THEY ARE A LUNCHING PAD.
GO WHERE THE CONVERSATION TAKES
YOU.
THINK ABOUT THE FULL DATA AND
MODEL DEVELOPMENT CYCLE.
SO THAT MEANS THINKING ABOUT THE
DATA ECOSYSTEM.
WE'RE REALLY THINKING ABOUT DATA
AS CREATED OR GENERATED PERHAPS
IN A CLINIC OR RESEARCH SETTING
TO WHERE IT'S DEPLOYED.
AND SO YOU SHOULD BE THINKING
ABOUT, YOU KNOW, DATA
REPOSITORIES, WORKSPACES, SEARCH
ENGINES, ALL THE COMPONENTS OF
OUR ECOSYSTEM IN A HOLISTIC WAY.
CONSIDER HEALTH APPLICATIONS AS
WELL AS RESEARCH APPLICATIONS.
I THINK IT'S VERY TEMPTING TO
THINK ABOUT A.I. MODELS IN A
HEALTH APPLICATION OR CLINICAL
SETTING, BUT THERE ARE MANY
TIMES AN A.I. MODEL, THE PURPOSE
IS TO BE DEPLOYED IN A RESEARCH
SETTING SO THINK ABOUT THAT TOO.
AND FINALLY, ONE OF THE GOALS OF
OUR BREAKOUTS IS TO MAKE SURE WE
HEAR FROM ALL OF YOU.
THAT'S WHY WE INVITED YOU AND
WHY WE'RE SO HAPPY TO HAVE YOU,
AND WHY THERE'S SO MANY EXCITED
FACES AROUND.
SO CHIME IN.
DON'T FEEL SHY.
OUR BREAKOUT LEADS WILL DO THEIR
BEST TO DRAW THAT OUT OF YOU.
SO I HOPE YOU WILL CONTRIBUTE.
WE HAVE THEMES FOR ALL OF THE
BREAKOUTS.
YOU SAW THIS ONLINE, WHERE WE
ASKED YOU TO PICK YOUR TOP
CHOICES, THESE ARE ROUGHLY
MAPPED OUT ACROSS THE DATA AND
MODEL DEVELOPMENT CYCLE.
AND SO THESE ARE AROUND
SYNTHETIC DATA, DATA SHARING FOR
GENERAL REUSE, MULTI-MODAL DATA,
FOUNDATION MODELS, PROXY
VARIABLES, AND WE HOPE TO SORT
OF COVER A WIDE RANGE OF THE
SPACE OF ETHICAL CHALLENGES AND
QUESTIONS THAT CAN ARISE FROM
A.I. BUT OF COURSE IT'S NOT THE
FULL SPACE SO USE IT AS A
LAUNCHING PAD, NOT A
RESTRICTION.
OKAY.
THESE ARE YOUR BREAKOUT LEADS.
I HAVE SOME NEW AND BREAKING
INFORMATION.
THESE ROOM NUMBERS ARE NOT
CORRECT.
SO, GET OUT A PEN AND I'M GOING
TO TELL YOU WHICH ROOM YOU'RE
GOING TO INSTEAD.
IF YOU ARE IN THE FOUNDATION
MODEL GROUP, AND BY THE WAY IF
YOU DIDN'T PICK IT UP, THE GROUP
ASSIGNMENTS ARE ON A SHEET
OUTSIDE.
IF YOU'RE IN THE FOUNDATION
MODEL GROUP, YOU WERE ASSIGNED,
YOU'RE NOW IN 260-C.
IN THE GENERAL REUSE GROUP
ASSIGNED TO 270B YOU'RE NOW IN
260D FOR DAVID.
IF YOU'RE IN MULTI-MODAL GROUP
AND WERE ASSIGNED TO 260-F,
YOU'RE NOW IN 260-E FOR EDWARD.
PROXY VARIABLES, YOU LUCKED OUT
AND YOU'RE IN THE ROOM I THOUGHT
YOU WOULD BE IN.
AND FOR SYNTHETIC DATA YOU'RE
NOT IN 270-A, YOU'RE IN 150-B.
SO IF YOU DIDN'T WRITE THAT DOWN
DON'T WORRY, WE HAVE IT OUT
FRONT.
I HAVE MY TRUSTY MAPPING HERE SO
I CAN LET YOU KNOW.
THAT'S FOR THE FIRST THREE --
THIS IS FOR THE FIRST THREE
SESSIONS, YOU'LL BE IN YOUR
THEMATIC BREAKOUT.
THE LAST SESSION ON FRIDAY,
WE'LL MIX YOU ALL UP SO YOU'LL
BE WITH A NEW GROUP OF FRIENDS
AND WE WILL REASSIGN YOU ROOMS
AS WELL.
I'LL SHOW YOU THE BREAKOUT FOR
THAT.
SO, I WANT TO END BY THANKING
AGAIN ALL OF THE CO-LEADS AND
THE BREAKOUT LEADS.
THIS HAS BEEN A JOYFUL
EXPERIENCE TO WORK WITH ALL OF
YOU.
I REALLY APPRECIATE IT.
I WAS TELLING THEM LAST WEEK
I'VE NEVER BEEN SO RELAXED ONE
WEEK OUT FROM A WORKSHOP BECAUSE
I HAVE SUCH FAITH IN THIS GROUP.
I THINK YOU'RE GOING TO HAVE A
LOT OF FUN.
I WANT TO THANK ALSO MY
COLLEAGUES WHO HAVE BEEN HELPING
THE ORGANIZATION AND REALLY
HELPED BRING THIS WORKSHOP INTO
BEING, THIS IS A SUBSET OF OUR
NIH A.I. ETHICS WORKING GROUP,
AND SO REALLY WANT TO THANK ALL
OF THEM.
AND FINALLY, THE SUPPORT TEAM
YOU'LL SEE TODAY, FEEL FREE TO
ASK THEM QUESTIONS.
JESSICA ST. LOUIS HAS BEEN
HELPING US, SIMON TWIGGER,
MICHELLE BAILEY, KELLY WILSON AS
WELL AS COLLEAGUES FROM SCG
MANNING THE REGISTRATION DESK
OUTSIDE.
WITH THAT, I WOULD LOVE TO HAND
IT OVER TO JULIA WHO IS GOING TO
INTRODUCE US INTO THE SCIENCE OF
TRANSPARENCY.
>> ALL RIGHT.
THANK YOU.
[APPLAUSE]
IT'S SO MUCH A PLEASURE TO BE
HERE.
AND TO HAVE THIS IMPOSSIBLE TASK
OF GIVING AN INTRODUCTION TO
TRANSPARENCY IN 20 MINUTES.
I DECIDED NOT TO FOLLOW THE
PROMPT PRECISELY BUT RATHER GIVE
MY OPINION ON WHAT TRANSPARENCY
MAY MEAN AND APPLY TO OUR
CONVERSATION IN THE NEXT THREE
DAYS, TO UNDERSCORE AND
REINFORCE SOME OF THE THINGS
SAID ALREADY BUT MAKE SURE SOME
TOPICS WITHIN TRANSPARENCY REALM
THAT I OFTEN SEE ARE MISSED THAT
WE PAY ATTENTION TO THEM AS WE
MOVE AHEAD.
I'M A COMPUTER SCIENTIST, I KNOW
I'M IN A MINORITY IN THIS
MEETING, IT'S REALLY EXCITING TO
MEET ALL THESE AMAZING PEOPLE
WHO WILL TEACH ME ABOUT HOW
TRANSPARENCY APPLIES IN THIS
EXTREMELY IMPORTANT DOMAIN OF
MEDICINE, OF HEALTH, RIGHT?
AND I SEE THIS IS REALLY AN
OPPORTUNITY TO ADVANCE THE STATE
OF THE ART BOTH WITHIN COMPUTER
SCIENCE AND WITHIN BIOMEDICAL
RESEARCH.
AS YOU'LL SEE MANY THINGS YOU
MAY ASSUME ARE KIND OF
OPERATIONALIZED AND DONE, JUST A
MATTER OF FIGURING OUT WHAT IS
THE RIGHT TOOL THAT I SHOULD BE
USING, THEY ARE WITHIN COMPUTER
SCIENCE, I'M GLAD THERE'S THIS
COLLABORATION AMPLIFYING AND
STARTING AND HOPEFULLY GOING
FURTHER.
TO START I WANT TO TELL YOU THAT
I HAD THE PLEASURE OF DIRECTING
THE CENTER FOR RESPONSIBLE A.I.
AT NYU, OUR GOAL IS TO MAKE
RESPONSIBLE A.I. AND A.I.
SYNONYMOUS, IN THE NOT-TOO-
DISTANT FUTURE.
OUR HOME PAGE, YOU'LL FIND
INFORMATION ABOUT OUR RESEARCH,
A LOT OF BASIC RESEARCH ON
COMPUTER SCIENCE, AS I
MENTIONED, THAT IS NECESSARY AND
ONGOING.
A LOT IS SOCIOTECHNICAL WORK, A
LOT IS WORK ON TECHNOLOGY
POLICY.
AND FINALLY, AS I'LL TELL YOU AT
THE END OF MY PRESENTATION WE DO
WORK ON EDUCATION AND TRAINING.
ALL OF THESE INTERVENTIONS ARE
IMPORTANT TO BE ABLE TO REALLY
ENACT LASTING SOCIETAL CHANGE IN
THE WAY WE USE A.I. AND RELATED
TECHNOLOGIES.
THE OTHER THING IS MY SLIDES
WILL BE VISUALLY INTERESTING,
AND THIS IS A BY-PRODUCT OF OUR
PUBLIC EDUCATION ACTIVITIES, AND
I ALSO THINK IT'S REALLY
IMPORTANT THAT WE BRING SOME
HUMAN, FUN, ART, AND OTHER WAYS
TO PRESENT INFORMATION INTO THIS
CONVERSATION.
BECAUSE THE CONVERSATION ABOUT
A.I. AND ETHICS WHICH
TRANSPARENCY IS A MAJOR
COMPONENT REALLY, IT'S A HEAVY
CONVERSATION FOR PEOPLE TO HAVE.
AND SO BY USING ART, THE GOAL IS
TO KIND OF SHORTEN THIS DISTANCE
BETWEEN PEOPLE AND
UNDERSTANDING, AND ACTIONS THAT
WE NEED TO BE TAKING
COLLECTIVELY.
SO, AS A DATA SCIENTIST I WILL
OF COURSE START WITH AN EXAMPLE.
I HOPE TO MOTIVATE A LOT OF WHAT
WE'LL BE TALKING ABOUT AND TO
MOTIVATE TRANSPARENCY, ONE ROOM
WILL CONTINUE ON DATA SYNTHESIS,
SYNTHETIC DATA GENERATION.
THE SPECIFIC EXAMPLE HERE IS
THIS PROJECT CALLED FASTMRI, A
COLLABORATION BETWEEN FORMERLY
FACEBOOK A.I. NOW META A.I. AND
NYU MEDICAL SCHOOL.
AND THE GOAL OF THIS PROJECT
ESSENTIALLY IS TO ACCELERATE AND
IMPROVE ACCESS AND IMPROVE
THROUGHPUT TO MRI TECHNOLOGY.
WHAT HAPPENS HERE?
THERE'S A CLEAR NEED FOR
IMPROVEMENT IN THIS DOMAIN.
ACQUISITION TIME IN MRI CAN
EXCEED 30 MINUTES, LEADING TO
PROBLEMS WITH COMFORT AND
COMPLIANCE AND ARTIFACTS FROM
PATIENT MOTION.
A CHALLENGE FOR MRI SYNTHESIS IS
THE LACK OF PUBLIC DATA AND
VALIDATION METHODS.
THIS IS WHAT THIS PROJECT IS
ATTEMPTING TO ADDRESS.
WHAT DO THEY DO?
THEY GENERATE SOME PRELIMINARY
SCANS THAT ARE GENERATED MUCH
MORE FAST THAN TRADITIONAL MRI
AND FILL IN THE BLANKS USING
DATA, RIGHT?
SO THIS IS A KIND OF SYNTHETIC
DATASET YOU GET AT.
THERE'S A CLEAR NEED FOR
IMPROVEMENT HERE.
THIS TECHNOLOGY HAS A RIGHT TO
EXIST AND THESE DATASETS ARE
VERY NECESSARY TO HELP US CREATE
SYNTHETIC MRI SCANS.
THE OTHER THING HERE THAT MAKES
THIS PARTICULAR PROJECT
APPROPRIATE WITHIN RESPONSIBLE
A.I. REALM IS WE CAN VALIDATE
THE QUALITY AND USEFULNESS OF
THE DATA AND OF ANY MODELS THAT
MAY BE TRAINED ON THIS DATA.
HOW DO WE DO THIS?
THERE'S A GROUND TRUTH, YOU CAN
COMPARE SYNTHETIC MRI IMAGE WITH
GROUND TRUTH IMAGE AND SEE IF
THEY DIFFERN WHAT WAY.
IN THIS PROJECT AND PUBLICATIONS
AUTHORS SPEAK ABOUT WAYS THEY
VALIDATE THE QUALITY OF THE
DATASETS GENERATED.
THESE VALIDATION METHODS ARE
BASED ESSENTIALLY ON STATISTICS,
BASED ON COMPARING WHETHER
PICTURES ARE SIMILAR AND
COMPARISONS BETWEEN BASIC
STRUCTURE, RIGHT?
ONE OF THE THINGS WE NEED TO
THINK ABOUT WHETHER THESE
COMPARISONS ARE APPROPRIATE,
WE'LL COME BACK IN JUST A
SECOND.
ANOTHER REASON WHY I LIKE TO
UNDERSCORE THIS PROJECT TO BRING
IT UP IS THAT WE ARE TECHNICALLY
READY TO BE ABLE TO GENERATE
THESE SYNTHETIC MRI SCANS.
THIS IS BECAUSE WE'RE ABLE TO
COLLECT THE DATA, WE HAVE
ALGORITHMIC TOOLS, THE HARDWARE,
TO BE ABLE TO ACTUALLY COME UP
WITH THESE DATASETS, RIGHT?
IT'S NOT A PIE IN THE SKY KIND
OF THING.
THIS IS SOMETHING WE CAN DO AS A
COMMUNITY.
AND THE FINAL POINT, EXTREMELY
IMPORTANT, IS THAT IN THIS
PARTICULAR DOMAIN UNLIKE MANY
OTHER DOMAINS WHERE A.I. AND
SYNTHETIC DATA ARE USED
AMBITIOUSLY, WE HAVE OR HOPE TO
HAVE DECISION MAKER READINESS.
THE CLINICIANS WHO ARE THEN
LOOKING AT SYNTHETIC SCANS OR
RESEARCHERS, THEY UNDERSTAND
WHAT THEY ARE DEALING WITH,
TRAINED IN MEDICAL ETHICS, IN
THE CASE OF RADIOLOGIST,
UNDERSTAND THE DECISION THEY ARE
GOING TO MAKE AS A RESULT TO
DIAGNOSE A PATIENT IN A
PARTICULAR WAY IS THEIR
DECISION.
SAYING THE COMPUTER TOLD ME SO
IS NOT GOING TO CUT IT, RIGHT?
AND THIS IS REALLY WHERE WE GET
TO OUR CONVERSATION OF
TRANSPARENCY, RIGHT?
IT'S BEING ABLE TO REASON
CAREFULLY ABOUT WHETHER THIS
PARTICULAR TECHNOLOGY OR
TECHNOLOGY LIKE IT AND WHETHER
THIS PARTICULAR DATASET IS
SOMETHING THAT A HUMAN DECISION
MAKER WOULD BE COMFORTABLE
USING.
RIGHT?
SO WHAT COULD BE SOME OF THE
CONCERNS THAT MAYBE SURFACE IN A
SYNTHETIC MRI DATASET?
DOES ANYBODY HAVE THOUGHTS?
IT MAY NOT BE REPRESENTATIVE,
RIGHT, SO THIS PARTICULAR STUDY
SAYS IS ABOUT KNEE MRI AND BRAIN
MRI, DOES NOT REPRESENT OTHER
TYPES OF ORGANS OR TISSUES.
IT MAY NOT BE REPRESENTATIVE OF
A VARIETY OF POPULATIONS OF
PATIENTS, THAT WE MAY WANT TO BE
USING OUR MODELS ON, ON WHICH
THE DATA IS TRAINED.
WHAT ELSE MAY BE A CONCERN HERE?
HOW GOOD -- RIGHT, HOW GOOD IS
THIS DATA ACTUALLY, IS IT
SUFFICIENT FOR US, IN OTHER
WORDS, TO BE USING EXISTING
METHODS TO EVALUATE DATA
SYNTHESIS TO TELL WHETHER THIS
DATA WILL ULTIMATELY BE -- THIS
IS THE HOLY GRAIL,
DIAGNOSTICALLY INTERCHANGEABLE
WITH GROUND TRUTH MRI SCANS,
RIGHT?
WHAT DO WE NEED TO DO TO MAKE
SURE WE CAN CHECK IN FACT THAT
THESE SYNTHETIC IMAGES ARE GOOD.
WHAT ELSE MAY BE SOME CONCERNS?
>> [OFF MICROPHONE]
>> YES, EXACTLY.
SO ARE WE ABLE TO GET PATIENT
BUY-IN, RIGHT?
ARE WE ABLE TO CONVINCE THE
PATIENTS THE DATA IS IN FACT AS
GOOD, THAT IT'S DIAGNOSTICALLY
INTERCHANGEABLE FOR ALL INTENTS
AND PURPOSES WITH THE REAL DATA.
OTHER STAKEHOLDERS, AND WE'LL DO
A LOT OF STAKEHOLDER ANALYSIS,
ARE THE CLINICIANS, RIGHT?
THE RADIOLOGISTS THEMSELVES, ARE
THEY TRUSTING THIS DATA.
AND WHAT DO WE NEED TO DO TO
MAKE SURE THEY ARE ABLE TO
REASON ABOUT THE QUALITY OF THIS
DATA AND FITNESS FOR USE FOR THE
TASK THEY HAVE AT HAND?
DO WE NEED TO TEACH THEM, TRAIN
THEM, SHOW SOMETHING IN THE WAY
OF CONFIDENCE INTERVALS FOR
EXAMPLE TOGETHER WITH THE DATA?
AND THEN ANOTHER CONCERN I WILL
RAISE IS PRIVACY.
SO THIS DATA THAT IS BEING
RELEASED HERE, GROUND TRUTH DATA
WAS ANONYMIZED BUT IS THIS
ENOUGH?
AND WE WILL TALK ABOUT LOTS AND
LOTS OF OTHER ASPECTS OF THIS
AND SIMILAR USE CASES, BUT WHAT
I WANT US TO BE THINKING ABOUT
FROM HERE IS THAT ULTIMATELY,
THIS USE CASE AND MANY OTHERS
WILL LEAD US INTO THE
CONVERSATIONS ABOUT
TRANSPARENCY, SO WHAT IS
TRANSPARENCY?
I WAS ASKED TO DEFINE THE TERM.
AND UNFORTUNATELY, THERE IS NO
CONSENSUS IN THE COMMUNITY ABOUT
WHAT TERMS TO USE TO DENOTE, IS
IT TRANSPARENCY INTERPRETABILIT?
EXPLAINABILITY?
INTELLIGIBILITY?
AND WHAT DO THE TERMS MEAN.
THEY ARE USED OFTEN
INTERCHANGEABLY.
USUALLY, THERE'S A DISTINCTION
BETWEEN TERMS THAT DENOTE
PROPERTIES OF A MODEL AS BEING
INTERPRETABLE.
SO, FOR EXAMPLE, CINDY RUDIN
SPEAKS ABOUT INTERPRETABLE
MACHINE LEARNING AS HAVING A
FOCUS ON DESIGNING MODELS THAT
ARE INHERENTLY INTERPRETABLE.
SO THIS IS A BIT OF A CIRCULAR
DEFINITION, RIGHT?
LIKE A DECISION TREE WOULD BE
INTERPRETABLE BUT THE NEURAL
NETWORK IS NOT.
AND A PERSON CAN PREDICT
UNDERSTAND OF A MODEL, MILLER
DEFINITION, SHIFTS THE FOCUS
AWAY FROM A MODEL AND BRINGS IN
THE PERSPECTIVE OF A HUMAN.
AND THIS IS A PERSPECTIVE THAT I
REALLY WANT US TO HAVE HERE.
THE WAY I MAKE SENSE OF THESE
TERMS IS ARRANGING THEM ON A
CONTINUUM, FROM TRANSPARENCY
AROUND INTERPRETABILITY
REFERRING TO PROPERTIES OF
TECHNICAL COMPONENT, ALGORITHM,
DATASET, SYSTEM, TO
EXPLAINABILITY AND INTEL
INTERCHANGEABLABLY, RATHER THAN
HOW IT WORKS.
I SUGGEST WE DON'T GET TANGLED
UP IN TERMINOLOGY, IF WE'RE TO
PICK ONE TERM LET'S TAKE
TRANSPARENCY BUT KEEP IN MIND
THAT OUR GOAL HERE IS TO ALLOW
PEOPLE, DIFFERENT KINDS OF
PEOPLE, PATIENTS, DOCTORS,
SOCIETY AT LARGE, REGULATORS,
AUDITORS, RESEARCHERS, TO
UNDERSTAND DATA, MODEL, OUTPUT,
TO ACT IN WHATEVER WAYS THAT ARE
APPROPRIATE FOR THEM TO ACT.
THE OVERARCHING GOAL IS NODES OF
RESPONSIBILITY TO PEOPLE, WHAT
LAURA TOLD US, TO SUPPORT
RESPONSIBLE DESIGN, DEVELOPMENT,
USE OF A.I., IN THIS CASE
COLLECTION AND USE OF BIOMEDICAL
DATA.
SO WHEN WE THINK ABOUT
TRANSPARENCY MORE CONCRETELY
THERE ARE A COUPLE QUESTIONS
THAT WE NEED TO BE ASKING.
ONE OF THEM IS TRANSPARENCY OF
WHAT?
TRANSPARENCY IS NOT ANALYTIC
CONCEPT.
WE NEED TO START SPLITTING IT UP
INTO COMPONENTS THAT WE CAN MORE
EASILY UNDERSTAND.
ARE WE INTERESTED IN DATA
TRANSPARENCY?
UNDERSTANDING HOW THE DATA CAME
TO BE?
WHETHER IT'S PRIVACY PRESERVING,
WHAT IT CONTAINS, WHAT IT'S
MISSING, WHAT IS APPROPRIATE
CONTEXT OF USE?
IS IT UNDERSTANDING THE MODELS
SO WE CAN ASSESS THEIR
PROPERTIES IN TERMS OF
ROBUSTNESS, IN TERMS OF
CORRECTNESS, IN TERMS OF
PERFORMANCE OVERALL AS SPECIFIC
SUBSETS OF DATA.
OR MAYBE WE'RE WANTING TO BRING
TRANSPARENCY OF THE OVERALL
PROCESS.
AND THIS IS SOMETHING THAT I
WOULD LIKE TO FOCUS ON HERE
BECAUSE WE DON'T REALLY THINK
HOLISTICALLY WHEN WE'RE STARTING
CONVERSATIONS.
WE'RE IN THE STATE OF DARK
THINKING ABOUT TRANSPARENCY, AND
DIFFERENT TOOLS AND ARTIFACTS
THAT HAVE BEEN DEVELOPED USUALLY
WE THINK ABOUT DATA FIX FOR
DATASETS OR MODEL CARDS FOR
MODEL PERFORMANCE, RIGHT?
BUT REALLY IT'S A BIG ECOSYSTEM
LIKE LAURA TOLD US ALSO, RIGHT?
THIS IS SOMETHING THAT I THINK
IS MISSING FROM THE
CONVERSATION, AND WHERE WE HAVE
AN OPPORTUNITY TO DEVELOP NEW
METHODS, AND THINK ABOUT
TRANSPARENCY THROUGH THE LIFE
CYCLE OF OUR ENTIRE SYSTEM
STARTING FROM THE QUESTION OF
WHY THE SYSTEM IS HERE, MOVING
TO DECIDING WHAT DATA TO
COLLECT, HOW TO ANNOTATE DATA,
PROCESS THE DATA, PREPARING IT
PERHAPS TO TRAIN A MODEL, AND
HOW TO REASON ABOUT MODELS'
PREDICTIONS.
SO, HERE I WANT TO GIVE ANOTHER
EXAMPLE.
HOW MUCH TIME DO I HAVE?
PERFECT.
AND THIS EXAMPLE -- I DON'T
ACTUALLY HAVE A LOT OF
EXPERIENCE IN THIS DOMAIN
SPECIFICALLY BUT I HOPE WE ALL
WILL BE ABLE TO RELATE TO THE
EXAMPLE I'M SHOWING.
AND IT'S FROM ALGORITHMIC
HIRING, A DOMAIN I'VE BEEN
DEEPLY ENGAGED IN.
SO, HERE CONSIDER ANN, A DATA
SCIENTIST AT A MAJOR RETAILER,
WHO WAS ASKED TO DEVELOP A
PREDICTOR TO DECIDE WHAT SALARY
TO OFFER TO INDIVIDUALS WHO
APPLIED FOR THE JOB AND WHO THE
COMPANY WANTS TO HIRE.
AND SHE HAS REPORT OF
DEMOGRAPHICS AND EMPLOYMENT
HISTORY OF JOB APPLICANTS.
FOLLOWING HER COMPANY'S BEST
PRACTICES SHE'S GOING TO SPLIT
HER DATA INTO TRAINING
VALIDATION AND TEST, THIS IS
WHAT WE DO IN ANY DOMAIN, RIGHT?
AND SHE'S GOING TO DO
PRE-PROCESSING OF DATA SO
SPECIFICALLY THE TYPE OF
PRE-PROCESSING I WANT TO FOCUS
ON SHE WILL FILL IN SOME MISSING
VALUES, INTERPOLATE MISSING
VALUES, MODEL TRAINING, TUNING,
VALIDATION.
WHEN LOOKING AT PER FORM
PERFORMANCE SHE'S GOING TO
NOTICE AND ISSUE, THAT ACCURACY
OF PREDICTION, SALARY LEVEL,
WILL BE LOWER SYSTEMATICALLY
LOWER FOR WOMEN WHO HAVE MORE
EXPERIENCE ON THE JOB.
THIS IS A FAIRNESS CONCERN,
RIGHT?
SO NOW BASED ON HER OWN LIVED
EXPERIENCE, ANN IS GOING TO TRY
TO FIGURE OUT WHY THIS IS
HAPPENING.
AND SHE WILL PAY ATTENTION
SPECIFICALLY TO THE DATA
INTERPOLATION STEP FOR THE
NUMBER OF YEARS OF EXPERIENCE.
AND SHE INTERPOLATES BY DEFAULT
IN THE PIPELINE PICKING THE MEAN
VALUE FROM ALL THE KNOWN VALUES,
RIGHT?
HER RESULT IS PRESENTED AT TOP
RIGHT, BUT GROUND TRUTH DATA IS
OLDER, NUMBER OF YEARS OF
EXPERIENCE IS A STRONG PROXY FOR
AGE, AND THAT'S WHAT WE'RE USING
TO DEPICT THE SITUATION.
IN THIS PARTICULAR DOMAIN, WHY
WOULD THIS BE HAPPENING, THAT
MISSING VALUE IMPUTATION THAT
ASSUMES VALUES ARE MISSING AT
RANDOM DOESN'T WORK LEADING US
TO TAKE THE MEAN OF THE
DISTRIBUTION, AND MISSING AT
RANDOM MEANS WHETHER OR NOT THE
VALUE IS MISSING DOESN'T DEPEND
WHAT THE VALUE IS.
BUT HERE VALUES ARE NOT MISSING
AT RANDOM BECAUSE OF AGE-BASED
DISCRIMINATION IN HIRING.
PEOPLE WHO ARE OLDER, WHO HAVE
MORE YEARS OF EXPERIENCE, THEY
ARE WITHHOLDING THAT
INFORMATION, RIGHT?
AND SO THINKING OF INTERPOLATION
STEP, SKEWING THIS DATA YOUNGER,
AND SALARIES ARE LOWER, AND IF
THESE PEOPLE MANY WOMEN WERE TO
ACCEPT THE OFFERS BEING MADE BY
THIS COMPANY, THEN THIS WOULD
REINFORCE THE GENDER WAGE GAP.
THIS IS SIMPLE BUT OFTEN
OVERLOOKED, BUT TRUE REALLY IF
YOU THINK ABOUT IT, IN THE WAY
THAT WE'RE ASSESSING THE QUALITY
OF THE DATA AND THE WAY IN WHICH
WE'RE PRE-PROCESSING DATA
LEADING UP TO MODEL TRAINING,
THERE ARE ADDITIONAL QUESTIONS
HERE.
FOR EXAMPLE, HOW ARE WE
INTERPOLATING WHERE CATEGORIES,
ARE WE INTERPOLATING MISSING
VALUES AS BELONGING TO ONE OF
THE RARE CATEGORIES, IF NOT
WE'LL BE SKEWING TO IMPUTE
VALUES ONLY IN THE MAJORITY
CATEGORIES.
EXAMPLE HER IS THE NATIVE
AMERICAN.
ARE WE REPRESENTING ALL THE
FUTURE VALUES, THE EXAMPLE HERE
IS WHAT IF GENDER, SOME IDENTIFY
AS NON-BINARY BUT INPUT FORM
SAYS MALE, FEMALE, OR LEAVE
BLANK OR OTHER, RIGHT?
AND EVERYBODY WHO IDENTIFIES AS
NON-BINARY IS GOING TO SELECT
OTHER OR LEAVE THE VALUE BLANK
AND INTERPOLATED AS MALE OR
FEMALE, JUST AN INCORRECT DATA
REPRESENTATION.
SO, SIMPLE THINGS LIKE THIS ARE
VERY IMPORTANT TO BUILD INTO OUR
THINKING ABOUT HOW WE PROCESS
DATA AND HOW WE DISCLOSE TO
OURSELVES AND ALSO TO THOSE
CONSUMING THE DATA, SOME OF
THESE STEPS THAT WENT INTO DATA
PROCESSING.
OTHERS ARE SIMPLE, FOR EXAMPLE
DATA FILTERING, WE CHANGE
DEMOGRAPHIC GROUP PROPORTIONS,
IF THESE OPERATIONS ARE AMONG
THE TENS OR HUNDREDS OF DATA
PRE-PROCESSING OPERATIONS YOU
THINK YOU'RE GOING TO LOSE THAT
INFORMATION BY THE TIME YOUR
MODEL IS TRAINED.
SO SOME OF THE CHALLENGES HERE
THAT WE'RE FACING IS BUILDING
TOOLS THAT WILL ALLOW US TO BE
TRANSPARENT EXPLAINING TO THE
DATA SCIENTIST BUT ALSO TO THOSE
CONSUMING THE RESULTS HOW DATA
PRE-PROCESSING AND WHETHER DATA
PRE-PROCESSING CHANGES SOME
BASIC DEMOGRAPHIC STATISTICS OF
THE POPULATIONS THAT WE'RE
WORKING WITH.
HERE I'M CORRELATING ONE OF THE
TOOLS THAT MY COLLABORATORS AND
I HAVE BUILT, THAT ALLOW YOU TO
TRACE INFORMATION ABOUT DATA
PRE-PROCESSING THROUGH THE
PIPELINE BUT THIS IS ONLY THE
FIRST STEP.
WE NEED MUCH MORE CAPABLE TOOLS
HERE.
THE OTHER QUESTION I WILL END ON
THIS, WE NEED TO BE ASKING, IS
TRANSPARENCY FOR WHOM AND WHY?
TO WHOM ARE WE EXPLAINING WHAT
WE NEED TO EXPLAIN?
AND WHY ARE WE EXPLAINING?
WHAT DO WE WANT THE RECIPIENTS
OF THE INFORMATION TO DO WITH
THIS INFORMATION?
AND FINALLY ARE THESE
EXPLANATIONS EFFECTIVE, ARE
PEOPLE UNDERSTANDING WHAT WE'RE
EXPLAINING TO THEM?
AND I HOPE THAT WE ALL WILL HAVE
TIME TO REFLECT ON THE
PARTICULAR TRANSPARENCY
MODALITIES IN OUR BREAKOUTS,
THAT WOULD BE APPROPRIATE FOR
DIFFERENT STAKEHOLDERS, ALSO
WILL REFLECT ON WAYS WE CAN
CHECK INFORMATION WAS ACTUALLY
REACHING THE RECIPIENT.
HERE TO COMPLEMENT WHAT AARON
WILL TELL US, I WANT TO SHOW TO
YOU THIS METAPHOR OF NUTRITIONAL
LABEL, THAT I THINK CAPTURES
BOTH PUBLIC FACING, PATIENT
FACING DISCLOSURE OF
INFORMATION, BUT ALSO CAN BE
USED VERY FLEXIBLY TO TELL US
PROPERTIES OF THE DATA, THE
MODEL, AND OVERALL PROCESS.
NUTRITIONAL LABELS SHOULD BE
COMPREHENSIBLE, SHORT, SIMPLE,
CLEAR.
THEY SHOULD BE CONSULTATIVE,
AND COMPUTABLE.
AUTOMATICALLY COMPUTABLE, TO
MAKE IT SO A PERSON WHO IS
IMPLEMENTING A DATA
TRANSFORMATION PIPELINE DOESN'T
ALSO HAVE TO WORK EXTRA HARD TO
DOCUMENT IT.
TO GENERATE THE NECESSARY
INFORMATION FOR TRANSPARENCY.
AND A CHALLENGE HERE IS BOTH
CREATING TOOLS THAT WILL ALLOW
US MANY STAKEHOLDERS TO HAVE AN
UNDERSTANDING OF THE DATA,
MODELS, THE PROCESS, BUT ALSO TO
MAKE SURE THAT THESE NUTRITIONAL
LABELS OR OTHER TRANSPARENCY
TOOLS WORK.
FOR THIS I'M HIGHLIGHTING
THERE'S A HUGE CHALLENGE, A BIG
GAP IN THE WAY THAT WE ARE
EDUCATING PEOPLE WHO ARE EITHER
BEING SUBJECTED TO A.I. OR ARE
PRODUCING OR CONSUMING DATA THAT
WAS COLLECTED AS WELL AS MODELS
AND RESULTS.
AND SPECIFICALLY HERE I WOULD
LIKE TO HIGHLIGHT THAT WE HAVE
BEEN DOING SOME WORK ON
ALGORITHMIC TRANSPARENCY AND
TEACHING DIFFERENT PEOPLE ABOUT
ALGORITHMIC TRANSPARENCY, BUT WE
HAVE NOT YET HAD THE PLEASURE OF
DEVELOPING WORKSHOPS AND
TRAINING PEOPLE IN THE
BIOMEDICAL DOMAIN SO THIS WOULD
BE THE NEXT FRONTIER.
HERE ARE A COUPLE REFERENCES
THAT I'M HAPPY TO SHARE AND SOME
PEOPLE WHO CONTRIBUTED TO THE
WORK THAT I DISCUSSED TODAY AND
IT WAS SUPPORTED BY THE NATIONAL
SCIENCE FOUNDATION.
I'D LIKE TO HIGHLIGHT THE
ALGORITHMIC TRANSPARENCY THAT IS
AVAILABLE IF YOU'D LIKE TO TAKE
A LOOK.
THANKS.
[APPLAUSE]
>> PLEASE PUT SOME QUESTIONS IN
THE SLIDO ONLINE OR IN THE ROOM
AND WE'LL TRY AND GET ANSWERS
AND NOW IT'S MY PLEASURE TO
INTRODUCE TINA
HERNANDEZ-BOUSSARD FOR THE NEXT
PRESENTATION.
>> THANK YOU.
OKAY, GOOD MORNING.
IT'S SUCH A PLEASURE TO BE HERE.
THANK YOU FOR THE ORGANIZERS FOR
HAVING ME LIKE START OFF THIS
WONDERFUL WORKSHOP.
IT'S A PLEASURE TO BE HERE WITH
FRIENDS, COLLEAGUES, NEW FACES
THAT I HOPE TO LEARN SO MUCH.
I WAS TASKED WITH GIVING A TALK
ABOUT WHAT CAN GO WRONG.
I WAS LIKE, WELL, GOSH, THERE'S
SO MUCH THAT IS WRONG AND
THERE'S SO MUCH TO TALK ABOUT, I
WASN'T SURE HOW TO REALLY TACKLE
THIS.
I'M GOING THROUGH A COUPLE
EXAMPLES AND HIGHLIGHT WHERE
THINGS HAVE GONE WRONG, AND WHAT
THIS IS THIS HAS CAUSED
REGARDING SOCIETAL AND
POPULATION HARM.
SO, WE KNOW WITH A.I. THERE'S SO
MUCH OPPORTUNITY, I DON'T NEED
TO EXPLAIN THIS TO THIS GROUP
BUT A.I. HAS THE OPPORTUNITY TO
REALLY TRANSFORM OUR ABILITY TO
DIAGNOSE, TREAT, AND REALLY
IMPROVE PATIENT OUTCOMES, CAN
HELP WITH CLINICAL DECISION
MAKING, GETTING THE RIGHT TYPE
OF INFORMATION, QUICKLY AT POINT
OF CARE IN THE HANDS OF
PATIENTS, CLINICIANS, HEALTH
CARE SETTINGS, AND GENERATIVE
A.I. IS A NEW ENDEAVOR THAT
WE'RE SO EXCITED ABOUT.
WE DON'T EVEN KNOW ALL THE
OPPORTUNITIES.
IT'S IMPORTANT TO NOTE THIS IS
REALLY A DOUBLE-EDGED SWORD.
BECAUSE WHILE A.I. HAS ITS
PROMISE TO IMPROVE DATA DRIVEN
MEDICINE HAS THE POTENTIAL TO
WIDEN DISPARITIES BEGAN AND
CREATE SOCIETAL AND POPULATION
HARM.
LAURA SAID WE'RE NOT JUST
FOCUSED ABOUT A.I. IN MEDICAL
PRACTICE BUT THROUGHOUT OUR
ENTIRE RESEARCH SYSTEM.
WHEN WE HAVE A.I. THAT IS
ETHICAL OR BIASED OR FLAWED IN
SOME SENSE IT TRICKLES THROUGH
THE ENTIRE HEALTH CARE SYSTEM.
SO WE'RE USING A.I. IN
BIOMEDICAL RESEARCH, AUTOMATED
EXPERIMENTS, SYNTHETIC DATA
GENERATION, ET CETERA, IN
TRANSLATIONAL RESEARCH
BIOMEDICAL DISCOVERY, DRUG
TARGETING, PRIORITIZATION, AND
THEN ALSO THIS ALL LEADS TO
MEDICAL PRACTICE.
HOW WE USE THESE RESEARCH AND
TRANSLATIONAL DATA TO REALLY
IMPROVE DIAGNOSIS, TREATMENT
SELECTION, ET CETERA.
SO, WHERE DOES HEALTH CARE --
A.I. AND HEALTH CARE GO WRONG?
THERE'S SO MANY EXAMPLES.
I DON'T WANT TO GIVE YOU A
SYSTEMATIC REVIEW OF ALL OF
THESE DIFFERENT ASPECTS BUT
THERE'S SO MANY AREAS AND
CONCRETE EXAMPLES WHERE A.I. HAS
ACTUALLY BEEN IMPLEMENTED AND
CAUSED HARM.
THERE'S BIAS IN ALGORITHMIC
DECISION MAKING, A PAPER FROM
2019 WHERE THEY SHOWED AN
ALGORITHM DEPLOYED IN A
HEALTHCARE SETTING, ACTUALLY THE
OUTCOME WAS USED, THERE WAS A
PROXY, SO THE ALGORITHM WAS
DEVELOPED TO PREDICT COST, THEY
IMPLEMENTED IT TO PREDICT NEED.
SO THEY USED COST AS A PROXY FOR
NEED WHILE WE KNOW THAT WAS
BIASED BECAUSE DIFFERENT
INDIVIDUALS IN A HEALTHCARE
SETTING HAVE LESS ACCESS TO
CARE, SPEND LESS MONEY ON CARE,
SO THAT ENDED UP THE RESULT OF
THAT WAS THAT BLACK AMERICANS,
FOR EXAMPLE, HAD TO BE TWICE
AS -- MUCH MORE SICKER THAN
THEIR COUNTERPART AS
NON-HISPANIC WHITE TO GET THE
SAME AMOUNT OF RESOURCES.
WHERE ELSE HEALTH CARE GOES --
OTHER PLACES WHETHER WE SEE
PROBLEMS IS IN PRIVACY BREACH,
DATA MISUSE.
SO THERE'S -- I'M IN SILICON
VALLEY, THERE'S A HUGE COMPONENT
OF HEALTH CARE COMPANIES, TECH
COMPANYING EATING UP AND REUSING
DATA.
WE SEE PROBLEMS WITH DATA
BREACHES.
UNAUTHORIZED DATA SHARING, AND
UNDERSTANDING HOW THIS CAN HARM
POPULATIONS AND IN PATIENTS AND
PEOPLE AND TRANSPARENCY HOW WE
THINK ABOUT THESE PRIVACY
BREACHES IS REALLY IMPORTANT.
WE HAVE FAULTY OR INACCURATE
MEDICAL DEVICES, WE HEARD ABOUT
PULSE OXIMETER IN COVID,
PIGMENTATION, WORKING BETTER FOR
FAIR-SKINNED PATIENTS AND SO
THERE'S A HUGE BIAS AND
DISCREPANCY THAT HAPPENED THAT
I'LL TALK MORE ABOUT.
THERE'S A RELIANCE ON TECHNOLOGY
SO WE'RE SEEING MORE AND MORE
EVIDENCE COMING OUT WHERE
CLINICIANS WITH BLINDLY USING
THE OUTPUTS OF A.I. ALGORITHM
EVEN WHEN ALGORITHMS GIVE THEM
FALSE INFORMATION THEY ARE
RELYING ON THIS INFORMATION,
REALLY IMPORTANT TO THINK ABOUT
OVERRELIANCE OF TECHNOLOGY AND
HOW THINGS CAN GO WRONG.
FIRST AN A.I. ALGORITHM, TRIAGE
TOOL IN E.D. TO DETECT PATIENTS
WITH HEART ATTACK SYMPTOMS AND
TRIAGE THEM TO DIFFERENT TIMES
OF CARE.
WE KNOW FROM MEDICAL LITERATURE
AND CLINICAL TRIALS MAJORITY OF
PATIENTS IN THE POPULATIONS ARE
MALE PATIENTS, FROM HISTORICALLY
LOOKING BACK IN CLINICAL, USING
DATA TO TRAIN THE ALGORITHM,
REALLY GOOD AT DETECTING HEART
ATTACK SYMPTOMS FOR MEN, LESS
ACCURATE FOR WOMEN, WOMEN HAVE
DIFFERENT SYMPTOMS.
THIS WAS DEPLOYED IN THE E.D.
SYSTEM AND THERE WAS THIS
SYSTEMATIC BIAS IN THE ABILITY
TO DETECT FEMALES WHO HAD HEART
ATTACK SYMPTOMS COMPARED TO
MALES.
DERMATOLOGY DIFFERENCES,
ACCURACY, DEVICES TO DETECT
MELANOMA.
A LOT OF THE A.I. TECHNOLOGY ON
SKIN LESIONS HAS BEEN TRAINED ON
FAIR SKINNED PATIENTS.
HEALTH CARE OR ANY TYPE OF
SETTING, THE ABILITY TO DETECT
MELANOMA ON A FAIR SKINNED
PATIENT IS MUCH MORE HIGHER THAN
THE ABILITY FOR THESE TOOLS TO
DETECT MELANOMA ON A PATIENT
WITH DARKER SKIN, OF THE
CLINICIANS ARE RELYING MORE ON
ALGORITHMS SO UNDERSTANDING WHO
THEY ARE TRAINED ON, HOW THESE
ALGORITHMS WORK ACROSS DIFFERENT
POPULATIONS IS KEY.
COVID-19 WE SAW NEED FOR A.I. TO
IMPROVE DECISION MAKING AND HELP
US UNDERSTAND THE UNKNOWN BUT
THERE WERE A LOT OF FAILURES FOR
A.I. IN COVID THAT HARMED
PATIENT TRUST, PROVIDER TRUST,
AND THE WHOLE AREA FOR THIS.
AND SO WHAT HAPPENED WAS A LOT
OF THESE ALGORITHMS IN THE NEED
TO GET SOMETHING OUT VERY
RAPIDLY WERE TRAINED ON SMALL
DATASETS, VERY
UNDERREPRESENTATIVE DATA SETS,
RAPIDLY DEPLOYED INTO HEALTH
CARE SYSTEMS, SO YOU SEE THINGS
LIKE THIS FROM THE "NEW YORK
TIMES" WHERE IS A.I. WORSENING
COVID-19 ON BLACK AMERICANS?
THESE COME OUT IN THE POPULATION
THAT BREAK DOWN THE TRUST
BARRIER.
MORE RECENTLY, WE'VE DONE SOME
WORK LOOKING AT RACE-BASED
MEDICINE, A LOT OF PEOPLE HERE
TODAY, WHERE WE WERE LOOKING AT
HOW OFTEN -- HOW IS RACE USED IN
THESE ALGORITHMS, AND WHAT IS IT
A PROXY FOR?
AND SO THIS WORK IS --
RACE-BASED MEDICINE IS MEDICAL
PRACTICE THAT'S GUIDED BY
ALGORITHMS THAT INCLUDE RACE OR
ETHNICITY.
BUT IT'S IMPORTANT TO NOTE RACE
IS MORE THAN LIKELY -- MORE
OFTEN THAN NOT A PROXY FOR OTHER
TYPES OF VARIABLES THAT
CONTRIBUTE TO DISPARITIES AND
OUTCOME OF DISEASE.
RACE FAILS TO ACCOUNT FOR OTHER
FACTORS ASSOCIATED WITH HEALTH
OUTCOMES, AND REINFORCES BIAS.
SO, IT'S REALLY IMPORTANT TO
UNDERSTAND HOW THIS HAS BEEN
USED HISTORICALLY IN SOME OF THE
ALGORITHMS THAT HAVE BEEN PUT IN
PLACE AND HOW THIS TRANSPARENCY
COULD REALLY IMPROVE THE
SITUATION.
LET ME GIVE TWO EXAMPLES.
SO, KIDNEY DISEASE AND EGFR,
THIS IS A VERY WELL-KNOWN
EXAMPLE IN THE LITERATURE.
SO EGFR IS USED TO DIAGNOSE AND
MONITOR KIDNEY DISEASE.
AND PREVIOUSLY IT WAS ADJUSTED
FORAYS. -- FOR RACE, BASED ON
ASSUMPTION BLACK PATIENTS HAD
INHERENTLY HIGHER LEVELS OF
CREATININE.
I DRILLED INTO THIS AND WE'VE
DRILLED INTO THIS AND IT GOES
BACK TO A STUDY OVER 200 YEARS
AGO THAT HAD I THINK MAYBE 30
PATIENTS IN THE STUDY, AND THIS
ASSUMPTION CARRIED FORWARD
THROUGH YEARS AND YEARS.
WHAT HAPPENED WAS THIS RESULTED
IN BLACK PATIENTS BEING
DIAGNOSED LATER AND MORE SEVERE
DISEASE WHICH THEN REDUCED THEIR
CHANCES OF BEING PLACED ON
KIDNEY TRANSPLANT LISTS.
SO THIS IS A CLEAR EXAMPLE OF
HOW THIS A.I. ALGORITHM HAS
HARMED ENTIRE POPULATIONS.
ANOTHER ONE IS VAGINAL BIRTH
AFTER CASAREAN VBAC CALCULATORS,
RISK OF ADVERSE OUTCOME FOR
VAGINAL BIRTH FOLLOWING CASAREAN
BIRTH.
AGAIN, THIS CALCULATOR WAS
ADJUSTED FOR BLACK PATIENTS
BECAUSE IT WAS ASSUMED BLACK
PATIENTS HAD A HIGHER RISK OF
ADVERSE OUTCOME FOLLOWING
CASAREAN BIRTH.
SO WHAT HAPPENED TO THIS WAS IT
OVERESTIMATED RISK OF THESE
ADVERSE EVENTS IN BLACK
PATIENTS, WHICH LED TO
DIFFERENTIAL RECOMMENDATIONS AND
LIMITED TREATMENT OPTIONS.
SO IT'S IMPORTANT TO REMEMBER
WHEN WE'RE USING THESE TYPES OF
VARIABLES IN DATASET THAT RACE
IS OFTEN A SOCIAL CONSTRUCT
WITHIN RACE THAN BETWEEN RACE.
WHEN WE TALK ABOUT TRANSPARENCY
IN DATA IT'S IMPORTANT TO
REMEMBER WHAT ARE PROXY,
CAUSALLY RELATED TO OUTCOME, HOW
WE'RE TRANSPARENT ABOUT THAT.
AND SO BECAUSE OF THESE ISSUES I
THINK THERE'S A REAL ETHICAL AND
SOCIETAL IMPLICATION OF A.I. IN
HEALTH CARE.
ONE KEY POINT I THINK IS WE'RE
REALLY BREAKING DOWN THE PUBLIC
TRUST AND PERCEPTION OF A.I. AND
HOW IT CAN BE USED.
THERE'S SO MUCH OPPORTUNITIES
FOR A.I. TO BENEFIT THE HEALTH
CARE SETTING BUT WE'VE BROKEN
DOWN THE TRUST.
THERE'S GROWING CONCERNS ABOUT
DATA PRIVACY AND SECURITY, LACK
OF STANDARDS TO COMMUNICATE HOW
DATA IS USED AND REUSED, RIGHT?
SO WHEN A PATIENT COMES IN, HOW
DO YOU COMMUNICATE WITH THEM
THAT WHILE I'M GIVING YOU THIS
DIAGNOSIS BASED ON A.I.
ALGORITHM OR I'M GOING TO TAKE
YOUR DATA AND SELL IT TO THE
NEXT COMPANY, TO MAKE A PROFIT
FOR IMPROVING A TOOL, SO THERE'S
LACK OF GUIDELINES HOW WE
COMMUNICATE THAT.
THE REGULATORY AND LEGAL
LANDSCAPE IS REALLY EVOLVING
RIGHT NOW TO ADDRESS THESE NOVEL
ISSUES BUT RIGHT NOW THERE'S
LIMITED POLICIES, REALLY TO
PROTECT INDIVIDUALS AND TO
ENSURE A.I. ALGORITHMS ARE
EQUITABLE ACROSS DIFFERENT
POPULATIONS.
IN ADDITION THERE'S SOCIAL AND
CULTURAL AND ETHICAL NORMS WE
NEED TO THINK ABOUT.
WE MENTIONED SOCIAL DETERMINANTS
OF HEALTH AND HOW WE INCORPORATE
THOSE INTO OUR ALGORITHM.
WHERE IS THAT DATA COMING FROM?
WHO IS IT COLLECTED FROM?
HOW ARE WE CAPTURING THAT
INFORMATION?
IT'S REALLY IMPORTANT TO THINK
ABOUT HOW WE CAN BE MORE
PRO-ACTIVE DEVELOPING
TECHNOLOGIES THAT CAN
SYSTEMATICALLY CAPTURE THIS
INFORMATION.
THERE'S A BALANCE BETWEEN
TECHNOLOGY AND INNOVATION THAT
WE NEED TO THINK ABOUT.
WE ALWAYS THINK ABOUT, YOU KNOW,
HAVING THE TECHNOLOGY WITH
HIGHEST PERFORMANCE, BEST
ACCURACY, AND WE OFTEN DON'T
THINK ABOUT, WELL, HOW IS THE
PARODY OF THE ALGORITHM, HOW
DOES ALGORITHM PERFORM ACROSS
DIFFERENT POPULATIONS,
OFTENTIMES WE TAKE A HIT IN
ACCURACY FOR AN ALGORITHM WHEN
WE MAKE IT MORE EQUITABLE ACROSS
POPULATIONS.
SO SOMETHING ELSE WE NEED TO
THINK ABOUT.
AND THEN ALSO WE NEED TO THINK
ABOUT INEQUITIES IN HEALTH CARE
ACROSS OUTCOMES AND DIGITAL
DIVIDE.
SO WE'RE DEVELOPING THESE
ALGORITHMS BUT WE DON'T PUT A
LOT OF EMPHASIS ON HOW WE CAN
GET THEM TO COMMUNITY HEALTH
CARE SETTINGS, HOW WE CAN GET
THEM TO RURAL HEALTH, WHAT ARE
TRANSPARENCY AROUND HOW THESE
TECHNOLOGIES MIGHT BE
INTEGRATED, PERFORMED, DEPLOYED
ACROSS DIFFERENT COMMUNITIES.
SO HOW DO WE DEVELOP FAIR AND
RELIABLE ALGORITHMS?
THIS IS WORK WE'VE DONE WITH
MADALENA WHO IS HERE, WE'VE
TALKED ABOUT THE LIFE CYCLE, HOW
IT'S NOT JUST ABOUT THE DATA,
NOT JUST ABOUT THE MODEL, IT'S
ABOUT THE WHOLE LIFE CYCLE.
THIS IS SOME WORK WE DID WHERE
WE'RE THINKING YOU HAVE TO THINK
ABOUT HOW THE DATA ARE CREATED,
HOW ARE THE DATA ACQUIRED ONCE
IT'S CREATED?
THE MODEL DEVELOPMENT IS A KEY
ASPECT.
MODEL EVALUATION IS SOMETHING WE
JUST DON'T HAVE ENOUGH
TRANSPARENCY ON.
AND THEN MODEL DEPLOYMENT WHICH
THEN GOES BACK INTO CREATING
MORE DATA SO I WANT TO TALK
ABOUT THESE STAGES IN RELATION
TO TRANSPARENCY.
OKAY.
SO, FIRST FOR DATA CREATION WE
NEED TO KNOW HOW THE DATA ARE
PRODUCED.
IS IT AN ACADEMIC MEDICAL
CENTER?
WHO HAS ACCESS TO THAT ACADEMIC
MEDICAL CENTER?
WHO DOESN'T, MORE IMPORTANTLY?
WHAT'S DATA QUALITY, DATA
DIVERSITY, FOR ACQUISITION WE
HAVE TO GET AN IRB TO ACCESS
DATA SO THERE'S CONSTRAINTS,
THERE'S REGULATORY AND LEGAL
MANDATES FOR USING DATA.
WHAT ARE THOSE?
HOW DO THEY CHANGE THE DATA YOU
ACTUALLY GET TO USE IN YOUR
MODEL?
OFTEN TIMES FOR IRBs WE HAVE
TO AGGREGATE GROUPS BECAUSE
NUMBERS ARE TOO SMALL.
THAT'S IMPORTANT TO UNDERSTAND
WHEN WE MOVE FORWARD BECAUSE HOW
DO WE SEPARATE THOSE OUT TO
ENSURE THAT ALGORITHM WORKS
ACROSS THESE SMALLER GROUPS?
JULIA MENTIONED GROUND TRUTH,
WHO IS DEVELOPING TRANSPARENCY
AND GROUND TRUTH, MEDICAL
STUDENTS ANNOTATING RECORDS?
IS IT CLINICIANS?
DIVERSITY IN DEVELOPING GROUND
TRUTH IS REALLY IMPORTANT IN
UNDERSTANDING THAT.
MODEL EVALUATION, I THINK
THERE'S IMPORTANT INFORMATION
NEEDED TO KNOW WHAT TYPES OF
EVALUATIONS WERE PERFORMED.
AND WHAT WERE PERFORMANCES?
SHOULDN'T BE STUCK IN A
MANUSCRIPT BUT HAS TO BE
TRANSPARENTLY REPORTED.
AND THEN DEPLOYMENT, THINKING
ABOUT TECHNICAL BARRIERS TO
DEPLOY THIS, WHAT ARE THEY?
COMING FROM A HOSPITAL, WHAT ARE
THE FINANCIAL CONSTRAINTS,
FINANCIAL MOTIVES, RETURN ON
INVESTMENT?
ALL THESE ARE REALLY IMPORTANT
ACROSS THE LIFE CYCLE OF A.I.
AND SO I'M GOING TO SKIP THAT
GIVEN TIME.
WE'VE TAKEN THIS, THIS IS THE
PAPER WE JUST GOT OUT WITH
MARSHALL WHO IS HERE AND OTHERS,
ABOUT HOW DO WE BUILD GUIDING
PRINCIPLES ONTO THIS LIFE CYCLE,
THINKING ABOUT TRANSPARENCY?
ONE OF THE OVERARCHING VARIABLE
S , TRANSPARENCY HAS TO BE AT
EVERY STAGE.
THERE'S A LOT OF NEED TO GET
THESE GUIDING PRINCIPLES INTO
CONCRETE RULES AND -- NOT RULES,
I GUESS CONCRETE GUIDELINES TO
MOVE FORWARD WITH DEVELOPING
MORE DIVERSE AND EQUITABLE A.I.
HEALTH CARE SYSTEM.
SO I'M GOING TO FINISH WITH
TALKING ABOUT TRANSPARENCY IN
THE LIFE CYCLE BECAUSE I THINK
THAT IT IS -- SO I'M GOING TO GO
TO THIS SLIDE AND SORRY I'M NOT
ARTISTIC BUT YOU HAVE TO HELP ME
WITH THIS BUT IT'S REALLY THIS
PSYCH -- CYCLE WHERE WE HAVE
LIMITED TRAINING DATA, MODELS
THAT WORK ON ALL POPULATIONS,
WE'RE NOT SURE WHERE WE CAN AND
CANNOT APPLY THESE MODELS.
THIS LEADS TO DISPARITIES IN OUR
OUTCOMES, LEADS TO TREATMENTS
INEQUITABLE ACROSS POPULATIONS,
THEY CAN BE PREJUDICED,
INACCURATE, NOT PERFORM WELL
ACROSS MINORITY GROUPS, BREAKING
DOWN TRUST, BREAKS DOWN THE
CONFIDENCE IN OUR A.I. SYSTEM.
THAT BRINGS US BACK TO THE FACT
THAT BECAUSE WE HAVE BROKEN THIS
TRUST, PEOPLE DON'T WANT TO
PARTICIPATE IN OUR RESEARCH.
THEY DON'T WANT TO PARTICIPATE
IN CLINICAL TRIALS.
SO WE BREAK DOWN THIS ENGAGEMENT
WHICH THEN GOES BACK TO THE FACT
THAT WE DON'T HAVE DATA TO
REPRESENT ALL THESE POPULATIONS
AND WE DON'T HAVE GOOD
TECHNOLOGIES TO GET THEM MORE
INVOLVED.
SO, WITH THAT, THANK YOU SO
MUCH.
I HOPE I KIND OF HELPED PUT
THINGS INTO SOME CONCRETE
EXAMPLES OF WHERE THINGS HAVE
GONE WRONG AND WHERE WE NEED TO
THINK ABOUT MOVING THE
TRANSPARENCY TO.
THANK YOU.
[APPLAUSE]
>> THANKS TO ALL OUR SPEAKERS.
WE'RE ABOUT TO HEAD INTO A
BREAK.
LET ME SKIP THROUGH THESE.
HERE IS WHERE WE ARE.
SO IN 15 MINUTES, YOU CAN TAKE
15 MINUTES AND YOU ARE
RESPONSIBLE FOR GETTING YOURSELF
INTO YOUR BREAKOUT ROOM, LET ME
JUST TELL WHAT YOU THOSE ARE
AGAIN.
THEY ARE NOT CORRECT IN WHAT WAS
WRITTEN DOWN.
FOUNDATION MODEL THEMES IN ROOM
260-C.
GENERAL REUSE IN 260-D FOR
DAVID.
MULTI-MODAL IN 260-E FOR EDWARD.
PROXY, YOU'RE IN THE ROOM YOU
WERE ASSIGNED, 150-A, SYNTHETIC
DATA IN 150-B.
>> [OFF MICROPHONE].
>> 260-C AS IN CAT, MY
APOLOGIES.
AND YOU HAVE A 15-MINUTE BREAK
NOW AND YOU'RE RESPONSIBLE FOR
GETTING INTO YOUR BREAKOUT ROOMS
BY 2:30.
BREAKOUT LEADS WILL START ON YOU
THE JOURNEY.
IF YOU'RE DIALING IN REMOTELY,
THEN THE NEXT OPPORTUNITY TO
COME BACK WITH US WILL BE AT A
LITTLE AFTER 4:00 FOR PLENARY
READ OUT AND DISCUSSION.
THERE ARE ELEVATORS.
THANK YOU.
>> WE'RE RIGHT ON TIME.
I HOPE YOU WERE ABLE TO GET
COFFEE AND ENJOYED THE BREAKOUT
SESSIONS.
I WAS ABLE TO BE A FLY ON THE
WALL, IT WAS AN INTERESTING
DISCUSSION.
BEFORE I HAND OVER TO AARON AND
THE DEBRIEFS FROM EACH OF THOSE,
I WANTED TO ANSWER A COUPLE
QUESTIONS THAT I HEARD THAT CAME
UP.
ONE IN PARTICULAR WAS JUST
AROUND THE ENORMITY OF THE TASK
THAT WE'VE JUST PUT IN FRONT OF
YOU AND HOW IT IS IMPOSSIBLE TO
SOLVE THIS IN THE 2 1/2, 3 DAYS
WE HAVE TOGETHER.
WE RECOGNIZE THAT.
AND SO MAYBE THE BEST THING I
CAN SAY IS JUST DON'T WORRY.
YOU ARE HELPING US BY -- HELPING
US GET HOWEVER FAR WE GET IN THE
NEXT DAY AND A HALF.
WE RECOGNIZE THAT WHAT WE COME
OUT WITH IS NOT GOING TO BE
PRISTINE GUIDANCE THAT WE CAN
HAND TO THE COMMUNITY.
WE'LL WANT MORE DISCUSSION ABOUT
IT, THAT'S EXPECTED.
WE HAVE THAT IN OUR PLANS SO
JUST MAYBE BE AT EASE WITH THAT
AND JUST KNOW YOU ARE STILL
HELPING US AS WE START TO PUT
THESE BONES TOGETHER.
SO WITH THAT I'M GOING TO HAND
OVER TO AARON WHO IS GOING TO
LEAD US THROUGH THE DEBRIEFING
SESSION AND HAVE SOME CLOSING
REMARKS FOR US.
>> ALL RIGHT.
GOOD TO SEE EVERYONE BACK.
WE'RE GOING TO DO THIS BY ASKING
ONE PERSON FROM EACH OF THE
GROUPS TO COME UP AND GIVE A 5
TO 6-MINUTE SUM RIF -- SUM MARY
OF THAT AND A FEW MINUTES FOR
QUESTIONS AND WE'LL GO THROUGH
EACH IN ORDER.
FIRST UP SYNTHETIC DATA GROUP,
HOPEFULLY YOU PICKED A PERSON TO
COME UP AND SPEAK.
>> SO THE RISK OF DOING THIS
EXTEMPORANEOUSLY, I MIGHT MISS
SOME THINGS, EVERYBODY WAS
CONTRIBUTING, A RICH DISCUSSION
AROUND CHALLENGES RELATED TO
SYNTHETIC DATA WITH RESPECT TO
ETHICS TRANSPARENCY.
SO WE HAD A NICE MIX OF PEOPLE
IN THE ROOM.
WE ACTUALLY STARTED WITH A
PROVOCATIVE QUESTION, WAIT A
SECOND, HOW DO WE DEFINE
SYNTHETIC DATA?
WE TALKED ABOUT PROVENANCE, CAN
YOU TRACE BACK TO COMPUTATIONAL
SOURCE OR A REAL WORLD SOURCE?
WE TALKED RAPIDLY, STARTED
TALKING ABOUT ISSUES OF TRUST,
ISSUES OF ACCESS, WE STARTED
TALKING, WE WERE ONE OF THE
GROUPS TALKING FRANKLY ABOUT THE
ENORMITY OF THE TASK, SOME -- WE
TALKED ABOUT FAST MRI, JULIA
ORIENTED THAT, OUR USE CASE AND
I'LL SPEAK TO THE STAKEHOLDERS
IN A MINUTE BUT USING THAT LENS
WE STARTED TALKING ABOUT THE
ENORMITY OF THE TASK TO SAY WHAT
GUIDANCE AND HOW DO WE MOVE FROM
UNDERSTANDING THE STAKEHOLDERS
AND THEIR NEEDS TO THINKING
ABOUT WHAT ARE THE CAPABILITIES
AND GAPS USING A COUPLE
EXAMPLES, RISK LEAVING THINGS
OUT BUT WE PUSHED FORWARD.
FAST MRI, AT THE END A COUPLE
WERE STILL TALKING ADDING
STAKEHOLDERS BECAUSE THERE WERE
MORE.
WE WEI BROUGHT IN THE IDEA OF
STAKEHOLDERS, THERE'S AN
IMPLICATION STAKEHOLDERS MAY
HAVE STAKE, MIGHT BE FINANCIAL
STAKE, MAYBE ANOTHER STAKE, BUT
AT THE END OF THE DAY WE HAD THE
INDIVIDUAL UNDERGOING MRI, A
PATIENT POTENTIALLY, THEIR
SUPPORT NETWORK, CLINICAL WORLD
AROUND THEM, RADIOLOGISTS AND
ANCILLARY FOLKS USING
TECHNOLOGY, WE MOVED INTO THE
REGULATORY PIECE OF IT,
INSURANCE AND PAYOR PIECE, WE
GOT TO THE COMMUNITY, TO THE
PUBLIC AND PERCEPTION, DEBRA
SAID THE PERCEPTION PIECE IS
CRITICAL BECAUSE IT TAKES ONE OR
TWO OF THESE TO GO WRONG AND
THEN THE KNEES ARE CUT OUT FROM
THE REST OF THE TECHNOLOGY WITH
BROAD IMPLICATIONS.
WE STARTED GETTING INTO THE
NEEDS THAT WILL FEED US INTO THE
GAPS.
AS FAR AS KEY THEMES ACCURACY
WAS CRITICAL, WE TALKED ABOUT
ERROR, UNCERTAINTY, QUANTIFYING.
ONE KEY THEME OF THE GROUP,
REALLY GREAT, IS THAT AT EVERY
STEP OF THE WAY WE WERE THINKING
IN THE BACK OF OUR MINDS, WE'LL
MAKE THIS EXPLICIT NEXT SESSION,
WHEN WE'RE WORRIED ABOUT THE
POTENTIAL GAP HOW DO WE MEASURE
AND QUANTIFY, IF IT'S
UNCERTAINTY OR FAIRNESS OR
ACCURACY, HOW ARE WE GOING TO
MEASURE IT?
THAT'S THE HIGH LEVEL THEMES.
IS THERE ANYTHING I MISSED,
FOLKS IN THE ROOM?
KEEP ME HONEST.
OKAY, THANK YOU.
>> BEFORE YOU LEAVE, ARE THERE
ANY QUESTIONS FOR THAT GROUP?
>> I WAS THERE FOR THE VERY
BEGINNING.
YOU WERE STARTING WITH WHAT WAS
MEANT BY THE TERM SYNTHETIC
DATA, WHERE DID YOU LAND?
>> WE PAINTED ATTRIBUTES OF SORT
OF MULTI-FACTORIAL ATTRIBUTES OF
SYNTHETIC DATA, DRAWN FROM A
REAL DISTRIBUTION, WE TALKED
ABOUT IS IT IMPOSSIBLE, RIGHT?
WE TALKED ABOUT THE FACT REAL
WORLD DATA INCLUDING ELECTRONIC
HEALTH RECORD CONTAIN IMPOSSIBLE
DATA.
WE CAN FIND REAL DATA THAT
EVIDENCES THINGS THAT DON'T
EXIST IN PHYSICS OR HEALTH, WE
FLESHED IT OUT, I DON'T THINK WE
SOLVED.
THE LINCHPIN IS THE PROVENANCE,
TRACE IT BACK, REPRODUCIBILITY
AND PROVENANCE, DOES IT
REPRESENT SOMETHING REALISTIC,
CAN YOU SHOW THIS COULD BE
STHOOG COULD -- SOMETHING THAT
COULD EXIST IN REALITY AND
FAIRNESS AND REPRESENTATIVENESS
BUT NOT DETERMINING IN THAT
SENSE.
PLEASE.
>> THERE'S A QUESTION FROM
ONLINE AROUND THE RELATIONSHIP
BETWEEN (INDISCERNIBLE) AND
TRANSPARENCY, SINCE YOU JUST
MENTIONED, HOW DO YOU THINK
ABOUT THAT ESPECIALLY AND
SYNTHETIC DATA (INAUDIBLE).
>> YEAH, ABSOLUTELY.
IN THE EXAMPLE WE UNPACKED WAS
FAST fMRI, IMAGINE THE
RADIOLOGIST INTERPRETING IT, THE
PATIENT UNDERGOING A SCAN,
ELEMENTS WILL BE DERIVED
DIRECTLY MEASURED AND GENERATED,
HAVING THE ABILITY TO TRACE BACK
WHICH PIXELS WERE PRODUCED OR
GENERATED AND MEASURED IS KEY.
THE OTHER THING THERE'S A ROLE
FOR THIS TECHNOLOGY TO DO THAT
SCAN WHERE EVERYTHING LOOKS
REALLY GOOD, IT FALLS WITHIN
NORMS AND PROBABILITY
DISTRIBUTION VERSUS THIS ONE
WHERE WE'RE NOT SURE, MEASURE
THIS ISN'T SUFFICIENT, NOW THAT
PATIENT NEEDS TO GO FOR THE HIGH
TESLA MRI SCAN TO GET THE NEXT
ONE DONE SO HOW FAR DOWN PATH
ARE WE DOING HERE?
IT FITS WITHIN A DIAGNOSTIC
WORKFLOW BUT IT WAS VERY CLEAR
THE ABILITY TO REALLY TRACE BACK
AND POINT YOUR FINGER AT EACH
STEP WAS CRITICAL IN THE FACE OF
TRANSPARENCY ESPECIALLY HERE
USING SYNTHETIC DATA FOR A
NUMBER OF USES, IF IT'S A
SYNTHETIC TRIAL GENERATING THAT
TRIAL ARM THERE'S A SYNTHETIC
PIECE, IS THIS THE PATIENT THAT
WALKED IN OR COMPUTATIONAL
DERIVED FROM A DISTRIBUTION OF
PATIENTS WHO MAYBE LOOKED LIKE
THIS ONE DID.
PLEASE, PLEASE.
I DO -- DO YOU WANT A MIC?
>> JUST TO ADD TO THE
TRANSPARENCY REPRODUCIBILITY
QUESTION, I THINK, YOU KNOW, THE
NOTION OF PROVENANCE THAT YOU
RAISED COMES, VERY INTERESTING,
BECAUSE THERE'S THIS QUESTION
OF, YOU KNOW, DOES MY PATIENT --
IS MY PATIENT REPRESENTATIVE OF
THE POPULATION FROM WHICH THIS
SYNTHETIC DATA WERE DERIVED IN
THE FIRST PLACE?
IF I HAD ASSESSMENT OF THAT
WHICH COULD BE ENABLED BY A.I.
OR AUTOMATED METHODS TO SAY, YOU
KNOW, THIS IS WHAT WE THINK YOUR
MRI LOOKS LIKE BUT THE
CONFIDENCE LEVEL IS VERY LOW
BECAUSE THIS PATIENT IN THESE
SPECIFIC WAYS DOES NOT ALIGN
WITH THE ORIGINAL PATIENT
POPULATION FROM WHICH THE
SYNTHESIS WAS CREATED, AND SO
THAT OF COURSE RAISED THE
TENSION OF PRIVACY.
SO WHEN YOU TALK ABOUT
PROVENANCE THERE'S ALWAYS THE
QUESTION OF ARE WE REVEALING
MORE INFORMATION THAN WE SHOULD
BE.
>> THANK YOU FOR HIGHLIGHTING
THE PRIVACY PIECE.
WE TALKED ABOUT THAT QUITE A
BIT.
>> ANOTHER HAND RAISED
(INAUDIBLE).
>> THE EXAMPLE OF IMAGING DATA,
DID YOU TALK ABOUT SYNTHETIC
DATA FROM PERSPECTIVE OF
CREATING IMAGING DATA THAT'S
HUMAN CONSUMABLE VERSUS CREATING
THE RAW SYNTHETIC DATA THAT IS
ACTUALLY GOING TO GIVE YOU THE
QUANTITATIVE NUMBERS?
>> THAT'S A REALLY INTERESTING
ONE.
WE DIDN'T TALK A LOT ABOUT THAT
THEME.
IF YOU HAVE THOUGHTS IT WOULD BE
GREAT TO DISCUSS BUT WE DIDN'T
GET INTO THAT.
>> ALL RIGHT.
>> GREAT, THANK YOU.
>> NEXT UP IS DATA SHARING FOR
GENERAL REUSE.
>> ALL RIGHT.
I WILL SUMMARIZE TODAY AS ONE OF
THE SESSION CO-LEADS BUT
TOMORROW WE'LL IDENTIFY A GROUP
MEMBER PERHAPS.
SO SOME OF THE SIMILAR THEMES TO
WHAT COLIN ADDRESSED, WE WERE AS
USE CASE A LITTLE BIT FURTHER
AWAY FROM A DIRECT PATIENT, YOU
KNOW, KIND OF LIKE AN MRI, SO WE
CAME AT IT FROM, YOU KNOW,
SECOND REUSE OR REUSE OF DATA
WHICH I THINK MADE DATA THE
ESSENTIAL PART OF WHAT WE WERE
DISCUSSING.
WE HAD STARTED WITH SOME
POTENTIAL USE CASES OF, FOR
EXAMPLE, CLINICAL TRIAL DATA
PLACED INTO A REPOSITORY VERSUS
REAL WORLD LIKE EHR SYSTEMS.
QUICKLY DISTILLED THAT DOWN INTO
DIFFERENCES BETWEEN DATA WHICH
WERE COLLECTED AS PART OF A
HYPOTHESIS-DRIVEN RESEARCH
EFFORT, VERSUS DATA WHICH MIGHT
HAVE BEEN COLLECTED LIKE, SAY,
IN THE PROCESS OF CLINICAL CARE
WHICH WERE THEN NOW HIGH
HYPOTHESIS GENERATING, TALKING
THROUGH THE NIH SHIFT HOW THEY
ARE APPROACHING THE FUNDING OF
DATA RESOURCES, MOVING FROM KIND
OF HYPOTHESIS-DRIVEN PROPOSAL
NOW INTO NEWER A.I. WORK,
POTENTIALLY JUST HAVING LARGER
HYPOTHESIS-GENERATING DATASETS.
QUICKLY CAME THE QUESTION OF
CONSENT FROM THAT.
SO, WHAT CONSENT -- FOR THE
SUBJECT WHOSE DATA WAS IN THESE
REPOSITORY, EITHER HYPOTHESIS
DRIVEN OR GENERATING, WHAT LEVEL
OF CONSENT DID THEY PROVIDE?
ONLY FOR THE SPECIFIC QUESTION
WHICH WAS DRIVEN BY THE
HYPOTHESIS?
DID THEY PROVIDE, FOR EXAMPLE,
IF THEY WERE IN A THE HEALTH
SYSTEM CONSENT FOR CARE AND DATA
TO BE USED BUT NOT IN RESEARCH
CONTEXT AND KIND OF EVERYTHING
IN BETWEEN.
AND WE JUST THOUGHT THAT THOSE
WERE REALLY IMPORTANT
CONSIDERATIONS FOR THE WORK THAT
WE WERE GOING TO BE DOING OVER
THE NEXT SESSIONS.
YOU KNOW, WHAT WAS THE ORIGINAL
INTENT OF THE DATA?
AND WHAT LEVEL DID THAT RESEARCH
SUBJECT AND THEIR DATA, LIKE TO
WHAT DEGREE DO THEY UNDERSTAND
HOW THAT REUSE WOULD HAPPEN.
THAT INFLUENCES WHETHER REUSE OF
DATA FOR SUBSEQUENT
INVESTIGATION SHOULD BE BROADLY
-- YOU KNOW, BROADLY AVAILABLE
OR DYNAMIC AND CASE SPECIFIC.
WHAT MECHANISMS WE EVEN HAVE TO
MAKE SUCH A THING REALITY.
WE TALKED A LITTLE BIT, MAYBE
I'LL SWITCH TO STAKEHOLDERS, SO
WE STARTED WITH THE -- KIND OF
FROM THE DATA PERSPECTIVE AGAIN
AND THEN GOT DOWN TO THE PATIENT
LEVEL.
WE TALKED ABOUT INSTITUTIONS,
WHETHER THOSE ARE RESEARCH
PERFORMING ORGANIZATIONS, LET'S
SAY A UNIVERSITY, OTHER ACADEMIC
CENTERS, MAYBE HEALTH SYSTEMS,
INSTITUTIONS COULD OBVIOUSLY
INCLUDE COMPANIES, RIGHT, WHO
ARE -- YOU KNOW, DEVELOPING OR
COMMERCIALIZING TECHNOLOGY THAT
ORIGINATED FROM
HYPOTHESIS-DRIVEN WORK OR
SECONDARY USES OF DATA,
INSTITUTIONS WOULD INCLUDE
FUNDING AGENCIES, OBVIOUSLY
INSTITUTION WOULD ALSO INCLUDE
PUBLISHERS.
WE TALKED ABOUT THE KIND OF
SPECIFICS OF THE GRANT REVIEW
PROCESS BECAUSE I FEEL LIKE
THAT'S WHERE WE'RE HEADING,
GUIDANCE ABOUT HOW AN APPLICANT
IS TO ADDRESS SOME OF THESE
ISSUES OR MAYBE A STUDY SECTION
MEMBER SO WE THOUGHT THERE WAS
STAKEHOLDERS THERE.
AND THEN WE ULTIMATELY TALKED
ABOUT, YOU KNOW, PROBABLY THE
RISING DOMAIN WHICH IS THOSE
ENTITIES RESPONSIBLE FOR
ORGANIZING AND MAINTAINING THE
DATA.
SO THOSE COULD EITHER BE
REPOSITORIES, THEY COULD BE
INSTITUTIONS, THEY COULD ALSO BE
BROKERS.
AND, YOU KNOW, THEY ARE GOING TO
BE IMPORTANT STAKEHOLDER HERE,
THAT IS KIND OF IN THE MIDDLE OF
THE GENERATION OF THE DATA,
CURATION OF DATA, ULTIMATELY
REUSE.
FINALLY DOWN TO PATIENTS, IN
SOME CASES THEY ARE RESEARCH
SUBJECTS, IN OTHER CASES THEY
ARE PATIENTS, IN OTHER CASES
THEY ARE CITIZENS.
SO, YOU KNOW, AGAIN, DEPENDING
HOW THESE DATA WERE COLLECTED
LIKE SOCIAL MEDIA DATA, RIGHT,
THEY ARE NOT PATIENTS.
THOSE MIGHT BE CITIZENS.
HEALTH SYSTEM SUBJECTS, THOSE
ARE PATIENTS.
AND THEN RESEARCH SUBJECTS WHO
REALLY RISE FROM SOME
HYPOTHESIS.
AND THEN WE EXPANDED OUT FROM
THERE.
YOU KNOW, TOWARDS WHAT IS THE
INTEREST OF DIFFERENT COUNTRIES,
LIKE E.U. AND SOME OF THEIR
GUIDELINES AROUND DATA REUSE OR
AGGREGATION.
THE QUESTIONS ABOUT TRIBAL
NATIONS OR, YOU KNOW, ENTITIES
WITHIN OUR UNITED STATES THAT
HAVE DIFFERENT TYPE OF
SOVEREIGNTY OVER DATA AND OTHER
PRACTICES.
FAMILY MEMBERS, YOU KNOW, UNITS
THAT SURROUND WHOSE PRIVACY OR
OTHER -- DATA REUSE COULD HAVE
IMPLICATIONS ON THEM.
AND THEN PATIENTS WITH SPECIFIC
CONDITIONS, WHO CONSENT TO
PARTICIPATE IN A TRIAL BECAUSE
THEY BELIEVE THAT THEY WILL BE
BENEFITING OTHER PATIENTS IN THE
FUTURE, WITH THEIR
CONTRIBUTIONS.
THERE'S A LOT MORE THERE,
FUNDING, RESPONSIBILITY,
CONSENT, LICENSING, BIAS, WHICH
I THINK I HOPE WE'LL GET TO IN
THE SUBSEQUENT SESSIONS.
>> ALL RIGHT, THANK YOU.
>> COULD I JUST ASK YOU A
QUESTION?
DID YOU ALL TALK ABOUT ABOUT THE
COMMON RULE PERMITS?
THERE'S A A GOOD BIT OF REUSE
PERMITTED UNDERSTOOD THE COMMON
RULE WITH AND WITHOUT CONSENT,
WHERE THE IRB CAN ACTUALLY
REDIRECT THE DATA.
I JUST AM WONDERING IF YOU ALL
LOOKED AT THAT AT ALL, NOT THAT
I THINK THAT IS -- I MEAN, I
THINK THAT'S THE FLOOR, NOT THE
CEILING, BUT I WAS CURIOUS.
>> WE DIDN'T SPEAK SPECIFICALLY
ABOUT THAT BUT I THINK WE'LL
BRING THAT UP IN OUR NEXT
BREAKOUT.
>> THANK YOU.
>> CLOSELY RELATED QUESTION,
THAT IS DID YOUR GROUP CONSIDER
THE CONSEQUENCE -- I UNDERSTAND
YOU LOOKED AT THE SCOPE OF
CONSENT IN TERMS OF DATA REUSE.
BUT DID YOU LOOK AT THE
CONSEQUENCES OF REQUIRING
CONSENT AS OPPOSED TO
ALTERNATIVE OF ETHICAL
MANAGEMENT OF UNCONSENTED DATA,
WITH RESPECT TO THE BIAS THAT
THAT INTRODUCES?
>> SO, YOU KNOW, I THINK WE WERE
TRYING TO JUST MAKE SURE WE HAD
A GOOD GRASP ON USE CASES AND
STAKEHOLDERS BEFORE WE
APPROACHED CHALLENGING QUESTIONS
LIKE THAT.
BUT, YEAH, WE DIDN'T SPEAK ABOUT
THAT SPECIFICALLY BUT KIND OF
DANCED AROUND IT.
>> I WAS WONDERING IF YOU GOT A
CHANCE TO DISCUSS LICENSING,
COPYRIGHTS, THINGS LIKE THAT.
THE OTHER ONE REGARDING CONSENT,
WHETHER YOU HAD A CHANCE TO
EXPLORE BLANKET STATEMENTS LIKE
WHO WILL USE THIS DATA FOR ANY
FUTURE RESEARCH, LIKE GIVEN THAT
AS YOU WRITE THESE UP, LIKE A.I.
CATAPULTS THIS SORT OF ALL KINDS
OF FUTURE USE CASES, SHOULD THE
USER BLANKET STATEMENTS BE
ALLOWED OR NOT, DID YOU EVER --
>> THAT CAME UP.
I DON'T KNOW THAT WE CAME TO A
CONSENSUS.
IN TERMS OF LICENSING, WE TALKED
ABOUT EXAMPLE FROM I THINK NASA,
WHERE THERE'S THE KIND OF
LICENSING SUCH THAT THOSE DATA
ARE FREELY AVAILABLE, THEY ARE
OPEN, THERE'S NO, YOU KNOW, KIND
OF LIKE CHECKS ON WHO GETS TO
USE THOSE DATA.
I DON'T THINK THAT'S PROBABLY
THE RIGHT PLACE FOR HEALTH DATA
TO GO, BUT WE DID DISCUSS IT IN
THAT CONTEXT.
>> ANY OTHER QUESTIONS?
>> (INAUDIBLE) DATA USE
AGREEMENTS BETWEEN ACADEMIC
ORGANIZATIONS?
>> WE DIDN'T DISCUSS THOSE.
>> (INAUDIBLE).
>> FOR THE ONLINE FOLKS COULD
YOU USE THE MIC?
SORRY.
MAYBE REPEAT THE QUESTION.
>> YEAH, I THINK THE QUESTION
WAS DID WE DISCUSS DATA USE
AGREEMENTS BETWEEN ACADEMIC
ORGANIZATIONS.
>> YES, I WAS THE OTHER OH
CHAIR.
WE DID LAY OUT THE STAKEHOLDERS,
CONCERNS, NEEDS, RECOGNIZING
TOMORROW WE'LL FOCUS ON THE
GAPS, CAPABILITIES AND GAPS.
SO EVERYTHING THAT YOU TALKED
ABOUT LIKE EVEN THE LICENSING,
WE KNOW THERE'S A GAP THERE
AROUND DATA USE AGREEMENTS, WE
KNOW THERE'S A GAP THERE.
CONSENTING, CONSENT VERSUS
UNCONSENTED DATA, THERE ARE
METHODS AND WE NEED TO IDENTIFY
METHODS AND GAPS TO MAKE IT
CONSISTENT ACROSS DIFFERENT DATA
TYPES.
>> NEXT UP MULTI-MODAL DATA.
>> ALL RIGHT.
WE ALSO HAD A VERY RICH
DISCUSSION AND HAD A REALLY
DIVERSE GROUP OF EXPERTS IN THE
ROOM.
I THINK TO START OFF WITH
MULTI-MODAL DATA, I THINK SOME
OF THE ISSUES THAT WE SURFACED
AS A GROUP WAS THAT CLEARLY WITH
MULTI-MODAL DATA LINKING BETWEEN
DATASETS IS OBVIOUSLY MORE
PRIMARY THAN MAYBE A FEW OF THE
OTHER USE CASES, SO A LOT OF
THOUGHTS ABOUT HOW LINKAGES TO
DATA REQUIRE MORE INFORMATION
ABOUT THE DATA FORMATS, DATA
DOCUMENTATION, SAMPLE SIZES OF
THE DATA, AND TRYING TO KEEP AN
EYE ON ALL OF THAT, KNOWING THAT
THE MORE YOU LINK, THE MORE YOU
CAN LOSE PATIENTS IN THE SAMPLE
AND LOSE INFORMATION WITH THE
TYPES AND AMOUNTS OF DATA
AVAILABLE TO YOU.
THAT CAME UP QUITE A BIT.
TRYING TO HIGHLIGHT THINGS THAT
HAVEN'T BEEN SAID IN OTHER
GROUPS.
WE ALSO TALKED A LOT ABOUT
MULTI-MODAL DATA ALSO PUTTING
THE COMMUNICATION BETWEEN TEAMS
ALSO CENTRAL TO THE ISSUE, SO
THAT CAME UP A NUMBER OF TIMES.
AND AFFECTED OUR STAKEHOLDER
LIST WHICH I'LL TALK ABOUT IN A
SECOND.
SOME OTHER MAIN COMMON
CHALLENGES AND ALSO
SUCCESSES/OPPORTUNITIES IN THE
SPACE I THINK HAVE MOSTLY BEEN
MENTIONED PREVIOUSLY BEFORE SO A
LOT OF NEEDS FOR GREATER DATA
AVAILABILITY, MORE DETAILING OF
MODELING TO BE ABLE TO REPRODUCE
MULTI-MODAL MODELS IN OTHER
SETTINGS OR INDICATIONS,
CONCERNS FOR PRIVACY AND
IDENTITY MANAGEMENT, ESPECIALLY
I WOULD SAY THAT ONE IS
PARTICULARLY FOR MULTI-MODAL
DATA GIVEN THE MORE YOU LINK
DATA POTENTIALLY THE HIGHER RISK
OF RE-IDENTIFICATION IS
POSSIBLE.
THE LACK OF PATIENT AND
COMMUNITY ENGAGEMENT ACROSS ALL
PHASES OF THE A.I. DATA
ACQUISITION TO DEPLOYMENT CYCLE,
AND THIS IDEA OF RISK AND RISK
APPROXIMATION, GENOMIC DATA
HAVING A DIFFERENT RISK THAN EHR
VERSUS WEARABLE DATA, ET CETERA.
THOSE ARE SOME THEMES THAT I
WOULD SAY SURFACED FROM OUR
GROUP DISCUSSION.
AND THEN IN TERMS -- WE SWITCHED
TO THE STAKEHOLDER CONVERSATION
LIKE MANY OF THE OTHER GROUPS
DISCUSSED, AND I WOULD SAY WE
ADDED MORE STAKEHOLDER GROUPS IN
OUR DISCUSSION, AND SOME OF THE
ONES I THINK ARE PARTICULARLY
INTERESTING IS WE TALKED A LOT
ABOUT DIFFERENT DISCIPLINARY
KNOWLEDGE, CREATION OF MODELS,
SO BASIC SCIENCE VERSUS HEALTH
SERVICES RESEARCH VERSUS
CLINICAL RESEARCH, AND WHEN
YOU'RE THINKING ABOUT
MULTI-MODAL DATA EACH OF THOSE
GROUPS HAS THEIR OWN LANGUAGE,
TERMINOLOGY HOW THEY ARE
THINKING ABOUT DATA AND
CONCEPTS, AND SO NOT JUST
LUMPING DATA SCIENTISTS TOGETHER
BUT TEASING INTO DISCIPLINARY
KNOWLEDGE SPACES, MOVING THROUGH
REAL ORGANIZATIONAL LEADERSHIP,
AND IN THAT BUCKET WE TALKED A
LOT ABOUT THE PEOPLE DEPLOYING
THE MODELS, MULTI-MODAL MODELS
ONCE DEVELOPED, MACHINE LEARNING
ENGINEERS BEING A DIFFERENT
GROUP THAN DATA SCIENTISTS
GENERATING MODELS, CREATING A
MODEL OR MANY TYPES OF DATA
MIGHT LOOK DIFFERENT THAN THE
FINAL MODEL DEPLOYED THAT CAN BE
DEPLOYED IN AN EHR AT THE POINT
OF CARE.
WE TALKED ABOUT REGULATORS, THAT
CAME UP QUITE A BIT BEFORE, THE
ABILITY FOR US TO HAVE A HIGHER
LEVEL OF CONTENT EXPERTISE
ACROSS DOMAINS INCLUDING
REGULATORS AND PAYERS AND
FUNDERS GIVING COMPLEXITIES
ACROSS DATA TYPES.
THE THEMES OF COMMUNICATION AND
TRUST I WOULD SAY WERE JUST
RELEVANT ACROSS EVERY
STAKEHOLDER GROUP THAT EMERGED.
I THINK I'LL STOP THERE AND KEEP
IT BRIEF.
YEAH, QUESTIONS?
>> SO I HAVE A QUESTION.
DID YOU GUYS TALK ABOUT -- WHEN
WE THINK ABOUT MULTI-MODAL WE'RE
PULLING TOGETHER DIFFERENT
TYPES OF DATA MODALITIES,
ASSOCIATED WITH ACCESS TO CARE,
WHICH IS GOING TO INFLUENCE
ACCURACY OF THE MODEL.
HOW DO YOU THINK ABOUT BEING
TRANSPARENT ABOUT LIKE THESE
MISSINGNESS, DIFFERENT TYPES
MODALITIES ACROSS A PATIENT
TRAJECTORY.
>> THAT CAME UP, WE STARTED OUR
CONVERSATION THERE.
AND INSTEAD OF RATHER THAN
TEASING APART, WE STARTED THE
CONVERSATION FROM OF COURSE WE
WANT TO USE ALL THE DATA
AVAILABLE TO US TO HAVE BETTER
DISCOVERY OR BETTER TRUTH OF
INDICATION BUT WE KNOW THAT
PEOPLE SHOW UP IN SAMPLES
DIFFERENTLY AND THE MORE YOU
LINK FRANKLY THE MORE YOUR
SAMPLE SIZE OFTEN DIMINISHES.
EVEN IF YOU JUST TOOK SIMPLE
EXAMPLES OF SPECIALTY CARE AND
PRIMARY CARE YOU'RE LOSING
PEOPLE WHO HAVE LOSS OF ACCESS
BUT IF YOU'RE ADDING MANY MORE
TYPES OF DATA, GENOMIC DATA,
WEARABLE DATA, OF COURSE THE
POPULATION LEVEL VIEW OF THE
DATA IS MORE LIMITED SO WE
STARTED FROM THAT, TO BE HONEST,
STARTED FROM THAT USE CASE AND
THAT WENT TO OUR COMMUNICATION
NEEDS, ABILITY FOR DIFFERENT
SCIENTISTS TO TALK TO ONE
ANOTHER AND REAM UNDERSTAND WHAT
IS THE DATA AVAILABLE TO THEM
AND WHERE IT WAS SOURCED AND HOW
IT CAME FROM, YEAH.
>> YOU MENTIONED HOW LINKING
DATA ACROSS DOMAINS, MODALITIES,
INCREASES RISK OF
RE-IDENTIFICATION.
I WAS CURIOUS HOW THAT
INFLUENCED YOUR LIST OR YOUR
TALK ABOUT TRANSPARENCY, HOW
THAT MAY HAVE INFLUENCED
STAKEHOLDER -- (INAUDIBLE).
>> I THINK IT INFLUENCED OUR
THEMES THAT CAME UP OVER THE
STAKEHOLDERS SO THIS IDEA OF
TRUST AND PRIVACY, NOT THAT IT
DIDN'T COME UP IN THE OTHER
GROUPS, BUT I THINK REALLY
CENTRAL TO OUR CONVERSATION, THE
MOST INTERESTING PART WHICH I
THINK IS ANSWERING YOUR QUESTION
IS THIS IDEA THAT EVEN THOUGH
YOU'RE DOING MULTI-MODAL DATA
THE SAME PERSON COULD HAVE A
DIFFERENT VIEW OF THE LINKING OF
THAT DATA DEPENDING WHAT TYPE OF
DATA AND WHAT MODEL IS BEING
USED FOR SO HOW NUANCED THAT IS.
WE HAD MORE CONVERSATION IN THAT
DOMAIN, THAT TRUST IS NEEDED BUT
FRANKLY ACTUALLY AS YOU'RE
LINKING ACROSS IT'S NOT THE
SAME.
IT'S NOT THAT EACH STAKEHOLDER
GROUP FOR EACH CASE WILL HAVE
THE SAME RISK TOLERANCE OR RISK
PROFILE.
I THINK THAT'S A ROUNDABOUT WAY
OF ANSWERING YOUR QUESTION.
>> ANY OTHER QUESTIONS?
THANK YOU.
>> AWESOME, THANKS.
>> ALL RIGHT.
NEXT UP IS FOUNDATION MODELS.
>> HI
I CO-LED THIS SESSION ON
FOUNDATION MODELS.
WE HAD A REALLY ACTIVE GROUP
THAT CONTRIBUTED TREMENDOUSLY
WITH A LOT OF ENERGY, SO THANK
YOU ALL FOR THAT.
I THINK THAT OUR DISCUSSION I
WOULD BREAK UP INTO TWO PARTS.
THE FIRST PART WE DEFINED WHAT
OUR SCOPE IN TERMS OF WHAT A
FOUNDATION MODEL IS.
AND IN PARTICULAR, WE AGREED
THAT FOUNDATION MODELS ARE
GENERAL PURPOSE, A.I. MODELS
CREATED OVER LARGE AMOUNTS OF
DATA THAT CAN THEN BE FURTHER
CUSTOMIZED FOR USE IN PARTICULAR
PURPOSES, AND THAT THE CHATBOTS
THAT WE LOOK AT ARE SPECIAL
CASE, USE OF FOUNDATION MODELS,
SO WE'RE NOT JUST THINKING ABOUT
CHATGPT IN TERMS OF -- THE SCOPE
OF OUR CASE.
WE WORKED THROUGH SEVERAL USE
CASES OF FOUNDATION MODELS
WITHIN THE BROAD SCOPE OF
BIOMEDICINE AND HEALTH CARE.
HAVING SAID THAT STAGE WE MOVED
ON TO OUR PHASE 2, WHICH WAS
IDENTIFYING STAKEHOLDERS, AND WE
WENT TO TOWN AND HAD, YOU KNOW,
40-SOME DIFFERENT STAKEHOLDERS
THAT WE IDENTIFIED.
[LAUGHTER]
AND THEN BEGAN TO START TO BOIL
THAT TO A MORE MANAGEABLE SIZE
WHEN WE RAN OUT OF TIME.
SO, WHAT WE ACTUALLY DID WAS
MOVE ON FROM THERE TO START
TALKING ABOUT THE OBJECTIVES,
THE INCENTIVES OF THE DIFFERENT
STAKEHOLDERS AND THE PLACES AS A
VIEW TO POINTING OUT PLACES
WHERE THERE ARE TRADEOFFS TO BE
MADE OR CONFLICTS TO BE
RESOLVED.
AND WE HAVE PARTIAL LIST OF HALF
A DOZEN OR SO SUCH THINGS THAT
WE'VE IDENTIFIED, WHICH WE PLAN
TO REPEAT TOMORROW MORNING AND
THEN USE THAT AS BASIS FOR OUR
GAP ANALYSIS AND GUIDANCE LATER
IN THE DAY.
>> ANY QUESTIONS?
>> THANK YOU.
>> LAST BUT NOT LEAST PROXY
VARIABLES GROUP.
>> THANK YOU.
WE ALSO HAVE A DIVERSE GROUP OF
PARTICIPANTS, A LIVELY AND
ENGAGING CONVERSATION.
SO, WHEN WE WERE TALKING ABOUT
THE USE CASE WE REALLY HASHED
OUT THOSE TWO EXAMPLES THAT TINA
GAVE IN HER TALK WHICH WERE THE
VBAC CALCULATOR AND OBERMEYER
STUDY.
PROXY VARIABLES, RACE AND
ETHNICITY IS A GOOD EXAMPLE OF A
PROXY VARIABLE.
WE SAW THAT IN THE VBAC
CALCULATOR BUT NOT ALWAYS RACE
OR ETHNICITY BE THE PROXY
VARIABLE.
SO THE OBERMEYER STUDY WAS
HEALTH CARE NEEDS, PROXY COSTS.
WE HAD A FUN CONVERSATION ABOUT
HOW EVEN IN THIS FIGURE PROXY
VARIABLES IS SOMETHING WE FIND
AT THE END OF THE DAY BUT REALLY
SOMETHING THAT COULD BE
IDENTIFIED EARLIER, SOMETIMES
USED INTENTIONALLY EARLIER, SO
REALLY EVEN BACK TO TINA'S SLIDE
ABOUT THE DATA LIFE CYCLE AND
HOW CONTINUOUS THIS IS, REALLY
UNDERSTANDING THE DATA FROM
WHICH MODELS ARE DERIVED.
WE FOCUSED ON STAKEHOLDERS.
AS WAS EXPECTED WE ENDED UP WITH
ALSO IDENTIFYING AREAS OF NEEDS
AND GAPS, SPILLING OVER INTO THE
NEXT SESSION, AND I INVITE
ANYBODY FROM THE GROUP TO CHIME
N WE MADE THESE SLIDES, A
REGURGITATION OF IDEAS.
I WON'T READ EVERYTHING, BUT
JUST TO START WITH FIRST STAKER,
WHETHER IT'S INVESTIGATOR OR
ANALYST, AND THE PRESSURES THEY
FACE MAY VARY BASED ON WHO IS
FUNDINGS, LENS AT WHICH THEY ARE
LOOKING AT DATA, REALLY IN THE
DATA CREATION STAGE FOR DATA
USERS TO UNDERSTAND EACH
VARIABLE AS WELL AS LIMITED TIME
AND RESOURCES TO COLLECT
VARIABLES OF INTEREST.
WE ALSO TALKED ABOUT WHAT WAS
THE INTENDED USE OF THE DATASET
VERSUS ACTUAL APPLIED USE.
THEN WE GOT INTO THIS
CONVERSATION ABOUT ARE WE
UNINTENTIONALLY USING PROXY
VARIABLES OR IS IT INTENTIONAL
AND WHEN IS IT JUSTIFIED, THOSE
ARE DEFINITELY UNRESOLVED ISSUES
I WOULD SAY.
WE STARTED TO HAVE EARLY
CONVERSATIONS ABOUT WHAT'S
NEEDED AND WHAT CAME UP WAS
AVAILABILITY OF NETWORKS AND
MULTI-DISCIPLINARY SETTINGS,
MANY EXPERTS, NEXT SLIDE, CAN
REALLY HELP IDENTIFY THESE PROXY
VARIABLES EARLIER ON IN MODEL
CREATION STAGES.
AND THEN, YOU KNOW, DATA USERS
ALSO HAVE A LOT OF COSTS OF
COLLECTING AND USING DATA THAT
MAY IMPACT HOW THEY USE DATA IN
A WAY THEY MAY NOT BE CATCHING
SOME OF THESE PROXY VARIABLES
THAT ARE HAPPENING.
WE TALKED ABOUT NEED FOR
REASSESSMENT AT THE END OF THE
MODEL LIFE CYCLE, IT'S A
CONTINUOUS PROCESS, NEED FOR
HYPOTHESIS-DRIVEN MODELS
UNDERSTANDING HOW VARIABLES IN
THE DATASET ARE RELATED TO
OUTCOMES OF INTEREST.
WE HAD A STAKEHOLDER HERE THAT
NO OTHER GROUP MENTIONED, I'M
REALLY EXCITED BECAUSE THIS WAS
ONE OF OUR CORE STAKEHOLDERS
WHICH WERE THIS GROUP OF
SOCIOLOGISTS, HISTORICAL
EXPERTS, PHILOSOPHERS,
DOT-DOT-DOT, PEOPLE WITH SIMILAR
EXPERTISE THAT NEED TO BE
INCLUDED AS BIOMEDICAL
RESEARCHERS BECAUSE THEY HAVE A
STRONGER UNDERSTANDING ABOUT HOW
VARIABLES WERE CREATED LIKE RACE
AND ETHNICITY FOR EXAMPLE, AND
THEY PROVIDE A REALLY VALUABLE
ROLE IN RISK EVALUATION,
UNDERSTANDING WHERE THE POSSIBLE
DOWNSTREAM IMPACT OF USING THIS
AT POPULATION LEVEL TRANSLATING
THE MODEL WHEN WE GO TO DO THAT
AT POPULATION LEVEL, SCOPING
INTENDED USE OF THE MODEL, BEING
ABLE TO IDENTIFY CLEARLY WHERE
THE MODEL SHOULD NOT BE USED AS
WELL.
AND THE GOAL AND THE WHY BEHIND
MODEL, STORIES BEHIND IT AS
WELL.
SO THIS WAS REALLY SOMETHING WE
SPENT A LOT OF TIME ON, AN
IMPORTANT CORE STAKEHOLDER FOR
US.
THE LAST ONE WE DISCUSSED IN
DETAIL WERE POPULATIONS AFFECTED
BY POPULATION MODELS, OFTEN THE
PATIENTS, WE HAD A SUBGROUP
ABOUT MARGINALIZED POPULATIONS
IN PARTICULAR WHO ARE MORE
LIKELY TO BE NEGATIVELY AFFECTED
GIVEN HISTORICAL
MARGINALIZATION.
SO WE TALKED ABOUT MANY OF THESE
OTHER ISSUES THAT CAME UP, I
WON'T REPEAT THEM, HOW MODELS
ARE EXPLAINED TO PATIENTS,
TRUSTWORTHINESS, CONSENT, DATA
SOVEREIGNTY, PRIVACY, DIDN'T GET
FAR TO GET INTO GAPS OR
SOLUTIONS BUT MANY SIMILAR
THEMES FROM THE OTHER SESSIONS.
AND THIS IS A LIST OF OTHER
STAKEHOLDERS THAT WE UNDERSTAND
ARE INVOLVED SIGNIFICANTLY WITH
THE USE OF PROXY VARIABLES
INCLUDING PAYERS, SPONSORS,
HEALTH SYSTEMS, AND
ORGANIZATIONS.
DOES ANYBODY ANYTHING TO ADD?
ANY QUESTIONS?
YES?
>> I THINK THIS WAS ALREADY
BROUGHT UP BUT I WANT TO
REINFORCE, I HEARD OBERMEYER
GIVE A TALK RACIAL PROXY AND
BIAS, AT THE TIME IT WAS
COLLECTED EVERYTHING KNEW IT WAS
AN IMPERFECT PROXY, WHAT
HAPPENED WAS THAT INFORMATION
GOT LOST BY THE TIME THAT THE
MODEL WAS BUILT, RIGHT?
SO MAYBE JUST A NUDGE WHEN YOU
THINK ABOUT TRANSPARENCY
INTERVENTIONS.
BUT THIS IS JUST A GREAT EXAMPLE
OF, YOU KNOW, TRANSPARENCY
THROUGH THE LIFE CYCLE.
>> YEAH, THAT'S REALLY HELPFUL
CONTEXT, THANK YOU.
>> I'M GOING ASK A LOADED
QUESTION.
SO, I TALKED ABOUT -- I LOVE
THAT YOU BROUGHT UP HISTORICAL
EXPERTS, THE FACT THAT IF YOU
DON'T LOOK BACK AT HISTORY
YOU'LL REPEAT THE SAME MISTAKE
OVER AND OVER AGAIN.
DO YOU THINK WE NEED TO FAVOR
FUNDED RESEARCH, NEEDS TO
PUBLISH NEGATIVE RESULTS, HOW DO
WE FACILITATE THAT?
ESPECIALLY LARGE LANGUAGE MODELS
COMBING PUBLISHED LITERATURE
ONLY COMBING POSITIVE RESULTS
BECAUSE THERE'S A PUBLICATION
BIAS, WHAT ARE WE GOING TO
REPEAT BECAUSE NEGATIVE RESULTS
HAVE NOT BEEN SHARED?
>> YEAH.
>> FOOD FOR THOUGHT.
>> SUCH AN IMPORTANT POINT.
THE LIVE IN THE GUIDELINE
DEVELOPMENT WORLD.
YOU KNOW HOW EVERYONE POINTS
FINGERS?
THE EVIDENCE BASE, YOU GUYS ARE
STUDYING ALL THESE THINGS,
THAT'S WHAT'S FEEDING INTO OUR
EVIDENCE BASE.
WE NEED TO DO A BETTER JOB IN
THE GUIDELINE WORLD OUTLINING
LIMITATIONS OF EVIDENCE BASE,
THERE'S A HUGE PUBLICATION BIAS
THAT I THINK NEEDS TO BE
ADDRESSED, AND I LIKE THOSE
IDEAS OF INTERVENTIONS, DEFINITE
SOMETHING WE CAN EXPAND ON IN
FUTURE SESSIONS.
THANK YOU.
>> A CONVERSATION FROM MY GROUP
WITH COURTNEY ABOUT WHO IS A
HISTORIAN, TRACKING DOWN IN HER
HOSPITAL NETWORK WHO KNEW WHY
THEY WERE RECORDING RACE A
PARTICULAR WAY, WHY THEY SHIFTED
FROM PATIENT REPORT TO PATIENT
REPORTED FROM THE CLINICIAN
MARKING SOMEONE DOWN AS
SOMETHING SO, YEAH, WE
OVERLOOKED THE HISTORIAN
STAKEHOLDER BUT DEFINING THAT I
THINK A LITTLE BIT CREATIVELY TO
WHO REALLY HAS THAT HISTORICAL
CONTEXT IS INTERESTING.
>> YEAH, ABSOLUTELY, THOSE
CONVERSATIONS CAME UP IN OUR
ROOM AS WELL.
>> SO YOU MENTIONED SOMETHING
EARLIER LIKE YOU HAD -- THERE
ARE SEVERAL PROXIES.
DO YOU THINK THERE'S GOING TO BE
A FINITE LIST OF PROXIES YOU'RE
GOING TO FOCUS ON, SEEING WHAT
THE PROXY IS FOR, OR DO YOU
THINK IT'S TOO BROAD, WE TALK
ABOUT RACE AS A PROXY OR HOW DO
YOU SEE THAT LIST GROWING OR IS
IT FINITE OR WHAT DO YOU THINK
THERE?
>> YEAH, NO, I THAT'S HELPFUL
POINT.
MAYBE HELPFUL TO THINK ABOUT,
YOU KNOW, SOME VERY COMMON PROXY
VARIABLES, HAVE GUIDANCE
SPECIFIC TO THOSE, UNDERSTAND
THERE MAY BE OTHER ONES
DEVELOPED THAT WE'RE GOING TO
CONTINUES TO BE IDENTIFY IT'S
THAT WE NEED TO SHARE THAT
INFORMATION BACK FOR OTHER
PEOPLE USING THE SAME DATASETS
OR SIMILAR MODELS AS WELL.
YOU KNOW, I THINK IT ALSO COMES
DOWN TO LIKE THE INTENTIONALITY,
TO THE COMMENT ABOUT WE KNEW IT
WAS A PROXY BUT THAT GOT LOST BY
THE PEOPLE WHO WERE IMPLEMENTING
IT, SOMETIMES PEOPLE ARE ALSO
USING IT AS A PROXY FOR
SOMETHING GOOD TO TACKLE
DISPARITIES.
WE'RE USING RACE AS PROXY FOR
SYSTEMIC RACISM, THEREFORE WE
KNOW THAT THERE IS, YOU KNOW,
KIDNEY TRANSPLANTS ARE BEING
DISPROPORTIONATELY ALLOCATED TO
CERTAIN GROUPS AND THEREFORE
SOCIAL DETERMINANTS ARE NOT
STRONG ENOUGH, THAT WE'RE GOING
TO USE RACE AS A PROXY FOR
SYSTEMIC RACISM BUT THERE'S
TRANSPARENCY, WHY ARE YOU
PUTTING IN THE MODEL, FOR THAT
REASON?
WE TALKED ABOUT SAY YOU BELIEVE
THE PROXY FOR A SPECIFIC GENE
THAT HASN'T BEEN IDENTIFIED,
LIKE WHEN THAT DATA BECOME
AVAILABLE HOW DO YOU MAKE SURE
THAT GETS REPLACED AND THESE
MODELS ARE CONTINUING TO BE
UPDATED.
>> I THINK THIS IS GREAT
DISCUSSION.
I WANT TO SUGGEST SOME THINGS
YOU MIGHT NOT THINK OF AS A
PROXY VARIABLE ACTUALLY ARE.
SEX IS ONE.
OBESITY IS ANOTHER.
I MEAN, YOU CAN DEFINE BOTH VERY
CLEARLY IN BIOLOGICAL TERMS BUT
WHAT THEY MEAN IS VERY MUCH
SITUATED IN WHERE YOU ARE IN
SOCIETY.
AND SO I JUST WANT TO THROW
THAT, WE TEND TO SAY WE CAN'T
USE PROXY VARIABLE BECAUSE WE
DON'T UNDERSTAND IT BUT I THINK
WE NEED TO REALIZE THAT LOTS OF
THINGS ARE PROXY VARIABLES, EVEN
IF WE HAVE A CLEAR BIOLOGICAL
DEFINITION.
SO I THOUGHT I WOULD SUGGEST
THAT FOR YOUR CONSIDERATION.
>> THANK YOU FOR THAT POINT.
>> SO YOU DID A GREAT JOB
SUMMARIZING OUR DISCUSSION.
>> THANK YOU.
>> WE HAD A GREAT DISCUSSION.
TO ELLEN'S POINT, THAT WHY IT'S
IMPORTANT WE TALK ABOUT
CONCEPTUALIZATION, PROXY
VARIABLES JUST EXIST, WE HAD TO
THINK ABOUT CONCEPTS WE'RE
TRYING TO MAP THESE THINGS ONTO
BUT BE CLEAR AGAIN, THIS WAS
SAID EARLIER, BEING CLEAR ABOUT
THE PURPOSE, ORIGINALLY, SO THAT
WHEN WE PASS THESE DATA ON,
THERE NEEDS TO BE NOTES ON THESE
DATA TO PROVIDE THAT TO USERS
AND THEN THAT CAN ADVANCE THE
SCIENCE AND NOT JUST ESSENTIALLY
REPLICATE MISTAKES THAT ARE
MADE, MAY HAVE BEEN MADE EARLIER
>> YEAH, THANK YOU, SUCH AN
IMPORTANT POINT.
ANYTHING ELSE?
NO?
OKAY.
>> GREAT.
>> OKAY.
LET ME SEE IF I CAN GET BACK TO
OUR SLIDES HERE.
LAURA, IF YOU'RE GOING TO HELP
ME, AM I GETTING CLOSE?
>> YEP.
YOU'RE GOOD.
>> OKAY, GREAT.
SO, I HAVE THE DOUBTFUL
PRIVILEGE OF TRYING TO SUMMARIZE
OUR FIRST HALF DAY OF
CONVERSATIONS.
I THINK, YOU KNOW, IT'S A REALLY
AMAZING OPPORTUNITY TO GET ALL
OF THESE -- ALL OF YOU HERE IN
THIS ROOM TO HAVE THIS
DISCUSSION.
IT'S A MOMENT IN TIME WE'RE AT
THE RIGHT TIME AND PLACE TO HAVE
THESE DISCUSSIONS, AND THEY ARE
BECOMING VERY, VERY PRESSING.
AND SO I'M JUST GOING TO GIVE
SORT OF MY MENTAL RECAP OF THE
DAY TODAY, HOPEFULLY A GOOD WAY
TO CLOSE OUT THE FIRST HALF DAY.
WE STARTED WITH A REALLY GREAT
KIND OF GROUNDING SESSION WHERE
WE HEARD THE GOALS OF WHAT THE
WORKSHOP ARE, TO EVENTUALLY
DEVELOP A DRAFT GUIDANCE THAT
COULD BE GIVEN TO THE NIH THAT
WOULD EVENTUALLY TURN INTO AN
RFI FOR PUBLIC COMMENT AND THEN
HOPEFULLY THEN SORT OF DEPLOYED
WIDELY ACROSS THE NIH GRANT
RECIPIENTS IN THE FUTURE.
JULIA GAVE US A GREAT TALK ON
THE DEFINITIONS OF TRANSPARENCY
AND SOME CHALLENGES OF THOSE
DEFINITIONS.
I THOUGHT THAT TALK WAS
EXTREMELY ILLUMINATING.
AND REALLY KIND OF SPELLED OUT
NUANCES OF SOME TERMINOLOGY.
TINA GAVE AN AMAZING TALK ABOUT
THE PROBLEMS WE ALL FACE, AS,
YOU KNOW, THESE ALGORITHMS AND
MODELS EVEN JUST TRADITIONAL
STATISTICAL MODELS HAVE BEEN
USED IN THE HISTORY OF MEDICINE,
AND, YOU KNOW, THERE'S A BOOK
CALLED "THE WEAPONS OF MASS
DESTRUCTION" THAT SORT OF CAME
TO MIND AS I HEARD HER TALK.
AND IT BOILS DOWN, YOU KNOW,
WHAT SOME OF THE CENTRAL ISSUES
ARE, AND I THINK THAT GENERATED
A LOT OF DISCUSSION IN THE
BREAKOUT SESSIONS THAT WE HAD.
I HAD AN OPPORTUNITY TO VISIT
ALMOST ALL THE DIFFERENT
BREAKOUT ROOMS.
AND WHAT I WAS STRUCK BY WAS THE
GREAT DIVERSITY OF WORK STYLES.
THAT WAS SOMETHING I DIDN'T
EXPECT.
THERE WERE SOME ROOMS THAT HAD
LIKE A FREE-FLOWING GROUP
DISCUSSION, AND THEN TRIED TO
SYNTHESIZE THAT INFORMATION.
THERE WERE OTHER GROUPS THAT
TOOK A VERY, YOU KNOW,
FLIP-BOARD STYLE LET'S WRITE
DOWN OUR THOUGHTS AS WE GO.
THERE WAS EVEN CROWDSOURCING,
THEY HAD ALL THE PARTICIPANTS IN
THE ROOM WRITE DOWN THEIR IDEAS
ON STICKIES AND INDEPENDENTLY,
AND THEN TRY TO PULL THAT ALL
TOGETHER.
AND THEN WE SAW THE RESULTS OF
THE LIVE GOOGLE SLIDE CREATION
THAT I SAW, WHICH IS A SKILL I
DON'T HAVE.
BEING ABLE TO, YOU KNOW,
SYNTHESIZE WHAT'S BEING SAID AND
PUT IT INTO BULLET POINTS ON A
SLIDE.
THERE'S ALSO DIFFERENCES IN
STRATEGIES LIKE ONE GROUP TOOK A
VERY DEPTH-FIRST SEARCH APPROACH
WHERE THEY REALLY ZOOMED IN ON
ONE STAKEHOLDER AND THEN HAD AN
IN-DEPTH DISCUSSION ABOUT WHAT
THAT STAKEHOLDER NEEDS WOULD BE,
THAT GAVE RISE TO OTHER
STAKEHOLDERS THAT WERE ADJACENT
TO THAT STAKEHOLDER.
OTHER GROUPS TOOK A MORE
PRINCIPLED BREADTH OF SEARCH,
LARGER FRAMEWORK, THINKING ABOUT
EACH ONE OF THOSE INDIVIDUALLY.
I HAVE TO SAY I THINK THE MOST
COMMON THEME I SAW IN THE
BREAKOUTS WAS THAT EVERYONE RAN
OUT OF TIME.
IT WAS ALMOST LIKE MORE THAN
HALF THE TIME WAS SPENT TRYING
TO UNDERSTAND THE SCOPE AND
DEFINITION OF WHAT THE GROUP WAS
SUPPOSED TO TACKLE, IN THE THREE
DAYS THAT WE'RE HERE TOGETHER.
AND THEN THAT LEFT VERY LITTLE
TIME AT THE END TO KIND OF
ACTUALLY TACKLE THE STAKEHOLDER
LIST THAT YOU GENERATED.
BUT IT WAS CLEAR THAT THERE WAS
A VERY COMMON, YOU KNOW, LIST
THAT EVERYBODY CAME UP WITH SO
EVEN THOUGH EVERYBODY TOOK LIKE
DIFFERENT APPROACHES TO TACKLE
THIS PROBLEM, THEY CAME UP WITH
A LIST THAT WAS FOR THE MOST
PART VERY SIMILAR TO ONE
ANOTHER, AND SO I THINK THAT
WILL BE REALLY INTERESTING.
I HOPE THAT THE BREAKOUT LEADS
WERE LISTENING TO THE OTHER
BREAKOUTS, AND THEIR LIST OF
STAKEHOLDERS, BECAUSE, YOU KNOW,
IT'S QUITE LIKELY THAT, YOU
KNOW, THERE'S A STAKEHOLDER THAT
YOU SHOULD PROBABLY INCORPORATE
IN YOUR DISCUSSIONS FOR
TOMORROW.
IF I COULD BREAK IT DOWN IT
SEEMED LIKE EVERYBODY WAS
THINKING ABOUT THE WHOLE LIFE
CYCLE PROCESS OF WHAT IT TAKES
TO BUILD AND DEPLOY A.I. MODEL
IN THIS SPACE SO EVERYTHING FROM
THE DATA GENERATION, DATA
CREATION, YOU KNOW, THE INPUTS
OF THAT DATA FROM CONSENT ALL
THE WAY TO THE HARMONIZATION OF
THAT DATA, AND THEN TO THE MODEL
BUILDERS, THE MODEL DEVELOPERS,
AND THEN THE FOLKS WHO MAY NOT
HAVE MODEL TRAINING EXPERTISE
BUT ARE IN CHARGE OF DEPLOYING
IN THE HEALTH CARE SYSTEM.
AND FINALLY THE PATIENTS
ACTUALLY AFFECTED BY THE
OUTCOMES OF THESE MODELS.
I WAS ALSO STRUCK WITH THE META
LEVEL DISCUSSION AFTERWARDS.
WE HEARD ABOUT THE
ANTHROPOLOGISTS WHO ARE
STAKEHOLDERS I HAD NOT
PREVIOUSLY THOUGHT ABOUT UNTIL
THAT WAS BROUGHT UP.
AND IT WAS ALMOST SO THAT IT
COULD ENABLE LIKE AN M AND M
TYPE DISSECTION OF WHAT WENT
WRONG IN THE FUTURE WHEN
INEVITABLY NO MATTER ALL THE
EFFORTS IN THIS ROOM, THERE ARE
GOING TO BE MISTAKES THAT WILL
HAPPEN, IN ORDER TO ENABLE A
CAREFUL DISSECTION OF WHAT WENT
WRONG AND FIX IT IN THE FUTURE,
I THINK, YOU KNOW, IDENTIFYING
THE NEEDS OF THOSE PEOPLE ARE
GOING TO BE REALLY CRITICAL.
ALL RIGHT.
SO, THAT WAS MY RECAP FOR THE
DAY.
AND I AM VERY EXCITED TO THINK
ABOUT WHAT'S GOING TO COME HOME,
AS YOU START TO THINK ABOUT THE
GAPS FROM THE CURRENT STATE OF
WHERE WE ARE WITH TRANSPARENCY
SO TOMORROW MORNING I'LL START
WITH A TALK WHERE I TRY TO DO MY
BEST BY EXPLAINING WHERE THINGS
ARE IN TERMS OF DATA SHEETS AND
MODEL CARDS, AND THEN YOU'LL GET
INTO YOUR BREAKOUT SESSIONS AND
I PROMISE YOU YOU'LL HAVE TIME
HOPEFULLY TO CHEW ON THESE IDEAS
AND HOPEFULLY EVEN DURING THE
EVENING YOU MIGHT HAVE
ADDITIONAL THOUGHTS THAT YOU CAN
BRING FORWARD TO YOUR GROUPS.
SO, THANK YOU.
AND LAURA, I DON'T KNOW IF YOU
HAVE ANYTHING YOU WANT TO SAY.
[APPLAUSE]
>> THANK YOU TO, AARON, ALL OUR
SPEAKERS, ALL OF YOU FOR THE
WORK YOU'VE BEEN DOING TODAY.
I THINK IT'S PAYING ENORMOUS
DIVIDENDS ALREADY, IT'S REALLY
EXCITING.
JUST A COUPLE LOGISTICS THINGS,
WE START TOMORROW AT NINE,
COFFEE OUT FRONT BEFORE THAT.
MANY OF YOU HAVE ORDERED LUNCH
WHICH WILL COME AND BE
DELIVERED.
IF YOU HAVEN'T ORDERED LUNCH,
YOU MIGHT WANT TO THINK ABOUT
HOW TO BRING SOMETHING TO EAT
BECAUSE WE ARE A LITTLE BIT OF A
FOOD DESERT.
COME TO US IF YOU HAVE QUESTIONS
ABOUT HOW TO SOLVE THAT PROBLEM
AND WE'LL GET CREATIVE WITH YOU
IF YOU NEED TO.
OTHERWISE IF YOU ORDERED LUNCH
ALREADY, IT WILL JUST COME
DURING THE LUNCH BREAK.
HAVE A VERY NICE EVENING.
REST UP.
ENJOY EACH OTHER.
AND RUMINATE ON SOME OF THIS
BECAUSE WE'LL BE PICKING IT UP
TOMORROW.
THANK YOU SO MUCH. 3472 02:03:41,680 [APPLAUSE]