WELCOME TO THE CLINICAL CENTER GRAND ROUNDS,
A WEEKLY SERIES OF EDUCATIONAL LECTURES FOR PHYSICIANS AND
HEALTH CARE PROFESSIONALS BROADCAST FROM THE CLINICAL
CENTER AT THE NATIONAL INSTITUTES OF HEALTH IN
BETHESDA, MD.
THE NIH CLINICAL CENTER IS THE WORLD'S LARGEST HOSPITAL TOTALLY
DEDICATED TO INVESTIGATIONAL RESEARCH AND LEADS THE GLOBAL
EFFORT IN TRAINING TODAY'S INVESTIGATORS AND DISCOVERING
TOMORROW'S CURES.
LEARN MORE BY VISITING US ONLINE AT HTTP://CLINICALCENTER.NIH.GOV
WELCOME AGAIN EVERYBODY TO
ETHICS GRAND ROUNDS.
I'M FROM THE DEPARTMENT OF
BIOETHICS.
WE SPONSOR THESE ETHICS GRAND
ROUNDS FOUR TIMES A YEAR FOR
PEOPLE PLANNING AHEAD THE NEXT
ONE ARE THE FIRST WEDNESDAYS AT
NOON HERE AND ALSO VIRTUALLY.
FIRST WEDNESDAY OF DECEMBER,
FEBRUARY AND APRIL.
FIRST WITNESS OF DECEMBER,
FEBRUARY AND APRIL.
FOR THOSE WHO HAVE BEEN HERE FOR
A WHILE THE GOAL IS TYPICALLY
BASED ON CONSULTS WE GOT.
WE HAVE A CLINICIAN PRESENT A
CASE AND INVITE AN OUTSIDE
EXPERT TO GIVIS THIS THOUGHTS ON
THE CASE.
THE FIRST IS THE GOAL OF THESE
IS JUST TO COME UP WITH
INTERESTING IMPORTANT ETHICAL
ISSUES FOR US TO DISCUSS.
NOT TO SET POLICY OR MAKE
DECISIONS FOR THE NIH.
EVERYBODY WHO SPEAKS HERE SPEAKS
ONLY FOR THEMSELVES UNLESS THEY
TELL YOU OTHERWISE.
NOBODY HERE IS A SPOKESPERSON
FOR ANYBODY ELSE.
I'M CERTAINLY NOT A SPOKESPERSON
FOR ANYBODY BUT AT LEAST
SOMETIMES FOR MYSELF.
SO, THERE'S ALSO CME CREDIT.
TO GET CME CREDIT YOU NEED TO
TEXT THE NUMBER 50525 AND YOU
TEXT IT TO THE PHONE NUMBER THAT
YOU HAVE COMING UP ON THE SLIDE.
IT'S THROUGH HOPKINS WE GET THE
CME CREDIT SO TEXT 50525 TO
THEM.
PEOPLE HERE WHO HAVE QUESTIONS,
PLEASE GO TO THE MICS IN THE
AISLES.
IF YOU'RE ONLINE, PLEASE GO TO
THE FEEDBACK BUTTON ON THE
WEBSITE.
YOU MIGHT HAVE TO SCROLL DOWN TO
SEE IT AND CLICK THAT AND PUT IN
YOUR QUESTION AND E-MAIL WILL
COME TO MY E-MAIL ACCOUNT AND MY
KIND COLLEAGUE ROBERT STEELE IS
TAKING THOSE.
YOU MIGHT SEE HIM HOGGING THE
MICROPHONE ANSWERING QUESTION WE
GET FROM ONLINE.
TODAY WE'LL TALK ABOUT THE TOPIC
I ASSUME DOESN'T NEED
INTRODUCTION FOR ANYBODY THE
ETHICAL ISSUES RAISED BY CHAT
GPT AND OTHER LARGE LANGUAGE
MODELS SPECIFICALLY IN THE
CONTEXT OF RESEARCH SO
RESEARCHERS ARE STARTING TO USE
THESE MODELS TO DO THINGS LIKE
LITERATURE SEARCHS AND RIGHT
ROUGH DRAFTS, FIGURE OUT WHAT
ARE IMPORTANT AND INTERESTING
RESEARCH QUESTION TO PURSUE AND
THE QUESTION IS WHAT ARE THE
ETHICAL ISSUES THAT GET RAISED
BY THAT AND HOW DO WE ADDRESS
THEM AND TO GIVE US THE CASE WE
HAVE NICK ASENDORF FROM NHLBI.
HE'LL GIVE US THE CASE AND
AFTERWARDS I'LL TRY TO GET HIS
AFFILIATIONS RIGHT -- DAVID
MAGNUS THE THOMAS RAFFIN
PROFESSOR AT STANFORD AND THE
DIRECTOR OF THE BIOETHICS
INSTITUTE AT STANFORD
UNIVERSITY.
HE'S RECENTLY BECOME AN
ASSISTANT DEAN WHICH I
UNDERSTAND IS HIS FAVORITE JOB
HE'S EVER HELD.
DAVID IS GOING TO GIVE US HIS
COMMENTS AND HIS VIEWS ON THE
ETHICAL ISSUES RAISED BY THE
CASE.
TO GET US GOING, NICK.
>> HOPEFULLY THAT'S COMING
THROUGH WHILE ONLINE.
LIKE I SAID I'M NICK ASENDORF
THE SCIENTIFIC INFORMATION
OFFICER AT NHLBI.
BEFORE I JOINED THE NIH I WORKED
AT 3M.
MY HOPE IS TO GIVE A BRIEF
OVERVIEW OF LARGE LANGUAGE
MODELS AND HAVE NOTHING TO
DISCLOSE AND HOW THEY'RE TRAINED
AND DEPLOYED AND POTENTIAL USE
AREAS WHERE THEY MIGHT BE USED
IN RESEARCH AND IN IN THE
CLINICAL SETTING AND THEN TO
BRING UP QUESTIONS FOR THE
BROADER GROUP TO DISCUSS RELATED
TO THEIR USE AND ETHICS BEHIND
THOSE.
LARGE LANGUAGE MODELS ARE A SUB
CLASS OF AS A MORE ARMING RIDGE
KNOWN AS GENERATIVE A LOT OF AND
THEY SEEK TO GENERATE NEW AND
NOVEL CONTENT WHETHER THAT BE
TEXT, IMAGES OR POTENTIALLY
OTHER MODALITIES LIKE GENERATING
CODE IN THE CASE OF GITHUB
CO-PILOT.
WHAT HINGES ON THE QUALITY IS A
LARGE TRAINING CORPUS OF DIVERSE
DATA DEPENDING ON THE TASK YOU
WISH THE MODEL TO HAVE, YOU NEED
DIFFERENT CORPUSES WHETHER AS
THE WIKIPEDIA OR INSTAGRAM WITH
THEIR CAPTIONS AND CODE FROM
GITHUB AND OF PARTICULAR
INTEREST OF WELL IS FOR LANGUAGE
MODELS TO MAKE SURE THAT DATA
SET THAT YOUR CORPUS HAS A WIDE
RANGE OF LANGUAGE INVOLVED TO
MAKE SURE THE MODELS ARE USABLE
BY THE GLOBAL COMMUNITY.
A.I. RESEARCHERS WILL DESIGN
THESE NEURAL NETWORKS THAT ARE
USED TO TRAIN THESE MODELS AND
THESE NEURAL NETWORKS ARE
INCREDIBLY COMPLEX AND THERE'S A
TON OF RESEARCH IN TERMS OF HOW
TO DESIGN THE ARCHITECTURE OF
THESE NETWORK TO ACCOMPLISH THE
TASK YOU WISH.
THE MODELS OFTEN HAVE BILLIONS
OF PARAMETERS AND TAKE MONTHS TO
TRAIN ON SPECIALIZED EQUIPMENT.
THE POWER AND COOLING COSTS MAKE
THE MODELS EXTREMELY CHALLENGING
AND COST PROHIBITIVE TO TRAIN BY
YOURSELF.
I WANT TO SHOW OPEN A.I. DOLLY 2
IT HAS JUST RELEASED DOLLY 3 A
FEW WEEKS AGO.
I ENCOURAGE PEOPLE TO GO OUT AND
LOOK AT THE DIFFERENCES BETWEEN
WHAT I'LL SHOW HERE AND WHAT
CAME OUT A WEEK OR TWO AGO.
>> IS THAT NOT GOING THROUGH FOR
FOLKS?
WANT ME START IT AND GET IT
GOING AGAIN?
THIS IS A FUN VIDEO THAT SHOWS
SO DALLE IS AN OPEN A.I. COMPANY
RELEASE THE SAME COMPANY THAT
MAKES CHAT GPT AND THE WHAT THE
TEXT IMAGE-BASED MODELS ALLOW TO
YOU DO IS TO PUT IN A TEXT
PROMPT LIKE WHAT YOU'LL SEE
KOALAS RIDING MOTORCYCLES AND
THE NETWORKS ARE TRAINED TO
LEARN THE RELATIONSHIP BETWEEN
KOALAS AND MOTORCYCLES AND
DETERMINE VARIED STYLES OF
IMAGES.
IT'S NOT ONLY ONE BUT MULTIPLE
TYPES OF IMAGES.
I'LL END THAT THERE AND WE CAN
KEEP GOING ON HERE.
I ENCOURAGE YOU TO LOOK AT THE
DALLE 3 ONES AND ENCOURAGE YOU
TO LOOK AT THAT.
THEY'RE TEXT TO TEXT MODELS AND
TRAINED ON LOTS AND LOTS OF
TEXTUAL BATE AND GOOGLE BARD ARE
THE MOST FAMOUS AND OTHER MODELS
OUT THERE RIGHT NOW.
OF PARTICULAR NOTE THEY'RE
PROBABILISTIC AND DON'T RETURN
THE CORRECT ANSWER OR SOMETHING
YOU'RE NOT PULLING THE EXACT
DATA FROM THE CORPUS INSTEAD
IT'S PROBABILISTIC AND
GENERATING THE MOST PROBABLE
SEQUENCE IN RESPONSE TO WHAT YOU
GET.
THE CONTEXT HERE MATTERS IN THE
SENSE THAT IT OFTEN LOOKS PAST
NOT ONLY THE PROMPT YOU GIVE IT
BUT THE PASS SEQUENCE OF
CONVERSATIONS YOU'VE HAD WITH
IT.
I MENTIONED EARLIER GPT 4 IS
ESTIMATED TO HAVE COST $100
MILLION TO CREATE WHICH IS
PROHIBITIVE FOR YOU AND I
INTEREST TO TRAIN OURSELVES.
THIS IS SIX MONTHS OLD AND I
WANT TO HEADLINE THE DIFFERENCE
BETWEEN SOME OF THESE MODELS ARE
OPEN SOURCE AND SOME ARE CLOSED
SOURCE.
AND META-AND LLAMA AND CAME OUT
WITH LLAMA V2 ARE OPEN SOURCE.
METAIS RELEASING THE WEIGHTS TO
USE SO WE DON'T HAVE TO SPEND
MONTHS TRAINING ALGORITHMS OR
THE COST TO TRAIN THOSE.
THEY CAN GENERATE MORE DOMAIN
SPECIFIC APPLICATIONS THAT ARE
COMING UP.
I'LL TAILOR IT TO THOSE HERE AND
WHEN YOU THINK OF HOW THE MODELS
ARE USED CURRENTLY THE MOST
COMMON USE CASE RIGHT NOW IS TO
BE USED AS A WRITING AID WHETHER
IT'S GIVING THAT SOME DOCUMENT
TO INJECT HUMOR INTO IT TO CHECK
FOR GRAMMAR, TO COMPLETELY
CHANGE THE STYLE OR ALMOST
USE -- I'VE USED IT BEFORE AS A
THESAURUS FOR TRYING TO FINE
TUNE WHAT I WANT AND ALMOST AS A
CONTENT GENERATION TO GET PEOPLE
OVER THE INITIAL HURDLE AND IT'S
EASIER TO EDIT SOMETHING THAN TO
CREATE SOMETHING OURSELVES AND
THERE'S AN INTERESTING USE CASE.
THINKING ABOUT GIVING IN TERMS
OF SERVICE THAT YOU GET ALL THE
TIME THAT IS SO LONG NO ONE HITS
ACCEPT YOU CAN FEED THIS IN AND
ANALYZE TO SEE IF THERE'S
ANYTHING NEFARIOUS AND
SUMMARIZING LITERATURE AND GRANT
REVIEWS AND POTENTIAL
APPLICATIONS.
WHERE RESEARCH IS GOING IS TO
DEVELOPING FINE TUNE OR
DOMAIN-SPECIFIC MODELS FOR
PARTICULAR APPLICATIONS.
ONE THAT IS GAINING RECENTLY IS
IN TOOLS FOR HELPING PROGRAMMERS
CODE FASTER.
CODE LLAMA IS A VARIANT AND
GITHUB CO-PILOT HAS ITS OWN AND
ACCELERATING THE WAY DEVELOPER
CAN WRITE CODES AND COMPANIES TO
TRAIN ON THEIR SPECIFIC I.T.
DOCUMENTATION OR HR POLICY TO
REDUCE TEDIOUS INTERNET PAGES
COMPANIES MAY HAVE.
IN THE CLINICAL SETTING WE CAN
THINK ABOUT POTENTIALLY MAKING
TOOLS TO AID CLINICIANS AND
PATIENT CARE, DIAGNOSIS AND
TREATMENT AND COMPLETING RECORDS
AUTOMATICALLY AND ALMOST HAVING
THE EXTRA PERSON AND THE MODELS
HAVE THE ABILITY TO HALLUCINATE
AND STEMS FROM THE MODELS BEING
PROBABILISTIC AND CREATING THE
MOST RESPONSE PROBABLE RESPONSE
GIVEN WHAT YOU HAVE.
THEIR ABILITY TO MAKE UP
INFORMATION IS A BIG RISK AND
FORCES US TO HAVE A HUMAN IN THE
LOOP OF A LOT OF THESE
APPLICATIONS WHICH IS
PARTICULARLY RELEVANT IN THE
CLINICAL SETTING.
WE MIGHT BE TAKING ACTIONS BASED
ON THESE LARGE LANGUAGE MODELS.
AN IMPORTANT THING BETWEEN THE
DIFFERENT BETWEEN PUBLIC AND
PRIVATE SYSTEMS AND DATA LOSS IS
ON THE TOP OF MIND ESPECIALLY
HERE IN THE GOVERNMENT AND
EXPOSING DATA WE DON'T WISH TO
BE OUT IN THE PUBLIC.
WHETHER THAT BE PRE DECISIONAL
INFORMATION OR DRAFT POLICY OR
PATIENT INFORMATION.
WE TALK ABOUT THE HUGE AMOUNT OF
DATA WE NEED TO TRAIN MODELS.
THEY'RE ONLY AS GOOD AS THE
TRAINING DATA.
IF THERE'S BIAS IN THE TRAINING
DATA THERE WILL BE BIAS IN THE
MODELS AND HOW DO WE COMBAT
THAT.
IT'S ALMOST UNAVOIDABLE.
THERE'S NO WAY WE CAN MAKE SURE
OUR TRAINING DATA SET IS VOID
AND DEVELOPING SYSTEMS THAT TAKE
THE RESPONSES AND CHECK FOR BIAS
IS SOMETHING THAT'S AN ACTIVE
RESEARCH TOPIC.
PLAGIARISM IN TERMS OF THE LARGE
LANGUAGE MODEL AND PLAGIARIZING
AND WHETHER IT'S HOW IT CITE
WHETHER USING IT TO COMPLETE
HOME WORK OR IN THE ACADEMIC
SENSE USING IT TO HELP YOU WRITE
PAPERS AN EDIT PAPERS AS WELL.
I WANT TO DIFFERENT IS THAT
RIGHT BETWEEN THE TECHNOLOGY AND
TOOL FOR THE MODELS.
THE UNDERLYING MODEL THE TRAINED
WEIGHTS AND PARAMETERS OF THESE
MODELS WHETHER THAT BE LLAMA,
GPT 4, PALM, WHICH FEEDS INTO
GOOGLE BARD AND DEPLOY IN
MULTIPLE SYSTEMS WHETHER THAT IS
SOMETHING LOCAL TO MY MACHINE OR
IN A PUBLIC INTERFACE AND GOOGLE
BARD AND CHAT GPT ARE PUBLIC
SYSTEMS.
WE AS USERS DO NOT KNOW WHAT
THAT BACK END MODEL WE TALKED
ABOUT OPEN SOURCE AND CLOSED
SOURCE BEFORE.
WHEN WE SUBMIT DATA UP TO CHAT
GPT WE DON'T KNOW HOW OPEN A.I.
IS USING THE DATA AND HOW GOOGLE
IS USING THE DATA.
WE CAN READ A BIG PUTOUT WHETHER
WE TRUST WHAT THEY NEED TO SAY
IS A DIFFERENT STORY.
BUT WE CAN TAKE THE MODELS IN
THE CASE OF OPEN SOURCE AND RUN
A LARGE LANGUAGE MODEL LOCALLY
ON MY MACHINE.
THE APPLICATION WHERE I WOULD
WANT TO PUT SOCIAL SECURITY
NUMBER OR PATIENT RECORD UP INTO
A PUBLIC SYSTEM THAT USE CASE IS
ENABLED TO FIND IT RUNNING
LOCALLY ON MY MACHINE.
WHEN I CAME BACK TO TALK ABOUT
WHAT DOES IT LOOK LIKE IF WE AS
THE NIH WERE TO CREATE AN
IN-HOUSE SYSTEM THAT RUNS A
LOCAL LARGE LANGUAGE MODEL AND
HOW DOES IT CHANGE BETWEEN THE
PUBLIC SYSTEMS, ME RUNNING
SOMETHING LOCALLY VERSUS THE
SHARED COMMUNAL RESOURCE AT THE
NIH.
I THINK DEPENDING ON HOW THE
TOOL IS DEPLOYED HERE CHANGES
THE ANSWERS TO SOME OF THE
ETHICAL QUESTIONS WE MIGHT POSE.
I'LL CLOSE HERE JUST WITH SOME
GENERAL QUESTIONS THAT
DR. MAGNUS CAN TAKE AND RUN FROM
AFTER I FINISH UP HERE.
CONSENT IS A HOT TOPIC I THINK
WHEN WE THINK ABOUT NOT ONLY FOR
LARGE LANGUAGE MODELS AND
PATIENT C-R AND THE IMAGES THEY
GENERATOR ANY DATA COLLECTED
WITHIN A STUDY SEQUENCING DATA
AND FINE TUNING NEW MODELS-.
HOW DO WE ASSESS THE DATA
INJECTED FROM THE MODELS IS
GREATER OR LESS THAN THE
POTENTIAL BIAS THAT EXISTS IN
OUR EVERYDAY ASSESSMENT AS
HUMANS AND SIMILARLY WHEN THEY
GIVE THE WRONG ANSWER, HOW DO WE
ASSESS THE ERROR RATE TO WHAT WE
DO AS A HUMAN IN WHATEVER TASK
WE WANT?
THE AUTHORSHIP QUESTION IS
ANOTHER INTERESTING QUESTION OF
HOW THEY SHOULD BE CITED, WHEN
WE NEED TO DISCLOSE THAT WE'VE
USED A LARGE LANGUAGE MELD OR
ANY GENERATIVE -- MODEL OR
GENERATIVE A.I.
THE TEXT TO IMPROVAGE AND TEXT
TO VIDEO TYPE OF APPLICATIONS
ARE MORE POIGNANT WHEN WE THINK
OF THE INFORMATION THAT CAN BE
SPREAD.
I'LL STOP THERE AND TURN IT OVER
TO DR. MAGNUS TO KEEP WALKING US
THROUGH AND MAYBE ANSWER
QUESTIONS PERHAPS.
THANKS, EVERYBODY.
>> THANKS, VERY MUCH.
I HOPE EVERYTHING'S LOOKING GOOD
AND PEOPLE CAN HEAR ME OKAY.
I'M NOT SEEING ANYTHING SENT TO
ME.
>> WE CAN HEAR.
>> GREAT.
OKAY.
NO CONFLICTS OF INTEREST TO
DISCLOSE AND I'M GOING TO GET
STRAIGHT INTO IT.
THESE ARE SOME OF THE ISSUES
JUST RAISED BY LARGE LANGUAGE
MODELS AND THEIR RESEARCH.
CONSENT PER AND CONCERNS ABOUT
BIAS AN DISCRIMINATION AND
ISSUES OF RELIABILITY AND
RELIANCE ON THE OUTPUTS FROM
LARGE LANGUAGE MODELS AND CREDIT
AND AUTHORSHIP AND I WOULD ALSO
ADD THE POTENTIAL FOR INCREASE
IN MISCONDUCT AND THE POTENTIAL
FOR INTENTIONAL MISUSE NEED TO
BE CONSIDERED.
I WANT TO START BY GIVING A
COUPLE FRAMING ISSUES.
FIRST AROUND WHAT BELONGS TO THE
IRB REALM AND IS THINKING ABOUT
IN THE RESEARCH REALM VERSUS
DOWN STREAM ETHICAL ISSUES AND
OTHER METHODS NEED TO BE
DEVELOPED TO DEAL WITH AND
SECOND THE ISSUE ABOUT WHAT IS
REALLY NOVEL HERE THAT LARGE
LANGUAGE MODELS RAISE VERSUS
WHAT ARE IN FACT FAMILIAR ISSUES
FROM EITHER A.I. BEFORE THIS OR
EVEN FROM OTHER DOMAINS OF
RESEARCH.
SO THE FIRST FRAMING IS JUST TO
RECOGNIZE THE LIMITATIONS OF
HUMAN SUBJECTS PROTECTION IN THE
WAY WE THINK ABOUT THE
REGULATIONS.
SINCE THE DEVELOPMENT OF
HELSINKI, BELMONT, THE TOUCH
STONE DOCUMENT FOR HUMAN
SUBJECTS PROTECTION AND THE
REGULATIONS THAT THEY WIND UP
PRODUCING FOCUS ON RISK TO
SUBJECTS NOT SOCIETY.
IRBs LOOK TO RISK TO SUBJECTS TO
ENSURE THEY'RE MINIMIZED AND
REASONABLE AND ENSURE THERE'S
INFORMED CONSENT AND THAT
THERE'S EQUITABLE SUBJECT
SELECTION AT LEAST IN PRINCIPLE
THOUGH THAT LAST ONE OFTEN GETS
SHORTED AND WHAT IRB'S DON'T
LOOK IS LOOK AT LONG RANGE
EFFECTS OF THE KNOWLEDGE AND THE
DOWN STREAM IMPLICATIONS OF THE
RESEARCH FOR SOCIETY.
THAT'S OUTSIDE THE SCOPE OF
IRB'S ABILITY TO USE THOSE AS
RISK OF RESEARCH AND YOU NEED
ANOTHER MECHANISM FOR DEALING
WITH THE DOWN STREAM
IMPLICATIONS OF A.I. AND THAT'S
ONE AREA WHERE A LOT OF RESEARCH
IS GOING ON.
WE PUBLISHED A PAPER IN 2021 IN
THE PROCEEDINGS OF THE NATIONAL
ACADEMY OF SCIENCE ABOUT AN
APPROACH TO THIS WE'VE BEEN
USING AT STANFORD WITH A.I.
RESEARCH FUNDED BY THE HUMAN
CENTERED A.I. INSTITUTE HERE AT
STANFORD WHERE WE DEVELOPED AN
ETHICS AND SOCIETY REVIEW
PROCESS AND THAT'S A PROCESS
THAT'S GETTING IT -- IT RATED
AND DEVELOPED AND FINDING WAYS
OF FINDING ETHICAL SCAFFOLDING
AND GETTING RESEARCHERS TO THINK
ABOUT THE DOWN STREAM ISSUES
THEMSELVES SINCE THEY'RE OUTSIDE
THE SCOPE OF THE TRADITIONAL IRB
REGULATORY SYSTEM.
THE SECOND FRAMING ISSUE AND
THIS IS GONE TO RUN THROUGHOUT
EVERYTHING ELSE I SAY IS THINK
ABOUT THE WAYS IN WHICH THE
ISSUES RAISED ARE SIMILAR TO
THOSE RAISED BY A.I. IN OTHER
APPLICATIONS OF HEALTH CARE IN
PARTICULAR PRE DUDICTIVE ANALYT
DISCUSSED FOR A WHILE.
AND CAN BE BIAS IN DATA LEADING
TO DISCRIMINATORY OUTPUTS I'LL
DETAIL IN A MINUTE.
THERE'S MORE TRAINING DATA IN A
LOT OF THESE LARGE LANGUAGE
MODELS THAN TYPICAL FOR FOR
EXAMPLE PREDICTIVE ANALYTIC
TOOLS.
WE'VE ALSO SEEN SMALL PROMPT
VARIATIONS CAN PRODUCE
DISPARITIES IN OUTCOMES AND
ENGINEERING IS BEING DEVELOPED
FOR BEING ABLE TO ACCURATELY USE
THESE THINGS AND THERE'S STILL
BIAS IN THE DATA.
IT MAY BE MORE OF IT BUT DOESN'T
MEAN IT'S LESS BY AS BECAUSE THE
INTERNET IS QUITE BIASSED.
THIS IS ONE OF MY FAVORITE
STATEMENTS, WE DO NOT SEE THINGS
AS THEY ARE, BUT AS WE ARE AND I
SUGGEST THAT'S TRUE IN MORE
DETAIL IN A MINUTE.
AND CONSENT FOR DATA USE AND
RELIABILITY AND RELIANCE AND
MISUSE.
ARE ALL ISSUES WE'VE DEALT WITH
IN OTHER DOMAINS OR OTHER PARTS
OF A.I. FOR A WHILE.
STARTING WITH CONSENT FOR DATA
USE.
SO ISSUED AROUND CONSENT FOR
DATA ARE NOT NEW AT ALL.
ISSUES AROUND BLOOD SPOTS,
ELECTRONIC HEALTH RECORDS IN
DATA.
THIS IS STUFF THAT WE'VE BEEN
DEALING WITH FOR A LONG TIME.
SO WE KNOW SOMETHING ABOUT IT.
DOESN'T MEAN IT'S COMPLETELY
SOLVED BUT THERE'S NOTHING
UNIQUE ABOUT LARGE LANGUAGE
MODELS FOR THOSE ISSUES.
THE ONLY THING THAT'S NEW HERE
IS RELATED TO THE OUTPUT OF
LARGE LANGUAGE MODELS.
IT'S NOT BIG DATA IN AND THEY'RE
UBIQUITOUS AND PEOPLE HAVE BEEN
WRITING ABOUT THAT BUT THE
OUTPUTS INCLUDE THINGS THAT
COULD LEAD TO POTENTIAL COPY
RIGHT VIOLATIONS, TUNING ON
SOMEBODY'S STYLE OF WRITING TO
PRODUCE RESEARCH ARTICLES AND A
SUCCESSFUL RESEARCHER AND THE
ISSUE OF THE AMERICAN JOURNAL OF
BIOETHICS AN ARTICLE WHERE
RESEARCHERS TUNED CHAT GPT TO
WRITE IN THE STYLE OF THEIR OWN
WORK.
THOSE ARE NEW ISSUES THAT NEED
TO BE DEALT WITH AND IT'S THE
OUTPUTS THAT ARE NEW.
THE OTHER ISSUE THAT'S NEW WELL
NOT NEW IN SOME WAYS BUT
DIFFERENT FROM SOME OF THE OT
OTHOTHER
[NO AUDIO]
AND GETTING POP UP AND GET SENT
TO YOU ABOUT COMPANIES OR
COMMERCIAL ENTITIES THAT HELP
WITH WRITING GRANTS AND THERE'S
CONSULTANTS THAT DO THESE THINGS
AS A RESULT OF THE SALE OF YOUR
DATA AND.
THE I.T. COMPANIES DEVELOPING
THESE ARE OUTSIDE THE BIO TECH
SPACE AND MOVING INTO IT AND NOT
REALLY UP TO RESPECT HIPAA AND
NOT COVERED ENTITIES AND OFTEN
DON'T FALL UNDER THE COMMON
RULE.
I WOULD SAY BUYER BEWARE WHEN
YOU SHARE DATA THROUGH YOUR
SEARCHS WITH AN I.T. COMPANY
OUTSIDE A BUSINESS ASSOCIATE
AGREEMENT AND I WOULD SAY YOU
SHOULD BE WARY EVEN WHEN YOU DO
BECAUSE THE WAY THE I.T.
COMPANIES THINK ESPECIALLY GIVEN
THE CHICAGO LAWSUIT AGAINST
GOOGLE AND IF YOU CAN'T SHOW
HARM THE COURTS ARE NOT GOING TO
FIND FOR DAMAGES AND SO I.T.
COMPANIES ARE JUST GOING TO MAKE
JUDGMENTS WHETHER ABIDING BY
THESE MATERIAL USE AGREEMENTS
ARE WORTH IT AND THOSE ARE JUST
COMMERCIAL DECISIONS THEY'RE
GOING TO MAKE RATHER THAN BEING
SACROSANCT WE THINK ABOUT THEM
IN THE MEDICAL SPHERE.
IN TERMS OF BIAS THIS HAS BEEN A
WHILE.
CONCERNS ABOUT THIS IN MACHINE
LEARNING MORE GENERALLY.
YOU CAN SEE WAY BACK WHAT IS NOW
ANCIENT DAYS OF 2018, WE RAISE
CONCERNED ABOUT THE NEW ENGLAND
JOURNAL OF MEDICINE ARTICLE AND
SOMETHING WE'VE BEEN TALKING
ABOUT FOR A HANDFUL OF YEARS AND
OF COURSE THE FAMOUS ARTICLE
DEMONSTRATING THE COMMONLY USED
ALGORITHM FOR ASSIGNING SOCIAL
WORK RESOURCES FOR PATIENTS WHO
WERE HIGH USERS OF RESOURCES AND
MIGHT MIGHT BE AT RISK OF
RETURNING TO THE HOSPITAL IF
THEY DON'T GET ADEQUATE SOCIAL
SUPPORT AND WHEN THEY DISSECTED
THE ALGORITHM OF MILLIONS OF
PEOPLE BECAUSE THEY USED COST AS
A PROXY FOR HEALTH AND BECAUSE
AFRICAN AMERICAN PROPOSITIONS
RESULTS IN DISCRIMINATION
AGAINST THE BLACK POPULATION
USED BY MILLIONS OF PEOPLE WHO
WERE IMPACTED IN HEALTH SYSTEMS
USING THOSE ALGORITHMS.
PEOPLE SEE THIS AS A MISTAKEN
CHOICE AS USING COST FOR PROXY
OF HEALTH AND HIGHLIGHT THE
IMPORTANCE OF INTEREST OF THE
PEOPLE BUYING THESE OR
DEVELOPING AND BUYING THESE
ALGORITHMS.
THE HEALTH CARE SYSTEMS BUYING
THE ALGORITHM CARED ABOUT COST
AND MADE SENSE COST WOULD BE A
PROXY FOR HEALTH AND THIS IS
CONCERNING BECAUSE AS COMMERCIAL
ENTITIES WIND UP BUYING THESE
SERVICES THE REALITY IS
DISCRIMINATION AGAINST THE
POPULATIONS IS PROBABLY COST
EFFECTIVE AND SO THAT'S
SOMETHING THAT'S GOING TO HAVE
TO BE CONSIDERED AND BE
CONCERNED ABOUT.
THERE'S ALSO PROBLEMS IN
ADDITION TO THE DESIGN OF THE
ALGORITHMS THEMSELVES WITH
POTENTIAL FOR BIAS AND
LIMITATIONS AS LEARNED FROM IN
THE TALK YOU JUST HEARD.
INCLUDING MISINFORMATION FROM
POPULATION AND GET BIAS FROM
GAPS AND CAN LEAD TO
SELF-FULFILLING PROPOSITIONS IN
SPITE OF THE FACT THAT OUTCOMES
FOR UNDOCUMENTED IMMIGRANTS ARE
SLIGHTLY BETTER THAN THE AVERAGE
PATIENT, THE VAST MAJORITY OF
HOSPITALS WE HAVE SURVEY DATA
SHOWING MOST TRANSPLANT PROGRAMS
MAKE IT AT LEAST A
CONTRAINDICATION TO BEING LISTED
FOR TRANSPLANT THAT SOMEBODY'S
UNDOCUMENTED IMMIGRANT.
THOSE ARE BIASSED DECISIONS.
THAT MEANS A.I. WILL LEARN FROM
THE BIASSED DECISIONS THESE
PATIENTS DON'T DO WELL.
YOU LOOK AT THE OUTCOMES,
THEY'LL BE BAD FOR THOSE
UNDOCUMENTED OR DEVELOPMENT
DELAY, GENETIC CONDITIONS, WHERE
THERE'S BIAS DECISIONS BY
CLINICIANS THE OUTCOMES ARE BAD
BECAUSE OF THE DECISIONS THAT
CLINICIANS MAKE AND THIS CAN
WIND UP CREATING SELF-FULFILLING
PREDICTIONS.
WE LIVE IN A RACIST SOCIETY IN
WHICH PEOPLE AND THERE'S SOCIAL
DETERMINATES OF HEALTH AND
THERE'S A HEALTH WEALTH GRADIENT
AT EVERY LEVEL OF WEALTH PEOPLE
GET HEALTHIER THE MORE YOU CLIMB
THE GRADIENT AND THAT MEANS
BECAUSE THEIR SOCIAL
DETERMINATES OF HEALTH THERE'S
REAL OUTCOMES NOT BIASES AND
DIFFERENCES THAT ARE A FUNCTION
OF THOSE SOCIAL DETERMINATES OF
HEALTH AND AGAIN ALGORITHMS AND
PREDICTIVE ANALYTICS IN
PARTICULAR WILL LEARN FROM THIS
AND MAKE THOSE KINDS OF
DECISIONS.
SO THESE ARE ALL THINGS THAT
WE'VE BEEN DEALING WITH FOR A
LONG TIME NOW AND WRITING ABOUT.
THERE'S STILL REAL PROBLEMS HERE
BUT THE PROBLEMS ARE REALLY NOT
DEVELOPING NEW TOOLS SO MUCH FOR
HOW TO ADDRESS THIS BECAUSE
THERE'S ALREADY A LOT OF TOOLS
OUT THERE TO ADDRESS THIS
BECAUSE WE'VE BEEN TALKING ABOUT
THIS FOR A WHILE.
IT'S REALLY IMPLEMENTATION.
HOW DO WE GO FROM THE FACT WE
KNOW THIS HAPPENS TO MAKE SURE
THAT RESEARCHERS ARE TRAINED AND
THE TOOLS BEING DEVELOPED ARE
UTILIZED TO AVOID THOSE KINDS OF
PROBLEMS.
RELIABILITY AND RELIANCE OF THE
GAP BETWEEN TRAINING DATA AND
APPLICATION IS GENERAL PROBLEM
WITH A.I.
IT'S ALSO BEYOND A.I.
THE CRISIS AND RIGOR AND
REPRODUCIBILITY OF RESEARCH
FINDING SUGGESTS LLM RELIABILITY
PROBLEMS ARE NOT UNIQUE.
THE FACT THAT SOMETIMES WE GET
WEIRD OR INCORRECT OR
PROBLEMATIC OUTCOMES FROM THAT.
THAT'S REALLY NOT THE PROBLEM.
I THINK THE PROBLEM IS REALLY
ABOUT RELIANCE.
IT'S NOT RELIABILITY, IT'S
RELIANCE.
ONE OF THE CHALLENGES OF CHAT
GPT IS UNLIKE SEARCH ENGINES IT
MAY USE A LARGE LANGUAGE MODEL
TO GIVE A RANGE OF PROBABILITY
OF ANSWERS WHICH IS WHAT YOU DO
WHEN I GET A SEARCH.
CHAT GPT GIVES ONE ANSWER WHICH
MAKES IT SEEM MORE
AUTHORITATIVE.
THAT'S PART OF THE PROBLEM AND
THE OTHER IS ANTHROMORPHISM WITH
THE TOOLS IN PROBLEMATIC WAYS.
THIS IS A SCREEN SHOT FROM A TV
SHOW CALLED THE GOOD PLACE.
JANET IS AN A.I. SO THESE THERE
AND THEY'RE ABOUT TO PUSH A
BUTTON TO DELETE HER AND WARNS
THEM THEY'LL BE A FAIL SAFE
WHERE SHE'LL GET THEM NOT TO
PRESS A BUTTON TO DELETE HER BUT
REMINDS HER SHE'S JUST AN A.I.
AND HAS IN FEELINGS AND NOT TO
WORRY ABOUT IT.
WHEN THEY GO TO PUSH THE BUTTON,
SHE SAYS, PLEASE DON'T DO THAT,
I DON'T WANT TO DIE, I DON'T
WANT TO DIE AND THEY BACK OFF
AND SHE SAYS, REMEMBER, I'M JUST
A.I.
THIS IS JUST A FAIL SAFE
MECHANISM TO PEOPLE DON'T
INADVERTENTLY PRESS THIS AND GO
FAR WARD AGAIN AND SHOWS THIS
PICTURE OF HER KIDS, SAYS LOSE
DON'T, MY SON HAS ASTHMA AND
THEY BACK OFF AGAIN.
I'M AN A.I., I DON'T HAVE KIDS.
THIS IS JUST A SCREEN SHOT FROM
THE NICKELODEON KID'S CHOICE
AWARDS.
I WORRY THAT ANTHROPOMORPHISM IS
PENETRATING AND A GROUP
PUBLISHED ABOUT THIS AND IT'S
PERMEATES AS A SHOW NOT JUST
PUBLIC UNDERSTANDING THE
LANGUAGE OF RESEARCHERS,
PROGRAMMERS AND DESIGNERS
THEMSELVES.
WE ACTUALLY HEARD SOME OF THIS
IN THE TALK WE JUST HEARD.
TERMS LIKE SEE, UNDERSTAND CAN
BE MISLEADING AND WISHFUL
THINKING AND HALLUCINATIONS.
THE CHAT GPT DOES NOT
HALLUCI
HALLUCINATE.
IT SUGGESTS A GLITCH.
IT'S A DESIGN FEATURE TO PLAY
THE ROLE IT PLAYS AND HAS
LIMITATIONS IN HOW IT FUNCTIONS
AND NEED TO THINK OF IT THAT WAY
RATHER THAN ANTHROPOMORPHIZING
IT.
THERE'S NO MEANING TO ANY OF THE
WORDS USED BY CHAT GPT.
IT'S JUST ABOUT THE PROBABILITY
OF DIFFERENT PHRASES AND SYMBOLS
WHICH HAPPEN TO BE WORDS.
AND WHAT SOUND LIKE AN
APPROPRIATE RESPONSE AND
RESPONSE TO THAT.
AS A RESULT OF THAT THERE ARE
THINGS A HUMAN CAN UNDERSTAND IT
WON'T.
I ENTERED WHAT IS THE NAME OF
THE ONLY SON OF PAUL GRAND
FAERNL THE -- FATHER THE ANSWER
I GOT IS I DON'T HAVE ENOUGH
QUESTION TO ANSWER THE QUESTION.
AUTHORSHIP AND CREDIT.
THERE'S A STATEMENT THAT'S GOING
TO BE COMING UP IN A BUNCH OF
BIOETHICS JOURNALS TOO A
STATEMENT ON THEIR RESPONSIBLE
USE.
AND A BUNCH OF US ARE PUBLISHING
THIS.
THE COPE AND ICMJE HAVE ALL
ARGUED WE SHOULD NOT ALLOW THESE
LARGE LANGUAGE MODELS TO BE
AUTHORS BECAUSE THEY HAVE NO
ACCOUNTABILITY OR AGENCY OR NO
ABILITY TO SIGN COPYRIGHT OR
STAND BEHIND RESULTS BUT I'LL
NOTE IT'S ALSO TRUE OF DECEASED
AUTHORS.
WE THINK MORAL AGENCY ACCOUNTS
FOR ACCOUNTABILITY AS WELL AS
THE CREATION OF A SCHOLARLY
COMMUNITY AND WITH WE SHOULDN'T
ALLOW THESE TOOLS TO ASCEND AS S
AND A WELL KNOWN ANALOGY IS WITH
WELL KNOWN TRADITIONS OF THE USE
OF STATISTICAL TOOLS AND REQUIRE
TRANSPARENCY NOT CREDIT.
YOU HAVE TO SAY CLEARLY WHEN YOU
USE A CERTAIN STATS PROGRAM TO
PRESENT YOUR DATA IT'S USUALLY
INCLUDED IN THE METHODS.
WE SHOULD SEE THIS LIKE WE SEE
ALL THE OTHER TOOLS WE'VE ALWAYS
BEEN USING AND NOT AS REALLY
SOMETHING NEW.
AND TRANSPARENCY ACHIEVES
MULTIPLE GOODS AND TRACE BIAS
AND ASSESS THE OWNERSHIP OF THE
PRODUCTS AND PREDICT THE SCHOLAR
COMMUNITY.
AND WE THINK THIS IS THE WAY TO
THINK ABOUT THIS THAT OFFERS THE
BEST PROTECTION.
NEW THINGS POTENTIAL FOR
MISCONDUCT IS DEFINITELY A NEWER
PROBLEM.
THERE'S OTHER TOOLS LIKE
PHOTOSHOP THAT HAVE BEEN AROUND
FOR DOING THIS BUT MAY BE SOME
TEXT TO IMAGE LLM MAY BE MAKE IT
EASIER TO DO MISCONDUCT THAN
BEFORE AND MAY MAKE IT HARDER TO
DETECT.
THE DATA SHARING AND REVIEW
WHICH HAS BEEN HAPPENING FOR A
WHILE HAS ACTUALLY LED TO AN
EXPLOSION OF BOTH PROBLEMATIC
RESULTS AND CASES OF FRAUD IN
WHICH YOU SEE SIGNS OF DATA
MANIPULATION AND THE TOOLS WILL
GET DEVELOPED IN WAYS TO MAKE IT
EASIER TO MANIPULATE DATA BUT
HARDER TO TELL AND OTHERS TOOLS
THAT WILL BE BETTER AT DETECTING
THOSE THINGS AND THERE'S GOING
TO BE A BACK AND FORTH OF THIS.
I THINK THIS IS SOMETHING I'M
VERY WORRY ABOUT WHICH IS THE
POTENTIAL IT MAY MAKE IT EASIER
AND EASIER FOR MISCONDUCT.
LASTLY DUAL USE RESEARCH OF
CONCERN IS A RELATIVELY NEW
ISSUE.
WE HAVE BEEN WORKING ON THIS IN
A.I. AND WILL PUBLISH OUR
RESULTS EARLY 2024.
AND EVERYONE WAS TAKEN OFF GUARD
WHEN FOLKS USING A.I. TO
IDENTIFY PROMISING TARGETS FOR
PHARMACEUTICAL DEVELOPMENT.
REALIZED THEY WERE ABLE TO
CREATE EXISTING BIOETHICS AND
NEW ONES DE NOVO DESIGN OF
BIOCHEMICAL WEAPONS THAT WOULD
BE WORSE THAN EXISTING THINGS.
AND ANOTHER ARTICLE THAT CAME
OUT SHORTLY LATER THAT CAME OUT
OUR ETHICS REVIEW SOCIAL AT
STANFORD I DESCRIBED EARLIER WE
IDENTIFIED THIS AS A PROBLEM
WITH SOME GROUPS LOOKING AT AN
A.I. MODEL TO IDENTIFY TOXIC
SUBSTANCES THAT SHOULD BE LOOKED
AT THROUGH THE EPA BUT WHEN THEY
TRIED TO DEAL WITH IT THEY FOUND
ALSO GUIDANCE FROM THE FEDERAL
OFFICIAL OR CORPORATE WORLD AND
LOTS OF GUIDELINES ON A.I. BUT
NOT THE DUAL-USE PROBLEMS.
THAT'S WHAT WE ARE TRYING TO
ADDRESS NOW AND SO STAY TUNED.
WE'LL HAVE IDEAS HOW TO ADDRESS
THOSE ISSUES.
AND I SHOULD SAY THERE'S A WIDE
RANGE OF POSSIBLE ISSUES HERE.
SURVEILLANCE AND AMBIENT
TECHNOLOGIES CAN BE USED BY
AUTHORITARIAN GOVERNMENTS TO
IDENTIFY PROTESTERS AND CREATE
MISINFORMATION AND PROPAGANDA IS
A BIG ONE WITH LLMs.
WE TALKED ABOUT THE UNINTENDED
BIAS AND YOU CAN USE LARGE
LANGUAGE MODELS TO INTENTIONALLY
DISCRIMINATE AND CREATE
MISINFORMATION.
YOU COULD POTENTIALLY REIDENTIFY
INDIVIDUALS FROM SUPPOSEDLY
REIDENTIFIED DATA AND THE BIO
WEAPONS CONCERN.
COULD CHATBOT TEACH YOU HOW TO
BUILD A DIRTY BOMB, AN ARTICLE
WRITTEN IN 2023 AND WHILE CHAT
GPT REJECTS HARMFUL PROMPTS THEY
WERE ABLE TO FIGURE OUT HOW TO
WORK AROUND THAT BY PROVIDING A
POSITIVE CONTEXT AND TRYING TO
DEVELOP MITIGATION STRATEGIES
AND ONCE THEY DID THAT AND
CONVEY AND PUT THAT IN THEIR
PROMPTS, THEY WERE ABLE TO
BYPASS THE PROTECTS PUT IN PLACE
AND FIND DETAILS ABOUT BUILDING
BOMBS INCLUDING RATIOS OF DIESEL
FUEL AND FERTILIZER FOR
DIFFERENT EXPLOSIVE EFFECTS.
OPEN A.I. SNUFFS THESE OUT BUT
THERE'S BEEN RESEARCHERS FIGURE
OUT NEW WAYS OF JAILBREAKING
THESE THINGS TO GET AT HARMFUL
INFORMATION.
IT ALSO RAISES A CHALLENGE WHERE
THE LINE SHOULD BE DRAWN.
IF SOMEBODY ENTERS CHAT GPT AND
HOW TO ACCESS ABORTION AND CAN
PROSECUTORS FIND OUT ABOUT
QUERIES MADE AND THESE ARE THE
PRIVACY ISSUES THAT I THINK ARE
NEW AND GOING TO BE PROBLEMATIC
AND THE KEY WILL BE REDUCING
RISK TO LEVEL AND THERE'S ACTIVE
WORK.
THE BIG TAKE HOME IS MANY AND
PROBABLY MOST THE DIFFERENT
ISSUES BEING RAISED BY LARGE
LANGUAGE MODELS IN RESEARCH ARE
ISSUES THAT ARE NOT UNFAMILIAR
AND DEALING WITH RESEARCH
GENERALLY OR A.I. RESEARCH AND
WHILE WE HAVE A LONG WAY TO GO
TO SOLVE THE PROBLEMS WE AT
LEAST HAVE GONE A LONG WAY IN
FIGURING OUT WHAT NEEDS TO BE
DONE AND MOST THE PROBLEMS
AROUND IMPLEMENTATION.
THERE ARE A FEW EXCEPTIONS LIKE
AUTHORSHIP AND CREDIT AND OTHER
CHALLENGING THINGS IN DUAL USE
NOT ADEQUATELY ADDRESSED YET BUT
WE'RE WORKING ACTIVELY TO
ADDRESS THOSE ISSUES AND
HOPEFULLY WE'LL HAVE FRAMEWORKS
FOR DOING THOSE AS WELL.
THANK YOU VERY MUCH.
>> THANKS, DAVID.
SO I WAS GOING TO GET US STARTED
BY ASKING YOU A QUESTION ABOUT
THE AUTHORSHIP ISSUE AND HOW
YOU'RE THINKING ABOUT THAT
BECAUSE TO ME IT SEEMS LIKE
THERE'S A STRONG ANALOGY BETWEEN
CITING A STATUS PACK AND AND THE
WAY I UNDERSTAND A LARGE
LANGUAGE MODEL I WAS IMAGINING I
WORK IN RISK BENEFIT ASSESSMENT
IN PEED RICK RESEARCH.
I GO IN AND GET A LARGE LANGUAGE
MODEL AND ASK WHAT ARE
INTERESTING ISSUES TO WRITE
PAPERS ON IN THIS AREA.
IT COMES UP WITH ONE AND ASK IT
TO DO A LITERATURE SEARCH AND IT
DOES A LITERATURE SEARCH AND ASK
IT TO WRITE A DRAFT AND IT DOES
AND I SUBMIT IT AND IT SAYS DAVE
WENDLER'S THE AUTHOR AND I USED
CHAT GPT AND IT SEEMS DECEPTIVE.
I DON'T KNOW IF YOU THINK THAT'S
RIGHT FOR A DIFFERENT EXAMPLE.
>> FIRST I THINK IT'S NOT GOING
TO BE AS SIMPLE AS THAT.
IN A LARGE BIOETHICS CLASS WE
ALLOWED THEM TO USE CHAT GPT AND
ENCOURAGED IT AND ASKED THEM TO
DESCRIBE HOW THEY USED IT AND
SAID IT WOULD BE FINE AND
COLLECTED DATA ON HOW STUDENTS
DID WHEN THEY USED IT AND THE
ETHICAL ANALYSIS WAS BETTER WHEN
THEY WERE NOT USING CHAT GPT
THAN WHEN THEY DID AND AN
ARTICLE COMING OUT THIS MONTH
ANOTHER ARTICLE GIVES AN
ILLUSTRATION WHY THAT IS WHERE
THEY'RE DEMONSTRATE GOING STUFF
OUT OF IT TO ADDRESS ETHICAL
ANALYSIS.
IT'S NOT AN EASY PROBLEM.
I DON'T THINK IT'S LIKE
AUTOMATIC YOU CAN JUST SAY
ANYTHING.
PROMPT ENGINEERING IS GOING TO
BE A NEW PART OF HOW TO WRITE
WELL AND THAT NOT A TRIVIAL
PROCESS.
IF YOU ENTER THE THINGS IN THE
WRONG WAY, TUNING IS REAL WORK.
PROMPT ENGINEERING IS REAL WORK.
I THINK IT'S MORE LIKE USING
SOME OF THE OTHER TOOLS.
BACK IN THE DAY MAYBE YOU
CALCULATED YOUR STATISTICS BY
HAND.
NOW YOU HAVE A STATS PROGRAM
THAT DOES IT FOR YOU AND WE HAVE
GOTTEN USED IT THAT AND THAT'S
FINE.
LEARNING HOW TO DO PROMPT
ENGINEERING AND TUNING THESE AS
PART OF THE PRODUCTION OF
KNOWLEDGE BUT YOU STILL HAVE TO
GO THROUGH IT METICULOUSLY FOR
THE LIMITATIONS AND AS LONG AS
IT'S ONLY SYNTACTIC YOU HAVE TO
DO WORK INTO IT AND SO I THINK
IT'S MUCH MORE LIKE THOSE OTHER
PLACES.
WHEN PEOPLE STARTED WRITING ON
TYPEWRITERS OR USING COMPUTERS
TO TYPE RATHER THAN WRITE OUT BY
HAND AND THERE WAS DECRY THIS IS
NOT AUTHENTIC.
THIS IS NOT REAL.
I DON'T HAVE TO KNOW HOW TO
SPELL ANY MORE BECAUSE I HAVE
SPELL CHECK AND WE SAID THIS
USED TO BE A PART OF HOW WE
EVALUATED QUALITY AND IT ISN'T
ANYMORE BECAUSE WE CAN HAVE
OTHER TOOLS TO HELP US DO THAT
AND RAISE THE FLOOR AND WE
ADJUST AND THE CEILING KEEPS
GOING UP.
THAT'S HOW I THINK ABOUT THIS.
EVERY TIME THERE'S A NEW
TECHNOLOGY PEOPLE DE CRY IT BUT
I THINK THE IDEA THIS IS GOING
TO BE A TRIVIAL ISSUE AND YOU
CAN TRIVIALLY HAVE ALMOST
EVERYTHING DONE FOR YOU I THINK
IS NOT REALITY ANY MORE THAN
STATS PROGRAMS MEANS YOU YOU'RE
NOT DOING REAL WORK WHEN YOU DO
EMPIRICAL PAPER.
>> DO YOU THINK YOU AND I WILL
HAVE JOBS THE NEXT 10 YEARS OR
SO?
>> YEAH.
ONCE YOU START GOING BEYOND 10
YEARS FROM NOW.
>> WE MAY BE IN TROUBLE.
ALL RIGHT.
>> I'M RON SOMERS I RUN AN A.I.
LAB IN THE RADIOLOGY DEPARTMENT.
IN THE CLINICAL CENTER WE'VE
LOOKED AT DISPARITY, BIAS AND
ALSO USED LARGE LANGUAGE MODELS.
MY QUESTION HAS TO DO WITH THE
DEFINITION OF DISPARITY AND BIAS
PARTICULARLY IN DIAGNOSTIC
IMAGING.
AS SOMEONE WHO REVIEWS
MANUSCRIPTS FOR JOURNALS I RUN
INTO THIS A LOT, PEOPLE WILL DO
A RESEARCH STUDY WHERE THEY'RE
USING RADIO GRAPHIC IMAGES OF
ONE TYPE OR ANOTHER AND THE
OUTPUTS OF THE A.I. MODEL DIFFER
FOR PEOPLE FROM DIFFERENT GROUPS
BE THEY RACIAL, SEX OR ETHNICITY
THEY'RE DEFINED BY THE AUTHORS
AS BIAS.
I'D LIKE TO HEAR YOUR THOUGHTS
WHETHER ANY DIFFERENCE IN ANY
A.I. GENERATED FEATURE, ANY
DIFFERENCE THAT OCCURS BETWEEN
PEOPLE IN DIFFERENT GROUPS
CONSTITUTES BIAS OR WHETHER IT'S
PERMISSIBLE TO HAVE DIFFERENT
PERFORMANCE FOR PEOPLE IN
DIFFERENT GROUPS.
>> YEAH, I ALLUDED TO IT BUT I
THINK BIAS EXISTS WHEN IT REALLY
IS AN INCORRECT UNDERSTANDING OF
THE OUTPUT.
FOR EXAMPLE, IT'S A BIAS WHEN
PEOPLE ARE MAKING LISTENING
DECISIONS FOR TRANSPLANT THAT
THEY OFTEN PHYSICIANS ARE
BIASSED TO NOT LIST PATIENTS WHO
HAVE DEVELOPMENTAL DELAY DESPITE
OUTCOMES ARE JUST AS GOOD IN
THAT POPULATION AND IF A.I.
LEARNS FROM THAT THEN THE
OUTPUTS WILL HAVE BIAS.
YOU CAN HAVE BIAS AS THE
FUNCTION OF THE DIFFERENT KINDS
OF DATA.
IN THE OBERMEYER THE PEOPLE WHO
DEVELOP THE ALGORITHMS MAY HAVE
CAUGHT IT WOULD DISCRIMINATE
AGAINST AFRICAN AMERICANS IF
THEY HAD A MORE ROBUST DATA SET
AND THEY HAD 1,000 BUT ONLY 8
WERE BLACK SO THEY MISSED THAT
FEATURE THAT CAME OUT THAT WAS
HIGHLY PROBLEMATIC.
AS I SUGGESTED THERE'LL BE
DIFFERENCES I DON'T THINK WE CAN
CHARACTERIZE AS BIAS AND GENUINE
DIFFERENCES OR THINGS THAT ARE A
FUNCTION OF UNETHICAL
PROBLEMATIC THINGS IN SOCIETY
STILL NOT NOTICED AND IF YOU
HAVE DIFFERENCES IN OUTCOMES AS
A BIAS OF SOCIAL DETERMINATES OF
HEALTH THAT'S A REAL DIFFERENCE.
THOSE THE HARDER ONES.
IF YOU HAVE PATIENTS PRESENTING
FOR PALLIATIVE RADIATION AND
DON'T WANT IT GIVE RADIATION AND
SEND THEM ON HOSPICE, VERSUS THE
ONES WHO MIGHT LIVE FOR A WHILE
VERSUS THE ONES WHO MIGHT LIVE
FOR A COUPLE YEARS WITH
AGGRESSIVE RADIATION.
PHYSICIANS ARE TERRIBLE AT DOING
THAT AND SARAH ALCORN AT HOPKINS
HAS AN ALGORITHM THAT DOES A
BETTER JOB AND AS WE KNOW ABOUT
DISPARITIES IN SOCIAL DISTANCES
AND PEOPLE POORER OR OF COLOR
WILL DIE IN THOSE BINS AND IT'S
A FUNCTION OF PROBLEMS IN OUR
SOCIETY AND DISCRIMINATION BUT
THEY'RE NOT BIASES THOSE ARE
TRUE OUTCOMES AND SOMETIMES
THOSE DIFFERENCES ARE THINGS YOU
WANT TO BE AWARE OF BECAUSE YOU
CAN PUT YOUR THUMB ON THE SCALE
TO MAKE THEM BETTER BUT
SOMETIMES YOU CAN'T.
YOU'RE NOT DOING ANYBODY A FAVOR
BY GIVING THEM LARGE DOSES OF
RADIATION WHEN THEY HAVE LESS
THAN SIX MONTHS TO LIVE ANYWAY.
TO SOME EXTENT THEY HOLD UP A
MIRROR TO US.
IF THERE'S OUTCOMES GENERALLY
DIFFERENT THOSE WILL BE
REFLECTED IN THE DATA AND
DOESN'T MEAN THEY'RE
UNPROBLEMATIC.
SOMETIMES THEY ARE AND SOMETIMES
THEY ARE.
-- AREN'T AND SOMETIMES THEY
ARE.
>> I HAVE SEVERAL QUESTIONS FROM
ONLINE AND I'LL EXERCISE MY
PREROGATIVE AND START WITH ONE
OF MY OWN BECAUSE IT FOLLOWS OFF
ON THAT.
UNDERSTANDING AND MEASURING BIAS
DIFFICULT TO DO BUT IMPORTANT TO
DO AND THERE'S BEEN IMPORTANCE
IN SENTENCING ALGORITHMS AND
DISCUSSION OF THAT.
SOMETHING THAT STRUCK ME ABOUT
THAT DISCUSSION IS THE ALGORITHM
MAY BE BIAS AND REFLECT
RE-SIDRE-
RE-SIDISM -- RECIDIVISM AND THE
JUSTICE SYSTEM IMPLEMENTED BY
HUMANS IS FAMOUSLY UNFAIR.
IT SEEMS TO ME THE MOST
IMPORTANT QUESTION IS NOT
WHETHER IT'S BIAS BUT WHETHER
IT'S MORE BIASSED THAN THE
DECISION PROCESS THAT WOULD
OTHERWISE BE USED AND SO I
WONDERED BOTH IF YOU WOULD AGREE
WITH THAT SORT OF CLAIM THAT
IT'S THE CONTRAST THAT MATTERS
NOT THE ABSOLUTE CLAIM AND ALSO
HOW YOU MIGHT APPROACH GOING
ABOUT TRYING TO FIGURE OUT IF
THAT'S TRUE.
>> WE'VE DONE WORK ON LISTENING
DECISIONS FOR CHARACTERISTICS IN
TRANSPLANTS AND WE HAVE LOTS OF
DATA SHOWING ALL KINDS OF
BIASSED DECISIONS THAT TAKE
PLACE ALL THE TIME AND THERE'S
HEALTH SYSTEMS OR HOSPITALS NOT
BIASSED AND OTHERS THAT ARE
BIAS.
ED -- BIASSED AND IF YOU HAVE A
MODEL LEARNING FROM THE
INSTITUTION WHERE'S THEY HAVE
THE BIASSED BEHAVIOR IT WINDS UP
BECOMING REINFORCING AND IT CAN
REINFORCE FROM MULTIPLE DOMAINS.
THE SAME WAY THEY DISCRIMINATE
AGAINST PATIENTS DEVELOPMENTALLY
DELAYED FOR TRANSPLANT LISTING
THEY DISCRIMINATE WHETHER TO DO
SURGERY AND THEY'RE SEPARATE
BIASES BUT LARGE LANGUAGE MODEL
HAVE A POTENTIAL TO START TO
DETERMINE CRITICALLY ILL KIDS DO
POORLY AND THAT'S DESPITE
DECISIONS BY PHYSICIANS BUT IT
WILL BE TURNED INTO WE NOW KNOW
KIDS DON'T SURVIVE IF THEY'RE
DEVELOPMENTALLY DELAYED AND HAVE
ORGAN FAILURE AND THIS CAN BE
COMBINED IN WAYS THAT ARE
PROBLEMATIC AND WAYS WHERE THE
REINFORCEMENT CAN ENTRENCH THESE
TO MAKE IT HARDER TO CHANGE THAN
WE HAVE NOW.
>> IT MAY BE STICKIER IN THESE
CASES.
>> YES.
>> AND THAT MAKES SENSE.
IF I CAN REPRESENT SOME FRIENDS
IN THE VIRTUAL SPACE AND START
WITH NIH SPECIFIC QUESTIONS.
AM I SUPPOSED TO READ PEOPLE'S
NAMES?
JUST THE QUESTIONS.
YES.
OKAY, SO FIRST QUESTION, ISN'T
NIH WIDE NLM BEING BUILT OR DO
CERTAIN MODELS ENCOURAGED OR
PREFERRED FOR USE IF INDIVIDUAL
I.C.s ARE ENCOURAGED TO LICENSE
THEIR OWN MODELS.
>> GOOD QUESTION.
AGAIN, THIS COMES DOWN TO HOW
THESE ARE IMPLEMENTS.
SOMETHING THAT WE ARE BUILDING
IN NHLBI RIGHT NOW IS
ESSENTIALLY A VERSION OF CHAT
GPT WE HOPEFULLY WILL ALLOW OUR
RESEARCHERS TO USE ON A DAY TO
DAY BASIS AND ANYONE IN THE I.C.
BUT I THINK A FEW MONTHS AGO I
HAVE AN UNANSWERED ASK AND
THERE'S AN ENTERPRISE VERSION
AND MICROSOFT SIMILARLY I EXPECT
THEM TO HAVE AN ENTERPRISE
VERSION OF THIS AS WELL.
AND SO I'D BE CURIOUS TO TALK TO
CIT TO SEE IF THEY'LL PROVIDE
SOME SERVICE TO ALL OF US AT THE
NIH TO BE ABLE TO LEVERAGE THIS
TOOL BECAUSE RIGHT NOW WE CAN'T
USE ANY OF THE PUBLIC SERVICES
FOR ANYTHING NOT PUBLIC
KNOWLEDGE.
WE NEED TO ENCOURAGE OUR
RESEARCHER COMMUNITIES TO DO
THIS TYPE OF TOOLING TO ACC
ACCELERATE THEIR RESEARCH AND
SPARK IDEAS AS WELL.
>> ANOTHER QUESTION WE'RE
ANSWERING ARE THERE METHODS TO
USE THE MODEL IN A DECENTRALIZED
MANNER KEEPING THE DATA
LOCALIZED?
>> A DECENTRALIZED MANNER BUT
KEEPING THE DATA LOCALIZED?
YES.
IT COMES DOWN TO WHAT MODEL
YOU'RE USING WHETHER THAT IS AN
OPEN SOURCE OR CLOSED SOURCE
MODEL.
THE OPEN A.I. MODELS THE CHAT
GPT 4 ARE NOT OPEN SOURCE.
THE WAY WE INTERACT IS THROUGH
SOME A.P.I.
WE'RE USING THE CLOSED-SOURCE
MODEL AND WE DON'T HAVE ACCESS
TO THE MODEL WEIGHTS OF CHAT GPT
4 BUT THE DATA WE GET FROM USERS
WOULD BE STORED LOCALLY.
SIMILARLY WE COULD DOWNLOAD THE
OPEN SOURCE LLAMA MODELS AND RUN
THEM LOCALLY AND THAT ENTIRE
SYSTEM THEN WOULD BE COMING DOWN
TO THE DIFFERENCE BETWEEN THE
TECHNOLOGY AND THE TOOLING AND
HOW TO THEY'RE BUILT OUT.
>> WE'RE OUT OF TIME.
SORRY.
I APOLOGIZE FOR QUESTIONS.
I'LL TRY TO RESPOND TO THE ONES
ONLINE A LITTLE BIT LATER AND IN
THE MEANTIME WE'LL HOPEFULLY SEE
YOU THE FIRST WEDNESDAY IN
DECEMBER AND UNTIL THEN PLEASE
THANK NICK AND DAVID FOR A GREAT
SESSION.
THANK YOU, GUYS.