WELCOME TO THE NIH NATIONAL
LANGUAGE SPECIAL INTEREST
GROUP EVENT.
TODAY WE HAVE A SPECIAL ONE.
WE'RE THRILLED TO WELCOME
DR. LARRY HUNTER PROFESSOR OF
PHARMACOLOGY AND COMPUTER
SCIENCE AT BOULDER, COLORADO.
DR. HUNTER RECEIVED HIS Ph.D. IN
COMPUTER SCIENCE FROM YALE
UNIVERSITY IN 1989 AND THEN
JOINED NIH AS A STAFF SCIENTIST.
FIRST AT THE NATIONAL LIBRARY OF
MEDICINE AND THEN AT THE
NATIONAL CANCER INSTITUTE BEFORE
GOING TO COLORADO IN 2000.
DR. HUNTER IS WIDELY RECOGNIZED
AS ONE OF THE FOUNDERS OF
BIOINFORMATICS AND PUBLISHED ONE
OF THE PAPER IN MACHINE LEARNING
PREDICTIONS OF MOLECULAR
FINDINGS.
HE SERVED AS THE FIRST
PRESIDENT OF THE INTERNATIONAL
SOCIETY FOR COMPUTATIONAL
BIOLOGY.
AND HE CREATED SEVERAL OF THE
MOST IMPORTANT CONFERENCES IN
THE FIELD INCLUDING ISMB, PSB
AND THE ROCKY MOUNTAIN
CONFERENCE ON BIOINFORMATICS.
HIS INTEREST SPANS COGNITIVE
SCIENCE TO DRUG DESIGN.
HE HAS PUBLISHED MORE THAN 200
SCIENTIFIC PAPERS, HOLDS TWO
PATENTS AND HAS BEEN ELECTED A
FELLOW OF BOTH THE ISCB AND THE
AMERICAN COLLEGE OF MEDICAL
INFORMATICS.
HIS PRIMARY FOCUS RECENTLY HAS
BEEN THE INTEGRATION OF NATURAL
LANGUAGE PROCESSING, KNOWLEDGE
REPRESENTATION, MACHINE LEARNING
AND ADVANCED VISUALIZATION
TECHNIQUES TO LOOKING AT HIGH
THROUGHPUT MOLECULAR BIOLOGY.
WITH THAT, DR. HUNTER, THE FLOOR
IS YOURS.
>>IT'S A REAL PLEASURE TO BE
BACK AT NIH EVEN VIRTUALLY.
I APPRECIATE YOU COMING TO THIS.
I'M GOING TALK LITTLE MORE
BROADLY BUT THERE'S GOOD CONTENT
IN HERE.
I WANT TO TALK MORE BROADLY
ABOUT KNOWLEDGE-BASED SYSTEMS.
I KNOW THERE'S A SWING BACK AND
FORTH IN THE HISTORY OF
ARTIFICIAL INTELLIGENCE BETWEEN
NEURAL NETWORK OR OTHER PURELY
DATA-DRIVEN AND KNOWLEDGE-BASED
SYSTEMS NOT ALWAYS SYMBOLIC BUT
OFTEN AND I WANT TO TALK ABOUT
WHY KNOWLEDGE-BASED SYSTEMS ARE
IMPORTANT IN THE PERFORMS FROM
NEURAL NETWORKS.
THE IDEA OF A KNOWLEDGE-BASED
SYSTEMS IS WRITING PROGRAMS AS
IF THEY KNEW ABOUT BIOLOGY AND
MEDICINE.
THAT KIND OF KNOWLEDGE AS WE'LL
TALK ABOUT IN A HUMAN BEING MAY
INCLUDE THINGS LIKE ANSWERING
QUESTIONS OR PASSING EXAMINES.
USING EXISTING KNOWLEDGE TO
EVALUATE OR RANK BIOMEDICAL H
HYPOTHESES MORE COMPATIBLE WITH
WHAT WE KNOW OR INTERPRET DATA
IN LIGHT OF EXISTING KNOWLEDGE
AND WHAT DOES IT TELL US ABOUT
THE OPEN QUESTIONS IN
BIOMEDICINE AND HOW DOES IT
INTERACT WHAT WE THOUGHT WE
ALREADY NEW.
THE HISTORY OF TAKING PRIOR
KNOWLEDGE AND USING IT AS PART
OF ANALYZING DATA GOES BACK TO
BAYESIAN INTERPRETATION BUT
THERE'S OTHER FRAMEWORKS AS
WELL.
I'M GOING TO TALK ABOUT
AUTOMATED REASONING THAT ACTS ON
COMPUTATIONAL REPRESENTATIONS OF
KNOWLEDGE.
THOSE ARE PRIMARILY SYMBOLIC BUT
ALSO SUBSYMBOLIC OR
MACHINE-LEARNING BASED
APPROACHES TO REPRESENTATION AND
PRECIPITATION LEARNING IS A HOT
TOPIC AND AN ARTICLE LAYS OUT
THE LANDSCAPE OF KNOWLEDGE-BASED
SYSTEMS.
WHY I THINK THIS IS SO IMPORTANT
RIGHT NOW IS BECAUSE THERE'S AN
AREA OF SCIENTIFIC WORK NOT YET
WELL SUPPORT AUTOMATION.
I CONSIDER THAT DISCUSSION
INFORMATICS.
BIOINFORMATICS THE LAST 30 YEARS
HAS BEEN GETTING INCREASINGLY
GOOD AT PROCESSING RAW DATA THAT
COMES FROM INSTRUMENTS, P AND A
SEQUENCE TO PRODUCE THE THINGS
THAT GO IN THE RESULT SECTION OF
YOUR PAPER.
WHAT ARE STATISTICALLY OVER
REPRESENTED TRANSCRIPT
ABUNDANCES.
WHAT ARE VARIANCES ASSOCIATED
WITH A PARTICULAR PHENOTYPE.
THREES A WIDE RANGE OF BIO
INFORMATICS TOOL THAT MAKE IT
POSSIBLE TO ANALYZE LARGE
AMOUNTS OF DATA AND TURN IT INTO
THE FIGURES THAT GO IN THE
RESULT SECTION.
HOWEVER, THE DISCUSSION SECTION
OF THE PAPER WHERE WE INTERPRET
THOSE RESULTS AND TRY TO
UNDERSTAND WHAT THEY MEAN FOR
HUMAN HEALTH AND UNDERSTANDING
OF THE FUNDAMENTAL MECHANISMS OF
BIOLOGY COMPUTATIONAL TOOLS ARE
NOT REALLY ALL THAT GREAT AT
HELPING SCIENTISTS UNDERSTAND
THE RESULTS AND SO WHAT I'VE
BEEN INTERESTED IN FOR THE LAST
FEW YEARS IS COMPUTATIONAL
SUPPORT FOR WRITING THE
DISCUSSION SECTION OF A PAPER.
THAT INCLUDES A WIDE VARIETY OF
DIFFERENT ACTIVITIES, GENERATING
HYPOTHESES THAT WOULD EXPLAIN
THE RESULTS WE COVER --
OBSERVED.
DRAWING DIAGRAMS THAT CAPTURE A
SET OF RESULTS SOMETIMES WE CALL
THOSE MOLECULAR CARTOONS THAT
ARE COMMON IN LOTS OF PAPERS IN
PATHWAYS AND ALSO DESCRIBING
EVENTS THAT UNFOLD OVER TIME AND
MAYBE EVEN AT THE TISSUE LEVEL
AND I'M GOING TO TALK MOSTLY AT
THE NLP PIECE OF THIS TALK ABOUT
PROVIDING CONTEXT FOR THE
RESULTS AND HOW DO THE RESULTS
WE FOUND IN THIS EXPERIMENT
RELATE TO THE EXISTING
LITERATURE AND ADDRESS OPEN
QUESTIONS OR PERHAPS CREATE NEW
ONES?
SO THE SINGLE MOST WIDELY USED
COMPUTATIONAL APPROACH FOR
HELPING PEOPLE INTERPRET RESULTS
IS CALLED ENRICHMENT AND WHAT WE
DO OFTEN STAKE A LIST OF GENES
OR GENE PRODUCTS OR METABOLITES
OR OTHER FACTORS THAT COME FROM
AN EXPERIMENT THAT SEEM TO BE
RELEVANT SOMEHOW TO THE QUESTION
THAT WAS BEING ASKED AND THEN
IDENTIFY A SET OF THINGS THAT WE
KNOW ABOUT THOSE GENES OR GENE
PRODUCTS OFTEN BASED IN ONTOLOGY
TERMS THAT HAVE BEEN ANNOTATED
TO THEM MANUALLY.
WE CALCULATE A BACKGROUND
DISTRIBUTION ABOUT WHAT WE WOULD
EXPECT TO SEE BY CHANCE SOME
BIOLOGICAL PHENOMENON ARE
COMMON.
INFLAMMATION IS A COMMON THING
ONE OBSERVES IN MANY
EXPERIMENTS, CHANGES IN
METABOLISM LOOK FOR TERMS THAT
APPEAR MORE FREQUENTLY THAN
WOULD BE EXPECTED BY CHANCE BY
WHATEVER THE PROCESS WAS THAT
THIS UNDERLYING LIST.
THERE'S A LOT OF DIFFERENT
ENRICHMENT ENGINES OUT THERE,
NIH RUNS ONE CALLED DAVID WHICH
IS POPULAR.
THIS IS A TYPICAL RESULT FROM
ONE OF THESE ENRICHMENT STUDIES.
YOU SEE A LIST OF TERMS THAT ARE
SIMILAR TO EACH OTHER AND MAY
NOTICE THESE ARE SOMEWHAT VAGUE
AS WELL AS OVERLAPPING.
THEY HELP US UNDERSTAND A LITTLE
BIT OF WHAT'S GOING ON IN OUR
EXPERIMENT RESULTS BUT NOT
HELPFUL IN THE WRITING
DISCUSSION SESSION.
WHAT CAN WE DO MORE THAN
ENRICHMENT?
WE CAN DO OTHER KINDS OF
STATISTICAL NORMALIZATION.
FOR EXAMPLE, LOOK AT TERMS THAT
TEND TO BE ENRICHES IN MANY
EXPERIMENTS RATHER THAN ONE
WE'RE LOOKING AT A DIFFERENT
BASELINE THAN LOOKING AT RANDOM
GENE PRODUCTS.
WE NEED SPECIFIC AND IMPORTANT
HYPOTHESES OF WHAT'S GOING ON IN
THE LARGE SCALE RESULTS.
WHAT REGULATORS NEED TO MAKE
DECISIONS ABOUT THE SAFETY OF
CHEMICAL COMPOUNDS OR THE
EFFECTIVENESS OF CHEMICAL ENTITY
MAY BE DRUGS ARE MECHANISTIC
ACCOUNTS.
WHO ARE THE PLAYERS?
HOW DO THEY INTERACT AND WHAT DO
THAT DO OR SUSTAIN OR CHANGE?
AND THERE'S A METALEVEL AT WHICH
DISCUSSION SECTIONS WORK.
WE DON'T JUST TALK ABOUT WHAT WE
BELIEVE THE FACTS TO BE BUT TALK
ABOUT WHAT THE EVIDENCE FOR THE
CLAIMS ARE.
CITATIONS TO THE LITERATURE
DISCUSSIONS OF OTHER DATA, KINDS
OF INFERENCE ARE ALL IMPORTANT
IN DISCUSSING RESULTS HOW CAN
COMPUTER PROGRAMS HELP US DO
THESE THINGS?
I LAY THIS OUT TO GIVE YOU A
LANDSCAPE OF WHAT OUR GOALS ARE
AND DISCUSSION INFORMATICS.
IN ORDER TO DO THIS WORK WE NEED
COMPUTATIONAL REPRESENTATIONS OF
WHAT IS ALREADY KNOWN.
DISCUSSION INFORMATICS HAS A
BROAD REPRESENTATION OF EXISTING
BIOMEDICAL KNOWLEDGE AND OUR
APPROACH IN MY LAB HAS BEEN TO
USE SYMBOLIC APPROACHES USING
ONTOLOGY AS THE FOUNDATION AND
NOT JUST ONTOLOGIES BECAUSE THEY
THEMSELVES ARE SORT OF AUTOMIC
AND I SHOW A CARTOON GIVING A
SENSE OF WHAT WE'RE LOOKING FOR
WHEN WE'RE TRYING TO REPRESENT
BIOLOGICAL KNOWLEDGE.
WE'RE TRYING TO TAKE CELLULAR
MUTATIONS, WHAT'S IN THE NUCLEUS
AND CYTOPLASM AND MOLECULAR
PLAYERS, PROTEINS,
POST-TRANSLATIONALLY MODIFIED
PROTEINS, HIGHER ORDER
COMBINATIONS OF THINGS,
MOLECULAR PROCESSES LIKE
OXIDATIVE STRESS AND THE ARROWS
INDICATE CAUSAL OR TEMPORAL
RELATIONSHIPS SUCH AS REGULATION
OR ENZYMATIC TRANSFORMATION OF
THE ENTITIES.
WE CAPTURE ALL THAT STUFF IN A
WAY THAT COMMUNICATES WELL TO
SCIENTISTS.
THERE'S BEEN A WIDE VARIETY OF
ATTEMPTS TO DO THIS.
THE COMPUTATIONAL METHODS ARE
OFTEN CALLED KNOWLEDGE GRAPHS.
THIS GOES BACK FOR A DECADE,
MAYBE A LITTLE BIT MORE, THE
EARLIER VERSIONS OF THIS WORK
AND WHAT MICHELLE CALLED A
MASH-UP.
SHE DIDN'T WORRY ABOUT THE
SEMANTIC CONSISTENCY OR THE
TERMS USED OR ONTOLOGICAL UNDER
PINNINGS.
HE IS JUST TRIED TO CAPTURE THE
RELATIONSHIPS BETWEEN DRUGS AND
DISEASES AND SYMPTOMS AND
BIOLOGICAL PROCESSES, SIDE
EFFECTS AND IS THE LIKE.
AND THERE'S BEEN CONSISTENT
IMPROVEMENTS IN THIS.
THE WORK AT UCSF AND THEN AT
PENN WAS IN TRYING TO ASSEMBLE
THESE INTO MORE SEMANTICALLY
COHERENT APPROACH AND THE WORK
IN SAUDI ARABIA LOOKED AT NEU
NEUROSYMBNEU
NEUROSYMBOLIC REPRESENTATIONS
AND OPENS DOORS TO THE LEARNED
REPRESENTATIONS AS INPUTS TO
NEURAL NETWORKS TO TAKE
ADVANTAGE OF THE POWER OF DEEP
LEARNING.
AND ALSO TO DO VICTORY SPACE
CALCULATIONS AND EDGE PREDICTION
IN THE GRAPHS AND THE LIKE.
THERE'S BEEN A GREAT DEAL OF
WORK IN THIS AREA OVER THE LAST
COUPLE OF DECADES.
THE REPRESENTATIONS OF KNOWLEDGE
AND GRAPHS OFTEN WHEN COMBINED
WITH NEURAL NETWORK KINDS OF
LEARNING GIVE US GREAT POWER TO
TRY AND DO THIS OPEN TASK OF
SUPPORTING THE WRITING OF
DISCUSSION SECTIONS AND THE
INTERPRETATION OF OUR MOLECULAR
BIOLOGY RESULTS.
THE GENE ONTOLOGY FOLKS HAVE
ALSO BEEN TRYING TO MOVE AWAY
FROM SIMPLE TRIPLES INTO MORE
COMPLEX GRAPHS.
AND THEY'VE BEEN FOCUSSING ON
CAUSAL ACTIVITY MODELS AND
THERE'S AN URL AT THE BOTTOM.
THE IDEA IS TO TRY TO TAKE A SET
OF MOLECULAR RESULTS AND TIE
THEM TOGETHER INTO
REPRESENTATIONS AROUND
REGULATION, TRANSPORT THE
UNDERLYING MECHANISMS THAT
DESCRIBE THE BIOLOGICAL PROCESS
ARE CAPTURED IN THE
REPRESENTATIONS.
THIS WORK IS SLOW IN THAT ALL
GENE ONTOLOGY IS BECAUSE IT
DEPENDS ON MANUAL ANNOTATION OF
THE LITERATURE BUT THEY'RE
POWERFUL AND MAKE POSSIBLE
CERTAIN KINDS OF INFERENCE AND
THIGHS ARE HIGHLY RELIABLE
DIAGRAMS COMPARED TO THE ONES
ASSEMBLED AUTOMATICALLY SUCH AS
THE OTHERS.
OUR WORK LIVE IN THE MIDDLE.
WE TRY TO BE CAREFUL WITH THE
WAY WE ASSEMBLE THESE.
THESE ARE NOT FULLY AUTOMATIC
BUT AND WE HAVE A SYSTEM THAT
WAS DEVELOPED BY A GRADUATE
STUDENT NAMED TIFFANY CALLAHAN
THAT HAS A MANUSCRIPT UNDER
REVIEW AND THIS LINK THE KIND OF
INFORMATION WE HAVE ABOUT
INDIVIDUAL PATIENTS, WHICH YOU
CAN SEE UP AT THE TOP OF THIS.
SO THINGS LIKE THE DRUGS THAT
WE'RE GIVING THEM AND THE
DISEASES THAT THEY HAVE THE
KINDS OF CHEMICALS AND
CO-FACTORS AND CATALYSTS THAT
PLAY A ROLE IN BIOLOGICAL
ACTIVITIES AND ALSO VACCINES.
WE DID WORK WITH COVID HERE AND
TIE THEM USING THE ELEMENTS IN
GREEN, THE RELATIONSHIPS AND THE
DEEPER KNOWLEDGE OF THE
UNDERLYING MOLECULAR BIOLOGY
THAT BOTTOMS OUT AT THE BOTTOM
IN DATABASES OF KNOWLEDGE ABOUT
MOLECULES SUCH AS THE HUMAN
PROTEIN ATLAS OR THE GENOTYPE
TISSUE EXPRESSION SUBJECT SO WE
KNOW WHAT GENES ARE BEING
EXPRESSED IN WHAT TISSUES AND
ALL THIS TIES TOGETHER ON A
MOLECULAR BASIS FOR THE
DIAGNOSES, LABS AND THERAPIES
PATIENTS GET.
THE IDEA IS TO TRY TO UNDERSTAND
MOLECULAR BASIS OF THESE THOUGH
WE DON'T HAVE MOLECULAR DATA
ABOUT THE PATIENTS.
IN SOME SENSE DIAGNOSES AND
DRUGS AND LAB RESULTS IMPLY
MOLECULAR INFORMATION ABOUT
INDIVIDUAL PATIENTS.
THIS IS HOW WE TIE THEM ALL
TOGETHER.
TIFFANY'S WORK DID AMAZING
STUFF.
I'LL TALK MORE ABOUT IT IN THE
FUTURE BUT YOU CAN SEE WHAT'S
GOING ON AT HER GITHUB ACCOUNT.
THIS IS CONTINUALLY KEPT UP TO
DATE AND HOPE THIS IS A RESOURCE
FOR THE COMMUNITY.
WE CAN DO A LOT OF THINGS WITH
THINGS LINE PHEKNOWLATER AND
THIS IS A QUERY WHERE WE TRY TO
LOOKED AT POTENTIAL DRUG-DRUG
INTERACTION AND TRY TO FIND
SYNERGIES AND FIND POTENTIAL
ADVERSE AFFECTS FROM TWO DRUGS
BEING PRESCRIBED SIMULTANEOUSLY
AND LOOKED AT PATHWAYS THROUGH
PROTEINS AND PROCESSES.
THIS QUERY CAN BE WRITTEN AS A
SPARKLE, A QUERY LANGUAGE FOR
THE WEB ONTOLOGY KNOWLEDGE AND
STANDARDS FOR EXPRESSING
KNOWLEDGE GRAPHS AND SO THERE'S
A GOOD SET OF UNDERLYING
COMPUTATIONAL TOOLS THAT HAVE
BEEN STANDARDIZED AND THERE'S A
LOT OF PUBLIC AND PRIVATE TOOL
FOR BUILDING THESE SORTS OF
KNOWLEDGE GRAPHS AND QUERYING
THEM.
SO SPARKLE IS THE EQUIVALENT OF
A SEQUEL QUERY AND YOU CAN SEE
ON THE FAR LEFT AND RIGHT ARE
DRUGS ON THE GRAPH AND THE
PURPLE ELEMENTS ARE PROTEINS --
NO, THE GRAY ELEMENTS ARE
PROTEIN, THE PURPLE ELEMENTS ARE
BIOLOGICAL PROCESSES.
YOU SEE SETS OF PROCESSES THAT
RELATE THE DRUGS THAT PROVIDE
MECHANISTIC HYPOTHESES ABOUT
THEIR SYNERGIES OR POTENTIAL
TOXICITIES.
THESE HAVE BEEN USED AND IT'S A
BASIC WAY OF USING THE KNOWLEDGE
GRAPH.
IT STILL HAS BEEN USEFUL IN
TRYING TO INTERPRET EXISTING
DATA.
A RICHER WAY TO GO AFTER THIS
WAS DONE BY ANOTHER FAIRLY
RECENT GRADUATE IN THE LAB,
AGNACIA CAPOTI WAS INTERESTED IN
THE MECHANISMS OF TOXICITY.
THIS IS IMPORTANT BECAUSE WE SEE
A LOT OF PROGRESS IN THE LAST
DECADE OR SO OF TRYING TO
PREDICT TOXICITY FROM CELLULAR
ASSAYS, GENE EXPRESSION OF
USUALLY HUMAN CELL LINES WITH
POTENTIAL TOXICANTS AND USING
THE GENE EXPRESSIONS TO TRY
THROUGH MACHINE LEARNING MAKE
PREDICTORS ABOUT TOXICITY.
SO TOXIC OR NOT OR PERHAPS TEXAS
IN PARTICULAR DOMAINS, CARDIO
TOXICITY AND SO FORTH.
THOUGH WE'RE FAIRLY ACCURATE AT
MAKING THE PREDICTS, REGULATORS,
FDA AND OTHERS HAVE BEEN
RESISTANT TO TAKE THEM UP AND
REGULATORS BEFORE THEY MAKE
DECISIONS THAT AFFECT
POTENTIALLY BILLIONS OF DOLLARS
IN REVENUE AND THE HEALTH OF THE
PUBLIC, REALLY DEMAND
MECHANISTIC ACCOUNTS OF
POTENTIAL TOXICITIES AND
THEREFORE ANIMAL EXPERIMENTS
UNTIL RECENTLY.
AND I LIKE TO TEASE HIM HE'S THE
ONLY ARGENTINIAN VEGAN AND
WANTED TO GET AWAY FROM THIS AND
TRY TO AFFIRM MECHANISMS OF
ACTION FROM THE SAME CELLULAR
TRANSCRIPTOME DATA.
WHAT HE DID WAS A MOLECULAR
TOXICOLOGY TEXTBOOK I'M SHOWING
HERE AND CURATED 11 DIFFERENT
MECHANISMS AND USED THE OPEN
BIOMEDICAL ONTOLOGIES AND THEIR
CAUSAL PATHWAYS FOR THEM AND
THEN HE COULD USE THE
TRANSCRIPTOMIC DATA TO INFER
WHICH MECHANISMS WERE OPERATING
IN SOME PARTICULAR EXPERIMENT.
THIS IS A COMPLICATED TASK
BECAUSE THE MECHANISMS DEPEND --
THERE'S NOT NECESSARILY 1 AND
WHICH ONE IS ACTIVE DEPENDS ON
DOSE AND TIME COURSE OF THE
PHENOMENON.
HERE'S A TYPICAL MECHANISM.
THIS IS A CAUSAL SEQUENCE OF
PROCESSES REPRESENTED BY GO
BIOLOGY PROCESSES AND EACH ONE
IN THE PURPLE BOX IS A GO
BIOLOGICAL PROCESSES AND IT'S
THE SIMPLEST ONE, AND THIS
STARTS WITH A SERIES OF STEPS.
A POSITIVE REGULATION OF
MITOCHONDRIAL MEMBRANE THAT GET
MORE POROUS AND BECAUSE OF THAT
IN THE INFLUX OF CALCIUM THERE'S
A RELEASE OF CYTOCHROME C AND
WHEN IT'S FOUND IN THE CYTOPLASM
IT ACTIVATED CASPASS IN THIS
PATHWAY AND THIS IS THE WAY THE
TOXICITY HAPPENS THEN HE
DEVELOPED A SYSTEM YOU CAN LOOK
UP HERE WHICH TAKES THE
KNOWLEDGE GRAPH APOPTOTIC
AND FILLS IN IMPLICIT ENTITIES
AND CREATES THE NEURAL NETWORK
REPRESENTATIONS THE GRAPHICAL
REPRESENTATION LEARNING APPROACH
AND DOES THE SAME THING FOR THE
GENE EXPRESSION SO WE RUN
TREATED VERSUS UNTREATED CELLS
IN TIME COURSES TO LOOK AT
DIFFERENTIALLY EXPRESSED GENES
IN THE ONES THAT HAVE TOXICANTS
VERSUS CONTROLS AND DID A
MECHANISTIC INFERENCE TO FIGURE
OUT WHICH MECHANISMS HE'S
REPRESENTED ALIGN BEST WITH THE
NODE EMBEDDINGS WITH THE DATA
HE'S ANALYZING AND THEN FROM
THOSE MECHANISMS HE CAN GENERATE
BOTH DIAGRAMS AND NARRATIVES CAN
ACCOUNTS OF WHAT THE
HYPOTHETICAL MECHANISM IS AND
CREATED DIMENSIONAL VECTORS.
YOU GET NICE SEPARATION.
THIS IS A DIAGRAM OF VARIOUS
DIFFERENT THINGS SORT OF RELATED
TO EACH OTHER, BREAST CANCER
GENES, PARTICULAR CHANGES THAT
HAPPEN AND BREAST CANCER ITSELF
AND THE LIKE AND YOU CAN SEE
NICE SEPARATION BUT STILL
RELATIONSHIPS BETWEEN THE
PHENOMENON AND SO THE IDEA IS
THOSE CELLS WITH A TOXIN EXPRESS
DIFFERENTIALLY EXPRESSED GENES
AND CALCULATE THE DIFFERENCE
FROM EACH GENE TO MECHANISM STEP
AND THERE'S A SEQUENCE PENALTY.
THE FIRST STEP SHOULD APPEAR
BEFORE THE LAST STEP OF THE
MECHANISM IN THE EXPERIMENTAL
DATA.
THOUGH THE MECHANISTIC STEPS
DON'T NECESSARILY ALIGN WITH THE
FOUR-HOUR DIFFERENCES IN THE
DATA EXTRACTED FROM THE CELL
LINE.
YOU DON'T WANT TO MAKE IT A
STRICT RELATIONSHIP.
IT'S A PENALTY FOR BEING CLOSER
OR FURTHER AWAY FROM THE ORDER
WE OBSERVED.
AND WHAT THAT DOES IS GENERATE A
SCALE OR SCORE FOR EACH
MECHANISM AND ALLOWS US TO RANK.
SO ONCE YOU'VE GOT A MECHANISM
ALIGNED TO A GRAPH YOU CAN SEE
WHICH THE SIMILAR ONES ARE WE
CAN GENERATE THESE NICE OUTPUTS
INTERPRETABLE BY THE MOLECULAR
BIOLOGISTS AND SEE IN THE
CARTOON THE MECHANISM OF THE
TOXICITY THE PURPLE BOX AES ARE
ALIGNED TO CERTAIN GENE PRODUCTS
IN THE GRAY AND CAN CREATE
NARRATIVES THAT DESCRIBE THESE
THINGS.
THE GENE WE SAW IN THE
EXPRESSION INTERACTS WITH SOME
OTHER GENE PRODUCT WHERE WE SAW
THE TRANSCRIPT AND EXPRESSION
AND INTERACTS WITH ANOTHER GENE
PRODUCT AND THOSE PARTICIPATE IN
THE STEP AND MECHANISM.
ANOTHER NICE THING ABOUT THIS WE
CAN GENERATE SUGGESTIONS FOR
EXPERIMENTS AND ANOTHER NICE
THING WE NEED IN THE DISCUSSION
SECTION IS NOT JUST SOME
HYPOTHESIS BUT SUPPORTING DATA
AND EVEN MORE IMPORTANT
POTENTIAL EXPERIMENTS THAT WOULD
CONFIRM IF THIS HYPOTHESIS WERE
TRUE OR NOT TO MAKE SUGGESTIONS
ABOUT GENE KNOCKOUTS YOU'D SEE
AFFECT THE PROCESS OR VARIANTS
IN THE HUMAN GENOME WE MAY
EXPECT WOULD AFFECT THE PROCESS.
OVERALL IGNACIO'S WORK HAS
GOTTEN UPTAKE AND EPA WHICH DOES
REGULATION OF THE INTOXICANTS IN
OUR ENVIRONMENT AS A CONTRACTOR
HIRED HIM TO BRING THIS IN THE
REGULATORY PROCESS IN THE UNITED
STATES.
SO THIS IS I THINK REALLY HIGH
IMPACT WORK.
HERE'S THE PERFORMANCE.
IT'S NOT TREMENDOUS.
SO WHEN WE LOOKED AT ALL THE
ASSAYS WHERE WE KNEW WHAT THE
RESPONSES WERE AND WE ALSO DID
SOMEWHERE WE DIDN'T KNOW WHAT
THE RESPONSES WERE AND THEN
LOOKED TO SEE IF THE PREDICTIONS
WERE CORRECT, IF WE LOOK AT
PRECISION, IT'S IN THE .7
NEIGHBORHOOD IF YOU LOOK AT THE
TOP THREE.
SO THAT'S OKAY.
THERE'S PLENTY OF ROOM TO
IMPROVE THIS BUT AS I SAID, IT'S
A COMPLICATED TASK.
MORE THAN ONE DIFFERENT
MECHANISM MIGHT BE AT PLAY.
IF WE SEE TWO CORRECT ONES IT'S
POSSIBLE TO DO AND IT DEPENDS ON
THE DOSE AND TIME AND THE LIKE.
BUT IT'S A COMPLICATED TASK.
I THINK THIS IS PROMISING WORK.
NOW, BEYOND WHAT IGNACIO DID AND
WHAT TIFFANY'S Ph.D. WORK WAS
CONNECTING BIOMEDICAL RECORDS TO
KNOWLEDGE AND BUILT THIS GRAPH
IN THE FIRST PLACE AND HER
PURPOSE WAS MOSTLY LOOKING AT
RARE DISEASE TRIES TO TAKE
PATIENT DATA STRAIGHT FROM THE
E.H.R. AND TRY TO INFER
MOLECULAR MECHANISMS UNDERLYING
THE PATIENTS AND HER ULTIMATE
GOAL WAS TO SUBPHENOTYPE RARE
DISEASE PATIENTS WITH THE HOPE
OF HAVING DIFFERENT TREATMENT
MODALITIES FOR PATIENTS HAVE
DIFFERENT MOLECULAR BASIS FOR
THEIR DISEASE.
SO MEDICAL RECORDS THEMSELVES
ARE CHALLENGING.
ONE OF THE THINGS SHE DID WAS
TRY TO FIGURE OUT MAP FROM
PATIENT RECORDS INTO OUR
KNOWLEDGE GRAPH.
THE KNOWLEDGE GRAPH IS
REPRESENTED IN OPEN BIOMEDICAL
ONTOLOGIES, THINGS LIKE GO AND
THE HUMAN PHENOTYPE ONTOLOGY AND
THINGS LIKE THAT WHICH ARE NOT
USED TO CHARACTERIZE PATIENTS IN
A CLINICAL SETTING AND DID APE
AMAZING MAPPING FROM A CLINICAL
DATA MODEL THAT IS OPEN AND THAT
MANY DIFFERENT CLINICAL
VOCABULARIES HAVE BEEN MAPPED TO
AND HER PAPER ON O MAP TO OVO
WAS PUBLISHED FAIRLY RECENTLY IN
NATURE DIGITAL MEDICINE IN A
WIDE CONTEXT.
FOR THOSE DOING WORK WITH
ELECTRONIC HEALTH RECORD I
ENCOURAGE TO YOU LOOK AT THE
PAPER I THINK YOU'LL FIND
USEFUL.
IT'S MAPPING MORE THAN 65,000
TERMS, ALMOST ALL OF WHICH HAVE
BEEN REVIEWED BY HUMAN EXPERTS,
PHARMACISTS, DOCS, NURSES, IT'S
AN AMAZING RESOURCE.
SHE THEN COULD MAP FROM THE OBOs
TO THE PHONOE -- PHEKNOWLATER.
WE TAKE EACH VISIT RECORD FROM
AN ELECTRONIC RECORD AND EACH
VISIT HAS A LIST OF DIAGNOSES OF
DRUG TREATMENTS AND OF LABS AND
LAB VALUES ALSO POTENTIALLY
VACCIN
VACCINES THAT WERE MAPPED AND
PUT THOSE IN PHEKNOWLATER AND
CAN COMBINE THE NODES LIT UP BY
A PARTICULAR PATIENT VISIT AND
COME UP WITH A REPRESENTATION IN
THE SPACE FOR THAT PARTICULAR
PATIENT VISIT.
WE CAN ALSO AVERAGE ALL PATIENT
VISITS AND COME UP WITH A
REPRESENTATION FOR THAT VISIT.
THIS HAS ALL KINDS OF VALUE.
HAVING A VECTOR REPRESENTATION
FOR A PATIENT, I'LL SHOW YOU
WHAT WE DO WITH THOSE CLINICALLY
BUT I WANT TO POINT OUT I WAS
SHOCKED BOTH THE HOSPITALS AND
OUR IRB HAVE DECIDE THE PATIENT
REPRESENTATIONS JUST VECTORS OF
NUMBERS ARE NOT PROTECTING
HEALTH INFORMATION.
WE CAN SHARE THEM PUBLICLY.
SO WE MADE THAT AVAILABLE AND
WHILE IT'S NOT A VERY DIRECT --
IT'S HARD TO DISENTANGLE WHAT
THE VECTOR REPRESENTATIONS MEAN
ABOUT THE PATIENTS.
THEY DO GIVE US A WAY TO TRY TO
GET SOME OF THE ADVANTAGES OF
THE BROADER COMPUTER SCIENCE
COMMUNITY TO LOOK AT THESE
TASKS.
AND WE'VE DONE A COUPLE OF
THINGS WITH THESE EMBEDDINGS
THAT ARE DIRECTLY USEFUL
CLINICALLY.
ONE IS WE TRIED TO DO
CLUSTERING.
WE LOOK AT THE PATIENT VECTORS
BOTH BY VISIT AND INTEGRATED
VISITS AND DO CLUSTERING ON TO
THE VECTORS.
IF YOU DO THE SAME THING,
CLUSTERING OVER THE CLINICAL
DATA ALONE AS YOU CAN SEE ON THE
LEFT IS FOR HALF A DOZEN
RELATIVELY RARE DISEASE.
DOWN AT THE BOTTOM THERE'S A
LEGEND BUT SO THIS IS SICKLE
CELL DISEASE AND CYSTIC FIBROSIS
AND THE LIKE.
IF YOU LOOK AT THE LEFT, WHEN WE
DO CLUSTERING ON THE BASIS OF
CLINICAL DATA ALONE THE CLUSTERS
SHOW UP TO RECAPITULATE THE
DIAGNOSIS WHICH IS FINE BUT NOT
INTERESTING AND WE CAN GET
PHENOTYPING AND SEE DIFFERENT
GROUPS OF PATIENTS WITHIN A
PARTICULAR DIAGNOSES.
SOME OVERLAP WITH KNOWN CLINICAL
SUB PHENOTYPES IN SICKLE CELL
PATIENTS THERE'S TWO BROAD
CLASSES THAT SHOW UP IN THIS.
WE WERE INTERESTED IN THESE
BECAUSE THEY TEND TO REFLECT
DIFFERENCES IN THE WAY THEY
TREATED AND IS AN EARLY WAY TO
IDENTIFY SUB PHENOTYPES THAT GET
DIFFERENT SORTS OF TREATMENT.
AND SO THEY'RE USEFUL IN
TREATING THE PATIENTS
CLINICALLY.
NOW I WANT TO TRANSITION A
LITTLE BIT TO NEW WORK.
THESE ARE PAPERS BY GRADUATE
STUDENTS LUCAS STONEWATER AND
BROOK SANTANGELO AND NOURAH SAIL
JEM AND WE WANT TO -- -- SALEM
AND WE WANT TO USE KNOWLEDGE
GRAPHS TO MAKE BETTER MOLECULAR
CARTOONS AND KNOWLEDGE GRAPHS
DON'T DIRECTLY TRANSLATE INTO
THE MOLECULAR CARTOONS.
THERE'S STEPS NOBODY SHOWS IN
THE CARTOONS AND THE CO-FACTORS
AND THE ATP BEING CONSUMED IF
YOU LOOK AT THE KNOWLEDGE BASES
LIKE REACTOME AND K AND IF WE'RE
TRYING TO LINK UP WHAT'S
DISTANT, BITS OF REGULATORY
PATHWAYS AND LOOKING AT DISEASE
LEVEL ACTIVITIES WE'LL ONLY
SAMPLE A SMALL BIT OF THE THINGS
WE COULD FIND HOW DO WE DECIDE
WHAT SHOW.
WE SHOW SEMANTIC ACTIONS.
IN LOOKING BACK AT DRUG-DRUG
INTERACTIONS AND WANTED TO FIND
A DRUG CONNECTED TO A PROTEIN
TARGET MAYBE CONNECTED TO A
PROCESS.
THAT'S A SEMANTIC ACTION.
WE CALL IT THAT BECAUSE THE
CONSTRAINTS ON THE PATH INVOLVE
THE KINDS OF NODE AND
RELATIONSHIPS IN THE PATH ARE
THE SEMANTICS OF THE NODE IN
ACTION AND WHY THEY'RE SEMANTIC
ACTION.
WE BELIEVE BY BEING ABLE TO
DESCRIBE WHAT WE'RE LOOKING FOR
IN BUILDING THE CARTOONS IN THE
SEMANTIC WAY, PERHAPS IN
ADDITION LOOKING AT THINGS IN
THE STRUCTURE OF THE NETWORK AND
LOOK FOR NODES WE CAN FIND THEM
FASTER AND WE WANTED TO
RECAPITULATE THREE EXISTING
CARTOONS FROM EXISTING KNOWLEDGE
BASES.
WE INDEX AND THEN TAKING THE
THINGS THAT WE FOUND IN THE
KNOWLEDGE GRAPH AND CONSTRUCTING
THE SUB GRAPH, IF YOU WILL, IF
WHAT WE WANT AND WE CONSIDER
CREATING THE SUB GRAPH THE NODES
AND EDGES IN THE CARTOON AND
LAYING THEM OUT AND MAKING THE
GLYPHS IS ANOTHER TASK AND
SORTED BYCYTE -- BY CYTO SCAPE
OR BIOREM BUT NEITHER PROVIDE
THE SEMANTIC SUPPORT FOR
QUERYING THE KNOWLEDGE GRAPH AND
THAT'S WHERE WE'RE FOCUSES.
THIS IS AN EXAMPLE AND ONE OF
THE INTERESTING THINGS ABOUT THE
SUBGRAPHS SIT INCLUDES THINGS
NOT IN THE ORIGINAL CARTOON BUT
MAY HAVE BEEN INTERESTING TO
INCLUDE IN THEM SUCH AS DRUGS
THAT AFFECT THE STEPS IN THE
CARTOON.
WE HOPE BY DOING THIS WE MIGHT
BE ABLE TO IMPROVE THE QUALITY
OF THESE CARTOONS OR DIAGRAMS BY
SUGGESTING TO THE AUTHORS OF THE
ADDITIONAL THINGS THAT THE
KNOWLEDGE GRAPH SUGGEST ARE
RELATED BUT WERE NOT IN THE
INITIAL QUERY THAT GENERATED IT.
SO WE HOPE THAT NOT ONLY WILL IT
MAKE IT EASIER AND FASTER TO
GENERATE THE CARTOONS BUT MAY
INCREASE THE QUALITY.
ALL RIGHT, SO NOW I WANT TO
TRANSITION TO SOMETHING IT
THAT'S DIFFERENT THAN NATURAL
LANGUAGE PROCESSING AND INSPIRED
BY HARVEY FIRESTEIN'S BOOK WAS
THE CHAIR OF MOLECULAR BIOLOGY
AT COLUMBIA FOR DECADES, RETIRED
ABOUT FIVE YEARS AGO AND HAD
BEEN RESPONSIBLE FOR TEACHING
MOLECULAR BIOLOGY TO GENERATIONS
OF COLUMBIA GRADUATE AND
UNDERGRADUATE STUDENTS AND HIS
METHOD OF TEACHING WAS TO FOCUS
ON WHAT ARE THE INTERESTING
QUESTIONS SCIENTISTS ARE GOING
AFTER.
RATHER THAN TRYING TO GET
STUDENTS TO MEMORIZE FACTS WHICH
ARE IMPORTANT BUT LESS
MOTIVATING, HE FOUND IT WAS
MOTIVATING TO WORK AN WHAT WE
DON'T KNOW AND WROTE A LOVELY
BOOK CALLED IGNORANCE, WHAT
DRIVES SCIENCE.
WE WERE INSPIRED BY THAT.
IF YOU LOOK AT THE HISTORY OF
NATURAL LANGUAGE PROCESSING
THERE'S BEEN A FAIR AMOUNT OF
WORK IN TRYING TO CHARACTERIZE
STATEMENTS THAT WERE NOT FACTUAL
IN THE LITERATURE USUALLY TO
EXCLUDE THEM FROM INFORMATION
EXTRACTION.
WORK IN HEDGING OR SPECULATION,
IDENTIFICATION, THINGS LIKE
THAT.
WE FLIPPED THAT ON ITS HEAD AND
DECIDED TO FOCUS ON THE
STATEMENTS IN THE LITERATURE
THAT WERE STATEMENTS OF
IGNORANCE.
THAT WAS INTERESTING NOT BECAUSE
THEY'RE STATEMENTS OF IGNORANCE
BUT GOALS FOR SCIENTIFIC
KNOWLEDGE.
WHAT DO WE WANT TO KNOW AND
THINK WE CAN FIGURE OUT AND SO
THAT'S IMPORTANT BECAUSE IT DOES
DRIVE SCIENCE FORWARD.
AND SO A GRADUATE STUDENT WHO
SHOULD FINISH IN JANUARY AND ON
THE JOB MARKET, FEEL FREE TO
REACH OUT TO HER, HER THESIS AND
WE HAVE A PAPER THAT'S PART OF
THIS UNDER REVIEW CALLED
IDENTIFYING AND CLASSIFYING
GOALS FOR SCIENTIFIC KNOWLEDGE.
WE'RE LOOKING IN TEXT TO TRY AND
FIND SENTENCES THAT EXPRESS
BASICALLY OPEN SCIENTIFIC
QUESTIONS.
AND WE'VE DONE THIS BY DOING
CLASSIFIERS OVER SENTENCES, WE
TRY AND CHARACTERIZE THE CONTENT
OF THE QUERY OR STATEMENTS ABOUT
GOALS OF SCIENTIFIC KNOWLEDGE
USING ONTOLOGY TERMS AND USING
CONCEPT RECOGNITION IN THE
SENTENCES.
AND THEN ONCE WE HAVE THIS WE
CREATE A GRAPH OF ALL THIS
IGNORANCE AND WE CAN EXPLORE
THAT GRAPH TO TRY AND IDENTIFY
WHAT ARE THE KINDS OF
INTERESTING QUESTIONS BEING
PURSUED IN A PARTICULAR AREA AND
WE CAN ALSO DO THE SORT OF
ENRICHMENT TASK AND ENRICH FOR
IGNORANCE INSTEAD OF KNOWLEDGE
AND TELLS US WHAT ARE THE OPEN
QUESTION A DATA SET MIGHT APPLY
TO RATHER THAN THE THINGS WE
ALREADY KNOW APPLY TO THE DATA
SET.
THIS IS PARTICULARLY INTERESTING
WHEN THE QUESTIONS THAT WE
IDENTIFY THAT WILL MIGHT APPLY
TO SOME SOME EXPERIMENTAL
RESULTS COME FROM OUTSIDE OF THE
DOMAIN OF THE EXPERIMENT AND MAY
INTRIGUE PEOPLE AND SEEMS IT MAY
BEAR ON THE OTHER OPEN QUESTION.
IGNORANCE STATEMENTS DON'T COME
WITH QUESTION MARKS AT THE END
OF THE SENTENCE.
THEY'RE OFTEN IN THE FORM OF
STATEMENTS ABOUT CONTROVERSIES
OR STATEMENTS ABOUT UNKNOWNS.
AND THEY SHOW UP THROUGHOUT
PUBLISHED JOURNAL ARTICLES.
THERE'S AN AWFUL LOT OF THEM IN
THE INTRODUCTION AS WELL AS THE
DISCUSSION SECTION OF THE PAPER
AND USUALLY THE STATEMENTS OF
IGNORANCE IN THE INTRODUCTION OF
A PAPER ARE THINGS THE AUTHOR IS
GOING TO CLAIM TO RESOLVE OR AT
LEAST PUSH FORWARD WITH THE
RESULTS FOUND.
YOU START OFF WITH A SET OF
STATEMENTS OR IGNORANCE OR
QUESTIONS OR THE LIKE, GOALS FOR
KNOWLEDGE.
THE RESULTS BEAR ON THE GOALS
AND THEN IN THE DISCUSSION WE
TALK ABOUT NEW QUESTIONS THAT
ARISE FROM THIS OR CONTROVERSIES
OR NEW HYPOTHESES.
SO THE STATEMENT OF IGNORANCE
SHOW UP THROUGHOUT DOCUMENTS.
THEY SHOW UP OCCASIONALLY IN
METHOD STATEMENTS AS WELL
SOMETIMES WHICH IS INTERESTING.
AFTER WE CAN DO THE ENRICHMENT
AND SAME IDEA TAKE THE
STATEMENTS OF IGNORANCE THAT
HAVE GO ONTOLOGY TERMS OR OTHER
ONTOLOGY TERMS IN THE SENTENCES,
MAP FROM THE RESULTING
EXPERIMENT TO THOSE ONTOLOGY
TERMS AND THEN LOOK TO FIND
STATEMENTS OF IGNORANCE THAT ARE
OVER REPRESENTED STATISTICALLY
SPEAKING, SAME ANALYSIS, THE
STATEMENT OF KNOWLEDGES ARE OVER
REPRESENTED IN.
SAME TECHNIQUE BUT OVER
REPRESENTED BITS.
THIS IS QUITE INTERESTING.
I CAN'T GO INTO DETAILS IT'S IN
THE PAPER BUT THEY DON'T OVERLAP
AND YOU HOPE STATEMENTS OF
KNOWLEDGE AND IGNORANCE WOULD BE
DIFFERENT.
AS THE A DIFFERENT THING FOR
AUTHORS TO THINK ABOUT AND
THEY'RE DIFFERENT FROM
STATEMENTS OF KNOWLEDGE.
THIS EXAMPLE TAKEN FROM
NUTRITION AND VITAMIN D WORK
FOCUSSED ON METABOLIC PROCESSES
IN PREGNANCY, GOOD THING TO
FIND, BUT THEY'RE QUITE
DIFFERENT FROM THE SORT OF
THINGS THE GO ENRICHMENT
GENERATED.
SO IN SUMMARY AND I'M HOPING TO
LEAVE QUESTIONS, WE'VE TALKED
ABOUT GENERATING MECHANISTIC
EXPLANATIONS OF EXPERIMENTAL
RESULTS, TALKED A LITTLE BIT
ABOUT SEMANTIC CARTOONING AND
ABOUT THE REPRESENTATION OF USE
OF IGNORANCE STATEMENTS USING
NATURAL LANGUAGE PROCESS.
SO THERE'S MORE TO BE DONE.
ONE THING THAT'S INTERESTING IS
USING TRANSFORMERS OR LARGE
LANGUAGE MODELS TO TRY TO
GENERATE DISCUSSION SECTIONS.
THERE'S BEEN A FAIR AMOUNT OF
INTEREST IN THAT THOUGH I HAVE
TO SAY THAT EVEN THE RECENT
RELEASE JUST THIS WEEK OF
META-A.I.s, SCIENCE TRAINED
GALACTICA TRANSFORMER IS
IMPRESSIVE AT THE SCIENTIFIC
TEXT IT CAN GENERATE BUT STILL
NOWHERE NEAR WHAT HUMAN BEINGS
CAN DO.
THE DISCUSSIONS ARE PRETTY
BASIC.
IT MAKES LOTS OF ERRORS SOME OF
WHICH ARE QUITE SUBTLE AND
ELUCIDATES WHAT DOESN'T EXIST
AND WE HAVE MORE TO DISCUSS BIT
IT SEEMS THE WRITING IS ON THE
WALL IS COMING AND FOR THOSE WHO
CARE ABOUT NATURAL LANGUAGE
PROCESSING APPLY TO BIOMEDICAL
JOURNAL STYLE TEXT THE IDEA OF
REAL DISCUSSIONS WITH COMPUTER
PROGRAMS OF INTEREST AS WE
INTERPRET OUR EXPERIMENTAL
RESULTS IS REALLY A VERY
PROMISING NEW FRONTIER IN
BIOMEDICAL NATURAL LANGUAGE
PROCESSING AND HOPE YOU'RE AS
EXCITED ABOUT THE ASPECT AS I
AM.
WITH THAT, I WANT TO THANK THE
WONDERFUL STUDENTS IN THE LAB
WHO ARE OUT AND VARIOUS PLACES
IN THE WORLD INCLUDING MAYLA
LOOKING FOR A POSTDOC OR OTHER
KIND OF JOB AND AT THIS POINT I
WOULD LOVE TO STOP AND TAKE YOUR
QUESTIONS.
>>THREW FOR THE AMAZING TALK.
CAN I ASK YOU TO BRING UP THE
SLIDE TO SEE THE E-MAIL FOR
PEOPLE TO SEND THE QUESTIONS?
THANK YOU SO MUCH.
SO IF I MAY IT WAS SO INTRIGUING
TO REALIZE AND I THINK THIS THE
FIRST TIME I HEARD IT YOUR
GENERATING VECTORS OF PATIENT.
YOU CAN REPRESENT EACH PATIENT
AS A WORD OR PATIENT VECTOR AND
IT'S SO AMAZING AND INTRIGUING.
THERE'S SO MUCH YOU CAN DO WITH
THIS.
IT'S ALMOST ENDLESS WHAT YOU CAN
DO WITH THAT INFORMATION IN
TERMS OF TREATING PATIENTS THAT
HAVE RARE DISEASES OR GROUPING
THEM BY WHATEVER METRIC THAT ONE
CAN THINK.
>>THE CLUSTERING WE DID IS ONLY
THE BEGINNING.
IT WILL BE INTERESTING TO USE
THE PATIENT VECTORS AS INPUTS
CLASSIFIERS TO TRY AND HELP
SUPPORT CLINICAL DECISION MAKING
AROUND TREATING SUCH PATIENTS.
YOU'RE RIGHT.
THE POTENTIAL IS HUGE.
THAT PART OF TIFFANY'S WORK IS
ALREADY PUBLISHED.
YOU CAN FIND THAT IF YOU -- I
DON'T HAVE THE PAPER CITATION
OFF THE TOP OF MY HEAD BUT IF
YOU LOOK AT THE OR THERE'S A
NODE ON PHEKNOWLATER
P-H-E-K-N-O-W-L-A-T-O-R THAT
SHOULD GET YOU TO THE LINK WHICH
HAS A BUNCH OF CONFERENCE
PRESENTATIONS AND SHORT PAPERS,
THE LONGER PAPER IS COMING UP.
I HOPE THE PATIENT VECTOR
EMBEDDINGS WILL WIND UP USEFUL.
>>AND WE DON'T HAVE TO DEAL
WITH PRIVACY ISSUES.
>>THAT'S WHAT OUR IRB IN OUR
HOSPITALS SAY.
NOT EVERYBODY HAS SIGNED ON TO
THAT BUT IT SEEMS IT'S AN EASIER
CASE TO MAKE OF PROTECTED HEALTH
INFORMATION NOT INDIVIDUAL
IDENTIFIABLE FOR SURE.
>>GREAT IDEA, VERY INSPIRING.
>>THANK YOU FOR THE GREAT TALK.
THE QUESTION IS ABOUT EVALUATION
ON THE KNOWLEDGE.
I THINK IT'S PROBABLY YOUR FIRST
SESSION OF THE TALK HOW YOU
GENERATE WITH KNOWLEDGE GRAPHS
AND THE AUDIENCE ASKED THE
RECOMMENDED WAY TO MAKE THE
GRAPHS AND THERE MAY BE
RELATIONSHIP OR CLASSIFICATIONS
AND HAVE MANY EVALUATIONS
GENERATED ON THE HYPOTHESIS.
WHAT IS THE RECOMMENDED WAY FOR
EVALUATION ON THE KNOWLEDGE
GRAPH?
>>THAT'S AN OUTSTANDING
QUESTION.
THAT'S A HARD TASK.
I RECOGNIZE A LOT OF KNOWLEDGE
GRAPHS ARE RECOGNIZED USING THE
LANGUAGE.
THERE'S NO IDEAL WAY TO EVALUATE
KNOWLEDGE GRAPHS.
THERE ARE AND THERE ARE VERY
SIGNIFICANT DIFFERENCES BETWEEN
THEM AND THEIR UTILITY.
AN IMPORTANT AREA OF RESEARCH IS
FIGURING OUT HOW TO EVALUATE
THESE.
SO THERE ARE AUTOMATED METHODS
THAT PEOPLE HAVE COME UP WITH TO
TRY AND COMPARE KNOWLEDGE
GRAPHS.
SO SOMETHING LIKE DOING LINK
PREDICTION AND MASK CERTAIN
PARTS AND TRY TO RECAPITULATE
AND WHAT PROPORTIONATE WHAT
LEVEL CAN WE REPRODUCE FROM THE
KNOWLEDGE GRAPH WE HAVE, CAN WE
REPRODUCE FROM BITS OF IT.
IT WORKS AT SCALE BUT IT'S NOT
THE MOST INTERESTING EVALUATION
GRAPH.
IT'S A LITTLE BIT OF HOW
REDUNDANT THE GRAPH IS RATHER
THAN THE QUALITY OF THE GRAPH.
A MORE MANUAL APPROACH TO SUCH
THINGS ARE USING COMPETENCY
QUESTIONS.
A GRAPH ABOUT MOLECULAR BIOLOGY
SHOULD BE ABLE TO ANSWER
QUESTIONS ABOUT RELATIONSHIPS
BETWEEN THINGS, WHAT'S THE
RELATIONSHIP BETWEEN GLUCOSE AND
GLUCOSE PHOSPHATE EXOKINASE OR
SOMETHING LIKE THAT AND YOU CAN
SET UPSETS OF COMPET AN
QUESTIONS A GRAPH -- COMPETENCY
QUESTION A GRAPH SHOULD HAVE
ANSWERS TO AND THAT'S A GOOD WAY
OF LOOKING AT QUALITY THOUGH YOU
CAN'T DO IT EXHAUSTIVELY SINCE
THE COMPETENCY QUESTIONS ARE
DRIVEN BY UNDERGRADUATE OR
MEDICAL INFORMATION.
HOW CAN YOU USE THE GRAPH TO
ANSWER MLE STAGE 1 QUESTIONS.
THAT WOULD BE A REAL CHALLENGE.
SORRY, THAT'S THE UNITED STATES
MEDICAL LICENSING EXAMINE ARE
THE KINDS OF QUESTIONS WE GIVE
TO DOCTORS AFTER CLASSROOM
TRAINING.
WE CAN USE COMPETENCY QUESTIONS.
WE CAN USE SYSTEMATIC APPROACHES
LIKE LINK PREDICTION, ALGORITHMS
AND THEN WE CAN USE EXTRINSIC
MEASURES.
HOW GOOD ARE THEY AT THINGS LIKE
SUBCLASSIFYING PHENOTYPES AND
RARE DISEASE PATIENTS RATHER
THAN TRYING TO MEASURE THE
QUALITY OF THE KNOWLEDGE GRAPH
ITSELF WE MEASURE THE QUALITY OF
HOW WELL WE CAN USE THE
KNOWLEDGE GRAPH TO DO SOME OTHER
TASK.
AND SO PERFORMANCE AT THE OTHER
TASK SPEAKS TO THE QUALITY OF
THE KNOWLEDGE GRAPH AND THAT'S
INTRINSIC EVALUATION RATHER THAN
INTRINSIC EVALUATION OF THE
GRAPH ITSELF.
KNOWLEDGE OF THOSE ARE PERFECT.
ALL THEM ARE HARD TO DO.
AND I THINK IT'S A REALLY
IMPORTANT OPEN QUESTION IN OUR
FIELD IS BOTH HOW DO WE EVALUATE
KNOWLEDGE GRAPHS AND RELATEDLY,
ONCE WE CAN EVALUATE THEM, HOW
DO WE BUILD THE HIGHEST QUALITY
ONES WE CAN POSSIBLY BUILD.
I THINK I CAN SAY
WITHOUT CLOSING THINGS I
SHOULDN'T DISCLOSE WITH
PHARMACEUTICALS THEY'RE
CONCERNED ABOUT THE QUALITY OF
THE GRAPH AND HOW RELIABLE THE
INFERENCES COMING OUT OF THEM.
IT'S A CRITICAL QUESTION TO THE
WIDESPREAD ADOPTION OF KNOWLEDGE
GRAPHS IN BIOMEDICAL RESEARCH.
>>LOOKING AT DIFFERENT
PERSPECTIVES.
>>I ULTIMATELY IT WILL BE LIKE
NATURAL LANGUAGE WHERE WE LOOK
AT PERFORMANCE ACROSS MANY TASKS
VERSUS ONE THE SAME WAY WE
EVALUATE KNOWLEDGE.
>>THANK YOU.
>>HOW WE IDENTIFY THE CONCEPTS
AND FOR EXAMPLE HOW DO THEY DO
REPRESENTATIONS.
>>WE DON'T IN THIS PARTICULAR
CASE.
THE PATH WITH INTERACTING WITH
RECORDS.
IT'S A NATIONAL SYSTEM FOR
STANDARDIZED OR LAYER OF
STANDARDIZED INTERACTIONS.
O MAP IS A VOCABULARY OF COMMON
DATA ELEMENTS AND ALL THE
VOCABULARY WE USED IN OUR
APPLICATIONS CAME FROM FIELDED
ASPECTS OF THE ELECTRONIC
PATIENT RECORDS.
FROM THE DRUG LIST AND DIAGNOSIS
LIST AND DID NOT DO NATURAL
LANGUAGE PROCESSING OVER
CLINICAL NOTES.
THE WORK I'M MOST HIGH ON IN THE
DOMAIN MAPS DIRECTLY TO OPEN
BIOMEDICAL ONTOLOGY PARTICULARLY
TO THE HUMAN PHENOTYPE ONTOLOGY.
THEY DON'T HAVE THE SAME AMOUNT
OF GRANULARITY BASIC SCIENTISTS
WOULD WANT TO SEE IN THE HUMAN
PHENOTYPE ONTOLOGY IS A BETTER
TARGET FOR NATURAL PROCESSING.
THERE'S DIFFERENT REPORTS AND
THE LIKE.
DOING EXTRACTION IS A BROAD AND
DIVERSE FIELD WHERE THERE'S BEEN
A LOT OF ACTIVITY.
YOU DON'T THINK WE'RE WORKING ON
DIRECTLY I DESCRIBED TODAY AT
LEAST DIRECTLY BEARS ON THAT.
EDEEP LEARNING VAENS ADVANCED T
STATE OF THE ART AND MODELS
TRAINED EXPLICITLY CLINICAL
PERFORM BETTER THAN THOSE
TRAINED ON GENERAL GOOD DATA
EVEN WHEN THEY'RE TUNED USING
CLINICAL OR SCIENTIFIC DATA.
>>I'LL ASK THE LAST QUESTION
FROM OUR AUDIENCE AND IT'S ABOUT
HOW PATIENT DATA IS CARRIED OUT
BEFORE PRODUCING THE PATIENT
VECTOR.
I GUESS THERE'S SOME
INDICATIONS.
>>WE HAVE -- WE BUILT THIS ON A
MINIMAL DATA SET BUT ON AN
IDENTIFIABLE DATA SET.
AND BECAUSE WE NEEDED TO GET
ACCESS TO THE DIAGNOSIS LIST,
THE DRUG LIST AND THE LABS.
THERE IS ELECTRONIC PATIENT
RECORD WAREHOUSE AT THE
UNIVERSITY OF COLORADO CALLED
CAMPUS THAT INTEGRATES THE
INFORMATION.
IT MOSTLY CAME FROM THE GENERAL
HOSPITALS SINCE THAT'S WHERE THE
RARE DISEASE INFORMATION IS AND
HAD TO GO THROUGH HOOP JUMPING
AND THE LIKE TO GET ACCESS IN
THE FIRST PLACE AND THEN AS I
SAID WE USED ONLY FIELDED DATA.
SO IT WAS RELATIVELY
STRAIGHTFORWARD.
THE O MAP DATA ARE
TRANSFORMATIONS AND WE GOT IN
THE BIOMEDICAL ONTOLOGY SPACE
AND DID THE GRAPH REPRESENTATION
LEARNING TO GENERATE THOSE
PATIENT VECTORS.
SO THOSE ARE THE STEPS IN OUR
WORLD.
WE HAD TO DO ALL THE HOOPS TO
GET ACCESS TO PROTECTING HEALTH
INFORMATION AND MAKE SURE WE
RIGHTED IT PROPERLY BUT THE
VECTORS WHICH ARE IRB AND THE
HOSPITALS THAT PROVIDE THE DATA
AGREED WE'RE NOT EHI.
>>OVER TO LANA.
>>I WOULD LIKE TO THANK YOU
VERY MUCH.
>>THANK YOU IT'S NICE TO BE
BACK AT NIH AND ENJOYED BY TIME
IN THE INTRAMURAL SCIENCES.
GOOD-BYE.