>> GOOD MORNING, EVERYONE.

WELCOME BACK TO DAY TWO OF

NATIONAL LANGUAGE PROCESSING

WORKSHOP.

I'M BELINDA SETO.

I'M THE DEPUTY DIRECTOR OF

THE NATIONAL INSTITUTE OF

BIOENGINEERING.

I HAVE FOUND THIS TO BE A

FANTASTIC EXPERIENCE NOT

ONLY FOR ME IN TERMS OF

LEARNING BUT ALSO IN TERMS

OF THE PARTNERSHIP WITH THE

NATIONAL LIBRARY OF MEDICINE,

WHICH HAS ALWAYS BEEN ONE OF

MY FAVORITE INSTITUTES BUT

THIS EXPERIENCE ACTUALLY

REINFORCES HOW GREAT A

PARTNER THEY ARE TO US.

AND SO YESTERDAY WE LEARNED

AT LEAST I FOUND IT

EDUCATIONAL, MAYBE OLD NEWS

TO MANY OF YOU.

I LEARNED A LOT ABOUT THE

STATISTICAL APPROACH, THE

LINGUISTIC APPROACH AND THE

VERY NICELY DRAWN PENDULUM

THAT MAYBE WE COULD END UP

SOMEWHERE IN THE MIDDLE WITH

A HYBRID APPROACH.

AND TODAY WE'RE GOING TO

LOOK -- WE'RE GOING TO HEAR

TALKS ABOUT APPLYING

APPLYING NATURAL LANGUAGE

PROCESSING TO EXTRACT

INFORMATION FROM HEALTH DATA,

INCLUDING ELECTRONIC MEDICAL

RECORD, AND APPLY THAT TO

CLINICAL DECISION SUPPORT.

NOW, I RECOGNIZE THAT IN THE

LATTER AREA, IN CLINICAL

RESEARCH SUPPORT, IT IS

STILL A VERY ACTIVE AREA OF

RESEARCH.

THERE ARE DEBATES WHETHER WE

SHOULD DO THIS BASED ON

EVIDENCE FROM LITERATURE,

BASED ON GUIDELINES, HOW

MIGHT WE USE NATIONAL

LANGUAGE PROCESSING TO HELP

US DECIDE TO TEST RESULTS.

SO I LOOK FORWARD TO VIBRANT,

ACTIVE, HEATED DISCUSSION

EVEN, AND SO AT THIS POINT,

IT IS MY PLEASURE TO

INTRODUCE DR. BLACKFORD

MIDDLETON, WHO WILL GIVE AN

OVERVIEW THIS MORNING.

DR. MIDDLETON IS THE

CORPORATE DIRECTOR OF

CLINICAL INFORMATICS

RESEARCH AND DEVELOPMENT AT

THE PARTNERS HEALTHCARE IN

BOSTON.

HE IS A SYSTEM PROFESSOR AT

MEDICINE AT BRIGHAM AND

WOMEN, ONE OF THE BEST

HOSPITALS THERE.

I HAVE A BIAS.

MY DAUGHTER IS THERE.

HE'S ALSO A LECTURER OF

HEALTH POLICY AND MANAGEMENT

AT THE HARVARD SCHOOL OF

PUBLIC HEALTH.

HE HAS A VERY, VERY

DISTINGUISHED CV AND I AM

REALLY ABBREVIATING THIS.

HE STUDIED BIOCHEMISTRY AND

MOLECULAR BIOLOGY AT THE

UNIVERSITY OF COLORADO,

BOULDER.

HE RECEIVED HIS MASTER'S

DEGREE IN PUBLIC HEALTH FROM

YALE UNIVERSITY OF PUBLIC

HEALTH.

WITH A DWUAL CONCENTRATION

IN EPIDEMIOLOGY, HEALTH

SERVICES ADMINISTRATION.

HE RECEIVED HIS MD FROM

SUNNI IN SKBOUFLE WAS A

RESIDENT IN INTERNAL

MEDICINE AT THE UNIVERSITY

OF CONNECTICUT HEALTH

CENTER.

SO YOU CAN SEE HE IS PREEMPTLY

QUALIFIED TO GIVE THE

OVERVIEW FOR THE MORNING.

DR. MIDDLETON?

[APPLAUSE]

>> THANK YOU VERY MUCH.

IF I CAN JUST GET BACK TO

THE PC.

IS IT SHOWING THE MAP?

THANKS VERY MUCH.

GOOD MORNING, AND THANK YOU

KINDLY FOR THAT INTRODUCTION,

DR. SETO.

IT IS A PLEASURE TO BE HERE

AND TO HAVE WORKED WITH

JAMES LUO AND THE TEAM FROM

AN N BIB TO HELP PUT

TOGETHER THIS DAY, WHICH I

THINK WILL BE AN OUTSTANDING

OVERVIEW OF CLINICAL SUPPORT,

CURRENT PRACTICES AND FUTURE

DIRECTIONS.

AND COMPLEMENTS SO NICELY

WHAT WE HEARD YESTERDAY IN

THE NLP DAY.

IT REALLY IS AN

EXTRAORDINARY OPPORTUNITY, I

THINK, TO BRING THESE

DISCIPLINES EVEN CLOSER

TOGETHER.

CERTAINLY MANY RELATIONS

ALREADY BETWEEN AN LP AND

C.D.S AND WE'LL SEE EXAMPLES

THAT HAVE THROUGHOUT THE DAY

TODAY IN THIS SORT OF

APPLICATIONS TRACK, IF YOU

WILL.

MY JOB IS TO TALK ABOUT C.D.S

FROM THE HIGHEST LEVEL AND

GIVE AN ORIENTATION OR

OVERVIEW TO WHAT IS CLINICAL

DECISION SUPPORT IN CLINICAL

PRACTICE, WHAT IS IT TODAY,

WHAT'S THE EVIDENCE BASE

SUGGEST THE UTILITY OR

IMPACT OF C.D.S AND BOTH

EVIDENCE FOR AND AGAINST

BECAUSE IT'S IMPORTANT TO

REALIZE WE'RE STILL

EVALUATING THE IMPACT OF

C.D.S IN MANY WAYS.

AND MAINLY GIVE SOME

IMPORTANT TO WHAT WILL C.D.S

BE IN THE FUTURE AND

QUESTIONS AS I SEE THEM.

I THINK THE FIRST THING TO

REMEMBER, THOUGH, ABOUT

DECISION SUPPORT IS SIMPLY

ACCESSING THE DATA AND

VISUALIZING THE PATIENTS'

RECORD IS PROBABLY THE MOST

IMPORTANT THING WE DO WITH

ELECTRONIC HEALTH RECORDS OR

PAPER-BASED RECORDS.

THIS, OF COURSE, IS THE

FAMOUS DIAGRAM FROM MINUTE

ARD, WHO APPLAUDED THE

PROGRESS OF NAPOLEON IN THE

ARMY IN 1812 TOWARD MOSCOW

AND THE SUBSEQUENT RETREAT,

OF COURSE, WHEN HE GOT TO

MOSCOW AND FOUND IT BURNED

TO THE GROUND.

IN THIS SINGLE DIAGRAM HE

SHOWED SIX DIFFERENT

DIAGRAMS SIMULTANEOUSLY.

WE CAN SEE LATITUDE AND

LONGITUDE.

WE CAN SEE TEM PRINCIPAL

PROGRESSION AND SIZE OF THE

ARMY BOTH IN ADVANCE AND IN

RETREAT AND THERE IS ONE

MORE I'M FORGETTING.

BUT THE WAYSTATIONS ALONG

THE WAY, GEOGRAPHIC

INFORMATION BILLION AND

TIME.

SO THIS INFORMATION, I

THINK, PUT TOGETHER LIKE

THIS, GIVES OBVIOUSLY A

BEAUTIFUL AND ELEGANT

INSIGHT INTO THE COURSE OF

THIS MARCH AND SUMMARIZES A

LOT OF DATA IN VERY

INTERESTING WAYS.

THIS IS THE FIRST STEP WE

HAVE IN CLINICAL DECISION

SUPPORT.

IS TO SIMPLY GATHER,

AGGREGATE AND DISPLAY

INFORMATION AND IT'S TIMES

TIMES FORGOTTEN AND

EXTREMELY IMPORTANT.

AND I AM GOING TO COME BACK

TO THIS AT THE VERY END.

SO IN THE U.S., HOW DO WE --

WHY DO WE NEED CLINICAL

DECISION SUPPORT IN THE

CURRENT DAY?

AND IN WASHINGTON MAYBE THIS

IS WELL KNOWN BUT I THINK

IT'S IMPORTANT THAT WE STILL

HAVE PATIENT DATA

UNAVAILABLE IN 81% OF CASES,

AS PAUL KANG FOUND HAD HIS

VERY FAMOUS STUDY.

LUSION LEAP FOUND IN THE

HARVARD MEDICAL OUTCOMES

PRACTICE STUDY, 80% OF

MEDICAL ERRORS ARE DUE TO

INADEQUATE MEDICAL

INFORMATION AVAILABILITY.

AND THOSE OVER 65 SEE ON

AVERAGE 6.4 DIFFERENT

CLINICIANS.

IT'S A FRACTURED AND UNWIRED

DELIVERY SYSTEM.

MANY HAVE WRITTEN ABOUT THE

DELAY TRANSLATION OF

KNOWLEDGE TO PRACTICE.

MARK OVERHAJ AND OTHERS

ESTIMATED WHEN YOU NET IT

ALL OUT, IT MIGHT TAKE 17

YEARS BEFORE A NEW

INNOVATION IS ACTUALLY

BROUGHT INTO CLINICAL

PRACTICE ROUTINELY.

AND WE KNOW FROM A VARIETY

OF CAREFUL STUDIES IN BOTH

URBAN AND RURAL AND ACADEMIC

AND COMMUNITY-BASED

PRACTICEITIONERS THAT

INFORMATION NEEDS ARE OFTEN

NOT BEING MET AT THE BEDSIDE

OR IN THE CLINICAL AM BLUETRY

CARE SETTING.

BILL STILLS AND OTHERS HAVE

WRITTEN ABOUT THE COGNITIVE

DEMANDS PROGRESSION THE

INFORMATION SUPPOSING THAT

WE'RE NOW AFFECTED WITH.

THIS CHART SHOWS THE WELL

KNOWN SAW THAT WE CAN HANDLE

ITEMS OR SEVEN PERHAPS ON

AVERAGE ON A GOOD DAY.

THE HUMAN COGNITIVE CAPACITY,

MAPPED AGAINST ALL THE OTHER

INFORMATION BITS THAT ARE

COMING AT US.

DECISION ON THE CLINICAL

PHENOTYPE, THAT IS THE

PATIENT'S CHARACTERISTICS,

PHYSICALLY AND WHAT NOT.

STRUCTURAL GENETICS, THE

ZMIPS APTYPES THAT MIGHT BE

RELEVANT, THE GENE

EXPRESSION PROFILES AND ANY

OTHER OMIC THAT'S MIGHT

APPLY TO THIS DECISION WE'RE

NEEDING TO MAKE THAT THE

POINT IN TIME BEFORE THIS

PATIENT.

REGRETTABLELY OUR DECISIONS,

THOUGH, DON'T ALWAYS RESOLVE.

IN FACT, ONLY ABOUT HALF THE

TIME RESULT IN APPLYING THE

BEST EVIDENCE TO PRACTICE.

ELIZABETH MCGLIN DID THE

FAMOUS STUDY LOOKING ACROSS

THE COUNTRY AT PRACTICE

PATTERNS, SURVEYING

PHYSICIANS AND EXAMINING THE

MEDICAL RECORDS OF THEIR

PATIENTS TO FIND THESE KINDS

OF DELIVERY PATTERNS, WHERE

COMPLIANCE WITH IDEALIZED OR

BEST PRACTICES EVIDENCE-BASED

GUIDELINES AND THE LIKE WENT

FROM 64.22 TO 14.21 PERCENT

REGRETTABLELY.

AND IN THE TAKE-AWAY, THE

HEADLINE WAS FOR THIS

ARTICLE "ON AVERAGE PATIENTS

RECEIVE 54.9% OF RECOMMENDED

CARE."

SO THE CALLED COIN TOSS

PROBLEM.

SO I THINK WE STILL HAVE THE

FLEX NER REPORT TO WREFLES

WITH 100 YEARS AFTER ABRAHAM

WROTE IT.

SOCIETY REAPS AT THIS TIME

AT THIS MOMENT BUT A MALL

SMALL FRACTION HAS THE POWER

TO CONFER.

WE HAVE SUCH A GREAT

KNOWLEDGE BASE.

WE HAVE EXCELLENT

PRACTICEITIONERS WHO HAVE

DOING THEIR BEST TO DEAL

WITH THE EXPLODING KNOWLEDGE

BASE AND DATEA SET BUT WE DO

NEED TO DO MORE TO BRING

THEM TOGETHERF WE DID BRING

THEM TOGETHER, A VARIETY OF

STUDIES WE DID LOOKED AT

USING THE EVIDENCE BASE

SYNTHESIZING BEST PRACTICES

AND EXPERT OPINION TO

ESTIMATE HOW MUCH HIGHLY

INFORMED AND INNER OPRABLE

CLINICAL MEDICINE WOULD BE

WORTH.

WHAT'S THE VALUE

APPROPRIATION, IF YOU WILL?

A VARIETY OF STUDIES HERE

LOOKING FOR EVIDENCE FOR AM

BLUETRY ORDER ENTRY, HEALTH

INFORMATION EXCHANGE,

CHRONIC DIABETES MANAGEMENT,

HEALTHCARE PHYSICIAN AND

PHYSICIAN AND PERSONAL

HEALTH RECORDS.

WE FIND ACROSS ALL THESE

STUDIES ABOUT EIGHT YEARS

WORTH OF WORK HOW A TEAM OF

TEN THAT ABOUT 150 BILLION

COULD BE SAVED IF ALL THIS

TECHNOLOGY WERE USED

ROUTINELY.

THIS MIGHT BE AN IDEAALIZED

ESTIMATE IN SOME EYES, BUT

WE THINK IT'S FAIRLY

CONSERVATIVE AND OTHERS HAVE

SUGGESTED IT MIGHT BE MORE

LIKE 30 PERCENT, INCLUDING

ALL OF THE ADMINISTRATIVE

WASTE AS WELL.

SO WHERE ARE WE?

[LAUGHTER]

WE ARE IN THE MIDST OF THIS

ERA WITH OUR FEARLESS

LEADER.

NO MATTER WHAT YOU THINK

ABOUT THE POLITICS OF THE

ISSUE, HEALTHCARE IS IN THE

MIDST OF REFORM, WHICH IS

JUST DESPERATELY NEEDED TO

SORT OF CORRECT THE COURSE

TO BEND THE CURVE, IF YOU

WILL, SO THAT WE DON'T HAVE

THIS INEXTRAABLE RISE IN

HEALTHCARE COSTS THAT SIMPLY

IS NOT SUSTAINABLE.

IT'S TOO BIG TO FAIL.

I DON'T KNOW HOW WE NEED TO

GET EVERYONE BEHIND THIS

IDEA, BUT HEALTHCARE REFORM

NEEDS TO BE ZREFD THAT WE

CAN BEND THE CURVE OF COST

AND TRY TO DO WHAT'S RIGHT

FOR PATIENTS AND MORE OF

WHAT'S RIGHT AND LESS OF

WHAT'S WRONG AND GET BETTER

OUTCOMES AT THE SAME TIME.

THE TRIPLE ITEM.

THE HEALTHCARE REFORM EFFORT,

AT LEAST THE O AND C

STIMULUS PACKAGE FROM

HIGH-TECH BILL, AIMS TO BOTH

STIMULATE THE ADOPTION OF

HEALTHCARE IT, STIMULATE

INOPRABLET OF THAT IT AND TO

THEN INFORM SUPPORT AND

ACHIEVE THIS TRANSFORMATION

OF HEALTHCARE.

A FOUR-YEAR TIMEFRAME IS

PRETTY QUICK.

MANY HAVE CRITICIZED THIS

MIGHT TAKE A GENERATION OR

TWO AND THE IMPLEMENTATION I

WOULD SUGGEST OF HEALTHCARE

IT IS OCCURRING WITHOUT THE

ESSENTIAL INGREDIENT OF THE

KNOWLEDGE BASE.

KNOWLEDGE IS NOT RESIDENT IN

ELECTRONIC MEDICAL RECORDS

WHEN IMPLEMENTED.

IT HAS TO BE ADDED AS A

COMPONENT EITHER FROM THE

CUSTOMER, THE IMPLEMENTER OR

FROM KNOWLEDGE VENDORS AND

THE LIKE.

SO I THINK ACTUALLY WE ARE

IN THE PERFECT STORM FOR

C.D.S.

MANY SAW THIS MOVE GIVE THE

FISHERMEN AND LOBSTERMEN OFF

THE COAST OF BOSTON, MASS,

FAMOUS STORY ABOUT THE

PERFECT STORM.

I FEEL SOMETIMES LIKE A

MEMBER OF THE CREW ON THAT

BOAT IN HEALTHCARE.

BUT WE HAVE LOTS OF CLINICAL

DATA GOING ONLINE, INCREASED

INTEROPRABLET GRADUALLY

COMING.

A TSUNAMI OF PERSONAL AND

SOCIAL DATA WILL BE ARREST

TO OUR CLINICAL DECISION AIR

AND MENTIONED ALREADY THIS

RISE IN THE NEED FOR

HEALTHCARE REFORM.

SO WHEN WE THINK ABOUT

DECISION SUPPORT, LET'S

THINK FIRST ABOUT HOW

PHYSICIANS REASON.

THERE IS, MANY DIFFERENT

WAYS TO LOOK AT THE

HYPOTHETICAL DEDUCTIVE

PROCESS THAT PHYSICIANS

EMPLOY TO ANALYZE A PATIENT,

ASSESS THE HISTORY AND

PHYSICAL ASSESS LABORATORY

ZPINGS THEN MAKE A

DIFFERENTTIAL DIAGNOSIS OR A

LIST OF PROBABLE CAUSES OF

DISEASE OR LESS THAN WELLNESS.

DAVID AMONG OTHERS, HAVE

WRITTEN ABOUT THIS.

PHYSICIANS ARE EXTRAORDINARY

PATTERN RECOGNIZERS AND

LISTENING HIGH POTTING SIES

TO VALIDATE OR DISPUTE THESE

HIGH POTTING SKPIS THEN

TAKING ACTION.

WE KNOW THIS DECISION MAKING

PROCESS IS SUBJECT TO A

NUMBER OF IMPORTANT BIASS.

FIRST HIGH POTTING SIES ARE

GENERATED VERY EARLY,

SOMETIMES IN SECONDS UPON

ENTERING A ROOM OR VISITING

A PATIENT'S BEDSIDE.

AND JUST A FEW ACTIVE HIGH

POTTING SIES ARE CONSIDERED

AT ANY ONE TIME.

BUT THESE BIASS ARE WELL

KNOWN AND I POINT YOU

TOWARDS DANIELLE CONMAN'S

BOOK THINKING FAST AND SLOW,

WHICH IS AN ACCESSIBLE

TREATY TREATISE ON HIS WORK

THAT ELUCIDATED SOME OF

THESE FINDINGS.

I WON'T GO THOSE IN GREAT

DENIAL BUT THE

REPRESENTATIVE WE CAN BE

BIASED BY OUR MISTAKEN

ASSUMPTIONS AROUND PRIOR

PROBABILITIES AND

INAPPROPRIATELY WAITING

DEPENDENT FACTORS OR

INDEPENDENT FACTORS, BOTH,

THE AVAILABILITY HERE, WHAT

DID WE SEE LATELY THAT KINDS

OF LOOKS LIKE THIS CASE AND

ANCHORING A-AND-ADJUSTING.

PHYSICIANS ARE WELL KNOWN TO

ANCHOR THEIR PERCEPTIONS AND

THEN HAVE DIFFICULTY ADJUSTING.

BUT BOTTOM LINE, FROM A

DECISION POINT OF VIEW, WHAT

WE AIM TO DO IS TO MAKE

DECISIONS THAT DISTINGUISH

THESE THREE ZONES.

THE ZONES OF DON'T TREAT,

DON'T INTERVENE, PERHAPS OR

CAST OR INTERVENE AND

FINALLY THE LAST TREAT,

WHERE THE PRIOR PROBABILITY

TEST PERFORMANCE

CHARACTERISTICS AND CLINICAL

INTERVENTION EFFICACY,

WHETHER IT'S DRUG OR

PROCEDURE, ARE ALL WEIGHED

IN THE BALANCE OF A NICE

DECISION TREE THAT CAN

DISTINGUISH THESE THREE

STONES AND HELP US DETERMINE

WHICH ZONE WE'RE IN AND WHAT

ACTIONS TO TAKE OR NOT.

WE NEVER GET TO DO THIS AT

THE BEDSIDE, OF COURSE.

IT'S TOO COMPLICATED.

THE PROBABILITIES AREN'T

KNOWN, AND IT'S NOT APPLIED

IN PRACTICE, SO IT'S ALWAYS

AN INTETUITION.

BUT THIS MIGHT BE VIEWED AS

THE GOAL.

SO CHUCK FRIEDMAN WROTE THIS

NICE PAPER ON THE

FUNDAMENTAL THIEROM OF

BIOINFORMATICS AND IT'S

PRETTY JURY.

BRAIN PLUS COMPUTER IS

GREATER THAN BRAIN.

I HOPE THAT'S TRUE.

[LAUGHTER]

I'M NOT SURE ALWAYS, BUT

HOPE SFLI THAT'S TRUE AND

WHAT WE'RE REALLY TRYING TO

DO I THINK WITH DECISION

SUPPORT US UNDERSTAND BOTH

THE DEFICIENCIES OF OUR

CLINICAL REASONING AND

UNDERSTAND THE HEALTHCARE

PROCESSES AND THE DELIVERY

SYSTEM AND THEN FILL THE

GAPS, FILL THE CARE GAPS AND

THE REASONING GAPS, FILL THE

PROCESS GAPS THAT PARTLY CAN

BE DONE DEPENDING WITH THE

COMPUTER'S ASSISTANCE.

I'D LIKE TO RECALL MARK AND

SCOTT'S DIAGRAM, THOUGH,

FROM 1980 A LONG TIME AGO,

WHO WROTE ABOUT THE

COGNITIVE FUNNEL AND WHEREIN

IS CLINICAL REASONING AND

THE PHYSICIANS' REASONING

MOST IMPORTANT?

MOVING FROM LEFT TO RIGHT,

IF YOU WILL, IN A COGNITIVE

FUNNEL, WE OPERATE ACTUALLY

VERY WELL AT THE POINT OF A,

AND NOT SO WELL AT THE POINT

OF B.

A IS WHERE UNCERTAINTY IS

RAMPANT.

THERE AREN'T CLEAR

GUIDELINES OR THE EVIDENCE

MAY BE CONFLICTING AND

MACHINES CAN OPERATE WELL AT

B, WHERE THE PROBLEM IS

WELL-DEFINED AND CONSTRAINED

AND CAN BE REALLY SUPPORT

ORDER IS AMENABLE TO COMP

PUTATION OR SYMBOLIC

REASONING OF ANY KIND.

I AM HUMBLED ALSO, THOUGH,

BY WHERE C.D.S IS LIMITEDN

MY OWN FELLOWSHIP AT STANFORD,

FOCUS ON HEALTH SERVICES

RESEARCH AND CLINICAL

INFORMATICS, I WAS A MEMBER

OF A TEAM THAT BUILT A NECK

FOR DIFFERENTTIAL DIAGNOSIS

AND MY WIFE HALFWAY THROUGH

THE PROJECT FOUND THIS

CARTOON AND MADE SURE I SAW

IT BECAUSE WE WERE GETTING

RESULTS LIKE THIS.

WE WOULD NIP A LARGE NUMBER

OF SIGNS AND SYMPTOMS AND

CASE FINDINGS INTO THE

DECISION THEORETIC PROGRAM.

OF APPROXIMATELY 700 DISEASE

CONDITIONS AND 40,000

FINDINGS AND ALL THE

PROBABLE CONNECTIONS BETWEEN

DISEASES AND FINDINGS WERE

CHARACTERIZED AND WE WOULD

GET NON-INFORMATIVE

DIFFERENT SHALES ALL OVER

THE MAP.

SO CREATING DECISION SUPPORT

REQUIRES THE CREATION OF

THIS KNOWLEDGE BASE.

IT IS DESCRIBED AS ANAWAY

PROGRAM WHOSE PERFORMANCE

DEPENDS ON THE EXPLICIT

LARGE BODY OF

ON PROCEDURES SKBRICHBLET

HERE IS IT'S REALLY ABOUT

THE KNOWLEDGE.

THIS CAME UP SEVERAL TIMES

YESTERDAY.

SMALL DOSES OF ENGINE IN AN

LP CAN INFORM THE STOMACHICAL

PROCESSS AND OPTIMIZE THEM

IN WAYS THAT ANY AMOUNT OF

COMPTATION MAY NEVER GET TO.

CREATING THESE KNOWLEDGE

BASES, THOUGH, IS DIFFICULT.

I AM GOING TO COME BACK TO

THAT.

THE KNOWLEDGE BASE, THOUGH,

CAN BE SUBJECTED TO ANY KIND

OF COMP PUTATIONAL PROJECT

AND YOU CAN RUN THROUGH THE

LIST.

BUT THE CHALLENGE IS THAT WE

ACTUALLY WANT TO BE ABLE TO

USE IT IN MULTIPLE DIFFERENT

ENGINES OR DIFFERENT SYSTEMS.

THIS IDEA OF SHAREABILITY OF

A KNOWLEDGE BASE IS CENTRAL

TO A LOT OF THE WORK OVER

THE YEARS BUT FRANKLY I

DON'T THINK IT HAS EVER BEEN

ACHIEVED.

LAST THING IS BEFORE DIVING

FURTHER IN FOO C.D.S IS EVEN

WITH ALL THE BEST EVIDENCE

WE HAVE, BRENT JAMES, AMONG

OTHERS WOULD, A WE ONLY HAVE

EVIDENCE FOR ABOUT 25% OF

WHAT WE DO.

THE REST OF IT, 75% ISN'T

SUPPORTED BY A PARTICULAR

GUIDELINE OR A COMBINATION

OF GUIDELINES AND IT'S

REALLY THE CLINICIAN

REASONING ACROSS UNCERTAINTY

WHERE THERE IS NOT A SOLID

GUIDELINE OR SOLID EVIDENCE

TO PURSUE.

SO I WOULD SUGGEST WE

ACTUALLY ARE ON THE VERGE OF

A DRAMATIC NEED FOR NOT JUST

KNOWLEDGE BASE CLINICAL

DECISION SUPPORT BUT

DECISION SUPPORT WHICH

ARISES FROM ANALYSIS IN

REALTIME OF PATIENTS LIKE

MINE OR DECISIONS LIKE MINE.

THE PATIENTS LIKE ME

PHENOMENON HAS BEEN

EXTRAORDINARY.

JAIME HAYWOOD AND THE TEAM

ALLOW PATIENTS TO PROVIDE

INFORMATION ONLINE AND GET

COMPARISON IN OTHER LIKE

PATIENTS.

WHY DON'T WE DO THE SAME

THING FOR THE PROVIDER?

SO AT THE POINT OF CARE IF

TLTS THERE IS NOT A PIECE OF

EVIDENCE OR A GUIDELINE

WHICH MIGHT APPLY, I MIGHT

BE ABLE TO SAY SHOW ME WHAT

THE PREVIOUS THOUSAND

PATIENTS AT THE BRIGHAM HAVE

DONE AT A SIMILAR SITUATION

OR THE THOUSAND PHYSICIANS

CARING FOR THOSE PATIENTS.

OUR KNOWLEDGE TRANSLATION

SPECIFICATION RESEARCH

HISTORY OVER THE YEARS HAS

MOVED FROM VERY EARLY

EFFORTS AT OKOSON AND

STANFORD WITH MARK MUSEEN

AND OTHERS THROUGH A VARIETY

OF PLAN-BASED AND SORT OF

NON-PLAN-BASED

REPRESENTATIONS OF KNOWLEDGE

TO TRY TO GET AT THIS NOTION

OF INNER OPRABLE KNOWLEDGE

BASE.

OUR GOAL IS TO COMBINE

EVIDENCE AND EXPERIENCE,

MAKE IT INTO A GUIDELINE AND

A PRINCIPLED AND UNAM

BIGUOUS WAY AND REPRESENT

THE KNOWLEDGE AND SHARE THAT

KNOWLEDGE AND SUBSEQUENTLY

EXECUTE UPON IT IN ANY

DIFFERENT ENGINE OR

RECEIVING ENVIRONMENT FOR

THE KNOWLEDGE.

THESE KNOWLEDGE BASES,

THOUGH, THAT WE BUILT OVER

THE YEARS THINGS LIKE MICEEN

FOR THE ANTI-MICROBIAL

THERAPEUTIC MENINGITIS AT

STANFORD OR DIFFERENT

DIAGNOSISS IN MEDICINE.

IN I A WAY THEY'D BEEN

CONSTRAINED TO A PARTICULAR

DECISION TYPE.

WHEN YOU THINK ABOUT THE

ACTUAL KNOWLEDGE STACK THAT

HAS TO GO INTO MODERN-DAY

CLINICAL DECISION SUPPORT,

IT HAS TO INCLUDE A WIDE

ARRAY OF COMPONENTS THAT

OFTENTIMES ARE NOT

CONSIDERED TOGETHER.

RANGING FROM THE TERMINOLOGY

SERVICES WHAT IS THE

CONTROLLED MEDICAL

TERMINOLOGY, THE UNDERLYING

INFORMATION MODEL AND ON THE

OLING AND SUBSEQUENT INTO

EXTRACTION OF MATTER USED

AND USED TO SIMPLIFY THE

KNOWLEDGE PROCESS.

INTERMED CONCEPTS, WHETHER

IT'S DRUG CLASSS OR PATIENT

STATES, ORDER CATALOGS AND

OTHER INFORMATION CLASSES

AND THEN STANDARD APP

TEMPLATES, ORDER SETS, RULES

AND THE LIKE.

SO WHERE IS C.D.S TODAY?

THE HOSPITALS ARE RIDDLED

WITH C.D.S.

OUR AUTOMOBILES ARE RIDDLEDED

WITH C.D.S NOW.

BUT COOPERMAN'S REVIEW

AROUND 2000, I THINK,

DESCRIBED THESE DIFFERENT

CLASSES, FORMATING,

INTERPRETING, CONSULTLINGING,

MONITORING, CRITICIQUE YGING

AND I WOULD ADD THIS WHERE A

LOT OF WHAT HERETOFORE WERE

CLINICAL DECISIONS ARE BEING

NOW MADE BY CONSUMERS

THEMSELVES ARE REAL

DISRUPTION, IF YOU WILL,

FROM A CLAY CHRISTENSEN

POINT OF VIEW WHERE THE

CONSUMER IS NOW BEING

EMPOWERED PREVIOUSLY

REQUIRED PROFESSIONAL

EXPERTISE.

SO WHAT'S THE EVIDENCE FOR

C.D.S?

GARTH DID A SERIOUS REVIEW

FINDING PRACTICEITIONER

PERFORMANCE IMPROVED IN 64%

OF STUDIES, 40% IN TEN

DIAGNOSTIC SYSTEMS, 76

PERCENT IN 21 REMINDER

SYSTEMS AND OF% IN DRUG

DOERGS PRESCRIBING SYSTEMS.

PATIENT OUTCOMES WERE ONLY

STUDYD IN SEVEN OF 52

STUDIES AND IMPORTANTLY

FACTORS ASSOCIATED WITH

SUCCESS WERE AUTOMATED

PROMPTS VERSUS ANYTHING THAT

REQUIRED THE USER TO DO

SOMETHING ELSE AND ALSO THE

BIAS PERHAPS OF STUDYING ONE

ONE-ZONE CHILD AND THE

AUTHORS WERE INVOLVED IN

EVALUATING THESE SYSTEMS.

A LOT OF WORK WE'VE DONE AT

THE BRIGHAM AND ELSEWHERE

SUGGEST, THOUGH, THAT C.D.S

CAN INCREASE GUIDELINE

ADHERENCE AT THE POINT OF

CARE.

WE CAN IMPROVE ORDER ENTRY

WITH CTO-RELATED PROMPTS AND

SHOWING THE COST OF TESTS

AND PROCEDURES AND WE CAN

LIKE THE AMAZON CHECK OUT

CHART DISPLAY, CHECK OUT THE

COST OF ALL THE NUMBER OF

TESTS OF ALL ORDER WHEN

SHOWING THE COST AS WELL.

THE PROBLEMS WITH HIT UP IN

MORE APPARENT OF LATE AS

WELL.

ROSS CAPEL I THINK DID THE

FIRST INTERESTING WORK IN

THIS SPACE LOOKING AT HOW

CTOE HAS FACILITATE

TRANSCRIPTION ERROR.

WHAT WERE THE PERCEPTIONS OF

IMPLEMENTATION OF THIS HIT?

AND 22 CATEGORIES OF

PERCEIVED INCREASED RISK

WERE DESCRIBED, BOTH

INFORMATION ERRORS AND ERRORS

ASSOCIATED WITH A POOR HUMAN

COMPUTER INTERFACE OR WORK

FLOW.

THIS REALLY LIT THE FUSE, I

THINK, THOUGH, UNDER HOW WE

HAVE TO VIEW HIT AND C.D.

SPECIFICALLY MUCH MORE]

CRITICALLY AND ESTABLISH A

DOSE RESPONSE CURVE F YOU

WILL, FOR HIT.

HOW MUCH OF A DOSE OF HIT IS

USEFUL VERSUS HOW MUCH IS

PROBLEMATIC?

ALERT FATIGUE IS A GREAT

EXAMPLE AND USUALLY COMES TO

MIND.

TOO MANY ALERTS AND THE

PHYSICIAN STARTS TO TURN OFF

AND IGNORE ANYTHING.

NOT ENOUGH ALERTS AND THE

PHYSICIAN MAY NOT BE PAYING

ENOUGH ATTENTION.

SO WHAT'S THE RIGHT AMOUNT

OF ALERTS IS REALLY OF

INTEREST.

PERHAPS SOME LARGEST CAN BE

BORROWED FROM ECONOMICS AND

BEHAVIORAL THEORY, THINKING

ABOUT THE THEORY OF SUBJECTIVE

NOVELTYY, FOR EXAMPLE.

DO WE ENTERTAIN AND INFORM

ENOUGH TO MAKE THE DECISION

SUPPORT INTERESTING TO THE

CLINICIAN?

RECENTLY AT DUKE A REPORT

WAS COMPLETED BY KENT

ROEBUCK, AND OTHERS AND JUST

A NUTSHELL HERE IS THAT TO

UNDERSCORE THE IDEA OF THE

AUTOMATIC PROVISION OF

DECISION SUPPORT AT THE TIME

AND LOCATION OF DECISION-MAKING,

MAKING A RECOMMENDATION THAT

IS ACTIONABLE AND THEN

INTEGRATING WITH THE WORK

FLOW AND PROMOTING ACTION,

NO NEED FOR ADDITIONAL DATA

ENTRY AND MAKE SURE THE

USERS INVOLVED IN THE LOCAL

KNOWLEDGE DEVELOPMENT

PROCESS.

SO THINKING ABOUT THE

DECISION SUPPORT RESEARCH

FOUND THAT 25% OF ALL THIS

DECISION SUPPORT RESEARCH

THAT HAS BEEN DONE HAS COME

FROM JUST FOUR DIFFERENT

INSTITUTIONS.

BRIGHAM, V.A. AND

INTERMOUNTAIN.

I'D LIKE TO BORROW HIS QUOTE

THE FUTURE IS ALREADY HERE,

IT'S JUST NOT EVENLY

DISTRIBUTED.

WE HAVE TO THINK ABOUT

DISTRIBUTING THIS KNOWLEDGE

AND HOW IT CAN WORK WELL.

LET ME TAKE A SMALL DETOUR

NOW AND JUST TALK ABOUT THE

OTHER THINGS I THINK IN

ENCROACHING UPON DECISION

SUPPORT AND NUMBER ONE, OF

COURSE, IS A BIG DATA.

HOW DO WE TAKE ADVANTAGE OF

THE QUANTITYIFIED CELL?

AS OUR PATIENTS AND OUR

CELLS ARE INCREASINGLY

QUANTITYIFIED WITH ALL TYPES

OF ONGOING MEASUREMENTS IN

THE INTECASTIES OF CARE, HOW

DO WE TAKE ADVANTAGE THAT

HAVE IN THE RECORD AND THE

BODY MONITOR, THE SMART

PHONES AND APPS AND OTHER

TYPES OF BIOSIGNAL MONITORS,

ET CETERA, BECAUSE THE

PATIENTS ARE BECOMING ACT

VACATE -- ACTIVATED PATIENT

AND CAN ACTIVATE THE

PROVIDER AS WELL.

THE NEXT ONE OF COURSE IS

PERSONALIZED MEDICINE.

WE'RE NOW BEGINNING TO

SEQUENCE THE ENTIRE JET

STREAM FOR 100 -- GENOME FOR

100 PATIENTS AT THE BRIGHAM.

HOW CONVENIENT IT IS WHEN

THE U.S. PROVIDES A PRISON?

HOW MUCH DO YOU WANT TO KNOW

WHEN YOU SEQUENCE AND JUR

GENE?

THERE IS THINGS THAT YOU MAY

NOT WANT TO DO TO KNOW AND

THE PROVIDER DOES NOT KNOW

WHAT TO DO WITH THE

INFORMATION ANYWAY.

SO SO IN ADDITION STUDYING

THESE POLY MORPHISMS NEED TO

ASSESS THE BEHAVIORAL

CHARACTERISTICS OF WHAT

PEOPLE WANT TO KNOW AND HOW

MUCH THEY WANT TO KNOW.

AND ZACH HAS WRITTEN ABOUT

THE -- THE PROBLEM WITH

SEQUENCING IS IT'S SIMPLY

THE TOO MANY TESTS PROBLEM.

IF WE HAVE A SENSITIVITY OF

100% AT A FALSE-POSITIVE

RATE OF ONLY 0.01 WITH TEN

THOUSAND TEST, THE RESULTS

WILL BE FALSE-POSITIVE.

SO WHAT HAVE SHH WE TELL THE

PATIENTS AND WHAT WILL

PATIENTS WANT TO KNOW?

AND HE HAS A LOVELY GRAPHIC

IN HIS TRANSLATIONAL

MEDICINE ARTICLE ABOUT THE

DIFFERENT DIMENSIONS AND AXES

THAT WILL HAVE TO BE

ADDRESSED BEFORE WE CAN USE

IN A DECISION WAY, PROVIDE

ANSWERS THAT MATTER TO THE

PATIENT AND MATTER TO THE

PROVIDER THAT HAVE UTILITY.

SO LET ME CLOSE NOW JUST

WITH A FEW WORDS ABOUT SOME

CURRENT RESEARCH THAT WE'RE

DOING AT PARTNERS WITH A

LARGE NUMBER OF

COLLABORATORS ACROSS THE

COUNTRY THAT'S CALLED THE

C.D.S CONSORTIUM FUNDED BY

THE HHRQ AND HAS THE GOAL OF

ACCELERATING THE TRANCE

LATION OF KNOWLEDGE INTO

PRACTICE, DISEMANATING

KNOWLEDGE AND EQUALITY VAULTING

ITS IMPACT.

WE HAVE A LARGE NUMBER OF

PARTNERS.

WE'VE BEEN FORTUNE TO HAVE

UMDM J, FRANK SONNENBERG

HERE, A NUMBER OF TECHNOLOGY

PARTNERS AND INTERNATIONAL

PARTNERS AS WELL

COLLABORATING IN THIS WORK.

WE HAD A FEW SIMPLE GOALS.

ONE WAS TO TAKE THE

KNOWLEDGE BASE DEVELOPED AT

PARTNERS OVER THE 60OD YEARS

THAT CLINICAL INFORMATION

TECHNOLOGY SYSTEMS HAVE BEEN

IN USE AND TO TRY TO

EXTERNALIZE IT FOR THE WORLD

TO USE.

SO WE'VE CREATED A PROT-TYPE

NATIONAL KNOWLEDGE

REPOSITIVETRY PORTAL WHERE

ONE CAN GO AND BROWSE THE

KNOWLEDGE ARTIFACTS USED IF

OUR CLINICAL DECISION

SUPPORT SYSTEMS AND EXAMINE

THEM AND SEE IF THEY'RE OF

INTEREST FOR YOUR OWN USE

FOR FREE.

SO WE'VE TAGGED ALL THIS

KNOWLEDGE, DEVELOPED A

MODEL, AND HOSTED IT NOW IN

THIS SAMPLE OF IT, I SHOULD

SAY IN THISER, TERNAL

KNOWLEDGE PORTAL.

THE C.D.S CONSORTIUM SERVES

UP THE KNOWLEDGE ARTIFACTS

IN THE THREE DOMAINS WE'VE

HAD UNDER STUDY FOR THE HH

IQ AND NOW SERVE UP A

KNOWLEDGE TRANSACTION, IF

YOU WILL, WITH REMOTE SITES

ACROSS THE WHICH,

INDIANAPOLIS, UMDM, G.E.,

EMINENTLY WITH DR. SONNENBERG

AND FRASER NEXTGEN CLINIC IN

PA, AND OREGON.

AT THE SAME TIME RELATED

PROJECTS A PEDIATRIC

RESEARCH COLLABORATIVE IS

USING THE C.D.SC SERVICES TO

PROVIDE DECISION SUPPORT TO

OVER HALF A DOZEN SITES NOW

I THINK, ALL OF WHICH ARE

USING EPIC EMR TECHNOLOGY.

SO IN CLOSING, WHAT ARE THE

GRAND CHALLENGES WE HAVE TO

ADDRESS?

SUMMARIZING INFORMATION

LEVEL INFORMATION,

PRIORITYIZING

RECOMMENDATIONS, MERGING

GUIDELINES WITH COMPETING

RECOMMENDATIONS, DISAM BIGUATING

GUIDELINE CONTENT, IMPROVING

THE COMPUTER INTERFACE,

MANAGING THESE LARGE

CLINICAL BASES, ACCESSING

AND MANAGING INFORMATION,

DISSEMINATION AND THE LIST

FROM MY PAPER WITH DEAN

SIDIC.

I PUT IT TOGETHER IN ANOTHER

WAY FOR THE BIOINFORMATITIONS

IN THE ROOM TO SORT OF DRAW

UPON A BIOLOGICAL ANALOGY.

I THINK IN MANY WAYS THIS IS

LIKE THE FUNDAMENTAL THIEROM

OF BIOLOGY.

WE NEED TO TRANSCRIBE ATOMIC

KNOWLEDGE OBJECTS BUILT UPON

COGNITIVE INFORMATION

FOUNDATIONS INTO KNOWLEDGE

REPRESENTATION OR KNOWLEDGE

ENGINEERING.

WE THEN NEED TO TRANSLATE

THAT INTO DECISION PROTEINS

OR ESSENTIAL CODES AND

STRUCTURES FOR SUPPORTING A

REFERENCE ARCHITECTURE.

WE NEED TO IMPLEMENT AND

ASSESS THE EFFECTIVE USE OF

PERSONALIZED DECISION

SUPPORT AT THE POINT OF CARE.

AND EACH ONE OF THESE

PROCESSES HAS A FEEDBACK

LOOP WHICH HAS TO BE

CONSIDERED AS WELL.

HOPEFULLY, THE N BIB AND

ANYONE ELSE WHO IS

INTERESTED WOULD LIKE TO

FUND THIS ONGOING WORK.

SO LAST SLIDE THEN.

I WANT TO RETURN TO NAPOLEON,

WOGOT TO MOSCOW, FOUND THE

CITY WAS BURNED TO THE

GROUND AND RETREATED ALL THE

WAY BACK TO THE GREENER

FIELDS OF PARIS AND FRANCE,

WHERE HE COULD REFLENISH HIS

ARMY.

BUT NM WAYS, I THINK, THE

EMPEROR HAS HAD NO CLOTHES

IN U.S. HEALTHCARE AND WE

NEED TO THINK ABOUT HOW TO

RECLOTHE THE PHYSICIAN

EMPEROR.

I THINK C.D.S IS THE

ESSENTIAL INGREDIENT.

WE CANNOT PROCESS THE

EXPLOSION OF KNOWLEDGE AND

DATE Y WITHOUT C.D.S AND

KNOWLEDGE SHARING IS THE

ONLY WAY TO SCALE THIS

ACROSS OUR COUNTRY AND

PERHAPS AROUND THE WORLD.

I'D LIKE TO SEE US NOT ONLY

SHARE DATA BUT SHARE

KNOWLEDGE AS WELL SEAMLESSLY

ACROSS THE LAND, AND THAT'S

IT.

THANK YOU VERY MUCH.

[APPLAUSE]

QUESTIONS?

>> ANY QUICK QUESTIONS FOR

BLACKFORD?

>> JIM WALKER.

I THINK YOUR PRESENTATION

WAS GREAT, BLACKFORD.

IT RAISES ALL KINDS OF

QUESTIONS FOR ME ABOUT

PROCESS ENGINEERING, ABOUT

THE ROLE OF THE CARE

DELIVERY ORGANIZATION,

WHETHER IT'S A LOCAL SORT OF

TRADITIONAL ORGANIZATION OR

INCREASINGLY SOME KIND OF

ACCOUNTABLE CARE ARRANGEMENT.

IT SEEMS TO US THAT A KEY

PART OF THIS IS GETTING

COMMITMENT OF ORGANIZATIONS

AND INDIVIDUALS -- PATIENTS

AND THE REST OF THEIR CARE

TEAM -- TO 100% PROCESSES.

AND IF YOU GET THAT, THEN IT

SEEMS TO ME WHAT YOU ARE

DOING BECOMES CRITICAL

INFRASTRUCTURE.

WITHOUT THAT, IT'S NOT CLEAR

TO ME HOW IT WORKS OUT.

BUT THAT'S A COMMENT, NOT A

QUESTION.

IS THAT ANY PART OF THIS,

PART OF THE KNOWLEDGE BASE

TO UNDERSTAND WHAT THE

LEVERS ARE FOR PATIENTS,

PATIENTS WITH LIMITED

EDUCATIONS, PATIENTS WITH

MODEST INTEREST IN WELLNESS,

IN SOME FUTURE STATE TO HOW

WE ENGAGE THOSE PEOPLE IN

RECEIVING THIS INFORMATION?

>> EXCELLENT QUESTIONS AND A

GREAT POINT TO BRING THIS

BACK TO REALITY.

HOW DO WE TUNE THE

ORGANIZATION OR FRAME THE

PROBLEM FOR THE ORGANIZATION

AND HOW DO WE MAKE THIS

GENERALIZABLE TO THOSE BEING

IN OUR HEALTHCARE SYSTEM AND

THOSE WHO ARE NOT?

ONE OF THE THINGS I AM

ENCOURAGED ABOUT IS JUST THE

PLETHORA OF SMART PHONE APPS

AND CLINICAL APP AS RISING

ARISING IN THE CONSUMERS'

HANDS WHERE THE VERY SAME

KNOWLEDGE NEED TO BE APPLIED

FOR LOW-ACTIVATED PATIENT,

ET CETERA.

AND I THINK THAT'S AN

EXCELLENT POINT AND PART OF

THE IMPLEMENTATION CHALLENGE.

>>

[INDISCERNABLE]

>> I GUESS WHAT I WOULD

SUGGEST.

WHEN YOU THINK ABOUT THE EKG

INTERPRETATION, PFTN IN THE

HOSPITAL, THE LABORATORY,

EVERYTHING COMES WITH AN

INTERPRETATION, AND THAT IS

NOW DECISION SUPPORT.

IF YOU LOOK AT THE

DIAGNOSTIC DIFFERENTTIAL, I

AGREE WITH YOU, THAT'S BEEN

A CHALLENGE BECAUSE I THINK

THE PROBLEM IS HARD.

BUT ON THE THERAPY SIDE, THE

PLANNING RADIATION ONCOLOGY

AND OTHER PROGRAMS ARE USED

ROUTINELY.

WE'VE MADE PROGRESS.

OF COURSE, WATSON IS GOING

TO SOLVE THIS PROBLEM?

>> THANK YOU VERY MUCH.

>> THANK YOU,-BLACKFORD.

[APPLAUSE]

>> MY NAME IS RICHARD CONRAD.

I'M THE PROGRAM DIRECTOR OF

N BIB AND WE'RE GOING TO

START THE FIRST PANEL OF THE

MEETING HERE.

IT'S ON CLINICAL

PERSPECTIVES AND WHEN WE

WERE PLANNING THIS MEETING,

WHAT WE WERE TALKING THIG

ABOUT WE WOULD LIKE TO HEAR

WHAT THE STATE-OF-THE-ART IS

FOR CLINICAL DECISION

SUPPORT FROM THE CLINICIANS

THEMSELVES AND PEOPLE WHO

ARE USING THESE SYSTEMS.

WHAT THE CHALLENGES THEY SEE

ARE, AND WHERE THEY SEE

OPPORTUNITIES FOR FUTURE

RESEARCH.

BUT ALSO WHERE THEY WOULD

SEE SYSTEMS HELPING THEM

WITH THEIR CLINICAL

PRACTICES.

SO WE'VE GOT FOUR GREAT

SPEAKERS, AND I AM NOT GOING

TO DO A LONG INTRODUCTION

BECAUSE I KNOW THEY'VE GOT

LOTS OF GREAT THINGS TO SAY.

BUT 2007 DR. THOMAS PAYNE

FROM THE UNIVERSITY OF

WASHINGTON.

JAMES WALKER FROM GEISINGER

HEALTHCARE SYSTEM AND SIEGEL

FROM THE UNIVERSITY OF

MARYLAND.

I AM GOING TO ASK EACH OF

THEM TO COME UP.

WE'LL TAKE QUESTIONS AFTER

EACH OF THE TALKS AND THEN

AT THE END WE'LL HAVE A

HALF-HOUR PANEL DISCUSSION.

SO IF YOU HAVE GENERAL

QUESTIONS, PLEASE SAVE THEM

TO THE END.

IF YOU'VE GOT SPECIFIC

QUESTIONS, PLEASE ASK THEM

AT THE END OF EACH TALK.

THANKS.

>> SO GOOD MORNING.

I AM THE FIRST PANELIST AND

I'M TOM PAYNE FROM THE

UNIVERSITY OF WASHINGTON IN

SEATTLE.

I DO HAVE SOME SLIDES, WHICH

ARE HERE ON THE SCREEN AND I

HAVE.

MY TOPIC TODAY IS AN ATTEMPT

TO BLINK SOME OF WHAT WE

HEARD YESTERDAY INTO THE

WORLD IN WHICH I WORK AND

WHICH MANY AMERICAN

CLINICIANS WORK IN A

CLINICAL SETTING AND TO KIND

OF SHOW HOW THAT RELATES TO

THE TOPIC THAT BLACKFORD

REVIEWED, WHICH IS DECISION

SUPPORT.

I THINK A PERSPECTIVE THAT

THE GROUP HERE CAN ADD TO

WHAT WE DISCUSSED YESTERDAY

IS THE WORK FLOW INTO WHICH

ALL OF THIS SITS.

THAT WAS SOMETHING I WANTED

TO ADD TO THE ANSWERS GIVEN

TO OUR QUESTIONER OF

YESTERDAY WHO ASKED WHAT

DOES IT TAKE TO TAKE WHAT

WE'VE LEARNED ABOUT NL.

AND TO IMPACT THE HEALTH OF

PEOPLE IN THE UNITED STATES?

AND THERE WAS A LIST OF

ANSWERS TO THAT QUESTION.

ONE MORE WOULD BE A

RECOGNITION THAT THE WORK

FLOW IN WHICH ALL THESE

THINGS APPLIED IS A CRITICAL

COMPONENT.

SO I AM GOING TO GIVE YOU A

LITTLE STORY ABOUT MY

EXPERIENCE WITH THAT WORK

FLOW, AND I WANT TO TELL YOU

FIRST HOW WE GOT FROM PAPER

TO ELECTRONIC NOTE, WHICH IS

THE MILL.

AND I WANT SHOW HOW THIS

RELATES TO SOME POTENTIAL

AND CLINICAL DECISION

SUPPORT.

I'LL GIVE YOU THREE EXAMPLES

OF HOW WE ARE USING NATURAL

LANGUAGE PROCESSING TODAY IN

VERY EARLY WAYS IN

UNIVERSITY OF WASHINGTON IN

SEATTLE.

AND THEN HOPEFULLY WE'LL

HAVE A FEW REMAINING MINUTES

FOR DISCUSSION.

I WORK IN A HOSPITAL SYSTEM

IN SEATTLE.

WE HAVE FIVE HOSPITALS.

WE HAVE ABOUT ONE AND A HALF

MILLION OUTPATIENTS VISITS A

YEAR AND 50,000 DISCHARGES

FROM OUR HOSPITALS.

SO IT'S A MODERATELY SIZED

ACADEMIC MEDICAL CENTER IN

THE NORTHWEST.

THIS IS WHERE WE STARTED,

WHICH IS THE STATE OF MANY

AMERICAN HOSPITALS,

CERTAINLY 30 YEARS AGO AND

TODAY AND MANY HOSPITALS.

IN OUR FIRST STEP ON THIS

JOURNEY TO ELECTRONICS WAS

TO PUT THAT INTO AN

ELECTRONIC MEDICAL RECORD,

WHICH HAS SOME ADVANTAGES

AND A FEW DISADVANTAGES AS

WELL.

SO THE NEXT STEP IN THIS

JOURNEY WAS TO CONVERT WHAT

CLINICIANS DOCUMENTED IN

THEIR CARE OF PATIENTS FROM

THAT TO THIS, WHICH IS AN

IMPROVEMENT.

IT IS LEGIBLE.

IT IS AVAILABLE TO MANY

PLACES AT THE SAME TIME, AND

IT'S USEFUL FOR MANY OTHER

PURPOSES BEYOND THE

IMMEDIATE PATIENT CARE

PURPOSE FOR WHICH THEY WERE

CREATED.

IT ALSO IS VALUABLE TO

OTHERS WHO HELP US WITH

COMPLIANCE ISSUES AND SO ON.

BUT IT HAS ITS OWN SET OF

LIMITATIONS.

WE STARTED OUR TRIP FROM

PAPER TO ELECTRONIC ABOUT

EIGHT YEARS AGO, AND ABOUT

SIX YEARS AGO, WE HAD

LARGELY FINISHED THIS

CONVERSION FROM PAPER TO

ELECTRONIC NOTES.

SO SINCE THEN, WE HAVE

THOUSANDS OF PHYSICIAN NOTES

WRITTEN EACH DAY FOR THE

HOSPITAL AND IN THE CLINICS

AND THAT TRANSITION HAS

INCLUDED ALL OTHER

DISCIPLINES NOW FROM NURSING

NOTES, BEDSIDE NURSING NOTES

TO NUTRITION, PHYSICAL

THERAPY AND INTERESTINGLY,

NOTES WRITTEN BY HOSPITAL

CHAPLAINS AS WELL.

IT'S ALL ELECTRONIC.

IT'S ALL IN ONE PLACE.

A REAL ADVANCE FOR US.

WHAT WE'VE FOUND IS THAT

NARRATIVE TEXT, THE CONTENT

OF THE NOTE THAT I JUST

SHOWED YOU, WE REGARDED TO

BE VERY VALUABLE.

AND SOME OF THE WAYS IN

WHICH IT'S VALUABLE I LIST

HERE.

FIRST OF ALL, THE NARRATIVE

CONTAINS THE HISTORY AND ITS

DETAILS, THE EXAM, AND THE

DISTINCTION OF THE CLINICIAN.

I HAVE TO SAY THAT THIS IS

ONE AREA IN WHICH THE NEEDS

OF CLINICAL CARE, THE NEEDS

OF OUR COMPLIANCE

REIMBURSEMENT CREW, AND THE

ZPAECHING RESEARCH

ENVIRONMENTS OVERLAP AND

THAT IF WE CAPTURE THE

THINKING OF THE CLINICIAN,

WE CAN ACCOMPLISH A LOT FOR

EACH OF THOSE GROUPS.

AND EACH OF THESE THOUSANDS

MAYBE 5,000 NOTES WRITTEN BY

PHYSICIANS EACH DAY CONTAINS

SOME SMALL AND MAYBE MORE

THAN SMALL KERNELS OF TRUTH.

SO THESE FINDINGS, THESE

EPISODES OR THESE COMPONENTS

OF THE STORY THAT THE

PATIENT TELL ARE IN THAT

NOTE.

NOW, IT'S NOT EASY TO FIND.

IT IS OBSCURED BY TEMPLATES,

BY THE ARTIFACTS THAT WE USE

TO PUT THE NOTE INTO THE

REPORTED SUCH AS COPYING AND

PASTING, DIRECT ENTRY AIDS

AND SO ON.

BUT IT'S THERE.

AND WE HAVE THE ABILITY TO

MAKE IT EASIER TO FIND AS

WELL.

AND IF YOU MULTIPLY THAT

SINGLE NOTE TIMES THE NUMBER

OF ENCOUNTERS THAT WE HAVE

IN OUR INSTITUTION, AND THE

NUMBER OF HOSPITALS IN OUR

COMMUNITY, THE NUMBER OF

COMMUNITIES IN OUR COUNTRY,

THERE IS A LOT OF

INFORMATION THERE LURKING

WITHIN THE NARRATIVE TEXT.

THERE HAS BEEN DISCUSSION

OVER THE LAST CERTAINLY 25

YEARS I'VE BEEN IN THE FIELD

BETWEEN THE TENSION BETWEEN

USE OF NARRATIVE TEXT AND

USE OF ENCODED OR STRUCTURED

NOTES.

BY THE WAY, BARNETT TAUGHT

ME YEARS AGO TO CALL IT

NARRATIVE TEXT AND NOT

PRETEXT.

HE SAYS NOTHING IS FREE

ABOUT FREE TEXT.

SOMEBODY PAYS FOR IT.

SO THE NARRATIVE TEXT

CONTAINS INFORMATION THAT WE

WOULD LIKE TO GET OUT.

WE HAVE TO LOOK FOR

STRUCTURED NOTE ENTRY AS

WELL.

AND YOU CAN SEE THAT THERE

ARE CONSTRAINED CHOICES FROM

WHICH THE CLINICIAN CHOOSES.

I HAVE ILLUSTRATE THERE HAD

IN A VERY SMALL BOX THAT

THERE IS A CHOICE THAT I CAN

MAKE TO INDICATE WHETHER

THERE IS SERUMIN IN BOTH

EARS AND WHEN I MAKE THAT

CHOICE, IT IS STORED AS AN

ENCODED ELEMENT.

THAT'S FROM ONE OF OUR

VENDORS.

HERE IS THE SAME IDEA

IMPLEMENTED BY A DIFFERENT

VENDOR.

SO THESE ARE ALTERNATIVES TO

NARRATIVE TEXT.

YOU CAN USE STRUCTURED

TOOLS.

AND THESE ARE VERY POWERFUL,

DEVELOPED WITH THE IDEA THAT

STRUCTURED OR ENCODED TEXT

IS REALLY THE FOUNDATION ON

WHICH WE BUILD CLINICAL

DECISION SUPPORT SYSTEMS.

HOWEVER, WHAT WE'VE LEARNED

IN OUR JOURNEY IS THAT THERE

ARE A LOT OF PROBLEMS WITH

STRUCTURED NOTE ENTRIES.

THE FIRST IS THAT IT'S

HARDER TO TRAIN PEOPLE TO

USE THEM.

ACE TEACHING INSTITUTION, WE

HAVE TURNOVER EVERY MONTH

WITH OUR TRAINEES, OUR

FELLOWS, OUR JUNIOR FACULTY

AND SO ON.

WE HAVE FIVE HOSPITALS, NOT

INCLUDING THE CHILDREN'S AND

THE V.A..

SO PEOPLE MOVE FROM ONE TO

ANOTHER AS THEY MOVE, THEY

HAVE TO LEARN HOW TO USE

STRUCTURED NOTE TEMPLATE

TOOLS AND THAT BURDEN IS

CONSIDERABLE TO US.

IN OUR EXPERIENCE, MOST

PHYSICIANS DON'T LIKE TO USE

THESE TWOTO WRITE THEIR

NOTES.

THEY PREFER ALTERNATIVES.

THEY WILL DO IT AND SOME

ULTIMATELY CHOOSE THIS.

BUT IT'S NOT THE MAJORITIES.

-- MAJORITY.

WHEN YOU ARE DONE WRIG A

 MOSTITH THESE TOOLS,

PHYSICIANS AREN'T TOO HAPPY

ABOUT READING THEM AND THEY

FOCUS ON THE NARRATIVE TEXT

PORTION, THE ASSESSMENT OR

THE VERY TERSE HISTORY AT

THE BEGINNING.

AND IN TREATING THESE WE

HEARD A LITTLE BIT ABOUT

THIS YESTERDAY, THAT IF YOU

USE THESE TOOLS, YOU LOSE

SOME VERY IMPORTANT DETAILS

THAT IS CONTAINED KM IN A

RICHER NARRATIVE.

AND SO FOR THESE REASONS, WE

FOUND THAT CLINICIANS ARE

MOVING MORE TO NARRATIVE

TEXT.

SO THAT'S WHY I VIEW A TOPIC

SUCH AS NATURAL LANGUAGE

PROCESSING AS TO IMPORTANT.

AND SO IMPORTANT FOR MANY

REASONS, INCLUDING CLINICAL

DECISION SUPPORT.

HERE ARE SOME -- A QUICK

WALK THROUGH A SINGLE NOTE

THAT WILL HELP ILLUSTRATE

WHY I BELIEVE THIS IS

IMPORTANT.

BUT BEFORE I SHOW YOU THAT

NOTE, I'D LIKE TO BRING UP

THE POINT THAT GORDY CHEF

AND DAVID BATES MADE IN THE

NEW ENGLAND JOURNAL ABOUT

THE PROBLEMS THAT WE FACE IN

CLINICAL MEDICINE TODAY AND

HOW CLINICAL DOCUMENTATION

MIGHT HELP.

THEIR OBSERVATION, WHICH I

FIND FIND TO BE VERY

ACCURATE, IS THAT THE

PROBLEM OF HAVING TOO MUCH

INFORMATION IS NOW

SURPASSING THAT OF HAVING

TOO LITTLE.

WHEN I WAS AN INTERN, 20

PERCENT OF THE TIME WHEN I

WENT TO THE CLINIC THERE

WOULD BE NO CHART.

THAT ISN'T THE PROBLEM

ANYMORE.

NOW I HAVE A CHART THAT'S

TOO LARGE FOR ME TO ABSORB.

AND THAT HAPPENS IN EVERY

ENCOUNTER IN WHICH I ENTER

THE ROOM UNLESS I KNOW THE

PATIENT WELL.

SO THIS IS ANOTHER NEED FOR

HELP AND I VIEW THIS AS AN

IMPORTANT ELEMENT OF CLINICAL

DECISION SUPPORT.

SO LET'S SEE HOW THAT MIGHT

COME TO BARE ON A -- BEAR ON

A TYPICAL NOTE.

THIS INCLUDES A LIST OF

MEDICATIONS THAT WAS ENTERED

BY THE INTERN.

IS THE RECONCILIATION OF THE

MEDICATIONS ACCURATE?

CAN WE HELP WITH THAT

PROBLEM?

IN A PHYSICAL EXAM, VITAL

SIGNS AND OTHER FINDINGS ARE

NOTED.

DO THESE HIDE CLUES TO AN

EARLY SIGN OF SUBSTANCE,

WHICH IF RECOGNIZED, COULD

HELP ALTER OF COURSE THAT

HAVE ACCEPTSIS?

THE IMAGING REPORT IS LISTED

IN A HISTORY.

WHAT OTHER RECOMMENDATIONS

LIE IN THOSE IMAGING REPORTS

THAT ARE RELEVANT TO THE

CARE OF THAT PATIENT?

HERE IS THE FORMULATION OF

THE THINKING OF THAT

PHYSICIAN, AND THEIR EFFORTS

TO ORGANIZE THE PROBLEM.

SHOULD THERE BE A BROADER

DIFFERENTTIAL?

IS THE CARE OUTLINE THERE

HAD APPROPRIATE?

THE CODE STATUS, IS THAT

ACCURATE?

IS THAT UP TO DATE?

ALL OF THESE ARE AREAS IN

WHICH WE CAN HELP THOSE

CLINICIANS WHO MAY HAVE 15

SUCH ADMISSIONS IN ONE

NIGHT.

AND LASTLY FOR OUR

COLLEAGUES IN CMS, ARE THE

COMPLIANCE RULES BEING

FOLLOWED?

WE'RE TENTATIVE TO THIS, AS

-- ATTENTIVE TO THIS, AS ARE

MOST INSTITUTIONS AND SPENT

SPEND A LOT OF TIME IN

TRYING TO IMPROVE OUR

COMPLIANCE.

ONE WAY TO DO THIS WAS TO

USE STRUCTURED NOTES.

THIS IS ONE OF THE

ADVANTAGES OF USING

STRUCTURED NOTES THAT YOU

CAN QUICKLY TRACK THE CARE

GIVEN TO PEOPLE WITH SAY

DIABETES MELITIS WITH

ENCODED ELEMENTS FOR FOOT

EXAM AND SO ON.

BUT AS I MENTIONED, THIS IS

LESS COMMONLY USED IN OUR

EXPERIENCE THAN THE

NARRATIVE TEXT REPORT.

AND IT'S NOT JUST US.

WHEN WE LOOKED AT THE CORE

MEASURES THAT ARE USED TO

MEASURE THE QUALITY OF THE

CARE WE DELIVER, THAT WERE

GATHERED TOGETHER BY THE U

AC, WHAT WE FOUND WAS THAT A

LOT OF THE INFORMATION FROM

THE CORE MEASURES IS FOUND

IN STRUCTURED ENCODED

INFORMATION IN OUR SYSTEMS,

BUT MOST OF IT IS NOT.

MOST OF THE INFORMATION THAT

WE USED TO MEASURE QUALITY

LIES IN THE NARRATIVE TEXT

OF THE NOTE AND NOT JUST

FROM ONE SOURCE BUT FROM

MANY SOURCES.

SO THERE IS WHERE WE NEED TO

DIRECT OUR FOCUS.

IF WE LOOK AT A BROADER

PERSPECTIVE ON THE SAME

ISSUE IN THIS PAPER FROM

PROCESS AND HER COLLEAGUES,

WE FIND THAT A LOT OF THE

QUALITY MEASURES THAT WE

SEEK TO APPLY TO THE CARE OF

PATIENTS ARE HARD TO GET TO,

AND AS YOU CAN SEE, ON THE

RIGHT-HAND SIDE OF THIS

GRAPH, SOME OF THE THINGS

THAT ARE HARD TO GET TO ARE

DISEASE-SPECIFIC HISTORY,

PHYSICAL EXAM, PATIENT

EDUCATION, SOCIAL HISTORY,

AND SO ON.

THOSE ARE VERY, VERY

IMPORTANT TO MEASURE IN

QUALITY.

AND YET THERE IS LARGELY NOT

EASY TO GET TO BECAUSE

THEY'RE IN NARRATIVE TEXT.

SO I SEE A HOST OF REASONS

THAT NATURAL LANGUAGE

PROCESSING CAN BE A BIG HELP

TO US.

SO WITH THAT BACKDROP, THE

STORY OF OUR TRANSITION FROM

PAPER TO ELECTRONIC NOTES,

OUR EXPERIMENTS USING

STRUCTURED AND UNSTRUCTURED

THE ADVANTAGES OF BOTH.

I'D LIKE TO SHOW YOU SOME

EXAMPLES WAVE PLIED TO THE

FIELD OF NATURAL LANGUAGE

PROCESSING IN OUR PRODUCTION

ENVIRONMENT.

NOW, A BRIEF ASIDE HERE TO

POINT OUT THAT, AS WE TAKE A

GREAT IDEA IN COMPUTING, IT

HAS TO PASS THROUGH SEVERAL

HURDLES, INCLUDING THE

RIGOROUS TESTING.

BUT THE HURDLES THAT I AM

FAMILIAR WITH IN MY DAY-TO-DAY

WORK IS THE CMS SYSTEM ARE

TO GET IT INTO A PRODUCTION

SYSTEM, IT HAS TO BE FIRST

OF ALL, INDUSTRIAL STRENGTH.

IT HAS TO BE HAVE EXTREMELY

HIGH PERFORMANCE OR

CLINICIANS WON'T TOLERATE

IT.

IT HAS TO COMPETE WITH LOTS

OF OTHER APPLICATIONS, AND

IT HAS TO FIT INTO THE WORK

FLOW OF THE BUSY CLINICIAN.

IF IT DOESN'T MEET THOSE

STANDARDS, IT REMAINS IN THE

LABORATORY.

NOW WE HAVE PASSED THOSE

HURD ENVELOPS TWO AREAS AND

WE'RE WORKING ON THE THIRD.

AND I'LL SHOW YOU THOSE

EXAMPLES TODAY.

AND THESE ADDED TO MY

ENTHUSIASM FOR THE FIELD OF

NATURAL LANGUAGE PROCESSING.

FIRST, COMPLYING WITH THE

LAW.

EVALUATION AND MANAGEMENT

CODES ARE DIFFICULT TO

ASSIGN.

FOR THOSE OF YOU WHO AREN'T

FAMILIAR WITH THIS, I'LL

GIVE YOU A BRIEF VIEW OF A

FEW OF THE PAGES OF THE BOOK

THAT COVERS HOW YOU ASSIGN

AN EMNM CODE TO YOUR NOTE.

AND BY THE WAY, THIS IS HOW

YOU ARE PAID SO IT'S

IMPORTANT THAT YOU MASTER

THIS.

SO YOU CAN SEE THAT THIS

BOOK, A FEW PAGES HERE ONLY

IS NOT EASY.

AND SO IT IS NOT SURPRISING

THAT PHYSICIANS HAVE

CHALLENGED COMPLYING WITH

THESE.

SO WE THOUGHT WHAT A GREAT

IDEA FOR THE USE OF NATURAL

LANGUAGE PROCESSING.

AND JUST AS WE HEARD

YESTERDAY, A COMPUTER SYSTEM

CODING WAS APPEALING TO US

BECAUSE OF THE COMPLEXITY OF

THESE RULES AND BECAUSE WE

ALSO HAVE THE NOTES IN

MACHINE-READABLE FORM.

TO OUR MEDICAL ELECTRONIC

RECORD CONTAINS A SET OF

TOOLS TO PROCESS THE

NARRATIVE TEXT, TAG THE

DOCUMENTS WITH CODES, APPLY

AG GARITHMS THAT ARE

PERTINENT TO ASSIGNING AN E

AND M EPISODE AND THEN YOU

HAVE A REASONABLE ESTIMATE

SUPPORTED BY THE DOCUMENT

YOU'VE JUST SIGNED.

THE TOOLS, HANDLE

NEGATIVEATION AND

QUALIFIERS.

WE CAN ADD RULES TO INCREASE

ITS PRECISION AND

INTERESTINGLY AND

IMPORTANTLY, THIS GIVES

FEEDBACK TO THE PROVIDER ON

EVERY NOTE THEY FIND WITHIN

THREE SECONDS.

SO IT FITS INTO THE WORK

FLOW OF THE PHYSICIAN.

IT'S CALIBRATED TO MEET THE

STANDARDS OF OUR COMPLIANCE

OFFICER AND AFTER

CALIBRATION, 93 OUT OF 100

NOTES WERE GIVEN THE SAME

CODE AS A TEAM OF

PROFESSIONAL CODERS ASSIGNED

TO THAT SAME NOTE.

I WILL NOT TELL YOU THE

PRECISE FIGURE THAT THE

CLINICIANS HAD IN ASSIGNING

CODES TO THE SAME NOTES, BUT

IT WAS SUBSTANTIALLY LOWER.

SO THIS IS VERY ENCOURAGING

TO US THAT IT GIVES

PHYSICIANS SOMETHING THEY

VERY MUCH VALUE, WHICH IS A

SAFETY NET TO COMPLYING WITH

THE LAW, AND IT ALSO HAS

BENEFITS.

THIS IS HOW IT ACTUALLY FITS

INTO THE WORK FLOW, WHICH IS

SORT OF A MODEL FOR HOW

DECISION SUPPORT

APPLICATIONS MIGHT BE USED.

FIRST OF ALL, WE AREN'T

PARTS TO HOW THAT NOTE CAME

TO BE BUT IT WAS DICTATED,

TYPED TO TEMPLATE, VOICE

RECOGNITION SOFTWARE.

THEN A FEE SHEET IS

COMPLETED.

THAT SIGNED NOTE GOES

THROUGH THE ELECTRONIC

MEDICAL RECORD INTO THE

PLACES ON THOSE RESIDE.

BUT A NOTE IS ALSO SENT TO

THIS ENGINE, WHICH IS USED

TO DERIVE THE CODE SUPPORTED

BY THE DOCUMENT.

BUT ALSO THE CODES THAT WERE

PULLED FROM THAT NOTES ARE

STORED AND ARE ALSO

AVAILABLE FOR USE FOR OTHER

PURPOSES.

AND I'LL SHOW YOU ONE OF

THOSE OTHER PURPOSES SHORTLY.

IF YOU HAVE A QUICK SCREEN

PRINT HERE SHOWS THAT ON THE

RIGHT SIDE OF THE SCREEN,

YOU SEE THE HISTORY OF THAT

NOTE.

AND I'VE HIGHLIGHTED AML.

IT CORRESPONDS TO LEUKEMIA.

YOU HAVE AU A CODE AND EVERY

PHRASE THAT IS IDENTIFIED

ALSO HAS THOSE.

AND AGAIN, THIS ANALYSIS IS

PERFORMED WITHIN THREE

SECONDS AND FED BACK TO THE

PHYSICIAN.

THE FULL NOTE LOOKS LIKE

THIS.

AND THOSE NOTES ARE READ FOR

NEGATION, GREEN FOR SOME

FORM OF PROBABILITY, AND

BLUE FOR POSITIVE.

SO THAT'S AN EXAMPLE OF A

PHRASE IDENTIFIED AND THE

CODE THAT IS ASSIGNED BY THE

SOFTWARE IS SHOWN HERE.

THE ONE ASSIGNED BY THE

PHYSICIAN IS SHOWN HERE.

SO THEY CAN UNDERSTAND WHERE

THEIR ESTIMATION VARIES FROM

WHAT THEptz� SYSTEM OFFERS AND

THIS ISX

EXPERIENCE TO THEM AND I

THINK A GOOD MODEL FOR

HELPING US DO A BETTER JOB.

SO THAT'S ONE EXAMPLE.

COMPUTER-ASSISTED CODING.

A SECOND IS THAT OUR NOTES

LOOK IN MANY CASES LIKE THE

ONE I'M SHOWING YOU HERE.

AND THIS IS A NARRATIVE TEXT

NOTE THAT INCLUDES A

DETAILED PROBLEM LIST

WRITTEN IN TERMS THAT ARE

FAMILIAR TO THE PHYSICIAN.

THE PROBLEM IS THAT NONE OF

THIS SYSTEMS THAT BLACKFORD

REFERRED TO CAN DO MUCH WITH

THIS UNTIL IT IS ENCODED.

SO WE HAVE TAKEN THAT VERY

PHYSICIAN-FRIENDLY PROBLEM,

WHICH YOU CAN SEE IN THIS

NOTE, AND ADAPTED THE

SOFTWARE SYSTEM THAT I

SHOWED YOU EARLIER FOR

COMPUTER-ASSISTED CODING AND

HELPED PULL THE PROBLEM LIST

FROM THAT NOTE.

SO YOU CAN SEE HERE A LIST

OF NOTES ON THE LEFT.

IT ANALYZES THE NOTE, AND IT

INDENTIFIES A SET OF

DIAGNOSES THAT ARE MENTIONED

BY THE PHYSICIAN IN THE NOTE,

INCLUDING SOME THAT ARE A

LITTLE BIT MORE CHALLENGING,

FOR EXAMPLE, L RENAL

LACERATION, T-11 COMPRESSION

FRACTURES.

FOLLOW THOSE ABBREVIATION

THAT'S WE HEARD SO MUCH

ABOUT YESTERDAY AND IT DOES

A VERY GOOD JOB OF PULLING

OUT THE CODES REPRESENTED IN

THAT NOTE FOR THE PHYSICIAN

TO REVIEW AND ADD TO THE

PROBLEM LIST WHICH THEY CAN

DO WITH A SINGLE CLICK.

SO THIS MAY SEEM LIKE A

SMALL ADVANCE AND I GUESS ON

THE SCALE OF THINGS IT IS,

BUT FOR THOSE PHYSICIANS,

FINISHING THEIR NOTE, DOING

THEIR JOB FOR DOCUMENTING

THAT CARE AND HAVING THE

PROBLEM LIST AUTOMATED AND

ENCODED AS A BY PRODUCT

THEIR WORK IS VERY POPULAR

TO THEM SO THEY DON'T HAVE

TO GO TO ANOTHER DIALOGUE

BOX, INTERRUPT THEIR WORK

FLOW TO ASSIGN A PROBLEM

LIST.

VERY POPULAR.

A THIRD EXAMPLE -- AND THIS

ONE HAS NOT YET ACHIEVED THE

STANDARD THAT I MENTIONED

FOR YOU OF BEING AVAILABLE

IN PRODUCTION WITH

PERFORMANCE STANDARDS THAT

CLINICIANS EXPECT.

AND THIS IS WORK I AM DOING

WITH MALIA YETISH IN OUR

BIOMEDICAL INFORMATICS

GROUP.

IN IT TACKLES THE

UNFORTUNATE PROBLEM THAT WE

FACE, WHICH IS THAT IMAGING

REPORTS ARE SO EXTENSIVE.

I'M NOT REFERRING TO THE

IMAGES THEMSELVES BUT THE

REPORTS WE GET BACK THAT

WITHIN THE REPORTS ARE MORE

INFORMATION THAT WE CAN

EASILY PROCESS OR REMEMBER.

THIS EXAMPLE SHOWS THAT THIS

TRAUMA VICTIM HAS MANY

SERIOUS PROBLEMS, INCLUDING

FRACTURES, PNEUMOTHORACS,

HEMATOMA IN THE ABDOMEN, A

LEFT HEMOTHORSAX AND THAT'S

ENOUGH TO GET YOUR

ATTENTION.

SO YOU HAVE LOTS TO DO TO

CARE FOR THIS PATIENT, AND

YOU MIGHT OVERLOOK THE FACT

THAT THERE IS ALSO A RIGHT

ADNEXAL CYT, WHICH THE

RADIOLOGIST RECOMMENDS THAT

WE REMEMBER AND FOLLOW UP TO

AVOID POSSIBLE OVARIAN

NEOPLASM DEVELOPING.

SO ALL OF THIS INFORMATION

IS IN THE NARRATIVE TEXT.

WHAT CAN WE DO TO HELP IT

WITH THAT?

OUR PROJECT IS LOOKING NOT

FOR THE CRITICAL

ABNORMALITIES BUT FOR THOSE

THAT ARE SUBCRITICAL,

FLAGGING THEM SO THAT

THEY'RE AVAILABLE FOR THE

NEXT PERSON TO REVIEW IN THE

CLINIC PERHAPS WHEN THEY'VE

RECOVERED FROM THEIR

ACCIDENT.

SO VOICE IS CATCHING ON.

VOICE AND SPEECH

TECHNOLOGIES ARE NOW

MAINSTREAM.

THIS IS NOTICED BY PHYSICIANS.

THEY'RE USING THIS

EXEXTENSIVELY.

I THINK THE THINGS THAT WE

CAN USE AS WE TAKE THE

EXPRESSIVENESS OF THE

PHYSICIAN AND PUT IT INTO

TEXT IS THAT WE HAVE THE

ABILITY TO SUMMARIZE, TO

SEARCH FOR EASILY, TO

EXTRACT KEY ENCODED

INFORMATION SUCH AS PROBLEM

LISTS.

WE CAN FOCUS OUR ATTENTION

ON THINGS THAT MIGHT BE

OVERLOOKED, AND SO AS THE

VOLUME OF NARRATIVE TEXT

GROWS, SO DOES OUR NEED FOR

THE TOOLS THAT WE LEADER

ABOUT DESCRIBED YESTERDAY.

I THINK THE TREND TOWARD

NARRATIVE GROWTH WILL

CONTINUE, AND THE NLP WILL

HELP US TO MAKE BETTER

DECISIONST FITS INTO THE

WORK FLOW OF MEDICAL RECORDS

AND AS WE IMPLEMENT MEDICAL

RECORDS MORE EXTENSIVELY

ACROSS OUR COUNTRY, WE NEED

TO MAKE SURE THAT WE MATCH

THOSE EMRS WITH HUMAN

STRENGTH AND WITH WORK FLOWS.

WE'LL STOP THERE.

[APPLAUSE]

>> WE HAVE TIME FOR ONE OR

TWO QUESTIONS FOR TOM.

>> UNIVERSITY OF UTAH.

I HAVE A SPECIFIC QUESTION

ABOUT YOUR IMPLEMENTATION OF

AN LP TO ENCODE THE

NARRATIVE PROBLEM LIST.

DO YOU HAVE SOME PERFORMANCE

NUMBERS OR DID YOU EVALUATE

THAT?

HOW WELL DOES IT PERFORM?

>> WE HAVE FOR THE CLINICIANS,

THE MOST IMPORTANT MEASURE

IS HOW FAST IT IS.

AND IT IS SECONDS.

SO IT'S VERY FAST.

THE AACCURACY IS NOT AS

GREAT AS -- IS NOT AS HIGH

AS A SAY PROBLEM MIGHT BE.

BUT AGAIN, IT ISN'T

AUTOMATICALLY ADDING IT TO

THE PROBLEM.

IT'S GIVING IT TO THE

CLINICIANS TO MAKE A

JUDGMENT ABOUT ADDING IT OR

NOT.

IT'S BEEN IN PRODUCTION ONLY

ABOUT SIX WEEKS BY THIS

EARLY DATA BUT IT'S HIGHLY

POPULAR.

AND VERY FAST.

>> SO ON THE SAME LINES, I

THINK IT'S A GREAT IDEA.

THE PROJECT -- THE IDEA IS

THAT YOU MIGHT -- THE

PHYSICIAN MIGHT SAY

DEPRESSION AND IT SAYS

RELENTING AND REMITTING

BLAH, BLAH DEPRESSION.

DOES IT RECOGNIZE THAT THERE

IS A MORE GENERAL, MORE

SPECIFIC ONE IN THE REAL

PROBLEM LIST?

>> PARDON?

>>

[INDISCERNABLE]

>> IT ATTEMPTS TO.

AND IT DOES THIS BY LOOKING

FOR EXISTING PROBLEMS WITH

THE SAME CONCEPT IN ITS FIFR

OVERLAPS.

IT ACTUALLY SHOWS THE SINCE

AND IT DOESN'T PROPOSE TO

ADD THEM IF IT FINDS

SOMETHING CLOSE.

YEAH.

>> HI.

PHILLIP RES KNICK,

UNIVERSITY OF MARYLAND.

THANK YOU FOR THAT.

GIVEN WHAT I WAS TALKING

ABOUT YESTERDAY, IT'S

INCREDIBLY EXCITING TO HEAR

THIS ESPECIALLY FROM THE

CLINICIAN PERSPECTIVE.

MY QUESTION IS ABOUT WHERE

THE STATE OF THINGS

>> NOW.

THIS IS VERY EXCITING.

ARE YOU RIGHT NOW A LONE

VOICE IN THE WILDERNESS?

ARE YOU STARTING TO SEE WHAT

YOU ARE SEEING HERE

PERCOLATE OUT THROUGH THE

COMMUNITY?

IN THE COMPUTER-ASSISTED

CODING WORLD, IBSEEN ONE

PERSPECTIVE BUT I'M ACTUALLY

WONDERING WHETHER RIGHT NOW

A BLEEDING EDGE AND NOEBS IS

DOING IT OR YOU ARE PART OF

A COMMUNITY OF OTHER PEOPLE

WHO ARE STARTING TO EXPLORER

EXPLORE THIS?

>> I THOUGHT LATTER.

I WON'T SAY IT'S WIDESPREAD

THROUGH THE UNITED STATES.

I WILL SAY THAT WITH

INSENTIVES FOR EMR USE, THE

AVAILABILITY OF ELECTRONIC

NOTES IS RISING QUITE

RAPIDLY.

THERE IS A RUMBLING

OCCURRING ACROSS THE COUNTRY

THAT WE'RE LOSING SOME OF

THE NOTES.

AND I AM BEGINNING TO SEE

THE PENDULUM SWINGING BACK

TO EXPRESSIST.

BUT I CAN'T QUOTE YOU DATA

ON THIS.

I AM NOT ALONE -- A LONE

VOICE IN THE WILDERNESS, BUT

THE GROUP OF PEOPLE WHO ARE

WORKING ON THIS IS A

RELATIVELY SMALL BAND AT THE

MOMENT.

>> WE HAVE TO FIGURE OUT HOW

TO GET THE RUMBLING LOUDER.

>> WILL THOMPSON,

NORTHWESTERN UNIVERSITY.

THE APPLICATION WHERE YOU

HAVE THE PROBLEM WITH BEING

AUTO-GENERATED GIVES THE

CLINICIAN AN OPPORTUNITY TO

EITHER ACCEPT OR REJECT THE

PROPOSALS MADE BY NLP.

ARE YOU USING THE

INFORMATION TO TRAIN THE

SYSTEM AND MAKE IT BETTER?

>> GOOD IDEA BUT NOT YET.

EARLY DAYS.

I HAVE TO SAY IT'S AN

INTERESTING ANECDOTE.

WE'RE DOING THIS AT THE SAME

TIME WE'RE IMPLEMENTING

C.P.OE IN THREE WEEKS AND

WORKING ON THE OTHER

CRITERIA.

IT'S AN EXCITING TIME BUT IT

ALSO TAKES AWAY YOUR FOCUS

FROM ALL OF THE THINGS YOU'D

LIKE TO DO.

OUR QOU IS GETTING BIGGER.

QUEUE IS GETTING BIGGER.

>> THE PROBLEM IS THAT IF

YOU GO BACK AFTER SAY FIVE

YEARS TO DO RESEARCH,

OBVIOUSLY THERE ARE SO MANY

ELEMENTS MISSING TO DO A

GOOD RESEARCH.

FOR EXAMPLE, THE IMPORTANT

THING IF YOU ARE LOOKING FOR

ANYTHING AND YOU MAY NOT

HAVE IT IN A NARRATIVE TEXT.

SO ON�TS WAY WE ADDRESSED THIS

WAS THAT IF ESPECIALLY IN AN

ACADEMIC INSTITUTION YOU CAN

HAVE A STRUCTURED NOTE BY

SOME JUNIOR RESIDENT OR

SOMEONE AND THEN YOU MAKE

THEM HOW TO THINK COG NITIVELY

ABOUT THE PATIENT AND PUT AN

IMPRESSION IF A FEW SENTENCE

WHAT'S THEY THINK ABOUT THE

PATIENT.

SO IF THAT WAY YOU SUMMARIZE

A FEW SENTENCES ABOUT A

PROBLEM THE PATIENT IS

FACING.

SO YOU HAVE BOTH A STRUCTURED

AS WELL AS A FREE TEXT NOTE.

SO HOW DO YOU -- WHAT DO YOU

THINK ABOUT THAT?

>> YEAH.

I THINK WE WILL HAVE A

MIXTURE OF NARRATIVE AND

ENCODED INFORMATION.

THE YES IS -- QUESTION IS

WHERE DOT BOUNDARIES FALL?

WHAT I CAN REPORT IS THAT

OUR CLINICIANS ARE PUSHING

THE BOUNDARY FARTHER AWAY

FROM HIGHLY ENCODED AND

CLOSER TO MORE NARRATIVE

TEXT SO THAT BOX THAT WE

PROVIDE IS BEING USED A LOT.

AND OUR REQUESTS ARE FOR TEM

PLATS THAT HAVE LESS

CLICKING AND MORE TYPING OF

VOICE.

SO THERE WILL BE A MIXTURE.

MY SENSE IS THAT PEOPLE

VALUE BOTH THE CREATORS AND

THE READERS, REALLY VALUE

THAT NARRATIVE AND SO THAT

PEND LUM IS SHIFTING MORE IN

THAT DIRECTION.

>> I'D LIKE TO THANK TOM

AGAIN.

[APPLAUSE]

>> I'D LIKE TO INTRODUCE OUR

SECOND SPEAKER FOR THIS

SESSION, DR. FRANK SONNENBERG

FROM THE UNIVERSITY OF

MEDICINE AND DENISTRY IN NEW

JERSEY.

>> OKAY, GOOD MORNING.

AND I WANT TO THANK

BLACKFORD FOR INVITING ME TO

THIS MEETING.

IT'S BEEN SE VERY

INTERESTING SO FAR.

I WASN'T SUPPOSED TO GO OFF

-- ALL THE WAY TO THE --

OKAY, I WANT TO FIRST

EXPLAIN THE PERSPECTIVE THAT

I AM SPEAKING FROM.

I AM A GENERAL INTERNIST.

I'VE BEEN PRACTICING FOR

ALMOST POYEARS, MORE THAN 30

YEARS IF YOU INCLUDE MY

RESIDENCEY.

SO I HAVE AN INTEREST IN

APPLYING THIS TECHNOLOGY TO

ME OWN PATIENT CARE.

I'M ALSO A MEDICAL DIRECTOR

OF CLINIC CHEMICAL

INFORMATION SYSTEMS FOR

MEDICAL SCOL AND A LARGE

FACULTY, MULTISPECIALTY

GROUP PRACTICE.

SO I HAVE A RESPONSIBILITY

FOR IMPLEMENTING CLINICAL

DECISION SUPPORT AND I'VE

COME UP AGAINST SOME VERY

REAL PRACTICAL CONCERNS THAT

I AM GOING TO ILLUSTRATE FOR

YOU IN MY TALK.

I AM ALSO A CO-INVESTIGATOR

IN THE CLINICAL DECISION

SUPPORT CONSORTIUM.

SO I'VE TAKEN AN INTEREST IN

THE FUTURE OF CLINICAL

DECISION SUPPORT, AND I AM

VERY WELL AWARE OF THE DPAP

BETWEEN WHERE WE ARE

CURRENTLY AND WHERE WE HOPE

IT WILL BE IN THE FUTURE, AS

BLACKFORD PRESENTED THIS

MORNING.

MY CONCLUSION IN OUR CURRENT

STATE-OF-THE-ART IS THAT

CLINICAL DECISION SUPPORT IS

STILL NOT READILY AVAILABLE

AND IT'S NOT EASY TO

IMPLEMENT WITH CURRENTLY

AVAILABLE TOOLS.

AND JUST GOING TO SHOW YOU.

THIS IS A FAIRLY TYPICAL

CLINICAL DECISION SUPPORT

THAT'S AVAILABLE.

THIS IS A COMMERCIAL LOENG

MEDICAL RECORD WHICH WE USE

ELECTRONIC.

AND I WOULD CHARACTERIZE THE

EARLY DECISION SUPPORT

THAT'S AVAILABLE AS LOW-HANGING

FRUIT.

YOU CAN SEE HERE SOMETHING

CALLED PREVENTIVE CARE

REMINDERS.

I'VE EXPANDED THAT PART OF

THE SCREEN SO YOU CAN SEE IT

IN MORE DETAIL.

IT'S JUST A LIST OF ITEMS

THAT THE EMR THINKS OR DO

FOR THIS PATIENT.

AND THEY ARE CHARACTERIZED

BY THINGS THAT CAN BE --

FIRST OF ALL, THEY ARE

DETERMINED BY VERY SIMPLE

RULES.

THE PATIENT IS A CERTAIN

AGE, AND THEY EITHER HAVE

BEEN DONE OR THEY HAVEN'T

BEEN DONE.

AND THE SYSTEM CAN'T

DISTINGUISH BETWEEN

SOMETHING THAT HASN'T BEEN

DONE FROM SOMETHING THAT

MAYBE HAS BEEN DONE BUT JUST

HASN'T BEEN RECORDED IN THE

RIGHT PLACE.

AND WHEN IT COMES TO MORE

COMPLEX DECISION SUPPORT,

THE SYSTEM REALLY HAS AT

THIS POINT RELATIVELY LITTLE

TO OFFER.

SO I THINK THAT THERE ARE A

NUMBER OF PROBLEMS THAT

PRESENT CHALLENGES TO

CLINICAL DECISION SUPPORT

CURRENTLY.

FIRST OF ALL, THE CURRENT

GENERATION OF ELECTRONIC

HEALTH RECORDS ARE NOT

DESIGNED TO OPTIMALLY

ORGANIZE PATIENT INFORMATION.

THEY'RE STILL VERY HEAVILY

DOCUMENT-BASED AND EPISODE

BASED SO THEY DON'T CAPTURE

VERY WELL SOME OF THE

ELEMENTS OF THE PATIENTS'

DATA THAT ARE LONGITUDENAL

AND THAT CARRY OVER FROM ONE

EPISODE OF CARE TO THE NEXT.

AND ALSO, AS BLACKFORD

POINTED OUT THIS MORNING,

PATIENTS TEND TO SEE A

NUMBER OF DIFFERENT

PROVIDERS.

THEY'RE NOT ALWAYS AT THE

SAME INSTITUTION.

AND SO OUR TOTAL MEDICAL

RECORD IS COMPRISED OF

INFORMATION THAT'S DERIVED

FROM A NUMBER OF DIFFERENT

SOURCES.

IN AND PARTICULAR HISTORICAL

FACTS ARE NOT READILY

ACCESSIBLE.

IT'S NOT EASY TO GO IN AND

SEE IF A PARTICULAR THING

HAS OCCURRED IN THE PAST.

AND CERTAIN KINDS OF DATA --

THIS IS MY OWN PERSONAL

OBSERVATION -- ARE NOT

CAPTURED VERY WELL.

ONE IS SYMPTOMS.

THESE TEND TO BE IN THE

NARRATIVE DESCRIPTIONS THAT

WE'VE BEEN TALKING ABOUT.

PHYSICAL FINDINGS.

THEY MAY BE DISCREETIZED BUT

SOME IN SOME CASES ONLY IN

PARTICULAR FIELDS.

SO THERE MAY BE A VARIABLE

THAT SAYS CARDIOVASCULAR

EXAM AND IT MAY BE A SPRING

OF WORDS BUT IT DOESN'T

ALLOW YOU TO EXTRACT FROM

THAT STRING WHAT THE ACTUAL

FINDINGS WERE.

PREVIOUS PROCEDURES ARE NOT

WELL-DOCUMENTED AS DISCREET

DATA.

AND ALSO MEASUREMENT

SUPPORTING DIAGNOSES.

I AM GOING TO ILLUSTRATE

SOME OF THESE THINGS IN A

MOMENT.

SO IN ORDER TO APPLY

DECISION SUPPORT OF THE TYPE

THAT WE'VE BEEN HEARING

ABOUT THIS MORNING, WHERE

YOU ARE APPLYING DECISION

RULES, TRYING IT APPLY

GUIDELINES, YOU HAVE TO HAVE

VARIABLES IN ORDER TO DECIDE

HOW TO EXECUTE THOSE RULES.

AND SUPPLYING VARIABLES

AUTOMATICALLY FROM EMR

REQUIRES CAPTURING THEM

DURING ROUTINE CLINICAL

CARE.

AND THEY HAVE TO BE

REPRESENTED SOMEHOW AS

DISCREET DATA.

YOU HAVE TO IDENTIFY WHAT

THE PERTINENT VARIABLES ARE.

AND ANOTHER PROBLEM THAT WE

HAVEN'T DISCUSSED A LOT AT

THIS MEETING BUT IT'S STILL

A BIG PROBLEM.

EMRS IN GENERAL STILL DO NOT

EMPLOY STANDARDIZED

TERMINOLOGY.

THAT'S BEGINNING TO CHANGE,

BUT WE'RE STILL NOT THERE

YET.

THE SYSTEM WE USE USES A

COMPLETELY PROPRIETARY

SCHEME.

AND IN FACT, IN OUR WORK AS

A DEMONSTRATION SITE FOR THE

CLINICAL DECISION SUPPORT

CONSORTIUM, WE HAVE A WHOLE

SUBPROJECT TO TRANSLATE OUR

DATA INTO THESE STANDARDIZED

CODING THAT THE KNOWLEDGE

SERVER REQUIRES.

THE BOTTOM LINE IS THAT MUCH

CRITICAL INFORMATION IS NOT

CAPTURED IN EMR VARIABLES.

SO I AM GOING TO WALK YOU

THROUGH A VERY DETAILED

EXAMPLE THAT WE ANALYZED AS

PART OF OUR WORK IN

IMPLEMENTING CLINICAL DPLINS,

AND I AM APPROACHING THIS

FROM THE OPPOSITE

PERSPECTIVE OF WHAT YOU

HEARD YESTERDAY.

GUIDELINES.

YESTERDAY, PEOPLE WERE

TALKING ABOUT LOOKING AT

NARRATIVE TEXT AND ASKING

WHAT CAN WE EXTRACT FROM IT?

I AM APPROACHING IT FROM THE

OPPOSITE POINT OF VIEW,

WHICH IS WHAT DATA DO WE

NEED AND WHERE CAN WE FIND

IT IN THE MEDICAL RECORD?

SO I LOOKED AT TWO EXAMPLE

GUIDELINES.

ONE IS THE J AND C SEVEN,

WHICH IS THE STANDARD AND

MOST HIGHLY ACCEPTED

GUIDELINE FOR MANAGEMENT OF

HYPERTENSION AND ALSO THE

NCEP CHOLESTEROL TREATMENT

GUIDELINES.

BOTH OF THESE -- WE SELECTED

THESE FOR TWO REASONS.

ONE IS THAT THEY'RE VERY

WELL-ACCEPTED.

THEY'RE NOT CONTROVERSIAL AT

ALL.

AND THEY'RE BOTH FAIRLY

COMPLEX.

THEY'RE EXTRAORDINARILY

COMPLEX, CONSIDERING HOW

COMMONLY THEY NEED TO BE

IMPLEMENTED.

SO WE LOOKED AT THE

VARIABLES AND WE IDENTIFIED

THE FOLLOWING TYPES OF

VARIABLES THAT ARE NEEDED TO

IMPLEMENT THESE GUIDELINES.

FIRST OF ALL, THERE ARE

SIMPLE VARIABLES WHICH I

DEFINED AS SINGLE

OBSERVATIONS, SOMETHING LIKE

A KRISTOLOLIC BLOOD

PRESSURE.

AND THERE ARE CALCULATED

VARIABLES, THINGS THAT ARE

NOT OBSERVED DIRECTLY BUT

ARE CALCULATED FROM OTHER

OBSERVATIONS.

SO A SIMPLE EXAMPLE IS AGE

THAT CAN BE CALCULATED FROM

THE BIRTH DATE OR A BODY

MASS INDEX WHICH IS

CALCULATED FROM THE HEIGHT

AND THE WEIGHT.

THEN THERE ARE OTHER COMPLEX

VARIABLES THAT ARE DEFINED

IN TERMS OF OTHER VARIABLES

OR OBSERVATIONS SUCH AS IN

THESE GUIDELINES TERMS SUCH

AS ELEVATED CORONARY RISK OR

MOT BOLIC SYNDROME APPEAR.

AND THEY'VE ILLUSTRATED AT

THE BOTTOM METABOLIC STROM

IS DEFINED IN TERMS OF

SEVERAL OTHER TERMS, ONE OF

WHICH OBDOMINAL OBESITY IS

DEFINED ITSELF IN TERMS OF

OTHER VARIABLES SUCH AS

ABDOMINAL CIRCUMFERENCE AND

DISCREET OBSERVATIONS.

AND IN LOOKING AT THESE TWO

GUIDELINES TOGETHER, WHICH I

THINK MOST PRIMARY PHYSICIANS

WOULD BE EXECUTING THESE

GUIDELINES MANY TIMES A DAY

EVERY DAY THEY SEE PATIENTS.

WE IDENTIFIED A TOTAL OF 169

DIFFERENT DISTINCT TERMS

THAT HAD TO BE SUBSTANTIATED

IN ORDER TO APPLY THESE

GUIDELINES.

THERE ARE 42 VARIABLE THAT'S

COULD BE CHARACTERIZED AS

DIRECT OBSERVATION.

THERE WERE 40 THAT

CONSTITUTED HEALTH ISSUES SO

THAT DIAGNOSIS THAT'S WOULD

BE ON A PATIENT'S PROBLEM

LIST.

FOUR WERE MEDICATION-RELATED.

THERE WERE 15 VARIABLE

THAT'S WERE CATEGORIZED AS

-- WE CATEGORIZED THESE AS

NEEDING TO ASK A CLINICIAN.

THAT MEANS THERE ARE THINGS

THAT WERE NOT ROUTINELY

CAPTURED AS PART OF A

CLINICAL ASSESSMENT BUT WERE

NEEDED IN ORDER TO IMPLEMENT

THIS GUIDELINE.

THERE WERE 32 VARIABLES THAT

REQUIRED CALCULATION AND 36

VARIABLE THAT'S WERE COMPLEX

TERMS, THAT IS, THINGS THAT

WERE DEFINED IN TERMS OF

OTHER VARIABLES.

SO WE FOUND THAT IN EMR ONLY

51% OF THE VARIABLES WERE

SIMPLE TERMS COLLECTED IN

THE FORMS NEED FOR

APPLICATION OF THE

GUIDELINES.

40% OF THEM DEPEND ON OTHER

SIMPLE TERMS AND THERE WERE

A LOT OF UNDEFINED TERMS,

THINGS SUCH AS CHILDREN,

ADOLESCENTS, ELDERLY, END

STAGE HEART DISEASE THAT A

CLINICIAN WOULD NEED TO

UNDERSTAND IN ORDER TO APPLY

THE GUIDELINE BUT ALL ABOUT

IT WERE NOT DEFINED

PRECISELY.

AND SOME TERMS WERE DEFINED

ONLY OUTSIDE OF THE

GUIDELINE.

THERE WAS ONE TERM ELEVATED

CORONARY RISK THAT WAS

REFERENCED BUT YOU ACTUALLY

HAD TO LOOK UP IN ORDER TO

FIND IT.

AND SO I WANTED TO FOCUS ON

JUST ONE OF THESE COMPLEX

VARIABLES THAT COMES FROM

THE GUIDELINE CALLED TARGET

ORGAN DAMAGE.

AND SO MANY CLINICIANS MAY

KNOW WHAT THAT MEANS, BUT IN

ORDER TO IMPLEMENT THIS

GUIDELINE, YOU HAVE TO KNOW

VERY PRECISELY.

AND SO IF YOU LOOK IT UP,

THERE IS A TABLE IN THE

GUIDELINE, AND IT CONSISTS

OF ALL OF THE FOLLOWING OR

ANY OF THE FOLLOWING.

LEFT VENTRICULAR

HYPERTROPHY, CHRONIC KIDNEY

DISEASE AND SO ON.

AND IN ADDITION, NOT ALL OF

THESE THINGS ARE SIMPLE

TERMS EITHER.

FOR EXAMPLE, LEFT

VENTRICULAR HYPERTROPHY IS

RARELY GOING TO BE RECORDED

IN EMR AS A TERM.

THAT DEPENDS ON SOME OF

THESE OTHER VARIABLES SUCH

AS THICKNESS OR LBH BY EKG

CRIER TERIA.

ANOTHER EXAMPLE IS

PERIPHERAL ARTERIAL DISEASE,

AND IT'S DEFINED IN THE

GUIDELINE AS CONSISTING OF

ONE OF THESE FOLLOWING

THINGS, INCLUDING AT THE

BOTTOM THAT ONE WAS JUST

DESCRIBED AS OTHERS.

IT WAS NOT SPECIFIED.

SO THESE VARIABLES COME FROM

DIFFERENT PARTS OF THE

RECORDS SORMENT FOR EXAMPLE,.

SO FOR EXAMPLE, I'VE

HIGHLIGHTED HERE THE

ENTITIES THAT WOULD BE

RETRIEVED FROM THE PATIENT'S

COMPREHENSIVE HISTORY,

WHETHER THEY HAD A PRIOR

STROKE OR NOT AND SO ON.

PRIOR CORONARY DEVASCULARIZATION 

DEVASCULARIZATION.

OTHERS DEPEND ON THE

PHYSICAL EXAM -- HEART

FAILURE, FOR EXAMPLE, STROKE

COULD BE SOMETHING DETECTED

FOR THE FIRST TIME ON

PHYSICAL EXAM, PERIPHERAL

ARTERIAL DISEASE CAN BE BOTH

HISTORICAL AND A PHYSICAL

FINDING.

AND RETINA OPTHY DEPENDS ON

THE PHYSICAL ASSESSMENT.

SOME THINGS DEPEND ON JRJSDS

SUCH AS ANGINA OR SYMPTOMATIC

HEART FAILURE AND SOME

THINGS DEPEND ON DIAGNOSTIC

TESTS.

THINGS I'VE HIGHLIGHTD IN

RED HERE.

THAT IS, SOMEBODY HAS LEFT

VENTRICULAR HYPERTROPHY OR

CHRONIC KIDNEY DISEASE,

THERE ARE DIAGNOSTIC TEST

FORS HEART FAILURE AND

STROKES.

SO SOME OF THESE ENTITIES

ARE CAPTURED BY MORE THAN

ONE MODALITY.

IN THE CASE OF PERIPHERAL

ARTERIAL DISEASE, CHRONIC

STENOSIS IS DETECTED BY

TESTS.

THE QUESTION IS WHERE ARE

THESE THINGS IN THE

CLINGICAL RECORD?

LET'S TAKE LEFT VENTRICULAR

HYPERTROPHY AND CONGESTIVE

HEART FAILURES AS EXAMPLES.

THE DATA ON LDH COMES EITHER

FROM AN ECOCARDIOGRAM.

YOU CAN SEE THERE ARE TWO

VARIABLE THAT'S I'VE

HIGHLIGHTED.

LEFT VENTRICULAR, POSTERIOR

WALL THICKNESS.

IN THIS CASE THEY'RE NORMAL

BUT THAT'S WHERE ONE WOULD

HAVE TO LOOK.

ANOTHER PLACE TO LOOK WOULD

BE IN THE EKG REPORT.

NOW, THIS REPRESENTS THE

DATA AS WE RECEIVED.

THIS IS FROM AN ACTUAL CASE.

THE ECOCARDIOGRAM WAS DONE

IN AFAST PHYSICIAN'S OFFICE

WHO IS NOT PART OF OUR

PRACTICE, AND THIS REPORT,

WHICH WE RECEIVED, YOU KNOW

THE LD WALL THINGS IS A

DISCREET VARIABLE.

IT WASN'T CAPTURED THAT WAY

BY OU SYSTEM BECAUSE IT'S

PART OF A TEXT REPORT.

AND THE COMMENT ABOUT THE

FRACTION WHICH RELATES TO

THE PATIENT HAVING HEART

FAILURE, IS NOT EVEN

RECORDED AS DATA IN THE

REPORT.

IT'S RECORDED AS A NARRATIVE

CONCLUSION.

AND THE ONLY WAY THAT WE

COULD POSSIBLY CAPTURE THAT

IS BY FINDING THIS TEXT AND

REVIEWING IT.

NOW, SIMILARLY, THE EKG.

THIS IS A COMPUTER-INTERPRETED

EQG.

BUT THE WAY WE RECEIVE IT IN

OUR ELECTRONIC MEDICAL

RECORD IS A SCANED REPORT.

SO THE TEXT IS THERE, BUT

THE FINDINGS, IF LEFT

VENTRICULAR HYPERTROPHY WERE

PRESENT, IT COULD ONLY BE

EXTRACTED FROM THE TEXT IN

THAT REPORT.

HERE IS ANOTHER EXAMPLE.

AGAIN, THIS IS FROM THE SAME

CASE.

THIS PATIENT HAD CORONARY

BYPASS SURGERY, AND THE ONLY

RECORD OF IT, THE ONLY

ACTUAL DISCREET RECORDING OF

THE PRIOR SURGERY IN THE

CHART WAS IN A LETTER FROM A

CONSULTANT.

THIS IS FROM THE PHYSICIAN

WHO ACTUALLY PERFORMED THE

SURGERY BECAUSE IT WAS DONE

IN AN OUTSIDE HOSPITAL.

THE OPERATIVE REPORT WAS NOT

AVAILABLE IN OUR MEDICAL

RECORD.

SO ONE WOULD NOT HAVE THIS

INFORMATION WITHOUT THE

ABILITY TO EXTRACT IT FROM

THIS LETTER.

AND IT ALSO PROVIDES THE

DATE THAT THE TEST WAS DONE

OR THAT THE SURGERY WAS DONE.

FOR EVIDENCE OF PERIPHERAL

VASCULAR DISEASE, WE TALKED

ABOUT DOING A -- THIS COMES

FROM A DUPE PLEX REPORT.

AGAIN, THESE ARE NUMBERS.

THEY COULD BE DISCREET DATA

ELEMENTS.

BUT WE RECEIVED IT AS A TEXT

TEXT-BASED REPORT.

IT TELLS US EXACTLY WHAT THE

STENOSIS IS, BUT AGAIN, THIS

WOULDN'T BE AVAILABLE TO ANY

OF OUR DECISION SUPPORT

UNLESS SOMEBODY EXTRACTED

THIS INFORMATION FROM THE

NARRATIVE TEXT AND PUT IT

INTO A VARIABLE.

NOW, THIS IS ONE THAT WAS

PARTICULARLY VEXING TO ME.

THIS IS AN ACTUAL -- THIS IS

A SCANED DIRECTLY FROM THE

GUIDELINE.

THEY TALK ABOUT PATIENTS

WITH SYMPTOMIC VENTRICULAR

DYSFUNCTION.

WE KNOW HOW TO DETERMINE

VENTRICULAR DYSFUNCTION BUT

HOW DO YOU DETERMINE IF IT'S

SYMPTOMATIC?

IT COMES FROM NARRATIVE

NOTES, IN THIS CASE A LETTER

FROM THE PATIENT'S

CARDIOLOGIST.

SHE'S DOING WELL.

WITHOUT ANY SHORTNESS OF

BREATH.

SO WE CAN CONCLUDE THAT

SHE'S NOT SYMPTOMATIC.

BUT THE ONLY WAY TO KNOW

WOULD BE TO READ THIS NOTE.

THERE IS NO VARIABLE IN OUR

EMR THAT SAYS SYMPTOMATIC

CONGESTIVE HEART FAILURE.

SO IN CONCLUSION, APPLYING

DECISION RULES REQUIRES

SUBSTHANGS OF A --

SUBSTANTIAL AND MANY OF

THESE VARIABLES ARE NOT

CAPTURED AS DISCREET DATA

EVEN IN OUR ELECTRONIC

HEALTH RECORDS.

AND MANY IMPORTANT DATA

ITEMS ARE AVAILABLE ONLY AS

TEXTULE ENTRIES OR NARRATIVE

REPORTS.

ONE APPROACH TO THIS IS

INCREASING THE DISCREET DATA

CAPTURE BY CLINICIANS.

I TEND TO AGREE WITH THE

CONSENSUS THAT THAT'S NOT

TOTALLY REALISTIC.

I THINK THERE ARE LIMITS AS

TO HOW MUCH WE CAN CAPTURE

DISCREETLY.

BUT CURRENT DECISION SUPPORT

SYSTEMS CANNOT MAKE USE OF

ALL THE INFORMATION THAT WE

HAVE.

SO I THINK FOR THE FORESEEABLE

FUTURE, NATURAL LANGUAGING

PROCESSING WILL BE THE ONLY

WAY TO CAPTURE THIS DATE Y

FROM THE ELECTRONIC HEALTH

RECORD.

SO THAT CONCLUDES MY TALK.

[APPLAUSE]

>> WE HAVE TIME FOR ONE VERY

QUICK QUESTION, IF ANYONE

HAS ONE.

AS WE SWITCH OVER

PRESENTATIONS.

OKAY, WE'LL MOVE ON TO OUR

THIRD PRESENTER.

DR. JAMES WALKER FROM

GEISINGER HEALTH SYSTEM.

>> THANK YOU.

IT'S A PLEASURE TO BE HERE

TODAY.

THANK YOU, BLACKFORD, FOR

THE INVITATION AND THANK YOU,

DR. LINBERG FOR HOSTING US.

IT'S ALWAYS A PLEASURE TO

WORK WITH THE HIGHEST

FUNCTIONING UNIT OF THE

AMERICAN GOVERNMENT.

HENRY ADAMS SAID THAT THE

ONLY OBLIGATION OF THE NOVEL

IS TO BE INTERESTING.

AND I THINK THAT'S THE ONLY

OBLIGATION FOR PRESENTATIONS,

SO I'LL TRY.

I WANT TO THANK JOHN BAYER,

WHO IS MY UNINDICTED

CO-CONSPIRATOR AND THE

PERSON THAT MADE ALL OF THIS

HAPPEN.

I'M THE CHIEF HEALTH

INFORMATION OFFICE AT

GEISINGER AN INTERNIST, A

COGNITIVE PSYCHOLOGIST AND A

STUDENT SYSTEMS ENGINEER.

AT GEISINGER, WE OPERATE A

COMPLETELY INTEGRATED IN

PATIENT, OUTPATIENT EHR USED

BY ALL CLINICIANS FOR

EVERYTHING.

WE HAVE A NETWORKED PHR THAT

ABOUT 38% OF THE PATIENTS

WITH WHOM WE HAVE SOME KIND

OF ONGOING RELATIONSHIP USED

AND DO THINGS LIKE RECEIVING

CLINICAL DECISION SUPPORT

DIRECTLY THROUGH THAT.

AND MANY OF WHOM OF COURSE

ACTIVATE IT AND GET THE

RESPONSES BACK WITHOUT ANY

OTHER HUMAN, EXCEPT THE LAB

INTERVENING.

WE LEAD HEALTH INFORMATION

EXCHANGE THAT LINKS TOGETHER

SEVERAL HUNDRED FACILITIES

AND I THINK 20 DIFFERENT

COMPANIES AND LEAD A BEACON

COMMUNITY, WHICH IS ENGAGED

IN TRYING TO LEAD AND

EXECUTE COMMUNITY-WIDE

EVIDENCE-BASED CARE

PROCESSES ACROSS FIVE

COUNTIES OF RURAL

PENNSYLVANIA.

SO I AM GOING TO TALK ABOUT

INFORMED CARE PROCESS

IMPROVEMENT, INCLUDING

CLINICAL DECISION SUPPORT.

IT'S OUR BELIEF, WE THINK WE

HAVE LEARNED THAT CLINICAL

DECISION SUPPORT ONLY IS

VERY USEFUL WHEN IT'S SET IN

THE LARGER CONTEXT OF CARE

PROCESS IMPROVEMENT.

AND THIS IS THE WAY WE THINK

ABOUT THAT.

THE GOAL IS NOT HELPING

PEOPLE MAKE BETTER DECISIONS

-- WHOEVER THEY ARE,

PATIENTS OR SOMEONE ELSE ON

THE PATIENT'S CARE TEAM, BUT

EXECUTING 100 PERCENT

PROCESSES.

AND THE ONLY NUMBER YOU NEED

TO KNOW ABOUT AMERICAN

HEALTHCARE IS THE ONE THAT

BLACKFORD CITED EARLIER,

55%.

RIGHT NOW -- AND THIS WAS A

VERY CAREFULLY DONE STUDY.

I HAVE NEVER HEARD ANY

QUIBBLE WITH THEIR METHODOLOGY.

THEY IDENTIFIED 30 EVIDENCE

EVIDENCE-BASED INTERVENTIONS

THAT APPLY TO ADULTS.

THEY SURVEYED ACROSS SEVERAL

CITIES AND COMMUNITIES IN

THE UNITED STATES AND CAME

UP WITH THIS APPALLING

NUMBER.

AND SO WHAT OUR GOAL IS IS

TO MAKE SURE THAT EVERY

PATIENT GETS EVERY

INTERVENTION AOFFERED TO

THEM AND IF THEY ELECT TO

TAKE IT, EXECUTED FLAWLESSLY

100% OF THE TIME.

BY THE WAY, THAT WOULD TAKE

CARE OF HEALTHCARE

DISPARITIES AS A SIDE

BENEFIT.

SO WHAT WHAT DOES THAT LOOK

LIKE FOR GEISINGER?

WE HAD SURGEONS SAT DOWN AND

SAID LET'S IDENTIFY

EVERYTHING HAS THAT'S BEEN

PROVEN IN A GOOD CLINICAL

TRIAL TO DECREASE THE RISK A

PATIENT HAVING AN ADVERSE

EFFECT WHEN THEY HAVE ELECTIVE

OPEN HEART SURGERY.

THEY FOUND 38 OR 40 -- I CAN

NEVER REMEMBER THE NUMBER.

AND THEY SAID OKAY, WE'RE

GOING TO COMMIT OURSELVES TO

DOING 100 PERCENT.

EVERY PATIENT GETS 100

PERCENT OF THOSE EVERY

SURGERY.

FIRST WE LOOKED AT OUR

EXISTING PERFORMANCE, WHICH

GOD HELP US, WAS BETTER THAN

NATIONAL BENCHMARK AND IT

WAS 59%.

SO THE TEAM -- AND THIS IS A

TEAM OF SURGEONS AND

OUTPATIENT CARDIOLOGISTS AND

PCPS AND POST DOC NURSES AND

PREOP NURSES AND EVERYBODY

ELSE INVOLVED IN THE PROCESS

CARDIAC REHAB DESERVE END TO

END -- REDESIGNED THE

PROCESS AND THEN DESIGNED

HEALTH IT TO REMIND

EVERYBODY WHAT THEIR PART OF

THE PROCESS WAS AND LET THEM

KNOW WHETHER IT HAD BEEN

DONE OR NOT.

WITHIN THREE MONTHS OF

IMPLEMENTATION WE WERE AT

100%.

WE'VE HAD BLIPS SINCE THEN

BUT THE -- THIS IS NOT THE

MOST RECENT DATA -- OUR RUN

RATE OVER THE LAST 36 MONTHS

IS 98%.

SO THAT'S WHAT WE'RE AFTER,

AND THEN CLINICAL DECISION

SUPPORT IS ANYTHING THAT

HELPS US GET THERE.

ALERT FATIGUE, JUST SO YOU

HAVE THE RIGHT DEFINITION OF

IT BECAUSE YOU PROBABLY

HAVEN'T SEEN THIS ONE

BEFORE.

ALERT FATIGUE IS DECISION

SUPPORT PROVIDED TO SOMEONE

WHO IS NOT COMMITTED TO A

100% PROCESS.

OUR EXPERIENCE IS THAT AFTER

EVERYBODY STACKS HANDS AND

SAYS WE'RE GOING TO DO THIS,

WE FIND THAT PEOPLE ACTUALLY

SCORES OF PHYSICIANS SENDING

US AN EMAIL AND SAYING NEED

TO PUT US OFF STOP ON THIS

PARTICULAR PROCESS BECAUSE I

KEEP CLOSING THE NOTE BEFORE

I REMEMBER TO DOCUMENT MED

REC RECONCILIATION, FOR

INSTANCE.

AND OBVIOUSLY, THE HEALTH IT

IS JUST AN EXPENSIVE MESS.

AND AS THE LITERATURE HAS

ABUNDANTLY DEMONSTRATED,

IT'S UNLIKELY TO IMPROVE

QUALITY EFFICIENCY OR

SATISFACTION.

AND OF COURSE, IF YOU ARE

ACTUALLY TRYING TO EXECUTE

100 PERCENT PROCESSS AND

THAT ENGAGEMENT RUNS ACROSS

ORGANIZATIONS, THEN IT'S

INDISPENSEIBLE AND THE

ARGUMENT GOES AWAY.

SO WHAT IS CARE PROCESS

IMPROVEMENT, INCLUDING

SUPPORT LOOK LIKE TO US?

FIRST OF ALL, THE TOTAL GOAL

IS TO IMPROVE EDUCATION, AS

YOU PROBABLY KNOW.

IT'S ESTIMATED THAT

HEALTHCARE ACCOUNTS FOR

ABOUT 10% OF THAT.

SO WE SHOULD BE A LITTLE BIT

MODEST AS WE START OUT.

BUT GRANTED THAT 10 PERCENT

IS WHAT WE CONTROL, WHAT IS

OUR TAKE ON IT?

WELL, WE THINK WE NEED TO

SUPPORT SHARED AND CLINICIAN

SENSE-MAKING, WHICH IS EVERY

CLINICIAN AND ALL PATIENTS

THAT HAVE SERIOUS PROBLEMS

KNOW.

IT'S AN APPROXIMATE,

INCREDIBLY HIGH ORDER

INTELLECTUAL TASK ON BOTH

THE PART OF THE CLINICIAN

AND THE PATIENT.

SUPPORT KNOWLEDGE

ACQUISITION, WHICH MAY BE

PART OF CLINICAL DECISION

MOTHER.

SHOULD SUPPORT SHARE-DECISION

SHARE-DECISION-MAKING, WHICH

IS A LITTLE DIFFERENT ANIMAL

THAN WHAT WE CALL CLINICAL

SUPPORT OFTEN AND TRANSLATE

THOSE DECISIONS INTO COST

COST-EFFECTIVE PROCESSES.

AND ONE OF THE THINGS THAT

WE DO TO TRY TO ACHIEVE 100

PERCENT PROCESSES IS WHAT I

AM NOT SURE IF YOU WOULD

CALL CLINICAL DECISION

SUPPORT OR NOT OF WHEN THOSE

-- WHEN THAT CARDIOLOGY TEAM

GOT TOGETHER AND DID

ELECTIVE OPEN HEART SURGERY,

WHEN WE DO CONGESTIVE HEART

FAILURE AND PERINATAL.

WHAT WE DO IS EMBED LOTS OF

DECISIONS INTO THE PROCESS.

SO IF WE'VE DECIDED THAT

EVERYONE WITH CKD, CHRONIC

KIDNEY DISEASE OR HAS A

FILTRATION RATE LESS THAN OF

0 SHOULD HAVE ONE VISIT WITH

A IF HE PHROLOGIST, THAT IS

BUILT FOO THE SYSTEM AND NO

HUMAN HAS EVER BOTHERED WITH

THAT.

NEPHROLOGIST.

IF WE OFFERED DECIDED THAT

ONE SHOULD BE OFFERED A

MAMMOGRAM EVERY YEAR.

THAT GOES TO THE PATIENT AND

IF THE PATIENT

SELF-SCHEDULES, GETS THE

AUTOMATIC REMINDERS, GETS IT

DONE AND IT'S NORMAL.

SHE GETS A MESSAGE BACK FROM

HER DOCTOR SAYS HALLELUJAH,

IT'S NORMAL.

NO GEISINGER CLINICIAN KNOWS

ANYTHING HAPPENED.

SO A LOT OF WHAT WE CALL

CLINICAL DECISION SUPPORT IS

EMTHE BEDED IN ALL -- EMBEDD

IN ALL KINDS OF SWIS THAT

CLINICIANS ARE SO THAT

CLINICIANS ARE LARGELY NOT

IRRITATED BY IT.

AND OF COURSE, WE'VE GOT TO

EXECUTE THE PROCESSES

RELIABLEABLY.

HOW WITH DO WE USE IT

CURRENTLY?

WHAT WE DO IS WHAT WE CALL

CLOSING CARE GAPS, WHICH WE

NEVER SAY IN PUBLIC BECAUSE

WE THINK THAT WOULD SOUND

APPALLING TO PATIENTS.

BUT WHAT WE DO IS WORK ON

CARE PROCESS RELIABILITY.

SO FOR INSTANCE WE WANT TO

REDUCE THE TIME FROM AN

ABNORMAL MAMMOGRAM TO BIOPSY

AND FROM BIOPSY TO TREATMENT

AND THE ACTUAL PERFORMANCE

STANDARD IS IF YOU HAVE AN

ABNORMAL MAM GRAM, YOU NEED

TO 100% PROCESS BE OFFERED A

VISIT WITHIN 24 HOURS WITH

THE BREAST CLINIC, WITHIN 12

HOURS OF THE TIME THAT

ABNORMAL MAMMOGRAM IS

REPORTED.

IF NOT, SOMEBODY GETS A

NASTY GRAM, NOT THE PATIENT,

OF COURSE.

WE DEFINE THE POSITIVES.

WHY?

BECAUSE THE 19th CENTURY

HISTOLOGY INFORMATION SYSTEM

WE USED DOESN'T UNABLE THE

HISTOLOGY -- ENABLE THE

HISTOLOGY TO FLIP AN

ABNORMAL FLAG ON IT.

WE'VE DONE THE ANALYST --

ANALYTICS AND THERE IS NO

SENSE DOING THE NLP IF WE

DON'T DO THE SIGNAL AND AN

ACTIONABLE, USEFUL WAY OF

DOING IT.

BUT ONCE THAT'S DONE, WE'LL

EXECUTE THAT ONE AND THEN AS

YOU CAN SEE, WE EXPECT TO DO

THAT WITH A LOT OF OTHER

PROBLEMS.

YOU WILL NOTICE THE PROCESS

REALTIME.

ONE OF THE THINGS THAT'S

IMPORTANT FOR US IS TO

UNDERSTAND HOW FAST

SOMETHING HAS TO HAPPEN FOR

IT TO BE USEFUL.

AND IT TURNS OUT THAT LOTS

OF THINGS DON'T HAVE TO BE

IN REAL TIME.

THEY HAVE TO BE FAST ENOUGH

OR WHATEVER THE PROCESS IS

AND SO THAT'S ONE OF THE

WAYS WE TRY TO BE EFFICIENT

IS NOT -- IS TO BE SMAMART

ABOUT WHAT THE TURN AROUND

TIME FOR DIFFERENT KINDS OF

IT IS.

THAT OBVIOUSLY IS AN

ADVANTAGE BECAUSE THEN WE

CAN PROCESS IT OFF-LINE.

WE DON'T HAVE TO HAVE

SUBSECOND SCREEN FLIPS,

WHICH OUR USERS RIGHTLY

DEMAND OF US AND OBVIOUSLY,

THIS IS NOT FAST ENOUGH TO

BE POINT OF CARE.

WE DON'T THINK IT MATTERS,

ACTUALLY, FOR REASONS THAT

I'LL TALK ABOUT IN A MINUTE.

OTHER POSSIBLE USE CASES.

YOU COULD IMAGINE TRACKING

ALL OF THESE DIFFERENT

PROCESS MEASURES TO MAKE

SURE THAT WE'RE TAKING CARE

OF PATIENTS APPROPRIATELY.

SO THIS IS WHAT NLP CARE

PROCESS IMPROVEMENT LOOKS

LIKE TO US.

OFTEN, PLACING A CLINICAL

DECISION INTO BUSINESS

PROCESS MANAGEMENT.

YOU ARE ALL AWARE OF THAT.

EVERYWHERE IN THE WORLD

EXCEPT HEALTHCARE THERE IS A

DISCIPLINE OF TAKING

PROCESSES, BUILDING THEM

INTO SOFTWARE SYSTEMS THAT

THEN MANAGE MAKING SURE THAT

THEY GET DONE OR THAT THE

RIGHT PERSON KNOWS THAT IT'S

THEIR TURN TO DO SOMETHING.

THEN USING AN LP AGAIN SO

THAT ONCE THE NLP IS DONE,

IT STILL HAS TO FIT IN A

PROCESS AND AN EXCUSE SYSTEM

THAT'S GOING TO MAKE IT

HAPPEN AND THEN GET TO 100

PERCENT PROCESS.

SO WHY NOT POINT OF CARE?

WE DON'T CARE THAT THAT

THING I TOLD YOU ABOUT ISN'T

GOING TO BE POINT OF CARE.

THE REASON IS BECAUSE WE

DON'T WANT ANYTHING POINT OF

CARE THAT DOESN'T HAVE TO BE

THERE.

IN OUR VIEW, THE PATIENT'S

TIME WITH THE PHYSICIAN

OUGHT TO BE FOCUSED ON THE

VERY HIGHEST ORDER, VERY

MOST IMPORTANT, VERY MOST

COMPLEX INTELLECTUAL TASKS

AND REMEMBERING THAT IT'S

TIME FOR THE MAMMOGRAM IS

NOT ONE OF THOSE.

AND SO OUR MODEL IS TO TAKE

EVERYTHING OUT OF THE POINT

OF CARE THAT COULD BE DONE

SOMEWHERE ELSE AS WELL OR

BETTER SO THAT THE PATIENT

AND THE DOCTOR CAN LOOK IN

EACH OTHER'S EYES AND DO

THAT SENSE-MAKING, DO THAT

SHARED DECISION-MAKING AND

NEGOTIATION OF WHAT THE CARE

PLAN IS GOING TO LOOK LIKE

AND SO THE PATIENT WALKS

OUT, A MY DOCTOR LOVES ME

AND KNOWS ME AND CARES WHAT

IS HAPPENING TO ME.

ONE OF THE THINGS IS THE

NETWORK-TO-PATIENT HEAT

RECORD IS ENORMOUSLY

VALUABLE FOR THIS.

LOTS OF PATIENTS IN OUR

UNDERSERVED, UNDEREDUCATED,

POOR, OLD, IMMOBILE --

UNDEREMPLOYED POPULATION,

ABOUT PO% OF OUR PATIENTS

ARE THRILLED TO ATTACK CARRY

OF ALL THIS STUFF

THEMSELVES.

WE SEND THEM ALERTS.

SAY IT'S TOM FOR AREA

DIABETES BLOODWORK.

WE DON'T CALL IT THAT, OF

COURSE.

AND MANY OF THEM GO TO THE

LAB, GET IT DONE, GET A

REPORT THAT GOES BACK

ELECTRONICALLY TO THE DOCTOR.<#�%P�j"

THEY SEND AN EMAILhmJ� MESSAGE

TO THE PATIENT AND THAT'S

DONE.

AND SO THAT IS ONE OF THE

THINGS THAT HELPS US UNLOAD

THE POINT OF CARE.

ONE THING WE FIND IS

AGGREGATION OF APPROPRIATE

INFORMATION TAKES TIME.

OUR DATA WAREHOUSE HAS 13

DATABASES FEEDING IT, AND

NOT ALL OF THOSE WORK ON THE

SAME TIME SCALE, AND SO NOT

WORRYING ABOUT POINT OF CARE

HELPS US TO AGGREGATE THAT

INFORMATION.

DO THE KIND OF HEAVY DUTY

DECISION ENGINE WORK THAT IS

OFTEN REQUIRED TO DO A GOOD

JOB THEN GET IT TO THE RIGHT

PERSON.

ARE YOU GOING TO FLIP UP

THAT FIVE MINUTES?

THAT'S REALLY SLICK AND THEN

ANOTHER THING -- WAS IT TWO

MINUTES AGO?

[LAUGHTER]

SO I'VE GOT THREE MINUTES.

I WANT TO TALK QUICKLY ABOUT

THIS BECAUSE I DISAGREE

FUNDAMENTALLY AND ALMOST

COMPLETELY WITH THE IDEA

WE'RE GOING BACK TO FREE

TEXT.

YOU REMEMBER THAT 55%?

THAT WAS BEFORE EHRS

DESTROYED THE EXPRESSIVITY

AND RICHNESS AND NARRATIVE

WORK OF THE DOCTOR'S NOTE,

OKAY?

THE DOCTOR'S NOTE IS NOT THE

ISSUE HERE.

IF DOCTORS CAN WRITE THE

AMERICAN NOVEL AND EXECUTE

100 PERCENT PROCESSES, WE'RE

ALL FOR IT.

IF NOT, NOT.

AND IT IS NOT A MEASURE.

AND SO I THINK IT'S AN

IMPORTANT THING TO GET PAST

THAT SURROGATE.

SO THIS IS ONE REASON WHY.

SO YOU WAKE UP.

AND YOU HAVE BED PAINS SO

BAD THAT YOU CAN BARELY GET

OUT OF BED AND YOU CAN'T

STAND STRAIGHT UP.

YOU CAN BARELY GET TO THE

DOCTOR WHO MIRACLECULOUSLY

IS WILLING TO SEE YOU TODAY.

THERE ARE 16 -- TWO KINDS OF

BACK PAIN BASICALLY.

THERE IS BENIGN AND IT'S

OVERTREATED AND

OVERMEDICATED AND ENORMOUS

AMOUNTS OF MONEY ARE WASTED

TON.

AND THEN THERE IS MALIGNANT

AND IT'S UNDERTREATED AND

UNDERRECOGNIZED.

AND SO THIS IS ABOUT

PRECISION.

THIS ISN'T ABOUT OVERVIEWS.

THIS IS ABOUT PRECISE CARE.

IF YOU ARE ONE OF THE 99.9%

WHO HAVE THE BENIGN KIND,

YOU WOULD PROBABLY LIKE TO

KNOW THAT AND KNOW THAT

YOU'VE NOT HAVE TO BE EXPOSED

TO X-RAYS THAT MAKES NO

DIFFERENCE IN OUR OUTCOMES.

THAT PAIN MEDS IF YOU NEED

THEM FINE AND DON'T, FINE.

THAT BED REST HAS BEEN SHOWN

OVER TRIALS TO BE USELESS

AND ACTUALLY BAD FOR YOU AND

SO YOU CAN DO WHATEVER YOU

FEEL LIKE AND 90% YOU ARE

GOING TO BE WELL IN FOUR

WEEKS.

YOU MIGHT WANT TO KNOW ALL

THAT.

IF, ON THE OTHER HAND, ONE

OF THE PEOPLE THAT HAS MET

STATIC CANCER AND IF YOU

AREN'T TREATED TODAY OR

TOMORROW, YOU ARE GOING TO

SPEND THE REST OF YOUR LIFE

IN BED IN ALL KINDS OF

MISERABLE SITUATIONS, YOU

WOULD PROBABLY LIKE TO KNOW

THAT AND PROBABLY LIKE TO BE

SENT FOR EMERGENCY MRI.

THERE ARE 16 QUESTIONS THAT

HAVE BEEN SHOWN IN A SUPERB

TRIAL TO DIFFERENTIATE THOSE

TWO STATES.

WE JUST ASK ALL THE DOCTORS

HERE.

YOU PIGEON HOLE A DOCTOR

AFTERWARDS AND HAVE HIM TELL

YOU THE 16.

NO ONE CAN DO IT.

IF YOU DO A CHART REVIEW,

YOU WILL FIND THAT THERE

ISN'T A CHART AT YOUR

HEALTHCARE ORGANIZATION THAT

HAS ALL 16 DATA ELEMENTS IN

IT, NATURAL LANGUAGE

PROCESSOR AND SO WHAT IS THE

SOLUTION AND HOW CAN NLP

HELP?

WELL, FIRST OF ALL, HELP BY

IF IT WERE FAST ENOUGH BY

IDENTIFYING LOW BACK PAIN AS

THE PROBLEM AND TEETH THINGS

UP.

IT COULD LOOK AND SEE THAT

THERE IS NO PRIOR BACK PAIN

ANYWHERE THE RECORD, NOTES

OR OTHERWISE.

IT COULD IDENTIFY THAT FOUR

OF THE 16-DAY CRITERIA ARE

KNOWN AND PREPLATE ZMOES

THEN IT COULD OFFER A

TEMPLATE TO SOMEBODY, THE

PATIENT, THE NURSE, THE

DOCTOR, WHOMEVER IS

APPROPRIATE AND THAT HUMAN

OR THOSE HUMANS COULD

COMPLETE THE 12 AND THEN

CHRONICLE DECISION SUPPORT

COULD CALCULATE THE

HIKEHOODS AND THE PROGNOSIS

AND THE PLAN AND ENABLE THE

PATIENT AND THE DOCTOR TO

WORK THAT THROUGH BECAUSE

THAT ISN'T SORT OF A -- IF

IT'S BENIGN --, THE ADVICE

GIVEN ISN'T INTUITIVE ADVICE

THAT YOU FEEL IS APPROPRIATE

WHEN YOU CAN'T STRAIGHTEN UP

BECAUSE OF THE PAIN.

AND THEN THE BUSINESS

PROCESS MANAGEMENT SYSTEM

CAN PUT ALL OF THAT INTO AN

AFTERVISIT SUMMARY THAT GETS

PRINTED BECAUSE YOU ARE

STILL A HUMAN BEING AND LIKE

CARRYING PAPER AROUND, IT

ALSO GOES TO YOUR ELECTRONIC

FILE FOR YOU TO LOOK AT

LATER WHEN THE PAIN GOES

DOWN AND YOU CAN THINK AND

THE B.P.M TO MAKE SURE THAT

YOU ARE FOLLOWED UP AND

REALLY ONE OF THE 90%.

ARE WE OUT OF TIME?

ALL RIGHT.

THAT'S ENOUGH.

WELL, ONE THING.

VERY QUICKLY.

[LAUGHTER]

LET ME SUGGEST TO YOU THAT

THE LEVEL OF AUTOMATION IS

ONE OF THE THINGS WE NEED TO

THINK ABOUT.

THIS IS ADAPTED FROM PAR

SURMON, WHO ACTUALLY

OUTLINED VERY USEFUL SET OF

LEVELS OF AUTOMATION, WHICH

AS FAR AS I CAN TELL WE PAID

ALMOST SO ATTENTION TO.

BUT IN PARTICULAR WOULD HELP

US A GREAT DEAL.

SO IF IT'S 90% RECALL AND

PRECISION, THEN MAYBE LEVEL

THREE IS APPROPRIATE.

IF IT'S 100% RECALL AND

PRECISION, THEN MAYBE IT'S

APPROPRIATE TO FULLY

AUTOMATE IT.

THANKS.

[APPLAUSE]

>> WE HAVE TIME FOR ONE

QUICK QUESTION FOR JIM.

>> YES, GO AHEAD.

>>

[INDISCERNABLE]

>> SOMEONE ELSE?

>>

[INDISCERNABLE]

>> THE DATABASE KNOWS EVERY

PATIENT AND THEIR PREFERRED

COMMUNICATION CHANNEL AND IT

AUTOMATICALLY ROUTES IT.

IF THEY STILL USE SNAIL MAIL,

IT GOES THERE.

WE DO THE APPROPRIATE THINGS

IN TEXT.

>> CAN IT PULL THEM UP ON A

-- PHONE THEM UP IN

TELEPHONE AND TALK TO THEM?

>> YES.

YOU'VE GOT -- CREATING A

CALL TEMPLATE THAT IS USABLE

FOR PEOPLE TAKES WORK BUT

YEAH.

>> YES.

>>

[INDISCERNABLE]

>> I NOTICED IN ONE OF THE

SLIDES IN THE MIDDLE THERE

YOU HAD THE LETTERS

CAPITALIZED BPM.

I ASSUMED THAT DOESN'T MEAN

BEATS PER MINUTE.

COULD YOU TELL US A LITTLE

BIT ABOUT BUSINESS PROCESS

MANAGEMENT AND HOW IT

RELATES TO THIS?

>> SURE.

BUSINESS PROCESS MANAGEMENT

IS A DISCIPLINE.

IF YOU SAY OKAY, GUYSINGER

HAS 120 CORE PROCESS THAT'S

WE NEED TO CHARACTERIZE AND

WHO IS RESPONSIBLE, WHO CAN

DO THEM AND WHAT THE TIME

LIMITS ARE, THEN THERE IS A

SOFTWARE SYSTEM THAT YOU CAN

PUT THAT PROCESS

CHARACTERIZATION INTO AND

THEN THE SOFTWARE SYSTEM

MANAGES IT AUTOMATICALLY.

SO IT SAYS OH, THIS WOMAN

HAS BEEN 12 MONTHS SINCE

LAST MAMMOGRAM.

SEND THEM A

THE WAY, THEY LIKE IT

THROUGH THE GIGSER NETWORK

AND BY THE WAY, SEND IT

THROUGH SMALE MAIL SO IT'S A

SYSTEM THAT IS JUST STARTING

TO BE USED IN HEALTHCARE

THAT MAKES -- THAT IS CRITICAL

INFRASTRUCTURE TO MAKING

100% PROCESSES A REALITY.

>> AND HOW DOES AN LP FIT

INTO THIS?

>> WELL, I THINK THERE WOULD

BE A STACK AND YOU'D SAY

LOOK.

YOU'VE GOT THIS PROCESS

DEFINITION AND YOU HAVE ALL

OF THESE TRIGGERS, BY THE

WAY, WHICH TELL YOU WHAT

INFORMATION YOU NEED TO

COLLECT ONE WAY OR ANOTHER

OR GO FIND AN NLP AND SAY IF

WE FIND THIS INFORMATION

WHEREVER IT IS AND IF IT'S

APPROPRIATE TO USE NLP FOR

IT LIKE ABNORMAL MAMMOGRAMS,

THINGS LIKE THAT, THEN USE

THE NLP IN THAT LAYER BUT

THEN HOWEVER IT'S COLLECTED

FROM THE PATIENT, FROM A

CLINICIAN, FROM A

PHYSIOLOGIC MONITOR, FROM NLP,

ALL OF THAT FEEDS INTO THE

B.P.M LAYER AND SORT OF RUNS

THE PROCESS.

>> FOR EXAMPLE, YOU COULD

USE THE PROCESS DEFINITION

TO DECIDE WHETHER TO PRESENT

A SPEECH RECOGNITION MODULE

OR A TEMPLATE BASED UPON A

PREFERENCE OR OTHER ANALYTIC

STUDIES?

>> YEAH, YOU COULD,

ABSOLUTELY.

>> OKAY.

OUR FINAL SPEAKER FOR THIS

SESSION IS ELLIOTT --

[APPLAUSE]

SORRY.

>> OUR FINAL SPEAKER FOR

THIS PANEL IS ELIOT SIEGEL

FROM THE UNIVERSITY OF

MARYLAND.

>> WELL, I'D LIKE TO THANK

THE NATIONAL LIBRARY OF

MEDICINE AND ANYBODYIB FOR

THE INVITATION TO MY

GUIDE AN ORGANIZED PHYSICIAN

ORGANIZED A HEAD CT STUDY

FOR DIZZINESS.

MINING THE DATA THAT I HAVE

AND LOOKING AT APPROPRIATENESS

CRITERIA, I CAN MAKE THE

DETERMINATION THAT EVEN

THOUGH A HEAD CT IS BEING

REQUESTED FOR THE INDICATION

OF DIZZINESS, IT LOOKS LIKE

AN MR WOULD BE SIGNIFICANTLY

HIGHER YIELD.

AND YOU CAN AGAIN, GET THAT

FROM EXPERT ADVICE OR FROM

THE LITERATURE OR YOU CAN

MINE YOUR OWN DATA AS FAR AS

PERCENTAGE OF POSITIVE

STUDIES AND AS FAR AS

PREVIOUS RECOMMENDATIONS.

AND THIS CAN ESSENTIALLY BE

THE SAME THING COULD BE USED

WHETHER IT'S A HEAD CT, HERE

IS AN EXAMPLE OF AN

EXTREMITY MR WHERE THERE IS

A REQUEST FOR AN MRI FOR A

PATIENT WITH ARTHRITIS.

AND HERE IS ANOTHER ONE

THAT'S RELATIVELY LOW

UTILITY.

ESSENTIALLY, A PATIENT WITH

BACK PAIN.

EVEN AN MRI IS RELATIVELY

LOW YIELD AS FAR AS HAVING A

SIGNIFICANT IMPACT ON THAT

PATIENT'S CARE.

AND SO IN THIS CASE, WE CAN

PROVIDE THAT FEEDBACK THAT

FOR THAT INDICATION MR MIGHT

BE A THREE OUT OF TEN AS FAR

AS INDICATION.

BUT IF WE ADD ABNORMAL

EXTREME.

REFLEXS TO THE HISTORY, THEN

GO BACK, THEN THAT'S

SIGNIFICANTLY INCREASES THE

UTILITY AND THE ADDED VALUE

THAT THE MR PRESENTS AND NOW

IT'S A NINE OUT OF TEN.

AND SO THIS CAPABILITY OF

BEING ABLE TO MINE THAT SORT

OF DATA AS -- ADDS A

TREMENDOUS AMOUNT, AS WAS

MENTIONED, THE RADIOLOGY

REPORTS CAN BE FAIRLY

COMPLEX AND BEING ABLE TO

MINE THE IMPORTANT CONCEPTS

IS REALLY, REALLY CRITICAL.

AND ONCE I CAN MINE THOSE

CONCEPTS, I CAN START

LOOKING AT ORGANIZED

PHYSICIANS, FOR EXAMPLE, AND

SEE HOW THEY CLUSTER AS FAR

AS DIFFERENT STUDIES.

FOR MR, HERE IS A CLINICIAN

THAT'S AN OUTLIAR THAT HAS A

LOWER INCIDENCE OF POSITIVE

FINDINGS BUT ALSO A HIGHER

INCIDENCE AS FAR AS

RECOMMENDATION FOR FOLLOWUP

STUDIES.

HERE IS ANOTHER ONE WHERE WE

COMPARE RADIOLOGISTS.

SO WE HAVE TWO AGENCIES

RADIOLOGISTS THAT KIND OF

CLUSTER WITH CT STUDIES WITH

A LOWER INCIDENCE OF

POSITIVE FINDINGS BUT A

HIGHER RECOMMENDATION RATE.

I DON'T KNOW IF THESE TWO

ARE BETTER OR WORSE THAN

ANOTHER RADIOLOGIST BUT IT'S

INTERESTING TO SEE HOW THEY

CLUSTER.

HERE IS CLUSTERING OF

POSITIVE FINDINGS VERSUS

RECOMMENDATION RATE FOR MR

CT AND X-RAY.

AND IT'S REALLY INTERESTING

TO SEE WHICH ONES HAD THE

HIGHEST YIELD OF POSITIVE

FINDINGS, PRESUME PABLELY

WHAT IS HAPPENING IS THE

RADIOLOGIST WOULD LOCK AT

THE X-RAY AND RECOMMEND A CT

STUDY AND THIS IS A

DIFFERENCE FOR FEMALES.

AND SO ONE OF THE THINGS I'D

LIKE TO SEE WITH EDM AND

NATURAL LANGUAGE PROCESSING

WOULD BE SOME HELP IN

EVALUATING THE INDICATION AS

FAR AS AUTOMATED PROTOCOLING.

ONCE AN EXAM HAS BEEN

ACCEPTED SUCH AS AN MR, THEN

IT WOULD BE HELPFUL FOR ME

TO HAVE ASSISTANCE AND IN

AUTOMATTICALLY PROTOCOLING

WHICH MIGHT BE BEST TO USE.

ALSO I MENTIONED FOR

UNEXPECTED FINDINGS.

WE HAD A LOT OF PATIENTS WHO

HAD, FOR EXAMPLE, A LONG

NODULE NOTED ON A TRAUMA CT

SERIES JUST LIKE THE OVARIAN

KRIS THAT WAS MENTIONED --

KRIST MENTIONED THIS MORNING.

KRIST.

SO CREDITING HAVING A SIT --

SYSTEM WOULD ALLOW US TO

CLOSE THE LOOP.

THE RADIOLOGISTS VARY IN

THEIR IDENTIFICATION OF

THESE QUOTE UNQUOTE

UNEXPECTED FINDINGS.

OF COURSE, THERE IS GREAT

WORK THAT HAS BEEN DONE IN

NATURAL LANGUAGE PROCESSING

AND DECISION SUPPORT AT MANY

FACILITIES.

HERE IS A GREAT ONE AT

INDIANA UNIVERSITY AND THERE

ARE SOME REALLY WONDERFUL

ONES THAT HAVE BEEN DONE.

THERE HAVE BEEN MANY

RADIOLOGY CLINICAL DECISION

SUPPORT TOOLS.

I'M HAPPY TO SEE DR. GREENES

IN THE AUDIENCE BECAUSE HE'S

AN EXPERT ON THIS AND I CAN

REMEMBER AS A RADIOLOGY

RESIDENT BACK IN THE 80S,

DR. GREENES COMING BY AND

TALKING ABOUT THE$1v WONDERFUL

TOOLS THAT WERE AVAILABLE

FOR DECISION SUPPORT.

AND BETWEEN MANY THAT ARE

EITHER RURAL-BASED OR ONES

THAT USE CASE-BASE

REASONING, THERE ARE A WIDE

VARIETY OF THESE.

ONE OF THE CHALLENGES,

THOUGH, IN BEING ABLE TO

CREATE THESE RULES-BASED

SYSTEMS, IS COLLECTING LARGE

AMOUNTS OF DATA.

AND SO IT'S GREAT TO HEAR

THAT PARTNERS IS DOING THAT

AND GUISING HERE IS DOING

THAT ALSO.

BUT I CAN'T THINK OF A

LARGER DATABASE OR

HEALTHCARE SYSTEM THAN THE

DEPARTMENT OF VETERANS FAIRS

WHICH HAS COLLECTED AND NOW

CONSOLIDATED DATA.

SO WE HAVE DATA NOW FROM 163

CLINICS AND ELECTRONIC AND

DIGITAL ESSENTIALLY FOR 20

YEARS.

AND IT HAS DATA THAT GOES

BACK FOR OVER 12 YEARS ON

OVER 28 MILLION PATIENTS.

AND SO THERE ARE HUGE

AMOUNTS OF VOLUME.

SO THE V.A. HAS TAKEN

ESPECIALLY A LAYERED

APPROACH.

IT REALLY LOOKS LIKE A PAPER

PAPER-BASED RECORD.

AND ALL THE INFORMATION

WITHIN THE SYSTEM IS THERE,

BUT YOU CAN'T MINE IT.

YOU CAN'T ASK QUESTIONS LIKE

SHOW ME ALL OF THE INCIDENTS

WHERE THIS PARTICULAR

PATIENT HAD A RASH OR SHOW

ME ALL RASHES, FOR EXAMPLE.

SO THE V.A. HAS CREATED A

TEAM OF DOZENS AND DOZENS OF

FOLKS WHO SUPPORT HSR AND D,

ESSENTIALLY A HEALTH

SERVICES RESEARCH WHO ARE

TAKING ADVANTAGE OF

CONSOLIDATING THESE DATA AND

TO HAVE A NUMBER OF NATURAL

LANGUAGE PROCESSING EXPERTS

WHO ARE CREATING PIPELINES

THAT ARE DOING PROCESSING

AND REPROCESSING OF THE DATA

THAT'S AVAILABLE TO MAKE IT

EASIER AND EASIER TO BE ABLE

TO DO DIFFERENT TYPES OF

STATISTICAL ANALYSIS.

SO I CAN'T IMAGINE ANY

BETTER SAND BOX THAT EXISTS

RIGHT NOW.

THE TEAM HAS FOCUSED

PREDOMINANTLY ON RESEARCH

APPLICATIONS.

WHAT I AM HOPING TO DO IS TO

BE ABLE TO UTILIZE IT TO A

GREATER EXTENT FOR DATA

SUPPORT.

AND WE'VE HAD TO DEAL WITH

ISSUES RELATING TO SECURITY.

THERE ARE MANY DIFFERENT

DATA TYPES AND I WON'T GO

THROUGH THOSE BUT PRETTY

MUCH EVERYTHING IN THE

ELECTRONIC MEDICAL RECORDS

STRUCTURED AND UNSTRUCTURED

IS AVAILABLE WITHIN VINCI.

SOME OF THE CHALLENGES,

BIGGEST CHALLENGES THAT WE

HAVE NATURAL LANGUAGE

PROCESSING INCLUDE THE FACT

THAT TEM PLATS ARE WIDELY

USED AND TEM PLATS CAN BE

VERY CONFUSING WHEN YOU TRY

AND DO NATURAL LANGUAGE

PROCESSING.

INCOMPLETE SENTENCES AND

JARGON, OF COURSE.

AS FAR AS NEXT GENERATION, I

AM NOT GOING TO GO INTO

DETAIL.

I'VE BEEN HERE IN THIS

AUDITORIUM PRESENTING SOME

OF THE WORK THAT WE'VE DONE

ALONG WITH IBM AND WATSON

UTILIZING THE JEOPARDY

SOFTWARE TO GET INVOLVED

EVEN BEFORE THEY PLAYED

JEOPARDY.

I AM LOOKING FORWARD TO

DAVID'S TALK ABOUT HOW IN

DETAIL ABOUT HOW THEIR

TECHNOLOGY WORKS.

BUT JUST SUFFICE IT TO SAY

THAT WE'RE REALLY INTERESTED

AND WE'VE BEEN WORKING WITH

IBM ON TRYING TO UTILIZE

WHAT THEY HAVE, WHICH I

THINK IS A FUNDAMENTALLY

DIFFERENT APPROACH USING

VERY, VERY HIGH SPEED.

IF YOU CAN IMAGINE THAT

YOU'VE HAD AN INFINITE

AMOUNT OF TIME AND

PROCESSING POWER TO FORM A

HYPOTHESIS WITH EVERY

QUESTION AND SEARCH A

DYNAMIC DATABASE EVERY TIME

YOU FORM LATE A QUESTION,

WHAT WOULD BE THE POTENTIAL

ASSOCIATED WITH THAT?

AND I THINK THAT TECHNOLOGY

THAT THEY HAVE THOFRS.

SO IN CONCLUSION, I THINK

RADIOLOGY AND DIAGNOSTIC

IMAGING WILL CONTINUE TO BE

A RICH SUBSPECIALTY FOR

IMAGE PROCESSING, NOT ONLY

THAT BUT ALSO COMPUTER-AIDED

DETECTION AND ALSO FOR

NATURAL LANGUAGE PROCESSING

AND ENHANCED CLINICAL

DECISION-MAKING.

USING THESE TWO TECHNIQUES

ON VERY LARGE DATABASES SUCH

AS THE VINCI DATABASE HAS

THE POTENTIAL TO HAVE A

FUNDAMENTAL MAJOR IMPACT ON

RESEARCH AS WELL AS OUR DAY

DAY-TO-DAY DECISION-MAKING

IN MEDICINE AND CERTAINLY IN

RADIOLOGY.

SO THANKS AGAIN.

[APPLAUSE]

LOOKING FORWARD TO THE PANEL

DISCUSSION.

>> CAN I ASK OUR THREE

PREVIOUS SPEAKERS TO COME UP

AND IF ANYONE HAS ANY

SPECIFIC QUESTIONS FOR

DR. SIEGEL?

OTHERWISE, WE CAN SPEND TEN

OR 15 MINUTES.

>> I LIKE YOUR FIRST SLIDE

THAT SAID "YOU'D LIKE TO

ALWAYS SEE POSITIVE OR

NEGATIVE."

IS THAT REALLY FEASIBLE

GIVEN SORT OF THE NATIONAL

SHRUB OF THE RADIOLOGIST AS

THE HEDGE?

>> SO I THINK THAT'S A GREAT

QUESTION.

AND WE KIND OF JOKE AMONGST

OURSELVES AS RADIOLOGISTS AS

HEDGERS AND IT BRINGS UP A

COUPLE FUNDAMENTAL QUESTION

IS WHAT IS A POSITIVE OR

NEGATIVE RADIOLOGY REPORT,

ESPECIALLY WHEN YOU MAY BE

COMMENTING ON MULTIPLE

DIFFERENT FINDINGS?

ANOTHER ONE IS HOW DOES ONE

DETERMINE LEVEL OF

CERTAINTY?

AND SO IN ROLLING, OUR --

RADIOLOGY, OUR PENDULUM HAS

SWUNG BACK AND FORTH AND

WE'VE GONE FROM STRUCTURED

REPORTS BACK TO

UNSTRUCTURED.

NOW WE'RE HEADING IN THE

DIRECTION OF STRUCTURED

REPORTS, WHICH INCREASINGLY

REQUIRE A LEVEL OF

CERTAINTY.

THE BEST EXAMPLE, OF COURSE,

IS BIRADS, WHICH IS THE

CLASSIFICATION SYSTEM FOR

MAMMOGRAMS WHICH FORCES THE

RADIOLOGIST TO CATEGORYIZE

EVERY SINGLE MAMMOGRAM AND

THAT HAS HAD MAJOR POSITIVE

IMPACTS.

AND SO I THINK TO YOUR

POINT, IT'S DIFFICULT WITH

FREE TEXT TO CONSTRAIN AN

INTERPRETER TO BEING ABLE TO

INDICATE LEVEL OF CERTAINTY

OF A PARTICULAR FINDING AND

I THINK IT'S REALLY

IMPORTANT TO DO THAT.

AND SO A LOT OF THE WORK

THAT I'VE DONE IN ADDITION

TO ON NATURAL LANGUAGE

PROCESSING HAS BEEN IN

SUPPORT OF STRUCTURED

REPORTING.

AND IN FACT, ONE THING THAT

WE PUBLISHED ON WAS COMPLETE

GRAPHICAL REPORTING, WHERE

ALL DO YOU IS ESSENTIALLY

TAKE A PEN ON AN IMAGE,

DELINIATE THE AREA OF

PATHOLOGY, AND THEN

ESSENTIALLY JUST PUT MARKERS

ON IT AND THAT BECOMES THE

ENTIRE REPORT, INCLUDING

YOUR LEVEL OF CERTAINTY.

BUT A LOT OF RADIOLOGISTS

HAVE OBJECTED BECAUSE THAT

DOES NOT ALLOW THEM THE FREE

EXPRESSION THAT THEY ENJOY

HAVING.

AND WHAT VALUE THAT FREE

EXPRESSION HAS IS SOMETHING

THAT ONE COULD DEBATE.

THERE IS VALUE IN THE

NUANCES ASSOCIATED WITH THE

ENGLISH LANGUAGE, BUT THE

STRUCTURE, I THINK, ADDS A

TREMENDOUS AMOUNT.

AND I THINK AS THE PREVIOUS

SPEAKERS HAVE SAID, WE'RE

GOING TO BE IN A SITUATION

WHERE WE'RE USING A

COMBINATION OF STRUCTURED

AND STRUCTURED DATE Y AS

TIME GOES ON.

>> YES.

SURE.

>> I WOULD LIKE TO KNOW IF

FOR MOST RADIO GRAPHIC

PROCEDURES, IF STANDARD SETS

OF FINDINGS OR FEATURES HAVE

BEEN DEFINED SO THAT YOU

COULD GO THROUGH A CHECK

POLICE AND SAY THIS FEATURE

IS PRESENT OR NOT?

>> YEP.

THAT'S A GREAT QUESTION.

SO THE RADIO LOGIC SOCIETY

OF NORTH AMERICA HAS

RECENTLY RELEASED ABOUT 100

TEMPLATED BEST PRACTICE

REPORTS.

AND ASSOCIATED WITH THOSE

THERE ARE CERTAIN FINDINGS

WITHIN THAT TEMPLATE THAT A

SERIES OF EXPERTS HAVE

GOTTEN TOGETHER, REVIEWED

THE LITERATURE, AND SAID IF

YOU ARE REPORTING AT AN MRI

OF THE KNEE, THESE ARE THE

THINGS THAT YOU NEED TO

INCLUDE.

IF YOU ARE REPORTING AT

THESE PARTICULAR OTHER

STUDIES, THIS IS WHAT YOU

NEED TO INCLUDE.

WE'VE TAKEN TAI STEP FARTHER

AT THE NATIONAL CANCER

INSTITUTE WHERE WE LOOKED AT

REPORTING BRAIN TUMORS.

AND AS IT TURNS OUT, IF YOU

LOOK AT FREE TEXT REPORTS,

MAYBE YOU GET SEVEN

PARAMETERS IN AN AVERAGE

REPORT.

BUT WHEN WE LOOKED AT THE

LITERATURE OF WHAT KORLETED

WITH GENOMIC AND PRODIOMIC

DATA, THERE WERE SOMEWHERE

AROUND 20 AND 30 THAT WE

EXTRACTED.

WE CREATE AID TEMPLATED WORK

STATION WHERE THE WORK

STATION WALKS THE

RADIOLOGIST THROUGH A MAKING

MEASUREMENTS, ASK SPECIFIC

QUESTIONS THAT KORLET WITH

THOSE PARTICULAR FINDINGS

AND NOW WHAT YOU HAVE IS AN

AUTOMATIC WORK STATION

INTERACTION THAT

AUTOMATICALLY GENERATES A

REPORT THAT'S TAILORED TO A

SPECIFIC DISEASE ENTITY

USING A NEW FORM OF

ANNOTATION THAT HAS BEEN

CREATED AT NCI CALLED AIM.

SO YOU CAN SEE VARYING

DEGREES OF ADDED STRUCTURE

AS TIME GOES ON.

ONE OF THE THINGS THAT WE'RE

STARTING TO SEE IN RADIOLOGY

AND WE'RE SEEING IT FOR

CARROTID STUDIES IS

>> I REMEMBER BURSEMENT.

YOU CAN CHARGE FOR THE

REPORT BUT WE'LL PAY A

LITTLE BIT EXTRA IF YOU MAKE

SURE YOU INCLUDE THESE

PARTICULAR ELEMENTS AND THIS

IS ONLY THE START IN

DIAGNOSTIC IMAGING.

TEN YEARS FROM NOW IN ORDER

TO GET PAID WE'RE GOING TO

HAVE TO BE ABLE TO DOCUMENT

THAT WE HAVE ANSWERED

CERTAIN QUESTIONS AND THE

ONLY WAY TO DO THAT IS

EITHER WITH STRUCTURE OR

WITH NATURAL LANGUAGE

PROCESSING THAT WILL ALLOW

KOERDS TO KNOW WHETHER OR

NOT THERE HAS BEEN A REPORT

THAT MEET CERTAIN QUALITY

CRITERIA AND CRITERIA WILL

BE INCLUDING THOSE ELEMENTS.

I THINK THIS RADIOLOGY BEST

AGREED TEM PLATS IS A FIRST

GOOD START BUT THERE ARE

MULTIPLE THINGS WE CAN DO TO

MAKE IT EVEN BETTER SO

THANKS FOR THE QUESTION.

>> ONE MORE QUESTION.

>> QUESTION FOR TOM PAYNE.

REALLY INTERESTING STUFF YOU

ARE GOING DOG THERE.

COMPOSITION OF THE PROBLEM

LIST AND YOU MENTIONED YOU

JUST ROLLED IT OUT SIX WEEKS

AGO.

ABOUT HOW LONG FROM THE TIME

THAT THAT PROJECT WAS A GLIM

NER YOUR EYE UNTIL SIX WEEKS

AGO LEPSED?

, ELAPSED?

>> WELL, WE HAD AN EARLY

EFFORT TO DO THIS ON OUR

OWN, AND WE ACTUALLY HAD

SHUTTLEE, WHO IS NOT HERE

BUT WAS HERE YESTERDAY.

THAT WAS THREE YEARS AGO.

WE HAD THIS IDEA AND HAD IT

IN A PROTOTYPE.

WHAT WE WEREN'T ABLE TO DO

WAS TO CONVERT THAT IDEA

INTO A PRODUCTION SYSTEM

THAT WORKED IN THE REAL

WORLD.

WHEN WE STARTED DOING THE

CODING, INTERESTING METAPHOR

HERE.

A LOT OF OUR HEALTH IT IN

OUR HOSPITALS BEGAN WITH

BILLING 40 YEARS AGO, 350

YEARS AGO AND THEN WE BROADENED

IT TO A BROADER PORTFOLIO.

SAME THING HERE.

WE STARTED WITH BILLING AND

AS A COR LARRY OF THAT

TAGGING THAT I SHOWED YOU,

WE REALIZED WE COULD DO IT.

THE PROBLEM LIST.

SO THAT WAS VERY QUICK AFTER

THE AGREEMENTS WERE SIGNED

LESS THAN A YEAR IT WAS IN

PRODUCTION.

>> SO WHAT I AM HEARING FROM

THIS SESSION IS A NUMBER OF

HYBRID SOLUTIONS IS NOT A

DEBATE OF WHETHER -- I THINK

WE NEED BOTH AND SOMEBODY IN

EACH PLACE SORT OF DRAWING

THE LINE IN SOMEWHAT

DIFFERENT AREAS WHERE YOUR

URGENT PROBLEMS ARE.

CAN WE CREATE IN A GENERAL D

-- ANY GENERAL PRINCIPLES

THAT CAN EMERGE AS STANDARD

APPROACHES?

ELLIOTT MENTIONED THE ACR

TEM PLATS, FOR EXAMPLE.

CAN WE DO THAT ACROSS OUR

SPECIALTIES AND BEGIN TO

HAVE TEM PLATS FOR THE

COMMENT OR THE HIGH RISK ORT

HIGH UTILIZED CONDITIONS

WHERE WE DO HAVE TO CAPTURE

THESE KINDS OF THINGS?

>> ONE GENERALIZATION THAT'S

IMPORTANT THAT THIS

DISCUSSION HAS BROUGHT UP IS

THAT THE CREATOR OF THE

INFORMATION IS ALWAYS MORE

INTERESTED IN EXPRESSIVITY

THAN THE RECIPIENT SO THAT

OUR DOCTORS OVER AND OVER

AGAIN SAY WITH THE RADIOLOGY

SAY CAN YOU PLEASE JUST PUT

THE IMPRESSION FIRST AND ALL

THE REST OF IT LATER?

AND THE RADIOLOGIST ACTUALLY

GET ANGRY AND SAY NO, THEY

HAVE TO READ THE WHOLE THING.

AND DOCTORS ACT THE SAME WAY.

ONE OF THE RULES IS THAT IF

YOU THINK IN TERMS OF THE

INFORMATION CONSUMER, YOU'LL

HAVE A DIFFERENT APPROACH TO

LOTS OF THIS.

>> JUST A COMMENT AND I

THINK ONE THING THAT WOULD

HELP A LOT IS SOME OF THE

REPORTS THAT WE GET, FOR

EXAMPLE, THE ECOCARDIOGRAM

REPORTS THAT HAVE

MEASUREMENTS IN THEM WERE

DELIVERED TO US AS DISCREET

DATA IN ADDITION TO THE

NARRATIVE REPORT.

FOR EXAMPLE, IF YOU WANT TO

KNOW LEFT ART RAL SIZE CAN

BE A DETERMINANT IN

SOMEBODY'S RISK IF THEY HAVE

HAVEATORIAL FRIBILLATION,

WHY NOT MAKE THAT AVAILABLE?

I'D LIKE TO SEE A LOT MORE

OF THAT DONE.

>> AND THERE WOULD BE

ANOTHER PRINCIPLEF YOU START

WITH 100 PERCENT PROCESS AS

YOUR MEASURE, THEN IT

BECOMES EASIER TO IDENTIFY

WHAT YOU WANT TO CAPTURE.

AND PEOPLE WHO ARE

RESPONSIBLE FOR CAPTURING

IT, PHYSICIANS AND OTHERS,

IN OUR EXPERIENCE, HAVE

MINIMAL RESISTANCE TO

CAPTURING THAT.

WHAT THEY PUSH WHAT THEY

HATE LIKE FIRE IS THEIR

EXPERIENCE OF WHAT 100 YEARS

OF BEING ASKED TO RECORD ALL

KINDS OF THINGS THAT THEY

KNOW FOR A FACT NOBODY'S

EVER GOING TO DO ANYTHING

WITH.

>> JUST SAY THIS DISCUSSION

WE'RE HAVING HERE IS NOT NEW.

IT'S BEEN GOING ON AT MY

CAREER.

THERE HAS ALWAYS BEEN A

TENSION BETWEEN THE WAY WE

PUT IN NOTES AND WHAT IS

DIFFERENT AND NEW TODAY IS

THE RAPID OPTION OF THESE

TOOLS AS THEY EXIST IN A

COMMERCIAL WORLD.

AND THEIR IMPACT ON THE WORK

FLOW AND THE TIME THAT

PROVIDERS HAVE TO SPEND WITH

A PATIENT.

THAT IS NEW, AND WHEN I AM

REFLECTING HERE IS THEIR

REACTION TO THIS ENORMOUS

CHANGE.

ANOTHER THING THAT'S NEW IS

THE POWER OF NLP, WHICH 30

YEARS AGO WASN'T THE DEGREE

AS IT IS TODAY.

SO I THINK THERE WILL STILL

BE A SPECTRUM.

TWHAR LINE FALLS IS GOING TO

CHANGE, AND WE HAVE TO

RESPECT THE FACT THAT REALLY

BUSY PEOPLE ARE PUSHING BACK

ON SOME OF THE THINGS.

>> BUT I THINK YOU ASKED A

GREAT QUESTION.

AND WHAT REALLY IS QUALITY?

AS TIME GOES ON, I THINK

THAT WE'RE GOING TO HAVE AN

INCREASED AMOUNT OF

ATTENTION PAID IN MEANINGFUL

USE TO WHAT REPRESENTS A

QUALITY NOTE.

I TALKED ABOUT WHAT'S A

QUALITY RADIOLOGY REPORT.

I REMEMBER WHEN I WAS A

STUDENT WRITING A DETAILED

THREE-PAGE NOTE NOT A

PATIENT AS AN ADMISSION AND

PHYSICIANS MAYBE CAN RELATE

TO THIS SORT OF THING AND

GOT A D ON IT SKPASHLE ASKED

WHY DID I GET A B AND THE

ANSWER WAS WELL, THE OTHER

STUDENTS WHO WERE IN YOUR

GROUP WROTE 20 AND 25-PAGE

ADMISSION NOTES.

[LAUGHTER]

AND I'M THINKING WHAT IS A

QUALITY NOTE AND WHAT SHOULD

BE IN IT AND HOW DO WE

DEFINE IT?

AND I THINK ONE OF THE

PROBLEM IS WE DON'T DEFINE

QULET AS THE THIS POINT,

WHICH MAKES IT DIFFICULT FOR

US TO REALLY KNOW WHAT ARE

THE ELEMENTS THAT SHOULD BE

IN A TEMPLATED GOOD QUALITY

NOTE� AND WHAT THINGS DO WE

NEED FOR THIS FUTURE OF THE

VISION THAT WE ALL HAVE.

>> JUST REAL QUICK ON THAT

ONE.

HENRIKIO HAD A GREAT ARTICLE

IN 2000 WHEN COMMUNICATION

IS BETTER THAN COMP PUTATION

AND I THINK HE ADDRESSES

THIS VERY EASILY.

THERE ARE SOME THINGS THAT

ARE SO HIGHLY CARCIZED, SOME

DATA THAT ARE SO HIGHLY

CHARACTER YIFD AND SO

COMPUTEABLE.

DID I GIVE THE PATIENT AN

ASPIRIN WHEN THEY HAD AN

ACUTE HEART ATTACK THAT THEY

SHOULD BE CAPTURED AND

STRUCTURED, STANDARDIZED

FORM AND COMPUTED ON.

AND THERE ARE OTHER THINGS,

PATIENT LITERALLY WALKS INTO

YOUR ROOM AND SAID IT REALLY

HURTS AND THEN IT EXPLODES

AND I FEEL FINE.

THAT WOULD BE FOOLISH TO TRY

TO PUT INTO EXECUTIONAL FORM

-- COM PUTATIONAL FORM.

THE QUESTION IS THIS DATEUM

-- DOES IT FIT INTO A

CLINICAL PREDICTION RULE?

IS IT USED IN SOME WAY TO

HELP INFORM PATIENT CARE, OR

IS IT SOMETHING HA WE'RE

BETTER OFF JUST ENHANCING

COMMUNICATION AND NOT TRYING

TO STANDARDIZE, COMPUTE,

TEMPLATE AND DO ALL THE

THINGS WE NEED TO DO TO

HIGHLY CHARACTERIZED

INFORMATION?

>> SO THIS WHOLE QUESTION

ABOUT NARRATIVE TEXT VERSUS

ENTERING SPECIFIC QUESTIONS

I'VE SUFFERED WITH, TOO, IS

A CLINICAL CAREGIVERS GIVER.

BUT THE THING I THINK IS

MISSING IN OUR FIELD AND

MEDICAL FIELD IN GENERAL.

SO IF YOU LOOK AT THINGS

THAT AREN'T STRUCTURED LIKE

ECOCARDIO GRAPHIC AND

CURIOUSLY ARE ALL HIGHLY

STRUCTURED AND RADIOLOGISTS

NOTES AND SOMEDAY WE'LL

FIGURE THAT OUT.

THEY'VE DONE RESEARCH FOR

YEARS AND THEY'VE FIGURED

OUT THE 40% MEANS THIS, THIS,

THIS AND THIS.

NOW EVERYBODY WANTS THAT

NUMBER BECAUSE THEY'VE

FIGURED OUT WHICH IS THE

IMPORTANT OF ALL THESE OTHER

THINGS THEY PROVIDE.

WE HAVEN'T DONE THAT WITH

CLINICAL DATA.

AND EVERYBODY BELIEVES

INTENSELY THAT THEIR

PHYSICAL EXAM SFINGSD THE

MOST IMPORTANT.

WE WE'VE GOT TO PUT THIS

20-PAGE PHYSICAL THING.

NO HUMAN SANE PERSON WOULD

EVER DO THAT.

BUT YOU ARE SUPPOSED TO

BECAUSE THE CARDIOLOGIST ARE

GOING TO CHECK THAT YOU'VE

GOT THIS PART WHICH YOU

NEVER NEED BECAUSE YOU GET

AN ECHO.

I THINK WE NEED TO INVEST

SOME MONEY IN RESEARCH IN

THE CLINICAL DATA ELEMENTS

THAT ARE NOT MEASUREMENTS

THAT FIGURE OUT WHICH ONES

MEAN ANYTHING.

AND WHICH ONES -- A LOT OF

STUFF WE DO AS PHYSICIANS IS

JUST TO HELP US GET TO THE

NEXT VISIT.

IT HAS NO USE TO ANYONE.

EXCEPT OURSELVES.

BUT WE HAVE TO ENSHRINE THIS

AND DO ALL THIS WORK IN

GETTING THIS INTO THE

DATABASE.

NOW THAT WE HAVE THE

COMPUTERS IN OUR WORK FLOW,

THAT MEANS THE

ADMINISTRATION CAN MAKE US

ANSWER ANY DUMB QUESTION

THEY THINK OF.

THAT'S GOING TO BE AN ISSUE.

>> A GOOD EXAMPLE, TAKING A

FAMILY HISTORY, CREATING A

PEDIGREE.

HOW IS THAT DONE?

WE HAVE A GENETICS CLINIC

AND THEY SPEND HOURS DOING

THAT IN VERY GREAT DETAIL,

WHICH THEY SHOULD.

AND WHEN THEY'RE DONE, IT'S

GORGEOUS.

BUT I SEE THESE PEOPLE DAY

AFTER DAY I GET A LITTLE

SNIPPET WITH A FAMILY

HISTORY EACH TIME.

IF IT WERE POSSIBLE FOR ME

TO PULL IT TOGETHER WITHOUT

SPENDING THREE HOURS EACH

TIME ASKING A LITTLE BIT

MORE PUT THAT STORY TOGETHER,

THIS ULTIMATELY DOES HAVE AN

IMPACT ON THAT PERSON'S

HEALTH.

BUT THE TOOLS TO CAPTURE

THAT RIGHT NOW ARE NOT FAST

ENOUGH FOR ME TO DO WHAT

THEY DO IN THE MEDICAL

GENETICS CLINIC.

SO I DO A LITTLE PIECE OF IT

AT A TIME AND A LOT OF IT IS

NO LONGER A SKETCH ON A

PIECE OF PAPER.

IT'S NOW IN TEXT.

WE NEED TO BE ABLE TO DO

BETTER WITH THAT BECAUSE

THAT TEXT DOES NOTHING THAT

IS GOING TO HELP ME COUNSEL

THIS PERSON ABOUT WHAT

FURTHER TESTING MIGHT BE

WARRANTED.

>> I JUST HAVE A QUESTION

FOR YOU ABOUT THE PROBLEM

LIST.

IN MY� EXPERIENCE IT'S BEEN

REALLY INTERESTING.

THERE IS NO SHERIFF OF THE

PROBLEM LIST, SO ANYBODY CAN

WRITE INTO THE PROBLEM LIST

AND THE PROBLEM THAT WE HAVE

AND I AM WONDERING HOW YOU

DEAL WITH IT -- IS WE HAVE

THINGS IT'S LOTS OF STUFF

GOES ON THE PROBLEM LIST BUT

IT'S LIKE A ROACH MOLT.

STUFF CHECKS IN AND IT NEVER

CHECKS OUT AGAIN.

YOU HAVE ONE EPISODE WHERE

SOMEBODY MEASURED A BLOOD

PRESSURE THAT WAS HE

WILLIATE VATED AND SOMEONE

WRITES IN THE PROBLEM LIST

HYPERTENSION.

BUT WHOSE RESPONSIBILITY IS

IT TO TAKE THAT OFF OR

SOMEBODY HAS AN ELEVATED

BLOOD GLUCOSE OR BACK PAIN

THAT COMES INTO THE ER?

WHOSE RESPONSIBILITY IS IT

TO TAKE IT OFF THE PROBLEM

LIST?

>> WE'RE DOING IT.

TESTIFY SAY MY FIRST JOB AS

A CLINICAL LOOSE IN A MASS

GENERAL WAS HELPING AND THEY

HAD OVER YEARS CREATED A

WORKING TOOL.

WE'RE NOT THERE YET.

AND MANY ORGANIZATIONS

AREN'T THERE YET.

I WOULD GUESS THAT'S ONE OF

THE REASONS IS THE MEANING

OF THE USE CRIER TERIA.

SO WE ARE CHANGING THE

CULTURE FROM PUTTING IT AS A

LIST OF MY THOUGHTS ABOUT

THIS PATIENT AND MAKING IT

ENCODED AND THAT'S GROWING

AND WE WANT ALL DISCIPLINES

INVOLVED.

WE THOUGHT THAT FAR AHEAD.

WE WANT EVERYONE WHO HAS A

VIEW ON THAT PATIENT TO

CONTRIBUTE.

AND THEN WE HAVE TO DECIDE

AS A COMMUNITY HOW WE'RE

GOING TO TAILOR IT AND TRIM

IT.

BUT I CAN'T GIVE YOU

EXPERIENCE ON THAT YET.

>> THANKS.

>> OKAY.

WE HAVE TIME FOR ONE LAST

QUESTION.

>> I WANTED TO MAKE ONE

COMMENT ABOUT THE PROBLEM

LIST.

IN ONE OTHER MEANING.

USE REQUIREMENT NOW IS THAT

WE MAKE ELECTRONIC

INFORMATION AVAILABLE TO THE

PATIENTS AND THAT INCLUDES

THE PROBLEM LIST.

AND SOME OF OUR PHYSICIANS

WERE REALLY UPSET WITH THE

IDEA THAT PATIENTS WOULD BE

SEEING THEIR PROBLEM LIST

AND WOULD SAY WELL, IT IS

OUTDATED AND INACCURATE

INFORMATION AND WE SAID WELL,

FIX IT.

SO SOME OF THEM WHO NEVER

CARED ABOUT IT BEFORE ARE

NOW GOING TO BE MOTIVATED BY

THE FACT THAT THEY KNOW THE

PATIENTS ARE GOING TO SEE IT.

>> OKAY.

VERY, VERY QUICKLY.

>> WE HAVE MAKING A

DISCIPLINE OF MANAGING THE

PROBLEM LIST SO AT SOME

LEVELS IT'S AN ENTERPRISE.

>> BUT WHO IS RESPONSIBLE?

>> SO AT ONE LEVEL THE

ENTERPRISE IS.

SO IF YOU'VE GOT A GFR OF

LESS THAN 60 AND CKD ON YOUR

PROBLEM POLICE AND THE DOCS

ALL AGREE TO THAT.

THAT IS JUDGMENT.

-- AUTOMATIC.

IF IT LOOKS LIKE YOU MAY

HAVE A PATIENT MIGHT HAVE

DIABETES, WE SEND A MESSAGE

TO THE DOC AND SAY LOOKS

LIKE THEY MIGHT HAVE

DIABETES, NOT ON THE PROBLEM

LIST.

PICK ONE OF THESE THREE.

IF THEY HAVE DIABETES BUT

THE NEUROLOGICAL EBBERS

EFFECTS AREN'T DOCUMENTED,

WE CAPTURE THAT.

SO THERE IS THAT LEVEL.

THERE IS A LEVEL THAT THE PCP

AND SOME RESPECTS OWNS THE

PROBLEM LIST AND SOME OF

WHICH IS A FOOD FIGHT.

>> OKAY.

>> IT'S MORE A PHILOSOPHICAL

QUESTION REALLY WITH THE

DIRECTION WE WANT TO GO.

SO IF WE THINK ABOUT

INFORMATION AND HOW DO WE

ACTUALLY PROVIDE TOOLS TO

IMPROVING -- PROVE THE

QUALITY OF CARE WE'RE

DELIVERING, THE SUPPORT, THE

CAREGIVERS AND THE CONSUMERS

WITH CARE, AS OPPOSED TO

MEASURING THEM AGAIN KIND OF

AN INTERESTING THING.

IF YOU REFLECT BACK ON THE

DAYS OF HETUS, WHERE THE

INTENT WAS TO DO PREVENTIFF

CARE, IT BECAME A WELL,

LET'S GO REPORT AND FOCUS ON

ACTUALLY GETTING THE

MEASURES.

AS WE START LEVERAGING THESE

TOOLS, LET'S THINK ABOUT

WHAT IS IT THAT WE'RE TRYING

TO ACTUALLY ACCOMPLISH AND

HOW DO YOU MOST EFFECTIVELY

DO THAT WITHOUT THE ADVERSE

CONSEQUENCES?

>> I AM A BIG FAN OF DOING

WHAT WE KNOW TO BE THE RIGHT

THING TO DO.

I ALSO PRACTICE AS WELL, AND

I HAVE -- THE OTHER

REQUIREMENT IS THAT MY

THOUGHTS, MY CONFUSION ABOUT

A PERSON'S SYMPTOMS ARE

HELPFUL TO ME AND TO MY

COLLEAGUES IN JUST SORTING

THINGS OUT TO KNOWING WHAT

THE HECK IS THIS CAUSING?

AND SOMETIMES I CAN FIGURE

THAT MYSELF.

SOMETIMES I CAN'T.

SO IN ADDITION TO MAKING

SURE THAT PREVENT PROBLEMS

THAT WE KNOW HOW TO PREVENT,

I ALSO WANT TO SUMMARIZE

WHAT I'VE HEARD AND FOUND SO

THAT I CAN EVENTUALLY FIGURE

OUT OR ASK A COLLEAGUE TO

HELP ME BECAUSE ULTIMATELY,

THAT CAN ALSO SAVE LIVES.

>> YOU MENTIONED PREVENTIVE

MEDICINE.

ONE THING THAT MOST OF US

EMPHASIZED IS THE ELECTRONIC

MEDICAL RECORD BUT THERE ARE

SO MANY SOURCES OF

INFORMATION.

AS TIME GOES ON, I THINK

WE'RE GOING TO SEE MORE AND

MORE PATIENTS ENTERING

INFORMATION, WHETHER IT'S

THEIR WEIGHT OR BLOOD

PRESSURE OR GLUCOSE MONITORING

ET CETERA INTO THE RECORD OR

EVEN TWEETING.

WE'VE BEEN MONITORING TWEETS

AT UNIVERSITY OF MARYLAND

ACROSS THE COUNTRY AND SO AS

FAR AS PREVIVE MEDICINE AND

SURVEILLANCE, I THINK THAT

WE SHOULD LOOK AT NATURAL

LANGUAGE PROCESSING AND

ENHANCED DECISION MAKING

BEYOND JUST THE ELECTRONIC

MEDICAL RECORD AND LOOK AT

THE ENTIRE CONTINUUM AND I

THINK IT'S AEXCELLENT POINT

THAT YOU MAKE.

>> OKAY.

WE JUST WANT TO ASK EVERYONE

ON THE PANEL FOR ONE THING

TOP THEIR WISH LIST, WHAT

WOULD IT BE IF CLINICAL

PRACTICE IF YOU COULD HAVE

IT, IT WOULD CHANGE YOUR

LIFE.

>> MY GIS -- I GUESS THE ONE

THING I'M JUST REFLECTING ON

THE SAFETY PROBLEMS THAT WE

WORK ON BUT WE DON'T FULLY

HAVE.

THE ONE THING I WOULD ASK

FOR WOULD BE A WAY TO

PROTECT US FROM NOT

FOLLOWING OUR OATH TO DO NO

HARM AND TO SAVE THE

PATIENTS BUT ALSO THE

PRACTICEITIONERS WHO SUFFER

WHEN HARM OCCURS.

A BETTER WAY TO DO THAT.

>> I THINK FOR ME WHAT WOULD

MAKE THE BIGGEST DIFFERENCE

IS TO HAVE TRULY SEAMLESS

HEALTH INFORMATION EXCHANGE

SO THAT EVERYTHING THAT WAS

AVAILABLE AND KNOWN ABOUT

THAT INFORMATION WOULD BE

AVAILABLE TO ME WHEN I SEE

THEM.

>> PARTICULARLY IN THE

CONTEXT OF THE DAY.

I WOULD LOVE TO HAVE NATURAL

NLP PROCESSING ENGINE THAT

RAN AGAINST ALL OF THE

DATERA, INCLUDING OUR

COMMUNITY DATA WAREHOUSE AND

WAS ABLE TO PROVIDE

APPROPRIATE INFORMATION WITH

THE BEST ESTIMATE OF ITS

RELIABILITY THAT WE CAN

DEPLOY WHEREVER AND WHATEVER

PROCESS WE NEEDED TO.

>> AND I AGREE WITH FRANK.

WHAT I WAS GOING TO ASK FOR

MAY SOUND LIKE A

CONTRADICTION IN TERMS.

BE ABLE TO HAVE A MECHANISM

TO MONITOR ALL HEALTH

INFORMATION RELATED TO ALL

PATIENTS WITHOUT ANY

COMPROMISE OF PRIVACY OR

SECURITY.

AND FIGURE OUT A WAY TO DO

THAT.

>> AND I THINK IT'S

SOMETHING THAT FOR US TO

STRIVE FOR.

>> LET'S THANK ALL OUR

SPEAKERS AGAIN FOR THIS

SESSION.

[APPLAUSE]

AND I THINK WE'LL TAKE A TEN

TEN-MINUTE BREAK AND

RECONVENE AT 10:50.

THANKS.

>> IN TERMS OF SUPPORT AND

THE PROCESSING.

WE HEARD A TOUB NEEDS AND

WHAT ARE THE CHALLENGES AND

NOW THIS SESSION WE'RE GOING

TO HEAR THE PERSPECTIVES

FROM THE NATIONAL PROCESSING

EXPERTS AND SEE HOW THE

NATIONAL PROCESSING WILL BE

ABLE TO ADDRESS SOME OF

THOSE CHALLENGES AND HELP

MEET THIS NEED.

SO WE HAVE AN EXCELLENT

PANEL OF SPEAKERS FOR THIS

SESSION, AND WE HAVE

DR. ROBERT GREENES FROM

ARIZONA STATE UNIVERSITY AND

DR. LI ZHOU FROM HEALTHCARE

AND DR. STEFAN MEYSTRE FROM

THE UNIVERSITY OF UTAH AND

DR. MARK MUSEN FROM STANFORD

UNIVERSITY.

SO WY LIKE TO CALL EVERYBODY

UP TO THE FRONT ROW, NOT

HERE.

I KNOW IT'S HARD FOR PEOPLE

TO WATCH THE SCREEN FROM UP

FRONT.

BUT ONCE WE GET INTO THE

DISCUSSION, A PORTION OF THE

SESSION, THEN EVERYBODY WILL

SIT IN THE FRONT.

WITH THAT, I WOULD LIKE TO

QUICKLY INTRODUCE THE

KEYNOTE OF THE SESSION,

DR. ROBERT GREENES FROM

ARIZONA STATE UNIVERSITY,

AND IRA FULTON, CHAIR

PROFESSOR OF BIOMEDICAL

INFORMATICS AND PROFESSOR OF

BIOINFORMATICS AT MAYO

CLINIC.

AND HIS RESEARCH IS FOCUSED

ON THE CLINICAL INFORMATICS

AND PARTICULARLY ON CLINICAL

DECISION SUPPORT AND

HEALTHCARE QUALITY

IMPROVEMENT AND THE

APPLICATION OF USEABILITY

AND CLINICAL CARE PROCESS.

SO THIS IS VERY, VERY SHORT

SUMMARY.

AND WITH THAT I WOULD LIKE

TO INVITE DR. GREENES TO

GIVE THE PRESENTATION.

>> THANKS, STEVE.

LET'S SEE HOW TO WORK THIS.

WELL, I'M NOT A LANGUAGE

EXPERT AND I WAS ASKED TO

PROVIDE A PERSPECTIVE ON

CLINICAL DECISION SUPPORT

MORE BROADLY.

I'M KIND OF -- IT'S KIND OF

A RESEARCH AGENDA TALK ABOUT

SOME OF THE REMAINING --

YOU'VE HEARD A LOT THIS

ALREADY BUT I'M GOING TO

KIND OF PROVIDE A

PERSPECTIVE ON THE GRAND

CHALLENGES FOR CLINICAL

DECISION SUPPORT FROM A

CLINICAL PERSPECTIVE.

AND THIS IS ACTUALLY NOT A

NEW THING.

ACTUALLY, THERE WAS A NICE

ARTICLE BY DEAN AND OTHERS

IN 2008 -- 2008 INDICATING

CHALLENGES.

THEY'RE NOT THE SAME AS

MINE.

IN FACT, IN FOUR YEARS A SO

A LOT+%r�A CHANGED ALREADY.

THERE HAVE BEEN SOME=G�S

STUDIES.

RECENTLY THERE WAS DAVE

NOVAK AND COLLEAGUES HAD A

REVIEW OF EFFECTIVENESS OF

C.D.S THAT OUR COMMISSION --

I DON'T KNOW IF IT'S NOT

ACTUALLY RELEASED YET OR NOT.

JONATHAN TISHA HAD A SUMMARY

OF SOME OF THE

STATE-OF-THE-ART OF CRINKICAL

DECISION SUPPORT.

THESE WERE ALL KIND OF

BEGINNING TO SET THE STAGE

FOR WHERE WE ARE NOW.

BUT I THINK THERE IS A LOT

NEW IN THE CURRENT

ENVIRONMENT.

FIRST OF ALL, WE'RE SEEING

FINALLY SOME CONVERGENCE ON

DATA MODELS AND DATA

INTEROPRABLET AND MANY

LADIES AND GENTLEMEN OF THE

JURY-SCALE PROJECTS ARE

BEGINNING TO WORK TOGETHER

TO TRY TO GET TO THAT AIM.

OBVIOUSLY, WE NEED THAT MORE

DECISION SUPPORT.

THERE IS ALSO INCREASINGLY

THE AVAILABILITY OF LARGE

DATABASES.

WE'VE HEARD ABOUT VINCI.

PARTNERS HAS A DATABASE.

THERE ARE MANY SOURCES NOW

WHERE WE CAN BEGIN TO DO

POPULATION-BASED KIND OF

EVALUATIONS.

THE OTHER -- ANOTHER KEY

THING THAT'S DRIVING

DECISION SUPPORT NEW

PERSPECTIVES IS THE IDEA OF

THE CONTINUITY OF CARE

ACROSS THE CONTINUUM AND

PATIENT-CENTERED MEDICAL

HOMES, THE ACCOUNTABLE CARE

ORGANIZATION IS BEGINNING TO

CHANGE THAT FOCUS TO LOOKING

AT THAT WHOLE ISSUE.

AND THIS IS ALSO BEING

DRIVEN BY MEANINGFUL USE

INITIATIVES THAT ARE

EMPHASIZING THE NEED FOR

CONTINUITY AND FOR QUALITY

AND FOR VALUE AND FOR

EFFICIENCY.

ANOTHER THING THAT'S NOT

OFTEN REALIZED AS THIS

EMERGE OF AN APP CULTURE

BECAUSE I THINK WHAT HAPPENS

WITH THAT IS IT NOW FREES

OUR THINKING ABOUT HOW TO

PRESENT AND UTILIZE DECISION

SUPPORT IN OTHER KINDS OF

WAYS TO INTERACT WITH OUR

PATIENT CARE DELIVERY TASKS

MORE EFFICIENTLY AND MORE

EFFECTIVELY.

SO IT CREATES AN INNOVATION

ENVIRONMENT.

AND I THINK WITH ALL OF

THESE AND WITH THE CONCEPT

OF CONTINUITY OF CARE, WE'RE

BEGINNING TO SEE GLIMERS OF

SOMETHING THAT SOME OF US

HAVE TALKED ABOUT FOR YEARS,

THE INTEGRATIVE LONGITUDENAL

PERMANENT CARE RECORD.

SO I ALWAYS LIKE THIS SLIDE

BECAUSE I'VE SHOWN IT FOR 20

YEARS AND SO MY TOP TEN LIST

OF CHALLENGES ACTUALLY FITS

IN THREE CATEGORIES.

ONE IS THE THE FRAMEWORK FOR

CARE, AND I'LL TALK ABOUT

EACH OF THESE BRIEFLY.

THE KNOWLEDGE SOURCES, HOW

WE DERIVE THE KNOWLEDGE AND

ASSEMBLE IT AND THEN HOW WE

USE IT.

SO WE'LL TALK ABOUT EACH OF

THESE.

I'M NOT GOING TO SPEND TIME

READING THROUGH THIS SLIDE.

BUT THE SCOPE OF THE CARE

PROCESS I'VE TOUCHED ON

ALREADY IS REALLY NOW

INCREASINGLY ABLE TO BE

CONSIDERED TO BE THE WHOLE

CARE CONTINUUM, INCLUDING

HEALTH AND WELLNESS.

WE'RE BEGINNING TO SEE AT

LEAST TALK ABOUT BUT THERE

IS NO EMERGENCE OF A

LONGITUDENAL PATIENT RECORD

INFORMATION MODEL.

WE DO FOCUS ON HOW<u TO GET

DATA ACROSS EHRS AND

CONTINUITY OF CARE

DOCUMENTS.

WE TALK ABOUT PHRS SOMETIMES

TETHERED TO A HEALTH SYSTEM

BUT INDEPENDENT SOMETIMES

THERE IS NO STANDARD FOR

THAT YET TO JUST

MENTION ROLES FOR TECHNIQUES

AND I DON'T THINK I HAVE

TIME TO ACTUALLY GO INTO IT

BUT I TALKED ABOUT ALL OF

THESE.

AND LASTLY, SO I THINK THAT

I CAN SAY THAT THE C.D.S

LANDSCAPE WILL CONTINUE TO

CHANGE.

WE HAVE A LOT OF

OPPORTUNITIES TO RETHINK THE

CARE PARADIGM BY INTEGRATING

THE DIVERSE DATA AND

KNOWLEDGE SOURCES, FOCUSING

ON USEABILITY AND CONTINUITY

AND COORDINATION OF CARE

PARADIGM AND DESIGNING

DECISION SUPPORT TO SUPPORT

THESE TASKS AND DEVELOPING

APP THAT'S WILL ALLOW YOU TO

FREE OURSELVES FROM THE

LOCK-IN OF CURRENT VENDOR

SYSTEMS.

AND IN ALL OF THESE

IMPORTANT ROLE FOR

TECHNIQUES.

OKAY.

[APPLAUSE]

>> MAYBE WE HAVE ONE FOR ONE

OR TWO QUICK QUESTIONS.

GO AHEAD.

>> THAT WAS A REALLY BROAD

BROAD-RANGING TALK.

IT WAS VERY HELPFUL IN

SETTING THE LANDSCAPE.

MY QUESTION IS WHEN YOU TALK

ABOUT THE POTENTIAL FOR THE

DEVELOPMENT OF APPLICATIONS

TO SUPPORT C.D.S OR PATIENTS,

PROVIDERS AND SYSTEMS, WHAT

DO YOU SEE AS THE ROLE OF

INNOVATION AND PERHAPS

PUBLIC-PRIVATE SECTOR

PARTNERSHIPS TO HELP

STIMULATE THE DEVELOPMENT OF

THESE C.D.S-RELATED

APPLICATIONS?

>> ACTUALLY, I THINK IT'S

ESSENTIAL.

WE ACTUALLY HAD A MEETING IN

SCOTTS DALE, FEBRUARY 1-3RD

AT THE SKPRAMPLET THE DODD

HEALTH MAYO CLINIC.

GEORGIA TECH TOOLS AND I

FORGET WHO HE IS.

HARVARD SMART PROJECT.

AND THE GOAL WAS REALLY TO

SEE IF WE COULD AGREE ON THE

NEED FOR A SET OF

SPECIFICATIONS BEYOND DATA

INTEROPRABLET.

BUT THERE WERE A LOT OF

THINGS LIKE CONTEXT

MANAGEMENT AND SO ON THAT

NEEDED TO BE ADDRESSED.

SO WE CAME AWAY WITH THAT

WITH A PRETTY RESOUNDING YES

AND IT SHOULD BE DRIVEN

PERHAPS BY THESE LARGE-SCALE

CARE PROVIDERS THAT COULD

ACTUALLY ARTICULATE THE NEED

FOR IT AND MAYBE BUILD

DEMONSTRATIONS.

SO ONE PART OF THAT IS TO

KIND OF CREATE SAND BOXES,

WHICH ARE SO THE CONSORTIUM

WOULD BE A PUBLIC-PRIVATE

DEVELOPMENT OF THIS SET OF

SPECIFICATIONS.

BUT THEN THE SAND BOXES CAN

BE SEMI COMMERCIAL.

THEY BASICALLY WILL BECOME

ENTREPRENEURIAL WORK SPACE,

ECOSYSTEM THAT'S CAN ALLOW

APP DEVELOPERS TO WORK WITH

LOWER-LEVEL SERVICE AND TOOL

PROVIDERS AND CREATE KIND OF

THE TECHNICAL PHONY

FRASTRUCTURE TO ALLOW THEM

TO WORK TOGETHER TO

PROT-TYPE, BUILD THE

APPLICATIONS.

SO IN OUR MAYO ENVIRONMENT

WE'RE FOCUSEDING ON THE

CONTINUITY OF CARE

COORDINATION ISSUES, THERE

IS MANAGEMENT THAT THEY ARE

TRYING TO COME UP WITH A

UNIVERSAL SOLUTION FOR AND

THERE IS OTHER THINGS THAT

ARE DRIVING IT.

I THINK WE'LL SEE OTHER

EXAMPLES LIKE THAT.

>> THANKS.

[APPLAUSE]

>> OUR NEXT SPEAKER IS

DR. LI ZHOU FROM THE

HEALTHCARE SYSTEM.

PARTNERS HEALTHCARE SYSTEM.

>> GOOD MORNING, EVERYBODY.

MY NAME IS LI ZHOU.

I'M FROM PARTNERS HEALTHCARE

AT HARVARD MEDICAL SCHOOL.

SO WHY DO WE NEED THE

PROCESSING IN C.D.S?

THE USE OF ELECTRONIC NOTES

IN OUR AM BLAHTRY ELECTRONIC

HEALTH RECORDS.

YOU CAN SEE THAT THE

AVAILABILITY OF THE

ELECTRONIC NOTES HAD

INCREASED DRAMATICALLY IN

THE LAST 20 YEARS.

AND WE HAVE 13 MILLION NOTES.

WE ALSO KNOW MUCH

INFORMATION ABOUT C.D.S IS

TEXTULE DATA AND THEREFORE

CANNOT BE REACHED WITHOUT SRP.

FIRST I WANT TO TALK ABOUT

THE PERSPECTIVE OF THESE TWO

AREAS.

WHEN I WAS A DOCTOR STUDENT

AND PURSUED BY PH.D IN

BIOMEDICAL INFORMATICS AT

COLUMBIA UNIVERSITY, MY

DISSERTATION FOCUSED ON THE

PROCESSING.

WE USED AN RP SYSTEM THAT

CONVERT THE NARRATIVE REPORT

INTO A SPECIAL FORMAT AND

DEVELOPED A SYSTEM CALLED

TEMPEST TO IDENTIFY THE TEM

PRINCIPAL ASPECTS OF MEDICAL

EVENTS TWO YEARS BEFORE THE

PATIENT WAS DIAGNOSED WITH

HEPATITIS.

THE PATIENT HAD A LIVER

TRANSPLANT ON JUNE, 199 IT.

HE UNDERWENT A STUDY AND

THEN PRESENT A FEVER FOR TWO

DAYS.

SO USING -- BY CONTACTING

PROCESSING AND PRECISIONING,

WE COULD IDENTIFY WHEN THE

EVENT OCCURRED, HOW LONG IT

LASTED, AND THE DURATION AND

DIFFERENT MEDICAL EVENTS AND

ALSO DEDEUCE THE TEM

PRINCIPAL RELATIONSHIPS SUCH

AS AFTER AND BEFORE.

SO I JOINED CLINICAL

INFORMATICS RESEARCH AND

DEVELOPMENT TEAM AT PARTNERS

A FEW YEARS AGO.

THE GROUP IS LED BY DR. GEN.

I AM PARTICULARLY IN THE

CLINICAL SUPPORT TEAM, WHERE

I HAVE A GREAT OPPORTUNITY

TO LEARN A LOT OF THINGS

REGARDING TO REALTIME

CLINICAL SUPPORT.

WE ARE INVOLVED IN

DEVELOPING AND MAINTAINING A

DIVERSE INCLUDING REMINDERS,

ALERTS, ET CETERA.

AND THOSE CLINICAL

INTERVENTIONS HAVE IMPLEMENTED

OUT-OF-PATIENT AND

IN-PATIENT SETTINGS.

INCLUDING C.P.OE PRESCRIBING.

IN A RESEARCH PROJECT LED BY

DR. MIDDLETON, WE

INVESTIGATE REPRESENTING THE

CLINICAL KNOWLEDGE AND ALSO

DID ELEMENTS FOR THOSE C.D.S

INTERVENTIONS TO SHARE

ACROSS DIFFERENT CLINICAL

SETTINGS AND DIFFERENT

SYSTEMS.

WE ALSO IMPLEMENTED SERVICE

SERVICE-ORIENTED

ARCHITECTURE TO CENTRALIZED

C.D.S SERVICE TO SHARE

KNOWLEDGE AND ALSO ACHIEVE

SYSTEM INTEROPERAABILITY.

SO BY BEING INVOLVED IN

THESE TWO AREAS, THERE IS

ONE QUESTION OF HOW WE APPLY

OUR RP INTO C.D.S.

THERE IS MANY AREAS WE CAN

APPLY, AS MENTIONED.

HERE I WANT TO FOCUS ON -- I

WANT TO -- I WILL GIVE A FEW

EXAMPLES IN THE FOLLOWING

THROW AREAS.

IMPROVING PATIENT SAFETY.

YOU HAVE EHR FUNCTIONS AND

REDUCE THE HEALTHCARE COST.

FOR PATIENT 50 I WANT TO USE

A MEDICATION LIST AS AN

EXAMPLE.

WE KNOW MEDICATION ERRORS

CAN CAUSE INJURIES.

MEDICATION ERRORS ARE

ESTIMATED TO COST THE U.S.

HEALTHCARE SYSTEM $177

BILLION PER YEAR.

MEDICATION-BASED PATIENTS

RECORDS ARE ALSO OUTDATED,

INCOMPLETE OR INACCURATE,

WHICH IS A MAJOR CAUSE OF

MEDICATION ERRORS.

RESEARCH FUND ACTIVE

MEDICATIONS ARE ALSO NOT

ADDD IN A TIMELY MANNER.

ONE FINDS DISCREPANCES

BETWEEN THE MEDICATION

THAT'S PATIENTS REPORT

TAKING.

IN ADDITION, ONE STUDY FOUND

7% OF MEDICATIONS WAS DUE

ONE CALENDAR DAY AFTER THE

INACTIVE WAS DOCUMENTED IF

THEIR CLINICAL NOTES.

MEDICATION APPLICATIONS

BECAUSE OF THOSE ERRORS.

APPLICATIONS HAVE BEEN BUILT

TO ADDRESS THIS ISSUE.

[INDISCERNABLE]

 THIS SCREEN SHOT SHOWS AN

EXAMPLE OF THE MEDICATION

LIST.

AND THIS SLIDE SHOWS YOU AN

EXAMPLE OF APPLICATION TO

RECONCILE A MEDICATION LIST

AFTER THE CHART.

HOWEVER, THOSE MED REC

APPLICATIONS USING HIGHLY

RELIED ON STRUCTURED DATA.

ONE OF OUR STUDIES WE FOUND

30% OF ACTIVE MEDICATIONS

WERE FROM PATIENTS

MEDICATION LISTS,

PARTICULARLY PROSCRIBED BY A

SPECIALIST OUTSIDE THE

INSTITUTION.

IN ADDITION, CLINICIANS

OFTEN NEED ADDITIONAL

INFORMATION BEYOND THE

MEDICATION LIST IN ORDER TO

MAKE JUDGMENT, CHANGES OR

MAKE OTHER DECISIONS.

FOR EXAMPLE, WANT TO EXTEND

THE HISTORY UNDERSTAND THE

PROGRESS OF THE DISEASE AND

LOOK AT THE CONSULTATION

NOTES FROM MEDICAL

SPECIALISTS.

ONE POTENTIAL APPLICATION WE

CAN USE EXTRACTS MEDICATION

AND WE CAN PUT -- THIS INTO

THE EXISTING APPLICATION.

[INDISCERNABLE]

 THIS ALSO PROVIDE A WARNING

TO THE PROVIDERS WHEN THEY

FOUND A HIGH LETTER OF

IMPORTANT MEDICATIONS.

ALSO WE CAN PROVIDE A NOTE

BUTTON ALONGSIDE OF EACH

MEDICATION JUST LIKE TO SEE

THE NOTES BETTER -- THE

MEDICATION WITH NOTES WAS

MENTIONED.

HOWEVER, THERE ARE MANY

CHALLENGES TO THOSE, THOUGH.

HOW WE INTEGRATE MRP WITH

C.D.S?

ONE IS THE INTEROPRABLET AND

THE TERMINOLOGY STANDARD.

WE KNOW MEDICATION LISTS MAY

BE CODED USING AN

INSTITUTIONAL OR COMMERCIAL

TERMINOLOGY WHERE MOST

EXISTING ENCODING CLINICAL

TESTS IS TERMINOLOGIES LIKE

PEOPLE MENTIONED.

SO THIS REQUIRES THE SYSTEM

ENCODED INFORMATION MODIFIED

TERMINOLOGIES.

IT REQUIRES THE SYSTEM TO

INTEGRATE A SYSTEM WITH

OTHER APPLICATION AND

CONTACT DATA INTERGATION,

AGGREGATION, NARRATIVE DATA

AND STRUCTURED DATA.

SO IT WOULD BE NICE TO

CONDUCT AND GIVING INFERENCE

USING, FOR EXAMPLE,

KNOWLEDGE BASE.

THE SECOND AREA I WANT TO

TALK ABOUT HOW TO USE NRP IN

HER FUNCTIONS.

HERE USE CPOE AS AN EXAMPLE.

SEVEN PERCENT OF ORDER

ENTRIES.

EVEN THE C.P.OS HAVE BEEN IN

PLACE FOR 20 YEARS HAS A LOT

OF FREE TEXT ENTRIES.

WE ALSO FOUND THE% OF

HYPOGLYCIC MEDICATIONS WERE

USED IN THIS CONTEXT.

75 OF THOSE FREE TEXT

ENTRIES HAVE AN EXACT MATCH

IN OUR MEDICATION DICTIONARY

AND THE REMAINING 25 COULD

BE CODED AS FORMULA

INFORMATION WAS ALSO PROVIDEED.

WE ALSO SAW 17% OF FREE TEXT

HYPOGLYC PEMIC MEDICATION IN

ORDER ENTRIES.

HERE I STILL USED SEVERAL

MOST COMMONLY MISSPELLED

TERMS, THEY LOOK AND SOUND

SIMILAR.

SO WHAT DO THIS AFFECT?

C.D.

C.D. IS NOT TRIGGERED AS A

MEDICATION IN FREE TEXT.

USING A SIMILAR DATA WE

FOUND TRIGGER -- WERE NOT

TRIGGERED DUE TO FREE TEXT

ENTRIES AFFECTING 84

DIFFERENT PATIENTS.

196 PATIENTS WHO HAD A FREE

TEXT ENTRY HAD THE SAME

EXACT DRUG IN THIS PERIOD.

AND 10% OF THEM HAVE HAD

IDENTICAL DRUG ENTRIES

ACTIVE USING THE MEDICATION

LIST AT THE SAME TIME.

THIS IS KIND OF DUPLICATE

THERAPY ERROR.

ONLY 26 OF THOSE PATIENTS

HAD DIABETES RECORDD IN

THEIR PROBLEM LIST.

SO WE HAVE AN ISSUE RELATED

TO PROBLEM LIST IF THEY ARE

NOT COMPLETED AS WELL WELL.

THOSE C.D.S ARE CRITICAL IN

PATIENTS SAFETY.

SO OUT OF SEARCH FUNCTION.

SUCH FUNCTION SHOULD NOT BE

LIMITED TO ONLY DETECTING

EXACTLY WHEN THE MEDICATION

ENDS.

IT SHOULD PROVIDE RELEVANT

AND A SMART LIST, NOT.

A LONG LIST TO SORT THROUGH.

IT SHOULD HAVE A SPANNING

ERROR DETECTION AND FRACTION.

IT CAN COMPRISE A LIST OF

SUGGESTIONS FOR THE

CORRECTION.

IMPORTANTLY, WE NEED TO

DESIGN EFFICIENCY TO ADDRESS

WORK FLOW ISSUES.

FOR EXAMPLE, A FEW FEATURES.

THEY ALLOW -- PROVIDE US TO

CREATE THEIR OWN LIST.

HOWEVER, THIS WILL BE

MONITORED.

AND WE ALSO SHOULD AVOID

NAVIGATING TO SAVE THEIR

TIME.

IT WOULD BE MORE EFFICIENT

IF WE CAN INCORPORATE SPEECH

RECOGNITION.

SO FOR EXAMPLE, IT MAY BE

ABLE TO REDUCE THAT.

THE LAST EXAMPLE IS HOW WE

ARE USING IT TO HAVE REDUCED

HEALTHCARE COST.

THERE IS A GREAT NEED TO

MINIMIZE THE COST OF CARE

DELIVERED MEETING QUALITY

INSTITUTES.

FIVE PERCENT OF PATIENTS

GENERALLY ARE UP TO 50 OF

THE COSTS.

ONE EXAMPLE IS HOW WE USE IT

TO PREVENT READMISSIONS BY

IDENTIFYING HIGH-RISK,

HIGH-COST.

MOST CURRENT RISK ASSESSMENT

USING STRUCTURED DATA.

CLINICAL NARRATIVE REPORT --

[INDISCERNABLE]

 ONE POSSIBLE SOLUTION IS TO

COMBINE DATA WITH DATA USING

NRP SO WE CAN IDENTIFY THE

TARGETED PATIENT'S

POPULATION.

AND THEN EMPLOY A METHOD

LIKE CLASSIFICATION

TECHNIQUES AS MODELS TO

INFORM PATIENTS AND PROVIDE

A C.D.S AND MAKE A

RECOMMENDATION BASED ON

CLINICAL DATA LINES.

NOW A BRIEF SUMMARY AND

DISCUSS THE POTENTIAL

OPPORTUNITIES AND ALSO

CHALLENGES IN THE FIELD.

HERE ARE DR. MIDDLETON'S TEN

COMMANDMENTS FOR AN

EFFECTIVE C.D.S.

AS WE SEE, THERE ARE MANY

OPPORTUNITIES TO APPLYING NRP

TO ENHANCED C.D.S.

WE NEED TO REMEMBER IT CAN

MAKE A BIG DIFFERENCE.

WE NEEDED TO MAKE THE SYSTEM

USEFUL.

WE NEED TO MAKE IT EASY FOR

A CLINICIAN TO DO THE RIGHT

THING.

HOWEVER, THERE ARE MANY

CHALLENGES TO IT.

IF IT TAKES TOO LONG TO

WORK, IT WILL BE USELESS.

[INDISCERNABLE]

 THE SYSTEM COULD BE ABLE TO

ANTICIPATE A NEED AND

DELIVER IN REALTIME.

WE CAN -- THE SYSTEM SHOULD

BRING INFORMATION TO

CLINICIANS AT THE TIMES THEY

NEED IT.

THE APPLICATION NEEDS TO FIT

INTO THE USER'S.

FOR EXAMPLE, IF A PATIENT

USES THE INTERFACE TO OUTPUT.

AS MENTIONED BEFORE,

STANDARD AND SYSTEM

INTEROPABILITY WERE

IMPORTANT.

AND COULD WE PROVIDE SERVICE

SERVICE-ORIENTED

ARCHITECTURE TO PROVIDE

CENTRALIZED MRP SERVICE TO

DIVERSE EHI SYSTEMS?

THERE ARE MANY OTHER ISSUES

LIKE ENCODING ORGANIZATIONAL

ISSUES, DIVERSE CLINICAL

DOMAIN, USERS MEETING THE

NEEDS OF THE USERS WITH

DIVERSE BACKGROUND AND NEEDS.

THERE ARE OTHER REQUIREMENTS

IN THE FIELD AS WELL.

SIMPLE INTERVENTIONS WORK

BEST.

SO WE NEED TO SIMPLIFY AND

MAKE IT USEFUL.

LASTLY BUT IMPORTANTLY, WE

NEED TO CONTINUE MONITOR THE

FEEDBACK AND MAKE PERMANENT

THOSE APPLICATIONS.

I AM DONE.

[APPLAUSE]

>> WE HAVE TIME FOR ONE OR

TWO QUESTIONS.

GO AHEAD.

>> DO YOU TRY TO USE THE

CONSUMER AS A SOURCE OF DATA

AND MAYBE MISSPELLED DRUG

NAMES FROM PHR?

AND EVEN BE ABLE TO VIEW

WHAT'S MY CURRENT MEDICATION

ON RECORD AND ADD THE

OVER-THE-COUNTER MEDICATION?

>> THIS IS A GREAT

SUGGESTION.

WENT DONE THIS YET.

BUT I THINK IT'S VERY

INTERESTING RESEARCH AREA.

>> DAVID CAROL, GROUP

HEALTH.

I'M CURIOUS IN THE C.D.S

AREA IF YOU ARE SEEING

ANYTHING THAT MIGHT BE

CALLED LIKE A GOOGLE EFFECT

WHERE WE'RE GETTING

ACCUSTOMED TO JUST BEING

ABLE TO REMEMBER A FEW WORDS

RELATED TO SOMETHING EITHER

IN OUR EMAIL AND OUR WEB

SEARCHES AND EXPECT THAT

THAT SHOULD BE THE STANDARD

OF OUR OWN PERFORMANCE IN

ORDER TO FIND THINGS THAT WE

NEED TO IN OUR RECORD?

>> YES.

THIS IS A GOOD SUGGESTION,

YES.

I THINK THE TECHNOLOGY IS

THERE.

WE JUST NEED TO BRING THOSE

KINDS OF TECHNOLOGIES TO

RETRIEVE RELEVANT

INFORMATION AND THE

INFORMATION WE REALLY NEED

IN REALTIME.

YEAH.

THANK YOU.

[APPLAUSE]

>> OUR NEXT SPEAKER IS

DR. STEPHANE MEYSTRE.

>> THANK YOU.

LET ME FIND THIS.

GOOD MORNING AND THANK YOU

FOR THE OPPORTUNITY.

I THINK THIS IS A REALLY

EXCELLENT WORKSHOP AND VERY

NICE MIX OF FEEDBACK.

AND WHAT I AM GOING TO TELL

YOU ABOUT FOR A CHANGE IS IN

LP AND CLINICAL DECISION

SUPPORT.

I AM FROM THE DEPARTMENT OF

BIOMEDICAL INFORM ARTICS AT

THE UNIVERSITY OF UTAH.

WE HAVE MANY COLLABORATIONS

WITH SALT LAKE CITY V.A. AND

SO I'D LIKE TO START ABOUT

ASKING A QUESTION THAT

SEVERAL PRESENTERS ALREADY

ADDRESSED ABOUT WHY NLP

COULD BE USEFUL?

WHY BOTHER?

THERE IS ONE SIDE OF A

REALLY FAST GROWTH OF

INFORMATION THAT'S BECOMING

AVAILABLE MORE AND MORE

SYSTEMS USE DHRS AND MORE

REQUIREMENTS FOR ELECTRONIC

DOCUMENTATION.

THERE IS ALSO A HUGE GROWTH

ON OTHER TYPES OF ELECTRONIC

INFORMATION FROM DIFFERENT,

NEW INVESTIGATIONS, GENETIC

TESTS, ET CETERA.

BUT THE PROBLEM IS THAT MOST

OF THE INFORMATION IS NOT

STRUCTURED AND CODED.

IT'S NAR NIV TEXT.

AND THE EXISTING STRUCTURED

INFORMATION AND CODED

INFORMATION FOR MOST OF IT

IS CREATED FOR

ADMINISTRATIVE AND

>> I REMEMBER BURSEMENT

PURPOSES, NOT FOR CLINICAL

CARE.

SO IF WE LOOK AT THE

ELECTRONIC HEALTH RECORDS,

MOST OF THE CONTENT IS

NARRATIVE TEXT.

IT'S DOCUMENTS.

HISTORY AND PHYSICAL, ORDERS,

PROGRESS NOTES, ET CETERA,

THAT WE HAVE SOME IMAGING

REPORTS, PRESCRIPTIONS.

THIS IS BECOMING MORE

STRUCTURED, LAB RESULTS,

ADMINISTRATIVE INFORMATION

THAT'S NOT STRUCTURED AND

YOU SEE THAT MOST OF THE HR

CONTENT IS UNSTRUCTURED,

NARRATIVE TEXT, AND THIS IS

NOT USABLE FOR CLINICAL

SUPPORT DIRECTLY.

CLINICAL DECISION SUPPORT

NEEDS STRUCTURED AND

DETAILED INFORMATION,

INFORMATION THAT'S

STRUCTURED USING SOME DATA

MODEL AND THAT IS CODED

USING SOME CENTERED

TERMINOLOGIES.

AND ACTUALLY, MOST DHR

CONTENT IS NARRATIVE TEXT

AND IT'S UNSTRUCTURED AND

NOT ACCESSIBLE FOR CLINICAL

DECISION SUPPORT.

AND ALSO INFORMATION NEEDS

TO BE DETAILED AT DIFFERENT

LEVELS.

AND THE ONLY EXISTING --

MOST OF THE EXISTING

STRUCTURED INFORMATION IN DHRS

NOW IS CODED FOR PUBLIC

HEALTH STATISTICS OR

REIMBURSEMENT AND DOESN'T

ALLOW FOR ENOUGH DETAILS AND

CLINICAL CARE CODING.

SO ONE WAY TO DEAL WITH THIS

IS TO USE EXTRACT TO

INFORMATION FROM ALL THE

NARRATIVE TEXT CONTEXT AND

ELECTRONIC HEALTH RECORDS.

AND THIS IS FOR MOST OF IT

CALLED CLINICAL INFORMATION

EXTRACTION.

INFORMATION EXTRACTION

INVOLVES THREE DIFFERENT

TYPES OF INFORMATION.

SO IT'S NOT THE WHOLE,

COMPLETE ANALYSIS OF

EVERYTHING THAT'S EXPRESSED

AND MENTIONED BUT IT'S

FOCUSING ON SOME SPECIFIC

TYPES OF INFORMATION OF

INTEREST FOR CLINICAL CARE.

AND THE DEVELOPMENT OF

INFORMATION EXTRACTION HAS

ALREADY QUITE A HISTORY IN

THE MEDICAL, BIOMEDICAL

DOMAIN, CLINICAL DOMAIN.

MUCH BUT MUCH MORE IN THE

BIOMEDICAL SIDE BECAUSE OF

AVAILABLE DATA.

CLINICAL DATA, AS WAS

MENTIONED YESTERDAY, IS

STILL VERY DIFFICULT TO

OBTAIN AND THIS AVAILABILITY

AND ALSO CHARACTERISTICS OF

THE CLINICAL TEXT MAKE IT

DIFFICULT FOR INFORMATION

EXTRACTION.

CHARACTERISTICS OF CLINICAL

TEXTS LIKE GRAMMATICAL

STRUCTURES, DEMOGRAPHIC

STYLE, A LOT OF ABBREVIATIONS

AND ACRONYMS AND THIS IS

BECOMING EVEN MORE IMPORTANT

WHEN THE INFORMATION IS

MANUALLY ENTERED BY THE

HEALTHCARE PROVIDERS.

THEY WANT TO DO IT FAST AND

SO THEY ABBREVIATE AND THESE

ABBREVIATIONS ARE OFTEN

SPECIFIC TO THEIR SPECIALTY,

TO THEIR INSTITUTION OR EVEN

TO THEMSELVES.

SO IT'S A DIFFICULT PROBLEM

ALSO.

SO NOW I AM GOING TO TELL

YOUE"JJT7u WHO:j TWO-EXAMPLES

THAT WE HAVE DONE.

THE FIRST ONE WAS CALLED THE

AUTOMATED PROBLEM LIST AND

THE PROBLEM THERE AND THIS

HAPPENED A FEW YEARS AGO.

THE PROBLEM IS THAT THERE

WAS AN ELECTRONIC PROBLEM

THAT WAS AVAILABLE BUT IT

WAS MOST OF THE TIME

INCOMPLETE, LIKE THIS ONE

HERE.

AND THIS ONE IS] -- HAS

ALREADY SOME ENTRIES AND WAS

NOT USED AT ALL.

AND AT THE SAME TIME THERE

WERE ALSO EFFORTS TOTHA

NEEDED INFORMATION FROM THE

PROBLEM LIST AND FROM

ADDITIONAL C.P.OE OF

CLINICAL ZPIRGS, FOR EXAMPLE

AND OTHER APPLICATION THAT'S

REALLY NEEDED A COMPLETE,

ACCURATE AND TIMELY PROBLEM

LIST.

SO WHAT WE DID IS TO THEN

DEVELOP A SYSTEM TO EXTRACT

POTENTIAL PROBLEMS FROM ALL

THE NARRATIVE TEXT INTO THE

HEALTH RECORDS AND THIS

SYSTEM USED DIFFERENT STEPS

THAT WERE DISCUSSIOND IN

SEVERAL EXAMPLES YESTERDAY

STARTING WITH DETECTING AND

STRUCTURE THE DOCUMENT, THE

SECTIONS, THE SENTENCES,

THEN DISAM BIGATING ALL

THESE ABBREVIATIONS AND

ACRONYMS I JUST MENTIONED.

AND THEN USED MM T X.

THIS WAS FROM THE NATIONAL

LIBRARY OF MEDICINE HERE TO

MAP CONCEPTS AND THEN ALSO

THE NEGATION DETECTION AND

DID SOME POST PERSISTING

ALSO TO TAKE INTO ACCOUNT

AND CORRECT THE FACT THAT MMTX

WERE FOR BIOMEDICAL TEXT.

SO THE AM BIGUT WAS DEALT

WITH DIFFERENTLY.

FOR EXAMPLE, DIFFERENT

COMMON ACRONYMS WERE

UNDERSTOOD AS MENTAL

DEPRESSION AND THIS WAS JUST

AN EXAMPLE.

MAPPED LOCAL CODES IN

HEALTHCARE AND THEN CREATED

AND FED BACK THE INFORMATION

INTO THE ELECTRONIC HEALTH

RECORD IN TWO FORMATS.

HL 7S.CA DOCUMENTS ALONG

WITH THE SECTIONS ENCODED

AND EACH INDIVIDUAL MEDICAL

PROBLEM USING THE LOCAL

INFORMATION MODEL AND

TERMINOLOGIES IN HEALTHCARE.

AND THEN THIS INFORMATION

WAS MADE AVAILABLE IN THE

H.R. AND WE IMPLEMENTED IT

AT THE HOSPITAL IN SKPLAX IT

LOOKED A LITTLE BIT -- SALT

LAKE CITY, AND IT LOOKED A

LITTLE BIT LIKE THIS.

IT'S NOT SUPPOSED TO BE

READABLE.

WHAT YOU SEE IS THERE IS

MANY MORE ENTRIES HERE AND

SO WHAT WE DID IS TO FIRST

MAKE SURE THAT USERS KNEW

WHERE THE INFORMATION CAME

FROM.

AND THEN WE ALLOWED THEM TO

CHANGE OR EDIT THE STATUS OF

THE PROBLEM THAT WAS

AUTOMATICALLY PROPOSED HERE

TO ALLOW THEM TO ASSESS

WHETHER IT WAS A CORRECT

CURRENT PROBLEM OR AN ACTIVE

PROBLEM OR A RESOLVED

PROBLEM, ET CETERA.

AND SO WE REALLY WANTED TO

HAVE THE HUMAN IN THE LOOP

HAVE THE FINAL DECISION FOR

ALL INFORMATION THAT BECAME

OFFICIALLY PART OF THE

HEALTH RECORDS.

AND THIS IS SOMETHING I'LL

MENTION AT THE END THAT WAS

REALLY IMPORTANT.

AND THEN ALSO TO ALLOW THEM

TO SEE AND HAVE AN IDEA

WHERE THE INFORMATION CAME

FROM BECAUSE SOMETIMES THEY

WERE NOT AWARE OF THE

INFORMATION WE WERE

PROPOSING.

THEN THEY COULD SLICK CLICK

ON THE SOURCE BUTTON AND

THEN SEE ALL THE SOURCE

DOCUMENTS WITH THE SENTENCES

AND THE PROBLEM HIGHLIGHTD

IN RED.

NOT VERY VISIBLE HERE.

BUT ALLOW THEM IN A FEW

SECONDS TO CHECK FOR

THEMSELVES WHAT INFORMATION

WAS EXTRACTED.

AND SO WE FIRST TESTED THE

NATURAL PERSISTING

EXTRACTION PART OF IT AND

COMPARED ALSO DIFFERENT

METHODS TO DO IT.

SO THIS IS A SYSTEM THAT I

AM TALKING ABOUT.

WE ALSO COMPARED IT AS A

BASELINE WITH A SIMPLE

SEARCH USING ALL ENTRIES AND

ALSO COMPARING THEM WITH THE

INDIVIDUAL HUMAN REVIEWERS,

THEY WERE CLINICIANS.

AND YOU SEE THAT HUMANS HAD

AN ADVANTAGE.

THEY WERE MORE PRECISE AND

WHAT WE FOUND WAS MOST OF

THE TIME CORRECT.

BUT THEY WERE NOT AS

SENSITIVE AS OUR SYSTEM.

AND THIS IS THE ADVANTAGE WE

BROUGHT.

ADD SOME CONTENT THAT HUMANS

WERE MISSING.

AND THEN WE IMPLEMENTED THE

SYSTEM IN SALT LAKE CITY IN

THE MEDICAL AND SURGICAL ICU

AND CARDIOVASCULAR SURGERY

AND EVALUATED IT IN A RANDOMIZED

CONTROLLED TRIAL.

SO THIS WAS AN VALUATION OF

THE IMPACT OF THE SYSTEM ON

THE CONTENT OF THE PROBLEM

WITH CONTROL, TEST GROUP AND

ABOUT 250 PATIENTS.

AND IN THE END WE OBSERVED

THAT, ESPECIALLY IN THE

INTENSIVE CARE UNIT, WE

FWRENT A SENSITIVITY, A

PROPORTION OF PROBLEMS THAT

WERE SUPPOSED TO BE IN THE

PROBLEM LIST AND ACTUALLY

FOUND IN THE PROBLEM LIST OF

ABOUT 9% TO 41% AND THEN IF

WE ALSO INCLUDED PROBLEMS WE

PROPOSED THAT WERE NOT

ASSESSED BY THE PHYSICIAN,

BY THE USER OF THE PROBLEM

LIST, THEN IT WAS ALMOST 78%

BUT THE SPECIFICITY WENT

DOWN A LITTLE BIT.

AND YOU SEE ALSO THAT IN THE

CARDIOVASCULAR SURGERY UNITS

THERE WERE NO EFFECT AND

THIS WAS SIMPLY BECAUSE THEY

DIDN'T USE IT.

THEY TOLD US THAT WE WOULD

USE IT.

AND SO THIS FINAL CONTROL

THE HUMAN IN THE LOOP DIDN'T

WORK HERE.

WE HAD NO IMPACT BECAUSE

THAT HAVE.

BUT POTENTIALLY IT COULD GO

UP TO 88% SENSITIVITY.

ANOTHER EXAMPLE IS A SYSTEM

THAT WE STARTED DEVELOPING

FOR THE I 2 B 2 MEDICATION

CHALLENGE IN 2009, REALIZING

THERE WAS A NEED FOR

DIFFERENT REASONS AND

OBSERVED THAT THE LIST OF

PROBLEMS OF MEDICATIONS WAS

INCOMPLETE MOSTLY BECAUSE OF

MEDICATION THAT'S WERE

PROSCRIBED IN ANOTHER

INSTITUTION, SOMEWHERE ELSE

OUT OF THE SYSTEM OR BOTH

OVER THE COUNTER BY THE

PATIENTS OR SIMPLY

PROSCRIBED BEFORE THE

INTRODUCTION OF THE ORDER

ENTRY SYSTEM.

SO WE ALSO DEVELOPED A

SYSTEM TO EXTRACT ALL

MEDICATION THAT'S WERE

MENTIONED IN THE NARRATIVE

TEXT IN CLINICAL TEXT.

AND FOR THIS CHALLENGE WHAT

WE FOCUSED ON WAS THE NAMES

BUT DETAILED INFORMATION

LIKE DOSAGE, FREQUENCY,

DURATION, AND A MORE

DIFFICULT ONE THE REASON FOR

THE PRESCRIPTION.

FOR THIS CHALLENGE IT WAS

EVALUATED FOR THE PURPOSE OF

MORE THAN 500 DOCUMENTS.

AND THE SYSTEM THAT WE

DEVELOPED INCLUDED ALSO MANY

DIFFERENT STEPS THAT ARE

SHOWN HERE.

SO IT STARTED WITH AGAIN

SOME ANALYSIS OF THE

STRUCTURE OF THE DOCUMENTS,

SECTIONS AND FILTERING OF

SOME SECTIONS, FOR EXAMPLE,

WE EXCLUDED THE MEDICATION

ALLERGY SECTION OBVIOUSLY

BECAUSE THE GOAL HERE WAS TO

FIND THE MEDICATION THE

PATIENT WAS TAKEN OR HAS

TAKEN SOMETIMES.

HOW TO SWITCH.

THEN ALSO SOME DISAM

BIGATION OF SOME OF THOSE

ACRONYMS.

LIKE HERE MG BECAME MILL

GRAM.

COULD BE MAGNESIUM OR

SOMETHING ELSE.

ID WAS NOT AM BIGUOUS IN THE

CONTEXT.

THEN WE EXTRACTED THE NAMES

AND POTENTIAL REASONS FOR

PRESCRIPTIONS.

SO THIS IS WHERE MOST OF THE

DIAGNOSIS PROBLEM AND

FILTERED SOME OF THOSE

PROBLEMS.

SO WE HAD LIKE IN THIS

EXAMPLE, 40 MILLIGRAMS.

WE HAVE A DRUG LACEIX AND A

REASON.

THEN WE WOULD ANALYZE THE

CONTEXT.

SO THE NEGATION, THE

EXPERIENCE WITH THE

MEDICATION ABOUT THE PATIENT

OR SOMEONE ELSE.

IF THE PATIENT DID NOT TAKE

THE MEDICATION, ET CETERA.

ALSO DETECTED ALLERGIES.

AND THEN EXTRACTED ALL THE

ADDITIONAL INFORMATION, THE

DOSAGE, THE FREQUENCY, THE

DURATION.

AND IN THE END RECONCILED

THOSE INFORMATION TO END UP

WITH SOME STRUCTURED ENTRY.

LIKE IN THIS EXAMPLE THE

NAME IS LACEIX, THE DOSE IS

40 MILL GRALES.

THE FREQUENCYY IS TWICE A

DAY.

DURATION IS NOT MENTIOND IN

THIS CASE.

AND WHEN EVALUATING IT FOR

EXACT MATCHES.

SO THIS MEANS WHAT WE FOUND

-- THE TERMS WE FOUND

CORRESPONDING EXACTLY TO THE

STANDARD, WE HAD PERFORMANCE,

THE RECALL IS EQUIVALENT TO

THE SENSITIVITY HERE THAT

RANGED FROM 17% ONLY TO

ABOUTLE 2% FOR SOME TYPES OF

INFORMATION.

PRECISION WAS MUCH HIGHER.

AND YOU SEE THAT FOR SOME OF

THESE CATEGORIES LIKE THE

DURATION AND REASON FOR THE

PRESCRIPTION, THE

PERFORMANCE WAS PRETTY LOW.

AND ACTUALLY EVERYONE

STRUGGLED WITH THAT.

AND EVEN HUMANS STRUGGLED

WITH IT.

AND SO FOR EXAMPLE, WE ADDED

SOME MANUAL ADAPTATIONS ON

OUR TEAM AND OUR AGREEMENTS

AT THE FIRST TASK FOR

DURATION ANNOTATIONS WAS

ONLY 16% OR 31% FOR REASON

FOR PRESCRIPTION, WHICH IS

VERY -- AND EVEN AT THE

CHALLENGE LEVEL, MULTIPLE

DIFFERENCES ANNOTATIONS BY

EXPERTS, THEN FOR DURATION

THEY ONLY AGREED ABOUT 40%

OF THE TIME.

AT ABOUT 47% OF THE TIME.

SO THIS IS A REALLY

DIFFICULT TASK NOT ONLY FOR

NLT.

SO AS A CONCLUSION, I'D LIKE

TO EMPHASIZE TWO ASPECTS

THAT I THINK ARE IMPORTANT

FOR USING CLINICAL DECISION

SUPPORT.

AND MORE SPECIFICALLY WHEN

USING NATURAL PERSISTING TO

EXTRACT CLINICAL

INFORMATION.

AND THIS IS REALLY FOR

HUMANS, FOR USERS TO TRUST

THE INFORMATION THAT'S

EXTRACTED.

AND WHAT WE OBSERVED AND

EXPERIENCED IS THAT THE FACT

THAT THERE IS A HUMAN THAT

EVENTUALLY DECIDES THAT

HUMAN IS CORRECT IS VERY

IMPORTANT AND ALSO TO ALLOW

THIS HUMAN TO SEE WHERE THIS

INFORMATION CAME FROM AND

MAYBE HOW IT WAS EXTRACTED.

AND SO DISCUSSION WE HAD

YESTERDAY ABOUT WHAT METHOD

TO USE AND FOR ACCEPTANCE.

AND IN THIS CASE IT SHOULD

BE SOMETHING THAT'S MORE

TRANSPARENT AND EXPLAINS HOW

AND WHERE THE INFORMATION

WAS EXTRACTED.

WHEN WE IMPLEMENTED THE

SYSTEM, WE HAD A LOT OF

TRUST AND ENDED UP DICTATING

SMALL NOTES TO HAVE THE

PROBLEMS IN THE LIST INSTEAD

OF ANSWERING IT ANNUAL

MANUALLY.

THIS WAS REALLY IMPORTANT.

AND ALSO ANOTHER ONE IS THAT

WITH THIS IN MIND, THEN

PERFORMANCE IS NOT AS

IMPORTANT.

IT SHOULD BE FAST, THAT'S

TRUE.

IT SHOULD HAVE ACCEPTABLE

AACCURACY, BUT EVEN IF IT

JUST ADDS A LITTLE TO WHAT'S

ALREADY AVAILABLE, IT'S

ALREADY GOOD.

THANK YOU.

[APPLAUSE]

>> EXCELLENT WORK.

ONE COMMENTS IS A QUESTION.

A COMMENT.

IT'S INTERESTING THAT I SEE

DOCTORS DICTATED AND A

PROBLEM TILLS A LITTLE BIT

ABOUT WHAT THEY'RE

COMFORTABLE WITH.

THE TELLS.

THE QUESTION IS WHY WOULD

THEY BE INTERESTED IN ADDING

TO THE COMMENTS?

>> WELL, IT'S FOR SEVERAL

REASONS.

BUT IN THE ICU IT'S BECAUSE

THEY SAW INTEREST IN THE

PROBLEM LIST FROM THE

BEGINNING.

THEIR CHAIR WAS INTERESTED

IN MAKING THE PROBLEM LIST

THE CENTRAL COMPONENTS IN

THEIR WORK FLOW, IN THEIR

CARE.

SO OBVIOUSLY, TO MAKE IT

USEFUL, IT NEEDS TO BE USED.

AND YOU SEE THE DIFFERENCE

WITH THE CARDIO VASC COLLAR

SURGERY DEPARTMENT WHERE

IMPLEMENTED IT, IT HAS

ABSOLUTELY NO IMPACT.

EVEN IF IT PERFORMED PRETTY

WELL BUT NO ONE USED IT

BECAUSE THEY STILL CONTINUED

THERE ARE USUAL WORK FLOW

THAT WAS REALLY NOT RELYING

ON THE PROBLEM.

BUT IN THE ICU, THEY WERE

WORKING ON -- THEY HAD A

STRONG EFFORT TO MAKE IT

MORE IMPORTANT, TO RELY ON

ENTRIES ON THE PROBLEM TO

BASE ALL THEIR DISCUSSIONS,

CASE DISCUSSIONS, ET CETERA

ON THE PROBLEMS, ET CETERA.

>> DAVID MENDLESTON IN NEW

YORK.

SO TO WORK THAT OUT A LITTLE

BIT FURTHER, THIS VALIDATION

STEP THAT YOU MENTIONED

BECAUSE PROBABLY YOU CAN'T

RELY ON THESE SYSTEMS AT

LEAST ON THIS POINT IN TIME

COMPLETELY.

AND WHAT WAS THE RESISTANCE

IN THE CARDIOVASCULAR UNIT?

WAS THAT IT WOULD TAKE MORE

TIME?

WE'VE HAD EXPERIENCE USING

CAT SYSTEMS.

SOME PEOPLE WON'T TOUCH THEM

BECAUSE IT TAKES MORE TIME

SO I WOULD BE INTERESTED IN

YOUR VIEWS IN HOW THIS WAS

SEEN IN YOUR SIGHT AND HOW

IT MIGHT EVOLVED INVOLVE.

>> INITIALLY WE'VE HAD

PILOTS EVALUATION WITH

SELECT USERS IN CARDIO VASC

COLLAR SKPIRJ THEY DIDN'T

COMPLAIN ABOUT THAT BECAUSE

FOR THEM IT MEANT ONE OR TWO

CLICKS.

SO IT WAS REALLY FAST.

BUT THE MAIN PROBLEMS IS

THAT THEY SIMPLY HAD NO

CINTIVE TO USE THE PROBLEM

LIST AT ALL.

SO THAT'S WHY THEY DIDN'T

USE IT.

YES?

>> JUST OUT OF CURIOUSITY.

AS A STUDENT, I USED TO

WONDER IF HUMAN EXPERT IN

THE LOOP.

IF I SEE A NOTE THAT SAYS

CIRRHOSIS OF LIVER, AS A

JUNIOR DOCTOR, I WOULD SAY

THERE IS CHRONIC CIRRHOSIS

OF THE LIVER.

SO I WOULD COME OUT WITH AT

LEAST FIVE, SIX, SEVEN

DIAGNOSES POSSIBLY.

WHICH ONE WOULD YOU PICK AND

WITHOUT ALL OF THEM IN THE

-- WOULD YOU WANT ALL OF

THEM IN THE PROBLEM LIST?

>> WE DID, TO FILTER ALL

THESE PROBLEMS, TO FIRST

MAKE SURE THAT THERE WERE NO

DUPLICATES.

WE DID DIFFERENT LEVELS

USING TERMINOLOGIES,

RELATIONS, ET CETERA, TO

MAKE SURE THAT FOR EXAMPLE,

IF THERE WAS ALREADY

DIABETES IN THE LIST AND WE

FOUND DIABETES MILLITIS TYPE

TWO, THEN WE WOULD NOT ADD

IT -- WOOPS.

A -- ALREADY VOICED.

AND WE ONLY INCLUDED SOME

TYPES OF CLINICAL

INFORMATION MOSTLY DIAGNOSES.

SO WE HAD A LONG LIST OF

MOST FREQUENT DIAGNOSES IN

THE DOMAINS CARDIO VASC

COLLAR SURGERY.

MOSTLY CARDIO VASC COLLAR SO

IT WASN'T EVERYTHING.

NOT ALL FINDINGS, SYMPTOMS,

ET CETERA.

>>

[INDISCERNABLE]

>> SO WHEN WE IMPLEMENTED IT,

IT WAS DONE IN REALTIME.

SO AS SOON AS THE PROVIDER

STORED, RECORDED OR SAVED A

DOCUMENT HE AUTHORED --

>>

[INDISCERNABLE]

>> NO.

IT WAS ONLY PERSPECTIVE.

ONLY PERSPECTIVE.

SO AS SOON AS THE PATIENT

WAS HOSPITALIZED ENTERED IN

THE ICU AND EVERYTHING WAS

NEW DOCUMENTS, WHERE PROCESS

AND PLATE AID PROBLEM.

IT WAS ONLY PERSPECTIVE.

>> ONE MORE QUICK QUESTION

HERE.

DID YOU CONSIDER USING NURSE

PRACTICEITIONERS OR NURSES

TO WHET THE PROBLEM LIST SO

THAT IT'S A LITTLE MORE

CLEANER IN THE WORK FLOW?

>> NO, WE DID NOT, BECAUSE

THE PROBLEM WAS ALMOST ONLY

USED BY PHYSICIANS.

IT WAS REALLY MORE LIKE A

DOCTOR'S PROBLEM LISTS.

>> WHAT ABOUT MEDICATION

RECONCILIATION?

>> THEY WERE NOT IN THE

PROBLEM LIST.

>> FOR THE OTHER ONE,, HE IN

GENERAL YOU MEAN?

>> YEAH.

>> NOT REALLY, BUT IT COULD

BE A GOOD IDEA, ESPECIALLY

IN SOME ENVIRONMENTS WHERE

NURSE PRACTICEITIONERS DO

MOST OF THE CONTACT WITH THE

PATIENT.

IT WOULD MAKE SENSE, YES.

>> THANKS.

[APPLAUSE]

>> OUR NEXT SPEAKER IS

DR. MARK MUSEN FROM STANFORD

UNIVERSITY.

>> THANK YOU, JAMES.

LET ME JUST ALL RIGHT.

WHERE IS THE -- ON THE

BOTTOM.

SEE, I'M A MAC GUY.

YEAH.

IT'S BETTER ON A MAC.

[LAUGHTER]

ANYWAY, THANK YOU.

I REALLY WANT TO THANK THE

NI BIB FOR BRINGING US

TOGETHER AND HAVING THIS

WONDERFUL CONVERSATION.

I WANT TO START BY

CONFESSING THERE WAS A TIME

WHEN I WAS A NON-BELIEVER.

IN THE 80S CHERRY WIN GRAD

HAD JUST MADE A SPLASH IN

VOLUME ONE OF HIS BOOK AND

IMMEDIATELY ABANDONED THE

IDEA OF VOLUME TWO WHEN HE

SAID WELL, THEY WOULD FIGURE

OUT SOME ANTICS SO WE SHOULD

JUST GIVE UP AND THAT'S WHEN

HE MOVED INTO COMPUTER

INTERACTION.

AND FRANKLY, IF YOU'D ASKED

ME THEN IF WE HAD THE

CAPABILITIES THAT WE HAVE

NOW IN NLP, I WOULD HAVE

BEEN VERY DOUFBLET.

AND SOMEONE WHO HAS SPENT

HIS CAREER MAINLY DEALING

WITH DECISION SUPPORT

ACTUALLY NOT DEALING WITH

NATURAL LANGUAGE PROCESSING

AS MUCH AS I CAN AT ALL, I

AM ACTUALLY VERY IMPRESSED

WHERE NLT HAS COME AND I'M

FRUSTRATED WHERE C.D.S HAS

GONE, PARTICULARLY IN THE

COMMERCIAL SECTOR.

AND I THINK WHAT I'D LIKE TO

DO IN THE NEXT FEW MINUTES

IS TALK ABOUT WHERE C.D.S

EXISTS AS IS ACTUALLY

DEPLOYED IN THE REAL WORLD,

WHERE I THINK IT CAN BE VERY

HELPFUL.

I WAS VERY INSPIRED BY BOB'S

NENOTE WHERE HE WENT ABOUT

THE LARGE GAMUT OF ACTIVITY

IN CLINICAL SUPPORT AND

BETTER MEDICAL DECISION

MAKING.

AND THEN GAIN TO THINK ABOUT

REALLY WHERE ARE WE IN THE

WORLD NOW AND FRANKLY, AS MY

SLIDE SUGGESTS, AT LEAST IN

THE COMMERCIAL SECTOR AND

LARGELY IN THE ACADEMIC

SECTOR AS WELL, WE TEND TO

LOOK FOR CLAIM GREAT SUCCESS

WHEN REALLY WE'RE MISSING

OUT ON GREAT OPPORTUNITIES

AND WAYS OF IMPROVING

HEALTHCARE AND HELP.

WHEN YOU LOOK AT THE WAY

VENDORS TALK ABOUT

DETERMINES IN TERMS OF

SUPPORT, THEY ADVERTISE

RULES.

THIS IS ACTUALLY A RULE

STOLEN FROM PRODUCT

LITERATURE THAT SENER DISHES

OUT WHICH TALKS ABOUT THE

KINDS OF RULES THAT YOU CAN

WRITE AND THEIR PARTICULAR

LANGUAGE FOR HANDLING RULES.

WE ALL KNOW ABOUT H.R. 7 AND

FRANKLY THESE RULES PROVIDE

THE OPPORTUNITY TO SAY IF

SOME BAD SITUATION EXISTS,

THEN DO SOMETHING ABOUT THAT

BAD SITUATION IN GENERAL.

AND THIS IS A TECHNOLOGY

THAT HAS EXISTED SINCE THE

1970'S.

HOYEARS AGO MCDONALD SHOWED

BRILLIANTLY HOW THIS KIND OF

TECHNOLOGY CAN HAVE A

TREMENDOUS EFFECT ON

AFFECTING THE KINDS OF THE

PROBLEMS THAT UNITELY AFFECT

PEOPLE IN AM BLUETRY CARE

AND AVERTING KINDS OF THE

SITUATIONS MOST OF US ARE

MOST CONCERNED ABOUT.

WE CAN DEAL WITH DRUG AND

DRUG INTERACTIONS AND DON'T

GET DRUGS THEY SHOULDN'T GET.

WE CAN SUGGEST ALTERNATIVE

MEDICATION THAT'S MIGHT BE

CHEAPER AT THE TIME OF ORDER

ENTRY.

WE CAN MAKE REMARKS ABOUT

ABNORMAL LAB RESULTS,

OPPORTUNITIES FOR

IMMUNIZATIONS, SERVICES THAT

WE BRING TO THE ATTENTION OF

PROVIDERS.

THIS IS ALL WONDERFUL STUFF,

ALL STUFF THAT WE SHOWED

HOYEARS AGO REALLY IS AT THE

HEART OF RULE-BASED SYSTEMS

WHEN THEY'RE USED REALLY

WELL IN CLINICAL CARE.

THE PROBLEM IS WE'RE

CONTINUING TO PROMOTE THESE

KINDS OF SYSTEMS AND

PROMOTING THEM TO DO THE

KINDS OF THINGS THEY DO WELL

WHILE WE REALLY IGNORE THE

POSSIBILITY OF ALTERNATIVE

FORMS OF DECISION SUPPORT

THAT ACTUALLY ADDRESS SOME

OTHER KINDS OF PRESSING

PROBLEMS WE JUST DON'T LOOK

AT AT ALL.

WE'RE GOOD AT SQUINTING AT

CLINICAL PROBLEMS AND SAYING

THIS IS WHERE RULES MIGHT BE

USEFUL.

BUT WE IGNORE ALL THE

PROBLEMS THAT RULE-BASE

SYSTEMS EMERGE WHEN WORK WAS

DONE IN THE 80S AND REALIZED

HOW THEY WERE BRITLE AND HOW

THEY DIDN'T ADDRESS PROBLEMS

AT THE EDGE OF THEIR

CAPABILITIES AND MOST

IMPORTANT, HOW WE LEARN WHEN

THEY BECAME LARGE WERE

ABSOLUTELY IMPOSSIBLE TO

DEVELOP, HOW QUALITY

ASSURANCE WAS IMPOSSIBLE AND

HOW WE REALLY NEEDED WAYS OF

STRUCTURING KNOWLEDGE IN A

MUCH MORE COHERENT WAY NOT

ONLY IT MAKE OUR SYSTEMS

MORE INTELLIGENT OU TO MAKE

US MORE INTELLIGENT IN BEING

ABLE TO MAINTAIN THEM AND

REPAIR THEM.

AND FRANKLY,S THE POPULATION

AGES AND WE START TALKING

ABOUT WHAT THE NEXT

GENERATION OF CARE IS GOING

TO REQUIRE, WE ARE NOT VERY

GOOD AT DEALING WITH WHAT

THE NEXT KIND OF DECISION

SUPPORT SYSTEMS NEED TO

PROVIDE.

THEY NEED TO BE ABLE TO DEAL

WITH GUIDELINES, AS BOB

SUGGESTED, THEY NEED TO BE

ABLE TO DEAL WITH TREATMENT

OVER TIME.

THEY HAVE TO DEAL WITH

INTERVENTIONS THAT ARE BASED

ON PREVIOUS RESPONSE OF

TREATMENTS.

WELL-PREVIOUSLY ON SOME DRUG,

IT MIGHT BE GOOD TO CONTINUE

THAT KIND OF DRUG.

IF SOMEBODY DID POORLY, WE

KNOW WE WANT TO TRY SOME

ALTERNATIVES.

AND MOST IMPORTANT AS THE

POPULATION AGES, WE HAVE TO

RECOGNIZE THAT 60% OF

PATIENTS OVER AGE 50350 HAVE

MORE THAN ONE DISEASE.

AND IN THE SETTING OF

MULTIMORBIDITY, EVEN OUR

EXISTING GUIDELINES WHICH

WERE ALL DEVELOPED WITH

CLINICAL TRIALS WERE

EXCLUDED IN THE FIRST PLACE.

WE HAVE TROUBLE ADOPTING OUR

GUIDELINES IN WAYS THAT

ACTUALLY MAKE SENSE WHERE

THE KINDS OF PATIENTS WERE

ACTUALLY FOR THE PATIENTS

WE'RE TREATING IN THE CLINIC.

CLINICAL CARE TASS EXISTS IS

MESSY.

PATIENTS HAVE MANY COMPLEX

PROBLEMS.

GUIDELINES THAT DON'T ALWAYS

APPLY, AND YET WE'RE REALLY

GOOD AT THINKING ABOUT THOSE

RULES THAT TELL US NOT TO

GIVE DRUGS TO WHICH PATIENTS

ARE ALLERGIC.

AND WHAT WE REALLY NEED TO

BE THINKING ABOUT IS THE

NEXT GENERATION OF CLINICAL

DECISION SUPPORT THAT CAN

SUPPORT TO DO THINGS THAT

ARE MORE INTERESTING.

I KNOW THIS SOUNDS IMMODEST

BUT IT GIVES YOU AN EXAMPLE

OF THE KINDS OF THINGS WE'VE

BEEN WORKING AT OVER THE

YEARS.

ONE IS I THINK IT GIVES A

GOOD EXAMPLE, TOO, AND I'M

OBLIGATED TO TALK ABOUT IT.

WE HAVE A SYSTEM AT STANFORD

THAT BLACKFORD ALLUDED TO

EARLIER THIS MORNING CALLED

ION, WHICH PROVIDES A

FOUNDATION WHICH MY

COLLEAGUE AT STANFORD USED

TO DEPLOY IN ABOUT NINE

DIFFERENT V.A. MEDICAL

CENTERS FAIR VARIETY OF

CHRONIC DISEASES FOR WHICH

GUIDELINE-BASED CARE MAKES

SENSE.

THIS SHOWS YOU AN

EXPERIMENTAL FRONT FOR THE

HYPERTENSION SYSTEM WHICH

SITS ON TOP OF CPRS, WHICH

SITS ON TOP OF VIST Y AT

V.A. MEDICAL CENTERS AND

WHAT VISTA AND ATHENA ALLOW

ONE TO DO TOGETHER IS TO

LOOK AT INFORMATION ABOUT

PATIENTS WHO ARE HYPER--

HAVE HYPERTENSION AND TO

APPLY THAT JOINT NATIONAL

COMMISSION GUIDELINE THAT

BLACKFORD MENTIONED OR MAYBE

IT WAS FRANK, TO

AUTOMATICALLY MAKE

SUGGESTION ABOUTS HOW

PATIENTS MIGHT BE TREATED IF

THE DOCTOR WANTS TO ASSUME

THAT THE GUIDELINE MAKES

STENS SENSE IN THAT

PARTICULAR SCENARIO AND HOW

DOES ATHENA WORK?

IT HAS A REPRESENTATION OF

THE GUIDELINES STORED AS A

KNOWLEDGE BASE AND THE

PATIENT DATA THAT ARE

AVAILABLE TO THE VISTA

ELECTRONIC HEALTH RECORD

SYSTEM AND ASSUMING THAT ALL

THE CODED DATA ARE ALL YOU

NEED, BRINGS THAT

INFORMATION TOGETHER TO MAKE

RECOMMENDATIONS SUCH AS FOR

EXAMPLE, CONSIDER ADDING AN

ACE INHINTOR BECAUSE THERE

IS COMPELLING INDICATION OF

HEART FAILURE.

ATHENA DOES THIS BECAUSE WE

HAVE UNDER THE HOOD -- AND

IT'S ALWAYS GOOD TO SHOW

WHAT'S UNDER THE HOOD AT

THIS KIND OF THE MEETING, AN

ONCOLOGY WHICH DESCRIBED

WHAT'S DEFINED IN TYPICAL

GUIDELINES.

THIS ONCOLOGY PROVIDES A

FRAMEWORK OF THINGS SUCH AS

A RECOMMENDATION

SPECIFICATION.

IT WILL HAVE AN ACTION

SPECIFICATION.

IT WILL HAVE AN ALGARITHM

THAT WILL SUGGEST HOW

VARIOUS TASKS HAVE BEEN

IMPLEMENTED OVER TIME.

AND IT WILL PROVIDE

PARTICULAR PROPERTIES OF

THOSE VARIOUS ENTITIES IN A

WAY THAT ALLOWS OUR SYSTEM

TO AUTOMATICALLY ACQUIRE

INFORMATION ABOUT THE

VARIOUS GUIDELINES THAT

MIGHT BE INITIATED USING

THOSE GENERIC ONTOLOGY.

SO IT MIGHT SUGGEST WHAT THE

INFORMATION YOU NEED TO

DESCRIBE THE GAC 7

GUIDELINE.

WHAT ARE THE WAYS IN WHICH

THE GUIDELINES APPLIES SO WE

KNOW WE ARE MEETING ITS

GOALS.

WHAT ARE THE KINDS OF DRUGS

WE WANT TO USE AND

INTERVENTIONS WE WANT TO

CONSIDER AND SO ON?

AND BECAUSE WE CAN DRAW OUR

GRAPHS, WE CAN SAY WHAT IS

THE TEM PRALE ORDER OF

INNOVATION THAT'S MAKE SENSE

IF WE WANT TO TREAT PATIENTS

IN ACCORDANCE TO THE

GUIDELINES?

THIS IS NOT THE KIND OF

SPECIFICATION THAT YOU CAN

SET A SITUATION CAN SPECIFY

IN A VERY CLEAN WAY.

THIS IS NOT THE KIND OF

SITUATION ACTION THAT MAKES

SENSE -- NOT THE KIND OF

SPECIFICATION THAT DEALS

WITH ONE SITUATION AND ONE

ACTION, RATHER, IT DEALS

WITH MULTIPLE SITUATIONS IN

MULTIPLE ACTIONS THAT UNFOLD

IN A RATHER COMPLICATED

CHAIN OF SEQUENCE.

BUT ONCE YOU HAVE THIS KIND

OF REPRESENTATION, ONCE YOU

HAVE THE DATA THAT ARE

AVAILABLE THROUGH VISTA,

ONCE YOU HAVE THE PATIENT

WHO IS BEING TREATED

ACCORDING TO THE GUIDELINE,

THEN YOU HAVE THE BASIS FOR

HAVING AN AUTOMATED SYSTEM

THAT CAN GO WELL BEYOND

IDENTIFYING VERY SIMPLE

PROBLEMS BUT CAN MANAGE

COMPLEX PATIENTS WITH

MULTIPLE, COMPLICATING

ISSUES IN THE AREA OF

HYPERTENSION OVER TIME.

THAT'S THE GOOD NEWS.

THE BAD NEWS IS THAT TO DO

THIS ACTUALLY REQUIRES A LOT

OF INFORMATION THAT'S NOT

ACCESSIBLE THROUGH THE CODED

DATA THAT WE ARE ABLE TO

BRING IN THROUGH VISTA.

SO WHAT DO WE MISS?

WE MISS UNDERSTANDING WHAT

ARE THE SPECIFIC PATIENT

PREFERENCE THAT'S MIGHT

CAUSE A PATIENT TO CHOOSE

ONE ACTION OVER ANOTHER IF

BOTH ACTIONS ARE RELATIVELY

INDICATED AT THE SAME LEVEL?

WHAT ARE THE PROVIDER

PREFERENCES OR THE

ORGANIZATIONAL PREFERENCE

THAT'S MIGHT MAKE ONE ACTION

EFFECTIVE OVER ANOTHER IN A

PERFECT SETTING?

WHAT ARE ALL THE INTANGIBLES?

WHAT'S THE PATIENT'S

TRANSPORTATION SITUATION?

CAN THE PATIENT GET TO THE

CLINIC FOR EVALUATION?

WHAT OVER-THE-COUNTER

MEDICATIONS MIGHT THE

PATIENT BE TAKING?

WHAT DRUGS MIGHT BE

PROSCRIBED ELSLSHERE THAT WE

DON'T KNOW ABOUT?

WHAT MORBIDITY EXISTS?

IN FACT, WHAT IS THE

CLINICAL PRACTICE GUIDELINES

ACCORDING TO THE PATIENT

BEING TREATED?

THAT KIND OF THING IS ALL IN

THE NARRATIVE TEXT AND THAT

INFORMATION IS NOT

ACCESSIBLE TO ATHENA IN ANY

WAY, SHAPE OR FORM.

AND AS YOU'VE JUST HEARD, A

LOT OF WHAT WE ASUME IS

GOING TO BE AVAILABLE

THROUGH THE CODED

INFORMATION MAY OR MAY NOT

BE THERE BUT WE'LL LEAVE

THAT FOR ANOTHER DAY.

WE'VE GOTTEN REALLY GOOD AT

TAKING THE ATHENA MODEL AND

REPLICATING IT.

SO MY COLLEAGUES AT THE V.A.

HAVE TAKEN VERSIONS OF

ATHENA THAT TAKE THE

HYPERTENSION VERSION AND

REPLICATE IT FOR USE IN A

GUIDELINE THAT TREATS HEART

FAILURE.

AND IF -VREPLICATED IT FOR

ENTERING INFORMATION ABOUT

HYPERLEP DEMIA AND DIABETES

AND MANAGEMENT OPOID THERAPY

AND ALL THAT IS REALLY

WONDERFUL.

BUT ALL OF THESE ARE SYSTEMS

AND AS I SAID A MOMENT AGO,

OUR BIGGEST PROBLEM IS

DEALING WITH MORBIDITY.

AND WHAT WE'RE REALLY TRYING

TO DO NOW IS IDENTIFY

MECHANISMS THAT MAKE SENSE

FOR TAKING THE COMPLEXITIES

WHERE WE HAVE TO MANAGE

PATIENTS IN ACCORDANCE WITH

GUIDELINES AND BEGIN TO

IDENTIFY WAYS IN WHICH WE

CAN BRING OTHER GUIDELINES

INTO THE IDENTIFICATION-MAKING

PROCESS.

AND SO A CONTRACT THAT WAS

SPOTTED THAT -- FUNDED,

WE'RE STARTING TO LOOK AT

THE WAYS IN WHICH MULTIPLE

GUIDELINES CAN BE

ADMINISTERED TOGETHER.

AND THE MODEL RIGHT NOW IS

VERY SIMPLE.

WE'RE VERY EXCITED ABOUT THE

POSSIBILITY OF LOOKING AT

THE PROBLEM AS IT EMERGES

WHEN YOU TAKE THE MULTIPLE

GUIDELINES USING THE ION

TECHNOLOGY, THE VARIOUS

ATHENA VERSIONS, RUN THEM ON

A PARTICULAR PATIENT WHO HAS

MORBID CONDITIONS AND THINK

ABOUT HOW CAN WE APPLY THOSE

GUIDELINES?

HOW CAN WE CONSOLIDATE THE

VARIOUS RECOMMENDATION

THAT'S THE GUIDELINES

INDIVIDUALLY WOULD

RECOMMEND?

AND THEN HOW CAN WE IDENTIFY

POTENTIAL INTERACTIONS AMONG

THOSE GUIDELINES, HOW CAN WE

REPAIR THOSE INTERACTIONS

AND ULTIMATELY HOW CAN WE

PRIORITIZE RECOMMENDATIONS

WHEN MULTIPLE GUIDELINES RUN

TOGETHER AND OBVIOUSLY NONE

OF THESE GUIDELINES WERE

CREATED WITH THE IDEA OF

ANTICIPATING ALL THE

POSSIBLE SIDE EFFECTS THAT

MIGHT OCCUR WHEN THE

GUIDELINES FOR OTHER

DISEASES ARE BEING RUN

SIMULTANEOUSLY?

I WON'T SAY THAT WE HAVE THE

SOLUTION HERE BUT I THINK

THIS IS AN EXCITING AREA OF

RESEARCH AND POINTS TO THE

COMPLEXITY OF THE CLINICAL

ENTERVISE AND THE PROBLEMS

THAT DECISION CERTAINLY WE

HAVE A LONG WAY TO GO IN

BEING ABLE TO DEAL WITH.

DEALING WITH THESE MULTIPLE

COMORBIDITIES IS HARD

BECAUSE AS I SAID EARLIER,

ALL THE CLINICAL TRIALS

TYPICALLY WILL EXCHRUT

PATIENTS WHO HAVE OTHER

DISEASES AND RARELY IS THERE

EVIDENCE THAT TELLS US WHAT

TO DO WHEN PATIENTS HAVE

MORE THAN ONE CONDITION.

UNDERSTANDING THE RELATIVE

EFFECTS OF COMORBIDITIES ON

FUNCTIONAL STATUS IS REALLY

OF AN IMPORTANT NUANCE.

WE NEED TO BE ABLE TO

APPRECIATE THAT YES,

TREATING A PATIENT IN

ACCORDANCE WITH ONE

GUIDELINE MIGHT HELP ONE

PROBLEM BUT MIGHT HURT SOME

OTHER PROBLEM AND BEING ABLE

TO ASSESS THAT TRADE-OFF IS

A DIFFICULT DECISION PROBLEM

THAT IS NOT SOMETHING THAT'S

BUILT INTO ANYIST EXISTING

GUIDELINE STRUCTURES.

BUT WHEN WE TRY TO DO THAT,

THOSE DECISION MODELS CAN

BECOME COMPLICATED AS WE

DEAL WITH ALL THOSE

POSSIBILITIES.

AND OBVIOUSLY, WHAT WE

REALLY NEED ARE WAYS OF

BEING ABLE TO MANAGE THE

COMPLEXITY IN A REUSABLE

SENSE.

AND THE DECISION-MAKING

BASICALLY IN THE ABSENCE OF

ANY KIND OF FORMAL EVIDENCE

WHICH USUALLY IS THE CASE

WHEN YOU DEAL WITH PATIENTS

WITH MULTIPLE MORBIDITY, HAS

TO BE INFORMED FROM OTHER

SOURCES.

AND INFORM THAT KIND OF

DECISION MAKING WILL

FUNDAMENTALLY BE THE KIND OF

INFORMATION THAT WE CAN

GREEN FROM ELECTRONIC HEALTH

RECORDS.

MAINLY INFORMATION THAT WE

CAN GREEN IN THE FORM OF NLP.

SO IN ORDER TO BASICALLY

HAVE THE KINDS OF HEALTH

SYSTEMS THAT CAN OFFER

EVIDENCE-BASED CARE, NOT FOR

SIMPLE SITUATIONS, NOT EVEN

FOR INDIVIDUAL GUIDELINES,

BUT FOR THE KINDS OF

PATIENTS WHO TYPICALLY WILL

BE OVERRUNNING THE

HEALTHCARE SYSTEM FOR THE

NEXT 40 YEARS, WE BASICALLY

NEED TO BE ABLE TO

INCORPORATE ALL THE

NON-CODED DATA THAT EXISTS

IN THE H.R. AND IDENTIFY

FROM HISTORICAL RECORDS

PATIENTS WHO ARE SIMILAR IN

THEIR COMORBIDITY, SIMILAR

IN TREATMENT SITUATIONS AND

TRY TO UNDERSTAND NOT IN THE

CONTEXT OF CONTROLLED TRIALS

BUT IN THE CONTEXT OF OUR

EXPERIENCE WHAT HAS TAKEN

PLACE FROM WHICH WE CAN

LEARN IN ORDER TO MAKE

DECISIONS ABOUT THESE KINDS

OF COMPLEX PATIENTS WHICH

ARE THE NORM IN CLINICAL

PRACTICE?

AND BASICALLY, WHAT CAN WE

DO TO BUILD THE KINDS OF

CLINICAL DECISION SUPPORT

SYSTEMS THAT WILL ADDRESS

ALL THE SITUATIONS FOR WHICH

PATIENTS AND PROVIDERS, NOT

JUST THOSE THAT COULD BE

FRAMED IN TERMS OF THE KINDS

OF SIMPLE RULES THAT ARE SO

PERVASIVE NOW IN THE KINDS

OF SYSTEMS THAT ARE

AVAILABLE COMMERCIALLY.

THERE ARE A NUMBER OF

WORKSHOPS THAT WERE HELD BY

THE INSTITUTE OF MEDICINE

FROM OTHER AGENCIES ON THE

IDEA OF A LEARNING

HEALTHCARE SYSTEM.

THAT WOULD BE ABLE TO LEARN

FROM THE EVIDENCE OF

PREVIOUS PRACTICE WHAT MIGHT

BE REASONABLE APPROACHES

WHEN FRANKLY THERE IS NO

EVIDENCE AND TO APPLY THOSE

KINDS OF INFERENCES IN THE

CARE OF PATIENTS WHEN THERE

MAY NOT BE A CLINICAL

PRACTICE GUIDELINE BUT WHERE

PROVIDERS ARE SEEKING MORE

THAN JUST AN INTUITION ABOUT

WHAT MIGHT BE THE BEST KIND

OF CARE.

AND I THINK AS WE THINK

ABOUT THE COMPLEXITIES OF

THE CLINICAL ARENA AND THINK

ABOUT HOW WE WANT TO APPLY

EVIDENCE-BASED PRACTICE TO

THE BEST OF OUR ABILITY AND

HOW WE WANT TO FILL IN THE

GAPS, I THINK IN WORKING

TOWARD THIS NOTION OF A

LEARNING HEALTHCARE SYSTEM,

WE CAN SEE A LOT OF BENEFIT

FROM TRADITIONAL CLINICAL

DECISION SUPPORT SYSTEMS,

THE KINDS OF SUPPORT SYSTEMS

THAT CAN INCORPORATE THE

COMPLEXITIES OF THE

GUIDELINES FOLKS HAVE TALKED

ABOUT THIS MORNING, BUT ALSO

UNDERSTANDING HOW SIMILAR

PATIENTS HAVE BEEN TREATED

IN AN EFFORT TO COME UP WITH

THE KINDS OF CARE PLANS THAT

COULD TAKE ADVANTAGE OF AS

MUCH INFORMATION AS POSSIBLE

IN DEALING WITH PATIENTS WHO

HAVE THE KINDS OF

COMPLEXITIES, WHICH USED TO

BE CONSIDERED RARE, BUT NOW

IN THE CURRENT AGING

POPULATION JUST WHAT WE DEAL

WITH ALL THE TIME.

THANKS.

[APPLAUSE]

>> I WOULD LIKE TO INVITE

THE SPEAKERS UP TO THE PANEL

FOR DISCUSSION.

>> GO AHEAD.

>> HEY.

WITH REGARD TO LOOKING AT

THE EXPERIENCE WITH SIMILAR

PATIENTS WELL, THE SAME

CONDITION OR COMBINATIONS,

I'D LIKE TO ASK YOU TO

ELABORATE ON HOW YOU USE

THAT INFORMATION AND DO YOU

ASSUME THAT THOSE PATIENTS

WERE TREATED CORRECTLY OR DO

YOU TRY TO SEPARATE THEM OUT

ACCORDING TO THE DIFFERENT

WAYS THEY ARE TREATED AND

LOOK AT THEIR OUTCOMES?

>> I WILL CLARIFY THAT THIS

IS NOT RESEARCH THAT WE'RE

CURRENTLY DOING AND VERY

EXCITED ABOUT MOVING IN THIS

DIRECTION.

AND I THINK WHAT ONE HAS TO

DO IS LOOK AT A VARIETY OF

PARAMETERS, INCLUDING THE

CHARACTERISTICS OF THE

PATIENT IN TERMS OF SIGNS

AND SYMPTOMS AND PROBLEMS,

INTERVENTIONS, INTERVENTION

HISTORY, AND TRY TO IDENTIFY

WHAT IS THE DISTANCE, FULL,

BETWEEN THE PATIENT TO BE

TREATED AND THE HISTORICAL

PATIENTS WHO MIGHT BE

AVAILABLE ON THE RECORD?

FRANKLY SOME OF THE BEST

WORK THAT I KNOW OF IN THIS

AREA WAS DONE 30 YEARS AGO

BY REYNOLDS WHEN HE LOOKED

AT THE PROBLEM OF REASONING

ABOUT PATIENTS WHOSE

DESCRIPTIONS WERE AVAILABLE

THROUGH LITERATURE AND NOT

TRIALS.

AND I THINK WE CAN LEARN A

LOT FROM THAT WORK AND

FRANKLY WITH THE ABILITY TO

GET ACCESS TIHOST OF

INFORMATION, I THINK WE HAVE

A RICH SET OF DATA THAT WE

CAN USE TO TRY TO DO THAT

KIND OF LEARNING.

BUT I DON'T KNOW OF ANYONE

WHO HAS DONE IT VERY WELL

YET.

>> YEAH.

>> HI.

I'M FROM THE MAYO CLINIC.

I UNDERSTAND THAT THE

GUIDELINES -- ONLY A PART OF

THE GUIDELINE CAN BE

IMPLEMENTED BECAUSE YOU ARE

DEALING WITH -- DATA AND I

UNDERSTAND PART OF THE

GUIDELINES CAN'T BE

IMPLEMENTED BECAUSE THE DATA

IS NOT AVAILABLE IN RECORDED

FORM.

CAN YOU ESTIMATE WHAT PART

THE FRACTION OF THE

GUIDELINE WHICH CAN BE

IMPLEMENTED BECAUSE OF THE

NEED FOR CODED DATA BECAUSE

DATA IS IN THE FREE TEXT?

>> IT TURNS OUT THAT THE

VAST MAJORITY OF THE DATA

THAT IS NEEDED CAN ULTIMATELY

BE INFERRED FROM CODED

INFORMATION, NOT IN A WAY

WHICH GUARANTEES THAT EVERY

POSSIBLE INFERENCE THAT

MIGHT BE AVAILABLE IN THE

RECORD COULD BE APPLIED TO

THE GUIDELINE, BUT IN A WAY

THAT IS REASONABLE.

SO AS I AM NOW TRYING TO

REMEMBER.

I THINK IT WAS FRANK WHO

TALKED ABOUT IT EARLIER.

WE CAN TAKE ADVANTAGE FROM

THE CODED RECORD AND USE IT

TO INFER ABSTRACTIONS THAT

MAY ALLOW US THEN TO

PREDICATE OTHER ACTIONS THAT

ARE SPECIFIED IN THE

GUIDELINE.

SO EVEN THOUGH WE MAY NOT

HAVE A PROBLEM CODE, WE CAN

PERFORM INFERENCE BASED ON

WHAT WE DO KNOW TO MAKE A

GUESS AS TO WHETHER THAT

SITUATION THERE IS.

AND IT'S IMPORTANT, TOO, --

WHAT IS IMPORTANT STOS THESE

KINDS OF SOPHISTICATED

REASONORS ARE ABLE TO INFORM

THE CLINICIAN HOW THE

INFERENCE WAS MADE AND WHAT

THE CERTAINTY IS WITH WHICH

THAT CONCLUSION WAS MADE.

THESE SYSTEMS ARE NOT

PERFECT, BUT I THINK IT'S

REALLY IMPORTANT TO

RECOGNIZE THAT PROVIDERS

SIMPLY ARE OVERLOADED WITH

GUIDELINES SUGGESTIONS AND

HAVE DIFFICULTY JUST

REMEMBERING WHAT GUIDELINES

MIGHT SUGGEST IN CERTAIN

CIRCUMSTANCES AND THE MORE

THAT WE CAN PROVIDE THESE

KINDS OF DECISION SUPPORT

SYSTEMS, THE BETTER OFF THEY

WILL BE IN BEING ABLE TO

DETERMINE WHETHER THE

GUIDELINE MIGHT APPLY AND IF

SO, HOW THEY WOULD WANT TO

APPLY IT.

>> I THINK JUST TO CLARIFY,

WHEN BILL AND HIS GROUP

TRIED TO INCLUDE A GUIDELINE

FOR HEART FAILURE ABOUT TEN

YEARS AGO, THEY PUBLISHED A

WONDERFUL PAPER SWRASHGS

PAPER THAT GOES INTO GREAT

DETAIL ABOUT THE INFORMATION

THE GUIDELINE REQUIRED, HOW

IT WAS NOT NECESSARILY

DREBLINGTLY AVAILABLE --

DIRECTLY AVAILABLE AND WHAT

INFERENCES THEY NEEDED TO

MAKE IN ORDER TO PROVIDE

THAT INFORMATION.

>> OKAY, THANK YOU.

[APPLAUSE]

>> THE NEXT, WE HAVE ABOUT

25 MINUTES FOR THIS PANEL

DISCUSSION, AND WE CAN REST

FOR QUESTIONS.

>> I HAVE A QUICK QUESTION

FOR DR. MUSEN.

I REMEMBER PARTICIPATING IN

A TECHNOLOGY EXPERT PANEL ON

QUALITY MEASURES AND A LONG

TERM CARE PHYSICIAN SAID

THAT ANY PATIENT CHART WHICH

HAS MORE THAN TEN MEDICATIONS

IS GUARANTEED TO HAVE A DRUG

DRUG-DRUG INTERACTION.

AND AS HE LOOKED AT THE

RETIRED PEOPLE AND BABY

BOOMERS, THEY ALL HAVE A

LONG LIST OF PROBLEMS.

ARE YOU LOOKING AT JUST TWO

CLINICAL GUIDELINES OR ARE

YOU LOOKING AT TEN DIFFERENT

PROBLEMS THAT THESE PATIENTS

HAVE, WHICH ARE ALL BEING

TREATED WITH MEDICATIONS SNK

>> THAT'S A GREAT QUESTION.

AND IN FACT, WE'RE LOOKING

BOTH OF THE PROBLEMS OF

GUIDELINES INTERACTING AND

SUGGESTING INTERVENTIONS AND

LOOKING OVERALL AT THE

QUESTION, PARTICULARLY IN

THE ELDERLY.

IT'S A REALLY SERIOUS

PROBLEM AND THEN IT BECOMES

A MATTER OF TRYING TO

IDENTIFY WHAT ARE THE

PREFERENCES OF THE PATIENTS

AND THE PROVIDERS IN TERMS

OF HOW TO TRIM SUCH A LIST

IN ORDER TO GIVE THOSE DRUGS

THAT ARE THE MOST REASONABLE

TO GIVE, RECOGNIZING FOR

EXACTLY THE PURPOSES THAT IT

MAY BE DANGEROUS TO GIVE

THOSE LONG LISTS OF DRUGS IN

THE KINDS OF SETTINGS WHERE

ACTUALLY A STRICT

APPLICATION TO GUIDELINES

MIGHT SUGGEST SUCH ENORMOUS

NUMBERS OF MEDICATIONS.

>> I HAVE A QUESTION MAYBE

PRIMARILY TO MARK MUSEN BUT

THEN TO THE OTHER PANELIST.

WE'VE HEARD FROM IN THE

KEYNOTE THAT WE DON'T HAVE A

GUIDELINE EXECUTION ENGINE

OUT THERE.

YOU MENTIONED THAT WE HAVE A

RULE-BASED LANGUAGE LIKE SIN

TAX.

WHERE DO YOU SEE THE MAGIC

OF WHAT'S SOMETIMES CALLED

TASK NETWORK MODELS LIKE

ATHENA IS USING?

WHERE DO YOU SEE THE KIND OF

WHERE THEY GO BEYOND THE

RULES WHERE THE RULES BECOME

BRILITY?

AND MAYBE -- BRITLE AND PUT

THIS INTO A PERSPECTIVE OF I

AM DEALING WITH

PHARMACOGENOMICS AND AS AN

INFORMATITION WHO IS

SUPPOSED TO BE GOOD AS

CODING, I CAN HAVE A PERFECT

OPPORTUNITY TO SHAPE THEM

TOWARDS A PARTICULAR

STANDARD AND THE BEST THING

I CAN TELL THEM IS OUT OF

SIN TAX AND THERE IS ALSO

THIS BOUNDARY, WHERE ONE

PHARMACOGENOMICS GUIDELINE

ENDS FOR ONE DRUG BUT

ACTUALLY SOME OF THE GENES

AFFECT OTHER DRUGS.

SO THE PROBLEMS OF THE BOUNDARY

BETWEEN GUIDELINES THERE IS,

TOO.

SO MAYBE SUMMARIZE.

SO THIS PROBLEM OF WE DON'T

HAVE A STANDARD OUT THERE

AND THE MODELS BRING SOME

MAGIC.

HOW DO WE ADDRESS IT?

>> I WANT TO ADDRESS THE

MAGIC.

BUT I GUESS THE FIRST

QUESTION WAS WHERE IS THE

INFLECTION POINT WHERE IT

BECOMES DIFFICULT TO MANAGE?

AND THAT'S A HARD ONE TO

ANSWER.

OBVIOUSLY, NS ONE IS EASY.

WHEN THE RULES ARE SUCH THAT

THEY DO NOT INTERACT, IT'S

VERY EASY.

THE PROBLEM OCCURS WHEN YOU

HAVE TO USE MULTIPLE

INTERACTING RULES IN ORDER

TO DEAL WITH COMPLEX

DECISION-MAKING PROCESSES.

AND I THINK THERE WAS A

PAPER IN AMIA ABOUT SEVEN

YEARS AGO WHERE ATTEMPTS

WERE MADE AT COLUMBIA TO BE

ABLE TO IMPLEMENT IN A RULE

RULE-BASED FRAMEWORK USING

ARDEN, I BELIEVE, PROTOCOLS

FOR MANAGEMENT OF CANCER

CHEMOTHERAPY.

AND THE BOTTOM LINE WAS THAT

THOSE RULES INDIVIDUALLY

MADE SENSE BUT COLLECTIVELY

ACTUALLY WERE VERY

PROBLEMATIC BECAUSE

COMPLEXITIES OF THE

INTERACTIONS.

WHEN YOU LOOK AT WHAT

HAPPENED IN THE 1980S WITH

ALL THE EXCITEMENT OF

RULE-BASED SYSTEMS, ALL OF

THE INDUSTRIAL APPLICATIONS

OF RULE-BASED SYSTEMS GOT

GREAT PRESS EARLY ON AND

ULTIMATELY THERE WAS AN

ENORMOUS RETRENCHMENT

BECAUSE PEOPLE FOUND THAT AS

YOU HAD MORE THAN 20 OR 30

OR 100 RULES TO REPRESENT

COMPLEX SITUATIONS, THE

MANAGEMENT OF THOSE RULES

BECAME ABSOLUTELY

IMPOSSIBLE.

AND SO I THINK AS WE THINK

ABOUT THE IDEA OF AUTOMATING

TASKS FROM THE PURPOSES OF

DECISION SUPPORT, THEN I

THINK IT'S MUCH MORE USEFUL

TO BE THINKING ABOUT TASKS

AS ABTRACTIONS, WHICH WE

WILL THEN IMPLEMENT USING

SOME PROGRAMMING FRAMEWORK,

WHICH COULD ACTUALLY HAVE

RULES UNDER THE HOOD BUT

WHERE THE COGNITIVE TASK FOR

THE DEVELOPER IS TO BE

THINKING ABOUT THINGS OF THE

HIGHEST LEVEL OF ABTRACTION

AND NOT HAVING TO BE WORRIED

ABOUT HOW INDIVIDUAL

PRODUCTION RULES INTERACT.

I THINK ARDEN HAS AN

IMPORTANT ROLE TO PLAY IN

ALL OF THE SITUATIONS WHERE

ONE IS DEALING WITH SAY

ALLERGIES OR DRUG-DRUG

INTERACTIONS OR

CONTRAINDICATED DRUGS.

AND THOSE SITUATIONS THAT

CAN BE STOVE-PIPED AND DEALT

WITH INDIVIDUALLY, I THINK

INDIVIDUAL RULES ARE

WONDERFUL, AND ALTHOUGH A

LOT OF PEOPLE GOT INTO

TROUBLE WITH LARGE RULE-BASED

SYSTEMS IN THE 80S, THERE

WAS AMAZING SUCCESSES WITH

SMALL RULE-BASED SYSTEMS IN

THE 70S.

AND SO MY ARGUMENT WOULD BE

ONE HAS TO KNOW WHEN TO USE

THE RIGHT TECHNOLOGY.

>> I WOULD JUST AMPLE PLI

FY-THAT.

A GUIDELINE TO ACTUALLY

EXECUTE IT.

PATIENTS DON'T COME READY-MADE

TO FIT A GUIDELINE.

THEY COME AT A CERTAIN POINT

IN TIME SO THERE IS AN

ELIGIBILITY POINT AT WHICH

MAY BE A GUIDELINE MIGHT

APPLY TO THEM.

AND AT THAT POINT THERE IS

SOME SITUATION ACTION RULE

THAT MIGHT APPLY.

SO WE TEND TO THINK OF

GUIDELINES AS IF WE HAD A

STANDARD REPRESENTATION OF

THE NETWORK THAT YOU COULD

TAKE INDIVIDUAL POINTS OF

THOSE AND TURN THEM INTO A

SITUATION RULE.

SO NOW YOU HAVE THIS

KNOWLEDGE BASE OF RULES OR

WHATEVER.

I PICKARDEN BECAUSE IT'S A

STANDARD.

WHY INVENT ANOTHER

EXPRESSION LINE?

BUT THOSE THEN CAN

COREFERENCE THE GUIDELINES

THAT THEY ARE PART OF OR THE

GOALS-DIRECTED SYSTEMS THAT

THEY ARE PART OF SO THAT YOU

KNOW, IF YOU NOW NEED TO

REFER TO THE FRAMEWORK, THE

MORE COMPLEX MANAGEMENT OF

THAT PROBLEM, MY GPS MODEL.

YOU COULD BASICALLY HAVE A

GOAL THAT THAT RULE IS PART

OF.

SO I DON'T I KNOW IT'S NOT A

COMPLETE ANSWER BUT RATHER

THAN SAYING IT'S RULES

VERSUS GUIDELINES, SOME

RULES -- TO ACT WAIT A

GUIDELINE, YOU HAVE TO

COMPOSE IT TO RULES AND

OTHER THINGS THAT YOU ARE

GOING TO HAVE TO IMPLEMENT.

1Y

>> TO CONTINUE WITH

DR. STEPHANE.

THERE ARE SOME SYSTEMS WHICH

ENFORCE MEDICATION

ASSOCIATION WITH THE PROBLEM

LIST.

SO FOR EXAMPLE IF YOU

PRESCRIBE SOMETHING LIKE

THAT, THERE SHOULD BE A

PRIMARY DIAGNOSIS OF SOME

COST.

IF WE PUT EVERYTHING CHRONIC

CIRRHOSIS OF LIVER AND

HYPERTENSION, DO YOU

ASSOCIATE THEM LIKE THAT?

DO YOU RATHER PREFER TO

ASSOCIATE EACH OF THE

MEDICATIONS FOR SPECIFIC

CAUSES OR SPECIFIC UTILITY

LIKE SAY HYPERTENSION?

>> I AM HAVING TROUBLE

HEARING YOU BECAUSE OF THE

ECHO.

PERHAPS YOU COULD SPEAK A

LITTLE MORE SLOWLY?

>> THE QUESTION IS I THINK

THE MEDICAL RECORD SYSTEMS,

THEY PROVIDE THE MEDICATION

ASSOCIATION WITH A

DIAGNOSIS.

IT'S A GREAT C.D.S.

SO IT MAKES YOU THINK.

WHETHER SAY I CAN ASSOCIATE

ALL THE MEDICATIONS I HAVE

FOR A CHRONIC LIVER-DISEASED

PATIENT WITH JUST SIR OYSIS

OR FLUID RETENTION OR FLUID

OVERLOAD OR THIS TREATMENT

IS FOR HYPERTENSION.

SO I CAN CLASSIFY LIKE THAT.

HOW DO YOU BALANCE IT?

>> I MEAN, CERTAINLY THOSE

KINDS OF ASSOCIATIONS ARE

THE PERFECT KINDS OF THINGS

TO ENCODE IN A RULE-BASED

APPROACH.

WHERE YOU GET INTO TROUBLE

ARE IN THE KINDS OF

GUIDELINES THAT DEAL WITH

SITUATIONS.

JUST TO PICK ONE, WHICH WE

LIKE TO TALK ABOUT A LOT

BECAUSE IT KOUSTS A LOT OF

TROUBLE AT STANFORD.

WE WERE CODING GUIDELINES

FOR MANAGEMENT OF H.I.V.,

WHICH WOULD HAVE A -- SOUND

LIKE A RULE, SAYING THAT IF

THIS IS THE SECOND EPISODE

OF ANEMIA THAT HAS FOLLOWED

THE USE OF THESE DRUGS, THEN

CONSIDER INTRODUCING OF

THESE OTHER -- SUBSTITUTION

OF THESE OTHER DRUGS.

AND THAT KIND OF A

SITUATION, WHICH SOUNDS LIKE

A RULE IS WHEN YOU GET DOWN

TO IT, IS VERY HARD TO

ENCODE IN A RULE-BASED

FRAMEWORK BECAUSE YOU HAVE

TO FIGURE OUT WHAT DOES IT

MEAN TO HAVE ANEMIA AND THE

SECOND EPISODE OF ANEMIA?

HOW DO YOU FIGURE TOUT

WHETHER THAT ANEMIA FOLLOWED

SOME INTERVENTION, WHICH NOW

YOU WANT TO BE CONCERNED

ABOUT IN PRESCRIBING IN THE

FUTURE?

AND WHEN YOU HAVE TO HAVE

THAT KIND OF REASONING,

ARDEN IS NOT A GOOD HAMMER

FOR THAT PARTICULAR NAIL.

AND SO WHAT I AM SAYING IS

NOT THAT THERE IS A -- ONLY

ONE HAMMER THROUGHOUT AND

IT'S REALLY COMPLICATED,

WHAT I'M SAYING IS THAT

THERE IS A LARGE SUITE OF

THINGS WE WANT TO DO IN

CLINICAL DECISION SUPPORT,

SOME OF WHICH ARE VERY

SIMPLE AND REPETITIVE AND

BEAUTIFULLY DONE IN A

SITUATION ACTION FRAMEWORK.

AND OTHER THINGS, WHICH ARE

HARD AND WHICH ARE BECOMING

INCREASINGLY MORE PREVALENT

IS PATIENTS FRANKLY GET

OLDER AND SICKER, WHERE WE

ARE GOING TO NEED TO TURN TO

OTHER KINDS OF DECISION

SUPPORT TECHNOLOGY IN ORDER

TO HAVE THE KIND OF

INFLUENCE ON THE WORLD THAT

WE WANT.

AND AS BOB ALLUDED, MANY OF

US OVER THE YEARS HAS BEEN

DEALING WITH WAYS OF TRYING

TO REPRESENT THESE MORE

COMPLICATED GUIDELINES TO BE

ABLE TO DEAL WITH THESE

SITUATIONS THAT ALLUDE THE

SAME RULE-BASED FRAMEWORK.

THERE IS NOT A WAY OF DOING

IT.

THERE IS A LOT OF EXPERIENCE

AND DAVE APPROACHES.

AND I THINK WE NEED A LOT

MORE OF THAT.

WE NEED TO LEARN A LOT MORE

OF THAT SO THAT WE CAN HAVE

MORE SOPHISTICATED WAYS OF

REASONING AND THEN WHEN WE

REALIZE WE ARE NOT THERE YET,

WE'RE FALLING PAYOFF -POFF A

CLIFF, WE NEED TO FIGURE OUT

WELL, WHAT'S THE NEXT BEST

KIND OF INFORMATION THAT WE

CAN GET TO HELP US.

>> DID YOU WANT TO HAVE MORE

ANSWER?

I THOUGHT YOU DIRECTED THE

QUESTION TO ME ORIGINALLY.

>> IF SO, I APOLOGIZE.

>> THAT'S FINE.

>> VERY GOOD.

I DIDN'T UNDERSTAND VERY

CLEARLY WHAT YOU WERE

ASKING, SO I AM VERY GLAD.

>> IT'S JUST THAT HOW

DISCREET AND HOW EXTENDED

YOU WOULD LIKE YOUR

DIAGNOSIS AT A PRIMARY --

PROBLEM LIST FOR FACILITATING

WITH MEDICATIONS?

SO CORRECT ME IF I DON'T

UNDERSTAND.

THIS IS ABOUT ORGANIZING

SOMETHING LIKE A LIST OF

PROBLEMS THAT INCLUDES ALSO

MEDICATIONS AND ORGANIZING

THEM, ORGANIZED THEM WITH

SPECIFIC PROBLEMS?

YOU SAY A QUESTION ABOUT

FUNCTIONALITY OR ABOUT

AUTOMATICALLY PROVIDING THIS

TYPE OF INFORMATION?

>> FUNCTIONALITY.

>> SO THAT'S SOMETHING THAT

WE HEARD SEVERAL TIMES THAT

WE REALLY DIDN'T INVESTIGATE

IT.

BUT THE NEED OR THE USE OF

THE PROBLEM VARIES A LOT

BETWEEN USERS AND

SPECIALISTIES.

SOME USE IT AS A VERY

EXTENSIVE DIFFERENTTIAL

DIAGNOSIS TOOL AND WOULD

LIKE TO INCLUDE ALL THE

RELEVANT NOGS IT, USE IT AS

THE MAIN TABLE OF CONTENT

FOR THE WHOLE H.R., WHICH IS

A BIT THE IDEA OF THE

PROBLEM IN MEDICAL RECORD.

AND OTHER ONES JUST USE IT

AS LIKE A TO DO LIST WITH A

LEVEL OF ABTRACTION THAT

MAKES SENSE FOR THEM BUT

OBVIOUSLY DOESN'T INCLUDE A

LOT OF INFORMATION, RELEVANT

INFORMATION THAT'S MENTIONED.

AND SOMETIMES THE MEDICATION

IS INCLUDED.

SOMETIMES THE PROBLEM BAUKS

BECOMES SOMETHING LIKE IT TO

DO LIKE CHECK VERIFIED

MEDICATION EFFECTS OR REMOVE

OR STOP MEDICATION, JUST AS

WITH A DATE SO THAT WHEN

THEY LOOK AGAIN THEY DON'T

FORGET TO LOOK AT THAT AND

ASK THE PATIENT IF THE

MEDICATION WORKS AS EXPECTED

IF THEY HAVE TO CHANGE IT OR

EVALUATE IT.

IT REALLY VARIES.

>> I HAVE A FEW COMMENTS.

ONE IS HOW WE USE ARP 2

MINING THE DATA TO FIGURE

OUT THE ASSOCIATION IS

ACCURATE.

-- ARE CURRENT MEDICATIONS

AND PROBLEMS, LIKE A LARGE

AMOUNT OF CLINICAL DATA AND

FIND ASSOCIATION BETWEEN

MEDICATION AND A PROBLEM.

ONE PROBLEM OF THIS IS YOU

CAN SPEND A LOT FALSE-POSITIVES.

SO THE FALSE-POSITIVES LIKE

IF YOU WANT TO FIGURE THOSE

TWO MEDICATIONS, THIS IS --

TOGETHER IT'S NOT TWO.

NEED A SOPHISTICATED MODEL

TO HANDLE THIS PROBLEM.

I THINK YOU CAN LOOK AT SOME

REFERENCE PUBLISHED FROM

COLUMBIA.

ANOTHER IS ABOUT HER

FUNCTION.

WE HAVE A VAPOR PROJECT,

WHICH YOU CAN USE BECAUSE

THE PROBLEM LIST IS NOT

COMPLETE.

SO YOU CAN ACTUALLY LOOK AT

THE PATIENT PROBLEM LIST.

IF THE PATIENT HAS TAKEN

MEDICATIONS FOR DIABETES,

THE PHYSICIAN SAY WOULD YOU

LIKE TO ADD DIAGNOSIS OF

DIABETES ON THE PROBLEM

LIST?

>> THANKS.

I HAVE I GUESS PARTLY A

PHILOSOPHIC QUESTION BUT

SOMETHING I'VE STRUGD WITH

SOME OF THE WORK THAT I'VE

DONE LOOKING AT THE

ELECTRONIC MEDICAL RECORDS

AND THAT IS IF I GET

ADMITTED TO THE HOSPITAL, I

REALLY WANT THE BEST DOCTOR

POSSIBLE TO TAKE CARE OF ME.

BUT THERE IS NO CONSUMER

REPORTS FOR THESE DECISION

SUPPORT SYSTEMS.

AND SO AS TIME GOES ON, IT

WAS INTERESTING TO SEE THE

VERSION FROM SER NER AND

SOME OF THE ESTABLISHED

ELECTRONIC MEDICAL RECORD

COMPANIES.

THERE ARE GOING TO BE MORE

AND MORE THIRD PARTIES THAT

ARE GOING TO HAVE SOFTWARE.

BUT HOW DOES ONE DETERMINE

THE QUALITY OF A SYSTEM LIKE

THIS?

IS THERE KIND OF A TEST SET

LIKE CONSUMER REPORTS MIGHT

HAVE WHERE YOU KIND OF KNOW

WHAT THE ANSWERS ARE AND YOU

JUST RUN YOUR CLINICAL

DECISION SUPPORT SYSTEM ON?

SO HOW WOULD I DETERMINE WHO

SHALL THE SYSTEM YOU GUYS

ARE USING OR ANYONE ELSE IS

REALLY ANY BETTER AND WHEN

YOU TRAIN IT, HOW DO YOU

KNOW THAT YOUR SYSTEM'S

BETTER NEXT YEAR THAN IT WAS

THIS YEAR?

>> CONTINUE SUPPORT DOCTORS,

HOSPITALS, AND DIFFERENT

LEVEL, RIGHT?

>> NOT SO MUCH TO DETERMINE

QUALITY OF DOCTORS.

I JUST MEANT IN GENERAL.

IN OTHER WORDS, CLINICAL

DECISION SUPPORT SYSTEMS.

HOW DO YOU KNOW WHAT THEY'RE

DOING WHETHER THEY PURPORT

TO DO?

HOW DO YOU GUYS KNOW WHEN

YOU DEVELOP IT IT'S GETTING

BETTER?

>> DO YOU WORK FOR THE

F.D.A.?

>> I'M SURE THEY WOULD LIKE

TO KNOW.

>> REGULATE THESE THINGS.

[LAUGHTER]

>> QUESTIONS.

>> I THINK THE PROBLEM IS

LARGER THAN THAT.

EVEN IF YOU HAVE A GREAT

PERFORMING CLINICAL DECISION

SUPPORT SYSTEM EVALUATED

INTRINSICALLY IN ISOLATION

WITH A SPECIFIC SET OF

DECISIONS AND DATA, ET

CETERA, HOW YOU IMPLEMENT IT

IN THE HEALTHCARE SYSTEM

DATA THAT HAS ZEESE IT AND

THIS IS PRECISELY THE

PROBLEM THAT WE MENTIONED

MANY TIMES HERE MAKES ALL

THE DIFFERENCE, I THINK.

AND SO THE MOST INFORMATIVE

EVALUATION WOULD PROBABLY BE

EXTRINSIX EVALUATION IMPACT

OF SUCH A SYSTEM IN A

SPECIFIC HEALTHCARE

INSTITUTION.

>> YEAH, AND THAT'S KIND OF

TRICKY, THOUGH, ISN'T IT?

I CAN TELL YOU WHO HAS THE

STRONGEST CHESS ENGINE AND

WHO PLAYS JEOPARDY THE BEST

BECAUSE I KNOW THE ANSWERS

TO THE QUESTIONS.

BUT HOW CAN I DETERMINE WHAT

THE QUALITY OF AN

INFORMATION DECISION SUPPORT

SYSTEM IS?

>> SO ONE OF THE ONGOING

ISSUES FOR A LOT OF APPS AND

OTHER H.R. IMPROVEMENTS IS

TO HAVE A ROBUST

DEIDENTIFIED PATIENT DATA

SET.

>> RIGHT.

SO IS THERE A DATA SET THAT

EXISTS?

>> WE HAVE TO MOVE AFOOT TO

CREATE SUCH A THING.

DID YOU WANT TO COMMENT?

>> THE OTHER THOUGHT IS THAT

I THINK THE EVALUATION

COMPONENT OF COURSE IS

ALWAYS IMPORTANT.

ONE MIGHT IMAGINE A TEST

CORP IS LIKE THE NLP FOLKS,

OF COURSE, WHO TEST THEIR

ENGINES AND WHO NOT.

-- WHAT NOT.

WE'VE COME UP WAY MEASURE

THAT CAN BE USED IN CLINICAL

PRACTICE.

WHAT'S THE ULTIMATE CORRECT

OUTCOME FOR DECISION SUPPORT

IS THE OUTCOME, NOT

NECESSARILY THE PROCESS

CHANGE, AS JIM WALKER WAS

TALKING ABOUT.

NOT NECESSARILY ORGANIZED

THE TEST PERHAPS, BUT THE

CORRECT CLINICAL OUTCOME OR

PERFORMANCE?

>> SO WE HAVE A MEASURE

CALLED NEVER NEEDED TO

REMIND WHICH LOOKS AT

CLINICAL PERFORMANCE AND

HAVE TO EQUALITY VAULT THE

SAME ENGINE WHERE THERE

MIGHT BE DIFFERENT REASONS

WHY CLINICAL PERFORMANCES

ARE BEING ACHIEVED.

AND I CAN GIVE YOU A

DIFFERENTTIAL MAYBE SQCH

>> CAN YOU EVALUATE

DIFFERENT CLINICAL SUPPORT

ENGINES SNAESHLINGS IS IT

NOT POSSIBLE TO BE ABLE TO

COMPARE ONE VERSUS THE OTHER

ON DATA THAT ONE MIGHT BE

ABLE TO ESSENTIALLY COMPARE

ONE TO THE OTHER ON?

>> YOU MIGHT FIND A

DIFFERENTIATION WITH THIS

NEED TO REMIND IDEA.

BUT I AGREE A SEPARATE TEST

CORPUS COULD BE HELPFUL.

>> IF THERE ARE THREE

VENDORS WHO ALL CLAIM TO BE

THE BEST, HOW DO I KNOW WHAT

TO BUY?

>> SO THAT'S THE SIMPLEST.

WHETHER IT'S EFFECTIVE IS

HOW IT'S IMPLEMENTED AND

TRIGGERED AND HOW IT

INTEGRATES.

I DON'T KNOW HOW YOU WOULD

TEST THAT OTHER THAN --

>> I THINK THERE ARE TWO

ISSUES HERE AND THAT IS IF

THERE IS, FOR EXAMPLE,

EVIDENCE THAT HAS BEEN USED

TO FORM LATE A GUIDELINE,

THEN IN AN ARTIFICIAL

LABORATORY SETTING YOU CAN

DETERMINE WHETHER THE

DECISION SUPPORT SYSTEM IS

ALLOWING ADVICE THAT FOLLOWS

THE GUIDELINE.

THAT IS THE PROCESS THAT BOB

ALLUDED TO AS BEING

IMPORTANT BUT NOT REALLY

WHERE THE MONEY IS.

ON THE OTHER HAND, OUTCOMES

IS VERY, VERY HARD BECAUSE

IT'S SO IDEOSIN CRATIC BASED

ON THE PARTICULAR PATIENT

AND PARTICULAR SITUATION AND

EVEN IF YOU ARE DOING A

TRIAL, WHERE YOU ARE LOOKING

AT A DECISION SUPPORT SYSTEM

ON A CADRE OF PATIENTS, THE

TIME IT TAKES AND CERTAINLY

IN THE AREA OF CHRONIC

DISEASE, TO BE ABLE TO KNOW

WHETHER YOU HAVE REACHED

OUTCOMES IS DIFFERENT IS SO

LONG YOU'RE AT A SITUATION

WHERE THE BEST THAT YOU CAN

DO IS AT LEAST SAY THAT I

HAVE FAITH IN THE GUIDELINE

REPRESENTING THE EVIDENCE

AND THEREFORE I'LL GO WITH

PROCESS BECAUSE IT'S

EXPEDIENT EVEN THOUGH IT MAY

NOT BE THE METRIC THAT

MATTERS THE MOST.

>> THE GUIDELINES HAS SOME

OF THE VALIDATION PROCESS.

[INDISCERNABLE]

 YEAH.

IF NOT, THEN THERE IS

ANOTHER PROBLEM.

>> ONE LAST COMMENT.

>> AS FOR THE V.A., THERE

WAS ENORMOUS DIFFERENCE IN

HOW DIFFERENT SITES USED THE

SAME TOOLS, HOW MUCH ENERGY

THEY PUT INTO USING THEM.

SO IT WASN'T REALLY THE

VENDOR SO MUCH AS OTHER

FACTORS.

AND I FOUND THAT TO BE TRUE

FOR THE COMMERCIAL VENDORS

WITH WHICH WE WORK AS WELL.

THEY HAVE INCREDIBLE SETS OF

TOOLS, BUT THEY'RE NOT FULLY

UTILIZED.

AND THE SITES OF OUR WORKING

ON THOSE THINGS.

IT WOULD BE A TRICKY THING

TO DO TO SAY THIS VENDOR IS

STRONGER, WEAKER THAN THAT

VENDOR UNLESS YOU WERE

SOMEHOW MAGICLY BE ABLE TO

CONTROL FOR THE PEOPLE WHO

ARE USING IT, HOW MUCH

ENERGY THEY PUT INTO IT AND

SO ON.

>> ONE QUESTION.

>> HI, I'M TONY WIDER.

MORE THAN ONE SPEAKER TODAY

HAS NOTED THE USE OF

STANDARD TERMINOLOGIES FOR

USING COMMUNICATION TO C.D.S

SYSTEMS AND I WANT TO

EMPHASIZE THAT WHAT WE

REALLY NEED ARE VALUE SETS

AND TERMINOLOGY SUBSETS TO

FOCUS INTERACTION.

ON THE ONE HAND, THIS GIVES

GUIDANCE TO THE NLP SYSTEMS

ABOUT WHAT WE'RE LOOKING FOR

IN TERMS OF DECISION SUPPORT

AND THEN IF A C.D.S

PERSPECTIVE, IT INDICATES

WHAT WE NEED TO FIND IN

ORDER TO RENDER DECISIONS.

AND SO WHEN BOB SPOKE

EARLIER ABOUT THE DIFFERENCE

BETWEEN QUALITY MEASURES,

WHICH ARE RETROPERSPECTIVE

-- TE TROSPECTIVE.

I THOUGHT ABOUT THE FACT

THAT INCREASING FOCUS IS

BEING PAID TODAY TO QUALITY

MEASURES AND AS A RESULT

MORE AND MORE VALUE SETS ARE

BEING VEPD DEVELOPED AND

PUBLISHED.

FOR EXAMPLE, WE'RE WORKING

ON DEVELOPING BEHAVIORAL

HEALTH VALUE SETS AND

QUALITY MEASURES.

SO MY QUESTION FOR BOB AND

FOR EVERYBODY IS CONSIDERING

THE DIFFERENCES BETWEEN

QUALITY MEASUREMENT AND C.D.S,

DO YOU FEEL THAT THE

CONCEPTULE BASIS THAT IS

KINDS OF CONCEPTS WE FIND IN

STANDARD TERMINOLOGIES ARE

SUFFICIENTLY SIMILAR BETWEEN

QUALITY MEASUREMENT AND

CLINICAL DECISION SUPPORT

THAT INDEED WE CAN DEVELOP

AND REUSE THEM FOR BOTH

PURPOSES?

>> WELL, I THINK THEY'RE THE

SAME DATA CLASSES.

CERTAINLY FOR PROACTIVE

DECISION SUPPORT MAY WANT TO

DO THINGS THAT ARE NOT

MEASURED IN THE NUMERATOR OR

DENOMINATER OF A QUALITY

MEASURE.

SO YOU MAY WANT TO CHECK ON

TEMPORAL RELATIONSHIPS,

EXISTENCE OF THINGS,

COMORBID CONDITIONS AND

OTHER THINGS LIKE THAT.

SOME OF THOSE BASICALLY AIM

TO TRY TO MAKE THE DECISION

SUPPORT AS TARGETED AND AS

SPECIFIC AS YOU CAN.

THE QUALITY MEASURE IS KIND

OF A COURSER NET THAT

PROBABLY DOESN'T HAVE ALL

THOSE REFINEMENTS IN IT.

AND SO I THINK WHEN YOU ARE

TRYING TO DELIVER IT AND

AVOID ALERT FATIGUE AND ALL

THESE OTHER THINGS THAT

DON'T APPLY TO YOUR PATIENT,

NON-SPECIFIC RECOMMENDATIONS,

YOUR OBLIGATION IS REALLY TO

TRY TO PULL UP MORE DATA.

SO I DON'T THINK IT'S A

DIFFERENCE IN CLASS THAT'S

WE HAVEN'T THOUGHT OF THAT

AREN'T IN YOUR -- BUT IT'S

PROBABLY GOING TO NEED MORE

PARAMETERS WITHIN THOSE

CLASSES.

>> I HAVE A QUICK COMMENT TO

MAKE ON THAT.

I SEE THE DISTINCTION IS

QUALITY MEASURES IS PRETTY

MUCH ENCODES THE CLINICAL

GUIDELINES, BUT AS CLINICAL

DECISION SUPPORT ALLOWS YOU

TO NOT ONLY MEASURE GIVEN

THAT YOU HAVE A PARTICULAR

DIAGNOSIS, YOU NEED TO TREAT

THEM IN A CERTAIN WAY, WHERE

AS CLINICAL SUPPORT SAY HOW

QUI MAKE A CERTAIN

DIAGNOSIS?

MAYBE YOU CAN HELP WITH THAT

TOO, WHICH IS NOT THE FOCUS

OF QUALITY MEASURES.

AT LEAST THAT'S HOW I'VE

BEEN LOOKING AT IT.

>> I HAVE A COMMENT TOO, AND

THIS IS BASED ON THE

EXPERIENCE WE HAD ALSO

TRYING TO MATCH WHAT CAN BE

FOUND IN THE NEXT USING

INFORMATION EXTRACTION IN

WHAT'S EVENTUALLY USED AND

NEEDED FOR PLATING THE

PROBLEM LIST OF DRIVING

CLINICAL SUPPORTS OR OTHER

APPLICATIONS.

SO WHAT WE FOUND IS THAT

THERE IS ACTUALLY A MISMATCH

OFTEN.

WELL, MOST OF THE TIME YOU

CAN FIND SOMETHING THAT

MATCHES WITH WHAT'S MOST OF

THE TIME MISSING IS

CONNECTIONS BETWEEN THE

CONCEPTS.

SO MOST OF THE TERMINOLOGIES

WE USE ARE JUST

TERMINOLOGIES THAT ARE NOT

REALLY WELL ORGANIZED AND

COMPLETE ONTOLOGYS WITH

MULTIPLE RELATIONS WE CAN

USE TO NAVIGATE DIFFERENT

LEVELS OF GRANUL ARITIES AND

DETAILS ABOUT CONCEPTS.

SO THERE IS SOME OF THAT BUT

WE ACTUALLY NEED TO ADD

LOGIC OIT TO NAVIGATE

TERMINOLOGIES AND THIS IS

PROBABLY WHAT I THINK WOULD

BE THE MOST USEFUL TO ADD

CONNECTIONS BETWEEN THE

DIFFERENT CONCEPTS BESIDES

WHAT'S ALREADY EXISTING AND

ALREADY VERY RICH BUT NOT

COMPLETE YET.

[INDISCERNABLE]

>> THE MEDICATION.

WE FIND LIKE WE USE IT.

SIGH LOT OF IT ACTUALLY HAS

CLASSES --

[INDISCERNABLE]

 THIS IS AN AREA THAT NEEDS

MORE WORK TO BUILD THIS KIND

OF STANDARD.

>> THANKS, EVERYONE.

FOR A GREAT DISCUSSION AND

WE CAN CONTINUE THE

DISCUSSION WHILE WE HAVE

SESSIONS DURING THE LUNCH

HOUR AND WE'RE RUNNING OUT

TIFLE.

SO THANK ALL THE SPEAKERS.

[APPLAUSE]

>> I WON'T DELAY YOUR SESSION

BUT I WANT TO PRESENT TO YOU ONE

MORE INSURMOUNTABLE OPPORTUNITY.

MAINLY IF YOU HAVE TIME LEFT

WHEN YOUR MEETING HERE ENDS,

THERE IS IN THE LIBRARY WHICH IS

TO SAY THE BUILDING YOU CAN SEE

NEAREST AN EXHIBITION CALLED

NATIVE CONCEPTS OF HEALTH AND

ILLNESS.

WHICH WE WORKED PRETTY HARD ON.

IT RELATES TO AMERICAN INDIANS

NATIVE ALASKAS, NATIVE HAWAIIANS

AND WE'RE PRESENTING THE MENTAL

MODEL, SO TO SPEAK, WHICH THEY

HAVE OF HEALTH AND ILLNESS.

ARGUING THAT, IF YOU MAY OR MAY

NOT AGREE WITH ME IF THE DOCTOR

AND PATIENT DON'T HAVE THE SAME

IDEA IN MIND THIS THERAPY

DOESN'T WORK BECAUSE THE PATIENT

DOESN'T FOLLOW IT.

SO THESE PEOPLE HAVE THEIR OWN

IDEAS.

SOME ARE QUITE WORTHWHILE

SHARING, THAT'S WHY WE HAVE THE

EXHIBITION.

YOU WOULD BE VERY WELCOME OVER

THERE, IT'S INTERACTIVE, YOU CAN

FIND YOUR OWN WAY THROUGH.

AND PLEASE JOIN US IF YOU GET

THE CHANCE.

THANK YOU.

>> MY NAME IS STEVE LOHR,

TECHNOLOGY REPORTER FOR THE NEW

YORK TIMES.

I HAVE COVERED -- THE REASON I'M

HERE IS JUST TO EXPLAIN, I HAVE

COVERED ELECTRONIC HEALTH

RECORDS FOR A NUMBER OF YEARS

AND HAVE REPORTED A FAIR BIT ON

ARTIFICIAL INTELLIGENCE

TECHNIQUES INCLUDING NATURAL

LANGUAGE PROCESSING.

FOR USES IN A NUMBER OF FIELDS.

SO I'M INTERESTED IN THE

SUBJECT.

THIS AFTERNOON'S PANEL IS

ENTITLED STAKEHOLDERS AND WE

HAVE A NUMBER OF FOUR

DISTINGUISHED PANELISTS, THREE

FROM OUR INDUSTRY STAKEHOLDERS

FROM AETNA, SIEMENS AND IBM AND

OUR INITIAL SPEAKER, JACOB RIDER

WHO IS A HYBRID, HAS IN TERMS OF

HIS PERSPECTIVE AS FAMILY

PHYSICIAN, THROUGH INDUSTRY AND

FOR SEVERAL YEARS, AND IS NOW

SENIOR POLICY ADVISER TO THE THE

OBAMA ADMINISTRATION IN THE

OFFICE OF THE NATIONAL

COORDINATOR.

JACOB GOT INTO INDUSTRY THROUGH

COMPLAINT I GATHER.

HE WAS A PRACTICING PHYSICIAN AS

WELL AS IT GURU AND ON HIS BLOG

I THINK IN 2004 HE COMPLAINED

ABOUT THE USABILITY PROBLEMS OF

ELECTRONIC HEALTH RECORD.

AT FIRST THE VENDOR CALLED HIS

BOSS TO TRY TO SHUT HIM DOWN AND

THEN THAT DIDN'T WORK BECAUSE HE

WOULDN'T TAKE DOWN HIS BLOG.

AFTER THAT THEY ENGAGED IN

CONVERSATION AND THEY HIRED HIM.

AS CHIEF MEDICAL DIRECTOR FOR

MISIS WHICH LATER BECAME ALL

SCRIPTS WHEN THE COMPANIES

MERGED.

LAST YEAR WHEN HE DECIDED TO

JOIN THE OBAMA ADMINISTRATION IN

AN INTERVIEW HE WAS ASKED TO HIS

ROLE AS SENIOR ADVISER AT THE

OFFICE OF NATIONAL COORDINATOR.

AND SAID MY ROLE IS TO LISTEN TO

THE MARKET.

TO DOCTOR, NURSES, HOSPITALS AND

VENDORS.

WELL TODAY JACOB IS HERE TO

SPEAK AND WE'LL ALL BENEFIT FROM

WHAT HE HAS TO SAY.

THANK YOU.

[APPLAUSE]

STEVE DEMONSTRATED WHAT A GOOD

REPORTER DOES WHICH IS TO DO

RESEARCH.

I'M IMPRESSED THAT YOU DO THAT

RESEARCH.

I SHOULD HAVE WORN MY RED SOX

HAT.

I GOT TO ENHANCE SOME OF THE

BACKGROUND, I DID DIB KNOW STEEF

WAS GOING TO DO SO MUCH

RESEARCH.

I'LL TALK ABOUT MY BACKGROUND,

SOME STUFF THAT STEVE DIDN'T

TOUCH ON, THEN TALK ABOUT THE

PERSPECTIVE OF THE OFFICE OF

NATIONAL COORDINATOR AND SOME OF

THE PROBLEMS THAT WE SEE THAT

PERHAPS CKS AND BY EXTENSION ARE

INCLUSION PERHAPS NATURAL

LANGUAGE PROCESSING MIGHT PLAY A

ROLE IN SOLVING SOME OF THOSE

PROBLEMS.

PERHAPS SOME REALITIES THAT WE

SEE.

SO THE PROGRAM SAID THAT DOUG

FRIDSMA IS TALKING NOW.

AND DOUG COULDN'T MAKE IT TODAY

SO I DID A LITTLE IMAGE SEARCH

ON DOUG FRIDSMA, THIS IS PAGE 1

AND BY THE WAY THERE'S A LITTLE

NATURAL LANGUAGE SPROASESSING

PITCH HERE BECAUSE GOOGLE MUST

BE DOING SOMETHING.

SO HERE WE SEE VARIOUS

ITERATIONS OF DOUG FRIDSMA AND

SOME WHO AREN'T DUG FRIDSMA.

THIS IS -- DOUG FRIDSMA.

HERE IS PAGE 2.

AND THIS GO ON GOOGLE AND DO

THIS.

DO WE HAVE A BLAZER?

HERE IS THE DO DOUG FRIDSMA

THERE.

SEARCH FOR DOUG FRIDSMA AND YOU

GET ME WEARING MY RED SOX HAT

WITH A LITTLE MORE FACIAL HAIR.

SO GOOGLE AND THE ORGANIZERS

TODAY KNEW THAT I AM DOUG

FRIDSMA.

I GREW UP IN BOSTON UNLIKE DOUG

FRIDSMA.

ON MY LINKED IN PROFILE WHICH I

KNOW STEVE LOOKED AT YESTERDAY,

IT LISTS ME AS THESE THINGS,

PARENT SON FAMILY BLOG PIONEER,

I DIDN'T SAY NERD BUT I THINK

IT'S THERE.

FINDING THE BENEVOLENCE IN

BUSINESS IS AN INTERESTING THING

CALLED GOOGLISM SO YOU GOOGLE

GOOGLISM AND FIND IT AND YOU CAN

ASK IT SOMETHING, GIVE IT A WORD

OR NAME OR SOMETHING AND IT SAYS

JACOB RIDER IS DOT, DOT, DOT,

LOOKING FOR BENEVOLENCE IN

BUSINESS, FROM ABOUT TEN YEARS

AGO.

THE APPLE DOESN'T FALL FAR FROM

THE TREE.

THAT'S AN P L TREE.

-- APPLE TREE.

MY DAD IS A PSYCHIATRIST AND HOW

I PRACTICED FAMILY MEDICINE AND

HEALTH IT IS FROM A PSYCHIATRIC

PERSPECTIVE.

I'M STILL A PRACTICING FAMILY

DOCTOR, THIS IS NOT AN

ENDORSEMENT OF ANY ORGANIZATION.

YOU HAVE TO SAY THAT, RIGHT?

YOU CAN SEE ALL FIVE PEOPLE WHO

ANSWERED THE HEALTH GRADE SURVEY

THOUGHT IT WAS A GREAT DOCTOR.

AWARDS AND RECOGNITIONS,

DR. RIDER THE HEALTH GRADES

RECOGNIZED DOCTOR.

MY WIFE IS A LAW PROFESSOR AND

WRITES RECOMMENDATIONS FOR

STUDENTS AND JOE SMO WOULD

DEFINITELY WAS ONE OF MY

STUDENTS, NOT ALWAYS THE BEST

RECOMMENDATION.

I SPENT TIME IN STARTUP AND THE

STARTUP THAT I WORKED FOR WAS A

COMPANY CALLED MED REMOTE, WE

WERE DOING MACHINE LEARNING AND

NATURAL LANGUAGE PROCESSING

ABOUT TEN YEARS AGO.

THE COMPANY WAS ACQUIRED BY

NUANCE AS WAS OUR IP.

SO THIS IS MAYBE A HOOK TO THIS

CONFERENCE.

I'M FASCINATED BY WHAT MIGHT BE

FEASIBLE AND I THINK THAT'S WHAT

WE WERE DOING BACK THEN.

DOING WORK THAT MIGHT MAKE STUFF

POSSIBLE SOME DAY.

THAT'S WHAT I'M HEARING, WE ARE

INCHING EVER CLOSER TOWARD WHAT

WE MIGHT BE ABLE TO DO AS WE

ALIGN CLINICAL DECISION SUPPORT

NATURAL LANGUAGE PROCESSING I

DID SPEND TIME AT HN EHR VENDOR

AND LEARNED A LOT WHEN I DID

THAT BUT MY FOCUS HAS BEEN ON

HOW WE CAN HELP PEOPLE BE

HEALTHIERER.

AS BOTH AS A FAMILY DOC AND AS

AN INFORMATICIAN.

I TRY AND KEEP THE END IN MIND.

FOR ME, THAT'S WHAT THIS IS.

I FOUND MY WAY TO THIS CITY,

SPECIFICALLY THIS BUILDING.

WHERE ONC IS HEAD QUARTERED AND

HHS.

FOR THOSE WHO DON'T KNOW, JUST A

LITTLE EXPLANATION WHERE WE SIT

AND WHAT WE DO.

THE MEANINGFUL USE STUFF IS

COMPRISED OF TWO REGULATIONS.

ONE FROM CMS, THAT'S THE

INCENTIVE PROGRAM.

THE OTHER FROM OFFICE OF

NATIONAL CONSIDERED NAYTOR.

AND WE -- COORDINATOR AND WE

DEFINE STANDARDS CERTIFICATION

CRITERIA FOR ELECTRONIC HEALTH

RECORDS.

SO EVERYBODY HAS SEEN THIS SLIDE

WE HAVE PLAGIARIZED INTO OUR

SLIDE DECKS.

DO YOU KNOW WHAT DID THIS ONE

ORIGINALLY?

WHO DID IT?

ESPECIALLY WITH STEVE IN THE

ROOM I HAVE TO MAKE SURE THAT WE

PROVIDE APPROPRIATE ATTRIBUTION.

SOMEONE TOOK THIS PICTURE AND

MADE IT INTO MANY, MANY SLIDE

DECKS.

SO THE MED FORE I USUALLY -- MET

FORE I USE IS TO -- METAPHOR I

USE IS DRIVEING A CAR.

THE MEANINGFUL USE PROGRAM IS

TAKING YOUR DRIVER'S TEST WHEN

YOU'RE 16 AND YOU HAVE TO DO THE

RIGHT THINGS, YOU HAVE TO DO A 3

POINT TURN AND YOU HAVE TO BE

ABLE TO STOP AT THE STOPLIGHT

AND JUST DO STUFF THAT OUGHT TO

BE DONE.

THAT IS DEFINED BY THE CMS

REGULATION IF YOU DO THAT IS ALL

YOU GET A LOT OF MONEY, QUITE A

BIT IF YOU'RE A HEALTHCARE

PROVIDER.

AS FRIEND OF MY SON'S FOUND OUT

WHEN HE WENT FOR HIS DRIVER'S

TEST AND HIS DAD'S UNINSPECTED

CAR, YOU NEED TO BE DRIVING AND

INSPECTED CAR OR YOU WILL

FAILURE DRIVER'S TEST.

THE ANALOG IS IN OUR PROGRAM YOU

CAN'T MEANINGFULLY USE AN EHR

UNLESS IT'S CERTIFIED SO THE

INCENTIVE PROGRAM SAYS YOU MUST

MAKE MEANINGFUL USE OF CERTIFIED

HEALTH INFORMATION TECHNOLOGY.

WE ARE THE SERVICE STATION.

ONC.

WE DEFINE THE INSPECTION CRY

TIER MAYA -- CRITERIA.

BECAUSE THERE'S TESTING BODIES

AT THE SERVICE STATIONS BECAUSE

WE DON'T SCALE, WHEREAS SHELL

AND JOE SMO OR THIS IS THE

INSPECTION STORE.

THEY DO SCALE.

SO WE DEFINE CERTIFICATION TRIE

CRITERIA AND SO WE SET THE

STANDARDS WHICH EHRs ARE

CERTIFIED.

BY EXTENSION THEN WE ALSO DEFINE

WHAT THE EHRs ARE CAPABLE OF,

MEANING MEANINGFULLY USED TO DO.

SO IF WE WERE TO EXPAND THE

CERTIFICATION CRITERIA IN SOME

WAY, THE MEANINGFUL USE CRITERIA

IS ALSO EXPANDED.

IF WE CONSTRAIN THE MEANINGFUL

USE CRITERIA, EHRs MAY NOT DO

SO MUCH CONSISTENTLY.

AS YOU TELL WE HAVE A LOT OF

LEVERAGE HERE.

PEOPLE PAY ATTENTION TO THINGS

THAT WE SAY IN OUR REGULATIONS.

SO WHAT HAPPENS IS PRODUCTS ARE

CERTIFIED.

WHEN A PRODUCT IS CERTIFIED IT

JOINS COMPETITORS ON THE P

CHAPEL THE CERTIFIED HEALTH IT

PRODUCT LIST ON OUR WEBSITE.

AND SO IF YOU ARE A CLINICIAN OR

HOSPITAL OR SOMETHING LIKE THAT

AND YOU WANTED TO PURCHASE AN

EHR YOU COULD GO TO THE THE

CHAPEL AND CONFIRM WHAT THE

SALESPERSON HAS TOLD YOU.

OR INVALIDATE IF THE EHR IS

CERTIFIED OR NOT.

SO THESE ARE GUIDEN PRINCIPLES,

THAT'S A PICTURE OF MY DAD, THE

PSYCHIATRIST, THESE ARE FIVE

PRINCIPLES.

A WORTHY STEWARD OF THE TRUST

MONEY AND TRUST, FOCUS ON YUT

COME, UNDERSTAND WHAT WE'RE

DOING AND WHAT THE END GAME IS,

NOT JUST WHAT'S IN FRONT OF US.

GO BEYOND WHAT WORK MEAN, LET'S

BE PRAGMATIC.

WE'RE NOT GOING TO DO CHITY

CHITY BANG BANG BECAUSE THAT'S

NOT WORKING TODAY.

IF WE THINK ABOUT WHAT WE MIGHT

DO IN DOMAIN OF CDS AND NATURAL

LAKE SPROASESSING, WHAT'S

WORKING TODAY?

HOW CAN WE LEVERAGE THAT TO MAKE

IT FEASIBLE FOR THE REST OF THE

INDUSTRY.

FOSTER INNOVATION AND SUPPORT

HEALTH IT BENEFITS FOR ALL, WE

TRANSLATE THAT INTERNALLY AS WE

LOOK OUT FOR THE LITTLE GUY.

IT'S NOT UNCOMMON TO BE VISITED

BY EXECUTIVES OF LARGE HNG

ORGANIZATION SO IT TAKES EFFORT

ON OUR PART TO MAKE SURE THAT WE

GO OUT INTO THE FIELD AND Id

TALK TO AN EHR VENDOR WHO HAS

$2 MILLION OF REVENUE AND NOT

200 MILLION OR $2 BILLION OF

REVENUE, THEY DON'T HAVE THE

BANDWIDTH.

THEY'RE TREADING WATER, THEY

CAN'T COME TO US AND ELOQUENTLY

EXPRESS THINGS IN POWERPOINT

DECKS SO A LOT OF FOCUS THE LAST

COUPLE OF YEARS HAS BEEN ON

MEASUREMENT.

AND AS WE THINK ABOUT QUALITY

IMPROVEMENT AND THE IMPROVEMENT

OF HEALTHCARE, NOTICE A LOT OF

EFFORT IN OUR RULE AND THE ONE

FROM CMS FOCUSED ON MEASUREMENT.

I THINK OF MEASUREMENT LIKE

WE'RE GIVING ALL THE KIDS IN

CLASS A CERTAIN GRADE BUT WE

HAVEN'T EMPOWERED THEM TO GET

GOOD GRADES.

SO EVERYBODY GETS A C MINUS AND

WE EXPECT THAT TO MOTIVATE THEM

TO GET AN A PLUS.

I THINK OF CDS AS PERHAPS THE

TOOLS THAT WOULD ENABLE THEM TO

GET A PLUSES BUT I GI DIGRESES.

HISTORICALLY QUALITY MEASURES

WERE DONE USING HUMAN WHOSE READ

PAPER RECORDS AND THEN MADE

DECISIONS BASED ON LITTLE SURVEY

TOOLS THAT THEY HAD ABOUT B

WHETHER QUALITY CARE WAS

RENDERED.

IT'S A RETROSPECTIVE VIEW.

WE'RE LOOKING IN THE THE

REAR-VIEW MIRROR, SOMETIME AS

LONG TIME AGO ESPECIALLY IF

WE'RE USING PAPER RECORDS AND

ABSTRACTERS LOOK AT WHAT CARE WE

RENDERED, 6, 12, 18 MONTHS AGO

AND WE DEVELOP AN INTERVENTION

FOR HOW WE CAN SOLVE THE

PROBLEM.

THAT WHOLE ITERATION TAKES 36

MONTHS IS NOT UNCOMMON FROM

DISCOVERY OF QUALITY ISSUE TO

RESOLUTION AND BY THEN IT MIGHT

HAVE GONE AWAY ANYWAY OR WE

CREATE OTHER ONES.

THE DECISION IS PROSPECTIVE AND

WE HAD SOME DISDISCUSSION DURING

THE LAST SESSION AND HOW

DECISION SUPPORT IS PROSPECTIVE

AND THERE ARE AS JODY MENTIONED

TOWARD THE END, THERE ARE

DIFFERENT QUALITIES TO DECISION

SUPPORT, NOT JUST THE MIRROR

IMAGE OF CLAM QUALITY MEASURES.

I WOULD LIKE TO TALK ABOUT THE

OTHER DECISION OF CDS, THAT'S

CONSUMER DECISION SUPPORT.

THIS IS OUR SURGEON GENERAL.

AT THE LAUNCH OF OUR CONSUMER

HEALTH PROGRAM LAST FALL.

WHEN YOU THINK CDS LET'S NOT

JUST THINK ABOUT THE PROVIDER AS

THE RECIPIENT OF CDS.

IT'S ALSO THE PATIENT OR THE

CAREGIVER OR OTHER MEMBERS OF

THE CARE TEAM.

IN FACT, THE PHYSICIAN MAY BE

THE RATE-LIMITING STEP HERE.

AS WE THINK CREATIVELY HOW WE'RE

GOING TO SOLVE THE HEALTHCARE

PROBLEM, WHY IS IT THAT A

MAMMOGRAM REMINDER SHOULD EVER

BE PRESENTED TO A PHYSICIAN.

WE LOOK AT ORGANIZES DOING

THINGS PROGRESSIVELY.

'S IDIOTIC TO REMIND ME ABOUT MY

PATIENT'S MAMMOGRAMS SO I CAN

ASK SOMEBODY ELSE TO ASK

SOMEBODY ELSE.

LET'S ASK THE PATIENT.

IF OUR SYSTEMS KNOW HOW TO DO

THAT THERE'S NO REASON FOR US

NOT TO DO THAT.

SO I THINK WE NEED TO THINK MORE

AND MORE ABOUT BEING

PATIENT-CENTERED.

I CAN REMEMBER WHEN I STARTED IN

THE VENDOR WORLD, THERE WAS ALL

THIS TALK ABOUT BEING PHYSICIAN

CENTERED.

THAT WAS LIKE A REVOLUTIONARY

THOUGHT TO STOP BEING HOSPITAL

CENTERED.

WE'RE GOING TO STOP THINKING

HOSPITALS.

WE'RE GOING DOWNSTREAM TO THE

THE DOCTORS.

I REMEMBER TALKING TO THE CEO

AND SAID YOU'RE NOT BACK TO THE

ROOT OF THE ISSUE YET.

AND HE LOOKED AT MY QUIZZICALLY

AND SAID THE PATIENT IS MORE

IMPORTANT THAN DOCTOR.

ANYWAY, THIS IS A BIT OF A

PORTRAYAL WHAT WE MIGHT SEE AND

IT'S A CARE CATURE AND

OVERSIMPLIFIES BUT FROM RESEARCH

TO GUIDELINES, TO CDS AND THEN

TO MEASUREMENT, CDS IS THE PART

THAT IN GENERAL HASN'T BEEN

PRESENT IN THE PAST DECADE OR

TWO.

BUT IT'S NOT A WATER FALL.

IT MIGHT BE BETTER THOUGHT OF AS

A CIRCLE.

SO RESEARCH TO GUIDE GUIDELINES

GOES TO INTERVENTIONS, WE HOPE.

GOES TO HEALTHCARE AND LIFE.

THAT WAS A DIFFICULT CIRCLE TO

DRAW BECAUSE IT ENCOMPASSES SO

MUCH.

THEN WE MEASURE IT, THE REASON I

DIDN'T SAY PHYSICIAN OR HOSPITAL

OR CARE PROVIDER, IDEALLY OUR

INTERVENTION MAY HAVE GREATER

SCOPE THAN JUST THE CARE

PROVIDERS AS I TALKED ABOUT A

SECOND AGO.

THEN WE MEASURE IT AND THAT

MIGHT BE PART OF A VERY TIGHT

LOOP, SO YOU SAY IT'S A FIVE

MINUTE LOOP OR A FIVE YEAR LOOP

OR SOMETHING IN THE MIDDLE.

NOTHING DICTATES HOW SLOW OR

FAST IT MUST BE.

AN IMPORTANT PIECE AS WE THINK

ABOUT ALL THE CONNECTIONS HERE,

CERTAINLY VERY IMPORTANT FOR US

AT ONC, IS ADOPTABILITY OR

USABILITY OF THE SYSTEMS.

SO WE HEAR A LOT ABOUT HOW CPOE

IS THIS HORRIBLE THING, THAT WE

HAVE IMPOSED ON THE INDUSTRY.

I HAD A PLEASANT CONVERSATION

WITH A PHYSICIAN LAST WEEKEND

WHO SAID THE MEANINGFUL USE

REQUIREMENT FOR CPOE

COMPUTERIZED ORDER ENTRY OR

PHYSICIAN ORDER ENTRY OR

SOMETHING LIKE THAT IS A

TERRIBLE THING.

SHE THINKS IT DOESN'T -- OUR

PRESUMPTION THAT THE PROVIDER

SHOULD BE INTERACTING WITH A

COMPUTER IS AN INAPPROPRIATE ONE

AND SHE CAN THINK OF LOTS OF

WAYS THIS WOULD WORK BETTER SO

SHE ENCOURAGED ME TO RETHINK

WHETHER CPOE SHOULD BE PART OF

MEANINGFUL USE.

AND THAT'SNARY GAME.

A KEY PART OF HER CONCERN WAS

THAT IT'S NOT USABLE.

CPOE DOESN'T FIT WITH THE WAY

SHE WORKS.

THIS IS A BIT OF A USER

EXPERIENCE FRAMEWORK THAT I LIKE

TO USE AS I THINK ABOUT

USABILITY.

SO WHEN I HEAR COMPLAINTS OR

CONCERNS ABOUT USABILITY THIS IS

THE FRAMEWORK I CONSIDER.

FUNCTIONAL MEANS IT DOES WHAT IT

WAS INTENDED TO DO.

REMIGHT BE LIEBL MEANS THAT WAY

-- RELIABLE MEANS THAT WAY EVERY

TIME SO SYSTEMS THAT GO DOWN OR

BREAK OR ARE DOWN EVERY NIGHT

FOR BACK UP.

MY FIRST EHR EXPERIENCE, I'M A

NIGHT OWL SO UNFORTUNATELY AFTER

KIDS SOCCER PRACTICE OR WHAT NOT

MAKE IT TO THE COMPUTER ABOUT

MIDNIGHT TO FINISH PROGRESS

NOTES FOR THE DAY, SO I'M ONE OF

THOSE UNFORTUNATE FEW WHO

COULDN'T FINISH DURING THE DAY

BUT OUR SYSTEM WOULD GO DOWN AT

12:30 FOR BACKUP AND STAY DOWN

MOST OF THE NIGHT.

ABOUT 4:30 IT CAME BACK UP.

F I WANTED TO NISH THE NOTES FOR

THE NEXT DAY WHICH WAS AN

IMPORTANT THING TO DO OR ELSE I

GET TOO FAR BEHIND I WAKE UP AT

4:30 TO FINISH THE NOTES WHICH

WAS NUTS SO THAT WASN'T A

RELIABLE SYSTEM.

USABILITY IS MEETS MY

EXPECTATIONS.

IT'S EASY.

I CAN INTERACT WITH IT WITHOUT

TAKING PROTON PUMP INHIBITORS.

VEENS CONVENIENT MEANS IT AN AT

THIS -- CONVENIENT MEANS IT

ANTICIPATES MY NEEDS AND

PLEASURABLE IS SELF-EVIDENT.

THIS IS A FUNCTIONAL SYSTEM

THIS, THIS IS A FUNCTIONAL

SYSTEM.

THIS MIGHT BE A USABLE SYSTEM.

THAT MIGHT BE A PLEASURABLE

SYSTEM SO WE LOOK AT ANY KIND OF

SYSTEM, EHR OR ANY NEW KIND OF

TECHNOLOGY AND WE CAN SAY THAT

FOLKS WILL ACTUALLY EXPRESS

EMOTIONS ABOUT SO I DID THIS IN

A SIMILAR SESSION WITH MEDICAL

STUDENTS LAST WEEK AND SAID

RAISE YOUR HAND IF YOU HAVE A

SMART PHONE.

THEY ALL WENT UP.

SO WHAT EMOTION DO YOU HAVE?

ALL THE ONE WHOSE PURCHASED THIS

COMPANY'S PRODUCTS THAT I LOVE

IT.

WHEN DO WE GET THERE WITH

ELECTRONIC HEALTH RECORDS?

MORE IMPORTANTLY THAN LOVING

SOMETHING IS?kP THAT THERE ARE SOME

THINGS THAT WE HAVE ARE SAFE.

SO THAT'S ANOTHER FOCUS THAT WE

AT ONC HAVE AND SO MANY OF YOU

KNOW WE COMMISSIONED A REPORT

FROM THE IOM WHICH WAS PUBLISHED

LAST NOVEMBER THAT LOOKED AT THE

SAFETY OF HEALTH IT SYSTEMS.

WE ARE ALSO THINKING USABILITY

AND SAFETY ARE TIGHTLY ALIGNED.

IF THE SYSTEM ISN'T USABLE IT'S

EASY TO MAKE MISTAKES AND IF

IT'S EASY TO MAKE MISTAKES WE

HAVE HEALTH IT CAUSED PATIENT

SAFETY PROBLEMSCH HEALTH IT CAN

BE THE SOLUTION TO PATIENT

SAFETY PROBLEMS SO WE NEED TO

BETTER UNDERSTAND SAFETY.

THIS IS AN EXAMPLE OF A FAIRLY

UNSAFE ENVIRONMENT IF BICYCLES

ARE RIDING ACROSS ROADS VERY

QUICKLY THEY MIGHT GET SMOOSHED

BY CARS.

THESE THINGS SLOW DOWN THE BIKE

RIDERS, THE CATTLE CRATE SYSTEMS

AND IN MY AREA IN UPSTATE NEW

YORK THEY HAVE PUT THESE IN MANY

OF OUR BIKE PATHS.

THE REASON FOR THAT IS THAT

PEOPLE WERE GETTING SMUSHED BY

CARS WHEN THEY WENT TOO FAST.

THE SAME CAN HAPPEN IN HEALTH IT

SYSTEMS AND THE REASON I PUT

THIS UP IS OFTEN WE'LL HEAR FROM

END USERS IS TESTIMONY IS TOO

SLOW.

THE FASTER IS SYSTEM GOES IN

SOME CASE IT IS MORE ERRORS

OCCURS.

RAISE YOUR HAND IF YOU HAVE USED

AUTOSUGGEST IN OUTLOOK AND SENT

THE WRONG MESSAGE TO SOMEBODY.

SENDING THE WRONG MESSAGE MAY

HAVE CAREER-ENDING IMPLICATIONS.

BUT SENDING THE WRONG

PRESCRIPTION TO SOMEONE OR

LOOKING UP THE WRONG PATIENT MAY

KILL SOMEBODY.

SO WE CAN END OUR CAREERS WITH

OUTLOOK BUT IF THE SYSTEM CAUSES

IT TOOB TOO FAST, IT CAN ALSO BE

UNSAFE.

DECISION SUPPORT IS A REALLY

IMPORTANT DOMAIN TO THINK ABOUT

BECAUSE ALL THOSE AUTOSUGGEST

SYSTEMS USE ALGORITHMS THAT ARE

DECISION SUPPORT.

THIS IS A SYSTEM IS HELPING YOU

DO SOMETHING FASTER AND MORE

EASILY AND IF YOU CAN DO IT TOO

FAST THAT'S A PROBLEM.

WE PARTICIPATED IN A SESSION

LAST SUMMER ATNIST WHERE THERE

WAS AN EXAMPLE ABOUT WHICH SIDE,

WAS AN OR FORM THAT A DOCK WAS

SUPPOSED TO FILL OUT, WHICH FOOT

ARE YOU GOING TO AMPUTATE.

THE QUESTION IS DO I DESIGN THIS

USING A RADIO BUTTON WHERE

THERE'S A .4% CHANCE THEY CLICK

THE WRONG ONE IN A HURRY OR MAKE

A FREE TEXT FIELD AND FORCE THEM

TO TYPE IN L E F T.

THE LATTER IS THE ANSWER BUT IN

THIS ATTEMPT TO DO THINGS

QUICKLY MANY VENDORS ARE NOW

MAKING THE FORMER SELECTION

BECAUSE CUSTOMERS ARE SAYING

HELP US DO IT FAST, HELP US DO

IT FAST.

THERE'S THE TRUCK TO SMOOSH

SOMEBODY.

THERE'S A PROBLEM WE HAVE BEEN

SEEING, I LOVE THE DISCUSSION

EARLIER ABOUT VALUE SETS.

IN THE QUALITY MEASURE DOMAIN

THIS IS MY RENDITION QUALITY

CHASM 2.0.

QUALITY MEASURES SAY THINGS

ABOUT WHAT OUGHT TO HAPPEN AND

THERE'S THE EXCLUSION AND

EXCEPTION CONCERNS.

SO THOSE WHO AREN'T ENTRENCHED

IN QUALITY MEASURES, A QUICK

QUALITY MEASURE 101.

QUALITY MEASURE ASKS DID THIS

HAPPEN, WHEN I'M JUDGED ON

WHETHER I HAVE DONE S>m

NOT, SO DID I DO THE FOOT EXAM

FOR MY DIABETICS?

TURNS OUT IF I'M A PHYSICIAN AND

I'M BEING JUDGED OR PAID BASED

ON MY PERFORMANCE, A PERCENTAGE

OF PATIENTS WON'T HAVE GOTTEN A

DIABETIC FOOT EXAM DONE,

ESPECIALLY ONES WHO DON'T HAVE

FEET.

SO ABSENCE OF FEET WOULD BE AN

EXCEPTION TO THIS.

BUT YET I NEED TO TELL MY EHR

THAT THE PATIENT HAS NO FEET.

SO THERE'S THIS BALANCE BETWEEN

THE QUALITY MEASURE WORLD THAT

SAYS OH, WE NEED ALL THESE

EXCEPTIONS TO BE EXPRESSED IN

THE QUALITY MEASURE SO THAT WE

CAN HELP PROVIDERS NOT GET

DINGED WHERE THEY SHOULDN'T GET

DINGED.

SO IN THAT CASE THE QUALITY

MEASURES ARE COMPLEX AND HAVE

LOTS OF EXPECTATIONS OF DATA

THAT MAY OR MAY NOT BE IN THE

ELECTRONIC HEALTH RECORD.

SO THAT'S WHY I HAVE THE QUALITY

MEASURE EXPECTATION ON THE LEFT

OF THIS CHASM AND THE CAPABILITY

OF THE EHR, THE FACT THE PATIENT

HAS NO FEET MAY OR MAY NOT BE IN

THE EHR, THERE ARE MANY

EXCEPTIONS MEDICATION, PATIENT

PREFERENCE T FACT THAT THE

PHARMACY WAS OUT OF THE

MEDICATION AT THE TIME AND

THAT'S WHY I DIDN'T PRESCRIBE

IT.

THERE ARE LOTS OF REASONS WHY A

PATIENT WON'T GET SOMETHING THAT

ARE EXCEPTIONS.

THE EHR MAY OR MAY NOT HAVE THAT

INFORMATION.

SO AS YOU THINK ABOUT THE NUMBER

OF ELEMENTS THAT COULD GO INTO A

CLINICAL QUALITY MEASURE AND THE

FOLKS THAT MAKE THESE MEASURES,

THERE ARE LOTS AND LOTS OF

THINGS THAT THEY COULD SAY IN AN

EHR OR I'M SORRY, IN A QUALITY

MEASURE SO THESE ARE I THINK OF

THESE AS THE MANY THOUSANDS OF

CRAYONS.

AND YET THE EHR MAY ONLY HAVE A

SUBSET OF THOSE THINGS AVAILABLE

TO US SO THERE MAYBE A

VOCABULARY THAT EXPRESSES A

SUBSET OF THINGS THAT THE

CLINICAL QUALITY MEASURES WOULD

EXPRESS SO THAT'S THE BIG GAP.

YET WE'RE TRYING TO SOLVE THE

PROBLEM WITH THE TOOL THAT WE

HAVE.

SO THERE'S THE EHR CAPABILITY SO

MY ATTEMPT TO THINK ABOUT HOW WE

MIGHT FILL THIS GAP, THERE ON

THE LEFT IS QUALITY MEASURE

EXPECTATIONS AND CAPABILITIES

THAT WE HAVE STANDARDS, WE HAVE

STANDARD TERMINOLOGIES.

WE HAVE STANDARD VALUE SETS.

WE HAVE STANDARD METHODS OF

CAPTURING AND EXPRESSING

INFORMATION.

WE MIGHT ACTUALLY HAVE TO MODIFY

THE EXPECTATIONS THAT WE HAVE OF

THE INFORMATION SO PERHAPS THE

EXCEPTIONS FOR QUALITY MEASURES

SHOULDN'T BE SO EXPANSIVE AND

STATEMENT WE MAY NEED TO ENHANCE

THE EHR CAPABILITIES.

SO WE ONC CAN ENHANCE THE EHR

CAPABILITIES SO I SEE MARC IN

FRONT SO I'LL PICK ON MINIMUM

HIM, THE CMAO OF SIEMENS.

BUT I CAN SAY TO MARC YOU MUST

CAPTURE THE FACT THE PATIENT HAD

A FOOT AMPUTATIONS, ANY TIME

THERE'S A PATIENT WHO IS

DIABETIC.

SO WHETHER OR NOT THIS PATIENT

HAS FEET NEEDS TO BE CAPTURED.

SO MARC COULD THINK ABOUT WAYS

TO IMPLEMENT THAT IN HIS SYSTEM.

EITHER USING NATURAL LANGUAGE

PROCESSING WHERE THEY WOULD GO

THROUGH NARRATIVE NOTES OR

DICTATED NOTES OR SOME OTHER

EVIDENCE OF THAT, MAYBE A CPT

CODE 17 YEARS AGO FROM THAT AM

AMPUTATION.

BUT I MAY REQUIRE THAT OF MARC

EXPLICITLY.

I COULD SAY FOR CERTIFICATION HE

ABSOLUTELY POSITIVELY HAS TO

IDENTIFY THE EVENT.

I COULD NAME A SNOW MED C THET

CONCEPT AND YET AT THE SAME TIME

THERE MAYBE 3,000 OF THOSE

THINGS AND I CAN'T EXPECT MARC

TO ADD 3,000 BY 2014 EHR

CERTIFICATION SOY MIGHT PICK

1400 OF THEM.

AND YET I WILL TALK TO FOLKS WHO

MAKE THE MEASURES AND SAY YOU

FOLKS HAVE 3,000 OF THESE

THINGS, PICK THE BEST 1400 AND

TELL ME WHAT THEY ARE SO I CAN

IMPOSE THEM ON MARC.

THIS IS THE ROCK AND HARD PLACE

ONC FINDS ITSELF IN TODAY AND

WORK FLOW AND USABILITY NEEDS TO

BE PART OF HOW WE THINK ABOUT

THIS.

NECESSARY ARE NOT JUST FORMS

INSTANTIATING EHR.

IN FACT, IF WE ENCOURAGE OR

ALLOW VENDORS TO INSTANTIATE

HARD CODED FORM INTO THEIR

SYSTEMS, WE WILL DESTROY THE

USABILITY IN THE WORK FLOW AND

THIS IS HARD WIRING SO ANY OF

YOU WHO HAVE BEEN PAYING

ATTENTION TO THE HIT POLICY

COMMITTEE, WHICH IS A FEDERAL

ADVISORY COMMITTEE THAT TELLS US

WHAT TO DO, YOU HAVE HEARD THEM

COMPLAIN ABOUT HARD WIRING SO

THIS IS A CASE IF I TOLL MARC

THIS IS A SNOW MED CT CONCEPT,

MARC WOULD ACTUALLY I THINK I

HAVE A THE PICTURE MAKE ONE OF

THESE AND HIS PHYSICIANS WILL

JUMP OFF A BRIDGE.

BECAUSE THIS IS TERRIBLE

USABILITY.

YET THIS IS WHAT HAPPENED DURING

OUR -- IN MANY CASES STAGE 1 OF

THE MEANINGFUL USE PROGRAM

STARTED IN 2011.

SO CURRENTLY TODAY WHAT WE HAVE,

AND PART OF WHY WE HAVE SO MUCH

HARD CODING IS BECAUSE WE HAVE

CDS AND CTUMs IMBEDDED INSIDE

OF EHRs.

SO MARC HAS HIGHER PROGRAM, I'M

REALLY PICKING ON MARC, HE'S IN

THE FRONT ROW.

HE'S USED TO IT TOO.

HE'S A BIG GUY.

SO THESE PIECES ARE PART OF THE

SAME SYSTEM AND SO YOU HAVE TO

HIRE PROGRAMMERS THAT THE TIME

THAT IT TAKES TO IT RATE AND

IMPLEMENT THESE THINGS IS QUITE

LONG BECAUSE YOU HAVE TO HIRE

HUMANS TO INTERPRET WHAT THESE

CONCEPTS ARE AND THEN PUT THEM

INTO YOUR SYSTEMS.

THIS IS WHAT WE HEAR FROM

PHYSICIAN, DOCTOR NOT A DATA

ENTRY CLERK AND WE HAVE TO HEAR

MORE ABOUT WHERE THE PUCK WILL

BE AND NOT WHERE IT HAS BEEN.

SO WE THINK ABOUT THE LEVERS

THAT ARE AVAILABLE TO US IN THE

FEDERAL GOVERNMENT, SPECIFICALLY

AT ONC.

STANDARDS ARE A LEVER, ONE OF

THE TWO THINGS ONC DEFINES WHEN

WE CREATE THE STANDARD AND

CERTIFICATION CRITERIA.

SO WE LOOK TO STANDARDS FOR SAY

HOW INFORMATION IS SHARED.

HOW INFORMATION IS INCORPORATED

INTO EHR, IS THERE A STANDARD

HOW QAL CLAWLT MEASURE OR

DECISION SUPPORT INTERVENTION

AND CDS FOLKS IN THE ROOM KNOW

IN OUR PROPOSED RULE WE REFER TO

THESE AS INTERVENTIONS NOT

RULES.

THE REASON IS WE WANT TO THINK

HOW IT EXPOSED AND ENTERACTS

WITH THE USER NOT HOW IT'S

INSTANTIATED IN THE SYSTEM

ELECTRONIC FORM SO IT MIGHT BE A

ROLE BUT IT MIGHT BE SOMETHING

ELSE, WE DON'T WANT TO MAKE

ASSUMPTIONS HOW THAT'S ENCODED.

SO STANDARD WAYS TO DO THINGS TO

ENHANCE THE LIKELIHOOD THEY

WOULD ACCELERATE.

THEN WE HAVE REGULATIONS.

POLICIES.

THESE ARE ALSO STANDARDS,

STANDARDS OF BEHAVIOR.

SO WE HAVE TECHNICAL STANDARDS

AN STANDARD OF BEHAVIOR.

THESE ARE THE TOOLS THAT WE THE

GOVERNMENT HAVE TO ACTUALLY

ACCELERATE PROCESS BECAUSE WHEN

YOU STANDARDIZE THINGS, THEY CAN

GO FASTER.

THAT'S WHAT I TRIED TO SAY HERE.

WHEN I TAKE THE TRAIN FROM BWI

TO DC IT'S MUCH FASTER THAN THE

THE BUS.

AND THE REASON IS THERE'S A

STANDARD.

STANDARD THINGS YOU CAN PLUG IN

AND UNPLUG.

SO THIS IS A STANDARD FOR HOW WE

PUT THINGS IN SOCKETS IN THE

UNITED STATES.

SO IF WE WERE TO STANDARDIZE

BOTH CLINICAL DECISION SUPPORT

AND CLINICAL QUALITY MEASURES

AND HAVE THEM EXPRESSIBLE IN

MACHINE INDEPENDENT END POINT

INDEPENDENT FORMATS SO MARC AND

COLLEAGUES FROM NEXTGEN AND EPIC

COULD CONSUME THESE PIECES OF

CLINICAL CONTENT AUTONOMOUSLY

FROM THE COURT SYSTEM, THEY

COULD IT RATE THEM MUCH MORE

QUICKLY AND IT WOULDN'T BE SO

HARD CODED, HARD WIRED.

SO DO WE HAVE AN OPPORTUNITY FOR

THIS ADS YOUR DATA -- AS YOUR

DATA ENTRY MECHANISM INSTEAD OF

THE FORM OR THE TEMPLATE AND

MAYBE THAT'S DICTATION, MAYBE

IT'S SOMETHING ELSE.

MIND READING OR INTERACTION WITH

THE PATIENT.

OR SOMETHING ELSE.

IS THIS ANOTHER WAY FOR US TO

UNDERSTAND HOW PATIENT CARE IS

RENDERED?

INTERACTED WITH A COMPANY A

COUPLE OF WEEKS AGO THAT'S

CREATING A SYSTEM FOR PHYSICIANS

AN PATIENTS TO INTERACT WITH

EACH OTHER ELECTRONICALLY.

A SECURE MESSAGING SYSTEM,

THEY'RE NOT JUST DOING THE

MESSAGES.

THEY'RE DOING ANALYSIS, DOING

MACHINE LEARNING AND THEY'RE

ACTUALLY LOOKING FOR OUTCOMES.

SO THE SYSTEM SENDS A MESSAGE TO

THE PATIENT 24 HOURS AFTER THE

LAST INTERACTION.

THE PATIENT SAYS HEY DOCK MY

BACK HURTS AND THE DOC GIVES AD

VOICE AND 24 HOURS LATER THE

SYSTEM FOLLOWS UP WITH THE

PATIENT AND SAYS ARE YOU BETTER?

THE PATIENT SAYS YES I'M BETTER.

WHAT THE DOC GETS IS A BETTER

UNDERSTANDING OF OUTCOMES.

SO THEY'RE READING THE TEXT OF

THESE THINGS AND THAT TEXT CARRY

ASSET OF DIAGNOSE KNOW SAYS, A

SET OF ACTIONS, A SET OF

INTERVENTIONS BUT THEN WE CAN

LEARN FROM AS WE IT RATE

FORWARD.

SO AS WE THINK WE ONC THINK OF

THESE THINGS WE'RE THINKING VERY

HARD ABOUT HOW WE CAN WRITE OUR

REGULATIONS WITHOUT MAKING

PRESUMPTIONS ABOUT HOW THINGS

ARE DONE.

WE WANT TO ACCOMMODATE NEW

INNOVATIVE WAYS OF DOING THINGS

THAT MAY INCLUDE DECOUPLED CDS,

THAT MAY INCLUDE NATURAL

LANGUAGE PROCESSING AN

INNOVATIVE WAYS OF INTERACTING

WITH PATIENTS RATHER THAN JUST

PROVIDERS.

HERE IS CDS 1.0, I DON'T NEED TO

READ IT.

IT'S ONE OF MANY.

WE DON'T THINK IN THOSE TERMS.

THIS IS MAYBE A 2.0 JUST IN TIME

INFORMATION.

WHERE I NEED IT WHEN I NEED IT

WHAT I NEED AND THE SYSTEM HAS

TO DO THAT PROPERLY.

SO WITHOUT REALLY COMPLEX THINK

ING WE CAN'T DEVELOP IT AT

THE RIGHT TIME AND THE RIGHT

PLACE.

THIS IS MY MASCOT FOR JUNIOR

(INAUDIBLE) AND I'M FINISHED.

SO THANKS FOR YOUR ATTENTION AND

LOOK FORWARD TO TALKING TO YOU.

[APPLAUSE]

>> WE'RE RUNNING OVER SO LET'S

SAVE QUESTIONS FOR THE END.

OUR NEXT SPEAKER IS DR. GREGORY

STEINBERG OF AETNA.

>> I'LL GREG STEINBERG, HEAD OF

CLINICAL AIF YEAR AGO AT AETNA,

I'M GOING TO SPEAK TO YOU FROM

THE PERSPECTIVE OF THE PAYER AND

DIRECTLY FROM THE EMPLOYERS WHO

ARGUABLY ARE INVOLVED IN THE

MUNDANE BUT NOT TRIVIAL TASK OF

PAYING FOR SOME OF THESE

PRODUCTS AND SERVICES WE'RE

TALKING ABOUT.

I'LL SAY FROM THE OUTSET THAT I

THINK A LOT CAN AND HAS BEEN

DONE ALREADY WITH EXISTING DATA

SOURCES, ADMINISTRATIVE DATA

SOURCES AND HIGH QUALITY CDS

SYSTEMS AROUND TODAY.

CLEARLY THERE'S POTENTIAL TO DO

MORE AND BETTER WITH NATURAL

LANGUAGE PROCESSING AND I'LL TRY

TO COVER SOME OF THAT.

I'LL ALSO SPEAK BRIEFLY ABOUT

SOME HIGH LEVEL STRATEGIC

THOUGHTS FROM THE AETNA

PERSPECTIVE, PARTICULARLY

RELATIVE TO THE NEW PARADIGM OF

THE MECHANICAL CARE

ORGANIZATIONS.

SO WHEN I TALKED ABOUT DATA AND

CURRENT ADMINISTRATIVE DATA WHAT

THAT MEANS AT LEAST TO US

OBVIOUSLY DIAGNOSTIC AND

PROCEDURE CLAIMS BUT DATA FROM

THE PBM LAB DATA SO WE CAN GET

AND HAVE FOR MANY YEARS NOW, NOT

JUST THE FACT OF LAB TEST BUT WE

KNOW WHAT THE LAB RESULT IS.

WE HAVE ALSO HAD PATIENT PATIENT

SELF-REPORTED DATA SO WE HAVE A

PERSONAL HEALTH RECORD, WHERE

DATA IS INPUTTED BY PATIENTS

ELECTRONICALLY AND COMES INTO

OUR DATABASE.

IN ADDITION WHEN WE HAVE CARE

MANAGEMENT NURSES SPEAKING TO

PATIENTS THEY'RE ENTERING THAT

DATA AND THAT DATA IS ALSO PART

OF THE LONGITUDINAL HEALTH

RECORD.

SO SOMEBODY TALKED ABOUT LACK OF

AVAILABILITY OF LONGITUDINAL

HEALTH RECORDS AND I WOULD POSIT

THESE ARE ACTUALLY AVAILABLE AND

HAVE BEEN AVAILABLE IN THE PAIR

WORLD FOR MANY YEARS.

MORE RECENTLY, WE'LL TALK MORE

ABOUT THIS LATER BUT, NOT SURE

THAT'S WORKING BUT THE HEALTH

INFORMATION EXCHANGE

CAPABILITIES WHICH ALLOW US

ELECTRONIC ACCESS TO EMRs IS

GOING TO BE AN INTEGRAL PART AS

WE GO[� FORWARD OF THE EXPANDED

DATABASE.

WHAT WE HAVE IN THE MIDDLE IS

THE CLINICAL DECISION SUPPORT

ENGINE THAT TAKES ALL THAT DATA

AND DEPENDING HOW THE ALGORITHMS

IN THE CDS SYSTEM ARE CONFIGURED

DOES A NUMBER OF THINGS, WILL

EITHER DEAL ON THE LEFT WITH

PATIENT-SPECIFIC SO-CALLED

PRECISION ALERTS, GAPS IN CARE

THAT WE HAVE BEEN TALKING ABOUT,

PEOPLE MENTION PATIENT

ENGAGEMENT SO VERSIONS OF OUR

GAPS IN CARE ARE FED BACK

ACTUALLY IN REAL TIME TO THE

PERSONAL HEALTH RECORDS AND TO

THE PATIENTS THEREFORE.

DIFFERENT VERSIONS OF CLINICAL

DECISION SUPPORT RULES CAN

INFORM REAL TIME ANALYTICS.

WE TALKED ABOUT QUALITY MEASURE,

REGISTRIES, ET CETERA.

MORE IMPORTANTLY OR MORE

RECENTLY I SHOULD SAY, WORK FLOW

RULES HAVE BEEN DEVISED TO HELP

IN TERMS OF CARE COORDINATION

AGAIN RELATIVE TO ACCOUNTABLE

CARE ORGANIZATIONS, PATIENT

CENTERED MEDICAL HOMES AND THE

LIKE.

SO A LITTLE CLICK DOWN IN TERMS

OF THE CLINICAL DECISION SUPPORT

SYSTEM.

WE GET ALL THE DATA THAT I

TALKED ABOUT, IT CREATES A

ROBUST LONGITUDINAL

PATIENT-CENTRIC ELECTRONIC

MEDICAL RECORD WHICH IS THEN

APPLIED AGAINST A DIGITIZED

VERSION OF THE EVIDENCE BASE

MEDICAL LITERATURE.

AND WHAT COMES OUT THE OTHER

END, IF YOU WILL, IS GAP

ANALYSIS ON A PATIENT SPECIFIC

LEVEL.

THAT LOOKS AT THE DIFFERENCE

THAT A PATIENT RECEIVING CARE IS

RECEIVING REFLECTED IN THE DATA

AND THE CARE THEY SHOULD BE

RECEIVING REFLECTED IN THE

LITERATURE THAT NEGATIVE

INFORMATION IS ENCAPSULATED INTO

VARIOUS FORMATS THAT WILL BE

TRANSMITTED THAT ARE TRANSMITSED

IN VARIOUS WAYS TO DOCTORS AND

PATIENTS.

SO CLINICAL DECISION SUPPORT

LIKE A LOT OF THINGS IN THE

WORLD VARY A LOT OF DECISION

SUPPORT LOOKS LIKE THIS.

THIS IS OBVIOUSLY RELATIVE TO

DIABETES AND FAIR HI BASIC IN

FAIRLY UNINTERESTING.

MORE IMPORTANTLY NOT REALLY A

REFLECTION OF THE TRUE VARIETY

AND COMPLEXITY OF THE SITUATION.

FOR US CLINICAL DECISION SUPPORT

IS THIS.

NOT JUST THERE'S MORE STUFF.

WHICH THERE IS BUT THIS IS A

REPRESENTATION OF HOW THINGS

INTERACT WITH EACH OTHER

PHYSIOLOGICALLY AND HOW

PHYSICIANS ACTUALLY THINK.

IT GOES TO THE POINT THAT WAS

MADE BY A FEW PEOPLE THAT

CLINICAL DECISION SUPPORT

SYSTEMS ARE NOT NECESSARILY ABLE

TO TAKE INTO ACCOUNT

COMORBIDITIES, MULTIPLE

MEDICATIONS AND SO FORTH.

THAT'S NOT NECESSARILY TRUE, AT

LEAST WITH RESPECT TO THE DO NO

HARM CAPABILITY.

SO YOU CAN CLEARLY HAVE THESE

ARE SOPHISTICATED RULES WITH

SOPHISTICATED RULE AUTHORING

CAPABILITIES BUT THEY EXIST.

SO YOU CAN CLEARLY HAVE ON THE

TOP YOU CAN CLEARLY IDENTIFY

DIABETIC WITH HEMOGLOBIN A 1C

NOT ON ANY MEDICATION, THE

AMERICAN DIABETES ASSOCIATION

POSITS THAT PERSON SHOULD BE ON

METAPHORMAN BUT THERE ARE A HOST

OF CONDITIONS METAPHORMAN MAY

NOT BE A GOOD IDEA, YOU CAN

WRITE RULES THAT WILL

PROACTIVELY LOOK FOR THOSE

CONDITIONS AND NOT SEND THE

OUTPUT TO A PHYSICIAN OR PATIENT

IF ANY OF THOSE CONDITIONS ARE

PRESENT BY LAB DATA, CURVES.

THERE'S A NUMBER THOSE STATIN,

INHIBITORS, YOU GET THE IDEA.

GIVE YOU A SENSE OF THE FACT

THAT WE HAVE BEEN DOING THIS FOR

REAL ON LARGE SCALE, THESE

STATISTICS ARE RELATIVE TO 2011.

ON THE LEFT SOME OF THE SOURCES

FOR SPACE STANDARDS THAT WE USE

FAIRLY LARGE GROUP OF FULL-TIME

DOCS, PHARMACISTS, ARE INVOLVED

IN BUILDING AND MAINTAINING

THESE RULES, THE TYPE OF RULES,

THE NUMBERS OF RULES IN THE

MIDDLE THERE AND IT GIVES YOU A

SENSE OF SOME OF THE ACTIVITY

AND THIS IS JUST IN ONE YEAR THE

RULES THAT WERE MESSAGES

GENERATED TO PROVIDERS AND/OR

PATIENTS.

ONE POINT WAS MADE ABOUT YOU SEE

MANY MORE PATIENT ALERTS, THAN

DOCTOR ALERTS, THAT GOES TO THE

FACT THAT AS SOMEBODY SAID

PROBABLY NOT A GOOD IDEA TO SEND

MAMMOGRAM ALERTS TO DOCS SO WE

DON'T BUT WE SEND THEM TO THE

PATIENT.

THE LITTLE THING AT THE BOTTOM

MAKE IT IS THE POINT WE HAVE THE

OPPORTUNITY TO DO A CHART

ANALYSIS WITH LARGE TEACHING

HOSPITAL IN NEW YORK CITY WHERE

WE HAD ACCESS TO MEDICAL RECORDS

AND WE WERE ABLE TO COMPARE BOTH

DIAGNOSTIC VALIDITY OF THE RULE,

DIAGNOSES THAT WE WERE IMPUTING

AND THE CLINICAL CONTENT OF THE

RULE AND HAD GREATER THAN 90%

CONCORDANCE.

SO WHAT HAVE WE LEARNED FROM

WORKING WITH THE SYSTEMS OVER

THE YEARS AND THE AREAS WHERE

NATURAL LANGUAGE PROCESSING MAY

HELP US?

THE REAL WORLD DATA WE DEAL

WITH, THOUGH IT'S USEFUL, VERY

USEFUL, IT HAS ISSUES.

SO DIAGNOSTIC CLAIMS CLEARLY ARE

OFTEN INACCURATE DUE TO A HOST

OF ERRORS.

IF I WANT TO RULE OUT DIABETES

IT'S THE SAME CODE AS IF THE

PERSON HAS DIABETES.

YOU CAN CONSTRUCT RULE LOGIC

INCLUSIONNARY CRITERIA AND

EXCLUSIONARY CRITERIA TO

MITIGATE THAT BUT IT'S AN ISSUE

AND CLEARLY CLAIMS LAG.

SENSITIVITY SPECIFICITY PROBLEM

SO THIS WAS MENTIONED BY A

NUMBER OF SPEAKERS.

WHEN YOU'RE DEALING WITH

PATIENT-SPECIFIC ALERTS GOING TO

DOCS YOU BETTER WITH RIGHT ALL

THE TIME.

DOCS HATE WRONG ALERT.

THEY HAVE NO PROBLEM TELLING US

THAT YOU HAVE LIED TO THEM.

AND POTENTIAL FOR ALERT FATIGUE.

YOU CAN DO THAT INCREASING

SPECIFICITY BUT T AT THE EXPENSE

OF SENSITIVITY.

ON THE OTHER HAND THIS WAS

MENTIONED EARLIER, WHEN DEALING

WITH POPULATION BASED QUALITY

MEASURES YOU PROBABLY HAVE TO

DIAL IT THE OTHER WAY

PARTICULARLY NUMERATOR BECAUSE

MANY PROVIDERS ARE GOING TO BE

MEASURED AND POTENTIALLY PAID

BASED ON HOW THE RULES ARE

STRUCTURED.

LASTLY AT THE BOTTOM OF THE

SLIDE, ABSENCE OF EVIDENCE DOES

NOT EQUAL EVIDENCE OF ABSENCE,

IS THE REAL PROBLEM.

SO HOW DO YOU GET TO ERRORS OF

OMISSION, THINGS THAT ARE NOT

THERE THAT SHOULD BE THERE.

IN OUR CURRENT PARADIGM WE USE

ELIGIBILITY DATA ESSENTIALLY AND

AS A SURROGATE FOR IF SOMETHING

IS NOT THERE, IF IT HAD BEEN

THERE WE WOULD HAVE SEEN IT.

THAT IS TRUE A LOT OF TIME BUT

NOT ALL THE TIME.

AND CLEARLY CAN LEAD TO FALSE

POSITIVES AND CLEARLY AN AREA

NATURAL LANGUAGE PROCESSING CAN

SIGNIFICANTLY HELP.

SO IS THERE DATA THAT THE CDS

SYSTEMS THAT DO NOT USE NATURAL

LANGUAGE PROCESSING, ONLY OTHER

SOURCES OF DATA I TALKED ABOUT

REALLY WORKED.

SO THESE ARE TWO PUBLICATIONS ON

ONE STUDY.

THIS WAS ONE OF THE ONLY

RANDOMIZED PROSPECTIVE

CONTROLLED TRIALS OF CDS THAT I

THINK IS IN THE LITERATURE.

THEY TOOK 40,000 MEMBERS IN A

HEALTH PLAN IN CLEVELAND AND

BASICALLY RANDOMIZED THEM, HALF

GOT THE CLINICAL DECISION

SUPPORT SYSTEM, HALF DIDN'T, IT

WAS WELL MATCHED, A ONE YEAR

PROSPECTIVE STUDY AND THE

PRE-DEFINED POINTS IN ADDITION

TO THE THE NUMBER OF ERRORS

DEFINED, HOSPITALIZATIONS AND

PAID CLAIMS.

REAL MONEY.

THESE WERE STATISTICALLY

SIGNIFICANT CHANGES, HOSPITAL

REDUCTION -- PAID CLAIMS REDUCED

BY $8.

PANEL ON THE RIGHT TOOK THE SAME

DATA BUT HAD AN EXTRA YEAR OF

DATA AFTERWARDS, AFTER THE STUDY

WAS OVER BECAUSE WHAT HAPPENED

IS THAT THE HEALTH PLAN

INVOLVED, FIGURED IT UNETHICAL

TO NOT HAVE THE CONTROL GROUP

HAVE CDS SO THE CONTROL GROUP

RECEIVED CLINICAL DECISION

SUPPORT IN THE YEAR SUBSEQUENT

TO THE STUDY.

AND THE AUTHORS OF THAT STUDY

PUBLISHED IN THE JOURNAL OF

HEALTH ECONOMICS LOOKED AT THAT

SO THEY LOOK AT CHARGES UNPAID

CLAIMS AN CONFIRMED CHARGES WERE

SIGNIFICANTLY REDUCED.

HOSPITALIZATIONIZATIONS WERE

REDUCED.

INTERESTINGLY THE

HOSPITALIZATIONS WERE ALL TO DO

WITH THE AREAS AND THE CAUSALITY

THING WAS INTERESTING.

WHAT HAPPENS IS YOU HAD THE TWO

GROUPS STARTED TOGETHER IN TERMS

OF CHARGES IN THE FIRST YEAR,

THEY DIVERGED DURING THE STUDY

YEAR AND THEN IN THE YEAR AFTER

WHEN BOTH GROUPS NOW GOT THE CDS

SYSTEM THE CHARGE DIFFERENTIAL

DISAPPEARED.

WHICH LED TO THE CONCLUSION THAT

THERE WAS PROBABLY A CAUSAL

EFFECT.

I SAID WE TALKED A LITTLE BIT

ABOUT ACCOUNTABLE CARE

ORGANIZATION.

SO WHAT THIS REALLY WHAT THE

SLIDE IS REALLY TALKING ABOUT,

WE'RE UNDERGOING A FAIRLY

SIGNIFICANT PARADIGM SHIFT RIGHT

NOW WHERE WE'RE MOVING FROM

PATIENT-SPECIFIC REACTIVE CARE,

TO POPULATION MANAGEMENT AND

PROACTIVE CARE.

MOVING FROM PAYING PEOPLE MORE

FOR DOING MORE TO PAYING PEOPLE

MORE FOR DOING BETTER.

THIS IS A FUNDAMENTAL TECH TONIC

SHIFT WE'RE INVOLVED WITH.

THE WAY WE TRIED TO START TO

LOOK AT THIS AND COBBLE TOGETHER

A SET OF SOLUTIONS THAT MIGHT

HELP IS IN ADDITION TO THE THE

DECISION SUPPORT STUFF THAT IS

THE BOTTOM RIGHT BUBBLE THERE,

WHERE YOU HAVE GAPS IN CARE AND

POPULATION BASED DECISION

SUPPORT TOOLS AND WORK FLOW

TOOLS.

WE HAVE THE HELP INFORMATION

EXCHANGE CAPABILITIES ADDED TO

THAT.

YOU HAVE ON THE TOP THERE A

LARGE HEALTH SYSTEM EXCHANGE,

HAS THE LARGEST FOOTPRINT WHICH

ALSO HAS THE INEX CAPABILITY

WHICH IS A CLOUD-BASED

APPLICATION STORE LIKE THE APPLE

STORE WHERE A HOST OF

APPLICATIONS CAN BE DEVELOPED

QUICKLY AND SIT ON THIS AND

DOWNLOADED BEHIND AN INDIVIDUAL

FIREWALL OFFICE.

LAYERED ON TOP OF THAT IS TOOLS

THAT I TRIAGE AS ONE OF THE

LEADING CONSUMER MOBILE APPS

THAT PROVIDES NAVIGATION SYMPTOM

CLINICAL DECISION SUPPORT,

ITSELF AND WRAP ALL THAT AROUND

WITH HEALTH PLAN SERVICES

BECAUSE THESE ACCOUNTABLE CARE

ORGANIZATIONS ARE GOING TO HAVE

TO START DEALING FAIRLY

SERIOUSLY WITH ALL THAT RISK

THAT IS SOMETHING NEW AN FOREIGN

TO THEM.

SO DOUBLE CLICKING MORE, I

APOLOGIZE, KIND OF BUSY BUT WHAT

THIS IS TRYING TO SAY IS, ONE

MORE LEVEL OF DETAIL.

AT THE BOTTOM YOU HAVE THE

DOCTOR, PROVIDED THE PROVIDER

DATA.

ON THE LEFT YOU HAVE THE

HOSPITAL DATA.

ON THE RIGHT IT'S CONNECTED

THROUGH THE MEDICITY HEALTH

INFORMATION EXCHANGE GRID

STRUCTURE ON THE BOTTOM.

IT GETS FED INTO THE ANALYTIC

ENGINE WHICH DOES A NUMBER OF

THINGS, IT SENDS OUT THE CARE

ALERTS AND DECISION SUPPORT

STRAIGHT UP.

IT TALKS TO THE PATIENTS IT

PROVIDES WORK FLOW TOOLS FOR THE

DOCTORS TO AFTER THE PATIENTS

PROACTIVELY.

AND A WHOLE HOST OF REPORTING

CAPABILITIES SOPHISTICATED AND

GOING TO BE NEEDED IN ORDER TO

SURVIVE AND THRIVE IN THE ARCCL

WORLD.

ONE WAY NATURAL LANGUAGE

PROCESSING HELPS IS BEING ABLE

TO TAKE ALL THE UNSTRUCTURED

DATA THAT WE HAVE TALKED ABOUT

AND ADD THAT TO EXISTING DATA.

THAT WILL ALLOW NOT CLINICAL

DECISION SUPPORT FUNCTIONS THAT

WE TALKED ABOUT, THE FUNCTION

BETTER AND MORE ACCURATELY BUT

PROBABLY OTHER ANCILLARY

BENEFITS IN TERMS OF 

EFFICIENCIES AND UTILIZATION

MANAGEMENT AND PRIOR

AUTHORIZATION, ET CETERA.

SO LAST SLIDE.

WE HAVE COME A LONG WAY, JUST

WITH THE EXISTING DATA SETS THAT

WE HAVE, WITH EXISTING CLINICAL

DECISION SUPPORT TOOLS THAT WE

HAVE.

UNSTRUCTURED CLINICAL DATA WITH

THE HELP OF NATURAL LANGUAGE

PROCESSING WILL ALLOW US TO GET

OVER THE BRIDGE THAT

PARENTHETICALLY HAPPENS TO BE A

BRIDGE WHERE I LIVE IN

PENNSYLVANIA T OLEST PRIVATELY

OWNED BRIDGE IN THE UNITED

STATES.

THANK YOU FOR YOUR ATTENTION.

[APPLAUSE]

>> OUR NEXT SPEAKER IS THE

PREVIOUSLY INTRODUCED DR. MARC

OVERHAGE FROM SIEMENS.

>> I SPENT 25 YEARS AT THE

REGULATORY INSTITUTE DOING

CLINICAL DECISION SUPPORT SO

THAT'S SORT OF MY LONG-TIME

HOME.

AND I HAVE BEEN AT SIEMENS FOR

JUST A YEAR NOW.

IT WAS HARD TO THINK ABOUT WHAT

WOULD BE USEFUL TO TALK ALL OF

YOU ABOUT SO POKE AT A FEW AREAS

THAT WE'RE BUMPING INTO AS WE'RE

DOING OUR WORK WHERE NLP AND

CLINICAL DECISION SUPPORT

COLLIDE AND IN PARTICULAR SOME

THINGS WE HAVEN'T SPENT TOO MUCH

TIME ON.

THE MESSAGE THAT THIS IS

IMPORTANT HAS GOTTEN THROUGH TO

PEOPLE, THIS SLIDE IS ONE THAT

PROFESSOR REICHARDT, HEAD OF OUR

BOARD FOR HEALTHCARE FOR SIEMENS

USED IN LONDON A FEW MONTHS AGO

AT A STOCK ANALYST MEETING, THIS

WAS THE ONLY SLIDE HE USED

TALKING FUTURE OF THE COMPANY.

AND TALKED ABOUT HOW

UNSTRUCTURED DATA DISEASE MODELS

AN THERAPY INTERACT TODAY AND

HOW THE EVOLUTION IS TO

STRUCTURE DATA PATIENT MODELS

INDIVIDUALIZED THERAPY AND

KNOWLEDGE O DOMAIN IN THE MIDDLE

GOING FORWARD.

SO PEOPLE EVEN AT THE SENIOR

LEADERSHIP LEVELS AND SPH POWER

HEALTHCARE AND TECHNOLOGY GET

THIS MESSAGE AND UNDERSTANDING

WHERE ALL OF US TWO WITH THIS

WORK.

SO THIS IS MARC'S MENTAL MODEL

HOW PIECES FIT TOGETHER.

I WON'T TALK ABOUT -- THE RED

STARS REPRESENT PLACES WHERE NLP

IN PARTICULAR POPS UP, I'LL

HIGHLIGHT EXAMPLES OF THAT.

I THINK OF IT AS COMING FROM TWO

DIRECTIONSCH ONE IS THE

LEFT-HAND SIDE WE HAVE HAD A LOT

OF CONVERSATION ABOUT TODAY

WHICH IS PATIENT DATA GETTING

TURNED INTO STRUCTURED DATA SO

IT CAN BE USED AN LIRNED FROM.

-- LEARNED FROM.

WE'RE RUNNING INTO THE RIGHT

HAND SIDE OF THIS EQUATION, HOW

DO YOU POPULATE THAT RIGHT SIDE

WHETHER IT'S PRODUCTION ROLES OR

WHETHER IT'S SOME EXPERT SYSTEM,

HOW DO YOU BUILD THE KNOWLEDGE

BASE THAT NEEDS TO UNDERPIN

THAT.

CONCRETE EXAMPLE IN A MINUTE.

ONE THING BRAG ABOUT, ONE OF

THOSE INTERESTING WAGING

COMMERCIAL SUCCESSES OF NLP

WHICH YOU MIGHT NOT KNOW EXIST

BUT LOOKED AT THE PROBLEM WE

JUST HEARD ABOUT QUALITY

MEASURES.

AND THE CHALLENGE MUCH OF THE

DOCK MEN TAIGHTS IS UNSTRUCTURED

TEK OF SOME KIND --

DOCUMENTATION OF UNSTRUCTURED

TEXT OF SOME KIND, DEFINED BY

FRIENDS AT CMS AND A SPECIFIC

AND CONCRETE WAY.

AND THE SYSTEM DOES THE USUAL TO

FIND AND DERIVE THAT DATA AND

PROVIDE FEEDBACK TO THE USER

ABOUT THE CONTEXT IT CAME FROM,

WHAT'S NEAT, THERE'S A COUPLE OF

THINGS ABOUT THE SYSTEM.

ONE THING TO HIGHLIGHT IS THE

LEARNING ASPECT OF THE SYSTEM.

WHAT I MEAN BY THAT IS THERE ARE

SEVERAL HUNDRED USERS ACROSS THE

UNITED STATES, SEVERAL THOUSAND

ACROSS THE WORLD WHO ARE USING

THE SYSTEM EVERY DAY IN

HOSPITALS AND HEALTH SYSTEMS TO

CAPTURE DATA FOR THE QUALITY

REPORTING.

ALL THE CASES THEY CHOOSE TO

CHANGE THE MACHINES

INTERPRETATION GETS FED BACK

INTO THE LEARNING ALGORITHM ON A

DAILY BASIS AND REFINE IT SO THE

PROCESSING THAT HAPPENS ON

ONGOING BASIS IS IMPROVED BY

FEEDBACK OF THESE THOUSANDS OF

INDIVIDUALS WHO ARE SOLVING

THEIR DAY TO DAY PROBLEM WHICH

IS HOW DO I CAPTURE THIS DATA

FOR A PARTICULAR QUALITY

MEASURE.

SO ONE BIT OF GENIUS THAT BROCK

AND HIS GROUP DID.

A SECOND WAY THAT WE BUMP INTO

PLA A LOT, THIS IS A DIFFERENCE

CLASS OF PROBLEM T RADIOLOGY

NOTE.

WE ALSO FIND NEED FOR SHORT SNIP

PETS.

WHICH IS A WHOLE DIFFERENT SET

OF CHALLENGES AND PROBLEMS AND

WE'LL HEAR A BIT ABOUT QUESTION

ANSWERING IN A BIT.

THAT'S ANOTHER DOMAIN WHERE

SHORT INPUT HAS TO BE

INTERPRETED.

ANOTHER ONE WE HAVE SPENT TIME

ON CARRIED OVER IN THE WORK I

WAS DOING ON REAGAN STREET IS

THE FACT YOU HAVE IN LABORATORY

RESULTS AN ANSWER, A SHORT SNIP

PET OF TEXT SO IF YOU'RE TRYING

TO DO PUBLIC HEALTH REPORTING OF

REPORTING CONDITIONS YOU MIGHT

HAVE A TEST FOR SHEGELLA WHICH

MIGHT SAY SHEGELLA ISOLATED BUT

MORE COMMON TO HAVE SOMETHING

THAT SAYS NO SHEGELLA,

SALMONELLA OR ECO LIE AND YOU

HAVE TO RECOGNIZE THE THREE

NEGOTIATIONS AND ALL YOU HAVE

ARE 6 OR 7 WORDS.

BUT VERY SHORT SNIP PETS OF

TEXT.

AND THIS IS JUST FROM PURPOSES

OF PUBLIC HEALTH REPORTING WHAT

WE HAD DONE AT REAGAN STREET WAS

THREE LAYERS, THAT'S A THEME

FROM THE WORKSHOP, NUMERIC

RESULTS ARE EASY, YOU SAID

THRESHOLDS AND THINGS, DISCRETE

RESULTS LIKE POSITIVE OR

NEGATIVE AND HARDER QAT GOIR OF

RESULTS WHERE THE ANSWER TO THE

QUESTION IS SOMETHING THAT'S A

SHORT SNIP PET OF TEXT AND NEEDS

TO BE TURNED INTO STRUCTURE

CONTENT IN ORDER TO PROCESS IT.

THE SECOND THING WITH THE SO

CALLED REMIND PLATFORM WHICH WAS�

INTRIGUING AND THERE WERE A

COUPLE OF REFERENCES THROUGHOUT

THE DISCUSSION, IS THE NEED

PARTICULARLY WITH UNSTRUCTURED

DATA TO BEGIN TO COMBINE AND

REASON PROBABLISTICLY ABOUT THE

DATA THAT WE GOT.

SO CLASSIC EXAMPLE HE WALKED ME

THROUGH WHEN HE FIRST DESCRIBED

THIS IS A PATIENT IN THE

HOSPITAL, YOU HAVE AMBULATORY

PHYSICIANS NOTE WHICH SAYS THE

PATIENT TRIED TO QUIT SMOKING

TWO YEARS AGO BUT FAILED AND

RESTARTED.

YOU HAVE THE INFORMATION FROM

THE PATIENT PHR THAT SAYS A YEAR

OLD THAT SAYS I'M NOT SMOKING

ANY MORE AN YOU HAVE DATA FROM

THE ADMITTING NURSES NOTE THAT

SAYS THE PATIENT SMOKES A PACK

AND A HALF A DAY BUT

INTERMITTENTLY.

HOW DO YOU PUT THAT TOGETHER TO

DECIDE WHETHER THE PATIENT

SMOKES OR NOT?

SO TAKING INTO ACCOUNT THE

TEMPORAL PATTERN.

WE HEARD FROM PARTNERS NLP GROUP

IN THE LAST SESSION AN

RELIABILITY OF THE REPORTER YOU

CAN CONSTRUCT PROBLEM LISTIC

INFERENCE ABOUT THOSE INDIVIDUAL

ELEMENTS AND DECIDE WHAT YOU

WANT THE ANSWER TO BE TODAY IN

PLAWR UNSTRUCTURED DATA BUT

STRUCTURED DATA AS WELL.

SO THIRD SNIP PET, AS -- WITHIN

SIEMENS IS ANOTHER SHORT TEXT

ASPECT OF NLP WITH A SYSTEM YOU

MIGHT BE RECORDING ENTERING

SYSTEMS AND FINDING DIAGNOSES IN

A FREE TEXT FORMAT FOR EXAMPLE

AND THE SYSTEM PROCESSING IS

EACH WORD IS ADDED TO THAT TEXT

TAKING INTO ACCOUNT WHAT WAS

THERE BEFORE AS WELL AS PATIENT

AND PROVIDER CONTEXT INFORMATION

IN ORDER TO DERIVE USEFUL OUTPUT

IN VARIOUS WAYS SO HERE IS A

CONCRETE EXAMPLE FROM WORK AT

REAGAN STREET APPLYING THIS ON

THE LEFT-HAND SIDE IS A FREE

TEXT BOX WHERE USER IS FREE TEXT

NARRATIVE TEXT BOX WHERE A USER

IS ENTERING THINGS AND ON THE

RIGHT HAND SIDE AS THAT TEXT IS

PROCESSED, THE SYSTEM IS MATCHES

THAT UP WITH CONCEPTS OF THINGS

THAT MIGHT BE REASONABLE TO

ORDER.

SO DIABETES THEY NOTED THE

OPHTHALMOLOGY CONSULT AND

ELECTROLIGHTS MIGHT BE

APPROPRIATE SO THOSE THREE

THINGS SHOW UP ON THE RIGHT HAND

SIDE TO MAKE THEM EASY FOR THE

USER TO GO AHEAD AND ORDER.

SOMEWHAT ANALOGOUS TO THE WORK

THAT TOM PAING DESCRIBED ADDING

PROBLEMS TO THE PROBLEM LIST,

MAKING IT EASY TO GET TO THOSE

THINGS AND NOT FORGET ABOUT

THEM.

ANOTHER WAY THAT WE'RE

LEVERAGING THIS DATA AND

ACTUALLY TRYING TO IMPROVE THE

DATA CAPTURE IS BY BUILDING

INDIVIDUALIZED PATIENT MODELS.

THAT COMES INTO PLAY IN

PARTICULAR IN THE CAPTURING OF

THE DATA.

IF WE HAVE VERY GOOD MODELS FOR

THAT INDIVIDUAL PATIENT THIS IS

ONE FOR HYPERTENSION, ONE OF MY

COLLEAGUES GLENN FUNG DID, AS

YOU BEGIN TO HAVE THIS YOU CAN

PLAY THAT BACK INTO THE DATA

THAT YOU CHOOSE TO USE OUT OF

THE PATIENT'S RECORD WHETHER FOR

CLINICAL DOCUMENTATION OR OTHER

SORTS OF USE, SO THAT'S ANOTHER

PLACE WE'RE LEVERAGING IT.

THE RIGHT HAND SIDE WE FOCUS ON

A MINUTE, I HAVEN'T HEARD A LOT

OF CONVERSATION TODAY BUT ONE

PLACE WE'RE SPENDING A FAIR

AMOUNT OF ENERGY FOR A VARIETY

OF REASONS WORKING FROM THESE

UNSTRUCTURED AS WELL AS

STRUCTURED DATA SOURCES AND

TRYING TO DRIVE THROUGH TO

USABLE KNOWLEDGE AND WAYS TO

CURE RATE AND MANAGE THAT

KNOWLEDGE SO WE CAN BEGIN TO USE

IT FOR ACTUAL REASONING AND

SEMANTIC SORTING OF DATA AND

THINGS OF THAT NATURE.

WE'LL HEAR ABOUT THAT IN A FEW

MINUTES.

AND IN PARTICULAR, ONE OF THE

THINGS WE TRY TO FIND IS I CALL

THE ART OF POSSIBLE.

IN OTHER WORDS, WHILE THESE

APPROACHES ARE IMPERFECT IN MANY

WAYS OUR ABILITY TO FIND

SEMANTIC STRUCTURE WITHIN

WHETHER GUIDELINES OR JOURNAL

ARTICLES AND SO ON THAT CAN

BEGIN TO INFORM HOW WE DRIVE

DECISION SUPPORT HAVE BEEN

INTERESTING, PROBABLY THE MOST

SOPHISTICATED APPLICATION OF

THIS THAT WE HAVE DONE TODAY HAS

BEEN WHERE WE'RE TRYING TO PULL

TOGETHER SEVERAL ASPECTS AND

TAKING DATA IN THE SEMANTIC

STRUCTURE AN ONTOLOGY DERIVED

FROM THE CARDIAC LITERATURE

ALONG WITH STRUCTURAL

OBSERVATIONS FROM CT MRI

ULTRASOUND OF THE HEART.

USING THOSE CONSTRUCTED DYNAMIC

MODELS OF THE CARDIAC

CIRCUMSTANCE LAS.

SO STRUCTURAL OBSERVATION, DATA

FROM LITERATURE HOW THESE

INTERACT AND CREATE DYNAMIC

MODELS OF THE HART THAT CAN BE

USED FOR DECISION SUPPORT FOR

CARDIAC SURGEONS WHO ARE

OPERATING ON COMPLEX CARDIAC

ABNORMALITIES TO SAY IF WE

OPENED UP THEt

OF 25%, WHAT MIGHT THAT DO TO

THE CARDIAC CIRCULATION?

NEONATES AND IN PARTICULAR BUT

IN OTHER -- EVEN IN OLDER ADULTS

FOR EXAMPLE, WHILE EXPERIENCE IS

A GOOD GUIDE, HAVING THE DATA

AND MODEL THAT UNDERPINS IT CAN

BE INCREDIBLY POWERFUL DECISION

SUPPORT TO SAY TURN THE DIAL

THIS WAY, WHERE WILL I LAND?

THAT MIGHT LEAD FOR EXAMPLE TO

CHOOSE A LESS AGGRESSIVE

PROCEDURE INITIALLY REASSESS AND

THEN FOLLOW-UP WITH A MORE

AGGRESSIVE PROCEDURE IF NEEDED.

AND THERE ARE O OTHER EXAMPLES

OF THAT.

SO THAT KIND OF DECISION SUPPORT

FOR THERAPEUTIC CHOICES CAN BE

DRIVEN BY THESE DATA DRIVEN OUT

OF THESE VARIOUS SOURCES.

SO BROADER SCALE WE LOOK

ATTENTION MINING AND NLP AS ONE

SOURCE ALONG WITH IMAGE

SEGMENTATION, FORMALIZATION OF

TREATMENT PLANS AND SO ON.

THAT SORT OF CREATE A CONTINUUM.

AND THAT'S -- THIS IS A -- NOT A

PRODUCT BUT A PROTOTYPE THAT

TRIES TO START TO PULL TOGETHER

ALL THESE DIFFERENT USES OF NLP

AND DECISION SUPPORT A LITTLE

BIT THE WORK BLACKBERG AND HIS

GROUP DID, TOOK A DIFFERENT

DIRECTION BUT ON THE TOP LEFT

HAND PANEL THERE PATIENT DATA

WITH IMAGING LINKED THROUGH A

SEMANTIC NETWORK SO

ABNORMALITIES AN IMAGE AND DATA

PATIENT RECORD ARE LINKED

TOGETHER.

MIDDLE RIGHT HAND PANEL, WHAT

HAPPENS TO THE LAST THOUSAND

PATIENT WHOSE LOOKED LIKE THIS

SO THE COMPARISON

RECOMMENDATIONS BASED ON SYSTEM

CASES DRIVEN OUT OF LOCAL RECORD

SYSTEM IN THE BOTTOM KNOWLEDGE

NETWORK LINKAGE BACK TO THE

CLINICAL GUIDELINES AN

LITERATURE THAT SUPPORT  THOSE

THINGS AND ON THE BOTTOM RIGHT

TAKING ADVANTAGE OF PUBLICATION

STREAMS AND PATIENT CARE

PATTERNS TO LINK TO

PRACTITIONERS THAT MIGHT HAVE

PARTICULAR EXPERTISE IN THIS

PATIENT'S CARE SO THIS IS CALLED

MEDICO, PART OF A EUROPEAN

PROJECT WE'RE JUST IN FEBRUARY

HAD THE FIVE YEAR WRAPUP FOR AN

EFFORT TO PULL TOGETHER PIECES

OF DECISION SUPPORT LEVERAGED BY

NLP TO CREATE A DASHBOARD IF YOU

WILL IN MY MIND PROPORTIONED

WRONG FOR ILLUSTRATION PURPOSES

THE PATIENT SORT OF NOT OVERLY

EMPHASIZED BUT IT STARTS TO HINT

AT THE POSSIBILITIES AS YOU PULL

ALL THESE THINGS TOGETHER IN

ORDER TO ENABLE DECISION SUPPORT

LEVERAGED BY NLP ACROSS THE

BOARD.

SO THANKS VERY MUCH FOR YOUR

ATTENTION.

LOOK FORWARD TO THE DISCUSSION.

[APPLAUSE]

>> OUR NEXT SPEAKER IS DR. DAVID

DON GEK OF IBM.

-- GONDEK OF IBM.

>> I'M DAVID GONDEK, PART OF THE

IBM WATT SEASON TEAM RESPONSIBLE

FOR THE MACHINE LEARNING IN THE

GENERALITY SYSTEM -- JEOPARDY

SYSTEM AND NOW LEAD FOR THE WE

USE WATSON IN THE HEALTHCARE

SPACE.

I WANT TO THANK THE ORGANIZE

RECOGNIZERS FOR OFFERING TO

OFFER A CHANCE TO TALK AND NLM,

IT'S ALWAYS GOOD TO COME DOWN

HERE.

WE'RE A BIG CLOSET NLM FANS BACK

IN THE LAB.

WE HAVE A LIST CALLED THERE'S A+p 

COE FOR THAT.

THE DISCUSSION LAST WEEK WAS

ABOUT SOMEONE FOUND SENSE OF

IMPENDING DOOM.

AND ANY SOUND IMPENDING DOOM.

YOU SAY THAT'S NOT ONE YOU WANT

TO SEE IN YOUR RECORD.

ANYWAY, SO I THOUGHT I SHOULD

START OFF THERE'S HOPE FOR WHAT

I CAN DO.

ARCHITECTURAL DIAGRAM.

ROUGHLY SPEAKING THE

ARCHITECTURE DIAGRAM LOOKS LIKE

THIS.

WHERE YOU HAVE A WHOLE BUNCH OF

COMPONENTS ON THE LEFT AND IN

THE MIDDLE A WATSON OCCURS AND

YOU GET AN OUTPUT.

THIS IS FUNNY AND SCARY BECAUSE

PEOPLE THINK THAT WATSON CAN DO

ALMOST ANYTHING WHICH I CAN'T.

WE'RE WORKING ON IT.

IT'S SCARY AND THIS IS SO FAR

FROM THE TRUTH IN THAT WHEN WE

BUILT WATSON WE RELIED ON A LOT

OF THE BEST EXISTING TECHNOLOGY.

SO WE WERE USING TECHNOLOGY FROM

THE LATEST LITERATURE, WE WERE

USING A PARSER IN DEVELOPER IBM

30 YEARS, LOOKING AT THE MEDICAL

DOMAIN WE USE UMLS WHICH IS A

HUGE AMOUNT OF EFFORT.

SO IT'S TRUE WATSON IS AN

ENSEMBLE SYSTEM AND DEPENDS ON

COMPONENTS THERE.

THE ORIGINAL SYSTEM IS PLAYING

JEOPARDY.

SOME SAY THAT'S'S WHEN A MIRACLE

OCCURRED.

WHAT DOES IT TAI TO PLAY

JEOPARDY.

THAT DICTATE AD NUMBER OF

CHOICES FOR THE SYSTEM.

I THOUGHT IT WOULD BE

INTERESTING TO THINK ABOUT

WHETHER THOSE CHOICES ARE USEFUL

IN THE MEDICAL CONTEXT.

WHAT YOU'LL SEE IS DUE TO THE

BACKGROUND THERE, WE'RE TAKING A

SOMEWHAT DIFFERENT APPROACH THAN

SOME OF THE OTHER APPROACHES YOU

HAVE HEARD ABOUT.

SO IN PARTICULAR WE LOOKED AT

BOTH UNSTRUCTURED APPROACHES AN

STRUCTURED APPROACHES FOR

JEOPARDY AND I THINK EVERYONE

HERE IS FAMILIAR WITH THE

LIMITATIONS AND BENEFITS TO

BOTH.

THE GROUP THAT I'M FROM MORE A

BACKGROUND DEALING WITH

UNSTRUCTURED DATA, SEMANTIC

SEARCH, CLASSIFICATION, THAT

SORT OF THING.

I THINK WE'RE FAMILIAR WITH KEY

WORD SEARCH WHERE IT HAS BROAD

COVERAGE, VERY FAST, CAN BE VERY

TIMELY, BUT OF COURSE THE

PRECISION IS LOW.

AND THE -- THERE'S BASICALLY NO

SEMANTICS.

SO VERY LITTLE SEMANTICS DOING A

KEY WORD SEARCH.

YOU CAN TRAP WITH MORE KNOWLEDGE

BASED APPROACHES OR RULE BASED

APPROACHES WHICH YOU CAN DO

ELABORATE IMPRESSIVE REASONING

AND YOU HAVE PRECISE SEMANTICS

SO WHEN YOU GET AN ANSWER YOU

CAN COME ONE THE PROOF, MAY NOT

BE READABLE BUT YOU CAN COME UP

WITH A PROOF OF WHY THE ANSWER

IS THERE.

AND LIABILITIES WHEN YOU HAVE TO

HAND CONSTRUCT THE MODELS

THEY'RE FAIRLY BRITTLE, HARD TO

MAP INTO THE MODELS.

DEALING WITH NATURAL LANGUAGE IT

WOULD BE HARD TO MAP IT TO

CONCEPTS.

HARD TO KEEP THEM UP TO DATE AND

IT ALSO CAN BE COSTLY.

SO THE JEOPARDY SYSTEM WAS A

COMBINATION OF THESE TWO

APPROACHES.

WE MERGE THE BEST APPROACHES TO

GET YOUR HANDS ON IN SEARCH WITH

KNOWLEDGE BASED TECHNIQUES.

AND THE HOPE THEN IS THAT YOU

CAN GET COVERAGE YOU NEED FOR

OPEN DOMAIN JEOPARDY WHERE THEY

MAY NOT ASK ABOUT ANYTHING AND

FOR CERTAIN AREAS YOU REALIZE

SEMANTICS WILL HELP YOU CAN

INVEST IN SPECIFIC AREAS.

I'M NOT CLAIMING THAT WATSON

DEEPLY REPRESENTS COMPLEX

MEDICAL GUIDELINE.

THAT IS NOT THE GOAL.

THE GOAL IS TO GIVE WATSON THE

SHY LOW SEMANTICS TO INTERPRET A

LARGE NUMBER OF GUIDELINES OR

DIAGNOSES WHERE WE CAN GET THE

COVERAGE WITHOUT MAYBE QUITE THE

SAME EXPLANATORY DEPTH.

SO ONE OF THE IMPORTANT REASONS

WHY WE THINK WE HAD SUCCESS WAS

THE ARCHITECTURE.

AND I'M ON A TEAM WITH 40

RESEARCHERS, FOCUSING ON THE

UNDERLYING TECHNOLOGY OF WATSON.

PART OF THAT, HALF OF THOSE ARE

ALGORITHMS, OTHER HALF SYSTEMS

SO I'M DEVELOPING THE

ARCHITECTURES FOR PLUGGING THE

ALGORITHMS TOGETHER.

HOW CAN YOU MAKE IT FAST.

DEALING WITH NORMALIZING THE

DATA.

STUFF LIKE THAT.

BUT IF YOU LOOK AT THE STATS ON

WATSON, JEOPARDY SYSTEM WAS

BUILT ON POWER 7 SYSTEM, I THINK

PEOPLE MENTIONED DISES DIZ AND

-- (INDISCERNIBLE) AND WE BUILT

A STATISTICAL LEARNING FRAMEWORK

AND NLP TECHNIQUES SO YOU LOOK

AT THE NLP TECHNIQUES WE

DEVELOPED, THEY'RE REAL

STATISTICAL, WE USE ANYTHING WE

CAN GET OUR HAND ON.

SOME THINGS YOU HEARD LIKE

QUESTION PARSING SENTENCE

PARSING DISAMBIGUATION, CODING,

RELATION EXTRACTION.

TRYING TO FIND THE TWO THINGS
 IN

TEXT.

LINGUISTIC FRAME EXTRACTION IS

WE BUILD KNOWLEDGE BASES FROM

LARGE CORPORATE.

SO IN THE JEOPARDY CASE WE MINE

TWO TERABYTES OF WEB DATA, ABOUT

10% LIBRARY OF CONGRESS, THAT

WAS TO GET FACTS AND KNOWLEDGE

BASE, SO WE KNOW WHO INVENTED

WHAT.

IT'S ALSO TO UNDERSTAND USAGE.

SO YOU CAN UNDERSTAND HOW

LANGUAGE IS USED, WHAT ARE

SELECTIONAL RESTRICTIONS,

PARTICIPATING IN THIS RELATION.

WE CAN LEARN RELATIONS, TO

EXPAND YOUR KNOWLEDGE BASE AND

PERHAPS YOUR HAND BUILT

KNOWLEDGE BASES(.t DON'T HAVE

ENOUGH COVERAGE.

THE COST WE HAVE A TALENT

FRAMEWORK AND THE PURPOSE OF

THAT IS TO IDENTIFY WHETHER ONE

PASSAGE OR SET OF PASSAGES

JUSTIFIES AN ANSWER TO A

QUESTION.

SO THAT MAY LOOK AT JUST LOOKING

AT A BAG OF WORDS MATCH, DO

THESE LEXCAL TERMS LIKE KEY WORD

SEARCH, DEEPER ANALYSIS LIKE

USING THE PRODUCT AND ARGUMENT

STRUCTURE, LOGICAL FORM, CODING,

DOESN'T HAVE TO.

THAT ALLOWS YOU TO -- YOU HAVE

ENSEMBLE OF DIFFERENT APPROACHES

THAT TRADE OFF SENSITIVITY AND

SPECIFICITY.

SO SOME THINGS ARE PURELY

LEXCAL, OTHER THINGS REQUIRE YOU

CODE CORRECTLY AND ABLE TO

IDENTIFY THE RELATIONS.

SO IT'S THIS ENSEMBLE THAT WE

FOUND HAD THE BEST PERFORMANCE.

THE WAY WE WERE ABLE TO DO THIS

IS WITH SOME OF THE INTEGRATION.

I MENTIONED UIMA BEFORE, THE

WATSON SYSTEM WAS BASED ON ONLY

TEXT DATA.

UMA ALLOWS A MULTI-MODAL

ANALYTICS FOR THINGS LIKE IMAGES

AND SPEECH.

IT MEANS THERE'S A SHARE ACROSS

THE SYSTEM.

IF YOU YOU HAVE A COMPONENT, IF YOU

HAVE A RESEARCHER OR KNOWLEDGE

BASE TO ADD IT IS FAIRLY SIMPLE,

YOU HAVE TO WRITE IT TO THE UIMA

API AND PLUG IT INTO THE SYSTEM.

THAT'S NOT ENOUGH BECAUSE WHAT

IS A SYSTEM GOING TO DO WITH

THAT.

IT'S NOT ENOUGH TO ADD YOUR NEW

PARSER OR YOUR NEW ONTOLOGY.

YOU HAVE TO FIGURE WHEN TO USE

IT, WHEN THE TRUST IT, WHEN NOT

TO TRUST IT.

THAT'S WHEN THE IMMIGRATION

FRAMEWORK COMES IN.

SO YOU HAVE A FRAMEWORK TO

REGISTER THE ANALYTICS AND IT

TRAINS SO USING TRAINING DATA,

JEOPARDY CASES WHERE JEOPARDY

QUESTIONS AND ANSWER, IT LEARNS

WHICH COMPONENTS ARE MORE

RELIABLE FOR WHICH QUESTIONS.

THEN AT APPLY TIME RUNNING THE

SYSTEM USE THE TRAIN MODELS TO

COMBINE THE OUTPUT OF YOUR NLP

TECHNIQUES.

THE NET OF THIS IS, THAT MEANS

THAT YOU DON'T HAVE TO

UNDERSTAND THE ENTIRE SYSTEM TO

IMPROVE IT.

SO YOU CAN WRITE A SPECIFIC

COMPONENT, PLUG IT IN, LET THE

SYSTEM LEARN HOW TO WEIGH IT.

AND IT'S SOMETHING THAT WE'RE

WORKING ON ALL THE TIME HOW TO

EXTEND THAT TO MORE ASPECTS OF

THE SYSTEM, EVERYTHING FROM

SCORING PASSAGES TO IDENTIFYING

THE TYPES OF THINGS TO

EVALUATING SOURCE QUALITY, DO I

TRUST THE SOURCE OR NOT.

ANOTHER IMPORTANT ASPECT OF THE

PROJECT IS WE HAD AN END TO END

EXPERIMENTAL EVALUATION.

SO WE HAD TOOLS AND WE HAD

STANDARD TEST SETS FOR

EVALUATING WHETHER A COMPONENT

WAS ADDING PERFORMANCE OR NOT.

THIS WAS HUGELY IMPORTANT

BECAUSE IT HELPED TELL PEOPLE

WHAT TO WORK ON.

THE BIGGEST PROBLEM WE HAVE IS

NOT THINKING OF IDEA OR EVEN

DOING THE WORK BUT TRIAGEING

WHAT'S MOST IMPORTANT AT THIS

POINT WHAT'S GOING TO HAVE THE

MOST EFFECT ON END TO END

ACCURACY.

SO ONE OF THE MOST IMPORTANT

THINGS WE LEARNED IN DEVELOPING

THE SYSTEM WAS HAVING THE

STANDARD SET OF QUESTIONS IN OUR

CASE JEOPARDY, WE CAN -- IF YOU

ASK ME I CAN TELL YOU HERE ARE

THE FIVE MOST IMPORTANT THINGS

TO WORK ON, I CAN LOOK AT THE

SET AND SAY WHAT IS THE HEAD

ROOM FOR POSSIBLE IMPROVEMENT.

IF YOU WORK ON PARSING HOW MUCH

IMPROVEMENT HOW MUCH IMPACT DID

THAT HAVE.

I SHARED TEST OR BENCHMARK

ALLOWED EVERYBODY TO TUNE

COMPONENTS TO BEST HELP THE

SYSTEM AND TELL THEM WITH WHAT

TO WORK ON.

FINALLY IT PRODUCES CONFIDENCE,

SO EACH ANSWER CAN GIVE YOU A

CONFIDENCE WHAT'S THE

PROBABILITY IS CORRECT, IT CAN

ALSO TELL YOU WHY IS THE ANSWER

THERE.

SO WHAT ARE THE DIFFERENT SORTS

OF EVIDENCE THAT IT USED.

IT CAN ALSO POINT TO SPECIFIC

PIECES OF EVIDENCE.

SO ONE NICE THING BECAUSE WE

WORK FROM TEXT WE CAN PULL UP

THE IS A PAJ AND SAY HERE IS THE

PASSAGE WE USE AND SEE IF IT

CORRECTLY INTERPRETED IT.

WE WANT THE FIND A SET OF

QUESTIONS.

DR. SIEGEL BROUGHT THIS UP, IT

WOULD BE NICE TO EVALUATE

DIFFERENT TOOLS CERTAIN WILL

AGREED IN A CLINICAL CONTEXT BUT

ALSO USEFUL FOR DEVELOPERS TO

DEVELOP A STANDARD BENCHMARK

SETS.

HOW DO WE TUNE THE ALGORITHMS TO

BEST IMPROVE PERFORMANCE.

SO AS SOON AS WE LOOK AT THE

MEDICAL DOMAIN WE SAID WE HAD TO

FIND DATA.

THE DATA THAT WE INITIALLY FOUND

WAS THIS AMERICAN COLLEGE OF

PHYSICIANS QUESTIONS, WHICH ARE

THE TRIVIA GAME WITH MEDICAL

RESIDENTS AND PROFESSORS.

FOR THESE THINGS LIKE SKIN RASH

ASSOCIATED WITH LAMB DISEASE OR

TYPE ASSOCIATION WITH THIS

(INDISCERNIBLE).

SO THESE ARE FAIRLY SIMPLE

QUESTIONS, THEY HAVE AN AGREED

CORRECT ANSWER, THEY TIP THINK

CLI HAVE A SINGLE ANSWER, NOT

ALWAYS RELEVANT INFORMATION IS

GIVEN AND ALL THE INFORMATION IS

GIVEN SO IT SHOULD UNIQUELY

SPECIFY AN ANSWER, IT'S A NICE

TEST TO SEE HOW DOES THIS

PERFORM AND WHAT WE NEED TO WORK

ON FIRST.

IN THE GRAPH OPT RIGHT I SHOW

PERFORMANCE OF A SYSTEM.

THATIx]� LITTLE PURPLE LINE IS THE

JEOPARDY SYSTEM.

SO NOTHING WAS CHANGED FROM THE

JEOPARDY SYSTEM, LIKE WE

LITERALLY TOOK OFF THE SET AND

GAVE MEDICAL QUESTIONS.

NOW, WHAT THE GRAPH IS SHOWING

YOU IS YOU CAN ANSWER BETWEEN

ZERO AND 100% OF THE QUESTIONS,

THE SYSTEM RANKED THE QUESTIONS

BASED ON CONFIDENCE SO IF IT

ANSWERED ALL 100% JEOPARDY

SYSTEM IS 20% CORRECT.

IF IT GOT TO PICK 10% CONFIDENCE

WE GET LIKE 40% CORRECT.

AS WE START TO IMPROVE THE

SYSTEM LIKE ADD SOURCES AT THE

RED LINE WE ADD AD FEW MEDICAL

REFERENCE TEXTS WE CAN GET OUR

HANDS ON.

WE RETRAINED SO INSTEAD OF

HAVING A SYSTEM TRAINING

JEOPARDY QUESTIONS WE TRAINED ON

(INAUDIBLE) QUESTIONS.

WE BEGAN THE WORK OF FUNCTIONAL

ADAPTATION.

SO THE ISSUE IS EVENTUALLY YOU

GET ABOUT AS MANY SOURCES AS

YOU'RE GOING TO GET, EVENTUALLY

MACHINE LEARNING SATURATES.

MORE DATA TRAINING DOESN'T HELP

YOU SO YOU HAVE TO IMPROVE WHAT

CAPABILITIES THE SYSTEM HAS.

YOU HAVE TO WORK ON THE MEDICAL

REASONS.

AND THAT'S THE BLUE LINE THERE,

SO RIGHT ABOUT 50% WHICH SOME OF

THE MEDICAL TELL ME AB WHERE

THEY ARE.

I DON'T HAVE FIRM NUMBERS ON

THAT.

EXAMPLES QUESTIONS

(INDISCERNIBLE) NASAL PASSAGES

CAUSE OF CONGENITAL HYPERBO LAR

ANEMIA, SO THESE ARE SOMEWHAT

USEFUL, I THINK WHAT MIGHT BE

MORE INTERESTING IS TECHNOLOGY

THAT GOES INTO SOLVING THEM.

SO YOU CAN USE IT FOR QUESTION

ANSWERING AND ALSO ASK WHAT ARE

THE COMPONENTS WE PUT TOGETHER

TO ANSWER THESE.

ANOTHER THING TOO, FROM JEOPARDY

WE INHERITED THIS INTERACTION

MODEL, YOU GET A QUESTION, YOU

GIVE AN ANSWER.

WE ALSO HAD THIS FOCUS ON

GETTING THE CORRECT ANSWER FIRST

IN YOUR ANSWER LIST.

SO MAKING SURE THE TOP ANSWER IS

CORRECT.

THAT MIGHT NOT BE AS IMPORTANT

IN OTHER DOMAINS SO YOU MIGHT BE

INTERESTED IN SOMETHING LIKE

THIS QUESTION WHERE THE CORRECT

ANSWER IS IN SECOND PLACE.

CANCER, IT DID HAVE THE CORRECT

ANSWER, SECOND PLACE.

SO HOW IS IT ACTUALLY COME UP

WITH THOSE ANSWERS AN JUSTIFY

THEM?

A QUESTION LIKE THIS, WHAT

NEUROLOGICAL

>>MS. O'CONNELL:TRA INDICATE IT

IS USE OF APROPRION.

WE HAVE ANALYSES LIKE YOU HAVE

SEEN, TOKENNIZING, DETECTION,

PARSING, DETECTION NEGATION

DETECTION, RELATION DETECTION,

SO FORTH.

SO WE HAVE TO UNDERSTAND HOW

THESE TERMS RELATE.

ALSO THINGS LIKE (INAUDIBLE)

DETECTION.

SO WE'LL SEE AHA, IT HAS A COI

SO WE KNOW WHAT THAT IS.

NOW, WE HAVE THINGS LIKE

RELATION DETECTION SO WE

TRANSSEMANTIC RELATIONS AHEAD OF

TIME ON THE MOST IMPORTANT

MEDICAL RELATIONS FOR THE SET.

SO WE IDENTIFY THAT, AHA, ASKING

FOR A CONTRAINDICATED RELATION.

BUT IF YOU HAVE A KNOWLEDGE

BASE, YOU CAN LOOK A THAT UP.

I SUPPOSE I HAVE UMLS RELATIONS,

SOMETHING LIKE THAT.

I CAN LOOK UP THE DRUG, I CAN

TRY TO SEE IF I HAVE ASSOCIATED

CONTRAINDICATION RELATIONS FOR

IT AND COME UP WITH THE ANSWER.

NOW, ONE THING WE LEARNED FROM

THE JEOPARDY EXPERIENCE WAS THAT

THIS IS PROPAGATE.

IF YOU DON'T GET THE PARSE RIGHT

AND YOU DON'T COME UP WITH A

CORRECT RELATION AND DON'T

SEGMENT YOUR ARGUMENT CORRECTLY

YOU'RE GOING TO GET MISTAKES.

SO THE SYSTEM DOES NOT DEPEND ON

PERFECTLY CODING EVERYTHING.

WE ALSO HAVE A NUMBER OF

PASSENGER SCORES SO WE'LL USE

TEXT.

TO LOOK AT UNSTRUCTURED CONTENT.

YOU CAN SEE DIFFERENT PASSAGES

WHICH JUSTIFY THE ANSWER.

SO THINGS LIKE BUPROPRION IS

CONTRAINDICATED NAN REXIA.

IT MATCHES -- THE PARSE MATCHES

CLOSELY WITH SOME ALGORITHMS

THAT WE USE.

YOU ALSO HAVE TO KNOW THINGS

LIKE IT HAS A LIST OF CONDITIONS

HERE WHICH ONE OF THEM IS A

NEUROLOGICAL CONDITION.

THEN YOU HAVE TO GO TO

STRUCTURED RESOURCE.

ANOTHER QUESTION LIKE

WELLBUTRIN, CONTRAINDICATED IN

ADULTS WITH SEIZURE DISORDERS.

YOU HAVE TO KNOW THAT'S A

COMMERCIAL BUPROPRION.

BACKGROUND KNOWLEDGE AGAIN.

WE'RE ALWAYS GOING BACK AND

FORTH BETWEEN USING UNSTRUCTURED

TECHNIQUES AND THE STRUCTURED

APPROACHES.

WITH THE SYSTEM ALLOWS YOU TO DO

IS PLUG THOSE IN AND SOME EXTENT

TRAIN ITSELF HOW TO USE THOSE.

THIS IS NICE BECAUSE YOU DON'T

CODE PERFECTLY.

ON THE STOCK ON THE QUESTION,

AND EVERYTHING THE NLP ON THE

SUPPORTING PACKAGE WILL TRY TO

MATCH IF WE CAN.

SO BEST CASE WE'LL EXTRACT A

RICH FRAME OR RELATION.

MAYBE NOT ABLE TO CORRECTLY

EXTRACT THOSE WE'LL USE THINGS

THAT ARE NOT AS SOPHISTICATED,

THINGS THAT MIGHT USE --

ENTITIES THAT APPEAR IN BO.

THINGS THAT MIGHT USE SOME PARSE

BUT NOT WHOLE PARSE AND NOT

REQUIRE A PERFECT PARSE AND THEN

YOU HAVE ENSEMBLE OF SCORES TO

PUT OUT A SCORE WHETHER IT

THINKS THE ANSWER IS CORRECT OR

NOT, JUSTIFIED BY THIS PASSAGE

AND SYSTEM WILL LEARN HOW TO

COMBINE THOSE.

WE HAVE TO DO THIS BECAUSE WE

COULDN'T STICK WITH A PURELY

KNOWLEDGE BASED APPROACH,

BECAUSE THE PROPAGATION OF

ERRORS.

SO WHAT THIS MEANS IS ANOTHER

IMPORTANT ASPECT THAT WE HAVE

WORKING ON IS THIS MATCHING

FRAMEWORK.

WHICH ALLOWS YOU TO PLUG IN

DIFFERENT ALGORITHMS FOR DOING

THESE MATCHINGS ON THE PARSE AND

RELATIONS AN CONCEPTS A THAT

OCCUR.

IF YOU'RE A MATCH THING YOU CAN

GET DIFFICULT AND HARD TO FIND

TRAINING SETS.

IN THE MEDICAL DOMAIN IT'S

SUBTLE.

AS I LOOK FOR SOMETHING LIKE

EROSION ENAMEL EROSION IS IT THE

SAME AS TOOTH ENAMEL EROSION.

IS THAT THE SAME AS

(INDISCERNIBLE)?

DOES IT MATTER IF IT'S YELLOW OR

NOT?

INCREASES WITH (INDISCERNIBLE)

VERSUS DECREASES WITH VALSALVA

IS VERY IMPORTANT.

SO DOING THIS MATCH IN BETWEEN

THESE TWO SETS OF SIGNS OR

SYMPTOMS IS VERY SUBTLE AND HARD

TO FIND TRAINING FOR BECAUSE IT

REQUIRES UNDERSTANDING OF THE

DOMAIN, SO THIS IS FIRST THAT WE

HAVE TAKEN ON KNOWLEDGE TRYING

TO COME UP WITH A TRAINING

SETTER FOR THIS, COME UP WITH WE

HAVE RULE BASED TECHNIQUES,

STATISTICAL TECHNIQUES.

WE TRY TO CODE THESE SO WE RUN

MEDIMAP AND IN EFFECT RUN OVER

IT AND NOT TRY TO CODE IT, JUST

TRY TO IDENTIFY HIS SYSTEM SO

TRYING TO BE ROBUST AND NOT LET

ERRORS PROPAGATE THROUGHOUT THE

SYSTEM.

YOU LOOK AT -- UP PER AIRWAYS OF

LOCATION, YOU NEED TO KNOW THAT.

WE MINE TEXT, WE ALSO USE UMLS.

WE DEAL WITH MORE DIFFICULT

PASSAGE JUSTIFICATIONS, SO

SOMETHING LIKE ATTACKS OF NEAR

DISEASE PRECIPITATED BY THE DIE

TEAR INDISCRETION A LOW CELL

DIET MAY HELP ALLEVIATING THE

SYMPTOMS.

SO QUESTIONS WHAT ALLEVIATES AN

ATTACK OR PRECIPITATES IT SO YOU

HAVE TO DO REASONING TO DECIDE

WHETHER IT'S JUSTIFYING OR NOT.

ANOTHER THING THAT OPENS UP IN

THE REAL CONTEXT IS YOU CAN

INTERACT WITH THE USER SO WATSON

COULD -- DOES IS ABLE TO SCORE

CONFIDENCE IN THESE

INTERPRETATIONS SO COULD ASK THE

USER I'M NOT SURE WHAT THE TWO

PHRASES MEAN DO THEY MEAN THE

SAME THING.

CONTRA INDICATE MEAN THE SAME AS

DO NOT USE?

BASED ON ALL THOSE NLP

TECHNIQUES WE CAN RUN THEM OVER

THE EMR, WE CAN TRY TO DO

DIAGNOSE KNOW WHICH IS WHICH

USES THE MATCHING, THE

EXTRACTION TOGETHER TO COME UP

WITH THE CORRECT DIAGNOSIS, ALSO

THINGS LIKE QUESTION ANSWERING.

SO YOU CAN USE THE EMR AND SAY

WHAT IS THE PATIENT ALLERGIC TO

OR MEDICATIONS USED FOR

NEUROPATHIC PAIN FOR THIS

PATIENT.

IS THERE ANY PAMELA HISTORY OF

HEART DISEASE SO YOU CAN PULL

RELEVANT FACTORS FOR PATIENT.

ANOTHER THING WE TRY TO DO IS

CREATE A FACTOR STRESSING TIME

LINE CONSTRUCTION TECHNIQUE

WE'LL SHOW YOU THE TIME LINE.

HERE ARE SYMPTOMS OF PATIENT HAS

HAD AND WHEN.

NOW, WE'RE THINKING ABOUT USING

WATT TON, THIS IS AN EARLIER

VERSION OF SYSTEM THAT WILL COME

BACK AND SAY HERE ARE OTHER

FACTORS I SAW IN THE

DESCRIPTIONS OF LAMB DISEASE,

CIRCULAR RASH, FATIGUE,

HEADACHE, WHAT WILL THE PATIENT

HAVE.

SO WE START THINKING AB WATSON

IN A FASHION MEANING WE'RE

CHANGING THE PARADIGM FROM BEING

QUESTION IN AND ANSWER OUT FROM

WATSON ITSELF GENERATE THE

QUESTIONS AND GET THOSE ANSWERS

THAT'S WHERE THE FOCUS OF THE

EFFORT IS NOW BOTH ON

IDENTIFYING WHERE THE IMPORTANT

MISSING INFORMATION, GAPS, AND

THEN ALSO WHAT'S A CONVENIENT

WAY, HOW DO WE SCORE THOSE, HOW

DO WE PRESENT THE USERS SO THEY

CAN ANSWER IN A USEFUL WAY FOR

WATSON.

SO TO SUM UP, GOING BEYOND

JEOPARDY WE'RE DEALING WITH MORE

COMPLICATED ARTIFACTS AND MUCH

MORE COMPLICATED REASONING

REQUIRED TO ANSWER THE

QUESTIONS.

THESE SIMPLE FACTOID O QUESTIONS

WE'RE DEALING WITH VERY LARGE AS

YOU KNOW EMRs.

WHERE WE LOOK AT INSTEAD OF

HAVING A QUESTION AND ANSWER HOW

DO WE INTERACT WITH WATSON, HOW

DOES WATSON PROPOSE CANDIDATES,

HOW TO IDENTIFY GAPS, HOW DOES

IT ASK YOU QUESTIONS, HOW DO YOU

ENCODE FOR WATSON.

ONE THING I HAVEN'T TALKED AS

MUCH ABOUT IS EXPLANATION.

WHEN WATSON COMES UP WITH AN

ANSWER IT CAN SCORE WELL HERE

ARE PIECES OF EVIDENCE THAT GAVE

ME EVIDENCE FOR THE ASPECTS OF

THE ANSWER.

I CAN TELL YOU WHY I THINK THE

CONDITION IS HARSH, WHY IT

INCREASES WITH INTENSITY, I CAN

SHOW YOU THE BEST PASSAGE FOR

THAT.

AND ASSESS HOUSE WATSON LEARNS.

WE TRAINED WATSON IN THE LAB,

CREATED A VERSION OF THE SYSTEM,

AND DEPLOYED ANYTIME THE

JEOPARDY CASE.

WATSON CAN LOOK IN THE RESPONSES

IT'S GETTING, USE THAT TO HELP

TUNE ALGORITHMS AND INCREASE

ACCURACY OVER TIME.

SO I THINK TO SUM UP WE'RE

WORKING ON THE FUNCTIONAL STAGE

AT THIS POINT AND HOPEFULLY

WE'LL GET TO SOMETHING THAT IS

EVENTUALLY USABLE AND EVEN

ENJOYABLE.

[APPLAUSE]

>> IF PANELIST ALSO COME UP.

YOUR DESCRIPTION OF WATSON MAKES

YOU THINK OF A LONG TIME

PREDECESSOR WITH (INDISCERNIBLE)

EXPLAINING EFFORTS TO HAVE

MACHINES DO WHAT PEOPLE DO.

HIS SIMPLE EXAMINATION WAS

AIRPLANES DON'T FLY AT THEIR

WINGS.

ALWAYS FOCUSES IT FOR ME.

SO WE'LL TAKE QUESTIONS AND IF

YOU HAVE ONE, IDENTIFY YOURSELF.

>> HI, THERE.

THANK YOU FOR A PHENOMENAL

PANEL.

FELL LIP RIZ NECK YOUTH OF

MARYLAND.

THIS IS -- PHILLIP REZNIK WITH

UNIVERSITY OF MARYLAND.

THESE TWO QUESTIONS, BUT I THINK

I CAN LINK THROUGH THE NOTION OF

INCENTIVES.

SO FROM NLP TO BE VALUABLE, IN

THIS CONTEXT NARRATIVE TEXT IS

IMPORTANT.

BECAUSE YOU HAVE TO HAVE

NARRATIVE TEXT TO OPERATE ON.

AND THE FUTURE OF NLP IS PUSHING

LIKE MANY OTHER THINGS IS MOVING

TOWARD BIG DATA BUT THERE ARE A

COUPLE OF THINGS GOING ON ON THE

INPUT SIDE THERE IS POTENTIAL

FOR THROWING NARRATIVE TEXT BABY

OUT WITH THE BATH WATER.

IN THE PUSH TO TRY TO ACCOMPLISH

STRUCTURED DATA FOR MEANINGFUL

USE.

ONE THING THAT'S VISIBLE HERE IN

MY EXPERIENCE AS WELL, THE

ACCESSIBILITY OF DATA TO DO

ESPECIALLY CLINICAL NLP SIDE,

YOU'RE SEEING THIS IN THE

INDUSTRY SIDE.

IN ACADEMIA AND OTHER FORMS OF

RESEARCH IT'S MUCH E LIMITED,

EXCEPT IN THE CONTEXT WHERE

SOMEBODY IS AFFILIATED WITH

ACADEMIC MEDICAL CENTER.

SO THE QUESTION IS, ONE, HOW DO

YOU INCENTIVIZE EHR VENDORS TO

RECOGNIZE AND PRESERVE

IMPORTANCE OF NARRATIVE TEXT?

AND HOW DO YOU INCENTIVIZE THE

FOLKS DOING THIS WORK AND

INDUSTRY TO BROADEN PERSPECTIVES

AN FIND MORE OF THE NATURAL

LANGUAGE PROCESSING IN THE

IMMUNITY THAN JUST PEOPLE IN

BIOINFORMATICS DPS OR ASSOCIATED

WITH ACADEMIC MEDICAL CENTERS?

>> I'LL TRY THE ONC PARK.

I DON'T KNOW WE WANT TO

INCENTIVIZE NLP PER SE.

WE WANT TO ALLOW FOR NLP PER SE

WHICH MEANS WE NEED TO BE

CAREFUL NOT TO DEFINE HOW THE

DATA GETS ENTERED SO WE DON'T

NEED TO SAY IT MUST BE

STRUCTURED WHEN IT COMES IN.

SO MUCH AS IT NEEDS TO BE

STRUCTURED AT SOME POINT.

SO IT COULD BE GOBBLEDEGOOK WHEN

IT COMES IN IF THAT'S HOW

PROVIDERS WANT TO PUT IT IN.

SO LONG AS THE SYSTEM PERHAPS

WITHIN NLP COULD CAUSE IT

UNGOBBLEDEGOOK.

THAT'S THE TECHNICAL TERM.

SO I'LL LOOK TO THE NLP EXPERTS

TO ANSWER THE SECOND HALF.

>> FROM THE PERSPECTIVE OF THE

PAYERS ALL I WOULD SAY IS WE

RECOGNIZE THE INCREMENTAL VALUE

OF NLP.

WHAT WE'RE STRUGGLING WITH IS

HOW DO YOU -- WHICH WE HAVE

TALKED ABOUT THE WHOLE DAY, HOW

DO YOU ACTUALLY GET IT INTO A

TERM -- INTO A FORM THAT WE CAN

USE.

WE HAVE THE INFRASTRUCTURE WE

BELIEVE TO BE ABLE TO USE IT.

WHERE WE'RE NOT QUITE THERE YET

AND HOPEFULLY WITH TECHNOLOGIES

LIKE WATSON BUT PERHAPS OTHERS

AS WELL, THIS WILL ALLOW US TO

DO THIS MORE EFFECTIVELY.

THE WILL IS ALREADY THERE AND

IT'S THERE -- PEOPLE ARE VOTING

WITH THEIR FEET IN WITH THEIR

WALLETS ULTIMATELY THE PEOPLE

PAYING FOR THESE SERVICES WANT

THE SERVICES TO BE GOOD, THEY

REALIZE THAT FOR THE SERVICE TO

BE GOOD, THE OUTPUT TO BE

ACCURATE, HAVING MORE DATA IS

BETTER.

SO THEY'RE ALREADY THERE

INTELLECTUALLY.

IT'S A MATTER OF THE MISSING

LINK IS GETTING FROM THE

UNSTRUCTURED DATA SUCH AS IT IS

TODAY TO A WAY WE CAN USE IT.

E   GLIF A FOLLOW-UP QUESTION ON

THE SAME TOPIC.

THE THE -- THIS F YOU LOOK AT

THE CERTIFICATION CRITERIA FOR

EHR, IT DOESN'T REALLY SAY THAT

YOU CAN USE NLP.

AND IF YOU HAVE SOME INDICATION

THAT YOU CAN USE NLP AND IT IS

BROGHT TO LIGHT AS PART OF THIS

CERTIFICATION CRITERIA, NOT ALL

THESE EHR EXCLUDED WILL NOT BE

GOING AROUND SAYING PHYSICIAN

SHOULD BE ENTERING INTO ALL

THESE WONDERFUL CHECK BOXES.

I KNOW YOU DON'T HAVE TO

EXCLUSIVELY SAY ANYTHING BUT AT

LEAST IF THE SPECIFICATION HAS

BIAS TOWARDS ALLOWING FREE TEXT

AND NARRATIVE TEXT, IT WILL MAKE

A BIG DIFFERENCE.

IF YOU GO AND LOOK AT THE MISSED

CRITERIA FOR USABILITY, IT

DOESN'T HAVE THE WONDERFUL

PICTURES YOU PUS PUT UP IN THE

-- AND IT REALLY READS LIKE THEY

WANT THESE THINGS TO BE ENTERED

ONE BY ONE AND WORRYING ABOUT

THE SAFETY OF IT AND THINGS LIKE

THAT.

SO NOTHING IN THE SPECIFICATION

FOR CERTIFICATION OF EHR REALLY

ALLOWS YOU TO -- IF YOU ARE AN

EHR VENDOR TO THINK OF THINGS TO

BE NARRATIVE TEXT.

THAT IS A FAILING I THINK ON THE

PART OF THE SPECIFICATION PART

OF IT.

AT LEAST THAT'S MY HUMBLE

OPINION.

>> NOTED.

STEVE WISEMAN DATA FORMATION.

MY QUESTION IS TO DR. GONDEK

WHERE YOU USE NATURAL LANGUAGE

PROCESSING IN THE LEARNING

SYSTEM DESCRIBED THIS MORN

SOMETHING

>> I THINK I SAID LALE BIT

DURING THE -- A LITTLE BIT

DURING THE TALK.

WE DON'T EXPECT NLP OR THE

WATSON SYSTEM TO HAVE THE DEEP

UNDERSTANDING A PRACTITIONER

WOULD HAVE UNDERSTANDING CONTEXT

TO REPRESENT DIFFERENT FACTORS.

WHAT WE THINK IT'S GOOD AT IS

RUNNING UP A LOT OF DATA, WE

THINK IT'S RICHER, HAS RICHER

MATCHING AND RICHER

UNDERSTANDING, SOMETHING LIKE

KEY WORD->U SEARCH.

WHERE IT GETS INTERESTING IS IF

YOU'RE SERVING FOR SOMETHING YOU

DON'T KNOW HOW IT'S REPRESENTED,

YOU DON'T WANT TO MISS IT SO

IT'S AN IMPORTANT QUESTION TO

YOU.

SO WE TALK ABOUT HIGH VALUE

QUESTIONS.

>> CAN I FOLLOW-UP WITH A

QUESTION, CAN YOU USE THE IBM

SYSTEM TO CODE MEDICAL RECORDS

USING UMLS, GIVEK THE CODES AND

IMBED THEM INTO MY MEDICAL

RECORD.

THE SYSTEM DOES CODING.

IT'S NOT OUR MAIN FOCUS BECAUSE

WE THINK OUR EXPERIENCE IT'S

DIFFICULT AND YOU NEVER MATCH IT

EXACTLY WHEN YOU NEED WHEN

YOU'RE USING IT SO EXAMPLES

TODAY WERE THAT YOU HAVE TO

TRIAGE, YOU HAVE TO PICK WHAT

ARE THE 1400 CODES YOU'RE

WILLING TO IMPLEMENT.

SO PART OF THE WATSON VIEW, I

THINK IS A LONGER TERM PICTURE,

CAN WE GET TO THE POINT WHERE

THE SYSTEM CAN MAKE THESE

COERCED DO THESE TWO THINGS MEAN

THE SAME THING, MAYBE NOT A

BINARY DECISION BUT A RICHER

REPRESENTATION TRY TO UNDERSTAND

SOME OF THE SAME SUBTLETIES

HUMANS DOING WHEN READING THE

TEXT.

I HAVE ANOTHER QUESTION ABOUT

WATSON.

VERY NICE PRESENTATION BY THE

WAY.

IN GENERAL SEEMS HUMANS HAVE A

HARD TIME TO DEAL WITH

PROBABILITIES.

WONDERING HOW YOU PRESENT

ANSWERS TO PHYSICIANS ANSWERING

QUESTIONS, DO YOU PICK THE BEST

T HIGHEST PROBABILITY OR GIVE

THEM ALL THE INFORMATION AND

THEN FIGURE HOW THEY DEAL WITH

THIS INFORMATION WITH THESE

PROBABILITIES.

>> SOMETHING WE HAVEN'T -- WE

DON'T HAVE A USER INTERFACE OR

CDS SYSTEM AT ALL.

OUR EXPERTISE IS ON THE NLP SIDE

AND WE'RE TRYING TO BRIDGE TO

THAT.

I THINK YOU'RE RIGHT YOU LOOK

HOW WATSON POPULATES ITS

KNOWLEDGE BASES WHEN IT DOES AND

WE USE STATISTICAL TECHNIQUES

BASED ON THOUSANDS OF PASSAGES.

YOU CAN'T SHOW THE DOCTOR

THOUSANDS OF PASSAGES, PROBABLY

NOT GOING TO PUT UP WITH THAT.

ON THE OTHER HAND, IT'S ALSO AN

ASSET IN THAT WHEREAS TYPICALLY

KNOWLEDGE BASE IS GOING TO HAVE

A BINARY RULE, AND WE HAVE

PROBABILITIES ASSOCIATED WITH

THESE, WE KNOW WHAT'S COMMON, WE

KNOW WHEN IT WAS SET, WE KNOW

WHAT SOURCES WAS SET.

ONE THING WE'RE LOOKING AT

DOING, IF I GIVE YOU A FACT, I

SAY THAT THIS IS A TREATMENT FOR

THIS, CAN WATSON FIND THE BEST

PASSAGE JUSTIFYING THAT FACT.

WHAT THAT MEANS IS EVEN IF

WATSON IS TO A COMPLEX

STATISTICAL TECHNIQUE IT CAN

RETRIEVE A PASSAGE WHICH SHOULD

IDEALLY BE CONVINCING TO THE

USER.

THAT REQUIRES SETTING UP

RELIABLE SOURCE IS IT TIMELY AND

SO FORTH.

IT'S SOMETHING WE HAVEN'T IN

JUST ANSWERING QUESTIONS YOU

DON'T DEAL WITH THAT, YOU JUST

CARE IF THE ANSWER IS CORRECT OR

NOT.

BUT WE'RE EXPANDING THE

FRAMEWORK TO EVALUATE IS THIS

EVIDENCE RELIABLE OR NOT, IS

THIS THE BEST PASSAGE WE COULD

HAVE SHOWN.

>> LOVE THE PRESENTATIONS.

THE WHOLE PANEL.

MY QUESTION THOUGH IS FOR MARC

AND DAVID.

I WONDER AT THE EDGES OF

REASONING BETWEEN THE NLP

METHODS, STATISTICAL METHODS,

HOW WILL WE GET AT THE DEEPER

REASONING METHODS LIKE ANATOMIC

REASONING OR PATHOPHYSIOLOGIC

REASONING OR OTHER FORMS OF

SYSTEMATIC REASONING?

EITHER SEMANTIC MODELING UNDER

THE RECORD OR KNOWLEDGE MODELING

UNDERNEATH WATSON?

>> I THINK THE MODEL THAT SEEMED

TO BE MERGING ACROSS THE BOARD

ARE THESE LAYERED KIND OF

APPROACHES OR ENSEMBLE

APPROACHES WHERE THERE'S NO ONE

OF THEM BUT YOU HAVE TO COMBINE

THOSE AND SOMETIMES IT RATE

THROUGH THEM SO YOU GET O TO

SOME KIND OF STOPPING ROLE.

SO I THINK AS THE -- WHEN WE

WILL GET TO THOSE IS WHEN WE

HAVE GOTTEN BASICS THAT ARE

BAKED SO THEN THOSE BECOME MORE

USEFUL AS A SUPPLEMENT TO THOSE.

>> I APPRECIATE MARC (INAUDIBLE)

TALK BECAUSE OF IN DEPTH MODELS

BILLING TO DEAL WITH

INTERACTIONS BETWEEN THE MODELS.

I SEE THAT WORK APPROACHING IT

FROM A RULE BASED BACKGROUND

MAKING THOSE BASES RICHER, MORE

LOOKING AT JOINTS BETWEEN THEM,

HOW THEY ENTERACT, WHEREAS I SEE

OUR WORK IS COMING WHERE WE DONE

HAVE THE RICH MODELS.

THIS IS A CONSCIOUS CHOICE WE

MADE SO WE TOOK THE DOCTORS --

WE COULD HAVE SAID WE'RE NOT

GOING TO DO DIAGNOSIS OF A

SPECIFIC CONDITION AND THEN WE

TAKE A VERY DIFFERENT APPROACH.

WE WOULD HAVE BUILT MODELS USE

MACHINE LEARNING, SO FORTH.

THE FACT THAT WE FOR THE TEAM WE

DECIDED WE'LL TAKE THE GENERAL

TASK MEANS THAT THE TYPE OF

TECHNIQUES WE'RE DEVELOPING ARE

DIFFERENT WHICH ARE MUCH MORE

BASED ON TRYING TO INTERPRET

THOSE MODELS FROM THE LANGUAGE

ON THE PASSAGE SIDE.  THERE'S NO

WAY IT CAN BE HUMAN AT THIS

POINT.

SO I THINK I WOULDN'T LOOK TO

WATSON TO BE DOING THAT DEEP

HAVING LARGE DECISION TREES

SORTS OF REASONING, RATHER LOOK

AT MORE FOR COVERAGE AND SEE IF

SHALLOW REASONING IS SMARTER

OVER TIME.

>> IT'S INTERESTING TO USE THE

PROBLEM LISTIC REASONING AS

WELL.

IS THERE A UTILITY FUNCTIONED

CONSIDERED IN TERMS OF GUIDING

SEARCH OR GUIDING QUESTION AND

ANSWERING?

SEEMS LIKE YOU MIGHT BE ABLE TO

PRIORITIZE CERTAIN DIRECTIONS

AND PATHS WITH EITHER A SINGLE

ATTRIBUTE OR MULTI-ATTRIBUTE

UTILITY MODEL.

>> DEFINITELY.

ONE THING WE DO WITH A QUESTION

IS BREAK IT UP INTO FACTORS

BASED ON SIN TACTIC PARTS AND

IDENTIFIES.

AND WE LEARN FROM DATA WHICH ARE

THE IMPORTANT FACTORS TO ANSWER.

SO MAYBE IF YOU HAVE -- YOU HAVE

SOME SYMPTOM THAT'S NOT VERY

SPECIFIC YOU WOULDN'T BE USING

THAT.

OR NOT SENSITIVE YOU WOULDN'T BE

USING THAT.

NOW, WE HAVE THOSE TECHNIQUES

IDENTIFY WHAT ARE THE IMPORTANT

PARTS OF A PASSAGE WHERE THE KEY

TERMS HOW IMPORTANT ARE THEY FOR

ANSWERING QUESTIONS.

THE PROBABLYISTIC -- ONE THING

THAT HAPPENS IN A PROBABLISTIC

COMPUTATION IS THAT IT TAKES

DIFFERENT INTERPRETATIONS OF THE

QUESTION.

SO WE CAN SAY WE INTERPRET THIS

AS A (INAUDIBLE) RELATION OR

INTERPRET AS A CONTRA INDICATES

RELATION AND SEE WHAT HAPPENS SO

IT ALLOWS YOU TO EXPLORE

DIFFERENT INTERPRETATIONS AND

THEN IN THE JEOPARDY CASE WE

DIDN'T GET A CHANCE TO REVISE

THE ANSWER, IT WAS BASICALLY

QUESTION IN ANSWER OUT.

BECAUSE OF THE PROBABILITIES YOU

CAN SAY HERE WHAT I THINK ARE

THE IMPORTANT FACTORS, HERE ARE

WHAT I THINK ARE THE IMPORTANT

RELATIONS AND A USER CAN SAY NO,

I DISAGREE WITH THAT, THAT'S NOT

AS IMPORTANT AS YOU'RE WEIGHING

IT.

>>

ONE NEED THING FROM MY

PERSPECTIVE ABOUT WATSON IS IT

BUILT ON THE SHOULDERS OF A

BUNCH OF GIANTS.

ONE OF THOSE GIANTS IS NLP AND

40 OR 50 YEARS OF RESEARCH ON

THAT.

ANOTHER IS ABOUT FIVE OR SIX

INTELLIGENCE COMMUNITY ADVANCED

RESEARCH PROJECTS THAT FUNNELED

MILLIONS OF DOLLARS TO DEVELOP A

TECHNOLOGY BEHIND EXTRACTION

QUESTION ANSWERING.

COUPLED WITH THE VALUATIONS THAT

SHOWED HOW YOU CAN TELL IF THOSE

THINGS WORKING AND MEASURE THE

PROGRESS IN THEM.

IT SEEMS TO ME, I'M NOT A

DOOKTOR OR NOT THAT KIND OF

DOCTOR, THAT THE COMMUNITY IN

THIS ROOM DOESN'T SEE HOW

CLINICAL DECISION SUPPORT MAKES

THE SAME LEAPS, GETS THE SAME

WAY FORWARD, EXCEPT FROM THE

COMPANIES SUCH AS SIEMENS OR

AETNA, WHO HAVE BUDGETS AND

RESEARCH TEAMS AN ACCESS TO HUGE

AMOUNTS OF DATA.

I WONDER WHAT IS THE RESEARCH

PROGRAM THAT GETS CDS TO WHERE

WATSON IS.

>> I'LL TAKE A STAB AT THAT.

WE HEARD ABOUT GRAND CHALLENGES

FROM OTHER SPEAKERS.

IT SEEMS TO ME THERE'S A COUPLE

OF MAJOR PIECES AN BOB HAD HIS

TEN AREAS OR WHATEVER, ONE AREA

OF RESEARCH IS CLEARLY ON THE

HUMAN COMPUTER INTERACTION

ASPECT WHICH IS A HUGE

CHALLENGE, HOW TO DELIVER THIS

IN A WAY THAT IS NOT

INTERRUPTIVE, WHETHER GPS OR

HOWEVER UP TO CONCEPTUALIZE IT,

THAT IS AN AREA THAT CLEARLY IS

UNSOLVED PROBLEM.

THE SECOND UNSOLVED PROBLEM IS

RELATED TO A LOT OF WHAT WE HAVE

BEEN TALKING ABOUT THIS

AFTERNOON, HOW DO YOU REPRESENT

THE KNOWLEDGE OR ROLES OR

WHATEVER IT ENDS UP BEING IN A

WAY THAT ARE EDITABLE AND

CREATABLE.

WHICH IS A REAL CHALLENGE.

EVEN IF WE COME UP WITH THIS

REALLY COOL WAY TO DO IT IT WILL

TAKE TEN YEARS TO TRAIN UP A

CADRE OF PEOPLE WHO UNDERSTAND

IT CONCEPTUALLY HAVE THE

CLINICAL KNOWLEDGE AN TECH NO

LOGIC TRANSLATION TECH NO LOGIC

TRANSLATION.

TECH TECH LODGE

I DON'T KNOW THAT WE HAVE THE

HUMAN AND PROCESS ENGINEERING

SOLVED ABOUT HOW TO DO IT.

THE THIRD THING I THINK TECHNO

LOGIC.

IT IS THE THIR LINE OF RESEARCH

IS HOW TO INTEGRATE THINGS INTO

WORK FLOW AND PROCESS AN HIGH

RELIABILITY WAY.

WE LEARNED A LOT ABOUT SOME OF

THE AREAS, HOW DO YOU MAKE THAT

SING IN A COLLABORATIVE

MULTI-USER, BOB ALLUDED TO THIS

A BIT, ENVIRONMENT.

YOU DID ALSO AS YOU TALK ABOUT

HOW YOU MAKE THAT FIT INTO AN

ENVIRONMENT WHERE IT'S NOT JUST

ONE PERSON YOU'RE SUPPORTING BUT

A TEAM AND COLLABORATIVE THAT

YOU'RE SUPPORTING.

SEEMS TO ME THOSE WERE THREE BIG

AREAS.

AND ANY ONE OF THOSE CAN HAPPEN

INDEPENDENTLY THEY DON'T MOVE

TOGETHER TO GET PROGRESS.

THOSE ARE THINGS THAT I WOULD --

IF I WERE ADVISING SOMEBODY WHAT

RESEARCH PROGRAM TO ADVANCE CVS,

THOSE WOULD BE THE THREE BIG

ONES.

>> I WOULD ADD I THINK YOU WERE

REALLY GETTING TO THE FUNDING

ISSUE, I THINK.

THAT WAS AT LEAST PART OF THE

QUESTION, RIGHT?

>> MY BIG HEDGE IS DATA BUT

SOMETHING WE CAN TALK ABOUT.

AT LEAST THROUGH THE FUNDING,

THE PUBLIC PRIVATE PARTNERSHIP

ROOT IS REALLY WHERE I BELIEVE

THIS IS GOING TO HAVE TO GO AND

IS ALREADY MOVING.

AS AN EXAMPLE, AETNA IS INVOLVED

IN THE MANY SENTINEL PROJECT

ALONG WITH A NUMBER OF ACADEMIC

INSTITUTIONS AND IT'S THAT KIND

OF PUBLIC, PRIVATE PARTNERSHIP

WHERE THERE'S FUNDING FROM

GOVERNMENT, FUNDING FROM

INDUSTRY, THIS IS GOING TO SOUND

LIKE MOM AND APPLE PIE BUT

TOWARDS THE GREATER GOOD THEN I

THINK THAT'S GOING TO BE THE END

OF IT.

>> THIS IS PROBABLY BE OUR LAST

QUESTION.

>> I GUESS THE OTHER THING IS TO

CONSIDER THE PERSPECTIVE HOW

BEST TO LEVERAGE BECAUSE IT CAN

BE AN AMAZING TOOL.

QUESTION IS WHAT'S THE BEST

APPLICATION.

IF WE KNOW 93% ACCURACY THAT IT

RELATES TO BILLING WHICH IS

BETTER THAN MOST CODERS AT

TIMES, IS THAT REALLY GOOD FOR

CODING?

AND ALSO BILLING BUT THEN ALSO

FOR REPORTING FOR EXAMPLE,

QUALITY REPORTS.

YOU START TALKING 93% ACCURACY

AND DOING CLINICAL DECISION

SUPPORT WITH THEm/v ALERTED

INTERMEDIARY, NOW MAKING MEDICAL

DECISIONS YOU MAY NOW SWAY

BECAUSE IN GENERAL WE DON'T

TOLERATE 93% IF THERE'S SOME --

IF IT'S A DEVICE.

SO AGAIN, I THINK HOW DO WE

APPLY IT, HOW DO WE LEVERAGE IT,

HOW DO WE UNDERSTAND THE

BENEFITS AND RISKS, I THINK IS

AN IMPORTANT PART OF IT TOO.

>> TO BE FAIR I THINK FIRST OF

ALL WE WOULD NEVER SAY THAT WE

DO CLINICAL DECISION SUPPORT.

WE ARE PROVIDING INFORMATION TO

A PROVIDER.

HE OR SHE MAKES A DECISION BASED

ON THEIR KNOWLEDGE AND JUDGMENT

WHETHER OR NOT THEY WANT TO

PROCEED.

WE SAY LOOK, WE HAVE ACCESS TO

INFORMATION WHETHER ON THE DATA

SIDE, KNOWLEDGE SIDE OR BOTH

THAT YOU MAY NOT HAVE.

HERE FOR YOU TO FIGURE OUT

WHETHER YOU WANT TO GO FORWARD

OR NOT.

AS ONE DOCTOR CALLED US UP AND

SAID LOOK, I'LL LISTEN TO YOU IF

YOU CAN TELL ME THE COLOR OF THE

O (INAUDIBLE) MUTATION BEFORE

ME.

WE CAN'T DO THAT.

>> ALTHOUGH I WOULD ARGUE THAT

THERE ARE EHR SYSTEMS, NOT

ANYONE AT THE TABLE HERE, THAT

ACTUALLY DO A LITTLE BIT MORE

THAN THAT.

SO YOU MAY IN A HOSPITAL SETTING

HAVE IF YOU'RE FAMILIAR WITH THE

WAY SYSTEMS WORK IN HOSPITALS

THERE'S DEPENDENT AND

INDEPENDENT PROTOCOLS.

INDEPENDENT PROTOCOL IS

SOMETHING THAT A NURSE OR THIS

SYSTEM CAN DO.

SO IF A PATIENT MEETS CERTAIN

CRITERIA WITHOUT AN ORDER FROM A

CLINICIAN, THE NURSE CAN

ACTUALLY DO SOMETHING.

I HAVE SEEN SYSTEMS THAT

ACTUALLY DO THAT WORK.

SO A PATIENT WILL HAVE A CERTAIN

DIAGNOSIS, BY SAY DIABETES, THE

DIABETIC ORDER OR MEAL PLAN GETS

AUTOMATICALLY ORDERED WITH NO

HUMAN INTERVENTION SO WHERE DO

THEY DRAW THE LINE BETWEEN ORDER

FOR DIABETIC MEAL AND THE KEY P

ARC YOU ADMINISTERTOR THE

PATIENTS MI.

CERTAINLY THERE IS A LINE

SOMEWHERE.

SO SOMETIMES DECISION SUPPORT

DISCUSSION EARLIER ABOUT SOMEONE

REGULATING THIS IN SOME WAY.

I WON'T NAME THE THREE LETTER

AGENCY BUT THIS IS IMPORTANT AND

DEEP INTEREST TO US AND IN FACT

THE PUBLIC.

SO I'LL ALSO PITCH SOMETHING

THAT I NEGLECTED EARLIER WHEN I

SAY NOTED.

I DID NOT OFFICIALLY NOTE THAT.

WE WERE IN THE RULE MAKING

PROCESS WHICH MEANS OUR PROPOSED

RULES ARE PUBLISHED AND FINAL

RULES HAVE NOT.

THE COMMENT PERIOD IS OPEN SO IF

ANYONE HAS OPINIONS HOW,NC OR

CMS SHOULD ENHANCE THE

LIKELIHOOD NLP IS INCORP RAYED

-- INCORPORATED, GO TO

REGULATIONS.GOV BY MAY 7 AND LET

US KNOW.

>> NOTED.

WE'RE RUNNING OVER ON TIME SO IF

YOU HAVE ANY QUESTIONS OR

COMMENTS CATCH THE PANELIST ON

THE BREAK.

SO THANK YOU VERY MUCH FOR YOUR

TIME.

[APPLAUSE]

SO THIS WILL BE THE LAST PANEL

FOR THIS WORKSHOP.

AND IN THIS PANEL WILL BE HAVING

ONE SPEAKER AS KEYNOTE, AND THEN

WE'LL HAVE A SESSION WHERE ALL

THE SPEAKERS OF THE DAY ARE

GOING TO SIT TOGETHER AND PEOPLE

CAN ASK QUESTIONS.

SO THIS PANEL SABT FUTURE

CHALLENGES AND OPPORTUNITIES FOR

NATURAL LANGUAGE PROCESSING AND

CLINICAL DECISION SUPPORT.

AND SPEAKER THE KEYNOTE SPEAKER

IS DR. JOHN WHITE, HEAD OF

INFORMATION TECHNOLOGY SECTION.

AS YOU CAN SEE HERE FOR AHRQ.

HE DID TRAINING IN MEDICINE AT

THE UNIVERSITY OF VIRGINIA AND

ALSO LANCASTER GENERAL, IN

PENNSYLVANIA.

HE HAS WON THE NATIONAL AAFV

AWARD FOR EXCELLENCE IN GRADUATE

EDUCATION AND ALSO ATTRACTING A

LOT ORNC AN CMS.

AND OHIO

[APPLAUSE]

>> TIEW SO MUCH.

THANK YOU FOR STICKING THROUGH

TO THE END OF THE DAY.

IT'S BEEN QUITE A DAY.

THERE'S A LOT OF GREAT

DISCUSSION.

I PROMISE YOU,LY NOT OVERLY

BURDEN YOU WITH THOUGHTS.

THIS IS THE END OF THE DAY AND

MEANT TO PULL TOGETHER AND TRY

TO LOOK A LITTLE BIT FURTHER

AHEAD.

I DO WANT TO SAY THAT THIS IS A

GREAT HONOR TO BE THE -- THIS IS

THE FIRST OPPORTUNITY TO SPEAK

AT LISTER HILL SO THANK YOU FOR

BEING WILLING TO LISTEN.

SO SO PREPARING FOR THIS TALK IS

A LOT OF FUN.

BECAUSE IT GOT ME THINKING ABOUT

MY LANGUAGE AND WAYS TO

INTERPRET IT.

GOING TO BE TO USE LANGUAGE.

SO USE OTHER PEOPLE'S LANGUAGE

BECAUSE MY LANGUAGE IS TERRIBLE.

SO YOU WILL SEE SPRINKLED

THROUGH HERE QUOTES ABOUT

LANGUAGE THAT I HOPE WILL GUIDE

US ON OUR TALK.

SO O I WAS MOST EXCITED ABOUT

THIS QUOTE.

THIS IS GAYLEN 2,000 YEARS AGO

TALKING LANGUAGE.

I THOUGHT MY WORD.

THE CHIEF MERIT OF LANGUAGE IS

CLEARNESS AND WE KNOW THAT

NOTHING DETRACTS SO MUCH FROM

THIS AS DO UNFAMILIAR TERMS.

THERE'S NOTHING NEW UP IN THE

SUN IF GAIL TALKED ABOUT THIS

2000 YEARS AGO.

IN TERMS OF THINKING ABOUT

CHALLENGES, RESEARCH

OPPORTUNITIES AND ALL THIS, I

TRIED TO FRAME IN MY HEAD, I

COME FROM AHRQ.

TO TALK A LITTLE BIT ABOUT WHERE

WE'RE TRYING TO GET TO, I WILL

NOT OVERLY BURDEN YOU WITH THIS,

NATIONAL QUALITY STRATEGY AND

HEALTHCARE QUALITY REPORTS.

WE TALKED ABOUT QUALITY FOR A

LONG PERIOD OF TIME, THE TOPIC

OF QUALITY IS (INAUDIBLE)

SIGNIFICANTLY.

PRIOR TO ABOUT TWO YEARS AGO WE

DIDN'T HAVE ONE SOLID THING WE

COULD SAY THIS IS WHAT WE'RE

TRYING TO DO, THIS IS WHERE

WE'RE GETTING AT, MOVES TO A

PATIENT FOCUSED APPROACH, IT CAN

B THE WHOLE THING YOU CAN FIND A

LINK AT THE BOTTOM.

THERE ARE THREE AIMS SIX

PRIORITY AND TEN LEVERS TO GET

THERE.

THE THREE AIMS YOU HER OF,

BETTER CARE, OVERALL QUALITY,

HEALTHY PEOPLE HEALTHY

COMMUNITIES AN THIRD IS

AFFORDABLE CARE.

WE CAN GET DEEP INTO THOSE IF

YOU WANT TO.

BUT THAT WILL MAKE EVERYBODY

LIKE MOM AND APPLE EYE AS ONE

PREVIOUS SPEAKER TALKED ABOUT.

UNDERLYING THOSE THEME HOUSE DO

YOU PRIORITIZE WITHIN THAT WHAT

TO DO AND THEY SAID HERE ARE SIX

PRIORITIES.

NOT WORD FOR WORD BUT POINT

THESE OUT BECAUSE THEY COME UP

LATER IN QUALITY AND DISPARITIES

REPORT REDUCING HARM, PATIENT

ENGAGEMENT, EFFECTIVE

COMMUNICATION, COORDINATION

TREATMENT AND PRACTICES, WIDE

USE OF BEST PRACTICES AND

QUALITY CARE MORE AFFORDABLE.

MAKES SENSE.

HOW DO YOU PRIORITIZE.

SO I WON'T GET TO THE TEN LEVER

BUSILY MENTION THAT HEALTH IT IN

PARTICULAR IS SPECIFICALLY

MENTIONED AS ONE OF THE

DEVELOPERS WE'RE SUPPOSED TO USE

TO MOVE QUALITY AHEAD.

WHY DO WE DO THIS?

TO IMPROVE QUALITY.

WHICH IS BETTER CARE, HELPING

PEOPLE HEALTHY COMMUNITY, SO ON.

SO THIS IS SOMETHING AHRQ DOES

EVERY YEAR, MOST RECENT ADDITION

MOST RECENT ADDITIONS WERE

RELEASED FRIDAY.

NATIONAL HEALTHCARE QUALITY

REPORT AND DISPAIR RAJ REPORT

SENT TO CONGRESS AS PART OF THE

AUTHORIZED LEGISLATION.

OVERALL QUALITY IMPROVE VRS

SLOWLY, A LITTLE AT A TIME.

NOTABLY THIS YEAR WE NOTED A

DISPARITIES REPORT THAT ACCESS

TO HEALTHCARE IS NOT IMPROVING

FOR MOST GROUPS AND IN SOME

PLACES MOVING BACKWARDS THAT'S

TAKING AWAY FOR YOU.

ON AVERAGE MOST PEOPLE GET MOST

OF THE CARE MOST OF THE TIME.

CARDIAC CARE IS A BRIGHT SPOT

PLACE WHERE WE HAVE BEEN MAKING

SOME SIGNIFICANT GAINS.

IF YOU CARE TO LOOK AT THE

NATIONAL HEALTHCARE QUALITY

REPORT, THERE'S ADOPTION OF EHR

WHICH IS SOMETHING WE'RE DOING

AS PART OF OUR ORGANIZATION HAS

GOOD INFORMATION TOOLS THAT HAS

HIGH QUALITY CARE.

AND AGAIN, I WANT TO POINT OUT

SIX PRIORITIES.

THE QUALITY REPORT IS GOING TO

BE ALIGNED WITH THOSE.

SO IT'S ALL GOOD AND WELL TO

TALK ABOUT THE STRATEGY AGAIN

YOU CAN'T PROVE WHICH MEASURES

SO WE'LL MEASURE RELEVANT TO

THAT.

SO TOLD YOU I WASN'T GOING TO

OVERLY BURDEN YOU BUT THAT'S

WHAT YOU'RE -- WE TALK ABOUT

QUALITY AND DIFFERENT COMPONENTS

AND STUFF LIKE THAT, SOMETIMES

(INAUDIBLE) I SAY PERFECT, IT IS

THE GREAT HISTORY OF LIFE ITSELF

WHICH IS AT THE BOTTOM OF ALL

MYSTERIOUS LANGUAGE WHERE IT

LIES TO EMPLOY CONCERNING IT

THAT'S WHAT I THINK ABOUT WHEN I

TALK ABOUT QUALITY, (INAUDIBLE)

AT THE BOTTOM OF THIS.

THOSE ARE THE CHALLENGES.

RESEARCH OPPORTUNITIES.

I DON'T KNOW IF THERE ARE ANY IN

THE AUDIENCE LOOKING FOR

RESEARCH OPPORTUNITIES BUT IF

THERE ARE LET US TALK ABOUT A

FEW OF THEM.

I'M PLEASED TO BE ABLE TO IT

WILL YOU THAT TODAY ON THE

WEBSITE THERE'S NEW EVIDENCE

REPORTS SYSTEMATIC ANALYSIS,

LONG TERM META ANALYSIS ON

IMPACT OF CLINICAL DECISION

SUPPORT SYSTEMS.

DONE TO UNIVERSITY NUMBER OF

FOLKS IN THE AUDIENCE

PARTICIPATED AND EXPERT PANEL TO

GUIDE THE WORK.

ARTICLES PUBLISHED YET ON LINE

IN ANNULS OF CONTROLLED MEDICINE

AT ANNULS.ORG AND BASICALLY

FINDS THAT AFTER 150 STUDIES

THROUGHOUT THERE, CLINICAL

DECISION SUPPORT IS SHOWN

THROUGH PROCESS MEASURES NOT

JUST ACADEMIC CENTERS PREVIOUSLY

BUT ACROSS A VARIETY OF

DIFFERENT SYSTEMS AND VARIETY OF

SETTINGS.

THIS IS GOOD, THIS IS THE FIRST

TIME TO LOOK AND SAY THIS DOES

MAKE US BETTER.

WHERE THERE IS LIMITED EVIDENCE,

IS IN TERMS OF CLINICAL

OUTCOMES.

ECONOMIC OUTCOMES.

WORK FLOW THAT'S ELECTRONIC,

SOCIOECONOMIC AND OTHER OUTCOMES

SO GETTING ALL THE WAY DOWN TO

LIVING LONGER, SUFFERING LESS,

THAT OUTCOME OR GETTING BETTER

VALUE FOR YOUR DOLLAR, THOSE ARE

NOT THERE YET.

DOESN'T SAY ONE WAY OR THE

OTHER, DOESN'T SAY IT DOESN'T

IMPROVE BUT THERE'S NOT ENOUGH

EVIDENCE ABOUT THAT.

SO THERE IS RESEARCH OPPORTUNITY

FOR YOU.

TOW TAKE A LOOK AT DECISION

SUPPORT AND HOW IT AFFECTS

CLINICAL OUTCOMES, ECONOMIC

OUTCOMES AND WORK FLOW AND

BASICALLY THERE'S SEVERAL

FEATURES OF IMPLEMENTATION THAT

ARE IDENTIFIED AS LEADING TO

IMPROVED IMPACT OF CLINICAL

DECISION SUPPORT.

SO ART.GOV OR ANNULS.GOV THERE'S

A NICE ARTICLE ABOUT IT.

SO THAT'S ONE OPPORTUNITY.

SO I HAVE THROWN UP A LOT OF --

THERE'S MORE NURTURING PEOPLE UP

THERE BUT YOU HAVE TO A LITTLE

POP BUSINESS PSYCHOLOGY IN HERE

TOO.

THERE IS A LOT OF GOOD THINGS TO

RECOMMEND IN THE INNOVATORS

PRESCRIPTION WHICH IS A

(INAUDIBLE) BUT THIS ONE IS THE

GRAVEYARD OF FAILED PRODUCTS AN

SERVICES POPULATED BY THING

PEOPLE SHOULDN'T HAVE WANTED.

UNDERSTANDING THE JOB

CUSTOMERRINGRY TRYING TO DO IS A

MAJOR ISSUE AT EVERY HEALTHCARE

INNOVATION.

HE TALKS ABOUT A MILK SHAKE AN

WHEN HE WHEN YOU BUY IT WHAT JOB

ARE YOU TRYING TO GET IT TO DO?

LET US NOT GO DOWN THAT ROAD BUT

WHEN WE TALK ABOUT DECISION

SUPPORT, THIS IS GREAT, WILL

CHANGE THE WORLD AND ONLY PEOPLE

USE IT, WHY AREN'T THEY USING

DECISION SUPPORT?

THERE MAYBE A REASON WHY, AND IT

MAY PROBABLY RELATED TO THE JOB

THEY'RE TRYING TO DO AND HOW IT

HELPS THEM DO THAT JOB.

LET'S TALK ABOUT ANOTHER FUNDING

OPPORTUNITY AND ANOTHER

OPPORTUNITY FOR FUTURE RESEARCH.

FUNDING OPPORTUNITIES WE PUT OUT

A LITTLE OVER ALMOST A YEAR AGO

BUT REALLY GOOD.

A LITTLE MORE BASIC SCIENCE USED

TO WRITE (INAUDIBLE) APPLIED

STUFF, DEMONSTRATION STUFF,

IMPROVING QUALITY, THESE ARE

OPPORTUNITIES TO DEFINE WHAT'S

THAT JOB.

RIGHT?

UNDERSTANDING THE JOB THAT THE

CUSTOMERS, WE TALK ABOUT

CUSTOMERS I'M NOT JUST TALKING

ABOUT THE DOCTORS.

BUT I'M TALKING ABOUT EVERYBODY

IN HEALTHCARE.

YOU HEARD THE FOLKS THAT HAVE

GONE BEFORE ME BUT TWO FUNDING

OPPORTUNITIES, UNDERSTANDING

CLINICAL INFORMATION NEEDS AND

HEALTHCARE DECISION MAKING

PROCESSES IN THE CONTEXT OF

HEALTH IT, NOT I HAVE A WIDGET

AND I WANT TO DO SOMETHING WITH

THE WIDGET.

UNDERSTANDING THE INFORMATION

NEEDS, DECISION MAKING PROCESS

WE HAVE ACCURATE INFORMATION

SYSTEMS HOW DOES THAT CHANGE OUR

JOB?

AS CLINICS?

AS PATIENTS?

AS CAREGIVERS.

RIGHT?

WHEN YOUR MOM CALLS AND SAYS I

HAD THIS WEIRD TEST RESULT HOW

DID THAT CHANGE BECAUSE OF THE

INFORMATION TOOLS AND SYSTEMS

THAT ARE AVAILABLE TO YOU?

SO THOSE TRUNTS OUT THERE, THEY

HAVE BEEN A WHILE AND WE HOPE

PEOPLE WILL FIND GOOD

OPPORTUNITIES FOR THAT.

THE BEST PART IS DERIEFD FROM

THE REFREX OF THE ACTS OF THE

MIND ITSELF, SO THESE WORDS

WE'RE TRYING TO PROCESS.

I'LL GET TO THE ISSUE WHETHER OR

NOT IT'S NATURAL LANGUAGE

BECAUSE I DON'T THINK IT IS.

WORDS ARE THOUGHTS MAY MANIFEST

IN DIFFERENT WAYS SO WHEN WE

MAKE THOSE THOUGHTS MANIFEST ARE

WE DOING THEM IN A WAY TO HELP

SUPPORT DECISIONS.

ANOTHER OPPORTUNITY FOR PEOPLE

TO THINK ABOUT HOW TO DO THIS.

THE IOM ISSUED A REPORT IN THE

LATE 2011 CALLED CLINICAL

GUIDELINES WE CAN TRUST SOME

COLLEAGUES WORKED ON THE REPORT

TWO RECOMMENDATIONS THAT ARE KEY

IN THERE.

FOR THE COMMUNITY.

GUIDELINES SHOULD STRUCTURE

FORMAT AND CONTENT OF CLINICAL

PRACTICE GUIDELINES TO REGULATE

CDS, THIS IS NOT I'M GOING TO

TAKE LANGUAGE AND (INAUDIBLE)

PROCESS THE HELL OUT OF IT.

WE NEED TO TAKE THE LANGUAGE

WE'RE USING AND WE NEED TO

STRUCTURE IT WITHOUT -- I'M NOT

GOING TO BEAT ON THAT ANY MORE.

YOU HEARD THAT HERE BUT THE IOM

IS SAYING THIS IS SOMETHING WE

OUGHT TO BE DOING.

THE SECOND RECOMMENDATION ALSO

KEY, GUIDELINE DEVELOPERS,

GUIDELINE IMPLEMENTERS AN

DECISION SUPPORT DESIGNERS

SHOULD COLLABORATE IN AN EFFORT

TO ALIGN THEIR NEEDS WITH ONE

ANOTHER BECAUSE BUT DON'T -- NOT

AS MUCH AS YOU SHOULD.

THERE WAS SOME BUT AS WE TRY TO

TAKE CLINICAL KNOWLEDGE AND

TRANSLATE INTO HOW WE SPORE

DECISIONS, WE DON'T TALK ENOUGH.

SO THAT'S SOMETHING WE NEED TO

DO.

OPPORTUNITY MOVING AHEAD.

OKAY.

LAST PART.

STREAKING ALONG HERE.

NATURAL LANGUAGE PROCESSING.

(INDISCERNIBLE) CHEMISTS IN THE

ROOM?

IT IS IMPOSSIBLE TO ASSOCIATE

LANGUAGE FROM SCIENCE OR SCIENCE

FROM LANGUAGE, TO CALL FORTH A

CONCEPT A WORD IS NEEDED TO

PORTRAY A PHENOMENON, A CONCEPT

IS NEEDED.

ALL THREE MIRROR ONE AND THE

SAME REALITY.

SO LANGUAGE THAT WE USE TO

DESCRIBE EVENTS AND CARE

TRADITIONS, WHAT'S GOING ON, IS

MEANT TO REPRESENT SOMETHING.

IT'S REPRESENTING HEALTH AND IT

REPRESENTS SICKNESS AND

REPRESENTING THINGS HAPPENING IN

HEALTHCARE IN PEOPLE'S LIVES.

I DON'T LIKE THE TERM BIG DATA.

'S FUZZY AND OVERUSED.

I THINK PEOPLE LIKE THE CLOUD I

THINK THAT THERE'S LACK OF

PRECISION IN THAT TERM.

BUT I DIGRESS.

WHAT DO WE HOPE TO GAIN FROM

NATURAL LANGUAGE PROCESS

SOMETHING WE TALKED ABOUT IT

HERE.

THERE'S A HUGE SWATH OF

HEALTHCARE DATA THAT IS NOT

STRUCTURED, IT IS A NARRATIVE

FORM.

THIS IS HOW I USED TO DO IT.

GOES OUT TO THE DICTATION AND

COMES BACK.

WE HOPE TO GET INFORMATION

THAT'S OUT THERE FROM THE

SYSTEM, PULL IT IN, THAT'S THE

AFFRONT LOOP, PROCESS IT AND

SPIT OUT THE OTHER END.

THERE'S TWO THINGS TO TRY TO GET

FROM THIS.

ONE WE'RE LOOK AGO CROSS BIG

DATA WE HAVEN'T BEEN ABLE TO

ANALYZE BEFORE.

AND WE'RE TRYING TO DISCERN

PATTERNS IN IT.

WE JUST HEARD SOME OF THAT UP

HERE, TRY TO IDENTIFY ISSUES

THAT US MIRROR -- MERE MORTALS

THAT CAN CAN ONLY KEEP 4,000

PATIENTS IN OUR HEAD A DAY AT

ONE TIME MIGHT NOT BE ABLE TO

DISCERN OTHERWISE AN ISSUES

CHECKING DATA TO TRY TO

DETERMINE EFFECT MAYBE WE CAN

FIND IT IN THE NARRATIVE.

SO I THINK THERE'S PROMISE TO BE

HAD THERE.

THEN THERE'S THE (INAUDIBLE)

LOOP, THERE'S JOHN THERE IN A

PATIENT AND HE CAN'T REMEMBER

SOMETHING SO HE TURNS TO WATSON

OR WHATEVER AND SAYS WATSON TELL

ME AND WATSON SAYS I HAVE BEEN

LISTENING TO WHAT YOU SAID AND

YOU OUGHT TO DO DA DA DA DA DA.

SO THERE'S THE OUT TO WHETHER

IT'S THE CLINICIAN OR WHETHER

IT'S THE PATIENT OR WHETHER

THERE'S THE POLICY MAKER OR

CAREGIVER OR WHOEVER, THERE'S

LOOP OUT.

RATHER THAN HAVE TO LOOK FOR IT

IN A CERTAIN WAY, WE LIKE TO

HAVE IT COME BACK TO US IN A WAY

THAT WE CAN UNDERSTAND OR AT

LEAST THAT ALEX TREBEK CAN

UNDERSTAND.

I KNOW THE WATSON WE DISCUSSED

ARE NOT (INAUDIBLE) WATSON, THE

IBM WATSON.

BUT SIX MONTHS AGO I'M AT

JONATHAN TESH'S TALK.

AND HE'S TALKING ABOUT WATSON.

SO HE SKETCHED WATSON.

THAT IS WATSON, THE NUMBER IS

IMAGINARY, I KNOW IT WAS 77,000,

NOT 36,000.

I'M THINKING DR. WATSON, TRY TO

IMAGINE SEEING DR. WATSON, I'M

THINKING YOU KNOW, THERE'S SOME

THINGS THAT DR. WATSON MIGHT

HELP ME WITH BUT THERE'S

SOMETHING MISSING IN JUST FROM

MY DAYS OF SEEING PATIENTS

TRAINED TO BE A CLINICIAN AND

DAYS OF PATIENT AND CAREGIVER,

THERE'S SOMETHING MISSING.

SO I START THE GET ON THE OTHER

SIDE AND START SKETCHING

SHERLOCK.

I'M LIKE OH, WHERE AM I GOING

WITH THIS?

SO THE QUESTION WAS ASKED WATSON

IS A FIRST YEAR MEDICAL STUDENT,

WITH THE WAY HE PHRASED IT.

WHAT WILL HAPPEN WHEN WATSON

GETS THROUGH MEDICAL SCHOOL?

I REALLY THINK THAT WATSON AND

TECHNOLOGIES LIKE WATSON HAVE

TREMENDOUS PROMISE.

I THINK DAVID CORRECTLY

OBSERVED, I DON'T KNOW IF YOU'RE

HERE OR GONE OFF TO TALK TO

PEOPLE, CORRECTLY OBSERVED THAT

YOU REALLY, THERE ARE CERTAIN

THINGS YOU CAN EXPECT FROM IT

AND CERTAIN THINGS THAT YOU

CAN'T.

THIS IS SOMETHING THAT I WOULD

LOVE FOLKS TO DISCUSS WHEN WE

GET DOWN HERE, WHAT IS IT IN THE

COUNTER PART TO WATSON AND THE

CLINICIAN THAT YOU NEED?

MY SENSE IS A COUPLE OF THINGSCH

THERE'S ACTUALLY A QUOTE FROM

SHERLOCK HOLMES THAT SAID 50 CAN

REASON SYNTHETICALLY FOR ONE

THAT CAN REASON ANALYTICALLY.

WE CAN TALK ABOUT IT MORE BUT

THE IDEA IS THAT IN ANALYTIC

THINKER OR AN ANALYTIC REASONER

STARTS WITH THE CONCLUSION OR

PREMISE OR WORKS BACKWARDS.

HOW TO GET THAT CLOSED -- VERSUS

A EQUALS B, B EQUALS C,

THEREFORE A EQUALS C, THAT'S

SYNTHETIC.

REASONING YOU PULL TOGETHER

THINGS.

SEEMS TO ME PULLING TOGETHER TO

APPLY IT TO WHAT YOU'RE SEEING

THERE AND HELP YOU DETERMINE

THAT OUTCOME.

ALL THE WAYS WE TALK ABOUT

WATSON SCIENCE OF MEDICINE AND

ART OF MEDICINE, THERE'S STILL A

LOT OF ART AND MEDICINE.

FRANKLY THERE'S A LOT OF ART AND

LANGUAGE WE USE IN HEALTHCARE.

AND ALL YOU KNOW THIS.

PART OF OUR LIMITATIONS IN

NATURAL LANGUAGE PROCESSING IS

THE IMPRECISION WHICH WE USE OUR

LANGUAGE AND FRANKLY TO GO BACK

TO LOVASIER, THE IMPRECISION IN

CONCEPTS WHICH ARE CONSTANTLY

EVOLVING AND UNDERSTANDING OF

THE PHENOMENA.

TO THE DEGREE WE DON'T HAVE

SOLID GROUNDING, SO THERE ARE

AMAZING THINGS HAPPENING.

I LOOK FORWARD TO DISCUSSION

FURTHER.

WHAT DO WE NEED TO DO, I WAS

DIGGING AROUND IN MY QUOTATIONS

AN RAN ACROSS (INDISCERNIBLE).

NOT SURE WHO (INAUDIBLE) HERE.

MUST BE DIFFICULT, THE PROGRAM

MUST BECOME COMPREHENSIVE,

ELUSIVE, MORE INDIRECT IN ORDER

TO FORCE DISLOCATE IF NECESSARY,

LANGUAGE INTO ITS MEANING AND

WE'RE GETTING AT THE MEANING OF

LANGUAGE, WHAT WE'RE TRYING TO

PULL OUT OF PROCESSING PART OF

IT IS THE MEANING.

BETTER HEALTHCARE.

SO I THANK YOU FOR YOUR TIME AND

ATTENTION AND LOOK FORWARD TO A

GREAT DISCUSSION.

[APPLAUSE]

>> ARE THERE ANY QUESTIONS?

>> CAN YOU GET ALL THE SPEAKERS

FROM THE PREVIOUS SESSIONS

TOGETHER FOR FINAL DISCUSSION?E+9

>> I THOUGHT THERE WERE MORE

SPEAKERS TODAY THAN THESE MANY

-- BASICALLY IT HAS SOME KIND OF

DISCUSSION WHERE WE SEE US GOING

AND I GUESS ONE QUESTION LIKE

WHAT WE SEE FOR THAT AND WHAT

ARE ADVANCES NEEDED IN BOTH

FIELDS FOR HEALTHCARE TO

BENEFIT.

AND THE OTHER QUESTION, WHERE DO

YOU SEE NIH FUNDING COMING IN OR

HELPING OUT FOR THIS PROCESS TO

LOOK FORWARD.

>> I DIDN'T WATCH THE QUESTION.

>> MY QUESTION, WHERE DO WE WANT

TO SEE NLP AND CES GO IN 5 TO 10

YEARS AND WHERE DO YOU SEE NIH

ROLE IN MOVING FORWARD.

>> IT'S BEEN A TERRIFIC DAY

LISTENING AND LEARNING FROM BOTH

NLP INVESTIGATORS AND CDS

INVESTIGATORS.

I GUESS IF I WAS REALLY TO TRY

TO PAIN THE PICTURE TEN YEARS

OUT NOT SURE O IF IT'S THE GREEK

ORACLE MODEL, NOT SURE IF IT'S

THE DIALOGUE WITH WATSON

DIALOGUE BETWEEN PATIENT AND

WATT SEASON OR THREE WAY BETWEEN

PATIENT WATSON AND PHYSICIAN.

THE NLP ROLE IS TO INFORM

MODELS, INFORM KNOWLEDGE

STRUCTURES AND DRIVE CORRELATION

S JUST LIKE GENOME

WIDE ARRAY AREA ASSOCIATION

STUDIES, WE'RE STILL TRYING TO

CONNECT THE DOTS IN LANGUAGE AS

JOHN WHITE JUST POINTED OUT.

AND FIND MEANING IN WORDS AND

FIND THE CONNECTIONS IN THESE --

IN THE CORPUS OF WORDS.

SO TEN YEARS OUT I THINK WE HAVE

TO HAVE A MUCH MORE INFORMED

CLINICAL PRACTICE.  THERE'S WAY

TOO MUCH TO KNOW.

WE NEED TO HAVE THE TOOLS I

THINK ASSISTING THE CLINICIAN

AND SYNTHESIZING SUMMARIZING THE

PATIENT STATE.

I THINK THE PHYSICIAN NEEDS TO

HAVE MANY, MANY MORE PREDICTIVE

TOOLS TO HELP HIM OR HER

INTERPRET THE PATIENT'S

TRAJECTORY THROUGH A PLAN AND

HEALTH AND WELLNESS AND I THINK

THE SAME SET OF TOOLS ACTUALLY

PROBABLY WILL BE OFFERED TO THE

PATIENT HEM OR HERSELF WHEN

APPROPRIATE OBVIOUSLY OR TO A

CAREGIVER PROXY FOR THE PATIENT

BECAUSE THAT'S FOR WHOM IT

REALLY MATTERS.

THE KNOWLEDGE BASE WHEN I WAS

LISTENING TO DAVID GONDEK, IF

PERHAPS THAT THING WILL BECOME

THE MEDICAL SIGH BORG.

REALLY IS TERMINATETOR.

BEST SENSE OF THE WORD BECAUSE

IF IT STARTS TO LEARN AND

CONTINUES TO LEARN WE MOVE

SIMILAR LAYERTY AND THE MACHINE

WILL BE WAY SMARTER THAN

DOCTORS.

THAT MIGHT BE OKAY.

WE WILL BE STILL IN A

CONSULTATIVE ROLE, THE HANDS-ON

ROLE IS STILL GOING TO BE

IMPORTANT.

IT WILL BE DIFFERENT HANDS ON

ROLE THAT WE HAVE NOW.

BUT I THINK WE NEED TO HAVE THE

NLP, AND THIS EVER-EXPANDING

KNOWLEDGE BASE, AND OH, BY THE

WAY THAT WILL LEAD TO DISCOVERY

IN WAYS THAT WE HAVEN'T THOUGHT

OF YET.

I PASS THE BALL.

>> SO I WANT TO TRY TO FOCUS YOU

BACK ON THE WHOLE JOB THING.

THERE'S SCIENCE TO BE HAD HERE.

UNDERSTANDING THE SEMANTICS OF

LANGUAGE AND MEANING MIND IT,

THEN THERE'S WHAT YOU TRY TO DO

IN HEALTHCARE.

AS A PATIENT I HAVE GOT A COUPLE

OF JOBS, I'M NOT WELL, I WANT TO

GET BETTER.

HOW CAN NLP TOOLS HELP ME DO

THAT.

I DON'T THINK WE TOTALLY KNOW

YET.

I DON'T THINK WE CLEARLY STATED

THAT.

AS A CLINICIAN I HAVE A JOB TO

DO.

THAT ROLE MAY CHANGE OVER TIME,

BUT I HAVE GOT A JOB TO DO.

AND WHAT DO I NEED THAT'S IN

LANGUAGE FORM NOW, NOT

STRUCTURED DATA FORM TO DO MY

JOB.

AND GIVE ME THE INFORMATION THAT

I NEED AS A PAYER.

TO MY IMMEDIATE RIGHT.

WHAT JOB DO I HAVE IN AS THE

STEWARD OF FOLKS RESOURCES TO

HELP THEM GET BETTER CARE WHAT

AM I TRYING TO GET AT, I CAN SEE

MORE CLEAR DIRECTIl

HOW TO SIT THROUGH THE DATA.

I NEED TOO TOOLS TO DO THAT.

PEOPLE SETTING POLICY FOR CARE

ORGANIZATION OR LARGE

ORGANIZATION OR PURCHASER OF

CARE.

IF YOU SAY HERE ARE PROBLEMS IN

HEALTHCARE AND HOW NLP FIXES

THAT, A PRECISE STATEMENT WILL

GET YOU A LONG WAY.

>> ONE OF THE REASONS THAT WE'RE

STRUGGLING WITH THIS QUESTION,

WE'RE STILL IN THE PROCESS OF

RE-ENGINEERING LEGACY HEALTHCARE

ENVIRONMENT.

SO A LOT OF INFORMATION NOW

CAPTURED IN PROSE THAT WE WOULD

LIKE TO HAVE STRUCTURED ISN'T A

LOT OF THE NUANCES THAT WE WOULD

LIKE TO BE ABLE TO CAPTURE THAT

ARE ALWAYS WILL BE IN NATURAL

LANGUAGE HAVEN'T BEEN

APPROACHED.

I THINK SO WHAT JOHN SAYS IS PUT

NEEDS FIRST, DESIGN YOU WOULD

LIKE TO HAVE.

IF WE CAN SUSPEND HEALTHCARE FOR

A DECADE AND DESIGN THE SYSTEM

WE WANT, COULD WE GET THERE, BY

INCREMENTALLY IMPROVING THE

SYSTEM WE HAVE OR SHOULD WE

RETHINK AND THEN CREATING THAT

GOAL ARCHITECTURE, ENVIRONMENT

AND TRYING TO MOVE THE SYSTEM

TOWARD IT.

WE'RE NOT GOING TO SUSPEND.

CAN WE COME UP WITH THAT VISION

AND MOVE TOWARD THAT.

I WAS STRUCK BY HALLWAY

CONVERSATION A FEW MINUTES AGO.

THAT REMINDED ME OF SOME OF THE

OPPORTUNITIES FOR USING THE

NUANCES MISWHEN WE TRY TO DO

STRUCTURED TEXT ENTRY.

TO MARC'S POINT EARLIER WE LOOK

AT THESE AS THERE ARE LOTS OF

DATA AND I WAS IMPRESSED THE

WATSON TEAM IS LOOKING WHERE

THERE'S DATA.

BUT THERE ARE INTERACTIONS

HEALTHCARE PROVIDERS HAVE WITH

EACH OTHER.

SIMILAR TO THE VIGNETTE I

DESCRIBED WHERE THE PATIENT AND

PROVIDER ARE ELECTRONICALLY

COMMUNICATING.

I THOUGHT ABOUT THE SIGN OUT

EVENT.

WE ACTUALLY ARE VERY EFFICIENT

IN CONVEYING INFORMATION TO EACH

OTHER.

SO SIGN OUT FOR THOSE WHO DON'T

KNOW IS WHEN I'M ON CALL FOR THE

WEEKEND AND MY COLLEAGUE TELLS

ME ABOUT THE 7 PATIENTS HE OR

SHE IS WORRIED ABOUT.

OR I'M A RESIDENT SIGNING OUT TO

THE NIGHT RESIDENT WHO IS ON

CALL TONIGHT AND I TELL THEM

ABOUT THE 26 PATIENTS IN THE

HOSPITAL AND OFTEN WILL WRITE

VIGNETTES ABOUT THEM AND BYPASS

MY INFORMATION SYSTEMS.

AND THOSE MIGHT BE FASCINATING

PLACES FOR US TO LOOK FOR VARY

VERY EFFICIENT USE OF LANGUAGE

IN CONVEYING IMPORTANT THINGS

ABOUT PATIENTS.

SO THAT MIGHT BE AN INTERESTING

PLACE TO LOOK THAT'S NOT QUITE

UNDER THE STREET LIGHT BECAUSE

WE'RE NOT CAPTURING THAT.

MAYBE THE OTHER RELATED QUESTION

IS ARE THERE BARRIERS TO THESE

SORTS OF THINGS.

AS WE MOVE TOWARD TOO MUCH OR

MOVED TOWARD CAPTURE OF TOO MUCH

INFORMATION I CAN THINK OF TWO

REASONS FOR IT.

ONE, WE INCENTIVIZED IT WITH

BILLING BASED ON CAPTURING TOO

MUCH DATA AND WE ALSO HAVE A

LEGAL SYSTEM DOCS ARE AFRAID OF

AND ACCUSTOMED TO CAPTURING TOO

MUCH DATA BECAUSE THEY THINK

THAT'S NECESSARY TO AVOID.

BEING SUED FOR SOME REASON.

ARE THOSE BARRIERS, IF SO, HOW

DO WE HANDLE THEM.

>> THANK YOU.

>> YOU THROW AWAY THE LAWYERS.

>> I HAD ANOTHER QUESTION, I

WANT TO BE DEVIL'S ADVOCATE.

SUPPOSE WE GET THE WHOLE THING

DONE, AND THIS IS BACK TO THE

ANALOGY OF THE GPS SYSTEM.

SHUTS DOWN ON THEM, THERE'S

STILL THE ANALYTICAL ABILITY

LEFT IN HIM OR HER TO MAKE

EFFICIENT.

SO I BOUGHT A GPS SYSTEM FOR MY

WIFE WHEN WE MOVED TO BOSTON

FROM THE WEST COAST.

SHE THOUGHT IT WAS COOL BECAUSE

YOU CAN GO TO DOWN TON BOSTON

AND NOT GET LOST, AND IT'S

HORRENDOUS IF YOU'RE NOT FROM

THERE.

SHE TOLD ME THIS GPS THING

REALLY STINGS.

SHE SAYS I'M PUTTING IN THE

LOCATION AN WHEREVER I HAVE TO

GO DOWNTOWN I'M GT TAKEN A MILE

AWAY REPEATEDLY.

I THOUGHT JUST YOU'RE NOT USING

IT RIGHT.

I GOT MY 20 LASHES FOR THAT.

I FIGURED IT WAS A KNOWLEDGE

BASE UPDATE PROBLEM.

THERE HAS BEEN A KNOWLEDGE BASE

UPDATE I NEGLECTED TO PUT IN AND

THUS ONCE THE NEW KNOWLEDGE BASE

WAS UP LOWED TO THE SYSTEM

EVERYTHING WAS FINE AND GPS

WORKED AND I WAS AD MEALS AT

NIGHTTIME AGAIN.

BUT TO ADDRESS THE QUESTION, I

THINK OFTEN TIMES WE THINK THESE

COGNITIVE AGE CRUTCHES.

I NEVER FOUND THAT TO BE THE

CASE USING ANY COMPUTING TOOLS

TO FACILITATE LEARNING AND

COLLEGE AN GRADUATE SCHOOL, ET

CETERA.

IT MAY CHANGE ONE ASPECT OF

REASONING OR DECISION MAKING TO

HAVE A DECISION AID OR COGNITIVE

SUPPORT TOOLS IN YOUR HANDS.

IT DRAMATICALLY EXTENDS ANOTHER

FORM OF REASONING, THAT MAYBE

CONSIDERS ALL POSSIBILITIES OR

DIFFERENT FUNCTIONS AN

FORMULATING AND WHAT NOT.

IT'S A WIN AND LOSS, THESE

DECISION AIDS WILL EXTEND

REASONING AN CAPABILITIES IN

SOME INTERESTING YET TO BE

DESCRIBED WAYS.

WHILE WE MAY ACTUALLY FORGET THE

KREBS CYCLE WHO REALLY GIVES A

DAMN.

IF I CAN REASON ABOUT THE KREBS

CYCLE, IT'S RELEVANCE TO THE

DISEASE AT HAND THAT'S FINE.

>> JUST HAD A COMMENT I AGREE

WITH WHAT BLACKFORD JUST SAID

ABOUT THE NOT THESE SYSTEMS NOT

REALLY COMPLETELY BEING A

SUBSTITUTE FOR OUR OWN REASONING

BUT IT THINK ONE THING THEY CAN

DO IS IF THEY FUNCTION WELL THEY

CAN TEACH US BETTER WAYS OF

DOING THINGS AS WE USE THEM.

THAT WILL MAKE US BETTER OFF

WHEN WE FIND OURSELVES WITHOUT

THEM TEMPORARILY.

>> I LIKE TO MAKE THE COMMENT --

YOU KNOW ABOUT WATSON, SUDDENLY

RISING UP NEW YORK TIMES AFTER

THAT GENERALITY THING CAP CHUED

WELL, WATSON DOESN'T KNOW IT

WON.

I WOULD BE INTERESTED BECAUSE

IT'S THE END OF THE DAY, A

LITTLE COMMENT FROM ANY PANEL

MEMBER HERE.

YOU ALL HAVE TOP DOWN POSITIONS

IN POLICY OR OTHER THINGS.

THERE IS A LITTLE BIT OF A TONE

HERE AND YOU MAKE THEM LOVE IT.

BOB'S IDEA CRYSTALLIZED THAT

VERY WELL, IF WE COULD ONLY

SUSPEND FOR TEN YEARS AND

PRODUCE CAM LOT AND EVERYONE

WILL LIVE IN IT.

I UNDERSTAND THE POINT THAT

YOU'RE MAKING, OF COURSE WE'RE

WORKING WITH FOSSIL SYSTEMS BUT

I DONE THINK WE'RE GOING TO GET

VERY FAR SITTING HERE IMPLYING

VAGUELY IF MEDICAL

PROFESSIONALS, DOCTORS AN NURSES

ARE SOMEHOW (INAUDIBLE).

I HAVEN'T HEARD ENOUGH ABOUT HOW

WE WORK WITH THEM TO SEE WHAT IT

IS THEY WANT TO DO THEIR JOB

BETTER.

>> I HAVE ANECDOTE ABOUT THAT.

WHEN OUR ORGANIZATION MOVED TO

MAYO CLINIC, OUR BMI DEPARTMENT,

IMMEDIATELY I HAD A PARADE OF

DOCS THAT WERE FRUSTRATED WITH

THEIR SHALL BE NAMELESS

INFORMATION SYSTEM.

AND THEY COULDN'T DO X OR Y OR Z

AND WERE FRUSTRATED WITH IT.

THEY CAME UP WITH IDEAS HOW THEY

COULD HAVE A WORKAROUND OR HOW

THEY COULD DO SOMETHING THAT

THEY CAN'T DO WITH THAT SYSTEM.

THE MESSAGE I GOD GOVERNMENT WAS

THEY WANT TO HAVE INFORMATION

SYSTEM WOULD BE USEFUL TO THEM.

>> I'M NOT TALKING INFORMATION

SYSTEMS BUT HELPING THEM DO A

BETTER JOB BY GIVING CLINICAL

DECISION SUPPORT.

>> THAT'S PART OF WHAT THEY'RE

ASKING FOR, ASKING FOR WAYS TO

ASSESS DATA THEY HAVE GOTC'� AND

WAYS TO ENTER THE DATA THEY NEED

TO ENTER, WAYS TO MANAGE THE

PATIENT'S PROBLEMS.

THESE ARE DECISION SUPPORT

POTENTIAL TARGETS.

THE MORE ADVANCED IS JUST AN

EXTENSION BECAUSE THEY CAN'T DO

THEM IN THEIR ENVIRONMENT.

SO WHAT YOU WANT TO DO IS CHANGE

THE DYNAMIC WHERE INSTEAD OF IT

SYSTEM LOOK AT SOMETHING THRUST

UPON THEM HAVE THEY BE SOMETHING

THEY CAN BUY INTO AND HELP

FOSTER THE NATURE OF, THEY CAN

ACTUALLY HELP BRING ABOUT.

I THINK THE DIRECTION YOU'RE

GOING IS NOT IN THE SCOPE OF THE

NIH RIGHT NOW HARVARD BUSINESS

SCHOOL, THIS IS A MARKETING

QUESTION, MARKETING WITH A

CAPITAL M I LEARNED WHEN I WENT

IN THE PRODUCT LEARN MARKETING

WITH A CAPITAL WORLD IS

UNDERSTAND THE NEEDS OF THE

MARKET AN ADDRESSING THEM.

MARKET COMMUNICATIONS PITCHING

THE STUFF WE MADE TO SOME DEGREE

I WAS REFLECTING ON THAT WHEN I

WAS SITTING IN THE BACK EARLIER

THAT SOME DEGREE THIS IS A

SOLUTION LOOKING FOR A PROBLEM.

WHAT ARE THE REAL PROBLEMS THE

MARKET UNDERSTANDS SO WE CAN

THEN WORK ON ADDRESSING THEM AN

OVER TIME PERHAPS THE TWO MAY

MEET.

CALLING NLP AND CDS IS THE RIGHT

WAY TO STOP THE CONVERSATION AT

THE BEGINNING.

WE NEED TO GO OUT AND LEAD WITH

OUR O IRES AS STEVE ACCUSED ME

OF DOING EARLIER.

AND LISTENING AN DEEPLY

UNDERSTAND WHAT THE MARKET NEEDS

AND WORK SLOWLY AND DELIBERATELY

TO SEE IF STUFF BUILT OVER THE

YEARS HAS APPLICATION.

SO SIRI IS A GOOD EXAMPLE OF

SOMETHING THE MARKET

UNDERSTANDS.

SO IF I ASK MY MOM HAS SHE EVER

USED NATURAL LANGUAGE PROCESSING

SHE'D SAY NO.

IF I ASKED MY MOM IF SHE USED

SIRI SHE WOULD SAY YES.

WE KNOW THE JOKES ANECDOTES THAT

SIRI ISN'T PERFECT YET BUT IT'S

AN APPLICATION OF WORK THAT MET

A CONSUMER NEED.

WHICH MAKES STUFF EASIER.

WHAT ARE THE OTHER STUFF THAT

HEALTHCARE PROVIDERS NEED TO BE

EASIER AND BETTER?

>> JUST REAL QUICK.

I PUT UP A (INAUDIBLE) QUOTE SO

I'M WITH YOU.

I WANT THE TO URGE OUT OF

CAUTION IF YOU ONLY THINK ABOUT

THE NEEDS OF CERTAIN SECTORS,

THE MARKET, YOU RUN THE RISK OF

IMMEDIATING THOSE NEEDS TO MY

POP CULTURE REFERENCE IS THE

SIMPSON'S EPISODE WHERE HOMER

FINDS HIS LOST BROTHER WHO RUN

IT IS CAR COMPANY.

HE SAYS HOMER DESIGN ME A CAR.

IT'S A BIG MONSTROSITY THAT'S

GOT A BUBBLE AND LIKE

COFFEEMAKER AND DONUTS AND THE

GUY IS LIKE YOU RUINED ME.

I WORRY IF YOU ASK ONE SPECIFIC

SEGMENT OF THE MARKET YOU'RE

GOING TO GET THE HOME WORK.

AND RIGHT?

>> WANTS AND NEEDS ARE

DIFFERENT.

SO WE DON'T BUILD THEM WHAT THEY

ASK FOR, WE BUILD THEM WHAT THEY

NEED.

WHICH MEANS WE HAVE TO ACTUALLY

THINK AFTER WE HAVE ASKED THEM.

ANYBODY WHO WORKED IN PRODUCT

MANAGEMENT KNOWS THAT.

WE ASK OUR CUSTOMERS AND ACT ON

H WHAT THEY DIDN'T SAY.

OTHERWISE STEVE JOBS WOULDN'T

HAVE BUILT AN iPAD.

RIGHT?

>> ONE OF THE CHALLENGES, IT'S A

GREAT QUESTION, I THINK ONE OF

THE CHALLENGES IS -- TWO

THOUGHTS.

ONE IS THAT ACTUALLY I HAVE SEEN

NOW IN SORT OF THREE EVOLUTIONS

OF EMRs I HAVE BEEN PERSONALLY

INVOLVED IN, ONCE THE TOOL IS

WELL UNDERSTOOD AND REALLY KNOW

HOW TO USE IT, THE PHYSICIAN

TAKES OVER.

AND THE CLINICIAN WHO IS

INTERESTED IN PATIENT CARE

INTERESTED IN POPULATIONS AND

ASKING QUESTIONS, ASSUMES THE

PROFESSIONAL ROLE OF CARING

ABOUT HIS PATIENTS, USING THE

DATA TO DO SO BETTER.

I HAVE SEEN THAT THREE TIMES

OVER.

NOT WITHOUT HICCUPS BUT THREE

TIMES OVER.

THE PROBLEM IS WE STILL LIVE IN

THIS WORLD IN THIS COUNTRY AND

IT'S GOT IT DAYS IN WASHINGTON,

HE WHO PAYS FOR HEALTHCARE IT IS

NOT HE WHO GAINS.

SO LONG AS WE HAVE AN ASE METRIC

RISK AND REWARD FOR INVESTING IN

HEALTH IT, BULK OF OUR

CALCULATIONS�'
 GOES TO THE PAYER

AND AMONG OTHERS, WHEN THE

PHYSICIANS ARE FOOTING THE BILL,

IT'S JUST NOT APPROPRIATE

PHYSICIAN IS NOT MOTIVATED TO

USE HIS MOTIVATION OF HEALTH IT,

PAYER IS USED TO OPTIMIZE THE

PHYSICIAN IT.

IN A STRANGE DYNAMIC THE PAYER

AN PHYSICIAN LIVE N. SO

FUNDAMENTALLY THINK ABOUT HEALTH

REFORM AN HEALTH IT IN THE SAME

BREADTH IN THIS COUNTRY BECAUSE

IN TEN YEARS WE COULD BE IN THE

SAME PLACE.

WE CAN HAVE COOLER AND SMARTER

TOOLS.

IF WE'RE NOT INCENSED CORRECTLY

OR WORRIED ABOUT VALUE IN SET OF

VOLUME, PHYSICIANS MAY SAY WHO

CARES.

>> AND THAT IS WHAT IS GOING TO

HAPPEN.

IT'S HAPPENING NOW AND IT'S

GOING TO HAPPEN AT INCREASING

SPEED IN THE NOT TOO DISTANT

FUTURE.

PHYSICIANS AND PAYERS ARE GOING

TO COME TOGETHER AS ONE.

RIGHT NOW IT'S AN ANTAGONISTIC

RELATION RELATIONSHIP FOR A LOT

OF REASONS WE'RE FAMILIAR WITH.

IT IS GOING TO BECOME A

SYMBIOTIC RELATIONSHIP.

AND THAT IS WHAT IS GOING TO

DRIVE A LOT OF THIS.

>> HOW WE MOTIVATE THE PATIENT

TO DRIVE WHAT ALL OF YOU WANT TO

DO?

RIGHT NOW THE FINANCIAL

INCENTIVES ARE ALLOCATED 4 1/2

BILLION I WOULD READ YESTERDAY

IS GOING TO PAYMENTS OF ONC TO

DOCTORS AND HOSPITALS SO FAR ON

ADOPTION OF EHR, BUT I DON'T

THINK THE PATIENTS NOR

UNDERSTAND THE BENEFITS OF EHR

AND EVERYTHING WE'RE TRYING TO

DO HERE.

AND EVERY CONFERENCE I GO TO

HERE AND EVERY OTHER

ORGANIZATION I GO TO.

I STILL DON'T SEE THE BOTTOM UP

PUSH WHERE PATIENTS ARE SHOWN,

LOOK, I SOLVED YOUR PROBLEM

BECAUSE.

THEN HE GOES TO ALL HIS DOCS,

QUADRUPLE BYPASS AND EVERYTHING

BECAUSE I WAS A KID WHO ATE

CANDY ALL THE TIME.

THE THING WE GOT TO DO IS GET

THE PATIENT TO SAY TO THE DOC

THAT DOESN'T HAVE THE EHR SYSTEM

I'M GOING TO ANOTHER DOC.

BECAUSE I WANT THE BEST

AVAILABLE MEDICINE AND WE

HAVEN'T MADE THE PATIENT, THE

DRIVER AND I COME FROM TEN YEARS

WORKING FOR THE ADVERTISING

INDUSTRY.

MY LAW FIRM WAS GENERAL COUNCIL

TO THE ADVERTISING FEDERATIONS

AND ASSOCIATION NATIONAL

ADVERTISERS.

AND WE KNOW HOW TO MAKE

CONSUMERS WANT THINGS IN THE

ADVERTISING INDUSTRY.

AND ONC HAS TO MAKE THE CASE.

I HATE TO DISAPPOINT YOU BUT ONC

PCORI, WE GOT TO MAKE THE CASE

TO THE PATIENTS THEY'RE GETTING

THE BENEFIT.

>> SO WE DO HAVE A MARKETING

PROGRAM THAT I REACH TO

PATIENTS.

IS THAT OUR GREATEST LEVER?

YOU SAW ME PUT THE SLIDES UP.

WE HAVE CAWP OF LEVERS AN LEVERS

ARE REGULATIONS.

DO WE HAVE EFFORTS?

WE HAVE A WOMAN THAT IS

(INDISCERNIBLE) CONSUMERISTA.

SHE FOCUSES ON CONSUMER PROGRAM.

IT EXISTS.

>> WE DON'T HAVE THE BUDGET OF

KELLOGG OR BUDGET OF GENERAL

FOODS.

>>� OF COURSE WE DON'T.

NOR SHOULD WE.

THAT'S NOT OUR GREATEST LEVER.

DO WE HAVE SOME EFFORT THERE?

YES, WE DO.

>> THE OTHER OBSERVATION IS P IF

YOU ASK AND SURVEY VERSUS DONE

SO, YOU ASK PATIENTS DOES YOUR

DOCTOR HAVE AN EMR?

IS YOUR RECORD ELECTRONIC?

AND PATIENTS TYPICALLY SAY YES,

IT IS.

THEY'RE SURPRISED WHEN THEY HEAR

ABOUT THE SLOW GRADUAL

PENETRATION OF EMR.

POINT NUMBER ONE.

POINT TWO, I THINK THE CONSUMER

ACCOUNTABILITY THING OR PERSONAL

ACCOUNTABILITY IS GOING TO PLAY

ITSELF OUT.

IT'S VERY DIFFERENT IN THIS

COUNTRY THAN LOTS OF COUNTRIES

AROUND THE WORLD AND OTHERS CAN

ATTEST TO THIS, PERHAPS.

OUR OWN SENSE OF ENTITLEMENT AND

ALL THAT STUFF, IN THIS COUNTRY

IS PART OF THE HEALTHCARE REFORM

CHALLENGE.

BUT THE CONSUMERS ARE VOTING

WITH THEIR FEET IN THEIR POCKETS

WITH THEIR FEET A THIRD OF

AMERICANS SEE AN ALTERNATIVE

CARE PROVIDER EVERY YEAR W. THE

POCKET THERE'S 20,000 NOW OR

MORE, iPHONE MEDICALLY

ORIENTED APPS PEOPLE ARE

DOWNLOADING AND USING SO

SOMETHING IS HAPPENING.

SOME WAYS I HAVE HEARD ONE

PERSON SAY WE HAD THE ARAB

SPRING AND CONSUMER MOVEMENT,

SOME SUGGESTED PERHAPS WE'RE IN

THIS MEDICAL SPRING.

IT'S THE SPRINGTIME OF A

CONVERSION OR TRANSITION TO A

NEW MODEL OF CARE THAT'S NOT

VOLUME BASED, IT'S VALUE BASED,

IT'S NOT PHYSICIAN ORIENTED,

IT'S PAY CONSUMER ORIENTED.

>> I GUESS I WOULD ASK, MAYBE

GREG, IF YOU FOLKS AND YOUR

COLLEAGUES ARE USING THE DATA

THAT WE ARE ACTUALLY MAKING

AVAILABLE ABOUT PROVIDERS WHO

ARE USING HER.

RIGHT NOW I CAN GO TO YOUR

WEBSITE I CAN PROBABLY GO TO

YOUR WEBSITE AND FIND MYSELF.

AND FIND CERTAIN FACTORS FOR

MYSELF.

CARE TO PROVIDE, WHICH HOSPITALS

I HAVE BEEN TO, PERHAPS USE

ELECTRONIC HEALTH RECORD AND

MAYBE IN A DECADE IT'S A

ELECTRONIC HEALTH RECORD WITH

CLINICAL DECISION SUPPORT WITH

PARTNERS OR MY OWE.

SO THERE MAYBE CERTAIN MAYO.

SO THERE MIGHT BE CERTAIN

ATTRIBUTES OF THE KIND OF

PRACTICE THAT ARE PUBLICLY

AVAILABLE DATA THAT ONC AND CMS

MAKE AVAILABLE THAT PAYERS,

LADIES HOME JOURNAL OR NEW YORK

TIMES MAKE AVAILABLE TO FOLKS TO

HELP IN DECISION MAKING ABOUT

WHICH PROVIDER THEY LEFER.

>> I NOTE WE HAVE ONE PRESS

PERSON IN THIS AUDIENCE SITTING

NEXT TO ME.

THAT'S ONE THING I HAVE FOUND WE

ARE SOMETIMES MISSING, AT LOTS

OF THESE MEETINGS HERE AT THE

LIBRARY, I MUST HAVE ATTENDED

TEN IN THE LAST YEAR SINCE THE

FIRST DISCUSSION IN WATSON.

WE NEED TO GET SOME PRESS PEOPLE

TO TRANSLATE WHAT YOU SAY IN

MORE DIFFICULT JAR GONE FOR THIS

AUDIENCE.

>> I WOULD AGREE WITH THAT,

>> I'M IN THE AWARE OF OF US

HAVING ACCESS TO WHETHER OR NOT

THEY USE ELECTRONIC HEALTH

RECORDS WITH CLEARLY THE QUALITY

MEASUREMENT RELATIVE TO

PHYSICIAN MOVEMENT AS ALIVE AND

WELL, MAYBE NOT WELL, BUT IS A

LIE ALIVE.

IT NEEDS TO GET BETTER BECAUSE

IT'S NOT ACCURATE AND WELL AND

IT'S A LOT OF WHAT WE TALKED

ABOUT HERE TODAY.

COULD AND SHOULD MAKE THAT

BETTER.

WHAT IS ULTIMATELY GOING TO

DRIVE ALL THIS?

THE CHANGE IN BEHAVIOR WHETHER

IT'S THE PHYSICIANS OR THE

PATIENTS, IT'S MONEY.

IT'S NOT COMPLICATED.

THE MINUTE YOU PAY DOCTORS

MORETOR DOING BETTER AND GIVE

THEM THE TOOLS IN A WAY THEY

BELIEVE IN AN AGREE TO, BEHAVIOR

WILL CHANGE.

THE MINUTE YOU MAKE PATIENTS PAY

MORE OUT OF PONGT FOR UNHEALTHY

BEHAVIORS, THEY'RE BEHAVIOR

CHANGERS, IT'S NOT COMPLICATED.

GOD BLESS AMERICA.

>> A LOT OF ARRA MEANINGFUL USE

REQUIREMENTS ARE FOCUSED ON

DIRECTLY ON PATIENTS FOR

EXAMPLE, THE REQUIREMENT TO

PROVIDE A VISIT SUMMARY AT THE

END OF THE VISIT.

THAT WAS ONE OF THE MOST

DIFFICULT THINGS FOR US TO

IMPLEMENT IN OUR PRACTICE,

PHYSICIANS RESISTED INITIALLY,

BUT ONE OF THE THINGS THE

PATIENTS APPRECIATE THE MOST AND

WE HAVE GOTTEN POSITIVE FEEDBACK

ABOUT VALUE OF WALKING AWAY WITH

A SUMMARY OF WHAT WAS DISCUSSED

AND ACCURATE MEDICATION LIST AND

WE HAVE HAD A NUMBER OF PATIENTS

WHO HAVE SWITCHED THEIR CARE TO

OUR PRACTICE SPECIFICALLY

BECAUSE WE HAVE THE ELECTRONIC

MEDICAL RECORD AND THEY LIKE THE

FACT THAT OUR PROVIDERS IN

DIFFERENT SPECIALTIES

COMMUNICATE WITH

SHARE INFORMATION.

>> THAT'S THE POINT.

AS PATIENT DO I REALLY CARE?

IF MY DOCTOR HAS EHR?

I DO BECAUSE THE DIRECTOR OF

HEALTH IT (INAUDIBLE).

I CARE THAT MY DOCTOR'S TALK TO

ONE ANOTHER, I CARE THAT MY

INFORMATION IS CAPTURED AND

PRESCRIPTION GETS WHERE IT NEEDS

TO AND I ALSO HAPPEN TO CARE

TIER 1 AND TIER 3 MEDICINE --

BECAUSE MAYBE THE INFORMATION

WASN'T AVAILABLE AT THE TIME.

DO I CARE HR NO.

DO I CARE BETTER CARE THAT I CAN

EMAIL MY DOCTOR AND GET A TIMELY

ANSWER TO MY QUESTIONS?

I CARE ABOUT THAT.

ABSOLUTELY.

>> JAMES, YOU HAD A QUESTION?

>> I WANT TO GET BACK TO

RESEARCH AGENDA.

PART OF THIS GOAL OF THIS

WORKSHOP WHERE WE ARE AND WHERE

WE WANT TO MOVE TO.

SO I WOULD LIKE TO HEAR WHAT ARE

THE OPPORTUNITIES FOR THE FUTURE

WHAT ARE NOW NEW RESEARCH

DIRECTIONS THAT ADVANCE THE

FIELD AND HOW TO MAKE THE IMPACT

ON HEALTHCARE.

WHAT I DESCRIBE.

UNDERSTANDING INFORMATION NEEDS

IS KEY.

WHETHER IT'S AGAIN, WHETHER IT'S

THE PATIENT OR WHETHER IT'S THE

CLINICIAN OR SOMEBODY ELSE,

UNDERSTANDING WHAT INFORMATION

THEY NEED IS IMPORTANT.

UNDERSTANDING WHERE YOU GET THAT

INFORMATION.

IF IT'S -- I GOT A CERTAIN SET

OF DATA THAT'S MY STREETLIGHT,

INFORMATION THAT'S NOT THERE

WHERE DO I HAVE TO GO TO GET IT.

THAT IS A FUNDAMENTAL NEED.

IN TERMS OF DRIVING THE EVIDENCE

THAT SHOWS THESE MAKE A

DIFFERENCE MORE RESEARCH THAT

ACTUALLY INVESTIGATE IT IS LINKS

TO OUTCOMES.

WHICH IS COMPLICATED.  THERE'S A

REASON WE DON'T HAVE A TON OF

STUDIES, PROCESSES ARE EASY TO

MEASURE AND CONFOUNDING FACTORS

THAT LEAD TO OUTCOMES ARE GREAT.

THANK GOD FOR SMART RESEARCHERS.

FINALLY, THIS IS MORE A

PRACTICAL QUESTION.

BUT PULLING TOGETHER THOSE WHO

CREATE MEDICAL KNOWLEDGE.

AND THOSE WHO TRANSLATE THAT SO

IT SUPPORTS CARE, IS A WHOLE

RIGHT NOW.

TO GET TO TEN YEARS FROM NOW

THAT HAS TO HAPPEN NOW.

>> I LOVE JN'S LIST AND WILL BE

SUBMITING A PROPOSAL P BEFORE

THE END OF THE DAY.

THE YOU HAVE HEARD KEY PREEING

ISSUES THROUGH THE COURSE OF THE

DAY TODAY AND YESTERDAY.

SEVERAL OF US HAVE CHATTED ABOUT

THIS KNOWLEDGE REPRESENTATION

PROBLEM.

STILL MANY DIFFERENT WAYS TO

APPROACH THAT AND PERHAPS

DIFFERENT WAYS NECESSARY FOR

DIFFERENT APPROACHES TO

INFERENCE.

BUT WE NEED TO ARRIVE AT A

STABLE KNOWLEDGE REPRESENTATION

SO WE CAN BEGIN TO BUILD
=� THE

CORPUS OF A KNOWLEDGE BASE THAT

WILL BE THEN SUITABLE TO

BROAD-BASED INTEREST AND THINK

ABOUT THE SHARING PROBLEM, AS I

ALLUDED IN MY DIAGRAM, IT MAYBE

ONE THING TO USE KNOWLEDGE BASE

IN EPIC, ANOTHER IN SIEMENS,

ANOTHER THING IN WATSON BUT THEY

ALL SHOULD BE POTENTIAL USES.

THE CDS INFERENCIAL PROBLEM,

RIGHT NOW WE'RE TAKING SUCH BABY

STEPS WITH OUR RULE BASED

SYSTEMS.

THIS IS A SITUATION ACTION RULES

OF THE MOST MUNDANE SIMPLE

ORDER, WE NEED TO THINK HOW TO

INCORPORATE PATIENT PREFERENCES

AND UTILITIES.

WHAT IS A UTILITY MODEL FOR A

PATIENT.

WHAT DO THEY CARE ABOUT WITH

RESPECT TO GENETIC TESTING OR

INFERENCE AROUND DISEASE AND

TREATMENT.

THE SAME MIGHT APPLY TO

PHYSICIANS.

THEY BOTH HAVE TO BE CONSIDERED.

THE OTHER THING WE'RE DOING NOW

IS STATE LESS REASONING.

IT'S JUST A CROSS SECTIONAL SNAP

SHOT SITUATION ACTION RULES FOR

PRODUCTION RULES APPLY TO

TODAY'S CHART.

AS OPPOSED TO CONSIDERING THE

PATIENT'S TRAJECTORY THROUGH

HEALTH AND DISEASE AND STATEFUL

INFERENCE TO REALLY THINK ABOUT

A PATIENT'S LONG TERM

TRAJECTORY, WORK FLOW INSERTION

POINTS, BOB GREEN AND OTHER

VERSUS DONE THINKING ABOUT

SITUATIONAL FACTORS, DECISION

SUPPORT HAS TO BE PROVIDED AT

THE MOMENT OF OPPORTUNITY.

THE TEACHABLE MOMENT FOR DOC AN

EDUCATOR, AT WHAT POINT DOES IT

HAVE THE LIGHT BULB GO OFF THAT

I NEED TO THINK ABOUT SOMETHING

ELSE AND OR THE PATIENT MATTER

TO AND WHAT IS THE COGNITIVE

MODEL TO SUGGEST A POINT OF

UNCERTAINTY WHICH DECISION

SUPPORT IS APPLIES AN REALLY

USEFUL.

WHERE DOES THAT OCCUR IN THE

WORK FLOW.

PRE-VISIT, POST VISIT IN THE

MIDDLE OF VISITS, IN THE

INTEREST INTERSTITIALTY IT IS OF

CARE.

THE DATA PACKAGE PROBLEM.

IF I SEND TO WATSON A CHART,

WHAT FORM DOES IT HAVE TO GO IN

TO SAY CCD OR GREEN CDA OR CCD

PLUS OR GREEN VMR OR WHATEVER.

WHAT IS THE MODEL FT PACKAGE

SHIPPED TO AN INFERENCE ENGINE

SO IT CAN BE INFERRED UPON AND

SOME INTERESTING RESULT COME

BACK AND WHAT'S THE NATURE OF

THE RESULT.

WHAT'S THE RECOMMENDATION, ARE

THERE WAYS TO STANDARDIZE THAT,

WHAT'S THE EXPLANATION.

WHO ARE THE ACTORS AND TARGETS

FOR THE INTERVENTION.

LASTLY, WE -- AS THE NATIONAL

RESEARCH COUNCIL REPORT THAT

BILL STEAD AND OTHERS WORKED ON

SUGGESTED WE HAVE THIS

TRANSACTIONAL APPROACH TO OUR

HIT NOW.

NOT SENSITIVE TO THE COGNITIVE

MODELS THAT PHYSICIANS HAVE.

I DON'T REASON ABOUT A

HEMOGLOBIN A 1C RESULT.

I USE HEMOGLOBIN A 1C RESULT TO

REASON DIABETES.

WE DON'T RESIGN SYSTEMS TO TAKE

ADVANTAGE OF IDIOPATHIC STATE TO

ALLOW CLINICIANS TO DO SECOND

ORDER ANALYSIS OF PATIENT

INFORMATION TO REASON MUCH MORE

EFFICIENTLY AND EFFECTIVELY.

THAT'S A SIMPLE EXAMPLE.

THERE ARE MORE THAT I DON'T

UNDERSTAND WELL BUT THE

COGNITIVE MODELS IS NOT WELL

UNDERSTAND.

THAT FEEDS INTO THE PHYSICIAN

INFORMATION NEEDS.

WHAT ARE CERTAIN COG MITIVE

MODELS OR THINKING PATTERNS

ASSOCIATED WITH DIFFERENT

INFORMATION NEEDS, ET CETERA.

Q.I WOULD LIKE FOLLOW-UP ON YOUR

ANSWERS AND JAIL'S QUESTION,

WOULD YOU SAY THAT ON THE WHOLE

THEN THE RESEARCH QUESTIONS ARE

MORE BEHAVIOR CULTURAL SOCIETAL

ISSUES THAN TECHNICAL?

>> I WOULDN'T.

I THINK IT'S DISTRIBUTED BOTH

MAYBE IF YOU FORCE ME TO GUESS

MAYBE A THIRD TECHNICAL ISSUE, A

THIRD CULTURAL AND A THIRD MODEL

IN ENGINEERING, MAYBE THAT'S

PART OF TECHNICAL.

IT'S AT LEAST HALF IN HALF.

(OFF MIC)

>> IN ADDITION TO THE BEHAVIORAL

PSYCHOLOGICAL COGNITIVE MODELING

WE'RE TALKING ABOUT, THERE'S A

SYSTEMS MODELING COMPONENT OF

THIS.

WE ARE BREAK DOWN BY INDIVIDUAL

USER.

YES NOT TAKING THE OPPORTUNITY

THAT DISWRAIK ALLUDED TO IN HIS

TALK, WHY AM I GETTING THE

MAMMOGRAM RIGHT IS WHAT SHE

SAID.

AND WE HAVE ACTUALLY DONE

RESEARCH ON THIS, IT'S -- WE

HAVE DID A DEMONSTRATION IN

NUMBER OF PRIMARY CARE PRACTICES

WHERE IT WAS CALLED STANDING

ORDERS WHERE ANY MEMBER OF THE

STAFF WITH CLINICAL -- IF

THERE'S AN INDIVIDUAL THAT NEED

PREVENTIVE SERVICES A MEMBER OF

THE STAFF POP UP IN FRONT OF

THEM AND SAY CAN I GET THAT

SCHEDULED FOR YOU NOW?

BECAUSE IT DIDN'T HAVE TO BE

DR. RIDER OR DR. WHITE TO BE

ABLE TO DO THAT.

SO THERE'S A SYSTEMS MODELING

COMPONENT OF THIS, YOU AND I ARE

JUST DISCUSSING EARLIER WE HAVE

BEEN TALKING WITH COLLEAGUES AT

NSF, THEY HAVE THAT BEHAVIORAL

COMPONENT T COMPUTATIONAL

COMPONENT AND THEY ALSO HAVE A

SYSTEMS MODELING COMPONENT THAT

WE HAVE BEEN WORKING WITH

CLOSELY.

SO ALL THOSE AGAIN GET DEEPLY AT

THE ISSUES DOWN THERE, THERE'S

MORE TO IT THOUGH.

>> ONC IS INTERESTED IN WHAT

WORKS.

SO BECAUSE WE ON SOME LEVEL WANT

TO SAY SO AHRQ AND PERHAPS NIH,

THESE ARE LONG-TERM OBJECTIVES.

THESE ARE THE KINDS OF THINGS

THAT WE EXPECT THE MARKET TO

NEED.

BACK TO MARKET CONVERSATION AND

FIGURE OUT FOR US RESEARCHERS

WHAT'S GOING TO WORK SO WE CAN

THEN IMPLEMENT THOSE THINGS AS

PERHAPS THE STANDARDS, THESE ARE

THINGS THAT MAY ACCELERATE THE

IMPLEMENTATION OF THIS STUFF

THAT WORKS.

SO BOTH FROM A STANDARD

PERSPECTIVE FROM A TECHNICAL

VIEW AND ALSO FROM BEHAVIOR.

SO THESE ARE THE KIND OF THINGS

THAT WE WOULD MOTIVATE VENDORS

TO DO.

AND TECHNICAL STANDARDS WE WANT

TO REQUIRE SO THAT THINGS GO

FASTER.

>> SECOND MORE.

SO WHEN I TALK ABOUT AHRQ AND

WHAT AHRQ DOES AN CONTRAST TO

NIH AND WHAT NIH DOES AND CDC

DOES, I TALK ANT AHRQ IN TERMS

OF HEALTH SERVICES RESEARCH.

AND HOW WE DO RESEARCH ABOUT

HEALTH SERVICES.

IF YOU LOOK ACROSS THE DIFFERENT

COMPONENTS OF NIH, THEY BOTH

FUND A LOT OF HEALTH SERVICES

RESEARCH JUST LIKE DIABETES

RESEARCH BUT IN THE CONTEXT OF

HEALTH SERVICES.

BUT I THINK THE DIFFERENT

INSTITUTES CAN ALSO GET MORE

DEEPLY INTO THE PARTICULAR

ISSUES THAT THAT INSTITUTE IS

THERE FOR.

AND THE INFORMATICS COMPONENTS

OF THOSE ISSUES, THEY HAVE DEEP

KNOWLEDGE ABOUT THAT DOMAIN SO

WHAT'S NEAT AT THE INTERFACE

BETWEEN AND BETWIXT SO I THINK

THAT DIFFERENT RESEARCH AGENCIES

HAVE DIFFERENCE ROLES TO PLAY.

NSF HAS A DIFFERENT ROLE TO

PLAY, CDC HAS A PUBLIC HEALTH

ROLE TO PLAY, THAT ARE ALL

RELATED TO ONE ANOTHER BUT THEY

DEFINITELY HAD THEIR OWN TWISTS

TO IT THAT MEET THE NEEDS OF

CONSTITUENCIES THAT OUR

DIFFERENT AGENCIES SERVE.

>> ANY OTHER QUESTIONS?

I HAD ONE QUESTION ABOUT

CLINICAL DECISION, RAISE THE

QUESTION ABOUT HOW DO YOU RATE

DIFFERENT SYSTEMS.

I WAS WONDERING DOES IT MAKE

SENSE TO HAVE CENTRALIZED

DATABASE FOR ACADEMIC

RESEARCHERS LIKE WHAT WE HAVE

FOR IMAGING LIKE AD ME WHERE

PEOPLE CAN TEST THEIR SYSTEMS

AGAINST MULTIPLE KIND OF

UNSTRUCTURED NOTES AND HAVE IT

OPEN FOR EVERYBODY TO USE.

I MEAN, INDUSTRY HAS THEIR OWN

DATABASES BUT SOMETHING TO

BENEFIT FROM HAVING A BIGGER

DATABASE TO WORK WITH.

>> IS THE GENTLEMAN FROM NIST

HERE?

DO YOU WANT TO TALK ABOUT TREG?

>> I MANAGE THE GROUP THAT RUNS

TREG.

SO TREK IS FOR THOSE WHO DON'T

KNOW IS EVALUATION WORKSHOP

SPEAR SERIES FOR INFORMATION

RETRIEVAL, WHERE WE MAKE DATA

AVAILABLE STRUCTURE USER FOCUS

TASKS AROUND THAT DATA.

EXAMPLE, QUESTION ANSWERING CAME

OUT OF TREK I SPEND MONEY FROM

THE WATSON.

THE IDEA TECHNOLOGY HUH TO

MEASURE IT AND WRITE THAT

TECHNOLOGY CAME OUT OF TRACK.

THE CHALLENGE RESEARCHERS HAVE

IN THIS DOMAIN RIGHT NOW IS THAT

THERE'S A CLS SAL AMOUNT OF DATA

AND NO ONE CAN GET TO IT.

TWO PEOPLE WITH DATA CAN'T TALK

ABOUT WHAT EACH OTHER DID.

THEY'RE REALLY GOOD LEGAL AND

PRIVACY AND IRB REASONS FOR

THIS.

WHICH I DON'T WANT TO FOR A

MOMENT IMPLY THAT COMPUTER

SCIENTISTS LIKE MYSELF THINK ARE

A BARRIER TO PROGRESS.

BUT THE CHALLENGE IS IF WE CAN

SOLVE THIS SECONDARY USE

SCENARIO, YOU WILL CHANGE

STATE-OF-THE-ART IN FIVE TO TEN

YEARS COMPLETELY.

EVERY SINGLE PROBLEM I HEARD

TALKED ABOUT TODAY, DONE.

BUT, BUT YOU HAVE TO SOLVE THE

DATA PROBLEM.

AND THE DATA PROBLEM SOLUTION IS

NOT SOMETHING THAT PEOPLE AT THE

RESEARCH LEVEL CAN SOLVE, NOT

PEOPLE AT THE MAYO LEVEL TO

SOLVE IT.

IT'S POLICY LEVEL PEOPLE TO SAY

IF WE'RE GOING TO PUSH THE STATE

OF THE ART IN HOW COMPUTERS

SUPPORT CLINICAL DECISIONS FOR

EXAMPLE, PART OF# THAT MIGHT BE

NLP, PART MIGHT BE DATABASE, ALL

KINDS OF STUFF, IF WE PUSH THE

STATE OF THE ART, WE WANT THE

MAKE A JUMP, WE'RE GOING TO

CREATE THE PHENOMENON WHERE

PEOPLE MORE THAN ONE SET OF EYES

CAN LOOK AT THE DATA AT ONCE AND

PEOPLE CAN COMPARE RESULTS

BETWEEN, YOU CAN MEASURE

PROGRESS.

THIS IS WHAT I GET FOLLOWING ON

EMAIL BUT NOT COMING IN PERSON.

>> YOU SAID SOLVE THE DATA

PROBLEM I WOULD LIKE TO

UNDERSTAND MORE WHAT YOU MEAN.

ARE YOU TALKING ABOUT BEING ABLE

TO AMASS AGGREGATE DATA IN

ADDRESSING THE PRIVACY ISSUES OR

ARE YOU TALKING ABOUT SOLVING

THE STRUCTURE?

>> ACCESS PROBLEM.

FOR EXAMPLE IN TREK WE HAVE A

MEDICAL RECORDS TRACK.

SO WE HAVE A TASK AROUND COHORT

FINDING AND WE'RE USING

DEIDENTIFIED MEDICAL RECORDS

FROM PIT.

IMAGINE BASED APPARENTLY IT'S

OKAY TO HAVE DEIDENTIFIED DATA

OUT FOR RESEARCHER AS LONG AS

NOT TOO MUCH IF TOO MANY PEOPLE

KNOW ABOUT IT, THAT'S BAD.

EVEN IF IT'S BEEN DEIDENTIFIED.

THAT'S THE PROBLEM THAT NEEDS

SOLVING.

>> 

>> THE TREK PEOPLE THAT THEY

WOULDN'T HAVE TROUBLE GETTING

THE DATA FROM AND CERTAINLY

TRUTH TO THAT IN ALL FAIRNESS A

LITTLE CONCERNED ABOUT THE LARGE

NUMBER OF PEOPLE TO WHOM THE

DISTRIBUTED FOR PURPOSES OF THE

TREK THING.

IN TERMS OF THEIR LAWYERS AND

THEIR IRBs THEY WERE ABLE TO

MAKE AGREEMENTS WITH TWO PERHAPS

THREE ORGANIZATIONS HOW THEY

KNEW WELL AND HAD CONFIDENCE IN,

IT'S NOT LACK OF CONFIDENCE IN

NIST AND TREK BUT THE FACT THAT

SUCH A LARGE NUMBER OF

RELATIVELY UNKNOWN GROUPS WOULD

BE LOOKING AT IT.

IT'S PERHAPS UNNECESSARY

COWARDICE BUT NOT NECESSARY TO

RUIN THIS.

>> I THINK THE CENTRAL PART OF

THE PROBLEM IS EVERYBODY IS

TRYING TO SOLVE THE PROBLEM AT

THE LITTLE PERSON LEVEL SO EVERY

LITTLE UNIVERSITY, MEDICAL

SCHOOL IS DOING A MEDICAL

INFORMATICS AND TRYING TO SOLVE

THIS PROBLEM, INDEPENDENTLY,

EVERYBODY'S IRB IS ASKING THIS

PROBLEM ON THEIR OWN.

, EVERYBODY'S CERTAINLY THOSE OF

US IN NATURAL LANGUAGE

PROCESSING INFORMATION RETRIEVAL

ARE CONFRONTING IT BECAUSE WE'RE

LEARNING WHAT IRB STANDS FOR.

WE HAVE TO SOLVE THE SAME

PROBLEM THEN WE CAN'T GET

SOLVED.

>> I WOULD LIKE TO SPEAK IN THE

SAME POINT, FROM BRANDEIS

UNIVERSITY.

IF YOU LOOK AT THE HISTORY OF

SPEECH RECOGNITION AND NAMED

ENTITY EXTRACTION AND ALL THESE

DIFFERENT NATURAL LANGUAGE

PROCESSING YOU WILL SEE EVERY

TIME DATA WAS COLLECTED,

RELEASED AND LET EVERYBODY IN

THE COMMUNITY WORK ACROSS THAT

DATA AND EVALUATE THEM YOU SEE

MOVE IN THE PERFORMANCE AT EVERY

TIME.

IT'S WELL PLOTTED.

UNFORTUNATELY THESE ARE FIELDS

THAT NEED A LOT OF DATA.

AND WE -- I HAVE BEEN IN THE

FIELD FOR MANY YEARS.

AND WE REALLY JUST KIND OF

PLAYED AROUND FOR ABOUT THE

FIRST DECADE OF MY WORK IN THIS

AREA, UNTIL WE BECAME

DATA-DRIVEN AND

EVALUATION-DRIVEN.

I KNOW IF YOU JUST PICK A COUPLE

OF PLACES AND SAY WE'RE GOING TO

LET THEM WORK ON IT, YOU'RE NOT

GOING TO MOVE THE FIELD FORWARD.

WE NEED TO FIGURE HOW TO GET IT

SO WE CAN WORK COMPARATIVELY.

AND THIS IS HOW DARPA DIDN'T

SOLVE SPEECH BUT MOVE SPEECH TO

THE POINT SAYING Y'ALL WILL WORK

ON THE WALL STREET JOURNAL AND

YOU'RE REQUIRED TO COME TO THIS

MEETING AND SAY WHAT YOU DID

THAT YOU GOT .4% IMPROVEMENTS

BECAUSE YOU DID ADAPTATION IN

THIS AND THAT.

AND EVERYONE HAD TO SHARE AND WE

ACTUALLY GOT SPEECH TO THE POINT

WHERE THE APPLE MARKETING

MACHINE MADE EVERYBODY WANT IT

BUT WE WERE ABLE THE DO THAT.

SO WE NEED EVALUATION AND HOW TO

GET M COP INTO PIPELINES TO HAVE

EXTRINSIC EVALUATION.

WE WANT TO EVALUATE PERFORMANCE

BUT THEN WHAT DO WE DO?

HOW DO WE GET THE SOFTWARE TO

YOU SO WE CAN TAKE A PIECE OF

THAT PROBLEM AND SAY LOOK, I

KNOW IT WORKS TO THIS DEGREE.

IS IT REALLY USEFUL IN THAT

CONTEXT.

SO THOSE ARE I THINK THE TWO

BLOCKS THAT THE NATURAL LANGUAGE

PROCESSING PARTICULARLY THE

UNIVERSITY COMMUNITY IS FACING.

>> THIS IS A BIT OF TANGENT AND

MAY HAVE BEEN DISCUSSED EARLIER

SO FORGIVE ME BUT I NEED TO PUT

A PLUG IN FOR THE BEHAVIORAL AND

SOCIAL SCIENCES.

IF YOU ARE CONCEPTUALIZING MORE

INFORMATION BETTER MEDICAL

DECISIONS, AND THAT'S THE BASIS,

SOMETIMES THOUGHT OF AS A

RATIONAL SORT OF WAY TO THINK

ABOUT THINGS, IF THAT'S THE

CONCEPTUALIZATION AND I HAVE

SEEN MORE COMPUTER SCIENTISTS

AND MORE BIOINFOMATITIONS, ET

CETERA, THINK THAT IF YOU JUST

HAVE BETTER INFORMATION YOU HAVE

BETTER DECISIONS, I'M HERE TO

TELL YOU THERE'S A WHOLE FIELD

OF KNOWLEDGE THAT WILL TELL YOU

THAT HUMANS ACT ON MORE THAN

JUST RATIONAL BASE KNOWLEDGE AND

AS YOU BUILD EXPERT SYSTEMS I

ENCOURAGE YOU TO CONTACTXE

LOCAL BEHAVIORAL SCIENTISTS.

>> STEVEN MARCUS NIGMS.

>> STEVE WISEMAN, (INAUDIBLE)

PREVIOUS QUESTION.

I COMMEND TO ALL OF YOU THE

KAUFFMAN FOUNDATION REPORT OF

THEIR TASK FORCE THAT CAME LAST

WEEK, KAUFFMAN WITH TWO Fs AND

ONE N.

IN THERE THERE ARE A SET OF

RECOMMENDATIONS INCLUDING FOUR

LAWYERS, TWO FROM DUKE, ONE FROM

YALE AND ONE FROM SOMEWHERE ELSE

CRITICIZING HIPAA AS IMPEDIMENT

TO THE RESEARCH WE ALL WOULD

LIKE TO DO WITH LARGE DATE SETS

AND AT A MEETING BACK IN JANUARY

MADE ONE COMMENT WHICH WAS

BEAUTIFUL.

WE HAVE GOT SILOS AND STOVE

PIPES.

AND NO COOPERATION.

>> I WOULD ASK ABOUT PATIENTS.

RAISE YOUR HAND IF YOU WOULD

DONATE YOUR DATA TO SUCH A DATA

SET.

IF YOU HAD CONTROL OF IT.

SO MY MOM SAID THE SAME THING.

WE CAN BRAINSTORM ABOUT WHAT

MIGHT BE THE POLICY LEVERS OR

OPTIONS OR WHATEVER BUT I WOULD

SAY THERE'S CERTAINLY A LOT OF

ENTHUSIASM FOR PATIENTS STARTING

TO TAKE CONTROL HERE AND IF

PATIENTS COULD TAKE CONTROL AND

DOE DONATE TO NIST OR BRAN DICE

OR WHEREVER, THAT MIGHT BE A

VIABLE OPTION.

>> I THINK ANYBODY HERE NOT A

PHYSICIAN BUT SPEND CONSIDERABLE

TIME IN A HOSPITAL WOULD AGREE

BEING THE PATIENT MAKES YOU

REALIZE YOU DON'T CONTROL THAT

DATA.

AND THAT'S SCARY FOR ANYBODY.

I AGREE, LET'S SOLVE THAT

PROBLEM.

BUT AT THE SAME TIME, THE PEOPLE

WHO ARE WORKING ON THE DATA

ISSUES, THE SECONDARY USE

ISSUES, ARE REMOVED FROM NOT

TRYING TO FIND OUT PRIVACY

REVEALING INFORMATION.

I UNDERSTAND THIS IS A VERY

DIFFICULT LINE TO UNDERSTAND.

OR TO DRAW.

LET'S THINK OUTSIDE THE BOX.

BECAUSE THE HIPAA PRIVACY, THE

DEIDENTIFICATION STUFF DOESN'T

WORK.

NOBODY BELIEVES THEY HAD SAFETY

BEHIND THAT NUMBER.

>> CLOSING COMMENTS?

>> NO.

>> OKAY.

[LAUGHTER]

>>

>> SO LET'S THANK THE PANELISTS

FOR THIS --

>> THERE'S ONE MORE QUESTION.

I WAS GOING TO JUST PUT THE

CHERRY ON TOP FOR WHAT IAN HAD

TO SAY.

THANK YOU FOR THAT AND MARIE,

THANK YOU FOR REINFORCING THAT.

I JUST WANTED TO REPEAT WHAT IAN

SAID AT THE BEGINNING, WHICH IS

THIS IS NOT A PROBLEM THAT INTO

GOING TO BE SOLVED BY THE

RESEARCHERS.

THIS IS A PROBLEM WHERE WE NEED

PEOPLE WHO ARE SEVERAL LEVELS

ABOVE US TO BE ADDRESSING THIS.

AND IT DOESN'T -- SEEMS TO ME

THAT THIS ROOM IS AS GOOD A

PLACE AS ANY TO START.

SO IT WOULD BE GREAT TO -- LET

US KNOW IF THERE SOMETHING WE

CAN DO TO HELP.

BUT OTHERWISE WE'RE SPEAKING

INTO THE WIND.

>> ON THAT PROVOCATIVE COMMENT,

I'M GOING TO TAKE 60 SECONDS TO

CLOSE.

THANKS TO ALL OF YOU FOR STAYING

TO THE VERY END.

THANKS TO THE SPONSORS NIBIB,

NLP JAMES LUKER AND DOND

LINDEBERG AND INCREDIBLE ARRAY

OF SPEAKERS YESTERDAY AND TODAY.

PHIL, YOUR TALK WAS EXTREMELY

INTEREST, NICE TO MAKE YOUR

ACQUAINTANCE, CHRIS MANNING,

CAROL FRIEDMAN, OTHERS I'M GOING

TO FORGET.

THANKS TO ALL WHO TRAVELED LONG

AND FAR AND I WON'T NAME NAMES

BUT SOME ARE GONE ALREADY.

I SUGGEST WE MEET AGAIN MAYBE

ABOUT TWO OR THREE YEARS OUT,

THE CDS COMMUNITY, NLP COMMUNITY

AN BEHAVIORAL SCIENTISTS THANK

YOU FOR THE IMPORTANT

PROVOCATIVE COMMENT THAT TO

ORIENT THIS TOGETHER WE NEED TO

THINK AB THOSE BEHAVIORAL

DIMENSIONS AND COGNITIVE ISSUES

THAT ORIENT US TO OUR DECISION

MAKING AN PERCEPTION.

THANKS TO ALL OF YOU AND SEE YOU

THEN AT THE NEXT TIME.

[APPLAUSE]