I'M SARAH DUFFY FROM THE NATIONAL INSTITUTE ON DRUG ABUSE AND I'M WELCOMING YOU. THE MEETING IS BEING BROUGHT TO YOU BY THE NIN COMMON FUND HEALTH ECONOMICS PROGRAM SUPPORT THE OFFICE OF STRATEGIC COORDINATION AND OFFICE OF THE DIRECTOR AT THE NATIONAL INSTITUTES OF HEALTH. AS A REMINDER THE MEETING IS BEING VIDEOCAST AND ARCHIVED ON THE NIN WEBSITE FOR FUTURE VIEWING. YOUR PRESENCE INDICATES YOUR ACCEPTANCE. AND IT'S BEEN LED BY LESS LIE DUR AND COORDINATED BY JOHN HAGA OF NIA AND THE OUTSTANDING CHANDRA KELLER ALLEN OF ROSE LEE AND ASSOCIATES. WORKING GROUP MEMBERS FROM ACROSS THE NIH MANY OF WHOM ARE HERE HAVE CONTRIBUTED TO GREATLY TO THE SUCCESS OF THE PROGRAM AS HAVE VARIOUS PROGRAM OFFICIALS AND CONTRACTS MANAGEMENT OFFICIALS, COMMUNICATIONS AND BUDGET STAFF AND OTHERS IMPORTANT IN DOING THIS WORK. AS ALWAYS, ON THE EXTRAMURAL SIDE OF NIN -- NIH IT'S THE OUTSTANDING RESEARCH WHO'S KNOWLEDGE AND HARD WORK ADVANCE THE FIELD. WE'RE LUCKY TO HAVE MANY OF THEM HERE TODAY. THE OVERARCHING GOAL OF THE COMMON FUND ECONOMICS PROGRAM IS TO BUILD CAPACITY AND SUPPORT ECONOMIC ANALYSIS AND HELPING IN TWO BROAD AREAS. ANALYSIS TO HELP THE CONTRIBUTIONS OF VARIOUS FACTORS TO POPULATION LEVEL HEALTH AND ANALYSIS THAT HELP US BETTER UNDERSTAND THE ADOPTION OF CUTTING-EDGE HIGH-VALUE INTERVENTION AND APPROACHES. THE SEVEN-YEAR PROGRAM IS COMING TO AN END AND WE THOUGHT THIS WAS A GOOD TIME TO SHOWCASE THE WAY THE FIELDS OF ECONOMICS IS IMPORTANT AND USEFUL TO THE ISSUES THAT ARE A CENTRAL CONCERN OF NIH. IN THE SESSIONS THAT FOLLOW pFOR NIH STAFF AND OTHERS, AS AU LISTEN TO THE PRESENTATIONS AND DISCUSSIONS THINK NOT ONLY ABOUT WHAT HAS BEEN DONE AND BEING DONE BUT ALSO ABOUT THE FUTURE. HOW CAN WE BEST EMPLOY THE METHODS OF ECONOMICS TO ACHIEVE THE MISSIONS OF NIH AND EACH OF THE INSTITUTES AND CENTER. WE ARE PRIVILEGED TO HAVE WITH US THE BEST MINDS OF ECONOMICS. AT THE END OF EACH PANEL THERE WILL BE TIME FOR QUESTIONS FROM THE AUDIENCE WHERE YOU MAY EXPLORE IDEAS WITH PANELISTS. BEFORE WE BEGIN THERE'S HOUSEKEEPING DETAILS. THE RESTAURANTS ON THE FLOOR ARE CLOSED SO YOU'LL NEED TO GO TO THE FIFTH OR FOURTH OR THIRD FLOOR. IT'S NICE TO SPREAD IT OUT, STAIRS ARE OUT HERE AND TAKE THEM DOWN OR TAKE THE ELEVATOR. NIH WIRELESS GUEST INTERNET ACCESS IS AVAILABLE. YOU DON'T NEED A PASSWORD BUT YOU DO NEED TO AGREE TO THE TERMS AND CONDITIONS AND IT'S FOT A SECURE NETWORK. FOR REFRESHMENTS YOU SAW THE VENDING MACHINES AND K-CUPS AND THE CAFETERIA ON THE FIRST FLOOR AND A SNACK SHOP. OUR GROUP DINNER IS TONIGHT AT 6:30 AT THE DAILY GRILL AT HYATT BETHESDA. THERE'S STILL I SIGN-UP SHEET YOU CAN DO BUT YOU NEED TO DO IT BY 12:30 TODAY. FOR TRANSPORTATION, WE ENCOURAGE YOU TO USE METRO OR GET A TAXI BECAUSE IT'S MUCH SIMPLER THAN GETTING A TAXI ON CAMPUS. IF YOU NEED A TAXI TO PICK YOU UP DIRECTLY AT BUILDING 31 ON FRIDAY, THERE'LL BE A SIGN-UP SHEET FRIDAY MORNING. ANOTHER THING FOR THE SPEAKERS, YOU HAVE 20 MINUTES. WE HAVE ONE OF THESE TIMER THINGIES. IT WILL BE GREEN. IT WILL TURN YELLOW WHEN YOU HAVE THREE MINUTES AND ADD CREDIT WOULD BE GREAT IF YOU CAN WRAP UP. UNLESS THERE'S QUESTIONS I'LL TURN US TO THE FIRST PANEL. JOHN. >> I'M JOHN HAGA FROM THE NATIONAL INSTITUTE ON AGING. THE FIRST PANEL IS LOOKING AT THE LIFE CYCLE OF INNOVATIONS AND THE HEALTH SECTOR. SOMETHING OBVIOUSLY OF INTEREST AND IMPORTANCE TO NIH. THE VALLEY OF DEATH IS USUALLY DESCRIBED AS WHAT IT TAKES TO GET BETWEEN RESEARCH AND FDA APPROVAL OF A DRUG OR DEVICE. THE EARLY RESEARCH SOME BY PEOPLE HERE SHOWED ANOTHER VALLEY OF DEATH TO WORRY ABOUT ON THE OTHER SIDE OF FDA APPROVAL GETTING EFFECTIVE TECHNOLOGY TO THE PEOPLE THAT NEED IT MOST. OBVIOUSLY A VERY IMPORTANT CONCERN FOR AN INSTITUTE WHO'S TAGLINE IS TURNING DISCOVERY INTO HEALTH. THE SPEAKERS ARE WHERE WE'RE GOING TO FOLLOW ROUGHLY A LIFE-CYCLE MODEL. BHAVEN SAMPAT THE ASSOCIATE PROFESSOR IN THE DEPARTMENT OF HEALTH POLICY AND MANAGEMENT AT THE MELMAN SCHOOL OF PUBLIC HEALTH AT COLUMBIA WILL START THE DESCRIPTON OF THE WORK HE'S DONE WITH M.I.T. AND OTHER COLLEAGUES HOW INCENTIVES DETERMINE WHAT GETS DEVELOPED AT ALL. WHAT EVEN GETS TO THE STARTING LINE. AND THEN I'LL INTRODUCE OTHER SPEAKERS AS WE APPROACH. >> THAT WOULD BE GREAT IF YOU SPEAK FROM UP THERE. >> YOU'RE NEXT TO THE LIGHT. WHILE BHAVEN IS GOING UP I'LL CHOICE TO TRY TO DO LOTS OF US - TOPICS OF INTEREST TO NIH IN A COMPRESSED SPACE OF TIME RATHER THAN GIVE ANY OF THEM REALLY THE TIME IT DESERVES. SO WE'RE GOING TO BE COMPULSIVE ABOUT TIME KEEPING AND APOLOGIZE AT THE END. >> IT'S OUR GREATEST HITS. >> THERE YOU GO. >> OKAY. VERY GOOD. WELL, NO TIME FOR SMALL TALK. SO UNTIL THE 1950s OR SO, ECONOMISTS OVER THE LAST TEN OR 15 YEARS THERE'S BEEN A BLOSSOMING OF WORK THAT LOOK AT NOT ONLY HOW SCIENCE AND TECHNOLOGY IMPACT THE ECONOMY BUT IT AFFECTS THE PROCESSES OF SCIENCE AND TECHNOLOGY AND FUSION WHICH WE'LL TALK ABOUT IN THE PANEL. I'M BHAVEN SAMPAT AND WILL BE PRESENTING AN OVERVIEW OF TWO PAPERS FROM HEIDI WILLIAMS' GRANT AND EACH IS ABOUT PATENTS WHICH IS THE MAIN POLICIES GOVERNMENTS USE TO TRY TO SPUR R&D IN MEDICINE AND ELSEWHERE. IT'S WORTH STARTING BY NOTING THAT TEASING OUT THE IMPACT OF PATENTS ON STUFF IS A VERY HARD THING TO DO. A PERENNIALLY HARD THING TO DO. IN FACT, ABOUT 60 YEARS AGO THE GREAT ECONOMIST AT JOHNS HOPKINS UNIVERSITY WAS ASKED BY THE U.S. SENATE TO SYNTHESIZE THE SUM OF ECONOMIC KNOWLEDGE AT THE TIME IN 1958 ON HOW TO PATENTS EFFECT INNOVATION AND DIFFUSION AND THINGS LIKE THAT. AFTER MANY YEARS OF WORK AND A 400-PLUS PAGE REPORT HE CONCLUDED TO THE SENATE IF WE DID NOT HAVE A PATENT SYSTEM IT WOULD BE IRRESPONSIBLE TO ABOLISH IT. THE QUESTION IS WHETHER WE CAN DO ANY BETTER TODAY. AND I THINK WE AND SO WE'LL DISCUSS TWO DIFFERENT QUESTIONS IN POLICY DEBATES. TODAY THAT I THINK ARE OF INTEREST TO THE NIH AND THE CROWD. THE FIRST IS A CLASSIC QUESTION. DOES STRENGTHENING OF PATENTS INCENTIVIZE INNOVATION. THE TRADE-OFF IS ECONOMISTS DON'T LIKE MONOPOLIES BUT WE'RE WILLING TO TOLERATE THEM TO GET INNOVATION IN THE FIRST PLACE BUT THE FUNDAMENTAL QUESTION IS HOW RESPONSIVE IS R&D TO IS THE STRENGTH AND LENGTH IN PATENTS. AND MAYBE MORE RELEVANT TO PEOPLE IN THE ROOM IS THE PATENTS WHERE INNOVATION IS CUMULATIVE. AS ISAAC NEWTON SAID IF I SEE FURTHER IT'S BECAUSE I'M STANDING ON THE SHOULDERS OF GIANTS. A LOT OF RESEARCH INCLUDING BIOMEDICAL SCIENCE IS CUMULATIVE. AND THERE PATENT POLICY IS HARDER BECAUSE EVEN IF YOU GET THE FIRST-STAGE INCENTIVES RISE TWO STRONG PATENTS TO GET THE FIRST STAGE MAY CREATE COST TO SECOND-GENERATION INNOVATION WHERE OUTPUT OF THE FIRST STAGE IS EFFECTIVE IN THE SECOND STAGE AND THIS IS IN THE CONTEXT OF GENES AND THAT'S THE PAPER I'LL TALK ABOUT. SO THAT'S THE KIND OF GENERAL SET UP. LET ME TURN TO SOME OF THE -- I'LL SAY, SOME OF THE DIFFICULTIES. THERE'S TWO FUNDAMENTAL DIFFICULTIES. ONE IS IT'S HARD TO MEASURE INNOVATION AND HARD TO MEASURE MISSING INNOVATION THAT WAS TECHNICALLY FEASIBLE THERE WAS DEMAND FOR BUT DID NOT TAKE PLACE. IT'S LIKE SHERLOCK HOLMES' DOG THAT DID NOT BARK. AND EVEN IF WE SEE A RELATIONSHIP BETWEEN PATENTS AND THE STRENGTH OF PATENTS AND INNOVATION IT'S HARD TO ATTRIBUTE A CAUSAL RELATIONSHIP BECAUSE THE STRENGTH OF PATENTS AND PATENTS ARE NOT RANDOMLY ASSIGNED. ANY PROJECT ON THE ISSUES HAVE TO OVERCOME THE ISSUES. I'LL SHOW YOU THE RESULTS FROM TWO THAT TRIED TO DO SO IN DIFFERENT WAYS. THE FIRST I WAS NOT DIRECTLY INVOLVED WITH A PAPER BY HEIDI WILLIAMS IN THE AMERICAN ECONOMIC REVIEW A FEW YEARS AGO. THE STARTING POINT FOR THE PAPER IS THE IDEA THERE'S A LOT OF DRUGS FOR TREATMENT OF CANCER. THIS IS ABOUT CANCER RESEARCH. AND DRUGS FOR TREATMENT OF CANCER. THERE'S VERY LITTLE DRUGS ON OR RESEARCH ON DRUGS TO PREVENT CANCER. AND THE QUESTION IS WHY. IT COULD REFLECT THINGS LIKE DEMAND. IT COULD REFLECT THINGS LIKE SCIENTIFIC OPPORTUNITY. THE DRUGS ARE HARDER. THE PAPER INTRODUCTIONS ANOTHER I'D IT COULD REFLECT PRIVATE SECTOR INCENTIVES OR DISINCENTIVES FOR DOING LONG-TERM RESEARCH. THEY'RE IMPORTANT IN THE CONTEXT OF PREVENTION BUT ALSO OTHER SORTS OF LONG-TERM RESEARCH. THE PAPER STARTS WITH A THEORETICAL MODEL I WON'T GO INTO AND THE PUNCHLINE IS THERE'S TWO REASONS WHY WE MAY SEE PRIVATE SECTOR FIRMS NOT WNTING TO DO LONG-TERM RESEARCH. THE FIRST IS JUST IMPATIENCE AND SHORT SIGHTEDNESS AND THE STRUCTURE OF THE PATENT SYSTEM. PATENTS GIVE 20 YEARS OF EXCLUSIVITY FROM THE TIME OF FILING. FOR A RANGE OF REASONS FILING IS AROUND THE TIME OF DISCOVERY BUT IT TAKES A LONG TIME TO GET FROM DISCOVERY TO COMMERCIALIZATION. SO THE IDEA IS THAT IF THERE ARE LONGER COMMERCIALIZATION LENGTHS IT COULD CUT INTO THE PATENT TERM AND DISINCENTIVIZE R&D AND WE'LL SEE MISSING R&D ON LONGER TERM CANCER PROJECTS. HERE'S THE PROBLEM, WE DON'T OBSERVE COMMERCIALIZATION LAGS FOR MISSING PROJECTS. SO IN DOING IT JUST FOR THINGS WHERE WE OBSERVE WOULD CAUSE ALL SORTS OF PROBLEMS. THIS IS ONE THING TO OVERCOME AND THE SECOND IS IT'S DIFFICULT TO TEASE OUT THE EFFECTS FROM OTHER FACTORS. HOW DO THEY DO SO? THEY LOOK AT CANCER RESEARCH AND THERE'S A LOT OF GOOD DATA ON CANCER AND THE KIND OF NATURAL WAYS TO THINK OF CATEGORIES WHERE THERE MAY BE MISSING R&D. SECOND AND THIS IS MORE FUNDAMENTAL, WE HAVE A GOOD PREDICTOR OF THE COMMERCIALIZATION LAG FOR CANCER WHICH IS SURVIVAL TIME. THE BASIC IDEA IS THAT CANCERS WHERE THERE'S A LONG SURVIVAL TIME, CLINICAL TRIALS TO SEE THE OUTCOMES WILL TAKE LONGER. CANCERS FOR WHICH THERE'S A SHORTER SURVIVAL TIME YOU'LL HAVE SHORTER TRIALS. YOU'LL BE ABLE TO DO SHORTER TRIALS. THAT'S KEY TO THEIR PAPER AND I'LL SHOW YOU TWO CHARTS HERE THAT SUMMARIZE THE RESULTS. THIS IS JUST SHOWING YOU ON THE LEFT ACCESS THE NUMBER OF CLINICAL TRIALS FOR THREE DIFFERENT STAGES OF CANCER AND SO THESE ARE NOT CANCERS AGGREGATED TO STAGES AND THE BOTTOM AXIS SHOWS THE FIVE-YEAR SURVIVAL RATE. YOU'LL SEE THERE'S A NEGATIVE RELATIONSHIP. THE METASTATIC CANCER HAVE LOW SURVIVAL RATES AND LOCALIZED CANCERS HAVE HIGH SURVIVAL RATES AND FEW TRIALS ECONOMISTS BELIEVE DEMAND MATTERS AND THE RIGHT AXIS AND IT DASH LINE LOOKS AT CLINICAL TRIALS PER LIFE YEARS LOST USING DATA AND YOU SEE A SIMILAR RELATIONSHIP. THIS IS ALL AGGREGATED TO THE LEVEL. WE CAN ALSO LOOK AT THE CANCER STAGE LEVEL. FOR EACH CANCER HAVE MULTIPLE STAGES. WHEN YOU UNPACK IT THIS IS SORT OF WHAT IT LOOKS LIKE. AND A NEGATIVE RELATIONSHIP BETWEEN SURVIVAL RATE AND NUMBER OF TRIALS. IN REGRESSION ESTIMATES IN THE BACKGROUND THEY ALSO DO THINGS LIKE CONTROL FOR MARKET SIZE AND THINGS LIKE THAT AND FIND A 10% INCREASE IN SURVIVAL TIME IS ASSOCIATE WITH A 9% DECREASE IN TRIALS. CONTROLLING FOR OTHER KINDS OF STUFF WE WANT TO CONTROL FOR. SO IT'S STILL POSSIBLE HOWEVER, WE CONTROL FOR DEMAND NOW IN REGRESSIONS I DIDN'T SHOW YOU BUT IT'S POSSIBLE THERE'S OTHER STUFF LURKING IN THE BACKGROUND. YOU PROBABLY THINK OF SOME THINGS YOURSELF. ONE IS OBSERVED COMPONENT OF DEMAND THAT AREN'T BEING PICKED UP AND SCIENTIFIC OPPORTUNITIES. SOME CANCERS MAY BE HARDER THAN OTHERS AND THAT'S CORE -- CORRELATED WITH SURVIVAL RATES AND IT'S A TOUGH NUT TO CRACK. THEY LOOK AT THE DIFFERENCE BETWEEN ALL CANCERS AND BLOOD CANCERS. AND THE ISSUE WITH BLOOD CANCERS IS FOR THOSE THERE ARE SURROGATE END POINTS. THERE'S GOOD SURROGATE END AND WELL ESTABLISHED POINTS. THERE THE LINK BETWEEN SURVIVAL TIME AND COMMERCIALIZATION LAG IS BROKEN. AND WHEN YOU BREAK THE LINK YOU DON'T SEE THE NEGATIVE RELATIONSHIP IN FACT YOU SEE A POSITIVE RELATIONSHIP. THE IDEA IS IF IT WERE JUST ABOUT SCIENTIFIC OPPORTUNITY AND DEMAND YOU'D EXPECT TO SEE THAT ONCE YOU TOOK OUT THE COMMERCIALIZATION LIVE FACTOR AND YOU DON'T. THAT GIVES US SOME CONFIDENCE IT'S IN FACT THE COMMERCIALIZATION PACK. LET ME WRAP UP THE DISCUSSION OF THIS PAPER AND A RECOMMEND IT TO YOU. IT'S MUCH RICHER THAN I CAN TELL YOU ABOUT IN A FEW MINUTES BUT BASICALLY THIS IS WORTH A LOT. THE BACK OF THE ENVELOPE SHOWS THE VALUE OF MISSING R&D FOR PATIENTS DIAGNOSED IS ON THE ORDER OF $93 BILLION THREE TIMES THE BUDGET OF THIS AGENCY. THEY ESTIMATE IF THIS IS HAPPENING DUE TO PATENTS ALONE A ONE-YEAR INCREASE IN PATTERN TERMS WOULD BE ASSOCIATED BETWEEN A 10% AND 20% INCREASE IN R&D. HERE'S THE ONE DIFFICULTY HEIDI TOLD ME TO EMPHASIZE. THE PAPER STARTED AT A PATENT PAPER AND THE WORD WAS IN THE TITLE AND GRANT AND IN DISCUSSION IT BROADENED OUT. IT'S NOT CLEAR AT THE END OF THE DAY IF IT'S JUST PATENTS OR SHORT TERMISM OR SOMETHING ELSE AND WHATEVER THE CAUSE IT'S NOT CLEAR LONGER PATENTS ARE THE ONLY ANSWER. THERE'S OTHER WAYS TO IMPROVE UPON THE CURRENT OUTCOME INCLUDING BETTER FUNDING OR BETTER SURROGATE OUTCOMES AND THIS TOO IS BEING EXPLORED. THAT'S THE FIRST PAPER AND I HAVE EIGHT MINUTES LEFT SO LET ME TALK ABOUT THE SECOND PAPER I WAS INVOLVED IN WRITING WITH HEIDI WHICH IS LOOKING AT THE AFFECTS OF PATENTS ON FOLLOW-ON INFORMATION INVOLVING GENOMICS. I TALKED ABOUT THE CLASSIC PATENT PROBLEM. IT'S WORTH NOTING THIS IS IMPORTANT NOT ONLY IN THEORY BUT IN PRACTICE. SO A RANGE OF SUPREME COURT DECISIONS HAVE BEEN MADE BASED ON THE CONCERN THAT PATENTS WILL TIE UP FUTURE INNOVATION INCLUDING MOST FAMOUSLY THE MYRIAD DECISION ON GENE PATENTS A FEW YEARS AGO. ECONOMIC THEORY GOES DIFFERENT WAYS. THERE'S ALSO SOME POLICIES WHERE IT'S BAKED IN THAT PATENTS MAY BE GOOD FOR FOLLOW-ON INNOVATION. SO IT'S AN EMPIRICAL QUESTION. HERE TOO WE RELY THERE'S GOOD DATA AND FOCUS ON DNA BECAUSE IT'S AN IMPORTANT PROBLEM AND FORTUITOUSLY WE HAVE DNA SEQUENCES WE CAN LINK TO PATENTS BECAUSE THEY LIST THE SEQUENCES, TO PUBLICATIONS, TO DIAGNOSTIC TEST DATA AND CLINICAL TRIALS. THOSE ARE OUR MEASURES OF FOLLOW-ON RESEARCH. THE HARDER PROBLEM IS CONSTRUCTING A COUNTERFACTUAL. WE DO SO IN TWO WAYS. FIRST, WE LOOKED AT SUCCESSFUL PATENT APPLICATIONS VERSUS UNSUCCESSFUL PATENT APPLICATIONS AND WHAT HAPPENS WHEN THEY HIT. SECOND, WE USE THE FACT -- THE TWO FACTS THAT A, THERE'S A LOT OF HETEROGENEITY ACROSS THEM IN PATENT RATES AND SOME ARE LENIENT AND SOME ARE STRICTER AND THE PROCESS AT LEAST IN THE UNITS THAT COVERED GENOMICS THE PROCESS THROUGH WHICH THEY'RE ASSIGNED ARE RANDOM WITH RESPECT TO THINGS WE CARE ABOUT SO WE ARGUE. WE CAN USE THOSE TO GET LEVERAGE ON THE CAUSALITY. HERE'S THE BASIC RESULTS. AGAIN, IN TWO OR THREE PICTURES. SO WE LOOK AT THREE SETS OF GENES. WE'RE LOOKING AT ABOUT 30,000 GENES. WE LOOK AT THREE SETS. THE FIRST SET AND ESSENTIALLY THE BOTTOM LINE IS GENES WHERE THERE WAS NEVER A PATENT APPLICATION. THE SECOND AND THIRD SETS ARE GENES WHERE THERE WAS A PATENT APPLICATION FILED BUT NOT GRANTED AND THEN A PATENT APPLICATION FILED OR GRANTED OR A SUCCESSFUL PATENT APPLICATION. THE KEY IS -- IT'S CONFUSION. THE PATENT APPLICATIONS IN OUR DATA ONLY START IN 2001. SO THAT'S WHEN THE PATENT APPLICATIONS ON THE GENES START. IF WE LOOK IN THE PREPERIOD TO APPLICATION WHETHER OR NOT SUCCESSFUL HAVE MORE OF THIS OUTCOME IN TERMS OF SCIENTIFIC PUBLICATIONS HAVE MORE SCIENTIFIC PUBLICATIONS AND MORE VALUABLE TO GENES NOT CLAIMED IN A PATENT APPLICATION. IT'S NOT SURPRISING THE GOOD STUFF YOU FILE FOR PATENT APPLICATIONS ON BUT IT'S IMPORTANT BECAUSE IF WE JUST COMPARED PATENTED GENES TO UNPATENTED GENES IN A NAIVE WAY IT WOULD SEEM PATENTS ARE ASSOCIATED WITH MORE INNOVATION. THAT'S DUE IN PART TO SELECTION IN WHAT YOU FILE AN APPLICATION ON. THAT'S IMPORTANT. AND THE TOP TWO LINES ARE SUCCESSFUL AND UNSUCCESSFUL PATENT APPLICATIONS DOESN'T MATTER. THERE'S NO DIFFERENCE IN FOLLOW-ON RESEARCH BETWEEN SUCCESSFUL AND UNSUCCESSFUL PATENT APPLICATIONS. THE SECOND THING WE DO IS USE THIS PATENT EXAMINER HETEROGENEITY, TAKE MY WORD FOR IT AND YOU CAN SEE IN THE CHART IF YOU DRAW AN ABOVE MEDIAN LENIENCY YOU'RE MORE LIKELY TO GET THE PATENT AND WE TREAT THIS CONTINUOUSLY BUT IN LOOKING AT FOLLOW-ON RESEARCH IN SCIENTIFIC PUBLICATIONS THERE'S NO DIFFERENCE. THIS WORKS IF WE TREAT LENIENCY CONTINUOUSLY AND WE LOOK AT OTHER OUTCOMES MEASURES INCLUDING DIAGNOSTIC TESTING AND CLINICAL TRIALS. SO BASICALLY, THIS WAS VERY SURPRISING TO ME, I MEAN, THERE'S A LOT OF PEOPLE AND I WAS ONE OF THEM WHO EXPRESSED CONCERNS TEN YEARS AGO OR SOMETHING LIKE THAT THAT PATENTS UPSTREAM ARE BAD. WE JUST DON'T SEE IT IN THE DATA HERE. SO WE CAN TALK MORE ABOUT THAT IN THE Q&A. SO TO KIND OF WRAP UP, FRITZ MACCLOM CONCLUDED THAT SOME DAY FACTUAL DATA WERE AVAILABLE WE AS ECONOMISTS CAN DO RESEARCH TO INFORM PATENT POLICY. THERE'S MORE TO BE DONE AND UNCERTAINTY SOME I TOLD YOU ABOUT AND SOME YOUR PROBABLY THINKING ABOUT IN THE PAPER AND WITH NEW DATA AND EMPIRICAL TECHNIQUES WE ARE MAKING PROGRESS ON THE QUESTION. ON BEHALF OF MYSELF AND THE HEALTH POLICIES I'D LIKE TO THANK HEIDI AND THANK THE NIH FOR HELPING SUPPORT IT. THANK YOU. [APPLAUSE] . >> THANKS VERY MUCH. THE NEXT SPEAKER IS HAIDEN HUSKAMP IN THE DEPARTMENT OF HEALTH CARE POLICY AT HARVARD MEDICAL SCHOOL. AND SHARON LISE-NORMAND A PROFESSOR OF HEALTH CARE POLICY AT BIOSTATISTICS IN THE SAME DEPARTMENT AND PROFESSOR AT THE HARVARD SCHOOL OF PUBLIC HEALTH TALKING ABOUT DIFFUSION OF MULTIPLE TECHNOLOGIES. THANK YOU. >> GOOD MORNING, EVERYONE. SHARON AND I WANT TO RECOGNIZE OUR MANY COLLEAGUES ON THE WORK AND THANK THOSE WHO HAVE HELPED US AND OUR PROJECT OFFICER AND SCIENTIFIC OFFICER. THE ADVANCES IN TECHNOLOGY DIFFUSION OF THE ADVANCES INTO PRACTICE ARE A KEY DRIVER OF SPENDING GROWTH. AND AS WE TRY TO IMPROVE THE VALUE WE DERIVE FROM OUR HEALTH CARE SPENDING WE NEED TO UNDERSTAND HOW TECHNOLOGIES ARE DIFFUSED INTO PRACTICE AND THE WORK DONE BEFORE THE GRANTS STARTED DIDN'T FOCUS ON THE ROLE OF THE PRACTICE OR LARGER PROVIDER ORGANIZATION. THERE'S BEEN A LOT OF CHANGE IN PROVIDER ORGANIZATION IN THE LAST SEVERAL YEARS WITH LOTS OF VERTICAL AND HORIZONTAL ORGANIZATION AND INCREASED RESPONSIBILITY FOR HEALTH CARE SPENDING AND PATIENT HEALTH OUTCOMES THROUGH CHANGES IN PROVIDER PAYMENT. THE GOAL OF OUR PROJECT WAS TO UNDERSTAND HOW ORGANIZATION-LEVEL CHARACTERISTICS AND FACTORS ARE RELATE TO DIFFUSION PAT ERTERNS AND WANTED TO LOOK AT DRUGS, DEVICES AND BUY -- BIOLOGICS AND THOSE HIGHER VALUE VERSUS THOSE LOWER VALUE. FOR THIS PRESENTATION SHARON LI LISE WILL LOOK AT THE TECHNOLOGIES WE'VE BEEN STUDYING. ONE IS THE FIRST WIDELY USED BIOLOGIC CANCER THERAPY A NOVEL TREATMENT IN 2004 WHEN IT WAS APPROVED FOR ADVANCED COLORECTAL CANCER AND THE CLINICAL BENEFITS VARY BY CANCER TYPE. IN SOME CASES MEDIAN SURVIVAL IS EXTENDED BY SEVERAL MONTHS AND IN OTHER TYPES OF CANCERS THERE'S NO DOCUMENTED EFFECT ON OVERALL SURVIVAL JUST AN EXTENSION OF SURVIVAL. WE STUDIED THE DRUG ACROSS ONCOLOGY PRACTICES USING CHEMOTHERAPY USERS. AND WE'RE USING REGISTRY DATA THE STATE OF MASSACHUSETTS TO LOOK AT FIRST AND SECOND GENERATION STINTS BECAUSE CLAIMS DATA DON'T PROVIDE ENOUGH INFORMATION ON WHAT'S USE THE ON THE MANUFACTURER WHETHER IT'S A FIRST OR SECOND GENERATION DRUG AND WE'RE USING REGISTRY DATA. THE THIRD EXAMPLE IS ANTIPSYCHOTICS DRUGS AND THEY'RE BEING USED BOTH ON AND OFF LABEL. WE'RE LOOK AT A MORE MATURE PHASE OF INNOVATION IF THAT'S WHAT YOU WANT TO CALL IT. NOT THE INITIAL DRUGS THAT CAME OUT BUT THE INNOVATION SINCE 2006 WHICH IS PRIMARILY GENERIC ENTRY. ALL THE GENERICS THAT HAVE COME OUT. SEVERAL REFORMULATIONS AND A FEW NEW BRANDS THAT CAME OUT IN THE LATER PHASE OF THE CLASS. LET ME START WITH BEVACIZUMAB. GIVEN INCREASES IN THE PRICES OF THE CANCER MEDICATIONS APPROVED IN RECENT YEARS AND RISING SPENDING ON MEDICATIONS BY PAYERS AND PATIENTS, IT'S IMPORTANT TO UNDERSTAND HOW THE NEW THERAPIES HAVE BEEN DIFFUSING. PREVIOUS RESEARCH SHOWS WIDE VARIATION ACROSS ONCOLOGY PRACTICES IN SPENDING IN CARE AND THE ONCOLOGY CARE MODEL SEEKS HIGH VALUE CANCER TREATMENT. MANY HAVE LOOKED AT DIFFUSION OF THE DRUGS. WE USE A RANDOM SAMPLE TO ASSESS DIFFERENCES TREATING ELDERLY MEDICARE BENEFICIARIES. WE ALSO WANTED TO LOOK AT DIFFUSION OF HIGHER VALUE VERSUS LOWER VALUE USES WHERE SURVIVAL HAD BEEN DEMONSTRATED. DURING THAT PERIOD THAT'S COLO RECTAL CANCER AND LUNG CANCER AFTER PRESENTATION OF KEY TRIAL FINDINGS IN 2005 AND LOOK AT USES APPROVED BY THE FDA VERSUS USES THAT WERE NOT AND CREATED EPISODES OF CHEMOTHERAPY USE AND ONCOLOGY PRACTICES USING TENS. HERE'S SOME OF OUR FINDINGS. THIS SHOWS THE PERCENTAGE OF USE ACROSS PRACTICES USING TIME TO FIRST USE IN THE PRACTICES AS THE MEASURE. BY MONTH TEN DOWN HERE AT THE BOTTOM YOU CAN SEE THAT ABOUT 40% OF PRACTICES USED THE DRUG FOR AT LEAST ONE PATIENT FOR COLORECTAL CANCER AND IN 2005 YOU START STO SEE IT IN THE CLAIMS. AND IT WAS USED FOR LUNG, BREAST OVARIAN CANCERS ALL PRECEDED I BY SEVERAL MONTHS. AND TO GET MORE ROUTINE USE NOT JUST TIME TO FIRST USE IN THE PRACTICE BUT WHEN IT WAS BEING USED AMONGST PATIENTS WE LOOKED AT TIME TO 10% AS WELL. AND THE PERCENTAGE WAS LOWER FOR OTHER PATIENTS FOR BREAST, KIDNEY AND OVARIAN CONSIDERED LOWER VALUE BY OUR DEFINITION. WE IDENTIFIED SEVERAL CHARACTERISTICS ASSOCIATED WITH DIFFUSION. IT WAS FASTER AT ONCOLOGY PRACTICES RATHER THAN ACADEMIC HOSPITAL PRACTICES. DIFFUSION WAS FASTER AT LARGER PRACTICES AND THOSE WITH HIGHER RATIO OF ONCOLOGISTS TO ALL PHYSICIANS AND FOR THE PROBABILITY OF A PATIENT GET THE BEVACIZUMAB WE LOOKED AT MODELS AND FOUND VARIATIONS IN THE USE. SO THE RATIO WAS 2.9 FOR ONE STANDARD DEVIATION ABOVE THE MEAN VERSUS THE PRACTICE ONE STANDARD DEVIATION BELOW THE MEAN AND LESS FOR USES APPROVED FDA APPROVED VERSUS NON APPROVED AND LESS VARIATION FOR WHAT WE CONSIDERED HIGHER VALUE USES OF THE DRUG RELATIVE TO LOWER VALUE USES OF THE DRUG. WE FOUND RELATIVELY RAPID DIFFUSION FOR COLORECTAL CANCER AND CHARACTERISTICS ASSOCIATED WITH DIFFUSION AND THE LIKELIHOOD THE PATIENT GETS THE DRUG DEPENDS ON THE PRACTICE WHERE THEY OBTAIN CARE WHICH RAISES CONCERNS. THIS IS ESPECIALLY TRUE FOR LOWER VALUE AND HIGHER VALUE AND NON APPROVED AND APPROVED CANCER TYPES AND INTERVENTIONS AT THE PRACTICE LEVEL INCLUDING THE NEW PAYMENT MODELS INTRODUCED IN RECENT YEARS OR FOR EXAMPLE CLINICAL USE -- EXPANDED USE OF CLINICAL PATH WAYS THAT PRIORITIZE THE USE OF CHEMOTHERAPY MODELS CAN HAVE THE POTENTIAL TO INCREASE HIGHER VALUE USE AND DECREASE HIGHER VALUE USE OF THE THERAPY. NEXT UP ARE STENTS. >> WE'RE DOING A TAG TEAM. WE GENERALLY HAVE THE SAME AIMS ACROSS ALL THE EXAMPLES WE'RE LOOKING AT SO I WON'T REPEAT THOSE. SO IF WE THINK ABOUT DRUG ALLUDING STENTS THEY WERE APPROVED FOR USE IN APRIL IN 2003. WE THEN LOOKED AT THOSE TYPES OF FIRST-GENERATION. THERE ARE WERE ONLY TWO TYPES OF DRUGS THAT COATED THE CORONARY STENTS. THE PLATFORMS WERE METALLIC STAINLESS STEEL AND 132 AND 140. THEY WERE FIRST APPROVED AND RAPIDLY DIFFUSED THROUGHOUT THE U.S. NOW, WHAT HAPPENED AFTER THEIR DIFFUSION WAS WAS THE ACKNOWLEDGEMENT THAT A VERY BAD ADVERSE EVENT OCCURRED AND THAT'S THROMBOSIS AND WASN'T SEEN IN THE FDA APPROVED CLINICAL TRIALS. SO THERE WAS AN FDA PANEL MEETING IN DECEMBER 2006. I SERVED ON THE PANEL TO REALLY LOOK AT THE EVIDENCE THAT DRUG ALLUDED STENTS WERE CAUSING HARM. AS A CONSEQUENCE OF THAT, A LOT OF RESEARCH AND WORK WAS UNDERTAKEN AND SO THEN WE SAW THE INTRODUCTION OF SECOND-GENERATION DRUG ALLUDING STENTS. THE SECOND GENERATION OF DRUG-ALLUDING STENTS HAD DIFFERENT DRUGS THAN THE FIRST GENERATION AND A DIFFERENT PLATFORM AND THIS WAS SMALLER. IF YOU THINK ABOUT TWO TIME PERIODS OVER WHICH WE ARE GOING TO EXAMINE THINGS IT'S GOING TO BE THE FIRST TIME PERIOD APRIL 2003 TO MARCH 2005. THAT'S ABOUT 4 MONTHS VERSUS FEBRUARY 2008 AND FEBRUARY 2015. THE LATTER PERIOD IS THE PERIOD AFTER THE STENT THROMBOSIS REALIZATION AND NEAR THE INTRODUCTION OF THE SECOND GENERATION OF DRUG ELUTING STENTS. I'M GOING TO LOOK AT THE STATE OF MASSACHUSETTS. THIS IS A CLINICAL REGISTRY AND WE'LL LOOK AT ALL ADULTS AT LEAST 18 YEARS OF AGE AND MANDATED AS A PART OF DOING THE BUSINESS AND THE HOSPITAL CAN DO THIS IS ONLY 35 AND THERE'S 80-PLUS ACUTE NON-FEDERAL HOSPITALS IN THE STATE. IF WE LOOK OVER THE PERIOD, THE COMPETING TECHNOLOGY WAS BARE METAL AND 74% OF THOSE ADMISSIONS INVOLVED A DRUG-ELUTING STENT AND 26% INVOLVED THE BARE METAL STENT. SO A LOT OF USE IN THE EARLY PERIOD. IF WE LOOK AT THE SECOND PERIOD, NOW WE HAVE THREE TYPES OF STENTS ON THE MARKET NEP FIRST GENERATION, THE SECOND GENERATION OF DRUG ELUTING STENTS AND THE BARE METAL STENTS ON THE MARKET. THIS IS A LONGER TIME PERIOD SO WE HAVE MORE ADMISSIONS. THE REASON WHY I'M SAYING PCI ADMISSIONS IS BECAUSE YOU CAN UNDERGO SEVERAL PCIs DURING THE HOSPITALIZATION. THE WAY WE COUNT WHETHER YOU HAD A DRUG ELUTING STENT IS IF WE SEE A SECOND GENERATION YOU'RE ASSIGNED TO THE SECOND GENERATION EVEN IF YOU HAD A FIRST GENERATION SO IT'S A HIERARCHY OF IT. AND THERE'S ENTHUSIASM FOR DRUG ELUTI ELUTING STENT. WE'VE LOST SOME PERCENTAGE POINTS IN TERMS OF IMPLANT AND POINT OF THE SPECIFIC DEVICE BUT IF WE LOOK AMONG ALL DRUG ELUTING STENTS WE SEE THE INTERVENTIONALISTS OR THE OPERATORS THAT IMPLANT THE STENT ARE STILL CHOOSING A RAPID UPTAKE OF THE SECOND GENERATION OF DRUG ELUTING SEND IT. I'M SHOWING A CURVE. I WANT TO NOTE THE ZERO POINT IS THE DATE OF LAUNCH. THE TIME IS APRIL 20, 2003 FOR THE FIRST GENERATION WHEREAS THE SECOND GENERATION IN THE RED, IT STARTS IN 2008 AND WE LOOKED AT PATIENTS IN BOTH PERIODS. THERE'S ABOUT 140 OR SO BUT 90 OF THOSE WERE IN PRACTICE IN BOTH TIME PERIODS SO WE'RE GOING TO LOOK AT AN OLDER GROUP OF OPERATORS. YOU'RE GOING TO SEE IN TERMS OF THE FIRST GENERATION VERY RAPID ADOPTION AND THAT'S BY MONTH TWO WE'RE ALMOST 17% OF OPERATORS ARE USING A FIRST-GENERATION DRUG ELUTING STENT AND AT LEAST 10% OF THE OF THE PCIs THEY'RE INVOLVED IN. IF YOU LOOK AT THE SECOND GENERATION, IT'S ALSO PRETTY QUICK IN GETTING 10%. IF I LOOKED AT FIRST IT WILL BE SLOWER BUT IF YEAR LOOKING AT ACCUMULATIVE USE WE SEE IT'S A LITTLE SLOWER. IF WE LOOK AT THE SLOPE IT'S NOT AS STRAIGHT UP AND RAPID AS THE FIRST GENERATION. NOW, IF I LOOK AT SECOND GENERATION AMONG DRUG ELUTING STENTS IN THE DENOMINATOR IN THE FIRST PERIOD RELATING TO BARE METAL STENTS AND FIRST GENERATION DRUG ELUTING STENTS AND I'M SHOWING YOU THEM RELATIVE TO ALL STENTS. THE GREEN CURVE IS THE SECOND GENERATION. THE RED CURVE IS IDENTICAL IN BOTH SLIDES AND AGAIN AS WE SAW, RAPID ADOPTION INTO CLINICAL PRACTICE. SO THAT'S WHAT THE GREEN LINE IS. WHAT ABOUT OPERATOR CHARACTERISTICS? THERE'S SOMETHING I SHOULD TELL YOU ABOUT MASSACHUSETTS MAYBE YOU DON'T KNOW. MASSACHUSETTS IS A STATE THAT HAS A LOT OF PATIENT THAT ARE WHITE AND A LOT OF DOCTORS THAT ARE ALSO OF WHITE RACE. WE'RE GOING TO ADJUST FOR A LOT OF CHARACTERISTICS BUT MASSACHUSETTS IS A HEAVILY REGULATED STATE THERE'S THRESHOLDS FOR CERTIFICATION TO IMPLANT THE DEVICE. WHAT YOU SEE HERE IS A LOOK AT A SETTING WHERE THERE'S REGULATION, DATA, QUALITY MANDATE WE HAVE TO LOOK AT THE PERFORMANCE OF THESE OPERATORS AND SO BUT WE HAVE DETAILED DATA BUT THE NUMBER OF OPERATOR CHARACTERISTICS THE VARIATION IS NOT AS WIDE AS YOU MIGHT FIND IN OTHER STATES. I THINK ONLY 6% OF THE OPERATORS ARE FEMALES THE MAJORITY ARE WHITE ETCETERA. WHAT YOU'RE LOOKING AT HERE IS A MONTH TO 65% ADOPTION GIVEN THEY GET RAPIDLY REPLACED. A NEW THING COMES IN AND THE OTHER STUFF IS NOT USED AGAIN. THE LIGHT BLUE LINE IS FOR THE LESS EXPERIENCED. I'M DEFINING IT AS BASICALLY YEARS SINCE GRADUATION FROM MEDICAL SCHOOL, LESS THAN 32 VERSUS GREATER THAN 32. YOU SEE AT THE BEGINNING THERE'S NOT THAT MUCH DIFFERENCE BUT YOU CAN SEE THAT THOSE YOU CAN THINK OF THEM AS YOUNGER OPERATORS CONTINUE TO USE MORE OF THE OPERATORS THAT IMPLANT THE DEVICE. WE SEE MORE AND MORE GOING UP TO 65% WHICH IS A LARGE NUMBER. IF I DO THE SAME THING LOOKING AT SECOND GENERATION STENTS WE SEE THE SAME PATTERN, YOUNGER AND LESS EXPERIENCED TO REFER TO LESS THAN 32 YEARS AFTER GRADUATION FROM MEDICAL SCHOOL. AGAIN, TEND TO USE THESE TYPES OF TECHNOLOGIES QUICKER AND IN MORE PEOPLE THAN MORE EXPERIENCED. NOW, WHAT ABOUT BETWEEN OPERATOR VARIATION. NOW WE'LL LOOK AT USE. DATA PATIENT AND DRUG ELUTING STENT, YES OR NO. WE FITTED HIERARCHICAL MODELS AND ONE MODEL IN THE FIRST TIME PERIOD AND WHAT I'M SHOWING IS THE SAME THING THAT HAIDEN SHOWED YOU IN TERMS OF ODDS RATIO. IF YOU'RE A PATIENT AND GO TO AN OPERATOR ONE STANDARD DEVIATION ABOVE THE AVERAGE OPERATOR IN THE STATE VERSUS ONE STANDARD DEVIATION BELOW YOUR ODDS OF GETING A DRUG ELUTING STENT IS THREE TIMES GREATER. THAT'S A LARGE DIFFERENCE. WHAT I'M SHOWING YOU HERE IS THE VARIATION FOR A SECOND DRUG ELUTING STENT. THE POINT IS YOU'LL SEE THE ODDS DECREASE BETWEEN THE OPERATOR VARIATION DECREASE IN UTILIZATION. I WOULD CHARACTERIZE THIS IN THE FIRST TIME PERIOD QUICK ON THE MARKET, RAPIDLY DIFFUSED, USED A LOT OF. AND THE SECOND TIME PERIOD, LESS. AND I WILL SAY THE FIRST GENERATION WAS FASTER THAN THAT OF SECOND GENERATION AND I'M GOING TO GIVE SOME TIME TO HAIDEN TO TALK ABOUT THE ANTIPSY ANTIPSYCHOTIC THERAPY. I'LL QUICKLY MENTION A FEW THINGS BECAUSE WE'RE EARLY IN THE PROCESS OF LOOKING AT THE SECOND GENERATION ANTIPSYCHOTICS. THOUGH THEY WERE ONE OF THE TOP SELLING CLASSES OF ALL MEDICATION CLASSES AND HAD THE TOP SELLING DRUGS IN TERMS OF SALES. WE ARE LOOKING AT A LATER PERIOD OF INNOVATION. MOST CARRY LOWER RISK OF METABOLIC SYNDROME. THE INTRODUCTION OF THE REFORMULATIONS IS INTERESTING BECAUSE THEY TEND TO HELP A SUBSET. THEY'RE INTENDED FOR A NICHE MARKET PATIENT WHO'S HAVE TROUBLE BECAUSE THEY'RE ORALLY DISSOLVABLE TABLETS OR LONG-ACTING INJECTABLES WHERE YOU ONLY NEED TO TAKE EVERY QUARTER INSTEAD OF DAILY OR MULTIPLE TIMES A DAY. THE HUGE FOR THE CLASS WAS GENERICS THAT STARTED IN 2008. IF YOU LOOK AT TIME TO FIRST USE IN THE PICTURE THE ONES THAT JUMP OUT AT YOU, SUPER FAST DIFFUSION ARE ALL THE GENERICS THAT CAME TO THE CLASS. ALL THE ORIGINAL BRANDS THAT WENT GENERIC DURING OUR TIME PERIOD. YOU SEE DOWN HERE A HANDFUL OF BEFORE YOU OF THE NEWER BRANDS THAT CAME UP THAT NEVER REACHED THE SAME LEVEL OF DIFFUSION BEING PICKED UP BY PRESCRIBERS AND YOU SEE ONE REFORMULATION AND ALL THE OTHER REFORM LAYING ARE MEANT TO BE NICHE DRUGS. IT DOESN'T MEAN HOW THEY'RE ALWAYS USED BUT WITH LOWER DIFFUSION RATES. YOU CAN SEE HOW DIFFERENT THE PATTERNS LOOK BY TYPES OF PRODUCTS WHEN WE COMBINE THEM TOGETHER. I THINK WE'RE OUT OF TIME BUT WE'LL BE DOING SIMILAR ANALYSES. DO YOU WANT TO ADD A FINAL COMMENT? >> SO ANOTHER ASPECT WE ARE WORKING ON ARE STATISTICAL MODELS TO ASSESS DIFFUSION AND BRIEFLY WE'RE LOOKING AT CATEGORIZING HOW DO YOU ACTUALLY MODEL DIFFUSION IN A WAY THAT APPROPRIATELY CHARACTERIZES ALL THE FEATURES. WHAT DO I MEAN BY FEATURES? I'M TALKING ABOUT THE DIFFERENT RATES OF ADOPTION ALLOWING TO VARY BY PRESCRIBER AND IT WILL PERMIT US TO COMPARE SLOPES OR ADOPTION BETWEEN DIFFERENT DRUGS OVER DIFFERENT PERIODS AND THINGS SUCH AS THAT. WE FITTED THESE BUT WE WILL NOT TALK ABOUT THE DETAILS TODAY SO THANK YOU VERY MUCH. [APPLAUSE] >> WE'RE ACTUALLY STILL IN TIME. WE'RE HOPING WE'LL PUSH MINUTES INTO THE DISCUSSION AS WELL. AND SHARON LISE AND HAIDEN REMINDED ME I SHOULD HAVE MENTIONED THE COOPERATIVE AGREEMENTS WHERE THE WORK IS BASED IS ONGOING. THERE'LL BE ANOTHER YEAR AT LEAST OF WORK AND PUTTING TIME DATA SETS AND WE'RE REAPING THE BENEFITS. THE NEXT SPEAKER IS JOHN SKINNER PRESIDENTIAL PROFESSOR OF ECONOMICS AT DARTMOUTH UNIVERSITY. AND JOHN IS LOOKING AT A CHANGE OF PACE OF HOW DO WE GET RID OF AND HOW FAST DO WE GET RID OF TECHNOLOGY PROVING TO BE INEFFECTIVE TECHNOLOGY. THANKS. >> THANK YOU, JOHN AND THANK YOU FOR YOUR LONG-STANDING SUPPORT ON THIS ON ISSUES BOTH OF DIFFUSION AND THANK YOU SARAH FOR SUPPORTING THE PROJECT. SO THIS IS -- I'M GOING TO TALK A LITTLE BIT ABOUT THE WHOLE IDEA OF THIS OF GETTING RID OF INEFFECTIVE TECHNOLOGY. IN PART OUR KEY ARTICLE ON THIS IS EMBARGOED. I CAN'T TALK ABOUT IT. I CAN TRY INTERPRETIVE DANCE. I DON'T HAVE A LOT OF SPECIFICS. BUT TO ME I'M DISCOVERING THERE'S LIKE SIGNS OF THIS QUESTION ABOUT WHAT HAPPENS WHEN TECHNOLOGIES BECOME OUT MOTIVE. ALL OVER THE LITERATURE AND IN MEDICINE AND OTHER AREAS TOO. IT HASN'T BEEN BROUGHT TOGETHER YET AND ONE OF THE GOALS HERE. AND FOR THE FIRST TWO TALKS THERE'S NO QUESTION AMONG ANY OF YOUR MINDS WHY THESE RESEARCHERS SHOULD BE LOOKING AT SUCH INTERESTING TOPICS. THIS IS A SPECTACULAR BOOK. IT'S NOW 13 YEARS OLD AND UNFORTUNATE AREALLY PROFESSOR ROGERS DIED AND SO THERE MAY NOT BE NEW ADDITIONS AND GOT 80,000 GOOGLE HITS AND THIS IS A FASCINATING TOPIC. I'M GLAD WE'RE GETTING INVOLVED IN THIS AND THAT WE HAD A LOT OF RESEARCH ON THIS BUT THE OTHER SIDE IS AFTER THE ENTHUSIASM IS GONE THIS IS A PAPER FROM 1981. THE ENTHUSIASTIC STAGES ARE NOT AS UNIVERSITY APPLICABLE BUT FOR DIFFERENT STAGES FOR AN ILLNESS. ALREADY WE'RE GETTING TO THE SEVENTH STAGE OF INNOVATION. AND THIS IS MY FAVORITE. I DON'T KNOW IF YOU'VE SEEN THIS BEFORE, THIS IS CALLED SCOTTS PARABOLA BUT AT THE LEFT WE START WITH A PROMISING IDEA, ENCOURAGING REPORTS, WIDESPREAD ENTHUSIASM, STRONG MEDIA PRESSURE FOR UNIVERSAL ACCEPTANCE, STANDARD TREATMENT, DOUBTS CREEP IN, DAMAGING SURVEY, WIDELY PUBLICIZED MEDICAL CASE AND USE IN ONLY SPECIALIZED CASES AND POSSIBLE USES FOR LARGE QUANTS OF VERY OLD SURGEONS AMAZE THEIR JUNIORS WITH STORIES OF THE OLD DAYS. THIS IS OUR CYNICAL MODEL FOR PROGRESS IN HEALTH CARE. DAVID JONES WHO'S A MEDICAL HISTORIAN AT HARVARD SAID IN THE LAST 100 YEARS 95% OF MEDICAL INNOVATIONS ARE NOT USED. IF YOU THINK OF ALL THE THINGS THAT USED TO BE USED, QUITE A FEW HAVE GONE BY THE WAYSIDE. IT TURNS OUT THERE'S MORE AND MORE LITERATURE. I HAVE TO MENTION SARAH'S WORK FROM 1992 ON THE PROTRACTED DEMISE OF MEDICAL TECHNOLOGY AND RECENTLY FROM AND AUTHORS HAVE LOOKED MORE AT THE ISSUE SO IT'S GETTING MORE AND MORE ATTENTION. THERE'S A DIFFUSION OF INTEREST IN DE-DIFFUSION. UNFORTUNATELY IT'S DIFFICULT TO STUDY BECAUSE THERE'S BABBLE OF DIFFERENT TERMS AND THERE'S 43 TERMINOLOGIES FOR THE PROCESS OF CONTRACTING WHICH MAKES IT HARD TO DO RESEARCH BECAUSE YOU GO SEARCH FOR IT AND IT'S LIKE YOU'RE GOING TO MISS A LOT OF OF THE STUFF. AND SO THEIR STUDY WAS LIMITED TO CLINICAL PUB MED BUT THERE'S A LOT OF LITERATURE IN THE ENVIRONMENTAL FIELD OF SAY FOSSIL FUELS. YOU MAY NOT WANT TO GET RID OF IT BUT YOU STILL WANT TO SCALE BACK. SO LET ME SUGGEST A TERMINOLOGY. WE'RE STILL AT THE STABLE OF TRYING TO GET OUR TERMINOLOGY RIGHT. ABANDONMENT IS SOMETHING YOU SHOULD STOP USING, PERIOD. LIKE IT'S A BAD IDEA. EX-NOVATION IS WHERE THEY SCALE BACK ON PROCEDURE. AND FINALLY LAST WEEK, JOHN MENTION THE A CONFERENCE AT NIH ON WHAT'S CALLED DEIMPLEMENTATION. AND I THINK THE FOCUS THERE IS FOR HEALTH SYSTEM DISCOURAGE THE USE OF A PROCEDURE. MAYBE IT'S A PIPE DREAM BUT IT WOULD BE NICE TO GET CLOSURE ON THE PROCESS. DE-ADOPTION AND EXNOVATION. SO FIRST IS WHY IS THE TOPIC INTERESTING? UNWARRANTED DE-ADOPTION AND EXNOVATION, IS IT TOO FAST OR SLOW OR SAY YOU SHOULDN'T DO CAROTIDS FOR OLDER PATIENT AND THEN DOCTORS START CUTTING BACK EQUALLY ON ALL PATIENTS. DO SOME CONTINUE TO DO IT, DO SOME NOT. AND ALSO WHY DOES IT VARY SO MUCH ACROSS REGIONS AND COUNTRIES. THIS IS AN EXAMPLE FROM A RECENT ARTICLE THAT SHOWS THE DECLINE IN CAROTID INSTENTING. THE NEWER APPROACH IS STENTING HAS NOT GOTTEN NEW EVALUATIONS OR TRIALS. THERE'S A DECLINE. WE STARTED ABOVE 300. I'M GOING TO SHOW YOU SOME NUMBERS BUT THIS IS PER 100,000. THREE IS THE EQUIVALENT FOR PER 1,000. YOU CAN SEE THE GENERAL DECLINE. I DECIDED TO LOOK AT IN-PATIENT CAROT CAROTIDS STARTED IN 1992 AND THE GRAPH IS FROM 2000 TO 2014 AND DOWNLOADS IN AN EXCEL SPREAD SHEET. YOU'LL SEE THREE IS THE AVERAGE. BUT LOTS OF REGIONS HAVE DIFFERENT RESPONSES. I SOME PLACES LIKE IF YOU LOOK AT SALT LAKE CITY OR THE BRONX THEY KIND OF STARTED LOW AND START OF DROPPED OFF PROBABLY BY ABOUT 30% ABOUT THE SAME. OTHER PLACES ACTUALLY DID NOT DECLINE AT ALL. AND THOSE IN ALABAMA PRETTY OF STAYED THE STATEMENT. THEY LIKED THEIR CAROTIDS AND STAYED THE SAME. MUENSTER, INDIANA DROPPED OFF. AND WE'LL COME BACK BECAUSE IT'S AN INTERESTING PLACE AND THEY STARTED WITH TWICE THE NATIONAL AVERAGE. I WANT TO MOVE TO ANOTHER DISEASE WHICH IS CONGESTIVE HEART FAILURE. HERE'S A RECENT INNOVATION TO TREAT AND THIS IS A DEFIBRILLATOR. PUT IT IN YOUR CHEST AND IT SHOCKS >> THIS IS ANOTHER PROJECT WHICH IS ALSO SOMEWHAT EMBARGOED. THE RANDOMIZED TRIAL WAS FAVORABLE IN 2005 AND AS A CONSEQUENCE AND THIS IS FOR PREVENTIVE REASONS FOR PEOPLE WHO HAVEN'T YET HAD A SUDDEN CARDIAC ARREST BUT APPEAR TO BE FEASIBLE CANDIDATES BECAUSE OF THEIR EJECTION FRACTION AND BECAUSE OF THE CLASS OF THE NEW YORK CLASS OF HEART FAILURE AND YOU CAN SEE THAT THE THERAPY LOOKS GOOD. WE SEE DIFFUSION FROM THE DARTMOUTH ATLAS DATA. SOME AREAS ARE JUST FLYING UP AND OUR FRIENDS IN MUENSTER ARE THE FASTEST. THEY END UP BEING AMONG THE MOST RAPID. THE U.S. AVERAGE GOES UP AND STARTS FLATTENING UP AROUND THE TIME THE TRIALS COME OUT. AND OTHER PLACES GET AS INVESTED. SEATTLE IS BEING CAREFUL. ACROSS THE UNITED STATES THERE'S VARIATION AND IT'S A PATCHWORK QUILT. IT'S NOT LIKE THE SOUTH IS MORE BUT THERE'S PLACES IN THE SOUTH THEY HAVEN'T ADOPTED THEM AT ALL AND OTHERS HAVE. GETTING BACK TO THE QUESTION OF WITH SOME ADOPT RAPIDLY AND SOME DON'T AND IT'S ON THE ORDER OF TEN TO ONE. THAT'S FROM .04 TO .54 SO AN INTERESTING VARIATION TO EXPLAIN. WELL, HERE'S THE EXNOVATION. IT'S NOT AS IF THE ELECTROPHYSIOLOGISTS ARE STOMP DOING THIS BUT THEY'RE SCALING BACK AND PICKING THEIR PATIENTS MORE CAREFULLY. SO THIS IS AN EXAMPLE WHERE THE DIFFUSION AND EXNOVATION ARE CLOSE TO ONE ANOTHER. ONE THING YOU'LL NOTE IS THAT THE PLACES THAT ADOPTED THE MOST RAPIDLY THAT DIFFUSED THE MOST RAPIDLY ALSO EXNOVATED AS RAPIDLY BUT STILL ENDED UP BEING ABOVE THE AVERAGE. IF WE LOOK AT SEATTLE AND SAN FRANCISCO WHICH ARE BENCH MARKS FOR REASONABLE HEALTH CARE THEY NEVER GOT EXCITED ABOUT THESE ICDs. DEF OTHER PLACES GOT EXCITED BIT AND CAME BACK DOWN. THAT'S INDICATIVE OF AT LEAST WORRISOME IN YOU MIGHT ASK ARE THE HIGH QUALITY PEOPLE THE ONES TO DIFFUSE MOST RAPIDLY AND TO EXNOVATE MOST RAPIDLY. SO I WANT TO TAKE A LOOK AT MUNSTER, INDIANA. WE'RE DOING RESEARCH ON THIS BECAUSE WE HAVE THE ICD REGISTRY FROM 2006 TO 2013 WITH INCREDIBLY DETAILED DATA ON ALL KINDS OF CHARACTERISTICS. THE PLACE DIFFUSE MOST RAPIDLY HAD THE LOWEST RISK-ADJUSTED OUTCOMES AND THE FACT THEY STILL END UP DOING MORE IS ALSO WORRISOME. MUNSTER IS INTERESTING ONLY BECAUSE THERE'S SOMETHING ON PUBLIC RECORD WHICH IS THERE IS THIS DOCTOR WHO IS JUST GOING AROUND DOING A LOT OF ICDs THAT WAS CLEARLY NOT QUALIFIED. THIS IS AN ARTICLE IN THE NEW YORK TIMES THAT THE DOCTOR DIXON RAIS RAISED CONCERN WHETHER THE DOCTOR OR OTHER PHYSICIANS WERE QUALIFIED AND HE WAS SHUT DOWN. IT WAS TOO PROFITABLE NOT TO STOP. THE HOSPITAL FELT IT WAS A NEW SOURCE OF REVENUE AND KEPT GOING. AL THE ISSUES ARE REALLY IMPORTANT TO SORT OF THINK ABOUT IN TERMS OF THE LINK BETWEEN DIFFUSION AND EXNOVATION WE FOUND IT TO BE FAR MORE COMPLICATED THEN WHEN WE FIRST STARTED OUT. NOW, I REALIZE THIS IS THE NATIONAL INSTITUTE OF HEALTH AND NOT THE TREASURY BUT I THINK IT'S ALSO IMPORTANT TO THINK ABOUT EFFICIENT EXNOVATION AS A WAY TO SAVE MONEY. THE WAY I USUALLY PUT IT WHEN I GO AND TALK IN OTHER COUNTRIES THEY HAVE THE SAME KINDS OF ISSUES. HERE WE HAVE ALL THIS PROMISING TECHNOLOGY THAT'S EXTREMELY EXPENSIVE THOUGH SOVALDI WE KNOW IS COMING DOWN IN PRICE QUITE A BIT BUT ALL THE NEW PRODUCTS FROM THE TECHNIQUE ARE GOING TO BE EXPENSIVE. I EXPLAIN IT TO MY KIDS THE FOLLOWING WAY. HERE'S A PICTURE OF MY CLOSET AND I STILL HAVE SOME DISCO SHIRTS FROM THE '70s. I HAVE TWO CHOICES WHEN I WANT TO BRING IN NEW ARMANI SUITS. I CAN EXPAND THE CLOSET OR GET RID OF SOME OF THAT STUFF. UNBENOUNCED TO MANY OF US THAT'S WHAT'S HAPPENING IN THE U.S. HEALTH CARE SYSTEM. HERE'S MEDICARE IN-PATIENT SURGICAL RATES. I REALIZE SOME OF THIS OCCURS BECAUSE THERE'S A SHIFT FROM IN-PATIENT TO OUT-PATIENT BUT EVEN IF YOU DO PCIs WHICH I HEARD ABOUT BEFORE. THERE'S A DECLINE OF COMBINED PCIs, A SMALL DECLINE. WE SAW A 35% DECLINE IN CAROTID ENDARTERECTOMY ALL SINCE 2006 HAVE BEEN FALLING IN TERMS OF RATES AND IT'S ALSO THE CASE THAT INFLATION ADJUSTED PER ENROLEE SPENDING CARE INCREASE HAS BEEN ZERO WHICH IS ASTONISHING IF YOU WANT TO THINK ABOUT HEALTH CARE COSTS THAT HAVE BEEN RISING WHERE EVER ELSE. MEDICARE PRICES ARE PRETTY MUCH FIXED AT THE RATE OF GENERAL INFLATION AND SO WHAT'S HAPPENING IS QUANTS FOR -- QUANTITIES ARE DECLINING AND THE PROCEDURES AREN'T AS GREAT AS THEY WERE FIRST THOUGHT TO BE. SO SUMMING UP, FIRST THE DE-ADOPTION AND EXNOVATION AND DE-IMPLEMENTATION ARE NOT JUST INTERESTING THINGS TO STUDY BUT REALLY IMPORTANT WAYS TO REDUCE WASTE. IT SEEMS LIKE WHAT WE'RE FINDING IS THERE'S A LOT OF PRACTICES THAT DON'T GIVE UP THE OLD STUFF. THEY KEEP DOING IT THE OLD WAY. WHEREAS OTHER PLACES, IN FACT WE'RE FINDING PHYSICIANS WITH MORE EXPERIENCE ARE DROPPING THE STUFF SOONER RATHER THAN LATER WHICH IS AN INTERESTING COUNTERPART TO YOUR WORK. BUT LITTLE IS KNOWN ABOUT EARLY AND LATE EXNOVATERS. WHAT IF THE TRIALS AREN'T ALWAYS RIGHT. THE INTERESTING THING IS ONE WHEN TRIAL SAYS DO THIS AND TWO YEARS LATER ANOTHER ONE SAYS DON'T DO THAT, DO THIS. WHICH PHYSICIANS FOLLOW THE TRIALS VERSUS FOLLOW THEIR INSTINCTS. PHYSICAL IMPLICATIONS COULD BE IMPORTANT AND IF YOU DON'T FUNDAMENTALLY CARE ABOUT THE DEFICIT YOU SHOULD STILL CARE ABOUT BUDGETS WHO ALLOW FOR NEW INNOVATIONS TO BE FUNDED AND DIFFUSED RAPIDLY. SOVALDI WAS NOT BEING DIFFUSED RAPIDLY BECAUSE OF THE HIGH COST. THERE WASN'T ANY MONEY THERE. FINALLY, I DON'T THINK SCOTT'S PARABOLA HAS BEEN RETIRED JUST YET. I THINK IT STILL HOLDS AND I THINK IT'S AN INTERESTING TOPIC AND I HOPE WE'LL SEE MORE RESEARCH ON IT IN THE FUTURE. [APPLAUSE] >> THANK YOU, JON. OUR DISCUSSANT WHAT WAS GIVEN THE IMPOSSIBLE TASK OF LOOKING BACK NOT ONLY AT THE KIND OF RESEARCH THAT'S GOING ON BUT THINKING ABOUT WHAT ELSE IS NEEDED IS DANA GOLDMAN A DISTINGUISHED PROFESSOR AT THE UNIVERSITY OF SOUTHERN CALIFORNIA AND THE FOUNDING DIRECTOR AND DIRECTOR'S CHAIR OF THE SCHAFER CENTER FOR HEALTH POLICY AND ECONOMICS AT USC. THANK YOU. >> THANK YOU. IT'S A PLEASURE TO BE HERE. I WAS INVOLVED AT THE GENESIS OF THE COMMON FUND AND WAS INVITE TO PROVIDE INPUT ON WHAT MIGHT BE DONE I WAS ALSO INVITE TO REVIEW PROPOSALS AND NOW INVITED TO DISCUSS THEM. HOPEFULLY NEXT TIME I'LL GET SOME MONEY. BUT, BE THAT AS IT MAY, WHAT I WANT TO SAY IS THIS IS A STORY OF AVERAGE VERSUS MARGINAL. AND PUBLIC POLICY GETS CONFUSED AND I'LL EXPLAIN AS I GO. I THINK THE SPEAKER HAVE DONE A GOOD JOB SHOEGS THE TENSION. IN THE SHORT RUN WE WANT UNFETTERED ACCESS TO NEW TREATMENTS. AND HIGH PRICES AS JON JUST SAID ABOUT SOVALDI AND THE MEDICARE PROGRAM AND ECONOMISTS KNOW IF WE HAVE A COMPETITIVE MARKET PRICES SHOULD BE SET AT THE PRICE OF PRODUCTION AND THE MARGINAL COSTS ARE OFTEN CHEAP THOUGH HE MENTIONED CRISPR AND OTHERS AND SOCIETY WANTS INNOVATORS TO DEVELOP NEW TREATMENTS. WE KNOW THE R&D IS RISKY AND NEED TO NOT ONLY INVEST IN BASIC SCIENCE BUT WE OFFER THINGS LIKE PATENTS AND I WOULD ARGUE RIGHT NOW ALSO IMPORTANT IS EXCLUSIVITY. YOU LOOK AT THE ORPHAN MARKET POWER. IT DOESN'T NEED TO BE A NEW INNOVATION JUST A NEW APPLICATION AND WE ALSO GIVE RESEARCH SUBSIDIES. IT CAN BE SUMMARIZED AS A TENSION BETWEEN PATENT RIGHTS AND PATIENT RIGHTS. I WANT TO GO BACK TO AN EXAMPLE BECAUSE I THINK IT SHOWS THE IMPORTANCE OF AVERAGE VERSUS MARGINAL. I WANT TO REWIND TO THE MID '90s. WHEN WE DISCOVERED HIV IN THE '80s, IT'S ONE OF THE MOST DEVASTATING DISEASES GLOBALLY BECAUSE IT HITS PEOPLE SO EARLY IN THE LIFE CYCLE. WHAT HAPPENED IS IN THE MID '90s WE INTRODUCED ANTIVIRAL THERAPY AND THERE WAS PROTEST IN THE PRICE. AS ECONOMISTS THAT'S SURPRISING BECAUSE I JUST MOVED TO L.A. AT THE TIME AND I REMEMBER MAGIC JOHNSON ANNOUNCING HE WAS HIV POSITIVE AND HAD RETIRED FROM THE LAKERS AND I THOUGHT THAT WAS THE LAST I WAS GOING TO SEE OF MAGIC JOHNSON. AND IN THE ULTIMATE IRONY, MAGIC JOHNSON NOW OWNS THE DODGERS IN LOS ANGELES. SO THE POINT IS EVEN WITH HIS ENORMOUS WEALTH, HE COULDN'T BUY HEALTH AT ANY PRICE. SO FROM THE ECONOMIST POINT OF VIEW THE PRICE OF LIFE WENT INFINITE TO ONE THAT WAS $15,000 PER LIFE YEAR OR WHATEVER. FOR US THAT'S AN INCREDIBLE SALE AND WE SHOULD BE BUYING AS MUCH AS POSSIBLE. BUT FROM THE BUDGETARY PERSPECTIVE, STATES WERE SAYING WELL, HOW ARE WE GOING TO PAY FOR HAART. BUT FROM THE NIH PERSPECTIVE THIS IS WHAT THE SURVIVAL CURVE LOOKED LIKE FOR HIV IN 1984. AND THE GOAL OF WHAT WE'RE GOING AT NIH AND WHAT WE WANT FROM OUR HEALTH SYSTEM IS TO SHIFT THE SURVIVAL CURVE OUT. THIS IS WHAT IT LOOKED LIKE PRIOR TO THE INTRODUCTION OF HAART. I PUT OUT 1994 BECAUSE THERE'S A SHIFT THAT GOES ON PRIOR TO THE INTRODUCTION OF THE NOVEL TECHNOLOGY. THIS WAS AZT AND OTHER FORMS OF CARE THAT PUSHED OUT SURVIVAL AND WE FORGET ABOUT THAT INCREMENTAL INNOVATION. ACTUALLY, IN SOME WAYS CANCER HAS BEEN VICTIM TO THIS BECAUSE THE STORY OF CANCER HAS BEEN INCREMENTAL INNOVATION BUT WHEN SUSTAINED OVER DECADES IT HAS A BIG EFFECT. THIS IS WHAT HAPPENED BY 2000. IF YOU LOOK AT LIFE EXPECTANCY WE ADDED 15 YEARS OF LIFE EXPECTANCY AS A RESULT OF THE INTRODUCTION OF THE DRUGS. NOW, THE PEOPLE NOT IN THE ROOM OFTEN SAY WHAT GOOD ARE ECONOMISTS? IT TURNS OUT WE'RE GOOD AT MULTIPLICATION. IF YOU TAKE ALL THE PEOPLE INFECTED BY HIV AND MULTIPLY THE NUMBER OF LIFE YEARS THEY GAIN BY THE RESULT OF THE TREATMENTS AND MULTIPLY IT BY A NUMBER THAT PETER NEWMAN TELLS ME WHAT TO USE, WHAT YOU GET IS $1.5 TRILLION IN BENEFITS EMERGED FROM HAART. THE PUBLIC POLICY DISCUSSION WAS THE REVENUES IF YOU DISCOUNT THE FLOWS IN VERY MUCH -- REVENUE. AND ABOUT 5% OF THE RETURN IS CAPTURED BY THE INNOVATORS. WHAT IS THE RIGHT RATE AND WHAT SHOULD IT BE. I PICKED A SUCCESSFUL DRUG BUT I THINK THE POINT IS NIH, FOR EXAMPLE, SHOULD BE EVALUATED THE SAME WAY WE EVALUATE INVESTMENTS FROM A LARGE PHARMACEUTICAL MANUFACTURER. YOU HAVE AN ENTIRE PORTFOLIO OF RESEARCH. SOME OF THAT'S GOING TO PAYOFF. SOME DOESN'T. I WOULD ARGUE IF 50% OF THIS INNOVATION IN HIV IS ATTRIBUTABLE TO NIH WHICH HELPED DISCOVER THE VIRUS AND LED TO THE DEVELOPMENT OF THERAPIES, IT WOULD JUSTIFY THE ENTIRE NIH SPENDING. ON AVERAGE IT'S BEEN WORTH IT. WHAT JON CORRECTLY POINTS OUT IT'S NOT JUST ABOUT AVERAGES BUT MARGINAL. SOME PEOPLE CALL US FLAT ON THE CURVE. HERE'S FEMALE LIFE EXPECTANCY AT BIRTH. THERE'S BEEN A DECELERATION IN IMPROVEMENTS IN HEALTH. AT THE SAME TIME WE'RE SPENDING A LOT MORE ON HEALTH CARE. IF YOU SAY TO PEOPLE WOULD YOU RATHER HAVE TODAY'S MEDICINE AT TODAY'S PRICES OR 1980 MEDICINE AT 1980 PRICES, THEY PROBABLY CHOOSE TODAY'S MEDICINE. THAT MENS -- MEANS ON AVERAGE WE'RE DOING WELL BUT WHAT ARE WE DOING AT THE MARGIN. JON POINTED OUT RIGHTLY THAT IF WE'RE SPENDING PRIVATE DOLLS OR DOLLARS AND YOU WANT TO WASTE YOUR MONEY AND BUY AN iPHONE 10 THAT'S YOUR PREROGATIVE BUT WE ARE SPENDING OTHER PEOPLE'S MONEY SO WE CARE ABOUT EFFICIENCY. THE MARGINALS ARE A BIG PART OF THE PROBLEM AND JON HAS SHOWN BOTH IN THIS WORK AND HIS COLLEAGUES AT DARTMOUTH THERE'S A LOT OF WASTE IN THE SYSTEM. THE PROBLEM THOUGH IS USE BY PATIENTS WHERE THE BENEFITS ARE SMALLER OR NEGATIVE. WE HAVE A LOT OF INAPPROPRIATE USE AND IT'S DIFFERENT PRACTICES NEP QUESTION. THE QUESTION IS WE DON'T KNOW WHAT IT DO ABOUT IT AND THAT'S POLICY DILEMMA. IN SOME WAYS WE'RE VICTIMS OF OUR OWN SUCCESS. WE'RE LIVING LONGER. ALZHEIMER'S FOR EXAMPLE IS A CONSEQUENCE OF SUCCESS. WE DID A GOOD JOB WITH CANCER AND OTHER DISEASES NOW WE HAVE IT DEAL WITH ALZHEIMER'S. SO PROGRESS IN ONE DISEASE IS LESS VALUABLE. WHAT BHAVEN TEACHES US IS THE LAYING FIELD IS TILTED IN THE WRONG DIRECTION WHEN IT COMES TO THIS. WHAT I WANT TO SAY IS IT'S NOT JUST ABOUT PATENTS BUT REIMBURSEMENTS WHICH IS SOMETHING ARTICULATED WHERE THE FINISH LINE IS NOT FDA REGULATORY APPROVAL BUT WHEN PATIENTS GET ACCESS TO THE DRUG. THE ENTIRE REGULATORY PROCESS IN THE PATENT SYSTEM IS FOCUSSED TOWARDS REWARDING TREATMENT NOT PREVENTION. IT'S VERY HARD TO TEST SOMETHING THAT KEEPS PEOPLE HEALTHY. AND I THINK WHERE THIS WILL BECOME A REALLY IMPORTANT ISSUE IS WITH ALZHEIMER'S DISEASE. WE'RE DOING TRIALS NOW. WE HAVE NEGATIVE RESULTS ANNOUNCED. THERE'S A LOT OF INVESTMENT IN DEALING WITH ALZHEIMER'S BUT IT APPEARS THE BENEFITS MAY EMERGE IF WE TREAT PEOPLE PRECLINICALLY NOT ONCE WE DEVELOPED THE PLAQUES AND TANGLES. AND WE ALL HAVE OUR OWN EXPERIENCE WITH COGNITIVE IMPAIRMENT. IF SOMEONE DEVELOPS AN INNOVATION HOW ARE WE GOING TO GET ACCESS FOR THE PATIENTS WITH WILD COGNITIVE IMPAIRMENT AND HOW ARE WE GOING TO BUILD THE MODEL? LIKE HE SHOWED IN CANCER WHERE WE LOOK AT METASTATIC DISEASE, IT TURNS OUT A LOT OF THE VALUE IS IN THE PREVENTION SIDE AND YET THE PLAYING FIELD IS TILTED AGAINST THAT. SO WE END UP UNDER INVESTING TO PREVENT DISEASE AND DISABILITY AND WE LOOKED AT SOME OF THESE. SUPPOSE WE CAN LOOK AT CALORIC RESTRICTION. THAT'S THE MOST IMPORTANT WAYS TO RESTRICT LIFE SPANNED. IF YOU RESTRICT THE CALORIC INTAKE OF A RODENT YOU CAN INCREASE THE LIFE EXPECTANCY BY 40%. IT'S HARD TO MEASURE QUALITY OF LIFE. A LOT OF ARE SAYING, OKAY. BUT IF YOU LOOK AT HOW MUCH THEY'RE ON THE WHEEL, THEY'RE ACTUALLY ON THE WHEEL MORE. WE TRIED TO DO A QUALITY OF LIFE. AND YOU'RE PROBABLY FAMILIAR WITH THE -- WHAT'S THE SUBSTANCE IN RED WINE -- >> RESVERTROLL. >> IN MICE THE EQUIVALENT I THINK DAVID SINCLAIR SAID 300 BOTTLES A DAY OF RED WINE. >> SPEAKING OF RED, YOU'RE LATE. >> YES, DONE. OKAY. I'LL STOP THEN. BUT IF I CAN POINT OUT THE LAST POINT THOUGH, WE SHOWED THERE'S ABOUT $7 TRILLION IN VALUE AND JON SKINNER IS RIGHT WHEN HE SAYS THERE'S FISCAL IMPLICATION. IF PEOPLE ARE LIVING LONGER THERE'S STILL GOING TO BE A $3 TRILLION INCREASE IN ENTITLEMENT COSTS AND LET ME CONCLUDE PREVENTION IS WHERE THE VALUE LIES BUT IT'S HARD TO REWARD IT. THERE'S ENORMOUS BENEFIT TO GETTING THIS RIGHT. THANK YOU. [APPLAUSE] . >> THANK YOU VERY MUCH, DANA. THE REASON I WAS RUDE IS WE'RE LEAVING TIME FOR GENERAL DISCUSSION. I HAD ONE QUESTION. LOOKING BACK TO THE ORIGINS OF THE PROGRAM, I REMEMBER THE DISCUSSIONS ABOUT A DIDFFUSION AND THERE WAS CYNICISM PARTICULARLY AMONG THE NON-ECONOMISTS IN THE ROOM AND SOME PEOPLE FELT THERE'S NOTHING TO STUDY. IT'S ALL ABOUT THE REIMBURSEMENT RATES AND PROFIT MARGINS AND THINGS THAT MAKE SOMEBODY MONEY WILL BE PUSHED AND DIFFUSED FAST AND THINGS THAT DON'T LIKE BETA BLOCKERS, OFF PATENT OR ASPIRIN WILL TRICKLE OUT SLOWLY AND THAT'S ALL THERE IS TO IT. I WAS STRUCK TODAY. ALMOST NOBODY WITH -- I THINK WITH ONE EXCEPTION FOR JON IN PART OF IT WAS TALKING ABOUT THE ROLE OF THINGS BEING PUSHED OUT TO DOCTORS OFFICES OR TO PATIENTS AND IT WAS A MUCH MORE SOPHISTICATED DISCUSSION OF CHARACTERISTICS THAN WE HAD WHEN WE SET THE PROGRAM UP. DO YOU FEEL THAT THE CYNICISM WAS WARRANTED? >> I THINK ECONOMISTS ARE SUPPOSED TO BE CYNICAL BUT THE CYNICISM COMES FROM A DIFFERENT DIRECTION. THAT THINGS THAT MAYBE AREN'T IN SOME CASES -- THINGS ARE TIED TO FINANCIAL REWARD BUT I'M SURPRISED AT HOW MUCH THERE'S VARIATION ACROSS USE OF THESE PROCEDURES AND MEDICARE WHERE MEDICARE PAYS THE SAME AMOUNT IN SEATTLE AS MUNSTER, INDIANA AND THEY RESPOND DIFFERENTLY TO INCENTIVES. PHYSICIANS AREN'T EVEN REALLY EVEN REIMBURSED. >> SO THE DRUG ELUTING STENT WAS INTERESTING BECAUSE THERE'S NO DIFFERENCE IN PAYMENT. IS REALLY HAS TO DO WITH THE SETTING. WE LOOKED AT OUTSIDE THE DRUG ELUTING STENT WHEN WE LOOKED AT ANTIPSYCHOTIC PRESCRIBING SAME WITH THE BEVACIZUMAB THE ENVIRONMENT MATTERS SO MUCH. >> I JUST WANT TO SAY I DIDN'T HAVE TIME TO GET INTO THIS BUT THERE'S A BIG DIFFERENCE ACTUALLY BETWEEN DRUGS AND BIOLODGICS AND PROCEDURES. THERE'S AN IRONY. THIS SUMMER, FOR EXAMPLE, BEVACIZUMAB WE ENDED UP GETTING FDA APPROVAL FOR TWO BIOSIMILARS. IN HIV, BILL GATES JUST ANNOUNCED HE'LL BE GIVING A DEAL WITH GENERICS FOR $75 A YEAR SO YOU CAN TREAT HIV. THESE ARE EXAMPLES OF INNOVATIONS WHILE THEY'RE EXPENSIVE IN NOMINAL TERMS INITIALLY THE PRICE COMES DOWN. ON THE OTHER HAND, IN DRUG ELUTING STENTS AND DEFIBRILLATORS IN THOSE INDUSTRIES WAGES ARE GOING UP. IN SOME CASES PEOPLE CALL THAT BOMELL'S DISEASE AND IN THOSE AREAS THE COST OF NOT DOING EXNOVATION IS HIGHER AND WE SHOULD DISTINGUISH THOSE SERVICES. >> ANY QUESTION OR POINTS? DAVID. >> A COMMENT/QUESTION FOR BOB. IF THE FIRST PAPER YOU TALKED ABOUT WITH HEIDI YOU TALKED ABOUT THE CHALLENGE OF TRIALS FOR CANCER WITH BETTER SURVIVAL RATES AND REMINDED ME OF A LITERATURE PAPER BY KEN WARNER A NUMBER OF YEARS AGO SOMETHING LIKE THE DESPERATION MODEL OF MEDICAL DIFFUSION. HE SHOWED IN THE DEVELOPMENT OF CHEMOTHERAPY FOR MALIGNANCIES ADOPTION WAS FASTER FOR ACUTE THAN MALIGNANT. I WONDER WHETHER SOME OF THE SAME STORIES IS WHAT'S GOING ON IN THE CANCER CLINICAL TRIALS. I THINK IT IS OF GREAT RELEVANCE TO NIH. WHEN YOU THINK OF HOW YOU PULL PEOPLE IN TRIALS YOU HAVE DIFFERENT MOTIVATION AND I WONDER IF THERE'S A ROLE GIVEN NIH'S IN PUT IN HOW TO DESIGN MORE EFFECTIVE TRIALS WHERE INDIVIDUALS AREN'T SO PERSONALLY MOTIVATED BECAUSE OF LACK OF OTHER OPTIONS. >> TO BE CLEAR THE FIRST PAPER I DID NOT WRITE. BUT I HAD THE SAME REACTION. I THOUGHT THERE'S SOMETHING GOING ON, SOMETHING GOING WE COULD CALL DEMAND BUT IT'S DIFFERENT. pGOOD TO THINK ABOUT.OMMENT AND GOOD, SO -- >> I SHOULD POINT OUT THAT HEIDI WAS GOING TO JOIN US AS WELL AS BHAVEN BUT IS ON OBSTETRIC LEAVE, EVERY TIME WE HAVE A BIG CONFERENCE IN THE PROGRAM IT CAUSES A PREGNANCY. >> I HAVE A QUESTION. I WONDER IF THERE'S AN OPPORTUNITY IT LOOK AT OR QUANT TATE WHAT THE IMPACT OF FURTHER RESEARCH AFTER INTRODUCTION OF TECHNOLOGY HAD ON DIFFUSION OR CHANGE IN POLICY OR GUIDELINES. WAS THAT HART OF YOUR INVESTIGATIONS? >> I'M ANSWER IN TERMS OF THE DRUG ELUTING STENT. THE POLICY OR INVESTIGATION RELATED TO THE FDA MEETING AND WHERE THERE WAS A STATEMENT TO BE CAREFUL ABOUT WHOM YOU'RE IMPLANTING THESE THINGS SO THAT WAS WHY WE LOOKED AT THE TWO TIME PERIODS. IN TERMS OF THE OTHER ISSUES IN TERMS OF BEVACIZUMAB, THE POLICY IS A LITTLE BIT EMBEDDED IN THE APPROVED OR NON-APPROVED USE. >> WE DID INCORPORATE THE DATES WHEN NEW INFORMATION WAS RELEASED WITH THE BREAST CANCER INDICATION WAS RESCINDED AFTER NEW INFORMATION WAS AVAILABLE. WE DID TRY TO INCORPORATE THAT AND SEE IT AFFECTED THE PATTERNS. >> SORRY, I DIDN'T ASK EVERYBODY AT THE BEGINNING TO INTRODUCE OURSELVES BECAUSE IT WOULD TAKE A LOT OF TIME. AS GRADUALLY WE'LL ALL GET TO KNOW EACH OTHER WHEN YOU SPEAK FIRST CAN YOU INTRODUCE YOURSELF. BHAVEN, PLEASE. >> THANKS, I'VE BHAVEN. AND THIS IS PROMPTED MOSTLY BY JON'S PRESENTATION BUT IT'S A QUESTION ABOUT DIFFUSION MORE GENERALLY. I WHEARD TACK IN THE '90s LOCK-IN ON VHS AND BETA AND YOU CAN GET LOCKED IN AND WHY YOU MAY NOT SWITCH TO SOMETHING BETTER OR MAYBE INCLUDING NOTHING. AND SO THE QUESTION IS LIKE HOW MUCH OF THE PROBLEMS WITH EXNOVATION IF YOU WANT TO CALL IT THAT ARE ABOUT BEING LOCKED INTO SOMETHING VERSUS -- IS THAT PART OF THE CONVERSATION? >> I THINK I'LL DEFER FOR THE DRUG ELUTING STENTS IT'S A GREAT EXAMPLE. >> SO PART OF IT IS -- SO THE SUPPLIES. SO IF YOU LOOK AT THE SUPPLY IN SWITCHING FROM ONE TYPE OF DRUG ELUTING STENT FROM FIRST TO SECOND GENERATION IS NOT PROHIBITIVE. AND THE SUPPLY IS UPDATED REGULARLY. SOMETHING LIKE THAT THERE'S NO LOCK-IN IN THAT CASE. I THINK SOMETHING -- LIKE WE DIDN'T TALK ABOUT IMAGING AND TEST. WHEN I INVEST IN BIG EQUIPMENT AND TOMOSYNTHETOMOSYNTHET -- >> THIS IS WHERE POLICY CHANGES CAN MATTER AND YOU TAKE CHANGES IN FORMU LARY DESIGN AND WHERE PAYMENT WOULD BE DIFFERENTIATED THEY PAY MORE ATTENTION. >> IF YOU WERE TRAINED TO DO SOMETHING AND DID A LOT OF THEM AND BY THE WAY, THEY'RE A HIGH FRACTION OF YOUR PROCEDURES THAT WAS YOUR BREAD AND BUTTER IT BECOMES HARDER TO DROP IT. >> I THINK WE HAVE TO BE CAREFUL ABOUT HOW FAST OF EXNOVATION WE WANT. BECAUSE LOCKING THERE'S LEARNING BY DOING AND IF YOU THINK ABOUT HETEROGENEITY IN TREATMENT AND EFFECT YOU HAVE TO UNDERSTAND MAYBE THEY'RE FIGURING OUT SOME PEOPLE WILL BENEFIT FROM THE OLD TECHNOLOGY AND IT'S GOING TO THE DIFFERENCE BETWEEN AVERAGE AND MARGINAL THAT OVER TIME THE TREATMENT MAY BE CHANGING AND YOU DON'T WANT TO FORCE PEOPLE TO EXNOVATE EVEN FROM THE MARGINAL PATIENT. >> CAN I -- >> ONE OF THE THINGS WE'RE FINDING IS A REMARKABLE LACK IN THE ICDs WHERE WE HAVE GOOD REGISTRY DATA. WE WERE EXPECT DOING SEE THAT SURGEON THE ELECTROPHYSIOLOGIST GET BETTER AT WHAT THEY DO FOR A PATIENT AND IN THINK IN PART IF YOUR A PRACTICING SURGEON YOU'D LIKE FOR THE WORST PRODUCERS TO DROP OUT. YOU WANT TO KEEP THE BEST DOING IT AND THE WORST TO DROP OUT BUT WE DIDN'T SEE A LOT OF EVIDENCE OF THAT. >> IN SOME SENSE THIS IS DUE TO REIMBURSEMENT BECAUSE WE HAVE A SYSTEM THAT SAYS WE'RE GOING TO REWARD SOMETHING AND IT'S CALLED VOLUME. SO WE DO A GOOD JOB AT THAT AND WE GET A LOT OF VOLUME. IF YOU HAD A SYSTEM THAT REWARDED PAYMENTS FOR HEALTH, I WAS SURPRISED LIKE HERE IN THE ELEVATOR THERE'S SIGNS EVERYWHERE SAYING TAKE THE STAIRS. I'M AN ECONOMIST. I THINK IF YOU PUT A COIN OPERAT OPERATED MACHINE ON THE ELEVATOR YOU'LL GET PEOPLE TO WALK QUICKLY. >> I WANTED TO COMMENT ON THINKING ABOUT THE POINTS RAISED RELATIVE TO THE TECHNOLOGY. YOU NEED TO THINK ABOUT IT THAT WAY. THE REASON WHY I'M SAYING THAT IS IF WE'RE TALK PROCEDURES OR DEVICES, I THINK WE NEED TO THINK DIFFERENTLY AND THE REASON WHY THAT MAY HAPPEN IF YOU HAVE AN IMPLANTABLE DEVICE SUCH AS A STENT IT'S THIS. IT'S TAKING REALITY -- REAL ESTATE IN YOUR BODY. IT'S NOT COMING OUT. IN SOME CASES WE NEED TO BE CAREFUL ABOUT LEARNING IN SOME SENSE. I THINK ANOTHER POINT TO MAKE IS EVEN AT THE TIME IT DIFFUSES, I DO A LOT OF WORK IN MEDICAL DEVICE. WHEN YOU LOOK AT DIFFUSION AND A NEW DEVICE THERE'S A HUGE TECHNICAL LEARNING CURVE. >> SINCE WE ARE RECORDING, IF YOU HAVE A QUESTION OR A COMMENT USE THE MIC. >> THE ONE PLACE -- >> BOB, SORRY. >> SORRY, ONE PLACE I THINK WE'LL SEE AN INCREASE IN SURGERIES IN THE JOINT REPLACEMENT. I THINK THE METAL ON METAL STORY IN TERMS OF DIFFUSION AND INNOVATION ARE INTERESTING. ANY COMMENTS THERE? >> SAY THERE WAS A NEW TYPE OF DEVICE METAL ON METAL AND RELEASED TO THE BLOOD AND WAS BAD FOR YOU. >> SO THE METAL ON METAL HAS BEEN AROUND A LONG TIME. AND IT WAS THAT SAFETY SIGNAL. THAT PROBLEM DIFFUSED QUICKER IN AUSTRALIA, OUTSIDE THE U.S. THAN WITHIN THE U.S. PART IS RELATED TO THE LACK OF INFORMATION. IF YOU LOOK AT BILLING CLAIMS YOU CAN SEE WHETHER IT'S METAL ON METAL, METAL ON PLASTICS. THERE'S AN INFORMATION LACKED AND IN BILLING CLAIMS PEOPLE CAN STUDY THE CLAIM. BUT METAL ON METAL IS BAD. THEY'RE SUPPOSED TO STAY IN FOR TEN YEARS -- >> THE SCOTT'S PARABOLA. >> I HEARD SOMEBODY THE OTHER DAY SAID SAYING I NEED A HIP REPLACEMENT AND WHAT KIND SHOULD I GET METAL ON METAL AND I SAID NO. >> I'M KAREN HUSK FOR THE NURSING ASSOCIATION OF MEDICAL RESEARCH. WE HAVE TO LOOK AT THE POPULATION AND THIS MEETING IS THROUGHOUT THE LIFE CYCLE. SOMETIMES OUR STUDIES ARE DONE WITH YOUNG INDIVIDUALS OR MIDDLE AGE. WHAT ABOUT THE USE IN THE ELDERLY OR WE DON'T CONSIDER USING THEM IN THE ELDERLY BECAUSE WE THINK NO, THEY CAN'T, WHATEVER SO I THINK IT'S PERSON TO MAKE SURE WE CONSIDER IN TERMS OF WHAT WE'RE TALKING ABOUT THE HEALTH AND WELL BEING OR THE HEALTH OUTCOMES TO INCORPORATE THAT DATA AS WE'RE LOOKING AT THE ECONOMIC PIECES TO THESE. >> I COMPLETELY AGREE. ONE OF THE THINGS THAT WE HAVE TRYING TO DO WITH THE IMPLANTABLE DEFIBRILLATORS IS GET HOLD OF THE RANDOMIZED TRIAL DATA WHICH SAY YOUNGER POPULATION THAN YOU SEE IT BEING PERFORMED IN IN THE MEDICARE POPULATION. ALSO MOVING TO THE UNDER 65 COMMERCIAL DATA IS SOMETHING WE'RE TRYING TO DO AND IS VERY IMPORTANT. BUT IT'S TRICKY. IT'S VERY DIFFICULTY TO GET ACCESS TO THAT DATA. >> SO I'LL JUST COMMENT. SO THE DRUG ELUTING STENT WAS DONE ON SOMEBODY 18 YEARS AND OLDER IN THE REGISTRY AND THERE ARE A LOT OF 85-YEAR-OLDS IN THE SET THAT GET DRUG ELUTING STENTS THOUGH THEY HAVE TO BE ON BLOOD THINNING THERAPY AND WORRY ABOUT STROKE. THAT'S ONE THING. I LIKE YOUR QUESTION BECAUSE I DON'T THINK THERE'S MUCH STUDY OF DEVICES IN CHILDREN. THE APPROVAL PATHWAY'S DIFFERENT AND IT'S SOMETHING I BELIEVE IS UNDER STUDIED. >> I HAVE A QUESTION FOR ALL THE SPEAKERS ON AND THE PANELISTS, ACTUALLY. WHAT ARE THE NEXT STEPS WE SHOULD BE LOOKING FORWARD TO? WHAT ARE THE NEXT STEPS OF RESEARCH WE SHOULD BE SUPPORTING TO SUPPORT THE HEALTH AND JON TALKED ABOUT THE WELFARE. BOTH AREAS. THANKS. >> HAIDEN. >> I'LL START BECAUSE IT'S SO SIMPLE AND IT MAY BE LAUGHABLE. WE DON'T HAVE THE INFORMATION WE NEED TO REALLY UNDERSTAND WHAT IS CAUSING THE VARIATION. AND WHAT ARE THE LEVERS IN ORDER FOR US TO PULL IN TERMS OF POLICIES. INFORMATION IS REALLY LACKING. DON'T HAVE THE CLINICAL DETAILS. HAVING ACCESS TO THE -- I SOUND LIKE A RESEARCHER WHICH I AM BUT HAVING ACCESS TO THE RANDOMIZED CONTROL TRIAL DATA. HAVING INFORMATION THAT INFORMATION THAT DEFINES THAT PROVIDER, ETCETERA. I START WITH A SIMPLE ONE. >> IF I CAN FOLLOW -- I COMPLETELY AGREE. ONE OF THE THINGS THAT WOULD BE INCREDIBLY HELPFUL IS BETTER ACCESS TO THE CARDIOVASCULAR REGISTRY DATA. CANNOT GET IT. IT'S EASIER TO GET INTO EQUIFAX THAN THIS IS TO GET THAT DATA AND THEY -- >> AND IT'S PRETTY EASY. >> AND THEY'RE JUST SITTING ON IT AND NOBODY LETS ANYBODY USE IT. ALSO I THINK I'M INTRIGUED BY SURVEYS AND YOU CAN IDENTIFY LIKE SOME OF YOUR PRACTICES WHERE THEY JUST JUMPED ON THE DRUG AND OTHERS WHERE THEY STEP pAND GIVING DOCTORS VIGNETTES AND SAYING WHAT WOULD YOU DO FOR THIS KIND OF PATIENT AND YOU CAN LOOK AT OBSERVABLE DATA BUT NOTHING SUBSTITUTES FOR GETTING INSIDE AND SEEING HOW PEOPLE THINK IT THROUGH. EVEN IF THEY KNOW THEY'RE RAPID ADOPTERS OR NOT. THEY MAY NOT EVEN KNOW THAT. >> THAT'S A VERY GOOD POINT ABOUT THE DEVELOPMENT OF DATA RESOURCES AND THINGS. I'LL POINT OUT LATER ON TODAY AT 3:15, WE'RE GOING TO COMBINE A BREAK WITH SHOW AND TELL AND SOME OF THE RESOURCES INCLUDING THE ONE THAT JON MENTIONED DEVELOPED UNDER THE COOPERATIVE AGREEMENT AND PEOPLE WILL BE IN ROOMS AND WE'LL BE HAPPY TO TALK ABOUT IT AND WE TRIED TO MAKE IT A START WITH DEVELOPING THE RESOURCES NEEDED. REBECCA, CAN I ASK YOU TO RESPOND ON THE QUESTION OF TILT. >> THIS IS COVERED. THE ISSUE OF TESTING -- >> REBECCA CLARK. >> THE NATIONAL INSTITUTE OF HEALTH AND HUMAN DEVELOPMENT. IT WAS FOUNDED TO LOOK AT THE ISSUES OF CHILDREN SEPARATE FROM ADULTS BECAUSE BEFORE IT WAS FOUNDED IN '62, I BELIEVE THE IDEA WAS CHILDREN WERE LIKE LITTLE DAULT -- ADULTS. NO PARENTS HAD BEEN ASKED AND THAT'S THE ISSUE. AND CONGRESS PASSED THE PHARMACEUTICALS ACT FOR CHILDREN WHICH IS RAN OUT OF HICHD. THERE'S A BRANCH THAT SPENDS ALL OF ITS TIME ON DRUG DEVELOPMENT FOR CHILDREN AND PREGNANT WOMEN AND I CAN'T OFF THE TOP OF MY HEAD RIGHT NOW REMEMBER THE NAME OF THE BRANCH BUT IF YOU JUST SUCH HICHD AND BEST PHARMACEUTICALS ACT FOR CHILDREN THERE'S A HUGE PUSH TO ADDRESS THE EXACT ISSUE ABOUT UNDER STUDIED CHILDREN AS AN UNDER STUDIED GROUP. >> THE PROBLEM -- THE PROBLEM WITH DEVICES AT LEAST IN RUN OF THE MILL DATA CLAIMS THERE'S NO WAY TO IDENTIFY THE DEVICE BECAUSE IT'S NOT LIKE AN NDC WHICH IS AVAILABLE FOR PHARMACEUTICAL. DEVICE IDENTIFIERS DON'T EXIST. I KNOW IF IT'S A BARE METAL STENT BUT I DON'T KNOW IF IT'S BOSTON SCIENTIFICS STENT VERSUS NOT. >> AND I'M MARIE BERNARD DIRECTOR OF THE NATIONAL INSTITUTE ON AGING. AND BUILDING ON WHAT REBECCA WAS SAYING IN THE CURES ACT SIGNED IN DECEMBER OF 2016 THERE WAS A MANDATE FOR NIH TO CONSIDER THE INCLUSION ACROSS THE LIFE SPAN. CHILDREN AND OLDER ADULTS AND IN FACT BY DECEMBER 13 OF 2017 THE NIH HAD BEEN DIRECTED TO MAKE DECISIONS WHETHER THERE'S NEEDS FOR POLICY CHANGES HERE. CERTAINLY THERE WERE A LOT OF DATA THAT WERE SHOWN TO NOT HAVE INCLUSION OF CHILDREN AND INCLUSION OF OLDER ADULTS IN KEEPING WITH THE DISEASE ARE COMMON IN THE POPULATIONS. SO STAY TUNED. SOME TIME SHORTLY AFTER DECEMBER YOU'LL PROBABLY HEAR ABOUT THAT. >> THANK YOU VERY MUCH. I'M AFRAID I'M GOING TO HAVE TO CUT OFF AND GIVE EVERYONE THE BREAK OTHERWISE I'M SETTING THE BAD EXAMPLE. THAT CLOCK IS WRONG. WE'LL REASSEMBLE AT PRECISELY 10:55. >> BOB CARTER. PART OF THE IMPETUS FOR THIS SESSION WITH ONE THING THAT CAME OUT THAT WAS NEWS TO ME WAS THE CONCEPT OF VALUE OF INFORMATION. A VERY SPECIFIC MEANING OF WHAT DO WE LEARN FROM DIFFERENT TYPES OF CLINICAL OBSERVATIONAL AND RANDOMIZED TRIALS AND IN THE SENSE OF THIS SESSION THERE'S THE MACRO AND THE MICRO. WHEN I TALKED TO DAVID ABOUT IT HE SAID I CAN TALK ABOUT THAT BUT IT'S A VEHICLE FOR WHAT I REALLY WANT TO TALK ABOUT. I'M INTERESTED WHERE HE GOES WITH THAT. >> I'M GOING TO DO WHAT'S ASKED AND TALK ABOUT VOI THOUGH I LIKE TALKING ABOUT OTHER THINGS TOO. I'M GOING TO TRY TO GET AN OVERVIEW FOR PEOPLE. AND SOME PEOPLE IN THE ROOM WILL KNOW EVERYTHING I'M SAYING AND FOR OTHERS IT WILL BE NEW AND VALUABLE. IT'S NO SURPRISE WE HAVE FINITE FUNDS FOR RESEARCH AND WE HAVE TO MAKE DECISIONS WHAT TO PAY FOR. . THE IDEA IS TO GET QUANTITATIVE ESTIMATES OF THE VALUE OF RESEARCH. I'LL START BY REVIEWING THE THEORY BEHIND THAT. THEN I'LL TRY TO DRILL DOWN A LITTLE BIT AND TALK ABOUT THE QUESTION OF WHETHER VOI CAN INFORM PRIORITIES FOR RESEARCH. IT'S NOT FREE. IT'S COSTLY. YOU HAVE TO THINK ABOUT ITS MERITS RELATIVE TO OTHER APPROACHES. THE EARLY APPROACHES ARE PRETTY COSTLY TO IMPLEMENT. A NUMBER OF US HAVE TRIED TO FIGURE OUT HOW WHEN VOI CAB -- CAN BE DONE MORE COST EFFECTIVELY. WE'VE LONG USED TOOLS LIKE COST EFFECTIVENESS TO ASSESS THE VALUE OF MEDICAL TREATMENTS AND WHAT COMES FROM INTERVENTIONS INCLUDING DIAGNOSTIC TESTS AND WE LOOK AT COST FOR QUALITY AND IDEAS SUCH AS NET HEALTH BENEFITS HAVE RISEN TO QUANTIFY THE NET VALUE THAT COMES FROM VARIOUS TECHNOLOGIES. WHAT THE NEW VALUE OF INFORMATION TECHNIQUES HAVE DONE HAVE EXTENDED THE IDEAS TO EXTEND THE VALUE OF MEDICAL RESEARCH. HERE'S THE BASIC INTUITION. YOU CAN THINK ABOUT IT THE WAY YOU THINK OF A DIAGNOSTIC TEST. YOU DO A TEST AND FIGURE OUT IF SOMEONE'S SICK OR HEALTHY AND GET THE UTILITY OF DOING THE RIGHT THING AND TREATING THEM IF THEY'RE SICK AND THE EXPECTED VALUE OF THE OUTCOME IS THE WEIGHTED PROBABILITY OF THE OUTCOMES OF ALWAYS THE RIGHT THING BUT IN SOME CASE TREATING WHEN YOU'RE SICK AND IF YOU'RE NOT SICK AVOIDING THAT. ON THE OTHER HAND, IF WE DIDN'T DO THOSE TESTS WE'D NOT KNOW WHETHER PEOPLE WERE SICK OR HEALTHY AND WOULD HAVE TO MAKE A DECISION. ALWAYS TREAT OR NOT TREAT AND MAKE CERTAIN MISTAKES. THIS EXPECTED VALUE IS ALWAYS GOING TO BE GREATER THAN THE EXPECTED VALUE SO THE VALUE OF A DIAGNOSTIC TEST IS THE DIFFERENCE AND IT'S THE SAME FOR THE PROPOSED RESEARCH STUDY. IT COULD BE A SUPER OR HORRIBLE IDEA TO DO THE STUDY AND THE DIFFERENCE IS THE SAME. THIS IS THE INTUITION BEHIND IT AND IN THE ABSENCE OF INFORMATION WE MAKE THE COPROMISED CHOICE ABOUT A QUESTION AND WITH PROBABILITY THAT WAS THE RIGHT DECISION TO pWITH PROBABILITY.ET THE VALUE ONE MINUS P WAS NOT THE RIGHT DECISION TO HAVE MADE AND WE GET THE LESSER VALUE. WITH INFORMATION WE ALWAYS MAKE THE DECISION THE PERFECT INFORMATION AND KNOW THE TRUE STATE OF THE WORLD SO WE ALWAYS GET THE BEST OUTCOME. VALUE OF INFORMATION IS THE DIFFERENCE BETWEEN THE EXPECTED OUTCOME WITH THAT INFORMATION RELATIVE TO THE EXPECTED OUTCOME WITHOUT THAT INFORMATION. AND IF YOU WORK THROUGH THE SIMPLE MATH OF THAT IT BREAKS DOWN TO TWO VERY SIMPLE AND INTUITIVE TERMS. ONE IS WHAT'S THE PROBABILITY WE MADE THE WRONG CHOICE IF WE MADE THE COMPROMISED DECISION NOT KNOWING THE TRUE STATE OF THE WORLD AND THE PRODUCT THE TWO TERMS IS THE VALUE OF RESEARCH. AND IF THERE'S ONE THING ECONOMISTS ARE GOOD AT IS MULTIPLYING THINGS AND THIS IS THE FIRST EXAMPLE AND COME TO ANOTHER LATER. THIS IS THE BASIC INTUITION. LET ME COMMENT, IF YOU LOOK CAREFULLY YOU SEE YOU NEED TO KNOW VALUE AND SOMETHING ABOUT THE PROBABLES OF VARIOUS STATES OF THE WORLD BEFORE YOU DO THE STUDY AND POTENTIALLY AFTER THE STUDY. SO YOU CAN THINK ABOUT THAT AS THESE CALCULATIONS DEPENDING ON A BUNCH OF INFORMATION. SOMETHING ABOUT THE VALUE OF VARIOUS OUTCOMES. YOU CAN THINK OF THAT AS URGENT ILLNESS AND SOMETHING ABOUT PRIORS AND POSTERIORS AND WHEN YOU DO A TRUE EXPECTED VALUE OF INFORMATION YOU WANT TO KNOW BOTH. YOU NEED INFORMATION ON BURDEN AS WELL AS PRIORS AND POSTERIORS. THERE'LL BE THINGS LEFT OUT LIKE SERENDIPITY AND THE BASIC RESEARCH QUESTION AND THE MORE IMPORTANT IT WILL BECOME AND IT GETS HARD IN THE CONTEXT TO USE THE METHODS. IT'S THE IDEAL SITUATION IN TERMS OF INFORMATION AND WE OFTEN DON'T HAVE THIS INFORMATION AND WE CAN OVERCHARACTERIZE THE BURDEN OF ILLNESS AND IF YOU THINK ABOUT THE WAY MOST SIGNIFICANT SECTIONS ARE WRITTEN IN MANY GRANTS SADLY THIS IS STILL THAT. A LOT OF PEOPLE DIE FOR CANCER THEREFORE EVERY CANCER RESEARCH SHOULD BE GRANTED. THAT'S NOT A LOGICAL APPROACH. HAVE TO THINK ABOUT HOW DO YOU REALLY KNOW NOW ABOUT THE PARTICULAR INTERVENTION WE'LL ENGAGE UP AND CAN WE GET MORE INFORMATION BASED ON THE STUDIES SO THE PRIOR AND POSTERIORS. OFTEN TIMES IT'S PEOPLE ARE NOT PRECISE ABOUT HOW MUCH CLARITY WE'LL HAVE AFTER A STUDY IS DONE. WE MAY NOT HAVE INFORMATION ON THE POSTERIOR AND IN CALCULATING ANALYSES IS PEOPLE ARE QUIET ABOUT THE POSTERIORS AND ONLY TALK WITH THE PRIORS AND ASSUME THEY CAN RESOLVE ALL UNCERTAINTY AND THAT'S MAXIMUM BOUND TONE VALUE OF INFORMATION. THE TRUE VALUE WON'T BE GREATER THAN THAT. A QUESTION IN THE TABLE YOU GET MORE AND MORE PRECISE ESTIMATES AND ONE IMPORTANT QUESTION IS HOW INFORMATIVE ARE THE BOUNDS BASED ON PRIOR RELATIVE TO THE REALITY YOU SHOULD EXPECT. LET ME SHIFT GEARS AND TALK ABOUT THE PACT CAL APPLICATION OF THIS IDEA. GOING BACK TO DANA'S ECONOMIST WE MULTPLY AND ADD. IT DRILLS DOWN TO THE INDIVIDUAL LEVEL OF INFORMATION WHICH I TOLD YOU ABOUT AND THE LIKELIHOOD OF A DECISIONS WOULD CHANGE AND YOU DON'T THINK ABOUT THE NUMBER OF PEOPLE INVOLVED. HOW LIKELY IS IT ANY DATA GENERATED WILL BE IMPLEMENTED. HOW LONG IS IT LIKELY AND HOW DURABLE IS IT LIKELY TO BE. THERE'S ANOTHER STUDY WITHIN SIX MONTHS. WHAT'S THE VALUE OF THIS ONE AND THEN SOME DISCOUNT FACTOR REFLECTING THE VALUE IN THE FUTURE OPPOSED TO THE PRESENT. YOU CAN THINK OF THE ELEMENTS AS COMING TO THESE SORTS OF EQUATIONS. WE'LL EVEN TALK ABOUT A COUPLE SPECIFIC METHODS AND I'LL ILLUSTRATE THESE. ONE IS TO USE VOI BASED ON DITION -- DECISION AND LOOK AT ALZHEIMER'S DISEASE TREATMENT WORKED ON. M MINIMAL APPROACHES THERE'S INFORMATION DIRECTLY FROM A CLINICAL TRIAL. THERE'S TWO FLAVORS. ONE IS LIMITED MODELLING. YOU CAN BOOTSTRAP AND EXTRAPOLATE OUT TO THE POPULATION LEVEL. IN THIS CASE WE HAVE TO DO A SMALL AMOUNT OF MODELLING TO BRING THAT OVER A LIFE TIME. WE'LL ALSO TALK ABOUT NO-MODELLING EFFECT WHERE'S THE FULL SET OF BENEFITS AND COST ARE DESCRIBED AND I'LL TALK ABOUT MORE LIMITED DATA. HERE'S AN EXAMPLE OF A FULL MODELLING APPROACH. THIS IS PETER'S WORK. THE QUESTION IS WHAT'S THE VALUE OF FURTHER RESEARCH ON INHIBITORS IN ALZHEIMER'S DISEASE. A MARK-UP MODEL DESCRIBING VARIOUS FORMS OF DISEASE TO DEATH. CHARACTERIZIZATION OF THE INCREMENTAL NET BENEFIT AND THERE'S LITTLE UNCERTAINTY FOCUSSED AROUND ZERO IN SHORT TERM BUT LARGER UNCERTAINTY FOR LONGER TERM FOLLOW-UP. YOU CAN SEE THIS IN THE CAL CALCULATIONS FOR RESEARCH AND IT'S LOW FOR SHORT TERM STUDIES AND HIGHER FOR LONGER TERM STUDIES. IT DROPS AT EITHER END BECAUSE RESEARCH ISN'T VAL YOU'LL AND IF HEALTH IS INFINITELY VALUABLE YOU DO ALL RESEARCH. SO THERE'S NO QUESTION AND YOU CAN CHARACTERIZE THE CONTRIBUTORS. ALMOST ALL THE VALUE OF INFORMATION CAME FROM THE UNCERTAINTY OF THE EFFICACY DURATION. IT'S A NICE EXAMPLE HOW TO DO THIS BUT YOU HAVE TO BUILD A MODEL AND THAT'S TIME CONSUMING. THERE'S MINIMAL MODELLING APPROACH AND THAT WAS A LARGE NIMH TRIAL AND THEY CONCLUDED THEY WERE ALL FAIRLY SIMILAR AND SINCE THE NEWER ATYPICALS WERE MORE EXPENSIVE THEY ARGUED YOU SHOULD USE THE CHEAPER DRUG. IF YOU LOOK AT SOME OF THE STUDY RESULT AND COST EFFECTIVENESS RESULT FOLLOW, WHAT YOU SEE IS ALL THE DRUGS LOOKED FAIRLY SIMILAR IN EFFECTS ON QUALITY AND THE ATYPICALS ARE MUCH MORE EXPENSIVE THEY CON INCLUDE -- CONCLUDED IS GO WITH THE CHEAPER DRUG. THERE'S A FAIR BIT OF VARIABILITY. IT MAY NOT LOOK LIKE MUCH AT THE INDIVIDUAL LEVEL. IT'S ON A QUALITY SCALE .05 VERSUS NOTHING BUT THERE'S A LOT OF PEOPLE WITH SCHIZOPHRENIA OVER A LOT OF YEARS AND THERE'S UNCERTAINTY OF COST YOU NEED TO TAKE IN ACCOUNT. WHEN YOU MULTIPLY THE NUMBER OF PEOPLE BY NUMBER OF YEARS YOU SEE EACH COHORT HAS $6.6 BILLION AND THE NEXT 20 WE'RE AT $342 BILLION SO YOU GET A DIFFERENT RESULT. A NICE THING YOU CAN DO WITH VOI IS LOOK AT OPTIMAL SAMPLE SIZES THEY SHOULD BE ON THE ORDER OF $90,000 AND THE OTHER TRIAL IS 800. BIG DIFFERENCES. QUICKLY, EXAMPLES OF THE NO-MODELLING APPROACH. FIRST EXAMPLE IS IN SINUSITIS. AND ONE IS MORE EFFECTIVE SHOWN IN SMALL RCTs BUT IT'S MORE EXPENSIVE. AND ONE OF THE NICE THINGS ABOUT SINUSITIS AND THE RCTs IT TURNS OUT IF YOU GIVE PEOPLE OTHER DRUG THEY GET BETTER. YOU CAN COVER THE OUTCOME TO COMPLETE RESOLUTION. A AND WE THEN DO A VOI CALCULATION BY CALCULATING THE NET BENEFIT WHICH ESSENTIALLY COMES OUT TO THE WILLINGNESS TO PAY FOR DAYS OF SYMPTOMS MINUS COST AND CALCULATE VOI FOR POPULATION. WE DISCOVERED THE VOI FOR IS IT BETTER OR COMPARED THE TWO IS NOT WORTH MUCH IF YOU'RE JUST LOOKING FOR EFFICACY. IT'S ABOUT $40 MILLION WORTH OF VALUE. NOT GOING TO GET EVERYBODY EXCITED IN THE CONTEXT OF HEALTH CARE. ON THE OTHER HAND, ONCE YOU BRING IN COST BECAUSE IT'S MORE EXTENSIVE IT BECOMES MORE UNCERTAIN THAN THE VALUE INCREASES TO $400 MILLION. THIS IS THE EXAMPLE THAT CAN BE DONE IN HOURS RATHER THAN MONTHS OF MODELLING. HERE'S ANOTHER EXAMPLE OF PANCREATIC CANCER. UNFORTUNATELY IN THIS CASE THE SURVIVAL CURVE TO COMPLETION MEANS DEATH BECAUSE ESSENTIALLY THEY ALL STUDY AND WE BUILT A MODEL. ORIGINALLY WHEN I DID THE WORK WE BUILT NICE ALGORITHMS AND EXTENDING THE WORK FOR NHLDI AND OTHER WAYS SO THAT WILL BE A NICE RESOURCE FOR PEOPLE. I WANT TO MENTION TWO LAST IDEAS. ONE IS WHAT WE'RE CALLING CONCEPTUAL VALUE OF INFORMATION. IT GOES BACK TO THE DANA GOLDMAN ECONOMIST MULTIPLY COMMENT. IF YOU THINK ABOUT THE EQUATION IT'S MULTIPLICATIVE AND IF THERE'S BETTER DATA IT DOESN'T MATTER. IF ANY OF THESE ARE ZERO YOU'RE DONE SO YOU GO THROUGH AND CHECK THEM. WE'VE DONE REVIEWS OF VARIOUS GRANT REVIEW PROCESSES AND OTHERS YOU'D BE SURPRISED THE NUMBER OF TIMES ONE OR ANOTHER OF THE FACTORS HAS BEEN NEGLECTED BUT NO ONE PICKED UP ON IT. IT'S A NICE WAY TO DO IT. AND VOI METHODS HAVE NOT BEEN STANDARDIZED. THEY TRIED TO STANDARDIZE METHODS ACROSS STUDIES AND A STRIKING THING YOU SEE IS HUGE VARIATION. SOME STUDIES AREN'T VALUABLE AND OTHERS LIKE IN MENTAL HEALTH IN PARTICULAR THAT ARE ABSOLUTELY HUGE NUMBERS SO THEY'RE MAGNITUDE EFFECTS. IF THE WORK FOR ARC WE LOOKED AT THE LIMITED MODELLING APPROACHES AND TRIED TO CHARACTERIZE THEIR VARIOUS ADVANTAGES AND DISADVANTAGES. ARC ASKS A QUESTION CAN WE USE VIO FOR SYSTEMATIC REVIEWS AND WE TRIED TO DEVELOP AN APPROACH FOR THE BENEFITS AND COSTS. WE SAW THE MINIMAL MODELLING AND FULL MODEL FROM LEAST TO MOST COSTLY WAS PROBABLY NOT THE RIGHT WAY TO DESCRIBE ONLY THERE WAS ANOTHER IDEA WHICH WAS MAXIMUM MODELLING. IT'S THE BUILDING OF A MODEL FOR AN ENTIRE DISEASE FRAMEWORK LIKE A CARDIOVASCULAR OUTCOME TO DISEASE MODEL. THOUGH IT'S COSTLY TO DO ONCE IT'S DONE IT MAY BE CHEAP TO APPLY IT FOR PARTICULAR STUDIES. HERE'S THE POLICY MODEL. ONE OF THE MODELS BUILT TO DO THIS. I THINK IT'S VERY PROMISING AND I THINK SOME OF THE WORK HAS BEGUN TO USE VIO IN THE FRAMEWORK. HERE'S ANOTHER EXAMPLE OF DIABETES IN COMPLEX MODELS WHERE IT CAN BE BUILT IN. THEN WE TRIED TO DEVELOP AN ALGORITHM FOR THINKING ABOUT HOW pTO USE VOI IN THE CONCEPT. YOU HAVE A POTENTIAL TOPIC FOR RESEARCH AND CONCEPTUAL VOI IS LOW. DON'T DO FURTHER VOI. IT'S NOT A GOOD IDEA AND THEY SUGGEST THE TOPIC COULD BE IMPORTANT. YOU ASK WHETHER IT CLUSTERS IN DOMAINS WITH OTHER TOP BEING AND IF COMPREHENSIVE OUTCOMES ARE MINIMAL YOU MAY BE ABLE TO DO MODELLING IF IT DOESN'T HAVE ANOTHER OUTCOME IS AVAILABLE YOU HAVE TO LOOK AT OTHERS. SO LAST MINUTE OR TWO LET ME JUST SUMMARIZE AND TALK ABOUT SOME OF THE FUTURE DIRECTIONS. I THINK VOI METHODS MAY HELP IN PRIORITIES AND I THINK THE ALGORITHM MAY BE USEFUL IN PRACTICAL APPROACHES USING VOI IN VARIOUS CONCEPTS. CONCEPTUAL VOI ALIGNS WHAT WE DO IN PEER REVIEW AND DECISION MAKING. AND THEY'VE TAKEN COMPONENT AND BUILT IT INTO A REVIEW PROCESS. HOW THEY USED IT I WON'T VOUCH FOR. IN NONE OF THESE CASES SHOULD THIS REPLACE JUDGMENT BUT THE APPROACH APPROACHES COMPLIMENT IT. IF YOU WANT TO THINK OF THE SIMPLEST WAY IS PRACTICAL AND QUANTITATIVE CALCULATIONS FOR VERY EXPENSIVE STUDIES AND THAT'S PARTIALLY HOW THEY'VE BEGUN TO THINK ABOUT THIS. IF THE STUDY'S BIG YOU MAY WANT TO DO THIS AND IT'S IMPORTANT TO REMEMBER THE EXPECTED VALUE OF PERFECT FFGS WHICH IS THE MOST COMMON WAY OF CALCULATING THESE IS UPPER BOUND. IT DOESN'T MEAN IT'S AN EQUALLY INFORMATIVE UPPER BOUND AND THAT'S OFTEN NEGLECTED. ULTIMATELY WE NEED TO LOOK AT THE USE OF VOI BECAUSE IT'S USEFUL BY RESEARCHERS AND OTHER DECISION MAKERS. DOES IT ACTUALLY CHANGE DECISIONS VERSUS THE STANDARD APPROACH AND WHAT'S THE HEALTH IMPACT AND MONETARY VALUE OF THE DECISIONS BY VOI. AND IT'S GREATEST WHERE THE STUDY IS EXPENSIVE AND DECISIONS MADE ARE CONSEQUENTIAL AND NIH PLAYS A HUGE ROLE IN GUIDING STUDIES. THE QUESTION IS CAN WE DO IT FAST ENOUGH BECAUSE WE DON'T WANT TO DELAY A STUDY FOR MULTIPLE YEARS STUDYING WHETHER IT SHOULD HAPPEN. WE CAN GO FAR IN THIS AND WITHIN THE TIME FRAME OF OTHER DECISIONS BEING MADE. LET ME STOP THERE. [APPLAUSE] >> DIRECTOR AT UNIVERSITY OF WASHINGTON. >> THANK YOU. SO WHENEVER I'VE BEEN ASKED TO SPEAK ABOUT REAL WORLD DATA AND WHEN I CAME ACROSS THE TERM I ALWAYS WONDERED ABOUT THE BUZZ WORD LIKE THE MANY WE HEAR IN THIS SPECTRUM OF GENERATION OF NEW DATA. OBVIOUSLY THE CURRENT FASHION IS LOOKING AT OTHER BUZZ WORDS LIKE BIG DATA AND THERE'S REAL WORLD EVIDENCE AND MY FAVORITE IS DATA SCIENCE. AFTER A HUNDRED YEARS IN THE DEPARTMENT OF STATISTICS THEY'RE THINKING OF CHANGING IT TO THE DEPARTMENT OF DATA SCIENCES. HERE HAVE YOU A HANDBOOK WITH A FANCY DIAGRAM ON THE COVER AND ONE THING I WANTED TO DO IS SEE HOW PEOPLE DEFINE REAL WORLD DATA. WE HAVE BEEN DOING SOME WORK ON THAT. AND THERE'S A LOT OF DEFINITIONS OF REAL WORLD OUT THERE. AND THE DEFINITION OF REAL WORLD EVIDENCE TOO. AND WE SAW LIKE WHAT CAME OUT ON THE TOP. AND THE GENERAL THEME THAT CAME OUT WAS THIS OVERARCHING TERM OF WHAT REAL WORLD DATA IS. PROBABLY APPLIED TO MOSTLY HUMAN DATA. WE GOT SOME PUSHBACK ON SECTORS THAT IT DOESN'T NEED TO BE HUMAN BUT THERE'S DATA OBTAINED IN THE NATURALISTIC SETTING SO THAT'S THE BROADEST DEFINITION OF REAL WORLD DATA WE FOUND. OBVIOUSLY THE REAL WORLD EMPHASIS OF DATA AND THE PURPOSE IS INFORMED DECISION MAKING TO KEEP IT EXPLICIT BUT IT'S USING REAL WORLD DATA AND THERE'S A GENERATION OF MORE COMPLETE UNDERSTANDING OF EFFECTIVENESS AND SAFETY AND VALUE FROM USING IT FROM HIGH QUALITY DIVERSE SOURCES. MOST THE TOPICS WE SAW IN THE MORNING ARE ACTUALLY USING REAL WORLD DATA TO GENERATE REAL WORLD EVIDENCE. SO SINCE I'VE BEEN ASKED TO TALK ABOUT THE PROMISE OF REAL WORLD DATA AND EVIDENCE. THE TWO THINGS THAT IMMEDIATELY COME UP ARE THE DISTINCTION BETWEEN EFFICACY AND EFFECTIVENESS AND ALSO CONNECTED TO THAT DISCUSSIONS ABOUT TREATMENT OF HETEROGENEITY. THERE HAS BEEN A LOT OF PAPERS AND DISCUSSION AD NAUSEAM THAT TALK ABOUT HOW WE CAN GET EFFECTIVENESS FROM REAL WORLD DATA AND THE ELECTION BUY -- BIASES AND I WON'T TALK ABOUT THOSE THINGS AND I'M SURE YOU'RE BORED IN THOSE DISCUSSION. I'LL TALK PLACES WHERE REAL WORLD DATA CAN BE USEFUL. ONE IS PROJECTIONS OF REAL WORLD TRIALS AND DATA AND SOME OF THE PREDICTIVE ANALYTICS WITH BIOMARKER PREDICTION. SO AS I SAID TRADITIONAL METHODS FOR REAL WORLD EVIDENCE AND COMPARATIVE EVIDENCE SUGGEST PRAGMATIC DESIGNS AND OBSERVATION METHODS ARE WELL ESTABLISHED. THEY, HOWEVER, REQUIRE TIME AND SUBSTANTIAL RESOURCE INVESTMENTS AND THE QUESTION HAVE THESE NECESSARY FOR EVERY NEW TECHNOLOGY AND COMPARATIVE QUESTION. THE RATIONALE IS THERE'S METHOD APROECH APPROACHES BASED ON REAL WORLD DATA AND IF THEY'RE VALID THEY CAN PROVIDE VALUABLE INFORMATION WITHOUT COMMITTING TO THE BROAD RESOURCES OF DOING BIGGER STUDIES IN REAL WORLD DATA. THE ONE MOST COMMONLY USED METHOD IS THE PROPENSITY SCORE METHOD THAT TAKES A TRIAL AND THEN ALL ESTIMATES OF PROPENSITY COMPARED TO REAL WORLD DATA AND PROJECT IT TO THE REAL WORLD DATA AND POPULATION. AND LIKE ANY MATCHING METHODS AND OBVIOUS WILLY THESE APPLICATIONS DON'T DEAL WITH OBSERVE FACTORS DEALING WITH SELECTION AND ONE ADVANTAGE THEY HAVE IS THEY HAVE ALL THESE VALIDATION AND FALSIFICATION AND WE WANTED WE WANTED TO PROJECT TO THE MEDICAID MANAGED CARE POPULATION. AND SO WE HAVE A DECISION MODEL THAT'S OFTEN DONE IN THESE CASE. WE HAVE THE LONG-ACTING ARM AND THE OTHER LONG-ACTING ARM. I'M GOING TO FOCUS MOSTLY ON THE ORAL ARM. BECAUSE THE ORAL PSYCHOTIC DRUGS ARM IS WHAT IS ACTUALLY MOSTLY USED AND WE HAVE OUTCOMES OF PEOPLE USING ORAL ANTIPSYCHOTICS. SO THERE'S DIFFERENT TRIALS THAT INFORM DIFFERENT PARTS OF THESE DECISION MODELS. THE MAIN TRIAL WAS A RANDOMIZED STUDY OF ORAL ANTIPSYCHOTICS WITH LONG-ACTING ANTIPSYCHOTIC DRUGS AND THE ORAL ANTIPSYCHOTIC DRUG AND CLINICIANS HAD THE FREEDOM TO CHOOSE THE ONE THEY WANTED TO USE AND IT'S THE SAME AS WE SEE IN THE MEDICAID POPULATION BUT THE PROPORTIONS ARE NOT THE SAME AS WE SEE IN THE TRIAL. THE REAL WORLD DATA WE IMAGINE OF THE CLAIMS DATA AND HAVE THE INCLUSION CRITERIA AND STUDIED THE MEDICAID POPULATION AND FOCUSSED ON PEOPLE WITH SCHIZOPHRENIA AND THIS WAS OUR TARGET POPULATION WE WANTED PROJECTED. IN THE ORAL ARM AND LONG-ACTING ARM THAT WAS RANDOMIZED WERE SIMILAR BUT LET'S LOOK AT THE MEDICAID DATA. THE MEAN AGE WAS SIMILAR BUT THEY HAD LOWER PROPORTIONAL FEMALES. A LOWER PROPORTION OF WHITES AND HIGHER PROPORTION OF HISPANICS THAN IN THE MEDICAID POPULATION AND IT WAS DIFFERENT AND THIS TRIAL HAD PEOPLE ON THE DRUG WHICH MAKES SENSE BECAUSE THEY WANT TO GET PEOPLE SO ONE THING THAT COMES OUT OF THIS IS I THINK IT'S A GOOD IDEA WHEN NIH IS THINKING OF DOING TRIALS SO HAVE DEMAND WE SEE HOW THE STUDY MATCHES WITH THE OVERALL POPULATION OR WHAT THEY WOULD BE. FOR EACH TRIAL WE HAD A BUNCH OF BASELINES WE MEASURED. AND WE PROJECT IT INTO THE POPULATION. LET'S QUICKLY GO OVER SOME OF THE RESULTS. IN HERE WE SAW THAT FOR THE ORAL ARM THE PRIDE TRIAL ON AVERAGE OR 18 MONTHS EACH PATIENT HAD .42 NUMBER OF HOSPITALIZATIONS AND THE MEDICAL DATA WAS 5.0 BUT WHEN WE REWEIGHTED IT IT PRODUCED THE OUTCOME OF .50 WHICH IS GREAT. WE'RE HAPPY IT WAS ABLE IT DO BUT IF YOU LOOK AT THE ALL-CAUSE HOSPITALIZATION IT WAS .55 AND THE MEDICAID POPULATION WAS A 3 AND WHEN WE REWEIGHTED IT INCREASED. WE DIDN'T HAVE COMORBIDITIES WE COULD WAY THE PROPENSITY SCORE ON. OBVIOUSLY THESE PATIENTS MAY HAVE OTHER ILLNESSES WERE WEREN'T ABLE TO MATCH AND PROJECT OUT. THAT'S ALSO ANOTHER SUGGESTION IS THAT MAYBE WITH NIH TRIALS WE NEED TO COLLECT MORE BIOMARKERS AT BASELINE AND RANDOMIZE ACROSS THE ARMS AND THE FACT WE NEED TO UNDERSTAND HOW THE TARGET POPULATION REALLY COMPARES. I SAID FOR THE IMPLICATIONS REAL WORLD DATA CAN SHOW GENERALIZABILITY AND PROJECTION METHODS AND WE NEED TO COLLECT ENOUGH BASELINE INFORMATION AND THE CONTROL ARM SHOULD COMPARE THE OUTCOMES WITH THE PROJECTION. NOW, THE PROJECTION DOES HAVE IMPLICATIONS FOR VALUE OF INFORMATION ANALYSIS. WITH PROJECTION UNCERTAINTY INCREASES. AS DAVID SHOWED ONE REASON NEW INFORMATION IS UNVALUED IS BECAUSE WARE UNCERTAIN AND NEW INFORMATION CAN OVERCOME THE UNCERTAINTY. SO THOUGH UNCERTAINTY CHANGES THE EFFECT ON VOI THE PROJECTS REMAIN AMBIGUOUS SO THIS IS A COST EFFECTIVENESS PLAY AND YOU'LL SEE THE NEW DRUG WAS NOT COST EFFECTIVE AND SO YOU WOULD ASSUME THE NEW DRUG SHOULD NOT BE USED BUT IT'S THE ERROR RATE THAT FALLS BELOW THE EFFECTIVENESS PLANE AND WITH THE DATA THE UNCERTAINTY INCREASES BUT IT SHIFT TO THE OTHER SIDE. THE NEW DRUG IS NOW COST EFFECTIVE BUT THE ERROR RATE COULD DECREASE. SO WITH PROJECTION VOI MAY DECREASE. IT'S INTERESTING TO SEE HOW THE VALUE OF INFORMATION OF A NEW STUDY DEPENDS IF YOU RELY ON THE PROJECTING AND REAL WORLD DATA. THE ASSUMPTION BEHIND THE VOI CALCULATION IS IT HAS POTENTIAL TO CHANGE BEHAVIOR. IF NOT INVESTMENT AND RESEARCH ACCORDING TO THE VOI LOGIC IS NOT VALUABLE. THIS MEANS THOSE AFFECTED BY DIFFUSION OF TECHNOLOGY WILL CONNECT TO BOTH WHAT WE SAW IN THE MORNING AND SOME OF THE RESOURCES WE'LL SEE THIS AFTERNOON. FOR EXAMPLE, WE HAVE BEEN WALKING ON UNDERSTANDING -- WORKING ON UNDERSTANDING THE EXPECT VALUE OF INDIVIDUALIZED CARE NETWORK AND APPLYING IT TO PRECISION MEDICINE AND YOU'LL SEE ABOUT UNDERSTANDING PREFERENCES FOR DIFFERENT STAKEHOLDERS AND ADOPTING A TECHNOLOGY IN THE FACE OF INFORMATION. AND IF THAT INFORMATION CHANGES HOW THAT ADOPTION CHANGES. FINALLY PREDICTIVE ANALYTICS HAS A LOT OF BUZZ AND IS SOMETIMES PROMISING AND SOMETIMES AN OVERSELL. I THINK IT'S HARD TO SAY IN THE pWORLD OF EFFECTIVENESS, CAUSAL INFERENCE IS STILL THE KING. YOU CAN THROW IN HUNDREDS AND THOUSAND OF CO-VARE -- VARIANTS AND YOU CANNOT RULE OUT CAUSAL IN FLENS. ONE INTERESTING IS BIOMONITORING. AND SOME OF THE WORK GOING ON RIGHT NOW IT'S MAKING THESE INFORMING COMPLETE AND THE TWO CHALLENGES TO THESE DATA ONE IS IT'S CONNECTED FOR A LOT OF PATIENTS WHICH IS BIG DATA BUT LESS NUMBER. AND OTHER DATA SHOWS CO-VARIANTS. THERE'S AMBIGUITY IN THE PREDICTIONS. MORE NEWER DATA SETS ARE BRIDGING THE GAPS AND HOPE FRY THEY'LL BE BETTER IN PREDICTING STUFF. THERE'S A SECTION ON CHALLENGES THAT GOES THROUGH DATA CONSISTENCY AND VALIDATION. THERE'S A HUGE CHALLENGE WITH PATIENT PRIVACY AND MORE PEOPLE WITH MORE INFORMATION HOW DO YOU MONITOR PRIVACY AND ALSO IMPROVING THE CLINICAL ACCURACY AND SETTINGS OF THE DATA. IN THE LAST FEW MINUTES WHAT I WANT TO HIGHLIGHT THAT AS A RESEARCHER THE BIGGEST CHALLENGE FOR ME TO USE REAL WORLD DATA IS THE ACCESS AND COST OF REAL WORLD DATA. DATA CURATION IS COSTLY BUT THERE'S NOT ENOUGH COMPETITION IN THE DATA MARKET THAT IS ABLE TO BRING DOWN THE PRICE OF THESE DATA ASSETS. OFTEN WE FIND THE DATA VENDORS THAT ARE OLIGOPOLISTIC AND THERE'S ANTHEM AND THERE'S A LUCRATIVE BUSINESS OF DATA SCIENCE SO EVEN IF YOU PAY THE FULL AMOUNT FOR THE FULL DATA OFTEN THEY DON'T GIVE YOU THE FULL DATA. THEY WANT TO KEEP SOME THINGS FOR THEMSELVES SO THEY CAN PRODUCE EVIDENCE AND SELL THAT. I THINK PART OF THE REASON IS THAT THERE IS NO REAL WAY FOR PATIENTS WHO I THINK OWN THE DATA TO CONTRIBUTE TO SOME NATURAL DATA DEPOSITORY AND IF YOU LOOK AT A HEALTH CARE SYSTEM LIKE NETHERLANDS AND FINLAND IT'S STORED IN ONE PLACE AND PEOPLE CAN ACCESS IT AND DO RESEARCH MUCH MORE COMPREHENSIVELY. AND PCORNET IS TRYING TO SOLVE THE PROBLEM BUT I THINK THEY'RE A LONG WAY OUT. WEHO WEHOOD WE HAD A CONVERSATION AND THEY COULDN'T GIVE ME A PROPER ESTIMATE OF WHAT DATA WOULD CAUSE BUT HAVING ALL THE DIFFERENT AGENCIES OR ORGANIZATIONS ARE WILLING TO SHARE DATA BUT THEY'RE NOT WILLING TO SHARE DATA WITH EACH OTHER. IT'S NOT LIKE THERE'S ONE REPOSITORY TO RUN A COMPLICATED ANALYSIS. THEY'LL GIVE YOU WHATEVER OUTCOMES THEY WANT AND THEY'LL GIVE YOU A MEASURE TO YOU AFTER YOU RUN IT. WHICH IS BETTER THAN NOTHING BUT THERE'S NOT MUCH YOU CAN DO WITH IT WITH PREDICTIVE MODELLING AND OTHER COMPLICATED THINGS YOU WANT TO DO. LAST I WANT TO LEAVE NIH WITH THIS QUESTION AS A DATA RESEARCHIST. WHY CAN'T GRANTS FUNDED THROUGH NIH ACCESS THE DATA FOR FREE? IT DOESN'T MAKE SENSE TO ME. IF YOU ARE SPENDING HALF YOUR RO1 BUDGET ANALYST IN THE FIRST FIRST PART OF THE YEAR AND IF YOU SAY IT'S A GOOD GENUINE GRANT YOU SHOULD GET ACCESS TO THE DATA. HOPEFULLY AS YOU DISCUSS GOING FORWARD NIH WOULD BE ABLE TO SOLVE SOME OF THE ACCESS TO THE REAL WORLD DATA PROBLEMS WE HAVE. THANK YOU. I'M GOING START WAY BROADER OVERVIEW BEFORE ZOOMING INTO THIS PARTICULAR PROJECT OF HOW INFORMATION PLAYS A BIG ROLE IN HOW PATIENT CARE IS DELIVERED. SO I'M GOING TO KIND OF SWITCH GEARS FROM THE PREVIOUS TWO TALKS AND BE EVEN MORE REAL WORLD AND MORE MICRO. INFORMATION PLAYS A CRUCIAL ROLE IN HOW CLINICIANS DIAGNOSE AND TREAT PATIENT. TO IMPROVE PATIENCE HEALTH WHICH IS THE MISSION OF THE HEALTH CARE MISSION AND NIH'S MISSION WE HAVE TO CHANGE HOW CLINICIANS ACHIEVE INFORMATION AND ECONOMISTS CAN SAY A LOT OF ABOUT THIS. MOST CLINICAL DECISION THE CORRECT ANSWER IS NOT KNOWN FROM EXISTING CLINICAL TRIALS. IF YOU'RE LOOKING AT TRAN FUSING PATIENT BLOOD OR GIVE MEDICINE TO A PATIENT WITH CONDITIONS THERE'S A WEALTH OF TRIAL DATA THAT MAY BE APPLICABLE BUT FOR A PARTICULAR PATIENT IN FRONT OF YOU HAVE TO USE JUMP BECAUSE THERE'S STILL UNCERTAINTY ABOUT HET HET HETROGENUS EFFECT AND HEALTH CARE IS DELIVERS BY ORGANIZATIONS. THERE'S SHIFT WORK AND SPECIALIZIZATION OF KNOWLEDGE. THEREFORE CLINICIANS MUST WORK IN TEAMS AND COMMUNICATE INFORMATION THEY HAVE WITH EACH OTHER AND STEPPING BACK FROM THE EVALUATION OF STUDIES, CLINICIAN HAVE TO MAKE DECISIONS IN MINUTES TO HOURS ON A DAY-TO-DAY BASIS. A FIRST ORDER FACT IS BOUNDED RATIONALITY. THEY HAVE LIMITED ATTENTION AND MAY BE PRONE TO DECISION BIASES AND CAN BE STUDIED THROUGH THE LENS OF ECONOMICS AND THERE'S A BROADER MEGASTATEMENT OF HOW IT MAY ALIGN WITH NIH'S ROLE BEFORE I CAN ZOOM IN IN THE TALK ABOUT MY PROJECT FUNDED BY THE NIH. NOW I'M GOING TO DISAPPOINTINGLY ZOOM INTO THIS PARTICULAR PROJECT WITH HEALTH I.T. AND HOW IT DELIVERS INFORMATION AND HOW CLINICIANS PROCESS THE INFORMATION. THOUGH IT'S A PARTICULAR PROJECT HOPEFULLY IT WILL GIVE YOU A BROADER SENSE OFHOW SOMEOFTHE THEMES APPLY TO RESEARCH THAT NIH MAY FUND. HEATH IT, ANY CLINICIAN CAN TELL YOU IT'S REVOLUTIONIZED AND IMPACTING PATIENT CARE AND CHANGED THE WORK FLOW OF WHAT CLINICIANS ARE DOING ON A DAY-TO-DAY BASIS. THERE'S WIDESPREAD AGREEMENT ACROSS THE AISLES AND CONGRESS THAT HEALTH I.T. SHOULD AN IMPORTANT PART OF GETTING DOCTORS TO PROVIDE QUALITY POPULATION-BASED MEDICINE AND HIS IS RESEARCH WE'VE KNOWN MORE THAN A DECADE MEDICAL ERRORS ARE PREVALENT. HUNDREDS OF PATIENTS DIE ON A DAILY BASIS BECAUSE CARE THAT SHOULD HAVE BEEN PROVIDED OR SHOULDN'T HAVE BEEN PROVIDED IS BEING PROVIDED OR VICE VERSA. WE NEED A SYSTEM THAT PROVIDES INFORMATION TO CLINICIANS SO THEY CAN PROVIDE THE BEST CARE. THERE'S CURRENTLY $35 BILLION INVESTMENT IN HEALTH I.T. AND IT'S PROMPTED BY RECENT LEGISLATION LIKE THE AFFORDABLE CARE ACT. WE KNOW HOW IT WILL OPTIMIZE HEALTH. THERE'S PERMUTATION AND FUNDAMENTALLY A DIFFICULTY IS THERE ARE MANY WAYS TO IMPLEMENT HEALTH I.T. AND WE TREATED IT AS A BLACK BOX. WE HAVE TO UNDERSTAND HOW HEALTH I.T. INTERACTS WITH HUMANS AND ECONOMIST HAVE A LOT TO SAY. IT MAY PROVIDE USEFUL INFORMATION CLINICIANS MAY OTHERWISE OVERLOOK BUT THERE'S BEEN ANECDOTAL STATEMENTS ABOUT INFORMATION OVERLOAD, ALERT FATIGUE. EVEN THE NON CLINICIANS CAN THINK ABOUT THE E-MAIL INBOX AND THE NUMBER OF E-MAILS ON A DAILY BASIS. THE LIKELIHOOD YOU'LL MISS AN IMPORTANT MESSAGE BECAUSE YOU'RE OVERWHELMED BY OTHER PIECES OF INFORMATION THE HEALTH I.T. SYSTEM MAY BE THROWING AT YOU. THIS.ED TO TAKE A CLOSER LOOK AT WE KNOW HUMANS CAN ONLY MEMORIZE SEVEN DID IN -- DIGITS AT THE MOST AND WITH NATURAL EXPERIMENTS WITHOUT DELVING INTO THE MICRODATA HOW DO YOU DESCRIBE OR COMPARE SYSTEMS OF HEALTH I.T. HOW DO YOU COMPARE GOOGLE AND YAHOO. THERE'S DIFFERENCES. IS THERE'S AN ESTIMATE OF THE EFFECT OF A SINGLE HEALTH I.T. INTERVENTION AND THERE'S CASE STUDIES OR THE HEALTH EFFECT OF ANY HEALTH I.T. INTERVENTION AND THE LITERATURE IS MIXED. WE FOUND SOME IMPLEMENTATIONS HAVE IMPROVED PATIENT CARE AND OTHERS REDUCED THE QUALITY OF PATIENT CARE AND SOME HAVE NO EFFECT AT ALL. WITH THAT, WHAT THE PROJECT IS ABOUT IS DELVING INTO THE MICRODATA OF A SYSTEM OF HEALTH I.T. AND USEFUL VARIATION. THIS STUDIES THE V.A. AS AN IMPORTANT PLATFORM. IT'S VISTA. IT'S ONE OF THE MOST WIDELY USED AND ACCLAIMED HEALTH SYSTEMS AND 60% OF PHYSICIAN HAVE TRAINED AND HAVE EXPERIENCE WITH VISTA. IT ZOOMS INTO A PARTICULAR COMPONENT AS WELL AS OTHER HEALTH I.T. SYSTEMS AND THESE ARE ELECTRONIC REMINDERS AND WHAT IT GIVES THE CLINICIAN ABOUT THINGS THE CLINICIAN SHOULD BE REMEMBERING. MAYBE YOU SHOULD THINK ABOUT A COLONCOSCOPY AND IF A PATIENT AIS DIABETIC MAYBE YOU SHOVED THINK ABOUT THE VARIOUS DRUGS. THE NEAT THING ON THE HEALTH I.T. SYSTEM SAY COMMON PLATFORM. THERE'S NOTHING -- IF YOU LOOK AT A VISTA SYSTEM IN CHICAGO VERSUS L.A. AND PALO ALTO THEY LOOK THE SAME AS IF YOU WERE TO GO TO THE CAFETERIA OF ANY V.A. THEY ALL HOOK -- LOOK THE SAME BUT THERE'S ELECTRONIC REMINDERS AT A GIVEN V.A. SYSTEM OVER TIME AND ACROSS THE SYSTEM VARIES A LOT. THERE'S RELATED INFORMATION ACROSS V.A. SYSTEMS. IF YOU LOOK AT 2000 TO CURRENTLY JUST TAKING THREE WEST COAST V.A. HEALTH CARE SYSTEMS, L.A. HAS INCREASED AT A SLOW PACE. PALO ALTO HAS BEEN A LITTLE BIT MORE AND SEATTLE HAS INTERNATIONAL ENGEINCREASED TWO OR THE TIMES OF PALO ALTO. THERE'S VARIATION ACROSS SYSTEMS AND TIMES. THIS IS HOW THE REMINDER LOOKS LIKE FOR A CLINICIAN. THE CENTRAL PORTION HERE IS SIMILAR ACROSS ALL V.A.s. BUT THE REMINDERS SUCH AS FOR THIS EXAMPLE WITH ALCOHOL USE OR DEPRESSION SCREENINGS THEY MAY DIFFER ACROSS STATIONS AND TIME. AND THERE'S PRACTICAL REASONS WHY WE FOCUS ON THIS. MANY OF YOU HERE KNOW DIABETES IS A COMMON CONDITION WITH A HYBRIDING OF DISEASE. THERE'S GUIDELINES FOR WHAT TO DO FOR DIABETIC PATIENT US BUT GETTING PHYSICIANS TO DO THEM IS A NON TRIVIAL THING AND HOW DO YOU TRACK WHETHER A PATIENT IS DOING WELL WITH DIABETES AND THERE'S A NICE LAB TEST WE CAN SEE IN THE REAL WORLD DATA OF THE V.A. SYSTEM AS A MEASURE OF THIS. IT'S UNIQUE AND IT CORRELATES TO AMTATION OR ANY DIABETES AND IT'S MEANINGFUL. AND IT'S BASED ON THE RED BLOOD CELLS. IT'S HOW MUCH RED GLUCOSE IS BINDED TO THE RED BLOOD CELL. THE IDEA IS STRAIGHTFORWARD WHETHER TO ASK IF THERE'S SPILLOVERS FROM ELECTRONIC REMINDERS OWN THE PERFORMANCE ON HOW PATIENTS ARE DOING WITH DIABETES. SO IT'S FOCUSSING ON A VERY SMALL BUT IMPORTANT PROBLEM OF THE INFORMATION OVERLOAD AND HOW THERE'S NEGATIVE SPILLOVERS ON HOW WELL CLINICIANS ARE TREATING PATIENTS AND PATIENT HEALTH OUTCOMES. HOW DO WE MEASURE ME BURDEN OF ELECTRONIC REMINDERS AND THE REAL WORLD DATA ARE THESE ARTIFACTS OF REMEMBERERS CALLED HEALTH FACTORS. WHEN IT CLICKS ON IT IT GENERATES THE HEALTH FACTOR AND YOU KNOW WHEN THEY CLICKED ON IT AND WHO THE PATIENT WAS AND WHAT THE REMINDER WAS FOR. COMPLICATION IN THE PROJECT IS THE ECONOMISTS WOULD NOTE WE CAN TALK ABOUT WAYS TO GET AROUND THE PROBLEMS AND THEY'RE POTENTIALLY IN -- >> AND THIS IS ABOUT 5.5 MILLION PRIMARY CARE OFFICE VISITS OVER A SIX-MONTH WINDOWS BECAUSE I'LL FOLLOW IT UP TO SIX MONTHS AFTER A PRIMARY CARE VISIT AND LINK IT TO A HEMOGLOBIN A1C AND DID A PATIENT INVOLVE MANY OR FEWERS. SO THE EXPERIMENT IS WE'D LIKE ELECTRONIC REMINDERS TO BE RANDOMLY ASSIGNED AND SEE HOW THE EFFECT OF NON ELECTRONIC REMINDERS AFFECTS THE PERFORMANCE OF DIABETES CARE AS* * BY HEME GLOBIN >> AS MEASURED BY HEMOGLOBIN A1C AND THERE'S DEMOGRAPHICS AND MEDICAL CONDITIONS AS ELIXHAUSER INDICES. AND STEPPING BACK FROM THE VARIATIONS I'LL FOCUS ON MICROLEVEL VARIATION THAT'S QUITE LARGE. EVEN HOLDING FIXED THE STATION AND HOLDING FIXED THE MONTH, A CLINICIAN WILL SEE A WIDE RANGE OF ELECTRONIC REMINDERS OVER HIS OR HER PATIENTS. A LOT OF THIS HAS TO DO WITH THE SEVERITY OR ILLNESSES A PATIENT MAY HAVE. A LOT OF IT IS EVEN HOLDING FIXED THE PATIENT AND THESE REMINDERS COME ON IN REGULAR TIMEINTERVALS. IF YOU CLEARED A REMINDER AND HAVE A VISIT FIVE AND A HALF MONTHS LATER YOU WOULDN'T GET A REMINDER BUT IF YOU SEE THE PATIENTS SEVEN MONTHS LATER YOU GET A REMINDER. THIS IS THE SPECIFICATION I'M RUNNING WHERE HEMOGLOBIN A1C WILL BE THE VARIABLE AND I'LL LOOK AT THE SIX MONTHS BEFORE THE LEVEL AND I'LL BE LOOKING AT ANY VISITS THE PATIENT HAD IN THE SIX MONTHS AND I'LL BE FOCUSSING ON THE BURDEN OF ELECTRONIC REMINDERS AS MEASURED BY THE ARTIFACTS CALLED HEALTH FACTORS. ONE SPECIFICATION IS TO FOCUS ON QUARTILES OF HEALTH FACTORS WHICH IS THE FIRST SPECIFICATION OR YOU CAN LOOK AT THIS EARLY. I'LL CONTROL FOR STATIONS AND I ALSO HAVE PATIENT FIXED EFFECT. YOU CAN SEE HERE THAT ALL OF -- AS YOU INCREASE THE NUMBER OF REMINDERS BY QUARTILE, THE HEME HEMOGLOBIN A 1C IS LARGER AND THAT'S BAD FOR DIABETES. IT MEANS THE SUGAR WAS NOT GOOD AND MONOCHRONIC WITH QUARTILES AND IT PEAKS. WHEN HAVE YOU AN IMPACT RELATIVELY SOON AFTER A VISIT MOST THE RED BLOOD CELLS YOU HAVE BEEN AROUND A LONG TIME. THEY'VE BEEN AROUND OBVIOUSLY BEFORE THE VISIT. IT SHOULD PEAK AT THREE MONTHS WHEN YOU HAVE THE HIGHEST CONCENTRATION OF RED BLOOD CELLS FROM THE VISIT BUT BECAUSE THEY ONLY LAST SIX MONTHS THEY WASH AWAY AND YOU SEE A NONMONOCHRONIC EFFECT. A NICE FEATURE AND WE CAN FIND MANY WAYS OF TESTING FOR CAUSALITY. SO WHAT WE DO IN THIS SETTING IS TO KIND OF DO A FALSIFYING TEST AND WHETHER IT'S LINKED TO THE A1C LEVEL AND IF YOU HAVE MORE HEALTH FACTORS AT A VISIT AFTER THE HEMOGLOBIN A1C IS DRAWN DOES IT AFFECT IT AND IT DOES NOT SHOW RELATIONSHIP BETWEEN THE TWO. AND IN THE SMALL SETTING OF DIABETES -- MANY HERE WOULDN'T SAY DIABETES IS STRONG ESPECIALLY IF YOUR CENTER IS FOCUSSED ON THIS BUT IT'S ONE OF THE DISEASES WE CAN FOCUS ON. IT'S AN IMPORTANT DISEASE AND WE FIND THAT HEMOGLOBIN A1C INCREASES IF YOU MOVE FROM NO HEALTH FACTORS TO THE FOURTH QUARTILE OF HEALTH FACTORS. IF YOU GO FROM A REMINDER BURDEN OF NO REMINDER BURDEN OR THE FIRST QUARTILE TO THE FORTH QUARTILE YOU HAVE THIS EFFECT THAT'S COMPARABLE TO TELLING PATIENTS TO KIND OF GO ON A PRETTY EXTREME DIET AND EXERCISE OR EVEN ADDING DRUG THERAPY. THE AVERAGE EFFECT OF DRUG THERAPY IN RANDOMIZED TRIALS IS AROUND .5 TO 1 POINTS IN HEMOGLOBIN A1C. AND IT COMES WITHOUT TELLING PATIENTS TO START ANY NEW DRUGS OR NEW DRAMATIC LIFESTYLE INTERVENTION. SO WE CONSIDER THIS AS PROOF OF CONCEPT TO THIS IDEA OF FFRGS OVERLOAD THAT MANY PEOPLE ANECDOTALLY SHOWN BUT THINK WE CAN GO FURTHER AT LOOKING AT THE DATA AND APPLYING TOOLS OF ECONOMICS TO QUANTIFY THIS. MOVING FORWARD THERE'S OTHER THINGS WE CAN DO IN ADDITION TO NEGATIVE EFFECTS. WE CAN LOOK AT THE POTENTIAL POSITIVE EFFECTS AND PHYSICIANS ARE FORGETTING TO PRESCRIBE THE DRUGS AND IMPORTANT IN CONTROLLING DIABETIC CONTROL ARE THEY NOT TALKING TO PATIENTS MORE ON WHAT THEY'RE DOING OR NOT DOING. IMPORTANTLY, MOST CLINICIANS WOULD SAY SOME PARTS ARE USEFUL AND SOME ARE ANNOYING AND SOME MAY BE DANGEROUS. EVEN WITHIN THE SPAN OF ELECTRONIC REMINDERS, SOME MAY BE MORE DIFFICULT TO DEAL WITH THAN OTHERS. SOME MAY BE MORE HELPFUL THAN OTHERS. WHAT TYPES OF PHYSICIANS DO A BETTER JOB OF DEALING WITH LEB TRONIC REMINDERS. -- REC ELECTRONIC REMINDERS AND WHAT SYSTEMS WITH NURSE AND SUPPORT DO A BETTER JOB USING HEALTH I.T. AND FINALLY IN MANY LIVING HEALTH CARE SYSTEMS THERE'S A LOT OF INTEREST IN SHOWING AND CHANGING THE WAY THEY DELIVER CARE. AND THIS IS TRUE AT THE V.A. THE V.A. IS CURRENTLY IN THE PROCESS OF REDESIGNING ITS HEALTH I.T. SYSTEM AFTER MER THAN 20 YEARS. THIS AT THE VERY LEAST PROVIDES AN OPPORTUNITY FOR ECONOMISTS TO WORK WITH HEALTH CARE SYSTEMS IN NATURAL EXPERIMENT ARE HAPPENING WITHOUT HAVING TO FUND THE AND A HALF EXPERIMENT. THIS IS AN OPPORTUNITY FOR RESEARCHERS TO AT LEAST COLLECT DATA ON WHAT RESEARCHERS ARE SEEING AND CHANGES THE HEALTH CARE SYSTEM IS IMPLEMENTING OR POTENTIALLY BETATESTING THINGS GROUNDED IN ECONOMIC MODEL IN THIS. SO TO SUM UP I WANTED TO TAKE A STEP BACK AGAIN AND SAY THIS IS BUT ONE PROJECT THAT I THINK IS HOPEFULLY LISTS A RANGE OF PROJECTS WE CAN DO IN HOW INFORMATION IS USED IN THE DELIVERY OF HEALTH CARE AND IDENTIFYING BEHAVIORAL MECHANISMS AND NATURAL EXPERIMENTS TO SHED LIGHT ON THE MECHANISMS AND POTENTIALLY EVEN DESIGN PROSPECTIVE TRIALS TO HELP UNDERSTAND THE IMPORTANT MECHANISMS. >> THE REAL WORLD IS THE OPPOSITE. IN SOME WAYS AS I UNDERSTAND VALUE INFORMATION IS BASED AROUND THE UNCERTAINTY AND AROUND YOUR OBSERVATION. IF NIH IS THINKING ABOUT INVESTMENTS IN THESE OR OBSERVATIONAL STUDIES AND WASN'T TO THROW OUT SOMETHING SPECIFIC WHICH IS THE PRECISION MEDICINE COHORT WITH HOW INFORMATION IS BEING SET UP NOW BUT EXACTLY WHAT WILL BE COLLECTED IS INFLUX AND BEING DETERMINED AS WE SPEAK. WITH THAT MAYBE THERE'S A SPECIFIC FOCUS ON WHAT INFORMATION NEEDS TO BE GATHERED. HOW DO WE CALCULATION THE UNCERTAINTY AROUND THE OBSERVATIONAL VERSUS THE RCT. >> I'M NOT SURE I 100% UNDERSTAND THE QUESTION. BUT I THINK THE THINGS YOU WANT TO FACTOR IN UNCERTAINTY IS HOW CLINICIANS AND OTHER DECISION MAKERS RESPOND TO EVIDENCE GENERATED FROM RCTs VERSUS OBSERVATIONAL DATA. >> SO I THINK HE WAS NEXT BUT LET ME MAKE A COMMENT RELATED TO YOUR QUESTION AND THAT IS MAYBE YOU COULD RESPONSE TO THE FOLLOWING WHICH I THINK IS ANSWERING UNCERTAINTY. IN THE RANDOMIZED TRIAL YOU'LL USUALLY LOOK AT THE AVERAGE EFFECT. HOW DO YOU WEIGH UP A GENERAL POPULATION? >> UNDER THE CONDITION YOU CAN WEIGH IT BY THE PROPORTIONS. THAT'S THE WAITING. DEPENDING ON HOW MANY BASELINE CHARACTERISTICS YOU HAVE, THAT MATCH COULD BE OBVIOUSLY BETTER, RIGHT? THE MATCHING OF TREATMENT IN THE REAL WORLD DATA WAS MATCHING TO THE POPULATION OUTCOMES IS OFTEN YOU MIGHT HAVE SOME OUTCOMES WHERE YOU CAN DO VALIDATION EXERCISES. THAT'S WHERE THE FIELD LIES AT THIS POINT. THIS IS NOT A SOLUTION TO ALL THOSE THINGS BUT AT LEAST ANALYST INTERIM THE MATCHING CAN BE DONE MORE EFFICIENTLY THAN DOING A BIG STUDY. AND I THINK INTEGRATING THE VOI METHODS WITH THE TECHNIQUES IS A GOOD WAY TO MAKE THE DECISION IN HOW TO INVEST. >> IF I CAN JUST -- SO BECAUSE IT'S IN THE OBSERVATIONAL SETTING WHERE WE KNOW THE ASSUMPTION WE CAN EXPECT THAT BECAUSE OF THE RICHER POPULATION. I THINK WORK IN THAT AREA IS IMPORTANT TO SOLVE. I SUBMIT WE DON'T WANT THE CONDITIONAL TREATMENT EFFECT AS A STAT STIISTICIAN I SAY WE WANT THE MARGIN OVER THE CO-VARIATES. YOU CAN NEVER SAY IT WON'T HAPPEN IN THE POPULATION. I'M CONCERNED OF THESE THEORIES WITHOUT THE REAL WORLD OF ACADEMICS AND CLINICAL PRACTICE. BECAUSE YOU'RE SAYING METHOD WISE MAKES MORE SENSE BUT IN YEARS THE POPULATION CHANGES AND THE DRUGS CHANGES AND THANK YOU DON'T HAVE THE ABILITY TO DRAW CONCLUSIONS ABOUT YOUR INDIVIDUAL PATIENT BECAUSE THEY WEREN'T POWERED TO DETECT THAT SO THE VALUE OF THE INFORMATION AT THE END OF THE DAY IS MAYBE YOU HAVE A QUICK TURNOVER OF THE END POINT BECAUSE PEOPLE WILL DIE WITHIN A YEAR BUT SAY FOR TREATMENTS OR EARLY STAGE CANCERS OR OTHER THINGS, PREVENTION, WE DON'T HAVE THAT. SO I THINK WE'RE NOT SOLVING REAL WORLD PROBLEMS. THE OTHER THING IS I REALLY TAKE ISSUE WITH CONSIDERING THE RETURN ON INVESTMENT AND THE CREATIVITY OF THE WORKFORCE. IT'S JUST A REFLECTION OF THE ACADEMIC REALITIES. PAIR A JUNIOR PERSON WITH A MID LEVEL PERSON BECAUSE THAT'S HOW THEY BOTH GET PROMOTED AND THAT'S JUST WHAT WE DO. THE PERSON WHO NOW HAS TENURE HAS THE FREEDOM TO TAKE RISK. PEOPLE DON'T HAVE TENURE CAN'T TAKE RISK. THOSE ARE DAY-TO-DAY PRACTICALITIES OF ACADEMIC CAREERS AND TO MAKE INFERENCE TO MAKE RETURN ON INVESTMENT AND CREATIVITY IS FLAWED WHEN YOU DON'T LOOK AT THE REAL-WORLD CONTINGENCIES AND I FEEL AS AN OLDER PERSON I'M MORE CREATIVE THAN I'VE EVER DONE AND GONE BACK TO DOING MOUSE MODELS AND I THINK IT'S COOL SO I TAKE THE ISSUE THAT YOU'RE MOST CREATIVE WHEN YOU'RE YOUNG. >> SO I THINK ONE OF THE THINGS TO REMEMBER ABOUT A VOI IS IF IT'S DONE CORRECTLY IT CAN QUANTIFY THE THINGS YOU'RE TALKING ABOUT OR AT LEAST FLATTEN THEM AT ISSUES. AND IT'S IMPORTANT FOR THE REASONS YOU DESCRIBED. SOMETIMES YOU'RE DOING A STUDY THAT'S GOING TO TAKE YOU TEN YEARS TO COMPLETE AND THERE'S ALREADY ANOTHER STUDY THAT'S ALMOST CERTAIN TO CHANGE. SAY IN YOUR FIELD THINGS CHANGE RAPIDLY AND YOU SHOULDN'T EXPECT THAT AND IT GIVES YOU A FRAMEWORK TO THINK ABOUT THAT AT LEAST AND OFTEN IT DOESN'T GET THOUGHT ABOUT OTHERWISE. ABOUT ALL OF US, ONE REASON I HESITATE IN ANSWERING BECAUSE WHEN I THINK OF ALL OF US IT HAS AN IMMENSE DISCUSSION AND HAS DISCOVERY IN IT. AND VALUE OF INFORMATION ANALYSIS IS ONLY AS GOOD AS YOUR ABILITY TO TIE THAT TO OUTCOMES AND THERE ARE MANY MORE STEPS FROM BASIC SCIENCE TO OUTCOMES FROM CLINICAL STUDIES. I THINK WE JUST NEED TO BE REALLY CAUTIOUS IN REMEMBERING WHERE THIS IS GOING TO BE REALLY USEFUL TO US. IT'S ALWAYS GOOD TO ASK THE QUESTION, CAN I BUILD A MODEL AND WHAT THE VALUE OF THE INFORMATION'S GOING TO BE BUT IN THIS PARTICULAR INSTANCE IT'S TOUGHER. LAST COMMENT. WE HAVE AN AWARD LOOKING AT PHARMACOGENICS FOR AFRICAN AMERICANS AND THERE'S PROSPECT TO DO VOI AND I THINK SCOTT AND OTHERS ARE DOING THAT SORT OF STUFF. >> I WANT TO FOLLOW ON THE COMMENTS OF ANDIRAN'S COMMENT ON DATA AND ACCESS TO MODELS. MAYBE PART OF THE WAY FORWARD IS SHAREABLE MODELS PERHAPS NIH REQUIRING SHAREABLE MODELS AND YOU TALKED ABOUT OLIGOPOLY OF DATA VENDORS. I'D BE CURIOUS FOR YOUR COMMENTS. >> IN MOST CASES YOU CAN CREATE YOUR OWN MODEL IF YOU JUST DEVOTE A LITTLE TIME. I THINK DATA IS A MUCH MORE SERIOUS. I HEAR YOU. SOME OF THE MODELS AND THERE ARE VENDORS THAT ARE BUILDING ON THE MODEL SIDE AND IF YOU HAD FREE MODEL IT WOULD BE MUCH BETTER. >> I THINK THIS IS GENE'S COMMENT IN A WAY WHICH IS THE FLIP SIDE IS PEOPLE INVEST IN THESE THINGS BECAUSE THEY THINK THEY'LL GET A PERSONAL LONG RUN RETURN SO TO THE EXTENT THEY'LL BE PUBLIC WE HAVE TO LOOK AT MODELS FOR AND CAREERS CONSISTENT WITH THAT. THAT'S THE TENSION THAT LED US TO THIS PLACE. >> IF I CAN FOLLOW UP AS WELL. PETER AND ANIRBAN THAT THERE'S AN OLIGOPOLY IN DATA AND WE NEED TO ASK WHY. MY FIRST READ IS THE PRODUCTION OF DATA IS THERE'S A LOT OF FIXED COSTS. VERY HIGH. WE KNOW FROM BASIC INDUSTRIAL ORGANIZATION WHEN HAVE YOU A FIXED COST IN INDUSTRY YOU'LL HAVE LESS ENTRY THEN YOU WOULD HAVE UNDER NORMAL COMPETITION. THERE'S ONE EXAMPLE OFFON OF JONATHAN'S POINT IN THE PRODUCTION OF DATA WE HAVE TO THINK THROUGH WHAT KIND OF MARKETING IMPERFECTION WE HAVE HERE AND HOW TO DO THAT. IT'S NOT GOING TO GO AWAY. I THINK THE MORE WE HAVE I.T. STUFF IT'S GOING TO GET EVEN WORSE. >> WASN'T TO MAKE TWO COMMENTS OF RANDOMIZED TRIAL DATA AND EXTENDING IT TO THE POPULATION. IF IN FACT TO GENE'S POINT ABOUT IT'S TEN YEARS LATER AND YOU CAN'T APPLY IT THEN WHAT'S THE BASIS FOR CURRENT PRACTICE? THEN WE'RE GOING ON AUTOPILOT. IN FACT YOU MAY FIND THAT IF YOU TAKE YOUR RCTs, THEY'RE OFTEN PLIKE YOU CAN SEE SIGNIFICANCE LEVELS OF 4%. WHEN YOU REWEIGHT THEM MY GUESS YOU'LL OFTEN FIND -- AND THIS GETS TO YOUR POINT, THEY'RE NOT SIGNIFICANT ANYMORE. YOU HAVE 11% FEMALES AND THEN 15% FEMALES AND YOU CAN'T SAY ANYTHING ABOUT HOW IT'S DONE AND HOW YOU PUBLISH THAT. LIKE IF IT'S NOT SIGNIFICANT YOU WON'T DO THE TREATMENT. AND EXTENDING OUT OF RCTs TO THE GENERAL POPULATION IS A LITTLE TRICKY BECAUSE THERE'S -- ONE OF MY FAVORITE PAPERS BY DAVID WEINBERG BY JACK'S SON HEAD TO THE COMPLICATIONS ARISE FROM CAROTID ENDARTERECTOMY ARE LOWER IN THE RANDOMIZED GROUP AND TRIPLE IN THE GENERAL COMMUNITY HOSPITALS. I THINK MAKING INFERENCES MAY TAKEAWAY IS THERE ARE WAYS WHERE YOU CAN USE SAY REGISTRY DATA WHERE YOU ONLY HAVE INFORMATION ON THE TREATED, NOT ON THE UNTREATED, BUT THE TREAT TO MAKE INFERENCES ABOUT THE QUALITY OF OUTCOMES ACROSS INSTITUTIONS BY VOLUME, BY ACADEMIC MEDICAL CENTER, ALL THESE FACTORS WHICH I THINK ARE IMPORTANT. >> I JUST WANT TO HAVE A QUICK FOLLOW UP TO THAT. ESPECIALLY WHAT YOU SAID I THINK HOW FAST SOMETHING IS OUTDATED DEPENDS ON WHAT THE THING IS. SO SOMEBODY WHO HAS A HIP IMPLANT IT'S SUPPOSED TO LAST FIVE YEARS AND FIVE YEARS AFTER THE FIRST METAL ON METAL HIP THERE'S DIFFERENCES BUT THE PATIENT STILL CARES ON WHAT WILL HAPPEN FROM THE HIP AND THE MANUFACTURERS SAY WE'VE GONE ON. IN TEN YEARS AND THE PATIENT ME WANTS TO KNOW. IT DEPENDS ON THE TECHNOLOGY. >> I TOTALLY AGREE. EFFICACY TRIALS ARE DIFFICULT TO PROJECT TO BECOME EFFECTIVENESS BECAUSE THE DATA ENVIRONMENTS WHICH WOULD NOT BE REPLICATED. IT GOES BACK TO THE TECHNICAL ASSUMPTION THE TREATMENT AFFECTS ARE IT SAME IN THE TRIAL THAN OFTEN NOT AND THERE'S A LOT OF PUSH TO DO BIGGER PRAGMATIC TRIALS AND WHEN THERE'S NOT MUCH PROTOCOL THEY OFTEN DON'T REFLECT REAL WORLD POPULATION. THERE'S AN NIH GUIDE ABOUT THE DISPARITY OF ACCESS OF TRIALS ACROSS THE COUNTRY. THERE I THINK THIS COULD BE ONE OR TWO THINGS TO COMPARE. >> AND I'LL START WITH DAVID, SO I'M GLAD TO HEAR YOU DISTINGUISH BETWEEN THE BASIC AND CLINICAL SIDE AND THERE'S INTERESTING POLITICS THERE. IF THERE'S SOME STUFF THAT'S MEASURABLE AND NOT HOW DOES THAT MIX AND STIR IF THE PROCESS. THAT'S MAYBE BEYOND YOUR MANDATE HERE. YOU TOUCHED ON A NUMBER OF CLASSIC THINGS. AND I WONDER IF THIS IS SOMETHING YOU THINK ABOUT THIS IS THE EXTEND OF MARKET FAILURE AND CROWD OUT. ITS THE ADDITIONAL VALUE OF INFORMATION RELATIVE TO WHAT OTHERS MAY BE DOING IS THE QUESTION. I THINK AS THE A LOT OF MARKET VALUE. YOU HAVE TO THINK ABOUT THE FINAL DEMAND SIDE ON THE CONTEXT OF INSURANCE. TECHNOLOGIES THAT MAY HAVE NEGATIVE VALUE AND WE HAVE DONE A PAPER ON INDIVIDUALIZATION WE WROTE AND THAT CAN TURN EVERYTHING ON ITS HEAD. THE OTHER QUESTION IS WHAT SHOULD BE FUNDED BY NIH IN THE CONTEXT OF WHETHER OTHER INVESTMENTS WILL BE MADE AND IT'S A VALID COMMENT. IT'S NOT SOMETHING WE WROTE ABOUT IN OUR WORK BUT IT'S REASONABLE AND WHERE THERE'S MARKET FALL USE FOR RETURN ON INVEST MANY I AGREE. AND PETER AND I AND OTHERS WERE ON THE PANEL ON THE COST EFFECTIVENESS AND ONE OF THE KEY MESSAGES OF THE REPORT IS NOT JUST TO THINK ABOUT THINGS FROM A SOCIETAL PERSPECTIVE BUT FROM MULTIPLE PERSPECTIVES. I THINK THAT MOVE COULD BE REFLECTED IN VALUE OF INFORMATION ANALYSIS AND COULD MAKE IT A RICHER TOOL FOR STRATEGIC DECISION MAKING. >> IT'S UNDERSTANDING HOW THE VALUE IS APPROPRIATED BY DIFFERENT STAKEHOLDERS. I THINK THAT HAS RAMIFICATIONS FOR WHAT KIND OF RESEARCH MONEY SHOULD GO IN VERSUS A CROWD OUT WHERE MANUFACTURERS CAN INVEST. THE V.A. MODEL WHICH IS BASED ON THE SEARCH COST. IT'S A COOL TO STUDY THESE. >> WE DID A STUDY WITHIN THE CLINICAL TRIALS GROUP AND JOSH AND DAVID AND ANIRBAN WAS PART OF THE STUDY AND POSITIVE NOTES AND LESS POSITIVE NOTES OF HOW V.A. PLAYS OUT. RESEARCHER CLINICIANS ARE TRAINABLE IN VOI. YOU CAN TRAIN THEM TO UNDERSTAND THE TECHNIQUES AND TRACK IT. IT'S ALSO FEASIBLE TO GET VOI TO THE DECISION MAKERS AT THE TIME THEY'RE MAKING PRIORITIZATION DECISIONS. ON THAT FRONT I THINK IT'S MADE INNOVATIONS WITH MINIMAL OR NO MODELLING. ON THE NEGATIVE SIDE I'D HAVE TO SAY WE'RE A LONG WAY FROM ACCEPTING VOY IN THE COMMUNITY. ONE OF THE BIG USE IN MY OPINION IS FACTORING COSTS OF THE TECHNOLOGIES AND DECISIONS. IN THE ONCOLOGY WORLD OUR CLINICIAN RESEARCHERS WERE VERY RESISTANT TO INCLUDING COST. THEY SAID IT WASN'T WITHIN THEIR PURVIEW AND THEY WERE BEING PENALIZED FOR DOING RESEARCH ON HIGH COSTS. AND MANY ARE SO EXPENSIVE AND THE MARGINAL BENEFIT IS MODEST. AND THE FINAL OBSERVATION IS WHERE DO WE PUT IT IN THE PROCESS? THE CONCEPTS WERE DEVELOPED FAIRLY FAIR ALONG. I'M NOT SURE THAT'S THE BEST PLACE TO DO VOI. BUT ON THE OTHER HAND HAVING US IN THE SCIENCES AND HEALTH ECONOMICS INTERACT WITH THESE RESEARCHERS I DON'T THINK WE'VE LOOKED AT HOW THEY'VE WORKED AND THAT'S THE CHALLENGE IN THE FUTURE. > DAVID CHAN SAID THAT ALL V.A. C CAFETERIAS LOOK ALIKE. THIS IS NOT TRUE OF NIH CAF CAFETERIAS. WE'RE IN THE BUILD THE BEST ONE AND THERE'S A SNACK BAR NEXT TO IT AND YOU CAN TAKE YOUR LUNCH OUTSIDE AND GET BACK IN WITHOUT SECURITY CLEARANCE. I HAVE NO IDEA WHAT IT'S LIKE OUT THERE BUT IT MIGHT BE GOOD. >> SO I GUESS WE'LL SEE EVERYBODY BACK HERE FOR PANEL 3 AT 1:25. WE'RE GOING TO START WITH OUR THIRD MANLE AND I WILL TURN IT OVER TO DR. LOWY. THANK YOU. >> GOOD AFTERNOON AFTERNOON I'M DOUG LOWY, ACTING DIRECTOR OF THE NATIONAL CANCER INNSITUTE AND THE SESSION CHAIR FOR SESSION NUMBER 3. AND I REALLY HAVE KIND OF 2 DIFFERENT INTRA DUCTORY REMARKS THAT I'D LIKE TO MAKE. FIRST WE ARE EXTRAORDINARILY FORTUNATE IN CANCER THAT THERE HAVE BEEN A LOT OF ADVANCES AND MANY OF THEM HAVE BEEN IN THE AREA OF PRECISION MEDICINE. BUT THE ISSUE OF COST AND COST EFFECTIVENESS, ET CETERA, IS FRONT AND IS IT FOR US ALL THE TIME AND EVEN MAKE ITS INTO THE NEWSPAPERS, WITH SOME REGULARITY. BUT I WILL SAY THAT FROM A PERSONAL PERSPECTIVE, I CERTAINLY AM PAINFULLY AWARE OF NOT QUITE THE ISSUE OF MY--SHOULD I TAKE MY MEDICINE OR PAY MY MORTGAGE, BUT INSTEAD, I HAVE BEEN INVOLVED IN THE HPV VACCINE SITUATION AND WE HAVE A VACCINE THAT WORKS WELL AND THAT HAS BEEN RELATIVELY WIDELY TAKEN UP IN THE FIRST WORLD. BUT THE PUBLIC HEALTH PROBLEM IS IN THE DEVELOPING WORLD.AND THE UPTAKE OF THE VACCINE IS ABOUT 3% AMONG ELIGIBLE PEOPLE IN THE DEVELOPING WORLD AND NCI IS STARTING A CLINICAL TRIAL THAT WHEN THE VACCINE WAS ORIGINALLY APPROVED, IT WAS FOR 3 DOSES AND THE TRIAL IS REALLY BASED ON SOME UNEXPECTED POST HOC ANALYSIS WHICH SUGGESTS THAT 1 DOSE MIGHT BE SUFFICIENT. SO THE PURPOSE OF THIS TRIAL IS REALLY TO SEE, DO YOU GET STRONG BENEFIT FROM IT BECAUSE AGAIN, BACK OF THE ENVELOPE ANALYSIS, NOT THE ACTUAL METRICS, JANE KIM IS OTHERS ARE WORKING ON THAT BUT CLEARLY IF WE COULD GET IT DOWN TO 1 DOSE, THE LIKELIHOOD OF WIDER DISSEMINATION WOULD BE MUCH GREATER SO LET'S COME BACK TO THE MATTER AT HAND WHICH REALLY IS THE AREA OF ECONOMIC BENEFITS IN THE CANCER SPACE. AND THE FIRST TALK IS GOING TO BE TANDEM PRESENTATION BY DAVID VEENSTRA AND JOSH CARLSON. AND FOR THOSE WHO CAN'T READ, THEIR TITLE IS DEMAND FOR PERSONALIZED MEDICINE AND IMPLICATIONS FOR RESEARCH PRIORITIZATION AND DELIGHTED TO HEAR THIS. THANK YOU. >> GREAT, THANKS VERY MUCH, WELCOME BACK FROM LUNCH, EVERYBODY, SO I WILL TRY TO SET THE STAGE FOR OVERALL PROJECT, THE WORK WE'VE BEEN DOING AT UNIVERSITY OF THE WASHINGTON AND JOSH WILL TALK A BIT MORE SPECIFICALLY ABOUT SOME OF THE DATA COLLECTION THAT WE JUST FINISHED. SO THERE'S MULTIPLE GOALS OF THIS, OUR PROJECT WHICH WE CALL PRIMER, I THINK THE BIGGEST OVERALL GOAL IS REALLY TO UNDERSTAND WHAT ARE THE DRIVERS FOR UPTAKE AND ADOPTION OF PERSONALIZED MEDICINE AND HOWEE CAN USE THAT TO INFORM RESEARCH PRIORITIZATION AS WELL AS BETTER UNDERSTANDING EVIDENCE THRESHOLDS. SO THESE ARE THE 3 PROJECTS, 3 AIMS, 1 HAS TO DEVELOP AN ECONOMIC MODEL FOR PERSONALIZED MEDICINE, BUILDING ON SOME EARLIER WORK THAT SUE HAS DONE. WE ARE ALSO ASSESSING INDIVIDUAL'S PREFERENCES FROM THE SOCIETAL PERSPECTIVE, PROVIDERS AND PAIRS, AND THEN, THAT INFORMATION WILL FEED INTO A THIRD AIM WHICH IS REALLY, DEVELOPING A PRAGMATIC FRAMEWORK FOR BETTER UNDERSTANDING THE IMPLEMENTATION, EVIDENCE THRESHOLDS, REIMBURSEMENT FOR PRECISION MEDICINE. SO THIS IS THE PUBLICATION THAT CAME OUT LAST YEAR ON THE FRAMEWORK FOR PRIORITIZING RESEARCH AND PRECISION MEDICINE AND I THINK THE GENERAL CONCEPT HERE IS THAT, YOU KNOW OBVIOUSLY PHYSICIAN PAIR AND PATIENT DECISION MAKING ARE THE DRIVERS OF ADOPTION OF PRECISION MEDICINE AND JUST THINKING ABOUT REALLY THE EVIDENCE THAT'S GENERATED AND GOING TO BE GENERATED IN THIS FIELD OVER THE COMING YEARS AND HOW THAT CAN INFLUENCE THE BEHAVIOR OF THESE DIFFERENT DECISION MAKERS. AND I BELIEVE ALSO WORKING WITH DAVID MELZER DEVELOPED THIS FRAMEWORK FOR TRYING TO UNDERSTAND THE VALUE OF INDIVIDUALIZED CARE AND ANIRBAN OBVIOUSLY LAID OUT THE WORK AND FACTORS THAT SHOULD BE TAKEN INTO CONSIDERATION, TREATMENT UPTAKE, IS IT COVERED BY INSURANCE, WHAT ABOUT UNCERTAINTY AND THE INFLUENCE OF EVIDENCE ON THAT UNCERTAINTY AND TO HAVE TRAJECTORIES OF ADOPTION OVER TIME SO WE CAN BETTER UNDERSTAND WHAT THE VALUE OF RESEARCH IN THIS SPACE MIGHT BE. SO WE REALLY NEED TO HAVE A PREDICTIVE MODELS FOR UPTAKE OF PRECISION MEDICINE AND THAT'S HA WE'VE DONE IN AIM 2 AND WE JUST FINISHED UP A LARGE NATIONAL SURVEY OF THE U.S. POPULATION AND I'LL TURN IT OVER TO JOSH KARLINGSON TO TELL YOU ABOUT THAT. >> THANKS, DAVID. HELLO, EVERYONE. SO BASICALLY AIM TO, SOUGHT TO ASSESS PROVIDER AND PAYER NEAR PERSONALIZED MEDICINE AND PROJECT DIFFUSION AND IT LINKS TO AIM 1 AND IT CAN BASICALLY INFORM THE FRAMEWORK THAT ANIRBAN ESTABLISHED AND YOU CAN SEE HOW IT LINKS TO AIM 3 WHICH IS MORE SPECIFIC ESTIMATES AND HOW PREDICTIONS CAN BE UPDATED WITH THAT TYPE OF INFORMATION. SO BY WAY OF BACKGROUND AND DAVE SPOKE A BIT ABOUT THIS, BASICALLY WE ESTABLISHED THE 3 KEY DRIVERS ARE PROVIDER BEHAVIOR, PROVIDER PREFERENCES, THAT GETS AT PROVIDER KNOWLEDGE AND THE RECOMMENDATIONS. PAYOR PREFERENCES, THAT ESTABLISHES ACCESS AND REIMBURSEMENT LEVELS AND OFTEN LINKED WITH PROVIDER RECOMMENDATIONS AND ALSO PATIENT PREFERENCES DRIVE THE VALUE OF PERSONALIZED MEDICINE AND PAYOR PREFERENCES AND PROVIDERS PREFERENCES CAN INFORM PATIENT DECISIONS SO THESE ARE INTERTWINED AND WE SOUGHT TO BASICALLY EVALUATE ALL 3 OF THEM SO WE COULD INFORM DIFFUSION OVER ALL SO WE ARE LOOKING AT POPULATION LEVEL DIFFUSION ESTIMATES RATHER THAN SMALL, LIKE INDIVIDUAL LEVEL DIFFUSION OR UPTAKE ESTIMATES. AND SO, UNDERSTANDING QUANTIFYING THESEPREFERENCES WILL HELP US INFORM THAT FRAMEWORK. AND ALSO ALLOW PREDICTIONS AGAIN AT BOTH THE INDIVIDUAL AND POPULATION LEVEL FOR SPECIFIC TECHNOLOGIES SO OUR FRAMEWORK IS BASICALLY GOING TO BE PLIED BROADLY SO IT WILL BE USED TO SEE AND INFORM SPECIFIC PERSONALIZED MEDICINE TECHNOLOGIES, SO THAT'S' KEY THING TO BE ABLE TO GET OUT OF THIS. SO OUR APPROACH WAS A MIXED METHODS APPROACH AND IT STARTS WITH DOING A BUNCH OF QUALITATIVE WORK. THAT WORK DID TAKE A WHILE TO PERFORM. BUT BASICALLY WE SAT DOWN, WE HAD FOCUS GROUPS AND A BUNCH OF INTERVIEWS WITH PATIENTS, PROVIDERS AND PAYORS AND TO UNDERSTAND THE KEY ATTRIBUTES THAT IMPACTED THEIR DECISIONS, SO WHAT ARE THE THINGS THAT DROVE THE DECISIONS, WHAT WAS MOST IMPORTANT TO THEM? AND THEN WE TOOK A LONG LIST OF STUFF THEY TALKED ABOUT AND GET THAT DOWN TO THE CORE STUFF THAT ULTIMATE GOAL MITTLY WE FELT WOULD DRIVE DECISIONS AND WOULD ALLOW US TO CREATE A MORE QUANTITATIVE SURVEY WHICH WE SUBSEQUENTLY DID. WE ALSO HAD TO ESTABLISH WHAT ARE CALLED ATTRIBUTE LEVELS, SO IF COST IS IMPORTANT TO PEOPLE, WE WANTED TO BE ABLE TO HAVE LEVELS OF COSTS THAT ARE SORT OF ACCURATE, AND REPRESENT THE RANGE OF OPTIONS SO IT MIGHT BE FROM A HUNDRED TO $4000 SO YOU HAVE TO ESTABLISH THOSE LEVELS AND THAT QUALITATIVE WORK CAN HELP INFORM THOSE TYPES OF THINGS AS WELL: SO WE TAKE THAT AND WE BASICALLY DEVELOP A QUANTITATIVE SURVEY, CALLED A DISCREET CHOICE EXPERIMENT AND WE USE THAT IN A LARGER SAMPLE OF PATIENTS, PROVIDERS AND PAYORS TO USE POPULATION LEVEL ESTIMATES WHEN WE GO THROUGH THIS AND I WILL SHOW YOU WHAT THIS LOOKS LIKE BUT WE'RE ABLE TO GET AT AGAIN PEOPLE'S UNDERSTANDING AND PREFERENCES FOR WHAT ATTRIBUTES ARE MOST IMPORTANT SO WE CAN RANK THEM. WE CAN ALSO UNDERSTAND MARGINAL RATES OF SUBSTIUTION SO HOW MUCH RISK PEOPLE ARE WILLING TO TAKE FOR CERTAIN LEVELS OF BENEFIT. HOW MUCH COST THEY'RE WILLING TO ASSUME FOR BENEFITS OR TO AVOID RISKS, ET CETERA. WE CAN ALSO, AND THIS IS KEY FOR DIFFUSION ESTIMATES UNDERSTAND THE PROBABILITY THAT INDIVIDUALS WILL USE AND RECOMMEND OR REIMBURSE INTERVENTION FOR A SPECIFIC TEST WITH THE GIVEN SET OF ATTRIBUTES. AND AGAIN, THIS IS LINK BACK TO PARAMETERS FOR THE EXPANDED EVIC WHICH CAN ULTIMATELY INFORM VALUABLE INFORMATION ESTIMATES AND ULTIMATELY THE RESEARCH PRIORITIZATION EXERCISES. SO NOW I WILL DIVE INTO OUR SPECIFIC DISCREET CHOICE EXPERIMENT AND THE SCOPE WE IDENTIFIED WAS FOCUSING ON GENETIC TESTS THAT CAN INFORM MEDICAL CARE, SPECIFICALLY SO GENETIC TESTS CAN BE USED FOR A LOT OF STUFF AND WE WERE FOCUSED ON 1S THAT CAN INFORM MEDICAL DECISIONS AND THE 1S WE ISOLATED WERE BASICALLY GENETIC TESTS IN TERMS OF SCREENING SO THIS IS ABOUT PROVIDING INFORMATION, PROVIDING FUTURE DISEASE RISK AND 1S THAT WERE MORE SPECIFIC TO TREATMENT CHANGE THAT, IS PAIRKTS WHO HAVE A SPECIFIC CONDITION RIGHT NOW FOR WHICH THE TREATMENT CHOICE IS BEING MADE AND CAN THIS INFORMATION HELP WITH THAT SPECIFIC DECISION, NOT NECESSARILY ABOUT SOMEONE'S FUTURE DISEASE RISK, SO WE WANTED TO BE ABLE TO TEASE THAT OUT AND WE STARTED OFF BY SETTING UP THE SCOPE FOR PEOPLE WHEN THEY TAKE THESE TYPES OF TESTS. THIS' INFORMATION WE HAVE TO PROVIDE PEOPLE TO ALLOW THEM TO HAVE ENOUGH INFORMATION TO MAKE CHOICES LATER ON. SO BASICALLY THIS IS HOW WE WOUND UP SCOPING IT. WE PROVIDE EXAMPLES, SO THE EXAMPLE OF KREENING IS BASICALLY PATIENT CANS HAVE A GENETIC MARKER AT HIGHER RISK OF DEVELOPING COLON CANCER IN THE FUTURE AND YOU MIGHT CHAIMPLE AND DO MORE SCREEN NOTHING THAT INDIVIDUAL. FOR TREATMENT CHANGE, WE USE THE WARFORIN, PEOPLE WITH A SERIES HEART CONDITION CAN RECEIVE A TEST THAT HELPS CHOOSE THE CORRECT DOSE FOR THAT INDIVIDUAL. WITHOUT THAT TEST THEY MAY HAVE GOTTEN A STANDARD DOSE FOR EXAMPLE. SO THE ATTRIBUTES THAT WE WOUND UP COMING UP WITH, THROUGH THAT PROCESS, WERE THE TYPE OF GENETIC TEST AGAIN SCREENING AND TREATMENT CHANGE OR COMBINATIONS OF BOTH. THE PROBABILITY YOU HAVELET MEDICAL MARKER. SO THIS IS--IF IT'S GENERATED GENETICS MAYBE THE PREVALENCE OF A GENETIC MUTATION. MEDICAL EXPERT AGREEMENT. SO THIS HAS TO DO WITH BASICALLY FOR EXAMPLE, PROVIDER GROUPS ARE RECOMMENDING THIS IF ALL PROVIDER GROUPS RECOMMENDING THIS SOME LEVEL TO WHICH THE MEDICAL EXPERT COMMUNITY SAYS THAT IF--IF THAT MARKER IS PRESENT WE RECOMMEND A MEDICAL CHANGE TAKES PLACE. WE GOT INTO IT WITH CHANGE AND HEALTH AND FUNCTION SO WE USE HEALTH AND FUNCTION. WE TALK TO PATIENTS, QUALITY OF LIFE WHEN WE WERE TALKING TO PROVIDERS THROUGH QUALITATIVE WORK FOUND OUT IT REZONING THAT NAILTED BETTER WITH PATIENTS IF THEY TALK ABOUT HEALTH AND FUNCTION WHICH IS QUALITY OF LIFE WHICH IS HARDER FOR THEM TO GATHER. CHANGE IN HEALTH AND FUNCTION AND LIKELY CHANGE IN LIFE EXPECTANCY, THIS IS THE MORTALITY COMPONENT. HERE IT'S PRETTY COOL, WE ADDED UNCERTAINTY TO THIS SPECIFIC PARAMETER AND YOU WILL SEE LATER THAT PEOPLE ACTUALLY RESPONDED TO DIFFERENT UNCERTAINTY LEVELS WHICH IS A KEY PIECE AND YOU CAN THINK ABOUT HOW EVIDENCE ACCUMULATES OVER TIME IF YOU TRACK THAT AND PEOPLE ARE WILLING TO HAVE A PREFERENCE FOR CERTAINTY, THEN THAT CAN CHANGE ESTIMATES OVER TIME AND THAT'S A KEY PIECE TO THINK BEING DIFFUSION OVER TIME AND ULTIMATELY COST OF THE TESTING TO YOU AND THE COST TO YOUR PLAN OR THE COST OF THE TEST IN GENERAL. SO THEN WE WENT THROUGH AND CREATED ATTRIBUTE DESCRIPTIONS, I WON'T READ THIS WHOLE 1, THIS IS THE DESCRIPTION FOR EXPERT AGREEMENT ON MEDICAL CARE CHANGE. WE DEVELOPED A LITTLE VIDEO ABOUT A 4 MINUTE VIDEO, HERE ARE GRAPHICS FROM THAT. AND THAT WORK HAS BEEN SHOWN TO HELP PEOPLE WITH UPTAKE OF INFORMATION SO EVERYBODY HAD TO WATCH A VIDEO, SLIGHTLY DIFFERENT FOR EACH GROUP BUT BASICALLY THE SAME THING, COVERING ALL OF THE DIFFERENT ATTRIBUTES WITH SUBSCRIPTIONS AND THEN WE PROVIDE THE TEXTURAL MATERIAL TO REINFORCE THAT. AND HERE'S WHAT THE ACTUAL TASK LOOKS LIKE. BASICALLY WE HAVE THE ATTRIBUTES ON THE LEFT AND THEN WE HAVE DIFFERENT OPTIONS. THEY CAN EITHER CHOOSE OPTION A WHICH HAS A SET OF OF ATTRIBUTES. SO HERE WE HAVE A SCREENING TEST, 1 OUT OF A HUNDRED PEOPLE WILL HAVE THE MARKER, MOST EXPERTS AGREE THAT IF THERE'S A MARKER THAT CHANGE SHOULD HAPPEN, YOUR QUALITY OF LIFE IS EXPECTED TO GO FROM POOR TO FAIR IF YOU HAVE A MARKER. YOUR LIFE EXPECTANCY IS AN AVERAGE OF 2 YEARS BUT WE HAVE AN UNCERTAINTY OF 1.5 TO 2.5. NARROW, COSTS $100. OR OPTION B HAS A DIFFERENT SET OF LEVELS FOR EACH ATTRIBUTE AND YOU CAN NOTICE THAT THE UNCERTAINTY LEVEL HERE, AVERAGE IMPROVEMENT IS 2 YEARS BUT THE UNCERTAINTY IS WIDER AND HERE WE HAVE THE SAME BENEFIT BUT YOU CAN CHOOSE BETWEEN THOSE UNCERTAINTY LEVELS SO WOE GO THROUGH THIS EXERCISE AND EACH PERSON HAD TO DO 16 OF THESE AND WE GO THROUGH MANY, MANY PEOPLE HAVING TO DO THAT WE ARE GETTING A LARGE NUMBER OF SAMPLES AND IN THIS CASE, THE PERSON CHOSE B -EXAMPLE, THEY WILL OPT OUT AND CHOOSE NEITHER TEST. SO ALL THE CHOICES ARE AVAILABLE TO THEM. SO WHEN YOU GO THROUGH THIS, YOU CAN SEE HOW PEOPLE ARE GOING TO BASICALLY MAKE AND TRADE OFF ATTRIBUTE LEVELS ACROSS EACH OF THE DOMAINS AND YOU DO IT ENOUGH TIMES AND CAN YOU BASICALLY CREATE A WAY TO UNDERSTAND THOSE MARGINAL RATES OF SUBSTITUTION. SO WHAT DID WE FIND? WE ARE IN THE PROCESS, WE WRAPPED UP THE PATIENT ELEMENT, WE ARE IN THE PROCESS OF DOING THE PAYOR AND PROVIDER DCs RIGHT NOW. BUT FOR THE PATIENTS WE USED A NATIONALLY REPRESENTATIVE SAMPLE AND AUDIOS AND TAKE THE SURVEY AND THEY DEMONSTRATE SIGNIFICANT DEMAND IF THERE WAS LOW MEDICAL EXPERT AGREEMENT, IF THERE WAS NO OR LITTLE CHANGE IN QUALITY OF LIFE, WITH INCREASING COST, SO THAT SOUNDED LIKE WE WOULD EXPECT, SO IT'S GOOD WHEN YOU'RE DOING THESE TYPES OF STATED PREFERENCE THINGS TO HAVE SOME RATIONALITY TO WHAT WE OBSERVE, BUT IMPORTANTLY WE CAN ALSO GET THE MAGNITUDE OF THESE EVENT WHICH IS IS IMPORTANT AND THAT GOES TO THE ULTIMATE DIFFUSION ESTIMATES, ET CETERA. INCREASING DEMAND IS ASSOCIATE WIDE GIED TREATMENT AND 1S THAT HAD A HIGHER PROPPABILITY OF HAVING AN INFORMATIVE MARKER AND WHAT WAS REALLY COOL WAS RESPONDENTS CAN MEANINGFULLY ESTABLISH BETWEEN HIGH AND LOW UNCERTAINTY, SO WE CAN GET--WHAT THEY MIGHT BE WILLING TO PAY FOR THIS, SO WILLINGNESS TO PAY FOR DECREASED UNCERTAINTY WAS ASSOCIATED BETWEEN 150 AND ABOUT $220. AND THE HIGHER ESTIMATES WERE ASSOCIATED WITH RESOLVING UNCERTAINTY FOR LOWER GAINS, AND SO, AND A LITTLE BIT LESS FOR WHEN THE AVERAGE GAME WAS HIGHER, MEANING AT THE LOWER--IF THE GAIN WAS LESS, THEY WERE WILLING TO PAY MORE FOR CERTAINTY AROUND THAT WHICH IS AGAIN WHAT WE EXPECT, BUT THIS HASN'T BEEN SHOWN TOO MUCH IN THE LITERATURE OUT THERE. SO THIS IS A PRETTY COOL FINDING. WE THOUGHT. SO ULTIMATELY WE CAN ALSO DO DIFFUSION PREDICTIONS SO HERE WE SHOW WHAT THIS LOOKS LIKE FOR THE ONK O TYPE DX EXAMPLE, IESM SURE NOBODY HEARD OF THE THIS EXAMPLE, SO WE OFTEN USE IN PERSONALIZED MEDICINE BECAUSE THERE'S DATA OUT THERE ON IT. IT'S USED, ET CETERA. SO WE BASICALLY GRABBED THE ATTRIBUTES AND SAW HOW THEY CHANGED OVERTIME AND THIS IS TIME FRAME FROM 2004 TO 2011 AND ON THE LEFT WE HAVE THE PROBABILITY OF UPTAKE FOR A GIVEN ELIGIBLE POPULATION AND SO, WE WERE ABLE TO SHOW OR PREDICT THE PREDICTED UPTAKE TO ABOUT 63% IN 2011 AND WE HAVE DATA--THE ACTUAL DIFFUSION I THINK THIS IS FROM INTERMOUNTAIN HEALTHCARE, WE ARE IN THE PROCESS OF ACCUMUMENTING DATA TO DO THIS IN OTHER SETTINGS BUT CAN YOU SEE THAT IT CORRELATES PRETTY WELL WITH WHAT THE OBSERVATION WAS. THIS IS ACTUALLY SOMETHING THAT WILL BE A TOOL THAT CAN SHOW HOW WE WILL DO THIS MORE PROSPECTIVELY. WE WERE AWLINGS ABLE TO LOOK AT AGE EFFECT, PART OF THE REASON WEB CONNECTED WENT UP THAT HIGH WAS TO ABLE TO GET A ELECTRIC AT SUBGROUPS AND SO HERE WE JUST SHOW THERE IS A SLIDE AGE EFFECT, SO PATIENTS WHO ARE OVER 65 HAD SLIDELY REDUCED ESTIMATES OF UPDIAMOND CIRCLE ACHIEVER VERSUS THOSE WHO WERE YOUNGER AND I THINK WE SAW THAT IN THE EARLIER DATA THAT WAS PRESENTED, THERE WAS AGE EFFECT RELATED TO THAT. SO WITH ONGOING WORK, WE'RE BASICALLY TAKING THIS, I SAID WE'RE ALSO DOING THE PAYOR AND PROVIDER DCEs, ALL OF THAT WILL COME TOGETHER TO TRY AND INFORM A PREDICTIVE MODEL THAT WE ARE GOING TO DEVELOP. WE WILL LOOK AT DATA FOR DIFFUSION FOR MULTIPLE GENETIC TESTS, WE HAVE A COUPLE IN THE PROCESS OF DOING THAT, TO TRY AND TEASE OUT EFFECTS THAT WILL NOT BE RELATED TO THE ATTRIBUTE CHANGES THEMSELVES. SO FOR EXAMPLE, IF THE ATTRIBUTES DIDN'T CHANGE AT ALL, WE WILL EXPECT TO SEE SOME DIFFUSION IN THE MARKET BECAUSE OF SOCIAL KONT GEN AND THE FACT THAT TECHNOLOGY WILL DIFFUSE OVER TIME WITHOUT A NECESSARY CHANGES IN CERTAIN ATTRIBUTES. SO WE CAN TEASE THAT OUT. WE GET AN UNDERLYING MODEL AND GET A LOOK AT HOW ATTRIBUTES ARE EXPECTED TO CHANGE ESPECIALLY AROUND EVIDENCE ACCUMULATION AND UNCERTAINTY WE CAN EXPECT TO CHANGE OR TIME AND THAT WOULD BASICALLY ALLOW US TO DO PREDICTIONS BASED ON CURRENT INFORMATION. SO ULTIMATE GOAL--ULTIMATELY WE HAVE A WEB BASED TOOL. IT'S NOT PARAMETERIZED TO ANY LARGE EFFECT BUT THAT'S 1 OF THE THINGS WE WILL DO AND THAT CAN BE USED BY RESEARCHERS AND IT CERTAINLY CAN BE LINKED TO VALUE OF INFORMATION TYPE ANALYSIS BECAUSE WE SAW THE KEY INFORMATION WAS VALUE OF INORMATION, AGAIN WITH THE 0, THERE'S NO VALUE THERE.& SO THIS CAN LINK BACK TO ALL THOSE TYPES ESTIMATES WHICH CAN BE USE INDEED RESEARCH PRIORITIZATION. I WILL TURN IT OVER TO DAVE AND HE WILL WRAP IT UP TO AIM 3 AND PROVIDE A SUMMARY. >> THANK YOU JOSH. SO THIS IS SOME WORK, MY GRADUATE STUDENT JUST FINISHED LATE LAST WEEK BASED ON SOME OF THESE RESULTS THAT JOSH DESCRIBED AS UPTAKE. WE DON'T HAVE PAYOR AND PROVIDER UPTAKE ESTIMATES, BUT WHAT WE SAW WITH PATIENTS ARE PREDICTED UPTAKE,Y WOOK LOOKED AT THE TESTING EXAMPLE AND WE ROUGHLY PLUGGED IN WHAT WE THOUGHT WERE ASHES TRIBUTES FOR THAT AND WAS PREDICTED UPTAKE WITH A $20 PATIENT COST OF ABOUT 70%. SO WE CAN USE THAT TO SAY, HEY IN THE FUTURE HOW MUCH VALUE MIGHT THIS HAVE FOR WARFORIN. NOW THIS A COMPLICATED SLIDE SO LET ME STEP YOU THROUGH THIS HERE,OT X-AXIS WHAT WE HAVE IS OVERTIME, SO AS EVIDENCE ACCUMULATES AND ON THE Y-AXIS HERE WE HAVE THE MAXIMUM POSSIBLE VALUE OF RESEARCH. SO EARLIER ON, YOU CAN SEE THAT THERE WAS A LOT OF POTENTIAL VALUE OF RESEARCH AND AS TIME WENT ON, THAT DECREASED AND I WILL EXPLAIN WHAT THESE 2 DIFFERENT LINES ARE IN A SECOND, BUT THE ORANGE LINE IS FOR THE PHARMACOGENOMICS EXAMPLE AND I WAS MOTIVATED TO PURSUE THIS CASE STUDY BECAUSE OF CLINICAL GUIDELINES THAT STATED FURTHER RESEARCH IS NEEDED BEFORE PHARMACOGENOMICS WILL BE RECOMMENDED AND AT HOW MUCH MORE RESEARCH AT WHAT POINT DO YOU SAY WE HAVE ENOUGH DATA? SO YOU MIGHT LOOK AT THIS AND SAY, HEY, LOOK THIS HAS GONE DOWN, THEY JUST PUBLISHED RCT, AND IT LOOKS PRETTY PROMISING FOR WARFORIN ISSUES, SO IT WOULD SUGGEST THAT FOR GUIDELINE COMMITTEES THAT MAYBE WE DON'T NEED TO DO RESEARCH. THE THING WE DID TO TRY TO HELP POLICY MAKERS THINK THIS THROUGH IS WE TOOK AN EXAMPLE OF A DRUG INTERACTION, AMYOAT OWN OR WARFORIN, WHERE THEY SAY AVOID THOSE DRUGS TOGETHER AND WE SAID WHAT KIND OF EVIDENCE IS THAT BASED ON? WE DID THE SAME CALCULATIONS AND WE FIND THAT KIND OF THE RESEARCH IS VALUE RESEARCH IS TRENDS ALONG SIMILAR LINES OVER TIME DON'T DO THE TESTING BECAUSE THERE'S NOT ENOUGH EVIDENCE SO THESE TYPES OF ANALYSIS, POTENTIALLY NOT ONLY CAN BE USED TO IDENTIFY GREAT AREAS TO INVEST IN RESEARCH, THEY CAN ALSO HELP THINKING AROUND POLICY DECISIONS AND SO, THIS IS SOME WORK WE'RE PRETTY EXCITED AND WE'LL HAVE TO WAIT UNTIL WE GET THESE PREDICTION ESTIMATES FROM THESE POPULATIONS BEFORE WE FULLY COMPLETE THIS STUDY. SO JUST TO SUMMARIZE, HOPEFULLY YOU WILL SEE PROJECT HERE AND ALSO FROM DAVID'S TALK ABOUT HOW VOI TYPE APPROACHES CAN BE USED TO PRIORITIZE RESEARCH AND ACTUAL ACTUALLY ESTIMATE THE VALUE OF PERSONALIZED MEDICINE AND THAT SLIDE, LAST EXAMPLE TRYING TO GET A BETTER UNDERSTANDING OF EVIDENCE THRESHOLDS IN GENOMICS VERSUS OTHER AREAS IN MEDICINE. WITH THAT I WILL THANK YOU IF ARE YOUR TIME. >> THANK YOU SO MUCH. WE WILL CONTINUE THE THEME OF ONK O TYPE TESTING, SCOTT FROM THE CANCER RESEARCH CENTER AND GEORGE WILL BE IN THIS 1. >> IN THE INTEREST OF TIME I WILL PRESENT THE WORK ON BEHALF OF MYSELF, SCOTT RAMSEY, TRACEY LIEU AND THE REST OF OUR TEAM AND WE TALKED ABOUT ONK O TYPE DX, IT'S 1 OF THE POSTER CHILDS FOR COMMONLY USED PERSONALIZED TREATMENT AND IT'S BEEN VALIDATE INDEED A PARTICULAR SUBSET OF BREAST CANCERS, AND IT'S BEEN MARKETED QUITE A BIT AS SOMETHING THAT WILL HELP ESTIMATE RISK OF RECURRENCE SO THAT KEY CAN SELECT CHEMO THERAPY FOR THOSE MOST LIKELY TO RECUR AND BENEFIT AND AVOID CHEMO THERAPY, AND THOSE WHO HAVE A LOW RISK OF RECURRENCE. IT'S ALSO SUPPOSED TO INCREASE THE CONFIDENCE AND THE DECISION ABOUT CHOOSING CHEMO THERAPY BOTH FOR THE PATIENT AND THE PHYSICIAN, AND ULTIMATELY, LOWER COSTS BY AVOIDING UNNECESSARY CHEMO THERAPY. SO WE'VE DONE A MULTILEVEL--WHAT I CALL PRAGMATIC APPROACH TO SEE WHAT'S HAPPENED. I WILL GO THROUGH AND PRESENT RESULTS WE GOT LOOKING AT THESE MULTILEVEL INFLUENCES, THE CULL COST EFFECTIVENESS. THERE WERE EARLY STUDIES SUGGESTING THAT ONK O TYPE WOULD COST SOMETHING ABOUT 8-$10,000 PER YEAR OF LIFE SAVED WHICH IS A REALLY GOOD VALUE FOR THE MONEY. IN PRACTICE IT SEEMS TO BE DIFFERENT AND I'LL SHOW YOU THAT. IN TERMS OF THE MARKETING, THERE'S IMPORTANT PATIENT OUTCOMES AND THINGS THE PATIENTS ARE LOOKING FOR FROM THIS TEST IN MAKING THEIR DECISIONS. AND IN THE PROCESS OF DOING THIS WORK, WE ALSO WORKED WITH CHUCK FELLPS TO SEE WHAT LESSONS WE HAD LEARNED FROM THINKING ABOUT VALUE OF FRAMEWORKS THAT ARE BEING PROMOTED IN ONCOLOGY AND A LOT OF OTHER DISEASES. SO I'LL FIRST TELL YOU THIS IS 1 OF OUR SECOND PAPERS THAT TALKED ABOUT THE ONCOLOGIST AND THE ORGANIZATION FACTORS THAT INFLUENCED THE UPTAKE OF ONCO TYPE AND THIS WAS DONE BY TRACEY LIEU AND HER CREW AT KAISER. OVER ALL OUR SETTING INCLUDED THE 17 KAISER NORTHERN CALIFORNIA PRACTICES, AS WELL AS A POPULATION BASED SAMPLE FROM WASHINGTON STATE BASED FROM THE SEER REGISTRY AND I'LL TALK ABOUT THOSE DATA IN A FEW MIN UTR -S. THE DATA ON THE MULTILEVEL USE COMES FROM THE KAISER 17 PRACTICES AND WE HAD BOTH THE CLAIMS, THE EHR AND THE WE HAD A SUBSET WE DID IN BOTH SITES SAMPLE OF PATIENTS WHO WOULD HAVE BEEN ELIGIBLE FOR THE TEST AND SO WE HAVE A SUBSAMPLE OF PATIENTS LINKED TO THIS DATA. WE ALSO SURVEYED THE ONCOLOGIST TO GET THEIR SENSE OF HOW TO DOES THIS TEST HELP IN THEIR DECISION MAKING. AND WHAT WAS VERY INTERESTING, THIS IS AMONG ALL OF THE PATIENTS, THERE WERE OVER 3000 WHO WERE ELIGIBLE TO HAVE AN ONK O TYPE TEST AND THERE'S CONSIDERABLE VARIABILITY AMONG THE SITES AT KAISER, AVERAGE OF 39% BUT 2 FOLD DIFFERENCE IN THE RANGE FROM THE LOWEST TO HIGHEST. SO THAT WAS CONSIDERABLE VARIABILITY IN A SYSTEM THAT HAS A SINGLE RECOMMENDATION AND NO FINANCIAL INCENTIVES. SO WE'RE INTERESTED IN UNDERSTANDING WHAT MIGHT RELATE, WHAT MIGHT ACCOUNT FOR THIS PATTERN. SO THIS IS--THESE ARE THE RESULTS, THERE ARE PATIENT FACTORS WELL KNOWN, COMMUNITY FACTORS AND I WANT TO DRAW YOUR ATTENTION TO THESE PRACTICE LEVEL AND PHYSICIAN. PATIENTS WHO ARE ELIGIBLE FOR THE TEST WHO WERE SEEN BY A FEMALE ONCOLOGIST AND A HIGHER PROBABILITY OF UNDERGOING ONCA TESTING AS THE CHIEF OF THE SITE AT THEIR INDIVIDUAL TESTERATE INCREASED THEN THE LEVEL, THE ODDS OF A PATIENT HAVING TESTING AT THAT SITE INCREASED AS WELL INDICATING AN INFLUENCE HERE OF THE CHIEF. SO THAT WE FELT, THOSE WERE ALL VERY INTERESTING TYPES OF THINGS. IT'S NOT UNUSUA THAT SORT OF A KEY DECISION LEADER SETS THE TONE FOR PRACTICE STYLE AND THAT THE PRACTICE FOLLOWS THAT KEY DECISION LATER. SO WE DECIDED FROM THIS, THAT THERE WAS AS WE'VE SEEN IN THE LITERATURE, WIDE VALID AND RELIABLE YAITIONZ IN THE USE AND IT WAS INTERESTING HERE BECAUSE REALLY THE PATIENTS DIDN'T HAVE ANY FINANCIAL BARRIERS TO GETTING THE TEST AND THE PROVIDERS HAD NO FINANCIAL INCENTIVES VIS A VIS THE TEST OR THE SUBSEQUENT CHEMO USE. IT WAS SOME INTERESTING YOU KNOW DIFFERENCES BY GENDER AND WHAT I ALSO DIDN'T PRESOANLT IS WHEN WE LEAD IN THE TESTING INCREASE AS THE PHYSICIANINGS PERCEIVE THE TEST, HELP INDEED THEIR ACTUAL CLINICAL DECISION MAKING. AND SO THIS IS QUITE INTERESTING. THE OTHER THING THAT PLAYS INTO THE VARIATION WE DIDN'T MEASURE HERE AND THAT WE'RE ALL KIND OF WAITING IS WE HAVE GOOD DATA ON HIGH AND LOW RISK, THE INTERMEDIATE RISK GROUP IS STILL SUBJECT OF CLINICAL TRIALS AND THERE'S SOME UNCERTAINTY, NO DOUBT THAT COMES INTO PLAY THERE, BUT WE THOUGHT THIS WAS VERY INTERESTING AND INFORMATIVE. REAL WORLD TYPE OF IMPLEMENTATION OF A PERSONALIZED TEST. SO NOW WE WANTED TO LOOK AT COST EFFECTIVENESS AND A DECISION TREE TO SEE WHAT WOULD HAVE THE BETTER VALUE IN DOING ONCO-TESTING VERSUS NOT. SO HERE WE WORKED WITH THE SYS NET MODELS AND WE WANT TO DO PRACTICE AND WE USED SOME OF THE RATES OF ONCO-TYPE TESTING AND USE OF CHEMO CONDITIONAL AND ONCO TESTING AND RESULTS IN KAISER ALONG WITH NATIONAL DATA ON THE UNDERLYING NATURAL HISTORY SURVIVAL PROBABILITY RECURRENCE, THE PUBLISHED ORIGINAL PUBLISHED DATA LOOKING AT THE ONCO-TYPE TEST IT SHOWS A STRONGLY STATISTICALLY SIGNIFICANT ASSOCIATION OF THE RISK CATEGORY WITH THE PROBABILITY OF RECURRENCE BUT AS A TEST PROPERTY THERE ARE LARGE--MOST PEOPLE DON'T RECUR AND THERE ARE A LARGE PEOPLE OF NUMBER WITH HIGH RISK SCORES WHO IN EVERY RECUR AND THERE ARE SOME PEOPLE WITH LOW RISK SCORES WHO DO RECUR, SO THE TEST DOES NOT HAVE PERFECT ACCURACY, SO WE MODEL THE ACTUAL ACCURACY FROM THE ORIGINAL TRIAL DATA. AND THE OTHER THING WE WERE INTERESTED IN THAT WAS TALKED ABOUT BEFORE IS SORT OF THE VALUE OF THE TEST INFORMATION FOR THE PATIENT. WOULD KNOWING YOU HAVE A LOW RISK SCORE SORT OF REASSURE YOU? WOULD YOU HAVE MORE WORRY IF YOU KNEW YOU HAD A HIGH RISK SCORE FOR RECURRENCE? THERE ARE NO PRIMARY DATA THAT WE'RE AWARE THAT HAVE COLLECTED THOSE UTILITIES SO WE ESTIMATED THOSE TO SEE IF THE ACTUAL COST EFFECTIVENESS WOULD BE SENSITIVE TO CONSIDERING A VALUE OF THE TEST OF THE PATIENT BEYOND THE ACTUAL TREATMENT CHOICE. SO, THESE ARE OUR RESULTS. WHAT'S NOTABLE IS WE DON'T EXPECT GETTING AN ONCO-TEST VERSUS NOT DOESN'T HAVE AN EFFECT ON QUALITIES ALTHOUGH YOU'RE TARGETING CHEMO IN THEORY WHO MAY HAVE A HIGHER RISK OF BENEFITING FROM IT BUT THE TEST IN AND OF ITSELF DOES NOT CHANGE YOUR PROBABILITY OF SURVIVING. THE COSTS WE WERE BASING THE ONCO TYPE TEST AND THE OTHER MEDICAL CARE COSTS ON NATIONAL DATA AND IT WAS A FAIRLY--AT THE RATE OF USE IN KAISER WHICH AVERAGED ABOUT 24% IN THIS STUDY PERIOD, IT WAS A FAIRLY EXPENSIVE TOAST PER YEAR OF LIFE SAVED. AND THERE WERE INTERESTING PATTERNS IN THIS THAT THE WOMEN WERE TESTED WHO WERE EITHER YOUNGER, HAD FEWER CO-MORBIDITIES, HAD HIGHER STAGES, AND IT LOOKED LIKE THE TEST MIGHT HAVE BEEN USINGED MORE IN SITUATIONS WHERE IT'S ACTUALLY GOING TO INFLUENCE THE TREATMENT DECISION AND IT WASN'T BEING USED AT ALL WHERE THE TREATMENT DECISION WAS FAIRLY OBVIOUS TO THE CLINICIAN AND/OR THE PATIENT. BUT THERE MAY BE A NUMBER OF THINGS THAT COULD EFFECT THIS PARTICULARLY SINCE THERE ARE A SMALL NUMBER OF QUALITIES, A LITTLE BIT OF CHANGE COULD HAVE A BIG IMPACT. SO IN THE SENSITIVITY ANALYSIS, THIS IS OUR BASE CASE HERE. IF THE PRICE WERE LESS THAN THE CURRENT MEDICARE REIMBURSEMENT RATE THEN THE TEST WOULD BE MORE COST EFFECTIVE IF THE PRICE WENT DOWN. CLEARLY I DON'T THINK THE PRICE OF THE TEST WILL GO UP. ALSO, THE TEST PROPERTIES WERE REALLY, REALLY IMPORTANT. IF THE TESTS WERE PERFECT IN ITS ACCURACY, THEN THIS WOULD BE A MUCH MORE COST EFFECTIVE RESULT AND I THINK IN THE PRIOR ANALYSIS THEY ASSUME IT WAS A PERFECT SENSITIVITY AND SPECIFICITY. THE OTHER THING THAT WAS QUITE INTERESTING IS THAT THE NET IMPACT OF REASSURANCE VERSUS WORRY, MOST PATIENT WHO IS WERE TESTED WILL BE LOW RISK, NOT AS MANY HAVE THE HIGH RISK SCORES AND THE NET BALANCE OF THAT CONSIDERING THAT VALUE, LOWERED THE COST PER YEARS OF QUALITY SAVED. TREATMENT ALSO, THE ADHERENCE MAKES A DIFFERENCE. IF YOU HAVE A LOW RISK SCORE BUT YOU OPT TO GET CHEMO YOU'RE INCURRING COST FOR A LOW LIKELIHOOD OF ANY BENEFIT WHEREAS IF YOU HAVE A HIGH RISK AND YOU DECIDE NOT TO GET CHEMO AND YOU DO RECUR, THEN YOU HAVE--HAVE YOU LOST YEARS OF LIFE YOU COULD HAVE GAINED. SO IF ADHERENCE WERE PERNECT TO WHAT THE TEST SAYS YOU SHOULD DO, THAT WOULD ALSO MAKE THE TEST MORE COST EFFECTIVE. SO WE CONCLUDED THAT IT--IN CURRENT PRACTICE, THE WAY DOCTORS AND PAIRKTS ARE CHOOSING TO USE ONCO-TYPE, IT HAS A HIGH COST EFFECTIVENESS RATIO AND REASONABLE CHAIMPLES TO ANY OF THESE KEY DRIVERS COULD EFFECT THE RESULTS IN GENERALLY TOWARDS MAKING THE TEST MORE COST EFFECTIVE. THE ACCURACY I MENTIONED AND I THOUGHT THE FACT THAT WE DID FIND, BASED ON OUR ESTIMATES OF VALUE OF WORRY VERSUS REASSURANCE, SUGGEST THIS IS IS WAS IMPORTANT AND THAT MEASURING PATIENT REFERENCES AND GETTING IT TYPE OF DATA FROM THE OTHER STUDIES THEY'VE TALKED ABOUT WILL BE IMPORTANT AND UNDERSTANDING THE NET IMPACT AND THE VALUE TO THE CONSUMER. SO I'M GOING TO SWITCH NOW TO THE PATIENT REPORTED OUTCOMES. FOCUS ON THE PATIENT IS IMPORTANT HERE IN THE ONCO-TYPE TEST, IT IS MARKETED TO PATIENTSADS WELL AS PHYSICIANS AND WE WANTED TO SEE IN OUR SURVEY OF PATIENTS HOW THIS WOULD EFFECT SOME OF THE PATIENT OUTCOMES. AND HERE WE'RE USING SURVEY DATA FROM BOTH KAISER AND THE SEER WASHINGTON STATE DATA AND FIRST OF ALL WE DO WE WANT TO KNOW WOMEN ARE TESTED, THEY'RE DONE, IT'S NOT DISCUSSED IT'S JUST ORDERED AND IF PEOPLE KNOW THEY'VE RECEIVED THE TEST, DOES THAT KNOWLEDGE AND HAVING THAT INFORMATION IMPROVE THEIR CONFIDENCE IN THEIR DECISIONS? AS WELL AS PERCEPTIONS OF THE RISK OF RECURRENCE? SO THOSE WERE DATA THAT WE PULLED. THIS ANALYSIS WAS LED BY DR. RAMSEY'S TEAM AND WE HAD A TOTAL OF 908 WOMEN AND IT WAS A SURVEY--EVERYBODY WAS ELIGIBLE FOR ONCO-TYPE TESTING AND THE RESPONSE RATES WERE SIMILAR AT EACH SITE. SO SOME RESULTS WERE NOT UNEXPECTED, THE OLDER YOU WERE, THE LESS LIKELY YOU WERE TO KNOW YOU HAD BEEN TESTED. OVERALL WE HAD ONLY 71% OF PEOPLE KNEW THEY HAD BEEN TESTED. EDUCATION WAS AS EDUCATION INCREASED ALSO YOU'RE MORE LIKELY TO KNOW YOU HAD BEEN--SAY YOU KNEW YOU WERE TESTED. THERE IS A REGIONAL SETTING AND WE ARE LOOKING AT THOSE AND THIS IS ADJUSTED FOR PATIENT CHARACTER CHARACTERISTICS. SO THESE ARE NOT SURPRISING BUT THEY DO SHOW THAT THE KNOWLEDGE OF HAVING THE TEST IS LOWER THAN WOULD BE OPTIMAL. NOW, DOES THE TEST, IF YOU KNOW YOU'VE HAD THE TEST, ARE YOU CONFIDENT IN YOUR CHEMO THERAPY DECISIONS. SO THESE ARE THE ODDS OF BEING--FOR WOMEN WHO ARE CONFIDENT VERSUS NOT CONFIDENT. OR COMPLETELY CONFIDENT VERSUS SOMEWHAT, LITTLE OR NOT AT ALL CONFIDENT BASED ON SELF-REPORTED TEST SCORES HERE AND WHAT WE SEE IS THERE'S NOT REALLY A SIGNIFICANT INFLUENCE AMONG WOMEN WHO PERCEIVED THEMSELVES WHO THINK THEIR RESULT WAS LOW OR HIGH RISK, BUT THAT THE INTERMEDIATE RISK GROUP IS WHERE YOU GET--THEY'RE LESS CONFIDENT AND THAT MAKES SENSE CLINICALLY BECAUSE AS I MENTIONED BEFORE, THAT'S THE GROUP FOR WHOM CLINICIANS AND PATIENTS, THE DATA ARE NOT IN YET. WE DON'T HAVE THE EVIDENCE. SO THAT MADE SENSE TO US. BUT WE DID--WE HAD THOUGHT THAT THE GROUP WHO WAS LOW RISK AND THE GROUP THAT WAS HIGH RISK, THEY WOULD HAVE MORE INFORMATION, HAVE MADE A MORE PERSONALIZED DECISION AND WE HAD EXPECTED THAT THEY WOULD BE MORE CONFIDENT ABOUT THEIR DECISION AND THE FACT THAT THE TESTING DID NOT HAVE THIS INTENDED RESULT IS WORRISOME. THE OTHER THING WE WANT TO SEE IS HOW DID THESE WOMEN ESTIMATE THEIR RISK OF RECURRENCE. SO WE LOOKED AT THEIR SELF-REPORTED TEST RESULT ASK WE HAD THE ACTUAL TEST RESULTS IN 1 SETTING BUT IN THE SEER AREA DATA THERE'S NO WAY TO KNOW THE ACTUAL TEST RESULT. SO, BASED ON THEIR SELF-REPORT OF WHAT THEIR TEST RESULTS WERE COMPARED TO WOMEN WHO HAD NOT BEEN TESTED, WOMEN WHO HADN'T BEEN TESTED ABOUT A THIRD OF THEM THOUGHT THAT THEY WERE GOING TO RECUR. NOW IN THIS GROUP THAT ARE ELIGIBLE, THE ACTUAL PROPORTION THAT ARE GOING TO ACTUALLY RECUR IS UNDER 10%. SO THEY'RE OVERESTIMATING THEIR RISK OF RECURRENCE. AND THOSE THAT HAD A HIGH RISK SCORE HAVE A SIGNIFICANTLY HIGHER PERCEPTION OF THE RISK OF RECURRENCE COMPARED TO THOSE WHO WEREN'T TESTED AND THE LOW RISK SCORE THEY HAVE A 5-POINT DECREASE. SO THE TEST IS EFFECTING FOLK'S PERCEPTIONS OF THE TEST RESULT AND THEIR ACTUAL RISK OVERALL EVERYBODY IS EVERYESTIMATING THE RISK OF RECURRENCE. WHAT'S DIFFICULT ABOUT THIS IS AMONG THOSE THAT RECUR, THEY EITHER DO OR DON'T RECUR. IT'S STILL WE DON'T KNOW REALLY WELL IN THIS EARLY STAGE GROUP WHO WILL ACTUALLY RECUR. SO OUR CONCLUSIONS FROM THIS PORTION OF OUR DATA WERE THAT WE HAVE ABOUT A THIRD OF PATIENT WHO IS DON'T EVEN KNOW IF THEY'RE BEING TESTED AND THAT THE TESTING IS NOT DOING WHAT WE ANTICIPATED IN IMPROVING CONFIDENCE IN THE DECISION AND--BUT THE TESTING DOES SEEM TO IN THE RIGHT DREKDZS EFFECT PER--DIRECTIONS EFFECT RISK. SO OVER ALL WE THOUGHT THEY APPLY A NEED FOR BETTER COMMUNICATION ABOUT GETTING THE TESTING, WHAT THE TESTING WILL DO, WHAT ARE THE IMPLICATIONS FOR TREATMENT AND THAT PATIENTS NEED TO HAVE A BETTER SENSE OF THE ACCURACY OF THE TEST AND THEIR ACTUAL RISK TO MAKE INFORMED DECISIONS. SO WE HAD A SUPPLEMENT THAT WE LOOKED AT TO LOOK AT THE COMMUNICATION NEEDS HERE. AND SUFFERING FROM CONFUSING CANCER GUIDELINE SYMPTOMS. THIS IS COMMON ACROSS MANY PORTIONS OF THE CANCER CONTROL SPECTRUM AND WE LOOKED AT THIS SUPPLEMENT WE INTERVIEWED INTERROGATIONS FROM A MIDATLANTIC COMMUNITY PRACTICE NETWORK AND ABOUT A HUNDRED OF THEM TO WHY DO THEY NOT TALK ABOUT, COMMUNICATE ABOUT GENOMIC TESTING AND IN SOME INSTANCES IT'S THINGS WHERE THEY DON'T FEEL IT WILL BE HELPFUL BECAUSE THE PATIENT IS TOO SICK, THEY THINK THEY ALREADY KNOW THE PREFERENCES INTERESTINGLY, THE PRACTICE NORMS WHICH WAS SIGNIFICANT IN EXPLAINING PRACTICE VARIATION, THEY DID NOT RATE AS A VERY IMPORTANT ISSUE. THE ISSUE ALSO WAS THEY WEREN'T THAT CONFIDENT IN USING THE TEST AND COMMUNICATING ABOUT IT. SO BASED ON THIS, WE ALSO ASKED WELL WHAT KIND OF A TOOL WOULD YOU LIKE TO BE ABLE TO USE IN YOUR CLINICAL ENCOUNTERS TO HELP YOU COMMUNICATE ABOUT THIS AND HAVING SOMETHING THEY WERE INTERESTED IN ALL OF THESE THINGS THAT WOULD BE HELPFUL WEB-BASED TOOLS FOR THE PHYSICIAN OR PHYSICIAN TOOL, WERE IMPORTANT. THEY ALSO WANTED TO THE PATIENT TO HAVE MATERIALS TO REVIEW BUT TO REVIEW THE MATERIALS WITH THE PHYSICIAN SO THAT SOMETHING THAT COULD BE DONE WITHIN THE CLINICAL ENCOUNTER TO REVIEW, WOULD BE HELPFUL AND THERE AREN'T--THERE REALLY AREN'T ANY TOOLS OF THIS TYPE. THE CLINICIANS HAD BEEN USING FOR THEM TO MAKE THEIR DECISION AND RECOMMENDATION, THINGS LIKE PREDICT OR ADJUVANT ONLINE. ADJUVANT IS DOWN--I HOPE TO RETURN WITH ONCO TYPE I HOPE IT WILL RETURN AND PREDICT DOESN'T HAVE IT YET BUT AT THE MOMENT CLINICIANS DON'T HAVE A GREAT TOOL TO USE TO HELP MAKE THESE DECISIONS NOR TO SUPPORT THEIR COMMUNICATION ABOUT THE TESTING BUT OUR RESULTS SUGGEST THAT THAT IS AN IMPORTANT CLINICAL NEED. SO IN TERMS OF TALKING ABOUT THE VALUE FRAMEWORK, SO THERE WERE A NUMBER OF ISSUES THAT CAME UP WHILE WE'VE ALL BEEN FUNDED THESE LAST FEW YEARS HAVE BEEN QUITE A FEW GUIDELINES COMING OUT. CERTAINLY CANCER CARE, ASSESSING THE VALUE OF CANCER CARE, VALUE OF PERSONALIZED CARE. AND SO, WE TRY TO PUT SOME OF OUR THOUGHTS AND LESSONS LEARNED TOGETHER IN LOOKING AT THAT. AND MANY OF THE THINGS ARE SORT OF OBVIOUS AND WE STUCK TO THE GOLD STANDARD OF COST EFFECTIVENESS FRAMEWORK BEING USEFUL AS A STANDARD WAY TO ASSESS VALUE. AND 1 OF THE KEY TECHNOLOGY THINGS IS WE WANT TO EMPHASIZE IS THAT THE SPONSOR DECLARE FINANCIAL STAKES. SOME OF THE VALUE FRAMEWORK HAD CONFLICTS, SOME OF THIS WAS FUNDED BI GENOMIC HEALTH SO IT'S IMPORTANT TO DECLARE ANY POTENTIAL OR PERCEIVED CONFLICTS OF INTEREST AND SO, THESE ARE--YOU KNOW NOT SURPRISING, I THINK THEY ECHO THE GUIDELINES AND THE NEW RECOMMENDATIONS ON THE USE OF COST EFFECTIVENESS AND HEALTH AND MEDICINE THAT PETER HELPED REISSUE. AND PUT IT IN THIS CURRENT CONTEXT. SO OVERALL OUR LESSONINGS LEARNED FROM 4 YEARS OF WORK IS THAT NOT SURPRISINGLY, ADOPTION OF THESE PERSONALIZED GENOMIC TESTS DOESN'T FOLLOW IN A STRAIGHT LINE. THERE IS AN ASCO GUIDELINE TO DO IT BUT IT IS INFLUENCED BY MANY TYPES OF FACTORS, NOT ALL OF WHICH WE UNDERSTAND AND IT MAY NOT IN FACT BE AS COST EFFECTIVE AS PREDICTED UNDER IDEALIZED CIRCUMSTANCES BECAUSE THERE WERE A LOT OF FACTORS THAT GO INTO THE ACTUAL USE AND I THINK BEING ABLE TO TRANSLATE IT IN ITS OUTCOMES IMPORTANT OF PATIENTS THAT WE NEED ADDITIONAL WORK AND COMMUNICATION AND WE NEED TOOLS FOR BOTH THE PATIENT AND THE CLINICIAN. THANK YOU. [ APPLAUSE ] >> THANK YOU DOCTOR, AND VERY INTERESTING AND DURING THE DISCUSSION I THINK IT WILL BE USEFUL TO PROBE ALITTLE BIT MORE HOW TO GET AROUND THE ISSUE OF WHO IS SUPPORTING THE TRIAL. OUR NEXT SET OF SPEAKERS ARE DAVID KENT, PETER NEUMANN AND DAVID KIM FROM THE EAST COAST FROM TUOF TUFTs UNIVERSITY. >> THANK YOU JOHN. IT'S REALLY A PLEASURE TO HAVE THE OPPORTUNITY TO PRESENT ON BEHALF OF OUR WHOLE TEAM. OUR PROJECT IS CALLED PERSONALIZED RISK INFORMATION AND COST EFFECTIVE STUDIES OR PRICES AND THIS IS A COLLABORATIVE PROJECT BETWEEN MY GROUP WHICH SPECIALIZES IN RISK MODELING OR PREDICTIVE MODELING. SO WE'RE KIND OF THE PERSONALIZED RISK INFORMATION PATH OF THE ACRONYM AND PETER'S GROUP THAT HAS DEEP EXPERTISE IN COST EFFECTIVENESS OF THIS ANALYSIS. SO MY JOB TODAY IS REALLY 2 FOLD. I WILL FIRST JUST BRIEFLY PRESENT THE CONCEPTUAL OVERVIEW OF THE IMPORTANCE OF INDIVIDUALIZING RISK, BOTH WHEN ANALYZING CLINICAL TRIALS FOR THE BENEFITS OF PHYSICAL THERAPY AND APPLYING IT TO PATIENTS. AND THIS EXTENDS ALSO, OR SHOULD EXTEND TO COST EFFECTIVENESS ANALYSIS, IF DECISIONS ARE BEING MADE 1 PATIENT AT A TIME WHICH FOR ME AS A DOCTOR IS AND FOR PATIENTS ALSO IS THE WAY DECISIONS, MANY MEDICAL DECISIONS TEND TO BE MADE. SO PEOPLE ARE INTERESTED IN AVERAGE CONDITIONAL TREATMENT EFFECTS, THAT IS I'M INTERESTED IN TREATMENT EFFECTS, CONDITIONAL ON ALL THE CHARACTER IEOF THETICS OF THE PATIENT WHO'S IN FRONT OF ME AT THE TIME , THE CHARACTERISTICS OF ANY GIVEN PATIENT THAT I'M TAKING CARE OF. SO WE'VE BEEN AWRPGING FOR MANY YEARS--ARGUING FOR MANY YEARS THAT FOR THIS REASON RISK BASED SUBGROUP ANALYSIS SHOULD BE ROUTINE BECAUSE YOU CAN NEVER REALLY TELL FROM THE OVERALL RESULTS HOW MUCH BENEFIT PATIENTS IN DIFFERENT RISK CATEGORIES ARE GETTING FROM A TRIAL AND RISK VARIES TREMENDOUSLY ACROSS ALL PATIENTS IN THE TRIAL AND THAT'S JUST OBSCURED WHEN OVERALL RESULTS ARE PRESENTED OR WHEN 1 VARIABLE AT A TIME SUBGROUP ANALYSIS OR DONE. SO THAT'S WHY WE'VE BEEN ARGUING THAT RISK SHOULD BE PRIVILEGED OVER THESE 1 VARIABLE AT A TIME ANALYSIS SUCH AS OLDER PATIENTS VERSUS YOUNGER PATIENTS, THOSE WITH AND WITHOUT DIABETES AND SO FORTH. AND THE REASON BASICALLY IS THAT UNLIKE THESE OTHER VARIABLES, RISK IS SIMPLY, A MATHEMATICAL DETERMINANT OF THE TREATMENT EFFECT AND THAT COULD BE SEEN BY EXAMINING ANY 1 OF THE USUAL MEASURES, EITHER ABSOLUTE RISK REDUCTION WHICH IS CLINICALLY THE MOST IMPORTANT MEASURE OF TREATMENT EFFECT, THE RELATIVE RISK REDUCTION, ODDS RATIO, THEY ALL ARE BASED DIFFERENT COMBINATIONS OF THE EXPERIMENTAL EVENT RATE AND CONTROL EVENT RATE. SO IF THE CONTROL EVENT RATE IS CHANGING, ACROSS THE TRIAL POPULATION THEN THE TREATMENT EFFECT HAS TO CHANGE. THIS IS SIMPLE ALGEBRA. AND WHEN YOU LOOK AT THE CONTROL EVENT RATE DOES CHANGE BASED ON KNOWN CHARACTERISTICKINGS OF THE PATIENTS AT BASE LINE, XT SO WE RECEIVED SEVERAL YEARS AGO, SUPPORT FROM PC ORI TO LOOK ACROSS CLINICAL TRIALS, WE USE TRIALS FROM BIOLINK, AND WE MODELED RISK AND WE LEARNED A LOT ABOUT HOW THESE RISK DISTRIBUTIONS WORK AND IN GENERAL, THERE'S JUST A LOT OF RISK HETEROGENEOUS O GENERATED AITY IN THESE TRIALS WHERE THE OUTCOME RISK AND THE LOWEST RISK QUARTER OF PATIENTS WILL BE 1 FOURTH/FOURTH, 1/10th, 120th , SO THE HIGHES RISK QUAR TILE WILL OFTEN HAVE 10-20 FOLD. THE OUTQUM RISK IS THE LOWEST RISK TORTILES AND ANOTHER WAY TO SAY. AND WHEN YOU HAVE PATIENTS THAT VARY SO MUCH IN THE RISK, THE RISK BENEFIT TRADE OFFS ARE NEVER THE SAME IN THE POPULATION. OKAY? SO, AMONG THE 18 TRIALS THAT HAD POSITIVE EFFECTS, THIS IS THE TREATMENT EFFECT. THIS IS THE TREATMENT EFFECT WITHLET Y-AXIS IS THE RELATIVE RISK OR HAZARD RATIO ACCESS AND WE DIDN'T SEE ANY PATTERN THAT CUT ACROSS ALL THE DIFFERENT TYPES OF INTERVENTIONS, THESE ARE MOSTLY CARDIOVASCULAR INTERVENTIONS BUT WHEN WE LOOKED AT ABSOLUTE RISK REDUCTIONS, WE SAW THAT GENERALLY, THE TREATMENT EFFECT GOT BIGGER AND HIGHER RISK PATIENTS. THAT'S NOT TERRIBLY SURPRISING IF YOU LOOK AT THE MATH. SO AND ALSO IF YOU USE YOUR CLINICAL INTUITION, AND IN 3 OF THESE CASES, THE VARIATION TREATMENT EFFECTS WAS VERY LARGE AND THE REST OF THE INITIAL TRIAL REPORTS WE'VE PUBLISHED CLINICAL MANUSCRIPTS ON THESE CASES AND I WILL SHOW YOU 1 OF THEM BECAUSE IT MAKES AN IMPORTANT POINT ABOUT THE SCALE DEPENDENCE OF HETEROGENEITY OF EFFECTS, THIS WAS THE DIABETES PREVENTION TRIAL, THEY ENROLLED 3000 PREDIABETIC PATIENTS AND RANDOMIZED THEM TO 3 GROUPS, EITHER USUAL CARE, MET FORMIN OR LIFESTYLE MODIFICATION AND THE OUTCOME, THEY LOOKED AT WAS THE DEVELOPMENT OF DIABETES AND HERE ARE THE RISK STRATIFIED RESULTS WE PUBLISHED ON THE LEFT-HAND SIDE, THIS IS LIFESTYLE MODIFICATION COMPARED TO USUAL CARE. IN LOW RISK PATIENTS TO HIGH RISK PATIENTS AND YOU CAN SEE, THIS IS SOMETHING THE STATISTICIANS WOULD CALL A HOMOGENIUS EFFECT BECAUSEOT HAZARD RATIO SCALE, THERE'S NO--EVERYBODY HAS ABOUT A 50% RELATIVE RISK REDUCTION. THIS GRAPH HERE IS METFORMIN, COMPARED TO USUAL CARE AND ON THE HAZARD RATIO SCALE, WE HAVE A HETEROGENEOUS ROW GENIUS TREATMENT FACT BECAUSE THE PATIENTS GET NO BENEFIT WHEREAS THE HIGH RISK PATIENTS GET ABOUT A 50% REDUCTION. THIS WAS HIGHLY STATISTICALLY SIGNIFICANT. BUT THE CLINICALLY IMPORTANT SCALE IS NOT THE RELATIVE RISK REDUCTION. IT'S THE ABSOLUTE RISK REDUCTION AND YOU HAVE TO REMEMBER, SOME OF THESE PATIENTS HAVE A RISK OF 5% OR LESS AND SOME OF THESE PATIENTS HAVE A RISK OF 50% OR MORE TO DEVELOP DIABETES IN 3 YEARS AND THEY LOOK LIKE DIFFERENT PATIENTS IN TERMS OF THE HEMODPLOABIN A-1-C THERE, BMI AND SO FORTH. SO WHEN WE LOOK AT THE CLINICALLY IMPORTANT SCALE OF THE ABSOLUTE RISK REDUCTION, WE SEE VERY SIMILAR PATTERNS BETWEEN LIFESTYLE MODIFICATION AND MET FORMIN WHERE IT'S REALLY THE HIGH RISK PATIENTS THAT GET THE MOST BENEFIT, EVEN THOUGH WE'RE THOALD THAT THE TREATMENT EFFECTS HERE ARE HOMOGENIUS AND THE TREATMENT EFFECTS HERE ARE HETEROGENEOUS ROW GENIUS. OKAY? SO WE DID A COST EFFECTIVENESS ANALYSIS WAS DONE ON THE NATIONAL LUNG SCREENING TRIAL. THIS WAS A LARGE TRIAL, WITH 50,000 PATIENTS BETWEEN THE AGES OF 55 AND 74, AND THE DATA IS PUBLICLY AVAILABLE WHICH IS WHY WE USED IT. THEY WERE RANDOMIZED TO LOW DOSE, CT SCAN OR CHEST RADIOLOGYRAPHY. THE MAIN OUTCOME MEASURE WAS LUNG CANCER, DEATH. THE OVERALL RESULTS OF THE STUDY, THERE WAS A 20% RELATIVE RISK REDUCTION. STEPHANIE [INDISCERNIBLE] AND A TEAM FROM THE NCI HERE ACTUALLY DID A RISK STRATIFIED ANALYSIS OF THIS TRIAL AND WHAT THEY FOUND WAS HOMOGENIUS TREATMENT EFFECT ON THE HAZARD RATIO SCALE. BUT ON THE--ON THE ABSOLUTE SCALE, IT WAS HETEROGENEOUS ROW GENIUS AND I WILL SHOW YOU THAT SO THIS IS MEANT TO REFLECT A LOW RISK PATIENT, WHO'S FEMALE AND HAD A RELATIVELY MINIMAL TOBACCO EXPOSURE, NO HISTORY OF LUNG CANCER, NO LUNG DISEASE AND SO FORTH AND THIS IS A HIGH RISK PATIENT WHO'S OLDER AND NOW AND HAS A VERY INTENSIVE SMOKING HISTORY, EMPHYSEMA AND SO FORTH. WHEN YOU LOOK ACROSS THOSE RISK STRATA, AGAIN 20% RELATIVE RISK REDUCTION FOR ALL PATIENTS BUT THESE PATIENTS GOT VERY LITTLE BENEFIT IN TERMS OF REDUCED RISK OF LUNG CANCER DEATH. THESE PATIENTS GOT A WHOLE LOT OF BENEFIT IN TERMS OF REDUCED CANCER DEATHS SO THIS IS ABOUT 12 LIVES SAVED FOR EVERY 5000 PATIENTS. THEY SAID THAT YOU CAN TARGET PATIENTS, THESE 3 QUAR TILES AND GET 88% TO THE BENEFIT BUT WE WANTED TO SEE WHAT THE COST EFFECTIVENESS OR THE NET MONETARY BENEFIT OF TARGETING VERSUS POPULATION WIDE TREATMENT WAS. TO DO THAT WE DEVELOPED A MULTISTATE REGRESSION MODEL. WHICH WAS BOTH A RISK PREDECLARATION MODEL, USED IT AS A SIMULATION MODEL AND THE FOLLOWING 8 RISK FACTORS AND WHAT THIS MODEL IS, IT'S A REGRESSION MODEL THAT JOINTLY MODELS THE TRANSITION BETWEEN THESE 4 DIFFERENT STATES FROM HEALTHY SMOKER TO LUNG CANCER DIAGNOSE, TO LUNG CANCER DEATH AND ALSO TO NONLUNG CANCER DEATH, SO, IT INCORPORATES ALL THE COMPETING AND SEMICOMPETING RISKS. AND THESE ARE THE RESULTS, SO IT COMPUTES 4 DIFFERENT REGRESSION EQUATIONS. AND WE CAN SEE THE CALCULATION WAS AND THE CALIBRATION WAS EXCELLENT. AND OUR RESULTS WERE VERY SIMILAR TO WHAT [INDISCERNIBLE] FOUND NOT SURPRISINGLY IN TERMS OF MORTALITY PREVENTED IN THE FIRST 7 YEARS. WE HAD A RATIO OF THESE EXTREME QUANTILES OF RISK OF ABOUT 11. SO FOR EVERY 2000 SCANS, YOU DO HERE, YOU ARE PREVENTING 11 LIVES OVER 5 YEARS WHERE HERE YOU'RE ONLY PREVENTING 1 LIFE, 1 DEATH, RATHER, OVER 5 YEARS. WHEREAS HOWEVER WHEN YOU FOLD IN--THESE PATIENTS ARE DIFFERENT IN THAT THEY'RE OLDER AND SICKER AND HAVE MORE LUNG DISEASE AND MORE TOBACCO EXPOSURE AND SO FORTH, AND WHEN YOU EXTRAPOLATE THAT TO A LIFETIME HORIZON, THE DIFFERENCE ATTENUATES QUITE CONSIDERABLY AND YOU'RE ONLY GETTING ABOUT A 4 OR 5 FOLD DIFFERENCE BETWEEN THE LOWEST VENTILE OF RISK AND THE HIGHEST VENTILE OF RISK. IT'S TRUE TOO THAT THE UTILITY, OF--THEY SPEND MORE TIME ALSO IN THE CANCER STATE IF YOU ARE HIGH RISK AND WHEN YOU FOLD THE UTILITIES IF, IT ATTENUATES THE DIFFERENCE EVEN MORE AGAIN AND BECOMES FLATTER AND NOW, IT'S DOWN TO ONLY 2 OR THROW FOLD RATIO BETWEEN THE EXTREME QUANTILES OF RISK AND FINALLY IF YOU LOOK AT COSTS, YOU WILL SEE THAT THE HIGH RISK PATIENT IT IS ARE ACTUALLY MORE COSTLY TO SCREEN BECAUSE THEY INCUR MORE PROCEDURES AND WHEN YOU FOLD IT ALTOGETHER, YOU GET AN ICER THAT'S RELATIVELY FLAT. SO EVEN THOUGH THE BENEFIT IN TERMS OF MORTALITY SEEMS TO BE MUCH, MUCH, HIGHER IN THE HIGH RISK PATIENTS WHEN YOU FOLD THESE OTHER DIMENSIONS IN AND YOU LOOK AT THE ICER, FOR EVERYBODY IT FALLS BETWEEN THE $50,000 THRESHOLD AND THE HUNDRED THOUSAND DOLLARS THRESHOLD. SO IN CONCLUSION, TARGETING LOW DOSE CT SCREENING TO THOSE WITH THE HIGHEST RISK OF LUNG CANCER DEATHS SUBSTANTIALLY IMPROVES EFFICIENCY IN TERMS OF LIVES SAVED PER PATIENT SCREENED OVER A 5-7 YEAR TIME HORIZON. HOWEVER, PATIENTS WITH HIGHER LUNG CANCER RISK, MORTALITY RISK ARE NOT SUBSTANTIALLY MORE COST EFFECTIVE TO SCREEN BECAUSE THEY'RE LONGER TERM SURVIVAL IS MORE LIMITED, THEIR QUALITY OF LIFE IS LOWER, SCREENING IS COSTLY IN THESE PATIENTS AND WE'RE REALLY USING THIS AS AN EXAMPLE OF MORE KIND OF GENERALIZABLE FINDINGS OF HOW RISK INFORMATION NEEDS TO BE USED AND COST EFFECTIVENESS STUDIES. SO SOME OF THESE LESSONS LEARNED ARE TO ESTIMATE THESE RISK BASED COST EFFECT INESS AND WE REALLY HAVE TO MODEL ALL THE LIFE EXPECTANCY FUNCTION UTILITIES AND COSTS, NEED TO BE CONDITIONED ON ALL THE SAME VARIABLES, YOU NEED TO HAVE THE SAME LEVEL OF GRANULARITY AND ALL THAT INFORMATION AND OFTEN THAT'S MISSING. WE THINK THAT MULTISTATE REGRESSION MODELS LIKE THE KIND WE USED HERE USING DETAILED IMPERICAL DATA CAN PROVIDE A USEFUL WAY TO JOINTLY MODEL ALL THE TRANSITIONS AT ESSENTIALLY THE SAME LEVEL OF GRANULARITY AND I'D LIKE TO TURN THE MICROPHONE OVER TO MY COLLEAGUES. THANK YOU. >> IN THE INTEREST OF TIME--OOPS, LET ME JUST INTRODUCE DAVID KIM AND SAY, AIM 3 OF THE PROJECT WAS DESIGNED TO PUSH THESE FINDINGS ON MISSED STRATIFICATION INTO POLICY IMPLICATIONS AND TO UNDERSTAND POLICY IMPLICATIONS AND TO SIMULATE INCENTIVE BASED POLICIES AND LOOK AT WHAT DIFFERENCE IT WOULD MAKE, SO WITH THAT, DAVID, LET ME JUST CALL YOU UP AND-- >> ALL RIGHT, THANK, PETER AND I'M VERY HONORED TO BE HERE. I REALIZE THERE ARE 5 SPEAKERS NAMED DAVID AND I'M THE LAST 1. SO THANKS FOR NIH TO INVITE ALL HEALTH ECONOMISTS NAMED DAVID. SO I WILL START WITH MORE LIKE OVERVIEW OF A CONCEPTUAL FRAMEWORK. SO, AS PETER BREFLY MENTIONED AND WHAT WE'RE TRYING TO DO HERE, THAT MOVE FROM THE VALENTINEDUATION, SPECIFIC TO THE INTERVENTION LEVEL TO THE EVALUATING MORE POLICY LEVEL. SO IF IMPLEMENTING THIS DIFFERENT CO-SHARINGS OR PAY FOR PERFORMANCE PROGRAM, THE ECONOMY RETURNS AND THE CONSEQUENCES OF IMPLEMENTING THOSE PROGRAMS. SO, START WITH THIS CONCEPTUAL FRAMEWORK, SO THERE'S MT2-TABLE SO YOU CAN THINK ABOUT THIS IS REALLY POLICY SPACE. IT CAN COME UP WITH THE DIFFERENT POLICY OPTIONS AND THEN REPRESENTING POLY-PERSONALIZED INFORMATION OR OR WE DON'T HAVE PERSONALIZED INFORMATION. AND TO ADD THE LAYERS AT THE COLUMN, SAYS WHETHER WE HAVE A POLICY BASED INCENTIVE PROGRAM OR NOT. SO IF YOU FOCUSED ON THOSE LOWER RIGHT CORNERS, THEN BASICALLY 1 SIDE FITS ALL, WE WILL COVER LOW DOSE CT FOR LUNG CANCER SCREENING FOR EVERYONE BECAUSE WE DO NOT HAVE A PERSONALIZED INFORMATION, WE ARE NOT GOING TO IMPLEMENT AN INCENTIVIZED PROGRAM. AND IF YOU MOVE TO THE LEFT, THEN, YOU KNOW WE CAN IMPLEMENT CERTAIN INCENTIVE PROGRAM WITHOUT THE PERSONALIZED INFORMATION AND PROMINENT EXAMPLE IS PAY FOR PERFORMANCE TO INCENTIVIZE TRADITIONS TO IMPROVE QUALITY OF CARE, ALSO THERE'S VALUE BASED INSURANCE DESIGN TO INCENTIVIZE PATIENTS TO UTILIZE LIER COST GENETICS OVER THE BRAND NAME DRUG. BUT WHAT--WE ERCH'S DOING HERE IS THAT WE ARE DOING A LOT OF GREAT WORK TO GENERATE ALL THESE PERSONALIZED INFORMATIONS DURING THE SUBGROUP ANALYSIS IN THE CLINICAL TRIALS EVEN IN THE COST EFFECTIVE ANALYSIS DUE TO GENOMIC TESTING, WE JUST HEARD THAT WE DEVELOP RISK PREDICTION MODEL TO GENERATE ALL THIS GREAT REACH PERSONALIZED INFORMATION. SO IN THAT CASE, WE DO HAVE A PERSONALIZED INFORMATION AVAILABLE BUT UNLESS WE TRANSFORM THIS INFORMATION INTO SOMETHING ACTIONABLE SUCH AS A POLICY INCENTIVE PROGRAM, THEN THERE'S A NO CHANGE IN POLICY LEVEL. SO OUR FOCUS IS REALLY HOW WE CAN UTILIZE THOSE RISK PERSONALIZED RISK INFORMATION AND TRANSLATE INTO POLYLEVEL. SO, FOR EXAMPLE, THE EXAMPLE IS REALLY WE CAN IMPLEMENT LIKE RECENT TARGETED PERFORMANCE OR RISK TARGETED VALUE BASED INSURANCE DESIGN. SO I WILL GOAL OF AIM 3 IS REALLY EVALUATING THE POTENTIAL CONSEQUENCES OF IMPLEMENTING THIS RISK BASED APPROACH IN THE HYPOTHETICAL SITUATION. IN TERMS OF EXAMPLE, DAVID CAN BRIEFLY TOUCH ON IT, BUT WE PICK--WE USE THIS LOW DOSE CT FOR THE LUNG CANCER SCREENING AND THIS IS PARTICULARLY EXAMPLE BESIDES WE HAD DATA GENERATED RESEARCH. THE ACA, AFFORDABLE CARE ACT MANDATES EVERY HEALTH PLANOT MARKETPLACE TO COVER LUNG CANCER SCREENING WITHOUT COST SHARING BECAUSE IT'S CONSIDERED AS A PART OF ESSENTIAL HEALTH BENEFIT AND RATIONAL THEY CITED ACTUALLY, THEY CITED THIS PREVENTIVE FIRST RECOMMENDATIONS AND WHAT THOSE RECOMMENDATIONS SAYS, SAYS WE LIKE TO TARGET THE HIGH RISK [INDISCERNIBLE] WITH A SMOKING HISTORY. THE WAY THEY DEFINE THIS HIGH RISK [INDISCERNIBLE] IS REALLY OLDER POPULATION WITH HISTORY OF HEAVY SMOKING. SO WHAT WE LIKE TO EXTEND FROM THIS HIGH RISK [INDISCERNIBLE] IS THAT EVEN AMONG THESE--AMONG THOSE CONSIDERED AS A HIGH RISK, IF YOU CAN UTILIZE THIS PERSONALIZED INFORMATION, WE CAN TEASE OUT WHO'S AT THE HIGHER RISK AND LOWER RISK SO WE CAN ACTUALLY PRIORITIZE WHO SHOULD BE TARGETED USING THOSE POLICY BASED INCENTIVE PROGRAMS. SO THAT'S THE GENERAL IDEA. AND DUE TO TIME CONSTRAINT I'M JUST ENDING WITH THE EVALUATION FRAME OF HOW WE ARE GOING TO EVALUATE THIS POTENTIAL CONSEQUENCES OF A DIFFERENT POLICY LABEL INTERVENTIONS RATHER THAN THE MEDICAL INTERVENTION LEVEL. SO [INDISCERNIBLE] AND I PROPOSED A NEW PRAIM WORK IN A RECENT PAPER AND OUR GENERAL ASSUMPTION IS THAT WITH THE HETEROGENEITY TREATMENT EFFECT, DIFFERENT FUNCTIONS OF TECHNOLOGY MAY HAPPEN IN PRACTICE. BASICALLY WHAT IT MEANS IS THAT EVIDENCE SHOWS THAT IA CERTAIN INTERVENTION WORKS BETTER FOR THE CERTAIN POPULATION OR THE SUBGROUPS FOR EXAMPLE, MALE OR FEMALE, OR THE OLDER POPULATION OR VERSUS YOUNGER POPULATION, THEN THOSE SUBGROUP WITH THE HIGHER ANTICIPATED BENEFIT, THEY'RE MORE LIKELY TO ADOPT THOSE TECHNOLOGY. SO IF THERE IS A HETEROGENEITY TREATMENT EFFECT, IN REALITY, WE EXPECT A DIFFERENTIAL ADOPTION OF TECHNOLOGY. AND IF YOU THINK ABOUT THIS ADOPTION, ANOTHER INTERESTING FACTORS TO INFLUENCE ADOPTION IS REALLY POLICY ITSELF, MAY INFLUENCE THE ADOPTION BEHAVIOR, I THINK PROMINENT EXAMPLE IS COST SHARING, SO HIGH COST SHARING REALLY DISCOURAGE PATIENTS TO UTILIZE--LESS UTILIZATION SO CAN YOU MANIPULATE THOSE ADOPTION BEHAVIOR THROUGH CERTAIN COVERAGE POLICY. SO IN THAT PAPER, WE PROPOSED A TUMOR SPECTRUM METRICS FROM POLICY EVALUATION PERSPECTIVE AND WHAT IT REALLY PROVIDED IS THAT--IT PROVIDED THIS WAY TO COST EFFECTIVE RATIO AFTER ADJUST THANKSGIVING SUBGROUP LEVEL ADOPTION RATES. UTILIZING THE RATE, WE WADED THROUGH THE HETEROGENEOUS ROW GENIUS TREATMENT EFFECT AND COST TO GENERATE THIS WAY THE POPULATION AVERAGE COST EFFECTIVE RATIO. SO WE UTILIZE THE FRAMEWORK AND THIS IS ONGOING PROJECT. SO I'M HAPPY TO SHOW THE RESERVE NEXT TIME. THANK YOU. [ APPLAUSE ] >> WELL THANKS TO BOTH DAVIDS AND PETER. WE WILL NOW SWITCH BACK TO COST EFFECTIVENESS FOR PHARMA CO GENETIC PANEL TESTING. AND THIS IS FROM VANDERBILT, JOSH PETERSON AND JOHN GRAVES. >> ALL RIGHT, WELL, THANKS VERY MUCH FOR INVITING US TO TALK. THIS WORK IT HAS ACTUALLY BECOME EVEN MORE RELEVANT NOW THAN WHEN WE STARTED. IT'S HARD FOR ME TO GO TO A GENOMIC MEDICINE MEETING AND NOT HAVE EFFECTIVENESS MENTIONED AND IN FACT THE PAPER THAT DAVID TALKED ABOUT IN JAMMA THIS WEEK, THE LAST SENTENCE OF THE ABSTRACT IS: FURTHER RESEARCH& ON COST EFFECTIVENESS NEEDS TO BE DONE. THAT WAS GRATIFYING. [LAUGHTER] SO THIS PROJECT IS CALLED THE RIGHT PROG EXPECT 1 WANTED TO ACCOMPLISH A FEW THINGS SO 1 IS TO DO MORE WORK AND HOW TO MAXIMIZE THE VALUE OF THE TESTING WITH A PHARMACOGENOMIC PANEL POLARIZEDDULATIONS THIS IS SOMETHING THAT A LOT OF INSTITUTIONS ARE TAKING ON, THEY'RE INVESTING THEIR OWN MONEY IN--IN MEASURING WITH THE PANELLIVITYS OR A PANEL TESTS PART OF THE POPULATION, THEY WANT TO SORT OF LEARN HOW TO USE THAT DURING CLINICAL PRACTICE AND THEY'RE VERY INTERESTED IN WHETHER IMPROVED CLINICAL OUTCOMES. SO IT'S HARD TO CALCULATE THAT VALUE WITHOUT TAKING A LONG-TERM VIEW BECAUSE IF YOU DO A PANEL TEST, MANY OF THOSE MARKERS MAY ONLY BE RELEVANT FAR INTO THE FUTURE SO THAT'S WHERE WE WANTED TO PROVIDE SOME NEW METHODS FOR THAT VALUE TO BE ASSESSED OVER THE LONG-TERM. WE ALSO WANT TO DEFINE THE INFLUENTIAL SPHSK AND FINANCIAL AND BEHAVIORIAL DETERMINANTS OF COST EFFECTIVENESS. SO THE OTHER THING THAT'S CHANGED IN THIS FIELD IS THAT WHEN WE STARTED, I THINK THE EVIDENCE WAS JUST STARTING TO GET ORGANIZED AROUND WHICH DRUG GENE RELATIONSHIPS ARE MOST IMPORTANT AND THIS HAS MATURED FAIRLY QUICKLY, REALLY THROUGH THIS ORGANIZATIONSINOIDSATION THROUGH CPICK, SO THIS IS A SCREEN SHOT OF THEIR WEBPAGE SHOWING THE TOP OF THEIR CPICK LEVEL A LIST AND WE ACTUALLY STARTED OUR PROJECT BY FOCUSING ON 3 DRUG GENE RELATIONSHIPS BUT AS JOHN WILL PRESENT IN THE SECOND PART OF THIS TALK, WE HAVE NOW STARTED TO EXPAND IT INTO THE ENTIRE CPIC LEVEL A-LIST BECAUSE THAT'S WHERE PEOPLE THINK THE VALUE IS TO MEASURE ALL THE MARKERS, ALL THE GENETIC MARKERS ON THE LEFT-HAND SIDE AND DO SOME SORT OF TAILORING OF THE DRUGS THAT ARE IN THAT SECOND COLUMN. WHEN YOU THINK ABOUT PANEL TESTING, IT OPENS UP DIFFERENT STRATEGIES IN TERMS OF HOW YOU TEST PATIENTS. SO, OF COURSE, THERE'S THE REFERENCE GROUP OF NO TESTING AND THE LINES THERE REPRESENT HOW MUCH TIME IT TAKES FOR A PATIENT TO HAVE A PARTICULAR INDICATION. SO FOR EXAMPLE, THEY MIGHT DEVELOP HEART DISEASE AND NEED AN ANTIPLATELET DRUG, AND THEN THEY MIGHT DRIEWBT ATRIOLE FIBBULATION LATER AND THEN THEY MIGHT NEED WARFORIN AND THEN A STATIN. SO THESE ARE INDICATIONS HOW THESE COULD PLAY OUT OVER TIME. YOU COULD MEASURE A GENETIC MARKER EACH TIME THOSE DRUGS ARE NEEDED AND THAT OF COURSE IS THE STATUS QUO. NOW, THERE'S A LARGE AMOUNT OF VARIABILITY IN CLINICAL PRACTICE HOW OFTEN THAT'S DONE. SO IN SOME PROFESSIONS LIKE INFECTIOUS DISEASE WHEN YOU'RE STARTING BACK HERE, THE COMPLIANCE TESTING WILL APPROACH 100% BECAUSE IT'S PART OF THE STANDARD OF CARE WHEREAS IN CARDIOLOGY, IN TERMS OF TESTING 5619 IT MIGHT BE AS LITTLE AS 10% OR LESS THAN THAT. WHEN YOU USE A PANEL, THERE'S CHOICES HERE, YOU COULD WEIGHT FOR THE FIRST TIME THAT A PATIENT NEEDS 1 OF THOSE MARKERS AND THEN MEASURE THE ENTIRE PANEL. OR YOU COULD MEASURE IT COMPLETELY AHEAD OF TIME BEFORE THEY NEED IT ALL. SO PREEMPTIVE TESTING VERSUS MULTIREACTIVE TESTING AND IF YOU LOOK AT THE LAST STRATEGY, THAT CAN ACTUALLY BE DIVIDED INTO MEASURING EVERYONE WHO WALKS THROUGH THE DOOR AND PEOPLE HAVE TALKED ABOUT THAT, YOU KNOW, NEWBORNS GETSING WHOLE GENOME SEQUENCING THAT WOULD BE AN EXAMPLE OF UNIVERSAL GENOMIC TESTING OR YOU COULD TARGET PEOPLE WHO ARE AT HIGH RISK AND SO WITH OUR INSTITUTIONAL PHARMACOGENOMIC PROGRAM, WE USED STATISTICALLY ISOLATED OR IDENTIFIED A GROUP OF PATIENTS WHO HAD CARDIAC RISK FACTORS AND WE THOUGHT WOULD BENEFIT FROM THIS PANEL TESTING. SO, 1 OF THE MEMBERS OF OUR GROUP, DID A LOT OF WORK TRYING TO IMPROVE THE TARGETING OF POPULATIONS WHO MIGHT BENEFIT FROM THIS PANEL TESTING AND SO WE TOOK A LOT OF PATIENTS FROM VANDERBILT, WE DID 1 QUARTER, 30,000 PATIENTS ANOTHER COHORT OF A HUNDRED THOUSAND PATIENTS AND WE'RE TRYING TO PREDICT FUTURE DRUG PRESCRIPTION. WE IMPLEMENTED THIS IN THE EHR AS A FLAG AND THAT DROVE GENETIC TESTING AND THEN OVER THE LAST COUPLE OF YEARS WE TRY TO IMPROVE ON THAT AND IMRIEWF OUR PERFORMANCE AND I WILL SHOW 1 SET OF RESULTS WHICH IS SORT OF THE IDEA THAT IF YOU DID IMPLEMENT THIS, OR SORT OF OUR BEST MODEL WAS TURNED OUT TO BE A LONGITUDINAL COX MODEL, THEN WHAT YOU WOULD FIND IS THAT--OVER THE COURSE OF 2 YEARS RELATIVELY RAPIDLY MORE THAN 50% OF THE PATIENTS WOULD BE EXPOSED TO MEDICATION THAT WE CARE ABOUT. SO THE POSITIVE PREDICTIVE VALUE WAS ACTUALLY VERY REASONABLE. THERE WAS A LITTLE BIT OF A FLAW THOUGH IN THIS APPROACH IN THAT YOU MISS A LOT OF PATIENTS WHO WOULD ALSO BENEFIT AND SO THAT DOES ALSO EFFECT THE OVERALL COST EFFECTIVENESS OF THIS APPROACH. THE NEGATIVE PREDICTIVE I VIRUS IS NOT AS GOOD AS YOU MIGHT WANT IT TO BE. SO, THIS BECAME A MODULE WITHIN OUR EVENT STIMULATION AND WE BUILT THESE FOR EACH DRUG, AND IT WAS SOMEWHAT OF A PAIN STAKING PROCESS SO FOR WARFORIN AND OTHERS WE HAD A SITUATION WHERE WE LINKED THEM TOGETHER SO THAT YOU HAD THE MODEL THAT DETERMINED WHETHER THE PATIENT SHOULD BE TARGETED FOR TESTING, THE MODEL TO SORT OF DETERMINE HOW LONG IT TOOK FOR 1 OF THESE DRUG INDICATIONS TO COME ALONG AND THE MODEL THAT ASSESSED FOR EACH OF THESE WHAT WAS THE OUTCOME DIFFERENCE. THE SIMULATION OF A LOT OF THE FEATURES YOU MIGHT WANT, OF COURSE, YOU'RE MODELING THE BENEFITS AND RISKS OF TAYLORED THERAPY, YOU ARE MANAGING COMPETING RISKS LIKE SECULAR DEATH, INDIVIDUALIZING CARDIAC RISK TO PREDICT THE TIMING OF SOME OF THESE INDICATIONS AND DID ALLOW US IN A LIMITED FASHION TO LOOK AT BEHAVIORIAL FACTORS, ALTHOUGH WE QUICKLY RAN INTEREST A WALL THERE. I WILL TALK ABOUT IT IN A SECOND. AND HERE'S A SUMMARY SLIDE OF WHAT WE FOUND. SO THIS IS OVER A LIFETIME. SO EACH PATIENT WAS FOLLOWED THROUGHOUT THEIR ENTIRE LIFE UNTIL THEY DIED ACCORDING TO LIFE TABLES AND NO GENOTYPINGS IS OUR REFERENCE GROUP AND BOTH REACTIVE SINGLE GENE AND REACTIVE PANEL ARE COST EFFECTIVE OVER THAT LIFETIME BY CONVENTIONAL MEASURES OF WILLINGNESS TO PAY. THE 1 THINK THIS I WILL MENTION IS THAT IF YOU LOOK AT SHORTER TIME FRAMES, LIKE 10 YEARS, THAT IS ACTUALLY A LOT HIGHER. SO WHEN EN--STRATEGIES TUITIONS ARE CONSIDERING DOING THIS PANEL TESTING, THEY NEED TO REALIZE THIS SAY LONG-TERM INVESTMENT GET AROI BACK IN A COUPLE OF YEARS. WE ALSO HAVE A NUMBER THERE FOR GENO TYPE WHICH IS A WAY TO TAKE THE COST OUT OF THE EQUATION AND SAY, WELL, HOW MANY PEOPLE DO YOU HAVE TO TEST IN ORDER TO AVOID 1 ADVERSE EVENT. I THINK THAT'S 1 WAY TO TALK TO CLIN IGZS WHO MAYBE AREN'T SO RECEPTIVE TO THINK BEING IT IN IT TERMS OF DOLLARS. THE TARGETED PREEMPTIVE PANEL AND WE DUG INTO IT AND WE DUG INTO WHY TARGETING WASN'T FINDING ENOUGH OF THE PATIENTS EVEN THOUGH IT WAS GOOD AT FINDING A SUBSET OF PATIENTINGS THAT HAD A HIGH EXPOSURE RATE. SO, OUR LESSONS WERE THE DISCREET SIMULATION WAS FEASIBLE AND ACCURATE FOR THIS SMALL PHARMACOGENOMIC PANEL WE PUT TOGETHER. WE CAN COMPARE PANEL TESTING STRATEGIES, THE NEGATIVES WERE IT WAS VERY TIME INTENSIVE TO SIMULATIONS. AND IT WAS COMPUTATIONALLY INTENSIVE SO THAT EVEN RUNNING ON A CLUSTER OR OPTIMIZED MULTIPLE THREADS WOULD HAVE TAKEN OUR LIFETIMES TO DO OUR PROBABILITY OF SENSITIVITY ANALYSIS. I WILL CHANGE COURSE AND I WILL LET JOHN TO TALK ABOUT THAT. >> SO I WILL TALK BRIEF LYE ABOUT HOW WE'VE TAKEN WHAT'S EFFECTIVELY WHAT JOSH PRESENTED EFFECTIVELY A 3 DRUG PANEL AND THE THING THAT'S UNIQUE AND PARTICULARLY ABOUT MULTIPLEX TESTING IF YOU THINK ABOUT IT, AS A COST OF YOU KNOW WHOLE EXOME SEQUENCING AND GENOME SEQUENCING AS STARTED TO CONVERGEOT COST OF DOING A SINGLE GENE TEST. THERE'S LITTLE COST OF ADDING ANOTHER GENE TO A PANEL AND SO THE QUESTION BECOMES AT 1 POINT HOW CAN YOU SCALE UP A PANEL TO CAPTURE THE VALUE? AND THEN THERE'S METHOD LOGICAL CHALLENGE WE HAVE TO SOLVE WHICH IS HOW IN THE WORLD CAN WE TAKE LET'S SAY THE LIST OF 46 CPIC DRUG WHERE IS THERE'S SCIENTIFIC EVIDENCE THAT REACHED AT LEAST ABOVE A BAR TO WHERE IT SHOULD BE ACTIONABLE INFORMATION AND SO HOW CAN WE POSSIBLY MODEL THIS ALL OUT TO FIGURE OUT WHAT'S THE ACTUAL VALUE OF THE GENETIC NNKZ THAT COULD BE POTENTIALLY CONVEYED THROUGH MULTIPLEX SEQUENCING? SO THE TAKE AWAY FROM THIS SLIDE IS NOT TO DIGEST THE WHOLE THING BUT THINK OF THAT LIST OF CPIC DRUGS, 46 DRUG-GENE PAIRS ON THAT LIST AND WE STARTED TO SIMPLIFY THE PROBLEM DOWN TO WHERE WE CAN MODEL OUT A SCENARIO AND WE COULD SIMULATE OUT THROUGH A LIFETIME PATIENTS BEING EXPOSED, HEALTHY QUOTE HEALTHY POPULATION OF PATIENTS, NAIVE TO ANY OF THE DRUGS BEING EXPOSED OVER THEIR LIFE TOO TO ANY NUMBER OF THOSE 46 DRUGS AND THE WAY WE'VE DONE THIS METHOD LOGICALLY IS TO COME UP WITH A CONONICLE SCENARIO FOR PHARMACOGENOMICS, THE IDEA BEING YOU HAVE SOME PATIENT PATIENT--OOPS--SOME PATIENT NAIVE TO A DRUG SCENARIO, WHERE THEY'RE EXPOSED, THEY HAVE AN INDICATION FOR DRUG OVER TIME. THERE'S STANDARD THERAPY THAT EVERYBODY'S PUT ON UNLESS THERE'S SOME EVIDENCE THROUGH A GENETIC TEST THAT THEY'RE A POOR METABOLIZER OF THAT DRUG AND THIS WILL BE IMPORTANT LATER, THE CLINICIAN UTILIZES THE INFORMATION AND CONVEYED IN THE GENERATEDET EGIC TEST. THIS HAS A TYPO AND THERE SHOULD BE ERRORS IN THIS TREATMENT, SO ON EITHER TREATMENT, YOU HAVE SOME ADVERSE EVENT THAT COULD HAPPEN TO YOU, THINK OF IT LIKE AS COMPOSITE ADVERSE EVENT OR SOME CASE FATAL FATALITY RATE RELATED TO THAT SO IN SOME CASES THAT COULD BE SIMPLE AS MUSCLE SORENESS OR IT COULD BE A FATAL BLEED OR A HEART ATTACK OR SOMETHING LIKE THAT IN THE CASE OF ANTIPLATELET THERAPY. BUT IN ANY CASE IF YOU'RE A GENOTYPE, YOU'RE A POOR METABOLIZER OF THE DRUG AND THAT IS UTILIZED IN THE FREE RADICALS SCRIBING DECISION AND THE THEN THE ALTERNATIVE PHARMACOGENOMICLY GUIDED CAN CONVEY BENEFIT AND THAT'S WHAT WE'RE MODELING AND THE NICE THING ABOUT DOING THIS IS 2 REASONS WHY WE HAD TO SIMPLELIFY A PROBLEM DOWN THAL VERY SIMPLE SCENARIO. ONE IS THAT WE COULD THEN SCALE IT UP TO 46. SO WE BASICALLY CAN MODEL OUT FOR A POPULATION, WE CAN POPULATE THIS SIMPLE PARAMETER OR THE SIMPLE MODEL WITH SPECIFIC PARAMETERS FOR EACH OF THOSE SCENARIOS AND SECONDLY WHAT WE COULD DO IS COMPUTATIONALLY THIS WILL TOUCH ON THE DAVID'S TALK FROM EARLY EARLY FROM VALUE OF INFORMATION. SO IN ORDER TO DO THE KIND OF FULL MODEL BASED VERSION EAND EVALUATION, YOU HAVE TO RUN THE MODEL A TON OF TIMES AND IT TAKES A WHILE TO RUN THIS ONCE AND YOU GET 1 POINT OUT OF THAT AND WE NEED TO RUN IT THOUSANDS AND THOUSANDS OF TIMES TO GET THAT VALUE OF INFORMATION AND WHAT WE CAN DO WITH THIS SIMPLIFIED VERSION OF IT IS TO EFFECTIVELY CONVERT THIS INTO A SCALABLE DELAY, DIFFERENTIAL UATION AND WE CAN SOLVE THAT AND WHAT THAT DOES IS REMOVE THE STOIKASTIC UNCERTAINTY YOU GET IN SIMULATION MODEL BEING THAT YOU'RE USING RANDOM DRAWS FOR WHO HAS THE PROBABILITY FOR WHAT. SO WITH THAT WE CAN BASICALLY RUN OUR MODEL FOR A FULL, WHAT WOULD BE A MULTIPLEX PANEL, AND GET VALUE OF INFORMATION WHICH PROVIDES REALLY USEFUL INFORMATION. WHICH I'LL GET TO HERE. SO THE FAVOR OF THESE RESULTS WHICH I WILL PRESENT TO YOU ARE BUILT OFF A MODEL THAT ANIRBAN AND DAVID PUT TOGETHER YEARS AGO AND IT HAS A FLAVOR OF INFORMATION AND THE WAY TO THINK ABOUT THAT CONCEPTUALLY IS WHAT'S THE OPPORTUNITY COST THAT WE ARE FOREGOING BY NOT UTILIZING GENETIC INFORMATION IN A DECISION, THAT IS IF WE CAN MAKE THE DECISION FOR EVERY PATIENT AT THE POINT OF PRESCRIBING AND THEN ACCRUE ALL OF THE HEALTH BENEFITS AND COSTS THAT COME DOWN STREAM OF THAT, WHAT IS--WE WILL PUT A DOLLAR VALUE ON THAT NEEVERGZ AND THAT'S WHAT THIS SLIDE IS SHOWING. SO IMAGINE THAT WE COULD KNOW AT NO COST FOR NOW, THE GENETIC DISPOSITION FOR EACH PATIENT AND CHOOSE THE RIGHT DRUG FOR THEM BECAUSE THEY ARE POOR METABOLIZERS AND THEY GET SHUNTED OFF ON THE ALTERNATIVE SO GIVEN SOME SOCIETAL WILLINGNESS TO PAY FOR A QUALITY ADJUSTED LIFE YEAR, WHAT IS THE DOLLAR VALUE GAIN OF ANY COST THAT THAT COST--THAT CAUSES DOWN STREAM. AND SO WHAT WE DID WAS SPLIT ALL OF CPIC DRUGS INTO BASICALLY 7 DISTINCT CATEGORIES AND THE 1 I WANT TO FOCUS ON IS THE 1 AT THE BOTTOM BECAUSE THIS HAS BEEN MENTIONED A COUPLE TIMES TODAY BUT THIS IS THE JAMA ARTICLE THAT CAME OUT THIS WEEK ON FARM--OR GENETICALLY GUIDED WARFORIN DOSING AND THAT--THE WARFORIN SCENARIO MAPS THROUGH THE HIGH, HIGH, HIGH SCENARIO MEANING A DRUG THAT'S PRESCRIBED FREQUENTLY AND HASSAN ADVERSE EVENT FREQUENCY SO COMPLIKAIGDZS OF WARFORIN IS 1 OF THE FREQUENT VISITS IN THE UNITED STATES AND THE ADVERSE EVENT SEVERITY IN TERMS OF HOW NASTY IS THE ADVERSE EVENT. IT'S NOT MUSCLE SORENESS IT'S HEART ATTACKS AND BLEEDS AND SUFFER FROM LIKE THAT SO WARFORIN CHECKS THE BOX ON ALL OF THESE, AND IF YOU THINK ABOUT SHOULD WE DO THE GENETIC TESTS FOR THE PHYSICIANS AT THE TIME THEY PRESCRIBE THIS GET THEM ON THE RIGHT DOSE QUICKER. THE VALUE OF THAT INFORMATION IS $757 IF WE TAKE THE STANDARD WILLINGNESS TO PAY A HUNDRED THOUSAND DOLLARS FOR QUALITY ADJUSTED LIFE HERE SO YOU CAN THINK OF THESE NUMBERS AS THEY MAP TO THESE VARIOUS SCENARIOS, NONAPOPTOTIC THE ONLY IN TERMS OF IF YOU'RE A REEMBUSKERRER, HOW MUCH DO WE PAY FOR A SINGLE GENETIC TEST BUT SOME OF THESE DON'T CHECK THE BOX OF BEING WARFIT, AND AS I MENTIONED EARLIER THAL SCENARIO OF PRECISION MEDICINE, AT ADDING THESE GENES TO THE PANEL IS EFFECTIVELY COST LITTLE BUT THERE ARE COSTS IN THE SYSTEM IN TERMS OF HOW THAT INFORMATION IS THEN DIFFUSED INTO THE ACTUAL DECISION MAKING. SO 1 THING WE CAN DO IS SIMULATE OUT WHETHER OR NOT IF YOU ADD THESE 46 DRUGS ON TO A SINGLE PANEL TEST, WHETHER PREEMPTIVE, OR REACTIVE MULTIPLEX TEST SUGGEST WORTH IT IN TERMS OF THE INCREMENTAL COST EFFECTIVENESS RATIO. AND THEN YOU CAN SEE OVER A LIFETIME, WE'RE KIND OF RIGHT ON THE MARGIN, STANDARD THRESHOLD IS'RE VARY FROM LET'S SAY A HUNDRED THOUSAND DOLLARS TO I DON'T KNOW WHAT THE OFFICIAL NUMBER IS TODAY, IT'S LIKE 138 OR SOMETHING LIKE THAT. BUT WE'RE KIND OF RIGHT ON THE MARGIN. IF YOU LOOK AT THIS OVER A 10 YEAR HORR RISEON AND ECHO JOSH'S POINT FROM EARLIER, YOU LET THIS PLAY OUT OVER TIME, OVER A LIFETIME REALLY TO,A CREW THE THE--ACCRUE THE VALUE THIS COULD CONVEY. IF YOU THINK ABOUT THE ENECONOMISTS IN IT THE ROOM THINK ABOUT WHY THIS INFORMATION OBTAINED UPSTREAM WILL NOT BE USED WE HAVE TERMS IN THE INSURANCE MARKETS THAT ARE NOT THE ON SAME INSURANCE PLAN THAT PAID FOR DRUG IN THE BEGINNING. WE HAVE PRICE SENSITIVITY TO IF THE ALTERNATIVE IS CONSIDERABLY MORE EXPENSIVE TMIGHT NOT BE WORTH IT TO A PATIENT TO PAY THOUSANDS OF DOLLARS MORE PER YEAR, EVEN THOUGH THE FARM O GENOMICS WOULD GIED THEM, ET CETERA. SO WAWE CAN DO IN OUR SIMULATION MODEL IS EFFECTIVELY DO A GIANT SENSITIVITY ANALYSIS AND LET THE PARAMETERS VARY AND THIS IS WHERE THE VALUE OF INFORMATION COMES IN BECAUSE WE CAN FIND THE PARAMETERS AND THE MODEL THAT HAVE THE HIGHEST LEVERAGE IN TERMS OF MIEWAING THE RESULTS AND WHERE VITAL INFORMATION IS REALLY IMPORTANT IS NOT BECAUSE THERE'S A PARAMETER THAT MOVES THE RESULTS AROUND. WHAT WE WANT TO KNOW IS WHAT ARE THE PARAMETERS THAT NOT ONLY MOVE RESULTS BUT ALSO CHANGE THE DECISION AND THOSE ARE THE PATHWAY GIVES RAMSTERS WE SHOULD PUT VALUE ON IN TERMS OF FIGURE OUT WELL, WE DON'T KNOW EXACTLY WHAT THE RELATIVE RISK REDUCTION FROM WARFORIN IS BUT WE SHOULD SPEND MONOFIGURING OUT WHAT IT IS BECAUSE IT'S EXTREMELY VALUABLE. SO WHAT THIS IS A TORNADO DIAGRAM SHOWING THE KIND OF RANK ORDERING THE PARAMETERS IN OUR MODEL, IN TERMS OF HOW THEY GUIDE OUR OPTIMALY CISION OF WHAT TREATMENTS OR TESTING STRATEGY WE SHOULD PUT IN THE POPULATION, AND SOME OF THESE ARE JUST DATA CHALLENGES. SO THE FIRST 1 RANKED HERE IS THE RISK OF GETTING ON A DRUG LIKE WAR FARIN, SO AT WHAT RATE DO PEOPLE GET ON THE DRUG THAT'S JUST INFORMATION. SO IF WE HAD GOOD DATA WE CAN NAIL THAT DOWN AND WE CAN START TO MOVE DOWN THE LIST IN TERMS OF PRIORITIZING ATTENTION TO REALLY FOCUS ON THE PARAMETERS THAT ARE DRIVING, NOT ONLY VARIATION IN THE OUTCOME OF OUR SIMULATION BUT MORE IMPORTANTLY HOW WE'RE MAKING DECISIONS OR WHETHER THE OPTIMAL DECISION CHANGES BASED ON VARIATION IN THE PARAMETERS. THIS IS A PLOT. IT'S EFFECTIVELY A 2 WAY SENSITIVITY ANALYSIS. I WILL WALK YOU THROUGH IT FOR WHAT MAY COME UP TODAY AND TOMORROW. SO YOU CAN HAVE NO TESTING WHATSOEVER IS OPTIMAL OR VARIOUS GENETIC TESTING SCENARIOS. THE X-AXIS IS THE RISK REDUCTION FROM THE ALTERNATIVE THERAPY THAT WAS--THAT WOULD BE SUGGEST FEDERAL YOU'RE A POOR METABOLIZER OF THE DRUG FOR EXAMPLE THAT WILL VARY FROM 5 TO 1. ONE MEAN THRG'S NO RELATIVE RISK REDUCTION, IT'S EFFECTIVELY THE SAME AS STANDARD THERAPY IN TERMS OF AVERTING ADVERSE EVENTS. VERSUS 5 WHERE THE IF YOU'RE A METABOLIZER IS IMPORTANT IN TERMS OF REDUCING THE RATE OF ADVERSE EVENTS. SO I MENTIONED THE WAR FORIN ARTICLE EARLIER, SO JUST TO GIVE YOU A SENSE OF WHERE THAT LIES ON HERE, JUST CAME OUT ON TUESDAY, IT HAD A COMPOSITE ADVERSE EVENT OF ABOUT .27. SO IT'S RIGHT ON THE BORDER. SO IF THAT RISK REDUCTION IS ABOVE .8 OR SO, YOU YIEWS--JUICE YOU GET IS NOT WORTH IT OUT OF PAYING FOR THE QUALITY. WHAT HAPPENS IS THAT MOST OF THE SCIENTIFIC EVIDENCE AND IN PARTICULAR KIND OF THE EDITORIALIZING AROUND PERSONALIZED MEDICINE, A NUMBER WILL COME OUT AND TELL BE ALONG THE X-AXIS AND WHETHER OR NOT WE THINK THAT THIS SHOULD BE DONE IN PRACTICE OR NOT DONE IN PRACTICE, DEPENDS ON WHERE YOU FALL HERE, BUT MOST OF WHAT THEY'RE DOING IS ASSUMING THAT THIS INFORMATION IS GOING TO BE IMMEDIATELY DIFFUSED INTO THE SYSTEM. SO WE'RE KIND OF OVERESTIMATING THE VALUE BECAUSE WE'RE ASSUMING THAT THE PHYSICIANS ARE GOING TO UTILIZE THE GENETIC INFORMATION ALL THE TIME. SO WHAT OUR MODEL CAN ACTUALLY DO IS SEE HOW SENSITIVE OUR CONCLUSIONS ARE TO HOW LIKELY IT IS THAT THE PHYSICIANS UTILIZE THE INFORMATION. ONE OF THE ARGUMENTS IN FAVOR OF DOING PREEMPTIVE TEST SUGGEST THAT YOU CAN LEVERAGE THE INSIGHTS OF PSYCHOLOGY AND BEHAVIORIAL ECONOMICS, THE IDEA BEING THAT IF YOU MAKE A PHYSICIAN ACTIVELY ORDER A TEST, THEY ARE LESS LIKELY TO UTILIZE THAT INFORMATION OR ORDER IT THAN IF YOU HAVE DONE IT UPSTREAM AND THAT INFORMATION IS PROVIDED TO THEM. ALTERNATIVELY, YOU HAVE DECISION FATIGUE IF YOU FLAG PEOPLE WITH DECISION AIDS THAT YOU TELL THEM THEY'RE A GOOD OR BAD METRICS TAB METABOLIZER YOU MAY IGNORE THE INFORMATION AND CYCLE THROUGH IT. THE LIKELIHOOD THAT THE PHYSICIAN UTELIZES THE INFORMATION DECLINES YOUR OPTIMAL STRATEGY WILL CHANGE TO WHERE IF IT'S A HIGH LIKELIHOOD THE PHYSICIAN UTILIZES IT, PREEMPTIVE GENERATEDEE TYPING STRATEGIC PLAN EDGE SEOPTIMAL VERSUS IN THE CASE OF--IN THE CASE OF WARFORIN, AS THE PROBABILITY GOES DOWN YOU'RE BETTER OFF DOING THE HUNDRED DOLLARS SINGLE GENE TEST RATHER THAN $800 FOR THE WHOLE GENOME SEQUENCING. I'M OVER TIME SO I'LL STOP THERE. SHI MENTION THAT OUR MODEL IS ONLINE, BOTH OF THESE MODELS WILL BE ONLINE. OPEN SOURCE SO I HAVEN'T DONE THE PROPER DISCOUNTING AS A JUNIOR PERSON OF--EXTRACTING MY OWN PERSONAL VALUE OUT OF THESE BUT THEY'RE OPEN SOURCE AVAILABLE ONLINE. I'LL BE WALKING THROUGH NEXT DOOR. >> [ APPLAUSE ] >> SO NOW WE WILL MOVE TO THORS THE DISCUSSION AND THE FIRST PART OF THE DOCTOR FROM M. I.T. WILL DISES SOME THINGS BEFORE WE THROW IT OPEN. >> THANK YOU. AS EXPECT ON A SESSION ON PERSONALIZE MEDICINE THERE'S A LOT OF HETEROGENEITY IN ISSUES ADDRESSED IN METHODS, BUT 1 COMMON THEME WE OBSERVED ALREADY IS THAT AS WE MOVE FROM UNCERTAINTY OR TRANSFORMING UNCERTAINTY TO MORE QUANT FIELD FUNCTIONSABLE RISK MORE PREICIZE RISK, WE'RE VERY INTERESTED IN WHAT ARE THE EFFECTS OF THE ADOPTION OF PERSONALIZED MEDICINE, ON UTILIZATION, ON PATIENT OUTCOMES, PATIENT SATISFACTION. I WILL BE FOCUSING MY REMARKS BASICALLY WHAT HAPPENS WHEN PRECISION MEDICINE ISN'T PRECISE? BECAUSE I THINK THAT'S WHERE WE'RE AT AND I THINK ECONOMIC HAS A LARGE ROLE TO PLAY. IT'S ALREADY AFTER LUNCH AND [INDISCERNIBLE] IS SETTING IN SO I WILL BE A BIT MORE VOCABULARY VACCAATIVE THAN I USUALLY AM. INSTEAD OF TAKING THE PERSPECTIVE OF INDIVIDUAL OR A PROVIDER OR A PAYOR OR AN EMPLOYER, OR EVEN SOCIETY, I WILL FOCUS ON THE DEVELOPER OF A PERSONALIZED MEDICINE. I WANT TO TRY AND CONVINCE YOU THAT GAME THEORY PROVIDES US A LOT OF INSIGHTS INTO WHAT'S HAPPENING IN THE LAST DECADE OR SO, AND WHAT WE SEE WITH PERSONALIZED MEDICINE. THE RESEARCH I'M GOING TO TALK ABOUT WAS NOT FUNDED BY THE NIH. THAT LETS ME BE PROVOCATIVE AS WELL. AND IN FACT A FEW YEARS AGO WHEN I APPLIED FOR MY FIRST NIH GRANT, JUST UNDER AGE 70, I WAS GIVEN A FEW EXTRA POINTS BECAUSE I WAS CALLED A YOUNG INVESTIGATOR. [LAUGHTER] AND SO THAT GIVES ME A LOT OF FREEDOM. OKAY. SO I'M GOING TO TALK BASIC LE ABOUT WHAT I WILL CALL THE INFORMATION FARMS RACE. IT'S JOINT WORK WITH MARK [INDISCERNIBLE], WE RESENTED IT A FEW WEEKS AGO AT AN NBER CONFERENCE AND IT WILL COME OUT AS AN NBER WORKING PAPER SOMETIME IN THE NEXT FEW WEEKS. IT'S BASED ON RESEARCH I'VE DONE WITH MARK OVER THE LAST DECADE. GIVEN TIME CONSTRAINTS, I WILL ONLY TALK ABOUT THE FIRST 3 OF THOSE POINTS TODAY. LET'S START WITH JUST DEFEIGNING A PRECISION MEDICINE. WITH COMBINED WITH SOME FORM OF BIOMARKER DIAGNOSTIC, THERAPIES THAT ARE ABLE TO IDENTIFY SUBPOPULATIONS THAT ARE LIKELY TO RESPOND DIFFERENTIALLY, BE IT POSITIVELY OR NEGATIVELY ARE VARIOUSLY CALLED PERSONALIZED MEDICINE, PRECISION MEDICINE, TAYLORED MEDICINE, AND STRATIFIED MEDICINE. REGARDLESS OF WHAT 1 CALLS IT, IT'S THE COMBINATION OF A THERAPY AND A COMPANION DIAGNOSTIC THAT RELIES CRITICAL LEOT ABILITY TREATMENT FROM RESPONDERS FROM NONRESPONDERSNONRESPONDERS AND AS WE KNOW FROM MANY, MANY MEDS CLINICAL EFFECTIVE RATIOS ARE PRETTY SMALL. SUPPOSE WE HAVE A PANEL OF PEOPLE WHO ARE ADDRESSED IN BLACK CLOTHES, SOME WHO ARE DRESSED IN BLUE CLOTHES. AND SUPPOSE THAT THE BLUE FOLKS HAVE HIGHER RESPONSE RATE THAN THE FOLKS DRESSED IN BLACK. WE'D LIKE OF COURSE, IS THAT THE BIOMARKER WOULD PERFECTLY DISTINGUISH THE BLUE DRESSED FOLKS FROM THE BLACK DRESSED FOLKS BUT LIKE DRUGS NO DIAGNOSTIC IS PERFECT. SENSITIVITY IS A PORTION OF TRUE POSITIVES THATTURE DIAGNOSED AS SUCH, SPECIFICITY IS THE PORTION OF TRUE NEGATIVES AND RECEIVE A NEGATIVE DIAGNOSTIC RESULT TO GIVE YOU BENCHMARKS THAT ARE 2 TESTS FOR RECEPTOR RESPONDERS AS AN 89% SENSITIVITY AND 83% SPECIFICITY. I DON'T REMEMBER WHAT THE ONK O TYPE NUMBERS WERE ANYONE REMEMBER HERE? WE TALKED ABOUT IT TODAY? ALL RIGHT. HOW DO WE GET TO THIS TO GAME THEORY? WELL COMPARING DIAGNOSTICS INHERENTLY REDUCE THE SIZE OF THE TREATED POPULATION. THE IDENTIFY SUBPOPULATIONS AND FRAGMENT. PRECISION MEDICINE IS THEREFORE TYPICALLY PROCESS RELATIVELY SMALL NUMBERS OF PATIENTS. MORE LIKELY THAT THEY'LL BE NICHE BUSTERS RATHER THAN BLOCKBUSTERS. IT'S PLAUSIBLE THEREFORE TO EXPECT THAT SMALL MARKETS WILL ATTRACT FEW ENTRANTS. TO DATE PRECISION MEDICINES GENERALLY INVOLVE SMALL NUMBER OF INTERDEPENDENT DIFFERENTIATED PRODUCTS OLIGOPLIFTS THAT BRINGS US TO GAIOF THE THEORY. OUR FOCUS HERE ON NONCOOPERATIVE GAMES WHERE THE NEGOTIATION AND ENFORCEMENT OF BINDING CONTRACTS VIOLATES ANTITRUST OR ANTICOMBINES, AND IS THEREFORE NOT POSSIBLE BUT WHERE PLAYER CANS ANTICIPATE, OBSERVE AND REACT TO EACH OTHER'S BEHAVIORS. A SOMEWHAT UNIQUE PERSPECTIVE OF GAME THEOR SETHAT TYPICALLY GAME THEORY ASSUME THAT PROPHET MAXIMIZATION IS THE FIRM'S IF NOT ONLY OBJECTIVE AND BUT THEY ASSUME THAT ALL PLAYERS ARE RATIONAL AND WHAT IS MEANT BY RATIONAL. FIRMS THINK THROUGH THE CONSEQUENCES OF THEIR ACTIONS AND HOW THEY WILL BE PERCEIVED BY COMPETITORS WITH EACH 1 ASKING, SINCE OUR COMPETITORS ARE RATIONAL AND ACT MYSELF THEIR OWN EXPECTED PROFITS, WHAT WILL THEY DO AND HOW SHOULD WE TAKE THEIR LIKELY BEHAVIOR INTO COUNT WHEN PAIK MAKING OUR DECISIONS. I MIGHT REMEEBD YOU THAT THE--REMIND YOU THAT THE HIOF HISTORIC GAMES OF MIND THEORY WERE NUCLEAR DETERRENTS, PRENORTH KOREA AND THE ARMS RACE. WE'RE GOING TO BUILD ON THAT AND LOOK TO UNDERSTAND WHAT I CALL THE INFORMATION FARMS RACE BETWEEN DRUG DEVELOPERS WHO WANT TO CREATE SCIENTIFIC FOUNDATIONSFOUNDATION S AND EVIDENTIARY PLATFORMS FOR DIFFERENTIATING MEDICINE AND PAYORS WHO WANT TO MINIMIZE THAT DIFFERENTIATION INSTEAD PLAY DEVELOPERS OFF 1 AGAINST EACH OTHER AND CREATE THE BIDDING WAR WHICH WE CALL BUTRON BUTRON COMPETITION. WHAT MAKES GAME THEORY APPLICABLE TO RESCISSION MEDICINE IS THAT THERE ARE COMPLICATIONS BEYOND THOSE IN TRADITIONAL ONE-SIZE-FITS-ALL DRUG DEVELOPMENTS THAT ARISE WHEN FIRMS CHOOSE TO USE A COMPANION DIAGNOSTIC TO STRATIFY PATIENTS INTO SUBPOPULATIONS IF THEY CHOOSE TO USE A COMPANION DIAGNOSTIC, THEN THEY HAVE A CAUGHT OFF VALUE FOR THE POM PANNION DIAGNOSTIC THAT ACT AS A GATE KEEPER EFFECTS THE CLINICAL OUTCOME SUCH AS EFFICACY AND THE MARKET SIZE OF POTENTIAL TREATMENTS OF POPULATIONS. A FIRM MAY OR MAY IF THE USE TO USE A COMPANION DIAGNOSTIC BUT IF IT DOES THE CHOICE OF DIAGNOSTIC CUT OFF VALUE TO SEPARATE LIKELY RESPONDERS FROM NONRESPONDER SYSTEM CRITICAL. IT LINKS THE UNDERLYING SCIENCE TO THE THERAPEUTIC RESPONSE TO EFFECT, OBSERVE EFFICACY WITH IMPLICATIONS FOR PRICING PARTICULARLY PRICING TAKES THE FORM OF VALUE PRICING OR SOMETHING LIKE THAT. OKAY, LET ME TAKE YOU THROUGH A DIAGRAM HERE. GRAB MY NOTES HERE. SUPPOSE WE HAVE ON A VERTICAL ACAXIS, THE NUMBER OF PATIENTS AND ON A HORIZONTAL AXIS, SOME CERTIFICATE OF COMPANION DIAGNOSTIC SCORE. AND LET'S SAY THAT THE YELLOW CURVE REPRESENTS THE NONRESPONDERS AND THE--OOPS AND THE BLUE CURVE REPRESENTS RESPONDERS. ASSUME WE HAVE 3 DRUGS THAT ARE VIRTUALLY THE SAME, SLIGHTLY DIFFERENTIATED AND EACH OF THEM IN THE POPULATION AS A WHOLE HAS A 33 AND A THIRD% RESPONSE RATE. SO THEY'RE IDENTICAL IN THAT SENSE. OKAY? NOW WE HAVE SOME POSSIBILITIES. COMPANION DIAGNOSTIC FIRST OF ALL IMPERFECTLY SCORES RESPONDERS, THE BLUE AREA WITH NONRESPONDERS, THE YELLOW AREA WITH THE BLUE AREA, SO THOSE--THE CURVES OVERLAP, OKAY? FALSE FALSE FALSE-POSITIVES ARE REPRESENTED TO THE RIGHT, AND THE FALSE-NEGATIVES BY THE PORTION OF BLUE CURVE TO THE LEFT OF THE CUT OFF LIKE C FOR EXAMPLE, EACH OF THE 33% WHO RESPOND TO TREATMENT GET THE ASSUMPTION OF 12 MONTHS OF HIGH QUALITY LIFE AND THE NONRESPONDERS GET 0. OKAY? SUPPOSE NOW A TRIAL USE TD 1 HELPED THOUSAND PATIENTS AND IT HAD NO TEST AT ALL. IT'S LIKE ASSUMING A CUT OFF OF THRESHOLD OF A. OKAY? AND ALL COMERS CASE IF YOU WILL, THE AVERAGE CLINICAL BENEFIT WOULD BE 4 MONTHS. IT WOULD BE 12 MONTHS FOR 1/THIRD WHO RESPOND AND 0 MONTHS FOR THE 2/THIRD WHO DON'T RESPOND. OKAY IN THAT CASE WE WOULD HAVE A HUNDRED% SENSITIVITY. IT WOULD SLECT ALL PATIENT WHO IS MIGHT RESPOND. 0 PERCENT SPECIFICITY, EXCLUDE NONE WHO WILL NOT BENEFIT AND THE POSITIVE PREDICTED VALUE WILL BE 33%. SUPPOSE INSTEAD, THAT WE HAVE A CUT OFF VALUE OF B. WHOOPS. HOW DO I GET THE POINTER TO WORK HERE. OTHER 1? OOH, SORRY. CUT OFF VALUE OF B, SAY IT SLEBTS NEARLY ALL WHO WILL RESPOND NOTICE THE BLUE AREAS JUST ALMOST ENTIRELY TO THE RIGHT OF B. SO LET'S SAY IT HAS 95% SENSITIVITY. SO OF THE 33,000 WHO WILL RESPOND, IT'S LIKE 31,500 PREEMPTIVELY BUT IT ALSO EXCLUDES A LOT OF THOSE WHO DON'T RESPOND. LET'S SAY IT HAD 64% SPECIFICITY SO OF THOSE 67,000 NONRESPONDERS IT RIERKS DENTIFIED 24,000 OF THEM. NOIA HAPPENS NOW, BY ENRICHING THE PATIENT SUBPOPULATION THROUGH THE USE OF THE COMPANION DIAGNOSTIC WHAT HAPPENS NOW IS WE HAVE 31,500 PEOPLE WHO HAVE A--SORRY--YEAH WITH 12 MONTHS BENEFIT AND 24,000 WITH 0 MONTHS BENEFIT SO THAT THE MEAN BENEFIT, NOW LOOKS FOR 6.7 MONTHS RATHER THAN 0.4 MONTHS NOT BECAUSE THE DRUG IS DIFFERENT. IT'S THE SAME DRUG, SENTIALLY BUT BECAUSE WE USED DIAGNOSTIC TO ENRICH THE PATIENT SUBPOPULATION, IT MAKES IT LOOK BETTER. NOW LET'S GO ON TO CUT-OFF C. CUT-OFF C EXCLUDES NEARLY ALL WHO DO NOT RESPOND, OKAY SO IT'S THE RIGHT CORNER OF THIS YELLOW CURVE. IT HAS 95% SPECIFICITY. SO THAT IT EXCLUDES 63,500 OF THE SCEIVE THOUSAND BUT BY ASUVENLINGS IT ALSO EXCLUDES A LOT WHO WOULD BENEFIT FROM TREATMENT. OKAY? ALL THESE FOLKS IN THIS LITTLE AREA DOWN HERE. WHAT WOULD THE MEAN RESPONSE LOOK LIKE? WELL, BY EXCLUDING--BY BASICALLY IDENTIFYING ALL THOSE WHO WOULD RESPOND, EFFECTIVELY, WE DRIVE UP THE MEAN EFFICACY TO 10.3 MONTHS WHICH IS CERTAINLY MUCH BETTER THAN MONTHS WITH NO DIAGNOSTIC AND CONSIDERABLY BETTER EVEN IN THE USE OF CUT-OFF VALUE B. >> ERNIE, WE ARE GOING TO HAVE TO MOVE FAIRLY SOON. >> I HAVE 2 MINUTE FIGURES I MAY. RIGHT. >> THANK YOU. >> NOTICE WHAT HAPPENS NOW IS THAT IF YOU USE A COMPANION DIAGNOSTIC, WHAT CUT-OFF CHOICE, IF YOU ASSUME THAT WHATEVER THE FIRM GETS IN TERMS OF WHICH DRUG IT GETS 1 BILLION DOLLARS IN ANNUAL REVENUES AND IF WE USE THE 138 K VALUE FOR QUALITY THAT WAS TALKED ABOUT A MINUTE AGO, WHAT YOU CAN SEE IS WHEN YOU CAN PRICE THESE 3 DRUGS VERY, VERY DIFFERENTLY. OKAY? AND THE--ALL COMERS WOULD BE PRICED AT 46,000 TREATMENT, DRUG B AT 77,000, DRUG C AT 119 EVEN THOUGH THEY'RE THE SAME DRUG. AND WAWE TALK A LOT MORE ABOUT PAPERS BASICALLY, THIS IS LIKE PRISONERS DILEMMA OUTCOME AND THAT FOLKS WILL BE DRIVEN BY COMPETITIVE REASONS TO LOOK FOR HIGHER QUALITY DRUG AS EVIDENT BY A HIGHER CUT OFF. WITH A GREATER OBSERVED EFFICACY RATE AT A MUCH HIGHER PRICE AND WE THINK THAT'S WHAT'S HAPPENING PARTLY IN THE LAST FEW YEARS. I'LL STOP THERE. [ APPLAUSE ] >> THANK YOU. >> THANK YOU DR. BERNDT. THE SESSION OPEN FOR DISCUSSION. I WOULD LIKE TO ASK DR. MANDELBLATT, ABOUT THE ISSUE OF ONCO-TYPING AND THE DIFFERENCES BETWEEN THE INITIALLY PUBLISHED STUDIES WHICH WERE PRESUMABLY SUPPORTED BY THE PHARMACEUTICAL COMPANY AND YOURS AND THE ENORMOUS DIFFERENCE. A SPECIFIC QUESTION I HAVE IS DO HAVE YOU ACCESS TO THE PRIMARY DATA FROM THE ORIGINAL ANALYSIS? WILL YOU ANALYZE IT DIFFERENTLY OR HOW DO YOU TRY TO EXPLAIN THOSE DIFFERENCES? >> THAT'S AN EASY QUESTION. THE ORIGINAL ANALYSIS ASSUMED ONK O TYPE TESTING WAS USED 100% OF THE TIME, IT WAS A HUNDRED% ACCURATE AND PATIENTS COMPLIED WITH THE TEST DICTATED RESULT A HUNDRED PERCENT OF THE TIME. SO UNDER THOSE CIRCUMSTANCES IT IS HIGHLY EFFECTIVE. AND SO WHAT WE DID WAS TAKE THE ACTUAL SELECTION FORCES THAT GO INTO THE CLINICAL DECISION MAKING WHERE PEOPLE FEEL THE TEST IS ACTUALLY USEFUL AND MIGHT CHANGE THEIR DECISIONS. HOW PATIENTS ACTUALLY CHOOSE TREATMENT GIVEN THE TEST AND WE USE THE ORIGINAL DATA TO ACTUALLY CALCULATE THE TEST PROPERTIES. AND SO, THE RESULTS ARE QUITE DIFFERENT. WE TRY TO REPLICATE THOSE RESULTS AND WE GET CLOSE. WE DON'T GET QUITE AS LOW BUT WHEN YOU CONSIDER INFLATION AND A LOT OF OTHER WIGGLE ROOM WE GET [INDISCERNIBLE]-- >> ARE THERE ANY LESSONS LEARNED FOR REVIEWERS OF SUCH A MANUCRYPT SCRIPT OR FOR THE REGULATORY AGENCY THAT MIGHT BE LOOKING AT THIS QUESTION? >> SCOTT WEST ANSWER-- >> I MEAN FOR REGULATORS THEY AREN'T GOING TO HAVE A REEL KEEPSAKES IN A SAY ON THE MODELING STUDY THAT TRIES TO ESTIMATE COST EFFECTIVENESS AND THE BUILDERS OF THOSE MODELS WILL MAKE THE CHASES. IT'S DISAPPOINTING THE REVIEWERS OF THOSE PAPERS DIDN'T CONSIDER MORE REALISTIC SCENARIOS RATHER THAN OPTIMAL SCENARIOS. YOU KNOW AT THE TIME THOSE PAPERS WERE PUBLISHED THERE WAS VERY LITTLE USE OF ONCO TYPE DX TO UNDERSTAND THE PERFORMANCE IN THE REAL WORLD BUT WHAT WE LEARNED IS THAT PHYSICIANS HAVE A SENSE OF WHO NEEDS AND WHO DOESN'T NEED THESE TESTS, OR WHO NEEDS AND DON'T NEED CHEMO THERAPY, SO THE INCREMENT AT VALUE OF THE TEST ON TOP OF THAT DOESN'T PROVIDE AS MUCH AS THE TEST OPERATORS WOULD HAVE LIKED AND THAT PATIENTS HAVE THEIR OWN PREFERENCES IN TERMS OF THERAPY AND NOT CHEMO THERAPY. THEY AREN'T ALWAYS DOING WHAT THE TEST SAID. SO COMBINE THOSE FACTORS AND IT'S JUST A LESS VALUE PROPOSITION. >> I THINK OFTEN VERY FEW THINGS ARE ACTUALLY OPERATED UNDER IDEAL CIRCUMSTANCES. AND SO, YOU KNOW IT WAS A STARTING POINT WHEN IT WAS DONE. LET CONFLICTS OF INTEREST WERE DECLARED IN THOSE ARTICLES IN TERMS OF THE SOURCES OF THE FUNDING. I WOULD HAVE BEEN LOATHE TO PUBLISH THEM OR WOULD HAVE BEEN A BIT MORE CRITICAL HAD I BEEN THE REVIEWER, BUT YOU KNOW, THERE ARE NO FATAL FLAWS IN THEIR APPROACHES EXCEPT THAT IT WAS AN UNREALISTICALLY OPTIMISTIC SITUATION AND I THINK IN GENERAL, WITH ADOPTION AND DIFFUSION OF TECHNOLOGY, IT'S NOT A SMOOTH LINEAR RELATIONSHIP AND I THINK WE'VE DEMONSTRATED THERE ARE A LOT OF FACTORS THAT GO INTO IT AND SO THE ULTIMATE SOFT PER QUALITY AS A MEASURE OF VALUE IS GOING TO BE DIFFERENT. >> DR. NORMAN? >> YES, I WANT TO ASK 1 OF THE DAVIDINGS A--DAVIDS A QUESTION, SO WHICH 1. SO DAVID KENT, I ENJOYED YOUR PRESENTATION. I HAD A QUESTION FOR YOU. YOU TALK A LOT ABOUT SCALE OF THE OUTCOME WHERE YOU'RE MEASURING THE OOSKTIVENESS OR EFFICACY. YOU KNOW LOOKING AT ABSOLUTE REDUCTIONS OR ABSOLUTE BENEFIT AND I WHOLE HEARTEDLY THINK THAT'S WHAT THE PEASHT'S GOING TO UNDERSTAND. SO I'M WONDERING ARE YOU SOMEONE THAT WOULD RECOMMEND THAT CLINICAL TRIALS USE THESE RATES RATHER THAN RELATIVE RISKS WHICH THEN OF COURSE INCREASES SAMPLES. I MEAN I WANT TO GET YOUR TAKE ON HOW WOULD YOU CHANGE CLINICAL TRIALS OR GIVEN--GIVEN THAT YOU SORT OF SHOW SOME VERY BIG DIFFERENCES, I THINK IN TERMS OF OUTCOMES AND BENEFIT. >> YEAH, SO I THINK THAT--I THINK THAT WE DEFINITELY NEED TO CHANGE THE WAY CLINICAL TRIALS ARE REPORTED SO THAT WE COULD UNDERSTAND THE RESULTS IN THE TERMS THAT LEVERAGE INTO CLINICAL DECISION MAKING. AS BEST--AS INFORMATIVELY AS POSSIBLE AND CURRENTLY THEY DON'T DO THAT. BUT I THINK WE COULD ALSO DECOUPLE THE METRIC IN WHICH THE RESULTS ARE REPORTED FROM HOW THEY'RE ANALYZED. SO, YOU KNOW I DON'T NECESSARILY THINK THAT YOU KNOW THE PRIMARY ANALYSIS FOR THE MAIN EFFECT OF ALL TRIALS SHOULD CHANGE BUT READING THAT AS A DOCTOR OR AS A PATIENT WHEN YOU HAVE A PATIENT WITH VERY SPECIFIC CHARACTERISTICS, YOU REALLY DON'T--YOU CAN'T TELL. IT'S TOTALLY OBSCURED IN THE MAIN TRIAL RESULT HOW MUCH BENEFIT OR WHAT THE CONDITIONAL ABSOLUTE RISK REDUCTION IS FOR YOUR PATIENT AND I MEAN, OBVIOUSLY THAT'S A HARD THING TO COME BY, SO THAT JUST DOING A SIMPLE RISK AND PRECEPTING RESULTS IN THE SIMPLE RISK STRATIFIED WAY WILL GET US 80% THERE. >> I MEAN I KNOW YOU KNOW THIS BUT IF YOU FIND HETEROGENEITY ON 1 SCALE, IT'S NOT TRUE YOU WILL FIND THEM ON ANOTHER SCALE AND MOST STUDIES ARE POWERED NOT TO DO THAT. >> YEAH, NO, MY--I ANTICIPATE THAT. YOU KNOW WHEN PEOPLE SAY HETEROGENEOUS O GENERATED AITY OF TREATMENT EFFECT, THEY OFTEN MEAN THAT SOME P-VALUE FOR INTERACTION WILL BE STATISTICALLY SIGNIFICANT AND IT'LL GENERALLY BE UNDERPOWERED SO IF YOU'RE TRIAL AND POWERED AT 80% FOR THE MAIN EFFECT, THEN, YOU KNOW FOR A SIMILARLY SIZED INTERACTION, YOU WILL NEED 4 TIMES THE MEMBER OF PATIENTS IN THE TRIAL TO ADEQUATELY POWER IF ARE THAT IRPT--INTERACTION. SO OFTEN IT COULD BE THAT WE WORK ON THE ASSUMPTION THAT THE PROPORTIONAL RISK REDUCTION WILL BE SIMILAR ACROSS ALL THE RISK GROUPS. I SEE YOU'RE SHAKING YOUR HEAD, I'M TOTALLY WITH YOU ON THAT. I DON'T THINK IT'S A SAFE ASSUMPTION. >> [INDISCERNIBLE] HAS A QUESTION OR COMMENT? >> OH. >> SORRY. >> I THINK PROBABLY THE LAST PART OF MY THOUGHT WASN'T SO IMPORTANT ANYWAY. [LAUGHTER] >> I JUST HAD A COMMENT THAT OCCURRED TO ME FROM SEVERAL OF THE PRESENTATIONS THAT WHEN WE THINK OF POLICY EMPLICATIONS, OR WHAT--WHAT'S GOING TO COME OUT THAT WE ALWAYS GR FOR THINGS LIKE VALUE BASED INSURANCE DESIGN OR OTHER WAYINGS OF CHANGING--WAYS OF CHANGING THE REIMBURSEMENT SYSTEM BUT I WAS REALLY STRUCK AT HOW DIFFERENT THE RATES OF THE ONCA TYPE TESTING WERE ACROSS KAISER SITES THAT KAISER MUST JUST BE COMPLETELY UNIFORMED FOR THE--FOR WHAT PEOPLE ARE BEING PAID AND HOW MUCH IT'S COSTING AND WHAT IT'S COSTING THE PATIENT AND YET, YOU END UP WITH, IT SEEMS SEVERAL YEARS AFTER IT'S INTRODUCED. IT'S A FAST VARIATION SO IT STRIKES ME WE NEED A WIDER SET OF POLICY TOOLS TO THINK ABOUT. >> IT DOESN'T SURPRISE ME. [LAUGHTER] >> I WANTED TO MAKE 2 QUICK COMMENTS ABOUT THE NATIONAL LUNG SCREENING TRIAL. WHEN IT WAS INITIATED, WHICH IS NOW ALMOST 15 YEARS AGO, IT WAS VERY CONTROVERSIAL WHETHER IT SHOULD BE DONE AND THE COST OF THE TRIAL WHEN IT FINISHED IN 2010 WAS 250 MILLION. AND I CAN TELL YOU AS A POTENTIAL FUNDER OF TRIALS. IF WE THINK ABOUT THE INCREASED PURCHASING POWER, YOU KNOW OF THE COST OF DOING THIS TODAY, IT REALLY IS SOMEWHAT DAUNTING TO THINK ABOUT THOSE KINDS THINGS AND THE SECOND THING IS I WANT TO COMPLIICATE YOUR LIVES FURTHER AND TO SUGGEST THAT WITH NOW, LUNG SCREENING BEING STANDARD OF CARE ALONG WITH THAT IS SMOKING CESSATION, SO ANOTHER IMPORTANT PARAMETER IS WHO ARE YOU GOING TO BE TESTING IN THIS AND HOW EFFECTIVE WILL YOU BE FOR SMOKING CESSATION AND IF YOU WEREN'T DOING THE TESTING, WHAT WOULD BE EFFECTIVENESS OF THE SMOKING CESSATION BE? I CAN TELL YOU ANECDOTALY THE PEOPLE ACROSS THE STREET AT THE [INDISCERNIBLE] CANCER CENTER ARE REPORTING THAT OVER 30% OF THEIR PATIENTS ARE STOPPING SMOKING WHO ARE CURRENT SMOKERS IN THEIR TRIALS ARE, ARE TOPPING SMOKING FOR AT LEAST A YEAR WHICH WOULD HAVE ANOTHER--YOU KNOW WE PREDICTED TO HAVE IMPACT ON EFFECT EVALUATION PROCESSIVENESS. >> YEAH, THAT'S ACTUALLY--THERE ARE TRIALS CURRENTLY BEING FUNDED OF CESS CISION IN LUNG SCREENING--SECESSION IN LUNG CARE SCREENINGS GIVEN GUIDELINES AND ALL GRANTEES ARE ASSESSING KUOF THES IN SOME WAY AND LUNG CYSNI C ARE TAKING THOSE RESULTS TO LOOK AT THE COST EFFECTIVENESS AND THE COST OF SMOKING CESSATION AND SMOKING RELATED DISEASES AND DEATHS AND LUNG CANCER DEATHS SO THAT'S AN EASY 1 TO GET AN ANSWER TO. >> BUT IT'S NOT JUST AN EASY 1 TO GET AN ANSWER TO, IT IS REALLY PHASE 4 AND POTENTIALLY CHANGING THE OUTCOMES QUALITIES OR HOWEVER YOU WANT TO DEFINE THEM FOR COST EFFECTIVENESS. >> THE ONLY THINK THIS I WOULD ADD IS PROBABLY DOES IT IN MULTIPLE DIFFERENT WAYS AND THE WAYS IN WHICH IT DOESN'T MIGHT BE COMPENSATORY SO IT MIGHT SOMEHOW LOWER THE RISK OF CANCER WHICH MIGHT LOWER THE BENEFIT OF SCREENINGOT 1 HAND, BUT ON THE OTHER HAND IT MIGHT RAISE THE LIFE EXPECTANCY FUNCTION, RAISE THE--RAISE THE UTILITY OF EACH ADDITIONAL YEAR, WHICH WOULD, YOU KNOW LOWER THE COST EFFECTIVENESS. SO, YOU KNOW IT'S--IT'S PROBABLY A VERY COMPLICATED TECHNOLOGY TRANSFER AND SPECIFICS WILL MATTER GREATLY, BUT YOU KNOW IT COULD BE THAT IT WASHES OUT IN THE END, SOMETHING LIKE THAT. >> SORRY. >> --COMMENT. SO I STILL SEE PATIENTS AND I RUN A CLIPICAL UNIT AND I--WE'RE DOING WORK ON PROSPECTIVE GENETIC TESTING, DONE STUFF ON WARFORIN:ICAL GENETICS AND THE PHYSICIAN IN ME IS IN CONFLICT WITH THE SCIENTIST PART OF ME BECAUSE I LOOK AT THE CLINICAL STUDIES AND SEE THE ISSUES THAT COME UP IN IMPLEMENTING THEM AND THERE FEELS TO ME A VERY LARGE CASM BETWEEN WHAT THE CLINICAL TRIALS SHOW AND THE REALITY OF WHAT I SPECT TO HELP AND THEY'RE VERY GOOD SCIENTIFICALLY DEFENSIBLE REASONS WHY I THINK THEY'RE SO DISCREPE ANT RELATED TO SUBTLE PATIENT SELECTION, IMPLEMENTATION ISSUES AND A WHOLE LOT OF LITTLER THINS AND THAT DOESN'T MAKE ME ANY MORE NOT TO DO THE RESEARCH BUT IT DOES MAKE ME WANT TO DO ANOTHER TYPE OF RESEARCH WHICH IS STUDIES OF IMPLEMENTING TECHNOLOGY EXPTION REPLICATION CLINICAL TRIALS. I WILL SAY I'VE RECENTLY BEEN DOING RESEARCH IN A TOTALLY DIFFERENT SPACE THAT'S BROUGHT ME IN CONTACT WITH THE ARNOLD FOUNDATION WHICH IS LASER-FOCUSED ON REPLICATION STUDIES AND I REALLY THINK THERE'S A LOT OF VALUE IN THAT AND THAT THERE'S AN OPPORTUNITY TO DO MUCH MORE OF IT IN THE BIOLOGICAL SCIENCES, PARTICULARLY AROUND ISSUES WHERE CLINICAL IMPLEMENTATION IS IMPORTANT. IT WOULD BE AN AREA THAT WOULD TOUCH ON A LOT OF ISSUES THAT ARE YOU KNOW OF GREAT RELEVANCE AROUND THE TABLE. IT WOULD CHALLENGE HOW WE THINK ABOUT SIGNIFICANCE AND INNOVATION AND OTHER ISSUES LIKE THAT THAT NIH HAS TRADITIONALLY VIEWED IN THE SAME WAY. BUT I REALLY THINK IT'S IMPORTANT BECAUSE I'M NOT CONVINCE THAD WE'RE GOING TO BE ABLE TO ACTUALLY GENERATE THE RESULTS AND GENERATED FITS FROM A LOT OF INTERVENTIONS THAT WE SORT OF THINK AND HOPE WE WILL BASED ON THESE TRIALS AND I THINK THAT WOULD BE REALLY GREAT. >> DR. MANDELBLATT, LAST COMMENT, IT'S JUST THAT WE'RE GETTING LATE. NOT FOR ANY OTHER REASON. >> THIS A QUICK COMMENT, I WILL FOLLOW UP ON THE CLENICAL TRIALS ISSUE, I THINK THE GOALS OF CLINICAL TRIALS AND PRODUCING HAZARD RATIOS AND DISSEMINATION INTO CLINICAL PRACTICES AND CLINICAL DECISION MAKING IS BASED ON DIFFERENT DATA AND I THINK TO THE EXTENT THAT CLINICAL TRIAL DESIGN COULD INCORPORATE OUTCOMES IT WOULD BE MORE RELEVANT. WHEN YOU--WHEN YOU HAVE AN END POINT THAT IS ANY FIRST LOCAL REGIONAL OR DISTANT RECURRENCE, THOSE DON'T MEAN THE SAME THING AT ALL TO ANYBODY. AND THEY'RE DONE TO GET POWER AND TO BE ABLE TO MOVE FORWARD WITH THE TRIAL. BUT THERE ARE A LOT OF ISSUE IN EMPOWERING, ON QUALITIES, ON ABSOLUTE BENEFITS, DOING THE SUBGROUPS, ALL OF THE TYPES OF THINGS SO THAT YOU COULD FEED INTO CLINICAL DECISION TOOLS AND IT WOULD ACTUALLY BE HELPFUL BECAUSE, THERE'S A DISCONNECT IN HOW WE USE EVIDENCE FOR A VARIETY OF REASONS. >> ALL RIGHT. THANK YOU. JOHN, DID YOU WANT TO SAY? >> JUST 1 ANNOUNCEMENT. DOUG MENTIONED AT THE OUTSET THE WORK ON THE HPV VACCINE AND HE WAS AWARDED THE LASKER PRIZE THIS YEAR WHICH IS--[ APPLAUSE ] ] FOR THAT LIFE SAVING WORK AND WE'RE ALL PLEASED TO BE HIS COLLEAGUES HERE. SECOND ANNOUNCEMENT IS THAT THAT CAFETERIA IS CLOSED NOW, BUT THERE IS A SNACK BAR ON FIRST FLOOR. >> SO NOW WE HAVE A BREAK BUT WE ALSO HAVE DEMONSTRATION OF THE DIFFERENT RESOURCES THAT WERE CREATED THROUGH THE PROGRAM AND THE FIRST 2 JOHN SKINNER AND JOHN GRAVES WILL BE IN ROOM 5 ON THIS FIRST FLOOR AND THE SECOND JOSH WILL BE IN ROOM 8. SO CONTINUE CONVERSATIONS OR COME SEE DATA OR GO GET COFFEE WE WILL RECONVENE AT 3:45. >> WELL WE'RE COMING TO THE HOME STRETCH FOR THIS AFTERNOON AND THE PLANNERS HAVE BEEN VERY THOUGHTFUL IN PUTTING THINGS TLGT, PLANNING THIS FINAL PANEL AS KIND OF THE CHERRY ON THE ICING OF THIS CAKE, WONDERFUL INFORMATION ABOUT WHAT'S GOING ON FROM THE HEALTH ECONOMICS COMMON FUND INITIATIVE. I'M MARIE BERNARD I'M THE DEPUTY DIRECTOROT NATIONAL EN--STRATEGIESITUTE OF AGING AND I HAVE THE PRIVILEGE OF MODERATING THIS SESSION. SOMEONE DID ASK ME WHY IS AGING INVOLVED OR INTERESTED IN THIS? AND THERE ARE SEVERAL YEANS. FIRST OF ALL JOHN HAGA, WONDERFUL HEALTH ECONOMIST IS A WONDERFUL LEADER OF THIS EFFORT AND DR. RICHARD HODES IS THE CO-LEAD AT THE DIRECTOR LEVEL. BUT REALLY IT'S WHERE HEALTH ECONOMICS HAVE BEEN DEMONSTRATED TO BE HELPFUL. WE HAVE DIABETES PREVENTION PROGRAM THAT WAS DONE YEARS AGO THAT WAS DEMONSTRATED THERE WAS A DIFFERENCE IN OUTCOMES IN OLDER ADULTS VERSUS YOUNGER POPULATIONS AND IT WAS A BEHAVIORIALLY BASED SORT OF DIFFERENCE. WE HAVE MULTIPLE ILLNESSES IN THE OLDER SEGMENT OF THE POPULATION THAT MEDE MODIFICATION AND MANY OF THE TECHNOLOGY TRANSFERS HAVE BEEN PRECEPTED HERE ARE THINGS THAT ARE APPLICABLE TO THE OLDER SEGMENT OF THE POPULATION SO LEARNING HOW TO USE YOUR KNOWLEDGE AND METHODOLOGY TO HELP PEOPLE TO HAVE BETTER HEALTH OUTCOMES WHEN THEY'RE OLDER IS REALLY IMPORTANT. WHEN THEY'RE MIDDLE AGENTED AND YOUNGER SO YOU START AGING FROM THE TIME YOU COME OUT OF THE WOMB. SO WE'RE VERY INVESTED IN THIS. FOR THIS PANEL, YOU GET A CHANCE TO HEAR A FEDERAL PERSPECTIVE ON HEALTH ECONOMICS, PRIORITIES IN HEALTH ECONOMICS FROM MULTIPLE HHS ENTITIES, THE FIRST PRESENTATION IS GOING TO BE BY DR. SHARON ARNOLD AND SHE IS LEADING HER AGENCY'S EFFORTS TO DEVELOP KNOWLEDGE, TOOL, DATA TO IMPROVE HEALTHCARE SYSTEMS. SHE IN THE PAST DIRECTED THE PATIENT POLICY AND FINANCIAL MANAGEMENT GROUP FOR THE CENTERS FOR MEDICARE AND MEDICAID SERVICES. SHE LED THE DEVELOPMENT OF PREMIUM STABILIZATION PROGRAMS FOR PRIVATE INSURANCE UNDER THE PATIENT PROTECTION AND AFFORDABLE CARE ACT. SO CLEARLY AN EXPERT AND WE WILL BE INTERESTED IN HEARING WHAT SHE HAS TO SAY FROM THE ARC PERSPECTIVE. >> THANK YOU VERY MUCH AND I'M PLEASED TO BE HERE. CAN YOU HEAR ME? SO I WANTED TO TALK THROUGH WHAT ARC'S PRIORITIES AND ARE TALK ABOUT OUR INTRA MURAL AND EXTRA MURAL RESEARCH PRIORITIES. SO, FIRST OF ALL OUR BROAD RESEARCH PRIORITIES ARE TO IMPROVE HEALTHCARE QUALITY, MAKE HEALTHCARE SAFER, INCREASE AFFORDABILITY, COST TRANSPARENCY WHICH ARE BIG GOALS FOR A VERY, VERY SMALL AGENCY. WE'VE ALSO BEEN FOCUSING ON KIND OF BRINGING ALL THOSE TECHNOLOGY TECHNOLOG Y TECHNOLOGY TRANSFERS TOGETHER IN SUPPORTING ORGANIZATIONS TO BECOME LEARNING HEALTH ORGANIZATION WHICH IS IS A VERY POPULAR TERM NOW. WHAT DOES THAT MEAN? THAT MEANS HELPING ORGANIZATIONS KIND OF USE THEIR INTERNAL AND EXTERNAL DATA TO IMPROVE QUALITY AND SAFETY AND BE CONSISTENT IF THAT PRACTICE AND THE DEVELOPMENT OF PREDICTIVE ANALYTICS. SO THOSE ARE OUR GENERAL RESEARCH PRIORITIES. WE HAVE SOME PARTICULAR INTERESTS IN TERMS OF HEALTH ECONOMICS BOTHOT DEMAND SIDE AND THE SUPPLY SIDE, OF COURSE WE DONE A LOT OF WORK AND ARE INTERESTED IN CONTINUING TO SUPPORT WORK ON INSURANCE COVERAGE, PREMIUMS, THE IMPACT OF COST SHARING, UNDERIN. SURANCE AND DISPARITIES, AND CERTAINLY AS WE CONSIDER CHANGES IN HOW WE PAY FOR INSURANCE AND PROVIDE INSURANCE WILL BE INTERESTED IN TRACKING CHANGES TO THAT AS WELL. THE SUPPLY SIDE WE'RE VERY INTERESTED IN PROVIDER INCENTIVES AND THE IMPACT OF THE MARKET AND ORGANIZATIONAL STRUCTURES ON OUTCOMES AND PRICES, SYSTEM CONSOLIDATION, IMPACT OF REGULATION, MEDICAL MALPRACTICE, PUBLIC REPORTING AND ACUTE AND SETTINGS THAT VARY FROM ACUTE TO LONG-TERM CARE. AND WE FOCUS ON ALL COMMERCIAL AND PUBLIC PROGRAMS. SO WE IS A LONG HISTORY OF INTRA MURAL RESEARCH AT ARC. INCLUDING STUDYING INSURANCE COVERAGE. SWEE HAVE BEEN--WE HAVE BEEN STUDYING TRENDS AND INSURANCE COVERAGE, PREMIUMS, ELIGIBILITY, WE PROVIDED EARLY AND ONGOING ESTIMATES OF THE UNINSURED AND UNDERINSURED AND RESEARCHERS AT ARC WERE THE FIRST RESEARCHERS TO PROVIDE IMPERICAL EVIDENCE OF JOB LOCK, WE ALSO LOOK AT MEDICAL EXPENDITURES AND LOOK AT THE CONCENTRATION OF EXPENDITURES BY CONDITION AND ALSO THE--YOU KNOW, CONCENTRATION OF EXPENDITURES IN TERMS OF POPULATION AND ALSO, EXPENDITURES BY CONDITION, ESPECIALLY CHRONIC CONDIPGZS AND WE'RE ALSO INTERESTED IN THE DETERMINANTS OF USE. SO THIS IS REALLY HARD TO SEE BUT THIS IS ALL THE FUNDING OPPORTUNITY ANNOUNCEMENTS WE HAVE. SO WE HAVE, YOU KNOW ALPHABET SOUP OF THE Rs HERE. AND WE DO PROVIDE GRANT FUNDING FOR EXTERNAL RESEARCHERS. WE ALSO HAVE ANOTHER--A NUMBER OF TRAINING OPPORTUNITIES BOTH PREDOCTORAL STUDENTIAL AND POST DOCTORAL AS WELL AS EARLY CAREER, I'LL MAKE NOTE THAT WE HAVE A NEW FUNDING OPPORTUNITY FOR TRAINING PROGRAM THAT IS A VERYIATION ON THE T32 EN--STRATEGIES TUITIONAL HEALTH SERVICE TRAINING PROGRAM. IN 1 IS DESIGNED FOR DEVELOPING HEALTH SERVICES RESEARCHERS THAT ARE EMBEDDED IN HEALTH SYSTEMS TO TRY AND DRIVE LEARNING AT THE HEALTH SYSTEM LEVEL SO WE'RE REALLY LOOKING FOR PARTNERSHIPS BETWEEN ACADEMIC, TRADITIONAL TRAINING INNS TUITIONS AND HEALTH SYSTEMS TO TRY AND TRAIN THIS NEW GENERATION OF HEALTH SERVICES RESEARCHERS THAT I THINK ARE NOT NECESSRILY WELL REPRESENTED BY THE TRADITIONAL TRAINING PROGRAMS. SO, WE FUND A FAIR AMOUNT OF EXTRA MURAL RESEARCH IN 2016 WE FUNDED ABOUT A HUNDRED MILLION DOLLARS IN GRANT AWARDS; VERY SMALL COMPARED TO NIH BUT BIG CONSIDERING OUR BUDGET. ABOUT HALF OF THAT WAS GENERAL INVESTIGATOR INITIATED SO NONDIRECTED AND ABOUT HALF OF THAT WAS DIRECTED AND I HAVE SOME SAMPLE GRANT AWARD TITLES THERE. SO WE--YOU KNOW WE FUND THE RANGE OF TOPICS THAT I DISCUSSED. AND SO IN ADDITION TO GRANT FUNDING, WE OFFER DATA. AND THE 2 I THINK BEST KNOWN DATA RESOURCES THAT WE HAVE AVAILABLE ARE THE MEPS DATA, THE MEDICAL EXPENTINE REGIMEN EDUCATIONALLURE PANEL AND THAT'S THE BEST USE OF COST AND HEALTHCARE COVERAGE IN THE UNITED STATES, AND WE HAVE JUST LEARNED FROM HEALTH AFFAIRS RECENTLY THAT IT'S THE NUMBER 1 SOURCE OF DATA FOR PAPERS ISSUES PUBLISHED IN HEALTH AFFAIRS AND WE ALSO HAVE THE HEALTHCARE COST AND UTILIZATION PROJECT DATABASE. THIS IS A COLLECTION OF ALL PAYOR AND COUNTER LEVEL HOSPITAL AND HOSPITAL-RELATED CARE IN THE UNITED STATES FROM 47 SPHAITS AND THE DISTRICT OF COLUMBIA. IT'S A VOLUNTARY RELATIONSHIP BETWEEN US AND STATE AGENCIES AND HOSPITAL ASSOCIATIONS. BUT IT'S AN INCREDIBLY VALUABLE RESOURCE BECAUSE AS I SAID IT'S ALL PAYOR. SO IT INCLUDES MEDICARE FEE FOR SERVICE, MEDICARE ADVANTAGE, MEDICAID, UNINSURED, DATA AND IT'S AT THE ENCOUNTER LEVEL SO A REALLY GREAT RESOURCE. WE ARE ALSO IN THE PROCESS OF DEVELOPING A COMPENDIUM OF HEALTH SYSTEMS AND WE ACTUALLY RELEASED THE FIRST PHASE OF THAT LAST WEEK. SO WE ARE PULLING TOGETHER A LISTING OF ALL THE HEALTH SYSTEMS IN THE COUNTRY AND IDENTIFYING HOSPITALS AND ASSOCIATED PHYSICIANS AND WE'VE GOT SOME DESCRIPTIVE INFORMATION ASSOCIATED WITH THAT AND WE HOPE TO BE ENHANCING THAT OVERTIME. SO ANOTHER GREAT RESOURCE FOR FOLKS TO CONSIDER USING. SO THAT'S IT. HAPPY TO TAKE ANY QUESTIONS IF FOLKS HAVE ANY? SURE. >> HCUP IS AN AMAZING DATA SET AND IT'S LIKE CHARGED BY THE STATE SO, I THINK, RIGHT? SO TO COLLECT ALL THE PIECES IS REALLY EXPENSIVE. >> SO WHAT WE DO FOR HCUP IS WE COLLECT STATE DATABASES AND WE MAKE STATE-LEVEL DATABASES AVAILABLE AND THEN WE ALSO PROVIDE SOME NATIONAL DATABASES WHICH ARE A SAMPLE BUT BESIGNED TO BE A REPRESENTATIVE SAMPLE SO RESEARCHERS CAN PURCHASE THE NATIONAL DATABASES FROM US AND WE CHARGE A VERY, VERY SMALL FEE FOR THAT IT'S REALLY JUST A SHIPPING AND HANDLING FEE ALTHOUGH WE ARE GIVEN OUR BUDGET CONSTRAINTS WE ARE REASSESSING THAT AND BUT IF YOU WANT TO PURCHASE THE STATE DATABASES, WE ARE JUST THE STEWARD OF THOSE, THEY BELONG TO THE STATES AND THE STATES DETERMINE THE PRICES FOR THOSE. SO WE JUST KIND OF HELP THEM OUT BY HANDLING THAT FOR THEM BUT THAT IS A STATE DECISION ABOUT HOW MUCH TO CHARGE. >> I SEE WE HAVE ANOTHER QUESTION FROM DR. VOLK AND THEN WE'LL TRANSITION TO-- >> HI, BEHE UNDERSTAND YOU. THANKS FOR THE PRESENTATION, OVER THE YEARS THE AMOUNT AND GRANTS FUNDED AT HRQ HAVE VARIED A BIT AND CAN YOU GIVE US A SENSE IN THE MOST RECENT YEARS THE 147 MILLION, WHAT PERCENTAGE OF APPLICANTS ARE TYPICALLY FUNDED AND WHAT YOUR SENSE OF HOW THAT'S GOING TO--I KNOW IT'S& HARD TO PREDICT THE FUTURE BUT DO YOU HAVE A SENSE OF LIKE HOW THAT'S LIKELY TO TREND OVER TIME GIVEN ALL THE OTHER DEMANDS ON YOUR LIMITED BUDGET? >> SO ACTUALLY SINCE I'VE BEEN AT ARC WHICH IS ABOUT 3 AND HALF YEARS THE INVESTIGATOR INITIATED POT HAS BEEN PRETTY STUDY AND THAT'S BECAUSE CONGRESS SEGGREGATES THAT OUT AND DIRECTS US TO HOLD THAT OUT FOR INVESTIGATOR-INITIATED WORK THAT IS NOT DIRECTED BY US AND WE HAVE SEEN THAT CONGRESS LIKES TO KIND OF KEEP THAT POT PRETTY STEADY. SO AS OUR BUDGET HAS BEEN SHRINKING THE INVESTIGATOR INITIATED POT HAS BEEN PRETTY STEADY. SO THAT'S THE GOOD NEWS FOR YOU. ALL OUR GRAND FUNDING MAY HAVE DECREASED BUT THE INITIATOR HAS BEEN HELD CONSTANT. I THINK WE HAVE A FUND RATE OF ABOUT 30% IN THE UPPER 20S. IT'S QUITE GOOD AND WHAT WE FOUND IS THAT IN TIMES OF PERCEIVEED LEENESS OF OUR BUDGET, THE NUMBER OF APPLICATION RATES DROP AND THE FUNDING GOES UP SO WHEN YOU HEAR THERE'S A POSSIBILITY OF A BUDGET CUT, SUBMIT YOUR PROPOSAL BECAUSE WOULD HAVE LESS COMPETITION BUT THAT'S THE WAY I'VE SEEN THINGS HAPPEN OVER THE LAST FEW YEARS. >> THANK YOU. A BUNCH OF ECONOMISTS HERE, I AM SURE WE HEARD THAT LOUD AND CLEAR. >> WE WILL TRANSITION TO OUR NEXT SPEAKER AND THEN GENERAL DISCUSSION AFTER ALL 3 SPEAKERS, DR. JOHN GRAHAM IS ACCOUNTAING ASSISTANT SECRETARY, DEPUTY PRINCIPLE SECRETARY IN PLANNING FOR HHS, HE LED RESEARCH ON TOPICS INCLUDING PAYMENT REFORM, REGULATIONAL DRUGS HEALTH INFORMATION TECHNOLOGY AND COMPARING INTERNATIONAL HEALTH SYSTEMS. HE'S BEEN INFLUENTIAL IN BOTH CONGRESS AND STATE CAPITALS AND HAS WRITTEN REGULARLY FOR THE MEDIA, INCLUDING WALL STREET JOURNAL, WASHINGTON POST AND FORBS WHERE HE JOINED THE ADMENSTRUATION IN MARCH, HE HAS HIS MBA FROM LONDON SCHOOL OF BUSINESS, TRAIN INDEED GRADUATE SCHOOL IN ECONOMICS AND WITH ALL OF 6 MONTHS EXPERIENCE IN THE FEDERAL GOVERNMENT WILL TALK TO US FROM THE ASPE PERSPECTIVE. >> THANK YOU VERY MUCH IT'S GREAT TO BE AROUND SO MANY GREAT ECONOMISTS. I HAVE READ MANY OF THEM, I WILL NOT CALL ANYONE BY NAME BECAUSE THEN FIELD FUNCTIONS DON'T CALL OUT SOMEONE THERE, ENH, BUT AS I READ THE ECONOMIC BOOKS AND ARTICLES AND MANY OF THE FOLKS AROUND THIS TABLE ARE WELL REPRESENTED SO I'M VERY HUMBLED TO JOIN YOU ALL. I'M NOT REALLY ANNEX CONMIST, I GOT AN BA AND MBA IN ECONMETRICS BUT I'VE BEEN CALLED AN ECONOMIST AND SO SINCE I CAME FROM CANADA I HAD TO HAVE A LABEL, AND SO ECONOMIST --I WAS A LABEL, SO IT INTERESTING WHEN MR. HAGA SAID COME DOWN AND SPEAK, I WAS HONORED I DIDN'T ASK WHY AND I DON'T ALWAYS ASK WHY BUT SOMETIMES I ASK WHY, BECAUSE IN THE OLD DAYS OF COURSE, WHEN YOU THINK THERE'S AN HONORARYIUM ASTACHED WE THINK THE RESERVATION PRICE SO YOU WANT TO FIGURE OUT, BUT SENTENCE I CAN'T DO THAT ANYMORE, OF COURSE THERE ARE 2 OTHER JOHN GRAMS WHO YOU MIGHT KNOW OIRA DOWN THE IN THE BUSH ADMINISTRATION AND THERE'S A JOHN GRAHAM AT DUKE, SO IF YOU HAVE CONFUSED ME WITH 1 OF THEM, I WANT TO MINIMIZE EMBARRASSMENT QUICKLY. >> JOHN, WE KNEW WHOA WERE. >> BUT THANKS VERY MUCH. I DIDN'T ASK WHY. I HAVE ASKED ABOUT TALKING TO THE ABOUT HEALTH ECONOMICS RESEARCH PRIORITIESA THE HHS. I WAS INTERESTED TO LEARN AT NIH HEALTH ECONOMIC SYSTEM A REGULATED TERM. THERE'S ACTUALLY A NOTICE, FROM NOVEMBER 2015, NOW THAT'S PREVIOUS MANAGEMENT OF COURSE, BUT, YOU KNOW, WHICH STATES HEALTH ECONOMICS IS HOW SCARCE RESOURCES ARE ALLOCATED AMONG ALTERNATIVE RESOURCES FOR SICKNESS, PROMOTION, MAINTENANCE AND IMPROVEMENT OF HEALTHCARE, COST, BENEFITS AND HOW IT'S DISTRIBUTED AMONG INDIVIDUALS IN SOCIETY. SO I DON'T KNOW HOW THAT WENT THROUGH THE REGULATORY PROCESS BUT IT SEEMS LIKE A FAIR ENOUGH DEFINITION TO ME AND OF COURSE AT NIH I THEY DON'T COVER ALL THAT STUFF. BUT AT HHS, WE COVER A BROAD--WE HAVE A BROAD INTEREST IN HEALTH ECONOMICS BUT I THINK I WILL HAVE TO KIND OF CHALLENGE THE FRAMING OF THE--OF WHAT I'VE BEEN INVITED TO SPEAK WITH THE DEPARTMENT OF HEALTH AND HUMAN SERVICES SO WE INTEREST IN ECONOMIC RESEARCH BEYOND THE HEALTHCARE PORTFOLIO. I WILL DUSES THAT A BIT. LET ME THROW OUT A FEW THINGINGS HERE AND AGAIN I AM TRYING TO BE CAUTIOUS, BECAUSE IT'S MY FIRST GOVERNMENT JOB SO I'M STILL LEARNING HOW TO GOVERN MYSELF AND MAKE STATEMENTS THAT WON'T GET THE SECRETARY IN TROUBLE AND STUFF LIKE THAT. I'M NOT AS--I KNOW SOME FOLKS MIGHT HAVE HEARD ME SPEAK PUBLISHING LICK BEFORE WHERE I THUMP THE BIBLE AND STUFF LIKE THAT YOU ABOUT I DON'T THAT ANYMORE, SOME WILL OFFER OPPORTUNITIES FOR EXTERNAL RESEARCHERS BUT SOME TO LET YOU KNOW WHAT WE'RE DOING. FIRST I'D LIKE TO MENTION IS OF COURSE, WE WERE WORKING QUITE HARD TO SATISFY 2 EXECUTIVE ORDERS ON DEREGULATION, 1370.71 REDUCING REGULATION AND CONTROLLING REGULATORY COSTS AND 13.77.7 AND DEREGULATION THERE. AND SHOW THE SHOATAL INCREMENTAL COST OF REGS NEW AND RESCINDED NOT EXCEED 0. SO, AND WE ARE TAKING THIS VERY SERIOUSLY. WE'RE GOING THROUGH REGULATIONS BACK TO I THINK THE TRUMAN PRESIDENCY AND FOR ALL I KNOW BACK TAKEN--THEY LINCOLN, I MEAN WE'RE PROWLING THROUGH EVERYTHING, ALL THE REGULATIONS THAT HAVE ACCUMULATED OVER THE YEARS AND SUBJECTING THEM TO REGULATORY IMPACT ANALYSIS, 1 OF MY COLLEAGUES--IS COLLIN HERE? INVOLVED IN THAT. AND IN ORDER TO MAKE SURE WE ACCOMPLISH THE GOALS OF THE EXECUTIVE ORDERS AND IT'S QUITE AN INTERESTING CHALLENGE. ONE OF THE THINGS SINCE I'M APPOINTED BY THE PRESIDENT AND WAS CHOSEN BY SECRETARY TRUMP, YOU WILL NOTE GET OUT OF HE WITHOUT ME MAKING COMMENDS ON THE AFFORDABLE CARE ACT. WE ARE FOCUSING EFFORTS ON THE BURDEN OF ACA, WE 2000 PAGES OF RULES, 2000 PAGES OF GUIDANCE 5000 PAGES OF PUBLIC COMEBTS AND WE'RE WORK HARD TO MAKE SURE WE DON'T OVERREGULATE. OF COURSE HHS AS A HUGE EFFECT IN THE ALLOCATION OF HEALTHCARE RESOURCES DIRECTLY OR INDIRECTLY. WE THINK AND WE THINK THIS IS GOOD ECONOMIC THINKING THAT THOSE RESOURCES ARE BETTER ALLOCATED BY PHYSICIANS AND INFORMED PATIENTS RATHER THAN THOSE OF US WHO WORK IN THE FEDERAL GOVERNMENT IN WASHINGTON D. C. AND OF COURSE WE HAVE TO BE GOOD STEWARDS OF TAX PAYOR'S RESOURCES. WE DON'T THINK WE HAVE ALL THE ANSWERS. I THINK THAT EVEN LESS SENSE I JOINED THE GOVERNMENT AND THAT IS 1 REASON WHY YOU WILL HAVE SEEN A WHILE AGO WE ISSUED A REQUEST FOR INFORMATION ON THE ACA EXCHANGES, HOW TO REDUCE THE BURDEN THERE AND JUST EARLIER THIS WEEK, I BELIEVE IT WAS, CMS ISSUED A REQUEST FOR INFORMATION ON A NEW DIRECTION FOR CMS, INNOVATION CENTER. CMMI, AFTER CAREFUL CONSIDERATION OF PUBLIC COMMENS ON PROPOSED RULES FOR SOME BUNDLED PAYMENTS, YOU KNOW THAT CMS DECIDED TO DELAY THE EFFECTIVE DATES UNTIL NEXT YEAR FOR SOME OF THE BUNDLED PAYMENTS AND YOU ALL KNOW WHAT I'M TALKING ABOUT. THIS NEW DIRECTION WHICH WILL BE DEVELOPED AFTER BROAD BASED PUBLIC INPUT, WE ARE COMMITTED TO MAKE SURE--INCORPORATES GOOD ECONOMIC THINKING SO I HOPE YOU WILL TAKE THE ADVANTAGE TO PARTICIPATE IN THE REQUEST FOR INFORMATION. WE ACTUALLY WANT TO INSURE THAT WHAT COMINGS OUT OF THAT WILL RESULT IN A PAYMENT SYSTEM THAT HAS MORE CHOICE AND COMPETITION IN THE MARKET, PROVIDER CHOICE AND INCENTIVES, PATIENT CENTERED CARE, WE'RE LOOKING FORWARDS BENEFIT DESIGN AND PRICE TRANSPARENCY. TRANSAPPARENT MODEL DESIGN AND EVALUATION OF NEW POSITION MODELS AND SMALL SCALE TESTING. MORE OPPORTUNITIES FOR GOOD ECONOMIC ANALYSIS, I THINK WILL ARISE AS STATES TAKE ADVANTAGE OF THE SECTION 115 WAIVER AUTHORITY TO ENABLE GREATER CHOICE AND BENEFIT DESIGN IN THEIR MEDICAID PROGRAMS MANY OF YOU WILL HAVE SEEN THE LET THEY'RE SECRETARY PRICE AND ADMINISTRATOR VERMA PUBLISHED IN MARCH WHICH STATED THAT STATES COULD CONSIDER GREATER ALIGNMENT BETWEEN MEDICAID DESIGN AND BENEFIT AND THE FEATURES OF COMMERCIAL HEALTH INSURANCE TO HELP WORKING AGE NONPREGNANT, NONDISABLED ADULTS PREPARE FOR PRIVATE COVERAGE. SO I THINK YOU WILL SEE STATES BE VERY INNOVATIVE IN RESPONDING TO THAT. I'M SORRY--IT'S TRAUMATIZING FOR ME TO MENTION THE AFFORDABLE CARE ACT GIVEN WHAT WE'VE BEEN THROUGH JULY AND LAST WEEK BUT AS YOU KNOW THE EFFECT OF THE ACA WENT BEYOND A SIMPLE NUMBER OF PERSONS WITH HEALTH INSURANCE, IT ALTERED THE SET OF CHOICES AVAILABLE TO CONSUMERS AND PATIENTS ELIMINATING THE MORE ECONOMICICAL INSURANCE PLANS THAT MANY CONSUMER HIS APPARENTLY PREFERRED, INCREASED TAXES WITH POSSIBLE EFFECTS ON INVENT VESTMENT INCENTIVES AND LABOR MARKET PARTICIPATION CHOICES FOR SOME WORKERS, THESE ARE ALL GOOD TOPICS FOR ECONOMIC RESEARCH AND WE WILL KEEP RESEARCHING THEM AS LONG AS WE HAVELET AFFORDABLE CARE ACT WITH US. A RESEARCH IN JULY FOUND THE PREMIUMS IN 2013, MLR, MINIMAL LOSS RATIO DATA AND THE 2017 MIDAS DATA SHOWING AVERAGE EXCHANGE PREMIUMS WERE 105% HIGHER IN THE 39 STATES USING HEALTHCARE .GOV IN 2017, THAN PREMIUMS IN 2013. THE SECRETARY'S CONCERNS REGARDING ACA ARE ROOTED IN THE EFFECTS ON CHOICE AND WE NEED TO REMEMBER THAT WHEN ADDITIONAL BENEFITS ARE MANDATED THE PRICES FOR THOSE MORE GENEROUS INSURANCE PLANS WILL BE HIGHER AND NOT ALL CONSUMERS WILL EXPERIENCE THAT AS AN IMPROVEMENT LAST SPRING CMS RELEASED INFORMATION ON--HOWEVER THESE ARE NO SUBSTITUTE FOR REPEAL AND REPLACEMENT WITH FURTHER AREY FORM. AND WE WELCOMEECONOMMIC ANALYSIS THAT WILL LEAD TO A BETTER MARKET FOR AMERICANS. NOW BEYOND HEALTH ECONOMICS WE NEED GOOD ANALYSIS OF THE THE ACA, UNEMPLOYMENT AND WAGE GROWTH. MORE ON MIER PAYING JOBS ARE INTERESTED TO THE DEPARTMENT BECAUSE ELIGIBILITY FOR MANY PROGRAMS DELIVERED BY THAT DEPARTMENT IS DEPENDENT ON HOUSEHOLD INCOME, THESE ENCLUED THE OBAMA CARE TAX CREDITS, COST SHARING REDUCTIONS, WHICH WE MAY OR MAY NOT CONTINUE DEPENDING ON MONTH TO MONTH BASIS, WE MAKE THAT DECISION, MEDICAID AND TEMPORARY ASSISTANCE FOR OTHER FAMILIES AND OTHER IMPORTANT EXAMPLES OF SUCH PROGRAMS. AS DR. PRICE MENTIONED AT THE SENATE CONFIRMATION HEARING, WE DON'T MEASURE THE PROGRAMS BY HOW MUCH MONEY WE SPEND ON THEM WE HOPE TO INSURE THE EFFECTS OF ALL THE MEANS TESTED PROGRAMS WHICH ARE DELIVERED TO BENEFICIARIES AND ARE EFFECTIVE ASK WE WANT TO UNDERSTAND ALL THE CENTER FOR EXCELLENCE ON AGINGS OF THEM. IF THEY HAVE A NEGATIVE EFFECT OR PARTICIPATION OR WAGE GROWTH, WE NEED TO INCORP RATE THEM INTO THE TOTAL COST OF THESE PROGRAMS AND OUR PROGRAM IS NOT JUST ABOUT COST, IT'S ABOUT INNOVATION SO NOW WE'RE GETTING CLOSER TO THE IN, IH FOLKS BUT WE'RE NOT THERE YET. WE'RE AT THE FDA. DR. GOTTLIEBHAS SIGNALED A NUMBER OF STEPS THAT FDA WILL SPEED THINGS TO MARKET. HE PUBLISHED AN ORPHAN DRUG MARKETTIZATION PLAN, WHICH EXPECTATIONS EMILYINATES THE 200 DRUG BACK LOG AND THE FDA AHEAD OF SCHEDULE AND THE FDA IS LOOKING AT PLACES WHERE RULES CONCERNING NEW DRUGS ARE USE INDEED WAYS THAT MAY CREATE OBSTACLING TO THE TIMELY ENTRY OF GENERIC COMPUTATION. WE WANT TO MAKE SURE OUR POLICIES ARE NOT MISUSED IN THE WAY THAT THWART CONGRESS INTENDED WHEN IT CREATED THE MODERN GENERIC DRUG WORK. WE KNOW A VIGOROUS COMPETITION HAD HELP BENEFIT PATIENTS BY LOWERING COSTS, IMPEDIMENTS PROVE MEDICINE, ALL DONE IN THE PUBLIC EYE, YES THERE'S AN RFI THAT THE FDA PUT OUT AND YOUR ECONOMIC ANALYSIS IS WELCOME. LET ME CLOSE BY MENTIONING A PROFESSIONAL PRIORITY'VE MINE. AS A VET VAN IN THE CAPITAL MARKETS IN 1 CAREER IS I WAS IN THE CAPITAL MARKETS APPROPRIATE INCENTIVES FOR NEW THERAPY WHEN IS MARKET INCENTIVES ARE NOT ADEQUATE. THIS ENCOMPASSES RARE DISEASES, OTHER THREATS TO PUBLIC HEALTH. ONE OF THE MOST EXCITING VENTURES IN WHICH WE'RE CALLED WAS THE CARB X, RESISTANT BIOFORM SUITICAL ACCELERATE WHICH COMPLETED FIRST YEAR OF IT'S CARB X IS DEDICATED TO ACCELERATING INNOVATIVE ANTIMICROBIAL RESEARCH TO ADDRESS THE URGENT THREAT TO DRUG RESISTANT BACTERIA, FUNDED BY THE NATIONAL EN--STRATEGIESITUTE OF ALLERG EXPE INFECTIOUS DISEASES, BART A, THE WELCOME TRUST, THE BROAD INSTITUTE AND OTHERS CARB X HAS AWARDED $42 MILLION TO 18 PROJECTS IN THE FIRST YEAR WITH AN ADDITIONAL $53 MILLION TO BE AWARDED IF PROJECT MILESTONES ARE MET. THE DEPARTMENT BELIEVES THAT THE FUTURE OF CARB X WILL LEAD TO FUTURE PLANS WITH MORE GOALS. THAT'S THE END OF MY PREPARED MARKS. I HATE READING FROM THOSE BUT I HAVE TO. I WILL TALK ABOUT ASPE, WE HAVE 13 AND 19 IN THE NEXT FISCAL YEAR, NONE GRANTS, ALL RESEARCH CONTRACTS AND WE ALSO SRO PPC ORI TRUST FUND MONEY, THAT WE KIND OF STAFF IT AND THEN WHICH URN IT OUT TO OTHER FOLKS. I WILL--I WILL CLOSE WITH 1 SUGGESTION. I GOT THIS FANTA FROM THE VENDING MACHINE, THIS IS BEHAVIORIAL ECONOMISTS SO THE VENDING MACHINE OUT THERE HAS 5 ROWS, THE TOP ROW IS SUGARY COKE, THE THIRD ROW IS DIET COKE, THE FOURTH ROW IS SUGARY SOCIETIA AND THE BOTTOM ROW IS WATER. AND I BET IF WE READ THE CONTRACT, THE TRIBUTER WAS NOT SUPPOSED TO STOCK IT THAT WAY. LET'S NOT TELL DR. COLLINS THAT'S THE WAY IT'S STOCKED THANK YOU VERY MUCH. [ APPLAUSE ] >> I THINK WE WILL TRANSITION TO THE NEXT SPEAKER AND OPEN UP FOR QUESTIONS AND GENERAL DISCUSSION. SO DR. MARK Mc CLELLAN, IS THE ROBERT J. MARGINAL MARGOELES, YOU CAN NOTICE HE'S NOT AT FED, AND HE WITH A 2 FER WITH HIM IN THAT HE'S PAST ADMIN SLATER FOR CMS AND PAST ADMINISTRATOR FOR FDA AND SINCE HE'S NO LONGER FED, HE CAN BE UNIMCUMBERRED. SO, DOCTOR? >> THANK YOU MARIE AND JOHN, AND I DID ACCEPT BEFORE REALLY LOOKING CLOSELY AT WHAT THE SUBJECT WAS AND THE REASON FOR THAT IS, ACTUALLY I PRETTY MUCH OWE MY CAREER TO THE SUPPORT I GOT EARLY ON FROM NIH AND THE NATIONAL INNSITUTE OF AGING AND AHCPR, NOW ARC, INCLUDING ON COLLABORATIVE PROJECTS WITH A NUMBER OF PEOPLE IN THIS ROOM. THE WORK BACK THEN, I HOPE IT WAS POLICY RELEVANT. CERTAINLY SEEMED RELEVANT TO IMPROVING HELT AND ON AT LEAST THE SECOND REASON ON HERE, AFTER I DID READ, THE REST OF THE INVITATION AND STARTED THINKING ABOUT REMARKS ON THE IMPORTANCE OF THE WORK BEING DONE WITH THE COMMON FUND FOR IMPROVING THE HEALTH OF POPULATION. IT'S MORE RELEVANT NOW THAN EVER. AND TO--TO THE POINT ABOUT BEING ABLE TO SAY WHAT I WANT. I AM NOT REPRESENTING ANYBODY'S CURRENT LIST OF PRIORITY PROJECTS SO I'M GOING TO TALK ABOUT WHAT I THINK YOU OUGHT TO BE SPENDING THEMONEY ON NOW FOR A FEW MINUTES. I WOULD LIKE TO START TO--BY THANKING EVERYBODY'S KRIEWKS TO THE MEDING, AND TO THE NIH'S MISSION WHICH IS NOT JUST ABOUT THE DEVELOPMENT OF CURES AND BETTER TREATMENT BUT AS THE TITLE OF THIS CONFERENCE EMPHASIZES IS TURNING DISCOVERY INTO HEALTH. AND IT'S TURN OUT TO BE I THINK INCREASINGLY CHALLENGING, SO GREAT DISCOVERIES, GREAT BREAKTHROUGHS, GOOD STEP'S HAPPENING AT FDA TO FURTHER ACCELERATE THE TRANSITION INTO TREATMENTS THAT ARE SAFE AND EFFECTIVE AND RELIABLE FOR PEOPLE TO USE. BUT, SIGNIFICANTLY RISING HEALTHCARE COSTS, YOU DON'T HAVE TO DO RESEARCH ON THAT. EVERY AMERICAN FAMILY IS EXPERIENCING THAT RIGHT NOW AS WELL AS THE FEDERAL GOVERNMENT. AND THANKS TO THE RESEARCH DONE HERE, SOME WORRY SOME TRENDS, I THINK IN POPULATION HEALTH, BUT ALSO MAYBE SOME HOPEFUL SIGNS ON WHAT MIGHT BE DONE ABOUT IT. SO, I THINK FIRST PRIORITY OR FIRST THING I WANT TO MENTION IS THE WORK THAT'S BEEN HIGHLIGHTED TODAY, PROVIDED SOME IMPORTANT NEW AND I THINK MAYBE EVEN STUNNING INSIGHTS ABOUT RECENT TRENDS AND CHANGES IN TRENDS IF POPULATION HEALTH OUTCOMES. WE KNOW SO MUCH HORMONE MAN WE DID BEFORE--SO MUCH MORE THAN WE DID BEFORE. I USED TO GIVE THESE TALKS EARLY IN MY CAREER ABOUT THE VALUE OF BIOMEDICAL INNOVATION, IT'S CLEARLY EVIDENCED BY THE SUBSTANTIAL REDUCTIONS IN POPULATION MORTALITY AND MORBIDITY PRETTY MUCH ACROSS ALL POPULATIONS LED BY A LOT OF RESEARCH BEING SUPPORTED BY NIH FUNDING HERE IN THE U.S. BUT REAL LYE BENEFITING THE ENTIRE WORLD AND I THINK MANY OF YOU IN THIS ROOM ARE FAMILIAR WITH WORK BY SOME, YOU KNOW, SOME NOBEL PRIZE WINNING ECONOMISTS AND OTHERS, BILL NORTHOUSE AND OTHERS, AND THE VALUE OF PRODUCTIVITY INCREASES ACTUAL AS WE COMPLAIN ABOUT RISING HEALTHCARE COSTS, THE INCREASES IN HEALTHCARE HAVE BEEN GRITTER THAN THAT IN ALL OTHER INDUSTRIES COMBINED. YOU KNOW NOT SURPRISING. IT'S NICE TO HAVE A LAPTOP AND 400 CABLE CHANNELS, I GUESS AND JET TRAVEL BUT WHAT PEOPLE REALLY ENJOY IS THE TIME TO SPEND WITH THOSE THINGS AND THEIR MOM BEING AROUND LONGER, INCREDIBLY VALUABLE,OT OTHER HAND AS YOU DISCUSSED EARLIER TODAY, I THINK WE ARE NOT CONTINUING TO SEE THOSE TRENDS AT LEAST IN ALL POPULATION GROUPS. THE U.S. STARTED TO DIVERGE FROM OTHER DEVELOP COUNTRIED AND MORTALITY IMPROVEMENTS AND MAYBE NOT SURPRISING AT ALL IS SOME LEVELING OFF OF THESE IMPROVE MENTS IN AGE SPECIFIC MORTALITY RATES PARTICULARLY FOR GROUPS LIKE MIDDLE AGED WHITES WHERE THERE'S BEEN AN INCREASE I'M SURE THERE'S INTERESTING FURTHER DISCUSSION ON THAT TODAY, I DON'T THINK WE'RE AT THE BOTTOM OF WHY THAT IS HAPPENING, I THINK WHAT WE CAN SAY FOR SURE, THIS IS A VERY WORRYSHIP TREND IN THE FACE OF WHAT SHOULD BE, YOU KNOW MORE AND MORE BIOMEDICAL PROGRESS AND IT'S A TREND THAT HAS HAD PRETTY DIRECT AND OBVIOUS POLITICAL IMPLICATIONS. THAT POPULATION IS 1 OF THE KEY POPULATIONS THAT WAS INTENDED TO BE HELPED BY THE AFFORDABLE CARE ACT AND THEY'RE BEING PROBABLY THE SWING VOTER POPULATION IN THIS PAST ELECTION FOR REASONS THIS KIND OF DISCREPANCY IS HAPPENING BETWEEN THEIR HEALTH AND WELL BEING AND OUTLOOK ON LIFE FOR WHAT LOOKS LIKE GOOD REASON FOR IF YOU LOCK AT RESULTS OF THESE STUDIES AND HEALTHCARE AND BIOMEDICAL SCIENCE IS OFFERING FOR THEM. SO MORE WORK ON UNDERSTANDING THESE TRENDS AND POPULATION HEALTH AND WHAT IS DRIVING THESE TRENDS SEEMS TO BE FUNDAMENTALLY IMPORTANT FOR THE COUNTRY, AND WANT TO PICK UP ON JOHN GRAHAM'S POINT ABOUT THE IMPORTANCE OF THINKING ABOUT NONMEDICAL FACTORS THAT ARE INFLUENCING HEALTH. I MEAN THE U.S. HAS GOTTEN TO BE AN OUTLIER IS NOT SO MUCH IN OUR HEALTHCARE SPENDING AND MEANS SOMEWHAT HIGHER PER CAPITA AND OTHER COUNTRIES AND OTHER STUFF THAT INFLUENCES HEALTH, SOCIAL SERVICES, THE HS PART OF HHS, THE EDUCATION PROGRAMS, THE INFRASTRUCTURE PROGRAMS, JON TRAINING PROGRAMS FUNDED AT EVERY LEVEL OF GOVERNMENT HAVE GOTTEN SQUEEZED DOWN AND THAT IS AN AREA WHERE THERE HAS BEEN BIPARTISAN CONSENSUS. STUDILY IF YOU LOOK BACK OVER THE LAST THOWRT, 40 YEARS, WE'VE HAVE SEEN INCREASE IN THE HEALTHCARE ENTITLEMENT SPENDING AND THAT HAS COME AT THE EXPENSE UNDER BIPARTISAN BUDGET AGREEMENT AGREEMENTS, FORMER PRESIDENT BUSH, PRESIDENT OBAMA NOW IN SPENDING ON THESE OTHER PROGRAMS. AND IF WE DON'T GET A BETTER HANDLE ON WHAT IS ACTUALLY INFLUENCING POPULATION HEALTH AND ALIGN RESOURCES WITH THAT I THINK THERE'S NO REASON TO THINK THAT THESE KINDS OF WORRISOME TRENDS WON'T CONTINUE. THERE'S A LOT OF DISCUSSION RECENTLY TOO ABOUT DISCRETIONARY BUDGET CUTS AND TO PUT THING FIST PERSPECTIVE HOW MUCH IS--YOU'RE 19 MILLION A YEAR WHICH IS LIKE ROUNDING HERE, BUT AND YOU HAVE 250 OR SOMETHING. >> 320. >> OKAY, GETTING UP THERE, SO, THE BIG CONTROVERSY AROUND PRESIDENT TRUMP'S BUDGET PROPOSAL THIS YEAR WITH A 2 BILLION DOLLAR CUT AT NIH AND SO FORTH, JUST FOR COMPARISON, MY FORMER AGENT, THE LATTER 1 CMS THIS YEAR WILL SPEND WELL OVER $1 TRILLION SO EVEN THESE NIH CUTS THAT'S LIKE LESS THAN A DAVID--THANK SPENDING NOW WITH THE INCREASING COSTS OF THE HEALTHCARE PROGRAMS, OUR ENTIRE FEDERAL BUDGET ON HOMELAND SECURITY, NATIONAL SECURITY AND DEFENSE COMBINED IS 600 BILLION AND 10% OF THAT IS HEALTHCARE COSTS FOR ALL THE PEOPLE IN THOSE PROGRAMS. SO, THIS IS NOT A SUSTAINABLE--IT'S JUST NOT A SUSTAINABLE MODEL AND FIGURING OUT MORE WAYS TO DEVELOP RESEARCH IDEAS THAT CAN HELP US UNDERSTAND, HOPEFULLY FIND A BETTER PATH FORWARD ON MAKING SURE THAT THESE BIOMEDICAL INSIGHTS DO ACTUALLY TRANSLATE TO BENEFITS FOR POPULATION HEALTH, YOU KNOW IT'S--I THINK WE'RE PASSED THE TIME AND WE CAN SIMPLY ARGUE THAT SPENDING MORE MONEY AT NIH IS GOING TO MAKE EVERYONE IN THIS COUNTRY BETTER OFF. THE RESEARCH WORK, THE VERY IMPORTANT RESEARCH WORK BEING DONE HERE IS--IT'S KIND OF DISSEMINATING, I'M LEADING US AWAY FROM THAT VIEW. SO OTHER IMPORTANT TOPICKINGS, THATIA A BIG 1. MORE INSIGHTS, AND RESEARCH ON INCENTIVES FOR TURNING BIOMEDICAL INSIGHTS INTO PRODUCTS AND SERVICES THAT ARE SAFE, EFFECTIVE AND RELIABLE FOR PATIENTS. YOU ARE ALL VERY FAMILIAR, PEOPLE LIKEERNY WITH THE UNCERTAIN PATH TO BRINGING NIH BASIC RESEARCH IDEAS INTO THROUGH DEVELOPMENT AND INTO SAFE EFFECTIVE AND RELIABLE USE. I WOULD ALSO HIGHLIGHT SOME OF THE IMPORTANT WORK ON--WHAT DO YOU CALL IT NOW DISINVESTMENT? >> [INDISCERNIBLE]. >> WHAT? >> [INDISCERNIBLE] >> EXOIVATION. >> EXNOVATION. >> WELL IT'S DEEM DEADOPTION EXNOVARTISSATION. >> OKAY. UNDERSTANDING INCENTIVES FOR THAT. IN LIGHT OF WHAT I TALK ABOUT BEFORE. YOU KNOW--LET ME KEEP--I'M GOING TO KEEP WORKING ON THAT. [LAUGHTER] IT DOESN'T QUITE ROLL OFF THE MEMBER OF CONGRESS' TONGUE, I THINK BUT THE CONCEPT IS GREAT. SO MORE OF THAT, I THINK RELOOKING AT SOME OF THE--SOME OF THE IMPORTANT WORK ON THE VALUE OF BIOMEDICAL RESEARCH INVESTMENTS PARTICULARLY HOW TO INCREASE THE VALUE, A LOT OF WHAT FDA IS DOING NOW IS ABOUT IMPROVING DEVELOPMENT SCIENCE AND IT MEANS NEW KINDS OF PRECOMPETITIVE COLLABORATIONS, NEW WAYS OF DOING RESEARCH, AND VERY IMPORTANT FOR ALL OF THE WORK THAT NIH WOULD LIKE TO BE ABLE TO SHOW, TO DEMONSTRATE THE IMPACT OF BASIC SCIENCE, TRANSLATIONAL SCIENCE ON ACTUAL HEALTH. ALONG THOSE LEANS, THIS IS REFLECTED IN THE GROWING DATABASES AND DATA SUPPORT THAT NIH, NIA AND ARC ARE PROVIDING. LOTS MORE OPPORTUNITIES FOR EXPANDING THINGS LIKE THE STATE RESEARCH DATABASES AND THE LIKE THROUGH MORE REAL WORLD DATA SOURCES AND OUR PROGRAM AT DUKE IS WORKING VERY CLOSELY WITH FDA AND OTHERS OF YOU IN THIS ROOM ON A BROAD CONCEPT OF REAL WORLD EVIDENCE WHICH ROLLS OFF THE TONGUE A LITTLE BETTER THAN EXNOVARTISSATION, BUT A LOT OF PEOPLE GENERALLY DON'T UNDERSTAND WHAT IT IS. IT'S CONFUSION BETWEEN REAL WORLD DATA AND REAL WORLD EVIDENCE, REAL WORLD DATA DOESN'T TELL YOU ANYTHING BY ITSELF YOU BUT THE CHALLENGES AROUND TURNING THAT DATA INTO DATA THAT YOU ACTUALLY UNDERSTAND AND APPLYING METHODS THAT ARE FIT FOR PURPOSE FOR THE RESEARCH QUESTIONS IN MIND COULD OPEN UP SOME VERY VALUABLE OPPORTUNITIES FOR UNDERSTANDING NOT JUST THE HEALTH BENEFITS BUT THE COSTS AND COST EFFECTIVENESS OF MEDICAL TECHNOLOGIES, BUILDING ON THE OTHER WORK THAT WAS DISKUTIONZED--DISCUSSED HERE. TODAY, OPPORTUNITIES FOR UNDERSTADING IMPACT OF TREATMENTS AND PARTICULAR SUBPOPULATIONS, REALLY SEEMS ESSENTIAL FOR PRECISION MEDICINE, JUST BECAUSE YOU'RE DOING REAL WORLD EVIDENCE DOESN'T MEAN YOU CAN'T RANDOMIZE BUT IT DOES MEAN SOME RETHINKING OF KIND OF FUNDAMENTAL RETHINKING OF THE WAY TO SUPPORT RESEARCH STUDIES AND NIH HAS STARTED TO DO THAT LIKE THE--YOU KNOW 1 MILLION PEOPLE IN THE PRECISION MEDICINE INITIATIVE HERE, BUT YOU KNOW THERE ARE MANY MORE MILLIONS OF PEOPLE WHO ARE NOW HAVING DATA COLLECTED THAT COULD CONTRIBUTE TO THAT, MUCH MUCH WHICH IS HAPPENING OUTSIDE OF THE CURRENT NIH SUPPORTED RESEARCH EFFORT AND THESE KIND OF RESEARCH DESIGN ISSUES MAY NOT SOUND LIKE HEALTH ECONOMICTHOSE WHO ARE WORKING IN IT, THESE ARE THE TECHNIQUES YOU ALL FOCUS ON. AND I JUST--I END WITH 2 OTHER AREAS OF EMPHASIS THAT ARE CLEARLY IMPORTANT TO THE ADMINISTRATION, HEARD THIS FROM JOHN, 1 IS STATE BASED RESEARCH OPPORTUNITIES SO THE STATE HEALTH PRACTICE DATABASE FOR RESEARCH, THAT'S A GOOD RESOURCE FOR UNDERSTANDING, YOU KNOW WHAT ABOUT STATE HEALTH POLICIES, VARY. YOU KNOW THAT'S ENRICHING THAT, LINKING IT WITH MORE SUPPORT FOR ACCESS TO REAL WORLD DATA SOURCES. I THINK IT WOULD BE VERY VALUABLE AS MORE STATES DO START TAKING ADVANTAGE OF 1332 WAIVERS AND MEDICAID WAIVERS, THOSE ARE VERY IMPORTANT PROGRAMS, THEY ALWAYS HAVE BEEN. UNFORTUNATELY, THE LEVEL OF EVIDENCE THAT COMES OUT OF MANY OF THESE STATE ACTIVITIES IS OFTEN PRETTY LIMITED BECAUSE OF FAILURE TO THINK EARLY ABOUT FEASIBLE REAL WORLD STUDY DESIGNS THAT COULD TURN THAT& INTO EVIDENCE AND THERE MAY BE DISAGREEMENTS ABOUT WHAT SHOULD HAPPEN NEXT WITH THE ACA BUT I THINK THERE SHOULD BE NO DISAGREEMENT THAT STATES NEED TO BE PLAYING A LARGER AND MORE EFFECTIVE ROLE IN IMPROVING POPULATION HEALTH BECAUSE YOU KNOW DOING THIS--THESE STEPS LIKE COMBINING MEDICAID SERVICES AND MEDICAID PROGRAMS WITH COMMERCIAL, COMBINING THOSE WITH NONMEDICAL PROGRAMS, BEHAVIORIAL HEALTH PROGRAMS, SOCIAL SERVICES, AND THE LIKE, THAT IS ALL GOING TO HAPPEN AND ONLY GOING TO BE LED AT STATE AND LOCAL LEVEL. IT WILL NOT BE DONE FROM A FEDERAL STANDPOINT AND THE STATE'S CURRENTLY ARE UNEVEN AT BEST AND LIMITED ACROSS THE BOARD IN THEIR CAPACITY AND INFRASTRUCTURE FOR LEARNING ABOUT WHICH OF THESE PROGRAMS WORK BEST. AND THEN FINALLY, YOU KNOW JOHN TALKED ABOUT THE FANTA AND THAT 1 HAS SUGAR IN THIS RIGHT? [LAUGHTER] SO WE HAVE TO WORK ON JOHN, BUT MORE OPPORTUNITIES AROUND BEHAVIORIAL ECONOMICS, CONSUMER ENGAGEMENT. I'M NOT SURE IF THIS IS NIA AND THE COMMON FUND BUT, PROBABLY MOST IMPORTANT PUBLIC HEALTH ANNOUNCEMENT THIS YEAR DOES GO TO SCOTT GOTTLIERKTS EB, WHICH IS NOT FOR ORPHAN DRUG APPROVE ALGORITHMS AS IMPORTANT AS IT IS IT'S THE REFRAMING OF HOW THE NATION SHOULD THINK ABOUT USE OF TOBACCO AND NICKEE TEEN CONTAINING PRODUCTS THAT IS BY FAR AND THE WAY THE NUMBER 1 CAUSE OF DEATH AND PREMATURE DISABILITY IN THIS COUNTRY IN THE WORLD AND WE'RE NOW IN A SITUATION WHERE IT'S BEEN DIVISIVE PUBLIC HELT ISSUE BEFORE BETWEEN HOW MUCH SHOULD WE REGULATE DOWN RIG SETS, HOW MUCH OF THE PUBLIC HEGHT COMMUNITY LET AMERICANS KNOW ABOUT POTENTIALLY SAFER ALTERNATIVES AND THE WORRIES THAT MIGHT LEAD TO MORE YOUTH SMOKING OR FEWER PEOPLE QUITTING AND THE TECHNOLOGIES HAVE CHANGED. THE REGULATORY STRATEGIES USED AROUND THE WORLD ARE IN THE PROCESS OF CHANGING, MANY COUNTRIES IN THE DIFFERENT DIRECTION OF THE U.S.S NOW, AND ACTUALLY SEEM TO BE SEEING PRETTY INTERESTING EARLY RESULTS IN TERMS OF SIGNIFICANT REDUCTIONS AND SMOKING RATES IN COUNTRIES LIKE THIS 1 HAVE PLATEAUED AT 16, 17% OR EVEN HIGHER FOR MANY YORES. THAT IS PARTLY A SCIENCE--SCIENTIFIC SET OF ISSUES AND TECHNOLOGICAL SET OF ISSUES BUT ALSO GETS TO BEHAVIORIAL ECONOMICS AND ECONOMIC ANALYSIS OF CONSUMER CHIES AND BEHAVIOR AND THAT REALLY NEED TOTION INFORM IN WHAT'S GOING ON IN THIS COUNTRY GOING FORWARD. ALL THIS WORK IS IMPORTANT. I HOPE YOU KEEP IT UP AND ALL THESE PROGRAMS ARE KIND OF ROUNDING ERRORS COMPARED TO THE CMS BUDGET. I WILL KEEP DOING WHAT I CAN TO SUPPORT THE EFFORTS HERE. THANK YOU VERY MUCH. [ APPLAUSE ] >> THANK YOU ALL 3. WE'VE HEARD A LOT OF VERY DIFFERENT PERSPECTIVES, I WAS IMPRESS THAD YOU LOOK AT THE HEALTH ECONOMICS AND WHAT'S IN THE PLAYING FIELD AND NOT IN THE PLAYING FIELD. CAN YOU HEAR ME? >> YES. >> SO WHAT I WOULD ASK OF EACH OF YOU TO GET THE DISKUTIONZ DISKUTIONZ--DISCUSSION START SIDE THIS IS AN ULTIMATE YEAR FOR THE COMMON FUND INITIATIVE AND THE INTENT HAD BEEN THIS IS THE CATALYST GOING FORWARD, LOW HANGING FRUIT THERE? PERSPECTIVES FOR NIH? WHOEVER WANTS TO START? D. >> I ALREADY KIND OF ANSWERED A BIT OF THAT SO MAYBE TURN TO THE OTHERS FIRST AND I CAN ADD ON IF YOU ALL HAVE-- >> SO I THINK THAT THERE IS A LOT TO UNDERSTAND ABOUT HOW NOT ONLY TO DISSEMINATE TECHNOLOGY, NEW TECHNOLOGY IN A SAFE AND APPROPRIATE WAY BUT ALSO HOW TO DIS--WHAT'S THE WORD AGAIN EXNOAFATE EXISTING PRACTICES. SO TO PLEA THAT'S A HUGELY FETTERILE WAY, AREA OF RESEARCH. I THINK THE FIELD IS MOVING INTO LOOKING AT ALTERNATIVE WAYS TO ASSESS HEALTH AND WELL BEING, PATIENT REPORTED OUTCOMES ARE INCREASINGLY AN IMPORTANT WAY TO ASSESS OUTCOMES AND I WOULD ENCOURAGE INCREASED EXAMINATION OF KIND OF EASY REPLICABLE WAY OF THE DATA AND USE IT ASSESS OUTCOMES. SO THOSE ARE THE THINGS THAT I WOULD FOCUS ON. >> I WOULD JUST SAY BEING THE NONGOVERNMENT ACTOR, I KNOW OBVIOUSLY THERE'S ACADEMICS HERE BUT ON THIS PANEL BEING THE GUY WITH POLICE GOVERNMENT EXPERIENCE, ALTHOUGH, REPRESENTING THE SECRETARY THAT I THINK YOU'RE ON THE RIGHT TRACK IN THE SENSE THAT IT'S THIS QUESTION, I THINK MAYBE A WORD TO DESCRIBE WHAT YOU'RE TALKING ABOUT WAS KIND OF SEAMLESSNESS. I WAS ENCOURAGED WHEN WE WERE TALKING ABOUT, [INDISCERNIBLE] AND THE ALZHEIMER'S PACKET AND ALL THAT STUFF, THERE A CONNECTEDNESS. DO IENT KNOW IF I WANT TO SAY MORE THAN THAT. JUST TO INSURE THAT WE SEE COME FROM THE BASIC RESEARCH TO PRACTICE WE SEE THE KIND OF PATH FROM A TO B TO C TO D. >> I DON'T HAVE MUCH TO ADD EITHER. JUST SAY IF I WERE YOU LOOKING BACK ON WHAT THIS COMMON FUND INITIATIVE ACCOMPLISHED IN TERMS OF HELPING TO TURN DISCOVERY INTO HEALTH AND UNDERSTAND WHAT REALLY CAN IMPROVE THE HEALTH OF POPULATION, YOU SHOULD BE--REALLY PLEASED WITH THE--WITH THE PAYOFF AND I HOPE THE NEXT ROUND OF EFFORTS ON THIS. THIS PROGRAM ENDING NEXT YEAR DOESN'T MEAN THIS DIRECTION IS GOING TO GET LESS SUPPORT AND I GUESS I JUST GO BACK TO 1 OF THE POINTS THAT I MADE EARLY IN MY REMARKS, TURNING DISCOVERY AND SUPPORTING DISCOVERY AND TURNING DISCOVERY INTO HEALTH SEEMS ABSOLUTELY CORE TO THE NIH MISSION AND I'M MORE, NOT LESS WORRIED ABOUT THAT AS TIME HAS GONE ON. THERE SEEMS TO BE GETTING--SEEMS TO BE MORE OF A DISCONNECT IN TERMS OF WHAT WE'RE SEEING IN POPULATION, HEALTH TRENDS, DESPITE WHAT SHOULD BE LIFE CHANGING AND WHAT IS LIFE CHANGING AND LIFE IMPROVING RESEARCH AND UNDERSTANDING THAT DISCONNECT BETTER, UNDERSTANDING THINGS LIKE EXNOVARTIS--EXNOVATION AND IT SEEMS VERY IMPORTANT. BUT THAT KEY THEME. DISCOVERY HEALTH, IT'S NOT AS CLEAR OF A RELATIONSHIP AS IT USED TO BE AND A LOT OF NIH IS DOING A LOT OF IMPORTANT WOORK AND THE WORK IN THE PROGRAM IS MOST RELEVANT RIGHT NOW FROM WHAT I CAN SEE TO HELPING UNDERSTAND WHY THAT'S NOT MORE THE CASE. >> HEALTH DISPARITIES, THE ELEPHANT IN THE ROOM, WHAT INSIGHT CAN ECONOMICS PROVIDE AND WHAT KIND OF RESEARCH WE SHOULD PURSUE AND I SHOULD CONSIDER SUPPORTING? THANKS. >> I THINK THE PEOPLE AROUND THE TABLE CAN ANSWER THAT BETTER THAN I CAN, BUT I THINK SOME OF THE RESEARCH WE TALKED ABOUT TODAY, AND THE RESEARCH YOU'RE ALREADY SUPPORTING, I WOULD LIKE TO SEE MORE ON UNDERSTANDING IF WE ARE GOING TO SPEND SO MUCH ON HEALTHCARE, ARE THERE WAYS TO BETTER LINK NONMEDICAL FACTORS THAT INFLUENCE HEALTH, WHAT OUR HEALTHCARE SYSTEM IS DOING. THERE ARE MANY HEALTHCARE ORGANIZATIONS NOW THAT ARE--USING THE DRUGS, THE DEVICES, THE APPROXIMATELY PRODUCTS, DIAGNOSTIC INSIGHTS THAT COME OUT OF NIH RESEARCH IN COMBINATION TO FOLLOW THE PRECISION MEDICINE APPROACH FOR MANY PATIENTS WHO HAVE RECURRENT COSTLY HEALTH PROBLEMS AND COMPLICATIONS, THE MAIN DETERMINANT IS NOT MEDICAL, IT CAN BE SOCIAL FACTORS LIKE EVERYTHING FROM TRANSPORTATION TO HOUSING TO SECURE ENVIRONMENTS AND THE LIKE. IF WE'RE TRYING TO GET TO A HEALTHCARE SYSTEM THAT IMPROVES HEALTH, WE'RE SPENDING THIS MUCH MONEY ON IT, I THINK RESEARCH AND UNDERSTANDING THOSE CONNECTIONS WOULD BE USEFUL. >> YES, AND I FULLY AGREE AND I THINK THE--IT'S INTERESTING YOU POINTED OUT THE MIDDLE AGED WHITE PROBLEM THAT'S NOT THE WAY WE WANT TO REDUCE DISPARITIES RIGHT? WE DON'T WANT TO GO THE OTHER WAY AROUND IT, RIGHT? SO I SHOULD MENTION JUST AS A MATTER OF FACT WE DID RELEASE OUR PROPOSED STRATEGIC PLAN FOR PUBLIC COMMENT EARLIER THIS WEEK AND IT VERY CLEARLY REITERATINGS REDUCING HEALTH DISPARITIES AS A GOAL AND CULTURALLY APPROPRIATE WORS LIKE THAT ARE IN A STRATEGIC PLAN AND BOTH HEALTH AND HUMAN SERVICES SIDE AND ASPE IS SOMETHING WE WANT TO FOCUS ON AS YOU WERE SAYING, I KNOW THE ESTIMATES OF WHAT DOES THE HEALTH SYSTEM CONTRIBUTE TO HEALTH AND YOU KNOW THAT BETTER THAN ME BUT I'VE HEARD 15% TO 25% TO 50% BUT MOST OF OUR HEALTH STATUS IS NOT THE HEALTHCARE SYSTEM, "-UNQUOTE. SO WE HAVE TO BREAK DOWN THOSE SILOS AND GET A LOT OF THOSE FUNDINGS AND IT'S SOMETHING I WILL BE WORKING ON AT MY TENURE AT ASPE, AND IF YOU WANT TO COMMENT ON THE STRATEGIC PLAN, IT'S OUT THERE FOR 30 DAYS. SO IF DISPARITIES IS SOMETHING YOU'RE INTERESTED IN, LOOK AT THE STRATEGIC PLAN AND WAY IN ON THE DOCKET AND WE'LL TAKE IT INTO CONSIDERATION, THANK YOU. >> I THINK DISPARITY SYSTEM SOMETHING THAT ARC IS VERY INTERESTED IN, IT'S BEEN INTERESTED, WE PROVIDE A REPORT EVERY YEAR TO LOCK AT DISPARITIES. THINK WAWE ARE TRYING TO FOCUS ON IS UNDERSTANDING KIND OF HOW WE MAKE SURE THAT EVIDENCE IS SPREAD EVENLY TO ALL PATIENTS, KIND OF PROPERTILY AND UNDERSTANDING THE BARRIERS TO HIGH RELIABILITY CARE AND MAKING SURE THAT EQUAL TREATMENT IS PROVIDED AS APPROPRIATE AND LOOKING AT VARIATIONS IN OUTCOMES. SO I THINK IT'S A REALLY IMPORTANT AREA. >> DOCTOR? >> MAY I ADD SOMETHING. >> YEAH, PLEASE. >> ALSO I THINK IT MIGHT BE IMPORTANT TO LOCK AT PREFERENCES FOR PARTICIPATION IN RESEARCH AND WAYS TO IMPROVE EQUAL PARTICIPATION IN RESEARCH BECAUSE I THINK THAT PEOPLE WHO PARTICIPATE IN RESEARCH, THE DISTRIBUTION IS SKEWED SO I THINK THERE CAN BE-- >> THIS IS TO PICK UP ON THAT, YOU KNOW I TALK A ABOUT REAL WORLD DATA AND EVIDENCE LITTLE SO MUCH MORE FEASIBLE NOW THAN IT USED TO BE TO TAKE THE RESEARCH STUDIES WHERE THE PATIENTS ARE RATHER THAN TO HAVE THEM COME TO THE TRADITIONAL KINDS OF SETTINGS BUT TO DO THAT WE HAVE TO RETHINK THE RESEARCH ENGAGEMENT PROCESS, THE IRB PROCESS AND NIH IS DOING A LOTS ON THAT CENTRAL IRB INITIATIVES AND LIKA AND THE PRECISION MEDICINE WORK, THAT'S GOING ON. I THINK WE'RE REALLY JUST SCRATCHING THE SURFACE AND I THINK THERE ARE A LOT OF WAYS NOW, EVEN WITH DISPARITIES AND INTERNET ACCESS AND THE LIKE TO GET TRADITIONALLY UNDER REPRESENTED PATIENTS MUCH MORE ENGAGED. YOU HAVE TO GET OUT OF THE TRADITIONAL CENTER, BRING THE PATIENTS TO YOU APPROACH TO RESEARCH STUDY DESIGN. >> I WANTED TO PICK UP ON A THREAD MARK THAT YOU PUT OUT THERE. SO IT'S GREAT THAT THE PLIGHT OF RISING MORTALITY IN MIDDLE AGED WHITES HAS DRAWN ATTENTION TO SOCIAL DETER ANTS OF HEALTH BUT STEPPING BACK, I THINK IT'S CLEAR TO ALL OF US FOR A LONG TIME THAT THERE'S A REAL MISMATCH TBEEN HOW MUCH WE SPEND ON HEALTH AND WAWE GET IN TERMS OF POPULATION HEALTH, SO WE ALL KNOW WE SPEND A LOT MORE THAN ANY OTHER COUNTRY. WE RANK 26th IN LIFE EXPECTANCY. SO THE REAL PUZZLE I WOULD LOVE THE 3 OF TO YOU TRY TO ANSWER AS PEOPLE HAVE HAD OR HAVE HIGH POSITIONS IS THE PUZZLE FOR US AS CITIZEN AND ECONOMISTS IS WHY DO WE SPEND A TRILLION DOLLARS PLUS A YEAR ON DELIVERING HEALTH SERVICES--[INDISCERNIBLE] >> WHY DO WE SPEND $3 TRILLION A YEAR ON DELIVERING SERVICES AND THE AMOUNT WE SPEND ON ACTUALLY TRYING TO UNDERSTAND THE PROBLEM OF WHY THERE'S THIS MISMATCH BETWEEN HOW MUCH WE SPEND ON OUR POPULATION HEALTH AND TINY? YOU KNOW IT'S PERHAPS THE ARC BUDGET, A SMALL PERCENTAGE OF THE NIH BUDGET AND VARIOUS FOUNDATIONS BUT LET'S SAY WE ADDED IT ALL UP AND I'M GUESSING IT'S MAYBE 2 BILLION DOLLARS? THAT MIGHT BE GENEROUS? SO 2 BILLION DOLLARS RELATIVE TO 3 TRILLION AND YOU LOOK AT THE IMPACT OF ENTITLEMENT SPENDING, IMPACT ON POPULATION HEALTH, WHY IS IT THAT WE HAVE TROUBLE FIGURING OUT HOW TO MAKE A MORE CONCERTED EFFORT TO DO SOMETHING ABOUT THIS? >> AGAIN THIS IS A TOPIC I'M SURE PEOPLE AROUND THE TABLE HAVE VIEWS BUT MAYBE A COUPLE THOUGHTS. ONE IS, PEOPLE OFTEN ATTRIBUTE FUNDING FOR THE TRADITIONAL KIND OF BIOMEDICAL RESEARCH, THE FACT THAT IT JUST KIND OF MORE VISIBLE, EASIER HEAD LINES AND CURES THAT CAN IMPACT CLEAR DISEASES AND UNREGISTERED QUESTIONABLY HAS BEEN VALUABLE RESEARCH, NOT SAYING ANYTHING AGAINST THAT. AND MAYBE THESE TYPES OF NONMEDICAL STUDIES OR NOT SO HIGH PROFILE. BUT IF YOU THINK ABOUT DOING RESEARCH IN THESE OTHER AREAS, IT'S REALLY HARD, RIGHT TO THINK ABOUT WHAT IS THE COMPELLING STUDY DESIGN, WHAT ARE THE INTERVENTIONS, THAT YOU COULD STUDY, THAT WOULD HAVE AN IMPACT, YOU KNOW THERE HAVE BEEN EFFORTS, INCLUDING NIA'S EFFORTS AROUND TO STUDY THESE FACTORS, YOU DID DID THESE LARGER LONG-TERM STUDIES THAT GET AT GOOD LONGITUDINAL DATA ON ALL OF THESE NONMEDICAL DETERMINANTS, HEALTH AND RETIREMENT SURVEYS AND GREAT EXAMPLE OF OF A RESEARCH BASE FOR SUPPORTING THOSE EFFORTS AND THAT'S HARD TO DO AND IT TAKES LONG-TERM FUNDING AND AN AREA OF WHERE I THINK THERE SHOULD BE A CHALLENGE BACK TO THE RESEARCH COMMUNITY ON HOW DO WE MAKE THIS KIND OF RESEARCH MORE PRODUCTIVE AND YOU KNOW MORE CLEARLY HAVING SOME PAYOFFS AND I THINK THAT'S WHERE THINGS LIKE USE OF REAL WORLD DATA, REAL WORLD EVIDENCE, THAT YOU KNOW CONSTRUCT THE MEANINGFUL POPULATIONS FOR TRACKING AND WELL INTERESTED DATA FROM THE BROADER SWATH OF OF--HALFTIME AT DUKE AND HALFEM THE TIME AT GOOGLE BECAUSE HE REALLY BELIEVES--HE'S THE GUY THAT BELIEVES IN CLINICAL TRIALS, OBSERVE AT DATA NEVER STILL DOESN'T PAY ANY ATTENTION TO MY WORK--[LAUGHTER] --BUT HE REALLY BELIEVES THOSE KIND OF FUNDAMENTALLY NEW WAYS OF DESIGNING STUDIES ARE NEEDED TO ADDRESS THESE KINDS OF REAL POPULATION HEALTH DETERMINANTS. SO IT'S SO FAR NOT A FULLY MET CHALLENGE TO THE FIELD OF EXPERTS? , SO CAN I JUST SAY, I THINK THERE'S A CHICKEN AND AN EGG PROBLEM AND MAYBE THAT'S WHAT KEVIN, IN ORDER TO REALLY DEVELOP A WAY TO MAKE IT YOU KNOW EFFICIENT, THOSE STUDIES EFFICIENT, YOU NEED TO STUDY THEM AND SO, I THINK THERE IS A REAL CHICKEN AND EGG HERE. >> MIGHT BE A GOOD NEXT COMMON FUND TOPIC. >> I WILL SPEAK TO THE RESEARCH QUESTION, I'VE SORT OF BEEN READING OUR STRATEGIC PLAN HERE, NOT DIRECTLY BUT TIMATHY BLOGGED ABOUT IT SO IT WAS QUICKER IF ARE ME TO GO TO HEALTH AFFAIRS BLOG AND READ WHAT HE PULLED OUT BUT 1 THING WE HAVE ON THE STRATEGIC PLAN IS BUILD OUT AND BROADEN MODELS THAT ALLOW BENEFICS, THE OPTION OF CONTROLLING THEIR HEALTHCARE DOLLARS SO THAT GOES TO THE STAND BY OF HEALTH SAVINGS ACCOUNTS AND ALL THAT STUFF, BUT TRYING TO DEAL WITH THIS THIRD PARTY PAYOR, WHICH MANY OF US WOULD DEFINE, WELL IT HAS A OF PROBLEMS, RELYING ON THIRD PARTY PAYORS, SO TO THE BEST WE CAN, WE ARE GOING TO FOCUS ON CONSUMER EMPOWERMENT. I KNOW THAT'S A BUZZ WORD BUT THAT'S--WE'RE GOING TO KEEP TRYING TO DO THAT BY WHATEVER WAY, WHATEVER MEANS WE CAN DO THAT. >> YEAH AND MAYBE THAT'S ANOTHER SOURCE FOR RESOURCES HERE. I MEAN PRETTY CLEAR THAT IF, YOU KNOW YOU GAVE WELL INFORMED PEOPLE THE CHOICE ABOUT HOW THEIR HEALTHCARE DOLLARS ARE BEING SPENT ON THEM VERSUS HOW THEY WOULD LIKE THEM TO BE SPENT, IF THEY HAD MORE CONTROL, I DON'T THINK WE'RE CLOSE TO MATCHING WELL INFORMED PATIENT PREFERENCES, AND SOME OF THAT MIGHT FIT INTO SUPPORTING OR CONNECTING THE MORE RESEARCH ON THESE NONMEDICAL FACTORS. WHEN I WAS AT CMS, WE DID--ACTUALLY BEFORE, THE WHITE HOUSE, WE DID INITIATIVE ON MONEY FOLLOWS THE PERSON IN MEDICAID WHICH BASICALLY TOOK THE MEDICAID ENTITLEMENT KIND OF A TRADITIONAL NURSING HOME SERVICES, THAT'S STILL WHAT YOU'RE ENTITLED TO GET IN THE MEDICAID STATUTE AND SET UP KIND OF A RISK ADJUSTED PROGRAM FOR GIVING PEOPLE WITH LONG-TERM DISABILITIES MORE CONTROL OVER THEIR RESOURCES AND THEY END UP SPENDING IT ON STUFF THAT'S NONMEDICAL, MODIFYING THEIR HOMES STUFF LIKE THAT BUT SAVE MONEY, CERTAINLY LED TO BETTER PATIENT EXPERIENCE AND OUTCOMES AND COMMUNITY PARTICIPATION AND MORE STEPS LIKE THAT WOULD BE AN IMPORTANT PART OF MAKING PROGRESS HERE, I THINK. >> YEAH, I MEAN, KEVIN, I THINK YOU CAN A REALLY IMPORTANT QUESTION AND IT'S 1 WE'VE BEEN GRAPPLING WITH BECAUSE WE HAVEN'T--FRANKLY HAVEN'T BEEN DOING VERY WELL MAKING THE CASE FOR KIND OF HEALTH SERVICES RESEARCH, EGHT ECONOMICS RESEARCH AND WE FOCUS ON IMPROVING QUALITY AND SAFETY AND WE FIND WHEN WE GO TO THE HILL, WE FACE PEOPLE WHO SAY, BUT WE HAVE THE BEST SYSTEM IN THE WORLD, THE SAFEST, THE HIGHEST QUALITY. AND SO IT'S HARD TO ENCOURAGE FOLKS TO SPEND A LOT OF MONEY WHEN THEY HAVE COGNITIVE DISSIDENCE ABOUT WHETHER THERE'S A PROBLEM. >> WE HAVE THE BEST LEGISLATIVE PROCESS IN THE WORLD. KTD--[LAUGHTER] >> CLEARLY. >> [LAUGHTER] DO WE HAVE OTHER QUESTIONS WE WOULD LIKE TO RAISE. >> I SHOULD JUST--A LOT OF OF OUR GUYS CAME FROM THE HILL RIGHT? I NEVER HAD A GOVERNMENT JOB, BUT WE HAVE A LOT OF GUYS FROM DR. PRICE'S OFFICE AND SENATOR'S OFFICE AND THEY--THEY'VE CHANGED THEIR MINDS ABOUT THE VALUE OF CONGRESS PRETTY QUICKLY ASK CONVERT TO OUR SIDE. [LAUGHTER] >> I'LL TURN IT OVER TO JOHN THEN. >> THANKS, I WOULD MENTION THE HEALTH ECONOMICS PROGRAM OF THE& COMMON FUND IS ENDING BUT THE COMMON FUND OF COURSE CARRIES ON, LESLIE DURR WORKS IN THE OFFICE AND MANAGING THAT AND 1 PROGRAM THAT AS A NEW LEASE ON LIFE IS CALLED SCIENCE AND BEHAVIOR CHANGE SO IT'S NOT THE AMOUNT SPENT ON THESE PROGRAMS ARE NOT GOING TO CHANGE MARK'S QUALITATIVE CONCLUSION THAT THESE ARE NOT TRILLION DOLLAR OR EVEN BILLION DOLLAR PROGRAMS BUT THERE ARE SOME INITTIAIVE ITS HERE THAT WE WOULD LOVE PEOPLE TO LOOK AT THE WEBSITE AND SEE WHAT'S GOING ON AS WELL. SARAH? >> SO I WANT TO THANK THIS PANEL AND ALL THE OTHER PANELS. I THINK WE HAD A WONDERFUL DAY. A LOT OF THINGS TO THINK ABOUT AND I KNOW SOME OF YOU WILL BE JOINING US FOR DINNER TONIGHT AT THE DAILY GRILL AT 6:30 RIGHT DOWN IF BETHESDA BY THE HYATT, OTHERS OF YOU COME BACK, WE RECONVENE AT 8:30. SO PLAN TO GET THROUGH, I WILL TRY NOT TO REMEMBER MY ID SO I CAN ACTUALLY GET HERE ON TIME SO THAT WOULD BE GOOD. BUT IT'S BEEN A WOBDERFUL DAY AND I THANK YOU ALL. [ APPLAUSE ]