>>> GOOD MORNING, EVERYONE. I HOPE EVERY ONE OF YOU WILL PROMISE ME NOT TO TALK ABOUT THE WEATHER. [ LAUGHTER ] WE START OFF TODAY WITH A GUEST, KENNETH MANDL IS PROFESSOR AT HARVARD MEDICAL SCHOOL, DIRECTOR -- WHAT A TITLE. DIRECTOR OF THE INTELLIGENT HEALTH LABORATORY. WOW! INTELLIGENT, THAT'S GOOD. AT THE CHILDREN'S HOSPITAL. >> INTELLIGENT HEALTH SYSTEMS. >> AS OPPOSED TO STUPID HEALTH SYSTEMS. HE RECEIVED HIS M.D. DEGREE FROM HARVARD, AND WAS AN NLM INFORMIC EARLY IN HIS CAREER AND HIS FIRST GRANT RECEIVED A PRESIDENTIAL EARLY CAREER AWARD FOR SIGNISTS AND ENGINEERS. HE HAS RESEARCHED ON THE USE OF I.T. AND BIG DATA FOR POPULATION HEALTH, DISCOVERY PATIENT ENGAGEMENT AND CARE REDESIGN, PRINCIPAL INVESTIGATOR OF A GRANT AWARDED BY THE PATIENT CENTERS OUTCOME RESEARCH INSTITUTE, ALSO MEMBER OF THE ADVISORY COMMITTEE TO THE DIRECTOR OF C.D.C., CURRENTLY THE CHAIR ON THE BOARD OF SCIENTIFIC COUNSELOR FOR THE NLM COMMITTEE, THE BOARD MEETS TWICE A YEAR AND REVIEWS PROJECTS AND SO ON. HE'S HERE TO PRESENT TO THE BOARD TODAY THE HIGHLIGHTS FROM THE PAST TWO YEARS OF THE LISTER HILL CENTER ESC ACTIVITIES. WELCOME, KEN. >> THANKS, EVERYBODY. IT'S AN HONOR TO BE HERE, TO HAVE BEEN ASKED BY CLEM TO SERVE AS THE CHAIR OF THIS COMMITTEE AND IT'S ONE OF THE TRUE PLEASURES COMING DOWN TO THE NLM. I SPENT FOUR YEARS AS WELL, IT'S A HIGHLIGHT COMING DOWN, SO THANK YOU FOR HAVING ME BACK. WE HAVE THE PRIVILEGE OF HEARING ABOUT THE EXCITING RESEARCH GOING ON AT LISTER HILL, EVERY SIX MONTHS. AND I'M GOING TO LIVE YOU AN OVERVIEW. IT WOULD BE IMPOSSIBLE TO GO INTO ALL THE DETAIL WE HEARD OVER THESE TWO-DAY MEETINGS, TIMES THREE, BUT WE'LL GIVE YOU THE HIGHLIGHTS AND TRY TO GIVE YOU A SENSE OF THE EXCELLENT WORK AND HOW THE BOARD HAS REACTED TO IT AND JUST A FEW THOUGHTS ABOUT WHERE THE BOARD HAS FELT FUTURE DIRECTIONS MIGHT BE, MIGHT GO. THESE ARE THE MEMBERS OF THE BOARD OF SCIENTIFIC COUNSELORS FOR LISTER HILL, A DIVERSE GROUP, SUE PERP SET OF COLLEAGUES FOR ME AND ALL PROVIDE UNIQUE PERSPECTIVES AND I THINK HAVE BEEN HELPFUL TOWARDS THE INVESTIGATORS HERE. WE GET REAL PERKS. ONE OF THE BOARD MEETINGS WAS SCHEDULED AT THE EXACT PEAK OF CHERRY BLOSSOMS IN THE SPRING, AND CLEM ARRANGED TO HAVE US PICKED UP IN CARS AND BROUGHT OUT TO THIS LOVELY NEIGHBORHOOD IN BETHESDA WITH THESE OVERARCHING TREES AND GOT TO SEE SOMETHING YOU DON'T SEE EVERY DAY, WHICH IS ED HAMMOND IN A TREE. SO WE HAVE SOME FUN TOO. AND WE GOT TO OUR MEETING ON TIME AS WELL. SO I DIVIDED UP MY COMMENTS AND OBSERVATIONS, I GROUPED THINGS TOGETHER, NOT EXACTLY BY INVESTIGATOR OR BY MEETING BUT IT FALLS OUT BY INVESTIGATOR HERE. AND I'VE CALLED THESE AREAS LEARNING TO READ, DIVINATION, PATIENTS ARE JUST SHY, MEDICINE 101 AND KEEPING THEM HONEST. AND, YOU KNOW, IN THERE, HOW TO KEEP THE DATA PRIVATE. LEARNING TO READ, SO THERE'S A MAJOR INITIATIVE HERE IN NATURAL LANGUAGE PROCESSING, SO WHAT IS ONE OF THE MOST PROMISING AREAS IN BIOMEDICAL RESEARCH? WELL, IT IS THAT WE CAN ACTUALLY RETEXT AND BEGIN TO MAKE INFERENCES FROM THE ENORMOUS COPPUS OF BIOMEDICAL LITERATURE AND ELECTRONIC HEALTH INFORMATION THAT WILL HELP IN DISCOVERY. RIGHT NOW IN HEALTH CARE, AS YOU KNOW, WE TURNED DOCTORS INTO CODERS AND IT'S ABOUT THE HEALTH OF THE CHART MORE THAN THE HEALTH OF THE PATIENT. WHAT WILL SAFE US IS THE ABILITY TO READ TEXT THAT'S NOT STRUCTURED. THERE'S NO STANDARDS ORGANIZATION THAT WILL BE ABLE TO KEEP UP WITH THE NEW WAYS TO EXPRESS OURSELVES AND NEW KINDS OF DATA. WE NEED A COMPUTER LOOK AT A CHART, LOOK AT THE BIOMEDICAL LITERATURE AND UNDERSTAND IT, AND HELP US SO THAT WE DON'T HAVE TO PUT EVERYTHING INTO AN ENCODED FORM OR TEACH ALL OUR AUTHORS TO USE THE COMPLETELY STRUCTURED WAY TO EXPRESS THEIR SCIENTIFIC FINDING. THIS IS IMPORTANT WORK. AN EXAMPLE OF THE WORK GOING ON HERE AT LISTER HILL IS A PROGRAM WHICH LEVERAGES UMLS RESOURCES TO EXTRACT SEMANTIC PREDICATION REGARDING CLINICAL MEDICINE FROM THE LITERATURE. HERE IS LITERATURE, WE CAN READ IT BUT CAN'T KEEP UP WITH IT. WE'RE PRETTY GOOD AS HUMANS AT BEING PATTERN DETECTORS, BUT REALLY UP TO A CERTAIN POINT. THERE'S DATA WE CAN PROCESS. SO SEMREP CAN PULL OUT CONCEPTS AND INSERT PREDICATIONS, SO IT CAN INFER FROM READING TEXT, FOR EXAMPLE, THAT TH THE AROMATASE INHIBITOR TREATS BREAST CARCINOMA, THIS IS ONE CONCEPT THAT PUTS THESE CONCEPTS INTO A META THESAURUS AND WE END UP WITH THIS RELATION BETWEEN THEM. THERE'S ALL KINDS OF RELATIONS BETWEEN TEXT CONCEPTS THAT NLM FOLKS LOOK AT, TEMPORAL, CAUSATIVE, PREDICATION. SO THIS LEADS TO VERY IMPORTANT DISCOVERIES. FOR EXAMPLE, HERE IN A HIGH IMPACT JOURNAL, SLEEP, THE INVESTIGATORS REPORTED DISCOVERING A LINK BETWEEN HYPOGONADISM AND DIMINISHED SLEEP QUALITY IN MEN, PULLED OUT OF LITERATURE READ BY COMPUTERS. SO IN MANY WAYS, ACTUALLY REALLY A VERY EXCITING DISCOVERY. SO HERE YOU CAN GO IN AND THEN THESE ARE EXAMPLES OF OTHER PREDICATIONS THAT ARE PULLED AUTOMATICALLY OUT OF LITERATURE. THIS IS ANOTHER PUBLICATION THAT IS ABOUT ACTUALLY READING THE LABORATORY DATA IN THE LITERATURE. ALL RIGHT. IT'S JUST ANOTHER EXAMPLE. SO LABORATORY DATA IS EXPRESSED IN LITERATURE IN DIFFERENT WAY, DIFFERENT UNITS, DIFFERENT TERMS, AND THE ABILITY TO PULL THAT OUT IS ACTUALLY QUITE IMPORTANT. SO THE BOARD GETS VERY EXCITED ABOUT THESE THINGS, AND WE ACTUALLY IN OUR RECOMMENDATIONS LISTED A WHOLE BUNCH OF DIFFERENT WAYS THAT THESE TYPES OF NATURAL LANGUAGE PROCESSING TECHNIQUES TO BE USED, LOOKING AT ADVERSE DRUG EVALUATIONS, ON THE EXTRAMURAL SIDE THERE ARE FUNDED INVESTIGATORS DOING THIS FROM THE NLM. YOU KNOW, DISPARTY OF OUTCOMES, ESTIMATING EFFECT SIZES IN SUBGROUPS TO TRY TO THINK ABOUT PERSONALIZED THERAPIES, THERE'S REALLY A TREMENDOUS RESEARCH AGENDA TO WHICH THESE TECHNIQUES COULD BE APPLIED. THE BOARD WAS VERY EXCITED BY THIS. AND, OF COURSE, LEARNING NOT ONLY TO READ THE LITERATURE BUT TO FALL IN LOVE WITH THE LITERATURE, JUST AS IF A GUY IS FALLING IN LOVE WITH HIS OPERATING SYSTEM, WE THINK OF SOMETHING TO TARGET NEXT. SO DIVINATION, DIVINATION IS HERE, THERE'S ACTUALLY SOME REALLY INTERESTING WORK THAT WAS PRESENTED TO US ON PROBABLISTIC MODELS AND MACHINE LEARNING FOR PREDICTION. IT TURNS OUT THAT THE FUTURE IS ACTUALLY LARGELY KNOWABLE FOR MANY PATIENTS. IN MY OWN WORK, WHEN I WAS IN THE WORK THAT WAS NLM FUNDED AND BY SURVEILLANCE WE FIGURED OUT EARLY ON THAT WE COULD TELL YOU HOW MANY PATIENTS WERE COMING IN TOMORROW WITH INFLUENZA. TODAY WE COULD TELL YOU HOW MANY ARE COMING IN TOMORROW WITH INFLUENZA WITH A NARROW CONFIDENCE INTERVAL. WE'RE FLYING BLIND DESPITE ENORMOUS DATA FROM PREDICTIVE MODELS AND LISTER HILL IS EXPERTING THE POWER OF COMPUTING IN THIS AREA. HERE IS ONE PAPER PREDICTING ICU OUTCOMES BASED ON PHYSIOLOGICAL DATA, AND HERE IS ANOTHER ONE THAT WAS -- I REMEMBER ALSO READING IN THE PRESS, ACTUALLY I REALIZED IT HAD COME OUT OF LISTER HILL, BUT THIS ONE LINKED ACTUALLY PREDICTIVE OF MORTALITY, WAS BMI, AND IT TURNED OUT THAT IF YOU WERE HEAVIER, YOU SURVIVED LONGER IN THE ICU, AND AFTER THIS I SAW CLEMENCLEM McDONALD WAS BRINGING DOUGHNUTS EVERY TIME WE CAME SO IT'S BEEN TRANSLATED INTO PRACTICE. [ LAUGHTER ] HERE IS ONE, PATIENTS ARE JUST SHY. SO THIS IS JUST ONE SCREEN SHOT, AND BY THE WAY THANK YOU, I'VE PILLAGED ALL YOUR SLIDES FOR THESE, BUT HERE IS THE ATTRIBUTION. THIS IS A -- THIS IS A SCREEN SHOT OF THE SCRUBBER, SO IT TURNS OUT TO BE VERY, VERY IMPORTANT TO -- IF WE'RE GOING TO HAVE -- IF WE'RE GOING TO SHARE TEXT FOR RESEARCH THAT WE DON'T SHARE PATIENT IDENTITIES, FIRST OF ALL IT'S ILLEGAL, AND ALSO IT'S BAD PRACTICE. SO THE TECHNOLOGY TO DO THIS IS BEING DEVELOPED HERE, THERE'S EXCITING WORK BEING DONE ON THE EXTRAMURAL SIDE ALSO FUNDED BY THE NLM, AND BETWEEN THESE TWO AREAS THERE REALLY IS I THINK THE POSSIBILITY TO GREATLY INCREASE THE AMOUNT OF DATA THAT CAN BE SHARED ACROSS -- ACROSS RESEARCHERS. I THINK THIS IS A GOOD SEGUE INTO ONE OF OUR IMPORTANT RECOMMENDATIONS THAT CAME OUT OF MORE THAN ONE OF OUR BOARD MEETINGS, AND THAT IS THAT I REALLY DO THINK THAT THE NLM IS IN A TREMENDOUS POSITION TO CURATE CORPUSES TEXT OF ELECTRONIC MEDICAL DATA, A TREMENDOUS LIBRARY ROLE, THE BOARD FELT THAT. IT CAME UP IN A NUMBER OF AREAS, FOR EXAMPLE, BOTH ON THE EXTRAMURAL AND EXTRAMURAL SIDE, I WAS JUST TALKING THE OTHER DAY ABOUT THIS, THERE IS -- THE NLP COMMUNITY, WHEN I READ PAPERS AND GRANTS, THERE SEEMS TO BE LESS ANNOTATED TEXT THAN I WOULD EXPECT. A COUPLE HUNDRED OR A COUPLE THOUSAND. THE NLP COMMUNITY NEEDS TENS OR HUNDREDS OF THOUSANDS OF NOTES IN ORDER TO GET THESE -- IN ORDER TO GET THESE TECHNIQUES UP TO THE NEXT LEVEL. THE NOTES HAVE TO BE DEIDENTIFIED TO BE SHARED WIDELY. AND IT'S -- I THINK IT'S AN AREA WHERE THERE COULD BE -- THERE COULD BE A TREMENDOUSLY ENABLING CONTRIBUTION. MEDICINE 101, THE NEXT TOPIC, SO THIS IS I THINK A TOPIC OF SPECIAL SIGNIFICANCE TO CLEM. I KNOW HE'S TAKEN A LOT OF PERSONAL INTEREST IN THIS. AND THIS IS REALLY QUITE GOOD. SO THIS IS ABOUT -- I CALLED MEDICINE 101, IT'S REALLY MEDICATIONS 101. THIS IS ABOUT GETTING MEDICATION DATA TO THE POINT OF CARE AND ACTUALLY TESTING WHAT HAPPENS USING A PHARMACY BENEFIT MANAGER MEDICATION FEED. IT TURNS OUT THE ONE PART OF HEALTH INFORMATION EXCHANGE THAT REALLY WORKS WELL IN THE UNITED STATES AT A NATIONAL SCALE IS MOVING MEDICATION DATA AROUND. WHY? BECAUSE THERE'S BILLIONS AND BILLIONS OF DOLLARS IN IT AND THERE'S THE APPROPRIATE REGULATION BEHIND IT TO HAVE MADE IT HAPPEN, E-PRESCRIBING, AND SO BEFORE A PHARMACIST FILLS YOUR PRESCRIPTION THEY NEED TO KNOW THAT YOU'RE ELIGIBLE TO HAVE IT FILLED AND REIMBURSED AND SO THERE'S A TREMENDOUS AMOUNT OF DATA. COMPANIES CAN DELIVER ESSENTIALLY YOUR FULL MEDICATION LIST. DOCTORS DON'T KNOW WHAT YOU'RE TAKING. BUT THESE GUYS DO. AND CLEM UNDERSTOOD THIS VERY EARLY, I THINK, EVEN WHEN HE WAS BACK STILL AT REGIO REGENTS, THIS IS A REAL APPLIED PROJECT IN AN EMERGENCY DEPARTMENT, AND RECENTLY PUBLISHED IN ANNALS OF EMERGENCY MEDICINE, TOP JOURNAL IN THAT FIELD. AND IT SHOWS THAT YOU ACTUALLY GET IMPROVED MEDICATION HISTORY DATA BY USING THESE DATA STREAMS. IT'S QUITE IMPORTANT. THERE'S ALMOST NOTHING MORE IMPORTANT IN MEDICINE THAN YOUR CARE TEAM KNOWING WHAT MEDICATIONS YOU'RE TAKE. I THINK THIS IS ACTUALLY A QUITE IMPORTANT FINDING. KEEPING THEM HONEST, THIS IS THE CLINICAL TRIALS INITIATIVES HERE AT THE NLM WHICH ARE OF COURSE ABSOLUTELY WORLD CLASS, AND BEING IMITATED BY THE REST OF THE WORLD. SO JOHN IOANNIDIS PUBLISHED AN ARTICLE A FEW YEARS AGO THAT SAID THAT 95% OF MEDICAL RESEARCH IS WRONG. IT WAS CONTROVERSIAL BUT IT'S NOT ACTUALLY TOTALLY RADICAL, IT WAS PERCEIVED AS RADICAL BUT WHEN I STARTED HARVARD MEDICAL SCHOOL IN 1985, THEY TOLD US THAT WHEN I GRADUATED, 30% OF -- THAT TO 30% OF WHAT THEY WERE TEACHING US WOULD HAVE BEEN PROVEN TO BE WRONG, AND THEN THEY TOLD US -- THEY SAID, BUT WE DON'T KNOW WHICH 30%, AND THEN THEY LAUGHED. AND THEN THEY CASHED MY TUITION CHECK. THAT'S WHAT WAS BEING TAUGHT IN A NOT PARTICULARLY RADICAL WAY A LONG TIME AGO, 95%, THERE'S ALL KINDS OF PROBLEMS LIKE PUBLICATION BIAS, RIGHT? WE ONLY PUBLISH POSITIVE STUDIES. IF YOU'RE A PHARMACEUTICAL COMPANY, WHO HAS THE TIME -- YOU KNOW, IS IT EVEN IN YOUR INTEREST TO WRITE THAT UP IF YOUR INVESTIGATOR CAN EVEN GET IT PUBLISHED. IF YOU HAVE 20 STUDIES, ONE IS POSITIVE, THAT'S THE ONE THAT'S PUBLISHED, BUT IT WAS A RANDOM CHANCE, WE USE A 95% CONFIDENCE INTERVAL, THAT'S WHAT WE'RE SEEING. THE WE DEFINED THE DEDOMINATOR OF STUDDIZ STUDIES. THERE'S THIS IDEA THAT THERE ARE SMALL AND HIGHLY SELECTED STUDY POPULATIONS, IN THE LITERATURE SEEN THROUGH CLINICALTRIALS.GOV AND OUTCOMES PUBLISHED, THE MAIN FINDINGS WERE NOT NECESSARILY THE OUTCOME VARIABLES THAT WERE PICKED IN THE TRIAL PROTOCOL DESIGN. AND NOW WE CAN START TO SEE THOSE DISCREPANCIES. WE CAN ALSO SEE THAT THERE'S A FUNDING BIAS, RIGHT? CERTAIN RESULTS FUNDED BY DIFFERENT FUNDERS ARE MORE IKELY TO BE POSITIVE THAN RESULTS FUNDED BY OTHER FUNDS. SO CLINICAL TRIALS.GOV WAS ENABLED BY LEGISLATION ABOUT REQUIRING REGISTRATION OF TRIALS, THE NLM PURSUED OPERATIONALLIZING THIS WITH TREMENDOUS SUCCESS, AND ULTIMATELY EVEN MOST RECENTLY IN -- AND THERE WAS A SIGNAL EVENT IN 2005 WHEN THE JOURNAL LETTER SAID IF YOU DON'T PUBLIC -- IF YOU DON'T REGISTER BEFORE THE STUDY STARTS WE'RE NOT GOING TO PUBLISH YOUR STUDY. THIS IS HUGE HUGE. IIN 2007 NEW THINGS WERE ADDED. HERE IS AN EXAMPLE OF SOME PAPERS, SO INTERESTINGLY, A LOT OF STUDIES ARE NOT PUBLISHED. SO WE'RE SEEING DOCUMENTATION OF PUBLICATION BIAS. PUBLICATION BIAS WAS A GREAT THEORETICAL CONSTRUCT UNTIL CLINICALTRIALS.GOV GAVE US THE DATABASE. NOT EVERYTHING WAS BEING PUBLISHED. REALLY IMPORTANT TO KNOW IT. WHEN YOU KNOW IT AND MEASURE IT, HOPEFULLY THE EFFECT ACTUALLY STARTS TO GO DOWN. AND THE MAGNITUDE OF THE PROBLEM STARTS TO REDUCE. HERE IS ANOTHER EXAMPLE. COMPARISON OF REGISTERED AND PRIMARY OUTCOMES, AGAIN, IT'S NOT ALWAYS -- THIS IS THINGS WE WERE SAYING BUT NOW YOU CAN ACTUALLY QUANTIFY THIS. WHAT THEY SAID THEY WERE GOING TO STUDY IS NOT EXACTLY WHAT THEY WERE REPORTING, REPORTING THINGS THAT WERE POSITIVE, THAT'S CALLED CHERRY PICKING AND IT'S A REAL PROBLEM BECAUSE IT MEANS YOUR STATISTICAL INFERENCES NO LONGER NECESSARILY APPLY. I ACTUALLY HAVE BEEN FASCINATED BY THE DATABASE, AND PUBLISHED A PIECE ON USING IT IN ANNALS OF INTERNAL MEDICINE SHOWING, AGAIN, IF YOU WERE FUNDEDDED BY A PHARMACEUTICAL COMPANY YOU WERE MUCH MORE LIKELY TO HAVE A POSITIVE OUTCOME IN YOUR TRIAL THAN IF YOU WERE FUNDED BY THE NIH OR NOT-FOR-PROFIT. AND WE'RE ACTUALLY DOING SOME WORK WITH DEBRA'S GUIDANCE TO USE CLINICALTRIALS.GOV AS A DATA SOURCE FOR CLINICAL TRIALS MATCHING, TO PROMOTE ACCRUALS OF TRIAL. WE'RE LOOKING AT THE WAY THE DATA IS COLLECTED IN CLINICALTRIALS.GOV, CAN BE USED, THIS WAS NOT SOMETHING THE ORIGINAL LEGISLATION OR ORIGINAL DESIGN OF CLINICALTRIALS.GOV FULLY INTENDED TO SUPPORT, YET PEOPLE ARE FINDING -- INVESTIGATORS EVERYWHERE ARE FINDING THIS IS A TREMENDOUSLY USEFUL DATABASE FOR MANY THINGS BEYOND THE INITIAL -- BEYOND THE INITIAL THEME. SO LET ME JUST STAY ON TIME BY STARTING TO WRAP UP, SO THESE ARE A SUMMARY OF IMPRESSIONS WE HAVE, YOU KNOW, AT THE BOARD, CONSISTENTLY ON MEETINGS. CONTINUE THE GREAT WORK. TWO IS WHERE TOOLS ARE BEING DEVELOPED, I THINK THERE'S A REAL OPPORTUNITY TO ENGAGE THE RESEARCH COMMUNITY MORE BROADLY TO DEVELOP REQUIREMENTS AND REFINE SOFTWARE. SO THERE'S TREMENDOUS VALUE IN THE NATURAL LANGUAGE PROCESSING AND DEIDENTIFICATION, THERE'S THE OPPORTUNITY TO REALLY ENGAGE COMMUNITIES OF USERS AND TO GO OUT AND TO FOCUS THE END PRODUCTS OF THE RESEARCH ON SOME VERY USEFUL IMPLEMENTATION. TWO, WE ALWAYS HAVE A KNEE JERK REACTION ON THE BOARD TO WANT TO THINK ABOUT SYNERGIES BETWEEN NLM SERVICES LIKE PubMed AND LISTER HILL ACTIVITIES LIKE RESEARCH ON BIBLIOMETRICS, AND NATURAL LANGUAGE PROCESSING LITERATURE. WE WANT TO SEE -- KEEP PRODUCING HIGH IMPACT PAPERS. I LISTED A BUNCH WE WERE IMPRESSED BY, IT'S A VERY GOOD FOCUS TO BE REALLY REACHING FOR THE STARS ON EVERY PROJECT, AND THINKING ABOUT WHAT THOSE HIGH IMPACT PAPERS ARE AT THE END. AND I'M NOT GOING TO DISCUSS THE TRAINEE PROGRAM BUT WE DO SPEND EVERY OTHER MEETING EVALUATING THE TRAINEE PROGRAM, THROUGH THE LENS OF AN HOUR-AND-A-HALF OR TWO-HOUR MEETING WITH A SELECT GROUP OF TRAINEES, REALLY FOCUS GROUPING WITH THEM, AND, YOU KNOW, GREAT TRAINEES ARE BEING ATTRACTED. THEY ARE ON GREAT PROJECTS, AND I THINK IF ANYTHING, THIS IS AN AREA THAT SHOULD BE CERTAINLY CONTINUED AT ITS CURRENT LEVEL OR EVEN EXTENDED. THERE'S A CLOSE-UP OF THE CHERRY BLOSSOMS, I'M GOING TO HOLD IT RIGHT THERE. >> THANK YOU. >> SURE. [APPLAUSE] >> QUESTIONS, COMMENTS? >> WE HAVE -- DO YOU WANT TO DO GENERAL QUESTIONS? >> EITHER WAY. >> LET ME START BY THANKING YOU FOR A GREAT SUMMARY. I'M KIND OF A NEWBY ON THE BOARD OF REGENTS, I'D LIKE TO UNDERSTAND A LITTLE BIT OF THE ROLE OF THE BOARD OF SCIENTIFIC COUNSELORS IN TERMS OF EVALUATING INDIVIDUAL PROJECTS FOR THE OVERALL MODE OF OPERATION OF THE LISTER HILL CENTER, HOW PROJECTS ARE SELECTED TO GO FORWARD WITH FUNDING INTERNALLY, AND WHAT THE ROLE OF THE BOARD IS IN EITHER DECIDING THAT THEY SHOULD CONTINUE OR NOT CONTINUE. >> SO THAT'S A VERY GOOD QUESTION. SO THE APPROACH WE'VE TAKEN, WE CERTAINLY ARE INTERESTED IN THAT, IS TO -- YOU RECEIVE A REPORT FROM -- TO MY UNDERSTANDING, I'LL LET THE NLM FOLKS CLARIFY, MY UNDERSTANDING THERE'S TWO PRINCIPAL INVESTIGATORS AT LISTER HILL, CLEM AND THERE'S DEBRA. AND THEN THERE'S A STRUCTURE OF MANY RESEARCH SCIENTISTS UNDERNEATH EACH ONE, AND WE EVALUATE IN GENERAL CERTAIN -- WE EVALUATE RESEARCH GROUPS THAT ARE UNDER ONE OF THOSE TWO STRUCTURES. SO WE GET A REPORT GENERALLY QUITE EXTENSIVE, ON THE WORK OF -- USUALLY A COUPLE SCIENTISTS IN A FOCUS AREA, WE READ THAT REPORT AND SEND ABOUT -- WE HEAR ABOUT TWO HOURS FROM THE GROUP. AND WE THEN DELIBERATE ON THE SCIENTIFIC MERITS OF THE PRESENTED WORK. WE THINK ABOUT THE CAREER TRAJECTORIES OF THE INVESTIGATORS, AND WE MAKE RECOMMENDATIONS BACK WHICH ARE READ OUT AT THE END OF THE MORNING, THE NEXT MORNING. OVER THE P.A. SYSTEM AT THE NLM. IT'S BEEN A PROCESS WHERE -- THE INVESTIGATORS -- IT'S BEEN A PROCESS WHERE THE INVESTIGATORS ARE QUITE APPRECIATIVE OF THE COMMENTS, IN ALL CASES THEY HAVE BEEN VERY RESPONSIVE. AND THEN WE -- AT THAT POINT WE ARE -- IT'S WRAPPED UP FOR THAT SESSION. THAT'S BEEN OUR STANDARD OPERATING PROCEDURE. >> SO GREAT OPPORTUNITIES THAT NLM FACES SHEPHERDING OUR KNOWLEDGE INDUSTRY IN HEALTH CARE, AND THE INTERNAL WORK, ONE QUESTION IS HOW KEY PRIORITIES ARE IDENTIFIED, AND ALSO HOW THEY CAN ALIGN WITH THE EXPERT IN TERMS OF WHO THEY CAN HAVE SYNERGISTIC IMPACT. >> I THINK THERE REALLY IS AN OPPORTUNITY THERE, AND I THINK, YOU KNOW, THE BOARD IS VERY ORIENTED TOWARD THAT OPPORTUNITY IN THE TIMES OF RECOMMENDATIONS MADE. IF YOU'RE SUGGESTING THERE SHOULD BE, YOU KNOW, SORT OF A PROCESS ON TOP OF THAT FOR ALIGNMENT, I MEAN, YOU KNOW, YOUR THOUGHTS, BUT THAT ORIENTATION IS WHAT THE BOARD HAS, REALLY THINK ABOUT INTRAMURAL/EXTRAMURAL RELATIONSHIPS DEVELOPED AND ALSO THE KIND OF NLM SERVICES SIDE, THE NLM PRODUCESES AN AMAZING SET OF SERVICES WITH DATA LITERATURE. >> . >> ENJOYED THE PRESENTATION. A COUPLE THINGS, WE'RE MORE CONCERNED ON THE HEALTH OF THE CHART RATHER THAN THE HEALTH OF THE PATIENT, I LOVED THE PLAY ON THE MOVIE "HER," LEARNING NOT ONLY TO READ THE LITERATURE BUT FALL IN LOVE WITH THE LITERATURE, THAT WAS GREAT. I WAS INTRIGUED WITH THE CONCEPT OF NLM'S TRANSFORM ATIVE ROLE IN IN THE MEDICAL DATA BECAUSE ALL OF OUR INSTITUTIONS MAKE DATA AVAILABLE FOR OUR RESEARCHERS, SO WHAT OTHER ROLES ARE IN PLAY IN HELPING WITH THAT? >> WELL, I THINK THAT EITHER THROUGH -- I DON'T WANT TO GET TOO SPECIFIC BUT IT'S A LIBRARY. INFORMATICS. I THINK IN ORDER TO CURATE AND MAKE AVAILABLE LARGE -- IT HAS THREE THINGS GOING FOR IT. IN ORDER TO MAKE AVAILABLE A CORPUS OF TEXT THAT WOULD BE USABLE, EITHER, YOU KNOW, EITHER PUBLICLY, WHICH IS HARD, OR UNDER A USE AGREEMENT, WHICH IS COMPLICATED BUT POTENTIALLY NECESSARY IN CASE THERE WAS SOME IDENTIFIER LEFT, THERE'S A LOT OF TALK ABOUT THIS IN THE COMMUNITY. THERE ARE MOVEMENTS TOWARD DATA DONATION AND EVEN FUNDED PROJECTS IN THAT AREA, BY RWJ, WHAT PEOPLE HAVE REALIZED IS THAT OUR DATA SOURCES ARE VERY FRAGMENTED AND THE PRIVACY CONCERN LIMITS SHARING, YET WE SEE AGAIN AND AGAIN INFORMATION IS STIFLED BECAUSE OF THE LACK OF TYPES OF DATA. SO IT IS NOT SOMETHING TO BE TAKEN ON LIGHTLY BECAUSE THERE ARE VERY SIGNIFICANT TECHNICAL AND ALSO REGULATORY CONCERNS. BUT IN REAL DATA, DEVELOPING THE NEXT GENERATION APPLICATIONS, THE NEXT GENERATION, NATURAL LANGUAGE PROCESSING, AND IF THERE WERE A HUNDRED THOUSAND NOTES OR A MILLION NOTES, OR, YOU KNOW, ACROSS DIFFERENT FIELDS, I THINK IT WOULD BE VERY VALUABLE. IT'S NOT A MINOR EFFORT. IT'S JUST ONE THING TO CONSIDER AMONGST OTHER PRIORITIES, BUT I DO THINK THAT THERE'S NOT ONLY A NEED WITHIN THE VARIED RESEARCH GROUPS THAT ARE FUNDED INTRAMURALLY AND EXTRAMURALLY BY THE NLM -- GNAWED. >NLM - [ INAUDIBLE ] >> I ENJOYED YOUR PRESENTATION AND I'M PARTICULARLY MOVED -- >-- >> THIS IS CLEM. CAN YOU HEAR ME? >> WE CAN HEAR YOU WELL. >> I'VE BEEN LISTENING. I'M TECHNICALLY CHALLENGED HERE. I CAME FROM SUCH A GOOD AND INTERESTING PRESENTATION, AND I HAVE TO AGREE WITH MUCH OF WHAT HE SAID, THAT THERE ARE PROBLEMS, AND ANYWAY I DON'T THINK FROM A REMOTE I CAN GET INTO IT TOO MUCH BUT I'M HERE LISTENING ANYWAY. >> GOOD. KEEP ON. MENT KEEP ON. >> OKAY, BACK TO HENRY. >> I'M MOVED BY YOUR RECOGNITION OF THE PHARMACY MANAGERS AND THEIR CAPACITY, ALTHOUGH THE DATA THEY HAVE ABOUT PARTICULAR PATIENT MEDICATION HISTORY AND HOW WE NEED TO BE ABLE TO UTILIZE THAT DATA FOR BETTER HEALTH OUTCOMES, WHETHER OR NOT THE PATIENT IS TAKING THE MEDICATION PROPERLY OR NOT. BUT I WAS ALSO MOVED BY THE -- I THINK I UNDERSTOOD YOUR STATEMENT CORRECTLY TO SAY THAT CLINICAL TRIALS FUNDED BY PHARMACEUTICAL COMPANIES HAVE GREATER PROPENSITY FOR POSITIVE OUTCOMES THAN OTHER TRIALS. >> YES. >> WHAT DO YOU ATTRIBUTE THAT TO? [ LAUGHTER ] >> TWO WAYS TO LOOK AT IT. PROBABLY BOTH. ONE IS THAT PHARMACEUTICAL COMPANIES ARE ADEPTH AT DOING TRIALS. THEY KNOW HOW TO GET THINGS DONE. THEY ALSO PUT A LOT OF THOUGHT INTO WHAT TRIALS THEY WOULD TAKE ON IN THE FIRST PLACE. AND THERE'S NO DOUBT THAT THAT IS PART OF IT BUT THERE'S ALSO CONCERN, IT'S BEEN DOCUMENTED IN SOME OTHER PIECES, THAT, FOR EXAMPLE, THE OUTCOMES REPORTED IN THE TRIALS, IN THE LITERATURE ARE DIFFERENT THAN THE OUTCOMES THAT WERE SELECTED FROM PROTOCOL. AND ONCE YOU'VE DONE THAT, THEN YOUR STATISTICAL THRESHOLD NO LONGER APPLIES TO FINDINGS SIGNIFICANT TO THE OUTCOME. SO IF YOU LOOK THROUGH 50 OUTCOMES AND REPORT ON THE TOP THREE, YOUR WORKING -- [ INAUDIBLE ] CHERRY PICKING IS ONE OF THE REAL AREAS WHERE CLINICALTRIALS.GOV REGISTRY HELPS ENFORCE GOOD BEHAVIOR. THERE'S ALSO A TREMENDOUSLY INTERESTING NEW SOURCE OF PROTOCOL DATA, THAT ARE BECOMING MORE PUBLICLY AVAILABLE, PUBLICLY AVAILABLE ACTUALLY AND THE FREEDOM OF INFORMATION ACT, CASES, AND SOME AVAILABLE THROUGH OTHER MEANS. WE'LL LEARN MORE AND MORE ABOUT THIS. THERE'S OPENNESS AND NLM HAS BEEN REALLY SO BEHIND THE OPEN DATA, OPEN INFORMATION ABOUT TRIALS, OPEN TOWARD SOFTWARE. THESE MOVEMENTS ARE BECOMING DOMINANT IN THE MEME OUT THERE, AND NLM HAS BEEN, YOU KNOW, ONE OR TWO DECADES AHEAD OF THE CURVE, AND I THINK STILL HAS THE OPPORTUNITY TO MAINTAIN A LEADERSHIP ROLE IN THESE AREAS. >> FOLLOWING UP ON WHAT IS THE ROLE OF YOUR COMMITTEE, THERE ARE A LOT OF GROUPS WORKING ON DEIDENTIFICATION TECHNIQUES. NLM COULD PRAY A ROLE AS ARBITER. DO YOU DESIGN TESTS TO SEE WHO DOES BETTER AND PUBLISH THAT SORT OF STUFF? IT WOULD SEEM RATHER THAN DEVELOPING YET ANOTHER METHOD FOR DEIDENTIFICATION, SERVING AS THE TRUSTED ARBITER WOULD BE MORE APPROPRIATE ROLE. >> I THINK THAT'S INTERESTING. ONE OF OUR RECOMMENDATIONS DID TOUCH ON THAT THEME. NONE OF THE DEIDENTIFICATION -- I'M SURE THIS HAPPENED IN DEIDENTIFICATION, BUT IN THE NATURAL LANGUAGE PROCESSING OF TEXT THERE WAS A TEST FUNDED BY -- SPONSORED BY ONE OF THE NLM PROJECTS. AND MANY DIFFERENT SOFTWARE PROJECTS COMPETED AGAINST EACH OTHER. I THINK THAT'S ABSOLUTELY RIGHT, THERE COULD WELL BE A ROLE FOR EVEN DEVELOPING STANDARDS FOR DEIDENTIFICATION, HIPA, FOR EXAMPLE, GIVES US A GENERAL HARD TO INTERPRET FRAMEWORK FOR WHAT IS SUFFICIENTLY DEIDENTIFYING AREAS, AND NLM IS INTERESTED IN STANDARDS, STANDARDS ARE BENCHMARKING, THAT'S A GREAT IDEA. >> I'D LIKE TO FOLLOW UP ON THAT COMMENT. I'M NOT QUITE SURE OF THE RIGHT LANGUAGE, BUT SOME PAPERS AND REPORTS ARE BIGGER THAN OTHERS. AND THIS MORNING IN THE "NEW YORK TIMES," THERE'S A LONG ARTICLE REPORTING ON A BMJ ARTICLE WITH REGARDS TO 90,000 WOMEN IN CANADA WHO WERE RANDOMIZED TO MAMMOGRAPHY AND THE RESULTS ARE NO BETTER. THAT IS A PAPER THAT WILL BE VERY BIG IN RADIOLOGY. I CAN SEE THE PARTIES LINING UP AT THIS VERY MINUTE. THERE'S COLLEAGUES OF MINE WHO ARE EMOTIONALLY INVOLVED WITH MAMMOGRAPHY THAT ARE GOING TO THROW THIS PAPER AWAY. AND I'M SURE THEY ARE GOING TO START TALKING ABOUT THE RANDOMIZATION WASN'T RIGHT, BLAH, BLAH, BLAH, BUT IS THERE A ROLE FOR SOMEONE TO BE UNBIASED IN IT A REVIEW OF THOSE KINDS OF ARTICLES? THE "NEW YORK TIMES," I HAVEN'T READ THE ARTICLE, BUT THE "NEW YORK TIMES" SAID THAT SWITZERLAND IS STOPPING MAMMOGRAPHY AS A TECHNIQUE. SO IT'S GOING TO BE VERY BROAD INTEREST IN THIS. IS THAT PART OF WHAT YOU'RE THINKING ABOUT? >> WELL, FIRST OF ALL, I THINK THAT'S A LANDMARK ARTICLE. YOU KNOW, I WILL DIGRESS FOR ONE MOMENT TO SAY IN 1992 OR SOMETHING, MY MOTHER HAD A MAMMOGRAM AND, YOU KNOW, IT WAS SUPPOSEDLY -- IT WAS SUPPOSEDLY POSITIVE, THE BIOPSY WAS SUPPOSEDLY POSITIVE, AND I GO TO THE LITERATURE. I REALIZE THERE IS NO LITERATURE. AND SINCE THEN I'VE SEEN MANY OF MY FRIENDS WITH FALSE POSITIVE SCARES, I'VE DONE THE NUMBERS IN MY HEAD BUT I DIDN'T HAVE THE TIME. THIS IS A STUDY I WOULD LOVE TO DO BECAUSE I THINK ACTUALLY IT'S A PERVASIVE THEME IN MEDICINE, THAT WE DON'T UNDERSTAND THE NUMBERS IN TERMS OF DIAGNOSTIC TESTING, SENSITIVITY, EVEN SIMPLE. AND WE IMPLEMENT SCREENING PROCEDURES THAT ARE NOT RIGHT. THERE ARE QUANTITATIVE -- I THINK THERE ARE SOME SIMPLE QUANTITATIVE CONCEPTS THAT ARE NOT WELL UNDERSTOOD BY PRACTICE INFORMATION. AND NLM IS CLEARLY IN AN EDUCATIONAL ROLE, DESIGNING TRAINING PROGRAMS, WHETHER IT COULD REVIEW THE LITERATURE I'M NOT SURE BUT WHETHER IT COULD ENFORCE THE BIG DATA THINKING, AND QUANTITATIVE THINKING, THAT WOULD HELP A GENERATION OF RESEARCHERS NOT MAKE THEM SAFE, I THINK IT IS IN THAT SPACE A BIT, BUT IT COULD BE FORMALIZED FURTHER. >> WE COULD ENCOURAGE SCREENING PROGRAMS. >> IT WOULD BE NICE. >> I THINK SO TOO. >> I'LL TELL YOU SOMETHING, I'M TEACHING A CLASS IN THE PROGRAM, I ASKED -- ONLY TWO OUT OF THE TWELVE PEOPLE KNEW -- THEY WEREN'T ALL IN THE TRAINING PROGRAM, TWO IN TWELVE PEOPLE IN MY CLASS KNEW WHO FRANCIS COLLINS WAS. SO, YEAH, I THINK THERE IS THE OPPORTUNITY WITH EACH TO THINK ABOUT THE RELATIONSHIP BETWEEN THOSE TRAINING PROGRAMS AND SAVING MEDICINE FROM ITS OWN $1.7 TRILLION BUDGET. >> THIS THING ABOUT THE DEIDENTIFIED GROUP, I AGREE COMPLETELY, THE IMPORTANCE. WHEN WE STARTED THE PROGRAM, 26, 27 YEARS AGO, ONE OF THE FIRST THINGS WE DID WAS COMMISSION ONE OF OUR COLLEAGUES TO GIVE US DEIDENTIFIED FACTS, THIS WAS RANDY MILLER, NOT A NEWCOMER TO THE WHOLE FIELD. HE SERIOUSLY UNDERTOOK TO DO THIS, THE NUMBER WAS 50, 75 CASES, SOMETHING LIKE THAT. HE WORKED LIKE A TROJAN AND CONVINCED HIMSELF AND US THAT THAT IS PLENTY HARD TO DO. HOWEVER LARGE YOU SCRUB THAT, IN THE MIDDLE OF TEXTS DOCTOR BLANK, BLANK, BLANK. SO I MEAN IT ISN'T THE CASE THAT THERE'S SIX WONDERFUL METHODS NEEDING TO BE TESTED, NO ONE IS DOING THIS THING RIGHT. THIS IS A VERY DIFFICULT PIECE OF WORK TO DO. AND TO CONSIDER IF NLM SHOULD UNDERTAKE IT, YOU GOT TO THROW SOME MONEY ON THE TABLE. >> THE MANUAL PROCESS. >> I'M JUST SAYING TO MAKE A NEW COMMITMENT, YOU REALLY LOOKED AT TWO THINGS THAT WE'RE GAMBLING ON. WE GAMBLED ON THIS SEMANTIC STUFF FOR 20 YEARS. NOW, THE CLINICAL TRIALS, THERE WAS LEGAL BASIS FOR IT, BUT IT WAS A BIG, BIG GAMBLE. WE CONTINUED TO STOKE THAT. NO ONE KNEW -- THAT'S ALL FUNDED OUT OF NLM APPROPRIATIONS, ZERO HELP FROM NIH IN SPITE OF THE FACT THEY WERE UNDER LEGAL OBLIGATION TO DO IT. SO THERE ISN'T A LOT OF GAMBLING MONEY LEFT. >> YOU REMEMBER THAT LADY AT CARNEGIE-MELLON WHO DEMONSTRATED THAT INFORMATION THAT SHE COULD REACT TO AS WELL. >> APPRECIATE IT. >> THANK YOU, KEN. WE NEED TO MOVE ON TO DR. LANDSMAN AND THE BOARD OF SCIENTIFIC COUNSELORS. >> HI. THANK YOU FOR INVITING ME. DAVID LIPPMAN ASKED ME TO DO THIS. I'M GOING TO DO A DIFFERENT JOB THAN WHAT WAS DONE PREVIOUSLY FOR LHC'S. I'LL GO THROUGH THE PROCESS MORE IN DETAIL AND THEN GO ON TO THE PROJECTS AND PEOPLE AND TALK ABOUT IT. I'M CHIEF OF THE COMPUTATIONAL BRANCH. I HAVE MULTIPLE ROLES AT THE NIH. I MANAGE THIS GROUP WHICH IS OVER A HUNDRED PEOPLE, THE COMPUTATIONAL BIOLOGY BRANCH, I HAVE A SMALL RESEARCH GROUP, SMALL BECAUSE I DO A LOT OF MANAGEMENT AS WELL. SO I'M ALSO CO-DIRECTOR OF NIH BOSTON UNIVERSITY GRADUATE PARTNERSHIP PROGRAM IN BIOINFORMATICS, GREAT STUDENTS AT THE NIH, SOME AT NCBI TOO BUT THAT'S AN NIH-WIDE PROGRAM. I CAN TALK A LOT ABOUT THAT IF YOU WISH BUT THAT'S ONE OF THE THINGS I DO. I'M ALSO REPRESENTING NLM AT THE TRAINING DIRECTOR COMMITTEE, I'M PART OF THAT TOO SO I'M QUITE INVOLVED IN SOME OF THE TRAINING THAT GOES ON IN BIOINFORMATICS. I'M ALSO THE DEPUTY SCIENTIFIC DIRECTOR OF NCBI, I DON'T HAVE A TITLE BUT REPRESENTED THE BOARD EVERY TWO WEEKS THEY HAVE A MEETING AND WE TALK ABOUT NIH POLICIES AND PROGRAMS IN GENERAL AND IN REGIONAL PROBLEMS TO. NIH INTRAMURAL PROGRAM, TIMES OF PEOPLE THAT ARE IN THAT PROGRAM WHO DO RESEARCH, I WAS A SENIOR INVESTIGATORS, YOU HAVE TO GO THROUGH THE TENURE PROCESS WHICH IS SIMILAR TO UNIVERSITY ONE. WE HAVE AN APPOINTMENT WHICH LASTS QUITE A LONG TIME, WE REVIEW RETRO SPECTIVEL. WE HAVE A SCHEDULE, EVERY YEAR WE HAVE PEOPLE PRESENTING. FOR THE TENURE TRACK INVESTIGATORS, CONVERTED AFTER 6-7 YEARS. WE HAVE TWO INDIVIDUALS, WE'VE HAD MORE IN THE PAST, WHO HAVE BECOME TENURED BUT I SHEPHERD AND MENTOR THEM THROUGH THE PROCESS OF TENURE. THEY GET REVIEWED IN THE MIDDLE, WHICH I'LL TALK ABOUT, BEFORE SIX YEARS AND THEN THEY SOMETIMES -- SOMETIMES MID-TERM REVIEWS CAN END A TENURE PROCESS IF AN INVESTIGATOR IS NOT SHOWING APPROPRIATE PROGRESS, THEN THEY CAN BE CONVERTED TO A STAFF SCIENTIST OR GIVEN AMPLE TIME TO FIND A NEW JOB. CONVERSION TO TENURE INVOLVES -- IT MEANS YOUR PROGRAM IS SUPPORTED AS LONG AS YOU MAINTAIN A CERTAIN LEVEL OF PRODUCTIVITY. THE TENURE PROCESS INVOLVES OUR BOARD OF SCIENTIFIC COUNSELORS AND TWO OTHER COMMITTEES OF ONE NCBI AND CENTRAL TENURE COMMITTEE. NIH TENURE IS UNIFORM ACROSS ALL INSTITUTES. OH, SORRY. THERE ARE ABOUT 900 INVESTIGATORS ON THE INTRAMURAL PROGRAM. I MENTIONED NIH IS POST-DOC FELLOW CENTRIC ENVIRONMENT. WE HAVE GRADUATE STUDENTS FROM ALL OVER THE COUNTRY AND THE WORLD. AND SO BUT THEY FAR OUTNUMBER. IT'S DIFFERENT FROM THE UNIVERSITY ENVIRONMENT. MOST OF THE PEOPLE THAT DO THE GOOD WORK IN OUR LABS ARE POST-DOCS. AND NCBI HAS, IN MY BRANCH, 40 TO 45% OF THE STAFF ARE POST-DOCS. GOING BACK AGAIN TO NIH, WELL KNOWN SCIENTISTS HERE THAT HAVE GOT NOBEL PRIZES, MARSHAL NIRENBERG, JULIUS AXLEROD. THE FIELDS ARE VERY VARIED AND WHEN YOU SEE THE TOPICS WE WORK ON AT NCBI YOU'LL SEE HOW DIFFERENT THEY ARE. NCBI HAS A DIRECTOR, DAVID LIPPMAN, DENNIS BENSON WHO YOU MET YET. THIS IS THE INFORMATION RESOURCES BRANCH WHICH SUPPORTS OUR ENDEAVORS, NETWORK, SOFTWARE, SECURITY, THINGS LIKE THAT. THERE'S A LIST OVER HERE. JIM OSTELL'S BRANCH IS THE LARGEST. WHEN YOU TALK ABOUT ALL THE GOOD THINGS ONLINE THAT'S PRODUCED OVER HERE. THE COMPUTATION BIOLOGY BRANCH IS WHERE WE DO THE BASIC RESEARCH IN A VARIETY OF FIELDS WHICH WE'LL TALK ABOUT. ALL THE STARS NEXT TO PEOPLE ARE THE PEOPLE WHO ARE REVIEWED BY OUR BOARD OF SCIENTIFIC COUNCIL, ALL THE INVESTIGATORS, TENURE TRACK INVESTIGATORS, AND THERE ARE SEVERAL SENIOR SCIENTISTS WHO REQUIRE REVIEW. THESE ARE STANDARD NIH PROTOCOL REVIEWS WHICH I'LL TALK ABOUT. IN ADDITION TO THAT WE HAVE OUR BOARD OF SCIENTIFIC COUNSELORS. THEY HAVE TERMS OF FOUR YEARS, PLUS ONE WE CAN EXTEND THEM FOR ONE EXTRA YEAR. CURRENT LIST IS OVER HERE. THIS IS CHRISTINE SEIDMAN, A CARDIOLOGIST INTERESTED IN HEART CONDITIONS. THIS IS DAVID LIPMAN, SEVEN MEMBERS ON THE BOARD, THE STARS, PATSY, CRAIG AND CHRIS ARE LEAVING THE BOARD AT THE END OF JUNE, REPLACED BY A NEW SET OF PEOPLE. THERE ARE A VARIETY OF TIMES OF PEOPLE ON THE BOARD, KIP GUY, CHRIS LEE IS A BIOCHEMIST, CHUNG WU IS A POPULATION GENETICIST, SCOTT EDWARDS VALERIE IS A GENOMICS PERSON, PATSY IS A FUNCTIONAL PERSON. AT TIMES WE ALSO ADD AD HOC MEMBERS. IF SOME PROJECTS NEED TO BE REVIEWED AND THEY DON'T HAVE EXPERT AMONGST BOARD MEMBERS THEY SAY WE NEED SOMEBODY TO HELP US. SO WE INVITE FOR ONE MEETING AD HOC MEMBERS AND THEY DO A VERY GOOD JOB. PREVIOUS MEMBERS, ARE LISTED HERE, WELL KNOWN NOBEL PRIZE WINNERS, ANDY FIRE WAS ON THE BOARD WHEN HE GOT THE NOBEL PRIZE. WE EXPECTED HE WOULD NOT BE COMING BECAUSE HE WAS ON THE RADIO EVERY OTHER DAY, AND HE WAS TRAVELING AND GIVING TALKS, AND WE SAID WHAT'S GOING ON, HE SAID I'LL BE THERE, I HAVE NO PROBLEM. HE WAS HERE AND SAT DOWN OVER HERE AND CONTINUED TO DO HIS JOB. THAT WAS GREAT. BUT AT THE TIME, IT WAS VERY INTERESTING HAVING HIM ON THE BOARD BECAUSE HE HAD THIS PASSION AND WE HAD SOME INTERESTING RESEARCH PROJECTS RELATED TO THAT. SO IT WAS VERY GOOD. SO OTHER PEOPLE ON THE BOARD, TRUDY MACKAY WHO IS I BELIEVE ON THE BOARD OF REGENTS USED TO BE ON OUR BOARD TOO. I DIDN'T HER HERE. >> SHE LEFT. >> BECAUSE OF THE STORM. >> OH. NOT A LONG WAY, NORTH CAROLINA. BUT THE BOARD HAS WORKED EXTREMELY WELL AND THEY COVER A WIDE VARIETY OF FIELDS FROM MEDICINE, DAVID GINSBURG, GENOMICS, CHEMISTRY, BIOLOGY, EVOLUTION, BOB SAUER, AND SO ON. ART LEVINE USED TO BE A SCIENTIFIC DIRECTOR. MEETINGS TWICE A YEAR, SPRING AND FALL. NEXT ONE IS IN APRIL. AND DECEMBER. WE SPLIT THE FUNCTIONS. THE FIRST MEETING OF THE YEAR IS REVIEWS OF RESOURCES, JIM OSTELL WILL PRESENT THAT, LIKE YOU'LL HEAR FROM WENDY ABOUT WHAT'S GOING ON ABOUT OUR RESOURCES, WHAT WE'RE PLANNING, AND THEY DIRECT VERY STRONGLY ON WHAT'S GOING ON AND ENJOY -- WE ENJOY THE INPUT AND THEY ACTUALLY HAVE WAY MORE TO SAY. IT'S VERY GOOD BECAUSE YOU HAVE YOUR USER -- HIGH END USER COMMUNITY COMING BACK. THIS IS WHAT YOU NEED TO DO. IN THE FALL MEETINGS WE RESERVE TIME FOR THE REVIEW OF–r THE RESEARCH PROJECTS. AND I'LL TALK A LITTLE BIT ABOUT THAT. EACH P.I., PROFESSOR, SENIOR INVESTIGATORS, IS REVIEWED EVERY FOUR YEARS, AS I SAID. AND THESE ARE THE SENIOR INVESTIGATORS, THE DATES THEY WERE REVIEWED. I, MYSELF, AND DAVID LIPPMAN WILL BE REVIEWED DECEMBER THIS YEAR, EVEN THOUGH WE'RE PART OF MANAGEMENT WE STILL GET TO PRESENT. ZHIYONG LU ABOUT PRESENT IN THE MIDDLE OF HIS TENURE. HE IS A COMPUTER SCIENCE Ph.D. AND WORKS ON TEXT MIMING AND HAS SEVERAL PROJECTS GOING AT THE UNIVERSITY OF COLORADO. IVAN WENT THROUGH HIS SECOND REVIEW FOR TENURE, OCTOBER OR NOVEMBER OF LAST YEAR, WE'RE IN THE PROCESS OF TRYING TO CONVERT HIM TO TENURE. IT'S A LABORIOUS PROCESS. WE ASSEMBLE A PACKAGE AND REVIEW THE PACKAGE TO COMMITTEE AND THEN SEND THAT TO THE COMMITTEE. HOPEFULLY THAT WILL BE DONE. I'M NOT GOING TO GIVE A DATE BECAUSE IT RELIES ON THE LETTER WRITERS AND IN THE RANGE OF 15 TO 20 LETTER WRITERS FOR THAT. AND THEN THERE'S THE SIX SENIOR s WELL, WENDY YOU'LL„i„i BE HEARING LATER IS ONE OF THEM. SO WHAT„i THEY DO, EVERYBODY HAS TO PREPARE A PACKAGE OF ABOUT 15 OR 20 PAGES EVERY FOUR YEARS, AND NOW YOU REVIEW RETRO SPECTIVELY. YOU HAVE TO SAY WHAT YOU PLAN TO DO. IT'S NOT A BIG„i PART OF YOUR PACKAGE BUT YOU NEED TO KNOW OU'RE S TOQj:‡Q YOU¨—¨Ki HAVE A CHOICE AND A 40-MINUTE PART. THEY HAVE THE WHOLE PACKAGE WHICH HAS ENORMOUS AMOUNT OF INFORMATION. AND THERE'S A LARGE INTERACTION BETWEEN -- LARGE IN TERMS OF TIME FOR THEM, AND THEY ALSO HAVE A PRIVATE TIME WITH EACH INVESTIGATOR. EVERYBODY -- ALL THE MEETINGS ARE OPEN, EXCEPT FOR THE PRIVATE TIME WITH THE P.I. AND BOARD WHERE THEY CAN FEEDBACK TO THE BOARD WHAT THEY ARE THINKING OR ANYTHING IN ADDITION TO THAT. THIS PROCESS TAKES PLACE ABOUT A MONTH BEFORE. THE QUALITY OF THE RESEARCH PROJECT IS -- PROJECTS ARE DISCUSSED AT GREAT LENGTH BY THE BOARD AND THAT'S WHAT THEY REALLY CONCENTRATE EFFORTS ON IN WRITING THE REPORT. THEY LOOK AT OTHER THINGS. NIH ALSO REQUIRES THAT THEY LOOK AT CERTAIN SPECIFIC THINGS LIKE MENTORING AND TRAINING OF FELLOW HAS BECOME A BIG THING ON CAMPUS HERE SO THERE'S INTERACTION OF THE BOARD WITH POST-DOCS AND STUDENTS WHICH HAPPENS DURING THAT NORMALLY -- BEFORE LAUNCH. THEY ALSO RECOMMEND AS THE CASE WITH THE TENURE TRACK AT THE MOMENT, AT THE APPROPRIATE TIME THEY WILL RECOMMEND WHETHER TENURE SHOULD BE INITIATED, THAT'S THE FIRST PART OF THE TENURE PROCESS ON THE NIH CAMPUS. AND SO UNTIL WE GET A POSITIVE SIGN FROM THE BOARD, WE WILL NOT DO ANYTHING ABOUT TENURE. SOME INSTITUTES GRADE THEIR REVIEWS, EXCELLENT, OUTSTANDING, SO ON. ADEQUATE, BELOW AVERAGE, SO ON. OUR BOARD DOESN'T USE SPECIFIC WORDS BUT THEY CONVEY THE MESSAGE, WHETHER THEY ARE NOT HAPPY OR HOW STRONG THEY FEEL ABOUT THE PROJECT. SO A VERY STRONG REVIEW COULD RESULT IN INCREASED SUPPORT, NOT LIKELY AT THIS STAGE, WITH A BUDGET WE HAVE. BUT CERTAINLY YOU GET A PAT ON THE BACK. STATUS QUO IS MAINTAINED WITH QUALITY REVIEWS, A VERY GOOD REVIEW YOURˇlD GROUP.„i¨—„iw3r ir„iZ9¨r¨'D HAP PENS TO POOR REVIEWS? THE NIH HASxD A„i DETENURING„i PROCESS THAT TAKES YEARS. IT'S FRAUGHT WITH LEGAL ISSUES BUT THERE'S A PROCESS THAT YOU CAN GO THROUGH, SCIENTIFIC DIRECTOR CAN DE-TENURE AN INDIVIDUAL WHO IS NOT PERFORMING WELL. IT HAS HAPPENED ON THE CAMPUS, NOT AT THE NCBI, AND IT'S RARE, I MUST SAY. AND ONE HAS TO BE REALLY CAREFUL IN DOING THAT. IT MIGHT BE CHEAPER TO TAKE THE RESOURCES AWAY FROM AN INDIVIDUAL THAN TO DE-TENURE. YOU COULD LOSE SOME RESOURCES, AND IN SOME CASES THERE'S BEEN AN AGREEMENT BETWEEN THE SCIENTIFIC DIRECTOR AND THE INVESTIGATOR, A STAFF SCIENTIST, A GOOD POSITION TO BE DOING RESEARCH BUT YOU'RE NOT A P.I. ON THE CAMPUS, YOU WORK FOR SOMEBODY ELSE. SO LET ME TALK ABOUT THE INDIVIDUALS A LITTLE BIT. IVAN IS THE GUY WHO IS GOING TO BE UP FOR TENURE SOON. HE'S VERY INTERESTED IN GENE REGULATION, WE DON'T USE ALL OUR GENES IN ALL OUR CELLS ALL THE TIME, WE USE A SMALL FRACTION IN THE CELL LINE. IN BLOOD, YOU WANT HEMOGLOBIN ISN'T -- SYNTHESIZED, IN OTHERS YOU DO NOT. HE COLLABORATES VERY STRONGLY WITH SEVERAL LABS ALL OVER THE COUNTRY ON PROJECTS RELATED TO REGULATORY GENOMES WHICH HELP IN A VARIETY OF THINGS, BUT SOME THINGS THAT DO HAPPEN IN THOSE REGIONS IS YOU GET MUTATIONS IN THE REGULATORY AREA. STEVE AN IS AN -- STEPHEN IS A MATHEMATICIAN KNOWN FOR HIS WORK WITH DAVID LIPMAN, MODELING SEQUENCE ALIGNMENT, STATISTICAL MODELS, CONTINUES TO WORK ON THE BLAST ALGORITHM. AND REFINEMENTS IN THAT HAVE BEEN TAKING PLACE FOR THE PAST 20 YEARS, REFINEMENTS, IMPROVEMENTS AND MORE ACCURATE STATISTICS HELP IN THE DIFFERENT TOOLS THAT WE NEED TO CREATE AT THE NCBI. WE OFFER THE BLAST AS A SERVICE, SO WHATxNi¨—„ixD HAPPENED WIT0D„i¨— THIS RESEARCH IT ENDED UP AS A RESOURCE IN INFORMATION ENGINEERING BRANCH AND PROBABLY ONE OF THE MOST USED -- PROBABLYA THE„i¨—„i„ixD–L¨E#N PART OF–r THE–r¨¨„–r NCBI. IN ADDITION, MOST PEOPLE SET UP WEBSITES ABOUT SEQUENCES HAVE A BLAST RUNNING FROM IM. THAT'S BEEN PRETTY PRODUCTIVE. THIS THEORETICAL PHYSICIST CONCENTRATES ON MASS CPECTROMETRY, YOU HAVE -- YOU TELL WHAT PATHOGENS ARE IN THE SAMPLE. HE'S WORKING CLOSELY WITH A LAB. IT'S COMPLICATED WORKING WITH A LAB IN MASS SPECTROMETRY BUT THEY WILL BE ABLE TO GET THIS GOING IN THE NEXT YEAR OR SO. HE'S ALSO INTERESTED IN ENTER -- INTERACTION BETWEEN PROTEINS. THOSE ARE ALL NETWORKS, SO WE SHOULD BE AWARE NO PROTEIN IS LOWLY. HE'S MAINTAINED A PROJECT THAT ONLY HE WORKS ON, I DON'T KNOW WHAT THIS MEANS REALLY. HE'S THE SECOND GUY TO WORK ON THIS. THE FIRST GOT THE NOBEL PRIZE, HE'S WORKING–r–rw3 ON–r„i ACTUALLY WRITING CODE TO SOLVE THE PROBLEM. THE IDEA CAME UP FROM THE FIRST GUY. SO WHEN„i HE WRITES HIS PAPER ACTUALLY HE TELLS ME, I CAN'T FIND REVIEWERS. THE GUY WHO GOT THE NOBEL PRIZE IS NOT AROUND ANYMORE. AND SO IT'S A TOUGH PROBLEM. BUT HE'S WORKING ON IT. HE'LL ARRANGE TO GET THAT PUBLISHED. EUGENE, YOU MIGHT KNOW, A GENOMEICIST, A PROLIFIC WRITER, PUBLISHES AN ENORMOUS AMOUNT. I WANTED TO HIGHLIGHT A PROJECT, ONE OF THE MOST -- RESULTED IN ONE OF THE NEW TECHNIQUES, CRISPR/CAS, REMOVING FOREIGN DNA FROM THE GENOME. AND THAT WAS DISCOVERED IN THE MID-2000'S AND THE KEY FUNCTION OF THE CRISPR/CAS WAS IDENTIFYING THIS GROUP. THEY COULD SPECULATE IN AN ARTICLE THAT THEY WROTE, THEY ADDED IN THE FACT THAT THEY THINK RNA IS INVOLVED IN THE SYSTEM AND WHEN PEOPLE READ THAT, THEY SAID, MY WORD, THAT'S THE KEY. AND RNA AND PROTEINS ARE -- AND NUCLEUS IS INVOLVED, WHY IS IT IMPORTANT? BECAUSE YOU CAN USE THAT SYSTEM NOW TO MAKE CRISPR/CAS SYSTEM WHICH TARGETS HUMAN CELLS, YOU CAN KNOCK OUT GENES QUITE SPECIFICALLY, AND SO THERE ARE PROJECTS PEOPLE WANT TO REMOVE GENES AND THEY CAN DO IT WITH A CRISPR/CAS SYSTEM. KOONIN GETS CREDIT FOR FORMALIZING FAMILIES OF PROTEIN THAT THE SYSTEM USES. INTERESTINGLY, UP UNTIL THAT TIME IN 2002, WHEN HE PUBLISHED THE FIRST PAPER, EVERYBODY THOUGHT IT DOESN'T EXIST IN MORE THAN A HANDFUL OF BACTERIA. TURNS OUT IT EXISTS IN EVERY BACTERIA BUT MANY ACTUALLY IT DOESN'T WORK FOR SOME REASON OR ANOTHER. E. COLI HAS A CRISPR/CAS SYSTEM BUT IT DON'T WORK THAT WAY. WE DON'T KNOW WHY AT THIS STAGE. JOHN, ANOTHER P.I. AT THE NCBI, WORKED WITH DAVID LIPPMAN IN THE EARLY '80s ON SEQUENCE COMPARISON ALGORITHM, DAVID LIPPMAN AND WILLEBRAND BILL PEARSON WAS THE THIRD PERSON. HE HAS CON TRAITED ON -- CONCENTRATED ON TEXT COMPARISON. I RECENTLY FOUND OUT THERE'S A YOUNG MAN, HE'S NOT DAVID, BUT IT'S IMPORTANT, AND PubMed CENTRAL INDEXING, PubMed CENTRAL FULL TEXT ONLINE, HE'S INVOLVED EXTENSIVELY. SO MANY OF THESE PROJECTS THAT JOHN WILBUR WORKS ON END UP AS RESEARCH, START AS RESEARCH PROJECTS IN THE COMPUTATION BIOLOGY BRANCH AND GET THEM NEW TOOLS FOR USE ON PubMed AND OTHER WEBSITES. VERY PRODUCTIVE ENVIRONMENT. MY RESEARCH, I'VE HAD A LONG STANDING INTEREST IN CHROMATIN STRUCTURE, I SPENT TIME LOOKING AT NUCLEAR PROTEINS AND SEQUENCES IN COMPARISONS IN THE '80s AND '90s AND STARTED LOOKING AT DNA REGULATORY SEQUENCES AND MORE RECENTLY WE HAVE LOOKED AT NEXT GEN SEQUENCE ANNUAL S.I.S., ChIP-SEQ. WE HAVE TWO PEOPLE, ONE IS GENOMICS. AP SOUTH AMERICAN FRUIT RELATED TO TOMATOES THAT WE DISCOVERED, THE FRUIT COMES IN A PACKAGE, IT'S LIKE A LEAFY PACKAGE, I DON'T KNOW IF YOU'VE SEEN IT. ABOUT A CENTIMETER AND A HALF IN DIAMETER. WHEN I WAS A CHILD GROWING UP IN SOUTH AFRICA, THESE THINGS WERE LIKE WEEDS ALL OVER THE PLACE. WE ASSUMED THEY WERE LOCAL BUT THEY MUST HAVE TRAVELED SOMEWHERE ON SOME BIRD PROBABLY MANY YEARS AGO AND STARTED GROWING THERE AND ARE PROLIFIC. THAT'S ONE OF THE PROJECTS. THE OTHER PROJECT WE ARE WORKING ON IS MORE WHERE WE LOOK AT WHERE PROTEINS BIND IN THE NUCLEI OF YEAST AND MICE RIGHT NOW. SPECIFIC PROTEINS WORKING WITH EXPERIMENTALS ON THE CAMPUS IN ALBANY AND NCI IN FREDERICK ON WHERE PROTEINS BIND AND WHY DO THEY BIND, WHAT'S THE DIFFERENCE. AND I THINK THAT WAS -- I THANK YOU FOR YOUR TIME. IF YOU HAVE ANY QUESTIONS, PLEASE ASK. THANK YOU. [APPLAUSE] >> COMMENTS OR QUESTIONS? THANK YOU VERY MUCH. NEXT, GENETIC TESTING REGISTRY. >>> WELL, THANK YOU VERY MUCH. I'M DELIGHTED TO HAVE THE CHANCE TO PRESENT TO YOU. I'M DELIGHTED AT THE CHANCE TO WORK AT NIH AND NIH, NCBI. YOU CAN TELL FROM SOME OF THESE LETTERS AFTER MY NAME I'M TRAINED AS AN INTERNIST AND AS A MEDICAL GENETICIST. AND PRIOR TO COMING HERE I WAS AT UNIVERSITY OF CHICAGO AFFILIATE, DIVISION CHIEF, VERY BUSY CLINICAL WORK DOING MOSTLY ADULT GENETICS AND CANCER GENETICS WITH TWO INTERNIST GENERALETTIST, COUNSELORS, CAME HERE AND WAS PUT IN INFORMATION INFORMATIONLAND, MY BRAIN IS WORKING MORE LIKE A LIBRARIAN AND DATA MANAGER BUT I CARE DEEPLY ABOUT THE CLINICAL ISSUES AND BRING A LOT OF THAT TO THIS WORK. SO I'LL TALK ABOUT THE GENETIC TESTING REGISTRY, OUR MISSION, HOW IT I GOT STARTED, TALK TO YOU ABOUT TRANSPARENCY REALLY MEANS. WE THROW THIS WORD AROUND, IT'S DEEPLY MEANINGFUL, AND GIVE YOU -- I CAN'T TAKE YOU ON A COMPLETE TOUR OF ALL THE BELLS AND WHISTLES AT THE REGISTRY BUT I'D LIKE TO GIVE YOU THE SENSE OF THE DEEP DATA THAT'S CONTAINED WITHIN, AND THE IMPORTANCE OF THAT IN THE LARGER WORLD OF THE LABORATORIES AND EVEN THE REGULATORS. SO I'LL GIVE YOU SOME INFORMATION ON THE USERS WHO THEY ARE AND USAGE AND INTERACT WITH SISTER AGENCIES AS WELL. I'LL GIVE YOU THE SENSE OF THE SCOPE OF THE GENETIC TESTING LANDSCAPE. I'LL SHOW YOU NEWS ARTICLES, BUT THIS REALLY INFLUENCED THE DEVELOPMENT OF THE REGISTRY. SO THE INITIAL CALL FOR THE GENETIC TESTING REGISTRY CAME FROM THE RECOMMENDATION BY THE SECRETARY'S ADVISORY COMMITTEE ON GENETICS HEALTH AND SOCIETY BACK IN 2008, THE PURPOSE TO DEVELOP A PUBLICLY AVAILABLE WEB-BASED REGISTRY ENHANCED TRANSPARENCY OF GENETIC TESTING, AN UNDERSTANDING MANY OR MOST OF THE TESTS AVAILABLE TO EVALUATE GENETIC CONDITIONS WERE LABORATORY BASED AND NOT SO CLOSELY REGULATED. THERE WAS CONCERN ABOUT THAT. SO I'LL SHOW YOU SOME DATA ON THAT. SO I'D LIKE TO GIVE YOU A SENSE OF WHAT WORLD WAS LIKE BETWEEN 2008 AND 2012 WHEN THE REGISTRY WAS LAUNC LAUNCHED. SANGER SEQUENCING HAD BEEN USED, SHORTLY AFTER THAT THERE WAS AN INFORMATION EXPLOSION BASED ON NEXT GENERATION SEQUENCING AND AMAZING AMOUNT OF DATA THAT IT PRODUCES. I'LL COME BACK TO THIS BECAUSE THIS HAS REALLY QUICKLY INFLUENCED THE CLINICAL REALM. AT THIS TIME, IT WAS BEGINNING TO BE RECOGNIZED AS VERY IMPORTANT IN DISCOVERY OF GENETIC CONDITIONS. GTR WAS BEING CREATED, THERE WAS STRONG OPINION ABOUT THE REGISTER, WHAT SHOULD BE IN IT, WHAT KIND OF DATA, WHO IS LOOKING AT THE DATA AND WHO IS SUBMITTING IT, IS IT ACCURATE. THIS IS AN ARTICLE FROM JUST TWO DAYS AGO, I'M SURE YOU'VE BEEN CATCHING WHAT YOU CAN OF THE OLYMPICS, UZBEKISTAN ANNOUNCED THEY WILL USE GENES TO PREDICT WHAT SPORT A CHILD IS BEST SUITESUITED FOR AND SELECT FUTURE CHAMPIONS. YOU CAN FIND THIS ON THE WEB, THERE ARE OTHERS LIKE IT, FOR A TEST OF A GENE IMPORTANT IN MUSCLE REGULATION, THAT PARENTS ACTUALLY WITHOUT THE BENEFIT OF HEALTH CARE INTERMEDIARY, THIS IS AVAILABLE DIRECT TO CONSUMER AND IS PURPORTED TO HELP PARENTS AND COACHES DETERMINE ATHLETIC POTENTIAL AND THE COMPANY SAYS IT'S SAFE TO USE AS EARLY AS AGE 1. SO WE HAVE THAT. AND I'LL INTERJECT AS LITTLE OPINION AS I CAN. THAT'S REALLY A GENE LET'S SAY FOR BRAWN, OF COURSE BRAIN IS QUITE IMPORTANT IN DETERMINATION OF ATHLETIC PRO TENSION. POTENTIAL. I DON'T KNOW OF TESTS FOR THAT. I'LL POINT OUT AS A PARENT OF A PERSON ON THE OLYMPIC DEVELOPMENT TEAM, FOR RUGBY, I AM ABLE TO PROVIDE AN OPINION I THINK PROBABLY BRAWN IS IMPORTANT, BRAIN IS MORE IMPORTANT. AND HERE MY SON IS SMILING AND HERE HE'S NOT SMILING, SO MUCH. KIND OF DIRTY AND HIS BRAWN IS PRETTY IMPORTANT THERE. AND THEN JUST A FEW DAYS AGO IN THE "NEW YORK TIMES" NIH AND NHGRI AND THE AMERICAN COLLEGE OF MEDICAL GENETICS AND GENOMICS, YOU'VE HEARD SOME OF THAT YESTERDAY ABOUT NEWBORN SCREENING AND NEXT GENERATION SEQUENCING, SO I THINK THEY ARE TRYING TO GO ABOUT THIS AROUND LEARN ABOUT IT IN A RESPONSIBLE WAY, STUDY THE ETHICS AS WELL AS TECHNOLOGY. NOW, WHEN YOU -- WHEN I SPEAK LATER TALKING ABOUT ANALYTICAL VALIDITY, THAT TRANSLATES INTO IF THIS BABY HAS A NEXT GENERATION SEQUENCING TEST AT A CERTAIN LAB, AND GETS A RESULT, WILL THAT RESULT BE THE SAME, IF THE CHILD IS TESTED AT THAT SAME LAB WITH THAT SAME TEST. THAT'S VERY MUCH IN QUESTION RIGHT NOW. MUCH LESS WHEN WE GET TO CLINICAL VALIDITY OF WHAT DISEASES DOES IT RELIABLY PREDICT AS WELL AS CLINICAL UTILITY. WE GATHER THAT DATA IN THE GENETIC TESTING REGISTRY, THAT'S WHAT IT MEANS TO GATHER EVIDENCE AND ENHANCE TRANSPARENCY. AND AT LEAST IN THIS AUDIENCE I DON'T HAVE TO EXPLAIN THE NEXT PICTURE. SO FRANCIS COLLINS ALMOST TWO YEARS AGO LED THE PUBLIC LAUNCH OF THE GENETIC TESTING REGISTRY AT RARE DISEASE DAY ON A RARE DAY, FEBRUARY 29, TO ENCOURAGE PROVIDERS TO ENHANCE TRANSPARENCY, PROVIDE INFORMATION ABOUT THEIR TESTS, AND AFTER MUCH DISCUSSION THIS WAS DETERMINED IT WOULD BE A VOLUNTARY REGISTRY WHICH HAS ITS EFFECTINGS OEFFECTINGSES ON THE TYPE OF DA TA WE'RE ABLE TO GATHER AND THIS REALLY WAS A LONG PROCESS, SO BETWEEN THE TIME THE REPORT CAME OUT AND THIS WAS LAUNCHED, VERY GLAD THIS TOOK A LOT, THERE WAS A LOT OF DELIBERATIVE ACTION TO GATHER STAKEHOLDER INPUT FROM A WIDE VARIETY OF STAKEHOLDERS WHO WHO HAD DIFFERING OPINIONS. WE ENABLED DATA COLLECTION AND DATA INFORMATION ABOUT THE TESTS, I'LL SHOW YOU METRICS ON THIS. WE DEVELOPED RESEARCH TEST REGISTRATION. THIS PAST DECEMBER WE'RE ACCEPTING REGISTRATION OF TESTS FOR CANCER TUMORS, WHICH IS SOMETHING THAT NO DATABASE THERE PROVIDES. WE INSTITUTE A REVIEW AND REQUIREMENT FOR ANNUAL REVIEW OF THE DATA. WE'VE BEEN FORTUNATE TO BE ADVISED BY MANY ERUDITE COLLEAGUES ENGAGED IN WHAT THE DATABASE SHOULD LOOK LIKE, AS WELL AS IMPORTANT ISSUES SUCH AS THE QUALITY OF THE DATA THAT COMES INTO THE GENETIC TESTING REGISTRY. SUBMITTEST MUST AGREE TO A CODE OF CONDUCT, WHICH WE RECORD THAT IT IS ACCURATE AND NOT MISLEADING. EACH TEST CANNING REFERRED TO IN SCIENTIFIC PAPERS, USED AS EVIDENCE, FOR EXAMPLE, FOR THE SISTER DATABASE WHERE PEOPLE MAKE ASSERTIONS ABOUT CLINICAL VARIATION AND THE IMPACT OF THAT, THE TEST IS ACTUALLY A METHOD. AND WE ENGAGE DTR USERS TO TELL US WHAT MAY BE INACCURATE OR MISLEADING AND HAVE A STANDARD OPERATING PROCEDURE WHICH IS THEN ENACTED TO SOME EXTENT EVEN. OF COURSE, YOU KNOW, ROME WASN'T BUILT IN A DAY. WE DO THIS IN PARTS. SO FAR WHAT IS AVAILABLE MENTIONED TO SOME EXTENT, SINGLE GENE TESTS, PHARMACO GENETIC TESTS, MOLECULAR AND CYTOGENETIC AND BIOCHEMICAL. WE DESCRIBE A WHOLE EXOME TEST IN FINE DETAIL AS WELL AS WHOLE GENOME TESTS, WE'LL USE OUR ADVISORY BOARDS TO GUIDE US. THESE ARE NOW IN SCOPE AS LABORATORY HE WALABSERVICES. WITH TIME, WE WILL BRING IN INFORMATION FOR MANUFACTURERS ABOUT THEIR KITS AND TACKLE THE DIRECT TO CONSUMER ISSUE. AT THE SAME TIME, NIH SUPPORTED THE GENE TEST LABORATORY DIRECTORY FOR MANY YEARS, AND SO WE WORKED WITH THE GENE TEST STAFF TO DEVELOP A TRANSITION PLAN AS NIH PLANNED TO END SUPPORT FOR THIS REGISTRY WHICH IS MORE LIKE A YELLOW PAGE OF -- YELLOW PAGES OF A LAB DIRECTORY BUT NOT DETAILED INFORMATION ABOUT SINGLE TESTS. YOU CAN FIND OUT ABOUT A LAB AND THE CONDITIONS THEY TEST FOR BUT NOT DETAILED INFORMATION ABOUT THE TEST. AND SO WE COMMUNICATED WITH LABS ABOUT THIS, THEY AGREED TO HAVE THEIR INFORMATION REDISPLAYED IN THE GENETIC TESTING REGISTRY WHEN IT WENT LIVE. GENE TESTS DECIDED TO CONTINUE THEIR SITE UNDER COMMERCIAL FUNDING, AND SO IF YOU REMEMBER A COUPLE OF THESE DATES, THIS IS WHEN GENE TEST WAS NO LONGER PUBLIC AT NCBI, JUNE 4 IS LAST YEAR, AND ON JULY 20, SEVEN WEEKS LATER, BIOREFERENCE LABORATORIES SUPPORTED THE LAUNCH OF THE COMMERCIAL VERSION OF GENE TESTS. SO I'LL SHOW YOU HOW THAT AFFECTED USAGE. DURING ALL THIS TIME, I'LL REFER TO SUBMITTER BURDEN, IT'S QUITE DIFFICULT FOR LABS TO SUBMIT. THERE'S NO EXTRA PAYMENT OR STAFFING. THIS WE CAME UP TO THE MINIMAL REQUIREMENTS IN THE GENETIC TESTING REGISTRY. THAT'S STRAIGHT OUT OF OUR APP LOG, STATISTICS, WHERE BACK IN LET'S SAY FEBRUARY, EARLIER THIS YEAR, THERE IS ROM RAN -- SOME RANDOM USAGE OF GTR. ON JUNE 4, WHEN THE GENE TEST USERS WERE REDIRECTED TO THE GENETIC TESTING REGISTRY, THE BOTTOM LINE REPRESENTS THE DISTINCT SESSION OR ACTUAL USERS BASED ON THEIR COOKIES AND YOU CAN SEE THE PATTERN THAT'S REFLECTED, WEEKDAY, WEEKEND USAGE, PEOPLE AT WORK, AND EVEN WHEN THE COMMERCIAL RENTAL INDUSTRY LISTING WENT BACK ONLINE, THERE'S NO REAL CHANGE IN USAGE, IT MAY HAVE GROWN DURING THE FALL, A TYPICAL DROP AS PEOPLE HAVE HOLIDAYS AND THEN UP AGAIN, WHICH IS A TYPICAL SEASONAL. WE GO OUT AND TRY TO MAKE PEOPLE AWARE OF THE REGISTRY, GO OUT TO MEETINGS WHEN WE CAN, USE SOCIAL MEDIA, WORD OF MOUTH. BUT ONE THING CAN DO IS USE OUR OWN RESOURCES. PubMed HAS AN EXISTING GENE SENSOR. 2007 I TYPED IN ONE OF MY FAVORITE GENES, TYPED THIS IN THE DATABASE, THE DATABASE KNOWS THIS IS A GENE AND PRESENTS VARIOUS SOURCE DATABASES YOU MIGHT WANT TO GO TO. GTR WITHIN A MONTH WILL BE ONE OF THOSE SEARCHES THAT YOU CAN DO AUTOMATICALLY, FOR NOW 67 TESTS AVAILABLE FOR THIS GENE. SO SUBMITTER PARTICIPATION, ENORMOUSLY IMPORTANT, AND I THINK IT'S BEEN PRETTY GOOD. THERE ARE CLOSE TO 400 REGISTERED LABORATORIES, AND TO MAKE THIS DISTINCTION FOR ALL THE DATA I'LL PRESENT NOW I'M ONLY PRESENTING DATA ON ACTIVELY ENGAGED SUBMITTERS, WHEN YOU COME IN, DECIDE TO REGISTER THEIR LAB AND PROVIDED MORE INFORMATION ABOUT THEIR TESTS. WE HAVE MORE DATA FROM THE LAB DIRECTORY, BUT THAT WILL BE PHASED OUT OR ENHANCED. I'M ONLY GOING TO DISCUSS GTR DATA. A WORLDWIDE EFFORT, MORE THAN HALF FROM THE U.S., AND ALTHOUGH WE HAVEN'T GONE OUT TERRIBLY ACTIVELY TO FIND OTHER LABS, THEY FOUND US, 11% ARE NEW, NOT IN GENE TESTS. WE'RE NOW MORE ACTIVELY GOING OUT BECAUSE WE HAVE TESTING SCOPE, BUT WE'VE BEEN ENGAGED IN A MIGRATION PROCESS UNTIL NOW. SO IF YOU LOOK AT THE ENGAGEMENTS IN TERMS OF THE PERCENT OF LABS, AS THE TOTAL THAT WERE IN GENERAL TESTS, 60% OF THEM HAVE ACTIVELY ENGAGED. IF YOU LOOK AT THOSE LABS IN TERMS OF THE TEST CONTENT THAT THEY REPRESENTED, IN THE GENE TEST LABORATORY DIRECTORY, PRETTY MUCH ALL OF THE LARGE LABS ARE PARTICIPATING. YOU HAVE TO GO TO THE RANK OF 30 TO FIND ONE THAT'S NOT EVEN IN, AND CONSEQUENTLY THAT'S MORE A METRIC OF 85% IN TERMS OF CONTENT. THIS IS WHAT OUR TEST GROWTH LOOKS LIKE. BACK FROM THE SPRING, LET'S SAY UNTIL NOW, IT'S REALLY SHOT UP ALMOST CLOSE TO 14,000 DETAIL TESTS, AND 4000 OR SO CONDITIONS TESTED WHICH WILL CONTINUE TO GROW AS OUR SCOPE ENLARGES. SO THE SUBMITTER BURDEN IS CONSIDERABLE BECAUSE OF COURSE THERE'S NOT JUST A GTR-LAND OUT THERE, THERE'S THE LABS NEED TO SEEK PAYMENT FROM CMS, AND THEN THERE ARE, YOU KNOW, POSSIBLE NEEDS TO SUPPLY DATA TO THE F.D.A. AND THE F.D.A. HAS BEEN TALKING A LOT ABOUT DRAFT GUIDANCE AND CHANGING THEIR VIEW OF LABORATORY DEVELOPED TESTS, AND SO WE ENGAGED OUR SISTER AGENCIES EARLY ON AND CONTINUE THE CONVERSATION, IN THE MIDST OF THIS CMS USED ONE OF THEIR MAIN CONTRACTORS, PALMETTO GBA, TO START TO CREATE WITH A SUN CONTRACTOR, McKESSON, A COATING WHICH IS AIMED TO IMPROVE THE TRANSPARENCY SURROUNDING WHAT GETS PAID FOR, AND SO WE BEGAN TO TALK WITH MKESSON AND WENT THROUGH MAPPING EXERCISES WITH THEM, THERE ARE NO BARRIERS TO -- WE BOTH AGREE THERE ARE NO BARRIERS TO REALLY MAPPING THE DATA, BUT, YOU KNOW, FROM A STANDPOINT OF THE LABS, THEY WOULD LIKE ONE STOP SHOPPING, OR DATABASE WOULD BE MORE COMPLETE IF THERE WERE ONE WAY TO GO IN, IT WOULD BE EASIER AND BETTER FOR THEM, BUT THERE NEEDS TO BE SORT OF A SWELLING OF NEED AND PURPOSE FOR THIS TO REALLY GO FORWARD. INTERESTINGLY IN THE MIDST OF ALL THIS, THE AMA WAS FIRST INITIALLY QUITE MIFFED ABOUT THE FACT THAT A NEW SET OF CODES HAS BEEN CREATED RATHER THAN THEIR CPT CODES, AND THEY INITIALLY THREATENED ACTION, LEGAL ACTION, AND NOW THEY ARE ACTUALLY PARTNERS AND THERE'S A CODE BRIDGING EFFORT TO MAP THESE CODES ON ONE OF A -- I'M ONE OF A DOZEN PEOPLE NAMED TO THIS GROUP TO HELP MATCH THE CODES. AND HOPEFULLY IMPROVE MY UNDERSTANDING OF ALL OF THIS IN TERMS OF GTR. I'LL COME BACK TO THIS A LITTLE BIT AT THE END. SO IN TERMS OF THE INFORMATION MANY TEST -- I DID WANT TO MENTION WE IN PARTNERSHIP, N.L.M. HELPED US TO FORM AN AGREEMENT WITH THE AMA SO WE COULD REPRESENT THE CPT CODE AND GTR, THAT PROVIDES A WAY FOR US TO DIRECTLY ENGAGE WITH CMS, THERE'S BEEN MORE RECENT INTEREST WHERE I WAS ASKED TO SPEAK ABOUT NEXT GEN SEQUENCING AND MAPPING WITH CPT CODES. YES? >> WHY CMS? WHY ARE THEY ENCOURAGING DEVELOPMENT OF YET ANOTHER SCHEME? >> WELL, WE'VE ASKED THAT QUESTION. I'M NOT SURE WHAT THE ANSWER IS. THEY MAY NOT HAVE INITIALLY EITHER RECOGNIZED OR BELIEVED THAT THIS EFFORT WOULD GO FORWARD SO FAST. I THINK THAT THEY ARE IMPRESSED WITH THE DEPTH OF DATA, PERHAPS REENGAGED BASED ON WHAT IS IN GTR. WHEN YOU LOOK AT FOR EXAMPLE METHODOLOGY, WE HAVE THREE LEVELS OF METHODOLOGY, AND THEY ARE VERY DISCREETLY CHARACTERIZED. WE HAVE -- WE USE THE APPROACH TO CATEGORIZE CONDITIONS, AGD IT'S BEEN LET'S SAY SHOCKING ON SOME LEVEL TO SEE WHAT THE APPROACH HAS BEEN ON THE OTHER END, FOR HOW TO CATEGORIZE CONDITIONS. THEY ARE BASICALLY NAMED NOT SO OF THE CONCEPTS OR DEFINITIONS. >> THEY MIGHT HAVE OPENED IT UP. >> WELL, YES. I DON'T KNOW HOW MUCH I CAN GET INTO THAT BUT I'LL MAYBE TALK AT THE END. I THINK WE REALLY WANT TO BRING THIS AND MAKE IT MORE COHESIVE AND I'D LIKE TO SEE, YOU KNOW, AS MUCH SORT OF VOICES THAT CAN BE APPLIED IN THIS AREA TO MAKE THAT GO FORWARD. AND THE LABS HAVE BEEN -- SORRY? >> THEY ARE ACTING LIKE WE'RE NOT PART OF THE SAME GOVERNMENT. >> WELL, THE LABS HAVE BEEN CAUGHT IN THE MIDST OF THIS, THEIR REIMBURSEMENT AS NOT BEEN AS GOOD AS HOPED BASED ON THE IMPLEMENTATION OF THE MOLECULAR PATHOLOGY CODES, USING STACKING CODES BEFORE WHICH WERE NOT INFORMATIVE AND THE METHODS USED TO DETERMINE WHAT THE PAYMENT WOULD BE WERE NOT SATISFACTORY TO THEM AND SO THERE'S REALLY A LOT OF STRONG EMOTION FLOATING AROUND, YOU KNOW, WE WOULD LIKE TO DO WHAT WE CAN TO MAKE THIS SMOOTHER AND THERE'S NO BASIC REASON WHY THERE SHOULD BE TWO OR THREE OR FOUR SETS OF TYPES OF INFORMATION. >> ABSOLUTELY NOT. >> WE GATHERED A LOT OF THIS INFORMATION UP FRONT WHEREAS EVEN THE McKESSON REGISTRY DOES NOT GATHER INFORMATION LIKE CLINICAL UTILITY EXAMINED AFTERWARD WHEN REIMBURSEMENT IS AT THE PALMETTO CMS LEVEL. I GTR IS THE NEXUS OF INFORMATION, PROVIDED BY THE LABORATORIES BUT WE DO A LOT OF CONNECTING, AND WE CONNECT TO THE LITERATURE SOURCES, MANY OF THEM FULL TEXT, ALSO THE GENE REVIEWS, PubMed, PubMed CENTRAL, I'LL SHOW YOU EXAMPLES OF THAT. WE'RE VERY ENGAGED IN UTILIZING VOCAB LATHERS -- VOCABULARIES BUT ADHERING TO THESE, ALSO TO ADVANCE THE CONVERSATION ON -- THERE'S A PAUCITY OF GENETICS TERMS AT THE GENE LEVEL, SAY, AND WE STARTED TO ENGAGE WITH EUROPEAN PARTNERS AND SO FORTH, TO MOVE THIS FORWARD. OF COURSE, USING NOMENCLATURES IS HUGELY IMPORTANT, IT LETS US KNOW WHAT THE LABORATORY IS TRULY TESTING, SURPRISING TO SEE HOW DIFFICULT IT IS SOMETIMES FOR THEM TO ACTUALLY DESCRIBE IT IN THESE TERMS. >> YOU DON'T SHOW IT IN THIS PICTURE. >> WELL, I SHOW HGNC WHICH IN THE TINY, TINY TEXT IS THE HUMAN GENE HUGO NOMENCLATURE COMMITTEE. NCBI DATABASES, QUALITY MEASURES SET BY THE COMMUNITY, WE DON'T SET THOSE QUALITY MEASURES, BUT WE REFLECT THEM SO THE COLLEGE OF AMERICAN PATHOLOGY ADMINISTERED THROUGH CMS AND F.D.A. TEST CODES, I MENTIONED AMA CODES, AND PRACTICE GUIDELINES WHICH WE HAND ASSEMBLE. I'D LIKE TO HAVE THE NATURAL LANGUAGE PROCESSING POWER AT OUR DISPOSAL TO FIND THESE ABOUT WE ATTACH THEM MANUALLY AS CURATORS. AND THIS IS A SIDE WORK BUT VERY IMPORTANT BECAUSE YOU CAN'T FIND THESE GUIDELINES IN ANY ONE PLACE, EVEN AT AHRQ BECAUSE THEY DON'T CATEGORIZE THEM. AND SO IF YOU ARE IN THE GTR OR SOME SISTER RESOURCES, YOU CAN HOVER OVER THE PRACTICE GUIDELINES WHICH WE PUT THERE AND READ THE TITLE AND GO STRAIGHT TO IT. THIS IS WHERE YOU CAN FIND THEM ALL ASSEMBLED. LOOKING AT THE DETAIL OF INFORMATION THAT LABORATORY PROVIDERS HAVE SENT, IN TERMS OF THE COMPLETION RATE OF VARIOUS FIELDS WE HAVE ANALYTICAL VALIDITY AS THE MINIMAL REQUIREMENT, WE HAVE THIS ON EVERY TEST, A„i HUNDRED PERCENT. CLINICAL VALIDITY AND UTILITY ARE PROVIDED AT 9 OR 10% RATE. WE WISH WE HAD MORE BUT IT WAS DECIDED LABORATORIES DON'T HAVE THIS IN HAND EVEN THOUGH F.D.A. WOULD REQUIRE IT IF THEY UNDERWENT THE F.D.A. PROCESS. AND CLINICAL UTILITY IS SOMETHING THAT ESSENTIALLY REQUIRED FOR PAYMENT BUT LABS HAVEN'T BEEN WILLING OR ABLE TO PROVIDE IT READY WILL. THEY DO, HOWEVER, FREQUENTLY PROVIDE TARGET POPULATION FROM THEIR RESOURCES. >> WHAT WOULD TYPICALLY BE A PAYMENT FOR A GENETIC TEST? >> WELL, SOME OF THE COSTS OF THE TESTS ARE IN THE ONE, TWO, THREE THOUSAND DOLLAR RANGE FOR INDIVIDUAL TESTS OR COMBINATIONS OF SMALL NUMBERS OF TESTS. AND REIMBURSEMENTS, LET'S SAY FOR BRCA-1 WAS CLOSE TO THE $2000 OR HIGHER RANGE. RECENTLY IT'S BEEN REDUCED TO ABOUT $1000. WHICH HAS BEEN ATTRIBUTED TO THERE BEING MORE COMPETITION NOW BECAUSE OF THE U.S. SUPREME COURT RULING. AND SO IN TERMS OF THE METHODS WE HAVE MOLECULAR AS PREDOMINANT TYPE BUT WE HAVE CYTO GENETIC AND BIOCHEMICAL, IT'S INTERESTING TO LOOK AT THE ADOPTION OF NEXT GENERATION SEQUENCING INTO THE CLINORIL -- CLINICAL REALM, ABOUT 10% OF THEM ALREADY USE NEXT GENERATION SEQUENCING AS METHODOLOGY. WE'VE HAD DOUBLE TAKES ON THIS METRIC AS I PRESENTED TO THE CLINICAL LABORATORY ORGANIZATIONS BECAUSE THE FACT IS THAT UNTIL OCTOBER, SORRY, NOVEMBER OF LAST YEAR THERE WAS NO INSTRUMENT THAT WAS F.D.A. APPROVED TO DO NEXT GENERATION SEQUENCING. NOW WE HAVE ONE BUT I CAN TELL YOU THAT THE LABS THAT HAVE SUBMITTED THESE TESTS ARE -- IT'S NOT AS IF THEY HAVE BEEN USING THIS–;ALL OFc SO THIS IS MOVING FORWARD AND THEN WITH RESPECT TO F.D.A. ON THAT TOPIC, WE'VE PRESENTED BASICALLY THE FIRST QUANTITATIVE DATA TO SHOW THE VAST MAJORITY OF GENETIC TESTS ARE LABORATORY DEVELOPED TESTS, NOT REGULATED -- WELL, NOT F.D.A. APPROVED OR CLEARED, AND ONLY EIGHT OF THE 6000 CLINICAL TESTS REGISTERED FROM U.S. LABS REPORT THEY ARE F.D.A. APPROVED OR CLEARED. THIS ISN'T A REQUIRED FIELD, WE DO HAVE ABOUT 14% OF LABS SAYING EXPLICITLY THE F.D.A. EXERCISES ENFORCEMENT DISCRETION WHICH IS THEY HAVE THE AUTHORITY TO EXAMINE THESE TESTING DETAILS BUT CHOOSE NOT TO. [ INAUDIBLE ] IT MEANS THEY CLAIM TESTS FALL END F.D.A. REGULATION, BUT THAT THEY CHOOSE NOT TO LOOK IN DETAIL AT THE VAST MAJORITY OF TESTS, AND THAT'S A WHOLE OTHER SWIRLING -- >> DOES THE LAB MAKE THAT CHOICE OR THE F.D.A.? >> THE F.D.A. MAKES THAT CHOICE. UH-HUH, YEAH, YEAH. SO THE LABS ARE IN THE MIDST OF, YOU KNOW, WORRYING ABOUT WHAT F.D.A. WILL DO AND THERE'S SOME CONCERN ALSO ABOUT LABORATORIES PROVIDING TESTING FOR VERY RARE DISEASES THAT THEY MAY BE THE ONLY LAB OFFERING IT. ACTUALLY WE HAVE DATA TO SHOW ABOUT THE RARE DISEASES THAT ARE REALLY ONLY OFFERED BY ONE LAB. SO THIS I THINK IS THE FIRST REAL VIEW OF A TEST, SO I'LL GIVE YOU A LITTLE DETAIL AND THEN I'LL SHOW YOU ONE OF THESE RARE F.D.A. APPROVED TESTS AND HOW WE DISPLAY THAT. BUT HERE YOU'LL SEE THE TEST NAME WHICH THE LABORATORY GIVES YOU, THE CONDITION IT'S FOR, AND THE LAB THAT PROVIDES IT. THERE'S SEVEN TABS OF INFORMATION WHICH I WON'T GO THROUGH HERE, THIS ONE HAPPENS TO BE ON THE PERFORMANCE WITH THE ANALYTICAL VALIDITY CITATIONS THEY CAN GIVE, LIMITATIONS, PROFICIENCY TESTING IN THIS CASE BY ACMG AND COLLEGE OF AMERICAN PATHOLOGIST, F.D.A. CLEARANCES, NEW YORK STATE TEST APPROVAL. ALL OF THIS IS WHAT THE LAB SAYS ABOUT THEIR TESTS FOR THE MOST PART, AND WE PROVIDE THE GTR SUCCESSION AND LAST UPDATE AND A HISTORY. HERE WE BRING IN INFORMATION, THIS IS MORE ABOUT WHAT THE COMMUNITY SAYS, IN THE LITERATURE, IN REVIEWS, SUGGESTED READING, IN FACT WE PUT IN THE RECENT PUBLICATIONS FROM THE NEW ENGLAND JOURNAL OF MEDICINE ABOUT WARFARIN, THIS IS OFFERED BY MANY LABS, AND PRACTICE GUIDELINE, YOUR CLINICALTRIALS.GOV, CONSUMER RESOURCES, IN PART BECAUSE PHYSICIANS, WHEN THEY WANT INFORMATION ABOUT TESTS THEY WANT INFORMATION TO GIVE TO PATIENTS TO PRINT OUT READILY. THAT'S AVAILABLE. >> THIS IS A MODEL OF WHAT WE MIGHT LIKE TO HAVE? >> THIS IS WHAT WE HAVE ON THE SIX THOUSAND, BUT THIS IS FILLED OUT MORE COMPLETELY THAN MANY TESTS. WE WOULD LOVE TO HAVE THIS BUT THE TESTS HAVE NOT GONE THROUGH F.D.A. CLEARANCE, WE WOULD LOVE TO HAVE PROFICIENCY TESTING BECAUSE ALSO REQUIRED, WE'RE NOTE DEMANDING IT. BUT MANY -- MOST LABS DON'T COMPLETE THAT. IF YOU CLICK HERE, YOU CAN GO EVEN DEEPER WHICH I'VE DONE TO GET ALL THE WAY IN. ANOTHER NEW YORK TIMES ARTICLE, YOU MAY RECOGNIZE THIS CAME OUT IN 2010, NICHOLAS WADE WROTE A DECADE LATER, GENETIC MAP YIELDS FEW NEW CURES, SO SORT OF WHAT IS THE HUMAN GENOME PROJECT -- WHAT HAS IT DONE FOR ME LATELY, THERE'S A LOT OF GNACHING ABOUT THIS. I'D LIKE TO SHOW YOU THE DEPTH AND DEBUNK THIS WHICH SOME OF THIS HAPPENED MORE RECENTLY BUT WE'RE PROGRESSING TO STOPPING DIAGNOSTIC ODYSSEYS AND DOING PREVENTION BASED ON THAT. IT'S NOT NECESSARILY CHANGING THE UNDERLYING GENETIC CARRIE UNDERPINNING. I'LL SHOW YOU A FEW OF THOSE. FIRST AND FOREMOST, NCBI MAINTAINS THE HUMAN GENOME ASSEMBLY WITHOUT WHICH WE COULD DO NO NEXT GENERATION SEQUENCING. SO THE LATEST RELEASE CAME OUT IN DECEMBER, THIS IS A POST ON THAT. WHEN SANGER SEQUENCING IS DONE YOU KNOW WHAT YOUR PRIMERS ARE, YOU KNOW WHAT GENE YOU'RE IN, WHAT YOU'RE GOING TO GET. BUT NEXT GEN SEQUENCING SEQUENCES EVERYTHING AND ANYTHING AND HAS TO FIGURE OUT WHERE ON THE GENOME IT RESIDES BY DOING A VIRTUAL ALIGNMENT CAN SOFTWARE AGAINST THE REFERENCE GENOME, YOU HAVE AICS INNED UP PUZZLE BUT BECAUSE YOU HAVE -- YOU HAVE AIC MIXED UP PUZZLE BUT YOU HAVE A TEMPLATE AND CAN QUICKLY IT WILL IT WILL IT WILL TAKE YOU LONG -- IT WOULD TAKE YOU LONGER TO PUT THIS PUZZLE TOGETHER IF YOU DIDN'T HAVE THE PICTURE IN FRONT MUCH YOU. HERE IS YOUR REFERENCE SEQUENCE, HERE IS THE CONSENSUS OF THE READS YOU'VE LOOKED AT THIS SEQUENCE OVER AND OVER AT THIS SITE AND DETERMINED IT'S AN A WHERE IT SHOULD BE A G, AND THAT'S THE SCIENTIFIC VIEW, TRANSLATED INTO LABORATORIES PROVIDING A STRIKING AMOUNT OF DETAIL ON EACH INDIVIDUAL TEST, AND SO HERE IS A GRAPH THAT SHOWS YOU THAT WE HAVE, IF YOU COUNT MINIMUM OF TWO TARGETS, TWO GENES, WE HAVE 342 NEXT GEN PANELS AS OF THE END OF LAST MONTH, GOING UP TO ABOUT 400 GENES ON THESE PANELS. IF YOU USE A MORE STRING CRITERIA, THEN YOU LOOK AT THE TYPES OF CONDITIONS AND HOW MANY TESTS WE HAVE, FOR A TALK I RECENTLY DID IN PITTSBURGH, WE HAVE 113 NEXT GEN TESTS FOR HEREDITARY CANCER. MANY PHENOTYPES, WHICH ARE COMMON, CARDIOMYOPATHY, HEARING LOSS, NEUROLOGIC AND METABOLIC CONDITIONS, EPILEPSY, INTELLECTUAL DISABILITY, SO THIS IS REALLY A RICH SET OF PHENOTYPES AND TESTS. SO I'D LIKE TO TELL PEOPLE INSTEAD OF MEMORIZING THIS LONG URL YOU CAN TYPE INTO GOOGLE N.I.H.GTR, THE GENETIC TESTING REG CITY WILL BE YOUR FIRST HIT AND THAT'S WHAT OUR HOME PAGE LOOKS LIKE. YOU CAN TYPE IN ANY TERM. WE DEFAULT TO CONDITIONS AND PHENOTYPES BECAUSE THAT'S WHAT PEOPLE LARGELY START WITH, YOU CAN GO STRAIGHT TO GENES, AN AUTO COMPLETE HERE WHICH I WON'T SHOW, YOU CAN LOOK AT GENE REVIEWS, LOOK AT YOUTUBE TUTORIALS ON HOW TO DO THIS, HOW TO SUBMIT AND LINKS TO OTHER RESOURCES AND EXTENSIVE HELP DOCUMENTS AS WELL, DOWN HERE HOW TO FIND GENETICS PROFESSIONALS. SO THE INFORMATION SUBMITTED TO THE REGISTRY REFLECTS RATHER QUICKLY WHAT HAPPENS IN THE WORLD OUT THERE, AS THE U.S. SUPREME COURT MADE THIS DECISION, STRUCK DOWN PATENTS ON NATURALLY OCCURRING GENES AND WITHIN THAT DAY, ONE OF THE LABORATORIES WE COULD SEE WAS POISED TO SUBMIT THIS TEST ON THE PANEL CONTAINING BRCA-1, BRCA-2 AND OTHER GENES TO LOOK AT HEREDITARY CANCER AND USING NEXT GENERATION SEQUENCING WHICH THE RULINGS SAYS YOU CAN USE THIS TECHNIQUE BUT THERE'S SUING AND COUNTER-SUING STILL GOING ON IN THIS SPACE. AND SINCE THEN IF YOU SEARCH BY GENE, YOU CAN FIND A HUNDRED TESTS IN 70 LABS, 34 IN U.S. LABS, IT HAS COME QUICKLY INTO THE CLINICAL REALM. LOOKING AT ANOTHER TEST TO GO BACK AND HOPEFULLY DEBUNK A LITTLE ABOUT WHAT'S THE HUMAN GENOME DONE FOR ME LATELY THIS IS ANOTHER PANEL WHERE YOU CAN SEE THERE ARE MANY GENETIC CONDITIONS BEING -- SYNDROMES BEING LOOKED AT, THIS IS THE SORT OF THING I USED TO DO IN CHICAGO, TARGETS INCLUDE BRCA-1 AND OTHERS. A SUGGESTED READING WHICH WE CULL FROM LITERATURE, THERE'S A PAPER ALONG WITH MANY PRACTICE GUIDELINES THAT SHOWS THAT IF YOU ARE A BRCA CARRIER, A WOMAN, YOU FIND OUT YOUR MUTATION STATUS, YOU ACT ON IT BY HAVING YOUR OVARIES REMOVED AFTER CHILD BEARING, REDUCE YOUR CHANCE OF DYING BY 60% AND I CHALLENGE ANYBODY TO FIND ANOTHER CLINICAL UTILITY THAT IS SO HIGH, MOST OF WHAT WE DO IN MEDICINE IS HOPEFULLY IMPORTANT BUT USUALLY MARGINALLY IMPORTANT. I'M PROUD TO SAY I WAS PART OF THAT WORK. SO LET'S TALK ABOUT, YOU KNOW, REALLY STRICTLY GENETIC DISEASE, THERE ARE 85 TESTS AVAILABLE FOR FABRY'S DISEASE. WE CAN USE THERAPIES TO FORESTALL OR REVERSE EFFECTS OF DISEASE, SERIOUS EFFECTS LIKE END STAGE RENAL DISEASE, COSTLY. AND CYSTIC FIBROSIS, SUGGESTED READING, 123 CLINICAL TESTS AVAILABLE, WIDE VARIETY OF METHODOLOGIES, AS YOU LOOK AT THE SUGGESTED READING, YOU SEE SOME OF THE BENEFITS OF THE OPEN ACCESS AT N.L.M., WITH THIS FULL TEXT PAPER AVAILABLE, IT WAS PUBLISHED IN TH IN THE NEW ENGLAND JOURNAL, SHOWING EFFICACY AND IMPROVED LUNG FUNCTION IN THESE PATIENTS, AND THIS IS A THERAPY THAT'S BASED ON THE SPECIFIC MUTATION, IT DOESN'T WORK ON THE COMMON ONE, DELTA 508 BUT WORKS ON ONE THAT AFFECTS 4% OF PATIENTS, A WONDERFUL PROOF OF PRINCIPLE THAT YOU CAN CHANGE OF COURSE OF A GENETIC DISEASE. IN THIS CASE BY WORKING WITH ADVOCATES AND SCIENCE AND INDUSTRY AND ULTIMATELY F.D.A. PUT THIS THROUGH VERY RAPID APPROVAL PROCESS. HERE IS ANOTHER TEST THAT REFLECTS THE -- THERE'S BEEN A LOT SAID ABOUT GENOME-WIDE ASSOCIATION STUDIES. A LOT OF DIRECT CONSUMERREST ITS RELY ON SMALL RISKS THAT ARE UP OR DOWN FOR COMMON DISEASES, AND IT'S BEEN DEBATED ABOUT THE UTILITY. HERE IS A TEST THAT WAS REGISTERED IN THE GTR FOR BREAST CANCER SUSCEPTIBILITY AND RELIES ON HITS, HERE THEY ARE LISTED ON THE METHODOLOGY TAB, AND THEY REFER TO THE PAPER ON WHICH IT'S BASED IN THE JOURNAL OF THE NATIONAL CANCER INSTITUTE, WHERE THESE SEVEN SNIPS WERE USED TO ENHANCE A WELL VALIDATED MODEL FOR QUANTITATING BEST CANCER RISK AND TAMOXIFEN USED, ASSESSING RISK ENHANCEMENT. ANOTHER EXAMPLE OF DISCOVERY IS GENES RELATED TO AGE-RELATED MACULAR DEGENERATION, THE MAJOR CAUSE OF BLINDNESS IN ADULTS, HERE WE HAVE 13 TESTS FOR AGE-RELATED MACULAR DEGENERATION GENES AND LOOKING AT THIS SUGGESTED READING HERE, IT'S INTERESTING NOW THERE HAVE BEEN STUDIES TO SHOW USING GENETIC ANALYSIS OF A RANDOMIZED CLINICAL TRIAL, WELL KNOWN TRIAL USING ORAL ANTI-OXIDANTS AND ZINC, DEPENDING ON THE GENOTYPE THERE WAS BETTER OR WORSE EFFICACY OF THE DIETARY SUPPLEMENTS. I HAVEN'T LEFT MUCH TIME TO TALK ABOUT CANCER. CANCER TESTS ARE PURPORTED TO BE ONE OF THE MAIN STRENGTHS OF GENETIC TESTING ON TUMORS, AND I THINK THAT'S THE CASE. BUT JUST TO SHOW YOU SOME OF THE EARLY TESTS THAT WE HAVE IN FOR SOMATIC VARIATION FOR THE GENES THAT CAUSE LEUKEMIAS, LYMPHOMA, IN THIS CASE A TEST FOR A WELL-KNOWN VARIATION IN THE BRAF GENE THAT THIS LABORATORY IS OFFERING FOR TREATMENT MANAGEMENT DECISIONS, GLIOMA OF THE BRAIN IN PEDIATRIC POPULATIONS. WE'RE WORKING WITH THIS LABORATORY FOUNDATION MEDICINE, THAT DOES TUMOR ANALYSES FOR ABOUT 200 GENES USING NEXT GENERATION SEQUENCING TO GUIDE THERAPIES, TARGETED THERAPIES BASED ON THE RESULTS AND YOU CAN SEE FROM THE SUBMISSION SITE THEY STARTED ENTERING INFORMATION. WE'RE FARTHER AHEAD WITH GENOMIC HEALTH WITH THE ARCHETYPE TEST FAMOUS FOR MAKING TREATMENT DECISIONS ABOUT TEAM CHEER FOR BREAST CANCER AND THEY ARE PRETTY MUCH ALL THE WAY THROUGH ON THE TAB AND ARE REVIEWING THE DATA. WE HOPE THIS GOES PUBLIC VERY SOON. IN ADDITION, PHARMACO GENETIC TESTS THAT EVALUATE THE PROPENSITY EITHER FOR TOXICITY OR FOR EFFICACY BASED ON GENETIC UNDERLYING VARIATION, WE HAVE 64 TESTS FOR 18 DRUG RESPONSES, THIS SHOWS ONE WHERE TOXICITY CAN BE PREDICTED BASED ON GENE ON -- GENOTYPE WITH ANOTHER IMPORTANT POTENTIAL WAY TO MAKE DECISIONS ABOUT THIS. HERE YOU CAN FIND THE INFORMATION ON THE GENES, YOU CAN GO TO THE PRACTICE GUIDELINES, IN THE PRACTICE GUIDELINES GET THE FULL TEXT AND YOU CAN FIND ACTUAL DOSING RECOMMENDATIONS BASED ON GENO TYPE GOING FROM WHAT TESTS TO POTENTIALLY GET TO WHAT WOULD YOU DO, COMING FROM AN AUTHORITATIVE SOURCE, THE CLINICAL PHARMACO GENETIC IMPLEMENTATION GROUP. BRIEFLY BACK TO THIS IDEA OF WHAT'S GOING OUT THERE, IN THE GENETIC TESTING WORLD, REALLY AS I ALLUDED TO THERE'S A LOT OF EMOTIONS FLYING AROUND BECAUSE MANY DEMANDS ARE BEING MADE. PAYMENTS GETTING LEFT. WEEDLIKE TWE'D LIKE TO MAKE THIS A MORE UNIFIED PROCESS. THERE'S NO -- IT'S AGREED BY CMS AND WORKING WITH THEIR CONTRACTORS, THERE'S NO BARRIER TO US MAKING CONNECTIONS AND MAPPING DATA AND WITH AGREEMENT BY THE LABORATORY HAVING EITHER THE DATA INITIALLY DEPOSITED AT NCBI OR INITIALLY DEPOSITED AT McKESSON LET'S SAY AND SHARED IN BOTH DIRECTIONS. AND WITH F.D.A. AS WELL, IN TERMS OF THE DRAFT GUIDANCE THAT THEY HAVE COME OUT WITH AND PUTTING UP THROUGH THE CHAIN, THEY HAVE 14 FIELDS THAT THEY WOULD BE INTERESTED IN GATHERING AND WE'VE GONE THROUGH MAPPING EXERCISE WHERE A THIRD OF THEM WE ALREADY TAKE IN EXACT FORM, A THIRD ARE PROBABLY THE SAME, WE HAVE TO COME TO A DECISION AND ANOTHER THIRD ARE DIFFERENT BUT WE CAN CERTAINLY GATHER THOSE FEW FIELDS LIKE PEDIATRIC USE, SOMETHING LIKE THAT. AND SO AT A RECENT VISIT ALONG WITH TERRY MANOLIO TO THE CMS HEADQUARTERS WHERE I GAVE A PRESENTATION ON BASICALLY THE METHODOLOGY OF NEXT GENERATION SEQUENCING, SO THEY COULD BETTER UNDERSTAND THAT AND MAKE DECISIONS, WE ALSO HAD A CONVERSATION ABOUT THE IDEA OF SHARING INFORMATION, IT'S VERY PRELIMINARY, AND IT WOULD NOT BE DONE WITHOUT THE APPROVAL OF THE LABORATORIES WHO PROVIDED INFORMATION. BUT IT REALLY COULD HELP THEM IN A NUMBER OF WAYS TO UNDERSTAND THE DEPTH OF THE METHODOLOGY, THE DETAILS ON THE TEST TARGETS, PARTICULARLY INTERESTED IN THE PURPOSES OF TESTS BECAUSE THEY DON'T OFFER PAYMENT FOR TESTS THAT ARE SCREENING, BUT WE DO HAVE CATEGORIES WE'VE ADDED THAT ARE COMING FROM CMS AND HRQ THAT THEY SUPPORT THE CATEGORIES WE'RE GATHERING. WE WOULD LIKE TO SEE THAT MOVE FORWARD. OUR ROLE IS TO ASSEMBLE, MANAGE AND FILL THIS EVIDENCE GAP. WE DON'T FUND STUDIES TO, YOU KNOW, ENHANCE THE, YOU KNOW, THE EVIDENCE BUT WE COLLATE IT AND MAKE IT AVAILABLE. MANY COLLEAGUES WHO CONTRIBUTED TO THIS WORK WHO I LOVE WORKING WITH, AND DIRECTLY ON THE PROJECT AND NCBI AND NLM, THE OFFICE OF THE DIRECTOR HAS BEEN DEEPLY ENGAGED ALL THE WAY THROUGH, WE COLLECTED RESEARCH A YEAR AGO. I'LL TAKE ANY QUESTIONS YOU MIGHT HAVE. >> THANK YOU VERY MUCH. [APPLAUSE] >> ONE QUESTION, AND THEN WE'RE GOING TO GO TO A BREAK. YOU'RE IT. >> WELL, OBVIOUSLY A PRODIGIOUS AMOUNT OF WORK, THIS IS OBVIOUSLY A MAJOR RESOURCE FOR ALL OF US WHO ARE INTERESTED IN HOW TO BUILD SUPPORT, THAT'S MY OWN INTEREST. IENT WIT WITH WITH AD -- I WANTED TO ASK THAT QUESTION WITH THAT WEALTH OF INFORMATION HOW DOES ONE -- WHO IS WORKING ON ACTUALLY TRYING TO TURN THIS INTO CASE SPECIFIC CLINICAL RECOMMENDATIONS, WHICH TEST IS SELECTED AMONG OR WHICH COMBINATIONS, THINGS LIKE THAT. >> SO THE HAT I WORE BEFORE, IN A FOUR-HOSPITAL SYSTEM THAT HAD USED EPIC, ONE OF THOSE PEOPLE CODING INTO THE ELECTRONIC MEDICAL RECORD, AND SO THE GENETIC TESTING REGISTRY, PART OF THE STRUCTURE OF IT, IS BUILD IN -- BUILT IN A WAY TO ENABLE THAT TYPE OF -- WELL, INFORMATION EXCHANGE, SO THAT WE REPRESENT THE ELECTRONIC MEDICAL RECORD THAT THEY PREFER, THE TYPE OF DATA TO TAKE IN AND CODES THAT WOULD INTERACT WITH LABORATORY MEDICAL SYSTEMS. AS FAR AS I THINK YOU REALLY ALLUDE TO THE IDEA OF WHAT'S A GOOD TEST. IS THAT RIGHT? SO WE SEE OURSELVES AT NCBI, THAT'S NOT EXACTLY OUR JOB TO DECIDE, AND EVEN THOUGH I'VE GOT A LOT OF LETTERS AFTER MY NAME, YOU KNOW, NOBODY SHOULD REALLY BELIEVE ME IF I'M THE ONE SAYING THIS IS A GOOD TEST OR BAD TEST. SO WE'VE TALKED WITH THE COMMUNITY ATG AMERICAN SOCIETY OF HUMAN GENETICS, TO THEIR BOARD, GONE TO THE AMERICAN COLLEGE OF MEDICAL GENETICS, MET WITH AMERICAN -- ASSOCIATION OF MOLECULAR PATHOLOGY, ACLA AND OTHER NAMES. PART OF WHAT WEČ COMMUNITY REFLECTS AS THEIR STANDARDS, CAP, GOING THROUGH PROFICIENCY TESTING AND WE THINK THAT'S HOW PEOPLE SHOULD REALLY MAKE THEIR DECISIONS. IN PART IT SHOULD BE BASED ON THE AMOUNT OF INFORMATION THAT PEOPLE ARE WILLING TO PROVIDE. AND TO SOME EXTENT WE ALSO PROMOTE THE TEST, IF YOU WILL, IN TERMS OF WHAT THE SEARCH RANKING IS, BASED ON THE RELEVANCY TO THE SEARCH, BUT THEN BASED ON THE DEPTH OF INFORMATION THAT'S BEEN PROVIDED WITHIN THE TEST.: SO WE HAVE SUGGESTED TO PROFESSIONAL ORGANIZATIONS THAT IF THEg 0– WERE WILLIN G TOHO REVIEW — TESTS, WE WOULD BE WILLING TOc PUT A GREEN STAR OR GOLD STAR OR TO SAY THIS.bq5 TEST HAS BEEN REVIEWED BY SUCH AND SUCHq/ w3 A PROFESSIONAL ORGANIZATION, AND IT WOULD BE FLAGGED. THE AMERICAN SOCIETY OF HUMAN GENETICS DIDN'T FEEL IT WAS THEIR PURVIEW. AMERICAN COLLEGE OF CLINICAL GENERALLEDDIC -- GENETICS HAS NOT DECIDED TO DO THIS. PART OF WHAT WILL PUSH IT, NOT SO MUCH CONCERN ABOUT LABORATORY DEVELOPED TESTS AS DIRECT TO CONSUMER TESTS. IF AND WHEN WE START SHOWING SHOWS, PEOPLE ARE GOING TO BE VERY ENGAGED IN PUTTING THEIR SAY-SO. v.o¨U (%v2—¨¨¨¨¨¨¨¨¨ THANK YOU VERY MUCH . >> THANK YOU SO MUCH. >> I'M AFRAID WE'RE GOING TO LOSE SOME PEOPLE AT THE BREAK. SO I WOULD LIKE TO ANNOUNCE THAT THERE'S SEVERAL OF US THAT MAY WILL BE OUR LAST MEETING, INCLUDING ME. SO I HAVE NAMED KATHY MENDENHALL TO BE THE CHAIR OF THE NOMINATING COMMITTEE, SHE'S GOING TO=/% BE HELPED BY KATHY MACE AND SIMON LU IF YOU HAVE NEW CHAIR, OR THE OLD CHAIR. [ LAUGHTER ] LET HER KNOW. WE'LL BREAK NOW AND BE BACK PLEASE AT 11:15 TO FINISH IT UP. >> MARY, ARE YOU READY? TIME. >> READY. >> I WAS REMINDED I SPOKE INAPPROPRIATELY. IT'S NOT THE OLD CHAIR. IT'S THE CURRENT CHAIR. [ LAUGHTER ] WE HAVE TO HAVE THE VOCABULARY JUST RIGHT. >> I REMEMBER WHEN PEOPLE WERE REFERRING, WHY DON'T WE CALL THEM EMERITUS TERM. >> ALL RIGHT. MARY? >> OKAY. >> WE'RE SHORT ON TIME. I'LL GO THROUGH THIS QUICKLY. THE SUBCOMMITTEE ON OUTREACH AND PUBLIC INFORMATION MET YESTERDAY MORNING FOR AN HOUR, AND WE HAD TWO PRESENTATIONS, ONE WAS BY DRDR. BARBARA RATH WHO IS NOT HERE TODAY, REPORTING ON K-12 PROGRAM INITIATIVES AND SHE REPORTED THAT NLM HAS BEEN ANALYZING THEIR K-12 PROGRAMS TO IDENTIFY AREAS FOR COLLABORATION AND COORDINATION, TO TRY TO FIGURE OUT IF THERE ARE ANY DUPLICATE EFFORTS. K-12 PROGRAM FALLS WITHIN THE CONSUMER HEALTH MISSION OF THE NATIONAL LIBRARY OF MEDICINE. THEY THINK INFORMING CHILDREN EARLY WILL BENEFIT THEM GIVING THEM MORE HELD INFORMATION EARLY THAT THEY CAN REACH OUT TO FAMILY MEMBERS SO THERE ARE A LOT OF BENEFITS TO WORKING WITH YOUNGER CHILDREN. THE K-12 PROGRAMS FALL INTO ONE OF FOUR CATEGORIES, PROVIDE INFORMATION RESOURCES FOR THE KINDERGARTEN THROUGH TWELFTH GRADE, PROMOTE LITERACY AND HELP GET YOUNG CHILDREN INTERESTED IN BEING HEALTH PROFESSIONALS. THERE ARE INFORMATION RESOURCES DEVELOPED SPECIFICALLY FOR THE K-12 AGE GROUP BY NLM LIKE TOX-TOWN, TOX-MYSTERY AND OTHER RESOURCES APPROPRIATE FOR KINDERGARTEN THROUGH TWELFTH GRADE, MEDLINE PLUS. HEALTH LITERACY IS THE SECOND PART OF THE K-12 PROGRAM, IT'S BEEN PROMOTED FOR KINDERGARTEN THROUGH 12th GRADE FOR SEVERAL PROJECTS INCLUDING TEEN HEALTH LEADERSHIP, AND THEN THERE'S SOME AFTER-SCHOOL PROGRAMS THAT HAVE BEEN DEVELOPED TO HELP PROMOTE INFORMATION. AND ALSO, WORKING WITH KINDERGARTEN THROUGH 12th GRADE IS AN IMPORTANT PART OF THE NATIONAL NETWORK OF LIBRARIES OF MEDICINE PROGRAMS, THEIR COMMUNITY-BASED OUTREACH PROGRAMS. THE THIRD CATEGORY OF ACTIVITIES FOR THE K-12 PROGRAM IS CURRICULUM SUPPORT, AND THIS INCLUDES DEVELOPING LESSON PLANS, IT CAN BE USED FOR THE CLASSROOM SETTING, AS WELL AS AFTER-SCHOOL PROGRAMS LIKE SCIENCE CLUBS AND THINGS LIKE THAT TO HELP CHILDREN BE BETTER EDUCATED. ABOUT THE SCIENCES. HEALTH CAREERS ARE PROMOTED THROUGH THE MENTORING IN MEDICINE PROGRAM WHICH WE HAD A PRESENTATION ABOUT IN THIS MEETING LAST YEAR, AND BARBARA SAID IT'S IMPORTANT TO REMEMBER THE KINDERGARTEN THROUGH 12th GRADE PROGRAM IS DESIGNED TO DEVELOP INFORMATION RESOURCES, NOT TO TEACH CLASSES. THEY ARE TRYING TO PROVIDE RESOURCES THAT CAN BE USED BY BIOLOGY TEACHERS, SCHOOL LIBRARIANS, BY OTHERS TO HELP MEET THE GOALS OF THE PROGRAM. MOVING FORWARD, THE PLANS ARE TO INCREASE CURRICULUM SUPPORT TO COVER ADDITIONAL RESOURCES, AND TO PROMOTE MORE WIDESPREAD AWARENESS OF THE RESOURCES ALREADY AVAILABLE. DR. MICHAEL ACKERMAN GAVE A FASCINATING PRESENTATION ON NLM IMAGES PROJECT. >> COULD I INTERRUPT? DOES ANYBODY IN THE GROUP TELL YOU ABOUT OMB? IT'S THEIR POLICY ALL THIS STUFF SHOULD STOP, ALL S.T.E.M. EDUCATION, ALL K-12 SHOULD BE FUNDED IN THE DEPARTMENT OF N.S.F., AND SPLIT WITH SMITHSONIAN. NOW, IT'S HARD TO IMAGINE AN OBAMA ADMINISTRATION THEY WOULD ALLOW THAT BECAUSE ALMOST EVERYTHING THAT WE DO IN THIS K-12 AND INSTITUTES AT NIH, THERE'S NO DUPLICATION, ALL OF IT IS REALLY AIMED AT RECRUITMENT OF MINORITIES IN HEALTH CARE PROFESSION, SCIENCE PROFESSIONS. AND THE OMB IS TOTALLY OPPOSED TO IT, ENDLESSLY HARASS US ABOUT THE REASON WE'RE PUTTING THIS TOGETHER IS BECAUSE WE WANT TO PUT IT TOGETHER SO THEY CAN BE CUT OFF THEY NECK. THAT'S WHY I MENTION THAT, IN THE APPROPRIATION, THE FIRST APPROPRIATION WE'VE HAD IN THREE OR FOUR YEARS, THERE IS LANGUAGE IN THE SENATE THAT SAYS YOU OUGHT TO CONTINUE THIS S.T.E.M. WORK. >> HLM IS NOT THE ONLY AGENCY OBJECTING TO THIS. >> ABSOLUTELY NOT. >> TO THIS OMB POSITION. >> YOU'RE POINTING OUT -- YOU'RE POINTING OUT GOOD THINGS WE'RE DOING. THEY TOTALLY HATE IT. THEY WANT TO TAKE THOSE DOLLARS AWAY. >> WHY? >> I THINK THEY ARE IDIOTS. >> I KNOW, BUT WHY? >> WELL, THEY THINK IT'S DUPLICATIVE AND IT WOULD BE MORE EFFICIENT, IT'S ALL STUFF, GIVE IT TO N.S.F., THEY WILL DO SOMETHING. THEY ARE NOT GOING TO RECRUIT DOCTORS AND NURSES BUT MAYBE A SCIENTIST OR MATHEMATICIAN, WHO KNOWS. N.S.F. IS NOT COMPLICIT IN THIS. ALSO ALL OMB. >> I GUESS THE NEXT QUESTION IS, IS THERE ANYTHING WE CAN DO? >> YES, ENDORSE WHAT WE'RE DOING. >> SHOULD WE ACCEPTED A LETTER LET -- SEND A LETTER TO THE BOARD OF REGENTS? THE WHOLE BOARD SHOULD HAVE A -- >> WELL, IF YOU HAVE TIME TO DO IT, THAT'S GREAT. THE REPORTS DO GO TO THE DIRECTOR OF NIH AND SECRETARY OF HHS, BOTH OF THOSE WILL BE FRIENDLY TOWARD YOUR POSITION, BUT I DON'T KNOW, I CAN'T IMAGINE WHY THEY ARE BEHAVING THIS WAY. WHITE HOUSE AND OMB ARE THE PROBLEM, STATE FORWARD. >> WOULD SOMEONE SEND US ADDRESSES? >> AND DRAFT LANGUAGE? >> SEND THE REPORT TO ME, I'M OBLIGED TO SEND IT TO NIH. IT'S AWFUL. >> WOW. >> COULD WE AT SOME FUTURE MEETING, I MEAN, I DON'T HAVE A SENSE OF HOW WITHIN N.L.M. EFFORT AND MONEY ARE SPENT. WE HEAR A REPORT. >> SURE. >> BUT I DON'T SEE HOW THE BUDGET ITSELF IS STRUCTURED AND WHAT FREEDOM -- I DON'T KNOW IF THE APPROPRIATION SAYS YOU SHOULD -- >> NO, IT DOESN'T. >> ONE LINE. >> OKAY. >> YEAH, WHEN WE REPORT, WE HAVE TO PRESENT TO CONGRESS A VERY -- A MORE DETAILED BREAKDOWN OF HOW WE'RE GOING TO SPEND THE GRANT DOLLARS IN DIFFERENT CATEGORIES OF GRANT. THAT'S A REQUIREMENT AT THE BEGINNING OF THE FISCAL YEAR AND WE -- NOT THE BEGINNING OF THE FISCAL YEAR BECAUSE IT NEVER HAPPENS. WHEN WE FINALLY KNOW WHAT OUR BUDGET IS, WE HAVE TO SAY THIS IS HOW MUCH IS GOING TO GO IN THE GRANTS BUDGET AND HERE IS IT DIVIDED AMONGST CERTAIN TYPES OF CATEGORIES. YOU KNOW, WHETHER IT'S GOING TO BE RO-1 GRANTS OR TRAINING GRANTS OR WHATEVER. BUT FORTUNATELY, THE INTRAMURAL PROGRAM IS BUDGET IS THE REP. >> IT WOULD BE GOOD TO SEE THAT. >> THERE IS OFTEN, THIS PARTICULAR YEAR THE APPROPRIATION DOESN'T SAY NLM IS ENCOURAGING BUT OFTEN THEY DO. USUALLY IT'S HELPFUL. SO THAT DOESN'T SAY SPEND THIS NUMBER OF DOLLARS, CONTINUE TO WORK ON THIS OR TRY TO WORK ON THAT. WE'RE QUITE RESPONSIVE TO THAT AT NIH. SO I'M SAYING AS IF THERE'S A FEW THINGS, BUT NIH OVERALL YOU COULD HAVE AS MUCH AS 150, 200, DON'T FORGET ABOUT THIS. THEY ARE CAREFULLY LOOKED AT BUT SOMETHING IS ACKNOWLEDGING THEM AND TRYING. BUT THEN AGAIN ALSO, IT'S LIKE IN UNIVERSITIES, WE USED TO LOVE THIS, WE WOULD WORK OUT THE MED CENTER BUDGET PROBLEM DOWN TO THE LAST TIME AND HE WOULD DECLARE A HOLIDAY. WELL, THAT MEANS EVERYBODY IS PAID TIME-AND-A-HALF, FOR GOD'S SAKE. THE MEDICAL CENTER DIDN'T SHUT DOWN BECAUSE HE[ (RR)xcRr/8O„?'skˇ]+(-KT„@TF]x nZ< N$ 2 k-P]V$E2 HOW DIFFICULT IT IS TO GET AN IMAGE THAT WILL BE IDENTIFIABLE BECAUSE IT DEPENDS ON WHAT ANGLE YOU TAKE A PICTURE FROM. >> THEY HAVE DONE A GOOD JOB. >> THE COLOR OF THE BACKGROUND THE PILL IS SITTING ON WILL AFFECT THE COLOR OF THE PILL, WHETHER OR NOT YOU COLORIZE IT TO GET RID OF REFLECTIONS, IT MAKES THE PILL LOOK FLAT. IT WAS A FASCINATING PRESENTATION. >> THERE'S ANOTHER WHOLE APPLICATION OF THAT THAT WAS INTERESTING, THAT IS FOR CONSUMER MEDICATION COMPLIANCE, PEOPLE THAT BASICALLY COULD SAY, WELL, THIS IS THE COLOR PILL I SHOULD TAKE NOW. >> RIGHT, RIGHT. IT'S A REALLY EXCITING PROJECT. THEY ARE GOING TO HAVE THIS CONTEST TO SEE WHO CAN DEVELOP THE BEST PROGRAM AND THEN THE GROUP THAT DEVELOPS THE BEST PROGRAM WILL WIN AN AWARD. AND THEN I'M ASSUMING N.L.M. WILL TAKE THAT PROBLEM AND USE IT TO TRY TO FURTHER DEVELOP THEIR PROGRAM. BOTH PRESENTATIONS YESTERDAY WERE REALLY GOOD. THE SUBCOMMITTEE IS RECOMMENDING THAT BOTH THESE PRESENTATIONS BE PRESENTED TO THE FULL BOARD OF REGIONENTS AT SOME POINT SOON DEPENDING ON HOW THE CONTENT MOVES ALONG, MORE APPROPRIATE FOR THE FALL MEETING. >> THAT WAS OUR THOUGHT WAS TO GIVE YOU ALL AN ADVANCE AND WHEN IT'S FURTHER ALONG. >> RIGHT, RIGHT. AND THEN THE MEETING ADJOURNS AT 8:45. >> I HAVE A COUPLE COMMENTS, THE REVIEW, I THINK THAT THERE ARE A COUPLE GROUPS OF PEOPLE THAT AT SOME POINT WE MIGHT THINK OF DOING THINGS SPECIAL FOR, BLIND PEOPLE, BLIND KIDS AND DEAF KIDS. BECAUSE THERE ARE THINGS WE COULD DO FOR THEM THAT NOBODY -- THE FINAL THING I WANT TO SAY, HAVE WE EVER THOUGHT OF DEVELOPING AN NLM APP? >> A WHAT? >> AN NLM APP. >> THERE IS FOR THIS. >> THERE ARE APPS. >> TAKE A PICTURE OF A PILL. >> I MEAN FOR NLM IN GENERAL, MEDLINE. >> YEAH. >> THERE ARE? >> SURE. >> I DIDN'T KNOW THAT. HOW COME WE NEVER HEARD ABOUT THAT? >> I DON'T KNOW. WITH RESPECT TO THE BLIND WHEN I CAME HERE THEY DEVOTED QUITE A LOT OF SPACE IN THE READING ROOM TO BLIND SUBSCRIBERS. I SAID I THOUGHT THERE WERE MORE MODERN THINGS THAT COULD BE DONE TO HELP THEM AND SHE ASSURED ME SHE WAS RIGHT, OUR BLIND PATRONS DIDN'T WANT ANY CHANGES TO ANYTHING. THEY FINALLY LEARNED HOW TO WORK THE STUFF WE GAVE THEM, DID NOT WANT CHANGES. SO THAT THEY ARE A VERY CONSERVATIVE GROUP, THE DEAF ARE MORE CONSERVATIVE. LOTS OF PEOPLE, LOTS OF -- (TECHNICAL DIFFICULTIES, PLEASE STAND BY) >> ALL RIGHT. THANK YOU. WE HAVE ONE MORE ITEM, KEN KOYLE AND DIGITIZING NLM HISTORY. >> AT HIGH SPEED. >> WELL, GOOD MORNING. I'M KEN KOYLE, DEPUTY CHIEF OF THE MOUNTAIN MEDICINE DIVISION. THANK YOU FOR THE OPPORTUNITY TO TALK WITH YOU FOR A FEW MINUTES THIS MORNING ON OUR CURRENT EFFORTS TO DIGITIZE SOME OF THE IMPORTANT MATERIALS WE HAVE AT THE NATIONAL LIBRARY OF MEDICINE AND OUR COLLECTIONS AND HOW THAT EFFORT IS GOING. SO BEFORE I GET INTO THE SPECIFICS ABOUT THE CURRENT PROJECT, I'D LIKE TO PROVIDE A QUICK INTRODUCTION TO NLM DIGITAL COLLECTIONS FOR THOSE WHO MIGHT NOT BE FAMILIAR WITH IT. THE PROJECT ITSELF WAS CONCEIVED IN 2006, LAUNCHED IN 2010, NOW NUMBERS ABOUT 10,000 ITEMS, IT'S THE NLM'S FREE ONLINE ARCHIVE OF BIOMEDICAL RESOURCES, AND IT'S A CONSTANTLY GROWING COLLECTION. ALL OF THE CONTENT IN DIGITAL COLLECTION IS IN THE PUBLIC DOMAIN, FREELY AVAILABLE WORLD ARMY MUSEUM. YOU MIGHT ASK IF THIS IS AN NLM COLLECTION WHY DID WE INCLUDE THE NATIONAL MUSEUM OF HEALTH AND MEDICINE PUBLICATIONS? WELL, SOME BACKGROUND ON THAT. BOTH OF THESE INSTITUTIONS SHARE A COMMON LEGACY, THEY BOTH CAME OUT OF THE ARMY SURGEON GENERAL'S OFFICE, HOUSED TOGETHER FROM THEIR INCEPTION UNTIL 1947 WHEN THE MUSEUM MOVED OUT OF THE OLD REDBRICK AND THAT'S PICTURED HERE, THE OLD REDBRICK BUILDING ON INDEPENDENCE AVENUE, THAT HOUSED BOTH THE -- THAT'S THE LAST BUILDING THAT HOUSED THE NATIONAL HUMAN OF HELD AND -- HEALTH AND MEDICINE, AND THE NATIONAL LIBRARY OF MEDICINE WHICH AT THAT TIME WAS THE ARMED FORCES MEDICAL LIBRARY AND THEY STOLE OUR MICROSCOPES. [ LAUGHTER ] THAT WAS APPROPRIATE BECAUSE THEY COLLECT ARTIFACTS, AND WE DON'T. YEAH, THEY HAVE A WONDERFUL MICROSCOPE COLLECTION WITH SOME OTHER THINGS. SO THE ARMED FORCES MEDICAL LIBRARY BECAME THE NATIONAL LIBRARY OF MEDICINE IN 1956, THE ARMY MEDICAL MUSEUM BECAME THE ARMED FORCES INSTITUTE OF PATHOLOGY IN 1946 AND THE NATIONAL MUSEUM OF HEALTH AND MEDICINE IN THE 70s WITH A BEAUTIFUL NEW BUILDING, THEY HAVE RECENTLY MOVED INTO WHEN THEY MOVED OFF THE WALTER REED CAMPUS. WE HAVE THIS SHARED LEGACY, YOU KNOW, GROWING OUT OF THE SAME INSTITUTION, WE HAVE SHARED GOALS AND SHARED FOCUS ON INFORMING THE PUBLIC ABOUT PAST, PRESENT AND FUTURE HEALTH AND MEDICAL ISSUES. SO THAT'S WHY WE DECIDED THAT WE SHOULD INCLUDE THEIR MATERIALS IN THIS NLM-SPECIFIC COLLECTION. LET ME GIVE YOU A FEW EXAMPLES OF THE CONTENT YOU'LL FIND IN THIS COLLECTION. FIRST OF ALL YOU'LL FIND SOME OLD PUBLICATIONS LIKE THE 1864 CATALOG OF THE SURGEON GENERAL'S LIBRARY, AND THE MEDICAL AND SURGICAL HISTORY OF THE WAR OF THE REBELLION PUBLISHED IN 1870 AND LESS OLD PUBLICATIONS LIKE THE FULL 61 VOLUME SET OF THE CATALOG FROM 1880 TO 1961. UP HERE ON THE SCREEN YOU CAN SEE THE TITLE PAGE OF THE FIRST AND LAST VOLUME OF THE SERIES. THESE IMAGES ARE TAKEN DIRECTLY OUT OF THE DIGITAL COLLECTIONS. AND YOU'LL ALSO FIND SOME NEWER THINGS, LIKE A 1963 BOOKLET ABOUT THE ORIGIN OF MEDLARS, VIDEO ABOUT THE HUMAN GENOME PROJECT AND VIDEO RECORDING A PRESENTATION OF THE FIRST 175 YEARS OF THE NATIONAL LIBRARY OF MEDICINE. YOU'LL ALSO FIND SOME EFEMORAL THINGS LIKE A 1946 PAMPHLET QUOTING DR. WILLIAM WELCH OF JOHNS HOPKINS DESCRIBING NLM AS AMERICA'S MOST IMPORTANT CONTRIBUTION TO MEDICAL KNOWLEDGE AND THE PROGRAM FOR THE SWEARING IN CEREMONY FROM 1984 FROM DR. DONALD LINDBERG. I'M NOT SURE WHY IT'S NOT AN ANNUAL HOLIDAY TODAY. OBVIOUSLY LEGISLATION IS PENDING. YOU'LL FIND FROM THE -- >> TIME-AND-A-HALF. >> IS THAT WHAT IT WAS? WE WOULD STILL HAVE TO WORK, RIGHT? WE'LL FIND SOME UNIQUE ITEMS FROM THE NATIONAL MUSEUM OF HEALTH AND MEDICINE SUCH AS THE LIST OF SKELETONS AND CRANIA THEY SEL HELD IN 1876 AND WHY THE THEY NEEDED TO EXIST, DEFINING THEMSELVES AT THAT POINT. SO NOW YOU KNOW ABOUT WHAT'S IN THE COLLECTION, WHAT THIS COLLECTION IS COMPRISED OF SO LET ME TELL YOU HOW WE SEE THIS BEING USED, OUR ENVISIONED USES FOR THE NLM PUBS AND PRODUCTIONS COLLECTION. THE MOST OBVIOUS USE IS A FUTURE HISTORIAN WRITING THE HISTORY OF THE NATIONAL LIBRARY OF MEDICINE WHICH WAS LAST DONE IN 198 BY WYNDHAM MILES. THIS IS NLM PUBS AND PRODUCTION COLLECTION WILL PRIDE A PRE-MADE SET OF PRIMARY -- PROVIDE A PRE-MADE AND SECONDARY SET OF MATERIALS A UMP WHYING OFF -- A JUMPING OFF POINT, SUCH A PROJECT WOULD BE BROAD IN SCOPE BUT WOULD PROVIDE A GOOD STARTING POINT FOR THAT. LESS OBVIOUS USES OF THE COLLECTION WOULD BE CULTURAL HISTORIANS LOOKING AT GOVERNMENTAL INFLUENCE ON PUBLIC HEALTH, OR HISTORIANS OF TECHNOLOGY MAYBE LOOKING AT EARLY DIGITAL FILE SHARING AND THE INTERNET BECAUSE NLM WAS KIND OF AT THE FOREFRONT OF THOSE AREAS FROM THE EARLY '80s. AND REALLY BACK TO THE BEGINNING OF MEDLARS. MILITARY MEDICAL HISTORIANS OF COURSE LOOKING AT EFFORTS BY THE OFFICE OF SURGEON GENERAL TO INCLUDE -- TO IMPROVE MEDICAL EFFICACY AND TRAINING, OR ANY NUMBER OF OTHER RESEARCH AREAS REALLY, LIMITED ONLY BY THE IMAGINATION. A QUICK RESEARCH EXAMPLE, HOW A RESEARCHER MIGHT USE THIS COLLECTION. IMAGINE IN THIS EXAMPLE YOUR A STAFF MEMBER AT JOHNS HOPKINS HOSPITAL, OF COURSE FOUNDED IN 1889. SO YOU'RE COMING UP ON THE 125th ANNIVERSARY OF JOHNS HOPKINS HOSPITAL IN MAY, AND YOU DECIDE YOU WANT TO DO SOMETHING TO COMMEMORATE THAT ANNIVERSARY. SO YOU ARE RESEARCHING THE HISTORY OF HOSPITAL. MAYBE YOU'RE NOT A HISTORIAN BUT YOU'RE TRYING TO DO SOME HISTORICAL RESEARCH, AND YOU KNOW THE ADMINISTRATION BUILDING AT THE HOSPITAL IS NAMED FOR SOMEONE CALLED JOHN SHAW BILLINGS, AND SO YOU DECIDE TO START YOUR RESEARCH BY FINDING OUT SOMETHING ABOUT BILLINGS. AT THE HOPKINS WEBSITE, IT MENTIONS BILLINGS WAS ONCE THE DIRECTOR OF THE NATIONAL LIBRARY OF MEDICINE, SO IN YOUR EFFORTS TO FIGURE OUT WHO BILLINGS IS, YOU DECIDE TO CONSULT THE NATIONAL LIBRARY OF MEDICINE COLLECTIONS. WHAT YOU'LL FIND WHEN YOU CONSULT THESE COLLECTIONS, SORRY ABOUT THAT. PULL UP THE PAGE, THIS IS THE FIRST THING YOU'LL SEE. YOU GO TO COLLECTIONS AND YOU CAN REFINE IT AND YOU KNOW WHAT YOU WANT IS SPECIFICALLY ABOUT THE NATIONAL LIBRARY OF MEDICINE, IT SHOULD BE IN OUR COLLECTIONS, THE COLLECTION WE'RE TALKING ABOUT TODAY, SO YOU SELECT THAT ONE, YOU SEE THAT YOU GOT 501 ITEMS. NOW, SPECIFICALLY, YOU'RE LOOKING FOR BILLINGS. YOU DO A SEARCH FOR BILLINGS, YOU SEE YOU HAVE 227 TEXTS OUT OF 500 THAT SOMEHOW MENTION BILLINGS. THIS ONE IS PROMISING, SELECTED PAPERS OF JOHN SHAW BILLINGS, MAYBE HE DISCUSSES INTERACTION WITH HOPKINS. SO YOU CHECK OUT THE 300-PAGE BOOK. YOU JUST WANT TO KNOW WHAT BILLINGS HAD TO DO WITH HOPKINS AND SEARCH WITHIN THE TEXT FOR HOPKINS, AND YOU CAN PULL UP SPECIFIC PAGES WHERE HOPKINS IS MENTIONED. OR SEARCH HOPKINS HOSPITAL OR HOPKINS UNIVERSITY OR WHATEVER KEY WORDS YOU WANT TO SEARCH FOR. NOW YOU CAN READ JUST THE TEXT THAT APPLIES TO THE RESEARCH YOU'RE TRYING TO DO. YOU CAN DO SEARCHES LIKE THIS ACROSS THE REPOSITORY, ACROSS A SINGLE PUBLICATION, WHATEVER IS NECESSARY FOR YOUR RESEARCH. >> YOU HAVE THE ARCHITECTURAL DRAWINGS FOR THE HOSPITAL, HOW DO YOU GET THAT? >> I SUPPOSE YOU COULD SEARCH -- I DON'T KNOW IF HE WOULD HAVE TALKED ABOUT THAT IN HIS OWN PAPERS. >> WHAT IF. >> YEAH, WELL, IT WOULD ENTAIL A BIT OF READING, THROUGH HIS PAPERS OR GO BACK UP AND PERHAPS SEARCH HOPKINS, OR BILLINGS AND ARCHITECTURE AND SEE WHAT PUBLICATIONS CAME UP. >> IT REFLECTED THE SORT OF VIEW AT THAT TIME OF CONTAGIONS, A PAVILION STYLE HOSPITAL, LOW LINES. IT WAS AN INTERESTING THING. THEY KNEW SOME STUFF BUT NOT AT ALL WHAT WAS KNOWN 20 YEARS SUBSEQUENT. >> I DON'T KNOW IF YOU PICKED THIS EXAMPLE CAREFULLY BUT I'LL BET YOU IF YOU GO TO GOOGLE AND TYPE IN BILLINGS MEDICINE YOU WILL NOT GET ANY OF THIS. >> WELL, ACTUALLY IF YOU GO TO GOOGLE AND I ACTUALLY DID DO THAT BEFORE THIS, IF YOU SEARCHED BILLINGS, I DIDN'T SEARCH BILLINGS AND MEDICINE BUT ONE OF THE FIRST THINGS IS THE HOPKINS PAGE MENTIONING THAT BILLINGS WAS THE DIRECTOR OF THE NATIONAL LIBRARY OF MEDICINE. SO THAT WOULD LEAD YOU TO US, IF YOU SEARCHED JUST JOHN SHAW BILLINGS, ONE OF THE FIRST HITS IS A PAPER THAT NLM PRODUCED AT THE CENTENIARRY OF THE TENURE AND THAT WOULD LEAD YOU BACK TO NLM. THERE ARE A FEW -- YOU'RE RIGHT, MOST RESEARCHERS ESPECIALLY CASUAL RESEARCHER WOULD START AT GOOGLE BUT THERE'S STILL A COUPLE PATHS THAT BRING YOU BACK TO NCLM. >> SEARCH INTERNALLY, IT WILL SAY SEE ST. JOHN. >> EXACTLY. >> ARE THERE ANY PLANS TO BE ABLE TO -- IF YOU ANNOTATE WIKIPEDAI ENTRIES? >> THE SHORT ANSWER IS YES, THE MORE COMPLICATED ANSWER IS THAT WE HAVE KIND OF AN ONGOING WITHIN THE HISTORY OF MEDICINE DIVISION, AN ONGOING PROJECT, ONE OF OUR REFERENCE LIBRARIANS, ANN ROTHFELT SPEARHEADED THIS, SHE'S BEEN SEARCHING THINGS IN WIKIPEDIA THAT RELATE BACK TO THINGS THEY CAN LINK DIRECTLY TO THE DIGITAL COLLECTIONS, AND SHE'S BEEN WORKING TO DO LINKAGE BACK FROM WIKIPEDIA TO OUR COLLECTION, IT'S BEEN VERY SUCCESSFUL. SHE'S MADE A LOT OF COLLECTIONS. >> DOES WIKIPEDI, A ALLOW PICTURES? >> IT'S AN ENCYCLOPEDIA ONLINE. >> I KNOW WHAT IT IS. >> LITTLE TINY ONES. >> YEAH, WIKI COMMONS WOULD HANDLE THAT. THAT'S ONE EXAMPLE. THERE ARE COUNTLESS EXAMPLES OF HOW THIS RESOURCE COULD BE USED. SO HAVING ADDRESSED WHAT'S IN THE COLLECTION AND HOW THE COLLECTION MIGHT BE USED, I THINK IT'S APPROPRIATE TO GIVE YOU A QUICK OVERVIEW OF HOW THINGS GOT IN THERE IN THE FIRST PLACE. SO HOW DID THIS PRINTED ANALOG CONTENT GET INTO THE DIGITAL FORM, INTO THE DIGITAL COLLECTIONS? IT STARTS WITH SOMEBODY PROPOSING A PROJECT. IN THIS CASE, IT WAS -- THE PROJECT WAS NLM PUBLICATIONS AND PRODUCTIONS, PRESENTING THAT PROJECT TO THE DIGITAL PROJECT SELECTION GROUP AND THAT GROUP MEETS WITH THE PROPOSER, THE PERSON WHO HAD THE IDEA FOR THE PROJECT AND THEY DISCUSSEDDED THE SPECIFICS OF IT AND WHAT THE VALUE OF THE PROJECT IS. AND THEN THEY GET THE APPROVAL. ONCE THE DIGITAL PROJECT SELECTION GROUP APPROVES IT, THEY CONFER WITH THE TECHNICAL EXPERTS AND THEY CONFER WITH THEM TO MAKE SURE THE PROJECT IS FEASIBLE, WITH EITHER THE CURRENT OR PLANNED WORK FLOWS THAT WE HAVE FOR DIGITIZATION. ONCE THE DIGITAL REPOSITORY WORKING GROUP SAYS YES, THIS IS SOMETHING WE CAN DO, THEY WILL PASS IT DOWN TO THE SCANNING WORKING GROUP AND THEY PUT IT INTO THE WORK FLOW, THE SCANNING WORKING GROUP WILL PREPARE THE ITEMS FOR SCANNING, AND COORDINATE AND PERFORM THE FIRST DIGITIZATION OF THE ANALOG ITEMS. AFTER WHICH THE DIGITAL REPOSITORY WORKING GROUP FINISHES THE PROCESS WITH A COORDINATED INGEST INTO THE DIGITAL REPOSITORY, OVERSEEN BY THE DIGITAL PROJECTS OVERSIGHT GROUP, THE SENIOR LEADERS FROM LIBRARY OPERATIONS HERE AT NLM, THEY OVERSEE THE ENTIRE PROCESS, THEY KEEP TABS ON THE SPECIFIC PROJECTS AND THEY RENDER DECISIONS ABOUT QUESTIONS THAT COME UP REGARDING THINGS LIKE CONTENT, META DATA AND OTHER ISSUES THAT MIGHT ARISE. THIS IS LINEAR AND SIMPLIFIED WAY OF EXPLAINING IS. DIGITIZATION IN PRACTICE IS A MORE INTERACTIVE PROCESS. SO EACH OF THESE GROUPS ALTHOUGH THEY ARE SPECIFICALLY FOCUSED ON ONE ASPECT OF DIGITIZATION, THEY ARE ALL IN VERY CLOSE CONTACT WITH EACH OTHER, THERE'S A LOT OF DISCUSSION BACK AND FORTH, THEY COMMUNICATE AND SHARE INFORMATION CONTINUOUSLY AND EVERYONE IS IN CONSTANT CONTACT AND MAYBE MEMBERS OF THE GROUP SIT ON MULTIPLE GROUPS FACILITATING COOPERATION BETWEEN THEM. THAT'S HOW WE TAKE THINGS FROM THE ANALOG, GET THEM THROUGH THE PROCESS AND DIGITIZE THEM INTO THE COLLECTIONS. SO WHERE DO WE SEE THIS GOING? WHAT'S THE FUTURE OF NATIONAL LIBRARY OF MEDICINE'S DIGITAL COLLECTIONS? IT'S AN OPEN-ENDED PROJECT, WE SEE IT CONTINUING ENDEFINITELY. WE'LL CONTINUE TO ADD AS NEW AND NEWLY DISCOVERED THINGS BECOME AVAILABLE AND RESOURCES ALLOW US TO DO SO. BY NEWLY DISCOVERED, AS WE DID THIS, WE IDENTIFIED THE THINGS THAT ARE CATALOGED AND IN OUR COLLECTIONS THAT WE COULD DIGITIZE AND PUT ONLINE. BUT PEOPLE ARE ALWAYS FINDING OTHER ITEMS THAT MAYBE WEREN'T CATALOGED, AS THEY CLEAN OUT THEIR OFFICE, THEY MIGHT FIND AN OLD STRATEGIC PLAN OR LIBRARY SERVICES BROCHURE THAT WOULD BE APPROPRIATE BUT WE DIDN'T HAVE A CATALOG BEFORE. WHEN THEY FIND THEM WE'LL CATALOG THEM AND ADD THEM TO THE PHYSICAL COLLECTION AND TO THE DIGITAL REPOSITORY. SOME OF THE POTENTIAL FUTURE ADDITIONS WOULD BE DIGITIZED MANUSCRIPT MATERIALS FROM THE NLM ARCHIVES, BOOKS PRINTED BEFORE 1501 AND OTHER RATHER BOOKS, INCUNABULA, AND ONLINE AND ON-SITE EXHIBITIONING EXHIBITIONS AND WE WANT TO LINK BACK TO THEM. THE NLM PUBLICATIONS WILL PROMOTE AND PRESERVE IMPORTANT WORKS CREATED BY AND FOR THE LIBRARY. SO IN CONCLUSION, THIS PROJECT REPRESENTS A VERY IMPORTANT STEP IN NLM'S DIGITAL FUTURE. AS WE DIGITIZE NLM'S HISTORY WE ACCOMPLISH THREE IMPORTANT THINGS, FIRST THE PERMANENT REDUNDANT STORAGE OF FACSIMILES OF HISTORICAL DOCUMENTS. SECOND, IMPROVED ACCESS TO MATERIALS FOR RESEARCHERS AROUND THE WORLD. AND THIRD, WE'RE ESTABLISHING A RECORD THAT CAN HELP US TO THINK CRITICALLY ABOUT THE PRESENT AND THE POTENTIAL FUTURE OF THE LIBRARY. SO WITH EACH NEW PROJECT IN THE DIGITAL REPOSITORY, NLM IS SETTING NEW STANDARDS FOR DIGITIZED CONTENT WITH MORE THOROUGH META DATA, CONTINUOUSLY IMPROVING ORGANIZATIONAL FRAME WORKS, AND HIGH RESOLUTION IMAGERY AND OF COURSE STRINGENT QUALITY CONTROL MEASURES. WITH THAT, I'LL DO MY BEST TO ANSWER ANY QUESTIONS YOU MIGHT HAVE. [APPLAUSE] ANY QUESTIONS? >> YES. >> I THINK IT'S REALLY SO INTERESTING, THIS REMINDS ME OF THE FACT THAT YOU'RE MAKING THIS AVAILABLE TO PEOPLE WHO WOULD NEVER BE ABLE TO GO AND FIND BOOKS AND NOT BE ALLOWED TO TOUCH IT, IT REMINDS ME HOW FORWARD THINKING IT WAS WHEN DR. LINDBERG STARTED TURNING THE PAGES, THIS IS AN EXTENSION OF THAT. AND I WAS VERY IMPRESSED WITH NOT ONLY THE IDEA OF USES BUT THE FACT YOU MENTION THAT IT WAS COMPLIMENTARY TO PubMed CENTRAL, WOULD YOU MENTION THAT MORE? >> YEAH, THAT WAS FROM THE INCEPTION OF THIS DIGITAL REPOSITORY ITSELF, NLM DIGITAL COLLECTIONS, THAT WAS A KEY ASPECT, THAT WE ALREADY HAVE THIS WONDERFUL DIGITAL REPOSITORY OF JOURNAL MATERIALS, YOU KNOW, FREE ACCESS TO THE JOURNAL MATERIALS THROUGH PubMed CENTRAL, AND WE WANTED TO MAKE SURE WE'RE NOT REPLICATING OR COMPETING. THE CONTENT WE'RE PUTTING INTO THE DIGITAL COLLECTIONS INCLUDES WE STARTED WITH A MONO GRAPHIC MATERIALS, AUDIO VISUAL MATERIALS, THINGS THAT WOULDN'T HAVE BEEN PART OF PubMed CENTRAL. WE WANT TO MAKE SURE THAT WE'RE NOT -- WE DON'T HAVE THAT UNNECESSARY REDUNDANCY OR COMPETITION. AS WE LOOK TO THE FUTURE AND WE START THINKING ABOUT DIGITIZING HISTORICAL SERIALS, AGAIN, WE'RE MAKING SURE THAT THE SERIALS THAT WE'RE DIGITIZING FOR OUR COLLECTION ARE NOT THINGS ALREADY IN PubMed CENTRAL. WOULDN'T MAKE SENSE TO REDO THAT WORK. AND THE INTENT OF THE TWO SYSTEMS, THOROUGH COMPLEMENTARY, THERE'S A DIFFERENT FOCUS FOR THE TWO PROGRAMS. WE DON'T HAVE A LINK TO PubMed CENTRAL FROM THE DIGITAL REPOSITORY. >> OR VICA VERSA. >> YEAH, IT'S SOMETHING WE CAN LOOK IT'S A. THAT'S SOMETHING TO EXPLORE. >> THERE'S WAYS TO FACILITATE VISITING A LIBRARY, SOMETHING YOU GET INTERESTED IN AND WANT TO SEE? THERE ARE ALL KINDS OF DIFFICULTIES THAT DIGITALIZATION DOESN'T SOLVE. >> ONE OF THE HOPES IS THAT SOMEBODY WILL FIND CONTENT DIGITALLY AND SAY THAT'S SOMETHING I NEED TO PUT HANDS ON, ESPECIALLY WHEN WE DIGITIZE THE INCUNUBALA. SOMETHING TO LOOK AT A BOOK FROM 1494 IN A PICTURE ON THE COMPUTER, BUT A SERIOUS SCULLER WHO I --SCHOLAR WHO WANTS TO KNOW THE HISTORY OF THE BOOK, THEY WANTS TO PUT HANDS ON IT. >> MEDICAL AND SURGICAL HISTORY OF THE REBELLION IS REALLY A MARVELOUS PUBLICATION, AROUND 12 OR 14 VOLUMES, WE HAVE ONE IN GOOD CONDITION. IT DOESN'T -- WE DON'T WANT TO ENCOURAGE PEOPLE TO MAN HANDLE IT. THERE'S A SUBSEQUENT PUBLISH SAYINGS SOLD COMMERCIALLY -- PUBLICATION THAT WAS SOLD COMMERCIALLY THAT THE LIBRARY BOUGHT. THEY DON'T OVERLIE. THE ORIGINAL HAS GOT BEAUTIFUL STUFF, THE DRAWINGS FOR INSTANCE OF GANGRENE ARE ORIGINAL WATERCOLORS, MAGNIFICENT. >> WHEN THIS IS DIGITIZED, AND PUT ONLINE, DOES GOOGLE GO THROUGH IT AND -- >> IS IT INDEXED BY GOOGLE, IS THAT WHAT YOU ARE ASKING? >> RIGHT. >> I DON'T KNOW THAT IT'S INDEXED BY GOOGLE BUT I DO NOT THAT GOOGLE SEARCH WILL RETURN SOME OF OUR ITEMS. I'M HONESTLY NOT SURE. >> TALK TO DAVID. HE HAS ACCESS. >> IN OUR DISCUSSIONS I WAS REALLY INTERESTED, I KNOW YOU SAID THE WORK FLOW IS IN PLACE, IT SEEMS TO ME IT'S CONSTANTLY UPDATED BUT THE FACT THAT YOU POINTED OUT THAT HOW CAREFUL YOU ARE MAKING SURE THAT THE THINGS ARE THE HIGHEST QUALITY, ALSO THAT ONE SINGLE PROJECT CAN ACTUALLY TAKE UP TO SIX MONTHS. >> YEAH, THAT'S CORRECT. THE DIGITIZATION PROCESS HERE AT THE NATIONAL LIBRARY OF MEDICINE IS DIFFERENT FROM WHAT YOU MIGHT FIND. THE BIG EXAMPLE IS GOOGLE BOOKS. MOST PEOPLE ARE FAMILIAR WITH GOOGLE BOOKS, IF YOU'VE EVER SEEN ANYTHING ABOUT IT, THEY HAVE THESE VERY ADVANCED MACHINES THAT FLIP THROUGH THE PAGES OF A BOOK IF RAPID SUCCESSION AND TAKE THESE IMAGES OF THE PAGES VERY QUICKLY, AND IF YOU'VE EVER TRIED TO READ A BOOK FROM GOOGLE BOOKS, OR USE THAT FOR RESEARCH, IT'S SLOPPY. I MEAN, IT'S QUICK, IT'S A VERY FAST PROCESS, BUT THE CONTENT THAT YOU GET FROM THERE, YOU MIGHT HAVE AN IMAGE WITH A THUMB IN THE MIDDLE OF IT BECAUSE THE PERSON WHO WAS DIGITIZING WAS HOLDING THE BOOK WHEN THE PICTURE WAS TAKEN. YOU KNOW, THINGS LIKE THAT HAPPEN. AND WE ARE A VERY DIFFERENT PROCESS. OUR APPROACH TO THIS IS THAT THESE -- THE MATERIALS THAT WE'RE DIGITIZING ARE DIGITIZED NOT ONLY FOR ACCESS, THAT'S CERTAINLY IMPORTANT, BUT ALSO FOR DIGITAL PRESERVATION, THESE ARE PRESERVATION QUALITY COPIES, SO YOU'RE NOT GOING TO FIND A THUMB IN THE MIDDLE OF THE PAGE, YOU'RE NOT GOING TO FIND PAGES OUT OF ORDER OR MISSING OR ANYTHING LIKE THAT. OUR QUALITY CONTROL CHECKS ARE STRINGENT. THAT DOES MEAN THE PROCESS TAKES A LOT LONGER. WE WON'T HAVE A BOOK AND PUT IT ON A MACHINE AND AN HOUR LATER IT'S ON THE INTERNET LIKE GOOGLE BOOKS CAN DO, BUT THE PRODUCTS THAT WE'RE PUTTING UP THERE I THINK ARE OF MUCH HIGHER QUALITY AND WILL HAVE MORE VALUE, LONG-TERM VALUE FOR RESEARCHERS. >> I DON'T THINK WE'VE EVER HAD THIS BUT MAYBE AT SOME POINT WE COULD HAVE A PRESENTATION OF THE HISTORY OF MEDICINE AND THE FUTURE. >> SURE. >> A SPECIAL PLACE FOR REBINDING. >> I'VE SEEN THAT. >> IT TAKES WEEKS AND WEEKS. >> OH, YEAH. OUR CONSERVATION, WE HAVE SOME CONSERVATIONISTS AND CONSERVATORS AND TECHNICIANS THAT DO AMAZING WORK. >> THERE WAS A GAL IN THAT DIVISION, IT WAS AN OLD BOOK IN A FAT BINDING, DEGENERATING, THEY HAD TO REDO IT. SHE SAID COULD I TAKE IT HOME? AND SHE DID. SHE PUT IT IN THE BATHTUB. >> THIS IS AN OLD HISTORY OF NLM. >> NOT IN THE LAST 30 YEARS. >> RIGHT. >> INSIDE IT, IN THE TIME IT WAS WRITTEN AND BOUND THEY MADE IT FAT WITH NEWSPAPERS OF THE TIME. IT TURNED OUT THAT THE NEWSPAPERS WERE OF MUCH GREATER HISTORICAL INTEREST THAN THE BOOK. [ LAUGHTER ] >> YEAH, THERE WAS A LADY I THINK IT WAS MORE LIKE IN THE '30s OR '40s, WE HAD SOMETHING THAT WAS CALLED THE BATHTUB COLLECTION. >> RIGHT. >> IT WAS A BUNCH OF STUFF THAT SHE HAD FOUND WHEN THEY WERE REBINDING BOOKS AND IT WAS ALL THE PAPERS THAT HAD BEEN IN THAT WERE STUCK IN THE BINDINGS OF WHATEVER THEY FOUND. IT WAS VERY INTERESTING BECAUSE WE DISCOVERED SOMEBODY DECIDED TO GO THROUGH THIS AND REALLY ANALYZED A FORMER COLLEAGUE OF OURS, AND SHE WAS A MUSICIAN AS WELL AS SOMEBODY WHO WORKED IN THE HISTORY OF MEDICINE DIVISION, AND DISCOVERED SOME OLD MUSIC WHICH SHE THEN -- WE THEN DONATED TO THE LIBRARY OF CONGRESS BECAUSE SOMEBODY HAD JUST TAKEN THE SHEET MUSIC OR SOMETHING FROM SOMEWHERE AND USED THAT AS WHATEVER WAS NECESSARY TO BIND THE BOOK, YOU KNOW, THE PAPER WAS VERY EXPENSIVE IN THOSE DAYS. >> YES. >> AN OBSERVATION, IT'S VERY IMPRESSIVE, BUT OUR EXPERIENCE IS FIND A WAY TO MAKE DATA ACCESSESSIBLE TO ALL OF THE SEARCH ENTRIES, PULL IT OUT, YOU DON'T WANT IT MUCKING AROUND IN YOUR APPLICATION. BUT THE BENEFIT OF CASTING THAT NET IS WELL WORTH THE TIME AND EFFORT. WE BRING SO MANY PEOPLE THAT WAY. WE DO IT WITH JUST PUTTING THE META DATA OUT. AND IT'S VERY IMPRESSIVE BECAUSE WE SPEND A LOT OF TIME IN THE SAME AREA. A TECHNICAL QUESTION, WHAT ARE YOU USING AS YOUR REPOSITORY? >> THE BACK END OF THE REPOSITORY IS FEDORA DATABASE, AND THE INTERFACE THAT YOU SAW IS WHAT USED TO BE CALL VIVISIMO, NOW CALLED DATA EXPLORER. >> OH, OKAY, EXCELLENT. >> CHRIS'S POINT ABOUT THE META DATA IS IMPORTANT, SOMETHING THAT HAPPENED IN THE LAST YEAR THAT THE HISTORY OF MEDICINE DIVISION AND THE CHIEF WAS PROUD, WE ALL WERE, WAS BASICALLY EXPOSING AN XML OF THE ENTIRE CONTENT OF THE INDEX CATALOG WHICH PREVIOUSLY WE HAD IN SOME DATABASE BUT IT WASN'T OUT THERE WHERE ANYBODY COULD FIND IT, AND WE'RE DEFINITELY ON THAT PAGE TRYING TO DO THAT WITH EVERYTHING WE DO HERE. >> TELL THE IMPLICATIONS WITH THAT. >> THE INDEX CATALOG WAS A STRANGE AND WONDERFUL THING, BUT IS CONSIDERED BY MOST HISTORIANS TO BE JUST A MONUMENTAL IMPORTANT THING IN THE HISTORY OF MEDICINE AND HEALTH, TREMENDOUS RESOURCE THAT WAS BUILT OVER THE YEARS, IN THESE BOOK CATALOGS. WHEN WE DIGITIZED ALL OF THIS DATA, THAT WAS A BIG STEP FORWARD. BUT WHEN JEFF CAME HERE, HE WAS LOOKING AROUND AND SAYING, YEAH, BUT WE HAVEN'T GOT THIS META DATA OUT IN XML FORMAT INGESTED BY OTHER PEOPLE, THAT IS USABLE AND CAN BE USED MORE GENERALLY, SO THAT WAS A BIG EFFORT TO GET THAT. >> WHAT DID THEY USE IT FOR? >> IT'S ONLY BEEN OUT, WE ONLY RELEASED IT THIS PAST SUMMER. WE HAVEN'T -- PEOPLE HAVE BEEN USING IT SO FAR FOR THINGS LIKE DATA VISUALIZATION. WHAT WE'RE LOOKING FORWARD TO IS WHEN INDIVIDUALS TAKE THIS INFORMATION AND FIND NEW WAYS TO PRESENT IT FOR OTHER USERS. SO FOR EXAMPLE SOMEBODY WHO CAN TAKE INDEX CAP AND PUT IT INTO A USER INTERFACE THAT HISTORIANS WHO MIGHT NOT HAVE FOUND IT OTHERWISE WILL HAVE ACCESS TO IT. >> SIMPLY HAVING THE IMAGES IS GOING TO BE SO MUCH MORE VALUABLE THAN HAVING"&zATH PROFESSIONAL RESPONSIBILITY BEING SHAPED FOR THE MEDICAL PROFESSION, AND THE BEAUTIFUL THING ABOUT THE INDEX CATALOG IS IT RUNS FOR TWO PRACTICE LIFE TIMES. SO THAT YOU CAN ACTUALLY SEE IN THE PHYSICAL LAYOUT AND THE ORGANIZATION OF TOPICS HOW PROFESSIONAL THINKING ABOUT THE PROFESSION'S RESPONSIBILITIES, THE PROFESSION'S ONLY INDICATIONS CHANGE -- OBLIGATIONS CHANGED IN THE MINDS OF THE ORGANIZATION COMMITTEE, IN SOME AREAS YOU CAN TRACE THAT THEN INTO INDEX MEDICUS OR CURRENT CONTENT AND GET A MORE REFINED UNDERSTANDING. SO HAVING THIS MATERIAL VISUALLY AVAILABLE IS GOING TO BE A HUGE HELP FOR THE EMERGING CONCERNS ABOUT HOW DO PROFESSIONAL RESPONSIBILITIES CHANGE. IT'S ONE THING TO GIVE EVERY MEDICAL STUDENT THE HIPPOCRATIC OATH BUT HOW DID WE COME TO THE DECISION THAT CONFIDENTIALITY IS SUPERSEDED BY REPORTING TUBERCULOSIS OR GUNSHOT WOUNDS IN THE E.R.? AND THESE KINDS OF SHIFTS IN PROFESSIONAL OBLIGATION THINKING ARE THAT PART DEMONSTRATED IN THE PHYSICAL LAYOUT OF THE TOOLS THAT WERE PART OF THE CONTINUING EDUCATION OF THE PROFESSION, A PHENOMENAL BENEFIT TO HAVE THESE THINGS IN A VISUAL FORM THAT PEOPLE CAN USE AND SEE THE INDIVIDUAL PAGES IN THEIR ORGANIZATION. IT'S SUCH A LEAP BEYOND SIMPLY BEING ABLE TO SEARCH. THAT WAS A GREAT BENEFIT. >> NOTHING STAYS THE SAME. >> WE'RE ADJOURNED.