>> GOOD MORNING. JANE'S HUSBAND IS GRAVELY ILL. SHE WAS AWARE OF THAT BUT HE ENCOURAGED HER TO COME YESTERDAY BUT SHE'S GONE HOME THIS MORNING TO BE WITH HIM. AND ALSO ON THIS DAY SINCE I'M FROM NEW YORK CITY, WHERE WE CONTINUE WITH THE 9/11 MEMORIAL, PEOPLE ARE STILL DYING OF HEALTH-RELATED EFFECTS OF 9/11, I'D LIKE US TO THINK ABOUT WHAT HAPPENED 18 YEARS AGO. SO, OUR FIRST ORDER OF BUSINESS IS ABOUT THE BOARD OF COLLECTIONS AND PRESERVATION POLICY, AND YESTERDAY AFTERNOON BEFORE WE LEFT WE WERE ASKED TO TAKE A LOOK AT THAT. AND SO JOYCE IS GOING TO START OFF WITH WHAT HAS BEEN ABOUT A YEAR OF WORK ON BRINGING POLICIES TOGETHER. >> WHAT BUTTON SHOULD I PUSH? PC BUTTON? >> HE'S COMING UP. HELP IS ON ITS WAY. >> IN THE PROCESS OF RENOVATING THIS ROOM THEY WANTED TO REMOVE THE SERVICE AREA. >> I'M HERE TO KICK OFF THE DISCUSSION ON BEHALF OF THE WORKING GROUP WHICH INCLUDES THE MEMBERS YOU SEE AT THE BOTTOM OF THE SCREEN BUT OVER THE LAST YEAR HAS INCLUDED VARIOUS BOARD MEMBERS AND WE THANK ALL OF YOU WHO POPPED INTO OUR WORKING GROUP DISCUSSIONS, CONTRIBUTED IN ANY WAY. THE PURPOSE OF THE BOARD OF REGENTS POLICY ON COLLECTIONS AND THE FIRST ONE WAS IN 1992, IT'S ACTUALLY STILL THE ONE WE'RE OPERATING UNDER. SO, I KNOW YOU HAD A LITTLE HOMEWORK LAST NIGHT, IN ADDITION TO OUR LOVELY GALA WITH FRIENDS AND PERHAPS YOU HAD A CHANCE TO LOOK AT IT AND YOU PROBABLY NOTICED THE LANGUAGE IS JUST A LITTLE ANTIQUATED, DOESN'T ACCOUNT FOR RESEARCH OUTPUTS OTHER THAN LITERATURE. IT'S VERY MUCH ABOUT THE LITERATURE. AND TODAY DECADES AFTER THAT INITIAL POLICY OUR COLLECTIONS INCLUDE MANY TYPES OF INFORMATION, FROM GENBANK TO CLINICAL TRIALS, YOU'LL HEAR FROM THAT WORKING GROUP LATER, TO RESEARCH OUTPUTS AND DATA AND OBJECTS WHETHER VIRTUAL OR PHYSICAL THAT WE NEED TO ACCOUNT FOR SO THIS POLICY IS PURPOSEFULLY BRIEF TO PROVIDE OUR STAFF THE FRAMEWORK UNDER WHICH WE CREATE A GUIDELINES DOCUMENT AVAILABLE TO THE PUBLIC TODAY, OVER 150 PAGES, AND WE CAN DESCRIBE A LITTLE BIT WHAT'S IN THAT AS YOU HAVE QUESTIONS. IN ADDITION TO THE POLICY ON COLLECTIONS, IN 1986 THIS BODY CREATED A PRESERVATION POLICY. ONE OF THE PURPOSES CREATE AN EFFORT TO JOURNALS TO BE PRINTED ON ACID-FREE PAPER AND WE LAUNCHED AN EFFORT SO MEDICAL JOURNALS WOULD BE PRINTED ON ACID-FREE PAPER AND WOULDN'T BE DETERIORATING IN LIBRARY STACKS, WHETHER OURS, NATIONAL LIBRARY OF CONGRESS OR HERE, THAT WAS ESSENTIALLY SUCCESSFUL BUT NOW HAS BEEN MADE OBSOLETE BECAUSE VERY FEW JOURNALS ARE PRINTED ON PAPER, TO MEET OBLIGATIONS OF MEMBERSHIP OR MAYBE TO IMPROVE REVENUES FOR ADVERTISING, BUT THERE ARE MANY, MANY JOURNALS THAT NEVER SEE PAPER, WHETHER ACID-FREE OR NOT. SO THE DRAFT FOR 2019 COMBINES THE POLICIES. SOME OF YOU ARE NEW BUT THE WORKING GROUP HAS BEEN CONSIDERING THIS OVER THE LAST YEAR AND BROUGHT THROUGH THE WORKING GROUP DISCUSSIONS, WE'VE TALKED WITH YOU AT THE FULL BOARD, ABOUT THINGS WE'RE CONSIDERING. I MENTIONED SCHOLARLY COMMUNICATION, ADVENT OF THE INTERNET IS SOMETHING WE NEED TO ACKNOWLEDGE. AND IT ALSO LOOKS FORWARD AT OUR CURRENT STRATEGIC PLAN LOOKING AT DATA AS OBJECTS, EMBRACING DATA AS SOMETHING WE NEED TO INCLUDE IN THE COLLECTION. AND ALSO, JUST THE NEW COLLECTION LANDSCAPE WITH PUBLIC ACCESS AND ALL THE CHANGES. SO, AGAIN, MEANT TO BE VERY HIGH LEVEL. IT ALSO IS INTENDED TO BE GENERAL IN MANY CASES TO PROVIDE THE STAFF FLEXIBILITY. YOU'LL SEE THAT SOME OF THE CLOSING PARAGRAPH MENTIONS THE STAFF WILL OPERATIONALIZE THIS. WE HAVE DONE THAT OVER THE YEARS WITH COLLECTIONS GUIDELINE POLICY AND VARIOUS GROUPS IN THE LIBRARY THAT ARE STAFFED TO BUILD COLLECTIONS, CLINICAL TRIALS OR PubMed CENTRAL OR PubMed, THE ACTUAL PHYSICAL AND VIRTUAL COLLECTION. I DON'T EXPECT IT WILL BE 30 YEARS BEFORE YOU DO THIS AGAIN BUT WE HOPE THIS WILL BE LASTING FOR A WHILE AND WITH THE WORKING GROUP WE'LL BE ABLE TO REVISIT IT ON A REGULAR BASIS AND SEE WHERE THINGS HAVE CHANGED. ONE OF THE KEY CHANGES THAT I THINK WAS REALLY HELPFUL TO US IS TO SAY THAT THE POLICY IS PROSPECTIVE AND WITH CHANGES AND EVOLUTIONS IN THE POLICY IT DOESN'T MEAN NLM IS GOING TO GO BACK AND REVIEW OUR ENTIRE COLLECTION BASED ON THE NEW REALITY. SO, WHEN WE DECIDE THAT WE NEED TO HAVE A NEW FOCUS IN A PARTICULAR AREA, OR DE-EMPHASIZE AN AREA, DOESN'T MEAN WE'RE GOING BACK TO THREE CENTURIES OF COLLECTING AND REVISE THE COLLECTION TO MEET THE NEW GUIDELINE. THE COLLECTION IS BUILD PROSPECTIVELY GOING FORWARD. WITH THAT INTRODUCTION I'LL OPEN IT UP TO COMMITTEE MEMBERS AND WE'RE READY TO -- OUR GOAL TODAY, WE WOULD LOVE TO HAVE YOU APPROVE THIS POLICY AS NEW COLLECTIONS AND PRESERVATION POLICY OF THE BOARD BUT WE'VE ALLOWED TIME FOR YOUR COMMENTS AND WE CAN TALK ABOUT IT LATER THIS MORNING OR WE CAN DO THAT NOW. >> I JUST WANT TO SAY WHY THIS IS SUCH AN IMPORTANT ACT. THE NATIONAL LIBRARY OF MEDICINE IS RESPONSIBLE FOR COLLECTING MEDICAL KNOWLEDGE OF THE COUNTRY AND THE COUNTRY IS THOUGHT OF RATHER QUITE BROADLY, 1956, THINK OF THE ERA AND TIME WE THOUGHT OF, INSIDE OF THE U.S. BOUNDARIES, BUT NOW WE RECOGNIZE THIS IS GLOBAL, MEDICAL KNOWLEDGE OF THE COUNTRY IS MEDICAL KNOWLEDGE OF THE WORLD. WE HAVE A RESPONSIBILITY THAT FRANKLY SUPERSEDES THE RESPONSIBILITY OF THE NIH, THE COLLECTION POLICY FROM THE BOARD IS THE ONE TOOL THAT WE HAVE TO INTERACT WITH NIH, TO EXPLAIN THAT WE HAVE PUBLIC INPUT, RESPECTED DEVELOPMENT OF THE COLLECTION, A SYSTEMATIC WAY OF DOING THIS AND A RIGHT TO SET UP THE COLLECTION. ARE WITH MANY CHANGES EVEN IN THE LAST YEAR, EXPANDING ELSEVIER CONTRACT, IF I CAN GET AN ARTICLE, I HAVE A LIBRARY. AND THAT DISTINCTION IS ONE WE WANT TO ACTUALLY PUT TO REST. WE WANT THE LIBRARY TO CONTINUE TO BE THE REPOSITORY OF INFORMATION THAT IS RELEVANT TODAY AND INTO THE FUTURE. SO YOUR WORK TODAY AND DELIBERATIONS GIVE US FRANKLY THE POWER TO INTERACT WITH NIH TO EXPLAIN WHY CHOICES MUST BE MADE IN AN INDEPENDENT SCHOLARLY MANNER DRIVEN BY LIBRARY PRINCIPLES. AND I HAVE TO SAY THAT WE'VE BEEN VERY SUCCESSFUL WORKING WITH DR. COLLINS AND DR. TABAK HELPING THEM UNDERSTAND THE COMPLEXITY OF THE COLLECTION, HELPING THEM TO UNDERSTAND THAT IT'S WAY MORE EXPENSIVE TO GET RID OF THINGS THAN IT IS TO KEEP THINGS THAT WE MUST PURPOSEFULLY COLLECT SOMETIMES THINGS THAT ARE NOT OF AT THE MOMENT THOUGHT OF AS BEING APPROPRIATE OR RELEVANT AND THEY REMAIN IN OUR COLLECTION. WE STILL HAVE ARTICLES IN PubMed, FOR EXAMPLE, THAT REFER TO VACCINES AND AUTISM, STILL HAVE -- I'M SURE WE HAVE HUNDREDS OF ARTICLES ABOUT STRESS CAUSING ULCERS. SCIENCE IS SELF-CORRECTING, IT'S NOT OUR JOB TO TRACK WITH SCIENCE BUT HAVE THE FOUNDATION FOR SCIENCE TO MOVE. SO I APPRECIATE THE THOUGHTFULNESS AND WANT TO REINFORCE WHAT YOU'RE DOING TODAY REALLY HAS ENORMOUS IMPLICATIONS ACROSS THE NIH AND I APPRECIATE IT. THANK YOU. >> NEIL? >> TO FOLLOW UP ON JOYCE'S COMMENTS, IT'S INTERESTING, ALL OF US DELIBERATED SINCE THE LAST MEETING, IN BETWEEN MEETINGS, 25 OR 30 YEARS AGO IT MEANT YOU TALKED ABOUT WHAT THE LIBRARY SHOULD ACQUIRE FOR PEOPLE TO COME HERE AND USE OR TO USE THROUGH INTERLIBRARY LOAN. EVEN 30 YEARS AGO, PHOTOCOPYING THINGS AND SENDING IT OUT CAUSED ENORMOUS FUROR AMONG PUBLISHERS WHO THOUGHT THAT WAS -- SHOULDN'T BE ALLOWED BUT NOW LOOKING AT THESE ISSUES AS JOYCE MENTIONED IT'S NOT JUST THE TRADITIONAL PRINT OR ONLINE LITERATURE COLLECTION BUT IT'S OTHER OUTPUTS FROM THE RESEARCH PROCESS. WHICH CAN TAKE VARIOUS FORMS. AND MOST NOTABLY, MORE RECENTLY, IT INCLUDES DATA IN VARIOUS FORMS, DATASETS, AND OTHER OUTPUTS, PROTOCOLS, ROUTINES, AND FIGURING OUT WHAT IT IS THAT IS SIGNIFICANT TO COLLECT AS PART OF WHAT GETS FOLDED INTO THIS COLLECTION POLICY AND WITH DBGaP AND GENBANK AND OTHER EXAMPLES, THE COLLECTION BECOMES THIS BROADER AND BROADER ENTITY THAT HAS IMPACT ON THE ENTIRE RANGE OF HEALTH CARE AND BIOMEDICINE. AND AS JOYCE ALSO MENTIONED, THE ADVENT OF SOMETHING LIKE THE INTERNET CHANGES EVERYBODY'S IDEA FROM WHEREVER YOU HAPPEN TO BE SITTING AND WHATEVER YOUR VOLLEY AS TO WHAT A LIBRARY COLLECTION IS. NLM HAS TO BE AWARE OF, YOU KNOW, WEBSITES THAT ARE NOT PART OF WHAT THEY COLLECT BUT IT'S PART OF WHAT THEY HAVE TO BE AWARE OF AND KEEP IN MIND IN TERMS OF THE LIBRARY'S COLLECTION AND WHAT IS SIGNIFICANT IN TERMS OF RESEARCH OUTPUT. SO I THINK THAT GRAPPLING WITH A VARIETY OF THESE -- THE RANGE OF THESE KINDS OF ISSUES THE GROUP CAME UP WITH WHAT WE THINK GIVES NLM, YOU KNOW, SOME GUIDELINES BUT OBVIOUSLY IS NOT PROSCRIPTIVE AND PROVIDES LEEWAY IN TERMS OF HAVING FLEXIBILITY AS ALL THESE CONDITIONS CHANGE IN THE FUTURE. >> I THINK NEIL BASICALLY CAPTURED WHAT WE'RE STRUGGLED WITH, AND IT'S DIFFICULT TRYING TO CREATE A POLICY THAT WOULD BALANCE THE NEED TO PROVIDE SOME GUIDANCE AGAINST BEING TOO PRESCRIPTIVE SO IT BECOMES OUT OF DATE OR NOT RELEVANT ANYMORE, IN A VERY SHORT TIME FRAME. AND IT WAS AN INTERESTING PROCESS IN THAT WE SORT OF DISCUSSED VERY SPECIFIC TERMS AND WORDS, AND TO TRY TO KEEP THIS PHRASE IN A WAY THAT WOULD BE HELPFUL AND NOT SORT OF LIMIT OR RESTRICT DECISIONS THAT NEED TO BE MADE AS A RESULT OF CHANGES OR EVOLUTION OF HOW INFORMATION IS PRESENTED IN THE FUTURE. AND SO WE SPECIFICALLY SAID ANY FORMAT THE MATERIAL IS PRESENTED IN. WE DON'T KNOW 10, 20 YEARS FROM NOW WHAT TECHNOLOGY WILL BRING FORWARD AND WHAT WE'LL BE USING TO PRESENT INFORMATION. AND SO THAT'S WHAT WE TRIED TO DO IN THIS DOCUMENT. >> THE TWO POINTS I WOULD MAKE, PREFACE BY SAYING I'M RATHER NEW TO THE GROUP, THREE WEEKS AGO I BELIEVE. [LAUGHTER] SO WHAT I APPRECIATE ABOUT THE PROPOSED POLICY IS THE CALLING OUT OF ACCESS IN AN EXPLICIT WAY. A LOT OF THESE POLICIES HISTORICALLY HAVE BEEN FOCUSED ON COLLECTION FOR COLLECTION SAKE OR A PURPOSE NOT CONNECTED TO THAT ACCESS POINT, AND I THINK THAT'S IMPORTANT AND RELEVANT TO WHAT IS WITHIN THIS PIECE. TO SOME OF THE EARLIER POINTS THAT OTHERS MADE ABOUT THIS BEING A PROSPECTIVE DOCUMENT THAT ALSO ANTICIPATE FUTURE NEEDS IN THIS AREA, I REALLY APPRECIATED SOME OF THE CONVERSATION WE HAD IN THE WORKING GROUP YESTERDAY WITH DALE IN PARTICULAR ABOUT PRE-PRINTS AND HOW THAT WOULD BE EVALUATED IN THE CONTEXT OF THIS POLICY. SO I THINK BOTH AT ACCESS POINT AND THE FRAMEWORK AND WHAT IS IN THE PROPOSED POLICY NOW, HOW IT HELPS US BE ABLE TO ADDRESS I'D SAY FUTURE BUT PRE-PRINTS ARE HERE NOW, IS VERY HELPFUL. >> QUESTIONS FROM THE -- >> THIS MAY BE OUTSIDE THE SCOPE OF WHAT'S IN THE POLICY BUT THINKING ABOUT ASSOCIATED SUPPLEMENTAL CONTENT FOR PAPERS, PART OF BEING ABLE TO COLLECT AND STORE, HOW DO YOU THINK ABOUT WHAT SHOULD BE WITH THE PAPER AS SUPPLEMENTAL MATERIAL VERSUS POINT TO OTHER MATERIALS, WHETHER DBGaP OR OTHER DATA SENSE, ENSURING CONNECTIVITY BETWEEN OTHER PLACES, CERTAIN DATA IS REQUIRED TO BE STORED AND MAKING SURE THERE'S LINKAGE BETWEEN A RECORD AND dbGAP, LET'S SAY, OR SOME OTHER DATABASE, AND THE PAPER ITSELF. >> SURE. I'M GOING TO ASK THE OVER SEER OF PubMed TO SPEAK. THERE WAS AN ARTICLE IN "NATURE" SUPPLEMENTAL MATERIAL WAS DISAPPEARING. I DON'T THINK WE'RE DONE BUT WE'VE BEEN EXPLORING. >> WHEN I CAME TO THE POSITION FIVE YEARS AGO, FIGURE OUT SUPPLEMENTAL MATERIAL. OH, THAT SOUNDS FUN. IT'S BEEN A CHALLENGE, EVER SINCE BASICALLY WE'RE BACK TO THE INTERNET, MADE IT SO THAT MATERIALS COULD BE PUBLISHED WITH AN ARTICLE REGARDLESS OF PAID RESTRICTION, PEOPLE STARTED SLAPPING THESE UP. AND WE'VE BEEN ACCEPTING SUPPLEMENTAL DATA AND REQUIRING ITS DEPOSIT WITH THE PAPER IF IT'S PART OF THE PUBLICATION RECORD SINCE PMC LAUNCHED IN 2000. BUT ENSURING THAT THOSE FILES ARE ACCESSIBLE LONG TERM IS NOT SOMETHING WE'VE MAINTAINED. IF YOU DIG INTO IT AND DO EVEN THE MOST CASUAL OF SURVEYS OF IT, YOU'LL FIND THAT IT'S NOT EVEN NECESSARILY THE PRESERVATION OF THE FILE TYPE BUT KNOWING WHAT'S IN THE FILE, HAVING ANY METADATA AS TO WHAT IS SUPPLEMENTAL TO THE ARTICLE. SO AS PATTY INDICATED, WE'RE DOING A COUPLE THINGS RIGHT NOW, ACTUALLY THE ASSOCIATES ARE WORKING ON GROUP PROJECT TO SURVEY THE LANDSCAPE AND MAKE -- TO ASSESS WHETHER WE CURRENTLY ON WHAT IS BEING MADE AVAILABLE IN SUPPLEMENTAL MATERIAL WITH ARTICLES. AND PERHAPS HOW WE CAN MAKE THAT MORE MEANINGFUL, AND WHAT MAYBE SHOULDN'T BE SUPPLEMENTAL. SHOULD WE STILL BE MAKING FIGURES SUPPLEMENTABLE IF YOU'RE AN ELECTRONIC-ONLY JOURNAL, IS THAT A PDF ITEM OR SHOULD IT BE PART OF THE ARTICLE? SO WE'RE LOOKING AT FIGURING OUT WHAT'S GOING ON AND WHAT RECOMMENDATIONS WE WOULD LIKE TO MAKE TO THE COMMUNITY. NISO MADE A FEW RECOMMENDATIONS SEVERAL YEARS AGO. >> NISO IS -- >> NATIONAL INFORMATION STANDARDS ORGANIZATION. THE I ALWAYS TRIPS ME UP. IT'S INFORMATION, NOT INSTITUTE. THERE'S THAT PIECE. AND THEN THERE IS TO YOUR POINT ABOUT WHAT'S BEING MADE AVAILABLE IN REPOSITORIES WE HAVE VERY LITTLE CONTROL OVER WHETHER CONTENT IS MADE SUPPLEMENTABLE OR AVAILABLE IN A REPOSITORY. MY STRONG RECOMMENDATION IF ANYONE ASKINGS ME IF THERE'S AN APPROPRIATE REPOSITORY, IT'S ALWAYS BETTER TO HAVE A RECORD WITH REAL METADATA, ACTUAL OBJECTIVE IDENTIFIER, PERMANENT IDENTIFIER, WE CAN BUILD THOSE LINKS AND GIVE PEOPLE SOMETHING EASY TO CITE. WHETHER PEOPLE ARE ALLOWED TO CITE DATA IN CERTAIN JOURNALS IS SORT OF IFFY TO ME. I KNOW SOME JOURNALS DON'T LIKE WHEN -- THEY HAVE A POLICY WHERE YOU SHOULD CITE PEER-REVIEWED CONTENT SO DATA LIVES IN A FUNNY WORLD AS FAR AS CITATION GOES BUT THAT'S LUCKILY TRANSITIONING. BUT HISTORICALLY IT'S BEEN YOU CITE THE PAPER AND HOPEFULLY YOU CAN FIND THE DATA THAT'S ASSOCIATED WITH THE PAPER THAT YOU CITED. SO, THERE'S A LOT TO DEAL WITH HERE. I DON'T KNOW WE'LL HAVE A PERFECT SOLUTION BUT WE'RE HOPING TO GUIDE THE COMMUNITY IN A MORE MEANINGFUL WAY. >> YEAH, I WANT TO ADD THAT'S TOTALLY THE KIND OF MATERIAL RESEARCH OUTPUTS THAT FITS UNDER THIS POLICY. AND THAT'S GREAT EXAMPLE OF HOW THIS POLICY SHOULD ALLOW US TO OPERATIONALIZE WHAT MAKES SENSE WITHIN THE GUIDELINES OF BEING A NATIONAL LIBRARY PROVIDING ACCESS. IT'S A GREAT QUESTION. >> WE HAVE A COMMITMENT TO BOTH SERVICE CUSTODIAN, FEWER AND SMALLER PARTS OF THE COLLECTION WE'LL BE CUSTODIAN TOO, AND A CONNECTOR. THERE'S SOME CONNECTIONS UNDER THIS POLICY WE WILL MAINTAIN IN PERPETUITY, WE ACCEPT THEM PROVIDED THEY HAVE A LONG-TERM STRATEGY TO STAY PRESERVED SO THERE'S A COLLECTION CALLED PORTICO, IT'S ACTUALLY PUBLISHER RESOURCE SO IF THE JOURNAL FOLDS& THERE IS A BACK ARCHIVE OF THAT JOURNAL SO THOSE ARE SOME THINGS WE BUILD INTO THE DECISION ABOUT ACCEPTING A JOURNAL. WE NEVER COMMIT TO KEEPING TRACK OF INVESTIGATOR WEBSITES. SO WHEN AN INVESTIGATOR SAYS AT THE END OF THE ARTICLE COME TO MY WEBSITE, AND HERE IT IS, WE DON'T EVEN FIGURE THAT OUT. SO THERE'S THE ABSOLUTE COMMITMENT, JOURNALS SAY WE'RE GOING TO DEPOSIT IN PORTICO OR SOME OTHER ARCHIVE AND THERE'S THE AUTHOR SAYING HERE'S MY WEBSITE. IN THE MIDDLE, DATA SHOULD BE DEPOSITED IN THE MAIN SPECIFIC OR GENERAL ARCHIVE. AND WE ARE WORKING TO FOSTER PRINCIPLES OF GOOD ARCHIVE, GOOD REPOSITORIES. THAT'S NOT JUST THE NLM DOING THAT. MIKE'S GROUP IS LEADING THAT ACROSS THE NIH. WE HAVE CONCERNS ABOUT COMMERCIAL ARCHIVES, WHAT LONG-TERM STRATEGIES ARE. AT THIS TIME THE GENOME DATA SHARING POLICY FROM THE NIH REQUIRES DEPOSITS IN CERTAIN ARCHIVES FOR THE SPECIFIC GENOMIC DATA. WE HAVE LESS COMMITMENT FOR DATA GENERATED IN THE COURSE OF RESEARCH OR IMAGES WE'RE IN THE PROCESS, NIH IS IN THE PROCESS OF RELEASING A NEW DATA MANAGEMENT AND SHARING POLICY WHICH IS A LITTLE BIT SILENT ABOUT SOME OF THESE ISSUES, TO BE QUITE HONEST. WE'RE TRYING TO MAKE -- POSITION THE LIBRARY TO REASONABLY SERVE AS CUSTODIAN WHAT'S RELEVANT FOR US TO PHYSICALLY HOLD AND MAINTAIN CONNECTIONS TO THINGS THAT ARE TRUSTABLE AND PERSISTENT AND THEN AS KATIE SAID TO WORK WITH THE COMMUNITY TO EMERGE, TO MOVE TOWARDS MORE STANDARDS ABOUT HOW TO ATTACH METADATA, WHAT DOES IT MEAN TO HAVE A UNIQUE IDENTIFIER, ONE COPY OF AN OBJECT, FOR EXAMPLE, THINGS LIKE THAT. AND IT'S KIND OF EXCITING, KIND OF WILD WILD WEST. >> THANKS. >> ANY MORE QUESTIONS OR DISCUSSIONS? WE WANT TO MAKE SURE THAT EVERYONE HAS ALL THE INFORMATION THEY NEED BECAUSE WE'RE GOING TO BE ACTUALLY ASKED TO VOTE ON WHETHER OR NOT WE'RE ACCEPTING THE POLICY. >> JUST THE PRESENTATIONAL COMMENT, LOOKING AT THE 1992, 86, 2019, IT WASN'T EXTREMEL EASY TO SEE WHAT THE ACTUAL CHANGES WERE BETWEEN THESE POLICIES, IT VARIED IN LOTS OF FACTS. >> (INAUDIBLE). >> TODAY, PLEASE FOR TODAY TOO BUT IN GENERAL YOU THINK THAT YOU SHOULD BE AT LEAST SOME WORDS OF TECH SAYING WE CHANGED THE POLICY FROM BEFORE BY MAKING THESE CHANGES. >> LOOKING AT PAGE 42 AND 43 IN YOUR PACKET. >> I'M CURIOUS ABOUT THE LATEST CHANGES EXACTLY. >> TO DO IT IN A TRACK CHANGES WAY BECAUSE A LOT OF WORDS CHANGED. JOYCE HIGHLIGHTED SOME. >> THERE WAS INTENTION BEHIND ALL THIS. NOW IT'S IN WORDS. >> YEAH, LIKE A COUPLE BULLET POINTS. >> YEAH. >> TWO MAJOR CHANGES WE PUT IN PLACE WITH THE NEW VERSION. >> RIGHT. >> I THINK WOULD BE HELPFUL. >> SUZANNE? >> CHIME IN. >> THE POLICY THAT'S BEING REPLACED, THE NEXT TWO PAGES IS WITH A BUNCH OF DEFINITIONS, DEFINES LITERATURE, SCHOLARLY. WE'VE TRIED TO MOVE AWAY FROM THOSE LIMITING DEFINITIONS. YOU'LL NOTICE THERE ARE NO DEFINITIONS OF WORDS IN THIS POLICY. AND THE BIGGEST CHANGE IS TO TRY AND MOVE FROM THAT STATIC DEFINITIONALLY LIMITED POLICY TO A POLICY THAT IS MORE OPEN AS THE DEFINITION OF SCHOLARLY MORPHS AND IT IS MORPHING LIKE CRAZY NOW WITH WHAT, IN MY OLD SOMEWHAT ANTIQUATED VIEW OF THE ACADEMY, IS FLY BY NIGHT PROFIT-MAKING PUBLICATIONS I WOULDN'T EVEN CALL SCIENTIFIC LITERATURE TO THE BANISHING OF THE ABILITY TO KEEP UP WITH -- VANISHING OF THE ABILITY TO KEEP UP WITH LOW LEVEL EVIDENCE, CLINICAL OBSERVATIONS. THE CASE REPORT, THE WISDOM OF A 20-YEAR PRACTITIONER, THOSE ARE NOW IN REVIEW JOURNALS, STATE MEDICAL SOCIETY JOURNALS, AND STUFF THAT CAN LESS AND LESS BE CONTROLLED BY INDEXING. SO THAT WE'RE LIVING IN THIS CHANGING WORLD SO THE BIGGEST THRUST OF THIS IS TO MOVE AWAY FROM LIMITING DEFINITIONS AND TO SECONDARILY ACKNOWLEDGE THAT IN THE LAST 20 YEARS THE NATURE OF THE COLLECTION IS CHANGED, AND WE CANNOT PERCEIVE WHAT IT WILL DO. AND RECOGNIZING IN THE SECOND PAGE, THE FIRST PARAGRAPH STARTS ON THAT PAGE, WHAT WE'VE GOT A POLICY TO ENSURE WE HAVE TO TURN IT OVER TO THE STAFF TO KEEP UP WITH INDUSTRY STANDARDS, BEST PRACTICES, AND ENGAGE IN RESEARCH AND CONSULTATION. WE JUST CAN'T DEFINE WHAT THAT'S GOING TO BE. AND IN 92 WE DEFINED IT, IT WAS THE JOURNALS THAT ARRIVED IN THE MAIL. AND IT'S NOW BECOME SOMETHING THAT'S GOT TO BE JUST MORE OPEN. >> YEAH, I WOULD SAY TO THE QUESTION, THE FIRST THING THE GROUP DID WHEN WE MET A YEAR AGO IN SEPTEMBER WAS TO REMOVE THE WORD "LITERATURE" AND TO EXPAND IT TO NOT JUST LITERATURE BUT RESEARCH. >> WE HAD TO PUT IT BACK IN A FEW PLACES. >> YEAH, WE DID HAVE TO PUT IT BACK, BUT IT WAS VERY, VERY TIGHTLY DEFINED AS LITERATURE, AND THE FIRST PARAGRAPH IS, I WOULDN'T SAY ESSENTIALLY UNCHANGED, BUT SORT OF ADMINSTERIA OF THE AUTHORITY AND BACKGROUND. ONE OF THE THINGS AS I BELIEVE PAUL NOTED WAS -- CONSUMERS WHO UTILIZED SOURCE. BEING ABLE TO UNDERSTAND THAT, BECAUSE YOU HAVE A SUCH MULTI-LAYER LEVEL HERE WITHIN THE LIBRARY, AND SO THAT BECOMES AN ISSUE. I DON'T KNOW IF THE POLICY KIND OF HAS THAT LITTLE COVERAGE OF THAT. >> THE SECOND PARAGRAPH TRIES TO REPRESENT THAT SAYING NLM COLLECTION, WRIT LARGE, RECOMMENDS INTELLECTUAL CONTENT. >> WE'LL BE BRINGING IN POTENTIALLY CONFLICTING CONTENT, AS WE DO. >> AS YOU MENTIONED EARLIER, PATTY, IT'S NOT NLM'S ROLE TO VOUCH FOR THE SCIENCE, BUT TO VOUCH FOR THE RECORD. >> FOR THE RECORD. >> AND IT'S GOING TO VARY. THERE'S GOT TO BE TRUST WITHIN THAT MISSION, NOT TRUST IN THE VERIFIABILITY OF ALL THE DATA. >> AND SO I THINK NOT NECESSARILY WITHIN THE POLICY BUT MAYBE IN SUPPLEMENTARY MATERIAL, ACTUALLY BEING EXPLICIT ABOUT THAT. >> WE BELIEVE IT'S PARTICULARLY IMPORTANT TO BE BROAD, TO RECOGNIZE THAT THERE MAY BE ETHICAL DEBATES ABOUT CERTAIN ASPECTS STEM CELLS BEING ONE WE NEED TO KEEP LITERATURE MOVING SO DEBATES CAN CONTINUE TO HAPPEN. IT'S A NUANCE BECAUSE TRUST MEANS WE TRUST THE RECORD, DOESN'T MEAN WE AFFIRM THE SCIENCE. >> BUT THERE'S ALSO THIS OBLIGATION TO THE PUBLIC. >> YEP. >> THAT HAVE INCREASING ACCESS, AND SO IF IT COMES FROM THE LIBRARY THE MEDICINE, NATIONAL LIBRARY OF MEDICINE, YOU'RE PUTTING THE IMPRIMATUR OF THE INSTITUTION ON THE INFORMATION. >> RIGHT. >> NOT ALL OF IT IS NECESSARILY TRUE, VALID OR SCIENTIFICALLY WORTH ANYTHING IS GOING TO BE I THINK IMPORTANT FOR THE BROADER. >> IT'S A NUANCE. >> WELL, IT'S A NUANCE BUT ALSO SOMETHING WE TEED UP FOR FUTURE DISCUSSION WITHIN THE GROUP, BECAUSE WE MADE THE DISTINCTION BETWEEN FOR EXAMPLE NLM COLLECTION WHICH IS BRAND COMPREHENSIVE BUT I THINK DOES HAVE SOME THRESHOLDS, AT LEAST IN ENSURING GOOD PUBLICATION OR ADHERENCE TO GOOD PUBLICATION PRACTICE WHICH DOESN'T GET US TO THE SCIENCE, DID YOU DO PEER REVIEW, DO YOU HAVE CONFLICT OF INTEREST AND SO FORTH. WHAT WE MAKE ACCESSIBLE THROUGH PubMed IS OF COURSE JUST A FRACTION OF THE COLLECTION, BECAUSE IT'S IN PubMed CENTRAL, NIH FUNDED OR OTHERS, OR HAS GONE THROUGH MEDLINE COMMENT AS MUCH ON THE QUALITY OF THE SCIENCE BUT DOES IMPOSE DIFFERENT QUALITY REVIEWS MORE ON THE SCIENTIFIC EDITORIAL AND SO FORTH POLICY THE JOURNAL. WE'VE HAD DEBATE ABOUT SHOULD WE BE MAKING MORE, HOW DO WE MAKE MORE OF THE COLLECTION AVAILABLE AND ACCESSIBLE? >> WHICH MAY PUT IS IN A POSITION WHERE WE'RE NOT JUST FOCUSING ON THE HIGHEST OF THE CROP BUT TRYING TO BRING MORE OF WHAT NLM HAS ALREADY TAKEN INTO ITS COLLECTION AND MAKE IT MORE READILY ACCEPTABLE, NOT JUST FOR PEOPLE WHO COME INTO A PHYSICAL LOCATION OR COME INTO THE INTERLIBRARY LOAN. I THINK THERE STILL IS THOUGH WITH THE COLLECTION SOME BASIC EXPECTATIONS WE HAVE FOR AT LEAST THE QUALITY OF THE EDITORIAL PROCESS AND FORTH, AND MAYBE THAT'S NOT EXPRESSED IN HERE AS WELL. OF OUR OTHER SERVICES WHEREANY- WE'RE COLLECTING CLINICAL TRIAL INFORMATION, THERE CAN STILL BE CONCERNS ABOUT QUALITY OF TRIAL BUT WE HAVE BASIC THRESHOLDS, WITH NO IRB APPROVAL. >> THOSE THINGS ARE WRITTEN IN -- MY VIEW, PEOPLE AROUND THE ROOM CORRECT ME OR ADD ON. THOSE ARE DONE AT I THINK THE BASIS OF THE INDIVIDUAL SERVICES, IF YOU WILL, SO I THINK THERE ARE CRITERIA MAYBE MORE IN COLLECTION DEVELOPMENT MANUAL THAN IN THE POLICY FOR THE COLLECTION, clinicaltrials.gov, PATTY TOLD ME SHE CAN NEVER FIND IT ON THE WEBSITE BUT HERE ARE THE BASIC THINGS TO GET INTO clinicaltrials.gov, IT'S NOT JUST OPEN TO ANYBODY WHO MIGHT INVENT A CLINICAL TRIAL THEY WANT TO REGISTER. >> WE TRY AND MAKE THAT EXPLICIT IN THE POLICY, IN THE PARAGRAPH ABOVE, THE FIRST PAGE BEGINS COVERAGE. COVERAGE OF THE AUTHORITATIVE SCIENTIFICALLY LITERATURE SHOULD BE COMPREHENSIVE, PEER REVIEWED SCHOLARLY LITERATURE, THAT'S WHAT GOES INTO MEDLINE BUT OTHER HEALTH-RELATED STUFF WE'LL DO THE BEST WE CAN. AND THAT'S WHAT THE LIBRARY HAS ALWAYS DONE, IT'S IN FACT ITS PRINCIPAL SOCIAL MISSION, AND THE BALANCE OF THAT PARAGRAPH GOES THROUGH TRYING TO COLLECT THE STUFF THAT I'M MOST INTERESTED IN WHICH IS THE FRINGE, THE THINGS THAT FALL OUT, AS A HISTORIAN I'M INTERESTED IN FOR A GENERATION NOW WE'VE HAD THIS THING THAT RELATES TO THE VACCINATION MOVEMENT OF THE 19th CENTUR, HOW MUCH IS FUNDAMENTAL TO AMERICAN LIBERTARIANISM IN HEALTH CARE. THERE ARE LOTS OF THINGS THIS LIBRARY HELPS ME DO BUT CAN'T GET IT YOU WILL. EVERY ANTI-VAX IS NOT GOING TO MAKE IT INTO THE COLLECTION. IT MIGHT NOT OUGHT TO BECAUSE WE WOULD NEED ANOTHER BASEMENT. >> PERHAPS THE DIFFERENCE IS MEDLINE PLUS IS TRUSTED AND VETTED INFORMATION THAT IS PUBLISHED. EVERYTHING ELSE TO SOME DEGREE HAS SORT OF THRESHOLD NECESSARY TO GET IN, BUT THEY ARE NOT NECESSARILY TRUSTED AND USED DIRECTLY BY FOLKS WHO ARE NOT PROPERLY TRAINED TO UNDERSTAND THAT LIMITATION SO I THINK THAT WOULD BE POTENTIALLY A WAY OF SEPARATING WHEREAS YOU CAN GET TRUSTED INFORMATION, VETTED AND PUT INTO MEDLINE, TO BE USED WITH THE SENSE OF SCIENCE WHICH IS, WELL, THIS MAY OR MAY NOT BE WHAT'S HAPPENING IN THAT CASE. >> THIS CAME UP IN RESPONSE TO SOME OF DALE'S COMMENTS. HE SAID WE'RE TRYING TO DO THE BEST WE CAN, WHAT WE'VE ALWAYS DONE. THAT IMMEDIATELY SPARKED IN MY MIND THERE'S A METRIC. OBVIOUSLY NOT IN THIS DOCUMENT. IT WOULD BE INCLUDED. BUT WHAT ARE THE METRICS TO SAY THIS IS THE BEST WE DO, AND WE'VE DONE OVER TIME? DOES THAT -- MIGHT BE A NAIVE QUESTION BUT HOW DO YOU KNOW OVER TIME AND MEASURE IN THE FUTURE? >> IS THERE AN INTENT FOR THAT? >> WHAT CARLOS SAID ABOUT MEDLINE PLUS, EACH INDIVIDUAL REPOSITORY, CLINICAL TRIALS, GENBANK, WILL HAVE ADDITIONAL GUIDELINES AND CRITERIA SO WE PUBLISH ALONG WITH THOSE. THIS ISN'T TO DO THE UMBRELLA POLICY, IT NEEDS TO COVER GENBANK AND ARCHIVES FROM THE PRIZE-WINNING SCIENTISTS, WE TAKE THEIR COLLECTIONS OR TAKE THOSE COLLECTIONS AND THEY ARE NOT PEER REVIEWED. THEY HAVE A DIFFERENT -- AND THIS POLICY NEEDS TO COVER ALL OF THAT. >> I THINK IT'S REALLY IMPORTANT TO DISTINGUISH THAT THIS POLICY IS ABOUT THE COLLECTION AND PRESERVATION, WHICH IS THE OVERALL OFFICE OF LIBRARY THAT UNDERLIES THE TOOLS AND DISTINGUISH THAT FROM RETRIEVAL AS YOU HEARD ARE THINGS LIKE ON THE WEB PEOPLE ARE ONLY GOING TO LOOK AT THE FIRST PAGE THAT THEY GET, WHICH MEANS ON THE BASIS OF OUR RETRIEVAL SYSTEM WE'RE PICKING THE 20 PubMed RECORDS THAT WE SEE, BASED ON YOUR QUERY, AND THAT'S -- USED TO BE INVERTED, BECAUSE THEY WERE FOCUSED ON CURRENCY AND THE LATEST ARTICLES, BUT RELEVANCE WEIGHT THE RETRIEVAL SINCE WE PUBLIC, WE ACTUALLY RANK UP THINGS LIKE REVIEW ARTICLES, AND THAT'S BECAUSE WHEN SOMEONE -- THAT IS THE WAY SCIENCE CORRECTS ITSELF. SO IT'S NOT THE LIBRARY IN A SENSE CORRECTING THE COLLECTION, IT'S ACKNOWLEDGING THE CONTEXT OF WHAT WE'RE DELIVERING AND WHO WE'RE DELIVERING IT TO, AND THEN TRYING TO USE THE PROCESS OF EITHER THE SCIENCE IN THE CASE OF THE SEQUENCE DATABASES OR THE WAY, YOU KNOW, LITERATURE HAPPENS, TO TRY TO GET THAT UP FOR A CERTAIN NAIVE QUERY AT THE TOP, PLACE WHERE SOMEBODY DOES SUMMARIZE, AN EXPERT DOES LOOK OVER THE FIELD AND SAY THIS IS WHAT'S PROBABLY TRUE TODAY AND THIS IS WITH A WE DISAGREE WITH. YOU CAN STILL SEE WAYS TO GET THE WHOLE THING BUT I THINK THAT'S AN IMPORTANT DISTINCTION AND WAY INDEPENDENT OF THE POLICY FOR THE TOTAL COLLECTION. >> OKAY. THANK YOU. WE'VE HAD A GOOD CONVERSATION, AND -- KEITH, DID YOU HAVE SOMETHING? >> I WORRY ABOUT YOU EXPLODING OVER THERE. >> I KEEP CHECKING WITH HER. WE HAVE A GOOD DISCUSSION. I DIDN'T HEAR ANYTHING THAT WAS A CHANGE IN THE WORDING OF THE DOCUMENT. AND SO FOR NOW WE'RE ACTUALLY GOING TO MOVE TO A COFFEE BREAK AND WE'LL HAVE BREAKOUT GROUPS, AND AT THE END WE'LL HAVE A VOTE. SO THIS WILL GIVE A LITTLE MORE TIME FOR YOU TO BE THINKING AND IF WE NEED MORE CLARIFICATION WE CAN DO IT AT THAT POINT. FOR NOW WE'LL GO OFF TO 15-MINUTE COFFEE BREAK. THANK YOU. >> AND THANKS TO THE COMMITTEE. THIS IS A LOT OF WORK. >> WE'RE MOVING ON TO THE WORKING GROUP BREAKOUTS, WE'LL START WITH STRATEGIC PLANNING, WHO IS REPORTING ON BEHALF OF STRATEGIC PLANNING? >> I AM THE SOLE SURVIVOR OF THAT. [LAUGHTER] >> WHAT DID YOU DO TO THEM? >> I THINK THEY ALL STEPPED BACK. DIDN'T WANT TO REMAIN. SO, WE HAD A ROBUST DISCUSSION THAT COVERED THE MAJOR ACTIVITIES THAT ARE ALIGNED WITH GOALS ATTACHED TO YOUR STRATEGIC PLAN. SOME-- JUST A FEW THINGS OF NOTE, YOU HAVE SOME NEW INITIATIVES THAT HAVE ALREADY YIELDED AWARDS. FOR INSTANCE THE EXPANDED FUNDING FOR EXTRAMURAL RESEARCH, AT WHICH I LEARNED THAT UNLIKE THE OTHER I.C.s ON THIS CAMPUS AND ELSEWHERE ONLY 15% OF YOUR BUDGET IS IN EXTRAMURAL RESEARCH. SO THAT'S HIGHLY COMMENDABLE AND YOUR SWIFT MOVE IN THAT DIRECTION WAS NOTED BY ALL. THREE THINGS WERE BROUGHT UP FOR YOUR CONSIDERATION. ONE WAS ATTACHED TO THE IDEA BUDGET ARE VOLATILE AND YOU HAVE A LARGE NUMBER OF ACTIVITIES THAT ARE ONGOING RIGHT NOW. A LOT OF THEM ARE IMPORTANT, SOME LARGE, SOME SMALL, BUT IT WASN'T CLEAR HOW THEY WERE PRIORITIZED AND WHICH OF YOUR CHILDREN YOU WOULD THROW OUT IF THE BOAT WERE TO SINK. IT WAS ALSO BROUGHT UP THAT IN ADDITION TO THE CONTINUITY OF OPERATIONS PLAN, SORT OF CURRENT BEST PRACTICE IN RISK MANAGEMENT TO THINK ABOUT IF IN THE UNTHINKABLE SITUATION THAT YOU ALL WERE INDISPOSED WHICH OTHER AGENCY WOULD PICK UP YOUR ESSENTIAL OPERATIONS. >> AGRICULTURE. >> SOUND LIKE NAL. >> IT'S SOMETHING I NEVER THOUGHT OF, MARK. >> YEAH, SO GOING BEYOND JUST, YOU KNOW, YOUR DELEGATION OF AUTHORITY PLANS, IF THE UNTHINKABLE, WHO OUTSIDE, AND THEN THINK IT WAS THE VICE ADMIRAL WHO POINTED OUT THEY HAND OFF TO CDC IN THE SEPARATE GEOGRAPHICAL LOCATION. AND I'LL STOP THERE. >> ALL RIGHT. PEOPLE WHO ARE NEW, I WANT TO REMIND YOU OUR STRATEGIC PLAN IS 2 1/2 YEARS OLD, IT'S THE KEY ELEMENTS, TO ENGAGE WITH USERS IN NEW WAYS AND BUILD THE WORKFORCE OF THE FUTURE. THIS EFFORT WE DID IN PREPARING FOR THIS MEETING HELPED US UNDERSTAND MARKERS MAKING ANY PROGRESS AT ALL IN THE STRATEGIC PLAN. WE FEEL LIKE WE'RE ACTUALLY MAKING VERY GOOD PROGRESS TOWARDS OUR KEY PRIORITIES, PARTICULARLY MY PERSONAL GOAL IN EXPANDING RESEARCH, AND THE AMAZING CONNECTIONS FOR OUTREACH AND ENGAGEMENT THROUGH THE OET, WE'VE MADE SOME REALLY PRETTY SIGNIFICANT STEPS WITH THIS. YOUR QUESTION ABOUT THE FINANCES IS A COMPLICATED AND VERY SERIOUS ONE. WE EXPLAINED TO YOU BEFORE THAT THE RENOVATIONS OF THIS BUILDING WE EXPECT WILL BE A $25 MILLION INVESTMENT, ALL OF WHICH HAS TO COME FROM OUR OPERATING FUNDS. THERE IS NO EXTRA MONEY COMING FOR THAT. AND SO WE'VE -- TODD AND EDDIE, BUDGET OFFICERS, ALL LEADERSHIP HAVE WORKED REALLY HARD TO FIGURE OUT WHO KEEP OPERATIONS GOING AND HAVE FUNDS AVAILABLE ONCE WE MAKE THE COMMITMENT FOR RENOVATION, WE'RE IN, SO WE HAVE A MAKE A MULTI-YEAR COMMITMENT KNOWING WE HAVE AN ANNUAL BUDGET. I ASKED MY MENTORS, GARY GIBBONS FROM NHLBI, WHAT HAPPENS WHEN YOU OVERSPEND YOUR BUDGET. HE SAID, YOU DON'T. WELL, BUT IT MIGHT JUST BE A LITTLE BIT. YOU DON'T. APPARENTLY YOU GO TO JAIL, I GUESS I'VE LEARNED. [LAUGHTER] AND IDENTIFY 85% OF OUR FUNDS ARE PRETTY MUCH COMMITTED. WE DON'T HAVE A LOT OF FLEX. IN OTHER INSTITUTES WHEN A BUDGET CRISIS COMES, MY FIRST YEAR WE WERE ASKED TO PREPARE AN 8% CUT MID-YEAR, EVERY OTHER AGENCY CUT THEIR EXTRAMURAL PROGRAMS A CERTAIN AMOUNT, WE COULD CUT THE ENTIRE PROGRAM AND NOT GET THAT MUCH MONEY. WE JUST DON'T HAVE THAT KIND OF FLEX. WE DID A LOT OF PLANNING THIS YEAR ON WHAT -- IN JUNE WHAT IF WE RAN SHORT OF MONEY IN AUGUST, HOW COULD WE GET FUNDS, AND EACH DIVISION WITH A LITTLE BIT OF ENCOURAGEMENT CAME UP WITH A MILLION HERE, A MILLION THERE WE COULD RECOVER. WE FEEL SAFE ON THAT BASIS. WE'VE STARTED THIS YEAR TO THINK ABOUT MULTI-YEAR BUDGET PLANNING. SO THAT WE AT LEAST KNOW WHAT'S COMING DOWN THE LINE, WHAT DO WE KNOW TO HAPPEN IN A MAJOR INVESTMENT WE'LL BE MAKING IN THE NEXT FEW YEARS TO MIGRATE OUR SERVICES TO CLOUD INSTANCES, IT'S NOT CHEAP. AND WHAT WE'RE BEGINNING TO UNDERSTAND IS ONCE YOU MOVE TO A NEW PLATFORM ALL YOUR LEGACY SYSTEMS HAVE TO BE REBUILT. IT'S NOT A QUESTION OF MAKING THE MOVE, BUT MAKING MICROSOFT AND UPDATE -- MAKING THE MOVE AND UPDATING YOUR PREVIOUS SYSTEMS. WE RECOGNIZE THE NEED TO BE READY FOR NEW KINDS OF -- IMAGE DATA PARTICULARLY, I'M TRIED TO BUILD A STRATEGY FOR MULTI-YEAR PLANNING KNOWING WE'LL HAVE ANNUAL ALLOCATIONS. ACROSS NIH WE'VE MADE SOME PROGRESS WORKING WITH THE BUDGET OFFICE ON HOW TO BEGIN TO THINK ABOUT MULTI-YEAR PLANNING. WHAT'S HAPPENING NOW, IT'S A GOOD AND BAD, SUCCESS OF NCBI, NIH WANTS TO THAT KEEP GROWING AND THINKS IT WILL KEEP GROWING BY CANNIBALIZING THE REST OF THE LIBRARY WHICH IS NOT GOING TO HAPPEN. SO, WE'RE NOW HELPING -- BECAUSE OTHER LARGER INSTITUTES AND WE'RE HALF A BILLION DOLLARS, NCI IS $9 BILLION, SO THE SCOPE IS JUST NOT EVEN COMPARABLE. WE'RE WORKING TO HAVE BETTER UNDERSTANDING, BETTER SUPPORT FROM THE NIH BUDGET PLANNING ABOUT HOW TO ADDRESS SOME OF THESE. THE QUESTION OF PRIORITIES IS REALLY AN INTERESTING ONE TO ME. WE'VE DEVELOPED SOME STRENGTH IN OUR LEADERSHIP TEAM TO BEGIN TO ACTUALLY HAVE GOOD DIALOGUE WITH EACH OTHER. NOW, I CAN'T TAKE MONEY FROM YOU AND GIVE IT TO HER. YET. BUT WE'RE GETTING TO THE POINT OF HAVING TO THINK ABOUT HOW WOULD WE, FOR THE GOOD OF THE ALL, MODIFY THE DIVISION LEVEL BUDGETS. THAT'S A NEW CONVERSATION. AND I'M PRETTY CONFIDENT SAYING I THINK WE'RE MATURING TOWARDS THAT. I DON'T THINK WE'RE THERE YET. WE HAVEN'T HAD THE CRISIS THAT IT NEEDED. BUT IF THE LIGHTNING STRIKES HAD BEEN WORSE THIS YEAR WE NEEDED $10 MILLION INSTEAD OF $3 MILLION TO FIX THE DATASET AND WOULD HAVE HAD THAT CONVERSATION PRETTY FAST. >> NOT TO GET TOO DEEP IN THE WEEDS OF THE DISCUSSION MANY OF YOUR GOALS ARE TO INCREASE, GET MORE, GET LARGER, DO MORE WITH LESS. BUT VERY RARELY WAS THERE A DENOMINATOR SO THAT YOU COULD KNOW WHEN YOU COULD STOP INVESTING IN THAT AREA AND MAYBE DIVERT ATTENTION SOMEWHERE ELSE. >> WE'RE DOING SOME OF THAT WORK, SOME PROMPTED BY REORGANIZATION AND TRANSITION OF THE SIS PROGRAM, INFORMATION SYSTEMS, SYSTEMATICALLY LOOKING THROUGH A NUMBER OF RESOURCES MAINTAINED BY SIS TO FIGURE OUT WHETHER THEY SHOULD BE SUNSET, INTEGRATED INTO OTHER PARTS OF THE NLM. WE'RE SEEING A BIT OF THAT ACTIVITY ACROSS NLM, PROMPT OR INFORMED BY MORE COMPREHENSIVE ASSESSMENT, WE DID A MAY OFFERING BUT TERMINOLOGY YESTERDAY WITH PRODUCT SERVICES, OPPORTUNITIES AND SO FORTH THAT NLM OFFERS INTERNALLY AND EXTERNALLY TO THE PUBLIC. I THINK WE'RE IN A BETTER POSITION TO THINK ABOUT THOSE THINGS NOW. WE HOPE BUDGET WILL BE POSITIVE SO WE DON'T HAVE TO MAKE HARD DECISIONS. WE HAVE BETTER INTERNAL LEADERSHIP STRUCTURE FOR DOING THAT, BETTER INFORMATION. >> COULD ONE OF YOU TALK ABOUT -- (INDISCERNIBLE) -- TALK ABOUT THE MTA AND HOW THAT'S HELPING US LOOK AT OUR RESOURCES. >> YEAH, SURE. >> WE STAND TO LOSE PLANNING EVALUATION. >> WE HAVE DEVELOPED PORTFOLIO ANALYSIS TO HELP ALL OF OUR OFFERINGS, SERVICES AND RESOURCES WE PROVIDE. ALL DIVISIONS, ON A DATA CALL, WE JUST RECEIVED ALL THE INFORMATION WE'RE COMPILING TO ALLOW US TO IDENTIFY AREAS OF OVERLAP AND WHERE WE CAN KIND OF STREAMLINE, LICENSING, THINGS LIKE THIS, AND WILL GIVE A GOOD DASHBOARD ACCOUNT OF WHAT WE'RE DOING WHERE AND WHAT WE'RE SPENDING WHERE ACROSS THE LIBRARY IN DIFFERENT DIVISIONS. >> THANK YOU. THINKING ABOUT MEASURES OF PERFORMANCE THAT ADDRESS VOLUME AND ADDRESS CUSTOMER SERVICE, BUT THEY DON'T REALLY ADDRESS PRIORITIZATION. WHAT YOU'RE ENCOURAGING US TO THINK ABOUT IS ONE STEP ABOVE, SHOULD WE BE DOING THESE THINGS. AND THAT'S A NEW LINE OF REASONING FOR THE LIBRARY. AND I THINK WE'RE MAKING SOME PROGRESS. >> THE THING THAT CAME UP I WOULD ADD, WAS YOUR IDEA PERHAPS PARTNERING WITH SOME LOCAL UNIVERSITIES THAT HAVE ESTABLISHED DEPARTMENT, PERHAPS NIH, WE COULD PARTNER WITH PEOPLE, GEORGE WASHINGTON, JOHNS HOPKINS, GEORGETOWN. >> GEORGE MASON. >> TO TAKE ADVANTAGE OF THEIR TRAINING, CERTIFICATION AND HAVE DATA SCIENCE PROGRAMS INSTEAD OF REINVENTING THE WHEEL AND CREATING OUR OWN WAYS , FIND WAYS TO WORK WITH THE LOCAL USERS. >> WAS THERE ANY DISCUSSION ABOUT THE PLAN TO REFRESH THE STRATEGIC PLAN, LONG RANGE PLAN? >> WE DIDN'T. >> WE DID TALK ABOUT IT A LITTLE BIT, NOT REALLY IN DETAIL. WE TALKED ABOUT HOW WE'LL NEED TO ADDRESS, WHICH INITIATIVE. WE FEEL LIKE WE'VE ALREADY ACHIEVED, AND WHICH WE MIGHT WANT TO SHIFT OUR FOCUS TO. >> SO THIS IS AN ACTIVITY THAT'S GOING TO COME BEFORE THE BOARD DURING A YEAR FROM NOW, IN THE FALL OF 20, START -- STRATEGIC PLAN IS 10-YEAR PLAN, COMING UP TO THE 5-YEAR MARK FOR THIS, AND IT'S TYPICAL AT THE NIH TO DO AN EVALUATION OF WHERE WE ARE, EVALUATION CAN BE DONE BROADLY ACROSS THE ENTIRE PLAN, IT CAN BE DONE REFLECTIVELY AS A LOOK TO SEE WHAT THE ENVIRONMENT IS TELLING US WAS IMPORTANT FIVE YEARS AGO OR STILL IMPORTANT OR IT CAN BE AN OPPORTUNITY FOR NEW INITIATIVES TO BE BROUGHT IN. SO WE WILL BE -- AS YOU RECALL, THE STRATEGIC PLAN IS THE RESPONSIBILITY OF THE BOARD. SO WE WILL BE WORKING WITH YOU THROUGH THIS YEAR TO FIGURE OUT HOW YOU WANT US TO SET UP FOR NEXT YEAR. WHEN WE SET THE STRATEGIC PLAN UP, MIKE HURDA'S GROUP DID THE PRIMARY STAFFING, WE USED EXTRA PANELS AND OUTSIDE JUDGMENT AND MAY WANT TO DO THAT AGAIN OR YOU MIGHT HAVE OTHER GUIDANCE BUT YOU'LL BE HEARING SMALL STEMS. IN THE FALL OF 21 HAVE A REAFFIRMATION OF REORIENTATION OF IT. THANK YOU. ANY OTHER QUESTIONS? IF NOT WE'LL MOVE TO RESEARCH. >> SO MANY OF THE MEMBERS OF OUR COMMITTEE ARE STILL HERE. [LAUGHTER] >> YOU HAD AN EASY JOB. >> WE HAD A LIVELY DISCUSSION, THE FIRST TOPIC OF DISCUSSION WHAT ARE WE HERE TO DO, I THINK PROBABLY THE MOST VALUABLE PART OF THE CONVERSATION. AND WE WENT BACK TO THE BLUE RIBBON PANEL AND WHAT THE BLUE RIBBON PANEL HAD LEFT FOR US AS UNDONE. AND SO THERE WERE THREE RECOMMENDATIONS THAT WE'VE BEEN TALKING SORT OF AROUND THE EDGES ON BUT HADN'T ADDRESSED HEAD-ON. ONE WAS THEY RECOMMENDED THAT YOU ALL DO AN ASSESSMENT OF YOUR CURRENT RESEARCH PORTFOLIO, AND THEN CREATE A STRATEGIC PLAN AROUND THAT RESEARCH PORTFOLIO. THE SECOND WAS THAT YOU CONSIDER AN AUDACIOUS GOAL, WHICH WE HAVE BEEN TALKING ABOUT, ALTHOUGH DOESN'T SEEM TO BE GETTING TRACTION YET. AND THEN THE THIRD RELATED TO BRINGING YOUR TWO RESEARCH GROUPS TOGETHER, LISTER GROUP AND NCBI GROUP, HOW THOSE CAN BE ONE. WE WENT AND TOOK THOSE ON IN OUR DISCUSSION. OUR REALIZATION WAS THE FIRST ONE, STRATEGIC PLAN PIECE, WAS CRITICAL UNDONE PIECE AND WITHOUT ADDRESSING THAT YOU COULDN'T ADDRESS THE OTHER TWO ISSUES. AND WE DIDN'T THINK THAT THE HOUR-AND-A-HALF THAT WE HAVE EVERY THREE MONTHS, FOUR MONTHS, WAS GOING TO BE ADEQUATE TO DO A GOOD JOB NOR WERE WE THE RIGHT GROUP TO ADDRESS THAT. SO WE'RE RECOMMENDING THAT YOU ALL CONSIDER A PROCESS, PARTICULARLY AS YOU'RE RECRUITING A NEW SCIENTIFIC DIRECTOR, NOT NECESSARILY ASSUME THAT NEW SCIENTIFIC DIRECTOR IS GOING TO COME IN WITH A VISION THAT YOU NEED BECAUSE IT COULD JUST BE PROMOTING THE RESEARCH THEY HAVE DONE THEIR WHOLE CAREER. BUT RATHER YOU TAKE THIS OPPORTUNITY TO THAT PERSON COULD BE THE SHEPHERD OF THE NEW VISION, AND HELP TO DEFINE THE PROCESS, BUT COULD AND NEEDS TO THINK MORE BROADLY ABOUT WHAT THE RIGHT FOCUS WOULD BE FOR RESEARCH PROGRAM, IF YOU WERE TO START FROM SCRATCH BUT ALSO ACKNOWLEDGING WHAT YOU HAVE TODAY, HOW THAT COULD EVOLVE. SO ALTHOUGH WE WEREN'T GOING TO DEFINE THAT PROCESS FOR SETTING STRATEGY FOR YOU, WE DID HAVE A LOT OF DISCUSSION ABOUT WHAT YOU MIGHT DO. SO I'LL SHARE THAT. WE THOUGHT THAT IT WOULD BE GOOD TO HAVE -- SOME FOLKS THAT ARE IN THE DISCUSSION YESTERDAY DO REPRESENT ENTITIES THAT SHOULD BE REPRESENTED IN THAT CONVERSATION, SO EXTERNAL ACADEMIC INSTITUTIONS, INDUSTRY, NSF, OTHER NIH, OFFICE OF THE DIRECT, OBVIOUSLY INTERNAL FOLKS. WE DIDN'T THINK IT SHOULD BE JUST INTERNAL BECAUSE, AGAIN, MAYBE NOT HAVE EYES OPEN TO ALL THE DIFFERENT WAYS IN WHICH THINGS COULD BE RECONSIDERED, AND ALSO UNDERSTANDING MAJOR POLITICAL FORCES THAT WANT TO KEEP THINGS THE WAY THEY ARE, AT LEAST FOR THE PEOPLE WHO ARE OVERSEEING CERTAIN PARTS, THAT WOULD NEED TO BE COUNTER BALANCED BY VIEWS FROM THE OUTSIDE. AND THEN WE TALKED A LITTLE BIT ABOUT HOW YOU MIGHT SET THIS UP. BUT, AGAIN, WE'RE NOT SUGGESTING ANYTHING. NSF DOES SOME INTERESTING SAND PIT, I DON'T KNOW WHY ANYBODY WOULD WANT TO JUMP IN A SAND PIT, SANDBX MAYBE WOULD BE THE U.S. EQUIVALENT TYPE APPROACH. ANYWAY, THE MULTI-DAY APPROACH, WHAT'S THE PROBLEM, WHAT ARE WE TRYING TO ADDRESS, GET FOLKS IN THERE AND ADDRESSING IT. WE THINK THAT KIND OF ATTENTION IS GOING TO BE REQUIRED TO DO THIS. WE DID TOO TALK ABOUT WHAT MIGHT THIS LOOK LIKE, AND IS IT WORTH IT, WHAT COULD COME OUT. THAT WAS A REALLY INTERESTING CONVERSATION. WE STARTED WITH THE FACT YOU ALL PRODUCE TOOLS THAT ARE REALLY IMPORTANT, AND MAYBE AREN'T GETTING ADEQUATE ATTENTION IN TERMS OF ABILITY TO INNOVATE ON THE TOOLS. AND SO ONE EXAMPLE, ONE EXAMPLE TO TALK ABOUT IN TERMS OF CLINVARE, WANTING TO ADVANCE THAT FURTHER. PubMed STILL IS ORDERING BASED ON TIME AND HOW LONG HAS PubMed BEEN UP, WHY DID IT TAKE SO LONG TO EXPERIMENT WITH ANOTHER APPROACH, SO REALLY SO THAT KIND OF THING, HOW DO WE STIMULATE INNOVATION ALONG THE TOOLS. ALSO UNIQUE POSITION THAT'S BEING BETWEEN ALL THE NIH INSTITUTES IN THE CENTER POSITION OF META SCIENCE, SO THINKING ABOUT ALL THE SPECIAL POSITION IN TERMS OF DATA COMING TOGETHER, CREATING THE PLATFORMS, THINGS THAT POTENTIALLY COULD BE DONE UNIQUELY HERE NOT JUST IN THE PRAGMATIC TOOL DEVELOPMENT STAGE BUT ALSO IN THE IMPLIED NEEDS FOR BASIC SCIENCE IN THOSE AREAS. SO HOW SHOULD DATA BE ORGANIZED, HOW DO YOU STORE IT, SENT OUT, BAYESIAN QUESTIONS, ALL THOSE THINGS STILL FIT WITH A FOCUS ON THE PLATFORM FUNCTIONS THAT EXIST HERE. WHAT DOESN'T FITNESSLY THOUGH ARE THINGS THAT -- WHAT DOESN'T NECESSARILY FIT, THINGS DONE ELSEWHERE, THINKING ABOUT HOW YOU CAN RATIONALIZE YOUR INVESTMENT MORE CLEARLY THAN OTHER INSTITUTES, MAKING THE ARGUMENT THEY CAN DO IT BETTER IS HARDER, YOU HAVE THE UNIQUE POSITION IN TERMS OF BEING AGGREGATOR, ALL THIS DISCOURSE, SO THAT WAS AN EXAMPLE OF HOW WE THOUGHT THIS ACTUALLY COULD BE AN ENABLING OPPORTUNITY FOR YOU ALL. AND THEN WE HAD SOME DISCUSSION WHICH WE LEFT UNRESOLVED ABOUT, WELL, OF COURSE TO DRAW PEOPLE HERE YOU'RE GOING TO WANT TO GIVE THEM SOME FREEDOM TO DEFINE THEIR RESEARCH AGENDA, HOW DO YOU DALE -- BALANCE WITH THAT A STRATEGY IN RESEARCH, COMBINATION PORTFOLIO. IN TERMS OF GRAND CHALLENGES UNLESS YOU HAVE A RESEARCH STRATEGY, GRAND CHALLENGES. WE'VE GIVEN YOU IDEAS UP TO NOW, THEY ARE NOT GOING ANYWHERE. THAT'S WHY THEY ARE NOT GOING ANYWHERE BECAUSE YOU ALL DON'T -- CAN'T SAY, WELL, THAT GRAND CHALLENGE IS MORE IMPORTANT, YOU CAN'T DO THAT WITHOUT RESEARCH STRATEGY SO THAT WAS OUR RECOMMENDATION THERE. AND REAL QUICKLY IN TERMS OF BRINGING -- WE HAD A BRIEF DISCUSSION AT THE BEGINNING OF THE MEETING ABOUT THIS. THE LISTER AND NCBI NOTIONS. WE THINK THAT IF THEY WERE WORKING ON DIFFERENT FLOORS, WORKING IN DIFFERENT SPACES, THAT'S NOT GOING TO WORK WELL. THEY DON'T KNOW WHAT EACH OTHER IS DOING, THAT'S NOT GOING TO WORK WELL, TWO EASY THINGS YOU CAN DO TODAY TO START TO BREAK DOWN THE BOUNDARIES OF THOSE GROUPS. >> (INAUDIBLE). >> MOVING PEOPLE AROUND. >> (INAUDIBLE). >> WE ARE ABSOLUTELY COMMITTED TO UNIFYING PROGRAMS WITHOUT COLLAPSING OR REDUCING THEIR COMPOSITION AND THEIR CONTRIBUTIONS. SO IF YOU HAVE THOUGHTS ABOUT HOW TO DO THAT, THE ONE, THE ONLY ONE THAT'S GOTTEN TRACTION IS THE COFFEE BAR. WE PUT A BETTER COFFEE BAR IN LISTER HILL WOULD PROBABLY GET MORE ENGAGEMENT. SERIOUSLY, DEEP AND STRONG IDENTITIES TO RESEARCH OPERATIONS THAT -- AND YET WE KNOW TO GROW IS GOING TO REQUIRE CROSS GROUPS. WE KNOW INVESTMENT IN COMPUTATION, DATA, WE HAVE TO BRING THIS TOGETHER. SO WE'RE REALLY OPEN FOR IDEAS. >> I LIKE THE NOTION OF BASICALLY SENDING PACKING BOXES SO PEOPLE CAN MOVE. WE'VE BEEN THINKING ABOUT RENOVATING, WE HAVE PLANS UNDERWAY TO RENOVATE ONE OF THE FLOORS OF THE LISTER HILL CENTER NOW. THAT'S WITHIN BUDGET. AS THE PLACE WHERE ESSENTIALLY NEW INVESTIGATORS WOULD GO, AND THAT WOULD BECOME A MUTUAL FLOOR, WHERE THE NLM AND RESEARCH PROGRAMS, AS OPPOSED TO LISTER HILL OR NCBI FLOORS, FUTURE ORIENTED, NEW INVESTIGATORS AS WE BRING MORE IN BUT WE HAVEN'T THOUGHT ABOUT TRYING TO SHUFFLE PEOPLE WHO ARE ALREADY THERE. WE'VE BEEN TRYING TO THINK MORE ABOUT CAN WE OPEN OUR SYMPOSIA SO WE DON'T HAVE NCBI RESEARCH SYMPOSIA AND LISTER HILL HILL SYMPOSIA. IT WILL BE AN INTRAMURAL RESEARCH TRAINING PROGRAM, JIM KNOWS, DAVID HAS PROMISED THAT FOR A WHILE SO WE'RE TRYING TO USE SOME OF THOSE OPPORTUNITIES AS WAYS TO BUILD THE BRIDGES BETWEEN PROGRAMS AS WELL. >> LIKE I SAID, TO ADD TO A COUPLE THINGS, WE CAN TALK ABOUT THIS, MORE ABOUT THAT DISCUSSION DIDN'T COVER EVERYTHING BUT A GREAT SUMMARY. THIS IDEA OF COMPOSITION, PORTFOLIO OF INTRAMURAL RESEARCH AND HOW LITTLE, WHAT COULD BE VERY UNIQUE TO NLM CAME UP. THE IDEA THERE'S SO MUCH TO BE DONE, RETRIEVAL OF RESULTS IN LITERATURE, ACCESS MODELS OF THE USER, DIFFERENT USER TYPES, GOING AFTER THE DATA. IT'S CORE SCIENCE. IT'S RESEARCH. IT SHOULD BE PART OF IT. SIGNIFICANT FILLER I THINK BUT THE GROUP THOUGHT OF FUTURE PROGRAM, FUTURE VISION FOR SOME OF THAT BUT IT'S BEEN I THINK MY OWN SENSE HAS BEEN THE LAST FEW YEARS MORE FOCUSED ON SPECIFIC TOOLS, DEVELOPMENT VERSUS CORE SCIENCE, DEEPER SCIENCE BEHIND INFERENCE, REASONING, THAT DIDN'T LIGHT UP FUTURE VERSIONS, PubMed 15 YEARS FROM NOW, OR WHATEVER COMES FROM THAT, AMONG OTHER KINDS OF TOOLS INVOLVING DATA, OTHER KINDS OF RESOURCES. THIS IS CORE RESEARCH THAT MICROSOFT RESEARCH, GOOGLE, AMAZON, UNIVERSITIES ARE NOT PROPERLY POSITIONED TO DO. AND THIS OTHER NOTION CAME UP, HOW DO YOU ATTRACT TOP TALENT TO COME PROBABLY LOWER PAY SCALE THAN INDUSTRY, FOR EXAMPLE, DOING THAT KIND OF RESEARCH FOR THE NLM INTRAMURAL PROGRAM, AND PART OF THE ATTRACTION PEOPLE WANT TO DO THE BEST RESEARCH AND HAVE THE MOST IMPACT. THAT'S ONE OF THESE AREAS WHERE THE UNIQUE PROPOSITION OF THE DATA RESOURCES ASSETS AND STRATEGIC PLAN OF THIS ORGANIZATION CAN LIGHT UP KIND OF A CAREER, CAREERS. >> I HAVE A DEEP INTEREST IN THIS CONVERSATION. [LAUGHTER] IT'S COMPLICATED AND DIFFICULT. I KNOW PEOPLE ARE AWARE LISTER HILL IS THIS TEENY IN TERMS OF RESEARCHERS VERSUS NCBI. I RESPECT THEM VERY HIGHLY. THEY ARE VERY DIFFERENT SUBJECT MATTER AND FOCI, AND SO IN LIFE WHAT HAPPENS WHEN YOU TAKE A SMALL GROUP AND A BIG GROUP, THE SMALL GROUP GOES AWAY. COME UP WITH A DEMOCRATIC PROCESS, BE SENSITIVE TO THAT. THE SECOND THING, NIH HAS NOT BEEN EXCITED ABOUT GENERAL INFORMATICS, MY EYES, FOR THE LAST TEN YEARS BUT SUDDENLY HAVE GOTTEN RELIGION IN THE PAST YEAR AND ARE FIERCELY INTERESTED IN FHIR AND OTHER THINGS. AT THIS JUNCTURE, IT WOULD BE NICE TO FAN THE INFLAMMATION OF INTEREST WITH RESEARCHERS WORKING IN THOSE SPACES. I APPLAUD YOUR POSITIONS TOO. IT'S HARD TO ARTICULATE THOSE AND GET PEOPLE WHO CAN AND WANT TO DO IT. THAT'S JUST MY STATEMENT. >> YOU MENTIONED TRYING TO FOCUS ON SOME OF THE MORE FUNDAMENTAL AREAS, SEARCH RETRIEVAL, MICROSOFT IS DOING, GOOGLE. >> BUT NOT FOR MEDICAL. >> THAT'S WHAT I WANTED TO HIGHLIGHT. WHERE IS THE UNIQUE NICHE FOR NLM, APPLYING THOSE TO MEDICAL -- >> IT'S BOTH. >> APPLYING TO RESOURCES?& >> IT'S ONE OF THESE. IT'S HARD TO IMAGINE BIOMEDICINE BEING VERTICAL. IT'S SO BIG. THE MISSION OF THE WORLD, IT'S A VERTICAL. THERE ARE SUBVERTICALS THAT ARE REALLY IMPORTANT. YOU CAN IMAGINE PIECES OF WORK THAT WOULDN'T HAPPEN IN OTHER AREAS, TOO WASHED OUT BY OTHER KINDS OF ATTEMPTS TO MAKE A GIGANTIC POLICY FOR THE WORLD, WHERE YOU CAN REALLY SUPER-CHARGE FOR UNDERSTANDING USER NEEDS, YOU KNOW, SCIENTISTS, CONSUMERS, HEALTH CARE PRACTITIONERS, UNDERSTAND IN A WAY THAT MIGHT BE GENERALIZED BUT FOR NOW SCIENCE BEING THE ACTUAL CUSTOM TAILOR OF THOSE TOOLS TO BIOMEDICINE. AND THINKING ABOUT, MY POINT OF VIEW, WHAT YOU CAN DO, MAKE LEAPS AND BOUNDS FORWARD, NOT TOO SMALL OR TOO BIG, IN TERMS OF APPLICATION AND NATURE OF THE LITERATURE. >> NO ONE IS TAKING CARE OF THE SCIENTISTS AND CLINICIANS ARE NOT BEING REALLY TAKEN CARE OF EITHER. AND SO ONE EXAMPLE IS WHO ELSE IS USING PubMed TOOLS NOW TO CREATE A BETTER SEARCH ENGINE. ALL YOU HAVE IS GOOGLE SCHOLAR, AND IT DOESN'T WORK. IT PULLS OLDEST REFERENCE TO THE TOP, DOES THE REVERSE, THEY ARE CITED MORE BUT THAT'S ABOUT IT. NO ONE IS SOLVING THAT PROBLEM. IT'S MORE THE CUSTOMERS. THEN THERE'S UTILITY. IN TERMS OF CONSUMER IT'S NOT CLEAR. BUSINESS CASES FOR ADDRESSING CONSUMER NEEDS BUT IF YOU LOOK AT IT, THE WAY BUSINESSES ARE DOING IT, TO DELIVER MORE DOLLARS TO THEM. SO GOOGLE HAS PROMOTED ANTI-VACCINE MESSAGES BECAUSE THEY DRAW IN PEOPLE, READERS, THAT THEN DRAW MORE EYES FOR ADS. NLM WOULD NEVER CREATE SCHEMES, YEAH, BEING FOCUSED ON THE SCIENCE ACTUALLY ADDS SOMETHING THERE IF THAT SCIENCE HAD BEEN DEVELOPED SOLVING THE SCIENCE AND CLINICIANS' NEEDS FOR DATA THEN WE MIGHT NOW HAVE BETTER TOOLS FOR GOOGLE TO NOW CLEAN UP ITS ACT. >> THERE'S A HUGE CHUNK OF GOOGLE TRAFFIC IN HEALTH CARE, RIGHT? AND TO BE HONEST, NOT FOR OUR MINUTES, BUT RESULTS TALK, IN MANY WAYS. AND THAT'S WHAT PEOPLE ARE SEEING IN THE WORLD. THAT'S IN THE PubMed, COCHRANE, MET A ANALYSES, SEEING TERRIBLE, WRONG INCENTIVE MODEL FOR BECOMING RANKED HIGHLY. IT'S AN INTERESTING COMMENT ON POSSIBILITIES. I THINK THERE IS THE EXAMPLE, THE ADVISORY BOARD, THEY STARTED THIS NEW ARTIFACT TOOL CALLED SEMANTIC SCHOLAR, YOU MAY HAVE SEEN IT, TRYING TO GO DEEP WITH THE TEAM. TEN OR ELEVEN ON THIS, SCIENTISTS DOING CORE SCIENCE, STARTED WITH COMPUTER SCIENCE, MOVED TO NEUROBIOLOGY AND NOW TRYING TO GENERALIZE. BUT IT'S THE KIND OF WORK THAT'S BETTER DONE AND MORE SERIOUSLY NLM INTRAMURAL RESEARCH. >> OKAY. THANK YOU. TIME TO MOVE ON, PUBLIC SERVICES. >> I WANT VALERIE AND JIM TO WEIGH IN BECAUSE THIS IS GUIDANCE WE'RE GOING TO NEED TO TAKE FORWARD, DIRECTED TOWARDS THE INTRAMURAL PROGRAM MOSTLY, SO I APPRECIATE YOUR REACTION TO IT. >> OKAY. SO IN THE DISCUSSION -- SO WHAT WE TALKED ABOUT, THERE WAS A PHRASE THAT ERIC USED, I CAN'T REMEMBER, MUCH LIKE WHAT WE THINK OF OUR GRANT PROGRAMS ARE, A PROOF-OF-CONCEPT. >> OH, YEAH. >> IN THE SPECIFIC PRIMARY, WHAT DID YOU CALL IT? I ACTUALLY WROTE IT DOWN, BECAUSE WE WERE -- TO RESPOND TO WHAT CLEM SAID, WE WERE TALKING ABOUT ABSORPTION, WE WERE TALKING ABOUT THE FOCUS OF RESEARCH. REALLY THAT'S WHERE WE SPENT OUR TIME. THE LIBRARY HAS A MISSION, THEN FLOWING FROM THAT, INTRAMURAL RESEARCH MISSION WOULD MAKE SENSE FOR THAT. AND SO THE FIRST PARTY INNOVATOR IS THE PHRASE. >> OH, YEAH. >> WE WERE SAYING -- WE'RE HAVING THE SORT OF ARGUMENT IS IT ALL ABOUT TOOLS? WHAT ABOUT DISCOVERY? RIGHT? THERE IS DISCOVERY. THERE'S PLENTY OF ROOM IN INFORMATION SCIENCES, WHATEVER YOU WANT TO CALL THIS FIELD, FOR DISCOVERY THAT CAN THEN BE APPLIED AND USED AND TAILORED AND USED FOR CUSTOMIZATION AND ALL. >> BACK TO MICROSOFT-SPEAK, WE OFTEN BUILD SOMETHING THAT WE CONSIDER LIKE A PLATFORM, WE WANT TO SHOW THE WORLD FIRST PARTY APP WE CALL IT THAT LIGHTS IT UP. BUT THEN INVITE THE WORLD TO BE CREATIVE AND PARTY ON IT BECAUSE THE WORLD TAKES IT SO MUCH FURTHER. BUT IT TAKES THE GENERAL THINKING IN THE PLATFORM AND DEMONSTRATION APP-LIKE PROCESS, LIKE EXCEL, USE THIS CLOUD OR -- AND THEN YOU THROW BOTH OUT THERE BUT THE IDEA IS WE CAN'T -- WE'RE A TINY PATCH OF THE WORLD, LET THE WORLD PARTY, IF WE HAVE THE BEST PLATFORM AND IDEAS, WE CAN GET LOTS OF CREATIVITY ON THE PLATFORM. >> JIM? >> SO I HAVE LOTS OF COMMENTS ABOUT THIS. SO WHEN WE SET UP THE ORGANIZATION OF NCBI IN THE FIRST PLACE, WE PURPOSELY SET IT UP WITH A RESEARCH BRANCH AND PRODUCTION BRANCH, NOT SO THAT THEY WOULD BE SEPARATE, SO THEY WOULD BE IN THE SAME ORGANIZATION BUT SORT OF WITH DIFFERENT INCENTIVE STRUCTURE. SO WE REALLY THOUGHT OF IT MORE AS POLES OF A MAGNET THAN JOINT THINGS, AND THE RESEARCHERS HAD TO HAVE THE FREEDOM TO EXPERIMENT WITH THEIR OWN IDEAS, BUT THEY WERE SELECTED BY SUBJECT AREAS THAT WERE SUPPOSED TO BE IN THE SCOPE OF NCBI, 20 YEARS AGO, A LOT WERE HIRED. AND THEN OF COURSE WE GOT NOTHING TO INCREASE THAT DOWN TO PEOPLE BUT OUR MISSION CHANGED DRAMATICALLY AND LOTS OF MORPHS. OUT OF THAT LOTS OF THINGS CAME, LIKE BLAST AND SOME FIRST VERSIONS OF PubMed AND THINGS THAT WERE BASED ON FUNDAMENTAL RESEARCH THAT WAS EXPLICITLY THEN CHANNELED INTO PRODUCTION. WE DID AN EXPERIMENT OF TRYING TO LEVERAGE THAT AND HAVE OUTSIDE GROUP INDUSTRY AS WELL AS ACADEMIA BUILD ON THOSE PLATFORMS AND NOTABLY GLASS, WHICH WE EXTERNALIZED AND MADE AVAILABLE. AND WHAT WE FOUND WAS A LOT OF WHAT CAME BACK THROUGH THAT, WHILE THEY MIGHT BE GOOD IDEAS, WERE SO FAR FROM PRODUCTION SOFTWARE THAT WE'VE ESSENTIALLY HAD TO RECODE THEM. AND SO THERE'S A HUGE INVESTMENT ON OUR PART TO MOVE IT, YOU KNOW, INTO SOMETHING MORE FUNCTIONAL, AND THEN OF COURSE OVER THE YEARS THINGS PARTED COMPANY IN SOME WAYS. RECENTLY WE'VE TRIED TO SORT OF RESURRECT SOME OF THAT WITH THE LATEST VERSION OF PubMed, WHERE WE EXPLICITLY COUPLED THE TEXT REVEAL RESEARCH GROUP THROUGH JOHN LOU'S GROUP AND HAD THEM WORK WITH A PRODUCTION TEAM SORT OF FROM THE BEGINNING AND TRYING TO USE OBJECTIVE MEASURES THAT MATTERED TO US, LIKE TESTING AGAINST PubMed USERS, WHICH SORT OF IMPROVED PERFORMANCE AND THAT SORT OF THING. AND THAT WORKED. AND ACTUALLY I THINK THERE'S A BETTER RETRIEVAL SYSTEM THAT CAME OUT OF IT. WE'RE STILL PRODUCT TYPING IT. THERE'S NOW ANOTHER ROUND THROUGH NLM LABS WHICH IS A BUNCH OF THE SAME GROUPS COLLABORATING WITH LISTER HILL CENTER EXPERIMENTING IN AUTOMATED TERM EXTRACTION, CONCEPT EXTRACTION, AGAIN WITH THE IDEA THAT WE'RE TRYING TO SET UP AN ENVIRONMENT WHERE WE'RE NOT -- WE'RE SORT OF COMMITTING A LITTLE MORE TO THIS RESEARCH PROJECT, BUT WE'RE NOT COMMITTING TO PUT IT INTO PRODUCTION UNTIL WE CAN PROVE THE VALUE OF IT IN A SOMEWHAT OBJECTIVE WAY. I THINK THAT IS THE RIGHT WAY TO GO. I CAN'T BELIEVE A LOT OF THE PEOPLE WORKING IN THIS FIELD WILL WORK FOR NLM SALARIES. YOU KNOW, WE'RE LOSING PEOPLE TO GOOGLE AND AMAZON ALL THE TIME IN SOFTWARE DEVELOPMENT AREAS, AND CERTAINLY THE DEMAND FOR PEOPLE WITH RETRIEVAL SYSTEM AND EXTRACTIONS AND THINGS LIKE THAT IS INCREDIBLY HIGH. SO I DON'T KNOW HOW TO DO THIS, BUT WE NEED SOME SORT OF MORE FLEXIBLE THING WHERE, YOU KNOW, YOU SORT OF CAN ADJUST THE STRUCTURE OF THE INTRAMURAL PROGRAM WITHIN THE BUDGET AS THE DEMANDS CHANGE, SO IT'S NOT A 30-YEAR CAREER OR 40-YEAR CAREER BUT THAT MAKES IT LESS ATTRACTIVE FOR PEOPLE. AND WE ALSO NEED SOME SORT OF LABORATORY IDEAS THAT DRAWS A BIGGER FIELD BUT COUPLED TO THAT HAS TO BE THE INVESTMENT AND PRODUCTIZING THOSE THINGS WHICH WAS PROMISING AND ACTUALLY A NUMBER OF THOSE EVEN IN THE COURSE OF PRODUCTIZING YOU REALIZE THEY ARE NOT A THOUGHT BECAUSE THEY HAVEN'T BEEN TESTED ON A MILLION QUERIES, THEY HAVE BEEN TESTED ON SOME SETS AND IT LOOKED GOOD. AND SO THERE'S AN INTERIM INDUSTRY CAN SEE THAT AS YOU GO THROUGH DEVELOPMENT PROCESS, ISN'T SOMETHING OUT OF THE RESEARCH GROUP THAT IMMEDIATELY GOES TO PRODUCTION AS BIG INVESTMENT. MORE WINNOWING ALONG THE WAY AS YOU GO. SO HOW YOU DO THAT WITHIN A FEDERAL BUDGET -- >> I WANT TO ADD ONE MORE POINT. IN TERMS OF PEOPLE WE ALREADY HAVE, RIGHT? WE WERE TALKING ABOUT THE MODEL WE HAVE IN OUR TRAINING PROGRAMS, BRINGING TOGETHER PEOPLE IN SEMINARS AND PRESENTATIONS. ORIGINALLY TRAINEES WERE IN DIFFERENT AREAS AND DIDN'T WANT TO LISTEN TO EACH OTHER, I'M NOT INTERESTED IN THAT KIND OF SCIENCE BUT THEY LEARNED FROM EACH OTHER AND IT HAS CHANGED THE WHOLE DISCUSSION THAT GOES ON IN THE MEETINGS. SO WE IMAGINED THAT THERE WERE MECHANISMS LIKE THAT, THAT COULD ACTUALLY -- THAT MAY BE IN PLACE AND WE'RE NOT AWARE OF BUT COULD FOSTER MORE CONVERSATION AMONG THE PEOPLE ALREADY HERE. >> THANK YOU. NO DOUBT WE'LL BE TALKING ABOUT THAT IN FUTURE. REALLY APPRECIATE THAT. WE'RE GOING TO MOVE ON TO PUBLIC SERVICE NOW, WHO IS GOING TO REPORT OUT ON THAT? >> PLEASE JUMP IN IF I LEAVE ANYTHING OUT. BOTTOM LINE FOR PUBLIC SERVICE, WE WOULD LIKE TO GET BOARD'S APPROVAL FOR THE clinicaltrials.gov WORKING GROUP. SO AS YOU KNOW, THE clinicaltrials.gov IS THE 2007, 2005, THE JOURNAL EDITORS DECIDED TO REQUIRE EVERYONE TO REGISTER THEIR CLINICAL TRIAL PRIOR TO CONDUCTING IT AND WOULDN'T PUBLISH YOU UNLESS YOU DID IT. AND THEN THROUGH clinicaltrials.gov, PARTIALLY RESPONSE TO THAT, TO HAVE A PLACE TO REGISTER YOUR TRIAL. OVER PAST 12 YEARS MORE USERS ARE USING IT, AND IT'S TIME TO LOOK AT THE NEXT -- LOOK AT MODERNIZATION OF clinicaltrials.gov. APPARENTLY SOME OF IT IS RUNNING UNDER WHAT, QUOTE, CHICKEN WIRE, UNQUOTE, UNDERNEATH THE SERVICE. >> I'M AFRAID THAT WAS MINE, IMPLY ROBUST -- [LAUGHTER] (INDISCERNIBLE). >> SOMETHING THAT WORKS WHEN THERE WERE ONLY A COUPLE CHICK ENS ON THE FARM, NOW THERE'S 300 CHICKENS, SOMETHING MORE ROBUST THAN CHICKEN WIRE. THE QUESTION WAS HOW DO WE DO THIS TO COME UP WITH A WAY THAT THE NEW clinicaltrials.gov WILL SERVE NEEDS OF OUR CONSTITUENTS. THEY HAVE PROPOSED A MODERNIZATION OF clinicaltrials.gov WORKING GROUP. AND THE GOALS OF THE WORKING GROUP ARE PROPOSED TO BE FOUR TOPICS, BUT NOT LIMITED TO MAINTAIN INTEGRITY OF CLINICALTRIALS.GOV AS A TRUSTED SOURCE AND WORKING GROUP WOULD HELP DEFINE WHAT INTEGRITY MEANS WITH THAT AS WE SAID A COUPLE TILES, NATIONAL LIBRARY OF MEDICINE DOES NOT SAY THIS IS THE BEST CLINICAL TRIAL VERSUS ANOTHER CLINICAL TRIAL, INTEGRITY, JUST THAT HERE ARE THE CLINICAL TRIALS AND HERE ARE HOW TO CONTACT THOSE IF YOU WANT TO GET INVOLVED. WHAT DOES INTEGRITY MEAN? MAXIMIZE UTILITY OF THE GROWING CORPUS OF INFORMATION, SUBMISSION PRACTICES, USER FOCUS, TECHNICAL FUNCTIONALITY, AND WITH THAT CONNECT WITH STAKEHOLDERS TO UNDERSTAND AND CONSIDER DESIGN ENHANCEMENT. THERE'S ACTUALLY QUITE A NUMBER OF PEOPLE WHO ARE USING clinicaltrials.gov. I BELIEVE YOU SAID IT WAS 50% ARE THE LAY PUBLIC WHO ARE GOING IN AND TRYING TO UNDERSTAND WHAT CLINICAL TRIALS ARE OUT THERE. SOMETIMES FOR THEMSELVES, SOMETIMES FOR FRIENDS OR FAMILY MEMBERS TO -- I IMAGINE PEOPLE FACING CANCER AND GETTING BAD NEWS, THEY MAY GO TO clinicaltrials.gov TO SEE WHAT'S THERE. AND WHAT DO THOSE USERS NOW WANT AND NEED FROM THE clinicaltrials.gov USERS, AND NOW THAT IT'S 2019, NOT 2007, CERTAINLY IT'S TIME TO DO SOME MODERNIZATION ON THAT. AND OF COURSE, YOU KNOW, THEY HAVE MET WITH 19 OF THE 27 INSTITUTES HERE TO SEE WHAT THEY WOULD WANT OUT OF clinicaltrials. gov. INDUSTRY, GROUPS LIKE BREAST CANCER, ET CETERA, HOW DO THEY USE clinicaltrials.gov. ACADEMIC SITES, IRB MEMBERS, SYSTEMATIC REVIEWERS, RESEARCHERS ARE VERY IMPORTANT. HEALTHCARE PROVIDERS AND SO FORTH, AND TO -- AGAIN, GET ENGAGEMENT FOR THE COMMUNITIES, EXACTLY WHAT THEY WANT AND NEED OUT OF clinicaltrials.gov, AND AFTER THE COURSE OF A YEAR WITH SEVERAL MEETINGS OF THE CLINICAL TRIALS WORKING GROUP WE CAN RACK AND STACK THESE TO SEE WHAT SHOULD BE THE HIGHEST PRIORITIES OF WHOSE NICHE SHOULD BE FILLED AND MAKE A LIST OF THINGS, SEE WHEN RESOURCES RUN OUT AND FROM THERE DEVELOP A NEW MODERNIZATION IN THE clinicaltrials.gov. >> WHAT IS THE RELATIONSHIP BETWEEN IT AND -- IS IT STILL A REGENTS WORKING GROUP OR SEPARATE WORKING GROUP THAT INCLUDES SOME PEOPLE FROM REGENTS AND OTHER PEOPLE? >> I THINK THE LATTER, SO MEMBERS OF THE REGENTS, STILL LEGALLY ALLOWED TO INVITE OTHER PEOPLE WITHOUT GOING THROUGH THE BIG RIGMAROLE. ALL THE WORKING GROUPS -- >> ACTUALLY HAVE PEOPLE OUTSIDE œSUBCOMMITTEE OF THE BOARD OF REGENTS IS ONLY BOARD OF REGENT MEMBERS, THE E P SUBCOMMITTEE IS THAT WAY. THIS ONE IS BEING EXPLICITLY SET UP WITHIN THE PUBLIC SERVICES WORKING GROUP SO WE DON'T HAVE JUST A PROLIFERATION OF DIFFERENT WORKING GROUPS WITH TWO OR THREE MEMBERS AT A TIME BUT IT WILL BE A DIRECT REPORT TO THE BOARD OF REGENTS THROUGH THE PUBLIC SERVICE WORKING GROUP. ANY RECOMMENDATION WOULD BE VETTED AT THIS LEVEL, NOT THE PUBLIC SERVICES GROUP ALONE. >> AND WOULD THIS REPLACE OTHER WORK THAT THEY WOULD DO? >> OTHER WORK? >> PUBLIC SERVICE. >> ONE OF THE PUBLIC SERVICES. >> YEAH. >> IT IS A FOCUS RIGHT NOW, ONCE THIS GETS ESTABLISHED AND STABILIZED WE MAY BRING IN OTHER ISSUES LIKE THE NPA, THERE MAY BE OTHER FACTORS, BUT FOR THE NEXT I WOULD IMAGINE TWO TO THREE MEETINGS AT LEAST THIS WILL BE THE BULK OF THE WORK THAT GROUP WILL BE DOING. >> LET ME ASK YOU JUST -- HAVEN'T HEARD ANYTHING FROM MY COHORTS. >> MY PERSPECTIVE, THE KEY ISSUES ARE WHAT WE STARTED TO DEVELOP, ALSO THE PLAN, HOW WE WOULD GROW IT OUT AND REALLY IN THE FIRST YEAR A LISTENING TOUR TO HELP UNDERSTAND WHAT IS IT THAT IS IMPORTANT FROM ALL THE PERSPECTIVES OF THE USERS AND WHAT HAVE YOU, BUT ALSO WE WANT TO BE OPEN TO THE INPUT, NOT ONLY OF THE PROFESSIONAL FOR EXAMPLE BUT FOR ACTION GROUPS AND OTHER COMPONENTS OF PEOPLE WHO USE IT. WE WERE ALSO EXPLORED POTENTIALLY COLLABORATIONS WITH GROUPS NOT PART OF THE NATIONAL LIBRARY OF MEDICINE BUT THEY CAN THEN USE OUR INFORMATION TO ADDRESSING INTO MAKE IT AVAILABLE TO PATIENTS. THERE'S A LOT TO DO. BUT THAT WAS A GOOD SUMMARY >> THE ONLY THING I WANT TO EMPHASIZE IS THE DISCUSSION ABOUT THE TRUSTED SOURCE. YOU KNOW, SO THAT IT'S SOMETHING THAT WE NEED TO REALLY THINK ABOUT AND THINK CLEARLY DEFINE AND MAKE IT UNDERSTANDABLE FOR THE CONSUMERS. BECAUSE I DON'T THINK CONSUMERS SEE IT THE WAY THAT IT IS REALLY SPELLED OUT. >> THE THING IS, SHOULDN'T THINK WE HAVE TO CHANGE THE SOFTWAREF YOU WANT TO GET PATIENTS TO FIND OUT WHERE THE STUDIES ARE, YOU GOT TO CHANGE THE RULES OUTSIDE OF THE SOFTWARE. SO IT'S A LOT OF TIMES VERY HARD TO UNDERSTAND WHERE YOU'RE GOING TO GET THE STUDY, IT'S NOT SOMETHING THEY CAN CHANGE, IT'S THE CONTENT. MAYBE YOU COULD -- >> YEAH, DEFINITELY CONTINUE ON THAT POINT. BASICALLY WE ARE REALLY AT SORT OF A TURNING POINT WHERE WE'VE HAD TO FOCUS THE LAST 10 TO 12 YEARS ON IMPLEMENTING SORT OF THE REGULATIONS AND POLICIES THAT ARE REALLY INTENDED TO PROVIDE BENEFIT TO A PRODUCT RANGE OF USERS, BOTH SORT OF THE PATIENT, HEALTHCARE PROVIDERS, AS WELL AS RESEARCHERS, AND REALLY OUR OBJECTIVE RIGHT NOW IS TO FOCUS ON HOW WE CAN EXTRACT THE VALUE OUT OF THESE& POLICIES, REGULATIONS AND LAWS THAT EXIST BUT ALSO AS WE'RE ENGAGING WITH THESE DIFFERENT COMMUNITIES TO UNDERSTAND THEIR NEEDS AND TO IDENTIFY DIFFERENT RULES AND RESPONSIBILITIES, AND THAT'S A LITTLE BIT WHAT CLEM WAS GETTING TO, SORT OF THE INFORMATION WE'RE REPRESENTING FOR EXAMPLE AT clinicaltrials.gov IS SUMMARY OF THE PROTOCOL DOCUMENTS. AND SO WHEN YOU THINK ABOUT WHETHER OR NOT THE INFORMATION THAT WE HAVE AVAILABLE IS MEETING THE NEEDS OF THE PUBLIC, YOU HAVE TO THINK ABOUT THE SOURCE OF THAT INFORMATION AS WELL, AND WHO ARE SOME OF THE DIFFERENT PLAYERS IN THAT SPACE AND WHAT WE'RE ALSO HOPING FOR IS TO REALLY UNEARTH HOW THIS FITS INTO THE BROADER CLINICAL RESEARCH LIFE CYCLE AND IDENTIFY NOT ONLY WITH USERS BUT SORT OF WHO ELSE HAS A ROLE TO PLAY AND CAN HELP US IN SUPPORTING HIGH QUALITY INFORMATION THAT WILL SUPPORT ALL OF THESE DIFFERENT USES THAT PEOPLE EXPECT. AND SO WHAT WE ENVISION OVER THE NEXT YEAR, WE'RE BUILDING OUT A STAKEHOLDER ENGAGEMENT STRATEGY AND WANT TO MAKE SURE THROUGH THE WORKING GROUP AND SOME EXTERNAL MEMBERS THAT WE'RE THINKING ABOUT THIS FROM THE RIGHT PERSPECTIVE AND HAVE THE RIGHT PEOPLE AT THE TABLE IN TERMS OF HOW WE'RE APPROACHING, ENGAGING WITH THE COMMUNITIES TO MAKE SURE WE'RE GETTING ALL THESE NEEDS OUT ON THE TABLE CLEARLY. >> IF I COULD INTERRUPT, THIS IS THE ACTING DIRECTOR OF clinicaltrials.gov. THAT'S WHY SHE SPEAKS WITH SUCH AUTHORITY. >> ONE PERSPECTIVE TOO THAT AS A CLINICAL TRIALSIST MYSELF, STRUGGLING WITH USING clinicaltrials.gov FROM MY PERSPECTIVE US A OF THE RULES WHO GETS TO CHANGE THINGS AND WHO SAYS THE DATA ARE OKAY, AND HOW SPECIFIC THE FORMAT HAS TO BE, THAT ALL NEEDS TO BE EVALUATED AS WELL. I THINK FOR ME, I'LL TRY NOT TO GET PERSONAL, TO GO THROUGH MULTIPLE SETS TO FIX DATA, IT DOESN'T MAKE SENSE. I'M THE P.I. RESPONSIBLE FOR THE TRIAL, LET ME MAKE THE CHANGES. IF YOU COULD LOOK AT AS WELL REGULATORY FRAMEWORK THAT YOU ALL PUT IN PLACE AND MAKE THAT PART OF WHAT YOU EVALUATE, I THINK YOU COULD TOO LIBERATE THE SYSTEM TO MAKE IT MORE USEFUL. >> CAN I SLIP, THE REGULATORY FRAMEWORK DRIVES US. WE DON'T SET THE REGULATORY FRAMEWORK. >> IN TERMS OF RULES AND RESPONSIBILITIES, THESE ARE ALSO SOME OF THE THINGS THAT WE WANT TO UNEARTH AS PART OF THIS TO HELP PEOPLE SEE WHERE THERE ARE SOME NEEDS RESTRICTIONS, WE HAVE TO FOLLOW THIS FRAMEWORK THAT WAS BASICALLY GIVEN TO US BY CONGRESS AND WHAT MAY NEED TO CHANGE IF PEOPLE WANT SOMETHING ELSE. >> GOOD. AND THEN I GUESS JUST THE APPROACH THAT YOU MIGHT CONSIDER IS REALLY HUMAN-CENTERED DESIGN APPROACH, REALLY GO BACK TO BASIC PRINCIPLES. YOU'VE GOT MULTIPLE USERS OF THIS DATA. ONE IS CONSUMER. AND 50% OF USERS ARE CONSUMERS SO REALLY UNDERSTANDING HOW CONSUMERS INTERACT WITH IT, TALKING TO THEM, WHERE WOULD THEY GO, KEY INFORMATION THEY WANT OUT OF IT. YOU KNOW, DETAILED INCLUSION/EXCLUSION CRITERIA FOR CLINICIANS MAY NOT BE AS USEFUL, WHO TO ASK QUESTIONS, ALL THESE THINGS ARE HIDDEN AND CONFUSING POTENTIALLY TO CONSUMERS. PEOPLE RESPONSIBLE FOR PUTTING IN THE DATA, HOWEVER THAT'S REGULATED BY CONGRESS, BRING THOSE VOICES TO TRY TO SIMPLIFY THAT PIECE. I THINK THERE'S BEEN PROBABLY AN OVEREMPHASIS ON RESEARCHERS WHO WILL USE THE DATA ULTIMATELY BECAUSE I WONDER, WE'VE HAD WONDERFUL USES OF CLINICAL TRIAL DATA, GREAT THINGS THAT CAME OUT OF IT, BUT MOST PEOPLE DON'T GO TO FIND RESULTS OF clinicaltrials.gov FROM clinicaltrials.gov, YOU DO A META-ANALYSIS, YOU NEVER RELY ON THAT. YOU GO BACK TO SOURCE OF PUBLICATION, SO I WOULD THINK ABOUT HOW THAT'S BEEN PRIORITIZED. WE WANT THE THING TO BE USED FOR RESEARCH PURPOSES AS WELL, BUT CREATING A WHOLE SERIES OF ADDITIONAL REQUIREMENTS UNDERNEATH THAT DON'T EVEN GET YOU TO THE LEVEL THAT RESEARCHER WOULD REQUIRE, I THINK IT NEEDS TO BE EXAMINED AS WELL IN TERMS OF THE KINDS OF -- HOW CLEAN, HOW TRUSTED DOES IT NEED TO BE. >> YOU HAD THOUGHTS HOW WE MIGHT HELP COMMUNITIES UNDERSTAND, FOR EXAMPLE, THE ISSUE THAT GOING TO THE TABLE IS COMMON IN MOST CULTURES, ONE-THIRD OF TRIALS ARE SUPPORTED BY NIH NEVER GET THERE. >> THAT'S AN AMAZING THING. >> HOW DO WE HELP RESEARCHERS UNDERSTAND THAT WE WANT TO WORK WITH THE PUBLICATION PROCESS, NOT IN PLACE OF IT? >> RIGHT. SO, I THINK THAT'S ONE OF THE MOST USEFUL -- WE KNEW THAT WOULD BE THE CASE WHEN clinicaltrials.gov WAS SET UP, THAT WE KNEW THAT WE COULD -- YOU KNOW, WE HAD DONE AN ANALYSIS, NINDS' TRIALS, AND DISCOVERED A BUNCH HADN'T BEEN PUBLISHED. AND WHEN YOU LOOK AT THAT, THAT ALONE, WE KNOW IN THE INDUSTRY THAT HAPPENS EVEN FAR MORE. BUT THE FACT AT NIH TRIALS THAT WOULD -- >> (INAUDIBLE). >> YEAH, SO THAT BY ITSELF IS INCREDIBLY USEFUL TO REVEAL. I WOULD SAY THE SOLUTION ISN'T NECESSARILY THEN TO BURDEN EVERY TRIAL WITH INCLUDING THEIR DATA ON clinicaltrials.gov, THE BURDEN SHOULD BE THAT EVERY TRIAL THAT'S COMPLETED ETHICALLY SHOULD BE PUBLISHED. I MEAN, TO ME IT'S UN ETHICAL NOT TO PUBLISH RESULTS. IT'S YOUR PROBLEM TO REVEAL THAT'S AN ISSUE, I DON'T THINK IT'S YOUR PROBLEM NECESSARILY TO BE THE SECOND TIER PUBLICATION FOR EVERY CLINICAL TRIAL. YOU'LL NEVER MEET THAT -- >> THAT'S THE SITUATION WE WERE PUT IN. >> RIGHT. >> BECAUSE OF THE LAWS OF THE LAST TEN YEARS TRYING TO IMELEMENT. THERE WERE CONCERNS ABOUT NOT JUST THE PERCENTAGE OF TRIALS THAT WERE NOT BEING PUBLISHED BUT EVEN AMONG THOSE PUBLISHED, WERE THERE PROBLEMS WITH SOME DATA ANALYSIS, SEEING COMPLETE RESULTS REPORTING AND HENCE I GUESS THERE WAS ENOUGH PUBLIC AND LEGISLATIVE OUTCRY ABOUT THAT. DOESN'T MEAN WE SHOULDN'T BE THINKING HOW WE CAN DO THAT AND MAKE SURE WE ADHERE TO REQUIREMENTS IN A WAY THAT'S THE MOST USER FRIENDLY FOR BOTH TO SUBMIT THE INFORMATION AND PULL IT OUT BUT WE DO HAVE CERTAIN CONSTRAINTS BASED ON PERCEPTIONS WHAT HAS BEEN A PROBLEM AND WHAT IS GOING IN THE PUBLISHED LITERATURE. >> MAYBE JUST TO REFRAME IT, I REALIZE YOU'RE HANDED THINGS AND HAVE TO DO, BUT IF THE GROUP COULD SAY IN TRUE SYSTEM, IDEAL SYSTEM, HOW WOULD THESE ISSUES BE HANDLED, WHAT WOULD BE YOUR RESPONSIBILITIES, OTHERS RESPONSIBILITIES, I THINK THAT WOULD BE EXTREMELY USEFUL. >> YOU BRING UP A REALLY IMPORTANT POINT, I DON'T THINK WE'VE MADE EXPLICIT YET WITH BECKY AND CARLOS THAT WE HAVE TO HAVE A PATH TO COMMUNICATE TO THE NIH CLINICAL TRIALS COMMITTEE THAT ACTUALLY IS THE POLICY DRIVER WE RESPOND TO. WE'LL HAVE TO FIND COMMUNICATION PATHWAYS BACK UP, SOME OF WHAT WE DISCOVER. BUT WE'RE NOT GOING TO RE-OPEN FDAAA, THAT WAS TEN YEARS OF PAIN DISCUSSION, AND TO ENCOURAGE THE FDA TO RE-OPEN LEGISLATION I HAVE TO SAY -- >> I THINK THAT'S A GOOD EXAMPLE, PATTY, WHAT YOU JUST DESCRIBED AND THEN I'LL SHUT UP. I THINK NIH HAS THE POTENTIAL, IF YOU WERE TO SAY IF YOU DON'T PUBLISH CLINICAL TRIALS RESULTS FROM AN NIH-SPONSORED TRIAL YOU'LL NEVER AGAIN BE PUBLISHED BY THE NIH -- >> THERE YOU GO. >> SOMETHING YOU CONTROL. >> NO, WE DO NOT. >> YOU DO NOT, BUT POLICY COULD BE CREATED TO DO THAT. THAT'S THE KIND OF THING THAT CAN BE MORE PERSUASIVE AND SOLVE THAT PROBLEM, INDUSTRY IS HARDER BUT AT LEAST THERE ARE OTHER ALTERNATIVES. >> AS WE MOVE TOWARDS DATA MANAGEMENT SHARING POLICY WE'RE LOOKING AT ENFORCERS, AND THERE IS AN ENFORCEMENT STRATEGY RIGHT NOW WITHIN THE NIH THAT IS IF YOU DO NOT PUT YOUR DATA INTO clinicaltrials.gov, WITHIN ONE YEAR OF THE COMPLETION OF YOUR STUDY, THERE ARE PENALTIES FOR YOU AND POTENTIALLY FOR YOUR INSTITUTION. >> YOUR ENTIRE INSTITUTION. >> WITHHOLD FUNDING FOR THE ENTIRE UNIVERSITY OF MICHIGAN, SO THERE IS -- ENFORCEMENT IS -- >> (INAUDIBLE). >> THERE ARE ISSUES ABOUT ENFORCEMENT. ONE OF THE THINGS WE ACTUALLY SHOULD HEAR FROM THE RESEARCH COMMUNITY WHAT ARE EXPECTATIONS OF ENFORCEMENT BECAUSE THAT MAY BE A LEVER. >> YEAH, I GUESS TWO OTHER POINTS. I THINK ALL THE THINGS YOU SEE ARE VALUABLE BECAUSE THEY ARE THING OTHER RESEARCHERS ARE SAYING AS WELL. THESE AREN'T UNIQUE. I'M NOT MINIMIZING THAT BUT THESE ARE IMPORTANT AND I THINK IN TERMS OF THAT VALUE PIECE, WE ALSO WANT TO UNDERSTAND WHY ARE PEOPLE NOT SEEING THIS AS A VALUABLE RESOURCE, IT WAS INTENDED TO BE, AND SO RIGHT NOW THERE'S A LOT OF PUBLICATIONS THAT SHOW HOW MUCH MORE COMPLETE THE ADVERSE EVENT INFORMATION IS ON clinicaltrials.gov. YOU HAVE TO HAVE ALL SERIOUS ADVERSE EVENTS REPORTED. THAT SHOULD BE OF GREAT VALUE TO META ANALYSTS AND SYSTEMATIC REVIEWERS, WHY ISN'T IT? AND DO WE WANT TO HELP UNDERSTAND SOME OF THOSE THINGS SO THAT IT CAN REALLY BE LEVERAGED IN THAT WAY? THESE ARE ALL REALLY IMPORTANT THINGS WE WANT TO EXPLORE. HEIDI, YOU MADE AN INTERESTING POINT YESTERDAY ABOUT LITERATURE AND HOW WE HAVE STRUCTURE AND THEN WE FOCUS ON A PUBLICATION AND REASSEMBLE THE STRUCTURE. THINKING ABOUT clinicaltrials.gov WE'RE SIMILARLY REQUIRING A STRUCTURE NEVER REQUIRED BEFORE RESEARCHERS, SOMEONE HAS EXCEL DOCUMENT, THEIR OWN DATABASE, THERE'S NO COMMON STRUCTURE IN THE WAY THAT PEOPLE ARE COLLECTING THEIR OWN DATA THAT THEN WOULD SUPPORT THAT DISSEMINATION IN A STRUCTURED WAY ON A DATA BASE SUCH AS OURS. WHEN WE THINK ABOUT ROLES AND RESPONSIBILITY WHERE IS ARE THE HAVE AN IMPACT IN ADDING MORE STRUCTURE TO ADD THE STREAMLINE BUT ALSO TO ADD MORE VALUE LATER IN THE SYSTEM. >> THINKING ABOUT WORKFLOWS, HOW TO INTEGRATE REPORTING INTO WORKFLOWS, clinicaltrials.gov LANDING AND CONDUCT, MAYBE ONE WAY OF GETTING AROUND MAKING THIS SEEM LIKE AN ADDITIONAL SET OF RESPONSIBILITIES. >> I WANT TO REINFORCE THE FACT AS A P.I. WE'RE TAKING THIS SERIOUSLY. >> THANK YOU. >> THE FINAL RECORD MUST BE SUBMITTED BY THE P.I. OF THE PROJECT. AND IN MANY CASES WE UNDERSTAND THAT PRESSING THAT BUTTON IS IN ITSELF A BURDEN AND WE'RE LOOKING AT WAYS TO MAKE IT MORE FEASIBLE FOR THE P.I. TO REVIEW THE MATERIAL A TRIAL NURSE OR SOMEONE PUT IN TO UNDERSTAND THE Q.C. PROCESS, SOMETIMES PERCEIVED AS ADDED BARRIER TO SUBMISSION BUT THANK YOU AS A P.I. FOR CARING AS AM. >> DO YOU ACTUALLY -- (INDISCERNIBLE). >> WE DO. WE DO NEED AN ACTION. WE CAN HAVE MORE DISCUSSION. THE DESCRIPTION OF WHAT THE WORKING GROUP WOULD LOOK LIKE IS ON PAGE 30 OF YOUR BOARD BOOK, AND THE ACTION IS REALLY MORE TO ACCEPT RESPONSIBILITY RATHER THAN HELP US FLESH OUT THE EXACT COMPOSITION BUT WE WELCOME CONVERSATION ABOUT THAT. >> (INAUDIBLE) IF THEY DON'T GET PUBLISHED. I'VE PERSONALLY AND YOU MUST HAVE HAD MANY ATTEMPTS, THEY SAID NO, NOT BRAND NEW, NOT RADICAL, JUST A REDO. I THINK IT'S GOING TO BE CRUEL TO PUNISH RESEARCHERS WHEN JOURNALS WON'T TAKE THE PAPER >> THERE'S A JOURNAL FOR EVERYTHING. [LAUGHTER] >> THAT'S WHERE EVERYONE IS TRYING TO GET TO. >> NEGATIVE TRIAL RESULTS, IT'S IMPORTANT TO PUBLISH. >> I AGREE. THE OTHER POINT REGARDING DATA AND COMMONALITY, NIH IS INTERESTED TO PROMOTE THE COMMONALITY. THE THIRD THING IS WHY COULDN'T WE ENCOURAGE THE JOURNALS TO TAKE YOUR FORMAT SO IT'S NOT MUCH DOUBLE WORK FOR THE RESEARCHERS. >> THAT COULD BE, YEAH, THAT WAS ONE, FDA IS THE OTHER. FDA REQUIRES THINGS IN CERTAIN FORMATS. THERE'S STANDARD AND THAT WILL BE ANOTHER OPPORTUNITY. >> I THINK THIS IS SUFFICIENTLY IMPORTANT THAT MAYBE EXPERTING THROUGH BUILDING 1 AND HAVING A STUDY WOULD BRING THE KIND OF NATIONAL ATTENTION TO THIS ISSUE. I THINK IT'S REALLY IMPORTANT? THAT'S AN INTERESTING POSSIBILITY. WAS THERE A NATIONAL ACADEMY STUDY ON CLINICAL TRIALS DATA SHARING, OR THERE IS A WORK ON DATA SHARING RIGHT NOW, BUT NOT SPECIFICALLY ON THE TOOL. >> THE REPORTING. >> ON THE REPORTING. >> YEAH, I THINK IT'S MORE A MATTER OF BEING CLEAR ABOUT THE GOALS AROUND SUCH A STUDY. >> SOME LEGISLATION WE WERE WORKING WITH WAS INFLUENCED BY PREVIOUS DRUG FORUM ACTIVITIES. SOME PRECEDENT FOR DOING THAT BUT ALSO BE CAREFUL. >> AND LET ME SAY WE HAVE -- WE ENVISION MULTI-YEA IN THE RANGE OF $30 MILLION FOR REBUILD. WE'VE GOT TREMENDOUS SUPPORT FROM BUILDING 1 ABOUT THE STRATEGY, JIM AND BECKY HAVE BEEN TERRIFIC, RECREATING NUMEROUS WORK PLANS. WE'VE HAD A YEAR AND A HALF OF ENGAGEMENT WITH MITRE TO DO ENGINEERING MODELING AS IT IS RIGHT NOW AND TO IDENTIFY SOME OF OUR CRITICAL VULNERABILITIES AND ADDRESS THEM IMMEDIATELY, SECURITY, PUTTING THE SYSTEM ON A STABLE PLATFORM, WORKING WITH DEVELOPMENT TEAMS AND INCREDIBLY ROBUST AND RESPONSIVE IN DEVELOPING EVERY PIVOT TO RESPOND TO LEGISLATION. SO WE HAVE STARTED THE PROCESS BUT THIS IDEA OF ENGAGEMENT, ESPECIALLY HEARING LIKE BECKY WAS SAYING, HOW TO WE MAKE IT USEFUL, WHAT ARE THE BARRIERS, A BIG THREAD TO THIS HERE. >> OTHER COMMENTS ABOUT THE GROUP, DALEEL. >> WE NEED TO GET THE GROUP STARTED. OTHER GROUPS NEED TO COME IN, IOM, BUILDING ONE BUT IF WE DON'T DO OUR HOMEWORK WE'RE SUBJECT TO CRITICISM YOU'RE PASSING TO US. I RECOMMEND WE ACCEPT, MOVE TO ESTABLISH THIS SEPARATE WORKING GROUP, AND LET THEM GET STARTED AND SEE WHAT THEY BRING BACK TO US AS NEXT STEPS. >> OKAY. SO WE HAVE A MOTION. >> SEPARATE WORKING GROUP, WORKING GROUP WITHIN THE -- >> RIGHT, AS DESCRIBED HERE. >> YES. >> A SECOND TO THE MOTION? >> SECOND. >> ALL IN FAVOR? >> DO WE VOTE? >> PUT YOUR HAND DOWN. (INDISCERNIBLE). >> YES. >> ARE THERE ANY NOs? OKAY, THANK YOU. >> THANK YOU. >> AND NOW WE'RE GOING TO MOVE ON TO THE REPORT OUT FROM THE COLLECTIONS GROUP FROM NEIL. DO YOU WANT TO GIVE SOME HIGHLIGHTS OF OTHER THINGS BEYOND THE COLLECTIONS POLICY IN THE WORKING GROUP? >> WELL, I THINK ONE THING THAT COMES TO MIND FROM THE PRIOR DISCUSSION WE HAD EARLIER THIS MORNING IS AROUND THE ISSUE OF THE ROLE OF TRUST, IN CONTEXT TO NATIONAL LIBRARY OF MEDICINE. AND IT IS REFERRED TO IN THE PROPOSED POLICY IN TERMS OF LIBRARY OF RECORD, THE TRUSTED ROLE AS LIBRARY OF RECORD. IN TERMS OF BROADER DISCUSSION WE HAD EARLIER, IT OCCURRED TO ME THAT I DON'T THINK THIS NEEDS -- I DON'T THINK THE POLICY NEEDS TO BE ALTERED TO ACCOMMODATE THIS, BUT YOU MIGHT WANT TO GIVE SOME THOUGHT TO THOSE WHO ARE MORE RESPONSIBLE TO THINK ABOUT PERHAPS A PREPARATORY STATEMENT ABOUT WHAT THE ROLE OF TRUST MEANS IN THE CONTEXT OF THE NATIONAL LIBRARY OF MEDICINE. IT DOESN'T MEAN THAT AS THE TRUSTED LIBRARY WE VERIFY EVERYTHING WE PUT IN OUR COLLECTION IS TRUE. OR WILL STAND THE TEST OF TIME. BUT THAT THERE IS A PROCESS BEHIND IT, THAT ENSURES THAT CRITERIA ARE APPLIED TO THINGS THAT GET INCLUDED IN THE COLLECTION OR JOURNALS THAT GET INCLUDED WITHIN MEDLINE, OR CLINICAL TRIALS GET INCLUDED WITHIN clinicaltrials.gov. AND THIS IS -- EACH MECHANISM HAS ITS OWN SET OF CRITERIA. IT MIGHT BE REFERRING TO THAT THERE ARE THINGS OUT THERE, THESE PROXIES THAT ARE USED TO GET TO THE IDEA OF QUALITY, EVEN IF THEY WERE MORE THAN PROXIES AND EVEN IF QUALITY WERE GUARANTEED, YOU KNOW, THEY WOULD STILL NOT BE EQUATING WITH PROOF. BUT A SYSTEMATIC PROCESS IN PLACE IS WITHIN THE ROLE OF WHAT THE NATIONAL LIBRARY OF MEDICINE IS ABOUT AND IT'S SORT OF PARALLELLING THE PROCESS OF SCIENCE IN THAT SENSE. AND DOES ENSURE THAT THE RECORD IS MADE, THE INTEGRITY OF THE RECORD IS MAINTAINED. AND THAT'S WHAT TRUST ACTUALLY MEANS. SO I DON'T KNOW, I THINK IT WOULD BE KIND OF INTERESTING TO SOMEHOW PUT OUT THERE SOMEWHERE RELATES TO THIS PARTICULAR POLICY BUT RELATES TO ALL THE OTHER MECHANISMS WITHIN NLM IN WHICH MATERIAL IS ACQUIRED AND MADE AVAILABLE. >> OVERARCHING THEME, DOCUMENTS RELATIVE TO IT. >> (INDISCERNIBLE). >> WE ENGAGED 18-F, PART OF THE EXECUTIVE BRANCH INNOVATION TEAM TO HELP US WITH INTERFACE DESIGN WITH clinicaltrials.gov, SPECIFICALLY AROUND THIS ISSUE. AND THE IDEA OF PLACEMENT, POP-UPS, LABELING WAS EXAMINED. >> LANGUAGE. >> AND LANGUAGE ITSELF. AND POP-UPS ACTUALLY ARE NOT -- AT LEAST WITH THE RESPONSE THIS GROUP HAD, BUT I THINK THE IDEA FIND WHAT WE MEAN BY "TRUST" IS IMPORTANT. >> OR EVEN IF IT'S A LINK THAT'S IN THE CORNER THAT SAYS HOW -- WE'RE A TRUSTED SOURCE, CLICK, BY TRUSTED SOURCE WE MEAN, SO NOT EVERYONE WILL NECESSARILY BE DISTRACTED BY THAT. >> YEP. >> BUT THEY MIGHT CLICK ON IT AS A WAY OF SORT OF SAYING, YOU KNOW, YOU CAN WORK ON THE LANGUAGE SO THAT IT'S ACCURATE BUT IT'S ALSO NOT -- IT'S NOT A LEGAL REQUIREMENT. JUST A QUALITY OF INFORMATION. CLARITY FOR THE USER. IMPORTANT TO HIGHLIGHT. WELL, YOU BRING THAT UP. YES, IT WAS IN THE NATIONAL LIBRARY OF MEDICINE. YES, BUT DID YOU READ THE TRUST PIECE THERE? IT DOESN'T GUARANTEE THAT THIS IS, YOU KNOW, DOGMA. HOW YOU WANT TO FRAME IT, YOU KNOW, GIVEN THE TENOR OF OUR CONVERSATION FOR THE LAST COUPLE DAYS, WORTH AT LEAST CONSIDERING THAT OPTION. >> AGAIN, I PUT THAT OUT THERE NOT IN TERMS OF GETTING IN THE WAY OF THIS PARTICULAR POLICY, BUT IT DOES AFFECT ISSUES THIS POLICY DOES GET TO BUT IT'S BROADER THAN THAT AS WELL. >> YES. >> SO, MARY -- >> BROADLY THAT THE ROLE OF THE LIBRARY IN DEVELOPING ITS COLLECTION. >> ESPECIALLY THE AGE OF DIGITAL OBJECTS AND COLLECTION WRIT LARGE. >> YES. >> I RECOGNIZE THAT YOU'RE NOT THE LIBRARIAN OF CONGRESS. >> NO, I'M NOT. [LAUGHTER] >> NOT EVEN A LIBRARIAN. [LAUGHTER] >> HOW COLLECTIONS ARE MANAGED IN YOUR UNIT SO CONGRESSIONAL RESEARCH OFFICER MORE BROADLY, HOW DO YOU ADDRESS THESE ISSUES OF TRUST? COMPREHENSIVENESS? COMPLETENESS? >> WE HAVE CORE VALUES THAT GUIDE OUR WORK. >> OKAY. >> AND IT'S VERY EXPLICIT. DOES NOT HAVE AN OFFICIAL PUBLIC MISSION OR PUBLIC FACE. >> OKAY. >> STATUTORILY OUR FOCUS REMAINS ON THE CONGRESS, AND ON PROVIDING THEM WITH RESEARCH AND ANALYSIS AND AT LEGISLATIVE ISSUES. WE ALSO WORK IN A CONFIDENTIAL MANNER. AND WE BILL OUR SELF AS A TRUSTED SOURCE FOR CONGRESS, AND I THINK THAT MEANS A LOT OF DIFFERENT THINGS. >> YEP. >> IT'S QUALITY OF OUR WORK. PRESIDENTS THE FACT THAT THEY CAN COME TO US AND ASK US QUESTIONS AND WE WILL HOLD THAT IN A CONFIDENTIAL MANNER. AND THAT'S ANOTHER COMPONENT OF IT. AND IT'S SOMETHING WE REINFORCE DAILY. WE MAKE IT CLEAR, EMPLOYEE GETS ONBOARDED, WHAT OUR STANDARDS ARE. >> MARY RAISED A KEY ISSUE, MENTIONED ANALYSIS, NOT WHAT NLM IS APPOINTED TO DO. >> RIGHT. >> I THINK THAT THAT IS A VERY MAJOR DISTINCTION RIGHT THERE. THAT NEEDS TO BE CLARIFIED. MANY CONSUMERS MAY ASSUME THE FACT THAT IT'S THERE BECAUSE IT'S BEEN VETTED AND HAS BEEN ANALYZED IN A WAY THAT, YOU KNOW, THE QUALITY HAS BEEN ANALYZED AND IT HAS NOT. >> BUT I THINK THE TERM AUTHORITATIVENESS MAY HAVE SOMETHING THAT WOULD RESONATE WITH THE NATIONAL LIBRARY OF MEDICINE. >> YOU'RE MAKING ME WONDER, JOYCE MIGHT KNOW OFFHAND, DOES OUR ENABLING LEGISLATION SPEAK TO SCIENTIFIC AND CLINICAL COMMUNITY OR BROADLY TO THE PUBLIC? >> IT'S SPECIFIC TO HEALTH PROFESSION. >> HEALTH PROFESSION. >> THERE'S A BOARD POLICY ABOUT CONSUMER HEALTH LATER BUT -- BUT THEN THEY ARE SUPPORTING THE LIBRARY, NOT THE NATIONAL NETWORK. BUT I WANT TO SAY THE WORKING GROUP DID SPECIFICALLY ADDRESS THE ISSUE THAT I THINK CONCERN IN ADDRESSING THIS, IN DETERMINING COVERAGE, NLM WILL TAKE INTO CONSIDERATION ITS ROLE AS TRUSTED LIBRARY OF RECORD AND NATIONAL RESOURCE, IN THE THIRD PARAGRAPH. SO THAT ALLOWS EACH RESOURCE TO DETERMINE WHAT THAT MEANS AND WHAT'S APPROPRIATE. AND WE LAUNCHED IN 1998, THERE HAVE BEEN QUALITY GUIDELINES LINKED TO FIVE PRINCIPLES, YOU CAN SEE THEM, THAT'S HELPED US OVER THE YEARS, HEY, THIS IS HOW WE'RE SELECTING. AND WE HAVE A COMPLETELY DIFFERENT PROCESS FOR MEDLINE AND PMC BUT PUBLISH WHAT WE'RE LOOKING AT FOR AND WHAT WILL BE PART OF THE CONSIDERATION. ALL RESOURCES BUT WHERE IT'S IMPORTANT, I DON'T KNOW GENBANK AND OTHER RESOURCES SAY THIS IS WHAT WE'LL TAKE FROM YOU AND THIS IS WHAT WE WANT, MAYBE MORE IMPLICIT IN SOME RESOURCES THAN OTHERS. >> I WOULD ADD WITHIN PubMed CENTRAL BECAUSE WE'RE A VISIBLE PART OF THE COLLECTION, PEOPLE HAVE BROUGHT UP THEIR CONCERNS WITH SOME OF OUR CONTENT. OUR RESPONSE IS ACTUALLY TO ADD ON EVERY ARTICLE DISCLAIMER STATEMENT, WE ADD TEXT TO THE STANDARDIZED DISCLAIMER THAT WAS SORT OF TRYING TO GET TO THIS. BUT I THINK ALMOST BY MAKING IT A DISCLAIMER WE DO OURSELVES A DISSERVICE BECAUSE WE'RE TRYING TO GET OUT OF BEING -- I WONDER IF WE REFRAME HOW WE THINK AND PRESENT IT, IT COULD BE MORE EFFECTIVE. SO I THINK THIS IS SOMETHING WE'VE BEEN GETTING AROUND BUT MAYBE HAVEN'T GONE AFTER IT IN THE BEST WAY. SO IT'S AN INTERESTING WAY TO HEAR THIS DISCUSSION PRESENTED AS YOU CAN TRUST US WHEN I FEEL LIKE WE'VE BEEN BACK PEDALING ON THAT A LITTLE BIT. >> YOU CAN GET TO THAT BY SAYING WHAT YOU CAN BE TRUSTED FOR. >> YES. THE TEXT IN THE DISCLAIMER KIND OF GETS THERE. BY CALLING IT A DISCLAIMER MIGHT UNDERMINE IT A BIT. MAYBE. >> A GUARANTEE. >> YES. >> REMINDED ME OF DISCUSSIONS, THERE'S SOMETHING CALLED THE INFORMATION QUALITY ACT, RIGHT? IN GOVERNMENT, YOUR GOVERNMENT AGENCY YOU'RE POSTING INFORMATION, MAKING IT AVAILABLE, NEED TO BE CLEAR ABOUT YOUR PROCEDURES FOR VETTING QUALITY BUT THERE'S A LOOPHOLE, INTENTIONAL ONE, EXCLUDES LIBRARIES PARTLY BECAUSE OF THE NOTION LIBRARIES CAN'T TAKE RESPONSIBILITY FOR CONTENT OF EVERY ARTICLE, DESPITE THE FACT THEY ALL HAVE COLLECTION POLICIES, AS WE HAVE HERE, WE HAVE PROCEDURES FOR TRYING TO IDENTIFY AUTHORITATIVE TRUSTED INFORMATION OFTEN BASED ON PROCEDURE OR SOMETIMES BASED ON FUNDED BY NIH. IT IS THIS TENSION BETWEEN KNOWING WE CAN'T VOUCH FOR EVERY WORD IN EVERY STUDY THAT'S EVERY PUBLISHED BUT WE DO HAVE SOME REASON TO SAY WE DO BELIEVE BY PROVIDING YOU ACCESS TO INFORMATION IT MEETS SOME LEVEL OF TRUSTABILITY AT LEAST IN TERMS OF SOURCE OR WAY IT WAS PEER REVIEWED OR OTHERS. THAT IS A CONSISTENT PRINCIPLE, WE'RE HEARING IN EACH INSTANCE TICKINGS ABOUT WHAT'S THE TREASURE HOLD AND WHAT WE INCLUDE IN THE EXCLUSION CRITERIA, APPROPRIATE SUBJECT MATTER, CONVEY IN A MORE GENERAL WAY. >> DALE HAS A COMMENT AND JIM. >> I THINK THIS GOES BACK TO MUCH OF OUR DISCUSSION YESTERDAY, THE COLLECTION POLICY IS THE EASY PART OF OUR WORK. NOBODY QUESTIONS THE VERACITY OF LOCATOR. IT DESCRIBES WHAT WE STUCK IN THE BASEMENT. AND IF YOU WANT TO GET THAT, THIS IS THE AUTHOR, THIS IS THE TITLE, THIS WAS THE PUBLISHER, IT'S WHEN YOU BEGIN TO MOVE TO THE OTHER PRODUCTS THAT ARE INCREASINGLY MORE UNIQUE TO NLM FROM OTHER LIBRARIES. MEDLINE, PubMed, PubMed CENTRAL, GENBANK, CLINICAL TRIALS, WE DISCUSSED YESTERDAY EARLY MANUSCRIPTS, PRE-PRINTS, NOT THAT I HAVE STRONG OPINIONS. >> OH MY GOD. [LAUGHTER] >> WHAT YOU PUT IN MEDLINE AND WHAT DO YOU NOT, THESE ARE WHERE TRUST BECOMES A VARIABLE BUT GOES TO THE SELECTION CRITERIA OF WHAT GOES INTO THE PRODUCT YOU'RE FACING THE PUBLIC WITH. AND IT'S ALL DIFFERENT. AND IT'S NOT, YOU KNOW, DIGESTED INFORMATION IN ANY SITUATION EXCEPT MEDLINE PLUS. THAT'S THE ONLY PLACE WHERE THE LIBRARY SHAPES THE CONTENT IN A SCHOLARLY WAY. THE REST OF THE TIME WE'RE FILTERING THE SYSTEM IN SOME WAY. AND THAT'S WHERE I THINK THIS TRUST AND OUR POLICIES ARE GOING TO BE INCREASINGLY INTENTIONED BECAUSE YOU CAN'T DO EVERYTHING FOR EVERYBODY. AND AS SOON AS YOU BEGIN TO MAKE DECISIONS, YOU BEGIN TO REQUIRE A DEFINITION OF WHAT YOU CAN BE TRUSTED FOR BY THE CRITERIA USED IN MAKING THOSE DECISIONS. SO I THINK AT THE COLLECTION POLICY LEVEL, TRUST IS EASY. YOU KNOW, THIS IS WHAT WE'VE GOT. BUT ONCE YOU BEGIN TO MOVE BEYOND THAT, TO ALL THESE PRODUCTS WHICH ARE ABSOLUTELY ESSENTIAL TO MODERN HEALTH CARE, I MEAN, YOU SCARED ME TO DEATH EARLIER. IF THIS STAFF COULD NOT WORK AND PRODUCTS COULD NOT BE PRODUCED ... HEALTH CARE IN THE WESTERN WORLD WOULD BE PERMANENTLY DAMAGED. JUST-- IT WOULDN'T BE POSSIBLE TO DO BIOMEDICAL SCIENCE ANYWHERE IN THE WORLD BEYOND THE PROJECTS AT WORK, UNTIL THIS WAS STOOD UP AGAIN. AND IT UNDERPINS SO MANY OTHER ENGINES AND OTHER THINGS. YOUR PRIMARY USER MODEL, YOU KNOW, FROM INDEX MEDICS AND MEDLINE FLOW, MOST OF THE ABILITIES TO DO EVIDENCE-BASED MEDICINE, MOST OF THE ABILITIES TO HAVE CONTINUING MEDICAL EDUCATION, CURRENT STANDARDS OF CARE, AND SO DEFINING WHAT GOES INTO THOSE IS A HUGE ISSUE BUT IT'S GOING TO TAKE A LOT OF WORK TO BE PRECISE ABOUT WHAT IT IS AND DISCLAIMERS ARE PROBABLY NOT IT. BUT WE DO NEED TO FIND A WAY TO CONVEY TO THE USER WHO NOW WE CANNOT REGULATE TO OUR STATUTORY USER, WHAT IT IS THEY ARE GETTING BECAUSE LIBRARY CONSCIOUSNESS DOESN'T FLOW VERY FAR IN THE CURRENT MEDIA-DRIVEN ENVIRONMENT. I MEAN, PEOPLE GREW UP WITHOUT EVER DARKENING THE DOOR OF THE LIBRARY EXCEPT TO USE A COMPUTER. I THINK THIS IS A HUGE ISSUE AND WE BEGAN TO SEE IN OUR WORKING GROUP YESTERDAY THERE ARE SEVERAL AREAS WHERE WE NEED TO THINK ABOUT OUR POLICY, AND TRUST WILL BE A BIG PART OF WHATEVER STANDARDS WE PUT IN PLACE. BUT IT WON'T BE SOLVED IN TIME FOR LUNCH TODAY. >> ONE OF THE OTHER THINGS WE DID SPEND TIME HEARING ABOUT RATHER THAN HAVING US REWORD IT WAS ABOUT THE PILOT STUDY THAT'S GOING ON RELATED TO -- (INAUDIBLE). >> DO YOU WANT TO REPEAT? >> SO THE ENTIRE BOARD CAN HEAR, BECAUSE ONE IN PARTICULAR, THERE HAS BEEN A RECOMMENDATION FOR CONSIDERATION THAT CAME FROM THE WORKING GROUP AS WELL SO I WANT TO MAKE SURE WE GET THAT OUT. >> RIGHT. AND PATTY AND JERRY, CORRECT ME IF I GET THIS WRONG. THERE'S-- NIH LEADERSHIP INDICATED THEY WOULD LIKE FOR NLM TO HELP FACILITATE DISCOVERY OF PRE-PRINT LITERATURE IN ORDER TO ACCELERATE ACCESS TO SCIENCE. WHAT BEEN EXPLORING RIGHT NOW WHAT WOULD BE INVOLVED IN IMPLEMENTING A SORT OF PILOT. WHETHER THIS WOULD RUN -- WE DISCUSSED HOW LONG A PILOT WOULD NEED TO RUN IN ORDER TO ASSESS ITS IMPACT. TO TAKE NIH-FUNDED OR NIH-SUPPORTED PRE-PRINT INTO PubMed IN ARCHIVE FORMAT, BIOARCHIVE IS WORKING ON GETTING THIS FORMAT, A FEW OTHER NEW PRE-PRINT SERVERS, AND THEN ONCE DISCOVERABLE IN PubMed, THIS FOLLOWS ON NIH ENCOURAGING INVESTIGATORS TO REPORT PRE-PRINT PRODUCT OF AWARD, 17 OR 20. >> 17. >> BUT SO WE'RE EXPLORING HOW WE CAN DO THIS IN A WAY THAT SUPPORTS DISCOVERY AND ACCESS TO NIH RESEARCH WITHOUT PERHAPS OPENING UP PANDOR'S A BOX. THERE'S STILL A LOT OF STRONG FEELINGS, LET'S SAY, ABOUT PRE-PRINT, AND WE WANT TO MAKE SURE THAT WE CAN DO THIS WITHIN OUR CURRENT WORK FLOW IN A WAY THAT DOESN'T WATER DOWN THE SCIENCE THAT'S AVAILABLE ON PubMed AND CLEAR TO USERS WHAT THEY ARE GETTING. SO IT'S SORT OF WALKING THE LINE OF EMBRACING SOME OF THE OPPORTUNITIES THAT OPEN SCIENCE PROVIDES BUT WITHOUT LOSING THE TRUST THAT WE'VE BEEN DISCUSSING HERE THAT USERS HAVE IN RESOURCES. SO SOMEWHAT EXCITING AND ALSO AFTER TODAY'S DISCUSSION TERRIFYING. BUT, YES, THAT'S I THINK ALL I HAVE TO SAY ON THAT. >> HOW IS PRE-PRINT ASSIGNED? >> THAT'S ACTUALLY PHASE ONE OF THE PILOT. WE DO NEED TO, BECAUSE -- >> NIH HAS A CLEAR DEFINITION. >> THE DEFINITION OF PRE-PRINT REPOSITORIES. >> REPOSITORY. >> YEAH, THERE ARE SOME WEIRD GRAYS IN MY MIND LIKE THE F1 000 MODEL, I'VE HAD A COUPLE (INDISCERNIBLE) TRYING TO DEFINE IT AND I KNOW THERE'S AN EFFORT WITHIN THE BROADER COMMUNITY TO LOOK AT WHAT DEFINITION SHOULD BE, A PERFECT WORLD THE COMMUNITY WOULD DEFINE IT AND THEN NLM WOULD SORT OF SUPPORT IT. I'M WORRIED THE TIMELINE OF THOSE TWO EFFORTS MAY NOT PLAY OUT THE WAY MY PERFECT WORLD EXISTS. SO WE'RE GOING TO HAVE TO SEE. THAT'S SOMETHING RIGHT NOW FOR NLM TO CONSIDER. IF YOU HAVE THOUGHTS YOU KNOW WHERE TO FIND ME. >> THE PILOT MAY HELP US COME UP WITH BETTER UNDERSTANDING OF WHAT WE WANT TO CONSIDER. >> YES. >> A VIABLE PRE-PRINT REPOSITORY. >> MY HOPE IF WE IMPLEMENT THIS AS A PILOT WE HAVE THE OPPORTUNITY TO EVOLVE AND IF WE DEFINE TOO NARROWLY OR TOO BROADLY TO REACT TO THAT. >> A CONCERN FROM THE GROUP WAS THAT ONE YEAR MIGHT BE NOT QUITE LONG ENOUGH AND MAY TAKE LONGER TO REALLY SEE THE IMPACT BUT IT WAS DEFINED I GUESS WITH A MINIMUM. >> KIND OF IMPACT DO YOU RECOMMEND WE LOOK AT? >> CITABILITY. THERE ARE SEVERAL HISTORY OF SCIENCE STUDIES THAT LOOK AT PHYSICS AND MATH COMMUNITY OF THE '60s WHEN PRE-PRINT STARTED, AND THEY MAKE A DIFFERENCE IN MATHEMATICS WHERE THE AVERAGE TIME FROM SUBMISSION TO PUBLICATION WAS ABOUT FOUR YEARS. >> YEP. >> THEY DIDN'T MAKE ANY DIFFERENCE IN THEORETICAL PHYSICS, WHERE THE AVERAGE PUBLICATION TIME WAS 6 TO 8 MONTHS. >> YEP. >> SO, THIS IS REALLY A PROBLEM OF THE JOURNALS PEOPLE WANT TO PUBLISH IN, HAVING SLOW PUBLICATION TIMES. WHICH THE JOURNALS ARE ALREADY GRAPPLING WITH BY ELECTRONIC PUBLICATION OF ACCEPTED MATERIAL, BUT TO KNOW ANY UTILITY OF THIS IS GOING TO HAVE TO BE CITED BY SOMEBODY ELSE. IF THE GOAL IS TO ACCELERATE SCIENCE, YOU'VE GOT TO ESTABLISH WHETHER YOU'VE ACCELERATED SCIENCE OR NOT, GIVEN THE AVERAGE PUBLICATION TIME THE HIGH IMPACT JOURNALS ON THE LINE I WOULD SUBMIT THAT ANYTHING LESS THAN THREE YEARS YOU'VE GOT NO DATA. AND PROBABLY FIVE-YEAR IS NECESSARY TO GET THREE DATA POINTS FROM THREE YEARS OF SUBMISSION TO KNOW WHETHER IT'S ANSWERING THE QUESTION THE SCIENCE IS SAYING WE WANT TO ANSWER YOU'RE TALKING ABOUT A FIVE-YEAR EFFORT. >> YOUR COMMENTS ARE HELPFUL, I SAW THIS AS AN INSTRUMENTAL PILOT, HOW MUCH WILL IT COST, HOW MANY PEOPLE WILL IT TAKE, CAN WE DO THIS, WHAT PARTNERS, AND NEVER CONSIDERED THE IMPACT. SO THANK YOU VERY MUCH FOR BRINGING THAT TO MY ATTENTION, THAT'S ANOTHER ASPECT THAT WE NEED TO THINK ABOUT AND MODELS OF VOLUME AND CUSTOMER SERVICE MIGHT GIVE US GUIDANCE TO THINK. JERRY AND I TALKED ABOUT ENGAGING OSI IN SETTING UP EVALUATION CRITERIA THAT MIGHT ACTUALLY FIT WITH SOME OTHER EVALUATION CRITERIA AND OFFERINGS. >> YEAH, AND DALE BROUGHT THIS UP YESTERDAY, INCREDIBLY USEFUL FEEDBACK, INVOLVING THE ENGAGEMENT OF PUBLISHING COMMUNITY ITSELF, OXFORD UNIVERSITY PRESS BROUGHT UP WHEN WE LAST MET WITH THE PUBLISHER'S GROUP, A LOT OF JOURNALS DON'T REALLY SUPPORT CITATION OF PRE-PRINTS BECAUSE THEY ARE NOT PEER REVIEWED, PREVIOUSLY ONLY HAVE BEEN -- ONLY ABLE TO CITE THEM AS PERSONAL CORRESPONDENCE, JUST NOW GETTING TO THE POINT YOU CAN CITE THEM AS A PAPER IN BIOARCHIVES. AND SO WHETHER EVEN THE METRICS ARE AVAILABLE YET FROM THE BOARD IS AN OPEN QUESTION, WHETHER NLM CAN FACILITATE THAT OR NOT DISCUSSION IS -- >> IT'S REALLY IMPORTANT. I'M SORRY CLAY ISN'T HERE BECAUSE IT'S A CASE WHERE POLICY MAY HAVE GOTTEN AHEAD OF THE PROCESS. AND SO WE HAVE THE REALITY NIH SAID YOU CAN USE AND TO TRY TO MAKE IT FUNCTIONALLY WORKABLE IS GOING TO BE A CHALLENGE. WE'VE HAD SOME CONVERSATIONS WITH BIOARCHIVES, A NUMBER OF CONVERSATIONS WITH BIOARCHIVES, SOME SIMILAR ISSUES WERE PUBLISHED, THEY WANT US TO DRIVE TRAFFIC TO THEIR SITE, DON'T WANT US TO PULL TRAFFIC ON OUR SITE. I WANT TO MAKE SURE THIS BOARD KNOWS ONE OF THE WAYS WE ATTRACT THE PUBLISHING COMMUNITY RIGHT NOW IS THROUGH THE ICMJE, JOYCE IS OUR REPRESENTATIVE TO THAT. THIS IS AN AFFILIATION OF PUBLISHERS. WE ALSO THROUGH THE FRIENDS OF THE NATIONAL LIBRARY OF MEDICINE HAVE OCCASIONAL MEETINGS WHICH THE FRIENDS BRING TOGETHER PUBLISHERS OF DIFFERENT LEVELS FROM SOCIETY OF PUBLISHERS TO SOME BIG AND WELL KNOWN PARTNERS LIKE ELSEVIER FOR SHARING CONVERSATIONS LIKE THIS. WE DON'T HAVE A SPECIFIC PUBLISHER ADVISORY GROUP TO US, WE HAVE DIFFERENT WAYS OF INTERACTING CONVERSATION WITH THEM. >> OKAY. THANK YOU. WE'RE FINISHED WITH OUR REPORT OUTS AND WE ARE NOW TIME ON OUR AGENDA TO MOVE TO HAVING THE ACTUAL VOTE ON THE NEW COLLECTIVE POLICY. ARE THERE ANY REMAINING QUESTIONS BEFORE I ASK FOR A MOTION? >> WE'VE GOT A QUORUM UNLESS YOU CALL ONE. >> ALL RIGHT, WE'VE GOT A QUORUM. THANK YOU, DALE. >> OKAY. COULD I HAVE A MOTION PLEASE? >> MOVE WE ADOPT THE POLICY AS RECOMMENDED BY THE WORKING GROUP. >> SECOND. >> SECOND? >> CARLOS. >> ALL IN FAVOR? ANY NOs? >> (INAUDIBLE). >> OKAY, THE MOTION HAS CARRIED. NOW WE -- >> CONGRATULATIONS. >> TWO VOTES IN ONE DAY. >> WE WON'T MAKE YOU VOTE AGAIN FOR FOUR MORE YEARS. >> THE LAST VOTE WAS THE STRATEGIC PLAN. >> DO WE HAVE OUR PHOTOGRAPHER HERE? >> LET ME SEE. >> THERE WE GO. >> THIS IS THE FUN PART OF THE MEETING. IT'S NOW TIME TO HAND OUT THE 2019 REGENTS AWARDS, SCHOLARSHIP FOR TECHNICAL ACHIEVEMENT, I'LL BE READING THIS. FOR SCHOLARSHIP TECHNICAL ACHIEVEMENT RECOGNIZED IN 1970, LEADING TO SCHOLARLY AND/OR TECHNICAL ACHIEVEMENTS ENRICHING BIOMEDICINE, EMPHASIS PLACED ON ORIGINALITY AND NEW PRODUCT OR METHODOLOGY THAT IMPROVES BIOMEDICAL KNOWLEDGE OR SERVICE. THE BOARD CONSIDERS THIS AWARD THE HIGHEST HONOR THAT IT CAN GIVE TO A MEMBER OF THE STAFF. PLEASED TO ANNOUNCE THE WINNER OF THE 2019 REGENTS AWARD, DR. WILLIAM KLEMKE. [APPLAUSE] I WILL TELL YOU JUST A LITTLE BIT MORE ABOUT HIM AND THEN WE'LL HAND OVER THE AWARD. HE IS A STAFF SCIENTIST AT THE INFORMATION ENGINEERING BRANCH OF THE NATIONAL CENTER FOR BIOTECHNOLOGY INFORMATION, RECOGNIZED FOR TECHNICAL AND INTELLECTUAL EXCELLENCE IN THE DEVELOPMENT OF IMPROVED METHODOLOGY USING GENOME SEQUENCE DATA FOR PATHOGEN DETECTION REALTIME SURVEILLANCE SYSTEM. UNDER THIS PROGRAM, THE FDA, CDC AND OTHER ORGANIZATIONS SUBMIT TO NLM THE WHOLE GENOME SEQUENCES OF BACTERIAL PATHOGENS FOUND IN FOOD, THE ENVIRONMENT AND PATIENTS. NLM PROCESSES THOSE, IDENTIFYING CLOSELY RELATED SEQUENCES AND ASSISTS IN MAKENING EPIDEMIOLOGICAL CONNECTIONS BETWEEN PEOPLE WITH FOOD BORNE DISEASE AND SOURCE OF DISEASE. PUBLIC HEALTH OFFICIALS NEED THIS AS QUICKLY AS POSSIBLE IN ORDER TO MINIMIZE MORBIDITY FROM FOOD-BORNE DISEASE OUTBREAKS. HIS TEAM TACKLED COMPLEX SCIENTIFIC AND TECHNICAL PROBLEMS TO BUILD A REALTIME SURVEILLANCE RESPONSE SYSTEM CAPABLE OF DELIVERING PRELIMINARY RESULTS WITHIN ONE HOUR, AND REFINED RESULTS WITHIN 24 HOURS. LET'S EXTEND OUR WARM CONGRATULATIONS TO DR. KLEMKE FOR THIS EXCELLENT ACHIEVEMENT. [APPLAUSE] >> SMILE. COME ON. [LAUGHTER] >> THAT'S GOOD. >> YOU WON AN AWARD, YOU GOT TO SMILE. [APPLAUSE] >> THANK YOU. >> LET ME THANK YOU FOR TAKING OVER AS CHAIR FOR THIS DAY. I APPRECIATE YOU STEPPING IN WITH VERY LITTLE NOTICE. >> WELL, THANK YOU. WITH THAT I CAN SAY THE MEETING IS ADJOURNED.