>> GOOD MORNING I WANT TO WELCOME TO YOU DAY ONE OF NIH BIG DATA TWO TO KNOWLEDGE WORKSHOP, THE BEST PRACTICES TO SUPPORT HEALTHY DEVELOPMENT. FOR THOSE WHO ARE JOINING US BY WEBCAST, YOU CAN USE THE #BD2K FOR QUESTIONS AND COMMENTS AND FOR THOSE IN THE ROOM, I ASK YOU TO PLEASE USE MICROPHONES. WE WILL HAVE THOSE FOR THE PANELISTS AND WE WILL HAVE A ROVING REPORTER WITH A MICROPHONE FOR THE QUESTIONS. I'M CINDY LAWLER AND I LEAD ONE OF THE BD2K WORK GROUPS THAT ORGANIZED TODAY'S MEETING AND I HAD LOTS OF HELP FROM THAT GROUP ESPECIALLY ASTRID HOWGEN, AND I WANT TO THANK THE TWO CO CHAIRS, AND I WANT TO INTRODUCE PHIL BOURNE NOW AND HE WILL PLACE THE WORKSHOP TODAY IN THE BIGGER CONTEXT OF THE LARGER NIH BD2K INITIATIVE. AS ASSOCIATE DIRECTOR OF DATA SCIENCE AT NIH, HE'S REALLY CHARGED WITH IDENTIFYING OPPORTUNITIES ACROSS THE DIFFERENT INSTITUTES TO ADVANCE THE USE OF BIG DATA TO SUPPORT SCIENTIFIC DISCOVERY. PRIOR TO JOINING NIH, PHIL WAS ASSOCIATE VICE CHANCELLOR FOR INNOVATION AND INDUSTRY ALLIANCE AND A PROFESSOR OF PHARMACY AND PHARMACEUTICAL SCIENCES AT THE UNIVERSITY OF CALIFORNIA AT SAN DIEGO. ALSO HAD AN ADJUNCT APPOINTMENT AT THE SAN BERNARDINO INSTITUTE. HE WAS LECTED TO THE ADVANCEMENT OF SCIENCE AND PAST PRESIDENT OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION, HE IS VERY WIDELY RECOGNIZED EFFORTS TO FOSTER FREE AND OPEN EXCHANGE OF SCIENTIFIC DATA, THE FOUNDING EDITOR OF OPEN ACCESS, COMPUTATIONAL BIOLOGY JOURNAL AND FITTINGLY HE WAS AWARD INDEED 2009 THE BENJAMIN AWARD, THAT RECOGNIZES SCIENTIST WHO IS EPITOMIZE SPIRIT IN WHICH FRANKLIN REFUSED TO PATENT HIS INVENTION, SO THANKS FOR JOINING US TODAY, PHIL. >> OKAY, THANKS I THINK THE INTRODUCTION IS LONGER THAN WHAT I'M GOING TO HAVE TO SAY. BUT IT JUST OCCURRED TO ME THAT ONE THING THAT WASN'T SAID WAS ACTUALLY MY OWN STANDARDS AND I WOULD LIKE TO BRIEFLY REFLECT ON THAT BECAUSE I KIND OF WANT YOU TO KNOW WHAT WE REALLY FEEL FOR WHAT YOU'RE DOING AND I THANK YOU ALL FOR BEING HERE AND I KNOW THAT DOING THIS KIND OF WORK IS VERY CONSUMING AND IT IS OF ENORMOUS VALUE TO THE COMMUNITY AND MY OWN GOES BACK A LONG WAY AND TO SORT OF ILLUSTRATE THE KINDS OF THINGS YOU'RE DISCUSSING TODAY. SO MANY YEARS AGO, I WAS INVOLVED WITH A STANFORD EFFORT AROUND MOLECULAR DATA AND THAT WAS DONE UNDER THE AUSSPICES UNDER THE CRYSTALLOGRAPHY SO IT'S PART OF THE MOTIVATION FOR WHY THESE THINGS GET DONE AND IN THIS CASE, IT WAS ACTUALLY DRIVEN BY SOCIETY WHICH I THINK IS AN INTERESTING ASPECT IN ITSELF. WE ACTUALLY WENT BASED OFF OF WHAT WAS ALREADY DONE FOR SMALL MOLECULES AND STARTED APPLYING IT TO MACROMOLECULES. WHEN WE STARTED THIS, WE THOUGHT IT WOULD ACTUALLY--WE WERE MEETING ON SUNDAY AFTERNOON, ACTUALLY AND I WAS AT COLUMBIA UNIVERSITY IN NEW YORK AND HAD ANOTHER MAJOR PERSON IN THIS WAS AT RUTGERS AND WE WOULD OSCILLATE AROUND AND WE WOULD MEET ON SUNDAY AFTERNOONS AND WE THOUGHT IT WOULD TAKE A FEW SUNDAY AFTERNOONS DURING THE COLD OF WINTER WEATHER TORE GET THIS RIGHT. I ACTUALLY ENDED UP TAKING SEVEN YEARS TO EVEN GET THE FIRST VERSION OUT OF THE DOOR. AND I THINK THERE'S A LESSON TO BE HAD THERE. I THINK THE SCOPE OF WHAT WE WERE DOING AND THE COMPLEXITY OF WHAT WE WERE DOING, WE DIDN'T PHASE IT WELL ENOUGH AND SO, THERE'S GOOD NEWS IN ALL THIS AS WELL. BUT IT WAS ALSO ULTIMATELY TURNED OUT IN LARGE PART TO THE WORK OF JOHN WESTBROOK AT RUTGERS TO BE VERY ELOQUENT. IT WAS SELF-DEFINING AND LOOKING AT WHAT WE CREATED A POINT OF VIEW, THE DATA FILE HAD NAMES AND PAIRS, THOSE NAMES WERE DEFINE INDEED A DICTIONARY WHICH WAS ALSO BASED ON THE SAME REPRESENTATION AND THEN THERE WAS A DICTIONARY DEFINITION LANGUAGE WHICH ACTUALLY FOUND DICTIONARYS ITSELF, SO YOU COULD WRITE TOOLS THAT WOULD ACTUALLY READ THE DICTIONARY DEFINITION, POPPAULATE A DICTIONARY AND THEN USE THAT DICTIONARY TO ACTUALLY DO ALL SORTS OF THINGS INCLUDING VALIDATION OF THE DATA FOR OURSELVES, NO ONE EXCEPT WE THEN BECAME THE PEOPLE RESPONSIBLE FOR THE RCSB PROTEIN DATA DECK, NO ONE USED THIS STANDARD EXCEPT US AND I THINK, PARTLY AS I SAID ALL RIGHT, PARTLY IT WAS THAT WE DIDN'T HAVE ENOUGH GOOD SOFTWARE TOOL TO USE THE STANDARD EFFECTIVELY WHEN WE BEGAN. HOWEVER ALL THAT SORT OF BAD NEWS IN A SENSE AND LESSONS LEARNED THE GOOD NEWS WAS THAT IT BECAME THE FOUNDATION FOR WHAT BECAME--I THINK REGARDED AS ONE OF THE SORT OF EXEMPLARS FOR THE BIOLOGICAL DATABASES IN THE BIOMEDICAL SCIENCES AND IT'S REALLY BECAUSE OF THAT STANDARD REPRESENTATION, IT WAS ONLY PROBABLY IN THE LAST YEAR OR SO THAT THE COMMUNITY IS REALLY GRABBED ON TO THIS IS THE STANDARD REPRESENTATION, IN PART BECAUSE OF THE WAY TAKEN--THEY DATA WAS BEING REPRESENTED BY THE EFFECTIVE STAN ARD OF TIME WHICH WAS ESSENTIALLY THE FORMAT ON A FORMAT, YOU CAN GUESS WHEN, THAT JUST NO LONGER WAS CAPABLE OF ACTUALLY DESCRIBING THE DATA IT WAS MEANT TO DESCRIBE. SO THAT PUSHED PEOPLE FINALLY INTO THINKING ABOUT HOW TO DO THIS AND THE COMMUNITY IS NOW CAUGHT UP. SO I THINK THERE'S LOTS OF LESSONS LEARNED THERE AND I'LL BE LISTENING CAREFULLY. I CAN'T BE HERE MUCH TODAY BUT I WILL BE HERE MOSTLY TOMORROW BUT I'LL LISTEN TO THE WEBCAST AS MUCH AS I CAN TO SEE HOW THESE ISSUES GET ADDRESSED SO BUT LET ME QUICKLY SAY WHAT THE OBJECTIVES HERE ARE. THESE ARE THE GOALS WRITTEN IN YOUR HANDOUT, YOU COME HERE KNOWING ALL THAT SO I WON'T GO REALLY INTO THAT. I WOULD SAY THAT WE'RE LOOKING TO TAKE THIS SORT OF STANDARD EFFORTS TOGETHER IN A MINUTE TO ACTUALLY FOSTER THIS ACTION PRETTY CLOSE AFTER THIS MEETING. SO WHATEVER TRANSPIRES HERE AND WHAT WE'VE GAINED ALREADY WILL LEAD TO SOME DEVELOPMENTS THAT I'M STILL SOME OF YOU ACTUALLY RESPOND TO BUT CERTAINLY THAT WE HOPE WILL FACILITATE THE COMMUNITY. SO WHERE DO THOSE KINDS OF THINGS FIT INTO THE BIG PICTURE OF WHAT WE'RE DOING HERE AT THE NIH. SO I'LL QUICKLY GO OVER THAT, I WANT TO SPEND A LOT OF TIME ON THAT AND I WILL BE HAPPY TO TALK TO YOU MORE ABOUT IT BUT WE'RE KIND OF WORKING ON THE OVERALL ASSUMPTION THAT GOES BACK FROM THE WORK OF A WORKING GROUP THAT REPORTED TO DIRECTOR FRANCIS COLLINS WHICH LED TO MY AREA HERE OF CONSIDERING HOW WE THINK ABOUT DATA SCIENCE AT THE NIH AND WHAT WE'RE TRYING TO DO, IT'S A GRANDIOSE TRY, IS TO ESSENTIALLY DEVELOP AN ECOSYSTEM AND WE'RE WORKING REALLY HARD TO AND THERE ARE MANY PEOPLE IN THIS ROOM FROM NIH WHO ARE INVOLVE WIDE THIS AND MANY OTHERS AROUND WHO ARE TRYING TO FACAS TILL STATE AS WELL HOW WE DO THAT, THE CENTRAL ELEMENT OF THIS IS THE IDEA OF CREATING A COMMONS AND NOT EVERYTHING OBVIOUSLY WOULD BE IN THAT COMMONS AND THERE'S A NONCOMMONS PART TELEVISION AND THAT COMMONS AND THE ECOSYSTEM AT LARGE CONTAINS A SET OF COMPONENTS THAT FORM THE RESEARCH LIFE CYCLE SO THEIR EFFECTIVELY, VARIOUS RESEARCH OBJECTS WITHIN THIS ENVIRONMENT THAT'S CURRENTLY BEING STOOD UP AND WE CAN TALK MORE ABOUT THAT OFFLINE AT SOME POINT. BUT THERE'S STANDARDS AND STANDARDS ARE AN IMPORTANT PORT FOR WILL FOUNDATION FOR ALL OF THIS. SO THIS ECOSYSTEM IS FED BY THE WORK OF NIH BUT ALSO AT THE COMMUNITY AT LARGE, SO THERE ARE CERTAIN DRIVERS FOR ALL OF THIS, OBVIOUSLY TRAINING OF A NEW WORKFORCE IS IMPORTANT TO THIS, POLICIES IN DATA SHARING IN PARTICULAR, BUT OTHER POLICIES AS WELL, THAT EFFECT THE ECOSYSTEM, THERE'S DEVELOPMENTS GOING ON THERE, SUSTAINABLE. THAT'S CLEARLY SOMETHING WE PAY A LOT OF ATTENTION TO BECAUSE WHAT WE'RE DOING RIGHT NOW JUST DOES NOT SCALE GIVEN THE GROWTH OF DATA AND THE FLAT BUDGET WE REALLY NEED TO BE THINKING ABOUT THINGS IN A DIFFERENT WAY. AND OF COURSE ALL OF THIS IS TO FOSTER A COLLABORATION IN THESE WORK SPACES AND ELSEWHERE. THERE IS ALSO PROTERBATIONS IN THE SYSTEM GOING ON. THE NATIONAL SYSTEM OF MEDICINE DR. LINDBERGH HAS ANNOUNCED HIS RETIREMENT, SO THERE'S A NEW DIRECTOR OF THE NIH--THAT PROVIDES OPPORTUNITY FOR THINKING ABOUT NLM'S ROLE IN THIS, NLM HAS BEEN ENORMOUSLY IMPORTANT TO ALL OF US OVER THE YEARS AND HOW CAN THAT FURTHER BE ENHANCED. THAT'S CURENDLY UNDER DISCUSSION AND THEN THE BD2K INITIATIVE ITSELF WHICH IS SOFTWARE, LOTTINGS OF PARTS OF THIS AND WHAT YOU'RE DOING THE STANDARDS PART BUT THERE'S OTHER COMPONENTS OF THIS SO THERE'S INITIATIVES AROUND THE CENTERS. WE FUNDED 12 CENTERS AND EACH OF THOSE WILL BE PILOTS WITHIN THIS COMMONS ENVIRONMENT AND WE FUNDED A DATA DISCOVERY INDEX CONSORTIUM WHICH IS CHARGED WITH ACTUALLY FINDING THIS ENVIRONMENT IN THIS CONTENT SO IT CAN BE FOUND AND ACCESSED FURTHER. AND OF COURSE ALL OF THIS DEPENDS ON THE STANDARDS AND SOFTWARE WHICH IS ALSO SOMETHING ELSE WHERE THERE WILL BE CALLS SHORTLY. SO JUST TO SAY THERE HAS BEEN PRIOR WORK AND EFFECTIVELY TWO PARTS OF WHAT'S GOING ON HERE, THERE'S A FRAMEWORK TO SUPPORT STANDARDS AND THEN THERE'S THE STANDARDS THEMSELVES AND THIS IS--YOU KNOW SOME WAYS I CAME TO THIS A LITTLE LATE SOME OF THESE WERE UNDERWAY WHEN I CAME HERE IN MARCH LAST YEAR, ALMOST A YEAR AGO NOW. SO, YOU KNOW WE'RE--WE'RE KIND OF RETROFITTING AND MOVING THINGS AROUND A BIT BUT NOT WITHSTANDING, YOU KNOW THESE ARE ALL IMPORTANT INITIATIVES AND DEVELOPMENTS. SO JUST TO SUMMARIZE, WE HAD PRIOR WORK AROUND THE FRAMEWORK, THAT START WIDE A WORKSHOP IN SEPTEMBER 2013 AND THEN MORE RECENTLY REQUEST FOR INFORMATION WHICH WAS VERY DIGESTED QUITE SIGNIFICANTLY AND WE'RE--SORT OF AT THE POINT WHERE WE'RE GOING TO DECIDE WHAT TO DO. S THAT DECISION IS GOING TO BE VERY MUCH INFORMED BY THIS WORKSHOP AND I SHOULD SAY THAT WE WILL BE ISSUING SOME FUNDING OPPORTUNITIES FOR THIS. I WOULD SAY THAT I THINK THIS ALL HAS TO COME TOGETHER AND I THINK IT ALSO HAS TO REALLY TAKE INTO CONSIDERATION WHAT ELSE IS GOING ON OUT THERE IN THE COMMUNITY. I MEAN I THINK THIS WAS--WHEN I LOOKED AT WHERE THINGS STOOD, I THOUGHT THERE WAS REALLY A NEED TO PAY ATTENTION TO OTHER ASPECTS OF WHAT'S GOING ON IN THE COMMUNITY AROUND FRAMEWORKS AND AROUND STANDARDS THEMSELVES. SO WHATEVER COMES OUT OF THIS, IS GOING TO BE THERE TO ENABLE WHAT IS ALREADY BEING DONE. IT'S NOT IN ANY WAY SHOULD BE CONCEIVED AS COMPETITIVE OR ANYTHING LIKE THAT, I ALSO POINT OUT THAT I SEE VERY MUCH AS AN INTERNATIONAL INITIATIVE. SO IT'S REALLY ALL I'M GOING TO SAY EXCEPT TO SAY THERE'S LOTS OF PEOPLE WITH RESPECT TO THE STANDARD OF INJURY THAT'S GONE ON IN THIS SORT OF FRAMEWORK WORK THAT'S LED BY SHERRY DECOR MAD O AND OF COURSE WHAT YOU WILL HEAR TODAY WHICH IS LED BY CINDY AND IT'S JUST--YOU JUST HEARD FROM. SO THE PERSPECTIVE IS, IT WILL INFORM US FAIRLY SOON FOR THE NEW FUNDING CALLS AND JUST IN CLOSING I WANT TO THANK SYND SCHEASTRID WHO GOTITOUS THIS POINT AND LET'S GIVE A ROUND OF APPLAUSE FOR THEM. [ APPLAUSE ] SO, THANK YOU. DO I HAVE ANY QUESTIONS, BRIEF QUESTIONS? STRAIGHT FORWARD AND OBVIOUS, I WILL BE AROUND AND I WILL BE HAPPY TO CHAT WITH YOU. >> HI, EVERYONE, THANKS SO MUCH FOR COMING. THANK YOU PHIL FOR KICKING US OFF, ALREADY I FEEL LIKE MY OWN STANDARDS WORK AS PART OF A GREATER ECOSYSTEM, SO THANKS FOR THAT INSPIRATION. SO DESPITE THE FACT THAT THEIR ENDED UP BEING MORE OF US HERE TODAY THAN MAYBE WE ANTICIPATED AT FIRST AND SITTING CLASSROOM STYLE. YOU'RE ALL INVITED HERE TO REALLY SPEAK YOUR MINDS AND ARGUE WITH ONE ANOTHER AND BRING YOUR PERSPECTIVES TO BEAR, SO I'M REALLY HOPING I CAN INSPIRE ALL OF YOU TO ACTIVELY PARTICIPATE IN THE CONVERSATION, WE'VE TRIED TO LIMIT THE TALKS TODAY SO WE CAN LEAVE AS MUCH TIME AS POSSIBLE FOR DISCUSSION AND FOCUSING ON GETTING DOWN AND WRITING IN THE GOOGLE DOC FOR HOW DO WE MOVE THIS FORWARD SO NIH IS REALLY LOOKING FORWARD FOR THE ADVICE FOR THE APPROACH THAT WILL ACTUALLY CONTINUE THIS IDEA THAT STANDARDS ARE REALLY PART OF THE FABRIC OF OUR DATA LANDSCAPE AND WE NEED TO THINK ABOUT HOW TO MAKE THAT PROCESS OF THEIR DEVELOPMENT MUCH MORE EFFICIENT. SO IT IS MY GREAT PLEASURE TO INTRODUCE PETER FOX TODAY PETER IS A PROFESSOR OF EARTH AND ENVIRONMENTAL SCIENCE, COMPUTER SCIENCE ASK COGNITIVE SCIENCE AND THE INFORMATION DIRECTOR OF WEB SCIENCE PROGRAM AND THE POLYTECHNIC INSTITUTE. HE HAS A Ph.D. IN APPLIED MATHEMATICS AND HIS RESEARCH IS IN OCEAN OCCASION AND ENVIRONMENTAL SEMANTICS AND SOLAR AND TERRESTRIAL PHYSICS SO WHY IS PETER HERE TODAY. PETER IS HERE TODAY BECAUSE HE HAS HAD AN ENORMOUS AMOUNT OF EXPERIENCE IN LEADING EFFORTS AND STANDARDS DEVELOPMENT AND DATA INTEGRATION TECHNOLOGIES IN THE EARTH SCIENCES AND BECAUSE I SAT NEXT TO HIM ON THE BUS FOR A VERY LONG BUS RIDE WAY BACK WHEN AND WAS INSPIRED BY HIS IDEAS ABOUT HOW COMMUNITIES WORK TOGETHER ABOUT HOW PEOPLE WORK TOGETHER TO REALLY BRING TOGETHER ALL THE DIFFERENT PERSPECTIVES THAT HELP MAKE SCIENCE MOVE FORWARD IN A MORE EFFICIENT AND EFFECTIVE FUN WAY, SO IT IS WITH GREAT PLEASURE THAT I WELCOME PETER. >> THANKS AND GOOD MORNING. I APPRECIATE THE INVITATION TO BE HERE SO MY JOB TODAY IS SORT OF NOT AS MY DAY JOB AT LPI, BUT I AM CURRENTLY PRESIDENT OF THE EARTH SCIENCES INFORMATION PARTNER WHICH IS IS A FEDERATION, COMMUNITY BASED ORGANIZATION, I WILL TELL YOU A BIT ABOUT THAT, A BUNCH OF MATERIAL TO GO THROUGH HERE, BUT TO SET THE CONTEXT HERE ESIP, WHICH I WILL REFER TO THE ORGANIZATION AS WAS FORMED OUT OF A NATIONAL ACADEMY REPORT. SOME OF YOU MAY HAVE REMEMBERED THE NASA MISSION WHICH IS AN OBSERVING SYSTEM WHICH WAS LED AS A LARGE CONTRACT TO A--TO A BELT WAY BANDIT WHICH SORT OF WENT VERY, VERY WRONG AND DIDN'T DELIVER ANYTHING AND LOTS OF DOLLARS WENT ASTRAY AND A BIG INVESTIGATION OF IT LED TO A NATIONAL COUNSELSIT RESEARCH REPORT THAT SAID YOU SHOULD BREAK IT UP INTO A DATA DISTRIBUTED SYSTEM AND THAT IS IN THE MID90S SO WHAT HAPPEN SYSTEM THAT NASA IN THIS THE SOMEWHAT WISDOM AT THE TIME SAID WE SHOULD NOT DO THE INSIDE NASA, WE SHOULD PUSH THIS OUT INTO THE COMMUNITY AND ASK OUR INVESTIGATORS TO WORK COLLECTIVELY AND THUS THE 24 INITIAL FUNDED ACTIVITIES INSIDE THIS OBSERVING SYSTEM D. I. S. DISTRIBUTED INFORMATION SYSTEM WERE PUT TOGETHER IN THE ESIP ORGANIZATION, SO 12 RESEARCH AND 12 APPLICATION AREAS. AND OVER TIME, SO 17 YEARS YEARS LATER I'M GOING TO SORT OF TELL YOU WHERE WE GOT TO, EARLY DAYS WERE A LITTLE BUMPY FORCING PIs TO WORK TOGETHER AND COLLABORATE WAS SORT OF AN INTERESTING EXERCISE, THEY DIDN'T WANT TO BE GOVERNED OR TOLD WHAT TO DO SO IT WAS VERY MUCH A PERCOLATION OF THING I THEY WERE DOING SO THROUGH THE EARLY 2000S THERE WERE LIGHT WEIGHT FORMALIZATIONS OF THE ORGANIZATIONS SO THERE WAS A CONSTITUTION AND A BYLAW, THERE WAS A FORMATION OF A FOUNDATION FOR EARTH SCIENCE WHICH IS THE LEGALLENTITY THAT THEN RELATES TO THE COMMUNITY AND TO THE AREA WHICH IS ESIP AND I'LL SHOW YOU THAT A BIT MORE. SINCE THEN IT'S BEEN AN INTERESTING RIDE, I'VE BEEN INVOLVED FOR THAT THE LAST 10 YEARS OF ESIP AND SO WE'LL TELL YOU A LITTLE BIT ABOUT IT BECAUSE THAT'S--THATTA WHY I WAS INVITED. WE'VE HAD ENOUGH TO TIME SORT OF THINK ABOUT LOOKING BACK WHERE WE GOT TO WHERE WE WERE GOING SO THAT'S VISION. SO THE LEADERSHIP AND COLLECTION AND USE OF EARTH SCIENCE DATA SO IF YOU WANT YOU CAN SUBSTITUTE SCIENCE WITH YOUR PARTICULAR APPLICATION AREA. THIS IS EVOLVED OVER TIME, THE EARLY DAYS, THE RESPONSIVENESS TO SOCIETAL NEEDS DIDN'T REALLY EXIST, RIGHT? IT WAS REALLY A SCIENCE PURSUIT BUT THIS HAS BECOME PART OF THE MISSION OF MANY GOVERNMENT AGENCIES AND SO IN SOME SENSE, THE VALUE CHAIN BETWEEN THE GENERATION OF THE DATA ITSELF THROUGH THE USE OF THAT DATA IN COMMUNITIES, RESEARCH COMMUNITIES AND SO ON, INCLUDING EDUCATIONAL COMMUNITIES OUT TO THE GOOD OF THE GENERAL PUBLIC AND REALLY PART OF THE VISION OF ESIP. THE MISSION ITSELF, SO THE THINGS THAT WE DO AS AN ORGANIZATION, IS REALLY PROMOTED THIS SHARED AGENDA, PEOPLE COME TOGETHER AND FINALLY REALIZE THAT AS A GROUP WE CAN DO THINGS BETTER TOGETHER THAN WE CAN SEPARATEERATELY SO NETWORKING DISSEM NATION, MEMBERSHIP, GLOBAL COMMUNITY, LINKING ALL THE DIFFERENT AREAS IN SUPPORT OF EARTH SCIENCE. SO ESIP IS A VERY BROAD BASE DISTRIBUTED COMMUNITY AND IT'S NOT BOTH THE DATA, PRACTITIONERS AND THE TECHNOLOGY PRACTITIONERS AND IT'S INTERMINGLED AS WELL AS RESEARCHERS AND LEVEL OF SCIENTIST AS WELL AND THAT HIS LITTLE SORT OF NETWORK DIAGRAM OF ESIP ITSELF AND I WILL COVER SORT OF WHETHER YOU LIKE IT OR NOT, IN DETAIL SOME OF THE ACTIVITIES THAT ESIP DOES BUT WE HAVE A NUMBER OF PARTNERS IN THIS ACTIVITY WHICH ARE IN THEMSELVES NETWORKS AS WELL. SO ESIP IS THIS PLACE WHERE A VARIETY OF DIFFERENT LEVELS OF ORGANIZATIONS COME TOGETHER, ALL THE WAY FROM SORT OF SINGLE INVESTIGATOR PROJECT TEAMS, ALL THE WAY UP TO VERY BIG PROJECTS UP TO THE NATIONAL ECOLOGICAL OBSERVING NETWORK, AND GENT AGENCIES WE QUICKLY EXPANDED BEYOND NASA AND THE WAY YOU SEE SPONSORING AGENCIES AND WE HAVE THE U.S. GEOLOGICAL SURVEY, ENVIRONMENTAL PROTECTION AGENCY, THE U.S. DEPARTMENT OF AGRI CULTURE AND SO ON AND SO ON. AND WE'RE ALSO AFFILIATED WITH OTHER PROFESSIONAL ORGANIZATIONS THAT ARE BOTH NATIONAL AND INTERNATIONAL. SO THIS IS SORT OF WHAT OUR ECOSYSTEM LOOKS LIKE, SO SORT OF INCAPSULATED INSIDE THIS DIAGRAM IS THE HEAVY EMPHASIS ON THE DATA AND TECHNOLOGY AND TOOLS BUT ALSO THE USERS ARE INSIDE THIS ECOSYSTEM WHICH I SINGLE A REALLY IMPORTANT THING AND IN A VERY SUBSTANTIAL WAY AND AROUND THE OUTSIDE, THE SECTORS IF YOU LIKE THAT WE--THAT WE REACH INTO SO MANY EVEN THOUGH IT'S EARTH SCIENCES, EARTH SCIENCES IS A HIGHLY INTERDISCIPLINARY FIELD BY ITSELF, THERE'S A STRONG ACADEMIC COMPONENT SO IT'S A PLACE FOR THE ACADEM AND I CAN THE AGENCY, THE MISSION AGENCIES IF YOU LIKE, THE ONES WHO ARE TASKED WITH CARRYING OUT OBSERVATIONS AND SIMULATIONS AND SO ON, INTERSECT BUT ALSO THERE ARE CORPORATE PARTICIPANTS ALL THE WAY FROM VERY SMALL START UP COMPANIES THAT ARE WORKING IN THE SPACE ALL THE WAY TO THINGS LIKE MICROSOFT RESEARCH AND SO ON. THE WAY IN WHICH WE'RE ORGANIZED SORT OF PARTLY INFLUENCED OR BASED ON THE INITIAL POPULATION OF ESIP IS AROUND FOUR TYPES OF MEMBERS. THE TYPE FOUR ALSO DEAL WITH FIRST WE'LL CALL THEM STRATEGIC PARTNERS, THAT MEANS THEY GIVE US MONEY, THEY GIVE US MONEY IN A COOPERATIVE AGREEMENT, TO BASE FUNDS ACTIVITIES THAT SEE DEVELOPMENT IN THE COMMUNITY AS A WHOLE AND THEN THREE MAIN TYPES, SO TYPE ONE OF THE DATA CENTERS, TYPE TWO ARE RESEARCH AND TOOL DEVELOPERS WHICH IS WHERE I FALL AND THEN TYPE THREE ARE APPLICATION DEVELOPERS AND IN PARENTHESES THERE IS THE FIRST NUMBER IS THE CURRENT NUMBER OF ACTIVE GROUPS INSIDE ESIP AND THE SECOND NUMBER IS THE TOTAL NUMBER OF MEMBERS IN THAT TYPE SO SORT OF AN INTERESTING STATISTIC HERE THE TYPE ONES SO THESE ARE INTERACTIVE AND THE LABS DATA CENTERS SO THESE ARE IMPORTANT BECAUSE THE DATA CENTERS FIND IMMENSE VALUE IN PARTICIPATING ESIP FOR A VARIETY OF REASONS. SO I'LL TO THOSE AS WE GO ALONG, SO THE TYPE TWOS YOU'LL SEE A LITTLE OVER HALF, BUT STILL ACTIVE AND ULRICH YOU'LL SEE THE NUMBERS ARE QUITE A BIT SMALLER AND REASON FOR THAT IS AS IS STILL GROWING ORGANIZATION, WE HAVEN'T FIGURED OUT A WAY TO DROP MEMBERS OFF WHEN THEY'RE NO LONGER ACTIVE SO THESE MEMBERS IN ACTIVE AND RESEARCH APPLICATIONS THAT HAVE BEEN GIVEN GRANTS FOR THOSE PROJECTS AND THOSE ARE NO LONGER ACCOUNTIVE AND NO LONGER FUNDED TO SHOW UP BUT IT GIVES YOU A SENSE OF DIT VERSITY OF THE ANNUAL BASIS. THERE'S ASSEMBLY IN JANUARY OR WASHINGTON D. C. AND OF THE--OF THE MEMBERS ONE PARTNER, ONE VOTE, AND THE LEADERSHIP OF ESIP IS PURELY VOLUNTEER LEADERSHIP. SO I GOT TALK INTOED BEING PRESIDENT WHICH IS GOOD. STRUCTURALLY, ESIP IS STRUCTURED AROUND GROUPS AND THERE ARE REALLY--SORRY THREE TYPES OF GROUPS. THERE ARE COMMITTEES, THERE ARE WORKING GROUPS AND THERE ARE CLUSTERS AND WITHIN THE COMMITTEES THERE ARE TWO TYPES OF COMMITTEES. THERE ARE STANDING COMMITTEES THAT ARE DIRECTLY AIMED AT KEY TOPICS IN THE COMMUNITY AROUND DATA AND TECHNOLOGY SO STEWARDSHIP, EDUCATION, I.T. AND INTEROPERABILITY AND PRODUCTS AND SERVICES AND I'VE GOT SLIDES ON EACH OF THESE TO SHOW YOU AND GIVE YOU A FEEL FOR THE TYPE OF ACTIVITIES AND GO ON, AND YOU WILL SEE THERE ARE THREE COMMUNITIES, THERE'S A RELATIVELY LOW ADMINISTRATIVE OVERHEAD AND WE NEED TO MAKE SURE WE'RE DOING THINGS IN WHICH OUR SPONSORS WANT AND WATCH THE BUDGET AND THE PARTNERSHIP COMMUNITY WHICH IS LOOKING AT NEW MEMBERS COMING INTO THE ESIP AND WHAT WE PROVIDE FOR THEM. THE STANDING COMMITTEES ARE ESTABLISHED BY A COMMITTEE AS A WHOLE. LONG LIVED. THE WORKING GROUPS, THERE'S FOUR OF THOSE AND I'LL FOCUS A LITTLE BIT ON THE QUALITY WORKING GROUP BECAUSE IT'S AN INTERESTING CASE STUDY, THESE ARE FORMED BY THE EXECUTIVE COMMITTEE AND I'LL SHOW YOU THE EXECUTIVE COMMITTEE AT THE END, SO THESE HAVE A FAIR AMOUNT OF AGILITY, THEY'RE LONGER LIVED BUT THEY ALSO DO COME AND GO VISIONEERS SET THE THEME FOR THESE AND SCENES COME AND GO, TOPICS COME AND GO AND THEN ON THE RIGHT IS SORT OF THE REALLY FUN ACTIVITY WHICH ARE THE CLUSTERS. CLUSTERS ARE FORMED PRETTY MUCH INSTANTANEOUSLY, IF YOU HAVE AN IDEA AND YOU WANT TO START A DISCUSSION, YOU CAN FORM A CLUSTER JUST BY SAYING TO OUR SCRIES PRESIDENT, I'M GOING TO FORM THIS CLUSTER AND OVER TIME YOU'VE HAD MANY CLUSTERS AND THIS A SAMPLE OF THE CLUSTERS ARE BACK IN 2006, I FORMED THE SEMANTIC AND THERE'S A VARIETY OF TOPICS THAT YOU CAN SEE THERE WHICH WHICH REALLY ARE MIX OF DATA AND TECHNOLOGY, TINY LITTLE BIT ON THE RELATIONSHIP BETWEEN THE FEDERATION AND THE FOUNDATION SO THE FEDERATION IS THE COMMUNITY AND THE FOUNDATION PROVIDES A LOGICICAL SUPPORT. AND THE INTERACTION BETWEEN THERE IS DEFINED BY THE SERVICE AGREEMENT SO THE WAY IN WHICH MONEY FLOW SYSTEM WELL-DEFINED. THE FOUNDATION IN TURN AND IT IS COMPRISED OF TWO PEOPLE, EXECUTIVE DIRECTOR AND A COMMUNITY, A COMMUNITY MANAGER, SO THE VERY LIGHT WEIGHT ORGANIZATION AND SO THOSE--THOSE ARE THE STAFF AND THE WAY IN WHICH THIS COALITION WORK SYSTEM WE HAVE THESE BIANNUAL MEETINGS IN THE WINTER AND IN THE SUMMER, PRETTY MUCH EVERY GROUP HAS THE SORT OF TYPICAL MONTHLY TELECOM AND CONAND THEN THERE'S SERIES OF ONLINE ACTIVITIES. AND OVERTIME, YOU NOTICE I HAVEN'T MENTIONED THE WORDS STANDARDS OR ANYTHING LIKE THAT. AND WE DIDN'T SORT OF THINK ABOUT IT THAT WAY BUT IT'S THE COORDINATION AND COMMUNITY CONVENTIONS PART OF ESIP THAT MAKE ITS REALLY SO STRONG. IF YOU LOOK AT THIS DIAGRAM AND I FORGET WHERE IT'S FROM BUT IT'S THE SHARING OF PEOPLE AND KNOWLEDGE THAT PROVIDE STRONG COHESION TO ESIP. SO THE OUTCOMES WE GET ARE--YOU KNOW A LOT OF THE MEETINGS ARE VERY MUCH ABOUT STIMULATING NETWORKING COLLABORATION, EXCHANGE OF IDEAS, FORUMS FOR THESE COMMUNITY DRIP COLLABORATIONS, PROFESSIONAL DEVELOPMENT AND INNOVATION AND I'LL SORT OF RUN THROUGH THIS. SO ESIP IS NOT A STANDARDS BODY. AND I'LL SAY IT'S ABOUT PRACTICES AND I THINK THE WORD BEST PRACTICE SYSTEM HIGHLY OVERUSED IF YOU WANT TO ASK ME ABOUT THE ENGLISH DEFINITION OF WHAT BEST IS IN THIS SITUATION, YOU CAN HAVE A RANT ON THAT SO ESIP IS A STANDARDS BODY AND YET WE GET A REMARKABLE AMOUNT OF THINGS DONE. BUT NOW I WILL RUN THROUGH ELEMENTS OF ESIP. WE WILL HAVE TRANSLATIONAL RESEARCH TEGGIC PLAN AND MAKE SURE A NUMBER OF AVAILABLE LINKS ARE IN HERE AND THE REST OF THIS PART OF THE TALK IS TAKEN FROM PRESENTATION I GIVE EVERY SIX MONTHS CALLED THE STATE OF THE FEDERATION. SO THERE'S SO MUCH GOING ON IN ESIP THAT NOT EVERYONE KNOWS EXACTLY WHAT IS GOING ON, SO WE SORT OF ROLL THIS UP AND INFORM THE MEMBERSHIP. WE HAD A PLAN OF 2014, AND WE'RE WE'RE SEEING WHAT PERCOLATING BELOW FROM THE MEMBERSHIP AND THE NEEDS IN THE COMMUNITY IS ALIGNED WITH AGENCY SPONSORS SO IT'S THE WAY IN WHICH WE MANAGE THIS BOTTOM UP TOP DOWN SYNERGY SO THAT OUR SPONSORS ARE GETTING WHAT THEY WANT OUR COMMUNITY IS GETTING WHAT THEY WANT AND THEN WE MAP THOSE STRATEGIC GOALS FOR THE WORKING COMMUNITIES TO GIVE THEM A SENSE OF HOW THEIR OUTCOMES HAVE THE THINGS THEY'RE DOING SUPPORT THE OVERALL STRATEGY. EMILY LAW IS THE VICE PRESIDENT ARE ESIP AT THE MOMENT AND SHE'S LEAD THANKSGIVING PARTICULAR ACTIVITY. SO LET ME GIVE YOU WHAT THIS CASE HAS DONE BACK IN MID2000S, THE GROUP ON EARTH OBSERVATIONS WHICH IS A VERY LARGE INTERNATIONAL ORGANIZATION WHICH IS ABOUT 10 YEARS OLD FORMED A NUMBER OF TASK AREAS IN APPLICATIONS THAT WERE RELEVANT TO THE GLOBAL OBSERVATIONS OF THE EARTH AND ONE OF THEM WAS THE AIR QUALITY AREA. AND IN THE UNITED STATES, THEY NEEDED A PLACE TO SORT OF CARRY ON THEIR ACTIVITIES, THEY NEEDED A PLACE TO COME TOGETHER AND MEET AND LEVERAGE EXPERTISE BECAUSE THEY DIDN'T HAVE ENOUGH PEOPLE. IT'S PRETTY MUCH A VOLUNTEER EFFORT, SO WHAT DO THEY CALL THEM? UNFUNDED MANDATES. YOU KNOW ABOUT THOSE. SO THEY CAME IN AS A CLUSTER FOR ESIP, THEY SAID WE WANT TO DO THIS, WE WILL FORM A CLUTTER AND SUDDENLY THE CLUSTER APPEARED AND IT HAD SOME AIMS. IT WAS AFTER A COMMUNITY OF PRACTICE SO PEOPLE WERE FIGURING OUT WHAT MEANT, BUT REALIZED IT NEEDED DATA PROVIDERS AND EFFECTIVE MECHANISM TO SORT OF SCIENCE, HIGH ASSAULT SCIENCE IN ATMOSHPHERIC EXPERIENCES AND SO THEY CAME TO ESIP AND OVER A PERIOD OF TIME THEY OPERATED AS A CLUSTER, THE GROUP ON THIS OBSERVATIONS RUNS AN ACTIVITY CALLED ARCHITECTURE INFORMATION PILOTS TO DEVELOP COMMON INFRASTRUCTURE THAT'S GOING TO WORK AND THROUGH ESIP, THROUGH VOLUNTEER EFFORT, THEY CAME UP WITH A CATALOG OF QUALITY DATA, IMPLEMENTED THAT THROUGH THE CLEARINGHOUSE THAT THE GEOWAS RUNNING AT THE TIME. THE WEB PRESENCE, THE PUBLIC PRESENCE OF THIS WAS ANOTHER COMPONENT OF IT, THERE WAS A HIGHLY PARTICIPATORY WEB SITE BOTHOT GEO SIDE AND ON THE ESIP SIDE WHICH BROUGHT TOGETHER THE PEOPLE, THE PROJECTS THEMSELVES AND THESE DATABASES SO IT WAS LIKE AN EARLY VERSION OF WIKIPEDIA FOR DATA FOR QUALITY AND N WHICH PEOPLE COULD PUT UP AND ADVERTISE DATA SETS AND ANNOTATE THEM ASK HAVE DISCUSSIONS AROUND THEM. AND THIS IS ONE OF THE LATEST OUTPUTS IF YOU MEMBER THE CLUSTER WAS MADE PERMANENT INTO A WORKING GROUP SO IT HAD SOME LEVEL OF PERM NANCY AND WE SEEDED IT WITH MORE FUNDING AND BY AND LARGE, IT IS PRODUCED A VERY SIGNIFICANT RESULT FOR THE AIR QUALITY COMMUNITY WITHIN THIS INTERNATIONAL--WITHIN THIS INTERNATIONAL SETTING. SO IT'S ONE EXAMPLE OF THE SUCCESS OF WHAT COMMUNITY BASED ORGANIZATION CAN DO. SO WHAT WE DO NOW, IS PRACTICES AND I'LL SHOW YOU A LITTLE BIT ON CITATION AND WE RUN TEST BED ACTIVITY. THE COMMUNITY CONVENTIONS, NOT STANDARDS, PROFESSIONAL DEVELOPMENT OUTREACH AND ACROSS THIS VENUE FOR COLLABORATION, SO, I'M JUST GOING TO RUN THROUGH A SERIES OF THESE AND I'M NOT GOING TO TALK THROUGH THEM ALL IN GREAT DETAIL, WE HAVE A STEWARDSHIP COMMITTEE STARTED AS A CLUSTER AND STARTED AS PRESERVATIONS STEWARDSHIP CLUSTER AND IT BECAME SUFFI IMPORTANT THAT IT BECAME A WORKING GROUP AND THEN IT BECAME A COMMITTEE AND THERE'S AN ENORMOUS AMOUNT OF WORK DONE BY THE STEWARDSHIP COMMITTEE AND IN PARTICULAR I WANT TO POINT TO TWO KEY THINGS ON CITATION AND IDENTIFIERS. NOW BACK IN THE LATE--LATE 2000S, THERE WAS AN ACTIVITY ON DATA CITATION. SO REALLY BEFORE A LOT OF THE REST OF THE COMMUNITY STARTED TO GET THAT AROUND DATA CITATION AND THAT WAS VERY NICE GUIDELINES FOR DATA CITATIONS THERE IN THAT LINK AND TAKEN--THEY LED US TO WHEN THE--THE MORE GLOBAL CONVERSATION ABOUT CITATION CAME ALONG THROUGH ORGANIZATIONS SUCH AS CO-DATA AND THE WORLD DATA SYSTEM AND FORCE 11 AND SO ON, ESIP PUT FORTH A RESOLUTION RELATED TO THOSE JOINT PRINCIPLES ON THE NEXT SLIDE BUT SORT OF REACHING BACK THERE, 'S ALSO QUITE AN INTERESTING SET OF WORK ON IDENTIFIERS, SO OF AS AN ENABLING CAPABILITY THAT YOU REALLY NEED WHEN YOU START TALKING ABOUT DATA AND OTHER SORTS OF COMMUNITY CONVENTIONS. AND MANY OF US ARE DOCUMENT INDEED PAPERS. SO HERE'S THE RESOLUTION. WE GOT LOFTY, YOU KNOW? ACKNOWLEDGING THE IMPORTANCE OF ACCESS AND RELIABLE DATA OF THE PROCESS OF REPRODUCIBLE SCHOLARSHIP AND SO ON AND O ON AND SO ON, LED TO THIS RESOLUTION WAS ADOPTED BY THE ENTIRE MEMBERSHIP, NANOG MOUSILY. THIS IS AN IMPORTANT THING, THIS TO ME COUNTS MORE THAN STANDARDS BECAUSE EVERYONE SORT OF HAS SIGNED UP FOR THIS, RATHER THAN TOSSING SOMETHING OVER THE FENCE AND WHETHER OR NOT YOU GET ADOPTION IS A BIT UNPREDICTABLE. THE STEWARDSHIP COMMITTEE IS ONE OF THE EARLY PROPONENTS OF DATA MANAGEMENT FOR SCIENTISTS, RAN A WHOLE PUNCH OF SHORT COURSES AT PHYSICAL UNION AND VARIOUS OTHER SETTINGS THAT'S LARGELY NOW PICKED UP BY PROFESAL SOCIETIES AND OTHER ORGANIZATIONS AND ALL THIS MATERIAL AND ALL THIS IS AVAILABLE AND USING BY ANYONE IN THE COMMUNITY. SO IT WAS INENTIALLY FUNDED BY NOAA, AND DEVELOPED AND MATURED BY THE COMMUNITY ITSELF. IN REQUEST TO SPONSOR AND YOU CAN SEE WHERE IT'S BEEN OFFERED. OPEN SEARCH, I DON'T KNOW HOW MANY ARE FAMILIAR WITH THE OPEN SEARCH CAPABILITY, THE WAY IN WHICH YOUR WEB BROWSER CAN CONNECT TO A VARIETY OF DIFFERENT CHANGES NOT JUST GOOGLE BUT BEING IN YAHOO, AND INDEX AND SO ON, WHAT WAS REALIZED EARLY ON IS FOR THE SCIENCES COMMUNITY, YOU WANT TO LEVERAGE AS MUCH OF THE WEB INFRASTRUCTURE AS CAN YOU SEE THERE WAS AN EFFORT THROUGH THE ESIP DISCOVERY CLUT THEY'RE BROUGHT TOGETHER A NUMBER OF ORGANIZATIONS AND HELP DEFINE AN OPEN SEARCH CONVENTION FOR OBSERVATIONS. SO NOW WE'RE GETTING REALLY, REALLY CLOSE TO THE POINT WHERE YOU CAN GO GOOGLE FOR YOUR EARTH SCIENCE DATA AND IT WILL ACTUALLY SHOW UP AND WITH SORT OF THIS FAR AWAY WHEN YOU DO A WEB SEARCH AND IT HAS WEB IMAGES, SO ON, THEY'RE CLOSE TO GETTING A DATA TAB THERE. WOULDN'T THAT BE COOL. I.T. INTEROPERABILITY, CAN YOU SEE THE SIDE HERE, YOU CAN SEE GEEKY THINGS THAT GO ON IN THIS COMMITTEE, ONE THING THEY'VE HAD A LOT OF SUCCESS IN ARE THESE THINGS CALLED RANT ASKS RAVES AND SO THEYVILLE A MONTHLY TELECONAND SOMEONE WILL GET AN IDEA OR BE FRUSTRATED BY SOMETHING AND THEY WILL GO TO THIS TELECONAND RANT AND RAVE AND SORT OF SAY, I WANT THIS, WHY IS THIS SO HARD? AND INTERESTINGLY, PEOPLE RESPOND. THEY LIKE THIS SORT OF--THEY LIKE THIS SORT OF CHALLENGE. PRODUCTS AND SERVICES IS ANOTHER AREA THAT IS OF SIGNIFICANT VALUE TO THE FEDERATION. THIS IS A PLACE WHERE IF YOU WANT TO THINK OF IT AS TECHNOLOGY TRANSFER OR VETTING THERE ARE PLACES WHERE IF HAVE YOU A BIT OF AN IDEA ON A SMALL SCALE BUT DON'T HAVE THE RESOURCES TO DO IT IN MY OWN INSTITUTION OR I DON'T HAVE THE PEOPLE TO DO IT THEN PROW POSE A TEST BED ACTIVITY. AND THESE TEST BED ACTIVITIES ARE ACTUALLY FUNDED THROUGH THE SPONSORING MEANS AND THE INCOME FROM OUR MEETINGS AND THESE ARE THINGS LIKE $5000, $10,000. AND THIS TEST BED AGGREGATES THE RESOURCES THAT ARE WITHIN THE COMMUNITY. THE PROJECTS THAT PEOPLE ARE LOOKING AT AND WE HAVE A FAIRLY SET OF ACTIVITY. SO APRIL OF LAST YEAR WE HAD ONTOLOGY TEST BED FOR MAPPING AND RECONCILIATION, THIS LAST PROJECT TOOL MATCH WHICH MATCHES DATA SETS AND APPLICATION TOOLS AND TOOLS AND DATA SETS SO THAT CAN YOU FIGURE OUT, I HAVE THIS DATA SET HA TOOLS CAN I POSSIBLY USE TO VISUALIZE THIS DATA SET AND THAT WILL BE SET UP AS A WEB SERVICE. OCTOBER OF LAST YEAR, CAN YOU SEE DISASTER LIFE CYCLE, MORE ON TOUR MATCH, ENTITY LINKING AND EXPERTISE, AND WE RUN THESE AT THE MEETINGS CALLED FUNDING FRIDAYS WHERE PEOPLE PRETTY MUCH GET OUT A PIECE OF PAPER AND A MARKER AND BASED ON WHAT THEY HEARD DURING THE MEETINGS PUT UP AN IDEA IN THE NEXT SIX MONTHS AND COMING OFF AND VOTE ON WHICH ONES THEY THINK ARE GOOD AND THOSE PEOPLE WHO WIN AND CAN YOU SEE IT'S A LIST OF TOPICS HERE, SEMANTIC ANNOTATION, MOBILE APPLICATIONS, MAKING EARTH DATA FUN AND THERE'S A MEMBER AND A TREAT COMPONENT TO THIS AND AT THE NEXT MEETING THEY PRESENT THE RESULTS OF THAT. APPLICATION ACTIVITY AND AGRI KILTURE AND CLIMATE. STRONG CONNECTIONS WITH LONG-TERM RESEARCH NETWORKS WHICH IS STARTING TO POP UP THROUGH THE U.S. DEPARTMENT OF AGRICULTURE. A DATA STUDY AND RESEARCH GROUP, AND DATA INFORMATION TO SAY, THEY'RE JUST GETTING PRETTY IMPORTANT. YOU GUYS HAVE BEEN DOING IT FOR A WHILE, WHAT DO YOU THINK ABOUT RUNNING A DATA STUDY TO SEE IF MAYBE, MAYBE IT'S A TIME TO HAVE A SURVEY FOR DAT KACCT. ALL RIGHT, SO YOU MAY OR MAY NOT BE FAMILIAR WITH THE SURVEYS IN ASTRONOMY AND EARTH SCIENCES THEY HAVE THESE 10 YEAR STUDIES THAT SORT OF PLAN OUT AND SCOPE OUT WHAT WE SHOULD BE DOING SO WE RAN A SERIES OF ACTIVITIES AND PUSHED IT OUT INTO THE COMMUNITY AND THERE'S--THERE'S A NICE LITTLE SESSION HERE THAT WAS RUN AT THE AMERICAN GEOPHYSICAL UNION THAT BASICALLY CONCLUDED THAT AT THE MOMENT A SEPARATE SET OF DATA IS PROBABLY NOT THE RIGHT THING TO DO BUT TO ELEVATE THE DATA IN ALL THE OTHER DISCIPLINE AREAS FOR WHICH DECADA L STUDY IS BEING DONE IS THE THING TO DO. SO THERE'S A NEW STUDY DECADA L SURVEY FOR EITHER AND THAT TELLS PEOPLE WANT TO DO AND NOW DATA WILL BE A FIRST CLASS CITIZEN NOW. SO THIS INITIALLY--THIS SCRIPT IS INITIALLY CALLED THE DISASTER CLUSTER AND I SAID MAYBE YOU WANT TO CHANGE THE NAME, IT SORT OF SENDS THE WRONG MESSAGE. [LAUGHTER] SO THEY CHANGE TODAY TO DECIDER LIFE CYCLE CLUSTER AND IT'S APPLICATION FOCUSED. LTS OF ACTIVITIES GOING ON IN THIS PARTICULAR AREA AND THIS ONE MIGHT BE OF INTEREST, SO INSTEAD OF HAVING A METADATA ACTIVITY, THE METADATA RUNS RIGHT THROUGH ESIP, EVERYTHING IS ABOUT METADATA. SO MATURING THE DISCUSSION IS WHAT WE'RE TALKING ABOUT IS DOCUMENTATION AND SO TED TRIMBLE HAD BEEN ABERMANN WORKED ON THIS AND THEY WORKED ON DISCOVERY AND THINGS LIKE THE U.S. GOVERNMENT'S BIG EARTH DATA INITIATIVE AND THEN PUSHING THAT OUT IN APPLICATIONS AND KEY CODING FORMATS LIKE THE HIERARCHICAL DATA FORMAT KLF-TWO ARE VERY PREVALENT IN EARTH SCIENCES AND YOU WILL SEE DOWN IN THE BOTTOM HERE, YOU KNOW INVITATIONS TO GET INVOLVED AND THEY WILL PULL US AND SAY, ESIP IS A MEMBER, YOU DON'T HAVE TO BE A MEMBER TO PARTICIPATE. ANYONE CAN SHOW UP IN THE MEETINGS, ANYONE CAN PARTICIPATE IN THE CONFERENCE CALLS. THERE'S NO WHO ARE YOU AND WHAT ARE DOING HERE AND AND TO A COMPLETES A GROUP. SO COLLABORATION THROUGH CONTENT MANAGEMENT SYSTEMS HAD ENORMOUSLY SUCCESSFUL WENT QUICKLY IF A CLUSTER TO A WORKING GROUP WHICH HAS LEVEL OF TERM NANCY AND THOSE--AND THAT'S REALLY A NICE SET OF WEBINARS THAT IS ATTRACTED THE ATTENTION OF THE PRIMARY DRUP A L S THEMSELVES AND COME TO THE MEETING. IMAGING CLIMATE AND THIS IS ANOTHER ACTIVITY THAT BASED ON THE SUCCESS OF THE QUALITY ACTIVITY IN THE OBSERVATIONS, ENKNOWLEDGE SCHECLIMATE ALSO COMES FROM GEOAND UNDER THE ACTIVITIES, YOU KNOW YOU WILL SEE KEY THINGS HERE THAT NEED TO COME FROM THE COMMUNITY. DATA PROVIDENCE, ACCESS PANZ APPEARANCE AS THEY RELATE TO ENERGY APPLICATIONS. AND THEN MY FAVORITE, TOM NAROCK AT MARIMOUNT UNIVERSITY, HE'S WORKED ON THE UNDERLYING CAPABILITIES OF SEMANTIC TECHNOLOGIES GET DISCUSSED AND PUSHED OUT INTO THE BROADER COMMUNITY FOR USE, SO THIS IS WHERE THE TOOL MATCH EFFORT COMES. THERE'S A BRANDING OF THE BIOPORTAL WHICH IS OUT OF STANFORD. WE'VE BRANDED A VERSION OF THAT FOR THE CLOUD FOR SCIENCES SO ESIP LARGELY IS THE GOVERNING BODY FOR ONTOLOGY IN THE EARTH SCIENCES COMMUNITY AND AGAIN COMMUNITY BASED AND COMMUNITY INPUT AND COMMUNITY FEEDBACK. I CAN SKIP THAT THAT'S THE ATTRIBUTE CONVENTION. AS I MENTIONED EARLY ON, IT REACHES OUT INTO A VARIETY OF DIFFERENT COMMUNITIES SO PROFESSIONAL COMMUNITIES LIKE THE AMERICAN GEOPHYSICAL UNION, SO WE'RE NOT A STANDARDS BODY BUT WE ABSOLUTELY TALK TO THEM AND PEOPLE SHOW UP AT OUR MEETINGS DOWN INTO EDUCATION THROUGH THE CLIMATE LITERACY NETWORK AND THEN COMMUNITIES THAT ARE ORIENTED AROUND THE AGENCIES SO THE ENVIRONMENTAL DATA MANAGEMENT COMMUNITY IS THE NOAA COMMUNITY AND THE USGS COMMUNITY DATA INTEGRATION AND THE NASA SCIENCE AND DATA INFORMATIONS AS WELL AS GLOBAL RESEARCH DATA ALLIANCE AND NEW ACTIVITIES SUCH AS THE NATIONAL SCIENCE FOUNDATION COUNCIL ON DATA FACILITIES AND THEN YOU JUST PICK YOUR FAVORITE ACRONYM AND THE SUM CONNECTION THAT ESIP HAS EITHER WE'VE APPROACHED THEM OR THEY AA PROACH US FOR A MUTUAL BENEFIT. SO WHAT'S JUST SORT OF SUMMING UP A BIT HERE THE ONE VIEW OF HOW YOU CAN THINK ABOUT ESIP IS WHAT WE LIKE TO THINK OF AS COLLECTIVE IMPACT, RATHER THAN INDIVIDUAL IMPACTS AND IT SORT OF HELPS, AGAIN, WE LOOK BACK AND SEE WHAT ESIP HAS BECOME AND WHY IT'S BECOME SO EFFECTIVE. SO CROSS SECTOR ACTIVITIES SHARE THE GENDER AND THEY CHANGE WHAT THEY'RE DOING SO THEY CAN SOLVE SOMETHING THEY CAN'T SOLVE ON THEIR OWN AND THAT IS MADE UP OF A VARIETY OF COMPONENTS. SO IT'S NOT JUST AN INDIVIDUAL IMPACT BUT IT'S A COLLECTION OF IMPACTS. AND THIS IS WHAT WE CALL THE BACKBONE ORGANIZATION. SOPHISTICATED A BACKBONE ORGANIZATIONS KNEESATION IF YOU'RE NOT FAMILIAR WITH THE TERMINOLOGY HAS A DEFINITION ON THE LEFT HERE, STRATEGIC COHERENCE, MAKE SURE THAT THINGS HAPPEN DAY-TO-DAY THAT WHEN YOU NEED A TELECON, THERE'S A TELECON, WHEN YOU NEED A WIKI, THERE'S A WIKI, AND MANAGES VARIOUS ASPECTS OF HOW THINGS GET IMPLEMENTED. RIGHT? AND THEN THE FUNCTIONS HERE ON THE RIGHT AND THESE ARE NOT SPECIFIC TO ESIP, THESE ARE GENERAL FUNCTIONS OF THE BACKBONE ORGANIZATION AND IT REALLY FACILITATES THE OPERATION OF THE COMMUNITY. YOU KNOW SO ENGAGEMENT, OUTREACH, FUND RAISING AND SO ON. SO THE KEY REALLY HERE IS IN MANY CIRCUMSTANCES AND NOW I'LL REFER TO STANDARDS ORGANIZATIONS AND STANDARDS ACTIVITIES, FOR THE MOST PART STANDARDS AC ARE LARGELY TECH THE IPADICAL ACTIVITY MPLET ADOPTION IS A SOCIAL AND CULTURAL ACTIVITY AND THEY'RE USUALLY WELL SEPARATED. WHEREAS WHAT ESIP HAS MANAGED TO ACHIEVE IS THAT THOSE TWO COMMUNITIES AND THE SOCIAL AND THE TECHNICAL ARE REALLY FROM THE START SO THAT PEOPLE DON'T EVEN THINK TWICE ABOUT WHETHER THEY'RE GOING TO ADOPT WHAT ESIP COMES UP WITH BECAUSE THEY'VE BEEN A PART OF IT AND THEY SEE THE VALUE IN IT AND THAT'S WHAT THE BACKBONE ORGANIZATION REALLY FACILITATES. SO IN TERMS OF SHARED AGENDA ISSUES THE VALUES OF ESIP, THINGS THAT WHY I KEEP ATTENDING IS IT'SAGEILE AND THINGS ON RELATIVELY SHORT NOTICE IN A FLEXIBLE WAY, IT IS HIGHLY COLLABORATIVE, NOT COMPETITIVE. AND IT'S FRIENDLY, IT'S GOT A STRONG SENSE OF COMMUNITY, THE INNOVATION IS ABSOLUTELY THERE AND THE REASON WHY IT'S GETS APPROACHED A LOT IS IT IS VIEWED AS A NEUTRAL ORGANIZATION EVEN THOUGH IT'S FUND FREE RADICALS GENERATED A COUPLE OF GOVERNMENT AGENCIES SEEN AS A PLACE WHERE VARIOUS AGENCIES AND PROJECT CANS COME IN A NEUTRAL SET WRING THERE'S NO SNOW, NO HIDDEN AGEND AS VERSES SHARED, AND THIS IS OPEN PARTICIPATORY AND VOLUNTARY SO WHERE WE GET A LITTLE EDGY AT TIMES, THERE'S ACTIVITIES AND THE SCHEME AT OUR ORGACTIVITY IS IT'S WORTH TAKING A LOOK AT AND WE SAID THIS IS THE ACTIVITY WHICH IS HOPEFULLY GOING TO LEAD TO THE DATA TAB ON YOUR WEB SEARCH SO SCHEMA.ORGHAS A DAT SET EXTENSION, A LARGE NUMBER OF EARTH SCIENCE DATA PROVIDER VS LIMP ELEMENTED THE DATA SET EXTENSION WHICH IS BASED ON THE WDTHREE CAT SO THEY GET ENCRAWLED BY THE MAJOR SEARCH ENGINES. DOCKER IS A GREAT TOOL THAT EMPLOYEES APPLICATIONS, WE JUST STARTED A DRONE CLUSTER, I'M NOT SURE WHAT A DRONE CLUSTER IS, BUT ONE OF OUR MEMBERS FROM JPL BROUGHT A LITTLE--A LITTLE DRONE THING AND SAID WE SHOULD START A DISCUSSION OF THIS, MAYBE THIS COULD BE USED FOR FOR OBSERVATIONS AND SO WE'RE VERY MUCH EMBEDDED IN THE OFFICE OF SCIENCE AND TECHNOLOGY POLICY. SO IN TERMS OF GOVERNANCE MODELS OR COMMUNITY ENGAGEMENT THIS IS SORT WHAT HAVE WE'VE LEARNED IS YOU HAVE TO MAKE IT EASY TO PARTICIPATE, I SORT OF CALL IT THE BUS, THE CAMPUS SHUT THEY'LL RUNS AROUND HERE AND IT SHOULD BE EASY TO GET ON AND GET OFF AT ANY PLACE YOU WANT AND WHEN YOU GET ON, YOU SHOULD HAVE A SENSE OF GOING IN THE RIGHT DIRECTION. THAT'S IMPORTANT. THE ALIGNMENT OF DAY JOB AND THROUGH WHAT YOU DO ON A DAILY BASIS AND COMMUNICATION IS OF COURSE REALLY IMPORTANT AND A VARIETY OF MECHANISMS FOR THAT AND THERE'S A WHOLE OTHER TALK THAT I COULD GIVE YOU ABOUT VIRTUAL ORGANIZATIONS, THE OTHER THING YOU WILL SEE IN ESIP A LOT IS USE CASES, USE CASES, USE CASES, ARTICULATIONS OF WHAT WE'RE THINK IN A FAIR AMOUNT OF DETAIL WHETHER IT'S SOME CITATION, ALL THE WAY SOPHISTICATED SEMANTICS ALL ABOUT WRITING DOWN AND GETTING A SHARED UNDERSTANDING OF WHAT WE'RE INTERESTED IN WHERE HIGHLY SOCIAL, STRONG PRESENCE, TWITTER PAGE AND THINGS LIKE THIS, PEOPLE GET THE FEELING OF BEING PART OF A COMMUNITY ON AN ONGOING BASIS AND NOT JUST A RANDOM E-MAIL NOW AND THEN SO TO SORT OF CONCLUDE HERE, GOING BACK TO THIS DIAGRAM AND COMMUNITY COORDINATION, SO THE SOCIAL MIXED WITH TECHNOLOGY, LIGHT WHITE OVERSITE, COMMUNITY BACKBONE AND COLLABORATION THAT LOOKS LIKE PART OF WHAT YOU DO EVERY SINGLE DAY IS THE VOLUNTEER LEADERSHIP, THIS IS OUR SLATE OF PEOPLE SO THERE ARE REAL PEOPLE BEHIND ALL OF THESE WHO HAVE STOOD UP AND BEEN ELECTED IN THESE PARTICULAR POSITIONS, STANDING COMMITTEE CHAIRS, ADMINISTRATION AND EACH OF THE TYPES, THESE TYPES HAVE A REPRESENTATIVE AS WELL, TO FACAS TILLITATE THE TWO WAY EXCHANGE OF INFORMATION BETWEEN THE STRATEGIC ASPECTS OF ESIP AND THE WORKING ASPECTS OF ESIP. YOU MIGHT RECOGNIZE THESE FACES. WE HAVE A SUMMER MEETING IF YOU WANT TO COME TO CALIFORNIA AND AT CAMP, IF YOU'VE BEEN THERE, IT'S A BIT RUSTIC BUT IT WILL BE A GOOD PLACE TO EXCHANGE. THE THE WINTER MEETING IS MORE FOCUS INDEED D. C. BUT THE SUMMER MEETING IS HIGHLY FOCUSED. IT'S A GOOD PLACE TO COME. WE'RE VERY ONLINE, WIKI COMMONS, FACEBOOK, TWITTER. YOU CAN CAN FOLLOW US AND OVER THERE ON THE LEFT AND SO AARON WAS A GRADUATE STUDENT WHO HELPED INITIATE THE AIR QUALITY CLUSTER THAT I TOLD YOU ABOUT FORMED IN--SAW IT THROUGH THE TRANSITION INTO A QUALITY WORKING GROUP, GOT HER DEGREE, CAME ON AS COMMUNITY DIRECTOR FOR ESIP AND WHEN OUR EXECUTIVE DIRECTOR MOVED ON, BECAME EXECUTIVE DIRECTOR AND SO, THIS IS AN EXAMPLE OF A PERSON WHOSE HEAVILY INVESTED IN THE COMMUNITY SUCH AS ESIP. AND THAT'S SCRATCHING MY HEAD ON THE RIGHT THERE. THANK YOU VERY MUCH AND I HOPE THAT WAS USEFUL. [ APPLAUSE ] >> I THINK WE HAVE TIME FOR MAYBE TWO QUESTIONS THAT ARE USED FOR WEBCASTING. >> SO THANK YOU THAT WAS VERY INFORMATIVE FOR THE GROUP. >> IS THAT MIC ON? IS IT? >> CAN YOU HEAR ME? >> I WILL HOLD IT CLOSE. SO THAT VERY INFORMATIVE AND I CONTINUING DOES SET A GOOD TEMPLATE FOR SOME OF THE CHALLENGES WE HAVE. ONE OF THE QUESTIONS THAT I WOULD LIKE TO ASK YOU IS HAVE YOU BEEN ABLE TO THROUGH THIS EFFORT TO TRY TO DEFINE A SORT OF CONSISTENT IDENTIFIER MODEL FOR THE DATA SO NO MATTER WHERE IT COMES FROM, THERE'S A UNIQUE WAY TO POINT TO IT IF YOU WANT TO BE ABLE TO SAY SOMETHING ABOUT IT OR A COMBINATION OF THINGS. BECAUSE WE HAVE A LOT OF THAT IN LIFE SCIENCES. >> SO THE ANSWER IS YES, ON THE CITATION IDENTIFY SLIDE WHICH I WILL BE HAPPY TO GIVE YOU A COPY, THERE'S A LINK TO THE PRACTICES ON IDENTIFIERS AND INFORMATION ON INFORMATICS ON EXACTLY THAT AND AGAIN WE'RE NOT A STANDARDS ORGANIZATION, SO THESE ARE GUIDELINES WITH RECOMMENDED APPROACHES BUT THERE'S NICE DISCUSSION OF THE FACTORS THAT GO INTO DECIDING WHICH FORMS OF IDENTIFIERS YOU WANT TO ACTUALLY TAKE AND THERE'S USE CASES BEHIND THEM AS WELL, SO YOU CAN IDENTIFY THE RELEVANCE VANCE OR APPLIC ABILITY BASED ON IDENTIFICATION OF THE YOUTH CASE IS ONE CAN YOU RELATE TO. SO ABSOLUTELY. >> THANK YOU. >> VERY IMPRESSIVE. I HAD A LOT TO LEARN FROM THAT. THANK YOU. JUST ONE SORT OF GENERAL POINT AND SOMETHING MORE SELF--SERVING, FIRST PART WHAT HASN'T WORKED? IN THIS AS IT EVOLVED. AND SUBBEDLY, I APPRECIATE THE FACT THAT EVEN THOUGH IT'S FUNDING, IT'S FELT THIS IS SORT OF OWNERSHIP BY THE FUNDING AS AND PART OF THE QUESTION THAT RELATES TO US IS WHAT COULD--WHAT COULD HAVE BEEN DONE BY THE FUNDING AGENCIES IN IN SORT OF NEUTRAL ROLE THAT WASN'T DONE THAT WOULD HAVE IN A SENSE PRECIPITATED THE DEVELOPMENT AND ACCELERATION OF ESIP. FOR WHAT I SAY, I CRITICIZE THE NATIONAL SCIENCE FOUNDATION ALL THE TIME SO LET'S GO. ONE OF THESE ACTIVITIES WAS THE WATER ACTIVITY AND THEY CAME INTO ESEING WHAT THE AIR QUALITY PEOPLE HAD DONE AND SAY WE'RE GOING TO DO THAT AND THEY CAME IN WITH A VERY, VERY SPECIFIC AGENDA AND SORT OF A WORK PLAN AND A SET OF PEOPLE THAT WANTED TO DO THAT AND THEY WANTED ALL THESE ESIP VOLUNTEERS TO PARTICIPATE AND THEY NEVER SORT OF FIGURED OUT THAT IT HAS TO BE A COMMUNITY ACTIVITY. SO ANYONE TRYING TO COME IN AND LEVERAGE VOLUNTEERS DOESN'T PARTICULARLY WORK VERY WELL, IT COULD HAVE BEEN TIMING THAT WAS PART OF IT, BUT THAT DIDN'T WORK AND THEY SORT OF, YOU KNOW NO ONE SHOWED UP TO THEIR MEETS AND THEY WERE SORT OF TALKING TO THEMSELVES AND THEY WENT OFF, THEY'RE ACTUALLY BACK NOW BECAUSE THEY LEARNED THEIR LESSON IN TERMS OF COMMUNITY AND ENGAGEMENT AND SORT OF TAKING MORE OPEN APPROACH. SO IN TERMS OF IT DOES, IT ECOSYSTEMS TAKE TIME, THEY'RE HARD TO DESIGN, YOU KNOW YOU CAN'T DESIGN ECOSYSTEMS AT LEAST AT THE MOMENT. YOU CAN'T ENGINEER THEM. THE FUNDING AGENCIES WERE I THINK SENSIBLE ENOUGH TO SAY ALL RIGHT WE WILL FUND A BUNCH OF PEOPLE TO PARTICIPATE INITIALLY AND THAT PUT ESIP STABLE FOR THE FIRST FIVE YEARS. SO THEN THEY SAID LET'S SEE WHAT HAPPENS. SO THE ESIP WILL BE VALUED BECAUSE IT'S ESTABLISHED AND BUILT IN. BUT WE STRONGLY RESISTED AGENCIES GIVING A SORT OF LINE ITEMS, THE THINGS THEY REALLY WANT DONE AND THEY RESPONDED EXTREMELY WELL TO THAT. THEY SAY BIG DATA INITIATIVE, THAT'S IMPORTANT TO US. PERIOD. THEN PEOPLE STEP UP BECAUSE THERE'S PEOPLE FROM THE COMMUNITIES REALIZING, ALL RIGHT, WE'VE BEEN TRYING TO GET INTERAGENCY COOP COOPERATION FOR A LONG TIME AND THERE'S A REASON WHY IT DOES NOT WORK, ALL RIGHT? LET'S DO THAT IN THE COMMUNITY. OH THAT WORKS. WE CAN ACTUALLY GET THINGS DONE AND WE CAN LOOK AT INNOVATIONS THAT ARE COMING FROM THE COMMUNITY AND WE CAN ADOPT THEM FOR THINGS LIKE THE [INDISCERNIBLE] AND THEN BRING THEM BACK INTO OUR AGENCIES BECAUSE ESIP HAS A PREMATURE NOW THAT WE'VE SORT OF SHOWN TO BE SUCCESS EXCLUSIVELY WE TEND NOT TO FOLLOW THINGS THAT ARE NOT DEEMED IMPORTANT. >> THANK YOU I'M AFRAID WE NEED TO CUT IT SHORT SO WE'RE NOT LATE. SO THE NEXT SESSION, FIRST SESSION, CHAIRED BY MELISSA HAENDEL. >> THANK SPECIALIZATION OF SPECIFIC ENDOTHELIAL MUCH PETER I HAVE A LOT OF QUESTIONS FOR YOU THAT I WILL BOMBARD YOU CAN LATER BUT IF WE COULD HAVE THE NEXT PANELIST COME UP SO THAT WE'RE HAVING CONVERSATIONS SO WE'RE NOT JUST GIVING TALKS HERE. SO, I WILL GIVE THIS BACK. OKAY, SO WE'RE GOING--THE WAY WE SET UP THE WORKSHOP IS TO KIND OF PROVIDE SOME OVERVIEW OF DIFFERENT TYPES OF STANDARD DEVELOPMENT IN BOTH THE BASIC SCIENCE AREAS AS WELL AS THE CLINICAL SCIENCE AREAS IN OUR SESSION LATER THIS AFTERNOON. AND SO THIS MORNING'S SESSION IS REALLY THE BASIC SCIENCE SESSION AND THE IDEA BEHIND SEPARATING THESE THINGS WAS ACTUALLY TO PROMOTE US GENERATING IDEAHOW DO WE MERGE THESE DIFFERENT COMMUNITIES WHICH IS FOR THE MOST PART WAY TOO SEPARATED SO AT THE END OF THE DAY WE'RE ALL HERE ON BEHALF OF NIH, WE'RE TRYING TO PROMOTE TRANSLATIONAL RESEARCH SO BASICALLY TRYING TO BRING THESE LANDSCAPES TOGETHER. SO I WILL TELL A STORY ABOUT MY OWN, FROM MY OWN EXPERIENCE ABOUT SOME STANDARDS DEVELOPMENT THAT IS HAPPENING IN THE ANATOMY COMMUNITY, AND YOU WOULD THINK THAT ANATOMY WOULD BICISM TOLL STANDARDIZE BUT IN FACT, MUCH LIKE MANY OTHER COMMUNITIES WE ARGUE VERY MUCH ABOUT HOW TO REPRESENT ANATOMY. SO EVERYONE'S HEARD THE QUOTE AND I WANT TO BE THE FIRST THING TO SAY IT AT THIS MEETING, THE NICE THING ABOUT STANDARD SYSTEM YOU HAVE SO MANY TO CHOOSE FROM. WHEN YOU LOOK FOR ANATOMY AND A VARIETY OF RESOURCES, YOU SEE THERE ARE VERY MANY OF THEM. THERE ARE AT LEAST 66 IN THE VITAL RESOURCE AS WELL AS IN BIOSHARING. THEY'RE NOT THE SAME 66 ODDLY ENOUGH AND SOME OF THEM ARE OBSOLETE AND SOME OF THEM ARE NOT ORGANISMS IN ANOTHER PLACE, SO THIS IS THE LANDSCAPE OF A FEW RESOURCES AND RESEARCH AND PLACES THERE ARE VERY MANY MORE. SO WE ACTUALLY WORK IN A LARGE COMMUNITY THAT STARTED OFF WITH A NUMBER OF SPECIES SPECIFIC ANATOMY ONTOLOGIES AND THE GOAL FOR THESE WAS LARGELY TO ANNOTATE GENE EXPRESSION AND PHENOTYPIC DATA WITHIN THE CONTEXT OF MODEL ORGANISM DATABASE RESOURCES AND LET'S INCLUDE THE ZEBRAFISH OR THE SWRKS FA WHICH I WAS INVOLVED WITH THE MOUSE ANATOMY ONTOLOGY AND A FEW OTHERS AND THIS--THIS DATED BACK ALL THE WAY TO ABOUT 2006. THESE EFFORTS WERE COMPLETELY UNCOORDINATED AT THAT TIME EVEN THOUGH OF CAUSER ZEBBIA FISH AND MICE SHARE COMMON ANAA TOMICAL FEATURES SUCH AS BRAINS, SPINAL CORDS, VERT BRA, ET CETERA, THESE WERE NOT RELATED TO ONE ANOTHER NOR TO ANY OTHER POTENTIAL VOCABULARIES THAT WERE OUT THERE. SO THEN A NUMBER OF OTHER RESOURCES STARTED TO BE DEVELOPED. WE SAW THE XAO, AS WELL AS EFFORTS TO START DEVELOPING MULTISPECIES TO REPRESENT LARGE SWATHS OF TAXA, SO THE AMPHIBIAN ONCOLOGY REALLY AIMED TO REPRESENT A LARGE NUMBER OF TAXA FOR THE COMPARATIVE EVOLUTIONARY BIOLOGY. THESE RESOURCE HIS A NUMBER OF DIFFERENT METHODS OF DEVELOPMENT THAT THEONATOLOGY BASICALLY CLONED THE ZEBRAFISH ANATOMY ONTOLOGY AND WENT AND DIVERGED THE AMPHIBIAN ONTOLOGY WAS ON THE OWN PATH AND HAD NO INTEROPERABILITY REALLY THAN ANY OTHER WHICH IS DIDN'T START WORKING WITH UNTIL QUITE LATE IN ITS DEVELOPMENT IN AROUND 2009, CHRIS MONTHLE IS I STARTED WORKING ON THE UBER ONTOLOGY AND WE REFER TO UBER-ON AND THESE TRY TO ALIGN THESE DIFFERENT EFFORTS TO SAY WE'RE GOING TO TRY TO SUBSUME THESE EFFORTS SO WE CAN WALK ACROSS SPECIES OF THE PURPOSE OF DISCOVERY, AND THE UBER-ON DESPITE THE NAME WAS MET WITH ENORMOUS SKEPTICISM AND WE HAD REGULAR BATTLES WITH MANY DIFFERENT PEOPLE TO TRY TO EXPLAIN WHAT IT WAS THAT WE WERE TRYING TO DO BECAUSE IT DIDN'T MEET X, Y, OR Z KRIST TERYARKS IN THE END WE TRY TO WORK WITH THE COMMUNITIES TO TRY TO UNDERSTAND HOW COULD WE MAKE THIS ONE SIMPLE SEMANTIC FRAMEWORK THAT EVERYONE COULD CONTRIBUTE TO AND EVERYONE COULD USE SO THAT THE EFFORTS THAT IT TAKES WHICH ARE VERY LARGE TO DEVELOP THESE VOCABULARIES COULD BE DISTRIBUTED AND SHARED AND MADE INTEROPERABLE SO BACK IN 2012 AFTER ABOUT TWO YEARS OF NEGOTIATIONS WE CONVINCED EVERYONE THEY COULD STOP THE MULTISPECIES IN ONTOLOGIES AND INSTEAD ALL WORK ON UBERON TOGETHER SO THE SIZE OF THE BARS REPRESENTS THE SIZE OF THE VOCABULARIES. SO IT'S--IT'S BEEN A LONG ROAD TO DO THIS SORT OF THING AND IT'S, YOU KNOW AS PETER SO NICELY SAID BEFORE THERE ARE HUGE TECHNICAL ISSUES IN HOW WE'VE MANAGED TO DEAL WITH THE--WHAT CHRIS LIKES TO REFER TO AS SVN TUNNELS FOR LACK OF PEOPLE PROVIDING QUALITY AVERS CONTROL, NOT HAVING THE RIGHT TOOLS TO MANAGE THEIR CONTENT, JUST A LOT OF TIME HAD TO BE RECONCILING THE SITUATION, BUT IT HA H TO BE LARGER SO THAT PEOPLE COULD UNDERSTAND THAT IT'S CONTRIBUTING TO I CENTRAL THING AND IT'S MORE RESPONSIBLE AND MORE EFFECTIVE THAN THE SINGLE PROTONLE OWNERSHIP OF ALL THE INDIVIDUAL PIECES AND THAT TOOK YEARS AND YEARS OF EXPEFORT THEN THERE'S THE FINANCIAL COMPONENT. MOST OF THIS WORK WAS NEVER FUNDED. SO THIS IS A LABOR OF LOVE BECAUSE THERE'S PEOPLE WITH DATA THAT WANT TO ADDRESS THEIR DATA NEEDS. SO FIGURING OUT HOW DO WE INSPIRE PEOPLE TO NOT TAKE THE YEARS AND YEARS FOR THE SOCIAL PAR, HOW DO WE GET PEOPLE INCENTIVIZED TO PARTICIPATE AND COLLECTIVE DEVELOPMENT AND ALONG THE LINES OF WHAT PEOPLE WAS TALKING ABOUT BEING AND TAKING A TEAM APPROACH HERE. SO THAT'S MY OWN PARTICULAR TALES OF STANDARD DEVELOPMENT WOES AND I WILL PASS IT ON TO CHRIS MONTHLE SO HE CAN INSPIRE HOW HE CAN FIX IT. >> OKAY, SO I WORK WITH MEL ICES ON THE MONARCH INITIATIVE BUT I'M ALSO A PART OF OTHER AND NONFUNDED PROJECT SUCH AS THE GENE ONTOLOGY PROJECT WHICH ARE INVOLVED FOR A WHILE AND OPEN BIOMEDICAL ONTOLOGIES ALONG WITH PEOPLE IN THIS ROOM AND AS PART OF MY WORK WITH THESE GROUPS, I DO A LOT OF WORK IN THE AREA OF STANDARDS RELATED TO ONCOLOGYS BOTH THE FRAMEWORKS UNDERTHE PINNING ONTOLOGIES AND AS A STANDARD THEMSELVES, BUT I'M ACTUALLY NOT GOING TO TALK ABOUT THAT, I WILL TALK TO YOU ABOUT ONTOLOGIES, I DO THAT ALL THE TIME SO I'M GOING TO KIND OF TALK FIRST OF ALL ABOUT A MUCH EARLIER STANDARD EFFORT I WAS INVOLVE WIDE AND THIS WAS UNDER THE AH SPIESS OF THE GROUP CALLED THE OMG WHICH WAS [INDISCERNIBLE] A MAJOR PLAYER IN THE KIND OF LIKE SOFTWARE STANDARDS ARENA, THEY'RE ACTUALLY THE CREATORS OF SOMETHING CALLED DML, THE UNIFIED MOLLLING LANGUAGE WHICH IS NOT TO BE CONFUSED WITH MLS, WHICH IS A MAIN FRAMEWORK FOR YOUR STANDARDS AND YOUR MODELS. SO WHAT--WHAT HAPPENED UNDER THE AUSPIESS OF OMG IS A TASK FORCE INITIATING LIFE SCIENCES AND A TASK FORCE ROUND ABOUT 19 DISMEVEN THEY WERE GOING TO CREATE A PLAN--1997 AND THEY WERE GOING TO CREATE A MODEL FOR ALL OF BIOLOGY SO WE COULD HAVE INTERLOCKING SOFTWARE COMPONTS WORK TOGETHER AND THEY'RE PART OF A GROUP CALLED RT-GTWO GENOME MAPS AND WE WERE CHARGED WITH THE POSITION TO REPRESENT GENES TO LINKAGE MAPS RADIATION AND NOT CYTOGENIC MAPS AND THIS WAS JUST BEFORE THE SEQUENCING OF THE HUMAN GENOME SO IF YOU IMAGINE AFTER THE HUMAN GENOME SEQUENCE, PEOPLE DIDN'T CARE SO MUCH ABOUT YOUR POSITION OF YOUR GENES, MAYBE NOT--TO A LARGE EXTENT. SO, I KIND OF ACTUALLY BAROD THESE SLIDES FROM ONE OF THE OTHER PRESENT ANTICIPATIONS UP ON THE SITE SO THEY HAD A PLAN OF EVERYTHING, KIND OF LIKE HAPPENING, YOU KNOW ALL THESE GROUPS WORKING ACCORDING TO THIS NICE TIME TABLE HERE BUT LIKE--LIKE THE BEST OF MANY PLANS THEY GO AWRY SO UNFORTUNATELY A LOT OF THE WORK THAT WAS DONE THERE WAS WASN'T REALLY ADOPTED AND LARGELY FIZZLED OUT AND UNFORTUNATELY MOST OF THE WEB SITE IS GONE SO CAN YOU SEE MANY OF THE EXCELLENT CONTRIBUTIONS THAT THE PEOPLE WHO WERE LIKE FLYING AROUND TO MEETINGS ALL OVER THE WORLD MADE TO THIS EFFORT. BUT AS WELL ONE WEBPAGE THAT IS STILL UP AND THAT'S THE FAQ AND IT'S UPDATED FOR A WHILE SO THAT THE LOVELY 1990 STYLE PAPPYEROUS BACKGROUND SO THE REASONS I WOULD ARGUE FOR THE LACK OF SUCCESS IS UNDER SECTIONS TITLED WHY SHOULD THIS STANDARDIZATION SUCCEED WHEN OTHERS FAILED SO THEREYA I HISTORY HERE. THERE'S REFERRING TO HISTORY BEFORE OF PREVIOUS ATTEMPTS OF STANDARDIZATION ATTEMPTS HAVE FAILED AND I WOULD ARGUE TWO OF THE MIN REASONS ARE RELIANCE AND TECHNOLOGY WAS TO EFFECTIVELY, IT WAS ORIGINALLY TOO COMPLEX TO BEGIN WITH AND IT WAS OBSOLETE AND PLANTED BY OTHER TECHNOLOGY AS IS OFTEN THE CASE WITH TECHNOLOGY, AND ALSO, I THINK IT IS IMPART DUE TO THE DEVELOPMENT PROCESS TO WHICH THESE STANDARDS WERE DEVELOPED AND THIS IS WHAT IS OFTEN CALLED KIND OF A WATER FALL MODEL AND A NUMBER OF STEPS, I MEAN THEY KIND OF--MEAN IT'S THAT WATER TRICKLES FROM ONE STEP TO THE NEXT SO MAYBE NOT QUITE WATERFALLS SO MAYBE MORE OF A STREAM OR A GLASSIER FOR THAT, EARTH SCIENCES. AND SO, WE'RE HERE, I'M SURE WE'RE HERE, LATER IN THIS SESSION, FOR SOME OF THE PROBLEMS THAT ARISE IN EACH OF THESE INDIVIDUAL KIND OF LIKE SEGMENTS AND WHEN YOU HAVE MULTIPLE PEOPLE INVOLVED, TRY TO COORDINATE ALL THIS AND EVENTUALLY THE IDEA IS AT THE END OF THESE SEQUENTIAL STEP THEY HAVE A BIG EXPLOSION WHICH IS YOUR PRODUCT ON THE MARKETPLACE, BUT BY THEN THE TECHNOLOGY THAT YOU ARE TARGETING IS OBSOLETE AND THERE'S NOT A LOT OF ROOM IN THE CYCLE FOR A FAST, DYNAMIC APPROACH, CHANGING REQUIREMENTS AND KIND OF LIKE DYNAMIC FEEDBACK AS YOU GO ALONG. SO THIS WAS BUILDING UP A PRIVILEGE AND SAY, I'M GOING TO MOVE IT TO THE LEFT SO WHEN YOU'RE WORKING YOU HAVE INFORMATION AND YOU CAN AFFORD IN THE BASIC SCIENCES TO BE A BIT MORE FLEX AND I BELIEVE THIS IS ACTUALLY, I WOULD ARGUE ONE OF THE DEFAULT MEANS OF PRODUCTION FOR MANY KINDS OF KNOWLEDGE ARTIFACTS AND NOT JUST SOFTWARE AND STANDARDS BUT IN THE LAST 15 YEARS THERE'S BEEN MORE OF A QUIET REVOLUTION IN SOFTWARE AND JUST KIND OF KNOWLEDGE WORK IN GENERAL AND KIND OF FIRST SHOW UP IN KIND OF LIKE THE WORLD OF--KIND OF LIKE THE KING OF THE SOFTWARE, STARTING WITH METHODOLOGIES, STREAM PROGRAMMING LIKE EVOLVED IN SOMETHING CALLEDDAGEILE AND A BUNCH OF RELATED METHODOLOGIES, MORE LIGHT WEIGHT, KIND OF STANDARDS, AND ALSO LIKE WAYS OF NEW DEVELOPING SOFTWARE THAT WE JUST CAN'T RELY LESS ON SOME OF THE MORE DIFFICULT ASPECTS LIKE UML AND THIS METHODOLOGY EMPHASIZES THE WATER MODAND HE WILL THE IDEA IS THAT YOU INVOLVE THEM IN THE AND YOU'RE TESTING IT ADAPTING IT AS YOU GO ALONG. AND SO THERE'S KIND OF AN ANALOGY IN HOW WE DISSEMINATE OUR KNOWLEDGE IN GENERAL AND WE'RE GONE FROM A TOP DOWN KIND OF APPROACH WHERE, YOU KNOW YOU HAVE A SERIES OF PUBLISHERS UP ON HIGH WHERE WE HAVE MORE MODELS LIKE WIKI AND ALTERNATIVE WAYS OF PUBLISHING AND NEW WAYS OF COMMUNICATION AND MORE HORIZONTAL TO TWITTER TO STACK OVERFLOW IN THE TECHNICAL REALM AND TO A CERTAIN EXTENT I WOULD ARGUE THIS AS BEING THE CASE FOR STANDARDS AS WELL, AT LEAST IN THE BASIC SCIENCES WE'VE SEEN--SEEN MORE AT A CONSCIENCELY OR UNCONSCIENCELY FOR ANAGEILE APPROACH WHERE I HAVE THIS NEED TO SHARE THIS DATA WITH MY COLLEAGUES AND MAKING UP A STANDARD PEOPLE START USING IT AND THEN THEY DISCOVER LIMITATIONS AND THEY KIND OF ADAPT AS THEY GO ALONG SO MANY OF THE GENOMIC STANDARDS I'VE BEEN INVOLVE WIDE LIKE GSS, FOLLOW THIS MODEL AND ALSO TO ARGUE THAT THEY'RE KIND OF LIKE COMMUNITY ONTOLOGIES STARTING WITH GENE ONTOLOGIES SUCH AS THE ONES THAT MELISSA MENTIONED BUT WHEN YOU HAVE THIS LACK OF A FORMAL MODEL AND LACK OF A TOP DOWN APPROACH, THE WORRY IS HOW DO YOU CONNECT THOSE EFFORTS, HOW DO YOU NOT JUST AN ARCH SCHEHOW DO YOU ENFORCE THE GENDER SENSE OF COMMUNITY AND PEOPLE WORKING TOGETHER, SO I THINK SOMETHING REALLY MAJOR I THINK IN SOFTWARE AND JUST KNOWLEDGE WORK IN GENERAL IS IT'S GOT NEW WAY OF WORKING WITH ARTIFACTS LIKE SOFTWARE AND SCHEMA AND RATHER LIKE KEEPING IT LOCALLY ON YOUR HARD DRIVE AND MAYBE SHARING OCCASIONALLY RELEASES WITH YOUR COLLEAGUES, THERE'S--THERE'S NEW WAYS OF DOING THIS INVOLVING DISTRIBUTED CONTROL SYSTEMS LIKE GIT AND I USE GITHUB BECAUSE I'M NOT PAID BY THEM OR ANYTHING BUT I THINK THEY'RE THE BEST TO EXEMPLIFY THIS NEW WAY OF SHARING KIND OF SOFTWARE AND SHARING WITH OUR COLLEAGUES AND WHAT IT DOES WHICH IS VERY NICE IS IT MEANS THE PROJECTS UP THERE, THEY'RE GENERALLY OPEN AND I CAN GO INTO ANY PROJECT AND GITHUB AND I CAN DO G TO FORK REPOSITORY SO I MAKE MY OWN COP NEUROECTODERMAL MY WORK SPACE AND THAT'S PUBLIC AND TRANSPARENT FOR EVERYONE TO SEE, KIMAKE LOCAL MODIFICATIONS, AND THE CHANGES ARE TRAPPED, I CAN THEN MAKE A POOR REQUEST BACK TO THE ORIGINAL SOURCE AND THEY CAN EITHER SAY, NO WE'RE NOT GOING TO FIX THIS OR OKAY THIS, IS GREAT, WE WILL PULL THIS IN AND THE ENTIRE CHAIN OF COMMENTS AND MODIFICATIONS I MADE ALONG THE WAY, IS PRESERVED IN THE AUDIT TRAIL. SO THAT IN THE FINAL PRODUCT, YOU CAN HAVE THE AUDIT TRAIL, THE FULL SET OF CONTRIBUTIONS THAT EVERYONE MADE FOR THIS, SO DOES THIS WORK IT SOFTWARE, I THINK IT DOES WORK FOR SOME STANDARDS EFFORTS AND I REALIZE STANDARDS ENCOMPASSES THE DIFFERENT TYPES OF KNOWLEDGE ARTIFACTS, ONE ONGOING THING THAT'S INTERESTING AN EFFORT WE'RE INVOF WIDE CALLED THE GLOBAL ALLIANCE FOR GENOMIC HEALTH AND THEY'RE USING GITHUB FOR MANAGING ALL OF THEIR DOCUMENTS AND THE ACTUAL APIs AND SCHEMAS THEMSELVES AND I THINK THE RELEASE OF INTERESTING, IT'S NOT REAL LYE AN EXPERIMENT THEY'RE DOING BECAUSE IT'S VERY, VERY PRACTICAL BUT I THINK WE'RE SEEING SOMETHING INTERESTING UNFOLD HERE. AND IF YOU JUST SEE A LOT OF KIND OF ACTIVITY THAT IS OFTEN INDEPENDENT--IT CAN HAPPEN INDEPENDENTLY AS PEOPLE ON TOP MANAGING THINGS, PEOPLE JUST NEED TO GET THINGS DONE OR WORKING ON THINGS, IF THEY HAVE THE CRAZY IDEA THAT WE WANT TO REPRESENT FEATURES RATHER--ON A SEQUENCE RATHER THAN USING LINEAR COORDINATES WE CAN USE THE MODEL, MAKE CHANGES AND THEN SEE IF THAT WORKS AND DISCUSS IT WITH PEOPLE OPENLY. SO, CAN WE--CAN WE EXTEND THIS MODEL TO OTHER FORMS OF STANDARDS AND I THINK WE NEED BETTER TOOLS HERE, SENTIALLY, I'VE BEEN USING GIT HUB A LOT FOR ONTOLOGY DEVELOPMENT AND IT WORKS TO SOME EXTENT BUT REALLY, IF YOU'RE--THESE SYSTEMS WERE DESIGNED PRIMARILY FOR MANAGING LIKE CHUNKS OF ASKY TEXTS SO IF YOU HAVE A PROGRAM THAT EXISTS LIKE A SOURCE COAT OR SCHEMA FOR EXCHANGING VARIANT DATA THAT WORKING WELL FOR A COMPLEX LIKE ONTOLOGY, WE NEED EXTENSIONS TO MAKE THAT WORK. SO JUST TO ROUND OFF, EVERYTHING APPLIES TO ALL KINDS OF SCIENCES AND THESE ARE TRANSLATIONAL MEDICINE, COMMUNITY STANDARDS ARE VERY BROAD TERM AND ONE OF THE ONGOING CHALLENGE WITH THIS PEER BASED MODEL IS HOW DO YOU--HOW DO YOU BALANCE THIS NOTION OF PEER BASED DEVELOPMENT WITH ANY--WITH SOMETHING A SET OF PRODUCTS THAT ARE CO HERENT AS A WHOLE. AND ONE THING I LIKED ABOUT WHAT WE TRY TO DO INITIALLY WITH THE OBO FOUNDRY BUT TRY TO MAKE A MAP OF THE TERRITORY TO SEE WHERE EVERYONE SITS RELATIVE TO EVERYONE ELSE, TO FIGURE OUT WHO NEEDS TO BE TALKING TO WHO, WHO NEEDS TO BE DEFINING BOUNDARIES WITH WHO ELSE AND I THINK WE REALLY NEED A KIND OF MAP LIKE THAT IN BASIC SCIENCES HERE AND IT'S ALWAYS DIFFICULT TO BUILD YOUR STANDARDS IN A WAY THAT WILL ANTICIPATE FUTURE DEVELOPMENTS AND THERE'S A CHALLENGE BETWEEN HAVING THEM BE EXTENSIBLE VERSES SOMETHING SIMPLE AND FECES THE NEEDS AND REQUIREMENTS OF TODAY, AND ONTOLOGIES ARE DEFINITELY A MAJOR HELP HERE, IF YOU PUT MORE OF YOUR STANDARD INTO AN ONTOLOGY, IT MAKE ITS MORE FUTURE PROOF AND THAT'S THE SEQUENCE ONTOLOGY WHICH DESIGNED 10 YEARS AGO AND MORE SUPPORT FOR GFTHREE FORMA IS REUSED WITHIN THE GLOBAL ALLIANCE SCHEME AS WELL. AND I GUESS JUST--JUST FINALLY, I THINK AT THE END OF THE DAY, WE NEED SPECIAL WAYS OF WORDING PEOPLE WHO WORK ON THESE STANDARDS. MOST OF US HERE, YOU KNOW WORK FOR PUBLICATIONS AND THAT'S HOW WE'RE--YOU KNOW OUR WORK SPACE IS MEASURED AND I THINK WE, YOU KNOW, IT'S ALL VERY WOW LIKE WE SAYING SOMETHING LIKE GIT HUB, OTHERS CAN SEE, YOUR CONTRIBUTIONS UNLESS THAT'S MORE WIDELY RECOGNIZED AND THAT'S ENOUGH INCENTIVES FOR MANY PEOPLE TO TAKE PART IN THESE EFFORTS. AND SO, THAT, I THINK WILL REQUIRE SOME KIND OF INVESTMENT FROM KIND OF LIKE FUNDING INSTITUTES AND POSSIBLY OTHER SOURCES AS WELL, LIKE VARIOUS KIND OF LIKE COMMERCIALENTITIES AND PHARMAS AND SO ON, BENEFIT FROM THESE EFFORTS SO MAYBE THERE'S WAYS IN WHICH THEY COULD HELP CONTRIBUTE AS WELL. >> WE CAN TAKE A QUICK COMMENT OR DISCUSSION BUT WE MAY SAVE MOST OF THEM FOR THE END. >> YOU GUYS HAVE IT ALL FIGURED OUT SO MICHELLE. >> [INDISCERNIBLE]. >> CAN YOU REPEAT THE QUESTION? >> WE HAVE SO MANY OF THESE ONTOLOGIES, I THINK 400 DIFFERENT ONTOLOGIES, SO MANY AREAS OVERLAP, HOW CAN WE PUT THIS ALL TO USE, I THINK WAS YOUR QUESTION. AND I WOULD SAY MAYBE IT'S A SLIGHT FAILURE TO BEGIN WITH THAT WE HAVE SO MANY ONTOLOGIES, THERE SHOULD BE LESS OF THOSE AND PEOPLE SHOULD NOT BE DOING THE SAME WORK WHERE THEY COULD BE USING EACH OTHER'S WORK SO THERE'S BETTER MECHANISMS FOR PEOPLE TO BE ABLE TO COMMUNICATE TOGETHER AND SAY, HEY I'M WORK ON THIS PORTION HERE, WHY DON'T WE--LET'S USE THIS PART FROM YOU AND YOU USE THIS PART FOR ME, LET'S NOT DUPLICATE THE EFFORT THERE SO THERE NEEDS TO BE BETTER MECHANISMS FOR SHARING. >> I THINK THAT'S TRUE FOR BOTH SOCIAL MECHANISM AS WELL AS TECHNICAL MECHANISMS AND I THINK THAT'S ONE OF THE DISCUSSION ITEMS THAT WILL COME UP AGAIN AND AGAIN. OKAY, SO NEXT I WOULD LIKE TO INTRODUCE CRIST DECK STOECKERT AND I'M INTERESTED TO HEAR WHAT HE HAS TO SAY,. >> THANK YOU MELISSA, SO I WILL SHARE THREE DIFFERENT KINDS OF EXPERIENCES I'VE HAD OVER THE PAST DECADE OR MORE STARTING WITH THE EARLY DAYS WHERE WE WERE STARTING TO DEAL WITH BIG DATA AS BEING GENERATED BY NEW TECHNOLOGIES AT THAT TIME. IT WAS MICROARRAYS AND SO THERE WAS A REAL PANIC GOING ON AT THAT TIME, WHAT ARE WE GOING TO DO WITH ALL THIS DATA. HOW ARE WE GOING TO MAKE SURE IT DOESN'T GET LOST, AND IT WASN'T JUST INVESTIGATORS GENERATING THE DATA BUT LEADING JOURNALS LIKE NATURE AND SELL AND SCIENCE AND SO FORTH, THEY WERE VERY CONCERNED AND THERE WAS A LOT OF PANIC ABOUT WHAT ARE WE GOING TO DO WITH THIS AND THAT LED US TO CREATE THE MINIMAL INFORMATION ABOUT A MICRO EXPERIMENT OR MIAMI BY I MEAN WE THE M-GEN SOCIETY AND WE COLLECTED INFORMATION ABOUT WHAT SHOULD BE A MINIMAL CHECK LIST FOR THE STANDARD AND THAT LED TO--THAT FOUNDATIONAL STANDARD OF MIAMI LED TO AA SERIES OF MINIMAL STANDARDS THAT ARE NOW COLLECT INDEED A WEB SITE CALLED MIBBI, BUT THAT WAS A CHECK LIST AND WE NEEDED TO PROVIDE THE SEMANTICS THAT WOULD GO ALONG WITH SHARING DATA SUCH AS MICROARRAY OR HIGH THROUGH PUT SEQUENCING DATA AND WHEN WE STARTED THIS, IT TURNED OUT THAT IT WASN'T JUST THE OMICS PEOPLE WHO NEEDED THESE SORTS OF STANDARDS IT WAS EVERYONE WHO WAS DOING BIOMEDICAL INVESTIGATIONS OF DIFFERENT TYPES WHICH LED TO OUR ONTOLOGY FOR BIOMEDICAL FBIOMEDICAL INVESTIGATIONS OBI WHICH LATER BECAME PART OF THE OBO FOUNDRY THAT CHRIS JUST MENTIONED. SO THESE ARE MORE SOPHISTICATED STANDARDS AND WHAT WAS GREAT ABOUT THEM IS THAT WE WERE USING BEST PRACTICES AND COMMON APPROACHES TO DO THIS. WE HAD WORKSHOPS WHERE WE BROUGHT TOGETHER VARIOUS COMMUNITIES WITH COMMON NEEDS. BUT FOR EVERY COMMUNITY THAT WE'RE ABLE TO SORT OF BRING TO BEAR TO WORK TOGETHER, THERE ARE MANY MORE COMMUNITIES WHO ARE DOING--HAVE THE SAME PROBLEMS AND DOING IT ON THEIR OWN. AND YOU KNOW THERE WERE GOOD REASONS FOR THEM TO DO IT ON THEIR OWN. THEY COULD DO IT MUCH FASTER, RIGHT? THEY COULD PUBLISH A PAPER, CLAIM VICTORY, GET CREDIT, MOVE ON. IT WASN'T REALLY ALWAYS CLEAR WHY THEY SHOULD JOIN US AND DO IT OUR WAY AS OPPOSE TO JUST DOG IT THEIR WAY. SO, ONE OF THE THINGS THAT HAS BECOME CLEAR IN WORKING ON THIS KIND OF STANDARD AND THE MOTIVATION IS TO SHARE DATA, IT BECAME CLEAR THAT TO DO THIS, YOU NEED TO SHARE IN A WAY THAT CAN BE SYSTEMATICALLY PROCESSED AND THAT'S LED ME TO BECOME INVOLVED IN ANOTHER TYPE OF STANDARD WHICH IS REALLY AN APPLICATION STANDARD AND SO, THAT'S WORK THAT'S BEING SUPPORTED BY ONE OF THE NIH INSTITUTES NIAID FOR THEIR WORK WITH GENOME SEQUENCING CENTERS OF INFECTIOUS DISEASES AND BIOINFORMATICS RESOURCE CENTERS. WE HAVE A METADATA WORKING GROUP. SO THE IDEA THERE IS TO--SENSE WE HAVE TO SHARE THIS DATA IN A WAY THAT CAN BE SYSTEMATICALLY PROCESSED, HOW ARE WE GOING TO DO THAT AS WE COLLECT DATA THAT COMES INTO THE SEQUENCING CENTERS AND GETS DISTRIBUTED THROUGH A BIOINFORMATICS RESOURCE CENTER, I'M INVOLVED IN ONE FOR PATHOGENS CALLED UP A TDB, SO THE WAY WE WENT ABOUT T WAS RATHER THAN TRY TO MAKE UP OUR OWN STANDARDS, WE TRIED TO TAKE EXISTING STANDARDS THAT COVERED OUR NEEDS AND AND THAT'S WHAT I MEAN BY AN APPLICATION STANDARD, RATHER THAN REINVENT STANDARDS WE WOULD TAKE IT LIKE OBI, WE WOULD MIX THE GENOMES STANDARDS CONSORTIUM A&E LIST AND SO FORTH TO COME UP WITH A REPRESENTATION OF WHAT IT IS WE NEEDED FOR THIS PARTICULAR APPLICATION. THE PROBLEM THAT WE'RE RUNNING INTO NOW IS HAVING THIS APPLICATION STANDARD, HOW DO WE GET METADATA FROM PROVIDERS TO FIT THOSE STANDARDs BECAUSE THEY'VE BEEN COLLECTING DATA THE WAY THEY COLLECT IT AND NOW WE'VE GOT THIS STANDARD, HOW DO WE BRING THE TWO TOGETHER. SO, JUST TO GO BACK AND LOOK AT THE PROS AND CONS, ASSOCIATED WITH THESE DIFFERENT EXPERIENCES, WHAT WORKED ABOUT MIAMI IS THAT IT WAS SIMPLE AND WE HAD A LARGE BUY-IN AND IMPACT, BUT EVEN THOUGH WE HAD SUCCESS IN A LOT OF WAYS, THERE--IT'S STILL--WE AND OTHER VS GONE BACK AND LOOKED AT COMPLIANCE, IT'S ONLY ABOUT ABOUT 50%, SO GLASS HALF FULL, GLASS HALF EMPTY, THERE IT'S GREAT THERE'S THAT MUCH BEING SAVED BUT THERE'S A LOT BEING MISSING. WHY IS THAT? WELL, EVEN THOUGH IF YOU GO TO ALL THESE JOURNALS AND YOU LOOK AT INSTRUCTIONS TO AUTHORS, THAT ARE REQUIRED TO MAKE THIS DATA AVAILABLE, IT'S NOT ENFORCED. REVIEWERS ARE ASKED TO CHECK ON THESE THINGS, BUT A LOT OF THEM SLIP THROUGH THE CRACKS. OUR EXPERIENCE WITH OBI, AND THE OBO FOUNDRY, WE'VE HAD A LOT OF BUY-IN AND THAT'S LED TO THE DEVELOPMENT OF A LOT OF TOOLS THAT OTHERS HAD BEEN ABLE TO USE FOR SHARING TERMS BETWEEN ONTOLOGIES IMPORTING TERMS, GETTING CREDIT FOR THOSE TERMS AND BEST PRACTICES BUT WHAT'S BEEN REALLY OUR BIGGEST OBSTACLE HAS BEEN NO CENTRAL FUNDING. SO WE HAD ABOUT A BACKBONE ORGANIZATION, BUT WE I DON'T UNDERSTAND HOW A BACKBONE ORGANIZATION FUNCTIONS LEAK OUTREACH AND SO FORTH CAN BE ALIGNED TO OUR DAY WORK. YOU KNOW I'M A PROFESSOR SO GETTING--SITTING ON THE SORTS OF ADMINISTRATIVE DAY-TO-DAY OPERATIONS THAT WE HAVE TO DO, JUST DOESN'T ALIGN WELL, AND THAT SHOWS BECAUSE AS WE HAVE OUR WEEKLY OR MONTHLY CALLS AND A NUMBER OF US ARE INVOLVED IN THIS, YOU KNOW IT'S--IT'S A CRAP SHOOT, IT'S WHO ACTUALLY HAS TIME TO SHOW UP THAT DAY. RIGHT? IT'S SOMETHING YOU FIT IN, AROUND YOUR DAY JOB. I JUST DON'T SEE HOW IT CAN BE ALIGNED. SO WHAT WE'RE DOING NOW, WITH THIS APPLICATION STANDARD BY THE NIAID, THAT IS PART OF MY DAY JOB. I AM FUNDED BY THE NIAID TO WORK ON THIS. AND I THINK WE'VE DONE SOME GOOD THINGS BY REUSING AND EXISTING STANDARDS AND WE HAVE THE BOF NIAID, BUT IT IS A WORK IN PROGRESS AND NOW WE'RE TYING TO TACKLE A VERY DIFFICULT ISSUE, CRITICAL METADATA AND MOW DO WE TAKE THAT AND INTEGRATE IT WITH THE OTHER TYPES OF THINGS WE'RE DOING AND SO, AGAIN PART OF THE PROBLEM IS STRUGGLING WITH TOOLS TO MAKE THIS, TO IMPLEMENT THIS. HOW DO WE TAKE THE DATA DICTIONARIES THAT PEOPLE HAVE BEEN USING ON THEIR OWN AND TAKE THAT DATA AND FIT IT TO THE STANDARD WE'VE COME UP WITH. SO WHAT'S THE CONCLUSION FROM THIS? I THINK DATA STAN ARDS ARE A LOT LIKE DENTISTRY, YOU NEED THESE TO KEEP FROM LOSING BITS GOING OUT OF ALIGNMENT, ATTRACTIVE FOR SHARING BUT IT CAN BE VERY LABORIOUS AND PAINFUL TO APPLY AND I MEAN, FACE IT WHEN YOU TALK TO PEOPLE, IT'S HARD TO GET THEM EXCITED NOW ABOUT ABOUT STAN ARDS. AS I MENTIONED EARLY ON,Y WITH MIAMI PEOPLE WERE SCARED ABOUT WHAT WOULD HAPPEN TO THEIR DATA BUT NOW I THINK THERE'S LESS OF THAT. SO ONE OF THE BIGGEST CHALLENGE SYSTEM HOW DO WE GET PEOPLE MOTIVATED ENOUGH TO PUT IN THE EFFORT TO MAKE IT WORTH INVESTING SO THAT--HOW DO WE--AND I THINK THIS KIND OF GOES TO ONE OF THE POINTS THAT WAS RAISED BEFORE IN MICHELLE'S QUESTION IS WE NEED TO HAVE WAYS THAT IF PEOPLE DO APPLY THESE STANDARDS, YOU KNOW THAT THERE'S SOME CLEAR OUTCOME OR BENEFIT THAT THINGS THAT PEARLY WHITE AND ATTRACTIVE AND SO I THINK WAWE NEED TO THINK ABOUT IS IF WE'RE GOING TO REQUIRE PEOPLE TO USE STANDARDS AND I THINK WE DO NEED TO DO THAT, WE NEED TO DO IT IN A CLEAR WAY SO THAT YOU SUBMIT YOUR STANDARDS, YOUR DATA EXCUSE ME, FOLLOWING SOME STANDARD, THEN HAVE YOU A CLEAR OUTPUMOF WHAT'S WRONG WITH YOUR DATA AND HOW IT CAN BE IMPROVED AND MAKE THESE SORTS OF TOOLS AVAILABLE TO REVIEWERS SO THAT THEY ACTUALLY CHECK COMPLIANCE SO THAT IF A JOURNAL DOES SAY, YOU NEED TO PROVIDE YOUR DATA IN SOME WAY MEETING YOUR STAN ARD EASY TO USE TOOL THAT HAS AN OUTPUT THAT EVERYONE CAN UNDERSTAND. WHAT THAT MEANS THOUGH IS THAT IT'S GOING TO HAVE TO RAISE THE BAR IN TERMS OF STANDARDS THEMSELVES. WE HAVE A LOT OF OVERLAPPING STANDARDS SO YOU CAN'T PROVIDE A GUIDANCE TO USE ANY OF THEM. I THINK WE WILL HAVE TO NEED TO HAVE A WAY TO ADDRESS WHICH ARE THE ONES THAT PEOPLE SHOULD BE USING BUT WE SHOULD BE USING GENERIC SOLUTIONS RATHER THAN ONE-OFF SO THAT WE DON'T HAVE 10 DIFFERENT STANDARDS THAT PEOPLE ARE USING WHEN THERE CAN BE ONE. AND THEY HAVE TO MEET SOME SET OF QUALITY IF WE'RE GOING TO HAVE A VALIDATOR, THEN THEY HAVE FOLLOW PRINCIPLES THAT CAN BE EASILY AND QUANTIFIABLY TESTED. SO, IF WE'RE GOING TO TAKE THIS APPROACH, THERE'S ALSO AN EDUCATIONAL COMPONENT TO IT. PEOPLE HAVE TO LEARN WHAT IT MEANS TO MAKE A GOOD STANDARD. PEOPLE GENERATE DATA DICTIONARYS ALL THE TIME IS THEYA STANDARD? WELL, WHY NOT? THAT IS PART OF THE CONVERSATION ABOUT WHAT GOES INTO A STANDARD AND SO, I THINK WE NEED TO COORDINATE, CERTIFYING STANDARDS, THINK ABOUT HOW TO DO OUTREACH, TO MAKE AND USE STANDARDS AND I THINK WE'RE ALSO GOING TO NEED TO FIND WAYS TO SUPPORT, MAINTAINING AND DISTRIBUTING STANDARDS ALONG WITH THE VALIDATORS TO CHECK THEM. THANK YOU. [ APPLAUSE ] >> DOES ANYBODY HAVE A QUESTION BESIDES MICHELLE, MAKE IT QUICK THOUGH BECAUSE I THINK WE'RE RUNNING LATE. >> CHRIS WHAT I REALLY LIKE ABOUT THAT IS THEY TALK TO THE INFORMATION THAT CHRIS TALKED ABOUT WITH RESPECT TO ONTOLOGY FOR DEVELOPMENT BUT HERE FORANOITATIONS BUT I'VE ALWAYS WONDERED IS IT SUFFICIENT AND HOW ARE MINIMUM INFORMATION MODELS EVALUATED AND HOW DO WE KNOW THAT IT REALLY TALKS TO THE YOUTH CASES THAT ARE OF CONCERN TO THIS COMMUNITY, THE DISCOVERY OF DATA SETS THAT ARE REUSED OF DATA SETS AND SO ON. SO WHAT CAN YOU TELL US ABOUT THAT? >> DO HAVE YOU A-- >> HELLO? SO, THEY'RE NOT SUFFICIENT. THEY TELL YOU WHAT IS NEEDED BUT THEY DON'T TELL YOU HOW TO PROVIDE IT SO THE INTAX AND SYMANTICS ARE STILL NEEDED SO WITH MIAMI WE CREATED MHML AND THE MGENONTOLOGY TO CREATE THE SEMANTICS SO YOU NEED THOSE EXTRA COMPONENTS. THE REASON WE START WIDE MIRROR IMAGE AMILLIO SEWE NEED TO KEEP IT SIMPLE. THERE ARE TOOLS SO ONE OF THE MAJOR DRIEVERS BEHIND THAT WAS ELVIS BRASMA, WHO IS RUNNING ARRAY EXPRESS, SO IF YOU GO, YOU CAN SEE THAT THIS DATA SET HAS, YOU KNOW A STAR FOR EACH OF THE FIVE OR SIX MAJOR COMPONENTS OF MIAMI YOU CAN SEE WHETHER IT'S GOT, FOUR STARS OR FIVE STARS OR WHAT WHAT HAVE YOU, WHAT'S MISSING AND WHAT'S THERE. SO THERE ARE TOOLS, THERE ARE WAYS FOR A SIMPLE CHECK LIST TO SEE WHAT'S VALLEY AND THAT'S A PRETTY LO LOW BAR AND ONE THAT I THINK EVERYONE CAN ACHIEVE. SO JUST A QUICK. GO AHEAD, JESS? >> I WAS GOING TO SAN FRANCISCO THAT I THINK THAT THE QUESTION OF, YOU KNOW CHRIS SPOKE MAKELY AFTER EVALUATION AND STANDARDS BUT I THINK ONE THEN THAT WE HEARD A LOT ABOUT THAT WE HEARD AND WE SHOULD CONTINUE TO BRING UP OVER AND OVER AGAIN THAT IS AT THE HEART OF MICHELLE'S QUESTION S&P HOW DO WE KNOW WHAT WE'RE DO SUGGEST ACTUALLY USEFUL AND THAT THE--THE DATA THAT ACTUALLY IS BEING STANDARD BY THESE METHODS IS ACTUALLY MAKING IT MORE REUSABLE AND THAT WE'RE ACTUALLY SEEING A SECONDARY ANALYSIS AND THERE HAVEN'T BEEN STUDIES THROUGH THIS AND AS WE GO THROUGH THE EXAMPLES, LET'S KEEP THAT IN OUR MIND BECAUSE THAT'S THE TARGET WE'RE SHOOTING FOR SO GO AHEAD. >> MY QUESTION, IT'S NOT QUITE RHETORICAL BUT IF YOU DON'T KNOW THE ANSWER, THAT'S OKAY, I WANT TO MAKE THE POINT OF THE 50% NUMBER STRIKING ABOUT MIAMI CANT IS THAT THE ONES THAT SHARE THE DATA OR IS THAT OF ALL OF THEM THAT IS--A LOT OF JOURNALS REQUIRE YOU TO SHARE DATA AND PEOPLE DON'T. >> THAT'S RIGHT. >> SO IT'S BOTH. WE AND OTHER VS DONE SYSTEMATIC STUDIES TO LOOK AT VARIOUS JOURNALS THAT HAVE PUBLISHED SAY HIKE ROUGH ATOM ARRAY STUDIES AND WE'RE DOING A STUDY NOW OF RNA-SEQ AND LOOKING, WELL, ARE THEY AVAILABLE AND ARE THEY AVAILABLE IN SUFFICIENT FORM THAT YOU COULD ACTUALLY FIGURE OUT WHAT THEY DID, AND SO THAT'S WHERE THE NUMBER COMES FROM. >> GREAT, THANKS SO I WOULD LIKE TO WELCOME GARY BADERSPIES WHO WILL TALK ABOUT THE LONG AND CONVOLUTED HISTORY PATHWAY STANDARDS. >>, THANK YOU VERY MUCH FOR THE INVITATION AND I'M HAPPY TO BE PART OF THIS DISCUSSION AND IT'S REALLY INTERESTING. I'VE BEEN PART OF TWO STANDARDS THAT I GUESS ARE PART OF SYSTEMS BIOLOGY WHICH IS THE PROTEOMIC STANDARDS INITIATIVE OR PSIMI, FOCUSED ON MODEL CITIZEN LEAKULAR INTERACTIONS, MOSTLY PROTEIN INTERACTIONS AND ONE OF THE LEADERS BIOPACKS FORMAT AND SO THAT'S A FORMAT FOR DESCRIBING BIOLOGICAL PATHWAY INFORMATION AND BOTH OF THESE STARTED AROUND THE SAME TIME IN EARLY 2000. THOSE WERE STARTED BY COMMUNITY LEADERS AND PRIMARILY DATA PROVIDERS WHO WANTED TO MAKE THEIR DATA AVAILABLE IN STANDARD FORMAT SO IT INCURRED USE BECAUSE IF YOU'RE MAKING DATA AVAILABLE YOU WANT PEOPLE TO USE IT AND STANDARDS MAKE IT EASIER FOR PEOPLE TO USE IT SO THAT WAS A BIG MOTIVATING FACTOR AND ALSO USERS WHO REALLY WANT TO USE THE DATA AND THEY WERE MOTIVATED ENOUGH ORE INCENTIVIZED ENOUGH TO MAKE THEIR LIVES EASIER BUT THAT WAS REALLY THE ONLY-HISTORY THE PEOPLE WHO REALLY SPENDING A HUGE AMOUNT OF TIME ON THAT, I THINK AND OVER TIME, THESE YEARS I WOULD SAY, THEY'VE FULFILLED A LOT OF INITIAL GOAL AND ILLEGALS STILL PEOPLE ARE WORKING ON THEM AND A LOT OF INITIAL THINGS HAVE BEEN COMPLETED AND THEY'RE SUCCE SO PSIMI, RELEASED NUMBER OF DIFFERENT VERSIONS SO THEY HAD LEVEL ONE AND LEVEL 2.5, THEY RELEASE INFORMATION ABOUT A MOLECULAR INTERACTION AND VARIOUS OTHER TYPES OF INFORMATION, THEY RATEAD SEMANTIC VALIDATOR, CONTROLLED CO VACULARYS, STANDARD PARSERS IN VARIOUS DIFFERENT COMPUTER LANGUAGES AND THEY HAD A SCORING MECHANISM AND THEY HAD A STAN ARD WEB SERVICE CALLED SIDE KICK AND A REGISTRY SO ANYBODY WHO MAKES AVAILABLE DATA VIA WEB SERVICE GETS REGISTERED AND CAN YOU GO AND DISCOVERY ALL THE PEOPLE THAT ARE SHARING DATA AND THERE'S DOZENS OF PEOPLE SHARING DATA THAT WAY. SO THIS HAS BEEN SUCCESS EXCLUSIVELY THERE'S EXTENSIVE DATABASE AND TOOL SUPPORT. AND SIMILARLY, THE LEVELED APPROACH RELEASING LEVEL ONE AND LEVEL TWO WHICH ADDED MORE FUNCTIONALITY AND LEVEL THREE AND MAYBE THE FUTURE, ANOTHER LEVEL FOLLOWED A LOT OF WHAT PSI MI DOING, AND ALL OF THESE COMPONENTS THAT WORK TOWARDS THE USABILITY AND SUCCESS OF THESE STANDARDS OF VALIDATED OR SOFTWARE LIBRARIES, ET CETERA, AGAIN THERE'S A LOT OF DATABASE SUPPORT AND TOOL SUPPORT, PSI MI WAS PUSHED BY THE HUMAN PROTEOME ORGANIZATION AND PROTEIN INTERACTIONS CAME OUT OF THE MAS SPRY COMMUNITY WHERE IT CAME MOSTLY FROM DAYA BASES AND FUNDS FROM D. O. E. AND WORKSHOPS IN JAPAN AND MOSTLY WORKSHOP FUNDING FOR BOTH OF THESE. MOST PEOPLE PAY THEIR OWN WAY TO GO TO THE WORKSH SO PEOPLE ARE MOTIVATED ENOUGH TO PARTICIPATE WITHOUT GETTING TRAVEL REIMBURSED FOR INSTANCE AND THEN ALL THE TOOL SUPPORT WAS SPECIFIC SEPARATE GRANTS THAT PEOPLE HAD AND THEY GOT PAPERS WHICH IS AN IMPORTANT MOTIVATING FACTOR FOR SOME PEOPLE INVOLVED. SO MELISSA ASKED ME--OR ASKS US TO COMPLAIN A LITTLE BIT ABOUT WHAT THE CHALLENGES ARE THAT WE FACED AND AND I'LL MENTION SOME OF THESE AND COMMENT ABOUT HOW WE MIGHT SOLVE THESE, WHAT WE LEARN FROM THESE. SO ONE OF THE--JUST IN--YOU KNOW JUST FOR BOTH OF THESE STANDARDS ONE OF THE PROBLEMS OR CHALLENGES FOR BIOPACKS WE CHOSE A LANGUAGE FOR EARLY ADOPTERS OF THE ONTOLOGY LANGUAGE AND THERE WERE NO--IT WAS BEFORE THERE WAS A STANDARD AND THERE WERE NO TOOLS AVAILABLE FOR US TO USE AND WE DEVELOP THOSE TOOLS OURSELVES AND IT TOOK YEARS PROBABLY EXTRA, MORE TIME THAN IT NEEDED IF THOSE TOOLS WERE READILY AVAILABLE AND I THINK THAT THERE'S STILL AN ISSUE WITH LACK OF TECHNOLOGY FOR HELPING STANDARDS COMES A LONG WAY SO WE NOW HAVE A GREAT FORMATS WE CAN USE AND LEARNED A LOT ABOUT HOW TO STRUCTURE VOCABULARIES AND DEVELOP VALIDATORS SOFTWARE METHODOLOGY WAS DISCUSSED BEFORE BUT FOR INSTANCE WE'LL REALLY WANT TO BUILD COMPOSED ONTOLOGIES FOR PARTS AS CHRIS MENTIONED BUT IT'S NOT AS EASY AS IT COULD BE, THERE'S MORE TECHNOLOGY DEVELOPMENT NEEDED THAT WOULD MAKE IT EASY FOR PEOPLE TO TAKE THESE CONTROLS FROM HERE AND THIS ONE FROM HERE AND HERE AND MIX MEMORY RESPONSE TOGETHER AND YOU KNOW THEY MIGHT, YOU KNOW THEY MIGHT EVEN BE DIFFERENT LANGUAGES THAT THEY'RE WRITTEN IN, SO SO THAT'S, I THINK THERE'S STILL NEED FOR DEVELOPMENT OF THAT THAT COULD RELY HELP FUTURE STANDARDS GO FORWARD. ALSO HAD SOME SOCIAL ISSUES. DIFFERENT PEOPLE WANTING DIFFERENT THINGS AND TEMPORARY EXPERTS IN THE COMMUNITY. ONE OF THE EXPERTS WOULD COME IN. ONE EXAMPLE IS THAT THERE WAS A DATABASE PERSON WHO WAS INVOLVED IN PSI MI2.0 AND THERE WAS SO MUCH SORT OF NONEXPERTS INFLUENCING THE THINGS WORKS AND SO THERE WAS NOTHING RELEASED SO THEY MOVED TO 2.5. SO THAT WAS--AND NOW PEOPLE WONDER WHY IS THERE 2.5. SO THAT'S NOT DISCUSSED TOO MUCH BUT THERE WAS, THE THING I WILL TALK ABOUT NEXT, THIS COMBINED INITIATIVE KIND OF SEEKS TO ADDRESS THAT TYPE OF THING WITH KIND OF PROCESS ORIENTED AND THEN I THINK PEOPLE MENTIONED BEFORE THAT THESE ARE MOSTLY VOLUNTEER EFFORTS AND INITIALLY THERE'S A LOT OF EXCITEMENT IN THE STANDARDS BECAUSE OF A LOT OF DATABASES INVOLVED, OVER TIME I LOT OF THOSE DATABASES DISAPPEARED THERE, 'S ONLY A FEW, THERE'S KIND OF I GUESS CERTAIN DATABASES BECAME BIGGER AND CERTAIN ONES PEOPLE LEFT AND FOR SOME REASON, SO WE--FEWER AND FEWER GROUPS WERE INVOLVED IN THE END PROBABLY BECAUSE OF FUNDING. SO ONE OF THE NICE THENS THAT CAME OUT OF THIS, WAS COMBINED, SO THIS IS LED BY MIKE AND NICK WHO ARE VERY INVOLVED IN THE SYSTEMS BIOLOGY MARK UP LANGUAGE AND THEY REALIZE THAD THERE'S ACTUALLY A LOT OF EFFORTS AT LEAST IN SYSTEMS BIOLOGY AND ANYTHING RELATED TO NETWORKS OR COMPUTATIONAL MODELING THAT OVERLAP. SO WE WERE ALL ORGANIZING OUR OWN WORKSHOPS, PEOPLE WERE INVESTING THEIR OWN STANDARD PROCESSES EACH TIME, WE WERE DEFINITELY LEARN FREE RADICALS GENERATED EACH OTHER SO PSI, FOLLOWED A LEVELED APPROACH WHICH REALLY HELPED CREATE A NICE SCOPE INITIALLY FOR STANDARD DEVELOPMENT AND THAT WAS TAKEN FOR WHENSOEVER HAD THAT IDEA WAS--THAT WAS A VERY VALUABLE IDEA TO HELP WITH DEVELOPMENT AND NOW COMBINE THESE THINGS IN DEVELOPING COMMON PROCEDURES AND TOOLS AND BEST PRACTICES FOR HAVING DEVELOPED THE STANDARD SO MAYBE A STANDARD ONE WAY OF ORGANIZING COMMUNITIES TO HAVE VOTED EDITORIAL BOARDS OF THE SIZE AND YOU SHOULD MAYBE USE THIS LEVELED APPROACH AND YOU SHOULD USE THESE FORMATS AND THESE STANDARD THIS IS WAY AND I THINK IT'S A BUNCH OF STANDARDS TO SHARE THEIR EXPERIENCE AND THEY COMBINE ALL THEIR WORKSHOPS INTO ONE WORKSHOP AND TRAVEL COSTS, ET CETERA AND SO THAT'S KIND OF ONE NICE THING THAT I WISH WE VALID HAD AT THE BEGINNING. SOME HEALTH TO FIGURE OUT HOW TO DEAL WITH CERTAIN ISSUES. SO I THINK THERE'S STILL WORK TO BE DONE TO SUMMARIZE WITH CODIFYING BEST PRACTICES, DEVELOPING TECHNOLOGY THAT REALLY HELPS PEOPLE WHEN THEY MAKE A STANDARD, NOT HAVE TO WORRY ABOUT THE TECHNOLOGY THAT THEY'RE WORKING WITH BUT THEY FOCUS ON WHAT THEY WANT TO DO WHICH IS HONDURAS DATA MODELING AND OF ALL THE ARGUMENTS YOU WANT IN THE WORLD ABOUT DATA MODEL, THAT'S WHAT YOU WANT THE ARGUMENT TO BE, NOT ABOUT TECHNICAL THINGS AND NOT WORKING ABOUT THAT AND THE COMMUNITY PROCESSES GITHUB OR WHATEVER'S WORKED FOR PEOPLE TO KIND OF SHARE THAT INFORMATION AND THAT SHOULD HOPEFULLY HELP. I GUESS THE OTHER THING IS INCENTIVIZING PEOPLE TO WORK TOWARDS FRAGMENTS BECAUSE IT'S BETTER FOR EACH PERSON TO CURRENTLY--IF WE'RE MOTIVATED BY GETTING A PAPER MOTIVATION OR FRAGMENT BECAUSE YOU WANT TO MAKE SHOULD GO NEW RATTLER THAN WHAT YOU COMBINE AND WORK TOWARDS THE COMMON GOAL. SO, OKAY, THAT'S IT. [ APPLAUSE ] >> THANK YOU VERY MUCH. I WAS INTERESTED IN YOUR COMMENT ABOUT NONEXPERTS INTERFERING WERE YOU TALKING ABOUT DOMAIN, DEVELOPMENT? THEY'RE ALL DIFFERENT THINGS AND DIFFERENT STAKEHOLDER FIST YOU LIKE SO COULD YOU EXPLAIN WHAT YOU MEANT BY NONEXPERTS? >> YES, MELISSA ASKED US TO TRY TO NOT BE AFRAID TO MAKE CONTROVERSIAL STATEMENTS. >> RIGHT. >> SO THE QUESTION WAS WHAT DO I MOON BY NONEXPERTS INTERFERING, SO IN THIS CASE, I WON'T--THE SPECIFIC EXAMPLE WAS THAT THERE WAS A NON--SOMEONE WHO WAS LIKE A MANAGER OF A GROUP WHO WAS NOT NOT ELIGIBLE IN BIOLOGY FOR INSTANCE COMING AND SAYING, YOU KNOW THIS IS HOW YOU SHOULD REPRESENT PROTEIN INTERACTIONS BECAUSE THEY WERE REPRESENTING A KIND OF MANAGEMENT LEVEL, A MANAGER WHO IS NOT A SCIENTIST OR ANYTHING REALLY TO BIOLOGY, BUT THEY'RE REPRESENTING THEIR DATABASE GROUP, AND THAT WAS THEIR JOB. SO YOU KNOW THE DATABASE GROUP CHOSE THAT PERSON TO COME REPRESENT THEM AT A MEETING THAT WAS REALLY ABOUT HOW TO DESCRIBE DATA AND WHAT THAT PERSON DID WAS BASICALLY TAKE THEIR DATABASE AND SAY THIS IS HOW IT NEEDS TO BE DONE BECAUSE IT WASN'T REALLY A DISCUSSION ORIENTED THING, IT WAS ONE GROUP PUSHING THEIR VERSION OUT BECAUSE THEY DIDN'T REALLY UNDERSTAND HOW THAT WAS WORKING. SO THAT WAS, I JUST--YOU KNOW IT WAS AN UNLUCKY THING, BUT IT HAPPENS AND AS A RESULT IT CREATED A LITTLE CRIPPLE IN THAT COMMUNITY. >> I WILL MOVE ALONG AND I THINK ONE COMMENT THAT WE SHOULD COME BACK TO AND I WAS SPEAKING ABOUT GARTO YESTERDAY IS THIS IDEA THAT HERE WE'RE LOOKING FOR YOU KNOW COMMUNITY EXPEFORT COLLECTIVE EFFORTS THAT MIGHT MEAN LESS PAPERS WHICH ARE CURRENCY OF BEING VALUED. AND SO IF THAT'S--IF THAT'S YOU KNOW LESS INNOVATIVE AND LESS PAPERS HOW WILL WE INCENTIVIZE PEOPLE THAT WORK TOGETHER SO THAT'S SOMETHING YOU HAVE TO FIX IN OUR REVIEWS SYSTEMS AND OUR GRANT REVIEWS AND TENURE PROMOTIONS AND THINGS LOAMACYIC THAT TO COME BACK TO THAT. SO NEXT I WOULD LIKE TO INVITE RYAN BRINKMAN TO COME UP HERE AND HE WILL TALK ABOUT FLOW CYTOMETRY AND WHY WE NEED SO MANY. >> BECAUSE I NEED LOTS OF PAPERS. SO SO I'VE BEEN INVOLVED SINCE 2005. SO THE WAY THE STANDARDS WORK IN THIS COMMUNITY IS UNDER THE OFFICE OF THE BRAGSAL SOCIETY FOR THE ADVANCEMENT OF CYTOMETRY, I HAVE ABOUT 56 MEMBERS OF THE TASK FORCE, WHICH IS OPEN TO ANYBODY WHO WANTS TO STEP UP TO THE PLATE. SO YOU GET PEOPLE INTERFERING, ANYBODY CAN JOIN IN AND THAT'S GOOD, SO THAT'S GOOD, ALL THE MANUFACTURES AND ALL THE THIRD PARTY SOFTWARE DEVELOPERS AND MEMBERS OF THE COMMUNITY, ACADEMICS WHO COME TOGETHER ON TELEPHONE CONFERENCE CALLS AND THIS GOES BACK TO WHAT CHRIS IS SAYING, AND ONE OF THE BIG LIMITATION SYSTEM, WE HAVE MONTHLY CALLS WHERE WE HAVE THE MEMBERSHIP OF THE TASK FORCE OF 15 PEOPLE AND REALLY THEY WANT TO SHOW UP AND THIS FLURRY OF ACTIVITY 24 HOURS BEFORE THE PHONE CALL OF WHO'S GOING TO BE THERE IN THE NEXT DAY FOR THAT CALL, WANT TO GET THEIR VOICE IN AND THEN THEY SHOW UP AND NOBODY ELSE SHOWS UP THIS LEADS TO WHAT EVERYBODY SAYS ON THESE SLIDES ARE THE LONG CYCLES WHICH IS DEPENT ON EVERYBODY PUTTING THEIR INPUT AND BECAUSE WE'RE FOLLOWING A FORMAL STANDARDS PROCESS, AND THE WAY THEY SIGN OFF AND CALLED THE WC-THREE MODEL AND PROPER INPUT ON THIS BECAUSE PEOPLE AREN'T SHOWING UP AT THE PROPER TIME AND PROPER INPUT. IT STRETCHES OUT THAT WHOLE CYCLE. AND SO, HOW DO--ONE OF THE THINGS WE NEED TO TALK ABOUT IS HOW DO YOU MEASURE SUCCESS, SO ONE WAY IS THEY ARE IMPLEMENTED SO WE'VE DONE THIS, SEVERAL DIFFERENT TIMES AND I WANT TO SAY THIS IS DEVELOPED FORMAL DATA FORMAT STAND ORDINANCE NUMBERS OR A TECHNICAL DOCUMENT THAT SAYS HOW WE ENCODE INFORMATION BETWEEN SOFTWARE TOOLS SO WE HAVE SEVEN OF THESE DONE FOR BOTH DATA AND METADATA, THE FIRST ONE IS THE FCF STANDARD WHICH DESCRIBES THE DATA THAT COMES OFF THE ACTUAL MACHINE AND WHILE, YOU KNOW IT'S FIRST IN 1984, IT'S BEEN THROUGH THREE DIFFERENT REVISIONS AND EACH OF THESE DIVISIONS SEEM TO TKE LONGER AND LONGER TO DO, AND EVEN HAVING TRIVIAL CHANGE LIKE HOW TO DEFINE A CARRIER HAS TAKEN ABOUT A YEAR SO FAR AGIN BECAUSE IT'S ONE VENDER WHO WANTS TO HAVE THAT INFORMATION AND THEY'RE A PROPONENT AND NOBODY SEEMS TO CARE AND GET EVERYTHING TO SIGN OFF AND SHOW UP HAS BEEN A STRUGGLE. AND FUNDING FOR THIS HAS BEEN A STRUGGLE BECAUSE IT'S ALL UNFUNDED MANDATES SO AND THEY HAVE THE FUNDING TO SHOW UP SO HOW DO YOU INCENTIVIZE PEOPLE TO PARTICIPATE HAS BEEN DIFFICULT. SO WE'VE BEEN LUCKY, WE HAD SOME FUNDING FOR A TECHNICAL WRITER WHO DOES THE WRITING OF THE ACTUAL DOCUMENTS SO WE AVERAGE THAT, ACTUALLY COST FOR A PERSON TO WRITE THESE DOCK UNLETS IS ABOUT A HUNDRED THOUSAND DOLLARS PER STANDARD IN TERMS OF TECHNICAL WRITING TIME, AND SOME ABOUT $800,000 HAS BEEN FUNDED BY THESE DOCUMENTS FOR THE EFFORT. SO ANOTHER EXAMPLE OF THIS WE DEVELOPED IS GETTING ML, NOW INFORMATION WE FINALLY HAVE OUR PAPER SUBMITTED FOR PUBLICATION IN JANUARY OF THIS YEAR, AND THIS IS THE WHOLE PROCESS, THE RECOMMENDATION, YOU GET THE COMMUNITY FEEDBACK AND THEN THAT'S WHERE THE SAY OKAY, THIS IS WHAT WE WANT AND THESE ARE THE CHANGES WE ORDERED AND THEY'RE MOTIVATED AT THE VERY END OF THE CYCLE AND SO THAT ENDED UP BEING A LOT OF DEVELOPMENT AND OKAY, SAY, NOW THEY'RE READY TO WE WILL PUT THIS FORWARD AND THEY WILL SIGN OFF AND THEY GO OH MY GOD, THESE ARE THE CHANGES WE WANT AND THOSE HAPPEN LATE AND THE REENGINEERING HAS TO BE DONE. NOT ONLY ARE THEY LATE TO SAY WE WANT THESE CHANGES BUT THEY'RE VERY EARLY TO START IMPLEMENTING SO WE GET A LOT OF FRACTURIZATION SO WE THIS EARLY CANDIDATE RECOMMENDATION THEY SAY OH THIS IS GREAT, WE WANT TO USE THIS AND THEY RUN OFF AND IMPLEMENT THAT BUT IT'S NOT READY BECAUSE IT'S NOT THE FINAL VERSION OF THIS STANDARD AND THEY GET PRACTICE RESEARCHURES AS A RESULT AND THEN YOU HAVE TO DEAL WITH THAT SO AGAIN LONG TIMELINES. WE ALSO DID MINIMUM INFORMATION STANDARD SO AGAIN HERE'S PEOPLE ACTUALLY PAYING MONEY TO DO THIS AND PEOPLE ARE PAYING MONEY ARE THE PUBLISHERS THAT WANT THIS. AND THE SOFTWARE DEVELOPERS WHO WERE THE TO INCORPORATE THIS, AND SO, SEEING THESE ACTUALLY REQUIRING THIS AND THEY GO INTO THE PROCESS OF MAKE THANKSGIVING PART OF THE WHOLE CYCLE OF THE PUBLICATIONS AND SO, SEEING UP TAKE BUT AGAIN IT'S A VERY LONG TIME BETWEEN WHEN WE START AND SEEING THAT UPTAKE ACROSS JOURNALS. AGAIN, ANOTHER ONE IS TRYING, A SIMPLE THING LIKE TRYING TO BUNDLE STUFF IN A ZIP FORMAT, YOU THINK IT WOULD BE REALLY EASY, THAT'S TAKEN EIGHT YEARS TO DO, PEOPLE CAN'T--WE'VE HAD SO MANY CALLS, WHERE PEOPLE JUST TRY TO AGREE. CAN WE CALL IT DOUG-ZIP AND THEN THE PROBLEM IS THEY CHANGE THEIR MIND AND THEY SAY YES THAT ZIP IS OKAY AND THEN WE COME BACK TWO MONTHS LATER SAY WEEING WENT BACK AND TALKED TO OUR INTERNAL PEOPLE WITHIN THE ORGANIZATION--THE PEOPLE WHO ARE EXPERTS IN THE FIELD WHO COME TO TEASE CALLS AND EVERYBODY WANTS TO HAVE THEIR SAY AND IT'S VERY DIFFICULT TO KEEP THINGS ON TRACK. WE HAVE A LOT OF EXPERTS WANTING TO HAVE THEIR SAY IS THE BEST WAY TO PUT THAT AND THEN VOLUNTEERS PROMISING TO HELP AND NOT STEPPING UP SO WE HAVE THIS MEET IT WOULD GO YEARS AGO AND YOU SAY OH YOU'RE HAVING THIS PROBLEM WITH THE STANDARD, WE WILL TAKE CARE OF THAT FOR YOU. IT WAS REALLY THAT AWKWARD AND THEN YOU GO BACK TO THEM AND SAY WE WILL DO IT, WE WILL HAVE DO IT, WE WILL DO IT AND BECAUSE THEY HAVE OTHER PRIORITIES THEY HAVE OTHER KINDS OF STUFF, IT DOESN'T GET DONE. AND THEN IT'S OKAY, WE WILL TAKE IT BACK BECAUSE WE WANT TO HAVE THIS DONE AND THEN THE PEOPLE HAVE THAT CHANCE TO DO THINGS. SO IT'S ALL THERE, I THINK I TALKED ABOUT THAT BEFORE AND KEEPING THINGS MOVING ALONG AND DOING THINGS COMPLICATED, SETTING THEMSELVES UP TO FATURE AND SO THIS IS HUGE, IT'S GOING TO SOLVE THE PROBLEMS BUT IT'S REALLY BEING COMPLICATED AND WE SPEND A LOT OF TIME FIGURING OUT THAT THING VS TO BE SIMPLE AND THAT TAKES US YEARS TO DO THAT. IT DOESN'T COME CHEAP SO AGAIN TRY TO GET FUNDING HAS TO BE DIFFICULT. WE HAD ONE FUNDED BY THE NIH HAS BEEN GREAT BUT TRYING TO GET THIS IN OTHER GRANTS HAS BEEN DIFFICULT AND YOU HAVE TO SNEAK IT IN AT THE BOTTOM AND IT'S FUND TAG COME OFF GRANTS BUT IT'S THE STANDARD STUFF SO THAT'S BEEN'S PROBLEM BECAUSE THEN THE STUFF YOU DID SAY YOU WERE GOING TO DO ISN'T REALLY BEING DONE AND TRYING TO GET THE FUNDING MECHANISM TO DO THAT HAS BEEN DIFFICULT. THANK YOU. [ APPLAUSE ] >> SO THANKS VERY MUCH, I BELIEVE WE NOW NEED ALL THOSE STANDARDS. SO WE'RE GOING TO TAKE QUESTIONS FOR THE NEXT 15 OR SO MINUTES AND HAVE AN OPEN DISCUSSION AND WHILE OUR PANELISTS ARE STILL UP HERE WE'LL QUESTION AND ANSWER BUT IT'S MEANT TO BE AN OPEN DISCUSSION, SO PLEASE JUMP IN AND ASTRID CAN HELP MANAGE THE QUESTIONS. >> OKAY. GO, MICHELLE? >> RIGHT. SO MY QUESTION PERTAINS WITH THE FACT THAT I LIKE EVERY--EVERY MONTH WE HAVE A NEW TECHNOLOGY, THEY NEW DATA FORMAT, A NEW THING TO STANDARDIZE AND I'M KIND OF CURIOUS, I CAN KIND OF IMAGINE GO THE WAY OF THE DATABASE, WE HAVE THOUSANDS OF DATABASES WHERE IT'S JUST LIKE GROWING AND GROWING AND GROWING AND WE'RE GOING TO HAVE ALSOA THOUSANDS OF DATA STANDARDS POTENTIALLY SO I LIKE THE IDEA OF COMBINE WHERE THEY TRY TO COORDINATE, IN SOME WAY, COORDINATION IS CLEAR BUT AT WHAT POINT DO WE DECIDE THINGS NEED TO DIE AND REFOCUS OUR EFFORTS. I KIND OF WANT TO SEE A CONVERSATION FROM YOUR EXPERIENCE ABOUT WHEN DO YOU SORT OF SAY, OKAY, THAT'S--WE'RE DONE WITH THIS AND LET'S MOVE ON TO THE NEXT CHALLENGE OR THERE'S SOMETHING MORE SALIENT WE THEY'D TO FOCUS ON. STANDARDS DON'T DIE, AND THAT'S A PROBLEM, AND THE SCF STANDARD HAS BEEN AROUND SO LONG AND THEY WANT TO SUPPORT THAT AND THERE'S THIS WHOLE INFRASTRUCTURE BASED AROUND THAT BUT IT SUCKS. IT'S AN AWFUL STANDARD, CAN YOU THINK OF WHAT WE'RE DOING WITH CRS BACK IN 1984, ALL THE TECHNOLOGY THAT HAVE MOVED ON SINCE THEN BUT IT'S A REAL HESITATION FOR PEOPLE TO ACTUALLY DROP THAT. AND SO I DON'T KNOW BECAUSE THERE'S MONEY DOING THAT AND MANUFACTURES REALLY DON'T WANT TO MOVE BEYOND WHAT IS DONE IN 1984. I MEAN, I LOT OF CENTERS PROBABLY NEED TO DIE. BUT I THINK THERE'S PERHAPS LESS OF A COST TO ME, ANYWAY IN MY EXPERIENCE WITH SUPPORTING LEGACY STANDARDS SO LONG AS THERE'S A WELL DEFINED TRANSFORM TO WHOEVER IS THE DESIGNATED CURRENT STANDARDS AND THE LEGACY FORMATS, YOU KNOW YOU CAN DEVELOP TOOLS TO CONVERT BACK AND FORTH BETWEEN THEM. IT DOES EXERT A COST BUT I THINK THAT'S A RELATIVELY LOW COST IN MY EXPERIENCE. BUT WHERE THE COST IS HIGH SER WHERE YOU HAVE MULTIPLE COMPETING STANDARDS WITH LOTS OF OVERLAP BETWEEN THEM, THERE'S JUST A LOT OF WASTED EFFORT IN CYCLES THERE. IN DOING THAT AND I THINK WE CAN DO MORE TO JUST REDUCE THE NUMBER, NUMBER OF EFFORTS AND MAYBE COME UP WITH NEW WAYS OF ACTUALLY BEING ABLE TO REDUCE THE STANDARDS AND WE'VE SEEN HOW IT WORKS WITH ONTOLOGIES AND IT'S BUILT INTO THE MAKE UP OF THE UNDERLYING MODEL OF ONTOLOGIES THAT THEY ARE REUSABLE THIS WAY. IT'S QUITE HARD TO REUSE AN XML STANDARD IF YOU'RE DEVELOPING ADJACENT STANDARD FOR EXAMPLE. SO IRB UNCLE WE NEED TO BE THINKING ON AN ABSTRACT ASK TECHNOLOGICAL LEVEL ABOUT HOW WE CAN BETTER REUSE COMOPPOSITE BEHAVIORIAL PHENOTYPENTS OF STANDARDS WITHIN OTHER STANDARDS AND MAYBE YOU WILL LIKE TO COMMENT ON WAYS OF DOING THAT BECAUSE WITH THE EXPERIENCE [INDISCERNIBLE]. >> YOUR COMMENT ABOUT [INDISCERNIBLE]? >> SO I THINK THAT IS A GREAT IMPORTANT POINT AND IT RELATES TO THE COMMENT IT WAS ALWAYS THE VAL UNSUPPORTED TEARS THAT SHOW UP, AND SO ESPECIALLY FOR PEOPLE IN THE ROOM, MAYBE ONE OF THE SOLUTION SYSTEM THAT WE SHOULD BE PAYING PEOPLE TO DEVELOP STANDARDS SO IT'S NOT JUST THE VOLUNTEERS BUT PEOPLE ARE OBLIGATED TO FOLLOW THROUGH WITH SOMETHING AND THEN A STANDARD WOULD DIE WHEN THE FUND SUGGEST STOPPED FOR THAT AND I SPEAK FROM EXPERIENCE HAVING WORKED ON A STANDARD BIOMEDICAL RESOURCE ONTOLOGY, I WILL TALK ABOUT LATER THAT ESSENTIALLY IS NOT DONE YET, IT SORT OF DIED WHEN THE FUNDING STOPPED, THAT IS THE WORK ON IT STOPPED BEING DONE, IT'S STILL OUT THERE, BY A PORTAL IN THEORY PEOPLE COULD BE VOLUNTEERING AND MAINTAINING AND UPDATING IT, BUT MAYBE WE SHOULD TAKE A MORE ACTIVE STAKE IN THE CHEST KIND OF THING. THERE WILL BE MORE POLITICAL ISSUES AND EAGY IN TURF AND PEOPLE SAY IT'S MY STANDARD IF PEOPLE WANT TO USE IT THEY SHOULD BE ABLE TO BUT FUNDING COULD RELATE AND I THINK THERE SHOULD BE MORE FORMAL DEATH RIGHTS FOR STANDARDS, SOMETHING LIKE THAT. >> OVER HERE. AT THIS POINT IT'S NOT ENOUGH WAYS WHETHER IT SHOULD DIAL HOW DO YOU KNOW IF THERE ARE OTHER DISCIPLINES IT WOULD BE GOOD TO KNOW THAT AND WHAT WE CAN LEARN FROM THAT? >> I THINK EVERYONE IN THE ROOM CAN TALK ABOUT THAT AND ONE OF THE STANDARDS WE USE IS THE FROM THE DATA THAT'S ANNOTATE FRIDAY THEM AND IT'S HARD TO TAKE THE STANDARD THAT JESSICA WAS TALKING ABOUT AND FIGURE OUT WHAT DATA ACTUALLY STILL EXISTS THAT'S USING IT AND I KNOW BECAUSE I ACTUALLY TRIED BUT IT'S TRUE FOR ALL OF THOSE THINGS, EVEN MERGING THOSE ONTOLOGIES I WAS TALKING ABOUT EARLIER, WHO'S USED THEM, HOW MUCH DATA IS THERE, WHAT ARE THE DOWN STREAM EFFECTS OF SAYING, THIS STANDARD IS OBSOLETE IN TAKING IT DOWN OFF OF THESE BIOPORTAL OR BIOSHARING OR OTHER SITES. WE DONE BE WHAT THE XEESKTS WILL BE OF SAYING WE PUT THE STAKE IN THE HEART OF A STANDARD. SO THAT'S PART ONE BEFORE WE EVEN UNDERSTAND LIKE HOW TO EVALUATE THINGS IS WE NEED TO UNDERSTAND WHO'S USING THEM. YEAH. >> I WAS GOING TO COMMENT ON CHRIS' GITHUB EXAMPLE, WHEN YOU LOOK AT IT, CAN YOU SEE WHO'S FORKED IT SO HOW MANY TIMES ARE PEOPLE ACTIVELY TRYING TO CHANGE SOMETHING AS WELL AS OTHER USAGE STATS, HOW OFTEN ARE PEOPLE COMMITTING AND THOSE PIECES AND WHATEVER WE THINK ABOUT FROM AN INFRASTRUCTURE PERSPECTIVE NEEDS TO ENGAGE AND MEASURE THAT ENGAGEMENT. >> Y I WANT TO ABOUT THE FUNDING STORY DATA COLLECTION WHICH ONE WONDERS IF IT'S TO IMPROVE THE HEALTHCARE OF THE PATIENTS. SO MY QUESTION IS SHOULD THE PATIENT BE OUT OF THE EQUATION OR ANY SAYING IN THIS PROCESS OF DEVELOPING STANDARD OR? >> I WOULD MENTION PATIENT IS AN EXPERT IN PATIENT INFORMATION, RIGHT? THEY HAVE EXPERTISE, SO ANOTHER BETTER EXAMPLE I COULD HAVE USED IS SOMEBODY, YOU 99 OUR COMMUNITY STARTED E-MAILING ABOUT PHILOSOPHICAL QUESTIONS THAT HAD UNRELATED TO THE DEVELOPMENT OF THE STANDARD AND ACTUALLY TOOK OUT DAYS AND DAYS OF THE COMMUNITIES TIME REPLYING TO THIS PERSON UNTIL THEY REALIZE THAT THE PERSON REALLY DIDN'T--WAS JUST REALLY KIND OF NOT REALLY SO IT'S JUST THE RISK AND LETTING IT OPEN, I GUESS THE POINT I SHOULD HAVE MADE MORE CLEARLY IS THAT YOU HAVE AN OPEN COMMUNITY IN SOME WAY, YOU'RE OPENING UP TO EVERYBODY'S INVOLVEMENT, ABDOMEN ORDER OF MICRONS PEOPLE COULD DO THING SPECIALIZAION OF SPECIFIC ENDOTHELIAL YOU NEED A WAY OF MANAGING THE COMMUNITY SO ALL I WAS REALLY, I GUESS WANTED TO SAY THAT WAS YOU NEED A MANAGEMENT FOR THE COMMUNITY, AND ALL I WAS TRYING TO SAY WAS YOU NEED AN EDUCATIONALITTOR AND ELECTION SYSTEM SO THERE'S SOME MODERATION THAT TAKES PLACE. >> SO IT'S THE CASE ONE OF THE GOALS OF THE SCIENTIFIC ENDEAVOR AS A WHOLE IS TO SHARE OUR FINDINGS AND GET FEEDBACK FROM THE COMMUNITY AT LARGE AND THE PATIENTS WITH RESPECT TO NIH ARE OBVIOUSLY ONE OF THOSE TARGET AUDIENCES AND THERE ARE A NUMBER OF DIFFERENT EFFORTS PCORI AND MANY OTHERS TO TRY TO SUPPORT THE INTERACTION OF PATIENTS WITH DATA STANDARDS, RESULTS AND ALL KINDS OF THING AND I GUESS CERTAINLY DREAM OF A WORLD WHERE WE CAN BYPASS THE LENGTH SCHECOMPLEX LITERATURE PROCESS. BUT I HEAR WHAT GARY SAYS IS THAT THEY'RE NOT GOING TO NECESSARILY JUMP IN AND DEVELOP THE STANDARDS THEMSELVES AND THEY ARE INVOLVED IN US UNDERSTANDING THE REQUIREMENTS AND FEEDING BACK THOSE RESULTS TO THEM BECAUSE THEY'RE REALLY THE CLIENTS AT THE END OF THE DAY. >> THANKS WHEN WE TALK ABOUT THIS, THERE'S SOCIAL ISSUE, THERE'S WHERE THIS LEADS SO DATA AND DISSEMINATION, AND DATA, SECONDARY USE. SO FOR THE METADATA FOR THE PRODUCTIONS AS WELL AS THE DATA SECRETARY REUSE, MY POINT IS NOT SAYING IT'S THE PIs JOB TO PROMOTE THE SECONDARY USE BUT YOU KNOW WE CAN SEE IN THE FUTURES THERE'S A LOT OF REQUIRE, REQUESTS FROM THE COMMUNITIE. LOOK, I WANT TO USE YOUR DATAS, BUT THE WEBPAGE DOES NOT WORK ANYMORE, YOU KNOW? OR, YOU KNOW THEY'RE MISSING SOME THINGS THERE OR HERE, SO SOMETHING MAY NEED TO BE ADDRESSED BIKE THE WHOLE SOCIAL INFRASTRUCTURE OR SOMETHING, YOU KNOW IN THIS, YOU KNOW THIS THESE TWO DIFFERENT DATA, TWO DIFFERENT KIND OF DATA PRACTICE FOR THE SAME SET OF DATA. AND THIS IS KIND OF A VERY DISTINGUISHED PIECE, THANK YOU. >> ANY OF YOU HAVE COMMENTS YOU WANT TO MAKE? >> I JUST ASSUMED THAT THE DATA SHARING AND REUSE WAS SOMEWHERE ELSE USING THIS, IS THAT NOT RIGHT? ARE WE FOCUSED ON DATA STANDARDS OR? BECAUSE THAT'S A WHOLE OTHER BIG ISSUE, I PROMOTE DATA SHARING AND YOU KNOW, IS IT BEST TO HAVE COMMUNITY RESOURCES THAT SHARE THE DATA OR EVERY INVESTIGATOR WILL SHARE THEIR OWN DATA? I'M I BIG FAN OF HAVING PUT IT ALL IN ONE PLACE BECAUSE I THINK IT GIVES A BETTER CHANCE FOR IT TO STICK AROUND. >> IT'S RELATED ESPECIALLY THE POINT THAT SOMEONE MAKES DATA AVAILABLE AND DISAPPEAR SPECIALIZATION OF SPECIFIC ENDOTHELIAL THERE SHOULD BE SOME WAY OF SOLVING WHERE THERE'S ARCHIVES BUT THAT--THAT IS A MUCH BIGGER DISCUSSION, I'M SURE IT'S ANOTHER WORKSHOP FOCUS ON A LOT OF THOSE ISSUES. >> THERE MIGHT BE STANDARDS INVOLVED ABOUT PUTTING THINGS IN CLOUDS AND MAYBE THIS' A GOOD PRACTICE FOR PEOPLE. >> SO I WANT TO ADD TO THESE EXCELLENT POINTS ABOUT THESE STANDARDS LIFE CYCLE AND JUST A SMALL COMMENT, THERE SHOULD BE AN OWNER, A CLEAR OWNER, IT SEEMS THAT FROM ALL YOUR CONVERSATIONS THESE WERE VERY OPEN PLATFORMS AND BECAUSE EVERYBODY WAS VOLUNTEERING THEIR TIME AND NOT FEELING LIKE THEY OWNED IT, NOBODY FELT THEY COULD YOU KNOW SAY NO TO ANYBODY AND SO, CONVERSATIONS LASTED ENDLESSLY, BUT I THINK IF THERE IS A CLEAR OWNER AND I REALLY LIKE JESSICA'S PLAN OF SAYING, WELL, SOMEBODY GETS PAID TO DO THIS. AT THAT POINT THEY SHOULD BE ENTITLED TO SIMPLY MAKE DECISIONS AND I THINK THAT WILL REALLY SHORTEN THE PROCESS, SO TOGETHER WITH A LIFE CYCLE WHICH MEANS THINGS CAN DIE THAT THERE'S ALSO AN OWNER AND THE MOMENT THE OWNER STOPS OWNING IT, THAT'S WHEN MAYBE THE STANDARD GETS RECYCLES OR DIES OR SOMETHING LIKE THAT. >> IN TERMS OF LIFE CYCLE I THINK ONE OF THE STANDARD LIFE CYCLE IS MUCH LONGER THAN A GRANT CYCLE. >> YES I ERNE JOY THE SESSION, BASICALLY THE POINT MOST OF YOU MADE ABOUT THE FRAGMENTATION OF STANDARDS AND COMPETING ELEMENTS SO HOW DO YOU COME ACROSS IN SITUATION WHERE YOU IDENTIFY VERY LATE IN THE PROCESS, A NEW GROUP THAT IS WORKING ON THE PARTICULAR THOUGHT BECAUSE WE HAVE TO THINK ABOUT BETTER IN TERMS OF GLOBAL INITIATIVES, OR SOMETHING THAT IS REALLY IS A COMMUNITY EFFORT, SAY HOW DO WE PLACE THAT OR IDENTIFY MECH NS, OR IDENTIFY SUCH INITIATIVES. >> SO HAVING A PLACE PEOPLE CAN GO, WE WOULD GO THERE AND FIND OUR STANDARD AND THEN WE FIND OUT SOMEBODY'S DONE THE SAME THING AND THEN TO TRY TO MERGE AND THE MIBI IS ONE GOOD EXAMPLE OF THAT WHERE THEY'VE DEN AND SOLVED THAT PROBLEM, FOR MINIMUM INFORMATION STANDARD BUT USE THRAG SAME APPROACH IN A GLOBAL FASHION FOR STANDARDS IN GENERAL IN THE LIFE SCIENCES WOULD BE A FANTASTIC IDEA. >> YOU I THINK THAT'S ONLY HALF THE PROBLEM BECAUSE THEN ONCE YOU DO IDENTIFY IT HOW DO YOU WORK TOGETHER? WE'VE HEARD THAT LIKE WITH PUBLICATIONS, THE MOTIVATION IS NOT TO WORK TOGETHER, ACTUALLY AND SO I DON'T THINK THERE ARE SUFFICIENT INCENTIVES IN PLACE TO REALLY FYOU DO WORK TOGETHER, THEN YOU GET THIS. I MEAN, IT'S STILL THIS VOLUNTARY GREATER GOOD KIND OF THING BUT WE NEED BETTER INCENTIVES THAN--THAT ARE MORE CONCRETE, I THINK IF WE REALLY EXPECT PEOPLE TO GIVE UP THEIR OWNERSHIP AND JOIN WITH OTHERS. SO THAT'S THE DOWN SIDE OF OWNERSHIP IS THAT IT CAN ALSO LEAD TO FRAGMENTATION. >> ANY LAST COMMENTS ON THAT THREAD OR WE WILL BREAK AND CONTINUE THE CONVERSATION? >> I THINK OWNERSHIP IS A GOOD IDEA BUT THAT'S A GOOD POINT BUT MAYBE THERE NEEDS TO BE SOME CODE OF CONDUCT LIKE THERE IS FOR ETHICAL PUBLISHING, LIKE WHAT'S AN AUTHOR, YOU KNOW THE PEOPLE, CULTURES DEFINED BY PEOPLE, A NUMBER OF PEOPLE WHO GOT TOGETHER AND SAYING THIS IS THE CODE OF CONDUCT THAT WILL BE BEST FOR THE COMMUNITY AND PEOPLE SHOULD ADHERE TO IT IF THEY'RE FOLLOWING BEST PRACTICES AND ETHICAL. >> SO ON THE CLINICAL SIDE, THERE'S A COUPLE OF STANDARD GROUPS AND ONE OF THEM, THERE'S NO AUTHORSHIP AT ALL. AND THE OTHER TWO, THE PEOPLE WHO WRITE IT AND THEY'RE USUALLY ASSIGNED ARE AUTHORS, THEY GET THEIR NAME ON IT SO THAT IT'S HOSEVEN AND IEEE, WHICH DOES GIVE PEOPLE PROPERTY AS SOMETHING THEY CAN THROW AROUND IN A CORE PUBLICATION AND IT ALSO FOCUSES THE WORK SO IF SOMEONE IS DOING THE MEETINGSING AND YELL, THE SAME CREDIT AS PEOPLE WHO SIT DOWN AND PUT IT ALTOGETHER, SO THAT MIGHT BE A MODEL. I WILL MENTION ABOUT THAT WHEN I TALK-- >> OKAY, THANKS SO MUCH. WE'LL HAVE A ROUND OF APPLAUSE FOR OUR PANELISTS HERE. >> THANK YOU. >> SO 10 MINUTES, BE BACK HERE AT 11:00 O'CLOCK. GREAT SO I'M ALEXIA Mc CRAY, AND OUR SECTION IS CHALLENGES OF THE USER COMMUNITY AND WE DECIDED AMONG US WE WOULD HAVE ONE SLIDE EACH SO EVERYONE PREPARE A SLIDE AND THEN I PUT THEM TOGETHER IN A SLIDE PRESENTATION AND WE WOULD REALLY LIKE THIS TO BE HIGHLY INTERACTIVE AND SO WE YOU COULD OPT AND ONE OF THE SPEAKERS SPEAKING AND YOU COULD WAIT UNTIL THE END OF THE PERSON'S TALK OR KUWAIT UNTIL EVERYONE'S GONE THROUGH, COMPLETELY UP TO YOU AS YOU LIKE. WE SAID WE WE WANT TO MAKE THIS HIGHLY INTERACTIVE AND OUR PANELISTS TODAY ARE OLIVIA GORDON WRITER WHO IS AT THE NATIONAL LIBRARY OF MEDICINE, AND FROM THE RAPID LEARNING PROJECT AND THEN WE HAVE JESSICA TENENBALM FROM THE DUKE TRANSLAGSAL MEDICINE INSTITUTE AND ANITA DE WAARD, FROM ELSEVIERE AND I WILL BE MODERATING SO WHAT HAS STRUCK ME SO FAR THIS MORNING AND WE'RE ONLY A FEW HOURS INTO THE MEETING IS THAT WE'VE ALL BYE-BYE USING THE TERM COMMUNITY AND I'M NOT SURE THAT WE DEFINED EXACTLY WHAT A COMMUNITY IS. SO I THINK THAT'S SOMETHING IMPORTANT BECAUSE OFTEN WHEN YOU TAKE ABOUT COMMUNITY BASED STANDARDS AND COMMUNITY BASED STANDARDS EFFORTS AND YOU HAVE THIS NOTION OF WELL DEFINED SCIENTIST OR COMMUNITY OF PEOPLE WHO HAVE SIMILAR GOALS AND AND IT COULD BE A VERY SMALL COMMUNITY, VERY SMALL SCIENTIFIC COMMUNITY OR A LARGE ONE OR COULD BE THE CONFEDERATION OR CONSORTIUM, BUT FROM MY PERSPECTIVE, I THINK WE HAVE ANOTHER WAY TO LOOK AT COMMUNITY IS THAT THERE ARE MANY TYPES OF COMMUNITIES OUT WELL BECAUSE WE'RE ALL TAXON MIST AND ONTOLOGYSTS THERE ARE THREE CATEGORIES OF COMMUNITY. AND I CAN'T STOP MYSELF. SO IT SEEMS TO ME THERE ARE DEVELOPERS OF ONTOLOGIES AND OF STANDARDS AND MANY PEOPLE TALKING ONTOLOGIES AND OTHERS TALKING ABOUT OTHER TYPES OF STANDARDS, AND SO THE DEVELOPERS OF ONTOLOGIES AND STANDARDS IN GENERAL COULD BE STANDARDS ORGATION SO WE HAVE AN ISO FOR EXAMPLE, OR HLSEVEN OR WE HAVE IHTSDO WHICH IS AN INTERNATIONAL GROUP THAT'S COME TOGETHER PRIMARILY TO EXTEND THE NOMENCLATURE OF MEDICINE AND DISSEMINATE IT AND THEN THERE IS THE SCIENTIFIC COMMUNITY ITSELF AND I THINK A VERY GOOD EXAMPLE THAT CHRIS BROUGHT UP THIS MORNING WOULD BE THE GENE ONTOLOGY WHERE THE MODEL ORGANISM COMMUNITY GOT TOGETHER AND SAID LOOK, WE HAVE A NEED, THEY WENT FOR FUNDING AND THEY GOT THE FUNDING AND THEN THE COMMUNITY BUILT THIS ONTOLOGY AND CONTINUES TO DO SO BUT IMPORTANTLY RELATING TO OUR CONVERSATION A FEW MINUTES AGO, THERE WAS IN FACT SOMEBODY IN CHARGE, SO THEREY A GROUP IN CAME BRIDGE THAT IS AN EDITORIAL BOARD AND THEY MAKE THE FINAL DECISIONS AND THEY MAKE SURE THAT EVERYTHING IS STANDARDIZED WITHIN THE STANDARD AS WELL. THEN THERE ARE SOCIETIES, WE HEARD ABOUT SOCIETIES AND THEN THERE ARE EFFORTS ON BEHALF OF THE SCIENCE OF THE COMMUNITY AND DARE I WOULD CALL AUGHT THE HUMAN PHENOTYPE ACTIVITY WHICH IS AN ONTOLOGY THAT'S BUILT FOR THE RARE DISEASES AND UNDIAGNOSED DISEASES COMMUNITY AND IT'S REALLY DONE ON BEHALF OF THAT COMMUNITY. OBVIOUSLY THE PEOPLE WORKING IN IT ARE ALSO ENGAGE INDEED DOING RESEARCH IN THAT AREA, BUT THE UPTAKE IS--WILL BE MUCH BROADER AND ALREADY IS BROADER. SO THAT'S REALLY THE FIRST CATEGORY OF YOU KNOW DEVELOPERS, PEOPLE WHO DEVELOP STANDARDS, THEN, AND N. I. S. T., THE FEDERAL AGENCY, WE HAVE A REPRESENTATIVE FROM N. I. S. T. HERE, OBVIOUSLY THERE ARE SDS THAT FEDERAL AGENCIES CREATE OR THE NATIONAL LIBRARY OF MEDICINE, THE MEDICAL SUBJECT HEADINGS AND OTHER ONTOLOGIES, THE ORGANISM TAXONOMY THAT NCBI CREATES. SO THAT'S ALL IN THE DEVELOPER AREA. THEN THERE ARE THE SESEMESTERINATORS OF STANDARDS THEMSELVES AND THERE I WOULD AGAIN POINT TO NLM, OLIVIA WILL SAY SOMETHING ABOUT THE UNIFIED MEDICAL LANGUAGE SYSTEM WHICH I WAS HEAVILY INVOLVED IN THROUGH A GOOD PART OF MY CAREER AS WELL AND THEN THERE ARE VEBD VENDOR WHO IS ARE THE DISSEMINATORS IN THE SENSE THAT THERE'S A STANDARD AND NOW THEY PUSH THAT OUT THROUGH THE--THROUGH THE SYSTEMS THAT THEY HAVE. AND THEN FINALLY THE USERS, SO THE SCIENTIFIC COMMUNITY ITSELF JUST AS A USER, VENDERS AS USERS, PATIENTS AS USERS AND ANYBODY WHO TOUCHES DATA. SO I THINK THAT YOU KNOW WHEN WE TALK ABOUT COMMUNITY, IT WOULD BE GOOD TO REALLY THINK ABOUT WHAT DOES THAT COMMUNITY MEAN BECAUSE THAT HAS AN IMPACT ON EVERYTHING, INCLUDING FUNDING AND BUY IN AND ALL OF THOSE THINGS. SO,--SO THAT LEADS TO THE NEXT QUESTION IS HOW DO COMMUNITIES DRIVE STANDARDS EFFORTS SO IT DEPENDS ON WHERE YOU SIT IN THAT AREA, ARE YOU A FEDERAL AGENCY THAT CAN PROMULGATE OLOGIATE THESE STANDARDS ARE YOU A COMMUNITY THAT WHERE IT BECOMES A DEFACTOR STANDARD BECAUSE YOU ARE--HAVE YOU SOMETHING IT MAY NOT BE PERFECT BUT EVERYBODY'S USING IT. AND SO THEN THAT DRIVES THE STANDARDS EFFORTS FORWARD AND THEN THERE ARE RECOLLECT STAKEHOLDERS AND WHAT ARE THERE ROLES AND SO THERE YOU MIGHT THINK ABOUT THE FEDERAL GOVERNMENT ITSELF, IN LOOKING TO SEE, YOU KNOW THEIR INVESTMENT IN RESEARCH PAID OFF AND THE ONLY WAY TO DO THAT IS TO SEE IF DATA ARE COMPARABLE AND THEY CAN SEE WHETHER--WHAT THEY'VE DONE HAS ACTUALLY MADE A DIFFERENCE AND THERE COULD OBVIOUSLY BE OTHER STAKEHOLDERS AND I HOPE WE HAVE DISCUSSION ABOUT THAT AND THERE'S ALREADY BEEN ALLUSION QUITE A FEW TIMES TO WHAT ARE THE INCENTIVES FOR LEADING AND SUSTAINING STANDARD EFFORTS AND AGAIN IT COMES DOWN TO WELL, YOU KNOW WHAT'S THE COMMUNITY. SO WHAT WE'RE GOING TO DO IS WE HAVE ONE SLIDE PER PERSON AND I'LL JUST ADVANCE THE SLIDES AND WE'LL START WITH OLIVIER, SO GO AHEAD. YOU WANT TO BE UP HERE, OH, GO AHEAD. >> IT SHOULD BE A STANDARD MEEK ROUGH ATOM PHONE THERE. IS IT NOT WORKING? >> THANK YOU SO AS I MENTION, I FIT IN ONE OF HER COMMUNITIES WHICH IS GOOD AND MY COMMUNITY WOULD BE THAT OF NOT SO MUCH ONTOLOGY DEVELOPERS BECAUSE THE INITIAL LIBRARY OF MEDICINE WELL, HAS SUPPORTED THE DEVELOPMENT OF RX-NORM AND SO NLM HAS PUT MORE EMPHASIS ON THE REUSING OF EXISTING STANDARDS AND ON FACILITATING THE THE ABILITY BETWEEN STANDARDS AND THAT'S ONE OF THE MAIN OBJECTIVES OF THE UNIFY MEDICAL LANGUAGE SYSTEM THAT ALEXIA BRIEFLY ALLUDED TO THE UMLS WHICH IS NOT THE ROLE OF UML, BY THE WAY, AS WAS MENTIONED THIS MORNING SO MY TAKE ON COMMUNITY SYSTEM ACTUALLY THAT THERE'S OTHER KINDS OF SUB DIVISIONS SO ALEXIA TALKED ABOUT THE SUBDIVISION BETWEEN THE USERS AND THE DEVELOPERS AND OTHER PARTS BUT THERE'S ALSO A HUGE SUBDIVISION IN THE COMMUNITY OF USERS FOR EXAMPLE, BETWEEN THE CLINICAL AND THE BASIC SCIENCE USERS OF ONTOLOGIES. AND THERE ARE ACTUALLY DIFFERENT SETS OF ONTOLOGIES USE INDEED THE DIFFERENT COMMUNITIES AND THAT'S WHAT PROMPTED MY TITLE FOR THE SLIDE ABOUT THE STANDARD DIVIDE IN THE COMMUNITY SO THIS IS A PROBLEM BECAUSE IF VARIOUS GROUPS ANNOTATE THE DAT WITH DIFFERENT--WO DIFFERENT ONTOLOGIES, NOW WE HAVE A PROBLEM WHEN WE WANT YOU TO DO, YOU KNOW INTEGRATION OF THIS NOW HETEROGENEOUS DATA SETS AND I'M NOT EVEN TALKING ABOUT YOU KNOW SHOULD WE USE THIS DATA ELEMENT VERSES THAT DATA ELEMENT BUT IF WE DON'T EVEN USE THE SAME KIND OF THE IDENTIFIERS FOR DISEASES FOR ORGANISMS AND THIS KIND OF THINGS TMAKE ITS VERY DIFFICULT. AND TO SOME EXTENT THAT'S ALSO WHAT MELISSA ALOUDED TO THIS MORNING WITH THE ONTOLOGY WITH THE ANATOMICAL ONTOLOGY. SO OF COURSE WHEN IT IS A FACT THAT THESE VARIOUS ONTOLOGIES EXIST, THE NEXT BEST THING THAT WE CAN DO IT TO TRY TO MAKE THEM INTEROPERATING GLOBALLY RAGERABLE TO, BRIDGE BETWEEN THEM SO NOW WE HAVE SUCH THINGS AS THE MAPPING CREATED AS PART OF THE UMLS AS PART OF A PORTAL THAT WAS MENTIONED THIS MORNING BUT OF COURSE, THIS IS NEVER AS GOOD AS THE PLAN COLLABORATIVE DEVELOPMENT OF STANDARDS AND AND FUNDING PROVIDED SOME KIND OF LAY OUT FOR THE DEVELOPMENT OF AND COORDINATION, THE TECHNICAL COORDINATION OF DEVELOPMENT ONTOLOGY IN THE BY O, IN THE BASIC SCIENCE, THE MAIN, WHAT I WANT TO POINT OUT IS THAT IS THAT SIMILAR EFFORTS HAVE EMERGED IN THE CLINICAL DO MAIN, IT'S MORPH AFTER THE FACT MORE THAN DEVELOPMENT IF YOU WISH, IT'S HARMONIZATION, BUT THESE EFFORTS ARE AND ARE IMPORTANT AND I WANT TO MENTION EFFORTS BETWEEN HARMONIZATION BETWEEN SN OMED-CT AND LOINC FOR EXAMPLE, AND I ALLUDE TO WHAT'S GOING ON BETWEEN SNOMED CT AND ICD11. SO THESE ARE ENCOURAGING BECAUSE IT SHOWS THAT FOR EVEN WHAT'S OFTEN REFERRED TO AS LEGACY ONTOLOGIES OR LEGACY STANDARDS THERE IS INDEED EFFORT TO HARMONIZE THEM AND DO MORE TOGETHER AND DO MORE PLANNING AND THE REUSE WHENEVER FLEXIBLE. I THINK I AM GOING TO STOP THERE. >> I'M HERE AS THE BIG DATA OF BRAVE NEW SET OF ORGANIZATIONS THAT'S TRYING TO USE ELECTRONIC HEALTH RECORDS AND INCREASINGLY BRING IN LEADING EDGE SCIENCE LIKE GENOMICS AND PROTEOMICS AND OTHER OMICS I HEARD ABOUT THIS MORNING THAT I DON'T EVEN BEGIN TO UNDERSTAND BUT TO TRY TO BRING THAT INTO NEW MAIN STREAM ORGANIZATIONS AND IN PARTICULAR DELIVERY SYSTEMS AND BEING A PARTICIPANT IN THIS DATA DRIVEN RAPID LEARNING THAT ADVANCES SCIENCE SO THOSE WILL BE ORGANIZATIONS LIKE THE VA, KISER PERMANENT A, AND ELECTRONIC HEALTH RECORDS WITH RESEARCH DEPARTMENTS LOTS OF STANDARDIZATION ORGANIZATIONS AND ENTHUSIASM RITES THAT COORDINATE ORGANIZATIONS LIKE THE COMMON DATA MODEL THAT NIH FDA PCORI HAVE BEEN PUTTING TOGETHER SO ALL THESE OF COURSE BECAUSE THEY'RE I.T. BASED STANDARDS ENFORCE US TO HAVE STANDARDS FOR DATA THAT WE DEPARTMENT NEED TO HAVE BEFORE BUT THE FLIP SIDE IS THAT IF THEY WORK WE CAN LEARN MUCH FASTER, COMBINE MUCH MORE DATA, VERYENTIOUS FICIENTLY MANY MORE PEOPLE CAN BE INVOLVED IN RESEARCH AND WE CAN DEVELOP THESE INTERACTIVE CYCLES. SO THE FIRST LESSON I DRAW FROM WORKING WITH SEVERAL DOZEN OF THESE ORGANIZATIONS IS TO JUST UNDERSCORE WHAT PETER FOX WAS SAYING EARLIER AND PARTICULARLY TO GO BEYOND RESEARCH TO THIS BROADER COMMUNITY WE NEED TO UNDERSTAND THE STANDARDS BECOME THERE'S WHOLE COMMUNITIES THAT NEED STANDARDs BECAUSE THEY WANT TO COLLAB, TO COMMUNICATE, TO WORK TOGETHER TO SHARE THEIR DATA AND LEARN FROM THEM SO THEY'RE PART OF A SYSTEM AND IF YOU WANT TO UNDERSTAND WHAT'S INVOLVED IN THOSE SYSTEMS I WOULD REFER TO YOU THE VA PRESENTATION FRANCIS COLLINS KICKED OFF LAST WEEK FOR THE MILLION PERSONINNISHIAATIVE. THERE'S AN AWFUL LOT OF OTHER THAN TECHNICAL STANDARDS FOR ANY OF THESE ORGANIZATIONS TO DIRECT TO CORRELATE AND CURE DATA, THERE'S MORE THAN TECHNICAL STANDARDS, THEY'RE STANDARDCOMMUNICATION ABOUT HOW CAN YOU FIND OUT THE DATA EXISTS STANDARDS FOR REGISTRIES AND SO FORTH. SO THE MOST SUCCESSFUL THOSE THAT HAVE BUILT NETWORKS IN LEARNING SYSTEMS OFTEN IS PART OF A BUSINESS PLAN IF I CAN USE THAT FROM KISER OR HMO OR KISER NETWORK OR ACADEMIC CENTERS THAT THEY'RE GOING TO COLLABORATE, NOT JUST STANDARDS BUT IDENTIFYING THE QUESTIONS AND USES AND USING THE DAT KACCT THE SHORT HAND SUMMARY HERE IS THAT THE DATA SUPPORT USED IT'S GOING TO BE GOOD, DON'T WASTE YOUR TIME IN A NUMBER OF PLACES THAT FAILED BECAUSE THEY DIDN'T USE DAT AND THEY USE RESEARCHERS TO BE VERY NEUROTIC IN QUALITY OF DATA IN DRAWING CONDILUTION CLIEWGZ FIST THEY DON'T TRUST THE DAT SO WE REALLY NEED TOW HAVE THIS KIND OF NETWORKS AND SYSTEMS THAT AND THE DATA SYSTEMS ARE WHAT PART OF WORK OF THE BROUGHTER COMMUNITY. THE SECOND MAJOR POINT HERE IS THAT WE'VE GOT MANY EXAMPLES OF SUCCESS FAILURE STANDARDS, INI - H IS GOING TO MOVE UP IN A SENSE TO BEING PART OF THE FEDERAL STATISTICAL SYSTEM YOU MAY THE KNOW IT EXISTS BUT THE GOVERNMENT SPENDS SIX BILLION DOLLARS A YEAR ACROSS 15 AGENCIES COLLECTING LARGE AMOUNTS OF DATA AND MAKING IT AVAILABLE. THERE ARE MANY STANDARDS INVOMPLE INDEED THAT INCLUDING A FEDERAL NOTICE FOR PROPOSED RULE MAKING AND ULTIMATELY OMB MY FORMER STAFF GETS INVOLVED IN HAVING TO CERTIFY BEFORE SOMETHING GOES OUT CERTAINLY TO DIRECT DATA FROM A MILLION PATIENTS ON AN ONGOING BASIS. SO THAT INVOLVES A LOT OF COORDINATION WITH OTHER AGENCIES IF YOU'RE GOING TO TALK ABOUT DISABILITY, SOMEONE WILL ASK ARE YOU USING THE SOCIAL SECURITY DEFINITION OF DISABILITY, YOUR DATA SYSTEM HAVE YOU CHECKED IN WITH PEOPLE WHO ALREADY PUBLISH GOVERNMENT STATISTICS ON AIR QUALITY, WATER QUALITY, GEOGRAPHIC AREA DEFINITION OF FAMILY, DEFINITION OF INCOME IF A LARGER COMMUNITY THAT YOU'RE MOVING UP, YOU'RE CREATING A LARGE DATA NETWORK THAT IS GOING TO BE USED AND MANY PEOPLE INTERESTED WHO ARE PART OF THIS BROADER COMMUNITY. I WILL GO THROUGH THE OTHER EXAMPLES EXCEPT TO MENTION WE HAVE THE ANSI STANDARDS, AMERICAN NATIONAL STAN ARDS INSTITUTE IS A BROAD CERTIFYING ORGANIZATION HAS 10,000 STANDARDS, THEY'RE 240 RECOGNIZED STANDARD SETTING ORGANIZATIONS IN THE COUNTRY, WE HAVE 10,000 STANDARDS, BUILDING CODES, ELECTRICAL CODES, COMMUNICATION CODES. MOST OF THESE IN THE UNITED STATES FALL WITHIN THE FRAMEWORK THAT REQUIRES A PROCESS THAT THEY ULTIMATELY CERTIFY AS BEING A DUE PROCESS OPEN STANDARD IMBALANCED PROCESS, SO A LOT TO LEARN FROM, NOT AN EASY PROCESS, BY THE TIME YOU GET TO LARGE COMMUNITIES THERE ARE ALWAYS PEOPLE WITH MULTIPLE AGENDAS AND NUMBER OF PROPRIETARY AND INTELLECTUAL BUT COMING TOGETHER IS NOT ALWAYS EASY AS YOU EXPAND OUT INTO FROM A PURELY RESEARCH NETWORK. THE THIRD I WOULD MENTION IS SINCE WE ARE TRYING TO INTEGRATE THIS, CLEARLY WE HAVE FAR MORE STANDARDS THAT CAN FIT WITHIN A CORE ELECTRONIC HEALTH RECORD AND EXPECT EVERYONE TO COLLECT IT, EVEN--WE STILL DON'T KNOW HOW TO COLLECT--MAYBE DOUG CAN EXPLAIN, MAYBE DOUG OR CHRIS BUT HOW WE GET THE THREE BILLION BASE PAIRS OF THE GENOME INTEGRATE WIDE ELECTRONIC HEALTH RECORDS. WE'RE GOING TO HAVE TO THINK ABOUT CORE STANDARDS AND ALSO ONES THAT ARE CERTIFIED FOR APPS OR MODULES THAT WOULD BE USEFUL FOR DIFFERENT PURPOSES AND COMMUNITIES AND THE FINAL POINT I WOULD MAKE JUST AN OVER VIEW IS AGAIN I KNOW NIH IS USED TO DOING A LOT OF THINGS ON THEIR OWN IS YOU BROADEN OUT THE AMOUNT OF DATA, THE USE OF DATA, THE GOOD NEWS IS THESE ARE USED MUCH MORE BROADLY BUT EVEN WITHIN THE DEPARTMENT THERE ARE MANY PARTS OF THE DEPARTMENT THAT ARE INVOLVED IN SETTING DATA AND MAKING USE OF IT, INCLUDING MEDICARE FOR PAYMENT PURPOSES THAT WILL CLEARLY COME UP VERY QUICKLY IN THE CANCER AREA AS WE REDEFINE HOW YOU PAY FOR IT AND HOW DO YOU ASSURE QUALITY THOSE ISSUES WILL COME UP SO JUST TO UNDERSCORE THAT THERE WILL BE A LOT OF COMMUNITIES THAT GET INVOLVED AND THEY NEED TO BE BROUGHT ALONG AND IN SOME CASES THERE WILL BE MISSIONARY WORK INVOLVED IN ORDER TO GET CONSENSUS. >> OKAY, JESSICA. >> THERE ARE SO MANY GREAT POINTS AND I CONTINUE TO CONTINUE WITH, I WEAR TWO DIFFERENT HATS BUT ONE IS A STANDARD IN A WAY BUT AS A NONEXPERT IN DIFFERENT AREAS AUT FIVE YEARS AGO WE PULLED TOGETHER DATA CLINICAL DAT AND WE PUT IT TOGETHER AND SAID WHAT DOES THAT MEAN. WE WANT TO BE STANDARDS AND WHAT SHOULD WE USE, TELL US? AND WE REACHED OUT TO THE CTSA CONSORTIUM IS SAID HAS ANYONE SOLVED THIS PROBLEM AND TOLD US MA TO DO? AND THEY SAID NOT REALLY. SO WE FIGURED OUT THE ARDS AND CREATED THE TAXONOMY OF THE COMMUNITIES WE TRIED TO CREATE AT LEAST FIGURE OUT A TAXONOMY OF STANDARD WHAT IS IS THE DATA STANDARD AND THERE'S A VERY USEFUL MODEL TO BENEFIT THESE AT THE UNIVERSITY OF OXFORD, DID A PRESENTATION YEARS AGO THAT FRAMED MY THINKING ON THIS WHERE SHE CATEGORIZED STANDARDS AS YOU GOT YOUR CONTENT, CHECK LIST, MINIMUM INFORMATION TYPE THING YOU HAVE YOUR FORMAT, SYNTAX, WHATEVER IT'S GOING TO BE SO TERMINOLOGIES, ANDONATOLOGYS AND THEN THERE'S MORE AS WE EXPLORE, THERE'S INFORMATION MODELS, DOMAIN ANALYSIS MODELS, I LEARN TODAY ABOUT APPLICATION MODELS AND IT'S SO DEFINING STANDARD IS A TRICKY THING TO DO SO THAT SAID WE TRY TO EXPLORE WHAT WERE THE STANDARDS AND SO IN JUST ONE AREA IN THE AREA OF GENOMICS WHAT ARE THE STANDARDS OUT THERE AND WE HAD PEOPLE IN THE ROOM TODAY GAVE USEFUL PRESENTATIONS BUT BEFORE WE KNEW--BEFORE WE HEARD FROM THE EXPERTS, JUST LOOKING AT THERE WE IDENTIFIED 15 STANDARDS THAT WERE RELATED TO GENOMICS, THIS GETS TO MICHELLE'S POINT EARLIER AND THESE WERE DEPRIVATIONRICATED FROM THAT POINT BUT FROM A NEWBIE PERSPECTIVE IT WAS HARD TO KNOW THAT SO WE TALK TODAY ABOUT HOW DO YOU EVALUATE THE GOOD STANDARD. WE REALIZE THERE'S NOT A GOOD STANDARD, IT DEPENDS ON YOUR USE CASE AND YOU DON'T WANT TO PUT IN MORE EFFORT THAN WHAT IS REQUIRED THAN FOR WHAT YOU WANT TO USE IT FOR SO IF YOU WANT TO BE STANDARDS COMPLOY ANT IN A WAY TO REUSE YOUR DATA WHERE A QUESTION IS A WALK DOWN THE HALLWAY AWAY, THAT MAY BE A DIFFERENT LEVEL OF RIGOR YOU NEED TO GO THROUGH THAN IF YOU WILL PUT IT OUT THERE FOR PUBLICATION AND HAVE SOMEONE TRY TO REPRODUCE YOUR DATA THEN AN EVEN HIGHER LEVEL, IF YOU WANT SOMEONE TO QUERY THE DATA AND FIND YOURS AND SAY SHOW ME THE DAT SETS FOR THIS DISEASE AND USE THIS PLATFORM. SO THERE'S NO RIGHT OR WRONG STANDARD. IT DEPENDS ON THE USE CASE AND THATADS THE OTHER LEVEL OF COMPLEXITY OF HOW THE HECK DO WE KNOW WHAT TO USE. A FEW YEARS AGO, WE PUSHED MELISSA AND I, WE CALLED THE SEA OF STANDARDS OF OMICS DATA, SINK OR SWIM, BUT WE TALKED ABOUT THIS TODAY, AND IN TERMS OF HELPING PEOPLE KNOW WHICH THAT ADOPT AND YOU COULD HAVE CONSUMER REVIEWS, AND PEOPLE LIKE THIS STANDARD USE THIS STANDARD AND BIOSHARING AND THIS AREA HERE TODAY DOES A LOT OF THIS STUFF. SO IN TERMS OF SHARING AND IN TERMS GOING IT ALONE I DON'T WE DON'T COME OUT OF THIS WORKSHOP REINVEBTING THINGS WE'RE DOING WELL IN OTHER COUNTRIES SO SO MY STANDARD SELECTION WIZARD, HERE'S WHAT I TRY DOJ THE RIGHT LEVEL, OR HAVE PEOPLE EXPERTS IN A GIVEN, FIELD OF STANDARDS SO THEY COULD TAKE A LISTEN TO WHAT IT IS YOU'RE TRYING TO ACHIEVE AND THEN MAKE RECOMMENDATIONS. I'LL STOP THERE. >> ANITA, WELL THIS IS VERY MUCH PART OF THE CYBER INFRASTRUCTURE AND WE CREATE USE AND CREATE TO DATA STANDARDS ALL THE TIME MY ON STABBED ARDS INVOLVEMENT STARTED AROUND 2001, I WAS INVOLVED WITH EFFORTS AND SUCH AND THAT WAS THE STANDARDS GROUP AND TRYING TO FIGURE OUT WHAT THE STANDARDS WERE IN RELATIONSHIP TO PUBLISHING WAS TRUCKY BUT WE SPENT TIME DOING THAT AND THEN THE EFFORT SERS DISBANDED AND WE REALIZED THERE WASN'T A GREAT LOSS IN THAT BECAUSE THE RELEVANT STANDARD WOULD STILL APPEAR. AND THE DISCOURSE GROUP AND WE DEVELOPED STANDARDS THAT WERE LARGELY UNUSED SO I SHARE THAT EXPERIENCE WITH MANY OF YOU IN THE ROOM. BUT I THINK I WANT TO ECHO WHAT TWO PANELISTS MEAN AND I WANT TO PUT THESE BRIEF MEANS OUT THERE, THE FIRST IS THE CONCEPT OF A MASLOW HIERARCHY, NOW THIS IS MOSTLY RESEARCH DATA BECAUSE I'M CURRENTLY MOSTLY INTERESTED IN HOW DO YOU LINK, INGEST, MANAGE, PUBLISH RESEARCH DATA AND I CAN THINK OF THAT FOR THE IN, ASLOW HIERARCHY, IT'S THAT YOU NEED TO BE SAVE BEFORE YOU CAN ATTEND TO YOUR HIGHER ORDER NEEDS SUCH AS SELF-FULFILLMENT, I THINK SIMILARLY IT'S CRITICAL IF WE'RE TALKING ABOUT RESEARCH DATA AND RESEARCH DATA STANDARDS. FIRST OF ALL DATA NEEDS TO BE SAVERRED AND ALIGNED BEFORE THE DATA START DOING ITSELF WORK SO IF WE TALK ABOUT DATA STANDARDS IT'S VERY IMPORTANT THAT WE THINK ABOUT WHAT IS THE USE CASE, HOW ARE WE GOING TO USE THIS STANDARD FOR USING RESEARCH DATA OR ANY OTHER TYPE OF CONTENT AND I THINK GENETICS AND I'M WAY OUT OF MY LEAGUE HERE BECAUSE THERE ARE MANY LIFE SCIENTISTS BUT GENETICS ONE GENE CAN PLAY DIFFERENT ROLES IN DIFFERENT PARTS OF THE BODY OR SYSTEMS AND IF WE CAN HITCH HIKE DOWN THE LINE THAT'S A GREAT WAY TO GO BECAUSE THEN YOU DON'T NEED TO ADD ANY NEW METADATA FIELDS OR STANDARDS. ONE EXAMPLE IS THAT ANITA LED THEASTY TO STANDARDIZE TED ORGANISMS AND RESOURCES USED WITHIN A PAPER AND WE HITCH HIKED ON THE AUTHOR KEY WORDS TAG THAT'S A TAG IN THE XML PUBLISHER AND IT'S INDEXED BY PUBMED AND DECIDED TO PUT THESE ENTITIES IN WE WOULD NOT ADD ANOTHER TAG TO THE XML FORMAT, WE WOULD HITCH HIKE ON THE NONUSEDTAL AND THAT'S WHAT I'M TRYING TO SAY HERE I THINK IT WOULD BE GOOD IF WE START WITH THE USAGE AND TART WITH THE DOWN STREAM PROCESS, HOW WILL THIS BE USED AND THEN DESIGN THE METASTREAM DATA AND THEN HAVE IT WHERE YOU BUY A TAG, SO YOU SIT AROUND WITH PEOPLE WHO USE THE OUTPUT OF YOUR SYSTEM THEY SAY WHICH THEY USE AND TALKING METADATA STANDARDS AND THEN THOSE ARE THE ONES YOU ADD UP FRONT SO DON'T JUST DUMP EVERYTHING UP THERE BUT ADD THE ONES YOU WILL USE DOWN STREAM SO THE SECOND ONE I WANT TO PUT IN THE AUDIENCE IS REDUCE, REUSE, RECYCLE MPLET I THINK YOU CAN THINK ABOUT STANDARDS AS WE THINK ABOUT PACKAGING YOU WOULD LIKE TO USE AS LITTLE AS CAN YOU SO YOU DON'T POLLUTE THE ENVIRONMENT WITH UNNECESSARY STANDARDS OR METADATA EVERY TAG NEEDS TO BE ADDED SOMEWHRE BY SOMEONE, BY SOMEONE OR SOMETHING SO IT'S GOOD FOR US TO THINK ABOUT THE METALOGICAL DATA AND REUSE WHAT OTHER PEOPLE HAVE DONE, EMPLOY COMMON ELEMENT SCHEMES AND USE MODULAR ADDITIONS AND I THINK A HUGE QUESTION AND I'M REALLY HAPPY THAT I HAD THIS UP THERE BEFORE THEY RAISED THIS POINT, THE GERONTOLOGYSTS QUESTION OF THE WORKSHOP IS WHAT ARE THE METRICS FOR SUCCESSFUL METADATA AND WHAT ARE THE STANDARDS HOW DO YOU KNOW THEY WORK SO YES, ADOPTION IS ONE BUT MAYBE REUSE IS ANOTHER ONE. >> THANK YOU. >> THANK YOU VERY MUCH SO THAT COMPLETES FORMAL REMARKS OF THE PANEL SO PLEASE COME UP TO THE MICROPHONES THAT ARE NOW SET UP IN THE AISLES TO MAKE YOUR COMMENTS AND I'LL PUT THE FIRST SLIDE BACK UP AGAIN. WOULD YOU ALSO INTRODUCE YOURSELF. >> I'M ERIC NEWMAN FROM CROWN NATION MEDICINE, I THINK THE LAST POINT ANITA BROUGHT UP FOR THE GENERAL QUESTION FOR THIS GROUP IS, IF WE'RE THINKING ABOUT WELL WE'RE TRYING TO COMBINE LOTS OF DIFFERENT INFORMATION, JESSICA MENTIONED OMICS, I THINK IT'S MAYBE ONE OF THE LARGER TARGETS IF WE WANT TO USE METADATA HOW DOES IT ALLOW US TO NOT JUST INTEGRATE FOR INTEGRATION SAKE BUT IF YOU WANT TO ANSWER TOUGH QUESTIONS IN MEDICINE, YOU NEED THOSE FOR MERGING DIFFERENT SETS OF DATA, SOME THAT MIGHT BE REALLY CLINICAL DATA AND OTHERS THAT MIGHT BE MORE LIKE KNOWLEDGE FROM THE LAST 20 YEARS OF ARTICLES BUT WHEN YOU PUT THAT TOGETHER, HAVE YOU THE TOOLS AND OTHER THINGS AND WORK WITH, AND THEY FIND OUT PATTERNS BECAUSE ONE OF THE THINGS WE DO WELL IS ALL THESE BRILLIANT COMPUTER SISTS WRITE THOSE OVER AND OVER AND OVER AGAIN WE HAD A TERM FOR THAT IT'S CALLED SHIMMING AND ONE OF THE THINGS THAT WOULD BE GREAT IS WITH DATA, METADATA STANDARDS IS IF ALL THESE THINGS WERE TRYING TO DO STILL WOULD BE MUCH EASIER AND THESE ARE DOZENS OF USE CASES THAT ARE POSSIBLE WITHIN A MATTER OF HOURS RATHER THAN MONTHS MAYBE I'M HOPING TO TOO WITH METADATA IS TO CONNECT AND FIND THE DAT BUT FIND IT AND SHARE THAT AND POINT OUT THE KNOWLEDGE GRAFFS THAT ARE WITHIN IT? >> SO I'LL TAKE A CRACK WITH THE OBAMA PRECISION MEDICINE WHICH IS NOT HIS, IT CAME OUT OF A REPORT, THE VISION THERE IS EXTRAORDINARY AND IT'S ALL ABOUT PUTTING TOGETHER DATA WHETHER IT'S CLINICAL, THREE THE EHR SYSTEM, THROUGHOUT YOUR ENTIRE LIFETIME OR GENOMIC DATA OR PUBLICATIONS, WHETHER IT'S ENVIRONMENTAL DATA, WHATEVER IT IS, AND IT'S A BEAUTIFUL VISION BUT HOW DO YOU REALIZE THAT, STANDARDS ARE ONE THING IF YOU'RE ABSOLUTELY RIGHT TO CREATE THE TOOLS TO CREATE THE KNOWLEDGE NETWORK, FOR THE COMMONS AND A KNOWLEDGE NETWORK, WHERE REAL KNOWLEDGE COMES OUT AT THE END AND THEN WE'RE WASTING OUR TIME AND IT IS OF COURSE REALLY TO CAPTURE THE CONTENT AND THE DATA FOR EVERY INDIVIDUAL BUT TO MANIPULATE THOSE DATA SO THERE'S SCIENTIFIC DISCOVERY AND I WHOLE HEARTEDLY AGREE WITH YOU SO I'LL LET OTHER PANELISTS CAME IN PLEASE,. >> YES YOU'RE ONTARGET AND THE BEST EXAMPLE OF BRING THIS IS TOGETHER IS THE COMMON DATA MODEL I MENTIONED WHICH HAS NIH FDA AND PCOREY COMBINING, THEY USE THE SAME DATA COORDINATION CENTER FOR $DISTRIBUTED DATABASE RESEARCH THAT'S RICH PLATS, HARVARD BILL GRIM AND THEY HAVE A CENTER WITH ROB, BUT RUNNING FROM DUKE THIS IS EXCITING BECAUSE I THINK THE PEOPLE HAVE BEEN WORKING THIS WITH THE THREE AGENCIES SEE IT AS THE PART OF THE FOUNDATION AND OTHER THINGS IS THEY CREATE A COMMON DATA MODEL WHICH IS THEY PULL THE DAT FROM THE ELECTRONIC HEALTH RECORDS TTURNS OUT 80 OR 90% OF ELECTRONIC HEALTH RECORDS YOU DON'T NEED FOR MOST RESEARCH CREATE A COMMON DATA STRUCTURE WHERE YOU CAN SEND AND WRITE ONE COMPUTER PROGRAM THROUGH THE THROUGH RICH PLATCENTER THAT WILL ACCESS ALL THOSE DATABASES, SAY 25 DIFFERENT SITES. AND PULL BACK IN THE SAME DATA SO YOU CAN DO THE PROJECTS FROM THREE YEARS FROM HAND COATING PAPERS CAN BE DONE IN TWO-THREE WEEKS THAT WAS PRESENTED AT THE NIH GRAND ROUNDS IN DECEMBER SO THAT'S A MODEL THAT'S COMING TOGETHER AND MAKES A LOT OF SENSE BUT YOU PULL EACH OF THESE PARTICIPATING INSTITUTIONS PULL THE DATA OUT OF ELECTRONIC HALTH RECORDS, CREATES A COMPARABLE FILE STRUCTURE TAKEN--THEY'S CURATED SO CAN YOU GO TO ONE SET OF TOOLS IN THE RICH PLATCENTER AND DESIGN YOUR COMMON QUERY AND RESEARCH PROJECT AND THEN THEN SEND THAT OUT AND THAT WILL BE ACCESSED. THE RAW DATA NEVER LEAVE THE PARTICIPATING INSTITUTION BUT THEN COMES TOGETHER AND GOES BACK TO THE USER SO THAT'S A PRETTY FAR ALONG STRUCTURE THAT'S TRYING TO ADDRESS SOME OF THE PROBLEMS YOU WERE RAISING. >> AND OF COURSE THE ISSUES OF UNDERLYING SEMANTICS AND MAKE SURE YOU ARE COMPARING APPLES TO APPLES. >> WELL, YEAH, THE COMMON DATA MODCELL ONLY ABOUT 10 ITEMS AND IT'S THE ONES THAT HAVE BEEN STANDARDIZED, OR WE HOPE HAVE BEEN STANDARDIZED BY DIAGNOSIS, PROCEDURE, LAB TEST VALUES AND VO ON. TRYING TO EXTEND THIS TO GENOMICS WILL BE A CHALLENGE BUT THAT'S THE--THAT'S THE FRAMEWORMES TOGETHE R. >> ANITRIC >> --ANITA YOU HAD A COMMENT? >> YES ONE THING I SEE IS THAT PEOPLE LIVE IN DIFFERENT TECHNOLOGY MIND SETS AND IN PARTICULAR IN THIS TYPE OF RESEARCH INTEGRATION YOU HAVE PEOPLE IN A DATABASE MINE SET WHICH IS AND SEMANTIC MIND SET INDGREATING DIFFERENT DATABASES AND THERE'S A THIRD GROUP THAT WILL BECOMING INCREASINGLY IMPORTANT AND THEY HEAR CONSUMER WEB SITES WAY AHEAD OF THE SCIENTIFIC INFRASTRUCTURE AND IT'S THE SEARCH AND RETRIEVAL MIND SET WHERE YOU'RE LOOKING AT USAGE AND A. B. TESTING AND WHAT WILL A USER DO AND YOU ARE LOOKING AT EVALUATIONS HOW GOOD IS A RESPONSE THAT I'M GETTING AND SO, ONE THING IS THE TECHNICAL INTEGRATION OF THOSE THINGS, BUT EVEN MORE IMPORTANT IS THE CULTURAL INTEGRATION OF DIFFERENT WAYS OF LOOKING AT A PROBLEM AND DIFFERENT WAYS OF CALLING IT A SUCCESS WHEN YOU FIND A SOLUTION AND I ACTUALLY THINK, SINCE WE'RE HERE AT THE NIH, THIS IS A GREAT OPPORTUNITY FOR EDUCATION, NOT JUST IN EDUCATION IN LEARNING TECHNOLOGIES BUT EDUCATION IN TERMS OF GOING OUT AND WORK NOTHING A GROUP WHERE THIS MIND SET EXISTS AS A DATABASE PERSON SIT NOTHING A RETRIEVAL GROUP AND EXPERIENCING THOSE METRICS BECAUSE THAT'S A LOT OF WHERE THE COMMUNICATION NEEDS TO OCCUR. >> THAT'S A LOT OF COMMUNITIES THAT WE'VE BEEN TALKING ABOUT. >> I THINK WE ALL, GHEE THIS IS A GLOBAL EFFORT SO MY QUESTION IS TO WHAT EXTENT DO WE TAKE INTO CONSIDERATION OR SHOULD WE SO THESE LARGE CONSIDERATION OUTSIDE THE U.S. SO WE CAN ADDRESS THAT AS WELL, GO AHEAD. >> SO IF I UNDERSTAND THE QUESTION, WHAT EFFECT DO WE NEED TO EXTEND TO THE U.S. WITH INTEGRATION AND DEVELOPMENT AND MANY OF THESE ACTIVITIES HAVE RAMIFICATIONS THROUGHOUT THE WORLD WOBDERFUL--WONDERFUL EXPRLD HOW IS THIS ADDRESSED? SO A LOT OF GLOBAL COMMUNITIES THAT ARE THERE FOR THE IHTO, THE INTERNATIONAL HEALTH TERMINOLOGY STANDARD DEVELOPMENT ORGANIZATION, THE DEVELOPERS OF SNOMED CT, THESE ARE CONSORTIA BASICALLY, THE ITO IS A CONSORTIUM OF 27 MEMBERS LAST TIME I COUNTED AND OF COURSE ORGANIZATIONS SUCH AS THE AUTOFOUNDRY ARE INHERENTLY RESTRICTED BY ANY COUNTRY OR ANYTHING, SO BY AND LARGE THESE INTEGRATION EFFORTS ARE ALREADY ADDRESSED SO THERE ARE NATIONAL INITIATIVES IF YOU WISH AND WHEN THE U.S. GOVERNMENT WANTS TO STANDARDIZE THE USE OF SOME TERMINOLOGIES AND ONTOLOGIES FOR EXAMPLE, FOR MEANINGFUL USE AND MAYBE DOUG WILL ALOUD TO THAT AND THEY ARE STANDARDS MORE RELEVANT TO THE U.S. AND THAT'S THE CASE FOR EXAMPLE, THAT ARE USED WORLD WIDE BECAUSE IT'S MEANT TO REPRESENT IN PRIORITY WHAT WHAT DRUGS ARE USED IN THE ON THE U.S. MARKET FOR EXAMPLE, BUT I WOULD SAY THAT BY AND LARGE THESE EFFORTS ARE INHERENTLY LOCAL. >> I JUST WANT TO RESPOND TO THE COMMENT OF THE NOTION OF A COMMON DATA MODEL THAT WILL BE WHAT WILL SOLVE OUR PROBLEM. I THINK IT IS FOLLY TO THINK THAT WE WILL HAVE ONE RING TO RULE THEM ALL. AND THANKLY MODELS HELP US AND AREN'T GOING TO HAVE A SINGLE SOLUTION TO ALL THE KINDS OF QUESTIONS THAT WE WANT SO WE THEY'D TO BE CAUTIOUS ABOUT CLAIMS THAT A SINGLE MODEL WILL SOLVE OUR PROBLEMS BECAUSE I DON'T TRUST ANYBODY THAT'S SINGLE WILL FIX ALL PROBLEMS. >> WELL WHAT IF YOU TELL US THAT YOUR SINGLE THING WILL FIX ALL PROBLEMS? >> WELL I WON'T THINK IT WILL AND WE CAN TALK ABOUT MY THING AT 1:00 O'CLOCK IT'S JUST REALLY IMPORTANT TO UNDERSTAND THAT WE ARE GOING TO GET LOTS OF THIS WRONG. AND IF YOU DO THIS WE WILL HAVE THE EQUIVALENT OF AOL, BUT NOT INTERNET, REMEMBER WE COULD GET INTERNET BUT WE COULDN'T GET AMAZON FROM AOL. SO WE HAVE TO THINK ABOUT HOW WE FRAME THE PROBLEM IF YOU AGREE WTH 90% OF THE INFORMATION, SCIENTISTS CREATE YET ANOTHER STANDARD THAT WILL HELP ADOPT THAT ADDITIONAL 10%. SO WE HAVE TO BE CAUTIOUS ABOUT HOW WE RUN THEM ALL. >> I WANT TO INTRODUCE THE CONCEPT OF BOUNDARY OBJECTS, I DON'T KNOW, IF EVERYONE IS FAMILIAR WITH THAT, I LEARN IT FROM TIM CLARK, IF YOU GOOGLE IT SAYS BOUNDARY OBLIGATIONS INTERSECTS ARE OBJECT WHICH IS ARE PLASTIC ENOUGH TO ADAPT TO LOCAL AND CONSTRAINTS OF THE PARTIES BUT ROBUST ENOUGH TO MAINTAIN A COMMON IDENTITY ACROSS SITES SO COMMON USE BECOMES STRONGLY STRUCTURE INDEED INDIVIDUAL USE AND I THINK THAT'S A MUCH BETTER WAY TO THINK ABOUT STANDARD AS BOUNDARY OBJECTS SO THE WAY FROM COMMUNITY AGENCY IN A WAY THAT WORKS FOR BOTH OF THEM. SO I COMPLETELY AGREE WITH YOUR POINT OF VIEW BUT THIS IS MAYBE A MORE USEFUL IMAGE THAN THE IMAGES. MAYBE THE COMMON DATA MODEL IS GREAT FOR WHAT IS TRYING TO DO SORT OF A LOWEST COMMON DENOMINATOR AND WE MAY BE ABLE TO BUILD ON THAT FOR DENATIONAL LIBRARY OF MEDICINICS DATA BUT AS I SAID IN MY THIRD POINT MUCH OF THE USERS OF THE USES ARE GOING TO HAVE TO REQUIRE SPECIALIZED DAILY BASIS AT COLLECTION AND SPECIALIZED COMMUNITIES THAT DON'T FIT THAT MODEL. SOPHISTICATED WOULD COMPLETELY AGROW, WE CAN'T FORCE TOO MUCH, I THEN IS A PROBLEM ONC DEALT WITH AND WHAT ARE THE CORE STANDARDS AND VERY QUICKLY YOU GET TO THE CARDIOISTS COME IN AND SAY WE WOULD LIKE ANOTHER 1100 PLEASE JUST FOR CARDIOLOGY AND THE CANCER THE ONCOLOGY SAY THIS IS OUR PERIODIC ELECTRONIC HEALTH RECORD AND OUT OF THAT STAGE, I THINK IF WE HAVE A CORE, THAT'S A CORE AND THEN WE THINK ABOUT THESE MODULES AND OTHER THINGS THAT FOR PARTICULAR COMMUNITIES AND PURPOSES AND THOSE WILL NEED TO EVOLVE. >> JESSICA YOU WANT US TO COMMENT ON THIS? >> YEAH IS THIS IS RECORDED SO I DON'T WANT TO--HYPOTHETICALLY IF THERE WAS A COMMON CORE NET MODEL AND IF ONE AT INSTITUTION WHERE ONE WAS INVOLVED AND YOU DID A TECHNICAL EVALUATION OF MULTIPLE OPTIONS FOR COMMON DATA MODELS AND CAME UP WITH SEVERAL TECHNICAL REASONS WHY A DIFFERENT MODEL WAS THE WAY TO GYOU MIGHT HAVE PRESSURE TO CHOOSE THE PCORONET ANYWAY, HYPOTHETICALLY THAT MIGHT HAVE HAPPENED, IT HASN'T BEEN DONE YET SO WE DONE KNOW HOW IT'S GOING TO TURN OUT BUT I WANT TO RAISE A POINT TO SAY I HAVEN'T DECIDE FEDERAL THAT'S A GOOD THING OR A BAD THING. IT COULD BE A BAD THING BECAUSE THE OTHER MODEL WOULD BE USEFUL AND MORE UPFRONT AND ON THE OTHER HAND IF WE CAN MAKE IT WORK FOR OUR EFFORTS AND IF WE CAN WORK WITH THE PCORNET PEOPLE MAKE IT WORK FOR OURS AND OTHER USE CASES MAYBE IT IS BET TORE BE TRYING TO COMTOGETHER TECHNICALLY. I WILL GIVE YOU THE FLOOR YOU I JUST WANT TO FOLLOW UP ON THE STORY AS JESSICA MENTIONED COMING IN AND SAYING WE WANT TO BE STANDARD COMPLIANCE, HOW DO WE DO HA? SO I'M WONDERING HOW ORGANIZED DO WE HAVE TO BE? THERE ARE SPECIFIC THINGS WE HAVE TO BE MORE EFFICIENT IN THE WAY THAT PROCESS WOULD WORK SO FOR INSTANCE TO PEOPLE, DO WE ABSOLUTELY NEED A TRAINING PROGRAM THAT TELLS PEOPLE JUST LIKE IF I WANT TO HIRE A PROJECT MANAGER AND THEY TRAIN SOMEONE TO GET IT LIKE FOR INSTARRED DEVELOPER AND DO WE NEED TO HAVE A CENTRAL PLACE WHERE THERE'S A NETWORK PATIENTS IN THE CLINIC THAT YOU ARE SHOWS THE STANDARDS AND THE ONTOLOGY OF ALL THE STANDARDS THAT PEOPLE CAN GO TO AND REGISTER STANDARDS AND KIND OF LIKE A WIKIPEDIA THING? >> YEAH, I WAS SHARING A LOT OF THAT BUT GO AHEAD. >> YEAH SO I WAS SHARING BUT I'M WONDERING LIKE THIS GROUP, WILL THIS MEETING AND AS A COMMUNITY WE NEED TO DO THESE THINGS MORE ORGANIZED AND EFFICIENCY IT WOULD BE NICE TO HAVE SOMETHING LIKE THAT. SO IF YOU'RE MAKING A RECOMMENDATION THAT GO INTO THE NOTES AND WE CAN FOLLOW UP ON THAT. YES, PLEASE, GO. >> A COUPLE POINTS ON THE COMMUNITY AND THE STANDARD MODELS, BECAUSE ONE EXAMPLE THAT HAPPENED WITH MIDI, SO THEY'RE THE COMMUNITY THAT THINK WE'RE CENTERED AROUND TECHNOLOGY, SO [INDISCERNIBLE], THAT, AND THE BIOLOGICAL [INDISCERNIBLE] FOR THAT. AND THEN WHAT HAPPENED IS THE DIFFERENT COMMUNITIES AROUND TECHNOLOGIES CAME TOGETHER IN THE UMBRELLA WHICH SAID THE COMMONALITYS THAT THEY'VE FOUND RATHER THAN TRYING TO COME AND SAY, MAYBE THESE START AND PUT THEM [INDISCERNIBLE] THESE ARE THINGS YOU HAVE TO ADDRESS BEFORE HAND. SO I THINK THIS IS MORE-- >> THIS IS SORT OF AND COMMONALITY WISHES AFTER THE FACT. >> ALL RIGHT, WHEN YOU A MIDI, WE TRY TO ORGANIZE SEVERAL MEETINGS, HAVING ALL STAKEHOLDERS OF THIS AND THESE INITIATIVES. AND THE OUTCOME WAS A BIT DISAPPOINTING AND WE THOUGHT THERE WAS A LOT OF RESISTANCE AND PEOPLE FELT THEY OWNED THAT AND ASSESSED IT AND REALLY WANT TO LET GO, EVEN THE SIMPLEST CONFIDENCE, NAME OF ORGANIZATION, AND THING OF THAT, SAY, IT'S AN UPHILL STRUGGLE BUT SOME OF THE OTHER GROUPS WERE COLLABORATING AND IT'S A MIXED RESULTS. >> OKAY, SO ONE OF THE THINGS THAT I KEEP HEARING SORT OF UNDERNEATH THE SURFACE OF DISCUSSIONS AND THAT I'D LIKE TO HIGHLIGHT AS PART OF THE WAY OF THINKING ABOUT STANDARD SYSTEM THE NOTION OF THINK BEING A STANDARD LIFE CYCLE AND THIS FLIES RIGHT IN THE FACE OF CHRIS' COMMENTTHE FACT OF WAY TOO MUCH WHAT WE DO IS COUPLED TOGETHER IN THE SPARE TIME IN THE MINIMUM STANDARD YOU NEED TO WORK IS A HIGH ASPIRATION, BUT I WONDER WHETHER IT MIGHT BE WORTH THINKING MORE EXPLICITLY ABOUT LIFE CYCLE AS WE DEVELOP AND REVISE STANDARDS AND AND WE HAVEN'T PUT A LOT OF CASE INTO'S METADATA DESCRIBING THE STANDARD AND THAT CREATES LONGEVITY PROBLEMS AND THE SECOND IS WE CLEARLY WANT TO REVISE STANDARDS AS IT MOVES FORWARD BUT AS EXAMPLES ARE ACROSS THE BOARD SHOW, PERHAPS, YOU KNOW SORT OF HARDWARE DEVELOPMENT IS A GREAT PICTURE. IF THERE'S NOT ATTENTION GIVEN TO HOW ONE STANDARD EVOLVES INTO ANOTHER OR HOW THE BODY OF DATA IN ONE STANDARD TRANSITIONS TO A NEWER STANDARD, YOU END UP WITH EITHER THE SCS PROBLEM, YOU END UP SUPPORTING A LEGACY STANDARD FOR DECADES BECAUSE THERE'S A HUGE BODY OF DATA IN THAT FORMAT AND YOU CAN'T LET GO OF IT OR YOU END UP WITH DATA GOING DARK, RIGHT? THE THIRD PROBLEM I THINK WE GET INTO IS THE NOTION WE HAVE ONE SHOT TO GET THE STANDARD RIGHT. SO EITHER WE FIGHT ABOUT IT FOR FIVE OR SIX YEARS, BECAUSE DOG GONE IT, I NEED TO GET PRINCIPLES, MY NAME, MY STANDARDS INTO THIS BECAUSE IF I MISS THE BUS IT'S NOT COMING BACK FOR ME OR WE LOSE THE ABILITY TO RESPOND TO NEW DATA AND I DO WONDER IF MORE EXPLICITLY MODELING, THE DEVELOPMENT, THIS ADOPTION, DISSEMESTER NATION, OBSOLESCENCE WITH RETIREMENT AND PLACEMENT OF THE STANDARD AS WE THINK ABOUT DEVELOPING THE STANDARD IN THE FIRST PLACE MIGHT HELP US IN SORT OF DEVELOPING A HEALTHIER ECOSYSTEM, SO WHEN THE TREE FALLS TO USE YOUR SUSTAINABILITY MODEL, THERE'S A WAY TO RECYCLE THAT WOOD. >> AND I THINK THAT THOSE ARE GOOD COMMENTS AND I THINK THAT DOES RELATE TO OUR TOPIC HERE WHICH IS, YOU KNOW WHAT'S THE INVOLVEMENT OF THE COMMUNITY IN THAT AND THE COMMUNITY WOULD HAVE TO BE VERY CLEARLY BEHIND ANY EFFORTS, YOU KNOW IN THIS LIFE CYCLE ESPECIALLY WHEN WE'RE TALKING ABOUT SUN SETTING AND STANDARD, BECAUSE IF THERE'S VALUABLE DATA OUT THERE, YOU KNOW PEOPLE ARE STILL USING IT, THEN THAT WOULD ARGUE, BUT AGAIN IT WOULD COME FROM THE COMMUNITY. >> THAT'S SORT OF HELPS. >> RIGHT, ANYBODY ELSE HAVE A COMMENT? , I COMPLETELY AGREE AND I THINK IT'S A GOOD MODEL AND JUST RESPONDING FOR A MOMENTOT POINT OF FIVE-SIX YEARS TO DEVELOP A SUMMER AND THE WAY TECHNOLOGIES ARE DEVELOG TODAY, YOU WILL NOT NEED ANY OF THE STUFF YOU'RE PUTTING INTO THE STANDARD IN FIVE YEARS BECAUSE WE WILL HAVE TOOLS TO PULL IT OUT AUTOMATICALLY AND I THINK THAT'S--THAT'S PERHAPS, BUT, YOU KNOW I THINK THAT'S THE OTHER PART IS THAT PEOPLE GET LOCKED IN AT THE BEGINNING OF THE CREATION OF A STANDARD AND DON'T STAY ALERT ENOUGH TO DEVELOPMENTS AROUND THEM BECAUSE THEY'RE TRYING TO MAKE THIS HAPPEN. SO I'M ABSOLUTELY IN YOUR IDEA OF FORMING AN IDEA OF DEVELOP WITH THE STANDARD AND MAYBE LIVE AND RETIRE AT SOME POINT BUT I WANT TO ADD THAT IT WOULD BE VERY USEFUL TO HAVE SOMEBODY MONITORING WHAT OTHER WAYS THERE ARE TO FULFILL THE USE CASE THAT THE STANDARD IS FOR. >> BUT JUST ADD ONE MORE THING AND OTHER AREAS OF SCIENCE THAT USE A DIFFERENT MODEL THAN NIH HAS, NIH FOCUSED ON RANDOMIZED CLINICAL TRIALS, THE THIRD ELEMENT THAT MOVES THINGS ALONG IS PREDICTIVE MODELS AND IT'S E-MC-SQUARED AND YOU GO BACK AND FORTH AND YOU USE WHAT DEFINITIONS AND DATA ARE USEFUL BECAUSE HAVE YOU THE FRAMEWORK OF THE MODEL AND THE AIM OF THE EXPERIMENTS IS TO PREDICT OBSERVATIONAL DATA AND AND WE SUDDENLY GIVE INTO QUIRKS AND LOT OF OTHER THINGS AND HAVE DATA BECAUSE THE MODELS ARE BETTER, THE MODEL WORKS BETTER THAT WAY SO SO I THINK AS WE EVOLVE, THIS HYPOTHESIS AS WE EVOLVE MORE PREDICTIVE MODELS, WE CAN BUILD A DYNAMIC WHERE THERE QUESTION ABOUT HOW GOOD IS OUR DATA, OUR ONTOLOGIES CORRECT, HAVE A MORE RIGOROUS TEST BECAUSE THE TEST IS DO THEY HELP US MAKE PREDICTIONS THAT UNDERSTAND REALITY BETTER. CHRIS WOULD BE THE EXPERT IN EVOLVING--INVOLVING CODING. >> CHRIS, GUILTY AS CHARGED BUT IN ANY EVENT I WANT TO PICK UP ON THE ISSUE OF NOTION OF COMMON DATA ELEMENTS BECAUSE ONE OF THE CRITICAL PIECES THAT IS OFTEN LOST IS THE GRANULARITY ISSUE ASSOCIATE WIDE STANDARDS. AS SOON AS WE TRY TO MAKE A FULL MODEL A COMPLETE PRISTINE STATEMENT OF EVERYTHING WE NEED TO SAY ABOUT SPECIFIC KINDS OF DATA, I THINK ALL IS LOST. THERE HAS BEEN AMPLE EVIDENCE AND AMPLE IOM PANELS AND OTHER KINDS OF THINGS THAT TALK ABOUT HOW LARGE SCALE SYSTEMS ARGUABLY BIOMEDICINE AND HEALTH CARE OF LARGE SCALE SYSTEMS IN TERMS OF COMPLEXITY AND SOPHISTICATION OF DATA IS ALWAYS BEST SERVED BY TARS MOANUOUS SET OF STANDARDS THAT ENABLE A MINIMUM OF CREATION OF CONTENT THAT CAN--THE LEGO PIECES IF YOU WILL THAT THEN CAN BE ASSEMBLED TO ACHIEVE THE GOALS OF THE MOMENT AND THAT YOU--THE INTERNET IS THE OTHER OBVIOUS EXAMPLE OF A LARGE SCALE SYSTEM WHERE THE STANDARDS THAT SUPPORTED ARE PARSIMONNUOUS. WHEN YOU LOOK AT THINGS LIKE THE COMMON DATA MODEL, IT IS ARGUABLY A RIGID FIXED STRUCTURE AND THAT IN ITSELF POSES SERIOUS PROBLEMS. I WOULD ALSO POINT OUT THAT IT'S JUST COMPLETELY DISCONNECT FRIDAY CLINICAL STANDARDS WHICH MAKE ITS COMMONALITY SOMEWHAT SUSPECT. NEVERTHELESS, I THINK THE MESSAGE IS WE HAVE TO LOOK CAREFULLY AT IDENTIFYING THE LOWEST LEVEL OF SPECIFICATION OF DATA WE CAN IMAGINE THAT SATISFIED THE CRITERIA OF CONSIST CONSISTENCY AND THEN ENABLE THEM AS USE CASES DEMAND WE TRY TO CREATE A LARGE MONOLITHIC COMPREHENSIVE DATA STANDARD IT WILL SURPRISE PROBABLY NOT SCALE OR BE USEFUL FOR THE NEXT USE CASE. >> OLIVIA. I WOULD LIKE TO ADD THAT THE SAME CAN BE MADE FOR TERMINOLOGY ONTOLOGY INTEGRATION AND I WASN'T THERE WITH THE UMLS WAS BORN, YOU WERE THERE, AND-- >> I GAVE BIRTH. >> [LAUGHTER] >> YOU CAN SEE IT BUT THERE WAS A CHOICE, A GENUINE CHOICE AT THE TIME BETWEEN CREATING TOP DOWN BIOLOGY FOR INTEGRATING MEDICINE AND ONTOLOGIES AND NLM SITED WHAT WAS PROBABLY LESS ELEGANT IF YOU WISH OR LESS PURE FOR SPECIAL PEOPLE WOULD HAVE SEEN IT BUT ENDED UP BEING MUCH MORE PRACTICAL AND DECIDED THEY WOULD GO WITH WHAT IS EXISTING STANDARDS AND THIS IS THE NOTION OF GOOD ENOUGH FOR A GIVEN USE CASE IS PROBABLY WHAT WE NEED TO KEEP IN MIND ALSO WHEN WE LOOK AT SOME OF THESE STANDARDS. >> MEL ICES AND AND THEN ONE MORE AND THEN WE HAVE TO STOP. >> GO AHEAD. >> SO I THINK I HAVE MORE OF A QUESTION AND MAYBE THIS IS SOMETHING THAT WILL COME BACK LATER TO THIS AFTERNOON, BUT YOU KNOW I REALLY LIKE THIS IDEA OF THE SORT OF STANDARD LIFE PSYCHE AND HE WILL WE THINK ABOUT RESEARCH AS HAVING LIFE CYCLES AND RESEARCH DATA AS HAVING LIFE CYCLES BUT WE DON'T HAVE A NUMBER OF DIFFERENT COMMUNITIES OF DOING THAT. THERE'S A CERTAIN--THAT'S A GREAT IDEA AND YOU KNOW WHEN DO WE PUT THE STAKE IN THE ORTOF THE STANDARD AND ALL THOSE THINGS WE TALKED ABOUT EARLIER. I THINK THERE'S A REAL COMPLEXITY IN HOW DO WE MANAGE THAT LIFE CYCLE PROCESS IN THE CONTEXT OF THE FACT THAT WE HAVE OUR DATA USERS SO THE CONSUMERRIZED DATA, WE HAVE THE STANDARD DEVELOPERS AND THEN WE HAVE A SERIES OF DATA INTEGRATE ONERS AND THE LIKE. SO IT'S KIND OF THESE THREE DIFFERENT GROUPS IF YOU WILL THAT BASICALLY HAVE TO SOMEHOW INTERACT WITH THAT STANDARD LIFE CYCLE AND SO, I'M WONDERING, LIKE HOW SHOULD WE--SHOULD FROM ALL THREE OF THOSE GROUPS LIVE IN A STANDARDS ABOUT THE OR SHOULD STANDARD PEOPLE LIVE IN THOSE GROUPS AND HOW DO THEY PART IN THE MOVEMENT OF THE LIFE CYCLE WHEN IT'S SUCH A DISTRIBUTED NETWORK AND SO MANY PERSPECTIVES THAT HAVE TO BE ADOPTED AND FROM THE IMPOSSIBLE TO BUILDIGRATION PERSPECTIVE WE SEE A LOT OF THINGS THAT ARE BROKEN ABOUT STANDARDS, BUT HOW--HOW DO WE COORDINATE THE FEEDBACK THAT WE HAVE IN THE NEXT INTEGRATOR AND THE NEXT INTEGRATOR HAVE ABOUT SEEING THOSE BROKEN BITS BACK TO STANDARD DEVELOPERS OR TO UNDERSTAND POTENTIAL END USERS REQUIREMENTS SO I THINK ALL THOSE COMMUNITY CONNECTIONS ARE STILL REALLY FRAGILE AND BROKEN AND WHEN YOU TAKE IT INTO THE GREATER CONTEXT OF THIS CYCLE, IT BECOMES COMPLEX AND DIFFICULT, I THINK. >> ONE OF THE THINGS WE DISCOVERED IN WILL EARLY DAYS OF THE UMLS IS AS WE'RE BEGINNING TO PULL ONTOLOGIES, TERMINOLOGIES REALLY INTO THE UMLS WAS THAT BY VIRTUE OF DOING THAT AND DOG THE EXERCISE OF MAPPING THEM TO EACH OTHER, IT WOULD SHINE A BRIGHT LIGHT ON TO WHATEVER TERMINOLOGY, ONTOLOGY WAS BEING PULLED INTO THE SYSTEM, SO IT WAS SORT OF ANOTHER WAY OF SAYING, WELL, YOU KNOW WHAT IS THE QUALITY OF THIS PARTICULAR STANDARD AND IN THE BEST CASES, THE--WE FED THAT BACK, WE ALSO FED IT BACK BUT IN THE BEST CASE, WHEN WE FEED IT BACK TO THE DEVELOPER OF THAT TERMINOLOGY OR THAT STANDARD THEY MADE ADJUSTMENTS SO THAT'S A REALLY--THAT'S SORT OF A PART OF A LIFE CYCLE ARE YOU CONTINUALLY IMPROVE WAG YOU'RE DOING, AND THAT'S JUST ONE WAY IS BY SHINING A BRIGHT LIGHT ON IN THIS CASE, THE TERMINOLOGIES. >> ANITA. >> VERY BRIEFLY ADDAD TO THAT, SO MIX THE CONCEPT OF LIFE CYCLE, HAVE YOU THIS CONCEPT OF VIRGINS WHICH NOT ALL STANDARDS AND THAT'S BEEN USED A LOT OF TIMES BUT IN A LOT OF STANDARDS THEY'RE NEVER EXPLICITLY SPELLED OUT SO THESE ARE THE CASES FOR WHOM THEY MATTER AND I THINK TPHAOEUGZINIZEATION--ORGANIZATIO N HAVE YOU ALLA OF THOSE TOGETHER, IT DOES LOOK ACHE LOT LIKE SOFTWARE DEVELOPMENT AND THERE'S GREAT PROJECTISES IN SOFTWARE DEVELOPMENT THAT COULD BE TAKEN OVER IN A LIGHT WEIGHT AND LESS FORMAL WEIGH FOR STANDARDS DEVELOPMENT BUT IT'S INTERESTING TO LOOK AT IT, IN THAT WAY. >> OKAY, VERY LAST QUESTION, COMMENT, PLEASE? Y NAME'S SHASHANI, WYNN, FROM NIDA, NIH, SO I THINK WE ALY TALK TO VERY, VERY BLURRED AREA BETWEEN [INDISCERNIBLE] MODEL AND THE MODEL ACTUALLY IS A CONCEPT AT SUBTRACTION, HOWEVER SENDER IS THE VALUE IS THE TERMS SHOULD BE CATEGORIZED IN THE STRUCTURE. SO I THINK THESE TWO AREAS THAT SHOULD GO DOWN NECESSARILY--NOT NECESSARILY COMPLETELY SEPARATE, BECAUSE IN THE PROCESS YOU FIND A LOT OF TERMS THAT [INDISCERNIBLE]. SO WHAT DO WE HAVE TO DO, ACTUALLY? I'M THINKING, IS REALLY SEPARATE THE DATA STRUCTURE AND THE DATA VALUE WHICH ARE THE STANDARDIZED, THE TERM, SO THAT WAY STANDARDIZE TERMS HAVE MUCH [INDISCERNIBLE] LIFE CYCLE. THEN MUCH MORE STABILIZE SEMANTIC MEANING OF THE GROUP OF PERHAPS OVER THE HISTORY OF PEOPLE WERE USED DIFFERENT TERMS TO DESCRIBE SOMETHING. SO MY QUESTION HERE IS, IS THERE ANY ADVERSE IN THIS AREA THINK ABOUT HOW TO STRUCTURE THE STANDARDIZED TERMS? I THINK WHEN YOU CREATE A NEW DATA ELEMENT FOR A CENTER, ONLY WHEN THE TIME, THE SOMATIC MEANING REALLY CANNOT GROUP, THERE'S A GROUP OF STANDARDIZED [INDISCERNIBLE] WILL CREATE A NEW DIALYSIS, OTHERWISE, THE DATA, THE COMMON DATA ELEMENT SHOULD BE RELATIVELY STABLE VERSES THE TERM CREATION OR-- >> THE VALUE OF DATA ELEMENTS, RIGHT? ANYBODY WANT TO COMMENT ON THAT. >> LYNN? >> WELL I THINK JUST UNDERSCORE IT'S ESSENTIAL IMPORTANT QUESTION BECAUSE AS I UNDERSTAND IT, THE WHOLE RESEARCH AGENDA OF PRECISION MEDICINE IS TO SAY THAT OUR CURRENT, MANY OF OUR CURRENT CATEGORIES AND ONTOLOGY IS A DISEASE DEFINITIONS ARE WRONG AND WE NEED TO MOVE RESEARCH ALONG TO A POINT TO UNDERSTAND WHAT THE NEXT ONE OR BETTER ONE IS. CHRIS IS VERY MUCH INVOLVED IN THAT, SO WE NEED A PROCESS TO MANAGE THAT AND I VIEW THAT AS I UNDERSTAND THE PRECISION MEDICINE INITIATIVE,A'S AN OBJECTIVE THAT THEY--THAT'S VERY IMPORTANT BUT I HAVEN'T SEEN ANYONE LAY OUT EXACTLY, EXCEPT FOR THE EMERGE NETWORKS, WORK, EXACTLY WHAT IS THE NATIONAL PROCESS GOING TO BE TO START WHERE WE ARE NOW, WHERE THE ICDA CODE AND SO FORTH AND THEN IT RAPIDLY INTEGRATE THE MORE SOPHISTICATED INFORMED PRECISION MEDICINE KINDS OF DEFINITIONS OF DISEASE AND WAYS OF THINKING ABOUT VALIDATING TREATMENTS. >> GREAT, OKAY, WELL, THANK YOU VERY MUCH TO ALL THE PANELISTS AND TO ALL OF YOU FOR PARE -PARTICIPATING. THANK YOU. WE WILL BREAK FOR LUNCH AND PLEASE RETURN TO THE ROOM BY 1:00 O'CLOCK. GOOD AFTERNOON, I'M CHRIS, IT'S MY PLEASURE TO INTRODUCE DOUG FRIDSMA, DOUG AS YOU KNOW THE OFFICE COORDINATOR FOR MANY YEARS AND AND RECENTLY IN THE FALL ASSUMED HIS ROLE AS PRESIDENT AND CHIEF EXECUTIVE OFFICER AFRONS THE MEDICAL INFORMATICS ASSOCIATION, I DON'T KNOW IF IT'S GOOD OR BAD, I KNEW DOUG FOREVER I KNEW HIM WHEN HE WAS A AND HE'S HAD AN ILLUSTRIOUS CAREER AND WE HAD A GREAT WOMAN TO LEARN FROM HIM, SO DOUG? >> THANKS CHRIS! AND SOME OF THE WORK WE HAVE TE, JUST TO GIVE YOU AN UNDERSTANDING OF THINGS SO WHEN HIGH-TECH WAS PASS INDEED 2009, I ACTUALLY STARTED AT THE OFFICE OF THE NATIONAL COORDINATOR FIRST ON THE H. I.T. STANDARDS COMMITTEE AND THEN I LATER MOVE INTOED UNDER BLOOMEN THAT WILL'S ADMINISTRATION AND THEN KAREN DESALVO. OMC WENT FROM A BUDGET OF ABOUT 61 BILLION OF THE TWO BILLION ONLY 60 MILLION WENT TO THE TECHNICAL STRUCTURES AND STANDARDS SO LESS THAN FOUR% OF THE DOLLARS SPENT AT OMC WENT TO THE TECHNICAL INFRASTRUCTURE. SO PART WHAT HAVE WE ARE GOING TO DO IS TALK ABOUT HOW DO YOU GET A LOT DONE WITH A LOT OF--WITH VERY FEW RESOURCES. AND TELL YOU ABOUT THE BRUISES I HAVE FROM MY EXPERIENCE--I WILL GIVE EXPLAN ANTICIPATIONS OF THESE, I KNOW WE HAVE 30 OR 40 MINUTES TO TALK AND I HOPE AT THE END WE WILL LEAVE ROOM FOR DISCUSSION SO CAN YOU ASK ME THE HARD QUESTION. I'M NO LONGER WORKING FOR THE GOVERNMENT SO NOW I AM UNCONSTRAIN INDEED WHAT IT IS I CAN SAY SO FEEL FREE TO ASK QUESTIONS AS WELL. IT'S JUST BASICALLY MY ONE, TWO, THREE, FOUR, FIVE, TALK. OKAY. SO AS WE GO THROUGH THIS, YOU CAN WRITE DOWN THOSE NUMBERS SO IT WILL BE A SERIES OF SECTIONS, WE'RE GOING TO TALK ABOUT ONE, TWO, THREE, FOUR, FIVE; OKAY, SO THE FIRST THING I WANT TO TALK ABOUT IS ONE CULTURAL ART SCALE SYSTEM AND WHY FRAMING THE QUESTION MATTERS. SO, THOSE WHO ARE CONNECTED TO THE NETWORK, IF YOU TYPE IN ULS, SEI, THE FIRST HIT YOU GET IS A REPORT DONE BY CARNEGIE MELON AND THEIR SOFTWARE ENGINEERING INSTITUTEOT ULTRA ART SCALE SYSTEM. I WOULD RECOMMEND EVERYBODY TAKE THE DOCUMENT, DOWNLOAD THE PDF AND READ ONLY THE 28 PAGES OF THAT DOCUMENT. IT'S ABOUT 150 PAGES BUT THE FIRST 28 PAGES FRAMES THE QUESTION THAT THEY WERE TRYING TO ANSWER AND THIS IS A REPORT THAT WAS GENERATED THROUGH THE DEPARTMENT OF DEFENSE THROUGH THEIR ACQUISITIONS TEAM. DONE IN 2006 AND BASIC ELEMENT AS WE THINK ABOUT WEAPONS ACQUISITIONS AND COMMUNICATIONS SYSTEMS HOW DO WE MAKE SURE WE CAN MANAGE A BILLION LINES OF CODE ALL OF WHICH HAS TO INTERACT TOGETHER AND ALL WORK TOGETHER WITH IT. SO IF YOU THINK ABOUT HOW WE WANT TO INTEGRATE OUR ENGINEERING PRIZE AND THE RESEARCH ENTERPRISE, IT'S IMPORTANT TO RECOGNIZE THAT WHAT WE'RE DEALING WITH AND NOT BUILING A BUILDING. THIS IS NOT ARCH ECTOMYOSINNURE AND ENGINEERING THIS, IS CITY PLANNING. AND SO, WITH WE THINK ABOUT HOW WE'RE GOING TO MAG MANAGE AND HAVE ALL THIS DATA WE NEED TO FRAME THE QUESTION AS ONE OF CITY PLANS AND NOT AS ONE OF ARCHITECT, ENTERPRISE ARCHITECTURE AND SORT OF BUILDING A BUILDING. SO IF YOU THINK ABOUT IT WHAT DOES THAT MEAN? IT MEANS WE CREATE AN ECOSYSTEM, PEOPLE WANT TO LIVE IN. WE WANT TO CREATE PLACES WHERE PEOPLE WANT TO BUILD BUILDINGS BECAUSE IT HAS THE NECESSARY INFRASTRUCTURE, IT'S A PLACE THAT'S SAFE TO LIVE. THAT IT HAS THE NECESSARY FACILITIES LIKE WATER, AND SEWER AND INTERNET AND ELECTRICITY AND THERE ARE ROADS TO GET IN AND OUT. AND THAT IN FACT, WE MAY SAY THAT FOR SAFETY SAKE, WE WANT TO MAKE SURE PEOPLE HAVE SOME SENSE OF WHAT THE BUILDING CODES ARE SO THAT THERE ARE INGREESES AND EGRESS RULES AND THINGS ABOUT HEIGHT LIMITATIONS AND THINGS ABOUT THE BUILDINGS BUT WE HAVE TO THINK ABOUT THIS NOT AS WE ARE GOING TO BUILD, YOU KNOW I SAID THE ONE--ONE RING TO RULE THEM ALL, RIGHT? WE'RE NOT GOING TO BUILD A GIANT BUILDING THAT EVERYBODY'S GOING TO WANT TO LIVE IN. WE'RE GOING TO TRY TO (A CITY IN WHICH THEY'RE GOING TO BE ZONING AND LITTLER THINGS LIKE THAT THAT WILL HELP BREAKPEOPLE TOGETHER AND CREATE A LIVELY ENVIRONMENT IN WHICH PEOPLE ARE GOING TO WANT TO SHARE DATA, WHY WANT TO WORK TOGETHER AND THEY WANT TO BE PART OF THOSE COMMUNITIES. SO THOSE OF YOU HAVE ALREADY DOWNLOADED AND SCANNED THROUGH THE 28 PAGES, CAN YOU JUST KIND OF GO OFF AND DO SOME OTHER THINGS BUT FOR THOSE WHO ARE WAITING FOR THE NIH NETWORK IT ALLOW YOU TO DOWNLOAD THINGS, I WANT TO GIVE YOU A CHARACTERISTICS AND THINK ABOUT IN TERMS OF THE PROBLEMS WITH REGARD TO DID THEA STANDARDS AND THE LIKE. SO THE FIRST CHARACTERISTIC IS DECENTRALIZED CONTROL, FEDERATION AND DATA SHARING, VIA STANDARDS WILL BE REALLY INT. PATIENTS ARE GOING TO BE A COMMON FEATURE ACROSS ALL OF THIS, BUT IN ANY ULTRA LARGE SCALE SYSTEM, THE ABILITY TO HAVE IT ALL IN ONE PLACE TO BE ABLE TO SOLVE WILL BE REALLY, REALLY HARD. SO FOR EXAMPLE, FROM A PRIVACY AND SECURITY PERSPECTIVE, DON'T BUILD A DATABASE ANY LARGER THAN YOU'RE WILLING TO BREECH. SO THAT MEANS THAT WE HAVE TO BE CAREFUL ABOUT HOW BIG OUR DATABASES MIGHT BE AND IT COULD BE THAT IF THERE'S A SINGLE WAY OF DOING THINGS THAT OTHER PEOPLE MAY HAVE DIFFERENT PRIORITIES, DIFFERENT WAYS OF REPRESENT TAG INFORMION AND SO, ULTRA LARGE SCALE SYSTEMS ARE USUALLY UNABLE TO ENTIRELY CENTRALIZE CONTROL AND COMMAND AND CONTROL OF THESE SYSTEMS. THE SECOND IS THAT THERE'S REQUIREMENTS AND FIGURE OUT WHAT WE NEED AND GET THE REQUIREMENTS DONE AND THEN WE'LL BUILD GENS THAT. SO IF YOU HAVE THE REQUIREMENTS THEY WILL CHANGE OVERTIME AND THAT MOST OF THE REQUIREMENTS THAT ARE GOING TO BE IMPOSSIBLE THAT ARE GOING TO BE UNNECESSARY WILL BE UNNOBLE WHEN YOU START. THIS BOOK DOESN'T TELL YOU WHAT THE SOLUTION IS BUT FRAMING AS A UNNOBLE AND DIVERSE SET OF REQUIREMENTS IT MEANS WITH HAVE TO DO ITERATIVE INCREMENTAL DEVELOP AND WANT WE LEARN AS WE GO YOU HAVE TO BUILD IT INTO THE SYSTEM WHEN YOU BEGIN. THIS WHOLE NOTION OF IF YOU HEARD THIS BEFORE, IF WE GET OUR ELECTRONIC HEALTH RECORDS INSTALLED EVERYTHING IS DONE AND WE SHOULD BE FINE, RIGHT? THE NEXT QUESTION I WOULD ASK IS THAT'S GREAT, WHAT'S THE NEXT SYSTEM YOU ARE GOING TO INSTALL? SO GOING THROUGH THE ONEROUS, STAFF, SO WE HAVE TO PICK AND SMALL CHANKS AND INFORMATION THAT WE WORK ON KNOWING WE'RE GOING TO EXPAND IT OVER TIME AS WE LEARN BETTER ABOUT HOW TO DO THIS AND THERE WILL BE RESEARCH CENTERS THAT ARE STRUGGLING HOW TO FIGURE OUT HO GET THE SYSTEM INTEGRATED IN AN ELECTRONIC WAY, SO THE THING IS IT'S BECAUSE THERE ARE GOING TO BE THAT WAY THAT WE HAVE TO BE ABLE TO TOLERATE SYNTAX AND SOPHISTICATION AND SO WE MAY NOT BE BE ABLE TO ALL AGROWOT SEMANTICS SOY WE NEED TO BE TOLERANT OF THAT BECAUSE NOT EVERYBODY'S GOING TO BE AT THE SEAM LEVEL OF SOPHISTICATION. NOW WHAT THAT MEANS WHEN YOU START ON A PARTICULAR PATH, I USE THIS NOTION OF WE NEED TO TAKE THE PATH OF LEAST REGRET. AND WHAT THAT MEANS THAT THINGS WILL ANTICIPATE ON THE FUTURE AND LET'S NOT GO DOWN A PATH THAT LOCKS OFF OTHER POSSIBILITIES OR VENUES SO AGAIN, YOU HAVE TO BUILD IT INTO THE WAY THAT SOFTS THE PROBLEM BECAUSE YOU HAVE TO FRAME THE QUESTION AS AN ALLOT RALARGE SCALE SYSTEM. THERE'S IN NOTION OF NORMAL FAILURES, DO YOU KNOW HOW FAIL SAFE SYSTEMS FAIL? THEY FAIL BY FAILING TO FAIL TO FAIL SAFE. AND WHEN YOU GET A BIG SYSTEM THAT'S REALLY COMPLICATED, IT'S IMPORTANT TO RECOGNIZE THAT YOU'RE NOT GOING TO BE ABLE TO ENTIRELY SECURE EVERYTHING. NOW SECURITY IS IMPORTANT BUT WE HAVE TO THINK ABOUT RECOVER SCHERESTORATION AND ALL THOSE OTHER THINGS AS ASSOCIATED WITH IT, SO THERE ARE PARTS OF THE SYSTEM THAT WILL BREAK AND WE WON'T NECESSARILY ALWAYS PREVENT THAT FROM HAPPENING BUT WE HAVE TO BUILD IN RESILIENCE SO THAT THAT IS--SO THAT WE CAN SORT OF DEAL WITH THAT. IT'S ABOUT ORCHESTERATION RATHER THAN COMMAND AND CONTROL AND THAT GOES BACK TO MY EARLIER POINT THAT THIS IS NOT ARCHITECT AND YOU ARE CITY ENGINEERING AND PLANNING. WE WILL HAVE TO BUILD BUILDINGS, RIGHT? THAT'S WHAT YOU NEED IN THE CITY AND YOU HAVE TO BUILD SOME THINGS BUT YOU HAVE TO DO IT IN THE FRAMEWORK OF SOMETHING THAT PROVIDES FOR ZONING LAWS AND BASIC INSTRAY JECTORY PRODUCTURE THAT ALLOWS ALL OF THE INFORMATION TO FLOW. AND YOU NEED TO THINK ABOUT WHAT ARE THE--THE INCENTIVES THAT WILL MAKE PEOPLE WANT TO BUILD SOMETHING IN YOUR CITY IF YOU WILL AS WELL. >> SO WHEN WE THINK ABOUT IT AS A ULTRA LARGE SCALE SYSTEM PROBLEM, WE ACTUALLY BEGIN TO APPROACH THE PROBLEM IN DIFFERENT WAYS THAN IF WE SEE THIS AS AN ENGINEERING TASK. SO NUMBER TWO, THAT WAS ONE OF THE ULTRA SYSTEMS, THE NUMBER TWO IS THE TWO PARTS OF DEFINITION OF INTEROPERABILITY, SO STANDARDS OF INTEROPERABILITY ARE HARD IN THE CONCRETE. THEY ARE IMPOSSIBLE IN THE ABSTRACT. AND THAT GOES BACK TO THE IEEE DEFINITION AROUND INTEROPERABILITY AND I LIKE THIS DEFINITION, THERE'S A TON OF DEFINITIONS OUT THERE, AND EVERY NATIONAL COORDINATOR HAS TO REDEFINE WHAT INTEROPERABILITY IS, BUT THIS IS THE ONE THAT I LIKE AND YOU PROBABLY HAVE YOUR OWN ONE, I WILL TELL YOU WHY I LIKE THIS ONE. SO INTEROPERABILITY HAS TWO PARTS, THE FIRST IS THE ABILITY OF TWO OR MORE SYSTEMS TO EXCHANGE INFORMATION AND THE SECOND IS THE ABILITY OF THOSE SYSTEMS TO USE THE INFORMATION THAT'S BEEN EXCHANGED. SO IT'S ABOUT EXCHANGE AND USE, SO IF IT'S ABOUT USE, IT MEANS THAT YOU CAN'T DEFINE INTEROPERABILITY ABSENT OF THE THING YOU WANT TO DO. SO ONCE YOU DEFINE WHAT YOU WANT TO DO YOU CAN DETERMINE WHETHER OR NOT YOU'RE INTEROPERABLE OR NOT, WE WANT THESE SYSTEMS TO INTEROPERATE, BECAUSE EVERYBODY'S GOING TO HAVE A DIFFERENT USE FOR WHAT THEY WANT TO USE THE INFORMATION FOR, I LIKE TO USE THE EXAMPLE OF--YOU KNOW IF I'M SEEING A PATIENT IN NIGH CLINIC THE NIGHT BEFORE WAS SEEN IN THE EMERGENCY ROOM AND THEY COME WITH A CRUMPLED YELLOW PIECE OF PAPEROT MEDICATION CARBON COPY OF MEDS THEY WERE PREJIBE AND I CAN'T READ IT BECAUSE IT'S BEEN SITTING ON THEIR DASHBOARD OVERNIGHT, INTEROPERABILITY IN THAT SETTING AND I WANT TO KNOW WHAT JUST HAPPENED LAST NIGHT. SO IF SOMEONE SENDS ME A PDF DOCUMENT THROUGH MY E-MAIL I HAVE INTEROPERABILITY BECAUSE I CAN USE AND EXCHANGE THE INFORMATION I HAVE. I CAN SEE WHAT HAPPENED BUT WHAT I REALLY WANT TO DO IS TAKE THAT INFORMATION AND INTEGRATE IT INTO MY HEALTH RECORD AND MAKE SURE THE NEXT PRESCRIPTION I WRITE WILL NOT INTERFERE WITH THE DRUG THEY GOT LAST EVENING, BECAUSE I CAN'T TAKE THAT PDF AND FOLD IT INTEREST OUR HEALTH RECORD AND I NEED STRUCTURED SIGNATURES AND I NEED ALL THESE THINGS SO DEFINING WHAT YOU WANT TO AKOCHLISH IS A CRITICAL STEP OF BEING ABLE TO DEMONSTRATE WHETHER YOU HAVE INTEROPERABILITY, SO AGAIN THOSE INCREMENTS WILL BE VERY CLEAR, SO I LIKE THIS DEFINITION BECAUSE IT'S MEASURABLE, IT'S TANGIBLE AND IT PROVIDES A FOCUS FOR THE WORK THAT'S DONE IF A WAY THAT ALLOWS US TO APPROACH INTEROPERABILITY BY HAVING INCREASING KINDS OF THINGS WE CAN DO WITH THE DATA GIVEN WHAT IT IS THAT WE'RE--THAT ARE SET IN FRONT OF US. SO THIS IS MEMBER NUMBER THREE. --NUMBER THREE. SO WHEN IT COMES TO THE THING TAC WE WERE DOING IN HEALTH I.T. AT THE OMC AND TRYING TO GET ELECTRONIC HEALTH RECORDS ADO, FRAME SOLUTIONS IN TERMS OF THINGS THAT MATTER TO PEOPLE BECAUSE IN THIS ROOM, WE MAY HAVE A LOT OF THINGS WE WANT TO DO BUT WE HAVE TO THINK ABOUT WHO OUR CONSTITUENCIES, WHO ARE THE PEOPLE IN CONGRESS AND PLACES THAT ARE GOING TO FUND THE WORK WE DO. HOW CAN WE DEMONSTRATE THE VALUE? SO THE THREE THINGS THAT WE FOCUSED ON WAS THE TRIPLE AIM, HEALTH, HEALTHCARE AND COST EFFECTS,ENTIOUS FICIENCYS, THOSE SORTS OF THINGS. SO THE GOAL IS TO IMPROVE HEALTH, IMPROVE THE DELIVERY OF HEALTHCARE AND TO CREATE EFFICIENCIES THAT WILL HELP US CREATE A MORE SUSTAINABLE CROSS MODEL AROUND HEALTHCARE TO FRAME WHAT YOU'RE GOING TO DO IN FRAMES PEOPLE CARE ABOUT, THAT'S A WAY YOU CAN COMMUNICATE THE VALUE TO THE FOLKS THAT ARE INTERESTED IN THE WORK THAT DO YOU. NOW IT COULD BE THE RESEARCH COMMUNITY THAT'S THE TARGET. IT COULD BE THAT IT'S THE PATIENTS THAT ARE THE TARGET, IT COULD IT'S ACADEMIC MEDIAL CENTERS OR OTHERS THAT ARE THE TARGET. BUT FIGURE OUT WHO WE WANT TO TARGET AND CREATE THE VALUE PROPOSITION FOR THEM IN TERMS OF THE THINGS WE WANT TO ACCOMPLISH AND THE WORK WE WANT TO DO. NUMBER FOUR, NUMBER FOUR IS THE FOUR SCALES OF ENGAGEMENT AND THERE ARE ACTUALLY SEVEN BUT WE WANT TO START WITH FOUR AND IT'LL BE LEFT AS I WRITE IN QUESTION FOR THE OTHER THREE LEVELS OF ENGAGEMENT BUT THATTA A WHOLE OTHER QUESTION. THIS IS THE ONLY PART OF MY TALK THAT HAS MATH, SO THOSE WHO ARE REALLY INTO MATH, YOU KNOW, IT'S OKAY, I'M GOING TO CALL ON YOU. THOSE WHO ARE NOT INTERESTED IN MATH, YOU KNOW, REALIZE THERE ARE GOING TO BE SOME THINGS COMING UP FOR YOU AS WELL IN TERMS OF SOCIOLOGICAL TECHNICAL AND WHAT NOT SO WE'LL GET INTO THAT. SO THE FOUR SCALES OF ENGAGEMENT, NOW A LOT OF PEOPLE TALK ABOUT THE LEARNING HEALTHCARE SYSTEM AND I USE THAT AS A FRAMEWORK BUT THERE'S A LOT OF PEOPLE THAT DRAW LOTS OF DIFFERENT DIAGRAMS AND FLOW CHARTS AND THINGS I USE IT TO TRY TO ARTICULATE THE SCALES OF ENGAGEMENT SO WHAT'S 10 TO THE ZERO? ONE, SEE THAT'S THE HARD EVALUATION PROCESS ONE, EVERYBODY SAYS ZERO AND IT'S NOT TRUE, RIGHT. SO 10-THE ZERO IS ONE. THAT'S THE PATIENT. SO WHEN WE THINK ABOUT WHAT WE WANT TO ACCOMPLISH AND SORT OF ESTABLISHING STANDARD FIST WHAT WE'RE GOING TO DO IS SUPPORT RESEARCH AND IT WILL COME FROM DEVICES AND PIT BITS AND OTHER THINGS LIKE THAT WE NEED TO BE COGNIZANT THAT PART OF THIS LEARNING ECOSYSTEM IS GOING TO INVOLVE CONSUMERS AND PATIENTS KIND OF AS SORT OF AN N-ONE. WHAT'S 10 TO THE THIRD? A THOUSAND, THAT'S ABOUT THE SIZE OF MEDICAL PRACTICE. SO IT'S PROBABLY BETWEEN A THOUSAND AND 3000 DEPENDING ON THE NATURE OF YOUR SPECIALTY OR THE LIKE. THIS IS WHERE A LOT OF MEANINGFUL USE DIRECTED ENERGY IS HOW CAN WE ENABLE ELECTRONIC HEALTH RECORDS TO BE EFFECTIVE IN SUPPORTING DOCTORS AND PATIENTS IN THEIR PRACTICE IN TERMS OF WHAT THEY DO. SO WE THEY'D TO MAKE SURE THAT PATIENTS CAN INTERACT WITH THE PATIENTS, WITH THE PRACTICES AND WE HAVE WAYS OF EXTRACTS INFORMATION FROM THAT AS WELL. SO, YOU KNOW FOR THE MEDICINE AND SOME OTHER THINGS THAT ARE GOING ON TRYING TO LINK THE ELECTRONIC HEALTH RECORD WITH OTHER THINGS IN THIS THE RESEARCH ENVIRONMENT IS SUCH AN IMPORTANT THING TO THINK ABOUT, WHAT'S 10 TO THE SIXTH? THE MILLION, THAT'S A POPULATION AND THAT'S ABOUT WHAT YOU WOULD EXPECT FOR SAY PRECISION MEDICINE, WE WANT TO GET A COHORT WITH THAT, IF YOU LOOK AT AN ACO, ACCOUNTABLE CARE ORGANIZATION, IT'S UPWARD OF A MILLION IN THE COVERED LIVES THEY WANT TO HAND AND HE WILL H. I. E. PUB PUBLIC HEALTH ALL DEAL WITH MILLION RANGE SORTS OF DATA POINTS AND THINGS LIKE THAT. WHAT'S 10 TO THE NINTH. YOU GUYS ARE KIND OF CATCHING OTHIS IS A GOOD CROWD. YEAH. SO THAT'S THE PUBLIC. AND I LIKE ILLITERATION, PATIENT, PRACTICE POPULATION, JUST BECAUSE IT HELPS ME REMEMBER IT, BUT THESE ARE THE THINGS WE DO IN RESEARCH THAT WE EXPECT TO BE APPLICABLE TO EVERYBODY WHOSE ON THIS PLANET. ALL THE HUMAN THAT ARE OUT THERE, AND THE OTHER THREE SCALES OF ENGAGEMENT, I WILL GIVE YOU A HEADS UP ON THAT, YOU KNOW I WANTED TO USE NEGATIVE NUMBERS BUT THAT'S NOT QUITE TRUE. IF YOU WENT FROM NEGATIVE 10 TO THE NINTH AND POSITIVE 10 TO THE MILEAGE AND BEAR WITH MEOT MATH OF THAT. 10 TO THE IN THE OTHER DIRECTION, AND WHAT'S 10 TO THE MINUS SIX, THE MILLION, THAT'S ABOUT THE NUMBER OF MOLECULAR COMPOUNDS WE HAVE OUT THERE TO HELP INFLU EXPENSE DIRECT DISEASE, SMALL MOLECULES AND THE LIKE. AND 10 TO THE THIRD IN THE OTHER DIRECTION, IS ABOUT THE NUMBER OF DISEASES WE HAVE, RIGHT? PROBABLY IN THE THOUSANDS, NOT IN THE MILLIONS NECESSARILY. SO THERE'S A WAY CAN YOU INTEGRATE ALL THE WAY FROM THE GENOME, THE SMALL MOLECULE, TO DISEASES, TO PATIENTS TO EHRs TO POPULATION TO PUBLIC AND THERE'S A SORT OF CYCLE THAT NEEDS TO HAPPEN. SO WHEN WE'RE THINK BEING THE KINDS OF STANDARDS WE THEY'D TO CREATE, FIGURE OUT AT WHAT SCALE WE EXPECT TO GET DATA AND MAKE SURE THERE ARE THINGS THAT WILL BE ABLE TO SUPPORT THAT. THERE ARE RELATIONSHIPS AND THERE ARE PROBABLY PEOPLE IN THIS ROOM BUT INFORMATICS, STANDARDS, TESTING, BUSINESS DRIVERS AND GOVERNANCE NEED TO BE ABLE TO SPAN THE THINGS WE HAVE THERE. SO ONE SYSTEM, TWO KINDS--PARTS AND ENGAGEMENT IN FIVE, IS THE FIVE TECHNICAL THINGS TO STANDARDIZE, TECHNICAL THING SYSTEM A TERM OF ART THERE, IS CLEARLY, WE ALL KNOW WHAT TECHNICAL THINGS ARE, RIGHT? VERY SOPHISTICATED BUT WHEN I WANT TO TALK ABOUT THE TECHNICAL STACK, THERE ARE REALLY FIVE THINGS THAT WE THEY'D TO THINK ABOUT. THIS IS WHAT WE DID WITH THE OFFICE OF THE NATIONAL COORDINATOR, PART TELEVISION WAS THAT WE DIDN'T WANT TO BUILD AOL AND WE TOOK OUR INSPIRATION FROM THE TCPIP STACK FROM THE INTERNET AND THERE'S SERIES OF LEVELS WITH EXTRACTIONS BETWEEN THEM. SO IF THINK ABOUT WHAT YOU STANDARDIZE, YOU STANDARDIZE MEANING, STRUCTURE, TRANSPORT, SECURITY AND SERVICES AND THE SERVICES REALLY LIKE AN API OR SOMETHING LIKE THAT TAKES MEANING STRUCTURE, TRANSPORT AND SECURITY AND PACKAGES IT TOGETHER FOR A PARTICULAR PURPOSE. SO THE ENTIRE INTERNET RUNS ON FOUR API. AND THE WORLD WIDE WEP. THE WHOLE WORLD WIDE WEB RUNS ON API, TWO OF WHICH WE USE AND TWO OF WHICH ARE DANGEROUS AND TEND NOT TO BE USED AS MUCH SO THERE'S ONE CAN YOU GO AND GET INFORMATION WHERE YOU CAN POST INFORMATION TO IS. BUT IT BASICALLY AS SPECIFIC WORDS TO USE THAT ARE CONTROLLED WORDS, THAT'S PART OF THE MEANING. IT HAS A PARTICULAR STRUCTURE WITH BRACKETS AND BACK SLASHS AND THINGS THAT HELP THEM GET FROM THE BEGINNING TO THE END, THERE'S AN TRANSPORT STANDARD IN HTTP, THERE'S SECURITY, TLS AS A TRANSPORT MECHANISM AND ALL ALL ND ALL THOSE BUNDLED TOGETHER PROVIDE AND THE FACT IS THAT THEY'RE GOING TO DISAGREEN CELLS ON MEANING, BUT THEY MEET AGREE ON MEANING BUT HAVE SOME QUIBBLES ABOUT WHETHER THE STRUCTURE SHOULD BE A OR B. THE FACT IS IF YOU WANT TO GET AGREEMENT, IT'S OFFENSE TIMES USEFUL TO BREAK UP THE PUZZ AND HE WILL REALIZE THAT I AGREE ON 90% AND 10% I DON'T AND ALLOW IN AN ALLOT RALARGE SCALE SYSTEM, SOME DEGREE OF HETEROGENEITY AND HOPEFULLY, BRING THINGS TOGETHER AS THINGS PROCEED. SO MEANS SEMANTIC INTEROPERABILITY, SO I HAVE FEW OTHER THINGS, THAT DIDN'T FIT INTO THE FIVE AND THE SIX OR SEVEN IT WAS TOO HARD. IT ISN'T A STANDARD BECAUSE WE SAY SO, IT'S A STANDARD BECAUSE PEOPLE USE IT AND THIS IS ONE OF THOSE THINGS, I WAS AT OMC WE COULD MANDATE STANDARDS, RIGHT? REMEMBER DIRECT? ANYBODY REMEMBER DIRECT? YOU DO, YES? WE SAID IT WAS SO. NO ONE IN THIS ROOM KNOWS ABOUT IT SO THE THING IS THIS IT'S A STANDARD AND IT'S PART OF THE INTEROPERABILITY PROBLEM, IN THE USE, IT SOLVES THE USE PROBLEM. SO, ONE OF THE THINGS THAT WE DID, OR THAT I DID WHEN I WAS AND I DIDN'T WANT TO BE THE GUY THAT EVERYBODY YELLS AT BECAUSE THEY DIDN'T LIKE THE STANDARD BECAUSE I DIDN'T PICK THEIRS, YOU KNOW, SOMEBODY SAID, THE THING ABOUT STANDARDS I HAVE ANOTHER ONE. SO STANDARDS LIKE TOOTH BRUSHES, EVERYBODY HAS ONE BUT NOBODY WANTS TO USE YOURS. AND THAT'S PROBABLY A GOOD THING, BUT WHAT I WANT TO DO IS I WANT TO ACTUALLY CREATE A WAY WHERE WE COULD ENGAGE PEOPLE IN HELPING TO ESTABLISH THE STANDARD. SO THIS WHOLE NOTION OF COMMUNITY DRIVEN STANDARD SYSTEM A REALLY GOOD IDEA BECAUSE IF YOU WANT PEOPLE TO USE IT, YOU HAVE TO SOLVE A PROBLEM FOR THEM AND YOU HAVE TO FIGURE OUT HOW THEY CAN HAVE OWNERSHIP OF WHAT THAT STANDARD MIGHT BE. SO WE CREATED THIS STANDARDS AND INTEROPERABILITY AS WELL RELATED TO THE BUDGET THAT I HAD SO ONLY HAVING ABOUT 60 MILLION DOLLARS OVER THE COURSE OF TWO YEARS TO ESTABLISH ALL THE STANDARDS WE NEEDED FOR THIS AND HE HIRED ME AND SAID HE DELAYED OUR PROPOSAL TO OMB, HE HIRED ME OCTOBER 1st AND IT WAS DUE BEFORE THAT. SO HAVE YOU 48 HOURS TO DETERMINE HOW YOU'RE GOING TO SPEND 60 MILLION DOLLARS. I CAN TELL YOU THAT--I CAN'T TELL YOU WHAT YOU HAVE TO BUILD AND I CAN'T TELL YOU WHEN IT'S DUE, KIBE PRETTY SURE YOU'RE GOING TO RUN OUT OF MONEY BEFORE YOU'RE DONE BUT YOU HAVE 48 HOURS THAT YOU CAN COME UP WITH SOMETHING THAT IS OMB, THIS IS MY INTRODUCTION GOVERNMENT SERVICE. SO WHAT WE DID IS WE SAID, WELL WHAT ARE THE THINGS WE HAVE TO DO, WE KNOW WE WILL HAVE TO CREATE THESE STANDARDS, WE WILL TEST THEM AND PILOT THEM, THERE WASN'T ANY ONE SINGLE ORGANIZATION THAT COULD DO ALL OF THAT AND WE SIT UP THE STANDARDS AND FRAMEWORK TO SORT OF LEVERAGE THAT AS WELL. AND SO IT REALLY GOT TOGETHER, YOU KNOW 3000 PEOPLE THAT REGISTER, THERE WAS PROBABLY, YOU KNOW 750 TO A THOUSAND PEOPLE THAT PARTICIPATED IN WEEKLY CALLS. AND IF ANYONE KNOWS ABOUT HL-SEVEN A CONTENTIOUS BALLOT MIGHT HAVE 30 OR 40 OF THOSE. AND SO WE REALLY CREATED A WAY BY ENGAGING THE COMMUNITY AND HAVE IT BE SOMETHING THEY OWNED. >> WE ALREADY KNOW THIS POINT THERE WILL NEVER BE ONE MODEL FOR THEM ALL, LET ME GIVE YOU THE EXAMPLE I LIKE TO USE. SO IF YOU THINK ABOUT INFORMATION MODELS AND THESE ARE INCREDIBLY SIMPLE INFORMATION MODELS WHEN WE THINK ABOUT CLINICAL CARE, WE THINK ABOUT PATIENTS WHO HAVE VISITS AND AT THOSE VISITS THEY HAVE LAB TESTS AND THEY GET RESULTS FROM THOSE LAB TESTS. SO IF YOU WANT TO GO IN AND ASK, TELL ME ALL THE PATIENTS THAT HAD A HEMOGLOBIN A-ONE C THAT WAS GREATER THAN EIGHT AND YOU HAVE A MODEL THAT LOOKS LIKE CLINICAL CARE. YOU HAVE TO LOOK AND GET RECORD VISITS AND HAD A TEST DONE AND YOU LOOK FOR RESULTS. NOW MOST ACADEMIC MEDICAL CENTERS, EXTRACT, TRANSFORM AND LOAD FROM THEIR CLINICAL CARE SYSTEMS INTO A DIFFERENT INFORMATION MODEL THAT SITS IN THEIR CLINICAL WAREHOUSE AND THEY ORGANIZE THINGS BY TEST, RESULTS AND PATIENTS SO THEY CAN SAY, HEMOGLOBIN AUMC, 8.0, HOW MANY PEOPLE HAD A RESULT LIKE THAT? AND IT IS THE QUALITY OF CARE OR THE KIND OF CARE THAT QUESTIONED. SO IN YOU DO STUDY, HAVE YOU STUDIES THAT HAVE ARMS AND THEY HAVE PATIENTS AND THEY HAVE RESULTS. NOW THE THING IS THAT NOT ANY ONE OF THESE MODEL SYSTEM INCORRECT, IN FACT THEY'RE BUILT FOR A PARTICULAR PURPOSE, BUT BUT A PARTICULAR KIND OF INFORMATION, WE NEED TO HAVING YOU WOULD AND IF EFFICIENT USE OF THE MODELS THAT MIGHT BE THERE, SO OTHERWISE WE END UP WITH SOMETHING THAT LOOKS LIKE THE V-THREE RIM IN HLSEVEN WHICH WAS A BIG ABSTRACT MODEL THAT WAS DIFFICULT TO IMPLEMENT, TOO, IT WAS THREE, TWO MINUTES. OKAY, I'M ALMOST DONE SO I ADDED THE SLIDE, TO TALK ABOUT MY VIEW OF THE CYCLE SO WE TALK ABOUT THAT IF YOU TAKE THE VALUE OVER TIME AND YOU HAVE A DOTTED LINE THAT INDICATE WHERE IS YOU GET ADOPTION, SORT OF WHERE IT GETS TO ADOPTION. YOU HAVE A LOT OF STANDARDS THAT LOOK LIKE THIS. THESE ARE WIDELY ADOPTED BUT DECLINE NOTHING VALUE. AND THERE ARE A LOT OF STANDARDS THAT ARE OUT THERE WHERE PEOPLE HAVE WIDELY ADOPTED BUT THEY DON'T NECESSARILY ALWAYS PROVIDE THE VALUE AS WE'RE GOING FORWARD AND I AM GOING TO PICK ON STANDARDS AND I APOLOGIZE BUT HLSEVEN V-TWO IS A WIDELY ADOPTED BUT DECLINING IN VALUE BECAUSE THERE ARE LOTS OF CHALLENGES ACROSS THE EGIES TUITIONS. THERE ARE SOME MODELS THAT ARE CURRENT AND STATE-OF-THE-ART. AND YOU ARE HERE. AND THE PROBLEM IS THAT THIS WILL CONTINUE TO MOVE FORWARD BUT AS THE NEW AND EMERGING STANDARDS BECOME CURRENT AND STATE-OF-THE-ART, THEY WILL BECOME WIDELY ADOPTED BUT DECLINE NOTHING VALUE. SO WE HAVE TO BUILD INTO THE SYSTEM, THAT THAT'S GOING TO BE A NATURAL PROGRESSION. NOW I WOULD SAY THAT MAPPING IS BETWEEN WIDELY ADOPTED AND CURRENT STATE-OF-THE-ART. AND THAT YOU SHOULD TRY TO ONLY FOCUS ON THAT BECAUSE THAT ALLOWS US TO SORT OF STAY ABOVE THAT ADOPTION CURVE IF YOU WILL AS WE GO FORWARD. TAKE THE PATH OF LEAST REGRET. SO IF A FEW MINAL THOUGHTS, MODULAR AND SUBSIDIARY CONSTITUTABLE, I LIKE THE WORD SUBSTITUTABLE BETTER THAN MODULE BECAUSE IT TELLS ME I TAKE SOMETHING OUT AND PUT SOMETHING ELSE IN. IHE MODELS THAT ARE DEVELOPED, THEY ARE GRADE STANDARDS THEY ARE MODULAR BUT NOT SUBSTITUTABLE, YOU CAN'T SWAP IT OUT AND PUT SOMETHING ELSE IN THERE, YOU HAVE TO CREATE A NEW PROTOCOL. AND THOSE GUYS THAT DEVELOPED ALL THE DATA GRAM SPECIFICATIONS FOR TCPIP. AND HIS PRINCIPLE WAS THAT IT WAS NOT ABOUT BACKWARD COMPATIBILITY, IT WAS ABOUT FORWARD PATTIBILITY. BASICALLY SAID WHEN YOU SEND, SEND CONSERVATIVELY, BUT WHEN YOU RECEIVE, RECEIVE LIBERALLY. THAT MEAN FIST YOU SEND BUT IF YOU RESEE, IF YOU GET AN OLD STANDARD OR NEW STANDA ARD THAT YOU WEREN'T AWARE OF, YOU GOT FOUR VALUES AND YOU GOT FIVE, DO YOUR BEST TO FIGURE OUT WHAT'S GOING ON, DON'T REJECT THE PACKET OUT RIGHT AS A REACTUALITY OF THAT. IT PROVIDES A STANDARDS AND IMIM YOU PUT IN AN OPTION, WHEN YOU SEND, THAT'S AN OR, BUT WHEN YOU RECEIVE, IT BECOMES AN AND IT ACTUALLY CHANGES THE EQUATION FOR HOW STANDARDS ARE DEVELOPED BECAUSE IF YOU HAVE TO GET CONSENSUS ON SOMETHING THAT'S GOING TO COST YOU MORE MONEY BECAUSE IT'S AN AND, IT YOU HAVE THE CONVERSATION WHERE AFTERWARD ITS BECOMES HARD. PAY ATTENTION OVER THE COURSE OF THE NEXT SIX-NINE MONTHS THERE WILL BE DISCUSSION ABOUT MU CERTIFIED PRODUCTS THAT ARE NOT INTEROPERABILITY. BECAUSE MANY OF OUR STANDARDS GOT OPTIONS AND THEY WE DIDN'T CERTIFY ON THOSEEM AND INITIATIVES TO WATCH, THERE'S A DATA OMC THAT IS TRYING TO DEFINE GRANULAR STRUCTURES AND SEPARATES STRUCTURE FROM MEANING AND ALLOWS PEOPLE FROM DEFENSES IN THEIR SEMANTICS BUT A COMMON STRUCTURE AND THEN CAN YOU CERTIFY EVERY PRODUCT IF YOU WILL TO THAT COMMON STRUCTURE AND LET THE EXPERT WHO IS ARE OUT THERE DEAL WITH WHAT THE MEANING IS. YOU KNOW THEY TALK ABOUT EXPERTS MEDDLING WITH THINGS, YOU KNOW, DOMAIN EXPERTS WHO MED KNEEL TECHNOLOGY ARE JUST AS BAD AS TECHNOLOGY FOLK WHO IS MEDDLE IN DOMAINS, AND JUST BECAUSE WE HAVE A Ph.D. IN BIOSCIENCE AND ONE OF THE BIOLOGIC SCIENCES DOES NOT MAKE US EXPERTS IN THE WAY IN WHICH THE TECHNOLOGY SHOULD BE DEPLOYED AND USED. SO IT'S IMPORTANT TO RECOGNIZE HOW WE CAN SEPARATE THOSE THINGS OUT AND THE LAST THING I WILL SAY IS FIRE, PAY ATTENTION TO THIS, IT WILL BE THE NEXT HEALTHCARE STANDARD IN SOME FASHION AND THE REASON IT HITS ALL OF THE POINTS, IT'S MODULAR, IT'S SUBSTITUTABLE, IT SOLVES REAL PROBLEMS, IT'S EASY TO IMP LEM AND SIMLE IT'S GOT A LOT OF FEATURES NORKS AT THE VERY TOP OF THE CURVE RIGHT NOW, AND IT PROBABLY CAN GOING TO--RIGHT NOW I THINK THE LATEST IS DPLOABAL WARMING, FHIR WILL HAVE A ROLE IT PLAY BUT OVER THE COURSE OF THE NEXT COUPLE OF YEARS IT'S PROBABLY GOING TO GO THROUGH THAT PIT OF DISPAIR BUT I REALLY BELIEVE IT'S GOING TO KIND OF COME OUT OF THAT VERY QUICKLY BECAUSE IT REALLY IS A GREAT NEXT STEP IF YOU WILL WITH THINGS. SO WITH THEY WILL END AND WE'RE HAVING A PANEL OUT HERE, CAN YOU ASK ME QUESTIONS OUT THERE AS WELL. AND IF THERE ARE A FEW HERE I AM HAPPY TO DO SO. IT'S ALL CRYSTAL CLEAR. I DIDN'T EVEN PRESENT MY THING. >> [INDISCERNIBLE]. >> I'M CHRIS CHUTE, I'M DELIGHTED WITH THIS PARTICULAR PANEL IT SHOULD PROVE TO BE ILLUMINATING. THIS IS INTENDED TO COUNTER POINT THE BASIC SCIENCE DATA STANDARD THIS IS MORNING. WE'RE SHAMELESS CLINICAL PEOPLE, FOR THE MOST PART, JESSE IS HONORS EMPLOYEESIARY--YES, YOU'RE SHAMEFUL--BUT OUR FORMAT WILL BE A LITTLE DIFFERENT IF THAT I REQUESTED THAT THESE SPEAKERS RESTRICT THEMSELVES TO A SINGLE SLIDE TO WHICH THEY WILL SPEAK TO BRIEFLY AND THEN WE WILL HAVE A DIALOGUE AMONG THE PANEL AND HOPEFULLY SOME LIVELY DISCUSSION AFTER WHICH, WE WILL OPEN IT UP TO DIALOGUE WITH THE ROOM, SO THAT'S HOW WE HOPE ON MANAGE THAT. SO WITH THAT, I GET--I WASN'T GOING TO USE A SLIDE BUT I WILL SPEND ONE MINUTE HERE. I THINK THE MESSAGE I WANT TO CONVEY IS, THE HEALTH INFORMATION TECHNOLOGY STANDARDS COMMUNITY HAS BEEN SOMEWHAT PAROCHIAL AND ICE SOMEWHAT ISOLATED FROM THE OTHER STANDARDS COMMUNITY WE HEARD ABOUT THIS MORNING. THERE HAS NOT BEEN A LOT OF INTERACTION BETWEEN AND AMONG THE STANDARDS COMMUNITY THAT WERE PRESENTED TODAY AND THESE CLINICAL COMMUNITIES. THAT'S UNFORTUNATE BECAUSE THESE CLINICAL COMMUNITIES HAVE BEEN BUSY PEOPLE FOR THE PAST 40-ODD YEARS DEVELOPING REALLY VERY ROBUST CONTENT AND VERY INTRIGUING INFORMATION AND I WOULD FURTHER ADD THAT OVER THE PAST DECADE THERE'S BEEN A REMARKABLE CONSOLIDATION OF STABBED ARDS AND 15 AND 2030 YEARS AGO, EVERY STANDARD'S ORGANIZATION WANTED THEIR OWN STANDARDS. IT WASN'T PRETTY AND I THINK THERE'S BEEN MORE COLLABORATION, EINVOLVED IN COLLABORATION THAT'S ACCELERATE INDEED THE PAST DECADE. SO I THINK--THEY'RE FACING THE PROBLEMS IN CONTEXT OF TRANSLATIONAL RESEARCH, THEYVILLE TO COLLABORATE BECAUSE YOU'RE DEALING WITH PATIENT INFORMATION AND BASIC SCIENCE INFORMATION AND IT IS THAT THE ADVANTAGES OF COLLABORATION ACROSS THESE COMMUNITIELL BE SUBSTANTIAL. WITH THAT, CLEM YOU HAVE THE BIGGEST BLOG, SO I THOUGHT IT WAS FIVE, THEY SAID I CAN ONLY RUN A SLIDE SO I PUT A WALL STREET JOURNAL PAGE ON THE ONE SLIDE. BUT I WANT TO EMPHASIZE FIRST THIS WAS VERY IMPORTANT TO ME BECAUSE THERE'S A BUNCH OF THINGS ON THE LEVEL THAT AREN'T OF INTEREST OR CONSIDERED BY THE STANDARDS AND I'M THINKING ABOUT THE ANATOMIC MODELS FOR ZEBRAFISH, I HAVEN'T TAKEN CARE OF ANY OF THEM LATELY BUT IT WAS IMPORTANT, IT WAS EDUCATIONAL AND THE OTHER THING I WANT TO SAY AND TALKING TO PEOPLE HERE THE BIG PROBLEM IS EVERYBODY WANTS THEIR OWN SPECIAL THING AND IT'S WITHIN THE STANDARDS GROUPS AS WELL. IT'S HAPPENS TO HL-SEVEN AND I WAS TALKING WITH THE GUY WHO WAS TALKING ABOUT FLOW CYTOMETRY AND WE GOT A JUST GET SIMPLER IF WE ARE GOING TO GET IT DONE AND GET ADOPTION AND ONE OTHER LESSON WAS INTEL MADE THIS CRUMMY CHIP, THE 286 CHIP, IT WAS A CRUMMY CHIP IT HAD A 24 BIT THING, HAD AIAN CHIP BETWEEN DIFFERENT THINGS AND I THINK SOMEONE ELSE HAD THE PERFECT CHIP. IT WAS A DECK, LIKE DIGITAL EWAIT A MINUTE, IT WAS A PERFECT CHIP AND IT DIDN'T MAKE IT BECAUSE THEY COULDN'T DO THE Z-WHATEVER IT WAS STANDARD. THEY WEREN'T BACKWARD COMPATIBLE, SO WE'VE DONE AN HL-SEVEN, NO ATTENTION, LITTLE ATTENTION TO BACKWARD COMPATIBILITY AND MAYBE YOU CAN -CAN'T DO IT BUT HL-SEVEN AS BEEN AROUND A LONG TIME BUT THE GUYS AT GE ARE BUILDING ONE THAT GOES INTO A WATCH. SO WHO KNOWS SO THAT'S A LONG STORY BUT MY MESSAGE IS TO SAY THERE'S A LOT GOING ON TO REIMPICIZE WHAT CHRIS SAID AND I PROBABLY CAN'T REALLY SAY THISOLE IN 20 SECONDS BUT THE HLSEVEN AS A NUMBER OF VERSIONS THE FHIR IS ATTRACTIVE, VTWO SINKS, V-TWO IS EVERY WHERE, NCPDP, AND IT'S TAKEN A SIMPLELE WORLD, FORMACIES AND IT ALWAYS WORKED THEY HAVE AN EASY LIFE DICOM DOES ALL THE MEDICAL IMAGES STUFF, SMART IS A NEW LOOKING DEVICE TO LOOK IN MEDICAL RECORDS AND LOOK STUFF UP. THERE ARE A VARIETY OF STANDARDS THAT ARE ESSENTIAL FOR POOLING BETWEEN INSTITUTIONAL RESEARCH. AND EVERYBODY'S USING THEM ON CODES INSTEAD OF STANDARDS AND IT DOESN'T WORK UNLESS YOU DO THAT NOIR WILL ANY OTHER STANDARDS SO LOINC IS FOR LABORATORY TESTS AND USED INTERNATIONALLY, 12 DIFFERENT LANGUAGE TRANSLATE INTO OUR RX NORM IS A MEANINGFUL USE STANDARD FOR DRUG NAME, VERY SOPHISTICATED, IT HAS ALL KINDS OF THINGS, INGREDIENTS AND NOW CLASSES AND ON AND ON AND MANDATED FOR USE IN MY WORLD ANYWAY, IT'S PROBLEMS, ORGANISMS, ALL THOSE, THE LITTLE THINGS YOU HAVE TO TALK ABOUT WITHIN MESSAGES, THERE'S THE CDE, YOU CAN HEAR ABOUT LATER AND STANDARDIZE THE VARIABLES YOU WANT TO HAVE USE INDEED STUDIES ACROSS THE SAME KIND OF THREAT. SO LIMITATIONS IN PROBLEMS, I THINK THE BIGGEST PROBLEM IS TWITCHY STANDARDS DEVELOPERS, THERE'S ONE THAT'S WORKING LET'S DO ANOTHER ONE. THERE'S LIKE 35 COMMITTEES AND NEB CAN GO TO ALL OF THEM. THERE'S--IT'S BIGGER THAN GOVERNMENT REGULATION WHEN IS YOU LOOK AT THE REVIEWS YOU HAVE TO DO ON THE HL-SEVEN OUTPUT WITH THE ROUND AND THEN LET'S AVOID THIS SOLUTION WHICH WE'RE REALLY IN RIGHT NOW, RESEARCH VERSES CLINICAL STANDARDS, AND I WAS A UNIVERSITY IN INDIANA, AND THERE WAS NO CONNECTION AT ALL BETWEEN THE RESEARCH MANAGEMENT AND DIDN'T LOOK UP PATIENT DATA THAT WAS ALL BY HAND AND I DON'T KNOW WHY THAT STAYED THAT WAY, IT STAYED THAT WAY FOR A LONG TIME AND THEN SLOW ADOPTION OF SOME CODE SYSTEM. SO MUCH OF WHAT RESEARCHERS WANT WANT WILL BE UNSTATED SO EXPECT FOR EVERYTHING TO COME OUT OF THE MEDICAL RECORD YOU WOULD LIKE, THEY WILL NOT DO THE FANCY INSTRUMENTS OR PARKINSON'S DEC AS THEY'RE DOING FOR A ROUTINE MATTER IF YOU DON'T DO IT ON TOP OF IT. AND THEN WE SHOULDS ARE--I DON'T THINK WE SHOULD STANDARDIZE WHAT'S EN ROUTE, FLUX, SO THERE WON'T BE ENOUGH TIME OR DATA UNTIL IT STUDIES DOWN SO IT'S STANDARDIZE THE NEWEST THING. ESPECIALLY IF YOU'RE IMAGINE WAG DOUG SAID. IF YOU DENT KNOW WHAT IT IS, WAIT, WAIT AN HOUR, WAIT A DAY, WAY A YEAR, IT WON'T KILL YOU ESPECIALLY IF IT'S CHURNING EVERY COUPLE OF DAYS, YOU WON'T HAVE A STANDARD EVERY DAY IT'S A DIFFERENT ONE. SO THAT'S ALL GOI TO SAY, DID I MAKE THE TIME? >> YES. >> YOU DID GREAT. >> OKAY, JESSICA IS THIS YOURS? WHO WOULDA THOUGHT? >> OKAY, AS WE'VE SEENOT TOKEN MOLECULAR PERSON, AND SOME OF THESE, I FEEL RIDICULOUS, DISCUSSING WITH CERTAIN PEOPLE IN THE ROOM, BUT. >> THERE'S SOME PRECISION ABOUT WHETHER IT'S FOCUSING ON MOLECULAR OR PERSONALIZED MEDICINE THAT INVOLVES YOUR FIT BITS AND YOUR WEATHER AND YOUR FOOD DESSERT INFORMATION BUT FOCUSING ON THE MOLECULAR SIDE OF THINGS THERE'S GENOMIC DATA, THERE'S A LOT OF STANDARDS IN IN BASE AS WE DISCUSSED WE I WILL NOT GO INTO DETAILS PARTLY BECAUSE I DON'T THINK YOU CARE AND PARTLY BECAUSE I DON'T KNOW A LOT WHICH IS AN ILLUSTRATION OF WHAT WE TALKED ABOUT THAT THESE ARE HARD, EVEN FOR PEOPLE IN THE FIELD, IT CAN BE COMPLICATED. SO, HLSEVEN CLINICAL DENATIONAL LIBRARY OF MEDICINICS GROUP IS WORKING ON STANDARDS FOR THE DELIVERY OF MOLECULAR CARE, SO WORKING ON THINGS LIKE NEXT GENERATED SEQUENCING OR INDIVIDUAL GENE SEQUENCING IF YOU HAVE THREE BILLION BASE PAIRS, DON'T NECESSARILY WANT TO SEEK THAT IN YOUR ELECTRONIC HEALTH RECORD, YOU DON'T WANT TO DO THEM IN YOUR ELECTRONIC HEALTH RECORD SO WHAT DO YOU WANT TO PUT IN THE ELECTRONIC HEALTH RECORD AND FIGURING OUT WHAT TO USE IN THERE, HCBS TO GIVE PEOPLE A PLAYER FOR THE KIND OFCHALLENGES YOU HAVEOT GENOMICS SIDE IS THAT THE REFERENCE GENOME IS CONSTANTLY CHANGING. SO YOU CAN SAY I'M TALKING ABOUT A VARIATION THAT HAPPENED AT SPOT ONE BILLION ON THE HUMAN GENOME BUT TOMORROW WHEN THEY FIGURE OUT THAT THERE'S ACTUALLY A LITTLE MORE OR A LITTLE LESS SPOT ONE BILLION CHANGED TO ONE BILLION AND THREE, AND COMING UP WITH A NOTATION THAT'S GOING TO LET YOU KNOW EXACTLY WHERE THAT WAS WHEN YOU REFERENCE TERMINOLOGY IS CHANGING, CAN BE TRICKY. STAR ALLELES ARE THIS COMPLICATED KIND OF NOSOLOGY THAT HAS BEEN DEVELOPED ON A FEW DIFFERENT GENES WHERE CAN HAVE A MUTATION IN ONE SECTION AND A MUTATION IN ANOTHER SECTION AND THAT MEANS YOUR STAR-ONE ALLELE AND THEN ANOTHER COMP BINNATION IS THE STAR TWO-LEGAL AND THERE'S NO SEMANTICS TO IT YOU HAVE TO KNOW THE HIRE ARCH SCHEKNOW WHAT IT MEANS, VERY COMPLICATED STUFF. SO THAT'SOT GENOMICS SIDE AND THAT'S MORE ON THE RESEARCH, SOME CLINICAL CHAIR SIDE, THERE'S A WHOLE--OT GENOMIC SIDE, NO, SORRY, LOOKING AT DRUG TERMINOLOGIES, GETTING INTO PROTEOMICS AND METABALOMICS THAT ON THE CLINICAL SIDE THE WILD WETTER WEST AND GARY I WOULD LOVE TO HEAR YOUR FEEDBACK ON THIS IF HAVE YOU OPINIONS BUT I'VE BEEN GETTING INTO THE METABALOMICS SIDE AND THEY'RE USING EXCEL SPREAD SHE'DS WITH COLUMNS AND ROWS THAT ARE ALL DIFFERENT AND EVEN THE PEOPLE INVOLVED IN THE METABALOMICS SOCIETY, STANDARDS INITIATIVE, AREN'T NECESSARILY USING ANY OF THE STANDARDS THAT HAVE BEEN PUT FORTH. DRUG TERMINOLOGIES, AGAIN ALIVIA IS HERE AND SO HOPEFULLY I WON'T SAY ANYTHING STUPID BUT YOU HAVE RXNORM WHICH IS GREAT FOR FORM CYSTS AND PEOPLE WHO KNOW TO KNOW THE SPECIFIC DETAILS OF THE DRUGS TELL TELL YOU DOSAGE AND PILLS AND THE NAME AND INGREDIENTS, IT DIDN'T UP UNTIL RECENTLY TELL YOU THE CATEGORIES, NOW THAT'S BEEN ADDED AS WE FOUND OUT FROM YOUR TALK LAST WEEK WHICH WAS TELEEXTENT. PREVIOUSLY NDFRT WHICH IS THE NATIONAL DRUG FIVE REFERENCE TERMINOLOGY, DID HAVE CATEGORIES YOU NEED TO MAP BETWEEN THOSE WHICH IS ESTABLISH INDEED A USEFUL THING AND PROBABLY THE FUTURE FOR MANY OF THESE MULTIPLE STANDARDS, ATC IS YET ANOTHER DRUG ONTOLOGY THAT GIVES MORE INFORMATION WHICH IS SORT OF A DOUBLE EDGED SWORD THAT IS VERY SPECIFIC FOR EXAMPLE, ASPIRIN WILL HAVE TWO DIFFERENT CODE FIST YOU'RE TAKING IT AS A BLOOD THINNER, IF YOU ARE TAKING AS A PAIN KILLER IT'S A DIFFERENT CODE WHICH IS GREAT IF YOU THAN INFORMATION BUT IF YOU DON'T KNOW THE INFORMATION YOU'RE GUESSING OR PUT NOTHING FALSE INFORMATION AND THEN DRON IS THE CHEMICAL ENTITIES AND THEY'RE TRYING TO SOLVE SOME OF THE--SOME OF THE LOGIC IN THE REASONING YOU CAN DO OVER THE EXISTING ONTOLOGIES AND I DON'T KNOW MUCH MORE OF THAT BEYOND SPACE SO FALLINGS PERSPECTLY ILLUSTRATING THE ISSUES WE SEE AROUND STANDARDS, SO WHAT IT IS IS THE ONTOLOGY OF SORT OF RESOURCE TYPES HAVE YOU IN RESEARCH--OR EXPERT IN THE FIELD RESOURCE WHO KNOWS STANDARDS FOR EXAMPLE IT I WORKED ON IT THROUGH THE SUPPLEMENTAL GRAPT THAT BROUGHT TOGETHER A BUNCH OF DIFFERENT GROUPS WHO HADN'T PREVIOUSLY BEEN WORKING TOGETHER AND IT WAS SORT OF BY THE LUCK OF WHO WERE WAS ASSIGNED TO WHAT IS EXTRA CYCLES THAT GOT THIS WORK DONE ON THE FINANCIAL SIDE, A DIFFERENT GROUP GOING FORWARD AND SO THE MONEY WENT AWAY AND THE STANDARDS THOUGHT BEING MAINTAINED AND THE TECHNICAL SIDE, IT WAS NOT SUPEREXCITING AND I STILL THINK YOU CAN RATIONALIZE IT BY SOME LEVELS, WELL IT'S SIMPLE AND WE TALK TODAY ABOUT THE PEOPLE ACTUALLY USE IT AND IF YOU ADD A LOT OF ABILITIES, SOME OF THE OTHER STANDARDS HAD, IT GETS MORE COMPLICATED, POTENTIALLY BLOTTED IT COULD BE POTENTIAL TO USERS AND THAT COULD BE A PROBLEM BUT THE SAME TIME IT WAS LESS ADVANCED THAN THE OTHER STANDARDS IN THE SPACE. AND THEN THE SOCIAL, IT REALLY CAME DOWN TO SORT OF WHO HAD ALLIANCES ALREADY, WHO KNEW WHO? WHO WAS WORKING WITH WHO? WHO WAS FAMILIAR WITH THE WORK, AND SO JUST THOSE SOCIAL NETWORS SAYING THIS IS THE STANDARD YOU MIGHT WANT TO USE OR THIS IS THE SOLUTION TO YOUR PROBLEM, YOU MIGHT WANT TO DO, I HOPE WE CAN MOVE PAST THAT, SO WE CAN MOVE PAST, AND THAT'S ALL FOR ME. >> SO MY COMMENTS ARE FROM THE PERSPECTIVIVE CLINICAL TRIALS, I'M STANDING IN FOR A BOSS TODAY, STAN HUFF CHIEF MEDICAL OFFICER AT IRPT HOW ABOUT THEAIN HEALTHCARE AND TWO OF THE INIT WHYATIVES HE STARTED AND START WITH THE CLINICAL INFORMATION MODELING INITIATIVE, WHICH IS SEEKING TO BRING TOGETHER ALL PEES PEOPLE AROUND THE WORLD THAT ARE DOING THIS KIND OF DETAILED CLINICAL MODELING AND COME UP WITH A COMMON SET OF THESE KINDS OF MODELS AND THE OTHER THING THAT HE SPEAR HEADED THAT'S HEALTHCARE SERVICES PLATFORM CONSORTIUM WHICH IS SEEKING TO STANDARDIZE THE LOWER LEVELS, THE HSPC, IS SETTING LEVELS BASED ON FIRE, AND AGAIN THE APPLICATION TO WHICH DEVELOPER CANS WRITE WITHOUT REGARD TO THE WHAT THE UNDERLYING PLATFORM IS, WHETHER IT'S ETHIC OR CERNER OR SOME OTHER VENDOR, SOME VA SYSTEM, TRYING TO STANDARDIZE THE WAY THAT APPLICATIONS AND BACK INTEROPERATE. SO MY THREE BULLET POINTS THERE ARE JUST SOME HURDLES THAT THESE EFFORTS FACE AS THEY TRUE TO STANDARDIZE THESE LEVELS. AND THEY'RE ALL THINGS THAT TALK TO ME THAT IMPRESS ME THAT I KNOW NEXT TO NOTHING ABOUT FLOW CYTOMETRY AND PROTEOMICS. BUT I'M IMPRESSED WE'RE COMING AND WE'RE HAVING THE SAME CHALLENGE AS WE TRY TO STANDARDIZE THESE THINGS FELT FIRST IS COMING TOTH, ALL THESE PEOPLE THAT ARE DOING THE DETAILED CLINICAL MODELS HAVE LARGE INVESTMENTS IN WHAT THEY'VE BEEN DOING, WHETHER IT'S POLITICAL OR FINANCIAL OR EMOTIONAL. AND IN TRYING TO GET PEOPLE TO TESTIMONY TOGETHER AND FORM A COMMON, WORK TOGETHER AND FORM A COMMON SET OF MODEL SYSTEM DIFFICULT, AND DIFFERENT TIMELINES, DIFFERENT INCENTIVES DIFFERENT USE CASES AND AND DIFFERENT BAND WIDTHS, AND THE NEXT ISSUE IS IP, WHO OWNS THE STANDARDS WOOY COME UP WITH, IF THESE PEOPLE HAVE PROPRIETARY IP IN THEIR STANDARDS AND THE THINGS THEY'RE BRINGING TO THE TABLE WHAT ARE THEY WILLING TO SHARE AND THEN WHAT ARE THE LICENSING ISSUES OF THE IP ISSUES FOR THE RESULTS AND THEN THE THIRD THING IS MAKING IT ALL WORK AND THE FIRST TWO ARE REALLY NONTECHNICAL ISSUES THAT THE THIRD ONE MAPPING THESE THINGS TOGETHER IS THE TECHNICAL ISSUE BECAUSE ALL THESE PEOPLE THAT ARE COMING TO THE TABLE HAVE CHUNKED UP THE WORLD DIFFERENTLY. THEY'RE ALL TALKING ABOUT MEDICATION ADMINISTRATION BUT SOME PEOPLE SAY, WELL, IF YOU HAVE AN IV AND YOU CHANGE THE RATE, WELL THAT'S TWO DIFFERENT ADMINISTRATIONS WHERE OTHER PEOPLE SAY, NO THAT'S ONE ADMINISTRATION, BUT TWO DIFFERENT TRIP RATE OPEN SOURCE SOFTWAREY PEOPLE SEE THE WORLD DIFFERENTLY AND THAT MAKE ITS HARD WHEN YOU COME TO THE TABLE AND TRY TO CREATE A COMMON SET OF MODELS. THEY DO PRECOOED NATION VERSES POST COORDINATION DIFFERENTLY, THEY PUT THINGS IN THE TERMINOLOGY MODAND HE WILL SOME THINGS IN THE INFORMATION MODEL, DRAWING THAT LINE BETWEEN THOSE TWO MODELS IN A DIFFERENT PLACE AND THAT JUST MAKES--MAKES MAPPING A CHALLENGE. AND I USED TO THINK HARMONIZATION WAS A GOOD WORD. BUT NOW, WHAT HE WOULD MUCH RATHER SEE IS PEOPLE COMING TOGETHER AND CREATING A COMMON MEL IN FROM THE BEGINNING INSTEAD OF GETTING TOGETHER WITH THEIR 10 DIFFERENT MODELS AND FIGURING OUT HOW THEY'RE ALL TALKING ABOUT THE SAME THING OR NOT. SO THAT'S ALL I HAVE. >> AND FINALLY WE HAVE KERRY. SO IF YOU LOOK FOR MEOT AGENDA MY NAME IS NOT LISTED ON THERE AND THIS IS PROBABLY ANOTHER PIECE OF EVIDENCE TO SUPPORT CHRIS'ORIGEINAL STATEMENT THAT THERE WERE MANY DIFFERENT COMMUNITIES WORKING AROUND STANDARDS THAT DON'T TALK TO EACH OTHER IN THE SAME WAYS OR DON'T TALK TO EACH OTHER AT ALL. THE COMMUNITY I WILL DOSES A BIT ABOUT IS WOP THAT'S FOCUSING ON CLINICAL RESEARCH STANDARDS, SO WITH SOME CONNECTION TO THE CLINICAL DATA STANDARDS AND SOME BACK TO SCIENTIFIC DATA STANDARDS BUT REALLY EFFORTS WITHIN THE CLINICAL RESEARCH COMMUNITY TO COME UP WITH COMMON APPROACHES OR DALA ELEMENTS AS WE'RE CALLING THEM TO COLLECT CERTAIN KINDS OF INFORMATION AND STUDIES AROUND A CERTAIN DISEASE, SO WHAT YOU'LL SEE ON THE LEFT-HAND SIDE OF THE SLIDE IS THERE ARE A NUMBER OF INITIATIVES ACROSS NIH ALREADY AND NUMBERS THAT ARE OUTSIDE OF NIH, THAT ARE ATTEMPTING TO IDENTIFY SETS OF COMMON DATA ELEMENTS OR COMMON MEASURES THAT SHOULD BE COLLECTED AND STUDIES IN OF A PARTICULAR KIND. >> WE'RE IN THE SAME SITUATION OF GETTING THE GROUPS IS WITH THE ROOM AND HOW TO MAP THESE TOGETHER BUT I WOULD ARGUE THAT MOST OF THESE FROM THE PHOENIX INIT AND THE NINDS AND PROMISE AND OTHERS ARE EXAMPLES OF COMMUNITY BASED DATA OF STANDARDS DEVELOPMENT BECAUSE IN EACH OF THESE CASES, GROUTS AND THE CERTAIN DISEASE HAVE IDENTIFY IN THE CASE OF PHOENIX WHAT ARE THE 15 RECOMMENDED MEASUREMENTS WE THINK SHOULD BE TAKEN IN IN GWAS STUDIES BUT NOW EXPANDING BEYOND GWAS STUDIES AND OTHER TYPES OF CLINICAL STUDIES WHETHER IT'S IN DIABETES OR CANCER OR NEUROLOGICAL DISEASE, THE NINDS MODEL IS IN A PARTICULAR DISEASE AREA, PARKINSON'S ALZHEIMER PATIENT HYMERS AND MS, AND NOT THE RECOMMENDED OFTEN RECOMMENDED DATA ELEMENTS ACROSS ALL STUDIES OF THIS DISEASE, THE MAIN PURPOSES OF WHICH ARE TO ENABLE SOME COMPARISON OF THE RESULTS OF STUDIES AFTERWARD, BECAUSE BOTH THE SAMENTITY WAS MEASURED WITHIN THE STUDIES MULTIPLE STUDIES AND IT WAS MEASURED IN THE SAME WAY SO IT SHOULD BRING COMPARABILITY. IN SOME CASES IT MIGHT BE COLLECT INDEED SUCH THE SAM WAY AND ACROSS SO MANY POPULATIONS, AND AGGREGATING DATA THERE ARE MOTIVATIONS AND DOING THIS WORK AND WE WILL SEE THIS HAPPEN NOW APPROXIMATE AND A GROWING NUMBER OF INSTITUTES AND CENTERS ACROSS NIH AND MY ROLE IN WORKING WITH THEM IS COORDINATING A WORKING GROUP, SEVERAL OF WHICH ARE SITTING HERE IN THE ROOM TODAY. THE WORKING GROUP TO THE TOP OF THE CDE PORTAL WE LAUNCHED TWO YEARS AGO WHICH WAS A ONE STOP SHOP FOR WHERE YOU CAN FIND INFORMATION ABOUT THESE VARIOUS INITIATIVES, WHERE ARE THEY, WHAT DISEASE IS COVERED IF YOU'RE INTERESTED IN ACCESSING THE DATA ELEMENTS IDENTIFIED YOU LINK BACK OUT TO THEIR WEB SITES TO GET ACCESS TO THEM. MOST OF THEM HAVE DEVELOPED SOME SORTS OF TOOLS OR PULL THE DATA ELEMENTS THEMSELVES OUT. SO WHAT WE'VE DONE IS REALIZE PARTLY BECAUSE OF THESE QUESTIONS ABOUT LACK OF HARMONIZATION AND RECOGNITION THAT'S THE SAME, SOMETHING VERY SIMILAR HAD BEEN COLLECTED IN TWO DIFFERENT WAYS AND TWO DIFFERENT INITIATIVES, WE'VE ACTUALLY CREATED NOW IN LAUNCHED IT'S AVAILABLE AT THE URL, THERE, A CDE REPOSITORY AND THAT'S TAKING THE ELEMENTS FROM THESE INIT QUAATIVE AND PUTTING THEM ON A COMMON PLATFORM WHICH HE HOPE TO BE ABLE TO USE TO ENABLE EASIER ACTESS TO THE COMMON DATA ELEMENTS WHERE CAN I GO TO FIND THE PLACE TO GET THE CDEs AROUND PARKINSON'S DISEASE IF I'M A FUNDED INVESTIGATOR. WHERE CAN WE GO TO COLLECT THESE ELEMENTS AND WHERE THIS' AN IMPERATIVE WHERE THE SYSTEM IS GUIDING YOU TO REUSE EXISTING COMMON DATA ELEMENTS AND TAKE AS MUCH FROM THE EXISTING STANDARDS AS POSSIBLE WITHOUT CREATING SOMETHING HOLY NEW THAT'S PROBABLY GOING TO BE A SLIGHT VARIANT OF AN EXISTING COMMON DATA ELEMENT, WHERE WE HAVE TOOLS TO HELP YOU CREATE COMMON DATA ELEMENTS. WE - THIS AS A TOOL THAT WILL ENABLE YOU TO START HARMONIZING SO WHERE THE NINES PROGRAM AND THE PHOENIX PROGRAM HAVE COME UP WITH SLIDELY DIFFERENT APPROACHES TO COLLECTING THE SAME KIND OF INFORMATION WHETHER AROUND AN OUTCOME OR WHETHER AROUND A PHENOTYPE, A MEDICATION LIST OR DEMOGRAPHIC INFORMATION CAN WE USE THIS TO HIGHLIGHT THE SIMILARITIES AND DIFFERENCES AND TRY TO WORK WITH THOSE COMMUNITIES TOWARD A MORE CONSISTENT WAY TO COLLECT THE DATA. AND WE'RE LOOKING AT THIS AS AN OPPORTUNITY TO THEN LINK TO WHAT THE SEVERAL OTHER SPEAKERS ON THE PANEL HAVE BEEN DISCUSSING HOW DO WE BUILD IN THE CLINICAL DATA STANDARDS THE EHR STANDARDS AND OTHERS BECAUSE AGAIN IN TERMS OF DIFFERENT COMMUNITIES WITH DIFFERENT PERSPECTIVES, A LOT OF THE CDE WORK HAS BEEN DONE BY RESEARCHERS, RESEARCH CLINICAL EXPERTS IN THOSE FIELDS, MAYBE WITH OR WITHOUT A LOT OF BACKGROUND IN UNDERSTANDING OF CLINICAL DATA STANDARDS AND WITHOUT MOTIVATION TO TAKE DATA FROM AN EHR, WE BELIEVE IN EMBED NOTHING THEM SOME OF THE EHR STANDARDS THAT CLEM HAVE MENTION EXPDZ GREATER OPERATING GLOBALLYERABILITY WITH ELECTRONIC HEALTH RECORDS AND LINKING THEM TO THE EHR STANDARDS SO THIS IS AN ONGOING EFFORT THAT SAY WE HAVE ABOUT 22 INCHES OF THE CENTERS ACROSS THE NIH AND WE'RE BEGINNING TO DEVELOP SOME GOOD LESSONS AND PRACTICES ABOUT HOW TO DO THESE KINDS OF THIS INFORMATION IN THE FUTURE. >> THANK YOU VERY MUCH I THINK YOU CAN WE DIDN'TLY DID NOT GIVE YOU TUTORIALS ON A WHOLE FLO TILLA OF CLINICAL DATA STANDARDS, WE COULD HAVE AND WE WOULD HAVE MORE THAN TAKEN UP THE AVAILABLE TIME. RATHER I THINK THE POINT OF THESE VERY RAPID OVERVIEWS WHICH IN FAIRNESS DON'T GIVE YOU AN APPRECIATION OF THE DEPTH OF THESE ACTIVITIES NEVERTHELESS OF THESE I THEY ARE RICH AND THE QUESTION I WANT TO ADDRESS WITH THE PANEL AND THEN WE'LL ADDRESS WITH ALL OF YOU AND THE PURPOSE OF THIS WORKSHOP IS TO EXPLORE MOW CAN NIH IDENTIFY METADATA ENABLING A GOODBYIZABLE BODY OF RESEARCH AND A TRUE BIG SCIENCE PARADIGM AND A BIG DATA PARADIGM. CLINICAL INFORMATION WILL BE PART OF THAT INEVITABLY IN THE CONTEXT OF TRANSLAGSAL RESEARCH. SO FOLKS SO FOLKS DO YOU SEE GIVEN CLINICAL INFORMATION OR RATHER THESE RESEARCH USE CASES. INVOKE DIFFERENT VIEWS AND EMPHASIS ON DATA THAN A LOT OF THE CLINICAL USE CASES THAT GENERATE CLINICAL DATA STANDARDS. WHAT WHAT DID YOU SEE IS THE OPPORTUNITY TO REUSE, MODIFY OR INTRODUCE A PROD BRADDER PERSPECTIVE OF USE IN THE CLINICAL DATA STANDARDS PROCESS AND DO YOU THINK IT'S POSSIBLE FOR THE CLINICAL DATA STANDARDS COMMUNITY TO REALLY EMBRACE THE SECONDARY USE AND UNANTICIPATED USES THAT IS THE NATURE OF RESEARCH. EASY QUESTION. >> I'M NOT SURE HOW MUCH I'M ANSWERING YOUR QUESTION OR ASKING ANOTHER BUT I SAW MYSELF WONDERING, ESPECIALLY WITH THE LAST SPEAKER, CARROTS VERSES STICKS THAT I'M AWARE OF THE CDE, AND WOULD BE A LEARNING CURVE FOR ME TO FIGURE IT OUT AND IF I WAS DOING SMALL TODAY OR LARGE STUDY, WHERE'S MY MOTIVATION TO GO FIGURE THAT OUT VERSES JUST PUTTING WHATEVER'S EASIEST FOR ME ON THAT CASE REPORT FORM AND USE TAG AND HOW DO WE MOTIVATE PEOPLE? >> THAT'S AN EASIER QUESTION, THEY ARE CARROT STICKS AND WHAT I WILL ENABLERS, THE ENABLERS ARE THINGS LIKE REPOSITOR THAT'S SHOULD MAKE IT EASIER FOR YOU TO FIND THESE ELEMENTS AND ONE OF THE PATIENTS, SOME OF THE INSTITUTES IN DOING THIS WAS TO ACCELERATE THE PACE OF GETTING NEW CLINICAL RESEARCH STUDIES STARTED SO RATHER THAN HAVING TO SEARCH FOR, AND EVALUATE AND DEVELOP, PERHAPS THE CASE REPORT FORMS OR SURVEY INSTRUMENTS COULD YOU FIND EASILY ONES THAT HAD BEEN AND I DON'T THINK I SAID THIS IN MY TWO MINUTES AND THESE HAD BEEN VETTED AND VALIDATED INSTRUMENTS SO THE'S A LINK BACK TO IMPROVING QUALITY, SO THERE'S THAT INCENTIVE TO TRY TO USE THEM TO DO THINGS FASTER AND MORE EFFICIENTLY. ONE OF THE STICKS IS THAT IN FACT SOMETHING WE JUST ADD TO THE PORTAL WAS A LIST OF FUNDING OPPORTUNITY ANNOUNCEMENTS THAT ENCOURAGE STRONGLY ENCOURAGE, EXPECT, OR EVEN REQUIRE THE USE OF COMMON DATA ELEMENTS VERY BROADLY. FIND SOMETHING IN THE PORTAL IN ESSENCE OR VERY SPECIFICALLY RELATED TO ONE OF THE INITIATIVES, SO THAT'S A WAY THAT INSTITUTES ARE TRYING TO PROVIDE A STICK TO ENCOURAGE THE USE OF THE CDEs. I THINK THE OTHERS ARE KIND OF LONGER TERM, THIS IS A COMMENT I THINK CHALLENGE WIDE DATA STANDARDS, RIGHT? THE INCENTIVE OR THE CARROT FOR SOMEBODY TO USE THEM IS THERE WHEN THEY'RE MORE PEOPLE WHO ARE MAKING USE OF THE STANDARD WHEN IT HAS WIDER PENETRATION, YOU MAY BE ABLE TO COMPARE DATA OR DO MET ANALYSIS AND I THINK THAT'S HARDER TO SEE IN THE BEGINNING FOR WIDE USE AND IT'S HARDER IN BROADER USE. >> BUT ALSO THAT'S A CARROT FOR THE PEOPLE WHO DO THE SECONDARY ANALYSIS, NOT FOR THE PERSON WHO INITIALLY DID IT. >> CORRECT WHICH IS WHY THE STICK MIGHT BE A GOOD WAY TO GET IT STARTED AND LET THE CARROT STICK TAKE OVER. >> LET ME THINK MORE ABOUT THAT. THERE ARE REALLY TWO FACES PEER, THIS PURE BIOLOGY WHICH A FREE FOR ALL, I MEAN THERE'S JUST--WHATEVER IT IS, IT IS THERE'S DIFFERENT SPACES AND THEN QUITE DIFFERENT FROM EITHER WHAT YOU SEE IN CLINICAL CARE OR EACH OTHER SO I WOULD CHALLENGE, YOU KNOW THE INSIDE OF A FLOW SIGNIFY TOM TERAND THE INSIDE OF A SEQUENCING MACHINE WILL HAVE QUITE DIFFERENT OUTPUTS AT THE LOWER LEVEL WHICH OFTEN GREAT ANTICIPATION OF RESEARCHERS BUT MOST OF THE CLINICAL DATA AND MAY BE YELLED AT ABOUT THIS RLLY BOILS DOWN TO A QUESTION AND AN ANSWER, YOU KNOW WHETHER IT'S A SURVEY INSTRUMENT OR A HEMOGLOBIN OR SOME MEASURE AND THE CHALLENGE THAT IS TO RECOGNIZE THAT--THAT CONCEPTUALIZATION AND SOME OTHER PIECES SO IT WOULDN'T WORK FOR A PRESCRIPTION. IT'S A DIFFERENT BASIC MODEL BUT AN AWFUL LOT OF STUFF FITS IN THIS MAIN VALUE PAIR OR THE OBSERVATION MODEL AND IF YOU LOOK AT NINDS WHICH HAS 11,000 OF THESE VARIABLES, YOU KNOW MOST OF THEM ARE SURVEY INSTRUMENTS FOR SPECIALIZED DISORDINANCE NUMBERERS WHICH IS INN VALIDATED BUT THEY'RE ALL THIS FORM, YOU KNOW THERE MAY BE A FEW THAT GET WICKEDY WHEN YOU FIGURE OUT HOW MUCH DRUG THIS PATIENT'S TAKING BECAUSE THAT MIGHT BE A DIFFERENT STRUCTURE BUT THEN THAT CONTEXT, YOU KNOW TO TRY TO JUSTIFY WHICH ONES TO USE SEEMS TO ME THAT IT WOULD BE EASIER TO GO TO THE FARM AND THEN FIND ONE. EVERYBODY GOES TO THE WEB TO FIND WHAT THEY WANT TO FIND F. WE DO IT RIGHT, IT SHOULDN'T BE HORRIBLE BUT FOR THAT STUDY, IT WILL NOT BE FOR NEW INSTRUMENTATION-- >> OKAY, WAIT JUST A MINUTE. YOU GUYS ARE HAVING A CONVERSATION-- >> FINE, I KNOW I KNOW JESS; I DON'T WORK FOR THE GOVERNMENT ANYMORE, SO I CAN SAY STUFF. >> I--YOU KNOW I'VE WRITTEN GRANTS AND THINGS LIKE THAT AND THERE A DATA SHARING PLAN YOU COULD PUT IN. I DID THAT WHEN I WAS ADDRESSING THE ENVELOPE AND SEND IT IN BECAUSE IT WAS A CHECK THE BOX KIND OF THING. YOU MAKE THE DAT SHARING PLAN A SCORABLE ELEMENT AND YOU MAKE IT PART OF PUTTING IN A GRANT. THAT IF YOU HAVE A ROBUST DATA SHARING PLAN THEN YOU SCORE HIGHER THAN SOMETHING ELSE. IF YOU WANT TO INCENTIVIZE PEOPLE TO COME UP WITH, AND IT DOESN'T MANDATE A PARTICULAR APPROACH BUT IT MAKES IT, THE REQUIREMENT OF THE PERSON ASKING FOR THE FUNDING TO HAVE A ROBUST WAY OF DESCRIBING THE DATA SHARING PLAN. THE SECOND THING IS THAT IT'S ALWAYS IMPORTANT WHEN WE TALK ABOUT COMMON DATA ELEMENTS TO SEPARATE OUT SYNTAX FROM SEMANTICSS, HAVE YOU LOOKEDDA THE LIST THERE, I SUSPECT ALL OF THEM HAD--THAT YOU HAD BOTH PEOPLE TAKEN--THEY HAD COMMON SYNTAX AND DIFFERENT SEMANTICS AND PEOPLE WHO HAD COMMON SEMANTICS AND DIFFERENT SYNTAX ASSOCIATE WIDE THAT. SO THEY HAD THE SAME THEN WITH THE SAME DEFINITION BUT THEY REPRESENTED IT DIFFERENTLY OR THEY HAD A COMMON REPRESENTATION BUT THEY HAD DIFFERENT DEFINITIONS APPLIED TO THAT. SO THE THING IS, YOU GOTTA SEPARATE OUT THE SYNTAX FROM THE SEMANTICS BECAUSE YOU HAVE TO HAVE DIFFERENT EXPERS TO COME TO THE TABLE TO DEVELOP THE TECHNOLOGY PART WHICH IS THEIN TAX AND THE DOMAIN SPECIFIC STUFF THAT IS THE SEMANTICS SO IF YOU CAN AGREEOT COMMON WAY OF DESCRIBING THINGS, THEN YOU CAN SUPPORT ALMOST A HIERARCHICAL VALUE OF INFORMATION. IT MAY BE THAT NHI WIDE, THERE ARE A SMALL SUBSET OF COMMON DATA ELEMENTS THAT WE ALL WANT TO COLLECT AND THESE MIGHT BE THINGS LIKE DEMOGRAPHICS AND BASIC WAYS OF DESCRIBING PEOPLE. IT COULD BE THAT NATIONAL CANCER INSTITUTE, COULD BE DESCRIBING THINGS IN ONE PARTICULAR WAY AND NHLBI, IF THEY'RE IN HERE--AND I BET YOU DO. AND IT MAY BE APPROPRIATE FOR THE NCI TO BE TALK ABOUT CANCKENER MORE DETAIL AND THE NHLBI TO TALK ABOUT LUNG DISEASE AND PULL MONITORROLOGY AND IN A BIT MORE DETAIL, THEY'RE GOING TO HAVE TO FIGURE OUT HOW TO AGREE ON LUNG CANCER FOR EXAMPLE AND IT MAY BE THAT THE INVEOR HAS NEW AND NOVEL WAY OF COLLECTING INFORMATION THAT DOESN'T FIT INTO ANY OF THOSE BUCKETS SO CAN YOU IMAGINE THIS NOTION OF IF THERE IS A-TELEVISION IF THERE'S A STANDARD THAT THE NIH HAS FOR THINGS THAT ARE INSTITUTE WIDE, LET'S USE IT. IF YOU'RE APPLYING TO THE NCI THEN USE THEIR EXTRA SET OF DATA ELEMENTS, BUT A ALLOW THE FLEXIBILITY THAT AS AN INVESTIGATOR YOU ARE COLLECTING NEW AND NOVEL THINGS AND THEN IF YOU ADD TO THE RESOURCES THAT HAVE YOU, SOME GOVERNANCE STRUCTURE THAT SAYS, WE'RE DOING A LOT OF STUDIES IN IN WAY, AND PEOPLE ARE DOING IT ALL IN DIFFERENT WAYS, LET'S GET THOSE RESEARCHERS TOGETHER AND MAYBE THAT BECOMES AN INSTITUTE WIDE STANDARD, AND THEN IF LOTS OF INSTITUTES ARE DOING THE SAME THING THEN CAN YOU POP IT UP TO A HIGHER LEVEL BUT IT'S ITERATIVE AND INCREMENTAL OVER TIME. >> LET ME DIRECT A LITTLE BIT AND I WANT TO BRING TOM INTO THIS. AND THE REASON I WANT TO BRING TOM IN IS THE SEMI, WHICH IS WHAT HE BRIEFLY TALKED ABOUT, THE CLINICAL INFORMAON MODELING INITIATIVE. AN A VERY LOGICAL ABSTRACT WAY SO THE QUESTION AND THEN, OTHER PRACTICAL CLINICAL STANDARDS CAN, MERGE FROM TFOR EXAMPLE, IT'S FIRE WHICH WOULD BE A PHYSICAL IMPLEMENTATION OF THE ABSTRACT MODEL IN SIMI, SO TOM IN THE CONTEXT OF TRYING TO REALLY UNIFY THESE DATA ELEMENTS IN SYNTAX AND SEMANTICS AND IN USE CASES DO YOU SOMETHING LIKE SIMI TO, EXTRACT AWAY WHAT'S SIMILAR AND ALLOW PEOPLE DOWN STREAM TO APPLY IT IN WAYS THAT MAKE SENSE? >> I DEFINITELY DO BUT IT'S, YOU KNOW THE DEVIL'S IN THE DETAILS BECAUSE AS WE TALK ABOUT EVERYBODY HAS THESE USE CASES THAT ARE A LITTLE BIT DIFFERENT AND SO WHAT IS SIMINIES AND WHAT WE'RE TRYING TO DO BUT THE PROOF'S IN THE PUDDING BUT HOW MANY OF THOSE ANALOGIES CAN I USE THAT--ANYWAY, IT'S KIND OF LIKE I THINK WHAT YOU WERE SAYING DOUG EARLIER THAT YOU HAVE TO HAVE SOME KIND OF ECOSYSTEM AND THEY'RE--THEY'RE--I THINK THERE ARE THESE DIFFERENT LEVELS OF ECOSYSTEM, EVEN. WE WITHIN SEMINEED AN ECOSYSTEM WITHIN WHICH PEOPLE CAN SPECIALIZE, EXTEND, CONSTRAIN ACCORDING TO THEIR USE CASES, WE NEED SOME KIND OF INFRASTRUCTURE WITHIN SEMI TO ALLOW PEOPLE TO DO THAT BECAUSE IT'S--YOU KNOW THE SAME THING'S GOING ON IN FIRE WHICH I'M GLAD YOU POINT THAD OUT CHRIS BECAUSE WHAT WE'RE TRYING TO DO IN SIMI IS CREATE LOGICAL MODELS AND YOU DO NEED A TECHNICAL INFRASTRUCTURE WHICH YOU CAN CREATE LOGICAL MODELS BUT THEY ARE LOGICAL MODELS AND WE EXPECT THEM TO BE IMPLEMENTED IN FIRE OR JAVA OR WHATEVER THE TECHNOLOGY OF THE DAY IS, BUT WE EXPECT THOSE FIRE THINGS TO BE IMPLEMENTATION AND NOW WITHIN FIRE, THEY'RE GOING THROUGH THE SAME THING BECAUSE THEY HAVE AN INFRASTRUCTURE FOR WHICH YOU CAN SPECIALIZE AND EXTEBD AND RIGHT NOW THERE'S HAND WAVING GOING ON BECAUSE THEY'RE SAYING, LYOU KNOW YOU YOU'LL JUST HANDLE THAT BY EXTENSION OR YOUTHFUL HANDLE THAT BY CONSTRAINT AND IF YOU THINK ABOUT IT, YOU WILL END UP WITH A WHOLE BUNCH OF SIBLING CONSTRAINTS AND SPECIALIZATIONS AND HOW WILL YOU MAKE THOSE WORK IF I WANT THE CLINICAL TRIAL, WOMENS AND NEWBORNS, COMMUNITY HOSPITAL CONSTRAINT OF THIS MODAND HE WILL SOMEBODY ELSE WANTS TO THE CLINICAL TRIALS WOMEN'S AND NEWBORNS LARGE ACADEMIC CENTER, CONSTRAINT MODEL, YOU HAVE ALL OF THESE PERMUTATIONS AND COMBINATIONS OF THESE CONSTRAINTS THAT ARE GOING ON AND SO THAT'S--THAT'S I GUESS IS IT A ROUND ABOUT WAY OF ANSWERING YOUR QUESTION, BUT I THINK CERTAINLY SIMI CAN BE KEY IN THIS EFFORT. BUT HAVE TO FIGURE OUT HOW WE HAVE THE USE CASE SPECIALIZES CORRECTLY TO MAKE THEM ALL INTO SOMETHING MANAGEABLE. >> I WOULD LIKE TO RESPOND, I THINK THAT DOUG IS RIGHT AND MAYBE HE'S NOT ALL THE WAY RIGHT. SO I LOOKED AT 11--NOT ALL 11,000, BUT MOST OF THE NINDS, ALL OF THE I-GENE, ALL THE CODES THEY'RE NOT ALL OVER THE PLACE. THIS THEY SAY THIS IS WHAT IT IS AND NCI HAS TAKEN THEM IN A DIFFERENT MANNER AND CREATE THEM DE NOVO, SO THEY DID HAVE THAT VARIATION, YOU TALKED ABOUT BUT IT SEEMS TO ME, THERE'S TWO WAYS TO DO THIS. JUST LET EVERYBODY DO EVERYTHING THEY POSSIBLY WANT AND SOMEHOW GLUE IT TOGETHER AND I THINK WE WILL OR AT THE TOP, NOT GOVERNMENT TOP BUT SOME TOPS, SAY THIS IS HOW WE WILL DO IT AND SEE IF PEOPLE WILL. I THINK THAT'S--THAT CAN BE A BETTER PATHWAY BECAUSE IT'S JUST MORE WORK TO GO OUT AND GET YOUR OWN THING, I THINK. AND THE ONES THAT EXIST, AND THERE'S ONE DATA STRUCTURE IN THE MODEL, YOU KNOW IT HOLDS ALL THE ONES, IT WON'T HANDLE THINGS RADICALLY DIFFER UNDERLYING MODEL AND THE MOSTLY OBSERVATIONS THAT WE'VE BEEN IN SO FAR, PROMISE, NEUROQUALITY, THESE ARE JUST LOCKED DOWN. THEY'RE JUST LOCKED DOWN, YOU DON'T HAVE--YOU DON'T DEAL WITH PEOPLE DECIDING THEY'RE GOING TO PUT DIFFERENT ANSWERS IN THAN WHAT'S BEEN PUBLISHED. >> THE FLIP SIDE, I'M THINKING OF SMOKING STATUS, YOU THINK IT WOULD BE SIMPLE, YES, NO, HEAVY, LIGHT, TWO P A XATOX A DAY AND 10 PACKS A DAY AND SEE SO MUCH EASIER FOR ME TO DECIDE ON ONE THAN TO HAVE TO WEB SITE A AND USE IT. >> [SPEAKING AT ONCE ] BUT WHEN YOU PACKS PER DAY AND HOW MANY MUTTS BEFORE YOU START SMOKING. >> THAT'S WHAT YOU THINK BUT ARE THEY IN THAT AND IF WOULD YOU PUT THEM IS THIS? >> WE'LL MAKE SURE. TEY'RE IN THE IOM RE. [SPEAKING AT ONCE ] >> A NUMBER OF THESE HAVE SMOKING STATUS, MARITAL STATUS, IT WOULD SEEM EASY TO INVENT YOUR OWN, SEX, GENDER, YOU PICK IT. THEY THEY DO INVENT THEIR OWN, IT'S EASIER FOR THE INITIAL INVESTIGATOR TO DO BUT WHAT DOES THAT DO? IT LIMITS YOUR ABILITY TO COMPARE DATA ACROSS STUDIES, THIS IS ONE OF THE THINGS WE TRY TO DONATE AND BIG AT THAT TIMA THEY TAKE THESE BIG DATA SETS BECAUSE THEY ARE--THEY'RE COLLECTED AND STRUCTURE INDEED WAYS THAT ALLOW THAT KIND OF COMBINATION. SO IF WHY CAN MAKE IT RELATIVELY EASY FOR YOU TO FIND THE ONE THAT'S RECOMMENDED BY, RIGHT NOW, YOU WOULD SAY RECOMMEND BY NINES, RECOMMENDED BY PHOENIX AND RECOMMEND INDEED PROMISE, AT LEAST WE NARROW THE SEARCH BASE TO THREE OF THOSE BUT IN FACT WHAT WE'RE TRYING TO DO THROUGH THIS GROUP NOW AND IT'S A TEST OF GOVERNANCE STRUCTURE AND SO FORTH IS TO SEE HOW YOU COULD REDUCE THE VARIATION DOWN TO MAYBE IT'S NOT ONE BECAUSE THERE ARE GOING TO BE LEGITIMATE REASON YES YOU MIGHT NEED TO COLLECT IT A DIFFERENT WAY AND A DIFFERENT STUDY BUT MAYBE FIND A PREDOMINANT WAY, AND ADVANCE WE'RE STUCK A BIT WITH THE LEGACY PROBLEM, RIGHT. COMMUNITY VS SELECTED THE WAYS THEY WANTED TO DO IT AND THEY'VE DEFINED THEM AND MANY OF THOSE COMMUNITIES ARE NOW OFF DOING SOMETHING SO TO COME IN AND TRY TO CHANGE WHAT THEY'RE GOING TO DO IS GOING AND HAVING A NEW PROCESS. >> IT'S MESSENTERY ESTIMATE THAD BUSINESS, THAT'S MY SUMMARY, I MEAN THE SMOKING STATUS IS A PERSPECT EXCEL SO MANY DIFFERENT WAYS TO STORE THAT, TO TALK ABOUT THAT AND WE'RE JUST NEVER GOING TO SATISFY EVERYBODY'S NEEDS AND DATING THAT SO WE NEED TO FIGURE OUT AND ALLOW SPECIALIZATION AND EXTENSION AND DO THOSE THINGS RATIONALLY. >> SO I THINK WE STRUGGLE SYNTAX FROM SEMANTICS AND I THINK IF WE CAN AGREE ON A COMMON STRUCTURE AND ALLOW VARIANT ABOVE THAT AS NEED BE, THAT WOULD BE A HUGE PROGRESS AND I THINK IT'S PROBABLY A EASIER THING TO ACHIEVE THAN TRYING TO GET UNIVERSALITY IN TERMS OF DEFINITION AND IN TERMS OF STRUCTURE. IF WE SEPARATE THE PROBLEM, IT WILL MAKE IT EASIER FOR US TO SORT OF START TACKLING IT AND CREATE MACHINE READABLE WAYS OF MAKING COMPARISONS ABOUT VOCABULARY CAR CABULARIES AND TERMINOLOGIES THAT ARE ENCODE IN A COMMON WAY OF LOOKING AT THINGS. >> I AGREE WITH ALL EVER THAT AND I THINK WE CAN MAKE IT AS EASY AS WE WANT TO, BUT IT'S STILL THE EASIEST THING TO WRITE ON MY WORD DOCUMENT, SMOKING PER YEAR OR WHATEVER IS IN MY HEAD SO I THINK WE NEED TO INSENT PEOPLE TO--FIRST OF ALL EDUCATE THEM ABOUT THE EXISTENCE OF THESE RESOURCES AND THEN INSENT THEM TO USE THEM. THERE'S ALWAYS PLENTY OF WORK SO WHY DON'T WE START WITH THE EASY STUFF, THE QUOTES WHERE IT'S PUBLISHED PAPERS, SOME CONSENSUS THAT THERE'S TWO OR THREE WAYS TO DO IT AND LOCK ON THAT AND THEN--IF YOU PUT FIVE DIFFERENT GROUPS AND FIVE DIFFERENT ROOMS YOU AUTOMATICALLY GET FIVE DIFFERENT STANDARDS IT HAS NOTHING TO DO WITH TRUTH ORS REALITY ITS HAS TO DO WITH THE VARIABILITY OF HUMANS SO WHY DOE DOGMAS WE WANT TO CONTINUE THAT? THERE'S HARD STUFF WE WON'T SELL THAT WAY. >> WELL JUST AS I LISTENING TO DOUG IN HIS 40 MINUTE TALK SO I WOULD SAY ALSO DON'T LET THE PERFECT BE THE ENEMY OF THE GOOD. IF WE CAN GET SOMETHING THAT WORKS FOR MOST PEOPLE MOST OF THE TIME LET'S MOVE FORWARD THAT AND WE CAN DEAL WITH THE OTHER EITHER LATE OR RECOGNIZE THERE WILL NEED TO BE VARIATION AND ONE OF HIS OTHER POINTS, YOU KNOW BUILD AN INCREMENT AND REVISE AND WE THEY'D TO CONTINUE TO SEE THIS NOD AS A ONE TIME EXERCISE, BUT AS TRYING TO MOVE FORWARD IN INCREMENTAL STEPS TO IMPROVE STANDARDIZATION OF DATA AND TO BRING GREATER CONSISTENCY BETWEEN THE STANDARDS AND THE DATA COLLECTION ACTIVITIES AND THE CLINICAL CARE WORLD AND THOSE THAT ARE IN THE CLINICAL CARE RESEARCH WORLD. >> [INDISCERNIBLE]. >> SO IF WE CAN AGREE ON REALITY BASED METHOD WE CAN TAKE EACH OF OUR DIFFERENT DATA STORAGE, DATABASES, ELEMENTS AND SO FORTH AND FIGURE OUT HOW THEY PLUG INTO THAT COMMON REALITY THAT WE ALL HOPEFULLY SHARE. SO MY SOAP BOX IS, WE CAN ADDRESS THE COMMON SEMANTICS COMMUNEALLY IF WE FOCUS ON A REALITY PROCESS BASED APPROACH. >> COMMENTS? >> SO I THINK IN THANATOLOGY COMMUNITY THERE'S TWO SCHOOLS OF THOUGHT, THERE'S RELIGIOUS AND WE SHOULDN'T GET INTO THE RELIGIOUS WARS BUT SOME PEOPLE THINK THAT ONTOLOGIES ARE A REP VERMEN INFECTED SENTATION OF WHAT THE KIND OF REAL TRUTH IS THAT'S OUT THERE AND THAT DEVELOPING AN ONTOLOGY IS JUST DISCOVERING WHAT THE--HOW THE WORLD IS ORGANIZED AND SO THERE'S THIS NOTION THAT THERE IS ONE TRUE ONTOLOGY AND THAT IS A REFLECTION OF WHAT TRUTH IS OUT THERE IN THE REAL WORLD, I THINK THERE'S ANOTHER SCHOOL OF THOUGHT THAT SAYS THAT ONTOLOGIES ARE ENGINEERING CONSTRUCTS THAT HELP US MAKE SENSE OF THINGS, MORE ALONG THE LINES OF KIND OF CONSTRUCTED FOR A PURPOSE. SO THERE ARE SOME PEOPLE THAT BELIEVE IN ONTOLOGY AS A TRUTH EXTRACTED THROUGH OBSERVATION OF SCIENCE AND THINGS LIKE THAT AND ANOTHER GROUP THAT SAYS ONTOLOGIES ARE ENGINEERED CONSTRUCTS THAT HELP US SIMPLIFY THE WORLD AND MAKE SENSE OF THING THAT WE'RE TRYING TO TAKE A LOOK AT. I DON'T WANT TO GET INTO SORT OF WHICH IS THE RIGHT WAY BUT I THINK THE NOTION OF A--KIND OF A SINGLE TRUTH MAY WORK IN SOME FIELDS, CERTAINLY I THINK THE GENE ONTOLOGY WORK HAS BEEN AN AREA THAT AND THAT IS A LOT OF PROPONENT THERE IS THAT SORT OF BELIEVE THERE IS A SINGLE ONTOLOGY THAT CAN BE REPRESENTATIVE OF TRUTH. I THINK THERE MAY BE OTHERS THAT PERHAPS IN THE CLINICAL WORLD TAKEN--THEY'S THE ONTOLOGY SYSTEM MORE OF AN ENGINEERING CONSTRUCT TO HELP ORGANIZE COMPLICATED THINGS SO IT'S IMPORTANT TO RECOGNIZE BOTH BOTH PERSPECTIVES ARE VALID BUT WE HAVE TO FIGURE OUT HOW TO MANAGE WORKING WITH THOSE. >> [INDISCERNIBLE]--IT'S BECOMING OBVIOUS THAT THE INFORMATION HAS IN LECULAR PHENOTYPES, BEING THAT THE VALID AND RELIABLE STAGES OF DIAGNOSTICS SO THE GROUPS OUT THERE WHO ARE GOING TO REGULATE DIAGNOSTIC TOOLS AND STUFF LIKE THAT SO THEY WORK WITH ACTUAL AND WHAT WE CURRENTLY KNOW FOR BIOLOGY AND ALSO WHAT THE INSTRUMENTATION FOR USING, SO, YOU PUT THEM DISASSEMBLING THIS THING AND WHAT WE ALREADY HAVE THE LIST OF [INDISCERNIBLE] AND TAKE ONE OF THESE STANDARDS OUT THERE, THESE OTHER PHENOTYPES TODAY'S [INDISCERNIBLE] TO CONSTRUCT THEM OR COMPOSE THEM TOGETHER COMPOSITIONAL APPROACH SYSTEM A WAY TO MIX THE ONTOLOGY BUT THERE'S A LOT OF AREAS WITH [INDISCERNIBLE] AND I MEAN JUST TO GIVE YOU AN EXAMPLE WHERE IT MIGHT BE APPLIED TO [INDISCERNIBLE]. TRY TO IMAGINE JUST IN THIS ROOM, HOW MANY DIFFERENT KINDS OF MATERIALS, ALL THE MOLECULES ALL THE SUBSTANCES, THINK ABOUT THE TABLE, THE CLOTH, WHAT THEY'RE MADE OUT OF EVERYTHING, IT'S PROBABLY UP IN THE MILLIONS, DIFFERENT COMPOUNDS, BUT THEN IF YOU LOOK AT IT, AND SAY, WELL, BUT IT'S CONSTRUCTED OF HOW MANY ACTUAL ELEMENTS, THAT'S BELOW 12, MAYBE 24, THE QUESTION IS ARE WE POSING AND CREATING DATA STANDARDS [INDISCERNIBLE]? WHICH COMPOUNDS OF MAKING THIS [INDISCERNIBLE] I MEAN BOTH RELATIONS--RELATIONS OF VERY, VERY IMPORTANT TO CONSIDER. IT WOULD BE INTERESTING TO TAKE ALREADY THE LIST OF 50 OR 60 OF THESE FOR THESE THINGS AND JUST [INDISCERNIBLE]. JUST A SMALL PART OF IT TO SEE IF WE CAN GET COMMON ALITY. I BET YOU [INDISCERNIBLE]. >> I'LL EXERCISE PREROGATIVE HERE AND SAY I RESONATE DEEPLY WITH THE NOTION OF WHAT WE WOULD CALL ATOMIC DATA STANDARDS AND NOR GRANULAR DATA STANDARDS WHETHER WE COULD DO IT IN A DAY, I MIGHT QUIBBLE BUT OTHER COMMENTS? >> WELL, I'M NOT SURE WHETHER YOU'RE SAYING, SHIFT TO THE PHENOTYPE BUT WHEN YOU STARTED OUT WITH GENES, GENES ARE--I MEAN SLAUGHTERED OR KIND OF EASY BECAUSE YOU KNOW THERE ARE ALL, FOUR DIFFERENT THINGS AND YOU HAVE A FEW--I KNOW. I KNOW. [LAUGHTER] BUT I'M ON THE HL-SEVEN, I'VE BEEN INVOLVE WIDE IT I AND I THINK WELL IS A FIVE-10 OR MAYBE 15 VARIABLES CAN YOU DESCRIBE MOST GENE VALID AND RELIABLE YAGS WITH, NOW WE GET INTO DELETION, DUPLICATION AND THAT STUFF, I WOULD LIKE TO TALK TO YOU AFTERWARDS, I THINK THERE'S A DIRECTION GOING THAT WAY. >> SO I WOULD LIKE TO--DOES THAT WORK, I WOULD LIKE TO FOLLOW UP ON SOME OF THAT CONVERSATION. SO I THINK THAT'S INTERESTING BECAUSE BOTH HAVE JUST STUCK TO THE ONTOLOGICAL ARGUMENT THAT I WON'T DIVE INTO THE [INDISCERNIBLE] AND I GENTLEMENLY TRY TO STAY OUT OF THAT BUT I DO THINK THAT A LOT OF THE STANDARDS THAT WE VTHERE ARE REALLY PROXIES FOR THE KINDS OF INFORMATION THAT WE WANT TO REPRESENT SO THAT WE CAN MAKE IT REUSABLE AND SHAREABLE AND CONTEXTURALLIZE IN OTHER PLACES, RIGHT? SO IT'S NOT TO SAY THAT WE WERE JUST HAVING THIS CONVERSATION OVER THE BREAK, THEIR PROBABLY WILL NEVER BE AN AGREED UPON DEFINITION OF WHAT IS A GENE, DOESN'T MEAN WE CAN'T HAVE GENE IDENTIFIERS IN THE DATABASES AND EXCHANGE DATA SO JUST TRY TO THINK ABOUT THE FACT THAT YOU KNOW WHEN WE--WHAT WE REALLY WANT IS A MORE GRANULAR THING THAT ARE THE UNITS OF EXCHANGE FOR THE THINGS WE REALLY DEEPLY WANT TO BE EXCHANGING AND SO I GUESS THIS IS MORE OF AERG THAN A QUESTION OR COMMENT BUT MAY BE WHAT WE NEED ARE SPECIFIC USE CASES THAT MAKE US DIVE INTO THESE DIFFERENT STANDARDS THAT WE HAVE FOR DIFFERENT THINGS SO I REALLY LIKE THE SMOKING QUESTION. SO FOR EXAMPLE, I HAVE DATA THAT'S ABOUT NICOTINE EXPOSURE FOR MODEL ORGANISMS AND I HAVE A NEED TO TRY TO FIND PATIENTS THAT HAVE CERTAIN GENERATED FETTIC CHARACTERISTICS AND HAVE CERTAIN SMOKING PATTERNS, HOW DO I GO AND FIND THOSE PATIENTS, HOW DO I--WHEN I GO TO THE CDE, SYSTEMS, I FIND ALL KINDS OF THINGS, I DON'T KNOW HOW IT'S CONNECTED TO THE DATA OR WHERE THOSE PATIENTS LIVE OR HOW I DO THAT, HOW DO I CROSS THAT TRANSLATIONAL DIVIDE, LET'S FIND USE CASES LIKE THAT ONE WHERE WE ACTUALLY, YOU KNOW BORROW FROM ACROSS THE BASE AND CLINICAL COMMUNITIES TO ACTUALLY TRY TO COMBINE DATA AND SO MAYBE THAT'S ONE OF THE OUTCOMES WE CAN THINK ABOUT AND SO MAYBE IF WE CAN--IF THAT'S GOOD, IF PEOPLE THINK THAT'S A GOOD IDEA, WE CAN GENERATE THOSE IDEAS IN THE GOGGLE DOCS. >> WELL YOU JUST CROSS THE LINE FROM WHAT'S STRUCTURED TO WHAT'S MOSTLY NARRATIVE SO THAT YEAH, WE HAVE THE CDEs BUT THE CLINICIANS AREN'T USING ANY OF THEM ALTHOUGH THERE IS A REQUIREMENT NOW FOR MEANINGFUL USE, CAN YOU GET ONE LITTLE NOTCH ON SMOKING SO WHEN YOU DO--WE DO THE NATURAL LANGUAGE PROCESSING AND NARRATIVE OF CLINICAL RECORD AND WE DID THAT TO GET SMOKING HISTORY FOR A STUDY WE ARE TRYING TO DO AND YOU CAN'T GET IT AT THE LEVEL YOU WANT IT BECAUSE IT'S WHAT'S THERE ALREAD THAT YOU CAN'T CHANGE BUT THE CDEs IF YOU DO THEM RIGHT, WE CAN ENCOURAGE THAT THIS THE ONE YOU SHOULD USE AND THIS IS OFF THAT'S CURRENTLY USED, IT'S JUST--WHAT IS IT LAST YEAR, TWO YEARS, THREE YEARS AND THERE'S NOT A CONTINUOUS VARIABLE AND THERE'S--IF THERE WAS ENOUGH PEOPLE SAYING HEY, WELL THESE 20, THESE ARE THE TWO THAT ARE REALLY GOOD, AND EVERYBODY--ENOUGH PE AGREE OFFICE OF DIVERSITY IT, YOU WILLN'T HAVE TO HAVE EVERYBODY, THEN YOU COULD GET THEM USING CLINICAL CARE AND THEN YOU COULD GET TO THEM FOR THE TRANSLATIONAL PURPOSES. >> SORRY ONE QUICK COMMENT, A COROLLARY TO THAT IS SYNTAX AND FORMATS SO FIELD FUNCTIONS WANT TO COMBINE THINGS I CAN'T GET A CDE IN THE FORMAT I CAN MATCH IT UP WITH ALL THE BASIC SCIENCE RESEARCH DATA THAT I HAVE. >> SO WE HAVE TO GET AGREEMENT ON THAT FIRST. ANYWAY, I THINK THERE'S TWO THINGS THAT WE NEED. AGAIN, YOU KNOW, I WAS THE RECIPIENT AND YOU WANT TO TAKE THE CONSUMER LETTERS BACK THERE, BUT YOU KNOW EVERYBODY HAD THEIR, YOU KNOW IF ONLY WE COLLECTED, YOU KNOW SMOKING CESSATION IN THIS PARTICULAR WAY, OR IF ONLY WE COLLECTED THESE 50 CARDIOVASCULAR DISEASE, WOULDN'T THIS BE SO GREAT AND IF WE ONLY DID THE MENTAL HEALTH AND YOU ADD IT UPA ULTIMATELY AND YOU GET 1100 DIFFERENT COMMON DATA ELEMENTS AND IF YOU TAKE A LOOK AT WHAT'S HAPPENING IN THE POPULAR PRESS, THERE'S A REVOLT AMONG POSITIONS THAT THEY'RE TURNING INTO A CHECK BOX, A SERIOUS OF THINGS THEY JUST NEED TO CHECK OFF AND WE CAN'T LET THAT HAPPEN, SO THAT WILL BE TO THE DETRIMENT OF EVERYONE. SO WHAT WE NEED TO DO IS FIGURE OUT WHAT THE PARSIMONNUOUS SET OF A CENTRAL DATA ELEMENT WE NEED AND MAYBE IT'S FOR DATA INTEGRATION AND DOING LONG NUDEINAL STUDIES AND IDENTIFYING PATIENTS OVER TIME OR WHATEVER, BUT IT'S A SMALL SUBSET. BUT THE OTHER THING WE NEED IS THAT WE CANNOT WAIT TO HAVE EVERYTHING STRUCTURED TO BE ABLE TO EXTRACT IT FROM THE ELECTRONIC HEALTH RECORD AND TO--YOUR POINT A LOT OF THIS STUFF WILL BE FREE TEXT. NOW THERE ARE A LOT OF REALLY SMART NATURAL LANGUAGE PROCESSING FOLKS OUT THERE THAT DO SOPHISTICATEDANISTICATED STUFF, WE HAVE ONE SITTING AT THE PODIUM AND THERE'S WAYS IN WHICH YOU CAN SORT OF DO THAT. SO, IF WHAT WE WANT TO DO IS ADVANCE THE SCIENCE AND THE INTEGRATION WITH THE CLINICAL SPACE, WE NEED TWO THINGS, ONE IS A PARSIMONNUOUS SET OF COMMON DATA ELEMENTS THAT ARE HELPFUL FOR DATA INTEGRATION AND MATCHING AND OTHER THINGS LIB THAT AND WHAT WE NEED IS WE NEED SOMETHING THAT WILL ALLOW US TO EXTRACT THE ENTIRE MEDICAL RECORD AND WHAT WHATEVER GLORIOUS STRUCTURE UNWAVED STORE AND IF YOU TAKE A LOOK AT HIPAA, THE P-STANDS FOR PORTABILITY, IT DOESN'T STAND FOR PRIVACY AND THE PROBLEM WE HAVE IS THAT THERE IS NO PORTABILITY, IF I GO TO MY DOCTOR, WHO USES A DISCERNER SYSTEM AND I DECIDE NOW THAT BECAUSE MY INSURANCE CHANGES, I'M NOW SEEING A DOCTOR THAT USES THE NEXT GEN SYSTEM, INTEROPERABILITY IN THE CLINICAL SPACE REQUIRES A PRINTER AND A SCANNER BECAUSE THAT'S THE WAY YOU GET THE INFORMATION OUT AND HOW YOU GET IT BACK IN. WE NEED TO HAVE HIPAA HAVE A PORTABILITY STANDARD THAT ALLOWS ME AS A PATIENT TAKE MY ENTIRE MEDICAL RECORD, STRUCTURED AND UNSTRUCTURED AND BE ABLE TO MOVE IT FROM ONE SYSTEM TO ANOTHER AND I THINK THAT BECOMES A WAY THAT WE CAN START RELEASING THAT DATA OVER TIME STRUCTURAL BECOMES RELEVANT AND LEAVE WHAT'S UNSTRUCTURED ALONE BECAUSE OTHER PEOPLE CAN DEAL WITH THAT. >> SO, THE IDEA OF AN IDRATIVE DEVELOPMENT THAT CAN DO SOMETHING CONCRETE THAT'S USEFUL FOR [INDISCERNIBLE]. AND THEN OVERTIME AS WE BUILD ON OUR SUCCESS, WE CAN INCREASE THE COMPLEXITY OR THE COVERAGE BUT NOT REALLY RESONATE, NOW THE CONCERN THAT I HAVE IS THAT THE HEALTHCARE DOMAIN HAS NOT BEEN VERY FRIENDLY. OR HAS NO REAL OPERATING PROCEDURES BY WHICH PEOPLE MIGHT SPEAK AND CONTRIBUTE TO [INDISCERN] WITHOUT SUBSTANTIAL INVESTMENT HOW DO WE GO FROM THE VERY TOP HEAVY INSTITUTIONAL APPROACH TO DEVELOPMENT OF STANDARDS TO SOMETHING THAT IS VASTLY MORE LIGHT WEIGHT AND MORE EASILY TAKEN INTO THE CONSIDERATIONS TO EXTEND OUR SCOPE AND OUR [INDISCERNIBLE]. >> [INDISCERNIBLE], YOU MIGHT FIND A HOME. >> I HAVE FOUR LETTERS FOR YOU FHIR, FHIR, BECAUSE IT AS ATTRACTED PEOPLE IN INCREMENTAL ITERATIVE DEVELOPMENT IT'S UNLIKE OTHER THINGS THAT I'VE SEEN THAT KREBS CYCLE COME OUT OF HL-SEVEN. >> BUT I DON'T WANT TO FOCUS ON THE TECHNOLOGY, I WANT TO FOCUS ON THE SOCIAL STRUCTURE THAT ENABLE THE CONTRIBUTION AND PROCESS BY WHICH THE ENGAGE-- >> I WILL SAY ONE THING ALL DEVELOPMENT IS SOCIAL, AND IF EVER GONE TO A SOCIAL STAN ARDS DEVELOPMENT MEETING WHILE IT IS--SOMEONE'S LAUGHING, THERE'S A RELL BIG SOCIAL COMPONENT OF THAT. WHAT OF WHAT I TRIED TO DO WAS RECOGNIZE WE NEEDED NEW PEOPLE TO ENGAGE, PART OF THE REASON WE SET UP THE STANDARDS IN THE INTEROPERABILITY FRAMEWORK, I DOPE FINISH ACHIEVED ALL OF ITS OBJECTIVES IN TERMS OF IDENTIFYING NEW PEOPLE TO COME IN, BUT THE BEST WORK THAT CAME OUT OF SOME OF THOSE ACTIVITIES WAS THE SMART PROJECT UP AT HARVARD, WHETHER THEY DID ENGAGE A NEW GROUP OF FOLKS INTERESTED IN THE INNOVATIVE APPLICATIONS AND REALIZING STANDARDS WOULD BE A WAY TO GET THERE. IT'S A REALLY HARD PROBLEM, WE'VE BEEN BRAINSTORMING ABOUT HOW DO YOU ENGAGE THAT INNOVATION COMMUNITY. WELL IF YOU JUST GOT YOUR SEED FUNDING AND YOU GOT A LIMITED TIME BEFORE YOU WILL BE ABLE TO GET THE PRODUCT TO MARKET OR GET YOUR NEXT ROUND OF FUNDING, YOU KNOW MAKE SOMETHING OF THESE MILESTONES, YOU DON'T TIME TO SPEND A WEEK IN HLSEVEN AND COMMITTEES TALKING ABOUT WHETHER SHARE VERSES STINGING YOU COULD ONLY HAVE ONE NAME IN THE ECTRONIC RECORD. YOU MEAN, THOSE ARE NOT THE KINDS OF THINGS THAT HAVE YOU TIME TO DISCUSS. YET MAYBE THAT IT'S THE INCUBATOR THAT IS YOU GO TO, THAT YOU FIGURE OUT WE NEED TO HAVE SOMEBODY AT THE INCUBATORS THAT HAVE A WHOLE SERIES OF FOLKS WORKING ON SIMILAR PRODUCT THAT CAN BE THAT LIAISON TO THE STANDARDS COMMUNITY. I DONE KNOW IF YOU HAVE IDEAS THAT WOULD BE REALLY GOOD TO HEAR. >> ARE YOU TALKING ABOUT--ARE YOU ASKING THE SOCIAL THINGS OF DEVELOPING A STANDARD, IT'S MORE OF THE SORT OF MAKING IT PART OF THE SYSTEM. TRYING IT OUT. IT'S A HIGH BAR TO GET THING INTO'S CLINICAL PRACTICES AND YOU DON'T WANT TO KILL PEOPLE WHILE YOU'RE EXPERIG WITH DOES THIS CLINICAL DECISION SUPPORT WORK? ARE YOU TALKING ABOUT THAT SOCIAL-- >> I'M TALKING ABOUT HOW DO WE AGAIN, GARNER THE INTELLIGENCE OF OUR COMMUNITY? THE FURTHER DEVELOPMENT OF EMERGING THAT AND HOW DO WE KACCT TININE PATHWAY THAT YOU ARE AND LOWER THE POINT FOR SMART PEOPLE CAN MAKE THE SMALL CONTRIBUTIONS THAT MOVE THE STANDARDIZATION FORWARD. I LIKE THE IDEA OF INCUBATORS [INDISCERNIBLE]. --THAT GETS US THERE. >> IT'S GOING TO BE TOUGH. I'VE DONE HL-SEVEN FOR 20 PLUS YEARS IT'S MORE LIKE A SCRUM THAN AN INTELLECTUAL PROCESS IF YOU MIOSES A MEANING--NORKS IN THE TECHNICAL DEVELOPMENT OF SCRUM, MORE LIKE RUGBY. >> THERE'S THERE CONTINUITY AND SOMETIME FIST THE MEETINGS BEEN WORKING AND YOU GET FIVE NEW GUYS COME TAG'S ALMOST THE WORST NEWS THAT MEANS YOU GET TWO EXTRA CYCLES BECAUSE UNLESS UNTIL THEY GET AUGHT UP WITH THE WORDS AND MEANINGS AND ALL THAT CRAZINESS THAT PEOPLE GET OBSESSED ABOUT, YOU KNOW THERE'S RELIGION EVERYWHERE. AND ALL THE DISCUSSIONS, AND SO I WOULD ACTUALLY SLEEP APNEA AND OBESITY THIS, ANYTIME FIND A MEETING THAT ENDS UP CLOSE AND GO TO IT AND EITHER BE DISGUSTED OR BE INFORMED OR--YEAH, BECAUSE YOU CAN COMMUNICAE AT THE BALLOT POINT BY E-MAIL. SO ANYBODY CAN PUBLICLY COMMENTOT BALLOTS, BUT THESE THINGS ARE TOO BIG. I MEAN--WELL, ACTUALLY ABOUT FIRE, IT'S SMALLER. YOU KNOW THE CDEIGHT THING SYSTEM 1200 PAGES, YOU GET SICK TRYING TO READ IT. DON'T READ THE CDA DOCUMENTATION. >> GOOD ADVICE. >> NO, I REALLY THINK THERE'S AN OPPORTUNITY WITH FHIR, AND THEY'RE DOING INNOVATION AROUND. THEY'RE TRYING TO DO THE 80/20 RULE. SO YOU SPEND 80% OF YOUR TIME TRYING TO SOLVE, THEY'RE DOING THE RYE THINGS NOW WHETHER OR NOT THEY'RE SUCCESSFUL OR NOT DEPENDS ON THE ENGAGEMENT OF PEOPLE OUTSIDE OF THE STANDARDS AND PEOPLE WHO HAVE BEEN THERE ARE 20 YEARS, YOU KNOW, IT'S--THERE IS A CERTAIN INERTIA THAT OCCURS. BUT I THINK THERE ARE SOME STUDENTS THERE, THE BIG PROBLEM IS THAT IT TAKES TIME, IT'S NOT CHEAP, YOU HAVE TO FIGURE OUT SOME WAY TO CONSOLIDATE AND SPREAD IT ACROSS A HOST OF INNOVATORS BECAUSE ANYONE PERSON I DON'T THINK HAS THE TIME TO DO IT. SHRKS WE PROBABLY HAVE TIME FOR TWO MORE QUESTIONS. >> [INDISCERNIBLE] NCI, SO FIRST I'LL THROW A MEAN IN BEFORE ANYONE ELSE DOES WHICH IS THAT ALL MODELS ARE WRONG [INDISCERNIBLE]. --HOW I USE THAT THE FIRST THING I KNOW ABOUT THE MODEL IS IF IT'S WRONG. IF IT'S TELLING ME IT'S WRONG, I KNOW IT'S WRONG, BUT [INDISCERNIBLE] BUT I THINK THE THING YOU TALKED ABOUT, ABOUT MAKING THE CONSTANT LEAP YOU LIKE, THINKING ABOUT THINGS IN AN ARCHITECTURAL LEVEL, TO THE PLANNING LEVEL, I THINK THAT PARALLEL PART [INDISCERNIBLE] WHICH IS WE FOCUS ON WHAT WE HOPE ALL HAVE HEAR FAR I LOIN TIME AND TALK ABOUT STANDARDS AND THEN I THINK A LOT OF DISCUSSION AROUND HERE IS THE SILVY OF ABOUT STANDARDS IS [INDISCERNIBLE]. THE LOGICAL MODEL AS WELL. VERY INTERCHANGEABLE IN THIS REGARD, AND WE'RE TRYING TO--A LOT OF THE DISCUSSION AROUND STANDARDS AND AT THAT LEVEL. SO THERE'S A CONSTANT LEVEL DIFFERENT OF THAT WHICH I THINK MAY HELP WHICH IS TO THINK ABOUT THINGS THAT THE METAMODEL, THE METAAND THERE'S A USEFUL FRAMEWORK WHICH COMES FROM THE OBJECT MANAGEMENT WHERE WE CAN [INDISCERNIBLE] WITH THE OBJECT AND IT RECOGNIZES THE [INDISCERNIBLE] AND THE MODEL AND EVEN AT BEST MODIFY THE STYLES AND [INDISCERNIBLE] THE METAMETAMODEL WHICH IS I DON'T--KNOW WHETHER WE NEED TO GO THERE PERHAPS WE DO BUT I THINK WHEN RECOGNIZES THAT LAYER AND IN THE PAST WE HAVE BLURRED INTO SOME OF OUR DOMAIN MODELS WHAT IS REQUIRED IN THE METAMODEL AND THE RIM THAT YOU REFERRED TO IS ONE OF THE PROBLEMS, AND ANOTHER MODEL THAT DOES THAT IS THE BRIDGE MODEL. AND I THINK WHAT TOM IS DESCRIBING, HE WAS DESCRIBING AN INITIATIVE TO I THINK WORK OUT TO HOW TO SUPERVISE THE EXTENDABILITY, OR THE CAPABILITY TO PROVIDE THE EXTENDABILITY, IF IT LIVES IN THAT METALAYER, THE OTHER LAYER, AND DEAL WITH THIS AND I THINK IF WE GET--IF WE BECOME LESS RESTRECTIVE, THE MAIN LAYER, ANYBODY PUT IN MODELS WE WANT BUT WE HAVE THE OTHER STANDARD THERE, THE BETTER LAYER WHICH ALLOWS THEM TO DO THAT, THAT BRINGS US TO THE POINT THAT CHRIS WAS--REFERRING TO EARLIER, YOU THEN GOT EVERYTHING ON A PLATFORM WHERE YOU HAVE THE GUIDELINE AND THEN YOU CAN SAY, WELL, WE'VE GOT DUPLICATES HERE AND HOW DO WE--HOW DO WE [INDISCERNIBLE]. >> RIGHT. >> VERY BRIEFLY SPONSE BECAUSE WE HAVE TWO MINUTES. >> SO WHAT CINDY'S DO SUGGEST WORKING ON THAT METAMODEL AND THEY'RE PASSING IT THROUGH OMG SET OF PROFILES ON UML THAT THEY'RE CALLING AML, ARCH TYPE MODELING LANGUAGE TO DEFINE THOSE FEATURES OF THE LANGUAGE OF THE MODELING LANGUAGE THAT ARE ALLOWABLE TO EXPRESS THESE SEMIMODELS. BUT SEE THAT'S A TALL ORDER TO EXPECT THE WHOLE WORLD, ALL THE OTHER EFFORT TO USE THAT SAME METAMODEL SO WE'RE STILL, WE STILL HAVE THIS PROBE DEVELOPMENT OF HOW DO WE ALL GET TOGETHER. >> THREE BRIEF COMMENTEDS, HAVING WORKED ON THE MODEL, I KNOW IT'S WRONG, BUT IN THE ACCENT, I KNOW ENGLAND HAS DONE A GOOD WORK THE MODELS AND LAYERS ARE QUITE NICE WORK AS THEY THINK ABOUT THAT AND YOU CAN SLOT SOME THINGS IN. IT'S HARD SOMETIMES TO DO THAT. YOU HAVE TO BE CAREFUL THAT YOU KNOW, I TRAINED AT STANDIFORD, YOU WE USED TO SAY THAT ANYTHING YOU CAN DO META, ANYTHING YOU CAN DO I METACAN DO FOR YOU, AND IT YOU CAN GET DISTRACTED. I THINK HAVING WORKED ON THE BRIDGE MODEL. I THINK THERE'S GREAT VALUE IN VERY SIMPLE INFORMATION MODELS AND LEVERAGING THE KIND OF ONTHAT LOGIC MODEL AND VOCABULARIES AND MORE TIME THAN WE SHOULD AT THAT INFORMATION MODELING LEVEL AND DO LATE BINDING AND WE SHOULD LEVERAGE A LITTLE BIT MORE THE ONTOLOGIES WE'VE GOT ALREADY THAT EXIST IN SOME OF THOSE VOCABULARIES. >> SO I WANT TO PUT ON THE TABLE, I THINK IT'S COME UP A LITTLE BIT, BUT THE ISSUE OF, WE NEED OTHER STANDARDS WHETHER IT'S METADATA OR WHETHER IT'S DATA, BUT WE NEED INFORMATION ABOUT PEOPLE AND ABOUT THE METHODS AND ABOUT THE MACHINES AND ABOUT THE OTHERS AND WE DON'T NEED IT BURIED DOWN, WE NEED TO BE ABLE TO SOLVE THE REPS OF THAT WITHA ALL OF OUR OTHER DAT AND THE DATA WE CAN BRING IN. I JUST DON'T KNOW WHERE IN THIS MEETING THAT FITS. BUT I JUST LIKE TO PUT IT ON THE TABLE. >> THANK YOU. LAST WORD? >> I THINK THAT'S A REAL PROBLEM AND MAYBE THE U. I.D. CODE WHICH WOULD IDENTIFY THE UNIVERSE OF THESE THINGS WOULD ALLOW THOME TO THINK ABOUT CLASSES OF THEM. DOES SNOMED HAVE CLASSES FOR ALL KINDS OF MACHINES? >> FOR MACHINES? NO. BUT THANK YOU. THIS HAS BEEN A VERY BROAD RANGING DISCUSSION. OUR CHALLENGE WILL BE TO DISTILL IT INTO SOMETHING THAT IS OPERATIONAL AND USEFUL TO NIH THAT PERHAPS WILL REMAIN IN THE FUTURE BUT THE SUBSTRATE ON WHICH WE BED BASE THOSE CONCLUSIONS HAS BEEN IN MY OPINION VERY RICH. SO LET US THANK OUR COLLEAGUES. [ APPLAUSE ] WE HAVE A BREAK. >> ALL RIGHT, SO THIS IS OUR FOURTH PANEL ON COMPLEXITY AND DATA DIVERSITY. I'M JAMES TAYLOR I'M FROM JOHN HOPKINS UNIVERSITY AND OUR PANEL WILL CONSIST OF MICHELLE FROM STANFORD AND CHARLES FROM UNIVERSITY OF PENNSYLVIA AND FILLIPPE FROM OXFORD AND THEN A REPEAT OF OLIVIER FROM NLM. WE WILL STICK WITH THE FORMAT OF A BRIEF PRESENTATION, ONE SLIDE, I'VE BEEN ALLOWED TO HAVE A SLIDE AS WELL AND THEN WE WILL HAVE A DISCUSSION AND THEN WE WILL TURN IT TO THE AUDA YINS AND HAVE A DISCUSSION. SO THERE WILL BE A LOT OF OVERLAP WITH THINGS WE TALKED ABOUT SO FAR BUT IF YOU PUT THEM IN THE CONTEXT OF THE TITLE OF THE SESSION WHICH IS DATA DIVERSITY AND COMPLEXITY, THEN I THINK QUESTIONS LIKE WHY DO WE NEED STANDARDS AND HOW DO WE DETERMINE IF THEY'RE USEFUL WHERE WHAT TYPES OF DATA SHOULD WE BE TARGETING, WHERE ARE WE GOING TO GET OUR BEST IMPACT FROM STANDARDIZATION EFFORTS AND THEN WE HAVE THE SECOND TWO QUESTIONS--THE SECOND AND THIRD QUESTIONS HERE, HOW DO YOU KNOW YOU NEED A STANDARD. AND HOW DO YOU KNOW A STANDARD IS WORKING WELL, AND SO, THESE ARE KIND OF VALUED QUESTIONS, RIGHT? CAN WE DETERMINE UPFRONT WHETHER WE SHOULD BE STANDARDIZING AND HOW DO WE DETERMINE AFTER THE FACT WHETHER STANDARDIZE SUGGEST WORTH THE EFFORT. AND THEN FINALLY HOW DO YOU KNOW WHAT IS THE RIGHT STANDARD FOR A GIVEN APPLICATION. SO THIS IS MY SLIDE, I'VE TAKEN THE LIBERTY OF REINTERPRETING THESE QUESTIONS. I AM NOT A STANDARDS PERSON. I AM A WEIRD CHOICE TO BE HOST THANKSGIVING OR TO BE FACILITATING THIS PANEL. IN FACT--[INDISCERNIBLE]--WELL, THERE'S PROBABLY TWO REASONS I'M HERE, ONE IS MY GENERALLY CONTRARY ATTITUDE ABOUT EVERYTHING, BUT, YOU KNOW ALSO, I HAVE KIND OF RELUCTANTLY BECOME A--BITS ACCIDENTALLY CONSUMER AND EVEN CREATOR OF STANDARDS FROM TIME TO TIME. YOU KNOW I USED TO DO A LOT OF SOFTWARE PROCESS WORK AND SO I WORK WIDE A LOT OF TECHNOLOGIES, I SAW EARLIER TODAY, THAT ACTUALLY LIKE THAT SENT ME RUNNING BACK TO ACADEMIC RESEARCH TO BURY MY HEAD IN BIOLOGY, BUT, FOR BETTER OR WORSE, I STARTED MAKING A SYSTEM THAT CALLED GALAXY THAT LETS PEOPLE ANALYZE DENATIONAL LIBRARY OF MEDICINIC DAT THAT'S ANALYZED BY 10S OF THOUSANDS OF PEOPLE AND PROBABLY MANY MORE AND IN THAT CONTEXT WE'VE HAD TO LOOK AT A LOT OF DATA INTEGRATION PROBLEMS PEOPLE HAVE AND WE HAVE BEEN DISESTABLISHING THE FACT OF STANDARDS FOR HOW YOU DESCRIBE AND REASON ABOUT ANALYSIS TOOLS. AND SO GENOMIC SYSTEM KIND OF INTERESTING BECAUSE IT'S A GREAT GREAT EXAMPLE OF TYPES AND FORMATS AND THE IDEA OF RESEARCH DATA AND STANDARDS AND VARYING QUALITY THAT TIES IN NICELY WITH THE STANDARDS LIFE CYCLE THAT WE'VE BEEN TALKING ABOUT, A BIT, GENOMICS HAS BEEN, PARTICULARLY INTERESTING IN THAT JUST THINK BEING ANALYSIS TOOLS RATHER THAN STANDARDS ONE OF THE TRUISMS OF THE LAST 15 YEAR SYSTEM THAT THERE'S ABSOLUTELY NO POINT IN EVER OPTIMIZING A BIOINFORMATICS TOOL BECAUSE ONCE YOU'RE DONE, THE DATA YOU'VE DONE IS WORTHLESS, AND THAT'S TRUE FOR STANDARDS AS WELL AND THERE ARE AREAS WHERE STANDARDIZATION IS STARTING TO MAKE SENSE SO FROM MY PERSPECTIVE I WILL LIKE TO ASK THE QUESTION, WHEN DOES IT MAKE SENSE TO CREATE A STANDARD IN THE FACE OF DATA THAT'S EVOLVING AND IN PARTICULARLY I'M CONCERNED ABOUT THIS IDEA, RIGHT? SO STANDAARE ALWAYS GOOD. IF YOU HAVE INFINITE RESOURCES IT IS ALWAYS BET TORE HAVE A STANDARD THAN NOT TO. I'M MORE INTERESTED IN THE QUESTION OF WHETHER ARE STANDARDS WORTH THE EFFORT AND I THINK THIS IS REALLY IMPORTANT AS WE THINK ABOUT SPECIFIC ADVISE WE CAN GIVE TO FACILITATE PROGRESS IN BIOMEDICAL RESEARCH AND BIOMEDICAL APPLICATION SYSTEM REALLY WHERE STANDARDIZATION IS ACTUALLY GOING TO HELP SCIENCE AND APPLICATION. AND IN PARTICULAR I WANT TO THROW OUT THIS THING OF INCENTIVIZATION IS A WEIRD QUESTION TO ME AND WE TALK ABOUT HOW STANDARDS NEEDS TO BE A PAID ACTIVITY. I WOULD PUSH BACK O THAT A LITTLE BIT BECAUSE IT'S STANDARDS DEVELOPMENT IS NOT OBVIOUSLY NECESSARY TO DO THE SCIENCE AND TO DO THE MEDICINE, THEN SHOULD WE--HOW DO WE--WHAT INSEBTIVIZATION NECESSARY, IS THAT STANDARD NEEDED IF THE COMMUNITY DOESN'T SAY THEY NEED IT DO GOOD SCIENCE AND GOOD MEDICINE? SO, THIS THEN, I JUST LIKE TO ASK THE QUESTION, WHEN STANDARDS ACTUALLY HELP VERSES HINDER RESEARCH, WE HAVE A LOT OF AREAS WHERE THINGS ARE CHANGING RAPIDLY AND FORCED TO CONFORM TO STANDARDS AND MANDATES THAT COME DOWN ACTUALLY SLOW DOWN PROGRESS AND YOU KNOW, MORE EFFORT GOES INTO FORMATTING DATA AND STANDARD WAYS AND SHARING IT. THAN THE DATA IS ACTUALLY PROBABLY WORTH. AND THEN FINALLY EEVALUATE THE QUESTION I WON'T HAMMER ON AGAIN HOW CAN WE PREDICT AND EVAL THE STANDARDS? IT CAN'T BE IGNORED WE NEED TO THINK ABOUT WHAT ARE THE'RE APPROPRIATE METRICS RICKS TO THINK ABOUT IF A STANDARD IS WORTH WHILE AND I WANT TO ASK WHERE ARE THE SUCCESS STORIES AND THE EFFORTS GOING INTO THE STABBED ARDS HELP US DO BETTER SCIENCE. PLEASE I WOULD LIKE TO LEARN MORE AND I'M PUT THANKSGIVING TO THE PANEL. I WANT TO HEAR AND THE AUDIENCE, I WANT TO HEAR LOTS OF CASES, YOU KNOW, THIS GENERAL THING OF, WE DO A LOT OF STANDARDS, WHAT FRACTION OF THAT EFFORT OVERALL HAS BEEN WORTH WHILE. AND HOW CAN WE DO A BETTER JOB OF INSURING THAT EFFORT GOES TO THINGS THAT ARE EFFECTIVE. I THINK--ALL RIGHT SO THAT'S ALL I'M GOING TO SAY AND I WILL TURN THIS OVER TO I BELIEVE PH ILLIPPE IS NEXT. >> OKAY, SO, I AM HERE WITH SEVERAL HATS ON MY HEAD. I AM REPRESENTING BIOSHARING AND THE BUILD NOTHING EUROPE, AND I LOOK FORWARD, ALSO I WOULD LOAMACY TOYING MENTION THE TALK BECAUSE THIS SOMETHING THAT'S DEVELOPED AS WELL, AND GOING BACK TO THE POINT OF STANDARDS AND PINPOINTS THAT WE HAVE FACED WHEN WE ARE WORKING BACK OUT TO THE LBDI AND WE'RE DEALING WITH THE PEOPLE DOING WHAT WAS SUPPOSED TO BE NONCONVENTIONAL EXPERIMENTS WHO HAD MANY, MANY DIFFERENT KIND OF TECHNIQUES AND METABALOMICS PROTEOMICS AND MICROARRAYS AND WHEN FACE WIDE THE POSITION OF THE DATA, HAD THE SUBMIT THE SAME INFORMATION THREE TIMES THAT WAS REALLY GOING DOWN VERY BAD WITH THEM. SO IN A WAY, WHY DO WE NEED THE STANDARD. THE FIRST POINT IS THAT WE WANT TO SHOW THE INFORMATION, WE WANT TO ENABLE COMMUNICATION, SEEMS TO BE OBVIOUS BUT USUALLY THIS IS WHAT PEOPLE WANT TO DO, THEY WANT TO RELEASE THAT DATA AND MAKE IT AVAILABLE FOR PRESERVATION, AND ALSO FOR EVALUATION AND THAT'S REALLY THE KEY POINTS AND KEY DRIVER, I WOULD SAY OF THAT STANDARDIZATION AND NOT RELEASING DATA AND RELEASE IT IN A FORM THAT IS NOT USE RESPONSIBLE THEN WE NEED ACTION BECAUSE WE HAVE PROBLEMS WITH REEVALUATING THE RESULTS AND WE TRY THE EVIDENCE AND THIS IS IN A WAY WHAT WE SHOULD BEAR IN MIND, THIS IS THE MAIN DRIVER OF THE STANDARDIZATION, IT IS NOT A CAREER PATH, IT SHOULD NOT BE INCENTIVIZED AS A CAREER PATH, AND WITH PEOPLE GOING INTO THE FIELD AND MAKING MORE STANDARD FOR THE SAKE OF MAKING STANDARDS AND PROBABLY NOT WHAT YOU WANT TO DO SO YOU WANT TO DO STANDARDS THAT ARE EFFECTIVE AND DELIVER VALUE FOR MONEY. SO WHEN DO WE KNOW AND RECOGNIZE THAT THERE'S A NEED FOR STANDARD, I WOULD SAY THAT THE FIRST THING IS MAYBE THERE IS A DISRUPTIVE TECHNOLOGY THAT COMES ASBESTOS ROUND AND CERTAINLY MANY, MANY LABS PRODUCE A LOT OF DATA. AND WHEN WE TRY TO ACCESS DAT AWELL, WE ARE STRUGGLING TO UNDERSTAND WHAT ACTUALLY WENT ON WITH THE OVERALL EXPERIMENT AND I WOULD LIKE TO DISTINGUISH TWO THINGS, TWO COINEDS OF EXPERIMENTS LOOKING AT THIS NEW TECHNOLOGY, THE KIND OF EXPERIMENT THAT TRY TO REALLY FINALIZE THIS AND FINE TUNE THE SIGNAL THAT'S PRODUCED BY THE TECHNOLOGY AND THEN KIND OF A VERY EARLY STAGE THAT SHOULD NOT DIGS RUPT THIS STANDARDIZATION, WHICH IS ABOUT HAPPENING THE MODEL AND THEN THERE'S THE EXPERIMENT THAT ARE USING IN TECHNOLOGY TO PRODUCE BIOLOGICAL SIGNAL COMES IS THE MOST IMPORTANT BIT FOR BIOLOGISTS AND THE REST BEFORE AND FINE TUNING AND GETTING RELEVANCE HOWEVER, IN THIS PROCESS, I THINK THERE ARE PATTERNS WE CAN NOW UNDERSTAND FROM THE EXPERIENCE FROM THE FIELD AND RNASEEK, THE SEQUENCING AND TECHNOLOGY, THE PROTEOMICS, THE TSA, THE METABALOMIC AND WE NEED TO WORK THROUGH THE MANUFACTURES CLOSE BECAUSE WE KTO THE [INDISCERNIBLE] THE IO, WHERE WE HAVE TO PAY FOR LICENSING TO TOOLS SO THAT'S [INDISCERNIBLE]. AND AGAIN IT'S A BARRIER TO THE DISSEMESTER NATION AND WORSE IN THE LONG-TERM THAT THE RESERVATION WE'RE TALKING ABOUT THE LEGACY SYSTEM, ABOUT HAVING TO SUPPORT THE SYSTEMS FROM 19, WELL, THAT'S THEART FACTS IN THE LONG RUN FWE NEED TO MAINTAIN ALL THE APIs, ALL THE TOOLS THAT'S A NIGHTMARE SO WE NEED TO OPEN SOURCE SO EVERYTHING IS ATTAINABLE. WE HOW DO WE KNOW IT'S WORKING, IT IS USED BASICALLY. PEOPLE ARE FINDING FUN IN USING THIS AND GOING THROUGH THE PAIN. SO BASICALLY WE CAN IDENTIFY THE STANDARDS IN IDENTIFYING THE PLAYERS WITH OF THE STANDARDS SO THE REPOSITORIES ARE BUILT AROUND THE STANDARD FOR THE REPOSITORIES OF THE AND THEY ARE NOW, THIS IS WHERE I THINK INDICATORS OF A SUCCESS AND INDUSTRY PLAYERS WILL PUCK IT UP, SO THAT DIRECTLY FROM THE MACHINE. DISREGARDING THE POSSIBILITILY, ALL INTERESTS, FINANCIALLY INTERESTS IN ADDING THE STANDARD OR THE FORMAT, I THINK WE SHOULD JUST AVOID THE STANDARD TO TALK ABOUT SPECIFICATIONS AND ONLY WHEN THEY ARE GOING THROUGH THIS ACCEPTANCE THEN WE HAVE THE SUCCESSFUL FORMAT. AND THEN, THEN REALLY ANOTHER INDICATION IS THE EXTENT OF HOW MUCH EXTENSION VS BEEN BUILT UPON THIS FORMAT, AND OF COURSE THE BEEN FIT AND I WOULD LIKE TO RESPOND FOR THE ELEMENTS AND YOU HAVE USER WITH REQUEST LIKE THIS DOCUMENTED EVERYTHING IS WELL DENTED WHICH BRING MEAS TO THE NEXT POINT HOW TO EVALUATE THE STATUS AND AGAIN, SOMETHING IN EXPERIENCE OF EITHER TOOL FOR INWAS LIKE KEEP IT SIMPLE, EASY TO USE WAS A MAJOR BOTTLENECK OF USABILITY BUT USUALLY ARE YOU KILLING AND A LOT OF EFFORTS. SO DELIVERING THE STANDARD BUT YOU ACTUALLY DELIVER THE TOOLS VERY, VERY, EARLY ON SO PEOPLE CAN INTERACT WITH WHAT YOU HAVE PRODUCED AND START MANIPULATING THINGS AT A VERY EARLY STAGE. >> SO IMPLEMENTATION GUIDELINES I THINK THAT'S ALSO DOCUMENT, REVOLVING AROUND THE DOCUMENTATION, IT'S VERY COMPLEX, THAT CAN BE AND SOMETIMES THERE ARE BETTER WAYS FOR GOOD INFORMATION, SO THAT'S AGAIN ADDING A LAYER OF HOW TO IMPLEMENT SITUATION PRACTICE INTO EXISTING STANDARDS, SO SAY LOOKING AT WAYS WE CAN ALSO PUT THIS CURATION INTO THIS ACTIVITY, IT'S PROBABLY VERY IMPORTANT SO YES, THE DEPARTMENT AND THE ABILITY, THE AVAILABILITY, FLEXIBILITY AND ABILITY, HOW DO STATUS PLAY WELL WITH OTHER EXISTING IN THE LANDSCAPE SO AGAIN THIS IS THE BIOSHARING HEART AGAIN, THIS IS SOMETHING AFTER THE RESULT WAS VERY IMPORTANT TO REALLY BUILD THIS MAP OF THE EXISTING, WHAT IS OVER THERE, THAT WE CAN ACTUALLY READILY REUSE AND/OR PLUG INTO SO THAT WE DON'T HAVE TO WORRY ABOUT AND WE HAVE THE APIs AND ALL THE COMPANIES READY TO USE. AND FINALLY, I WILL--I WILL FINISH OFF ON THE--I MEAN WHAT STANDARD WORKS BET WEST WITH THE APPLICATION AND AGAIN, SOMEHOW IT'S A CHICKEN AND EGG PROBLEM, BUT WE NEED TO KNOW WHAT EXISTS OUT THERE SO WE KNOW HOW TO REGISTER INITIATIVE IT MAKE IT APPEALING TO PEOPLE TO ENGAGE AND ALSO WE NEED TO CREATE A METRICS FOR THAT, SO IN A BASIC ATTEMPT OF THE BIOSHARING AND TRYING TO GAUGE THE UPTAKE AND TRY TO SEE HOW MANY RESOURCES ACTUALLY RELYING ON THE SPECIFICATION IN THEIR IMPLEMENTATION AND THAT'S VERY LOW HANGING FRUIT AND IT'S ALSO AN AMOUNT OF WORK THAT'S DONE IN CURATION, ACCESSING THE RESOURCE, CHECKING THE DOCUMENTATION TO SEE IF THEY LIKE PARTICULAR CERTIFICATIONS THAT THEY RELY ON. SO ALL THIS KIND OF ELEMENTS ALSO HINT OF SOME OTHER KEY ELEMENT OF THE LIFE CYCLE OF THAT WHICH WOULD BE, HOW PEOPLE DOCUMENT THE LITMUS TEST OF THE STANDARD DEVELOPMENT SO WE GO BACK TO THE USE CASES, HOW CAN THEY--SOFTWARE ENGINEERING IS ALWAYS THE CONTEST AND WE PROVIDE A FEATURE OF THE REQUEST, AND YOU NEED TO DEFINE HOW YOU HAVE THE SITUATION IN CAN THE WHAT AND I THINK HAVING THAT IN A STANDARD DOCUMENTING THE USE CASES, SHOWING PRACTICAL APPLICATION AS A KIND OF WAY TO DOCUMENT WHAT YOU HAVE AIMED FOR, HAS BEEN DELIVERED. SO, OF COURSE, WE WOULD NEED SOMETHING IN NEUTRAL TERRITORY WHERE PEOPLE WOULD SUBMIT A STANDARD, COULD YOU EVALUATE AGAINST THE CERTIFICATION JUST A PROTEIN HERE, AND JUST A BODY, THE BODY COULD BE ARKS DENTIFIED AND EVALUATION OF THE FITNESS OR PURPOSE OF THE FORCE THAT'S CARRYING OUT. I WILL STOP HERE. THANK YOU. >> OKAY SO WHAT I WANT TO DO IS SHARE WITH YOU TWO EXPERIENCES OF THE MANY OF HAVING TO DEAL WITH TERMINOLOGIES AND DATA INTEGRATION AND SORT OF PROVIDE A PERHAPS A SPECIFIC RECOMMENDATION TO THE NIH AS TO HOW THEY COULD HELP US DO OUR JOBS BETTER. SO FIRST IS WORK THAT I DO, THROUGH THE W-THREE Cs LIFE CARE AND LIFE SCIENCES GROUP. I CO-CHAIR THAT GROUP AND WE'RE INTERESTED IN USING SEMANTIC TECHNOLOGIES TO SOLVE PROBLEMS IN LIFE SCIENCES AND HEALTHCARE AND THE INTEGRATION DATA AND REUSE. AND WE STARTED A WHOLE BUNCH OF US GEEKS THAT GATHERAD A WORKSHOP IN JAPAN ANDY WEILS THAT ALL OF US WERE PRODUCE THANKSGIVING DATA IN RDF, BUT WE DIDN'T HAVE DESCRIPTIONS FOR IT AND WE DIDN'T HAVE COLLECTION OF DATA WE WERE DOING AND WE WERE GENERATE THANKSGIVING DAT FLAY THE SAME ORIGINAL DATA FILES AND WE DIDN'T KNOW. SO, WE WE CLEARLY RECOGNIZE THAT WE NEEDED SOME WAY TO STAY HERE ARE THE DATA THAT I HAVE, AND ALSO TO DESCRIBE THEM IN SUCH A WAY THAT WE COULD FIND THAT INFORMATION THAT WE HAD SUFFICIENT METADATA, SO, FROM THAT WORKSHOP, WE BROUGHT THIS INITIATIVE TO DEVELOP A GUIDELINE, A RECOMMENDATION FOR HOW TO DESCRIBE DATA SETS USING OUR TECHNOLOGY PLATFORM CALLED RDF. AND, I REMEMBER DISTINCTLY THAT WHEN WE SAT DOWN AFTER THE INITIAL MEETING IT WAS LIKE, YEAH THIS, IS GOING TO TAKE US A COUPLE OF MONTHS, TOPS, THAT'S IT. WE'RE JUST GOING TO DO A QUICK SURVEY, WE APPROXIMATELY PUT IT IN THE DOCUMENT, PUSH IT OUT THERE AND WE'RE DONE. AND NOW WE'RE TWO YEARS LATER AND WE'RE HOPING, I'M HOPING, I'M REALLY HOPING THAT NEXT WEEK WE ACTUALLY PUBLISH THE DRAFT RECOMMENDATION. WHAT DID WE DO? SO FIRST WE INVITED THE MEMBERS OF OUR COMMUNITY, WE HAVE A HUNDRED PEOPLE ON THE MAILING LIST, WE ASKED THEM TO PARTIC MATE CONFERENCE CALL AND 25 PEOPLE SHOWED UP. AND SORT OF ARTICULATED HERE'S THE DATA, HERE'S THE METADATA WE CARE ABOUT AND HERE'S THE DATA WE THOUGHT THINK OUTTO BE ASSOCIATED AND THEN WE SAUR VEYED EXISTING VOCABULARY CABULARS, THE RDF VOCABULARIES SO WE CAN USE THEM TO AMBIGUOUSLY IDENTIFY THE ELEMENT AND DESCRIBE THEM USING OUR FORMAT, OKAY? SO WE DID OVER 12 VOCABULARIES AND WE LOOKED AT WHAT DID THEY OFFER AND WE MASK THAD WITH WHAT THEY WERE LOOKING FOR SO YOU CREATE AID MATRIX AND THEN WE WENT ROW BY ROW AND WE VOTED. WE SAID EVEN IF THERE'S OVERLAP, THERE'S MULTIPLE VOCABULARY CABULARS AND THEY PROVIDE A RECOMMENDATION FOR THIS TERM. WE PICKED ONE AND SO, IT'S OUR COMMUNITY RECOMMENDATION, RIGHT? OTHER PEOPLE CAN DO IT DIFFERENTLY BUT FOR THIS COMMUNITY, WE JUST WANTED TO DO IT TOGETHER AND TO TO DO IN ONE WAY. SO WE DID THAT, THAT ALL WENT RELATIVELY WELL, BUT OVER THE COURSE OF TWO YEARS, WITH WEEKLY TELECONFERENCE CALLS DEALING AT THE PROBLEMS THAT NOT EVERYBODY COULD BE THERE EVERY WEEK AND SO ON. GAPS EMERGED, RIGHT? MONTHS WOULD GO ON AND THEN, YOU WOULD ASK PEOPLE TO DO SOMETHING THAT WOULD COME BACK AND YOU HAVE TO KEEP PROMPTING THEM AND SOMETIMES YOU YOURSELF GOT BUSY, I WOULD GET BUSY AND SO O. SO THIS JUST DRAGS ON AND ON AND ON AND ON AND. AND I HAVE TO TELL YOU THAT A TWO MONTH THING GETS TO BE A YEAR YEAR THING WHEN YOU DON'T HAVE SOMEBODY IN CHARGE. I SHOULD HAVE BEEN IN CHARGE OBVIOUSLY BUT I'M BUSY I'M TRYING TO DO RESEARCH AND I'M TRAINING PEOPLE AND SO ON, SO I THINKER MY SPECIFIC RECOMMENDATION IS THAT TO SUPPORT THESE INITIATIVES THAT THIS COMMUNITY FEEL SYSTEM IMPORTANT, I THINK WE NEED TO DEDICATE PROJECT STAFF TO MANAGE IT. AND I'LL GIVE OF AN EXAMPLE, I WORK WITH IN THE CONTEXT OF THE W-THREE C, I AM THE REPRESENTATIVE FOR STANDIFORD AND I'M WORK NOTHING THIS RDF WORKING GROUP AND THERE'S A GUY IN THERE, HE'S AWESOME. HE LIKE--HE HAS THIS SCHEDULE. HE KEEPS TRACK OF THE ISSUES. HE NAILS YOU DOWN ON STUFF, AND THERE'S A TIMELINE. AND HE'S THERE, HE TAKES THE VOTE, YOU'RE THERE, NOT THERE, WE KEEP MOVING. AND I LOVE IT. IT'S SUCH A REFRESHING CHANGE AND I THINK THAT'S WHAT SHOWS TAKEN--THEY WE AS SCIENTISTS, WE'RE SUPER INCLUSIVE WE DON'T WANT TO HURT ANYBODY'S FEELINGS BUT WE CAN'T DO THIS IN THIS STANDARDS EFFORT. WE NEED TO HEAR IF YOU'RE AVAILABLER IF NEURNOT, WE NEED TO MOVE ON SO OUR HOPE IS THAT AS WE LEARN HOW ON DO THIS, RIGHT? WE WILL GET BETTER AT IT AND WE WILL BE MOARKS EFFICIENT AT IT AND HOPEFULLY WE REACH OUT TO GREATER AND GREATER COMMUNITIES TO SOLICIT INPUT AND TO REVISE OUR RECOMMENDATIONS IN SIX MONTHS AND 12 MONTHS AND ON A CONTINUING BASIS. SO THE SECOND EXPERIENCE THAT I WANT TO SHARE WITH YOU, IS OUR WORK IN DATA INTEGRATION AND SO WE RUN A PROJECT CALLED BIOTWO RDF, WITH WE CREATE THE DATA SCIENCES SO WE TAKE THE DATA, HAVE THE FILES, XMLFILES AND RELATIONAL DATABASES, AD HOC FILES AND TURN IT INTO THE RDF, SO WE CREATE A BIG CONNECTED GRAPH OF DATA SO THAT LETS US NAVIGATE FROM DATABASE TO DATABASE WITHOUT GOING THROUGH FORMS OR, YOU KNOW SPECIAL APIs OR ANYTHING LIKE THAT. SO WE THINK IT'S REALLY COOL AND WE USE THAT FOR OUR RESEARCH: SO I HAVE THIS COMMENT HERE WHERE, WE I WOULD TALK TO PEOPLE AND I WOULD SAY WHAT DO YOU AND CAN THEY WOULD SAY, WELL IT'S KIND OF LOW QUALITY? WHAT DO YOU MEAN LOW QUALITY? I JUST TOOK SOMEBODY'S DATA AND I TURNED INTO THIS OTHER FORMAT AND THEY'RE ALL LINKED TOGETHER AND ISN'T THAT AMAZING? AND THEY'RE LIKE YEAH, IT'S NOT OUT OF DATE, AND I'M THINKING THE PEOPLE PROVIDED THE DATA SO ARE YOU SAY HANDWRITING YOU RELEASE THE DATA IT WASN'T GOOD QUALITY, RIGHT? SO THERE'S A BIT OF A PROBLEM THAT EFFORTS LIKE HOURS THAT ATTEMPT TO INDGREAT DATA SOMEHOW WE HAVE TO KEEP UP WITH THE LATEST DATA IN ORDER TO BE HIGH QUALITY, WELL THAT'S ONE MEASURE OF HIGH QUALITY BUT I'LL TELL YOU RIGHT NOW, WHEN WE DO THIS DEMONSTRATION PROJECT WHEN WE PUBLISH THE DATAOT SERVERS FOR PEOPLE TO UNDERSTAND THE TECHNOLOGY AND SEE WHAT IT'S ABOUT, I DO THIS ONE A YEAR. OKAY? AND SO ONCE A YEAR, EVERY SINGLE STRIP THAT WE USE, WE HAVE TO UPDATE BECAUSE THE DATA PROVIDERS CHANGE THEIR NORMAT. EVERY SINGLE ONE OF THEM AND SOME OF THEM WHO I SHOULDN'T NAME DO IT EVERY OTHER WEEK. HOW CAN WE POSSIBLY INTEGRATE THE WORLD'S DAT ABYEEE MEDICAL DATA WHEN I HAVE THIS ONE PREJUDICE ECTOMYOSIN WITH 35 DATABASES AND I CAN'T DO IT EASILY. SO, MY SPECIFIC RECOMMENDATION IS THAT WAAD DATA INTEROPERABILITY INTO DATA SHARING PLANS AND IT HAS TO BE THERE, IT HAS TO BE THE CASE THAT WE, THE COMMUNITY WHO USE AND WANT TO REDUCE DAT DON'T FACE THIS BARRIER, IT NEEDS TO CHANGE. THAT'S IT. THANKS. >> ALL RIGHT, I'M THE THIRD GUY IN AN AFTERNOON PANEL IN A ROOM FULL OF SMART PEOPLE SO BECAUSE WE'VE BEEN THINK BEING THIS FOR A WHILE, I WANT TO TRY TO INVERT THE QUESTION A BIT. I LET ME GIVE YOU BACKGROUND TO WHAT'S GOING ON HERE. MOST OF THE THE WORK THAT WE DO IS INVOLVED IN THE SECONDARY USE OF DLINICAL DAT AFRONS REGISTRY DATA, OF PATIENT REPORTED DATA, FOR CLINICAL RESEARCH OR QUALITY IMPROVEMENT PURPLE ONS, SO I WHAT I WANT TO DO IS THINK A LITTLE BIT ABOUT WHAT ARE THE REALLY PRAGMATIC CONSIDERATIONS THAT STAND IN THE WAY OF CREATING STANDARDS AND STANDARDIZING DATA BECAUSE, YOU KNOW AS--EXCUSE ME AS JAMES SAID EARLIER, IF YOU GET EVERYBODY TOGETHER IN A ROOM AND YOU WIPE THE BOARD CLEAN AND YOU SAY, OKAY, IN THE ABSTRACT, WHEN IS A GOOD STANDARDIZE DAT ATHE ANSWER IS YES, RIGHT? UNLESS, UNLESS, I HAVE TO GET THAT RO-1 IN, NEXT WEEK, RIGHT? UNLESS I NEED TO GET A PROGRESS REPORT DONE, UNLESS I NEED TO FIGURE OUT WHERE THE REST OF THE BUDGET'S GOING TO GO, UNLESS I REALLY NEED TO KNOW ABOUT DATA X IN EXCRUCIATING DETAIL AND SO ON. SO WHAT ARE THOSE UNLESSS, AND WHAT ARE THOSE KINDS OF THINGS WE KEEP RUNNING INTO AS A COMMUNITY AS BARRIERS TO MOVING FORWARD WITH THE STANDARDIZATION THAT WHEN WE ALL GET TOGETHER AND TALK IN THE ABSTRACT IS A GOOD THING. ALL RIGHT, SO I'M GOING TO START EASY, THIS IS NOT SO MUCH A POINT FOR THIS GROUP BECAUSE I THINK, IT'S KIND OF ASSUMED IN THE ROOM, BUT IN PRACTICE, ONE OF THE FIRST PLACES THAT WE SEE PEOPLE STUMBLE IS AROUND THE NEED FOR COMMON TERMINOLOGIES OR EVEN AROUND STARTING A CONVERSATION ABOUT COMMON TERMINOLOGIES AND I MEAN THAT NOT NECESSARILY BECAUSE THE STANDARDS DON'T EXIST, BUT BECAUSE WHEN YOU START TO HAVE A CONVERSATION ABOUT THE SEMANTICS, THE STANDARDS COME IN WITH THEIR OWN INFORMATION MODEL OR EACH WITH HIS INSPIRATION MODAND HE WILL YOU NEED TO HAVE THE STEP ZERO CONVERSATION HA DO YOU WANT OUT OF A COMMON TERMINOLOGY, YOU KNOW WHY? DO WE NEED A STANDARD HERE? I THINK, YOU KNOW EVEN THE ABSTRACT, YES, BUT AGAIN FROM MY WORLD, SOME OF THESE REASONS ARE SORT OF CRISTICAL CLEAR, YOU KNOW ONE I'LL GIVE A CLASSIC EXAMPLE THAT WE SAY FAIRLY FREQUENTLY, HAVE YOU A COUPLE OF VERY LARGE PROSPECTIVE STUDIES, I'M A PEDIATRICIAN, WE LOOK AT BIES A LOT, LOOKING THE AT CHRONIC LUNE DISEASE AND IN NEOINATES. ENROLLED HUNDREDS OF PATIENTS IN EACH COHORT AND LOOKED AT THE EFFICACY OF A PARTICULAR TYPE OF OXYGEN THERAPY, FOR PREMATURE BABIES WITH CHRONIC LUNG DISEASE AND CAME TO OPPOSITE CONCLUSIONS. THEY SET OUT TO ANSWER THE SAME QUESTION, THEY USED VERY SIMILAR METHODS, THEY HAD A SLIGHTLY DIFFERENT DEFINITION OF THE--OF THE INTERVENTION, THE SLIGHTLY DIFFERENT DEFINITION OF THE OUTCOME AND SLIGHTLY DIFFERENT WAY AND WHAT YOU ENDED UP WITH WERE TWO VERY LARGE PILES OF NONINTERROPEERABLE DATA THAT CAME TO DIFFERENT CONCLIEWPGZS AND ANALYZED AND AND ISOLATION AND THERE WAS NO WAY TO RECRECONCILE THE TWO. SO UNLESS AGAIN YOU'RE THE PERSON WHO GOT TWO RO-1S OUT OF THIS, THIS WAS NOT A GOOD THING. ALL RIGHT? BUT, IN ORDER TO GET TO THE--THE STANDARDIDESSATION, NOT ONLY DO YOU NEED THE EFFORT OF TERMINOLOGY, PUTTING IN THE EFFORTS OF THE STANDARDIZING BUT YOU NEED TO THINK ABOUT WHAT'S THE POINT OF THE COMMON TERMINOLOGY, OR WHAT ARE THE THINGS YOU WANT TO KACCT TININE PATHWAY NUR THOSE TERMINOLOGIES. IN PARTICULAR, IN OUR WORLD, HOW DO SECONDARY USE PLAY INTO THIS, RIGHT? SO WE'RE CONSTANTLY DEALING WITH THE DIFFERENCE BETWEEN SOMEONE WHO IS TRYING TO RECORD DATA BECAUSE THEY'RE TAKING CARE OF PATIENTS, AND SOMEONE WHO IS TRYING TO RECORD DATA BECAUSE THEY'RE TRYING TO CONDUCT THE STUDY, AND THAT GOES TO THE QUESTION, WHAT STANDARDS DO YOU USE, RIGHT? BECAUSE PART OF WHAT WE WANT TO FIGURE OUT IS OKAY, IF YOU WANT A STANDARD, IF WE NEED TO USE TERMINOLOGIES AT LEAST WE AGREE ON A SET OF TERMINOLOGIES, WE AGREE ON A LANGUAGE, WHERE ARE THE PLACE WHERE IS WE REALLY DO NEED TO INVEST THE EFFORT IN VERY CLOSE ALIGNMENT AND WHERE ARE THE CASES WHERE YOU REACH THE POINT OF DISCIPLINARY MINERBING RETURNS EARLY BECAUSE THERE'S ENOUGH PLAY IN OTHER ASPECTS OF THE QUESTION YOU'RE ASKING THAT IT'S NOT WORTH A PRIORI FIGURING OUT WHETHER YOU HAVE A RESEARCH READY DEFINITION OF BPD OR HYPERTENSION OR OBESITY. THAT SORT OF THING. THE OTHER PIECE OF THIS IS WHEN YOU TALK ABOUT TERMINOLOGIES, WHERE DOES THE INFORMATION LIVE, RIGHT? WHERE HAVE PEOPLE BEEN ACCUSTOMED TO RECORD THANKSGIVING AND YOU KNOW BOTH IN THE SEMANTIC SPACE OF TERMINOLOGY AS WELL AS ORGANIZATION OF YOUR DATA SET AND THAT GOES TO THE QUESTION OF HOW MUCH OF YOUR STANDARDIZATION SPECIFIC DETAILS ON YOU NEED TO INVEST IN STANDARDIZING YOUR METADATA OR AT LEAST IN MAKING YOUR METADATA EXPLICIT SO THAT, YOU CAN ACTUALLY CONNECT THE SEMANTIC CONCEPTS, THROUGH YOUR METADATA. ALL RIGHT SO THAT'S CONTERMINOLOGIES SO THE SECOND THING THAT I THINK GOES HAND IN LOVE WITH THE QUESTION OF DATA STANDARDIZATION IS GENERATEDIGATING AGREEMENTS METHODS FOR CHARACTERIZING THE DATA ONCE THEY MOVED INTO THE STANDARD OR IDEALLY BEFORE AND AFTER, RIGHT? SO, SOME OF THIS IS A QUESTION AGAIN OF DATE OF PROVIDENCE AND LOOKING AT THE OPERATING CHARACTERISTICS OF DATA THAT COME FROM DIFFERENT SOURCES AND FIGURING OUT WHEN THOSE OPERATING CHARACTERISTICS REALLY SKEW THE WAY THE DATA ARE BEHAVING AND BUT SOME OF THIS IS ALSO REALLY SORT OF VERY PRAGMATIC POINT THAT YOU DON'T NEED TO JUST GET YOUR DIAGNOSIS IN OUR WORLD FOR INSTANCE, TRANSFORMED INTO A AN ONTOLOGY LIKE SNOMED THAT GIVES AWE COMMON SET OF CONCEPT IDENTIFIERS FOR THINGS BUT YOU NEED TO HAVE AGREEMENT ON HOW TO ASSESS HOW THAT TRANSFORMATION WENT. RIGHT? AND I'M NOT NECESSARILY PROPOSING THAT THERE HAS TO BE ANOTHER STANDARDIDESSATION PROCESS, BUT THERE HAS TO BE AN EXPLICIT DISCUSSION AND THERE HAS TO BE REACHING CONSENSUS IN THE COMMUNITY ABOUT THE KINDS OF DATA CHARACTERIZATION THAT ENABLE YOU AS A USER TO SAY, YES, I HAVE A DATA SET THAT CONFORMS THAT THIS FORMAT WITH THIS SET OF MATTA DATA MAKE IT T I WANT. LOOK LIKE IT'S DESCRIBING THE DOES IT OPERATE IN THE WAY I WANT. AND THEN THE THIRD THING THAT I WANTED TO CALL OUT WAS, EXCUSE ME, THE INTERACTION BETWEEN THE VAL YOU OF STANDARDIZE AGES BECAUSE IT PROVIDES THE ABILITY TO TRANSIT ACROSS DATA SETS OR KRASES DATA SOURCES IN ABLE TO DO ANALYSIS MORE EFFICIENT LEER OR IN A MORE GENERALIZABLE WAY WITH THE NEED TO ACTUALLY INTOPERATE SOME DOMAIN SPECIFIC REQUIREMENTS, ALL RIGHT? AND THIS IS IN SOME WAYS [INDISCERNIBLE] AND NOT. TAKEN TO IT'S SORT OF FULLY EXPRESSED CONCLUSION, WE VALID NO STANDARDS BECAUSE IN MY STUDY, I NEED TO HAVE THIS, THIS PARTICULAR ELEMENT DEFINE INDEED PRECISELY THIS WAY, PERHAPS BECAUSE IT'S CRUCIAL TO MY OUTCOME OR PERHAPS BECAUSE I CAN, I HAVE THE ININSTRUMENTITATION TO MEASURE LEVEL AT THIS DETAIL AND SO WE WON'T TALK TO EACH OTHER. SO IN PRACTICE, IT IS POSSIBLE TO REACH AGREEMENT ON A LOT OF THE SPECIFICS OF STUDIES PARTICULARLY IF THE DATA ARE WELL CHARACTERIZE INDEED ADVANCE AND YOU CAN UNDERSTAND WHERE THE GAPS MAY BE HIDING IN THINGS THAT LOOK LIKE THEY CONFORM TO A STANDARD AND YOU CAN USE IT IN A WAY YOU NEED TO WITHOUT COMPLETELY JETTISONING THE OPTION OF USING STANDARDS, RIGHT? AND SO, I PICK THESE UP FROM PEDIATRICS AGAIN BECAUSE I THINK ABOUT KIDS A LOT, ONE OF THE BARRIERS WE HAVE TO GETTING PEOPLE WHO DO CLINICAL TRIALS AND DRUG STUDIES TO ADOPT COMMON DATA MODELS OR DATAA DESCRIPTIONS AROUND DRUG TRIALS ARE THAT MOST OF THE EXISTING STANDAR ARE DEFINED AROUND ADULT USAGE OF MEDICATION, RIGHT? AND THERE'S JUST, ONE OF THE THINGS THAT'S BAKED INTO MOST OF THOSE MODEL SYSTEM FLAT DOSING, RIGHT? WHICH IS NOT MEANINGFUL, WHEN YOU TALK ABOUT KIDS. RIGHT? I'LL PICK A VERY SIMPLE EXAMPLE, LYNN WAS HERE TALKING ABOUT PC ORI'S COMMON DATA MODEL AND THE SPRING BOARDING OFF THAT, THE FIRST STUDY LOOKING AT THAT IS THE LOOKING AT THE RELATIVE VALUE OF LOW DOSE VERSES HIGH DOSE ASPRICK DOSING TO PREVENT CORINARY ARTERY DISEASE, LEAVING ASIDE THE CORONARY ARTERY DISEASE IS NOT A MAJOR MORBIDITY IN KIDS; EVEN SO THE TRIAL IS COMPARING 40-MILLIGRAM VERSES 8-MILLIGRAM. THAT HIGH VERSES LOW IF YOU'RE AN ADULT. WHICH IS THE LOW DOSE IN YOU'RE A FOUR KILOINFANT. SO PART OF "IS WHEN YOU ASK HOW SUCCESSFUL IS YOUR STANDARD IS HOW AMENABLE IS IT TO EXATTENTION TO THESE ADDITIONAL ANALYTIC USE CASES WITHOUT HAVING TO BEND THE STANDARD OUT OF SHAPE AND MAKE IT NOT--NOT USEFUL FOR INTEROPERABILITY. ALL RIGHT, I'LL STOP THERE AND WE CAN MOVE ON, THANKS. >> HELLO, I'M A REPEAT OFFENDER TODAY. [LAUGHTER] WHEN THEY TOLD ME SINGLE SLIDE, I TOOK IT LITERALLY SO THAT'S PRETTY MUCH THE SLIDE I SHOWED THIS MORNING ALREADY. I'M GOING TO EMPHASIZE THE LAST ONE TALKING MOST OF WHAT I HAVE TO SAY HAS BEEN SAID ALREADY SO I WILL EMPHASIZE A COUPLE OF POINTS. DO WE NEED ADDITIONAL STANDARDS AND I THINK THAT IT'S BEEN SAID ALREADY BUT WE NEED TO--REUSE, IT CAN BE AS SIMPLE AS, YOU KNOW MAKING THESE THINGS TO DISCOVER AND TO ADOPT, BUT SOMETIMES IT GOES TO MAPPINGS AND SO BOTH MAPPING AND POST DOOREDINATION BASICALLY THE SIMPLE IDEA IS THAT PUTTING THE CONCEPTS TOGETHER YOU CAN CREATE NEW ONES. SO, THE NOTION OF THAT'S MYOCARDIAL INFARCTION, HAVE MYOCARDIAL INFARCTION AND YOU STICK THEM TOGETHER. IT'S NOT AS EASY AS IT SOUNDS BECAUSE IF YOU WANT POST DOORED NATION TO BE DONE, PUTTING TOGETHER, NEEDS TO OBEY SOME RULES, OF SYNTAX AND SEMANDATORYIC SPECIALIZATION OF SPECIFIC ENDOTHELIAL THAT'S WHERE IT GOOD LUCKS DIFFICULT AND HOW--IT BECOMES DIFFICULT TO DO AND IT'S BETTER THAN CREATING SUM OTHER INFARCTION BECAUSE IT DOESN'T EXIST IF THE TERMINOLOGY YOU WERE USING AND YOU NEED IT BECAUSE OF THE MAIN SPECIFIC GRANULARITY THAT'S REQUIRED FOR A GIVEN STUDY, SO, ALL THESE THINGS ARE WORKED ON CONSIDERING AND SOME TERMINOLOGIES ARE ONTOLOGIES SUCH AS [INDISCERNIBLE] FOR EXAMPLE, ARE BUILT WITH MOST COORDINATION INSIDE IF YOU WISH THERE, 'S ALL THE POSSIBILITY OF CREATING POST COORDINATION, I'M NOT SAYING IT'S EASY, IT IS AWFULLY HARD AS I MENTION, BUT IT IS POSSIBLE WHEREAS WITH OTHER TERMINOLOGIES, IT'S IN A NUT SHELL. SOMETIMES IT'S ALSO A MATTER OF FACILITATING USE OF THE TRMINOLOGY ITSELF OR THE ONTOLOGY AND AGAIN IT'S NOTHING NEW OR THAT HASN'T BEEN SAID BEFORE SO WE THAT ARE CREATING THESE ARE NECESSARY TO CONSUME BY THE END USERS AND WE MAY WANT TO THINK OF DUMBING DOWN, QUOTE-UNQUOTE THESE ARTIFACTS SO THEY ARE EASIER TO LOOK AT AND AND THEY ARE CONSUMED BY THE END USERS, THAT'S ONE THING. IS IT THE SPECKIBILITY OF CREATING SERVICES AROUND THOSE TERMINOLOGIES, I'M TALKING ABOUT APIs MEKSED THIS MORNING AND I JUST WANT TO SHARE WITH YOU MY EXPERIENCE WITH RXNORM, WITH THE RXNORM IS THE NERMALITATION SYSTEM FOR THE DRUG, SO THE DRUGS ARE VERY FROM THE BANKS AND SNOMED--IT CAME IN A DATABASE FORMAT WHERE WE USE THE FORMAT, SO ALTHOUGH THEY HAVE A USEFUL LOGICAL STRUCTURE IT'S A GLASS MODEL MA MAKE ITS USEFUL WHEN YOU LOOK AT IT, AND IT'S TABLE FORMATIONS WHEN YOU NEED TO DO A LOT OF DRAWINGS THAT YOU FIND YOUR WAY FOR THE PHOTOGRAPH IF YOU WISH, IT MAKE ITS DIFFICULT FOR USER WHO IS HAVE GOOD TECHNOLOGY, MAKE ITS VERY DIFFICULT FOR THEM TO FIND THEIR WAY AROUND. SO WE CREATED A FEW APIs AND A FEW FUNCTIONS TO EXPOSE THE LOGICAL STRUCTURE AND SO LESS THAN 10 YEARS AGO, THAT'S SLOWLY ABDUCTED. FOR A VARIETY OF USES, INSTITUTIONAL USER WHO IS DECIDED TO USE IT INSTEAD OF USING THE DATABASE BECAUSE IN PART BECAUSE THE DATABASE IS DATED WITH GOOD-OF GOOD SET OF UPDATES AND WITH WEEKLY ADDITIONS ON TOP OF THAT SO IF YOU WANT TO USE THE DATABASE, YOU ARE ALWAYS LOADING THE STATS INTO YOUR SYSTEM. SO THE NEWS IS ALSO BY AND MODEL APP DEVELOPERS SO THE POINT BEING THAT IT'S NOT THE MATTER OF CREATING A HUGH ONE, IT'S A MATTER OF RETHINKING THE WAY WE PRESENT THE INFORMATION, WE DISTRIBUTE INFORMATION AND MAKE IT EASIER FOR PEOPLE TO GET AT IT, AND THE LAST POINT I WANT TO MAKE, HAS TO DO WITH I'VE HEARD DIFFERENT THINGS. AND THE LAST THING IS MONEY, AND HOW WE WILL PAY FOR THINGS. SO WE PAY PEOPLE TO DETERMINE OUR ONTOLOGY OR TERMINOLOGY, AND THAT'S THE CASE WITH SNOMED, CTM AND THE HSDO, AND OTHERS. AND ESCAPE, AND THE PRICE TO PAY, AND IT'S DONE HERE BUT THE TYPE TO PAY FOR THAT IS THAT, THEY ARE NO LONGER OPEN SOURCE. AND THERE NEEDS TO BE MECHANISM FOR US TO GET ACCESS TO THEM. SO IN THE U.S., EVERYBODY CAN ACCESS SNOMED BECAUSE YOU ARE A MEMBER OF THE CONSORTIUM WHICH MEANS THAT MOONS THAT OUR ROLE PLAY, AND BUT, BUT, IF YOU WANT TO ACCESS SNOMED, IT DOESN'T COST YOU ANYTHING, BUT YOU NEED TO SIGN THE UML LICENSE AGREEMENT AND IT'S ALSO A BARRIER TO ACCESSING THE TERMINOLOGY BECAUSE ALTHOUGH THEY'RE FREELILY AVAILABLE, YOU CANNOT GET TO THEM DIRECTLY AND WE NEED TO HAVA I TOKEN TO THE API, TO GET ACCESS TO THAT TO SHOW CREDENTIALS TO GET ACCESS, SO THESE ARE SOME OF THE ISSUES THAT HAVE BEEN TALKED ABOUT LITTLE BIT THAT THAT'S HAVING BEEN COMPLETELY EXPLICITLY MENTIONED IF YOU WISH AND I'M NOT CLAIMING I HAVE A SOLUTION OR RECOMMENDATION BUT THESE ARE THINGS THAT ARE TAKE INTO ACCOUNT WHEN WE YIELD WITH THE OPEN SOURCE VERSES THE WEP DEVELOPERS AND THAT'S IT. SO LET'S OPEN IT UP, YET. GO FOR IT. >> $90,000 TO WORK ON OBI, IT'S STILL OPEN SOURCE, STILL ONTOLOGY. I DON'T KNOW [INDISCERNIBLE]. >> I CAN'T RESPOND TO THAT BUT THE RESPONSE OF THAT IS WAY OVER THAT. >> I JUST WANT TO RESPOND TO SOMETHING JAMES SAID JUST ABOUT, YOU KNOW HOW WHEN YOU SHOULD MAKE A STANDARD HOW MUCH EFFORT YOU SHOULD PUT INTO TTHAT DAY IS KIND OF A DEINFECTOR STANDARD AND DIDN'T TAKEN--THEY THAT LONG TO CREATE, AND YOU COULD MAKE SIMPLE STANDARDS FOR THE FIRST PERSON TO CREATE SOMETHING AND THEY CREATE THIS THING AND IT ENDS UP BEING USEFUL THAT'S YOU KNOW, THAT'SA THE POINT IT DOESN'T HAVE TO TAKE YEARS AS WE'VE DESCRIBED IT, SO MAYBE-- >> GREAT, I AGREE THAT'S A VERY INTERESTING CASE BECAUSES TOTAL DEFACTOR AND BILL MADE IT UP AND EVERYBODY ELSE HAD TO USE IT SO, ON THE OTHER HAND PEOPLE ARE WASTING HOURS AND HOURS AND DAYS AND MONTHS TO TRY TO COME TO A CONSENSUS, AND JUST AS A COMMENT IF YOU LOOK AT CHEMICAL SPACE, THERE ARE--THERE ARE LIKE 90 CHEMICAL FILE FORMATS. SO, O O SO IN SOME SENSE, YEAH, IT'S GREAT IF SOMEBODY COMES TO THE SPACE AND DEFINES, A LITTLE COMPONENT BY WHICH YOU CAN ACCESS THE DATA BUT THEN EVERYBODY DOES TOO, AND THEN YOU HAVE, I THINK THE TOOL IS CALLED OPEN BABBLE SO YOU CAN CONVERSE FROM ONE TO ANOTHER THE NOW I DON'T MIND PROVIDED SUCH A TOOL EXISTS AND WE COULD EASILY TRANSFORM FROM ONE TO ANOTHER AND YOU HAVE THIS EXPERIENCE WITH GALAXY AND YOU HAVE INPUT AND OUTPUT FORMAT AND HAVE YOU SOMETHING, LITTLE TOOL TOKEN FROM ONE TO THE OTHER MAYBE YOU'RE GOOD BUT THERE ARE ARGUMENTS WHERE YOU MIGHT BE MISSING INFORMATION OR IF YOU LOOK AT GENO TYPE ENCODE WRING HAVE YOU VERY DIFFERENT DESCRIPTION OF GENOTYPES, AND SO IF YOU HAVE ONE STANDARD MAYBE YOU DON'T HAVE THE FULL TICKETTURE EITHER. AND SO WE DO HAVE TO EVALUATE, I THINK FORMATS, A LITTLE BIT MORE CAREFULLY AND IT'S OKAY IN SOME SENSE, TO THE PATH TAKEN--THEY MAKING SURE THAT THAT'S ADEQUATE FOR MULTIPLE NEEDS, MAYBE? >> THE FIRST RESPECTS IN POST OR CHILD WITH THE CASE WHERE THE TOOL DEVELOPS THE FORMAT AND YOU KNOW THERE'S A GOOD HISTORY THAT THE FIRST COUPLE WIDELY AVAILABLE TOOLS IN THE FIELD TEND TO DEFINE THINGS AND ONE OF THE QUESTIONS BECOMES HOW DO YOU--DO YOU BELL A FORWARD LOOKING STANDARDS DEVELOPMENT METHOD THAT LETS YOU USE THE TOOL WHEN IS YOU HAVE NOTHING MORE THAN CARROT AND A BLOCK OF LETTERS AND YOU KNOW NOT FALL A PART WHEN. WE NEED MORE INFORMATION. >> SORRY I WILL MAKE ONE MORE COMMENT ON THAT. PART OF THE ISSUE OF INTEROPERABILITY FOR DATA IT WAS REFERRED TO THE PREVIOUS SESSION IS THAT YOU HAVE TO THINK ABOUT THE SYNTAX AND THE SEMANDATORYICS BUT IT'S ALSO THE MODEL, RIGHT SO MOST OF US HAVE THIS CONCEPTUAL MODEL ABOUT HOW THINGS ARE CONNECTED TOGETHER, AND HOW YOU ACTUALLY PUT THAT INTO AND MAKE THAT CONCRETELY AVAILABLE TO ANOTHER PERSON, IS EYE CHALLENGE, RIGHT? BECAUSE WE HAVE THESE DIFFERENT FORMATS WHETHER IT'S JASON OR XML OR SO ON, SO THERE'S ELEMENT OF SIN TACTIC OPERATING GLOBALLYERABILITY WHICH IS A BARRIER TO REUSE AND THEN THERE'S CODING DATA WITH OR WITHOUT A TERMINOLOGY SYSTEM AND MOST CASES I WILL TELL YOU FROM EXPERIENCE, MOST DON'T USE A TERMINOLOGY SYSTEM OF ANY KIND IT'S AD HOC, EVEN IF YOU DO USE ONE THEN WE HAVE A VAST CO COLLECTION AND THEN HAVE YOU AN EXAMPLE EVER WHERE YOU HAVE A WHOLE BUNCH OF SIMILAR TERMS AND NUMBER'S DONE THE WORK OF MAPPING THOSE TERMS TOGETHER, SO IF YOU HAVE A SNOMED TERM, YOU CAN MOVE THAT OVER TO THE ONTOLOGY TERM OR AN ICDNINE TERM OR WHATEVER IT IS AND THAT HELPS AND THAT'S PART OF THE ECOSYSTEM THAT WE THEY'D TO FOSTER, IF WE'RE GOING TO LET PEOPLE DO WHATEVER THEY WANT AND WE WILL CULTURE OR CAP, MAKE USE OF WHAT THE RESEARCH EFFORTS ARE. >> SO YOU SAID TWO THINGS ON THE NHIERARCHIES H SLIDE WHICH IS THE IDEA THAT THE PROJECT STARTS AND STAY AND THE PROJECTS GOING ON FOR SOMETIME AND FIRST OF ALL I THINK IT'S GOOD TO SEE THE OPPORTUNITY TO HAVE TWO HITS TO ONE HIT WHICH IS KNOCKED DOWN ON THE WEEDS IN ONTOLOGY OR MODELS OR STANDARDS THAT I MEAN IT'S SOMETHING THAT IT'S SORT OF ALMOST PART OF THE ENABLING FRAMEWORK IN WHICH IT CAN HAPPEN, AND ADMINISTRATIVE THING IF YOU LIKE, I MEAN ONE LEVEL, I THROW THAT RIGHT BACK AND SAY, THAT ANY PROPOSAL CAME INTO NIH THAT SHOWED THAT ASPECT OF BEING THOUGHT ABOUT WOULD BE WELL RECEIVED OF PEOPLE AT NIH LOOK FOR THAT COMPONENT IN THE PROJECTS THAT YOU WOULD SEE AND THE OTHER THING WOULD BE TO EXPAND EITHER WAWAY AND EXPAND IT ON WHAT YOU SEE OR WHAT ARE THE RIGHT KIND OF PEOPLE FOR THAT WHERE THIS SHA ACTIVITY COME FROM, AND SHOULD IT COME FROM WITHIN THE ORGANIZATIONS THAT ARE DOING THE WORK SHOULD IT BE PART OF THE ECOSYSTEM, BECAUSE IT COULD BE SELF-REGULATING IF IT'S ELSEWHERE, TO BE WITHIN THE GOVERNMENT, COULD BE WITHIN CONTRACTORS WITHIN THE AWARDEES, ET CETERA, WHAT DO YOU THINK. >> SO THERE ARE DIFFERENT FACILITIES AND SORT OF A COMMITMENT TO LONG-TERM PUBLICATION CLONING JEFFITY SO WHEN WE CREATE THE DOCUMENT WE PUT IT ON THE WEB SITE, TELL BE THERE FOR HOPEFULLY AS LONG AS THE WEB EXISTS AND SO AND THIS IS IF WE PUT PROPOSALS IN FRONTS OF YOU AND WE SAY, WE WANT TO TAKE THIS EMERGING DATA FORMAT OR TERMINOLOGY OR WHATEVER IT IS AND WE WANT TO PUSH IT TO THE INCOME LEVEL, ALL IT'S TRYING TO SAY IS ALL OF OUR EXPERIENCE SYSTEM THAT WE'RE DOING IT AD HOC AND WITHOUT PROFESSIONAL SUPPORT, THAT COULD VASTLY IMPROVE OUR EXPERIENCE AND MAKE IT WAY MORE EFFICIENT. NOW HOW WE DO THAT, I SEE THE EXPERTS AND I SEE WHAT THEY'RE DOING IN THE WORKING GROUPS I'M IN INTEREST GROUPS SO WE DON'T HVE ACCESS TO THOSE INDIVIDUALS BUT IF WE CAN HAVE A POOL OF SUCH AGENTS THAT CAN HELP US, AND SORT OF MAKE HERE'S YOUR START PLAN AND YOUR MILESTONES AND HERE'S THE WHOLE PLAN AND I'M GOING TO MARSHAL YOU THROUGH IT WITH YOUR COMMUNITY THAT WOULD BE, I THINK INVALUABLE AND I THINK IT'S A FORMULA FOR SUCCESS. THEY'RE JUST GOING TO CHAIR IT AND GET US THROUGH IT. BUT FOR US TO BE RESPONSIBLE FOR IT, CAN YOU SEE WHAT THE RESULTS ARE. >> I MEAN, I'M FULLY GRATEFUL FOR EVERYBODY WHO SPENDS TIME DOING IT BUT REALLY WE SHOULD BE DOING OUR SCIENCE. SO WE NEED SUPPORT IN THIS--IN THIS PARTICULAR CAPACITY AND I'M SURE RYAN CAN PROBABLY SPEAK TO THAT EXPERIENCE. [LAUGHTER] >> IF THE NIH WAS GOING TO SUPPORT YOU, AND THE STANDARDS SHOULD IT BE A RHYME THAT THOSE--ENVIRONMENT THAT THOSE STANDARDS BE FREE OF [INDISCERNIBLE] >> YES, I WANT TO BE CAREFUL HOW A SAY THAT IS THAT THE IMPORTANT HOW I SAY THAT AND I DON'T WANT TO GET TANGLED UP IN DISCUSSIONS THAT ARE HELD IN TRUST BY THE NIH OR SOME OTHER STANDARDS BODY, BUT I THINK WE TALK ABOUT MECHANISMS AND ALL OF YOU MENTION THAT IT COSTS MONEY AND MONEY TENDS TO CLOSE DOORS BUT YOU KNOW ONE OF THE MECHANISMS THAT WE HAVE AS A SCIENTIFIC COMMUNITY TO KEEP THE DOOR OPEN WHEN MONEY IS SPENT IS FUNDING PROCESSES LIKE THE NIH--FOR AD HOC, WHAT DO YOU IS SOME OTHER KIND OF SCIENCE AND IT'S HARD TO WEDGE THAT IN, SOME OF IT TOO IS THAT'S SCIENCE, RIGHT? THERE IS A--THERE'S A BODY OF KNOWLEDGE AND A SET OF PRACTICES AND TESTABLE RESULTS WE DO A DISSERVICE WHEN THAT GETS RELEGATED TO SORT OF THE SPARE TIME OF SOMEONE WHO'S REAL FOCUS IS SOMEWHERE ELSE SO PART OF WHAT WE MAY NEED TO CONSIDER IS TO WHAT EXTENT IS THIS PROCESS SCIENCE IN AND OF ITSELF AND DOES IT MERIT IT'S OWN CONSIDERATION. >> I HAVE TO SAN FRANCISCO SHOULD OF IN DISCUSSION YOU WOULD NOT FIND IN OTHER AREAS OF SCIENCE. I JUST WANT TO REFLECT THAT, IT'S A SHORT HAND AND BIOSCIENCES AND IT'S JUST DATA CRAZY. OKAY? THE IDA, PRODUCES MORE AND MORE DATA AND THE IOM REPORT 10 YEAR AGO THAT SAID THAT BI SOLING A THEORY DEFICIENT AREA OF SCIENCE. BECAUSE IN MANY OTHER AREAS YOU BEGIN TO ANSWER THESE QUESTION SYSTEM WHERE DO WE NEED DATA, HOW COULD THIS BE THESE QUESTIONS BY STARTING WITH THE THEORY AND THE TEST. YOU START WITH THE THEORY AND THE IDEA IS THAT TESTING COMPOSING AND IN A SUSPECT WAY TO DEVELOP A SCIENCE. SO YOU START THERE. AND THEN YOU DERIVE THE QUESTIONS LIKE, YOU KNOW IF YOU'RE ON A--DOG THE PARTICLE ACCELERATOR ON THE POSTS AND IF YOU START WITH LOOKING FOR AN EXPOSEON, WHAT DATA DO WE NEED, YOU WILL DECIDE WHAT TO EXPOSEON AND THOSE ARE THEORIES, SO I WANT TO SUGGEST THAT WE'RE GETTING INTO THIS, I THINK WE'RE GET INTERESTING THIS IS HOW DO YOU DECIDE WHAT DATA YOU NEED AND WHAT WE'RE DOING THIS FOR, AND I THINK YOU NEED FRAMEWORK IN SAYING WHAT IS THE THEORY WE'RE TRYING TO TEST AND PUT IT THAT WAY BECAUSE THOSE ARE THEENTIOUS FICIENT WAY TO DECIDE WHAT DATA YOU NEED BECAUSE OTHERWISE YOU PRODUCE WHAT YOU SEE IN THE UNIVERSITY LIBRARY, IS EVERY BIOLOGY COURSE, THE THOUSAND PACKAGE TEXTBOOK, AND THEY KEEP GETTING BIGGER, EVERY INITIATIVE. SO, YOU NEED TO BE ABLE TO GET THROUGH I THINK, E-EQUALS MC SQUARED AND FORMULATE A RICHER INCENTIVE AND SET OF THEOR EVERIES AND THAT MAY HELP YOU, RESOLVE THE DATA QUESTIONS. PROVOCATIVE QUESTIONS. >> WOW, SO I'LL ASSURE YOU THAT BIOLOGIST DOS HYPOTHESIS DRIVEN SCIENCE IN THE CONTEXT OF THEORY. THAT IS UNAMBIGUOUSLY TRUE AND WE COLLECT KES AND MORE OVER EMORE IMPORTANTLY I THINK WHEN PEOPLE THIS IS SUPPORT REALLY AN ISSUE FOR US, THE BIGGER ISSUE IS HOW DO WE DEAL WITH THE HUGE DIVERSITY OF DATA THAT WE COLLECT AS A RESULT OF THE HUGE NUMBER WITH A THING. >> CAN WE ADDRESS THAT PINT AND HISTORICALLY CONTINGENT AND THERE ARE NO BRAND THEORIES BECAUSE EVERY TYPE OF ORGANISM DOES THINGS DIFFERENTLY IN THE WEEK. AND THEY ALL DO THINGS DIFFERENTLY. YOU CAN'T START WITH THE EXPOSE ON, AND THEORY AND GO BUILD A MILLION DOLLAR COLLIDERS THAT FIND IT, IT'S A DIFFERENT BEAST. SO IT'S ALSO WHY SO MANY KINDS OF BUSINESSES COME INTO THE FIELD AND SAY WE'RE GOING TO SOLVE ALL YOUR PROBLEMS FOR YOU AND THEY SOON LEAVE WITH THE TAILS BETWEEN THEIR LEGS. >> ALL RIGHT WE'RE TOWARD THE END SO I WILL TAKE THE LIBERTY OF LAST QUESTION AND BRING US DOWN FROM PHILOSOPHY OF SCIENCE TO SOMETHING VERY SPECIFIC SO I WOULD LIKE TO ASK THE PANEL AND MICHELLE YOU PUT UP AN ANSWER BUT YOU'RE WELCOME TO ANOTHER ONE. >> WHAT I HEARD IS THAT WE NEED A PROJECT MANAGER, BUT BUT AS YOU NEED TO KNOW WHAT TO WORK ON AND EACH HAVE A REGISTRY OF THINGS THAT ARE THERE, AND THEN IDENTIFY THE GAPS AND FOR THOSE THAT EXIST. >> THIS WILL SOUND LIKE SOMETHING FOR ME TOO AND THIS WILL FALL BACK ON THE SUGGESTION THAT THAT ONE OF THE THINGS I NEED TO DO IS THINK ABOUT STANDARDS IN THEIR OWN RIGHT AS OPPOSE TO THE LAST TWO SCIENCES OF DATA MANAGEMENT PLAN AND YOU KNOW THAT--THAT GETS US TO A BETTER PRODUCT. >> I'M NOT SURE IT'S F IT'S APPROPRIATE FOR ME TO GIVE RECOMMENDATIONS FROM MY INSTITUTION BUT I'M GOING TO DO IT NEVERTHELESS. [LAUGHTER] NO, I THINK WHAT'S USEFUL IS KIND OF THING THAT WAS THE DEMONSTRATED EARLIER, SO IF YOU WANT PEOPLE TO USE DATA, ELEMENTS, THE FIRST THING YOU NEED TO DO IS PROBABLY TO PROVIDE SOME KIND OF REPERTOIRE AND USEFUL INFORMATION AND DEMONSTRATION TO USE. SO BASICALLY TRYING TO START THE PROCESS OF COLLECTING THESE THINGS AND EXPOSING THEM AND ALIGNING THEM, AND THEN, THE COMMUNITY WILL PICK UP THE GOOD ONES AND WITHOUT THE OTHERS, I THINK THAT GOES INTO THE RIGHT DIRECTION. ALL RIGHT, SO-- >> I WANT TO ADD IF WE DO THAT WE HAVE IT BE CAREFUL WE DON'T CREATE STANDARDS FOR STANDARDS SAKE, THEY'RE USE CASE AND BIOSHARING IS ASSOCIATE WIDE STANDARDS SO THEY KNOW WHY THE STANDARDS THERE. SO. >> THIS IS HARD TO DOCUMENT SO THAT THEY BE A RECOMMENDATION, DOCUMENTED USE CASE SYSTEM SPECIFICALLY AND CLEARLY. THIS IS REALLY THE LAST POINT THEY WANT TO FOCUS ON, WHICH IS EVALUATION. AND I FEEL LIKE WE--YOU KNOW WE JUSTIFY OUR STANDARDS BUILDING WITH THE PROMISE OF UTILITY. AND I THINK WE HAVE TO FOLLOW THROUGH, AND WE HAVE TO DEMONSTRATE WITHIN SOME PERIOD OF TIME, SO CAN YOU IMAGINE THAT PHASE ONE IS BILLED THE STANDARD, PHASE TWO IS EVALUATE THE STANDARD AND YOU START TREAT THANKSGIVING MORE AS AN ENTERPRISE, RIGHT AND WE GET SERIOUS ABOUT THIS, WE WILL DO REALLY WELL AND WE WILL LEARN THINGS. ONE OF THE THINGS THAT I THINK HAS LARGELY BEEN OMIT FRIDAY OUR EFFORT SYSTEM THE INCLUSION OF SOCIAL SCIENTISTS IN OUR STANDARDS DEVELOPMENT AND THEIR EVALUATION AND SHOW WE SHOULD BE STUDYING OW PEOPLE USE THIS AND WHY THEY DON'T USE CERTAIN THINGS AND WHAT WOULD MAKE IT EASIER AND SO ON. ALL OF THIS WILL HELP US IN THE LONG-TERM, BUILD THE BETTER STANDARD OF THE FUTURE WHEN THAT COMES AROUND. >> LAST COMMENT. >> AT THE RISK OF BEING PROVOCATIVE, THE NOTION OF USE CASE DRIVEN STANDARD SYSTEM A VERY SLIPPERY SLOPE PARTICULARLY WHEN WE THINK IN THE CONTEXT OF NIH AND WITH THE WHOLE DATA INTEGRATION, UNANTICIPATED QUERIES ALIKE BECAUSE A CONTRARIAN VIEW, SO IT'S CONTARRIAN, IS THAT PERHAPS WE NEED STANDARDS THAT CAN UNAMBIGUOUSLY REPRESENT DATA FULL STOP WITH THE EXPECTATION THAT WE CAN'T EVEN ANTICIPATE WHAT KIND OF USES THE DATA WILL BE PUT TO. >> ALL RIGHT. LOOKS LIKE THAT'S AN OKAY PLACE TO STOP FOR FOLKS, SO IF YOU DON'T MIND THANKING THE PANEL. [ APPLAUSE ] >> I SENT COORDINATOR FORS THE GROUPS, FIND A CORNER AND WE'LL COME BACK TOGETHER AND REPORT ON OUR BREAK OUTS FOR THE FIVE MINUTE BREAK AND THEN, BREAK OUTS FOR THOSE WATCHING THE WEBCAST, AROUND 4:30, FOR THE SUMMARY? RIGHT? YEAH. SO CAN YOU WATCH AT 4:30. OKAY, SO, I GUESS WE'LL GET STARTED. SO WE TALKED ABOUT A LOT OF OF THINGS AS DID A LOT OF OTHER GROUPS AS L. THERE'S A LOT TO HAVE TALK ABOUT. ONE OF THE TOPICS THAT JENNY BROUGHT UP TO ME FOR A SHORT TIME WAS WHEN IN THE RESEARCH CYCLE DO STANDARDS COME TO PLAY AND FOR BASIC RESEARCH SCIENCE, IT USUALLY HAPPENS WHEN YOU EITHER GO TO SUBMIT A GRANT REPORT PROPOSAL AND YOU HAVE TO WRITE A DATA MANAGEMENT PLAN OR YOU GO TO PUB LIBRARY A PAPER AND YOU'RE JOSHAL TELLS YOU HAVE TO DO SOMETHING MORE THAN YOU'VE DONE AND WE CONJECTURES THAT IT REALLY OUGHT TO HAPPEN EARLIENER THE CYCLE AND THAT, I THINK WE KNOW THAT AND WE DIVERGED INTO A LONG DISCUSSION ABOUT EDUCATION AND HOW DO WE EDUCATE RESEARCHERS ON HOW TO INCORPORATE THIS INTO THEIR RESEARCH LIFE CYCLE AND YOU KNOW THE THING IS THAT, THE VALUE OF WHAT RESEARCHERS DON'T REALLY SEE THE VALUE OF USE OF STANDARDs BECAUSE MOSTLY THEY'RE NOT ACTUALLY REUSING PEOPLE'S DATAA SO MUCH YET AND THERE'S REAL NEED. SO ONE RECOMMENDATION IS THAT WE NEED TO REALLY SHOW THE UTILITY NOT JUST IT CAN'T BE A STICK, THERE HAS TO BE A REAL CARROT THERE, THAT YOU WANT TO BE PART OF THAT ECOSYSTEM BECAUSE YOU KNOW IT YOU'RE A GOOD CITIZEN THEN YOU'LL GET MORE BACK OUT OF IT AND YOU WILL GET RSS FEEDS ON YOUR DATA, YOU WILL GET OTHER PEOPLE ADDING THINGS TO YOUR DATA ANDRY USE THE DATA THOSE KINDS OF THINGS. ONE OF THE THINGS THAT CAME UP THIS WAS IDEA THAT OUR STUDY SECTIONS REALLY DON'T HAVE, INFORMATION SCIENCE EXPERTISE ON THEM AS AWIVE AS WE MEET LIKE AND THAT WHEN WE BRAINSTORM SOME RECOMMENDATIONS AROUND THIS, I WILL SCROLL DOWN FOR A SECOND. THIS IS NOT REALLY ORGANIZED. I'M NOT SPEAKING IN ORDER HERE. SO BASICALLY THAT THE STUDY SECTIONS NEED TO INCLUDE REVIEW OF DATA MANAGEMENT PLANS AND INFORMATION SCIENCE EXPERTS AND THEN WE BRAINSTORM SOME NONTRADITIONAL WAYS THIS CAN BE DONE. SO MAYBE IT'S ACTUALLY A SEPARATE PANEL SO TO REVIEW THE DATA MANAGEMENT PLANS BY A PANEL OF INFORMATION SCIENTISTS, EXPERTS. MAYBE THAT'S A LOT OF OVERHEAD, MAYBE WE GO TO A PROCESS WHERE THERE'S PRESCREENING OF GRANTS FIRST THAT IF THEY MAKE THE GRADE, THEN THEY HAVE TO DEVELOP T MANAGEMENT PLANS. MAYBE EVEN THEY DON'T SUBMIT A DATA MANAGEMENT PLAN AT ALL AND IT'S PART OF THEIR FUNDING TO WORK WITH AN ASSIGNEDKS PETTER OR--ASSIGNED EXPERT OR THEY FIND THEIRON EXPERT AND THAT'S PART OF THE MECHANISM AND THAT'S PART OF THE GRANT ITSELF IS TO HAVE A DAT DATABASE THEA MANAGEMENT EXPERT INVOLVED IN THE GRANT. SO THAT'S A RECOMMENDATION OF CHANGING THE WAY WE INCLUE THOSE THINGS WITHIN THE GRANTING PROCESS. SO COMING BACK TO THE EDUCATIONAL PIECE, WE YOU KNOW BOTH CHRIS AND I HAVE BD2K R25S FOR EDUCATIONAL DEVELOPMENT OF EDUCATIONAL MATERIALS, MANY ANOTHERS IN THE ROOM ARE PARTICIPATE NOTHING THAT. YOU KNOW AND ONE OF THE GOALS OF THAT IS REALLY TO EDUCATE RESEARCHERS BUT NOT NECESSARILY TO EDUCATE THE EDUCATOR SPECIALIZATION OF SPECIFIC ENDOTHELIAL ONE RECOMMENDATION IS TO HAVE THAT PROGRAM AND THINK ABOUT WAYS IN WHICH WE CAN GARNER THOSE ENERGIES, ROUND EDUCATING, YOU KNOW WHAT WE REFER TO AS CONCIERGES AND THOSE CONCIERGES LIVE IN LIBRARIES THAT HELP EDUCATE THEIR RESEARCH COMMUNITIES ABOUT THE DIFFERENT LANDSCAPE OF STANDARDS AND MATCH THEM TO THE STANDARDS THAT MEET THEIR DATA MANAGEMENT NEEDS. AND THEN, LET'S SEE, IDIDN'TIFY WHERE TRANSLATIONAL--I THINK I MISSED THAT ONE. WHO ELSES WANT TO SPEAK TO THAT ONE? THAT ONE HAPPENED AFTER I LEFT IDENTIFY WHERE TRANSLATIONAL RESEARCH IS BLOCKED OR HAS CONFLICTING STANDADS. OH, OKAY. SO MAYBE THAT'S OUR TOPIC FOR TOMORROW. OUR TOPICS AND OUR RECOMMENDATIONS ARE STILL MESSED UP. THRU WAS ANOTHER RECOMMEND ABOUT NEEDING TO CREATE A TIGHT CONNECTION BETWEEN THE DAILY BASIS ATA DISCOVERY INDEX AND THE STANDARDS INDEX AS WELL AS THE DATA SO THAT THOSE AND REALLY WHEN YOU THINK ABOUT THE EDUCATIONAL PIECE THAT ALSO HAS TO FIT IN THERE, TOO BECAUSE THEY HAVE TO HAVE ACCESS TO THAT FULL LANDSCAPE IN ORDER TO PROVIDE THE EDUCATION SO THAT WAS ANOTHER RECOMMENDATION THAT THERE SHOULD BE A TIGHT COORDINATION BETWEEN, YOU KNOW STANDARDS DEVELOPMENT CAN'T EXIST WITHOUT THE DATA TO EVALUATE FUNCTIONING AND YOU KNOW WHETHER OR NOT WE'VE GOTTEN OVER THE CURVE OF USEFULNESS AND SO, THOSE NEED TO BE TIGHTLY COORDINATED. DID I FORGET ANYTHING, SHERRY? DOES ANYBODY WANT TO COMMENT ON ANY OF THOSE BEFORE WE MOVE ON TO THE NEXT GROUP? >> IT'S REALLY GREAT, I THINK ONE SIDE THOUGH OF THINKING ABOUT HOW WE GO ABOUT EMBEDDING THAT IN THE STUDY SECTION SYSTEM A REAL OPPORTUNITY TO EDUCATE FOR SUBMITTING GRANTS WHAT WE EXPECT FROM THEM. BUT THEN ALSO AN OPPORTUNITY TO RAISE THAT AWARENESS INSIDE STUDY SECTIONS AS WELL, SO THAT'S REALLY IMPORTANT AND THEN SOMETHING THAT I'VE SEEN OVER AND OVER AGAIN, LOOKING ACROSS THE STUDY SECTIONS THROUGHOUT NIH, IS THAT WE PARTICIPATE IN HOW WE APPROACH THIS AND THIS IS AN OPPORTUNITY TO GET IN. >> - >> SO--STAY AFTER THE FACT. >> THAT'S TRUE BUT BUILD BUILDING THAT AWARENESS IS PART OF IT. THAT HAS AN IMPLICATION AND IS AN ATTENTION TO IT, THAT WITHOUT HAVING IT WRITTEN DOWN AS A REQUIREMENT, PEOPLE WILL WEIGHT AND PUT IT OFF. SO I THINK THERE'S--IT'S NOT NECESSARILY THAT THERE'S A RIGHT AND WRONG, SO I SAID ON NHGRI STUDY SECTIONS FOR A LONG TIME AND I APPRECIATE THEM BECAUSE THEY PAID MORE ATTENTION TO THEM. AND WHAT THAT--THAT HAD REAL RAMIFICATIONS OF PEOPLE FOR THOSE OF US WHO WERE REVIEWING PAID ATTENTION TO THE FACT THAT THEY HAD ONE. DIDN'T REALLY PAY AS MUCH ATTENTION TO THAT AS A GOOD PLAN OR A BAD PLAN WILL BE SUCCESSFUL BUT THEY HAD A PLAN AND THEY THOUGHT ABOUT HOW TO DISSEMINATE INFORMATION. SO I THINK AGAIN WE WANT TO BE CAREFUL WHAT WE ASK FOR BUT ASKING FOR SOME KIND OF DATA PLAN IS REALLY IMPORTANT. IN HOW THEY WILL USE STANDARDS. >> TWO COMMENTS, ONE IS IT SEEMS HARD TO GET PEOPLE WHO ARE REALLY EXPERTS IN INFORMATION SCIENTISTS AREN'T NECESSARILY PIs AND INVITED TO THE REVIEW PANELS SO THAT'S ONE ISSUE THAT WE VHOW TO INVOLVE THE ACTUAL IN THE TRENCHES INFORMATION SCIENTIST WHO IS ARE THE PEOPLE WHO ARE GOING TO FIX THE DATA LATER EVEN IF THEY MET THE DATA MANAGEMENT PLAN, THEY HAVEN'T ACTUALLY--THOSE AREN'T NECESSARILY THE SAME PEOPLE. THE SECOND THING IS, ASK WE INCLUDE A DATA MANAGEMENT PLAN BUT ALSO SOMEHOW INCLUDE THIS IDEA THAT YOU WILL HAVE SOME PERSON THAT WILL BE WORKING WITH YOU THAT IT'S NOT--IT'S NOT A ONE TIME THING THAT YOU DON'T--DATA MANAGEMENT PLAN SHOULD CHANGE AS YOU GO ALONG, AS YOU DO YOUR SCIENCE, YOU GET DIFFERENT DATA AND NEW STANDARDS COME OUT AND THINGS CHANGE AND MAYBE WE CAN HAVE OW CAKE AND EAT IT TOO AND DO BOTH OF THOSE THINGS. >> GOOD TO HAVE A DATA MANAGEMENT PLAN, THAT'S IMPORTANT BUT A LOT OF DATA MANAGEMENT PLANS I'VE STEIN HAS BEEN ON GROUPS I'VE BEEN PART OF, IS GOOGLE SHARE, WITH THE DATA SO WE PUT IT ON A SERVER, YOU NIGHED TO GO BEYOND THAT, AND THINK ABOUT THE STANDARDS AND THE PEOPLE REVIEWING THE GRANTS ARE NOT IN A POSITION TO TALK ABOUT THAT. >> WE TALKED ABOUT THIS INURE GROUP, THAT MIGHT BE A MORE CONCRETE THING WE DID DO RIGHT NOW TO SAY HERE ARE THE THINGS WE THEY'D TO SEE OUT OF A DATA MANAGEMENT PLAN OR DATA SHARING PLAN. >> I OUR PLUG FOR PI DAY AND WE NEED TO HAVE DIFFERENT KINDS OF INVESTIGATORS COMING IN. AND EXPERTS. >> ALSO, [INDISCERNIBLE]. SNOW LONGER iPEOPLE, YOU WANT PI-PEOPLE. >> OKAY, SORRY, PI-DAY IS MARCH 14th. YES. BUT THERE'S--A DISCUSSION OUT THERE IN THE COMMUNITY ABOUT HOW WE TRAIN PEOPLE AND WHAT WE DO IS WE--WE RATE PEOPLE WHO ARE iS SO THEY'RE DEEP IN ONE--THEY'RE VERY--THEY DON'T HAVE MUCH KNOWLEDGE OF THAT, SO THE IDEA OF A PI PERSON IS THEY HAVE TWO DOMAINS AND ONE NEEDS TO BE COMPUATION. AND BUILDING IN THAT AWARENESS OF [INDISCERNIBLE]. AS LONG AS WE STILL HAVE COMPUTERS AND WE NEED TO BE TRAINED. AGAIN THAT'S SOMETHING THAT YOU CAN START TO DO IN MANY DIFFERENT WAYS. ONE OF THOSE SUBTLE REMINDERS, YOU KNOW WHAT YOU, YOU YOU NEED COMPUTATIONAL PEOPLE TO GO AWAY. >> THIS PIECE OF HAVING A DATA SCIENCE IS PART OF TRADITIONAL RO-1S CAME UP IN OUR GROUP AS WELL AND HAVE IT BE A COMPREHENSIVE PIECE OF RO-1. >> I WANT TO SECOND THAT. I WANT TO BE CAREFULLY'M I'M NOT SPLITTING SYMANTIC HAIRS BUT IT STRUCTURALLY ME AS A LITTLE OFFCENTER WHEN THEY WE WANT TO REVIEW THIS SEPARATELY FROM THE SCIENCE THIS, IS NOT PART OF THE SCIENCE. RIGHT? BECAUSE AT SOME LEVEL, IT IS PART OF THE SCIENCE, RIGHT? IT'S PART OF THE SCIENCE LIKE YOUR HUMAN SUBJECT SAFETY PLAN IS PART OF THE SCIENCE SORE THAT SORT OF THING. SO I TOTALLY GET THAT YOU WANT IT SAY, LET'S LOOK AT HOW YOU'RE MANAGING THE DATA SEPARATELY FROM DID YOU GENERATE A VALID HYPOTHESIS, DID THE EXPERIMENTS YOU PROPOSE ADDRESS THE HYPOTHESIS, OR ARE THEY FEASIBLE, BUT IT'S ALMOST AN OPTICS THING TO SAY IT'S NOT PART OF THE SCIENCE D THAT MAKE ANY SENTENCE? >> [INDISCERNIBLE]. >> OR DO IT YOURSELF IF THAT'S YOUR SKILL SET BUT WHEN YOU ASSEMBLE THE TEAM WHO WILL SEE THIS THROUGH TO COMPLETION AND PRIOR TO WHAT YOU NEED TO ACCOUNT FOR, DOES THIS PIECE GO TOO. >> WE HAVE 30 SECONDS TO REVIEW THE OTHER TWO GROUPS. SO,--I WAS ABOUT TO SAY WRAP IT UP. >> SO I WILL TRY TO NOT TO HIT ON THE TWO MAIN TOPICS WE'VE ALREADY HIT ON AND HIT ON THINGS WE TALKED ABOUT THAT HAVEN'T BEEN BROUGHT UP, ONE THAT WE'VE HEARD A LOT ABOUT IS HOW WE NEED A SHORTER LIFE CYCLE. HOW IT'S TAKEN WAY TOO LONG TO DEVELOP THESE STANDARDS AND MAYBE WE SHOULD DISRUPT THE ENTIRE WAY TO DEVELOP STANDARDS, AND DISRUPT THE MODAND HE WILL ONE WAY TO DO THAT IS TO PARTNER MORE WITH INDUSTRY. BUT ONE THING THAT CAME UP ABOUT THAT IS MAKING SURE WHO--WHERE THE IPs RESIDES WHENEVER YOU'RE WORKING WITH INDUSTRY BECAUSE THAT'S ONE WAY TO GET THEM TO THE TABLE IF YOU'RE CLEAR WHERE WHERE THE INTELLECTUAL PROPERTY RESIDES. WE ALREADY TALKED ABOUT MORE SPECIFIC DATA, WE TALK ABOUT EDUCATION, LEADERSHIP IS NEEDED IN STANDARD DEVELOPMENT SO WE TALK ABOUT HOW PEOPLE SORT OF--OH NIH NEEDS TO START SO THE JALS NEED TO DO THAT, IT NEEDS TO COME FROM THE SCIENTIST, SO WE NEED EVERYONE TO COME TOGETHER AND SORT OF A SAY, HERE'S WHAT IT NEEDS TO BE BECAUSE EVERYONE'S PUNTING THE BLL TO SOMEONE ELSE TO TAKE THE LEADERSHIP ROLE. >> AND THEN AS FAR AS BEST PRACTICES GO, WE DECIDED THAT IT'S ALL ABOUT OPENNESS, SO, STANDARDS HAVE TO BE OPEN SO THAT PEOPLE CAN MAKE CAREERS OUT OF OPEN DATA STANDARDS AND THAT DATA NEEDS TO BE OPEN SO THAT'S SORT OF WHERE WE LANDED. ALEXIA DID I MISS ANYTHING BIG THAT YOU WANT TO HIT ON THAT HASN'T BEEN TOUCHED ON? >> [INDISCERNIBLE] >> RIGHT. >> [INDISCERNIBLE]. >> SO WE TALKED ABOUT INNOVAVE WAYS TO MAKE SOME OF THIS HAPPEN SO IT'S CAPTURED HERE IN THE NOTES BUT WE'RE OUT OF TIME. >> THAT'S THE PROBLEM OF BEING THE LAST ONE A LOT OF POINT VS BEEN MADE ALREADY. >> [INDISCERNIBLE]. >> EXACTLY. YES. >> [LAUGHTER] UNLESS YOU'RE IN THE ROOM. BUT I WILL SAY THE THING IS THAT WE--THE GROUP HERE WE HAD A FANTASTIC GROUP ACTUALLY SO WE COVERED THE LONG DISCUSSION ABOUT BASICALLY HOW TO DEFINE THE SCOPE OF THE EFFORTS SO AT ONE POINT, THAT WAS RELATED TO ONE WHEN PEOPLE STARTED IN THE PART OF THE DEVELOPMENT OF THE AVOID THE SLIPPERY SLOPE AND ALL THESE THINGS SO WE HAD A DIGS CUSHION ABOUT WHAT CAPTURES SOME OF THESE MODELS THAT ARE PRODUCED AND THAT WAS THE POINT BY THAT WAS MADE ABOUT THE RISK OF SPECIFICATION, ON THIS PARTICULAR INTERESTING POINT I THINK LEADS TO THE HOW TO ASSESS THE STANDARDS AND THAT POINT OF REGULATION OF THE RESOURCES, AND WHAT WE WANTED TO COMMAND AND IBT--INTEGRATE CLIEWGZ EVER UNIQUE TESTS LIKE SYSTEMS, SO WHERE THE USE CASE WOULD BE ALWAYS PRESENT IN THE IDENTIFICATION DOCUMENT, DOCK ULTRAVIOLET BING THE SPECIFICATIONS AND ALSO THIS WOULD BE COUPLED AGAINST TWO INDENT IMPLEMENTATIONS OF THE STANDARD BY TWO DIFFERENT USES AND THAT WOULD SHOWCASE BUT THIS CAN BE DYSFUNCTIONAL AND DELIVERIES SO THESE TWO WERE FROM OUR INTERTWINED DISCUSSION THAT WE HAD FOR SOMETHING ACTIONABLE, TRACEABLE AND FUNCTIONAL. STILL UNRELATED TO THE--SO THIS WAS THE POINT I WAS MAKING ABOUT THE UNITEST, AND AGAIN, THE POINT THAT WAS MADE BY RYAN, SO THIS IMPLEMENTATION SHOULD BE MADE FREELILY AVAILABLE. BUT THAT'S A GOOD POINT. SO WE WE DID KUSSED WAS SOME DISWOGS ABOUT THE INCLUSION OF WHICH DATA SHOULD BE USED IN THE TERM NATION PLAN, THIS IS THE PART OF THE [INDISCERNIBLE] AND RAISING THE AWARENESS WHEN PEOPLE [INDISCERNIBLE] RO-1S PROPOSALS THAT THEY HAVE A GOOD UNDERSTANDING OF WHAT THE STANDARDS ARE AVAILABLE IN THEIR FIELD AND TO EXPLAIN AT LEAST WHY THEY WOULD NOT USE THE STATUS SO THIS IS ABOUT CREASING THE LOOP OF INTERACTION AND THOSE STANDARDS AND USERS CONSUMERS, AND WE IDENTIFIED TWO KINDS OF CONSUMERS WITH PEOPLE ACTUALLY ACQUIRING THE DATA, THE ONE COLLECTING AND THE OTHER KIND OF USERS DOWN STREAM, PEOPLE CONSUMING DATA FOR DATA MINING AND THE ANALYSIS AND THEY WOULD COMPLAIN, I COULDN'T FIND THIS DATA ELEMENT AND THIS ELEMENT BUT GOING BACK TO THE POINT OF THE ELIMINATION PLAN, I THINK HAVING A LIST OF WHICH STANDARD WE USE AND HOW SERIOUSLY THE DETERMINATION PLAN IS TAKEN, BUT ALSO TO HAVE A FEEDBACK LOOP IN THIS EFFORT, THAT PEOPLE ARE ABLE TO SEND FEEDBACK ABOUT WHY WE ARE USING THE STANDARD, IT IS ENOUGH OR IT PROPOSES WOTOO MANY CONSTRAINTS WHICH ARE NOT ACCOMMODATED IN THE FUNDING LEVELS WE OBTAIN, THE COST OF COMPLYING TO THE STANDARD AND PRODUCING DATA WITH THE STANDARD SHOULD BE FACTORED IN THIS THE KIND OF WORK THAT THE SCIENTISTS ARE DOING THIS, I A NEW TASK ON THE SHOULDER. AND THIS IS SHOULD BE WORRIED ABOUT WELL. SO KIND OF INTERACTION OF THE FEEDBACK MECHANISMS THAT WERE NEEDED AND [INDISCERNIBLE]. WOULD BE AN IMPORTANT METHOD RECOMMENDATION WE WOULD MAKE TO THE NIH. AND YES, AND AGAIN, LINKS TO INIT CANNATIVES AND THE BY O BIC, AND SO THEREFORE, IN A WAY ASSESSING NOW THESE HAVE BEEN PUBLISHED AND REALLY BY SCIENTISTS THAT ACTUALLY WE USED TO THESE TWO DIFFERENT FORMS OF THE TYPE OF MUTATIONS, DO THEY COMPLY WITH THESE AND KONT REALLY HELP REUSE AND AGAIN AND THIS IS A CLOSING THE LOOPS AND THIS IS TALKING ABOUT HOW DO WE ASSESS THE SITUATION, AND THIS SAY MESSAGE WE WERE DISCUSSING IN THE GROUP. ANY QUESTIONS? >> OKAY, THANK YOU. >> OKAY, SO WE ARE GOING TO CLOSE THE SESSION NOW AND WE WOULD HOPE THAT ALL OF YOU WOULD CONTINUE TO CONVERSE OVER COCKTAIL HOUR AND DINNER AND CONTINUE TO JOT YOUR IDEAS DOWN IN NAGOOGLE DOC, IT'S--THAT GOOGLE DOC, TELL BE HELPFUL IF WE CAN ORGANIZE THE CLEAR RECOMMENDATIONS BY THE END OF THE MEETING. WE ARE STARTING AGAIN AT 8:30 IN THE MORNING TOMORROW. DO WE HAVE ANY OTHER ADMINISTRATIVE ANNOUNCEMENTS, ASTRID? OKAY, AND THERE'S DINNER VEESERVATION, YOU SHOULD RE--RESERVATION, YOU SHOULD RECEIVE AN E-MAIL ABOUT THAT IF YU HAVEN'T LET US KNOW YOU WANT TO COME, PLEASE LET ASTRID KNOWS WE WOULD LOVE IT IF EVERYONE COULD JOIN US. THAT WOULD BE GREAT. OKAY, LET'S GIVE A ROUND OF APPLAUSE FOR ALL OF YOUR EFFORTS TODAY. THANKS. [ APPLAUSE ]