CAPTIONS START LATER TO IMPROVE ACCESS TO AND QUALITY AND SAFETY OF CARE FROM THE PATIENT, PHYSICIAN OR SYSTEM PERSPECTIVE SO MEASURING THE QUALITY OF CARE, THE SAFETY OF CARE TO PARTICIPATE IN RESEARCH, TO DEVELOP NEW THERAPIES AND PARTICIPATE IN RESEARCH TO EVALUATE AVAILABLE THERAPIES OR SYSTEM LEVEL APPROACHES, THAT'S PCORI'S BUSINESS, TO COMPARE AVAILABLE THERAPIES TO FIND OUT WHICH TREATMENTS WORK THE BEST FOR WHICH PATIENTS. THOSE ARE AMONG THE KEY DRIVERS AMONG THE SEDATION TO START A REGISTRY. THESE TEND TO BE LED BY SPECIALTY SOCIETIES. THERE'S ONE HERE, THIS IS A COALITION OF 21 DIFFERENT PHYSICIAN SPECIALTY SOCIETIES, AND THIS IS A NETWORK OF SOMEWHERE UPWARDS OF 60 PHYSICIAN ORGANIZATIONS USUALLY SPECIALTY ORGANIZATIONS, WHO HAVE BUILT REGISTRIES FOR ONE PURPOSE OR ANOTHER. THESE ARE JUST OUTPATIENT CARDIAC REHABILITATION G.I. DIGESTIVE HEALTH RECOGNITION PROGRAM REG CITY AND RENAL, DRIVEN BY MEDICARE REQUIREMENTS THAT YOU'LL HAVE AN EASIER TIME GETTING PAID IF YOU'RE PARTICIPATING IN A REGISTRY, DEVOTED TO PERFORMANCE IMPROVEMENT, DRIVEN BY THE STRONG DESIRE OF THE MEMBERS OF THESE ORGANIZATIONS TO HAVE A REGISTRY TO CONTRIBUTE TO. WHEN I FIRST GOT TO D.C. 4+ YEARS AGO AS HEAD OF PCORI, AMONG MY FIRST VISITORS WERE A LARGE NUMBER OF PHYSICIAN SPECIALTY ORGANIZATIONS, EXPECTING THAT PCORI COULD PROBABLY HELP THEM FUND THE REGISTRY, AND IT DID NOT TAKE ME TOO LONG TO REALIZE THERE WERE 60, 70, 80 OR REGISTRIES. IT WAS VERY SAD, DISAPPOINTING TO THE PHYSICIAN SPECIALISTS THAT PCORI COULDN'T SUPPORT ALL THESE. REGISTRIES TAKE A LOT OF SUPPORT. THEY ARE NOT CHEAP AND CONTINUE TO TAKE MONEY OVER TIME. JUST AN EXAMPLE OF ONE, TO SHOW YOU HOW SPECIALIZED THEY GET, THIS IS ONE THAT'S CONDUCTED BY TWO OF THE OLDEST REGISTRY BUILDING SPECIALTY SOCIETIES COLLEGE OF CARDIOLOGY, AND THEY HAVE MULTIPLE REGISTRIES BETWEEN THEM AND LINKED BUT THIS ONE IS ABOUT TRANSVENOUS -- TRANSCATHETER VALVE REPLACEMENT IS WHAT IT IS, TWO SPECIALTY SOCIETIES FOLLOWING THE OUTCOMES OF PEOPLE WHO GET ONE PARTICULAR PROCEDURE. THAT'S ABOUT ALL I'M GOING TO SAY ABOUT PHYSICIAN ORGANIZATION DRIVEN REGISTRIES FOR THE MOMENT. I WANT TO TALK NOW ABOUT PCORnet, WHICH WALTER MENTIONED. THIS IS A PCORI EFFORT, WE'RE AIMING TO BUILD A CLINICAL RESEARCH NETWORK NATIONALLY BASED ON TWO INTEGRAL TYPES OF MEMBERS, CLINICAL RESEARCH NETWORKS, WE FUNDED 11, WE'RE NOW UP TO 13, AND EACH ONE OF THOSE COMMANDS A POPULATION, MANY MILLIONS, IT STARTS WITH ELECTRONIC DATA ON MILLIONS. 18 NETWORKS, I WILL SHOW YOU THE NETWORKS IN A MINUTE, THESE TEND TO BE LIKE REGISTRIES, SOMEBODY ALREADY IN EXISTENCE, SOMEBODY BEING BUILT WITH PCORI FUNDS STARTED BY PATIENT ORGANIZATIONS, ADVOCACY ORGANIZATIONS, OR BY RESEARCHERS PARTNERED WITH PATIENT ORGANIZATIONS. ANOTHER THING THAT'S UNIQUE ABOUT PCORnet, WE AIM TO SUPPORT CLINICAL TRIALS AND BIG DATA OBSERVATION, AND RESEARCH TO BE CONDUCTED ON ANY CONDITION OR QUESTION WITH ENHANCED POWER, ACCURACY AND EFFICIENCY AND BRINGS HEALTH CARE SYSTEMS INTO THE PROCESS OF RESEARCH SO WE AIM TO BRING PATIENTS IN BUT ALSO THEIR CLINICIANS AN DELIVERY SYSTEMS, HEALTH PLANS, IN THE GOVERNANCE OF PCORnet AND RESEARCH IT CONDUCTS. WE SEE THE WORLD WILL BE DIFFERENT, CLINICAL TRIALS AND OBSERVATION STUDIES WILL BE MORE EFFICIENT, PARTICULAR BY BY PARTNERING WITH DELIVERY SYSTEMS AND USING ELECTRONIC HEALTH RECORDS DATA, WE WILL SUPPORT MULTI-CENTER STUDIES, SO THE SIZE IS AN IMPORTANT FACTOR FOR US, WE'LL CAPITALIZE ON SHARED INFRASTRUCTURE, COMMON DATA, POLICIES AND PROCEDURES. A GREATER COMPARABILITY, WE'LL PUSH IN EVERYTHING WE DO TOWARDS SHARED DEFINITIONS, COMMON DATA MODELS AND STANDARDS. RESEARCH PRIORITIES WILL EVOLVE, CRUCIAL IS CONSULTATION, IT'S IN OUR MISSION, CONSULTATION WITH PATIENTS, CLINICIANS, DELIVERY SYSTEMS, PAYERS AND PURCHASERS. THEY ARE CONTRIBUTING TO THE RESEARCH TO ALSO BELIEVE THAT IT'S WORTH DOING, THE RESEARCH IS WORTH DOING, HOPEFULLY THE FINDINGS, IF THEY ARE DEFINITIVE, WILL BE EASIER TO DISSEMINATE AND IMPLEMENT. SO HERE I JUST WANT TO SHOW, AND I KNOW THAT THE REPRESENTATIVE FROM AT LEAST TWO, I BET MORE OF THESE PPRNs ARE IN THE AUDIENCE ON MULTIPLE SCLEROSIS, THESE ARE TEN RELATIVELY COMMON DISEASES, AND I'LL GIVE YOU A MINUTE TO LOOK DOWN THE MIDDLE COLUMN SO YOU SEE THE CONDITIONS, NINE OF THEM HERE, THE ONLY NEUROLOGIC ONE IS MULTIPLE SCLEROSIS. ADRENOLEUKOLEUKODYSTROPHY IS IN THE AUDIENCE. PAUL GROSS IS A PARTICIPATE IN GENETIC ALLIANCE, DOZENS OF DISEASES, SOME NEUROLOGIC, IMMUNODEFICIENCY, MUSCULAR DYSTROPHY, DUCHENNE, AUTISM AND SEIZURES, AND VASCULITIS. AND THEN WE JUST ADMITTED FIVE NEW PPR, INs, PARKINSONS, FOX INSIGHT, AUTISM AND ALZHEIMER'S, IN ADDITION TO THE COMMUNITY AND PATIENT-CENTERED PATIENT PARTNER CENTERS OF EXCELLENCE, A LOW INCOME COMMUNITY, VERY LARGE LOW INCOME COMMUNITY MOBILIZED, THE LAST IS PRIDE PPRN, RESEARCH IN IDENTITY AND DISPARITIES FOR EQUALITY, BUT IT'S GAY, LESBIAN, BISEXUAL, TRANSGENDER COMMUNITY EXPLORING THE RESEARCH -- HEALTH RELATED RESEARCH QUESTIONS OF THAT COMMUNITY. OUR FIVE NEW ONES. YOU SEE QUITE A BIT IN THE AREA OF NEUROLOGIC DISORDERS. IN ADDITION, THE CDRNs THAT I MENTIONED HAVE A COMMON COHORT AND A RARE COHORT THAT THEY ARE VERY INTERESTED IN, HERE I WILL JUST POINT TO -- LET'S SEE, ALS AS ONE OF THE RARE DISEASES RELATIVELY RARE DISEASES THAT ONE OF OUR CDRNs IS BUILDING A VERY LARGE COHORT IN, SO A LOT OF ACTIVITY IN NEUROLOGIC DISORDERS. WHAT DO WE TELL THE PPRNs? YOUR MAIN JOB IS TO GROW. THIS IS WHAT IT LOOKS LIKE NOW. HERE IS THE CDRN, HERE IS THE CDRN, THE PPRN HAS MODEST OVERLAP WITH MEMBERS GETTING NEAR CLINICAL CARE IN THE PURVIEW OF THIS CDRN AND PROBABLY THIS ONE TOO. WE TELL THEM TO GROW, PARTICULARLY TO GROW INTO THE CDRN. SO PARTICIPATION IN PCORnet IS AN OPPORTUNITY FOR PPRNs TO PARTNER AND GET LARGER. WE HAVE EXAMPLES WHERE THEY ARE BEGINNING TO REACH OUT, CAN YOU IMAGINE REACHING OUT TO KAISER PERMANENTE WITH LET'S SAY MULTIPLE SCLEROSIS. THE POINT HERE IS THAT ONCE YOU'VE GOT ALL THE HEALTH CARE DATA THROUGH A CDRN IT MAKES SENSE FOR REGISTRIES TO LOOK AT THAT AND GATHER THE INFORMATION THEY NEED. PCORnet, ALL 13 NOW CDRNs, CIRCLED BY THE HUNDREDS AND THOUSANDS OF REGISTRIES IN THE U.S., WE WOULD LOVE TO SEE THE DAY WHEN MANY OF THOSE REGISTRIES PARTNER WITH PCORnet TO ACTUALLY EXPAND THEIR REACH TO GROW AND CONDUCT RESEARCH TOGETHER. IT MAY BE THAT PCORnet -- ANY CDRN IN PCORnet DOESN'T HAVE ENOUGH PATIENTS WITH THIS RARE DISORDER BUT WORKING WITH THE REGISTRY WE CAN GROW THE REGISTRY AND GET THE SAMPLE SIZES THAT ARE BIG ENOUGH, AND ADD ALL THE DATA ON COMORBIDITY AND OUTCOMES THAT THE REGISTRIES MAY NOT BE SO ABLE TO GET. SO EFFICIENCY IN COLLABORATION IS THE THEME HERE. WE WORK VERY CLOSELY WITH THE FDA SENT UNTIL AND CTSAs UNDER PETRA. PETRA'S SLIDE IN WALTER'S SLIDE WAS EXACTLY RIGHT AND WE WORK CLOSELY WITH THEM, AND I SEE NO REASON WHY REGISTRIES DON'T ULTIMATELY DON'T AIM FOR THE DAY, WE'RE EXTRAORDINARILY COORDINATED WITH SENTINEL, WE USE THE SAME COMMON DATA MODEL AND WE'RE MAKING TO MAKE CERTAIN OUR COMMON DATA MODELS ARE COMPATIBLE, WE EACH MAINTAIN TWO THAT WE CAN WORK TOGETHER. BUT IN EVERY WAY, IN CONTRACTING, IN IRBs, AND IN DATA, WE'RE WORKING WITH THE CTSAs BECAUSE THEY ARE SUCH A RESERVOIR OF RESEARCHERS AS WELL AS PATIENTS. SO WHAT KINDS OF DATA ARE BEING COLLECTED FOR REGISTRIES? YOU KNOW THIS AS WELL AS I DO, PATIENT-REPORTED DATA IS OFTEN A REAL ADDITION TO REGISTRIES THAT STARTED WITH CLAIMS DATA OR ELECTRONIC MEDICAL RECORD DATA, SO PATIENT-REPORTED DATA BOTH QUALITATIVE ON HOW ARE YOU DOING AND ALSO QUANTITATIVE, SCALES AND BEHAVIORAL INFORMATION AND QUALITY OF LIFE, SYMPTOMS, ALL OF THAT, ELECTRONIC HEALTH RECORDS DATA ADDS THINGS WE'VE NEVER HAD BEFORE, ACCURATE INFORMATION ON THE MEDS BEING TAKEN, ACCURATE INFORMATION ON LAB VALUES, ACCURATE INFORMATION -- OR INFORMATION FROM PROGRESS NOTES, IMAGING REPORTS, DISCHARGE SUMMARIES, SOMETIMES ACCESS THROUGH NATURAL LANGUAGE PROCESSING. PRIMARY DATA COLLECTION IS A PART OF A LOT OF SPECIALTY REGISTRY SO SOMEBODY HAS AN IMPLANT, A NURSE OR ANOTHER PERSON GOES INTO THE HOSPITAL THE DAY OF IMPLANT AND RECORDS A LOT OF INFORMATION ON THE IMPLANT AND ON THE SURGICAL PROCEDURE, AND THAT BECOMES A PART OF THE REGISTRY SO PRIMARY DATA COLLECTION BY -- NOT BY THE PATIENT IN THIS CASE BUT BY THE PROFESSIONAL IS SOMETIMES AN ESSENTIAL PART OF A REGISTRY. CLAIMS DATA IS THAT INCREDIBLY IMPORTANT ELEMENT THAT CAPTURES -- HAS THE PROMISE OF CAPTURING INFORMATION ON PATIENTS YOUR REGISTRY AND EMRs MISS, THEY GO TO CARE AT OUTLYING EMERGENCY DEPARTMENT OR ANOTHER DOCTOR IN ANOTHER HOSPITAL. CLAIMS DATA REALLY GIVES YOU THE DENOMINATORS AND KNOWS WHO -- HELPS YOU KNOW WHO IS STILL UNDER OBSERVATION, BIOSPECIMEN INFORMATION INCREASINGLY IMPORTANT IN MANY CDRNs AND SOME PPRNs HAVE COMMUNITY SPECIMENS. COMMUNITY LEVEL DATA IS IMPORTANT, SOCIOECONOMIC STATUS OF A NEIGHBORHOOD, HEALTH STATUS OF NEIGHBORHOOD IS PART OF REGISTRIES AND IMPORTANT. WHO DO WE PUT IN THE REGISTRY? WELL, THIS IS JUST QUICK TO SHOW ONE COULD START OVER HERE AND JUST GET THE PEOPLE WHO ARE RECEIVING A SPECIFIC TREATMENT, OR PRESCRIPTION. YOU CAN THEN FOLLOW THE OUTCOMES OF THESE PEOPLE AND DESCRIBE THEM, AND FIND OUT HOW MANY PEOPLE, YOU KNOW, ESSENTIALLY YOU CAN SIMPLY MEASURE THE OUTCOMES IN PEOPLE EXPOSED TO TREATMENT. MUCH BETTER HERE, YOU GET A LOT MORE INFORMATION, YOU CAN ADD TO YOUR INFORMATION, IF YOU'VE GOT ALL THE PEOPLE WITH THE CONDITION SO YOU'VE GOT THOSE THAT ULTIMATELY WIND UP GETTING EXPOSED TO THE TREATMENT OR PROCEDURE AND THOSE THAT DON'T, AND YOU CAN BEGIN EVALUATING THE EFFECTIVENESS OF THE PROCEDURE IN THOSE WHO GOT IT VERSUS THOSE WHO DIDN'T. EVEN BETTER IS TO MOVE BACK AND HAVE UNDEROBSERVATION OF POPULATION THAT HAS ALL THE PEOPLE WITH THE CONDITION BUT ALSO HAS ALL THE PEOPLE AT RISK FOR THE CONDITION SO YOU CAN BEGIN STUDYING RISK FACTORS FOR DEVELOPING THE CONDITION, HOW T O PREVENT THE CONDITION. ALL THINGS EQUAL, ONE WOULD PREFER THIS, ANSWERING A NARROWER RANGE OF QUESTIONS BUT BECAUSE THEY ARE SMALLER YOU CAN GET MORE SPECIFIC INFORMATION ON THAT. THE NICE IDEA I THINK IS TO HAVE THIS STUDY WITH THE DETAIL DATA EMBEDDED IN A REGISTRY LIKE THIS. WHAT ARE THE CHALLENGES? THIS WE'LL TALK ABOUT A LOT IN THE PANEL. HOW BROAD SHOULD THE POPULATION BE? YOU HEARD ME SAY IDEALLY IT'S NICE TO BE VERY BROAD. HOW BROAD SHOULD THE REGISTRY'S PURPOSE BE? WHAT DATA SOURCES AND ELEMENTS SHOULD BE CAPTURED? WHAT DATA STANDARDS AND PLATFORMS SHOULD BE USED? HOW SHALL INFORMED CONSENT BE OBTAINED, HOW SHOULD THE REGISTRY BE FUNDED INITIALLY AND SUSTAINED. SOMETIMES IT'S EASIER, MAYBE AN OBSERVATION BIAS BUT EASIER TO GET THE THINGS FUNDED INITIALLY THAN TO KEEP THEM SUSTAINED. IDEAL ANSWERS, I DON'T MEAN TO SAY YOU SHOULD WRITE THIS DOWN AND GO HOME AND DO IT. I MEAN IF EVERYTHING WENT OUR WAY WE WOULD LIKE TO HAVE EVERY DISEASE SHOULD HAVE A REGISTRY OR A PLACE IN A LARGER REGISTRY, AND THERE SHOULD BE ONE. WE SHOULDN'T HAVE SIX OR SEVEN REGISTRIES ALL WITH INSUFFICIENT NUMBER OF PEOPLE, OR WITH OVERLAPPING PEOPLE COLLECTING DATA TWICE. REGISTRIES OUGHT TO BE LARGE AND THEY SHOULD BE MULTIPURPOSE, IN PART BECAUSE THAT HELPS SUSTAIN THEM. ALL DATA SOURCES ADD VALUE, SO ALL SHOULD BE COLLECTED. IT INCREASES THE FLEXIBILITY OF THE REGISTRY. REGISTRIES OUGHT TO BE EFFICIENT AND THEY OUGHT TO TAKE FULL ADVANTAGE OF EXISTING DATA FROM ELECTRONIC HEALTH RECORDS, CLAIMED, STORED TISSUE, ENVIRONMENTAL CONTEXT DATA SO DON'T DO IT YOURSELF. PATIENTS MUST PLAY A CRITICAL ROLE. WE BELIEVE THIS IS AN ESSENTIAL PART OF ON A REGISTRY, THAT THE PATIENTS ACTUALLY HAVE AN IMPORTANT ROLE IN DECIDING WHAT THE REGISTRY WILL BE USED FOR, THE RESEARCH PURPOSES. WE'VE DONE THAT, WE'VE DEMANDED THAT IN PCORnet AND IT MAKES A LARGE DIFFERENCE, WE BELIEVE, IN THE ABILITY TO EXPECT, ASK FOR AND EXPECT PARTICIPATION IN TRIALS AND OBSERVATIONAL STUDIES THAT NEED PATIENT INPUT. INFORMED CONSENT, I THINK OTHERS KNOW MORE ABOUT THIS THAN I WILL SO I'M GOING TO LEAVE IT TO OTHERS, BUT IN GENERAL REGISTRIES OUGHT TO SEEK BROAD INFORMED CONSENT, FORWARD-LOOKING INFORMED CONSENT, TIERED SO PATIENTS CAN GIVE DIFFERENT AMOUNTS DEPENDING ON COMFORT LEVEL, ALL DONE UNDER ONGOING GOVERNANCE POLICIES THAT INVOLVE PATIENTS IN GOVERNANCE. REGISTRIES SHOULD USE COMMON DATA ELEMENTS AND WE HAVE SO MUCH WORK TO DO IN THIS AREA, SO MUCH COMPETITION AND STRUGGLE ABOUT WHAT THOSE COMMON DATA ELEMENTS SHOULD BE AND THE DATA STANDARDS THAT GO WITH THEM. BUT WE SHOULD ALL WORK TO FIGURE OUT HOW TO PROCEED TOWARDS STANDARDIZATION, IN EHRs AND CLAIMS, IT WOULD MAKE THIS WORK OF CREATING COMMON DATA MODEL SO MUCH EASIER REALLY BUT SO MUCH LESS, REGISTRIES SHOULD BE SUSTAINED BY THE RESEARCH THEY DO. IF YOU'RE GOING TO BUILD A REGISTRY AND NOT DO RESEARCH, OR NOT CREATE OTHER PRODUCTS THAT HAVE VALUE, THERE'S ALSO USE TO START THE REGISTRY. EVERY USER HELPS SUPPORT THE REGISTRY THAT IS EVERY PERSON THAT DOES RESEARCH CONTRIBUTES TO THIS INFRASTRUCTURE OF THE REGISTRY AND EVERY REGISTRY REMAINS OPEN TO MULTIPLE USERS, THAT'S PRETTY CRUCIAL. GETTING TOWARD THE END HERE, TO GET STARTED, WHERE DOES THE FUNDING COME FROM? IT MAY WELL COME FROM THE ORGANIZATION OF THE FOUNDATION. IT MAY WELL COME FROM PRODUCING A PRODUCT. SO MANY REGISTRIES ACTUALLY WIND UP CREATING PERFORMANCE REPORTS THAT SOMEBODY, PHYSICIANS, DELIVERY SYSTEMS, HEALTH PLANS, PURCHASE. AND EXTERNAL FUNDING FOR DOING RESEARCH, FROM INDUSTRY, FROM PCORI, FROM NIH. FROM ARC. STUNNING FUNDING, PROBABLY THE ORGANIZATION WILL SAY WE CANNOT SUSTAIN THIS FOREVER, YOU'RE LEFT WITH PRODUCING PRODUCTS THAT MATTER AND HAVE VALUE, SPECIFIC RESEARCH. NIH, INDUSTRY, PCORI, CDC AND FOUNDATIONS CAN ALL WE HOPE CONTRIBUTE TO RESEARCH ACTIVITIES OF A REGISTRY. YOU MAY FOLLOW A MODEL LIKE CYSTIC FIBROSIS, PUTTING THE MONEY IN TO DEVELOP A NEW TYPE OF MOLECULE FOR CHILDREN OR PERSONS WITH CYSTIC FIBROSIS, A NOVEL MOLECULE THAT WORKED FOR SMALL PERCENTAGE OF PATIENTS. I THINK I READ THAT THE INITIAL INVESTMENT WAS IN THE LOW MILLIONS AND THEY SOLD THE RIGHTS ONCE KALYDECO PENT PUBLIC FOR $3.3 BILLION, SOLVING YOUR RESOURCES TO GET OTHER REGISTRIES LAUNCHED. THEY ARE THINKING CREATIVELY AND MAKING MAJOR CONTRIBUTIONS TO PATIENTS IN THE PROCESS. AND THEN A LITTLE BIT ABOUT PCORI FUNDING, WE HOPE YOU'LL APPLY TO PCORI FOR RESEARCH, CAN YOU APPLY FOR ANY CLINICAL AREA IN WHICH PRACTICE COULD BE CHANGED. YOUR RESEARCH, IT'S NOT -- WE DON'T WANT TO LEAD TO THE BETTER UNDERSTANDING OF THE CAUSES OF DISEASE. WE'RE AT THAT END OF THE TRANSLATION SPECTRUM WHERE WE NEED RESEARCH THAT WILL CHANGE PRACTICE AND IMPROVE OUTCOMES, COMPARATIVE, HAS TO BE PATIENT CENTERED, YOU'VE GOT TO BE ENGAGED WITH STAKEHOLDERS, LIKE RO1s, 1.5 MILLION, 3 YEARS, YOU CAN TALK US UP AND CALL US AND TALK US UP MODESTLY THERE. THAT'S BROAD. WE HAVE CHANGED IN THAT WAY. NOW THE TARGETEDS ARE SINGLE CLINICAL AREAS SO WE NOW HAVE ADVISORY PANELS THAT TELL US TO GO HERE. NARROW QUESTIONS, AREAS LIKE HYPERTENSION IN MINORITY AND RURAL POPULATIONS WITH NINDS AND NHLBI, HEPATITIS C DRUGS, GOT TO BE CER, PATIENT CENTERED, MUCH LARGER VARIABLE FUNDING AMOUNTS, THREE TO FIVE YEARS, FIVE IS THE LONGEST. IN FACT THEY ARE TYPICALLY FIVE. FUNDING HAS GONE UP TO $30 MILLION FOR A SINGLE STUDY. AND THE PRAGMATIC MOST EXCITING ONE, WE GIVE OUT UP TO 18 OF THESE A YEAR, COMPARATIVE EFFECTIVENESS. WE ACTUALLY HOPE YOU'LL RESPOND TO A SET OF HIGH PRIORITY TOPICS THAT WE LIST IN THE ANNOUNCEMENT BUT YOU CAN ALSO BRING YOUR OTHER HIGH PRIORITY QUESTIONS. THEY HAVE GOT TO BE PRAGMATIC, EMBEDDED IN DELIVERY SYSTEMS, ALMOST ALWAYS TRIALS BUT WE WILL PROVIDE UP TO $10 MILLION IN DIRECT COST, $15 MILLION TOTAL, FIVE YEARS. SO YOU CAN GO TO OUR WEBSITE. WE FUNDED 14 OF THOSE SO FAR, AND WE FUND SIX MORE, SIX TO NINE MORE, EVERY SIX MONTHS. SO I REALLY INVITE YOU TO LOOK THERE. IF YOU'RE JUST GETTING STARTED, THINK ABOUT OUR ENGAGEMENT AWARDS, MUCH SMALLER, UP TO $250,000 IN TOTAL, THE OBJECTIVES ARE SPECIFICALLY TO BRING NEW GROUPS INTO RESEARCH, THAT HAVEN'T PREVIOUSLY BEEN INVOLVED IN PATIENT CENTERED OUTCOMES RESEARCH, DEVELOP MECHANISMS FOR DISSEMINATING, FINDINGS AND SUPPORT IN PARTNERING OF PATIENTS AND CLINICIAN WAS RESEARCHERS, WE HAVE THREE TYPES OF AWARDS BASED ON KNOWLEDGE, TRAINING, DISSEMINATION AND IMPLEMENT BUT READ OUR WEBSITE THERE. YOU MAY FIND SOMETHING THAT WOULD HELP GET YOUR REGISTRY OFF THE GROUND OR AUGMENT YOUR EFFORTS. AND JUST IN GENERAL, STAY IN TOUCH, LOOK FOR OUR FUNDING OPPORTUNITIES, YOU'LL FIND ALL THREE THERE. I WILL HUSH UP NOW AND TAKE MY PLACE ON THE PANEL, WALTER, IS THAT RIGHT? THANK YOU VERY MUCH FOR YOUR ATTENTION. [APPLAUSE] >> AS WE SAID, THE STEERING COMMITTEE THOUGHT THIS AREA OF REGISTRY WAS OF THE MOST HIGH IMPORTANCE FOR THE GROUP TO DISCUSS, AND SO LUCKILY TO HAVE OUR TWO MODERATORS, PAUL GROSS AND DEAN SUHR. DEAN IS CO-FOUNDER OF OF MLD FOUNDATION, AND PAUL GROSS WAS THE PAST CHAIRMAN FOR THE HYDROCEPHALUS ASSOCIATION BOARD OF DIRECTORS. SO DO YOU WANT TO GO DOWN THE LIST OF THE OTHERS? (INAUDIBLE) >> YES, WE DO, OKAY. I SEE ERIKA AUGUSTINE, DEPARTMENT OF NEUROLOGY, UNIVERSITY OF ROCHESTER. AND SHE'S WORKED WITH EXPERIMENTAL THERAPEUTICS, RARE PEDIATRIC NEUROLOGIC DISORDERS. AND THEN LUCIE BRUIJN, ALS ASSOCIATION. SORRY. AND JOHN WILBANKS, CHIEF COMMONS OFFICER FOR SAGE BIONETWORKS, SPENT HIS CAREER WORKING TO ADVANCE OPEN CONTENT, OPEN DATA AND OPEN INNOVATION SYSTEMS, A SENIOR FELLOW OF FASTER CURES. AND WHO DID I LEAVE OUT? OH. (INAUDIBLE) OKAY, GREAT. WHY DON'T WE GET STARTED THEN. >> HAVE WE GOT THE RIGHT PRESENTATION UP? SO WE'RE GOING IN ORDER. >> OKAY. >> I GUESS WHILE THEY ARE STARTING THERE, WE HAVE A HANDOUT FOR YOU, BEING THE WEST COAST GUY THAT FLEW THE RED EYE IN I WAS NOT EARLY BUT SOME DEFINITIONS OF REGISTRIES AND SECOND PAGE IS A FULL LIST OF RESOURCES, SO A NUMBER OF THINGS THE FOLKS HERE WILL REFER TO, SOME OF THE THINGS THAT JOE ALREADY REFERRED TO, WE'VE PUT THE WEBLINKS AND SO ON, ON THIS. WE WANT TO MAKE SURE YOU GET COPIES OF THESE I LITERALLY JUST CAME OFF AN AIRPLANE. >> WE'RE TRYING TO GET TO THE SLIDES. SO I GUESS JUST TO GIVE YOU A LITTLE BIT OF OVERVIEW OF WHAT WE DID WAS TRY TO JUST ALL ANSWER A COMMON SET OF QUESTIONS AROUND WHAT WAS SORT OF THE OVERARCHING PURPOSE OF THE REGISTRIES THAT WE HAVE EXPERIENCE WITH, WHAT WERE THE TYPES OF QUESTIONS THAT WERE BEING ANSWERED, AND THEN LESSONS THAT ARE LEARNED. WE FIGURED FOR AN ORDER, WE HAD A CREATIVE STRATEGY. WE COULD SEE WHAT ORDER THEY PUT US ON THE TABLE AND COME AND SPEAK IN THAT ORDER. SO... GREAT, THANK YOU VERY MUCH. THE MOUSE IS MY CLICKER? PUSH THE KEYBOARD, OKAY. I'M PAUL GROSS, AS WALTER INTRODUCED ME. I'M THE INTERIM CEO FOR THE HYDROCEPHALUS ASSOCIATION, AND I CO-FOUNDED A CLINICAL RESEARCH NETWORK CALLED THE HYDROCEPHALUS CLINICAL RESEARCH NETWORK BACK IN 2006, WITH A DOCTOR WHO HAD DONE THE MOST CLINICAL TRIALS IN THE BRAIN CONDITION MY SON HAS, MY 10-YEAR-OLD BOY, SINCE BEEN INVOLVED IN THE CREATION OF THREE ADDITIONAL NETWORKS, ANOTHER ONE IN HYDROCEPHALUS FOR ADULTS, THEY SHOULDN'T BE SEPARATE BUT THEY ARE, I'VE BEEN WORKING ON CEREBRAL PALSY NETWORK WHERE I HAVE 32 DOCS WORKING FROM OVER 23 INSTITUTIONS AND SO IT FOLLOWS A BIT OF THIS MODEL BUT THEN IS GOING IN THE DIRECTION OF WHAT JOE SELBY TALKED ABOUT, CPRN, AND PPRNs COMING TOGETHER. SO QUICK BACKGROUND IS HCRM FOUNDED WITH SOME PRIVATE PHILANTHROPY TO START, STARTED AS JUST THREE CENTERS, IN UTAH, ALABAMA, AND TORONTO, AND WE TOOK A BUSINESS-LIKE APPROACH, SO I CAME OUT OF SOFTWARE, I WAS A HIGH TECH GUY, AND, YOU KNOW, MY INITIAL PUSH WAS FOR PEOPLE TO MEET LIKE REGULARLY, HAVE ACTION ITEMS THAT CAME OUT OF MEETINGS AND CHECKED ON THE ACTION ITEMS AT THE NECESSARY MEETING, THAT DISCIPLINE DID NOT EXIST, SO THAT BUSINESS LIKE APPROACH TO ACCOUNTABILITY WAS AMAZING, IF YOU SCHEDULED A CALL EVERY TWO WEEKS, IF ONE OF THE P.I.S OR ONE OF THE NEUROSURGEONS HADN'T DONE THE ACTION TWO WEEKS IN A ROW THE PEER PRESSURE WAS AMAZING AND MADE FOR INCREDIBLE PROGRESS. WE BROUGHT IN ADVISERS FROM CHILDREN'S ONCOLOGY, THERAPEUTIC DEVELOPMENT NETWORK, THE NETWORK THAT CYSTIC FIBROSIS FOUND, ADVISERS FROM NINDS AND HYDROCEPHALUS FOUNDATION, WE GOT TOGETHER COLLABORATING AND DEFINED A SET OF STANDARD OPERATING PROCEDURES AND BEGAN TO WORK ON A REGISTRY. ITS FIRST PUBLIC FUNDING BEYOND BUILDING OFFER THE PHILANTHROPIC BASE WAS 2009, SUCCESSFUL AWARDEE FROM FROM NINDS FROM THE FUNDING AND THE AMERICAN RECOVERY, I FORGET THE SECOND R ACT, REHABILITATION -- REINVESTMENT, THANK YOU, AND IN 2012 A PARTNERSHIP WITH THE HYDROCEPHALUS ASSOCIATION FOR SUSTAINING FUNDING OF THE ORGANIZATION, I WAS IN THIS INTERESTING POSITION, CHIEF FUNDRAISER FOR HCRN AND CHAIRMAN OF THE BOARD FOR HYDROCEPHALUS ASSOCIATION, WOULD GO TO DONORS, WHERE SHOULD I PUT MY MONEY, HCRN OR H-A, MERGING THEM WAS STRATEGIC. IN 2014, HCRN GOT FUNDING FROM PCORI AND THAT'S BECAUSE THE PRODUCT OF THAT REGISTRY SUPPORTS SUCH HIGH QUALITY RESEARCH ABLE TO TUNE ITS QUESTIONS IN A WAY THAT FIT WITH PCORI PRIORITIES AND WAS COMPELLING. SO IT'S NOW UP TO NINE CENTERS, A LIST OF THE CITIES ACROSS NORTH AMERICA, TWO IN CANADA, A DATA COORDINATING CENTER THAT IS AT THE UNIVERSITY OF UTAH, AND THAT DATA COORDINATING CENTER COORDINATES RESEARCH FOR A LOT OF DIFFERENT NETWORKS, IT HAS A NUMBER OF THINGS BASED OUT OF PEDIATRIC EMERGENCY AND SEE IT'S A VERY LEVERAGED RESOURCE FOR BUILDING CLINICAL RESEARCH CAPABILITIES. SO HYDROCEPHALUS IS AN EXCESSIVE ACCUMULATION OF FLUID IN THE BRAIN TREATED BY NEUROSURGEONS, PRIMARY TREATMENT IS PLACEMENT OF A SHUNT OR CATHETER IN THE BRAIN TO DRAIN EXCESS FLUID REEIGHTED IN THE '50s BY A DAD, HASN'T ADVANCED MUCH SINCE. IT'S NEUROSURGEONS THAT STUDY THE CONDITION, SO THIS IS A SURGICAL DATA COLLECTION REGISTRY, ITS OBSERVATIONAL, FOR SEVEN OUT OF NINE CENTERS THERE IS NO CONSENT REQUIRED, SO EVERYBODY THAT GETS TREATMENT, EVERYBODY THAT GETS A SURGERY, ENDS UP IN THE REGISTRY. AP FORM IS FILLED OUT THAT DESCRIBES THE PATIENT AND SURGERY BY A CLINICAL RESEARCH ASSISTANT, A HEAVYWEIGHT WAY TO GO, WITH EMRs AND FUTURE THERE'S A WAY TO GET OUT OF LEVERAGING CLINICAL RESEARCH ASSISTANCE FOR REGISTRY-LIKE STUDY. AND THEN THAT DATA IS VALIDATED BY THE NEUROSURGEON SO THEY HAVE TO SPEND VERY LITTLE TIME FOR DATA TO GET INTO THE REGISTRY. IT CAPTURES EVERY SINGLE HYDROCEPHALUS-RELATED SURGERY, ONLY TWO -- THREE VARIANTS OF SURGERY AND THEY ALL GET CAPTURED. SO THE REGISTRY WAS DEFINED BETWEEN 2006 AND 2008, A LITTLE BIT OF A SLOW START, STARTED COLLECTING DATA IN 2008, NOW HAS OVER 4700 PATIENTS, AND 9600 EVENTS. SO IT'S PRETTY -- VERY RICH HIGH QUALITY DATA SET. SO THE GOAL IS REALLY TO CHARACTERIZE THE POPULATION, AND TO IDENTIFY VARIATION, BOTH IN THE POPULATION AND IN PRACTICE. AND THE WHOLE CONCEPT OF CENTER EFFECT OR WHERE ONE INSTITUTION DOES SOMETHING DIFFERENT HAS BEEN -- BECOMES SO APPARENT, IT'S FAIRLY AMAZING HOW THE RESEARCH THAT GETS DONE BY THIS GROUP PROVES OUT THAT THE NEED TO DO MULTI-CENTER STUDIES IS KEY. ITS IDEA IS TO GENERATE STUDY QUESTIONS AND PILOT DATA. THERE'S THREE SUBPROJECTS WITHIN THE REGISTRY, SHUNTS, ENDOSCOPIC THIRD VENTRICULOSTOMY AND SHUNT INFECTION, AND PROTOCOLS. TO SUMMARIZE, VOLUME IS A HUGE ADVANTAGE FOR STUDY PLANNING AND ANALYSIS, AND THEN THE COLLABORATIVE ENVIRONMENT IS NOT JUST A REGISTRY, IT'S A NETWORK, A SET OF CENTERS THAT WORK TOGETHER, THEY COME TOGETHER BY A CONFERENCE CALL ONCE EVERY TWO WEEKS, THEY COME TOGETHER WITH A NETWORK MEETING TWICE A YEAR, AND THEY WORK TOGETHER TO DESIGN STUDY QUESTIONS, TO CREATE THE DATA FORMS, TO DO ANALYSIS, AND TO WRITE THE MANUSCRIPTS, IT'S A GREAT TRAINING ENVIRONMENT WITH SENIOR GUYS AND YOUNG GUYS TRYING TO ESTABLISH THEMSELVES, MARVELOUS. IT SETS THEM UP TO BE PREPARED FOR OPPORTUNITIES LIKE THE PCORI FUNDING OPPORTUNITY THEY WERE ABLE TO PURSUE. SOME OF THE LEARNINGS THAT WERE LIKE WHAT WE WOULD DO DIFFERENTLY, IT'S SURGICAL, DOESN'T REGISTER PATIENTS THAT COME THROUGH CLINIC THAT MAY OR MAY NOT NEED TREATMENT. IT WAS FOUNDED AS VERY MUCH SO A CDRN, A CLINICAL RESEARCH NETWORK WITHOUT MUCH INVOLVEMENT FROM PATIENT. I WAS THE SORT OF SOLE VOICE OF THE PATIENT, THAT WAS THE THING WE'VE BEEN TRYING TO SHIFT WITH A PARTNERSHIP WITH HYDROCEPHALUS ASSOCIATION. LOTS OF CHALLENGES AROUND ACADEMIC CREDIT AND MAINTAINING A MOMENTUM FOR A SINGLE DISEASE FOR THIS MANY YEARS BUT IT'S GOING QUITE WELL. COST AND SUSTAINABILITY ARE AN ISSUE. I'LL TURN IT OVER NOW TO DEAN. [APPLAUSE] >> IT'S MAGIC WHEN THE TECHNOLOGY WORKS. I'M DEAN SUHR THERE THE MLD FOUNDATION, PAUL AND I ARE YOUR CO-MODERATORS TODAY AND WE COME FROM DIFFERENT PARTS OF THE WORLD IN THE SPECTRUM. WE'RE A TWO-PERSON ORGANIZATION, MY WIFE AND I, UNPAID, NO STAFF, RUNNING AROUND DOING ALL KINDS OF DIFFERENT THINGS, AND STRONG ON THE PATIENT PERSPECTIVE AND WHAT I HOPE TO SHARE OVER THE NEXT FEW SLIDES WITH YOU IS THAT WE BELIEVE THAT THE PATIENTS CAN DRIVE RESEARCH AND DO THEIR OWN RESEARCH AND WE CAN BE A FEEDER FOR SOME OF THIS VERY SPECIFIC CLINICAL WORK, THIS DOWN TO THE P-VALUES AND THOSE SORTS OF THINGS. I'M GOING TO SHARE A CONTRAST TO WHAT PAUL JUST TALKED ABOUT. WHAT WE'VE BEEN DOING FOR REGISTRIES, WE'RE A FILE MAKER, PHP WEB-BASED SHOP, IT WORKS AND DOES A GREAT JOB. WHAT IT DOESN'T DO, IT WORKS WELL FOR ME BUT IT DOESN'T DO THE SORTS OF THINGS THAT WE EXPECT HERE IN, YOU KNOW, THE 2010s, WHERE WE WANT TO BE ABLE TO SHARE DATA OUT BECAUSE THIS SYSTEM WAS DESIGNED FOR US TO BE ABLE TO MANAGE OUR FOUNDATION AND TO CAPTURE DATA ABOUT THE FAMILIES THAT WE HAVE DATA. YOU'LL NOTICE IT'S FAMILY DEMOGRAPHICS, THERAPY AND TREATMENT DATA, WHAT HAVE YOU HAD DONE AND WHEN. WE KNOW IF THE CHILD -- MOST PATIENTS ARE CHILDREN, IF THEY ARE ALIVE AND HOW LONG THEY LIVED. NOTABLY SECOND FROM THE BOTTOM, BUSINESS MANAGEMENT, CONTACT AND MARKETING. FOR US AS A NON-PROFIT, WE NEED TO KNOW WHO WE'RE SERVING, SO WE CAN SERVE THEM, BUT ALSO SO WE CAN REPRESENT THEM INTO THE FDA OR THE INDUSTRY, WE GO TO ANY SORT OF MEETING OR CONFERENCE, WE NEED TO DO BASIC ANALYSIS. FAMILIES ARE A MISSION, WE HAVE TO SERVE OUR FAMILIES, THAT'S ALSO THE VALUE WE BRING TO EVERY TOUCH WE MAKE WITH INDUSTRY OR WITH THE REGULATORY AGENCIES, KNOWING ABOUT THE FAMILIES, NOT JUST WHERE THEY ARE BUT SOME BASIC CHARACTERISTICS AS WELL. WE DON'T WANT TO GET TOO FAR AWAY FROM THAT. SO I SPENT OVER FIVE YEARS WATCHING THE WHOLE REGISTRY SPECTRUM EVOLVE AND MANY OF YOU HAVE THAT SAME JOURNEY OR ARE ON THE SAME JOURNEY. I SEE PEOPLE TAKING NOTES AND PHOTOS. SOME OF YOU HERE DON'T HAVE YOUR OWN REGISTRIES YET. THE CONCERNS THAT WERE BOTHERING ME, WHO WAS OWNING THE DATA, WHOO IS MANAGING THAT DATA, WHAT'S THE OPERATIONAL COST AND SUSTAINABILITY, WE'VE HEARD THAT, FREE IS GOOD TO GET STARTED BUT FREE ISN'T ALWAYS THE SAME IN YEAR ONE, YEAR TWO, YEAR THREE. EVEN IF THERE'S NOT DOLLARS ATTACHED TO THE BOTTOM LINE YOU HAVE TO HAVE STAFF AND RESOURCES AND OUTREACH PROGRAMS AND THOSE THINGS. THE BIGGER PICTURE, WE WANTED A PLATFORM THAT ALLOWED US TO PARTICIPATE ACROSS DISEASES BECAUSE WE KNOW WE DON'T LIVE IN A VACUUM. OUR DISEASE, WE'RE CONCERNED ABOUT OUR DISEASE BUT THINGS THAT HAPPEN IN RELATED DISEASES, EVEN UNRELATED DISEASES CAN BE VERY IMPACTFUL FOR US, AND VICE VERSA. WE'LL TALK ABOUT CONNECTING SILOS BUT WE BELIEVE, WE'RE NOT GOING TO END UP WITH ONE NATIONAL DATABASE. JOE REFERRED TO THE IDEA WE CAN COMMUNICATE, THIS IS THE IDEA THAT SILOS OF DATA, THERE MAY BE ELECTRONIC HEALTH RECORDS, RESEARCH RECORDS, PHARMACOLOGICAL RECORDS, YOUR MRIs, WHATEVER DATA YOU HAVE, IT'S STILL GOING TO STAY IN THE SILOS, BUT WE NEED TO COMMUNITY AND IDENTIFY BACK AND FORTH. IT'S ALL ABOUT THE PATIENT AND WE WANT TO BENEFIT THE PATIENT WHILE THE RESEARCH IS GOING ON. WE'RE NOT JUST INTERESTED IN A CURE. WE'RE INTERESTED IN QUALITY OF LIFE ALONG THE WAY. PCORI CAME ALONG, WE GIANT THE CENA PPRN, EVERYTHING HAS AN ACRONYM IN THIS WORLD. COMMUNITY ENGAGED NETWORK FOR ALL, PATIENT-POWERED RESEARCH NETWORK, AND WE'VE TALKED ABOUT PATIENT VERSUS CLINICAL RESEARCH NETWORK. GENETIC ALLIANCE, UCSF IN SAN FRANCISCO AND UC DAVIS AND PRIVATE ACCESS ARE THE THREE EXTERNAL NON-DISEASE PARTNERS THAT WE'RE WORKING WITH. THEY ARE OVERLAYING IN SOME OF THE CDRNs, AS WELL, UCSF IS. ELEVEN ORGANIZATIONS FROM RARE TO FAIRLY COMMON, HEPATITIS BEING ONE OF THOSE DISEASES, ACROSS OUR DISEASES WE'RE STUDYING OBESITY SO WE'RE DOING THINGS THAT ARE VERY BROAD ACROSS 11 ORGANIZATIONS IN ONE PPRN, A MINI LITTLE PCORnet GOING ON OURSELVES. AND PCORI SUPPLIED THE FUNDING, BUT MOST NOTABLY AND JOE DIDN'T TOUCH ON THIS BUT PCORI DOESN'T ISSUE GRANTS. THEY HAVE MONEY, THEY WILL WRITE CHECKS BUT THEY ISSUE CONTRACTS AND THEY ARE PERFORMANCE BASED, AND YOU DO HAVE TO PERFORM, YOU DO HAVE TO PROVIDE THE UPDATES AND MAKE THE PROGRESS AND STATUS, I THINK THAT'S REALLY CHANGED THE WORLD CERTAINLY OF REGISTRIES AND IT'S IMPACTING HOW RESEARCHERS GO ABOUT THEIR WORK AS WELL. THE CENA PLATFORM, IT'S VERY COST EFFICIENT, BECAUSE IT'S SHARED ACROSS SO MANY ORGANIZATIONS. FAST MOVING, SHARON IS FAMOUS FOR SAYING WE'RE BUILDING THE AIRPLANE WHILE WE FLY IT. WE CAN GET STARTED, WE CAN DO THINGS AND ADD ON, REFINE IT. OBVIOUSLY YOU CAN'T CHANGE HOW QUESTIONS ARE ASKED AND HOW THOSE ANSWERS ARE CAPTURED BUT YOU CAN CHANGE THE METHODOLOGIES AND CONTINUE TO ADD ON AND REFINE. INSIDE CENA THERE WILL BE -- ACTUALLY NOT IN CENA BECAUSE THAT'S PCORI BUT THE GENERAL SIDE, IT'S DESIGNED TO BE SCALABLE AND HAVE MANY GROUPS, THE EXCITING THING IS IT'S NOW GARNERING FINANCIAL RESOURCE, RESOURCES FROM OUTSIDE PCORI. WE APPRECIATE THE PCORI FUNDING WE'RE NOT SOLELY DEPENDENT, SHARING THE OPERATIONAL AND DEVELOPMENT COSTS ACROSS A BROADER ARRAY OF DISEASE GROUPS. AND MOST IMPORTANTLY WITH REGARD TO THE DATA, SHARED QUERIES, COMMON DATA ELEMENTS, BROAD DATA SET, WE'RE ABLE TO LOOK AT THINGS THAT CROSS DISEASE BOUNDARIES WHICH I THINK IS VERY, VERY IMPORTANT. THE PRIVATE ACCESS PARTNERSHIP HAS TO DO WITH HOW WE'RE DOING OUR PRIVACY MANAGEMENT AND SECURITY FOR THE INDIVIDUAL PARTICIPANTS, AND WE CALL THEM PARTICIPANTS BECAUSE THEY ARE NOT ALWAYS THE PATIENTS. OFTENTIMES THERE'S A CAREGIVER ENTERING DATA ON BEHALF OF A PATIENT. AND AS JOE POINTED OUT ON ONE OF THE NEW PPRNs, WE'RE STARTING TO STUDY POPULATION GROUPS THAT AREN'T NECESSARILY PATIENT GROUPS. THEY HAVE SOMETHING IN COMMON AND WE'RE LOOKING FOR HEALTH CHARACTERISTICS OF THOSE GROUPS AND STARTING TO SEE THE DIFFERENCES WITH THAT. WE'RE ALSO SHARING QUESTION SETS WE DEVELOPED. WE DEVELOPED A QUESTION SET ON MEDICAL CANNABIS, MEDICAL MARIJUANA, THAT'S NOW AVAILABLE TO THE REST OF THE PEOPLE IN THE CENA PLATFORM, TO ANYBODY ELSE THAT WANTS TO USE IT, IT'S ALREADY ENTERED INTO THE CEN AX PLATFORM SO WE CAN GATHER ANSWERS AND ASK QUESTIONS ACROSS THE DISEASES WHERE WE WANT TO LEARN INFORMATION. VERY PATIENT CENTRIC, A GAMEIFIED FEEDBACK. WE START TO DIFFER FROM THE CLINICAL NETWORKS, A PATIENTS GETS FEEDBACK, THEY ARE GOING TO GET A BELL CURVE OR A BAR THING THAT SAYS ANSWERED YES TO, YOU KNOW, I LIVE IN A TWO-STORY HOUSE AND I ANSWERED YES, WE FIND OUT 67% OF THE OTHER POPULATION DOES THAT, OR I'M TAKING THIS DRUG TO IMPROVE QUALITY OF LIFE, MAYBE IMPROVE TONE OR SOMETHING LIKE THAT, AND MY CHILD IS HAVING THIS PARTICULAR REACTION, IS THAT CONSISTENT WITH EVERYBODY ELSE? IF IT IS, THAT'S INTERESTING. IF IT'S NOT THAT MAY BE EVEN MORE INTERESTING. IF IT'S WORSE YOU WANT TO LOOK INTO IT. IF IT'S BETTER YOU PROBABLY WANT TO LOOK INTO THAT AND WE COULD LEARN AS WELL. THE IDEA THERE WITH THIS FEEDBACK IS IT'S AN ENTICEMENT FOR THEM TO PARTICIPATE IN THE REGISTRY AND IMPROVING QUALITY OF LIFE. PRIVACY LAYER IS WHAT PRIVATE ACCESS IS PROVIDING WHERE WE HAVE GRANULAR PRIVACY AND SHARING CONTROL. I COME OUT OF THE RARE DISEASE WORLD SO WE DON'T TALK ABOUT PRIVACY. WE TALK ABOUT SHARING. HOW MUCH DATA CAN WE SHARE WITH RESEARCHERS AND WITH EACH OTHER TO IMPROVE QUALITY OF LIFE AND TO EXPEDITE RESEARCH? IT'S ALL THE SAME THING, DIFFERENT SPECTRUMS AND SOME THINGS I DON'T WANT TO HAVE SHARED OR WANT CONTROL OVER, THE SYSTEM MANAGES WELL. I'LL SHOW YOU THAT IN A MOMENT ON THE NEXT SLIDE. NOTABLY, THIS PRIVACY IS DYNAMIC. YOU AS A PARTICIPANT CAN CHANGE IT AS ANY POINT IN TIME, CHANGE IT IN THE MORNING, IN THE AFTERNOON, YOU HAVE VERY -- I'M GOING TO JUMP TO THE CHART. YOU'LL BE ABLE TO SEE IT IN THE SLIDE. THAT TOP SECTION, MLD, THE FOUNDATION AND RESEARCHERS RECOMMENDED BY MLD, WE HAVE A FILTERING PROCESS, YOU'LL SEE ON THE RIGHT GREEN IS ALLOWED, GREEN IS ASK ME, RED IS DENIED. THREE WAYS ACROSS THIS WHOLE MATRIX, IT'S A 3 BY 6, ORGANIZATIONS, RESEARCHERS, BROAD DATA ANALYSIS PLATFORMS LIKE THE GRD AND THE PARTICIPANT IS ABLE TO SET ALL OF THOSE PREFERENCES AND DO THAT OVER TIME. A TYPICAL PARTICIPANT MIGHT BE CONSERVATIVE OR PROBABLY IS FAIRLY CONSERVATIVE UNTIL THEY BUILD UP TRUST IN A PLATFORM LIKE THIS SO WE HOPE THAT EVEN IF THEY START PUTTING DATA IN AND SAY THEY DON'T WANT TO SHARE IT OVER TIME WE CAN ENCOURAGE THEM TO SHARE AND THEY CAN GO BACK AND CHANGE THAT SO THAT DYNAMIC CONTROL IS VERY POWERFUL. AND THEN THE ADVOCACY GROUPS ARE THE INITIAL CREATORS OF QUESTIONS. WE BY NO MEANS THE SMARTEST PEOPLE IN THE ROOM BUT WE LIVE WITH THE DISEASE, A GOOD PLACE TO START, AS WE CONTINUE TO GATHER DATA AND GET FEEDBACK FROM RESEARCHERS, AN INDUSTRY WHERE THEY HAVE THE QUESTIONS. THE CHALLENGES, THIS REPEATS BACK. FOR US EVERYTHING IS ABOUT IMPROVING QUALITY OF LIFE. AND A THERAPY IS AWESOME TO IMPROVE QUALITY OF LIFE BUT HAVING A BETTER WEDNESDAY IS GOOD AS WELL. WE WANT TO ADVANCE RESEARCH AND POLICY, CONSENT AND ACCESS MUST BE ALIVE, MUST BE DYNAMIC, AND THEY MUST CONTINUE OVER TIME. REGISTRIES REALLY KEY POINT, LAUNCHING A REGISTRY TO SOLVE A PARTICULAR SINGULAR PROBLEM, EITHER FUNDED BY ONE INDUSTRY GROUP OR ONE ACADEMIC PARTNER, AND NOT KNOWING WHERE THAT FUNDING IS GOING TO COME LATER ON, THAT REGISTRY WILL DIE. YOU HAVE TO THINK LONG-TERM, IF YOU'RE GOING TO GATHER DATA, YOU WANT TO REUSE AND ADD AND REFINE. DATA IS VALUABLE. DON'T LET IT GO AWAY. YOU DO HAVE TO MANAGE DATA. YOU CAN'T JUST PUT DATA IN THERE AND SAY THAT IT'S GOING TO BE THERE AND EVERYBODY CAN GET AT IT. YOU'VE GOT TO MANAGE ON BEHALF OF USERS, OWNERS AND THE RESEARCHERS. AND NOTABLY, ADVOCACY GROUPS CAN DO THINGS THAT OTHER ORGANIZATIONS CAN'T. YOU NEED TO RESPECT PRIVACY. YOU NEED TO RESPECT HIPAA BUT SOMETIMES WE CAN MOVE FASTER AND SOMETIMES WE CAN BE MORE ADAPTABLE THAN OTHERS THAT WOULD HAVE ACCESS TO DATA. WE DON'T LIVE IN A VACUUM. I TALKED ABOUT THAT IN TERMS OF THE SILOS. I FLEW IN FROM WEST COAST, IT WAS DARK. THIS PICTURE IS FROM SOME OTHER TIME. I DON'T KNOW WHAT YOU SEE WHEN YOU LOOK AT THAT BUT I SEE DOZENS OF FARMS AND ALL THOSE SILVER THINGS THERE ARE BUILDINGS AND SILOS, AND TO ME THAT REPRESENTS OUR MEDICAL DATA. IT'S LOCKED UP IN THIS SILO, THAT SILO, ALL THOSE FARMS ARE DOING DIFFERENT THINGS, TALK ABOUT IT DIFFERENT WAYS, PROBABLY NEXT TO EACH OTHER GROWING THE SAME CROP IN DIFFERENT WAYS, DIFFERENT VARIANTS. YOU CAN SEE A JUMBLE AND A DISCONNECT, IT'S TOUGH TO SEE IN THAT SLIDE BUT CONNECTING THOSE FARMS ARE ROADS, ALL THOSE ROADS ALLOW THE FARMERS TO COMMUNICATE AND MOVE GOODS BETWEEN EACH OTHER, THEY ARE ABLE TO HELP EACH OTHER WHEN THEY NEED TO HELP EACH OTHER, SOME FARMS HAVE GATES, PRIVACY, BUT THEY OPEN THE GATES WHEN THEY NEED TO, WHEN A FARMER IS IN TROUBLE THEY WILL BRING THE TRACTORS FROM NEXT DOOR AND HELP. IT'S NOT ALWAYS CLEAR WHERE WE'RE GOING. THE IMPORTANT THING IS TO KNOW WHERE IT IS, TO HAVE ACCESS AND TO BE ABLE TO CONNECT IT BACK TO THE INDIVIDUALS. WHEN WE ARE ABLE TO DO THAT AND MIX THE INGREDIENTS, WE END UP WITH GOOD STUFF, ALWAYS DIFFERENT AND YOU NEVER KNOW WHAT IT'S GOING TO BE, BUT REALLY THAT END GOAL FROM ALL OF THAT DATA COMES THIS INFORMATION OR IN THE CASE HERE THE GOODY. I THINK THAT'S THE CHALLENGE THAT'S IN STORE FOR US HERE. I'LL TURN IT OVER TO THE OTHER PANELISTS. THANK YOU. APPLAUSE. [APPLAUSE] (INAUDIBLE) >> SO I'D LIKE TO THANK THE ORGANIZERS. I'M JOHN WILBANKS, SAGE BIONETWORKS, A NON-PROFIT INSTITUTE IN SEATTLE. AND WE COME FROM A FAIRLY DIFFERENT ANGLE. ACTUALLY WE DON'T RUN REGISTRIES AT SAGE. WE RUN RESEARCH STUDIES. BUT THE THING I WANTED TO TALK ABOUT IS THE WAY THAT RESEARCH STUDIES ARE STARTING TO TAKE ON THE CONTOURS OF REGISTRIES AND VICE VERSA, IMPLICATIONS THAT MIGHT HAVE AS WE THINK ABOUT REGISTRIES GOING FORWARD. THE FIRST SORT OF THING, WE TALK ABOUT PURPOSE FOR OUR STUDIES AT SAGE, A BIG PART OF THE PURPOSE IS THE REALIZATION THAT WHAT JOE CALLED PATIENT-REPORTED DATA, IS ABOUT TO GO EXPONENTIAL, AND THE THINGS ABOUT EXPONENTIAL, YOU DON'T NOTICE UNTIL YOU'RE OVERWHELMED, THAT'S WHAT HAPPENS WHEN YOU DOUBLE AT A CERTAIN PRE-DETERMINED RATE. SO WHEN YOU LOOK AT WHAT PERVASIVE COMPUTING DEVICES, SMARTPHONES FOR NOW, BUT INCREASINGLY OTHER DEVICES CAN TELL YOU ABOUT PEOPLE'S HEALTH, IT'S REMARKABLE HOW MUCH PATIENT-REPORTED DATA THEY GIVE. THE BEST EXAMPLE IN THIS CONTEXT, WE'RE RUNNING A STUDY IN PARKINSON'S DISEASE CALLED EMPOWER, WITH iPHONES THAT SCRAPES DATA OFF THE PHONES, WE ENROLLED 18,000 PEOPLE IN THE FIRST 20 DAYS OF RUNNING THE STUDY, BECAUSE OF THE LACK OF FRICTION IN THE WAY OF DOING ENROLLMENT WE DID INFORMED CONSENT WORK THAT ALLOWED TO US DO THAT. WHAT THE STUDIES DO IS PROVOKE YOU TO DO CLASSIC PATIENT REPORTED OUTCOMES ABOUT YOUR PHONE THROUGH SURVEYS AND SUCH BUT WE HAVE SPECIFIC SENSOR ACTIVATED TASKS, HOLDING THE PHONE IN YOUR HAND OR POCKET, TAKE 20 STEPS FORWARD AND BACK AND RECORD ACCELEROMETER AND GYROSCOPE WHICH GIVES GAIT AND BALANCE AND WOBBLE AND ALL OF THAT IN A PURELY QUANTITATIVE WAY. WE ASK PARTICIPANTS TO SAY AH INTO THE MICROPHONE FOR TEN SECONDS WHICH GIVES US MUSCLE TONE FROM THE VOICE BOX, TAPPING ON THE SCREEN FOR DYSKINESIA, MORE HIGH RESOLUTION THAN THE CLINIC BECAUSE WE DON'T GET THE NUMBER OF TAMS, WE GET YOUR MEAN, MEDIAN, MAX, MINIMUM, WE GET YOUR LOCATIONS, WE CAN FIND PEOPLE WHO HAVE BENEFITING FROM EL DOPA PREVIOUSLY UNVISIBLE BECAUSE THEY WERE NOT GETTING MORE TAPS BUT MORE ACCURATE TAPS. WE DO THIS IN A PARTNERSHIP WITH APPLE TO THE RESEARCH PLATFORM, WE ALSO ARE RUNNING A STUDY POST CHEMO COGNITIVE RECOVERY, HOSTING THREE MORE STUDIES IN CARDIO, DIABETES AND -- I'M BLANKING ON THE THIRD ONE, ASTHMA, SORRY, ASTHMA. WHAT'S UNIQUE IS WE GIVE ALL THE PARTICIPANTS THEIR DATA BACK AT THE END OF THE STUDY AND GIVE THEM THE RIGHT AND TECHNICAL CAPACITY TO EXPORT THAT DATA INTO SOMETHING VERY MUCH LIKE THE REGISTRY. THIS KIND OF REALLY HIGH RES LONGITUDINAL DATA ABOUT PEOPLE ALLOWS YOU TO BEGIN DOING THINGS DIFFERENTLY. WE'VE NEVER HAD THE SAMPLE SIZE OF PEOPLE TO USE CONSUMER MATHEMATICS LIKE WE USE AT AMAZON OR GOOGLE ON HEALTH DATA, HAVEN'T HAD THE CONCENTRATION OF DATA PER TIME POINT TO USE THOSE, AND HAVEN'T HAD THE LONGITUDINALITY, BUT PATIENT REPORTED DATA ALLOWS US TO BEGIN COLLECTING THE SAMPLE SIZE, DENSITY AND LONGITUDINALITY FOR DEEP LEARNING AND THEY ARE NOT CONTAINED BY OLD TAXONOMIES. WE DON'T WONDER WHAT IS A TREMOR AND WRITE ONTOLOGY, WE'RE RECONSTRUCTING FROM 25 COMPUTATIONAL FEATURES THAT THE DATA TELL US ABOUT, USING THOSE TO DETECT CLUSTER OF PEOPLE IN OUR STUDY WHOSE QUANTITATIVE SYMPTOMS ARE MOST AFFECTED BY WATCHING THE NEWS, BECAUSE PEOPLE WHO GET STRESSED HAVE MORE DIFFICULTY WITH PARKINSON'S SYMPTOMS, PEOPLE WHO GET STRESSED OUT BY RACE RELATIONS IN THE UNITED STATES AND REPORT THIS THROUGH THE APP, WE HAVE A SIGNIFICANT CLUSTER BADLY AFFECTED BY WATCHING THE NEWS ABOUT RACE RELATIONS IN THE UNITED STATES. THIS IS NOT SOMETHING VISIBLE ANY OTHER WAY, CREATING A DIFFERENT WAY FOR THINKING ABOUT REGISTRIES AND DATA, AND DIFFERENT INTERVENTIONS. WE LAUNCHED SIX MONTHS AGO TOMORROW, WE JUST RELEASED A VERSION OF THE STUDY WITH INPUTTED FROM THE 13,000 COMMENTS FROM STUDY PARTICIPANTS, TELLING US WHAT WE NEEDED TO CHANGE FOR THEM TO BE WILLING TO KEEP GIVING US THEIR DATA. WE HAVE OFFERED EVERYONE IN THE STUDY THE OPPORTUNITY TO DEDICATE A COPY OF THEIR DATA TO A COMMONS, NOT A REGISTRY, A DATA COMMONS, FOR QUALIFIED RESEARCHERS TO USE WORLDWIDE, 75% ACROSS ALL FIVE OF THE STUDIES WE HOST AGREED TO DO THAT. SO WE HAVE THE ABILITY TO START THINKING OF STUDIES AS VECTORS FOR REGISTRIES AND VICE VERSA. BUT IT'S PRETTY HARD TO DO IF YOU DON'T START FROM THE BEGINNING WITH THE ASSUMPTION THAT THE PARTICIPANTS AT THE CENTER, THE PARTICIPANTS GETS A COPY AND HAS A RIGHT TO DEDICATE THE COPIES TO OTHER PLACES INCLUDING REGISTRIES OR COMMONSES AND SO IT'S A REALLY INTERESTING PLACE TO GO, BUT IT'S REALLY NOT THINKING ABOUT THE INDIVIDUAL AS SOMEONE ABOUT WHOM WE WANT DEMOGRAPHIC AND SURVEY INFORMATION BUT INDIVIDUALS WHO DECIDE THEY ARE ACTIVATED ENOUGH TO BEGIN DEDICATING PASSIVE SENSOR DATA FROM THEIR LIVES ABOUT THEIR HEALTH, IN ADDITION TO THE MORE STRUCTURED INFORMATION YOU CAN GET OUT OF H.R. STYLE SYSTEMS. THE LAST PART, NONE IS THAT USEFUL IF THERE'S NOT A PLAN TO GET DATA ANALYZED, VERY UNLIKE LI THAT THE POSTDOC WE HIRE IS GOING TO BE THE BEST IN THE WORLD AT ANALYZING AND IF SHE IS SHE'LL GET HIRED BY A HEDGE FUND IN A MONTH. THINKING ABOUT LONG-TERM DISTRIBUTION TO MULTIPLE GROUPS UNDER NEUTRAL TERMS AS OPPOSED TO SAYINGOME ELITE ACADEMICS WHO CAN PASS dbGAP STRICT CONSTRAINTS, RIGHT VEHICLES , WE HAVE TO GENERATE HYPOTHESES AND LET PEOPLE WHO ARE PASSIONATE AND PERHAPS QUALIFIED AT DATA ANALYSIS BUT NOT INSIDE THE ACADEMIC CULTURE TO ACCESS THAT DATA BECAUSE THE BEST DATA SCIENTISTS IN THE WORLD RIGHT NOW DO NOT WORK THAT THIS SPACE. THEY WORK IN FINANCE, SOCIAL MEDIA AND IN TECH. BUT THEY ALL HAVE FAMILY MEMBERS WHO ARE SICK. SO IF WE DESIGN SYSTEMS THAT LOCK THEM OUT OF REGISTRY DATA OR STUDY DATA, IT IS UNLIKELY WE'LL GET ANYTHING OTHER THAN PAPERS AT THE END OF THIS. THANK YOU. [APPLAUSE] >> GERALDINE? >> I CAN START TALKING. I'M GERALDINE BLISS, I CHAIR THE RESEARCH COMMITTEE OF THE FEE LAN McDERMID SYNDROME AND I HAVE A 17-YEAR-OLD SON, CAUSING AUTISM AND EPILEPSY. MY SON HAS SEVERE EPILEPSY, NOT NECESSARILY COMMON IN FEE LAN McDERMID BUT COMPELLED ME TO GET INVOLVED. I CHAIRED THE FOUNDATION, THE FIRST THING WE WANTED TO DO IS DEVELOP A PLAN FOR RESEARCH, AT THE TOP WAS TO DEVELOP A REGISTRY, SO WE LAUNCHED THIS PROJECT 4 1/2 YEARS AGO CHAIRED BY OUR WONDERFUL MEGAN O'BOYLE IN THE BACK OF THE ROOM. YOU'VE MET HER, SHE'S DONE A FABULOUS JOB CREATING OUR REGISTRY AND PCORnet, PHELAN McDERMID NETWORK. IT'S OPERATED BY OUR FOUNDATION, BY THE PARENTS. WE WORK WITH THE RESEARCH COMMUNITY TO CRAFT QUESTIONNAIRES AND OUR FAMILIES GO ONLINE AND ANSWER QUESTIONS AND THROUGH THAT WE WERE ABLE TO -- WE WERE IN A GREAT POSITION TO APPLY FOR A PPRN THROUGH PCORnet, WE'RE LABELING IN ELECTRONIC HEALTH RECORDS AND HOPE IN THE FUTURE DATA FROM APPS AND POTENTIALLY DEVICES AND JUST LIKE JOHN WAS TALKING ABOUT WITH AUTISM, THEY MIGHT BE TRACKING LANGUAGE, COMMUNICATION EFFORT, REPETITIVE MOVEMENT, MANY THINGS, AND WE'RE EXCITED, THROUGH WHAT WE'RE BUILDING HERE WE HOPE TO ENABLE THOSE TYPES OF PROJECTS. THE OTHER MAJOR PROJECT THAT WE'RE INVOLVED WITH IS THE RARE EPILEPSY NETWORK, SOME GROUPS PLAY BE ELIGIBLE TO EVENTUALLY BECOME PARTNER IN THE RARE EPILEPSY NETWORK, A GROUP OF NOW I THINK WE HAVE ABOUT 20 DISEASES THAT ARE IN IT, THEY ARE ALL DISEASES IN WHICH EPILEPSY IS COMMON, ALL RARE DISEASES, WE'RE DOING THE SAME THING COLLECTING DATA FROM PATIENTS. SO WE WERE ASKED TO THINK ABOUT CONSIDER SOME OUR LESSONS LEARNED, THERE HAVE BEEN MANY LESSONS LEARNED ALONG THE WAY. I THINK GOING BACK, I THINK REALLY THE FIRST THING IS TO REALLY THINK ABOUT WHAT ARE THE GOALS YOU NEED TO ACCOMPLISIH WITH YOUR REGISTRY, WHERE IS YOUR DISEASE NOW, WHERE ARE THE GAPS, WHAT DATA DO YOU NEED TO ADVANCE YOUR DISEASE? SO WE HAD MULTIPLE GOALS. WHAT WE DID WAS VERY AMBITIOUS, I WOULD NOT NECESSARILY RECOMMEND THIS MANY GOALS FOR REGISTRY, BUT OUR FIRST WAS TO SUPPORT RECRUITMENT FOR STUDIES AND CLINICAL TRIALS, THIS IS A FAIRLY EASY ONE TO DO. AS YOU'RE DOING THIS THINK ABOUT, OKAY, HOW AM I GOING TO RECONTACT PEOPLE TO LET THEM KNOW ABOUT STUDIES AND CLINICAL TRIALS? SO IN YOUR INFORMED CONSENT FORM THAT YOU'LL PROBABLY WANT TO HAVE, YOU PROBABLY WANT A CHECK BOX, CAN WE RECONTACT YOU TO LET YOU KNOW ABOUT STUDIES OR TRIALS THAT ARE COMING UP? FOR US WE THOUGHT THIS WAS IMPORTANT TO DO. WE ALSO KNEW THAT RESEARCHERS WOULD BE LIKELY TO SAY I WANT TO DO A STUDY WITH PATIENTS JUST WITH EPILEPSY OR SMALL DELETION PATIENTS SO WE NEEDED ENOUGH DATA TO SORT OF SLICE AND DICE OUR COMMUNITY ALONG MULTIPLE KIND OF -- IN MULTIPLE WAYS, AND WHAT WE DISCOVERED ALONG THE WAY IS WE ALSO NEEDED A RECRUITMENT POLICY, OUR INVOLVEMENT IN PCORI IS CHALLENGING US TO STEP UP OUR GAME AND TO REALLY FORMALIZE OUR POLICY, WHAT CONSTRAINTS DO WE HAVE TO MAKE DATA AVAILABLE TO RESEARCHER, DO WE CARE IF THEY REIDENTIFY OR TELL THEM NO, YOU CAN'T TRY TO REIDENTIFY, ARE WE GOING TO MAKE DATA AVAILABLE TO EVERYBODY OR GOING HAVE HAVE AN APPLICATION PROCESS? THESE ARE THINGS YOU WILL NEED TO CONSIDER WHEN YOU'RE DEVELOPING A REGISTRY. THE SECOND THING WE WANTED TO DO WAS DEVELOP HYPOTHESES, I GET CALLS ALL THE TIME, I'M THINKING ABOUT DOING A STUDY IN EPILEPSY, DO YOU HAVE PATIENTS WITH EPILEPSY OR ISSUES WITH GAIT? AND BEING ABLE TO HAVE SOME DATA, IT'S HIGH LEVEL PHENOTYPIC DATA TO SHARE WITH THEM IS EXTREMELY HELPFUL. WE WANTED TO BETTER UNDERSTAND OUR PHENOTYPE AND RELATE THAT TO THE GENOTYPE, IN OUR DISORDER YOU CAN HAVE A TINY MUTATION OF SHANK 3 GENE OR MISSING HUNDREDS OF GENES, WE WANTED TO UNDERSTAND SOME OF THE VARIABILITY WE SEE IN THE PHENOTYPE COULD BE EXPLAINED BY THE GENETIC CONDITION PEOPLE HAD. SO WE COLLECT GENETIC DATA AND PHENOTYPIC DATA, COLLECTED THROUGH ABOUT 200 QUESTIONS, TWO QUESTIONNAIRES, AND WHAT WE NEEDED TO THINK ABOUT REALLY IS HOW WERE WE GOING TO LET FAMILIES KNOW THEIR DE-IDENTIFIED INFORMATION WILL BE SHARED? OUR GOAL IS TO MAKE IT AVAILABLE, WE NEED TO MAKE SURE FAMILIES ARE OKAY WITH DATA BEING SHARED. IN OUR INFORMED CONSENT FORM WE TELL PEOPLE YES WE'RE GOING TO -- IF YOU WANT YOUR DATA SHARED, IT WILL BE DONE SO IN A DE-IDENTIFIED FASHION, DO YOU UNDERSTAND THIS? WE ALSO NEEDED TO HAVE A DATA ACCESS POLICY. WHAT ARE THE ACCEPTABLE USES OF THIS DATA BY RESEARCHERS? AND ARE THERE OTHER CONSTRAINTS THAT YOU WANT TO PUT ON THEM? DO YOU WANT THEM TO RETURN DATA TO YOU ONCE THEY HAVE DONE THE ANALYSIS? WE REQUIRE THAT. AND FINALLY WE WANTED TO UNDERSTAND THE NATURAL HISTORY OF OUR DISEASE, AND IN ORDER TO DO THIS WE NEED TO BE ABLE TO RECONTACT FAMILIES AND HAVE THEM PARTICIPATE IN THE REGISTRY ON AN ANNUAL BASIS, SO ONE OF THE CHECK BOXES ON OUR CONSENT FORM IS ASKING THEM IN WE COULD RECONTACT THEM AND LETTING THEM KNOW WE'LL BE REACHING OUT TO THEM, FOR THEIR FUTURE PARTICIPATION. AND WE THINK THAT'S A REALLY IMPORTANT AND NICE FEATURE OF OUR REGISTRY. THESE ARE A FEW OF THE THINGS WE'VE CONSIDERED, AND KIND OF HAVE BEEN LEARNING ALONG THE WAY. JUST A FEW LAST CONSIDERATIONS FOR THOUGHT. EVERY DISEASE THAT'S CONTEMPLATING CREATING A REGISTRY NEEDS TO THINK ABOUT WHAT'S ALREADY OUT THERE YOU CAN USE. IF THERE'S ANOTHER RELATED DISEASE THAT'S GOT A GREAT REGISTRY ALREADY, I THINK FIRST START BY TALKING WITH THEM AND FINDING OUT IF YOU CAN USE THEIR PLATFORM, USE THEIR REGISTRY, BECOME A PARTNER WITH THEM. THERE ARE RARE EPILEPSY NETWORKS YOU MIGHT BE ELIGIBLE TO JOHN, THE CLINICAL DISEASE NETWORK FOR DISEASES PART OF THAT THERE IS A CONTACT REGISTRY THAT'S AVAILABLE. THAT MAY BE A GOOD OPTION FOR SOME GROUPS. I'M SORRY, RARE DISEASE CLINICAL RESEARCH NETWORK IS -- AM I THINKING ABOUT -- OH, ALSO THE OFFICE OF RARE DISEASE RESEARCH THAT WALTER MENTIONED EARLIER, AND THEY'VE GOT SOME RESOURCES TOO THAT MAY BE WORTH CHECKING INTO. WE USE PATIENT CROSSROADS, AND THEY HAVE A FREE REGISTRY OPTION CALLED CONNECT, SO FOR THOSE UNSURE YOU CAN FUND ONE THIS MIGHT BE A GOOD OPTION TO LOOK INTO. RESOURCES ARE OUT THERE. THERE'S -- REGISTRIES TAKE A LOT OF WORK, MOST OF US START NOT AS EXPERTS BUT PARENTS OR PATIENTS ORE FAMILY MEMBERS WITH A PATIENT, AROUND WE HAVE A STEEP LEARNING CURVE. TAP INTO RESOURCES. WE TRIED TO DEVELOP CONSENT FORM DATA ACCESS AGREEMENTS, AND WE TALKED WITH OTHER GROUPS, SHARE WHAT THEY HAVE, SHARE WHAT WE HAVE, SO SPEND SOME TIME JUST TALKING WITH THOSE OTHER GROUPS AND FINDING OUT HOW THEY ARE DOING THIS. COMMON DATA ELEMENTS IS IMPORTANT, MENTIONED EARLIER. IT DOESN'T MAKE SENSE FOR YOU TO DEVELOP QUESTIONS SO SPECIFIC TO YOUR DISEASE AND NOT BE ABLE TO COMPARE HOW YOUR DISEASE IS -- HOW IT COMPARES TO OTHER RELATED DISORDERS. IF YOU CAN TAP INTO COMMON DATA ELEMENTS THAT HAVE ALREADY DEVELOPED, BEEN DEVELOPED, IT'S HUGELY BENEFICIAL, SO NINDS HAS SOME COMMON DATA ELEMENTS THEY HAVE ALREADY DEVELOPED. THERE MAY BE -- IF YOUR DISEASE CROSSES MULTIPLE -- IF YOU'RE ALSO IN MENTAL HEALTH, LIKE PHELAN-McDERMID YOU WANT TO EXPLORE COMMON DATA ELEMENTS. AND USE OF GLOBAL IDENTIFIER, THE I.D. GETS ASSIGNED, IT'S GENERATED AND ASSIGNED TO EACH PATIENT SO YOU CAN SORT OF TRACK THAT PATIENT IF YOU DUMP YOUR DATA INTO ANOTHER LARGER DATABASE, LIKE THE NATIONAL DATABASE FOR AUTISM RESEARCH, YOU DON'T WANT OUR PATIENT COUNTED MULTIPLE TIMES AND YOU WANT TO KNOW IF YOUR PATIENTS ARE SHOWING UP IN MULTIPLE PLACES. THAT'S ACTUALLY A REALLY IMPORTANT WAY, THAT'S AN IMPORTANT WAY TO ENABLE PROJECTS WITH RELATED -- WITH OTHER PROJECTS, WITH OTHER RESEARCH GROUPS, SO YOU MIGHT CONSIDER THAT. I DON'T KNOW HOW NINDS IS -- MAYBE SOMEBODY CAN ADJUST THE SLIDE, USE OF GLOBAL DEIDENTIFIERS, THROUGH NIMH IT'S THROUGH NDAR, COMMONLY USED AT NIH. THAT'S IT. THANK YOU. [APPLAUSE] >> OKAY, LUCIE, ALS ASSOCIATION. >> SO THE NATIONAL ALS REGISTRY IS DIFFERENT FROM THE ONES WE'VE HEARD ABOUT NOW IN THE SENSE THAT THIS IS REALLY DRIVEN THROUGH PUBLIC POLICY. NOT ONLY THROUGH THE ALS ASSOCIATION BUT THE MDA WHO IS REPRESENTED HERE AND MANY ALS GROUPS. IN FACT, IT STARTED AROUND 2005 WITH APPROPRIATIONS, IT WAS AROUND 2008 THAT THE ALS REGISTRY ACT WAS ENACTED, AND THIS THEREOF ORIGINATES FROM A DIFFERENT ANGLE, DRIVEN BY PATIENTS IN OUR PUBLIC POLICY TEAMS WITH THE GOVERNMENT, AND IT WAS MANDATED AT THAT TIME THEN THERE WOULD BE AN AGENCY WHO WOULD BE INVOLVED IN RUNNING THE REGISTRY AND AT THE TIME IT SEEMED QUITE A GOOD FIT TO WORK WITH ATSDR, PART OF CDC, WHO REALLY HAD THEIR EXPERTISE AND SURVEILLANCE REGISTRIES, THAT'S HOW IT INITIATED. IT'S INTERESTING THE CONCEPT OF SETTING GOALS AND DRIVING WHAT YOU WANT THAT REGISTRY TO DO. THIS WAS VERY MUCH DRIVEN BY THE PATIENT'S PASSION TO UNDERSTAND IF INDEED ALS WAS MUCH MORE PREVALENT AND FREQUENT THAN DESCRIBED. IN FACT, THERE WERE MANY CASES OF HOT SPOTS OF ALS, SOME THOUGHT TO BE ASSOCIATED WITH LAKES AND DIFFERENT AREAS, AND SO A LOT OF THE QUESTIONS COMING TO OUR ASSOCIATION AND OTHERS LIKE THE MDA WAS CAN WE REALLY ESTABLISH IF THERE ARE ANY RISK FACTORS INVOLVED? SO IT WAS QUITE AN AMBITIOUS TASK, BUT ONE OF THE FORTUNATE THINGS, IT'S AN INCREDIBLY WELL-FUNDED REGISTRIES BECAUSE EVERY YEAR THROUGH PUBLIC POLICY EFFORTS THEY HAVE MAINTAINED FUNDING FOR THE PROCESS. AND IN FACT, WORKS SLIGHTLY DIFFERENTLY IN THE SENSE OF HOW DATA IS COLLECTED BECAUSE IT'S DONE IN TWO FORMS. ONE IS THE WEB PORTAL WHICH WAS STARTED IN 2008 WHICH REALLY GOT LAUNCHED IN 2010, AND THIS IS SELF REGISTRATION, PEOPLE LIVING WITH THE DISEASE, CAREGIVERS, WILL ACCESS THROUGH THE COMPUTERS, THE SITE TO ADD IN THEIR DETAILS, AND THEN OF COURSE A LOT OF ANALYSIS DONE TO BE SURE THESE ARE TRUE ALS PATIENTS, BECAUSE IT'S ALSO VERY DIFFICULT WHEN YOU START TO COLLECT DATA AND YOU GET DATA POINTS AND YOU'RE NOT GETTING A PURE REGISTRY, SPECIFICALLY FOR ALS ONLY. AT THE SAME TIME, THE CDC HAD BEEN VERY ACTIVE IN WORKING WITH NATIONAL DATABASES TO GET A SENSE OF NUMBERS FROM THESE NATIONAL DATABASES. MOST RECENTLY A PUBLICATION CAME OUT FROM THESE EFFORTS TO GET NUMBERS, QUITE A LOT OF UPROAR AMONGST THE ALS COMMUNITY BECAUSE THE NUMBERS SEEMED LESS THAN WHAT WAS ACTUALLY SEEN SO THEY WERE DRIVING WITH THE THOUGHT THERE'S MORE ALS. JUST HIGHLIGHTS THE LIMITATIONS AND CHALLENGES OF THE REGISTRY BECAUSE THE SELF REPORTING IS GOING TO BE AS GOOD AS WHOEVER REPORTS, AND WHAT WE RECOGNIZED WAS THAT IN FACT MOST OF THE REPORTING CAME FROM ALPHA CERTIFIED CENTERS, MDA, SPECIALIZED CENTERS, LARGE AREAS UNDERREPORTED, THE CDC DID METRO CENTERS, NEW YORK, MIAMI, TO SEE IF THERE ARE CERTAIN POPULATIONS BEING MISSED, SOMETHING TO KEEP IN MIND WHEN THINKING OF REPRESENTING A FULL POPULATION, FOR A NON-REPORTABLE DISEASE SUCH AS ALS YOU'RE NOT GOING TO CAPTURE EVERYONE. WHAT IS STRIKING ABOUT OUR ORGANIZATIONS TOGETHER WITH MDA, INVOLVED IN DOING -- COLLECTING DATA FROM DIFFERENT STATES, AND IN FACT SOME STATES ARE VERY UNDERREPRESENTED AND IN MANY CASES PATIENTS ARE JUST NOT CONNECTED WITH COMPUTERS AND TECHNOLOGY. SO ALTHOUGH WE THINK WE'RE IN THE COMPUTER AND TECHNOLOGY AGE, THIS IS A DISEASE IN GENERAL OF THOSE IN THEIR 50s, 60s AND ABOVE, THOSE ARE THE KINDS OF EFFORTS WE HAVE TO ENSURE TO GET MORE INVOLVEMENT INTO THE REGISTRY. WE ALSO CONNECTED IT WITH A RISK FACTOR SURVEY, AND THIS TOOK QUITE A LOT OF ENGAGEMENT, ONE OF THE OTHER CHALLENGES AND LESSONS LEARNED THAT I WOULD POINT OUT IS YOU DO HAVE TO CONTINUALLY ENGAGE A STAKEHOLDERS. STAKEHOLDERS ARE PEOPLE LIVING WITH THE DISEASE. RESEARCHERS ARE GOING TO WANT TO USE THIS RESOURCE AND IF WE DON'T BUILD UP THESE REGISTRIES, SUFFICIENTLY WELL, SO THAT THE RESEARCH CAN BE PROPERLY DONE, YOU'RE LIMITED AS WELL. AND THE SURVEYS WERE DEVELOPED THROUGH EPIDEMIOLOGISTS AND THROUGH CAREFUL DISCUSSIONS AND MANY STRATEGIC MEETINGS, BUT THEY ARE LIMITED BECAUSE OF COURSE CERTAINLY THROUGH CDC AND REQUIREMENTS BY OMB, SURVEYS HAD TO BE MANAGEABLE NOT TOO LONG, VARIOUS CHALLENGES THAT OTHERS WILL HAVE RECOGNIZED, AND SO ONE OF THE THINGS I THINK IS THE REALITY, THAT THESE SURVEYS ARE VALUABLE IN SELECTING PATIENTS FOR FUTURE STUDIES BUT TO DO A FULL STUDY WITH THE NATIONAL REGISTRY WOULD BE LIMITING SO YOU WOULD HAVE TO REALLY GAIN THE ACCESS TO THOSE KINDS OF PATIENTS AND THEN DO A BETTER DATA. ANOTHER VERY IMPORTANT POINT TO NOTE IN DOING THESE KIND OF RISK FACTORS ANALYSES, THERE ARE NO CONTROLS IN THESE STUDIES SO YOU HAVE TO BE CAREFUL WHEN DOING ANALYSES. AGAIN, IT'S WHAT IS YOUR GOAL, WHERE DO YOU WANT TO GET TO, AND WHAT ARE YOUR LIMITATIONS WITH THE REGISTRY YOU HAVE. IN FACT, I THINK IT'S QUITE IMPORTANT TO HAVE MULTIPLE REGISTRIES, VERY HARD TO IMAGINE ONE. THE DEFINITION OF REGISTRY IS SO BROAD. WE HEARD ABOUT NATURAL HISTORY, THIS DATABASE, THIS REGISTRY DOESN'T QUITE DO THAT ALTHOUGH WE'VE NOW INTRODUCED THE ABILITY TO GO BACK A FEW TIMES OVER THE LIFE OF THE PATIENT, IT REALLY ISN'T AS RIGOROUS PERHAPS AS A FOLLOW-UP IN ALS PROGRESSION, FOR EXAMPLE, TO NOTE THE PHENOTYPIC CHANGES, CAREFUL OVER TIME. SO IT HAS HUGE VALUE BUT THOSE KINDS OF LIMITATIONS. SO OVER TIME, THE GOALS HAVE EVOLVED, AND WHAT WE CAN DO IF THE REGISTRY HAS CHANGED, I THINK ACTUALLY THE SECOND POINT HERE IS PROBABLY THE MOST VALUABLE AND MOST IMPORTANT, IS THE RESEARCH NOTIFICATION TOOL. WHAT'S BEEN DONE UP FRONT IS EVERYONE GOING INTO THE WEB PORTAL HAS THE OPTION TO BE ASKED AND INVITED INTO RESEARCH STUDIES. THAT'S BEEN INCREDIBLY VALUABLE BECAUSE WE HAVE A WEALTH OF INFORMATION ON EACH OF THESE PATIENTS SO BY GOING INTO THAT AND REQUESTING FOR YOUR STUDY THE RIGHT KINDS OF PATIENTS THEY CAN BE NOTIFIED. CURRENTLY THE WAY OF REGISTRY IS SET UP AND RUN BY CDC, THERE ISN'T A CONNECTION BETWEEN THE PATIENT AND THE RESEARCHER. THE RESEARCHERS GOES IN THROUGH CDC AND REQUESTS THEY WANT TO GET NOTIFICATION AND CDC WILL INVITE PEOPLE TO PARTICIPATE SO THERE ISN'T A DIRECT EXCHANGE OF DATA AND CONTACT WITH PATIENT AND RESEARCHER. THE OTHER THING THROUGH THIS PROGRAM IS ACCESS FOR CLINICAL TRIALS. AND IT HAS BEEN A CHALLENGE IN MANY OF THE DISEASE AREAS BUT CERTAINLY IN ALS TO REGISTRY TO GET ENOUGH PARTICIPANTS FOR CLINICAL TRIALS, AND THIS HAS BEEN VERY EFFECTIVE IN GETTING THE RIGHT KINDS, KNOWING MORE AND ENGAGING THEM. IN ADDITION WITH THE FUNDS POURING INTO THE REGISTRY WE HAVE ALSO HAD SPECIFIC STUDIES FUNDED BY THE REGISTRY WHICH HAS BEEN VERY HELPFUL. I THINK THAT'S MY LAST SLIDE. I THINK THAT THE KEY POINTS THAT I WOULD EMPHASIZE AGAIN IS YOU DO HAVE TO ENGAGE A STAKEHOLDER, YOU HAVE TO BE REALISTIC ABOUT WHAT THE INITIATIVE IS GOING TO PROVIDE, BECAUSE I THINK IT'S BEEN VERY MUCH EMPHASIZED BY THE PANEL, THE COST AND THE TIME TO BUILD THESE REGISTRIES IS HIGH AND LONG, SO PEOPLE PARTICIPATING ESPECIALLY LIVING WITH ALS, SOME MIGHT SURVIVE ONLY TWO TO THREE YEARS, FIND IT INCREDIBLY FRUSTRATING IT'S TAKEN THIS LONG TO SET IT UP, THEY MIGHT NOT BE ALLOWED TO SEE THEIR DATA, WHAT IS THE DATA THEY ARE GOING TO RECEIVE, IT'S NOT GOING TO BE AN IMMEDIATE -- THIS IS THE RISK FACTOR THAT IS AFFECTING MY DISEASE AND THAT'S REALLY THE PASSION THAT PEOPLE LIVING WITH IT HAVE, WHAT CAUSED MY DISEASE, IS THERE A HIGHER PREVALENCE, SO I THINK WE HAVE TO BE CAREFUL ABOUT WHAT WE COMMUNICATE AND HOW EFFECTIVE IT IS. IT'S A HUGELY VALUABLE TOOL FOR VARIOUS THINGS BUT WE HAVE TO BE EFFECTIVE AS TO WHAT THAT IS THAT IT DOES. I'M OPEN IN THE DISCUSSION TO ANSWER MORE QUESTIONS ABOUT IT BUT THAT GIVES AN OVERVIEW. THANK YOU. [APPLAUSE] >> AND ERIKA? >> GOOD MORNING, EVERYONE. THANK YOU FOR THE OPPORTUNITY TO BE PART OF THIS YEAR'S FORUM. AND I REPRESENT THE ACADEMIC PERSPECTIVE IN TERMS OF THINKING ABOUT REGISTRIES. AND OUR RESEARCH GROUP CAME OUT OF A CHANCE INTERACTION SO I THINK IT'S USEFUL IN THINKING ABOUT THE GOALS OF OUR REGISTRY AND HOW THEY WERE ESTABLISHED TO GO BACK TO THAT ORIGINAL MEETING. SO ALL OF US GO TO DIFFERENT KINDS OF EVENTS WHERE WE TALK WITH PEOPLE ABOUT WHAT WE DO, AND THIS RESEARCH GROUP EVOLVED OUT OF SUCH A CHANCE ENCOUNTER, WHO ARE YOU AND WHAT DO YOU DO, BETWEEN A MOLECULAR GENETICIST AND GERIATRICIAN. RESEARCH INTO JUVENILE BATTEN DISEASE, A RARE DISORDER, HE WORKED WITH YEAST AND MOUSE MODEL AND GERIATRICIAN WHO ARE EXPERIENCE IN OTHER RARE DISEASES AND THERAPEUTIC DEVELOPMENT, IF YOU FOUND A TREATMENT, IF YOU FOUND A YOU'RE, HOW WOULD YOU KNOW, WHAT HAPPENS WITH THE CHILDREN WHO HAVE THIS DISEASE? AND ALTHOUGH THIS IS A DISEASE THAT HAS BEEN IN EXISTENCE, IT HAS BEEN KNOWN AND RECOGNIZED FOR A COUPLE OF CENTURIES, THERE'S RELATIVELY LITTLE AT THIS POINT IN TIME GOING BACK ABOUT 11 OR 12 YEARS IN TERMS OF WHAT WAS KNOWN ABOUT HOW THE DISEASE UNFOLDS OVER TIME. AND THIS IS A DISORDER WHERE THERE ARE NO AVAILABLE DISEASE MODIFYING TREATMENTS, WHERE THE DISEASE IS FORMALLY FATAL, IN THE SECOND OR THIRD DECADE OF LIFE. WE THOUGHT ABOUT THE CONCEPT OF DEVELOPING NEW THERAPEUTICS AND WHO WE WOULD TARGET, HOW WE WOULD UNDERSTAND WHAT SYMPTOMS TO TARGET AND WHAT HAPPENS IN TERMS OF THE DISEASE OVER TIME AND WHAT A TRIAL WOULD LOOK LIKE, THERE WERE MANY QUESTIONS THAT CAME ABOUT, THIS LED TO NEW COLLABORATIONS BETWEEN THIS GERIATRICIAN, MOLECULAR GENETICIST, PEDIATRICIAN AND THOSE WITH BATTEN DISEASE, THINKING ABOUT CLINICAL TRIALS OUTCOMES AND UNDERSTAND NATURAL HISTORY, NOW LED BY JONATHAN MINK, AT THE TIME OF INCEPTION OF THIS RESEARCH GROUP THERE WEREN'T TREATMENTS ON THE HORIZON OR IN DEVELOPMENT BUT A DECADE LATER WE'RE FINALLY AT THAT POINT AND HAVE BEEN WORKING FOR THE PAST TEN YEARS TO BE READY FOR THIS POINT. EVEN IF THERE AREN'T NECESSARILY CLEAR THERAPEUTICS ON THE HORIZON, START THINKING NOW ABOUT HOW YOU MIGHT GATHER INFORMATION THAT WILL BE USEFUL WHEN THE TREATMENTS ARRIVE. WHAT DO WE COLLECT? AT THE HEART OF OUR REGISTRY IS A RATINGS SCALE, A UNIQUE DISEASE SO SOME COMMON MEASURES IN PEDIATRICS DON'T NECESSARILY APPLY IN JUVENILE BATTEN DISEASE PATIENTS BECOME BLIND, THEY DEVELOP DEMENTIA, AKIN ALZHEIMER'S, MOVEMENT AKIN TO PARKINSON'S, THE MEASURES WE USE PEDIATRIC DISORDERS NOT SO USEFUL OR ACCURATE. WE NEEDED TO START FROM SCRATCH WITH WHAT WE THOUGHT WE KNEW AND DEVELOP SOMETHING. SO WE DEVELOPED A CLINICIAN-DIRECTED INTERVIEW, ESSENTIALLY A SEMI STRUCTURED INTERVIEW CONDUCTED WITH PARENTS AND THEN ALSO HAS AN ELEMENT OF PHYSICAL EXAMINATION, WITH THE AFFECTED CHILD. THROUGH PARTNERSHIP WITH THE PATTEN ASSOCIATION, THEY HAVE BEEN HELPFUL HELPING US UNDERSTAND WHAT WE KNEW WAS NOT ACCURATE, EARLY YEARS WERE SPENT HONING THE RATINGS SCALE AND UNDERSTANDING OTHER THINGS THAT HAPPEN AND ARE IMPORTANT IN UNDERSTANDING DISEASE PROGRESSION WITH THE IDEA OF OUTCOME OF IDEAS AND THERAPEUTICS IN MINE. WE USE A FILE MAKER DATABASE, AN ASSOCIATE DOES DATA ENTRY, THERE'S NO DIRECT PATIENT ENTRY AGAIN BECAUSE SOME PATIENT OUTCOMES ARE IN SOME WAYS NOT NECESSARILY RELEVANT. SO WHAT HAVE WE DONE OVER THE COURSE OF THESE YEARS? I AM GOING TO TALK ABOUT PAPERS, NOT BECAUSE THE PAPERS THEMSELVES ARE THE IMPORTANT OUTCOME BUT THE INFORMATION THAT THEY REPRESENT IS HELPFUL. AT THIS POINT IN TIME A DECADE IN WE'VE SEEN PERHAPS OVER A HUNDRED PATIENTS WHO HAVE JUVENILE BATTEN DISEASE AND THIS WAS DEVELOPED AT A NATURAL HISTORY STUDY IN MIND, SOME PATIENTS, WE'VE SEEN AS MANY AS 11 OR 12 TIMES IN TERMS OF ANNUAL VISITS, SO 11 OR 12 YEARS OF DATA, APPROXIMATELY 100 OR SO REPRESENTS PROBABLY A THIRD OF THE INDIVIDUALS IN THE UNITED STATES WHO ARE AFFECTED BY JUVENILE BATTEN DISEASE. FOR A RARE DISEASE WE'VE SEEN A GOOD PROPORTION OF INDIVIDUALS. AS IT RELATES TO THE PRIMARY OBJECTIVE THINKING ABOUT CLINICAL TRIAL OUTCOMES WE'VE DEVELOPED A RATINGS SCALE, WE'VE BEEN ABLE TO VALIDATE AND SHOW IT'S RELIABLE, IT'S BEEN USEFUL FOR UNDERSTANDING WHAT HAPPENS WITH THE DISEASE OVER TIME. BUT ALSO IMPORTANTLY, WE'VE BEEN ABLE TO USE THE REGISTRY TO BUILD OUR OWN KNOWLEDGE AS WE ENGAGE WITH CLINICIANS, AROUND THE COUNTRY, AROUND THE WORLD, TO TALK MORE THAN JUST ABOUT OUR DIRECT EXPERIENCE WITH THE PATIENT FOR WHOM WE PROVIDE CARE, WE CAN PROVIDE MORE SPECIFIC GUIDANCE. WE'VE BEEN ABLE TO DISPEL MYTHS, SOME THINGS WHERE WE WERE WRONG, WHERE NEUROLOGISTS WERE WRONG. WE ONCE THOUGHT IN JUVENILE BATTEN DISEASE PATIENTS HAVE AN EXTREMELY REFRACTORY EPILEPSY AND PATIENTS WERE STARTED ON TREATMENTS RIGHT AWAY BUT THROUGH BEING ABLE TO SEE HOW SYMPTOMS EVOLVE OVER TIME AND WHAT HAPPENS WITH PATIENTS WE CAN REALIZE ACTUALLY SYMPTOMS ARE NOT SO REFRACTORY AND WE CAN GIVE CONCRETE INFORMATION ABOUT THAT. SIMILAR WE'VE BEEN ABLE TO AN QUESTIONS FROM FAMILIES IN A MORE SPECIFIC WAY, SO AS IT RELATES TO THE IDEA THAT PERHAPS THERE ARE SOME SEX DIFFERENCES OR GENDER-BASED DIFFERENCES WE'VE HAD PATIENTS COME TO US TO SAY WE THINK THAT SYMPTOMS ARE MORE AGGRESSIVE IN GIRLS AS COMPARED TO BOYS, THAT'S WHAT WE'VE SEEN AND WE WENT BACK TO THE NATURAL HISTORY DATA, BACK TO THE REGISTRY TO GIVE A CLEAR AND INFORMED ANSWER ABOUT THAT. PATIENT DIRECTED QUESTIONS ARE THINGS WE SPENT TIME THINKING ABOUT AND USING OUR REGISTRY IN CONCRETE WAYS. FINALLY, NOT ONLY DO WE CONTACT PEOPLE YEAR AFTER YEAR FOR PARTICIPATION IN THE ONGOING NATURAL HISTORY STUDY, WE ALSO HAVE PERMISSION TO CONTACT PEOPLE WHEN THERE ARE NEW OPPORTUNITIES FOR PARTICIPATION IN RESEARCH IN PARTICULAR OF CLINICAL TRIALS SO ALTHOUGH AT CONCEPTION THERE WERE NO CLINICAL TRIALS ON THE HORIZON THERE'S NOW AT LEAST ONE WITH A FEW MORE IN DEVELOPMENT AND WE HAVE BEEN ABLE TO USE THE REGISTRY TO CONTACT PEOPLE ON DAY ONE OF RECRUITMENT TO BE ABLE TO LET PEOPLE KNOW THAT THERE'S A STUDY, WHETHER OR NOT THEY ARE ELIGIBLE TO PARTICIPATE, AND MAKE SURE THE COMMUNITY REMAINS VERY INFORMED. I TALKED ABOUT FUTURE DIRECTIONS HERE BUT REALLY THESE ARE ISSUES WE'RE GRAPPLING WITH AT THE MOMENT. ONE IS TO THINK ABOUT CONTINUED PATIENT PARTNERSHIP. OUR REGISTRY IS BUILT ON FACE TO FACE INTERACTION WITH DIRECT INTERVIEW AND DIRECT EXAMINATION OF THE PATIENT BUT THERE ARE A LOT OF REASONS WHY THAT'S NOT NECESSARILY SO PRACTICAL, SO HOW DO WE LEVERAGE TECHNOLOGY AND THINK ABOUT MORE PRACTICAL, MORE CONVENIENT WAYS TO ENGAGE WITH OUR PATIENTS? SO WE'RE THINKING A LOT ABOUT THAT. WE SPENT A LOT OF TIME WITH PATIENTS ABOUT THE CONTENT OF OUR REGISTRY AND WE ALWAYS WANT TO MAKE SURE WE'RE GATHERING THE INFORMATION THAT GIVES US THE MOST USEFUL DATA, NOT ONLY FROM A CLINICAL TRIAL STANDPOINT BUT AGAIN IN TERMS OF WHAT'S VERY MEANINGFUL FOR PATIENTS. DATA MANAGEMENT IS A CRITICAL ELEMENT TO CONSIDER WHEN STARTING A NEW REGISTRY, CERTAINLY, AND IN ITS ONGOING EXISTENCE GETS REVISITED, A CULTURE OF EVALUATION, A CULTURE OF CONTINUOUS IMPROVEMENT AND THESE ARE THE THINGS WE SEEK TO DO IN DATA MANAGEMENT BEING KEY PART OF THAT. WHEN WE STARTED, WE WERE THE KEY OR ONLY REGISTRY AS IT RELATES TO PATIENT WITH JUVENILE BATTEN DISEASE BUT NOW THERE ARE A FEW, THERE'S AN INTERNATIONAL COLLABORATION OUT OF EUROPE, ANOTHER CONSIDERING STARTING A REGISTRY, WE'RE A SMALL COMMUNITY SO CROSS-TALK WILL BE IMPORTANT IF NOT UNIFYING INTO A SINGLE REGISTRY FOR PATIENTS. MR. WILBANKS TALKED ABOUT USING DATA, SO TO GATHER THIS DATA AND NOT MAKE REGULAR USE OF IT WOULD BE A WASTE. MAKING SURE THAT WERE WE ARE -- AND PATIENTS ARE MAKING THE MOST COMPREHENSIVE USES OF AVAILABLE DATA, THAT WE UNDERSTAND THE INFORMATION THAT WE HAVE AND WE'RE MAKING THE MOST OF THE DATA WE'VE GATHERED, SOMETHING THAT'S A CONTINUOUS PROCESS. AND AS WE LOOK TOWARDS EFFICACY TRIALS, SO UNDERSTANDING NOT ONLY ARE TREATMENTS SAFE, APPROPRIATE TO GIVE, BUT THEN DO THEY WORK, HAVE THE DATA THAT WE'VE COLLECTED OVER TIME, ARE THEY APPROPRIATE AND SUFFICIENT FOR USE OF THE NATURAL HISTORY CONTROL WHICH IS SOMETHING AGAIN PATIENT FOUNDATION THINK ABOUT CLINICAL TRIALS, VERY SMALL NUMBERS, CHALLENGES AND RECRUITMENTS, MAKE SURE WE THINK CREATIVELY ABOUT HOW WE DESIGN THOSE STUDIES AND SOMETIMES REGISTRIES CAN PROVIDE THE BACKGROUND INFORMATION NOT ONLY FOR WHAT THAT STUDY'S DESIGN SHOULD LOOK LIKE BUT IN TERMS OF A CONTROL GROUP OR NATURAL HISTORY COMPARISON. IN THINKING ABOUT -- THIS IS A GROUP EFFORT BUT IN THINKING ABOUT FUNDING, WE'RE VERY THANKFUL TO NINDS WHO PROVIDED INITIAL FUNDING FOR THIS REGISTRY IN THE SETTING OF NATURAL HISTORY STUDY. WE'VE HAD ADDITIONAL FUNDING FROM THE INSTITUTION FROM THE UNIVERSITY OVER TIME TO HELP SUPPORT THIS ONGOING WORK. OUR CLINICAL TRIALS ARE FUNDED BY THE FDA, AND LAST BUT NOT LEAST IS THE BATTEN DISEASE SUPPORT AND RESEARCH ASSOCIATION. THEY ARE THE MOST INVALUABLE PARTNER, TRUE PARTNER THAT WE HAVE, NOT PRIMARILY FROM A FUNDING STANDPOINT, THEY HAVE BEEN VERY GENEROUS WITH FUNDING BUT REALLY MORE FROM A COLLABORATION STANDPOINT. EVEN THOUGH THE INITIAL IDEA FOR THIS NATURAL HISTORY STUDY AND REGISTRY CAME OUT OF ACADEMIA, IT'S BEEN SOMETHING WE COULD NOT MOVE FORWARD WITH IN ANY WAY WITHOUT PATIENT PARTNERS, AND SO WE REALLY -- THEY ARE EQUAL PARTNERS IN THIS WORK AND HAVE DRIVEN MANY OF THE QUESTIONS AND MANY OUTCOMES. I'LL BE HAPPY TO TAKE QUESTIONS ABOUT RARE DISEASES IN PARTICULAR AS WE MOVE TO THE PANEL. [APPLAUSE] >> I HAVE TO OPEN TO QUESTION AND ANSWERS, GO TO A MIC SO PEOPLE THAT ARE WATCHING RECORDED CAN HEAR QUESTIONS. >> I'M NOT A RESEARCHER. THE THING THAT OCCURRED TO ME IS THE WORST PROBLEM THAT WE GET WITH RARE DISEASES IS GETTING THEM DIAGNOSED BECAUSE DOCTORS DON'T KNOW THEM. AND PARTICULARLY MR. WILBANKS SEEMS TO ME THERE MUST BE A WAY, YOU'RE COLLECTING SO MUCH DATA, IS THERE ANY WAY TO HELP DOCTORS, I KNOW THIS IS A BIG QUESTION, TO BE ABLE TO NARROW DOWN AND FIND AND HELP DOCTORS DIAGNOSE WITH SOME OF THE DATA YOU'RE GETTING? >> SO I'M SCARED TO OVERSTATE THE CASE BECAUSE THIS IS REALLY NEW AND IT'S PARTICULARLY WELL FITTED TO MOVEMENT DISORDERS COMPARED TO OTHER KINDS OF DISORDERS. BUT WHAT WE HAVE SEEN IS THERE'S A REAL DESIRE FOR THE PARTICIPANTS TO HAVE SOMETHING LIKE A SCORE CARD THAT WOULD BE DOCTOR READABLE. THAT'S THE IDEA. PARTICIPANTS SHOULD HAVE A SCORE CARD ON THE PHONE DESIGNED WITH SOFTWARE INVESTMENT AND USER EXPERIENCE DESIGN SO A DOCTOR CAN LOOK AT THE PARTICIPANT'S DATA TO HELP THE CONVERSATION BUT THAT'S AS FAR AS WE'VE GONE. WHAT WE DON'T WANT TO DO IS OVERREAD THE FIRST SET OF RESULTS FROM THE PROCESS. BUT THERE'S IS CLEARLY DEMAND BY THE PARTICIPANTS HAVE TO THE ABILITY TO SAY HERE IS HOW MY GAIT LOOKED OVER THE LAST MONTH, WHAT CAN WE DO ABOUT THAT? SO THAT'S REALLY AS FAR AS WE'VE GONE BECAUSE WE'RE TRYING TO BE CAUTIOUS AND NOT, YOU KNOW, JUST ASSUME THIS IS THE ONLY WAY TO GO FORWARD. >> LET ME THROW OUT THAT THERE IS A PROGRAM AT NIH, THE UNDIAGNOSED DISEASE PROGRAM, AND IT'S BASICALLY NOW A NATIONAL PROGRAM, MULTIPLE CENTERS WHERE PEOPLE WHO HAVE CONDITIONS THAT NO ONE KNOWS WHAT IT IS GO AND BASICALLY GET AN INTENSIVE WORKUP, 40% ARE NEUROLOGICAL. >> ONE WAS ATAXIA. >> ONE OF THE PROJECTS THAT'S BEEN DONE THROUGH PCORnet IS COMPUTABLE PHENOTYPES FOR INDIVIDUAL DISEASES AND ACROSS P CORONETTED, SO GROUPS LIKE U.S. TAKING DATA FROM REGISTRIES AND ELECTRONIC HEALTH RECORDS AND RESEARCH PARTNER AT HARVARD IS TAKING THAT AND USING IT TO COMPUTABLE -- MAYBE PHELAN McDERMID -- (INAUDIBLE) -- THEN YOU WOULD HAVE USERS, WHAT DOES PHELAN McDERMID LOOK LIKE AND YOU CAN LOOK AT IT IN LARGER DATASETS, ONE WAY TO MOVE RESEARCH FORWARD AND HOPEFULLY GET WHITE PAPERS THAT COME OUT OF THAT AND OUR CLINICIANS LEARN MORE ABOUT IT. >> YEARS AND YEARS TO A DIAGNOSIS, PHYSICIANS WERE TRAINED TO LEARN BASED ON PATTERN RECOGNITION, THAT IF SOMEONE HAS A AND B AND C, THAT THIS IS THE DIAGNOSIS THAT THEY SHOULD HAVE. BUT IN THIS ERA OF IMPROVING GENETIC DIAGNOSTICS, AGAIN I THINK THERE ARE A LOT OF THINGS THAT WE'RE LEARNING NOT NECESSARILY SO RIGHT AND PEOPLE DON'T FIT THAT RULE OR FOR SOMETHING PEOPLE HAVE LIKE ATAXIA WHERE THERE ARE DOZENS OF CAUSES OF ATAXIA, JUST THINKING ABOUT GENETICS ALONE, NOT NECESSARILY OTHER CAUSES, IT'S REALLY HARD TO SAY, MY PATIENT HAS ATAXIA, WHICH ONE OF THESE 40 DISEASES SHOULD I LOOK FOR? I'M NOT SURE. OR I'M GOING TO DO SOMETHING LIKE WHOLE EXOME SEQUENCING OR ANOTHER TEST, AND I'VE GOT AN UNCERTAIN RESULT, WHAT DOES THAT MEAN? AND YET ANOTHER EXAMPLE IS NEWBORN SCREENING, WHERE VERY EARLY ON WELL BEFORE PEOPLE EVERY WOULD HAVE A DIAGNOSIS, NOW THERE'S THIS FINDING ON THE NEWBORN SCREENING AND WE'RE NOT SURE WHAT IT MEANS. SO I THINK WHEN IT COMES TO REGISTRIES, SOMETIMES WE CAN HELP WITH THE PATTERN RECOGNITION PART, IN THE WORLD OF BATTEN DISEASE, IF YOU HAVE A SCHOOL-AGE CHILD WHO HAS BLINDNESS, EPILEPSY, BEHAVIORAL PROBLEMS, YOU SHOULD THINK ABOUT JUVENILE BATTEN DISEASE AND THERE AREN'T A LOT OF OTHER THINGS THAT THE PERSON SHOULD HAVE, BUT WHEN WE GET SOME OF THOSE UNCERTAIN ANSWERS OR SOMEONE HAS A NEW DIAGNOSIS, SOMETIMES WE CAN LOOK TO REGISTRIES TO THAT COMPREHENSIVE DATA TO BE ABLE TO GIVE MORE BACKGROUND ON WHAT TO EXPECT, WHAT ELSE TO CONSIDER, OTHER DIAGNOSES THAT HAVE BEEN CONSIDERED FOR PEOPLE SO HOPEFULLY WE CAN PUT THOSE THINGS TOGETHER AND WORK TOGETHER TO GET PEOPLE FASTER DIAGNOSES AND TO BE ABLE TO GET THEM TO TREATMENTS FASTER TOO. >> YEAH, THANK YOU VERY MUCH. >> I HAVE A TWO-POINTED QUESTION. I'M MARK BECKWITH FROM NEVIS OUTREACH INCORPORATED. THERE'S LOTS AND LOTS OF TALK ABOUT REDUNDANCY IN REGISTRIES SO DR. AUGUSTINE MENTIONED THREE SHE'S TRYING TO ROPE TOGETHER I IMAGINE, AND WE'RE NO STRANGER TO THAT. THERE WAS A PAPER PUBLISHED ON FOUR NEVIS REGISTRIES, OUR PUSH IS TO MAKE THERE BE ONE GRAND AND MASSIVE DATABASE FOR PEOPLE WHO HAVE THIS ALL OVER THE WORLD. MY QUESTIONS ARE, IS THERE ANY MOVE AT THE NIH TOWARD CENTRAL IRB AS WAS ALLUDED TO EARLIER BECAUSE THAT'S GOING TO MAKE OUR JOB THAT WHOLE LOT EASIER TO GET THAT KIND OF APPROVAL, AND THE OTHER QUESTION IS DO ANY OF THESE FUNDING OPPORTUNITIES, DR. SELBY, ARE THEY ABLE TO ACCOMMODATE INTERNATIONAL PARTNERS? >> I THINK IN TERMS OF CENTRAL IRB IT'S A MATTER OF TIME BEFORE IT'S STANDARD FOR MULTI-CENTERED TRIALS BUT IT DOESN'T MAKE IT -- YOU STILL HAVE TO GO THROUGH THE PROCESS, BUT THE GOOD NEWS IS YOU SHOULD GO THROUGH IT JUST ONCE. >> RESEARCHERS FROM OUTSIDE MAY APPLY TO PCORI PROVIDED THE RESEARCH REQUEST IS OF INTEREST TO U.S. POPULATIONS, SO WE HAD A CALL FOR RESEARCH IN RARE DISEASES A FEW MONTHS AGO AND MADE IT CLEAR OFTENTIMES YOU NEED TO GO INTERNATIONALLY TO GET ENOUGH PATIENTS FOR THE PURPOSES YOU WERE SAYING BECAUSE YOU REALLY WANT ONE AS LARGE AS POSSIBLE REGISTRY. SO YES. >> ACTUALLY MY QUESTION IS ABOUT INTERNATIONAL AS WELL, SO THANK YOU. I WONDER IF ANY OF YOU HAVE THOUGHT ABOUT, MUCH LESS STARTED, TO TRY TO TAKE YOUR DOMESTIC REGISTRIES INTERNATIONAL? IT'S SOMETHING WE'RE IN THE PROCESS IN FTD, DESIGNING OUR FIRST, WE HAVEN'T GONE LIVE YET BUT HAVE BEEN URGED TO WALK BEFORE WE CAN RUN AND ABSOLUTELY KNOW WE'RE GOING TO NEED TO HAVE AN INTERNATIONAL REGISTRY TO RUN SUCCESSFUL TRIALS AND I'D LOVE TO HEAR WHAT THE CHALLENGES ARE OR IF YOU STARTED TO ADDRESS THEM. >> WE'RE CONCERNED ABOUT THAT. THE U.S. IS PRETTY ENTER NATIONAL ON ITS OWN, STARTING WITH LANGUAGE, CULTURE DIVERSITY BUT NOT LIKE NATION TO NATION. THAT WAS IMPORTANT TO US AS WE JOINED THE CENA PLATFORM, A PLACE WHERE THE UNDERLYING ARCHITECTURE ALLOWED DIFFERENT LANGUAGES TO BE USED. AND SO THAT WAS FUNDAMENTAL IN THE PLATFORM. NOW ALL WE HAVE TO DO IS TRANSLATE, AND ADAPT TO THE CULTURES, AND ADAPT TO THE LEGAL REQUIREMENTS OF OTHER COUNTRIES, SO IT'S NOT AS STRAIGHTFORWARD AS JUST CHANGING, YOU KNOW, S'S TO C'S, BUT IT'S TRUE WITH THE CHRONIC DISEASES AS WELL, THERE'S A LOT TO BE CONCERNED ACROSS THIS WORLD. I MADE A NOTE EARLIER, HEALTH IS ALL OF US, WE ALL HAVE SOMETHING TO CORRECT, IN THE U.S., 10% OF THE POPULATION, TO KNOW IT ALL OR THINK WE CAN QUANTIFY IT ALL IS FOOLISH. >> WE TRANSLATED OUR REGISTRY INTO SPANISH, FRENCH, ITALIAN, AND WE ARE AT A POINT WHERE WE NEED TO GO BACK AND MAKE CHANGES, WE FIND THE COST OF TRANSLATING IS JUST PROHIBITIVE RIGHT NOW. SO THAT'S ONE OF THE CHALLENGES OF ENGAGING. MAYBE LOOK AT WHERE -- IF YOU'RE GOING TO GO TO THE EXPENSE PERHAPS PRIORITIZE LIKE SPANISH WOULD BE APPLICABLE TO MANY PATIENTS IN THE UNITED STATES AS WELL, THAT MAY BE ONE THAT WE COULD TEND TO PRIORITIZE SPANISH OVER OTHER LANGUAGES. >> AND AND/OR PARTNER TOO, AN ADVOCACY GROUP. >> WE WORK WITH GROUPS IN NINE OTHER COUNTRIES, AND THEY ARE WATCHING US TO GO FIRST, ON ONE HAND WE'RE IN A GREAT POSITION, BUT WE'RE LOOKING FOR SOME KIND MUCH MODEL TO SHOW US THE ROAD MAP FOR HOW TO GO FROM HERE THERE THERE, DO WE BUILD SEPARATELY KEEPING AN EYE TO BUILD SOMETHING THAT CAN CROSS-TALK EVENTUALLY OR MORE SUCCESSFUL TO BUILD ONE AND REACH OUT INTO THE OTHER COUNTRIES? I'VE NOT HEARD SUCCESSFUL STORIES. >> I CAN COMMENT ABOUT ALS. WHEN WE STARTED TO TALK ABOUT THE NATIONAL REGISTRY HERE, THERE WHY IN EXISTENCE SEVERAL EUROPEAN REGISTRIES, A WELL-ORGANIZED EUROPEAN ALS REGISTRY, A STRONG IRISH REGISTRIES BECAUSE ONE OF THE SINGLE NEUROLOGIST SEES ALMOST ALL OF THE PATIENTS IN IRELAND, STRONG REGISTRIES WHICH WE BROUGHT IN AND GOT MANY SCIENTISTS AND CLINICIANS TO ENGAGE SO RISK FACTOR MODULES ARE SIMILAR. IT'S BEEN SET UP, THEY ARE COMPATIBLE AND WILL FEED IN TOGETHER BUT THEY ARE NOT ONE REGISTRY BECAUSE THE ISSUES OF LANGUAGE AND CULTURE ARE ENORMOUS, I KNOW THE EUROPEAN REGISTRY STRUGGLES WITH THAT, THAT'S HOW THE ALS REGISTRY EVOLVED WITH MULTIPLE OF THEM LIKELY TO FEED INTO EACH OTHER. >> WE RUN A LARGE TRAUMATIC BRAIN INJURY PERSPECTIVE STUDY AND INITIALLY WORKED WITH EUROPEANS, HOPED TO HAVE ONE BIG DATA CENTER, BUT THAT REALLY DIDN'T HAPPEN IN THE END, FOR A LOT OF REASONS LUCIE SAID. SOME THINGS WE WERE ASKING WERE COMPLETELY NONSENSE IN THE EUROPEAN COMMUNITY, AND SO IT'S REALLY THE PLAN B WAS TO MAKE SURE THEY WERE COMPATIBLE AND YOU COULD PUT THEM TOGETHER. WHICH I THINK IS PROBABLY A REASONABLE FIRST STEP TO GO. NOW, ANYBODY IN HERE KNOW ABOUT THE RARE DISEASE CONSORTIA? THEY HAVE INTERNATIONAL COMPONENTS, RIGHT? [OFF MIC] >> YOU NEED TO USE THE MICROPHONE SO THE WEBCAST PEOPLE CAN HEAR >> FOR SOME REASONS PEOPLE ARE ALLUDING TO, THERE'S DUPLICATION IN DATA, CAN WE SET UP INTERNATIONAL NETWORKS TO BE AWARE OF REDUNDANCY, REDUCE REDUNDANCY TO SHARE DATA AND PERHAPS NON-COMPETITIVE SPACE AND I WOULD ENCOURAGE ASSOCIATIONS HERE TODAY TO LOOK INTO IT, A MULTI-STAKEHOLDER CONSORTIA, TO ENSURE THINKS ARE NOT BEING DUPLICATE AND BRINGS INDUSTRY TO THE TABLE, AS WELL AS ADVOCACY AND IT'S SPEARHEADED AND THAT'S ALL I HAVE TO SAY. >> I WILL SAY THAT ANYTHING IN EUROPE, YOU HAVE TO HIRE A GOOD LAWYER, IT'S GOING TO AFFECT YOUR INTEROPERABILITY. TRANSLATION WAYS, TRADITIONAL TRANSLATION SERVICES IS EXPENSIVE, YOU CAN SAVE A TON OF MONEY BY GOING THROUGH SOME OF THE MORE WBBY SERVICES TO GET A FIRST DRAFT AND WORK WITH A LEAD ADVOCATE NATIVE IN LANGUAGE TO POLISH THE TRANSLATION. MANY INDICATIONS YOU HAVE TO GO THROUGH A LOCAL BOARD, DO SOME GOOGLE WORK OR YOU'LL SPEND A LOT OF MONEY ON LAWYERS LATER. IT'S POSSIBLE, IT'S JUST NOT EASY, IT'S VERY HARD TO REPLICATE. THE STUFF WE'RE DOING IN LUXEMBOURG GIVES NO BENEFIT TO THE STUFF IN THE NETHERLANDS DESPITE HOW CLOSE THEY ARE CULTURALLY, GEOGRAPHICALLY AND LINGUISTICALLY, THERE'S NOT AN EASY ANSWER. BUT I WOULD SAY IN ANY POPULATION, YOU'RE GOING TO GET DIFFERENT LEVELS OF ENGAGEMENT WITH YOUR REGISTRIES ARE STUDY, PEOPLE COMMITTED ARE WORTH THE EFFORT OF INTERNATIONALIZATION. IN LUXEMBOURG A COHORT OF PEOPLE WITH PARKINSON'S DISEASES IN INVESTED, GETTING THEM INTO THE STUDY IS WORTH THE EFFORT OF PROCESS BUT IN THE ABSENCE OF A PARTNER ON THE GROUND SCIENTIFIC PARTNER WITH AN IRB YOU CAN GO THROUGH AN ADVOCATE TO TRANSLATE, IT'S A HIGH TRANSLATION COST. >> CHDI FOR HUNTINGTON HAS AN INTERNATIONAL REGISTRY, THEY MIGHT BE THE PEOPLE TO TALK TO, I DON'T KNOW IF ANYONE IS HERE. >> THANKS, PAUL. THESE ARE SUCH IMPRESSIVE PRESENTATIONS, ON SUCH IMPRESSIVE BODIES OF WORK, AND THANK YOU ALL FOR BOTH PRESENTATIONS AND THE GREAT WORK FOR YOUR PATIENTS. SO IMPRESSIVE IN FACT I FEEL THE NEED TO SAY SOMETHING ON BEHALF OF THE PEOPLE IN THE ROOM, OR ON THE WEBCAST, THAT MAY FEEL A LITTLE INTIMIDATED OR OVERWHELMED BY IMPRESSIVENESS OF WORK YOU'VE DONE. I'D LIKE TO SAY TO EMPHASIZE WHAT WE HEARD FROM SEVERAL SPEAKERS, ESPECIALLY LUCIE AND ERIKA, THAT THIS IS NOT AN ALL OR NOTHING PROPOSITION HERE. YOU DON'T NEED TO GET A BIG CONTRACT OR A BIG GRANT TO BUILD YOUR BOEING 767 AT THE OUTSET. YOU CAN REALLY BUILD YOUR LITTLE BIPLANE WHILE YOU FLY IT, AND YOU CAN START YOUR PATIENT REGISTRY, THE DEMOGRAPHIC PART THAT ERIKA EMPHASIZED IS SO IMPORTANT AND LUCIE TOO IN RECRUITING FOR YOUR RESEARCH, BASIC TRANSLATIONAL AND CLINICAL. SO WE STARTED OURS ON AN EXCEL SPREADSHEET AND NOW IT'S ON A BIG COMPUTER DATABASE, BUT YOU CAN DO GREAT THINGS FOR YOUR PATIENTS WITH BARE BONES. AND, YOU KNOW, COMMUNICATE WITH YOUR ACADEMIC INVESTIGATORS, FOR EXAMPLE, TO BUILD A NATURAL HISTORY DATABASE WITH THE GRANTS THAT YOU'RE GIVING THEM AND SO FORTH. I WONDERED IF ANY OF THE PANELISTS WOULD LIKE TO COMMENT FURTHER ON THE IMPORTANCE OF GETTING STARTED, DON'T WAIT FOR THE 767. GET GOING. >> I'D BE QUICK TO COMMENT, WE WENT THROUGH A LOT WHEN WE WERE THINKING ABOUT THE REGISTRY AND HAD SO MANY DIVERGENT VIEWS, EVERYONE HAD THEIR BEST GUESS, AND IF YOU GET IT PERFECT IT WILL TAKE FOREVER SO I COMPLETELY ECHO THAT. IT'S WHERE I COME FROM THAT I THINK THERE'S BEEN A LOT OF SUCCESS WITH MULTIPLE REGISTRIES, AGAIN THE TERM REGISTRY ESPECIALLY FOR THE COMMUNITY, WE HAVE BIOREPOSITORIES THAT WE STARTED, WE HAVE REGISTRIES, WE HAVE NATURAL HISTORY, IN A WAY THEY ARE ALL INTERLINKED SO I DON'T KNOW THAT IT'S POSSIBLE TO NECESSARILY GET ONE THING, HAVING MULTIPLE THAT CAN TALK TO EACH OTHER COULD BE JUST AS EFFECTIVE. >> ONE THING I WOULD ADD IS FIND A DOC THAT TREATS THE CONDITION THAT YOU'RE FOCUSED ON, AND IF THAT DOC DOESN'T HAVE ONE FIND ANOTHERRING DOC DOC NEAR IT ORE WITH AN M.PH., FOCUS ON EPIDEMIOLOGY IN PARTICULAR, WITH A COMBINATION OF DOCTORS THAT UNDERSTAND CLINICAL EPIDEMIOLOGY, I SPENT SEVEN MONTHS WITH THESE DOCSES, PHYSIATRY, ORTHOPEDIC SURGERY, NEUROSURGERY, PT, OT, DONATING TIME TO DEFINE A REGISTRY, ALL IN WITHOUT DIME ONE BEING SPENT, I USED FREE CONFERENCE CALL.COM TO GET THEM TOGETHER ON THE PHONE AND GOOGLE CALENDAR TO SCHEDULE THEM, GOOGLE GROUPS TO CREATE A DISTRIBUTION LIST, ALL FOR FREE. AT SOME POINT YOU'VE GOT TO DEFINE THE DATABASE, YOU NEED EXPERTISE, YOU NEED TO DO THAT IN SOMETHING THAT WILL SCALE BUT YOU CAN GET GOING BECAUSE JUST ENGAGING WITH YOUR, YOU KNOW, RESEARCH FOCUS DOC ON THIS TOPIC CAN GET YOU SO FAR, ESPECIALLY IN CONJUNCTION WITH SOCIETIES AND WHAT NOT. TREMENDOUS OPPORTUNITIES. >> I WOULD AGREE COMPLETELY WITH THAT. SO YOU NEED PEOPLE WHO ARE PASSIONATE AND COMMITTED TO CONTINUING TO GO FORWARD AND SOMETIMES THAT INVOLVES MONEY, OFTEN IT DOES NOT. SO PATIENTS BY DEFINITION ARE GOING TO BE PASSIONATE ABOUT THEIR OWN CONDITION, AND IF YOU CAN FIND PHYSICIAN PARTNER, IF YOU NEED A PHYSICIAN PARTNER, THEN FINDING SOMEONE WHO IS PASSIONATE ABOUT WHAT'S HAPPENING ALSO WILL GO A VERY LONG WAY. THERE WILL BE EBBS AND FLOWS IN TERMS OF GOOD FUNDING YEARS AND NOT BUT YOU WANT PEOPLE COMMITTED TO SEEING THIS THROUGH AND HAVING A LONG-TERM PLAN FOR THOSE TOUGH YEARS, WHAT ARE THE ABSOLUTE CORE ELEMENTS THAT ARE NEEDED, AND WHAT ARE THE THINGS THAT ARE HELPFUL TO HAVE AS WELL. IF THERE'S ONE PLACE TO INVEST, RELATIVELY EARLY ON, IT PROBABLY IS ON THE DATA MANAGEMENT SIDE BECAUSE YOU MAY -- YOU MAY WITHIN YOUR OWN FOUNDATION HAVE SOMEONE WITH PARTICULAR EXPERTISE IN DATA MANAGEMENT BUT IF YOU DON'T THE PHYSICIANS MAY OR MAY NOT HAVE THAT EXPERTISE, HAVING SOMEONE WITH KNOWLEDGE ABOUT BUILDING REGISTRIES, DATABASES, CROSS-TALK, YOUR GOALS MAY CHANGE OVER TIME, WHAT YOU NEED FROM YOUR REGISTRY WILL LIKELY CHANGE OVER TIME AND HOPEFULLY IT WILL GROW AND THE DATA MANAGEMENT PIECE WILL HELP YOU BE ABLE TO MAKE THAT GROW, WHEREAS YOUR OTHER PARTNERS MAY OR MAY NOT HAVE THAT SKILL SET SO IF THERE'S ANY INVESTMENT TO BE MADE THAT MIGHT BE THE ONE UP FRONT WHETHER THE DOLLARS SHOULD GO. >> I WOULD ADD ONE OF THE THINGS THAT JOHN SAID WAS THIS REDEFINING DISEASE TAXONOMY, REDEFINING HOW WE LOOK AT DISEASE JUST AS YOU JUST SAID, THE EXPERTISE YOU NEED MIGHT BE DATABASE. THAT'S NOT A DOCTORATE, NOT A POSTDOC, IT'S YOUR TEENAGER AND THEIR PHONE AND THEIR COMPUTER, PRESCHOOLER, I DON'T KNOW, YOU NEED TO SOMETIMES JUST STEP AWAY FROM IT AND TAKE THOSE BABY STEPS. I'M REMINDED WE HAVE TO WATCH THE CLOCK. >> I HAD A COMMENT ABOUT INTERNATIONAL REGISTRY. AND HOW TO PURSUE THAT. WE'RE EARLY IN THE REGISTRY PHASE, I REPRESENT A (INAUDIBLE) DISEASES, IT'S ONE THAT LARGELY INCREASINGLY AFFLICTS JEWISH JEWS, SECOND LARGEST CONCENTRATION OF PATIENTS IS IN ISRAEL OUTSIDE THE UNITED STATES, HAPPENS THEN ON LARGELY THOSE TWO COMMUNITIES OF PATIENTS OUTSIDE THE UNITED STATES, AND IN THE U.S., REALLY HADN'T BEEN CONNECTED WITH EACH OTHER BOTH IN A COMMUNITY SENSE OR REALLY ANY SENSE, AND LARGELY BECAUSE THAT'S NOT WORKING FOR OUR COMMUNITY, WE WANTED TO INCREASINGLY UNITE THE COMMUNITIES AND AS WE BEGAN DEVELOPING THE REGISTRY WE REALIZED THE REGISTRY COULD BE ONE OF THE WAYS OF DOING THAT AND THROUGH A VARIETY OF WAYS DEVELOPING COLLABORATION WE REALLY USE ONE OF THE LEAD CLINICAL RESEARCHERS THE ISRAELI COMMUNITY WORKING WITH THE LEAD RESEARCHERS IN THIS COMMUNITY TO HELP IN BUILDING THAT REGISTRY TOGETHER, WHICH REALLY TURNED OUT TO BE VERY SUCCESSFUL, AND SO THE REGISTRY ITSELF IN THE FORMATION OF IT, THE WAY THE REGISTRY QUESTIONS THEMSELVES WERE BUILT, HELPED TO BECOME A REALLY POSITIVE FUNCTION FOR UNITING THOSE COMMUNITIES, WHEN THE REGISTRY ITSELF WAS LAUNCHED IT BECAME ONE OF THE INTERNATIONAL SORT OF FUNCTIONS FOR THE WHOLE COMMUNITY. I THINK FOR US WE FEEL LIKE IT'S SORT OF A HOPEFUL NEW STEP, AND WE'LL SEE HOW THE ENROLLMENT PROCESS GOES. I JUST OFFER IT AS ANOTHER WAY OF THINKING ABOUT IT. >> LAST QUESTION. COMMENT? >> CHUCK MOHAN, WE'RE REPRESENTING OVER 250 PRIMARY MOTOCHONDRIAL DISEASES, THE QUESTION I HAVE IS HAVE ANY OF YOU CONSIDERED OR HAVE YOU EXPERIENCED CREATING A SEPARATE DISEASE-SPECIFIC PORTAL FOR YOUR REGISTRIES? >> ENGAGING, BETTER ENGAGING PATIENT PARTICIPATION THROUGH SIMILAR SYMPTOMS, PHENOTYPES. OKAY, I'LL ANSWER MY QUESTION. [ LAUGHTER ] WE HAVE, AND WE'RE FINDING IT VERY SUCCESSFUL, NOT ONLY IN ENGAGING THE PATIENTS BUT ALSO IN BREAKING DOWN BARRIERS BUILT BY EGOS AND OWNERSHIP, SO WE'RE USING A SIMILAR PLATFORM, PRIVATE ACCESS AND PEER PLATFORM THROUGH GENETIC ALLIANCE, WE FOUND THAT BY IDENTIFYING AND BUILDING THE PLATFORM, AND GIVING TO THE MITOCHONDRIAL COMMUNITY, WE'VE BEEN ABLE TO ENGAGE OTHERS TO PARTICIPATE. VERY SIMILAR, WE HAVE FOUND THE MOST ATTRACTIVE MECHANISM TO MOTIVATE PEOPLE TO PARTICIPATE IN A REGISTRY IS THE PRIVACY, ENABLING THOSE REGISTRANTS TO BE ABLE TO SELECT THE VARIOUS LEVELS OF PRIVACY TO THEIR DATA, AT EVERY QUESTION. AND THE SECOND SUBCOMPONENT TO THAT WOULD BE THE ABILITY TO KEEP IT DYNAMIC. AS YOU INDICATED, TO BE ABLE TO HAVE NO ACCESS TO MY DATA TODAY, BUT TO GET A COMFORT LEVEL WITH THE REGISTRY AND PROVIDE ADDITIONAL ACCESS TOMORROW. THANK YOU. >> YEAH, THANK YOU, CHUCK. AND I WOULD ADD ON THAT, I REPRESENT A REALLY SMALL DISEASE AREA, SO WE DON'T HAVE KIND OF MULTIPLE DISEASES, BUT WE'RE PUTTING MULTIPLE FACES ON THAT SAME DATABASE, SO A CAREGIVER OF A JUVENILE MAY ANSWER DIFFERENT THAN A MOTHER OF AN INFANTILE VERSION, AND WE THOUGHT YOU DON'T HAVE TO HAVE EVERY PARTICIPANTS ANSWER ALL 200 OR 2000 QUESTIONS FOR THAT DATA TO BE VALID. IF THEY ANSWER 20, IF THEY ANSWER 150, IF THEY SKIP SOME AND GO TO OTHERS, THAT'S STILL VALID DATA. YOU WANT TO GO BACK AND ENCOURAGE THEM TO FILL IN BLANKS WHERE APPROPRIATE BUT EVERY PIECE ADDS TO THIS DATA SET THAT WE IS THEN DO THESE THINGS ON OVER TIME, AFTER WE GATHER THAT DATA. >> THANKS, DEAN. IN THE INTEREST OF KEEPING US ON TIME, GO AHEAD, DR. SELBY, WRAP US UP. >> I WANT TO THANK THE PANELISTS AT NINDS FOR SPONSORING AND THE AUDIENCE FOR BEING PATIENT LISTENERS AND ADDING GREAT COMMENTS. MUCH OF WHAT I'M GOING TO SUMMARIZE, WHAT WAS SAID FROM THE MICROPHONES, WE FOUND OURSELVES IN THE MIDST OF A MOVEMENT, MAYBE A REVOLUTION IN TERMS OF DATA, REPRESENTED BY THE FORMATION OF REGISTRIES, OFTEN DRIVEN BY PATIENTS AND FAMILY MEMBERS, AND THIS LEADS AMONG OTHER THINGS TO NEW KINDS OF RESEARCH, RESEARCH THAT JUST WOULDN'T HAVE BEEN THOUGHT OF BEFORE, IN PART BECAUSE OF PATIENTS ARE INVOLVED IN ASKING QUESTIONS, IN PART BECAUSE THERE'S NEW KINDS OF DATA SUCH AS THE EXAMPLES THAT JOHN GAVE, FROM USING WEARABLE TECHNOLOGIES AND RESEARCH KIT AND ITEMS LIKE THAT. MANY PERSPECTIVES AS REGISTRIES GET STARTED OUT OF PATIENT, RESEARCHERS, THE ORGANIZATION, SOME AIM FOR CURES, SOME AIM FOR NEW TREATMENTS, FIRST TREATMENTS EVEN, SOME FOR SYMPTOM RELIEF, SOME AIM FOR QUALITY AND SAFETY. VERY IMPORTANT TO NOTE, IT WAS MENTIONED FROM THE MICROPHONE, WE DIDN'T TALK TODAY ABOUT A LOT OF REQUIREMENTS, ALL OF WHICH YOU HAVE TO HAVE IN PLACE, INCLUDING AN INTERNATIONAL FOCUS, ON DAY ONE. WE TALKED ABOUT IDEAS FOR -- WE TALKED ABOUT WHAT WOULD BE IDEAL AND WHERE IT WOULD BE REASONABLE AND OUR EXPERIENCES POINT US WANTING TO GO LONGER TERM. FOR YOU JUST STARTING OUT IN THE GARAGE, WITH YOUR SPOUSE, TO START A NEW REGISTRY, JUST HOPEFULLY THIS DISCUSSION MADE YOU AWARE OF SOME OF THE THINGS THAT YOUR FORERUNNERS HAVE RUN INTO AND RECOGNIZED AND ARE DEALING WITH SO WHEN YOU SEE OPPORTUNITIES AROUND THE AREA OF LINKING WITH LARGER POPULATION DATABASES, AROUND THE AREA OF STANDARDIZING DATA, AROUND THE AREA OF TALKING TO INTERNATIONAL FRIENDS, YOU'LL SEE IT. COMMON DATA ELEMENTS CERTAINLY CRUCIAL. THE IMPORTANCE OF BROADER DATA WAS POINTED OUT, IF YOU'VE GOT A BROADER POPULATION FROM WHICH YOUR PATIENTS COME, THAT BEGINS TO GIVE YOU THE ANGLE FOR UNDERSTANDING WHAT PRECEDES GETTING THE DIAGNOSIS OF THE DISEASE, YOU CAN BEGIN IDENTIFYING DIFFERENT MARKERS TO SEE HOW PREDICTIVE THEY ARE FOR WHO IS GOING TO EVENTUALLY GET A DIAGNOSIS. YOU CAN'T HAVE THAT IF YOU'RE ONLY WORKING FOR PEOPLE -- WITH PEOPLE WHO HAVE ALREADY GOTTEN THE DIAGNOSIS, THE IMPORTANCE OF THINKING INTERNATIONALLY HAS BEEN MENTIONED, I THOUGHT JOHN'S POINT WAS A PEARL ABOUT TRANSLATION SERVICES, GET YOUR FIRST DRAFT OF THE TRANSLATION CHEAPLY BECAUSE THAT'S WHAT ALL PEOPLE ALWAYS SAY, IT'S SO DARN EXPENSIVE, I CAN ONLY DO SPANISH, BUT IF YOU CAN GO ON THE WEB AND GET SOMETHING CHEAP AND HAVE SOMETHING REFINE IT, LOVE IT. SO THAT'S REALLY IT, PAUL. I THINK IT WAS A GREAT DISCUSSION, AND I HOPE WE INSPIRE ANOTHER TWO OR THREE HUNDRED REGISTRIES ON THE BASIS OF OUR DISCUSSION THIS AFTERNOON. [APPLAUSE] SO THIS WILL BE WHAT ARE WE'LL CALL THE STRAGERY SECTION OF THE FORUM. I KNOW A GOOD PART OF THE OPPORTUNITY THAT PEOPLE LOOK FORWARD TO HERE IS THE CHANCE TO INTERACT WITH PROGRAM DIRECTORS, AND ONE OF THE THINGS THAT PROGRAM DIRECTORS ARE VERY INVOLVED WITH HERE AT NINDS AND NIH IN GENERAL IS HELPING SET PRIORITIES AND MAKE DECISIONS, AND WE'RE ALWAYS LOOKING FOR INPUT FROM PEOPLE FOR HOW TO MAKE THE WISEST DECISIONS POSSIBLE ABOUT HOW TO USE THE TAXPAYERS MONEY. AND SO IN ORDER FOR YOU TO BE ABLE TO INTERACT WITH US AND HELP US MAKE BETTER DECISIONS WE THOUGHT IT WOULD BE WORTHWHILE TALKING ABOUT THE WAY, THE NORMAL WAY THINGS WORK AT NIH, IN TERMS OF HOW WE GO ABOUT MAKING DECISIONS AND WHAT ARE THE THIS ACTIVITIES THAT KEEP US BUSINESS. UP FRONT, ONE OF THE MOST IMPORTANT THINGS TO KNOW ABOUT NIH IS THAT THAT I WORD IN THE MIDDLE IS PLURAL. IT'S NATIONAL INSTITUTES OF HEALTH, 27 INSTITUTES AND CENTERS, 24 HAVE GRANT-MAKING AUTHORITY. JEREMY USED TO SAY WHEN YOU'VE SEEN ONE NIH INSTITUTE, YOU'VE SEEN ONE NIH INSTITUTE. EVERY INSTITUTE WAS CREATED AT SOME POINT IN CONGRESS, SOME POINT BY CONGRESS, FOR VARIOUS REASONS, EACH WITH A COMPELLING MISSION AT THE TIME, NAMES AND MISSIONS AND IDENTITIES HAVE CHANGED OVER THE YEARS. NINDS HAS BEEN LOSING ITS SENSES, BACK IN THE '50s THEY TOOK HAVE VISION AND CREATED NATIONAL EYE INSTITUTES, IN THE 80s THEY TOOK AWAY DEAFNESS, NATIONAL INSTITUTE OF BIOENGINEERING, IN THE LAST RECENT TIME THE CENTER FORMERLY KNOWN AS NCCAM HAS BEEN RENAMED THE NATIONAL CENTER FOR COMPLEMENTARY AND INTEGRATIVE HEALTH, IT'S AN EVOLVING COLLECTION OF INSTITUTES. AS I SAID, IT'S VERY IMPORTANT TO KNOW WHICH ONE YOU'RE WORKING WITH BECAUSE WE ALL HAVE DIFFERENT THINGS WE VALUE, DIFFERENT GRANTS WE SUPPORT, DIFFERENT WAYS WE GO ABOUT MAKING DECISION SO I'M GOING TO KEEP AT THIS A HIGH LEVEL AND TALK GENERICALLY ABOUT THE WAY THAT DECISIONS GET MADE ACROSS NIH. ONE THING YOU NEED TO KNOW IS THAT EVERYTHING WE DO REVOLVES AROUND THIS GRANT CYCLE, AND JUST AS IT TAKES NINE MONTHS TO MAKE A BABY UNDER HAPPY CIRCUMSTANCES, THAT'S WHAT IT TAKES TO GET A GRANT, ALTHOUGH IN THE CASE OF THE GRANT IF IT'S DELIVERED EARLY THAT'S ALWAYS A GOOD SIGN. SO THERE ARE SOME DIFFERENCES. I'D LIKE TO POINT OUT THREE DIFFERENT CATEGORIES OF ACTIVITIES WE CAN THINK ABOUT, ONE IS IN THE PLANNING PHASE BY THE P.I. OR TEAM THAT IS DEVELOPING A PROPOSAL. AND THEN THERE'S THE INSTITUTION ORGANIZATION WHO SUBMITS IT, SO THAT'S THE FIRST PART. ANOTHER LARGE PART, WE'LL SPEND SOME TIME TALKING ABOUT THIS, THE REVIEW CYCLE, APPLICATIONS NOW I THINK EVERYTHING COMES IN ELECTRONICALLY THESE DAYS, WHICH IS AN ENORMOUS TECHNOLOGICAL IMPROVEMENT OVER ONLY A DECADE AGO. SO THEY COME INTO A DIVISION OF RECEIPT AND REFERRALS WHERE THEY GET ASSIGNED BOTH TO ONE OF THOSE 24 GRANT-MAKING INSTITUTES OR CENTERS, THEY GET ASSIGNED TO A REVIEW GROUP, AND THEN THEY GET REVIEWED AND THERE'S A SCORE AND SUMMARY STATEMENT, WE'LL SAY MORE ABOUT THAT. AND THEN WE GET TO THE INTERESTING PART WHERE THERE IS THE FUNDING PLAN, THE DECISION. THIS IS WHERE PROGRAM STAFF, INSTITUTE STAFF AND COUNSEL ADVISE THE INSTITUTE DIRECTOR WHO MAKES A FUNDING DECISION AND THEN AT SOME POINT THIS AWARD IS MADE. THIS LASTS NINE MONTHS BUT HAPPENS THREE TIMES A YEAR, THREE STAGGERED CYCLE. THE MOST INTERESTING IS NOW, AT THE END OF THE FISCAL YEAR, WE HAVE TO SPEND THE MONEY OR IT GOES BACK TO THE TREASURY SO THERE'S A SCRAMBLE BETWEEN NOW AND OCTOBER TO USE THE MONEY AS WISELY AS POSSIBLE. WHEN SOMEONE HAS AN IDEA, AND IS THINKING ABOUT A GRANT, HOW DO THEY ACTUALLY COME IN? I THINK ONE COMMON MISCONCEPTION IS THAT IN ORDER TO SUBMIT AN APPLICATION ON A PARTICULAR TOPIC, THERE HAS TO BE A SPECIFIC SOLICITATION ON THAT TOPIC. I'VE TALKED TO THE SURPRISINGLY LARGE NUMBER OF PEOPLE WHO THINK I WISH THEY WOULD PUT OUT AN RFA, ALL APPLICATIONS COME IN IN RESPONSE TO FUNDING OPPORTUNITY ANNOUNCEMENTS BUT MANY ARE GENERIC SO THIS LAST BULLET POINTS OUT PARENT FUNDING OPPORTUNITY ANNOUNCEMENTS, THESE ARE FOR THE STANDARD KINDS OF GRANT MECHANISMS WE USE, I'LL SHOW YOU AN EXAMPLE IN THE NEXT SLIDE, AND THEY CAN BE ON ANY TOPIC IN THE MISSION OF A PARTICULAR NIH INSTITUTE. THIS IS HOW THE MAJORITY OF APPLICATIONS COME IN. THESE PARENT FUNDING OPPORTUNITY ANNOUNCEMENTS HAVE THREE RECEIPTS PER YEAR, SO THERE ARE PLENTY OF OPPORTUNITIES FOR PEOPLE TO COME IN, AS LONG AS WHAT WE WANT TO DO IS WITHIN THE MISSION OF SOME INSTITUTE. THAT IS A MESSAGE THAT I'M GOING TO KEEP REINFORCING AS IT'S VERY IMPORTANT TO CHECK WITH THE INSTITUTE THROUGH THEIR WEB PAGE, THROUGH ACTUAL LIVING HUMAN BEINGS LIKE PROGRAM DIRECTORS AT THOSE INSTITUTES, TO FIND OUT HOW A PARTICULAR RESEARCH IDEA IS ALIGNED WITH THE MISSION OF THAT PARTICULAR INSTITUTE. THERE ARE SOME FUNDING OPPORTUNITY ANNOUNCEMENTS THAT WILL SOLICIT RESEARCH IN SPECIFIC AREAS, OFTEN CALLED REQUESTS FOR APPLICATIONS, TYPICALLY ONE-TIME SET-ASIDE POTS OF MONEY WHERE THE INSTITUTE IDENTIFIED A PARTICULAR RESEARCH PRIORITY, SOMETHING WHERE WE THINK WE CAN REALLY MAKE A BIG DIFFERENCE, AND SO WE'LL DO THOSE SOLICITATIONS. PART OF WHAT I'M GOING TO TALK ABOUT TODAY IS HOW DO WE COME UP WITH THESE IDEAS, IT ISN'T JUST SOME ONE PROGRAM DIRECTOR IN A ROOM WAKING UP WITH THE IDEA LET'S DO AN RFA ON THIS PARTICULAR TOPIC. IT REALLY IS A PROCESS, AND I WANT TO SHARE A LITTLE BIT OF THAT WITH YOU. NOW, WHEN FUNDING OPPORTUNITIES ARE ANNOUNCED THEY ARE ACTUALLY PUBLISHED IN THE NIH GUIDE, THIS COMES OUT EVERY WEEK, AND IT'S FOR ANY OF YOU OR ANY OF THE PEOPLE YOU WORK WITH WHO MAY BE INTERESTED IN KNOWING IF THERE ARE PARTICULAR FUNDING OPPORTUNITIES. IT'S WORTHWHILE SIGNING UP. YOU CAN JUST GO TO THE NIH GRANTS PAGE, AND CONVENIENTLY ENOUGH GRANTS.NIH.GOV AND YOU CAN FIND IT, THERE'S ALSO A LIST SERVE TO SIGN UP FOR AND THEY DON'T SPAM YOU A LOT. ONCE A WEEK YOU'LL GET AN E-MAIL WITH SUMMARY OF FUNDING OPPORTUNITY ANNOUNCEMENTS NEW THAT WEEK ALONG WITH MANY IMPORTANT POLICY ANNOUNCEMENTS. WE'RE A BIG GOVERNMENT AGENCY. WE HAVE LOTS OF POLICIES. WE DON'T WANT TO GET STALE SO WE CHANGE THE POLICIES A LOT AND SO LUCKILY ANYTIME WE CHANGE A POLICY WE HAVE TO TELL THE PUBLIC ABOUT IT. SO THERE WILL BE AN ANNOUNCEMENT IN THE GUIDE IF THERE IS SOME POLICY THAT CHANGES THAT COULD MAKE YOUR LIFE EITHER EASIER OR MORE COMPLICATED. THEY ARE INTENDED TO MAKE YOUR LIFE EASIER BUT SOMETIMES EASIER IS COMPLICATED. OKAY. HERE IS JUST AN EXAMPLE OF A LISTING, AND I'M JUST POINTING OUT HERE IS ONE THAT NINDS PUBLISHED AT THE END OF AUGUST, AND WE ACTUALLY ANNOUNCED INTEREST IN LARGE RANDOMIZED EMERGENCY CARE CLINICAL TRIALS, SO THAT'S AN EXAMPLE OF ONE SPECIFIC FUNDING OPPORTUNITY THAT WILL BE COMING. OKAY. NOW LET ME TALK ABOUT THE SECOND PART OF THAT TRIANGLE THAT I SHOWED YOU, WHICH IS PEER REVIEW, SO IN HERE, SO SOMEONE FOUND A FUNDING OPPORTUNITY ANNOUNCEMENT AND SOLICITED APPLICATIONS, THEY HAVE COME IN, THEY HAVE BEEN ASSIGNED AND NOW WHAT HAPPENS DURING THIS REVIEW PHASE? THE PROCESS IS ACTUALLY IN PLACE SINCE THE '40s, I SEE HERE A PICTURE OF A STUDY SECTION IN THE '40s. YOU'LL NOTICE LOTS OF PAPER. ALSO NOTICE LOTS OF MALES IN SUITS. MORE RECENTLY, THINGS ARE ELECTRONIC. YOU WILL SEE A MUCH MORE DIVERSE SET OF PEOPLE SITTING AROUND A TABLE, TYPICALLY WITH LAPTOPS. AND THE GOAL OF THIS IS TO GET INDEPENDENT PEER REVIEW OF THE IDEAS. OKAY. SO THIS IS IN THE SLIDES THAT I GAVE YOU. THIS IS JUST FACTS, I'M NOT GOING TO READ THEM TO YOU ABOUT THE REVIEW COMMITTEES. ONE POINT I DO WANT TO MAKE HERE IS THAT THE CENTER FOR SCIENTIFIC REVIEW WHICH DOES MUCH OF THE RESEARCH FOR NIH, THERE ARE 25 SETS OF STUDY SECTIONS ON PARTICULAR TOPICS, EACH OF THOSE INDIVIDUAL SETS, YOU COULD THINK MUCH THAT AS A GENUS WITH MULTIPLE SPECIES, SO IF THE INTEGRATED REVIEW GROUP IS A GENUS, INDIVIDUAL STUDY SECTION IS THE SPECIES, DISINTEGRATED REVIEW GROUP HAS SOMEWHERE BETWEEN 8 AND 12 STUDY SECTIONs SO IF WE SAY AVERAGE OF 10 STUDY SECTIONS, 25 INTEGRATIVE REVIEW GROUPS, 250 STANDING STUDY SECTIONS THAT MEET THREE TIMES A YEAR AT THE CENTERS FOR SCIENTIFIC REVIEW, EACH HAS ON AVERAGE ABOUT 20 MEMBERS, THERE ARE 5000 PEOPLE WHO ARE INVOLVED IN STANDING STUDY SECTIONS AT THE CENTER FOR SCIENTIFIC REVIEW AT ANY GIVEN TIME. ONE OF THE POINTS I WANT TO MAKE IS THAT ONE OF THE WAYS THAT YOU CAN BE HELPFUL TO US IS BY SUGGESTING REVIEWERS AND ENCOURAGING PEOPLE WHO ARE EXPERTS IN FIELDS OF INTEREST TO YOU TO VOLUNTEER TO BE REVIEWERS IF THEY HAVEN'T BEEN, I'LL SHOW YOU HOW AT THE END OF THE TALK. THE CENTER FOR SCIENTIFIC REVIEW IS ALWAYS LOOKING FOR NEW EXPERTS, AS ARE THE REVIEW GROUPS WITHIN ANY INDIVIDUAL INSTITUTE. SO NINDS HAS A SCIENTIFIC REVIEW BRANCH, IT HAS A NUMBER OF STUDY SECTIONS THAT IT RUNS. THE CENTER FOR SCIENTIFIC REVIEW STUDY SECTIONS COVER APPLICATIONS THAT COME IN IN RESPONSE TO THE PARENT ANNOUNCEMENT SO THEY COVER THE VAST MAJORITY OF APPLICATIONS THAT COME INTO NIH. WITHIN THE INSTITUTES THERE ARE MORE SPECIALIZED ONES SO THESE WOULD BE CENTERS. SO FOR EXAMPLE OUR UDALL CENTER FOR PARKINSON DISEASE REVIEWED IN HOUSE IN THE NINDS REVIEW BRANCH, IN ADDITION THERE ARE OTHER APPLICATIONS THAT HAVE SPECIAL ELIGIBILITY REQUIREMENTS SUCH AS VARIOUS CAREER DEVELOPMENT AND TRAINING AWARDS AND THEN OTHER THINGS LIKE CONFERENCE GRANTS OR VARIETY OF CONTRACTS TO GET REVIEWED IN-HOUSE. THIS IS ANOTHER LARGE REVIEW ACTIVITY USING LOTS OF REVIEWERS AND SO WHAT I'M SAYING IS NIH IS ALWAYS ON THE LOOKOUT FOR EXPERTS TO COME AND HELP US REVIEW GRANTS. OKAY. WHAT DO ENTHUSE PEER REVIEWERS DO? I'M GOING TO TALK ABOUT THIS JUST A LITTLE BIT BECAUSE THIS DOES HAVE SOME IMPACT ON HOW WE SET OUR PRIORITIES. I'M A SIMPLE-MINDED GUY FROM VERMONT. REALLY IN MY VIEW THERE ARE THREE QUESTIONS THAT ANY REVIEW PANEL IS ASKING REGARDLESS OF WHAT KIND OF THING THEY ARE REVIEWING. FIRST THEY ARE TRYING TO UNDERSTAND WHAT IS IT THAT SOMEONE WANTS TO DO. IT'S PATHETIC, IF YOU CAN'T FIGURE THAT OUT FROM THE APPLICATION, BUT THERE ARE SOME PEOPLE WHO HAVE A CERTAIN GIFT FOR WRITING IN A WAY THAT'S VERY HARD TO FIGURE OUT WHAT THEY WANT TO DO. THIS IS GENERALLY NOT A GOOD THING. ASSUMING THAT PEOPLE CAN FIGURE OUT WHAT IS BEING PROPOSED, THEY WOULD ALSO LIKE TO KNOW WHY IS IT IMPORTANT, WHY SHOULD TAXPAYER DOLLARS GO TO SUPPORT ANY ACTIVITY, AND THEN ASSUMING THAT THEY KNOW WHAT YOU WANT TO DO AND ARE PERSUADED IT WOULD BE IMPORTANT TO DO THAT, THE OTHER IMPORTANT QUESTION THAT REVIEWERS ARE ASKING IS CAN THIS PARTICULAR SET OF APPLICANTS IN THEIR INSTITUTION WITH THE RESOURCES AND TEAMS THEY HAVE, CAN THEY DO IT. OKAY. THOSE ARE SIMPLE QUESTIONS A GUY FROM VERMONT MIGHT REQUEST. WE'RE IN THE FEDERAL GOVERNMENT, SO WE ACTUALLY GIVE SOMEWHAT DIFFERENT WORDS TO IT, BUT REALLY SO THERE'S A SET OF REVIEW CRITERIA USED, AND THERE ARE DIFFERENT WAYS OF ASKING THE QUESTIONS I JUST MENTIONED. ONE THING THAT IS GOING TO CHANGE FOR THE POSITIVE -- THAT'S BEEN A CHANGE FOR THE POSITIVE IN THE LAST FEW YEARS IS TRYING TO ALIGN THE INSTRUCTIONS FOR PEOPLE APPLYING FOR GRANTS WITH THE INSTRUCTIONS WE GIVE THE PEOPLE WHO ARE REVIEWING THE GRANTS. IF WE'RE GOING TO ASK REVIEWERS TO ADDRESS THESE QUESTIONS LET'S TELL THE APPLICANTS SO THEY CAN PROVIDE THE ANSWERS WITHIN THEIR APPLICATIONS, AND SO THIS IS AN EXAMPLE OF HOW THAT HAPPENS, SO IF WE'RE ASKING WHY IS THIS IMPORTANT, WELL, WE CALL THAT SIGNIFICANCE, AND LO AND BEHOLD THERE'S A SECTION IN THE APPLICATION CALLED SIGNIFICANCE. SO YOU CAN ACTUALLY TELL REVIEWERS AND NIH STAFF WHY YOU THINK THIS IDEA IS IMPORTANT IN SIGNIFICANCE, AND LIKEWISE THERE ARE A VARIETY OF OTHER WAYS PEOPLE CAN DEMONSTRATE THAT THEY HAVE THE EXPERIENCE AND APPROPRIATE TEAMS TO CONDUCT THE RESEARCH THAT THEY WANT TO DO. OKAY. ONE OTHER IMPORTANT THING TO KNOW ABOUT THE WAY REVIEW IS DONE IS THAT TYPICALLY THE SIGNIFICANT NUMBER OF APPLICATIONS FROM TEN TO A HUNDRED ARE REVIEWED BY A GIVEN STUDY SECTION, IT WOULD BE NICE IF EVERY PERSON ON A PANEL HAD TIME TO READ EVERY APPLICATION, VERY THOUGHTFULLY AND WRITE DETAILED QUESTION TEXT, A NUMBER OF PEOPLE WE WOULD GET SIGNING UP WOULD BE EXTRAORDINARILY SMALL, AS A RESULT IN ORDER TO KEEP THE WORK LOADS MANAGEABLE IN MOST CASES ONLY THREE TO FIVE PEOPLE ARE ASSIGNED TO REVIEW A PARTICULAR GRANT IN DETAIL, THEY DO WRITE WRITTEN CRITIQUES, AND THEN AT THE MEETING THEY WILL ACTUALLY DESCRIBE IT TO THE REST OF THE PANEL, I TOLD YOU THERE MIGHT BE 20 PEOPLE ON THE GROUP SO MOST OF THE PEOPLE WHO ARE VOTING ON HOW IMPORTANT AND DOABLE THEY THINK AN APPLICATION IS HAVEN'T ACTUALLY READ IT. THEY ARE LISTENING TO THEIR COLLEAGUES DESCRIBE WHAT THEY SEE AS THE STRENGTHS AND WEAKNESSES OF THE APPLICATION, AND ALL OF THAT TAKES PLACE TYPICALLY IN SOMEWHERE BETWEEN SEVEN AND 15 MINUTES PER APPLICATION SO IT'S A VERY INTENSE PROCESS BY WHICH THE APPLICATIONS ARE REVIEWED, AFTER SOME DISCUSSION OF THE STRENGTHS AND WEAKNESSES OF THE APPLICATION EVERYONE AROUND THE TABLE WHO IS NOT IN CONFLICT ACTUALLY GETS TO VOTE. AND SO GRANTS ARE SCORED ON A SCALE OF ONE TO FIVE, GOLFERS RATHER THAN BOWLERS CAME UP WITH THE METHOD BECAUSE ONE IS A PERFECT SCORE, FIVE A BAD SCORE YOU, AVERAGED, MULTIPLIED BY 10, THE CLOSER TO 10 WOULD LIKELY BE PAID. SO MORE ABOUT THAT IN A MOMENT. NOW LET'S TALK ABOUT FUNDING PLANS. WE HAD GRANTS SUBMITTED, WE'VE HAD THEM REVIEWED, I SHOULD HAVE SAID IN ADDITION A SCIENTIFIC ADMINISTRATOR IS KEEPING TRACK OF THE DISCUSSION AND WRITES A SUMMARY, POINTING OUT THE IMPORTANT STRENGTHS AND WEAKNESSES MENTIONED DURING THE DISCUSSION, I.E. WHAT DID THE 17 PEOPLE WHO DIDN'T READ THE APPLICATION WHO WERE STILL ALLOWED TO VOTE, WHAT DID THEY ACTUALLY HEAR, WHAT WAS DISCUSSED AT THE MEETING, AND THEN APPLICANTS ALSO DO GET COPIES OF THE CRITIQUES THAT THE THREE PEOPLE WHO WERE ASSIGNED TO THE APPLICATION WROTE. DESCRIBING WHAT IN PARTICULAR THE STRENGTHS AND WEAKNESSES RELATED TO THE REVIEW CRITERIA THAT I MENTIONED. SO WE HAVE SCORES, WE HAVE THE SO-CALLED SUMMARY STATEMENTS, AND NOW INSTITUTE STAFF, PROGRAM STAFF, ARE GOING TO WORK ON COMING UP WITH THE FUNDING PLAN. SO HOW DO WE GO ABOUT THAT? WHAT DO WE DO TO COME UP WITH A FUNDING PLAN? WELL, ONE OF THE MOST IMPORTANT FACTORS IS SCIENTIFIC MERIT, SCIENTIFIC MERIT IS HIGHLY RELATED TO THE PEER REVIEW SCORE. SO YOU CAN START OUT WITH A LIST OF SCORES IN RANK ORDER, I TOLD YOU THERE WERE 250 DIFFERENT STUDY SECTIONS. YOU MIGHT NOT BE SURPRISED TO KNOW THAT NOT EVERY GROUP OF 20 PEOPLE BEHAVES EXACTLY THE SAME. SO SOME GROUPS WILL BE TOUGH GRADERS, SOME GROUPS WILL BE EASY GRADERS. AND SO YOU'VE GOT AN APPLICATION TO GET A SCORE OF 23, WHAT DOES THAT MEAN? IS THAT GOOD? SOUNDS GOOD. CLOSE TO 10. BUT IF YOU FIND OUT THAT PARTICULAR GROUP WAS A BUNCH OF EASY GRADERS, AND ACTUALLY 60% OF THE APPLICATIONS THEY LOOKED AT HAD A SCORE OF BETTER THAN 23, THEN MAYBE THEY JUST HAD AN EXTRAORDINARILY STRONG SET OF APPLICATIONS, OR MAYBE THEY ARE JUST REALLY EASY GRADERS, AND ON THE OTHER HAND YOU MAY HAVE ANOTHER GROUP WHERE 23 WAS THE BEST SCORE THEY GAVE, AND MAYBE THEY DIDN'T HAVE MANY GOOD ONES OR MAYBE THEY ARE HARD GRADERS. IN ORDER TO BE ABLE TO COMPARE SCORES ACROSS STUDY SECTIONS WHAT YOU REPORT IS ALSO A PERCENTILE, WHICH IS HOW MANY APPLICATIONS IN THE STUDY SECTION OVER THE LAST YEAR HAD A SCORE BETTER THAN WHATEVER SCORE YOU GOT. SO IF IN THE FIRST CASE I GAVE, IF A 23 IS 60th PERCENTILE THAT'S NOT SO IMPRESSIVE. IN THE OTHER CASE IS 23 IS FIRST PERCENTILE THAT'S IMPRESSIVE, ONE WAY OF NORMALIZING, SO WHEN WE TALK ABOUT PAY LINE, WHAT WE'RE TALKING ABOUT IS WHAT PERCENTILE ARE WE WILLING TO AUTOMATICALLY PAY, AND MOST INSTITUTES DO HAVE SOME PAY LINE, ANYTHING IN THE TOP X PERCENT WE'RE GOING TO PAY. SO THAT'S ONE THING. THAT'S RELATED TO SCIENTIFIC MERIT AS DETERMINED BY A GROUP OF INDEPENDENT PEER REVIEWERS. NOW, AS WALTER SAID, WE HAVE A VERY GENEROUS BUDGET, BUT WE HAVE AN EVEN LARGER UNIVERSE OF REALLY QUITE GOOD IDEAS FROM TALENTED INVESTIGATORS WE COULD FUND IF WE HAD ENOUGH MONEY, AND THE SCALE THAT WE USE, A ONE TO FIVE SCALE, ACTUALLY NOW ONE TO NINE SCALE, ISN'T THAT PRECISE, AND SO YOU HAVE A LOT OF VIRTUAL TIES, A LOT OF APPLICATIONS THAT HAVE VERY SIMILAR SCORES OR PERCENTILES, AND SO AROUND THOSE MARGINS ANOTHER IMPORTANT THING TO TAKE INTO ACCOUNT ARE PROGRAM CONSIDERATIONS. SO THAT REALLY MEANS THAT THE PROGRAM DIRECTORS WHO KNOW WHAT'S GOING ON IN THEIR FIELDS, ARE LOOKING AND YOU'VE GOT THREE APPLICATIONS, TIED AT THE SAME POSITION, AND ONE OF THEM WOULD BE THE 14th GRANT ON A PARTICULAR TOPIC, THAT'S OF INTEREST AND IMPORTANT, AND THE OTHER WOULD BE ONLY THE FIRST GRANT ON A DIFFERENT TOPIC THAT MAYBE WE THINK IS JUST AS IMPORTANT, AND IF WE'RE GOING TO BREAK THE TIE WE MIGHT BREAK IT IN FAVOR OF THE FIRST GRANT ON THIS NEW TOPIC RATHER THAN THE 14th GRANT ON ANOTHER VERY POPULAR TOPIC. AND SO THERE ARE A NUMBER OF CONSIDERATIONS LIKE THAT. AND THOSE ARE INTERESTING THINGS TO HAVE CONVERSATIONS WITH PROGRAM STAFF ABOUT, IN TERMS OF HOW DO THEY ABOUT ASSESSING THAT. ONE IS ATTENDING MEETINGS LIKE THIS, AND WORKSHOPS WITH MORE SPECIALIZED WORKSHOPS ON PARTICULAR SCIENTIFIC OR CLINICAL TOPICS, TO IDENTIFY WHERE ARE THE NEW OPPORTUNITIES, WHAT ARE THE CHANCES FOR COLLABORATIONS? WHO ARE PROMISING NEW INVESTIGATORS COMING ALONG IN A FIELD? HOW CAN WE STIMULATE NEW COLLABORATIONS BETWEEN SETS OF PEOPLE WHO HAVEN'T WORKED TOGETHER BEFORE? WE HEARD QUITE A BIT ABOUT THAT IN THE EARLIER SESSION ABOUT THE NEED TO GET PEOPLE TOGETHER. OBVIOUSLY, THE OTHER THING THAT DETERMINES WHICH AWARDS GET MADE OR HOW MANY FUNDS ARE AVAILABLE. OKAY. I THINK I'VE TALKED THROUGH SOME OF THIS ALREADY. SO GIVEN THAT -- SO WITH PAY LINE, THIS IS ONE WAY OF NOT HAVING A FIGHT OVER EVERY GRANT THAT WE PAY. AND SO THERE IS SOME ARTICLE OF FAITH THAT IF SOMETHING MAKES IN THE TOP X PERCENT, ODDS ARE HIGH IT'S A GOOD GRANT, NOT TO SAY SOMETHING THAT'S IN THE NEXT 10% ISN'T ALSO VERY GOOD BUT MAYBE IT'S NOT QUITE AS GOOD AS SOMETHING IN THE TOP 10%, SO ZERO TO TEN ON AVERAGE MAY BE BETTER THAN 10 TO 20 OR 20 TO 30, BUT STILL EVEN SOMETHING UP TO A 30 PERCENTILE CAN BE A REALLY VERY GOOD IDEA, VERY LIKELY THAT THE RESEARCH WILL SUCCEED, COMPETENT TEAM, AND SO NOW THINKING ABOUT OTHER THINGS SUCH AS HELPING EARLY INVESTIGATORS GET GOING, HELPING A LAB WHERE THIS IS THE ONLY GRANT THEY'VE GOT GOING VERSUS A LAB WHERE THIS MIGHT BE THE THIRD GRANT THAT THEY'VE GOT, TAKING ADVANTAGE OF SPECIAL OPPORTUNITIES, THESE ARE THINGS THAT FALL IN HIGH PRIORITY AREAS. IN ADDITION, YOU MAY REMEMBER WHEN I TALKED ABOUT FUNDING OPPORTUNITY ANNOUNCEMENTS I MENTION AN RFA. WE DO SAID ASIDE FUNDS FOR SPECIAL SOLICITATIONS, AND SO FREQUENTLY THOSE ARE ACTUALLY REVIEWED SEPARATELY. THERE MAY BE A SPECIAL STUDY SECTION THAT'S CONVENED JUST TO REVIEW APPLICATIONS THAT CAME IN IN RESPONSE TO ONE PARTICULAR ANNOUNCEMENT,BECAUSE WE HAVE IDENTIFIED THAT AREA AS SOMETHING THAT'S SO IMPORTANT WE'RE WILLING TO PULL MONEY OUT OF THE GENERAL POOL AND SET IT ASIDE SPECIFICALLY FOR THIS TOPIC, THEN WE WILL LOOK AT THAT SET OF APPLICATIONS SEPARATELY AND ASSUMING THAT SOME OF THE ONES THAT COME IN ARE OF SUFFICIENTLY HIGH QUALITY, WE'VE ALREADY DETERMINED WE'RE GOING TO PAY SOME APPLICATIONS IN THAT AREA. NOW, OBVIOUSLY SETTING ASIDE MONEY FOR SOME SPECIFIC SOLICITATION IS BYPASSING THE NORMAL PROCESS, AND SO FOR THAT REASON WE WANT TO BE -- YOU DON'T WANT JUST SOMEONE DECIDING LET'S SET ASIDE MONEY FOR THIS WITHOUT GETTING A LOT OF INPUT, SO THIS IS WHERE WE USE COUNSEL, AMONG OTHER THINGS, TO DO CONCEPT CLEARANCE, SAYING WE WANT TO -- WE'RE THINKING ABOUT SETTING ASIDE MONEY IN THIS GENERAL AREA, ARE YOU, COUNSEL, PERSUADED THIS WOULD BE A GOOD IDEA, IF THEY GIVE US CONCEPT CLEARANCE WE CAN GO AHEAD AND WORK ON THAT. OKAY. LET ME TALK ABOUT ONE OTHER THING IN TERMS OF PAYLINE, IT'S RELATED TO THE APPROPRIATION, ONE OTHER THING I HAVEN'T MENTIONED IS THAT ON AVERAGE, THE GRANTS WE MAKE ARE FOUR YEARS LONG. WE HAVE TO SPEND ALL THE MONEY BY THE END OF THE FISCAL YEAR SO COME OCTOBER 2, ASSUMING THE GOVERNMENT IS OPEN AND WE HAVE A BUDGET, WE'LL BE FIGURING OUT HOW TO SPEND IT, AND, LET'S SEE, WE'VE GOT THREE YEARS OF OUT YEAR COMMITMENTS, IN SOME WAYS 3/4 OF THE MONEY IS COMMITTED BEFORE THE YEAR IS STARTED, IF WE HAPPEN TO GET A CUT IT GETS EVEN TOUGHER, AND SO WE HAVE TO BE CAREFUL WHEN WE'RE MAKING FUNDING DECISIONS ABOUT THE BALANCE OF LONG-TERM APPLICATION, AND IMAGINE WE MAKE A NUMBER OF SEVEN-YEAR AWARDS, MAYBE WE ALSO MADE SOME TWO-YEAR AWARDS. WE'RE TRYING TO NOT COMPLETELY TIE OUR HANDS IN THE OUT YEARS SO WE HAVE MONEY TO TAKE ADVANTAGE OF NEW OPPORTUNITIES EVERY YEAR. OTHER THINGS TO PRESERVE THE PAYLINE BEATING SIX OUT OF SEVEN ISN'T GREAT BUT BETTER THAN THE WORST YEAR TEST NINTH PERCENTILE, NOT AS GOOD OF THE DOUBLING YEARS AT THE 25th PERCENTILE, SO PAYLINES HAVE VARIED, AND WE HAVE WORKED VERY HARD TO NOT LET IT DROP BELOW THE 14th BECAUSE WE FEEL LIKE THAT IS REALLY A FLOOR AT WHICH, YOU KNOW, PEOPLE, IF THEY CONTINUE TO ENDEAVOR, MAY BE ABLE TO MAKE THAT, IF IT GETS MUCH WORSE, EVEN MORE PEOPLE ARE GOING TO GET DISCOURAGED AND DROP OUT. IF YOU'RE MAKING DECISIONS TO MAINTAIN THE PAYLINE THAT MEANS YOU'RE DOING FEWER SPECIAL SET-ASIDES, IN RECENT YEARS WE'VE HAD TO MAKE ADMINISTRATIVE CUTS SO SOMEONE ASKED FOR $100,000 A YEAR, WE MAY ONLY GIVE THEM $87,500 A YEAR IF WE'RE TAKING A 12 1/2% CUT. THAT GENERATES SOME ADDITIONAL MONEY AS WELL. OKAY. SO NOW LET'S TURN TO COUNSEL. SO HERE IS AN ACTUAL PICTURE OF COUNCIL, WALTER AT THE PODIUM. MANY COUNCIL MEMBERS PAYING RAPT ATTENTION TO LAPTOPS, SO IT MUST HAVE BEEN A BREAK. A STUDY SECTION LOOKS AT A SET OF GRANTS IN ONE SCIENTIFIC AREA OR IN RESPONSE TO ONE SPECIFIC SOLICITATION, FOR NINDS IN ANY GIVEN ROUND THERE ARE PROBABLY AT LEAST 50 DIFFERENT STUDY SECTIONS THAT LOOKED AT GRANTS THAT WERE SELECTING FROM AMONG TO PAY, SO WE WOULD LIKE SOME GROUP THAT'S ACTUALLY LOOKING AT THE WHOLE UNIVERSE OF GRANTS THAT WE'RE PROPOSING TO PAY AND THAT'S THE ROLE OF COUNCIL. SO OBVIOUSLY THEY TAKE A MUCH HIGHER LEVEL ROLE, THEY ARE LOOKING AT BROAD AREAS THAT WE'RE PROPOSING TO FUND, SO THAT'S PART OF WHAT THEY DO. IF WE'RE GOING TO SET ASIDE PART OF OUR MONEY FOR SOLICITATION, WE WOULD LIKE FEEDBACK FROM THEM THAT THIS SOUNDS LIKE SUFFICIENTLY IMPORTANT TOPIC THAT IT'S WORTH SETTING ASIDE SOME MONEY. THEN FINALLY COUNCIL PROVIDES VERY IMPORTANT ROLE IN TERMS OF ADVISING US ABOUT THE OVERALL GOALS AND PRIORITIES OF WHAT WE'RE TRYING TO DO, AND EACH ROUND IN OPEN SESSION WE'LL HAVE A NUMBER OF IDEAS ABOUT WAYS THAT WE COULD DO THINGS DIFFERENTLY OR BETTER OR MORE EFFICIENTLY. OKAY. SO I'M NOT GOING TO READ THESE. THESE ARE IN THE PRINTED SLIDES THAT YOU HAVE. IT'S JUST FACTOIDS WILL WHO IS ON COUNCIL. OKAY. AND SO THEN IN TERMS OF WHAT COUNCIL DOES, SO WE PROVIDE THEM THESE DETAILED FUNDING PLANS OF WHAT WE'RE THINKING ABOUT DOING. THEY CAN CONCUR WITH THEM AND MAKE SOME SUGGESTED MODIFICATIONS. SO IN PARTICULAR, IN AREAS WHETHER WE HAVE A SET OF GRANTS THAT ARE AROUND THE AREA OF THE FUNDING LINE, WE MAY HAVE -- WE MAY GIVE THEM A LARGER SET OF POSSIBLE ONES TO FUND THAN WE'RE GOING TO BE ABLE TO DO, WE ASK FOR THEIR FEEDBACK ON ONES THEY THINK ARE PARTICULARLY GOOD OPPORTUNITIES ONES THEY MEANING MIGHT BE REDUNDANT. IF THERE'S SOMETHING THEY THINK WOULD BE A PARTICULARLY EGREGIOUS WASTE OF TAXPAYER MONEY, THEY CAN DISAPPROVE THINGS AND SAY THOU SHALT NOT FUND, AND IF THEY SAY THAT WE CANNOT FUND IT. OKAY. LET ME JUST CLOSE OUT THIS TALK AND GET INTO THE DISCUSSION BY JUST TALKING ABOUT THINGS THAT INDIVIDUAL ORGANIZATIONS CAN DO TO HELP IMPACT OUR DECISION-MAKING PROCESS. AND SO I'M GOING TO POINT OUT A FEW THINGS HERE. I'M GOING TO TALK ABOUT WORKING WITH P.I.s, REVIEWERS AND COUNCIL. SO ONE ISSUE IS WHAT GRANTS DO WE SOLICIT? SO IN ADDITION TO DOING WORKSHOPS, ANOTHER THING WE CAN DO IS ELICIT REQUESTS FOR INFORMATION, AND I HAVE PUT UP ONE HERE THAT WE PUBLISHED RECENTLY, AND YOU ACTUALLY HAVE UNTIL SEPTEMBER 18th, AND THIS HAS TO DO WITH DEVELOPMENT OF A FEDERAL PAIN RESEARCH STRATEGY, SO THERE IS AN OFFICE AT NIH HEADED BY LINDA PORTER OF NINDS, THAT IS LOOKING AT COORDINATING PAIN RESEARCH ACROSS NIH. THERE IS NO INSTITUTE WHERE PAIN IS IRRELEVANT OR NOT PART OF THEIR MISSION, BUT THERE HAD NOT BEEN AN INSTITUTE WHERE THE STUDY OF PAIN WAS THEIR ONLY MISSION OR PRIMARY FOCUS OF THEIR MISSION AND SO IN ORDER TO MAKE SURE THAT IMPORTANT AREAS OF PAIN RESEARCH DON'T FALL THROUGH THE CRACKS, THERE IS THIS PAIN RESEARCH COORDINATING COMMITTEE, AND THERE IS NOW DEVELOPING A FEDERAL PAIN RESEARCH STRATEGY. AND SO THIS COULD LEAD TO SPECIFIC SOLICITATION, SET-ASIDES OF MONEY FOR PARTICULAR THINGS, ONE OF THE THINGS THAT MIGHT HAPPEN WOULD BE IF THERE WERE FEEDBACK FROM THE PUBLIC ABOUT PARTICULARLY IMPORTANT OPPORTUNITIES THAT THE PUBLIC DOESN'T THINK THAT NIH IS SUPPORTING AS EFFECTIVELY AS IT COULD BE NOW, THAT'S ONE EXAMPLE OF A WAY THAT PEOPLE CAN PROVIDE INFORMATION. AND WHEN WE DO RFIs WE CAN TELL WHEN WE'VE HIT A NERVE WHEN WE GET 5000 RESPONSES, WE KNOW IT'S SOMETHING PEOPLE CARE ABOUT A LOT. SO AN EXAMPLE WAS A FEW YEARS AGO THERE WAS A PROPOSAL UNDER SERIOUS CONSIDERATION TO COMBINE THE NATIONAL INSTITUTE OF DRUG ABUSE AND NATIONAL INSTITUTE OF ALCOHOL, ALCOHOLISM AND ALCOHOL ABUSE. IT TURNED OUT PEOPLE CARED A LOT ABOUT THAT AND THERE WERE MANY, MANY, MANY COMMENTS, PRO AND CON. THERE HAVE BEEN OTHER ONES WHERE THERE WERE THINGS WE WEE THOUGHT WERE OF INTEREST, I WON'T SAY WHAT THEY ARE BUT THE PUBLIC NOT SO MUCH AND MAYBE WE GOT A DOZEN RESPONSES, SOME INDICATION OF INTEREST TOO. SO IT IS VERY HELPFUL TO US WHEN PEOPLE PUT IN THEIR COMMENTS. OFTEN ORGANIZATIONS DO BRAINSTORMING AND SEND CUMULATIVE THOUGHTS FROM EXECUTIVE BOARD, FOR EXAMPLE, HELPFUL AS WE THINK ABOUT HOW TO GO ABOUT PLANNING HOW TO USE THE MONEY. OBVIOUSLY PEOPLE CAN ALSO -- ORGANIZATIONS CAN WORK WITH INVESTIGATORS. IF YOU HAVE A SET OF INVESTIGATORS, WHO ARE WORKING IN AN AREA THAT'S OF PARTICULAR INTEREST TO YOUR ORGANIZATION, YOU MAY BE ABLE TO WORK WITH THEN AND HELP THEM AS THEY ARE DEVELOPING THEIR PROPOSAL. IT'S NOT UNCOMMON FOR PATIENT-CENTERED ORGANIZATIONS TO COLLABORATE WITH INVESTIGATORS TO HELP THEM WITH SUCH THINGS AS SUBJECT RECRUITMENT AND REALLY GETTING THE WORD OUT THERE'S A TRIAL GOING ON. THAT CAN BE POWERFUL. IN THE CASES OF CONVENING A WORKSHOP THAT'S NOT UNCOMMON FOR PATIENT ORGANIZATIONS TO ACTUALLY SUBMITTING A FOR A SMALL GRANT, R-13 TO BRING TOGETHER PERHAPS BASIC SCIENTISTS AND CLINICIANS FROM TWO FIELDS THAT NEVER SEEMED TO GET TO MEET AT THEIR OWN PROFESSIONAL ORGANIZATION MEETINGS, AND BRING THEM TOGETHER TO ADDRESS MERGING OPPORTUNITIES FOR RESEARCH IN SOME FIELD WHERE IT HASN'T HAPPENED BEFORE. I'VE ALREADY MENTIONED BEFORE THAT IT IS POSSIBLE TO RECOMMEND STUDY SECTION OR COUNCIL MEMBERS. I MAYBE DIDN'T MENTION THAT PEOPLE ON COUNCIL AND STUDY SESSION SERVE STAGGERED FOUR-YEAR TERMS, EVERY YEAR 25% OF COUNCIL AND STUDY SECTIONS GRADUATE, JUST AS YOU REALLY GET 'EM TRAINED AND THEY ARE GOOD IN OFFERING INCREDIBLY WISE ADVICE THEY BOLT FOR THE DOORS AND NOW THAT THEY KNOW SO MUCH AT NIH THEY ARE EAGER TO GET BACK WORKING FULL TIME ON RESEARCH OR WHATEVER CAUSE. WE'RE ALWAYS IN NEED OF GOOD PEOPLE, PLEASE SUBMIT NAMES TO ERNIE LANZ, THE CHIEF OF NINDS, BOB FINKELSTEIN, I'M HAPPY TO TAKE YOUR SUGGESTION, AS IS WALTER. FOR MORE SPECIALIZED SUGGESTION THE CENTER FOR SCIENTIFIC REVIEW HAVE A WEBSITE WHERE PEOPLE CAN VOLUNTEER OR YOU CAN VOLUNTEER SOMEONE. THERE'S 5000 OR MORE REVIEWERS A YEAR. I'VE TALKED ABOUT HOW THE WAY THEY RATE APPLICATIONS AND SCORE THEM HAS A BIG IMPACT ON WHAT WE PAY, AND SO IT'S IMPORTANT THAT THERE BE GOOD PEOPLE AND IT'S IMPORTANT THERE BE NEW PEOPLE. WHAT WE DON'T WANT IS THE SAME OLD SET OF PEOPLE CONSTANTLY EVALUATING APPLICATIONS. OKAY. THAT WAS A VERY WHIRLWIND TOUR OF SOME OF THE WAYS THAT WE GO ABOUT DECIDING WHAT WE'RE GOING TO DO, AND I DIDN'T WANT TO SPEND TOO MUCH TIME TALKING ABOUT THIS SO AT THIS POINT I WOULD LIKE TO PAUSE AND TAKE QUESTIONS, COMMENTS, WHATEVER ELSE YOU WOULD LIKE TO TALK ABOUT. [APPLAUSE] >> HOW DO YOUR QUESTIONS FOR INFORMATION GET DISSEMINATED AND HOW CAN WE LEARN ABOUT THEM. >> REQUESTS FOR INFORMATION ARE ALSO PUBLISHED IN THE NIH GUIDE. YET ANOTHER REASON YOU SHOULD JOIN THAT LIST SERVE. YOU CAN ALSO -- WE TWEET THEM OUT. I'M LOOKING FOR SHANNON GARNET, WHO WAS HERE EARLIER, EVERY MONDAY SHE TWEETS OUT FUNDING OPPORTUNITIES AND OTHER THINGS THAT WE THINK ARE OF INTEREST, AND SO YOU CAN -- AND ONE OF THE SLIDES WE HAD A LIST OF TWITTER FEEDS THAT YOU CAN FOLLOW, SO THERE ARE LIST SERVES, TWITTER FEEDS, THOSE ARE THE TWO MAIN ONES. [OFF MIC] >> SO ONE OF THE BEST PIECES OF ADVICE I HAVE HEARD SOME SEASONED INVESTIGATORS GIVE BEGINNING PEOPLE IS IS IT WORTHWHILE GETTING A PROFESSIONAL WRITER TO HELP, ESPECIALLY THE INITIAL DRAFT, PARTICULARLY GIVEN THAT A LOT OF THE PEOPLE WHO ARE DOING RESEARCH HERE WEREN'T NECESSARILY BORN IN THIS COUNTRY, ENGLISH MAY NOT BE THEIR FIRST LANGUAGE, BUT EVEN FOR PEOPLE WHERE IT IS, NOT EVERY SCIENTIST IS -- WAS ALSO MEANT TO BE, YOU KNOW, A 3000 WORD A DAY EDITORIAL WRITER, AND SO, YES, THAT'S SOMETHING THAT CAN BE OF TREMENDOUS HELP. I KNOW A NUMBER OF FOLKS WHO DID HAVE PROFESSIONAL EDITORS WHO HELP THEM, AND SOME PEOPLE FIND IT MORE HELPFUL TO DICTATE SOMETHING AND THEN HAVE SOMEONE CLEAN IT UP. SO THAT'S A MATTER OF TASTE AND CONTROL, BUT I WOULD SAY REGARDLESS OF WHETHER ONE WRITES IT, ONESELF OR HAS AN EDITOR HELP, ONE OF THE MOST IMPORTANT THING BEFORE SUBMISSION IS HAVE SMART PEOPLE NOT IN THEIR IMMEDIATE FIELD LOOK AT THE APPLICATION, EVEN LOOKING AT THE ABSTRACT, JUST SAY LOOKING AT THIS, CAN YOU UNDERSTAND WHAT IT IS I WANT TO DO AND WHY IS IT IMPORTANT? SO I ALWAYS TELL PEOPLE, THIS IS WHERE THEIR GRUMPIEST COLLEAGUE CAN BE THEIR BEST FRIEND. >> I THROW OUT THAT MANY OF THE ORGANIZATIONS HAVE PEOPLE WHO ARE REALLY GOOD DOCTORS AND THEY REALLY WANT TO, YOU KNOW, MOVE THE SCIENCE FORWARD BUT HAVE NEVER WRITTEN A GRANT, AND THEY HAVE NEVER GOTTEN A GRANT, AND SO I WOULD SAY, YOU KNOW, AS AL WAS SAYING ABOUT A PROFESSIONAL WRITER, PROBABLY AN IMPORTANT THING IS TO GET SOMEBODY IN THE GROUP WHO HAS ACTUALLY WRITTEN AND BEEN AWARDED AN NIH GRANT, BECAUSE THERE'S TWO LEVELS OF GOING THROUGH THE GRANT PROCESS. ONE IS THE SCIENTIFIC RATIONALE THE BASIS, THE OTHER IS, YOU KNOW, GRANT WRITING, WHICH IS AN ART FORM IN AND OF ITSELF, AND THE PEOPLE WHO FIRST GET INTO THIS ARE USUALLY -- THEY USUALLY LOSE POINTS BECAUSE THEY ARE INEXPERIENCED AT GRANT WRITING ITSELF. SO THERE'S NO REAL SUBSTITUTE FOR SOMEONE WHO IS ACTUALLY SUCCESSFUL IN THE SYSTEM TO HELP YOUR GROUP GET THEIR FIRST -- >> IDEALLY SOMEONE WHO HAS BEEN SUCCESSFUL RECENTLY, BECAUSE IT IS MUCH MORE COMPETITIVE NOW THAN IT WAS, SAY, 15 YEARS AGO, AND I MENTIONED BEFORE WE KEEP CHANGING THINGS. WE CHANGED REVIEW CRITERIA, WE CHANGE THE ACTUAL FORMS, WE'VE SHORTENED THE APPLICATIONS DRAMATICALLY, THERE'S MORE EMPHASIS NOW ON GETTING A MESSAGE SUCCINCTLY AND CLEARLY, THERE ARE PEOPLE WHO ARE EXTRAORDINARILY GOOD AT THAT WITHIN TWO PARAGRAPHS OF READING WHAT THEY WRITE YOU'RE READY TO GIVE THEME THEIR MONEY, AND THE CON IS ALSO TRUE. SO IT IS WORTHWHILE HELPING PEOPLE PAIR UP SO PEOPLE ARE ARE GOOD AT COMMUNICATING IS PAIRED WITH SOMEONE WHO HAS GOOD IDEAS. SOMETIMES THOSE ARE THE SAME PEOPLE BUT SOMETIMES THEY ARE NOT. [OFF MIC] >> THERE ARE ALSO A NUMBER OF WORKSHOPS THAT WE AND OTHER ORGANIZATIONS PUT ON AT PROFESSIONAL MEETINGS, TALKING IN CONSIDERABLE DETAIL ABOUT THE ART FORM OF WRITING GRANTS, AND THERE ARE ACTUALLY SOME COMMERCIAL ORGANIZATIONS, SOME OF WHOM ARE VERY GOOD. THEY ARE VERY TUNED IN TO THE LATEST RULES AND FORMS AND WE HAVE -- I'VE SEEN SOME WORKSHOPS WHERE PEOPLE GIVE EXTRAORDINARILY GOOD ADVICE, AND SOME OF THEM ARE ACTUALLY AVAILABLE AS CONTRACTORS WITH INDIVIDUAL ORGANIZATIONS TO HELP PEOPLE WRITE GRANTS, SO THERE IS A PRETTY GOOD SET OF PEOPLE WHO CAN GIVE VERY EFFECTIVE ADVICE OUT THERE SO THIS ISN'T SOMETHING THAT SOMEONE NEEDS TO DO ALONE. >> I UNDERSTAND THE DIFFICULTY IN TRYING TO ENGAGE REVIEWERS BUT IS THERE A METHOD TO ALIGN THE DISCIPLINE A REVIEWER HAS WITH THE APPLICATION OF THE DISEASE OR THE LOI THEY ARE REVIEWING. >>S THAT ONE THING THE SCIENTIFIC OFFICER DOES IS BECAUSE IN ADDITION TO -- I DESCRIBED IT AS THOUGH YOU HAVE THE COMMITTEES THAT ARE FIXED MEMBERSHIP THAT MEET THREE TIMES A YEAR, VIRTUALLY EVERY COMMITTEE WILL NOT HAVE ALL THE EXPERTISE THEY NEED FOR A PARTICULAR SET OF APPLICATIONS IN A PARTICULAR ROUND, AND SO THEY WILL INVITE IN ADDITIONAL REVIEWERS, SO-CALLED AD HOC REVIEWERS WHO BRING THE SUBJECT MATTER EXPERTISE TO COMPLEMENT THE PEOPLE WHO ARE ON THE PANEL. THE OTHER THING THAT HAPPENS IS YOU SUBMIT A GRANT, MAYBE THERE ARE TWO PEOPLE WHO ARE REAL EXPERTS IN YOUR AREA BUT LO AND BEHOLD THOROUGH CLOSE COLLABORATORS. YOU WOULD LOVE TO HAVE THEM REVIEW THE GRANT BUT IF THEY ARE COLLABORATE IT'S A CONFLICT AND THEY WILL BE OUT OF THE ROOM. WE HAVE TO FIND PEOPLE TO DO THE REVIEW FOR APPLICATIONS WHERE A CONFLICT WITH A MEMBER OF THE STANDING COMMITTEE IS, THERE'S A JUGGLING ACT BETWEEN GETTING A BALANCE, APPROPRIATE EXPERTISE, I WOULD SAY ONE COMMON ERROR THAT BEGINNING INVESTIGATORS TEND TO MAKE IS THAT THEY THINK THEY ONLY WANT THEIR APPLICATIONS REVIEWED BY ABSOLUTELY THE MOST EXPERT PEOPLE IN THE FIELD. OFTEN, IF THE APPLICATION IS WELL WRITTEN, IT WILL BE QUITE APPEALING TO INTELLIGENT PEOPLE WHO AREN'T QUITE IN THEIR FIELD, AND THOSE PEOPLE CAN BE EFFECTIVE ADVOCATES AND THE PERSON IN THE FIELD MAY BE TOO CLOSE AND FOCUSING ON RATHER MINOR POINTS WITHIN THE FIELD, SOMETHING THAT IS AN ISSUE THAT'S OF CONCERN OF THE SIX PEOPLE WHO WORK ON THIS TOPIC, BUT ISN'T PERHAPS THE MOST IMPORTANT ISSUE IN THE BIG PICTURE CONSIDERATION OF THAT RESEARCH PROJECT. SO SOMETIMES REALLY CLOSE EXPERTISE CAN BE OVERRATED, AND WHAT YOU WANT IS A SMART PERSON WHO CAN FOLLOW A WELL-ARGUED APPLICATION. >> WHAT ALAN IS TRYING TO SAY, HE'S BEING POLITIC ABOUT IT, IS THAT THE PERSON WHO SUBMITS THE GRANT GETS THE CRITICISM BACK BUT THEY DON'T ACTUALLY KNOW WHO WROTE WHICH CRITICISM, AND THE GENERAL RULE IS THEY SAY IS THERE WASN'T EXPERTISE IN MY AREA ON THE PANEL BUT THEY DON'T KNOW IT'S THE PERSON WITH THE MOST EXPERTISE THAT KILLED THE GRANT. I THINK ALAN'S POINT IS RIGHT, THAT YOU WANT TO BE ABLE TO WRITE IT SO THAT ALL THESE PEOPLE ARE REALLY SMART PEOPLE THAT IT'S REALLY CLEAR AND THAN THERE'S A KIND OF CONSENSUS THAT GETS AROUND THE GRANT AS BEING A REALLY GOOD THING TO FUND. >> YOU REMEMBER THE MAJORITY OF PEOPLE AROUND THE TABLE WILL BE IN THE GENERAL AREA, BUT WILL NOT BE IN THE SPECIFIC AREA. THESE ARE ALL PEOPLE, YOU KNOW, MOST OF THEM ARE ALSO WRITING GRANTS OF THEIR OWN SO THEY KNOW WHAT IT'S LIKE AND THEY WANT TO BE ABLE TO DO A GOOD JOB. THEY ARE TRYING TO UNDERSTAND THE APPLICATION AND GIVE IT AS FAIR A SCORE AS THEY CAN. [OFF MIC] >> IN THE GENERAL AREA REVIEWING THE ACTUAL GRANT THEY MAY NOT BE IN THE DECISION-MAKING PROCESS BUT THEY ARE STILL GIVING COMMENTS? >> YES, THEY ARE STILL GIVING COMMENTS. MAYBE I MISUNDERSTOOD YOUR QUESTION THEN. >> WELL, IT SEEMS TO ME IF THEY ARE NOT VOTING -- (INAUDIBLE) >> OH, NO, IF THEY ARE NOT ELIGIBLE TO VOTE, THEY WOULDN'T BE EVEN SHOWN THE GRANT. ALL RIGHT. RIGHT. IF YOU'RE IN CONFLICT YOU DON'T EVEN SEE THE APPLICATION. >> YOU HAVE TO LEAVE THE ROOM WHEN IT'S DISCUSSED. >> THAT'S RIGHT. YOU GET TO GO HUNT FOR THE COFFEE THAT WE'RE NOT ALLOWED TO PROVIDE. >> THANKS. MY NAME IS ROBERT McBURNY, PRINCIPAL INVESTIGATOR OF THE MULTIPLE SCLEROSIS PATIENT POWERED RESEARCH NETWORK THAT JOE REFERRED TO. A TOPIC HAS COME UP WITHIN OUR NETWORK, AND IT'S NOT SO MUCH ABOUT THE NIH GRANT FUNDING MECHANISM BUT IT'S ABOUT CONFIDENTIALITY, AND SO I WONDER IF YOU COULD SAY WHAT ARE THE CONFIDENTIALITY PARAMETERS AROUND THE GRANT PROCESS, AND THEN I'LL POSE OUR PROBLEM. >> OKAY. SO EACH REVIEWER WHO PARTICIPATES DOES SIGN A DOCUMENT UNDER FEDERAL PERJURY PENALTY THEY RECOGNIZE THESE ARE CONFIDENCE MATERIALS AND WILL NOT SHARE WITH ANYONE ELSE. THEY WILL NOT ASK A POST DOCTORATE GRADUATE STUDENT TO GIVE COMMENT, THEY WILL DO IT THEMSELVES AND WILL NOT TALK ABOUT THIS WITH ANY OF THEIR COLLEAGUES. AND THEY WILL NOT TALK ABOUT WHAT HAPPENED IN THE STUDY SECTION MEETING. SO WHAT HAPPENS IN STUDY SECTION STAYS IN STUDY SECTION, IS THE LAW. NOW, THESE ARE HUMAN BEINGS, AND I'M AWARE THAT PEOPLE DO TALK AND GOSSIP. >> NO, SO THE POINT I WANTED TO MAKE, AND I UNDERSTAND THAT SITUATION, I'VE BEEN A REVIEWER, BUT SOMETHING HAS COME UP WITHIN OUR PATIENT POWERED RESEARCH NETWORK, SO ONE OF THE IDEAS ABOUT CREATING A PATIENT POWERED RESEARCH NETWORK IS THAT -- ARE YOU NOT HERE? OKAY. IS THAT WE WANT THERE TO BE A GROUND SWELL OF RESEARCH TOPICS THAT COME FROM THE INTERESTS OF THE PATIENTS. AND IT'S VERY INTERESTING BECAUSE THE INTERESTS OF PROFESSIONAL RESEARCHERS IS OFTEN ON SOMETHING THAT'S COOL THAT COULD BE PUBLISHED IN NEW ENGLAND JOURNAL OF MEDICINE, NATURE, SCIENCE, ET CETERA, AND MAY NOT BE EXACTLY THE INTEREST OF A PERSON LIVING WITH THE DISEASE. IN FACT, WHILE THEY ARE VERY INTERESTED IN SOME DEGREE OF NEW DRUG TARGETS OR DEVELOPMENT OF SOME NEW THERAPEUTIC, MANY OF THEM WANT TO KNOW ABOUT REAL RESEARCH ON THINGS THAT THEY MAY BE ABLE TO DO TODAY THAT IMPACT THE COURSE OF THEIR DISEASE. SO IT'S VERY IMPORTANT THAT THOSE SORTS OF -- FOR EXAMPLE, DIET, EXERCISE, ALL SORTS OF OTHER ALTERNATIVE MEDICINES, ET CETERA. IT'S VERY IMPORTANT THAT THEIR IDEAS GET EQUAL WEIGHTING. SO WE SOLICIT IDEAS FROM OUR NETWORK AND WE ALSO SOLICIT IDEAS FOR RESEARCH FROM THE RESEARCH COMMUNITY. THINGS THAT COME IN FROM THE RESEARCH COMMUNITY ARE PRETTY WELL FORMED RESEARCH STUDIES. THINGS THAT COME FROM THE NETWORK ARE USUALLY IDEAS THAT NEED TO BE WORKED UP TO BE RESEARCH STUDIES. SO IT'S VERY IMPORTANT FOR ENGAGING THE PATIENT COMMUNITY, I'M SORRY, THIS IS A LONG-WINDED QUESTION, IT'S VERY IMPORTANT FOR ENGAGING THE PATIENT COMMUNITY THAT THERE'S VISIBILITY OF WHAT'S HAPPENING TO THEIR IDEAS, AND THAT WE PUT IN PLACE, IF YOU LIKE, A PROPOSAL DEVELOPMENT ENGINE THAT MAYBE PEOPLE WHO ARE DISINTERESTED IN THAT PARTICULAR RESEARCH ARE PREPARED TO CONTRIBUTE TO MAKE A GOOD PROPOSAL SO THAT GOOD PROPOSALS ARE JUDGED EQUALLY, NOT IDEAS AGAINST REALLY WELL-RESEARCHED PLANS. ONE OF THE WAYS OF ENGAGING PEOPLE IS TO MAKE THAT PROCESS TRANSPARENT. SO WE'VE SAID TO THE RESEARCHERS WHO ARE SUBMITTING PROPOSALS, WELL, WE'RE GOING TO MAKE YOUR PROPOSALS TRANSPARENT AS WELL. HORROR! >> YOU MEAN PRIOR TO REVIEWER AWARD? >> PRIOR TO THEM BEING ACCEPTED AS A RESEARCH STUDY AND THERE'S ANOTHER ASPECT OF THIS, IF YOUR RESEARCH IS GOING TO INVOLVE, YOU KNOW, SURVEY INSTRUMENTS FOR PATIENTS, ET CETERA, LIKE THAT, WE WANT OUR PATIENT COMMUNITY TO SEE THOSE INSTRUMENTS AND COMMENT BACK TO THE RESEARCHERS, SO WE DON'T HAVE THE PATIENT COMMUNITY ALL SIGNING CONFIDENTIALITY AGREEMENTS. >> RIGHT, BECAUSE WE ALSO, WHEN PEOPLE SUBMIT GRANT APPLICATIONS TO US, IT'S WITH THE UNDERSTANDING THAT WE KEEP THEM CONFIDENTIAL UNTIL SUCH TIME AS THEY ARE AWARDED, AT WHICH POINT THEN IT WILL SHOW UP IN NIH REPORTER. >> ON THE ONE HAND, WE'RE ENGAGING THE PATIENT COMMUNITY, WE WANT TO LET THEM KNOW EVERY RESEARCH PROJECT THAT IS COMING DOWN THE PIKE, IF YOU LIKE, WHETHER IT'S AN IDEA OR A FULLY BLOWN RESEARCH PROPOSAL. ON THE OTHER HAND, WE HAVE THIS DILEMMA THAT MOST PROFESSIONAL RESEARCHERS DON'T WANT ANY OF THEIR IDEAS EXPOSED, PUBLICLY. >> RIGHT. SO ONE THINK THING WE CAN USE IN THAT ARENA IS WE KNOW THAT THERE ARE ORGANIZATIONS THAT MIGHT BE INTERESTED IN PAYING SOME OF THE APPLICATIONS WE WERE UNABLE TO PAY, AND WE CAN'T JUST SEND TO TO THE ORGANIZATION. HOWEVER, WE CAN TELL THE APPLICANTS, YOU MIGHT WANT TO TALK TO THIS ORGANIZATION AND LET THEM KNOW THAT YOU HAD AN APPLICATION THAT DID PRETTY WELL BUT MAYBE DIDN'T QUITE MAKE IT AND IF THE APPLICANT CHOOSES TO SHARE THEIR SUMMARY STATEMENT OR APPLICATION WITH ANOTHER FUNDING ORGANIZATION, THAT'S FINE. WE CAN'T DO THAT BUT THEY CAN. WE CAN CERTAINLY MAKE AVAILABLE INFORMATION ABOUT ORGANIZATIONS THAT ARE INTERESTED IN FUNDING AREAS OF RESEARCH SO THAT'S ANOTHER PLACE WHERE WE CAN PARTNER IS PARTICULARLY, SAY, FOR BEGINNING INVESTIGATORS WITH SMALLISH GRANTS, IF WE KNOW THAT SOME OF YOUR ORGANIZATIONS MIGHT BE INTERESTED IN HELPING THEM, IF YOU AREN'T ALREADY IN CONTACT WITH THOSE FOLKS, THAT'S SOMETHING ELSE THAT OUR PROGRAM STAFF IN PARTICULAR OUGHT TO KNOW ABOUT WHAT YOUR ORGANIZATION IS DOING AND CAN HELP PEOPLE GET IN TOUCH WITH -- HELP OUR APPLICANTS GET IN TOUCH WITH ORGANIZATIONS THAT MAY BE ABLE TO WORK WITH THEM. >> MY POINT IS THAT THE RESEARCH CONFIDENTIALITY PROCESS IS SOMEWHAT AT LOGGERHEADS WITH AN OPEN TRANSPARENT PROCESS THAT WE WANT TO OFFER THE PEOPLE IN THE NETWORK WHO ARE PROPOSING RESEARCH STUDIES. SO I DO THINK THAT THERE'S A DAY WHERE JUST LIKE IN CLINICAL TRIALS.GOV THERE WILL BE MORE OPENNESS ABOUT WHAT STUDIES ARE ABOUT, WHILE THEY ARE ONGOING ON THE BASIC RESEARCH SIDE. >> ON THE BASIC RESEARCH SIDE IF WE'RE SUPPORTING IT, IT IS OUT THERE, IT'S IN NIH REPORTER. EVERYTHING WE SUPPORT IS OUT AND YOU CAN ACTUALLY SEARCH IT AND THERE ARE GOOD SEARCH TOOLS FOR THAT. THAT'S ACTUALLY SOMETHING I COULD HAVE MENTIONED, IF YOU'RE NOT AWARE OF THAT, IT'S IN THE LAST THREE OR FOUR YEARS, NIH HAS MADE A REAL EFFORT TO MAKE THE DATABASE OF ALL THE THINGS THAT WE SUPPORT AVAILABLE GOOGLE NIH REPORTER. IT WILL TELL YOU WHAT INSTITUTE SUPPORTED IT, WHAT STUDY SECTION REVIEWED IT, WHO THE PROGRAM OFFICER IS, AND SO YOU CAN GET A LOT OF GOOD INFORMATION TO START FINDING OUT HOW TO ALIGN YOUR RESEARCH IDEA WITH A PARTICULAR INSTITUTE OR SET OF INSTITUTES AT NIH. SO ONCE IT'S FUNDED, THERE CONSIDERABLY TRANSPARENCY. IF WE'RE SPENDING TAXPAYER MONEY ON IT, WITHIN HHS, IT'S NOT A SECRET. IT'S OUT THERE. >> WE DON'T PROHIBIT ANYBODY FROM SHARING APPLICATIONS WITH OTHER PEOPLE BEFORE THEY COME IN. IF THEY BLAME US -- >> RIGHT, WE'RE THE ONES WHO PLEDGED TO KEEP IT SECRET. WHAT THEY DO WITH IT IS UP TO THEM. >> ALONG THE LINES YOU EMPHASIZED IN TERMS OF IMPORTANCE OF GETTING THE PATIENT GROUPS INVOLVED, YOU MENTIONED HOW IMPORTANT IT IS TO GET THEM INVOLVED WITH THE PRINCIPAL INVESTIGATORS AND SO FORTH. TWO RELATED QUESTIONS. ONE, THERE WAS A TIME WE HAD COOPERATIVE AGREEMENTS, U-GRANT MECHANISMS THAT REQUIRED THE PARTICIPATION OF A PATIENT GROUP TO BE A CO-APPLICANT, ALONG WITH AN INDUSTRY PARTNER, FOR EXAMPLE. I'M WONDERING IF THERE ARE ANY VESTIGES OF THAT APPROACH IN OUR CURRENT FUNDING MECHANISMS. >> THE FUNDING MECHANISM STILL EXISTS. I'M SCRATCHING MY HEAD FIGURATIVELY TRYING TO THINK IF WE HAVE ANY ACTIVE ONES RECENTLY. >> RARE DISEASE CONSORTIA REQUIRE PATIENT GROUPS AS PART OF IT. >> THERE YOU GO. [OFF MIC] >> THE WEALTH STONE CENTER GRANTS, MUSCULAR DYSTROPHY, AND UDALL CENTERS HAVE MANDATED OUTREACH TO PATIENT ADVOCACY GROUPS. THOSE ARE THE ONES THAT COME TO MIND. WHAT WOULD YOU SUGGEST, RON? >> THAT WAS THE KIND OF MECHANISM I WAS RECALLING, FOR EXAMPLE, WHEN NINDS WAS THE PRINCIPAL INSTITUTE BEHIND THE RAID GRANTS, OR RAID PROGRAM, RAPID ACCESS TO INTERVENTION DEVELOPMENT, AND THAT PROGRAM REQUIRED THAT THE CO-APPLICANTS BE ACADEMIC INVESTIGATOR AS P.I., INDUSTRY PARTNER INVOLVED IN THE PROGRAM, AND PATIENT ADVOCACY ORGANIZATION. THAT PROGRAM IS NOW MOVED TO NCATS, NOW CALLED BRIDGES, BUT IN A RELATED WAY, ALAN, TO WHAT EXTENT WOULD YOU SAY THE REVIEW PROCESS IS INFLUENCED BY OR IN WHICH THE PATIENT, THE INVOLVEMENT OF THE PATIENT GROUP, IS ACTUALLY A PLUS IN THE REVIEW PROCESS, THE REVIEWERS OR THE COUNCIL OR THE FUNDING DECISION IS MADE, IN PART INFLUENCED BY THIS PATIENT GROUP, BEING INVOLVED, BEING READY TO HELP WITH THE TRANSLATIONAL, CLINICAL EFFORT AND SO FORTH. >> WELL, ONE OF THE BIGGEST REASONS SOME CLINICAL STUDIES HAVE FAILED HAS BEEN INABILITY TO RECRUIT, SO IF YOU'VE GOT A GROUP THAT'S ON BOARD AND IT LOOKS LIKE THEY ARE GOING TO BE ABLE TO MAKE IT WORKS, THAT'S A HUGE PLUS JUST TO PICK ONE EXAMPLE. >> A BUNCH OF PROJECTS WHERE THE PATIENT ADVOCACY GROUP IS PART OF A GRANT, THEY ARE ACTUALLY DOING A PIECE OF THE WORK, I MEAN THE REVIEWERS LOOK AT THAT AND THEY LIKE ON THE SIGNIFICANCE SECTION SAY, OH, YEAH, PATIENTS SAY IT WAS GREAT SO THEY GIVE THEM A CHECKED BOX ON THAT VERY QUICKLY AND ALSO FOR FEASIBILITY, IN TERMS OF ENROLLMENT, IN THE PATIENT ADVOCACY GROUP HAS A LETTER SAYING THEY WILL HELP THEM ENROLL THAT HELPS A LOT. FOR ALL THE CLINICAL STUDIES, I THINK IT'S A BIG PLUS, IF THEY HAVE PEOPLE COMING IN, SUPPORTING THEM, IT'S NOT JUST MY LITTLE IDEA, YEAH. >> I SHOULD SAY ON SOME STUDY SECTIONS THEY DO HAVE, YOU KNOW, PATIENTS AS REVIEWERS, THAT'S NOT UNPRECEDENTED EITHER. NIMH DOES THAT A LOT OF THE ONE OF THE THINGS A PATIENT CAN BRING AS PERSPECTIVE ON WHAT WOULD OR WOULD NOT BE AN UNACCEPTABLE BURDEN FOR PARTICIPATING IN THE TRIAL, SO YOU HAVE A VERY DIFFERENT PERSPECTIVE IF YOU HAVE THE DISEASE IN TERMS OF WHAT YOU'RE WILLING TO DO TO ANSWERS A QUESTION VERSUS WHAT WOULD A SO-CALLED NORMAL CONTROL BE WILLING TO DO. >> SO I'M BRAD MARGUS, I RUN BIOTECH COMPANIES IN MY DAY JOB, NIGHT JOB VOLUNTEER I RUN THE A-T CHILDREN'S PROJECT THAT FUNDS RESEARCH ON ANOTHER BRUTAL DISEASE, THAT TWO OF MY KIDS HAVE. I WAS ACTUALLY ON COUNCIL A LONG TIME AGO, I WAS MUCH YOUNGER, AND REALLY NAIVE IN AND PEOPLE IN THIS ROOM TOURED ME OVER UNTIL THE LAST MEETING BEFORE YOUR FOUR YEARS WAS UP YOU FIGURED OUT WHAT THE ACRONYMS MEAN. >> YOU LEARNED FASTER. >> I OFFENDED EVERYBODY, PROGRAM DIRECTORS WHISPERED BEFORE THE MEETING STARTED, CAN YOU PLEASE ASK ABOUT THIS? ANYWAY, I ASSUME YOU HAVE -- I'VE HEARD YOU HAVE SOME VERY GREAT CANDID PEOPLE. THE ONLY THING BETTER WOULD BE IF WE WERE SERVING ALCOHOL NOW BUT IF YOU COULD BE MORE CANDID, WE HAVE THE RIGHT MISSION AND GOAL IN THE ROOM, COULD YOU GIVE MORE COLOR COMMENTARY ON WHAT YOU WOULD LIKE FROM US, DISEASE ADVOCATES TO DO. FOR EXAMPLE, YOU'VE REMINDED THE NIH HAS TO SCRAPE TOGETHER 5000 EXPERTS ON THESE STUDY SECTIONS, AND FROM WHAT I'VE HEARD, YOU KNOW, MOST INVESTIGATORS, MANY INVESTIGATORS SUBMIT GRANTS THAT ARE UNFUNDED OVER AND OVER AGAIN, PROBABLY CLOGGING UP THE WHOLE SYSTEM, RUMOR HAS IT IT'S A GOOD STRATEGY AND EVERYONE UNDERSTAND YOU HAVE TO SUBMITTED MORE THAN ONCE TO GET FUNDED. ARE THERE WAYS TO MAKE THIS MORE EFFICIENT AND CAN WE HELP AND TELL THE PATHETIC INVESTIGATORS BACK OFF AND COOL IT OR WE CAN HELP BY MEETING WITH PROGRAM DIRECTORS SAYING THIS IS A REALLY IMPORTANT AREA TO US IN THIS AREA, ONE WHERE WE'RE STUCK, THIS IS AN AREA WHERE IT'S THE TREND. YEAR AND WE'RE NOT KEEN ON IT, BUT IS THERE ANYTHING ELSE WE CAN DO, YOU KNOW, BE CANDID. WE WON'T TELL, RIGHT? [ LAUGHTER ] THE SAME THING GOES WITH RFAs. YOU KNOW, CAN I ASSUME YOU CAN ALSO DO RFAs, SOMETIMES YOU DON'T ASK FOR A SET-ASIDE IN THE CASH BUT WHERE WE THINK AN AREA NEEDS IT AND WE'LL FIND YOU THE EXTRA -- >> RFA IS A ONE-TIME THING. THERE ARE OTHER THINGS WE CAN DO THAT CAN BE A MULTIPLE TIME SET ASIDE JUST FOR GRANTS THAT JUST MISS THE PAYLINE FOR EXAMPLE, A PROGRAM SET ASIDE, THAT RUNS FOR THREE YEARS, AND THAT'S WHERE WE THINK MAYBE AN AREA OF RESEARCH IS IN NEED OF A BOOST AT THIS POINT, AND WE DON'T KNOW EXACTLY WHEN THE APPLICATIONS ARE GOING TO COME IN, WE'LL RUN FOR THREE YEARS, AND SO THERE MAY NOT TAKE THAT MUCH MONEY TO BE ABLE TO PAY A GRANT OR TWO BEYOND THE PAYLINE EACH ROUND AND YOU RUN THAT FOR THREE YEARS AND MAYBE YOU'VE PAID ANOTHER NINE OR A DOZEN GRANTS IN THAT AREA, AND YOU MAY HAVE STIMULATED THAT AREA OF RESEARCH AND MADE A DIFFERENCE. WE'VE DONE A LOT OF THOSE OVER THE YEARS. >> IS THERE ANYPLACE WHERE ADVOCATES DRIVE YOU CRAZY BECAUSE WE'RE NAIVE AND DON'T KNOW HOW IT WORKS OR ANYTHING LIKE THAT? [ LAUGHTER ] >> I CAN ANSWER THAT. [ LAUGHTER ] [OFF MIC] >> IT SEEMS THIS ISSUE OF PATIENTS WANTING TO HELP WITH SYMPTOMS, WHEREAS THE ISSUE OF -- (INAUDIBLE) -- WHICH DOES NOT LEAD TO -- (INAUDIBLE) AND I THINK THAT'S WHY I THINK -- (INAUDIBLE) -- AND FIGURING THIS OUT. [OFF MIC] >> I WANT TO SAY IT'S A TWO-WAY STREET, IT'S REALLY IMPORTANT. >> USE THE MIC SO THE PEOPLE ON THE VIDEOCAST CAN HEAR YOU. >> I WANT TO SAY IT'S A TWO-WAY STREET, IT'S REALLY IMPORTANT FOR THE INVESTIGATORS AND FOR US TO HEAR THE PATIENT VOICE AND WHAT CLINICAL MANIFESTATIONS ARE REALLY IMPORTANT TO YOU AND THE FAMILY BECAUSE WE CAN GET INTO OUR LITTLE BUBBLES AND IT'S REALLY IMPORTANT TO SEE WHAT ARE THE PRIORITIES THAT THE PATIENTS HAVE. >> JUST TO GIVE A SIMPLE EXAMPLE, BEFORE I CAME TO NINDS IF I THOUGHT OF SOMEBODY WHO HAD HAD SPINAL CORD INJURY AND WAS IN A WHEELCHAIR, I ASSUMED THE BIG THINK THEY WANTED TO DO WAS TO BE ABLE TO WALK. WHEN I GOT HERE AND STARTED TALKING TO PEOPLE ABOUT SPINAL CORD INJURY, A BIGGER ISSUE WAS BOWEL AND BLADDER FUNCTION FOR EXAMPLE IN TERMS OF WHAT WAS THE HIGHEST RANK. NOT ALL PATIENTS ARE THE SAME BUT PEOPLE HAVE DIFFERENT PRIORITIES IN TERMS OF WHAT REALLY MAKES A DIFFERENCE FOR DAY-TO-DAY QUALITY OF LIFE, AND YOU CAN GO ON FOR ANY NUMBER OF DISORDERS, THERE ARE THE OBVIOUS ONES THAT THE PERSON WHO DOESN'T HAVE A DISORDER SEES BUT THERE ARE OTHER ONES WHERE THE PERSON WITH THE DISORDER EXPERIENCES AND THAT WOULD MAKE A MUCH BIGGER DIFFERENCE IN THEIR QUALITY OF LIFE, SO THOSE ARE -- >> THIS IS SOMETHING THAT THE -- JOHN PORTER, MUSCULAR DYSTROPHY, PPMD DID A BENEFIT RISK STUDY DETERMINING THE CAREGIVERS OF DUCHENNE PATIENTS, THE THING THEY VALUED MOST WAS SLOWING THE PROGRESS OF DISEASE, AFFECTING A CONDITIONAL APPROVAL OF A DRUG IN EUROPE. I THINK THAT'S SOMETHING PATIENT GROUPS CAN DO THAT HELP INFLUENCE THE PROCESS AND I THINK YOU WANT TO LOOK AT THE ENTIRE THERAPY LINE AND ASK YOURSELF WHERE ALONG THE PIPELINE CAN WE EXPERT THE MOST INFLUENCE, SOMETIMES IT'S A DRUG EFFORT, SOMETIMES IT'S MORE BASIC THINGS THAT HAPPEN THERAPY DEVELOPMENT ACROSS THE LANDSCAPE, AND I THINK RON'S PANEL THAT LAURA AND I ARE ON TOMORROW IS GOING TO ADDRESS SOME OF THESE ISSUES THAT WERE JUST RAISED. >> LET ME GET TO BRAD IN TERMS OF DRIVING US CRAZY. I THINK THE THING TO REALIZE WHEN COMING TO NIH FOR FUNDING IS THAT IT'S AN OPEN COMPETITION. AND THE MISTAKE THAT'S EASY TO BE MADE IS THAT THE GROUP IS LISTENING TO A SMALL GROUP, AND YOU'RE GETTING GROUP THINK THAT ISN'T REALLY VETTED IN A WIDER -- IN A WIDER GROUP OF EXPERTS AND SO YOU GET PEOPLE WHO COME IN AND THEY ARE PASSIONATE ABOUT ONE THING BUT IT'S REALLY -- IT MAY BE EVEN A MISGUIDED THING, AND THE PROBLEM IS THEY JUST HAVEN'T VETTED IT WITH ENOUGH PEOPLE BEFORE THEY COME IN WITH IT SO I WOULD SAY WHAT YOU WANT TO DO IS TO GET YOUR -- GET REALLY BROAD INPUT IN YOUR SCIENTIFIC ADVISERS AND THEN TO SQUASH THINGS THAT ARE REALLY NOT GOING TO FLY AND NOT WASTE A LOT OF TIME TRYING TO DRIVE, YOU KNOW, A ROUND PEG THROUGH A SQUARE HOLE SO I THINK THE DISEASE ORGANIZATION CAN REALLY DO THAT. SOME OF THEM ARE VERY GOOD AT THAT, THEY GET BROAD EXPERIENCE ON BOARD, REVIEW THINGS CAREFULLY, AND THEY ONLY COME IN WITH THE STUFF THAT IS WELL VETTED AND IF YOU HAVE A GROUP OF EXPERTS THAT ARE JUST THE SAME AS THE PEER REVIEW EXPERTS THAT WE HAVE, AND YOUR GROUP THINK IT'S GOOD, IT'S PROBABLY GOING TO DO WELL IN PEER REVIEW. IF YOU HAVE A NARROW GROUP OF PEOPLE ADVISING YOU, WHO REALLY, YOU KNOW, DON'T HAVE THE SAME LEVEL OF WISDOM AS PEER REVIEW COMMITTEE, IT'S PROBABLY GOING TO GET KILLED WHEN IT COMES FOR PEER REVIEW. >> I THINK IT WAS AN INTERESTING SUGGESTION FOR ADVOCACY GROUPS TO PICK UP FUNDING FOR THE 10%ERS OR WHO FALLS OUTSIDE THE PAYLINE. YOU TRACK OUTCOMES, STATISTICS, IS THERE ANY DIFFERENCE BETWEEN NINTH PERCENTILE AND 10th OR THE 25TH PERCENTILE THAT'S NOT IN THE PAYLINE, WHAT DOES THAT FALL? >> THAT'S BEEN A VERY INTENSE AREA OF DISCUSSION RECENTLY. I WOULD SAY IF YOU GO BACK TO THE TIME WHEN THINGS GOT FUNDED ON THE THIRD TRY, THE 25th PERCENTILE, SOME OF THOSE PROBABLY DIDN'T DO AS WELL AS SOMETHING THAT GOT 10th PERCENTILE ON THE FIRST TRY, FOR EXAMPLE, SO THERE ARE SOME DIFFERENCES IF YOU LOOK, BUT THIS IS A CASE WHERE PEOPLE CAN DO VERY SELECTIVE DATA MINING AND IT'S ALSO PRETTY HARD TO SAY EXACTLY WHAT IS A SUCCESSFUL OUTCOME, DOES THAT MEAN THEY RENEWED THE GRANT, BIBLIOMETRICS STATISTIC, WHAT IS SUCCESS? IT'S EASIER TO DETERMINE SUCCESS WHEN WE ACTUALLY DO CURE SOMETHING OR WE ENABLE PEOPLE TO LIVE LONGER HEALTHIER LIVES, IN SOME MEANINGFUL WAY, AND THE REALM OF RESEARCH OFTEN THERE MAY BE SOMETHING THAT WAS INCREDIBLY IMPORTANT, WE DON'T KNOW IT NOW BECAUSE IT ISN'T UNTIL THAT FINDING WAS USED TO ENABLE SOMETHING ELSE THAT MAKES A BIG DIFFERENCE THAT WE KNOW IT. SO WE HAVE HAD MANY, MANY, MANY DISCUSSIONS ABOUT HOW EXACTLY YOU MEASURE A SUCCESSFUL OUTCOME OF A GRANT. IF SOMEONE JUST TOOK THE MONEY AND WENT ON VACATION TO TAHITI FOR FOUR YEARS, THAT'S NOT A SUCCESSFUL OUTCOME FOR ANYONE OTHER THAN THAT PERSON, BUT OTHERWISE IN TERMS OF JUST LOOKING AT WHAT RESEARCH WAS DONE, THERE'S I THINK ONE OF THE CHALLENGES WE'VE TALKED ABOUT A LOT IS WHAT HAPPENS WHEN THINGS DIDN'T WORK AS THE INVESTIGATOR HOPED? WELL, IF THEY DEFINITIVELY SHOWED THAT THIS MECHANISM IS WRONG, YOU SHOULD STOP LOOKING AT IT, IT'S A SO-CALLED NEGATIVE OUTCOME BUT IT'S REALLY IMPORTANT FOR THE FIELD TO KNOW. IF THEY JUST GOT AMBIGUOUS RESULTS AND THEY DID A BUNCH OF EXPERIMENTS AND SPENT A BUNCH OF MONEY AND NOTHING HAPPENED, THEN THAT'S AN ABJECT FAILURE, IT'S PRETTY RARE THAT THINGS FALL INTO THOSE EXTREMES, SO I DON'T THINK ANYBODY WOULD SERIOUSLY ATTACH MEANINGFUL SIGNIFICANCE TO A DIFFERENCE OF MORE THAN AT LEAST FIVE PERCENTILE POINTS IN PER REVIEWED OUTCOME, SOME PEOPLE WOULD SAY 10 OR 15, I THINK WE'RE ALL KIND OF ARGUING THROUGH THE TOPS OF OUR HEADS BECAUSE I DON'T KNOW WHAT DATA WE WOULD SEE THAT WOULD CONVINCE US THAT WE WERE WRONG IN OUR BELIEFS ON THAT. >> IT'S DIFFICULT FOR US, FOR ME ANYWAY, TO WRAP MY MIND AROUND THIS ONE WHEN I THINK ABOUT THE OTHER OPPORTUNITY FOR US. THE WAY I SEE IT, THE NIH GRANT PROCESS SORT OF MOVES THINGS BY DEGREES, RIGHT? YOU FUND THE BEST SCIENCE, THE BEST SCIENCE IS GOING TO HAPPEN, IS GOING TO BE WHERE THERE'S ALREADY A GOOD BODY OF EVIDENCE, ADVANCING AREAS OF RESEARCH WHERE A LOT IS ALREADY KNOWN, AMONG THE MOST SUCCESSFUL PROJECTS, IT'S HARDER SOMETIMES FOR NOVEL IDEA TO GET FUNDED BY THE NIH. WELL, THAT'S AN OPPORTUNITY WHERE WE, THE PATIENT ADVOCACY GROUPS, CAN PERHAPS FUND SCIENCE. DO I FUND THE PROJECT THAT'S REALLY NOVEL THAT I KNOW THE NIH WOULD NEVER REALLY CONTEMPLATE FUNDING OR DO I GO FOR THAT 10th OR 11th PERCENTILE PROJECT THAT COULD BE GOOD, THOSE ARE THE STRUGGLES WE HAVE. >> I WOULD SAY, YOU KNOW, IF YOU'RE PERSUADED THAT ONE THAT LOOKS NOVEL AND IS DIFFERENT AND YOU'VE LOOKED THROUGH NIH REPORTER AND WE'RE NOT DOING ANYTHING LIKE THIS, AND WE'RE ALREADY DOING A LOT IN SOME OTHER AREA, YOU MIGHT WANT TO GO FOR THE THING THAT'S DIFFERENT. THAT COULD BE AN ENORMOUS CONTRIBUTION. >> IS THERE A WAY TO FEED THAT BACK TO US? WILL YOU LET A RESEARCHER KNOW, HEY, THIS IS NOVEL, WHY DON'T YOU GO HE TO THE FOUNDATION, NOT JUST FOR THE ONES THAT SCORED WELL BUT FOR THE ONES THAT -- >> I WILL DEFER TO A LOT OF PROGRAM OFFICERS HERE BUT I THINK THEY ALREADY DO THAT QUITE A BIT. AND ALSO SOMETIMES THINGS SCORE WELL, OR SCORE POORLY, FOR SCIENTIFIC REASONS AND SOMETIMES THEY SCORE POORLY FOR LACK OF SOPHISTICATION IN BEING ABLE TO PRESENT AN IDEA, THOSE ARE RATHER DIFFERENT OUTCOMES, IF IT'S JUST SOMEONE'S GOT AN IDEA THAT'S SO PROVOCATIVE THAT NOBODY IS GOING TO GIVE THEM A CHANCE UNTIL THEY HAVE SEEN SOME HINT OF FEASIBILITY DATA, THAT'S WHERE IF IT SOUNDS PROVOCATIVE AND INTERESTING A FOUNDATION COULD MAKE AN ENORMOUS DIFFERENCE AND THEY MAY NOT BE ABLE TO HELP THEM GET ALL THE WAY TO THE FINISH LINE BUT IF THEY CAN GET SOMETHING IT'S MORE THAN JUST A GOOD IDEA, IT'S AN IDEA WITH SOME MINIMAL AMOUNT OF CREDIBLE DATA TO BACK IT UP, THAT MAY BE ALL THEY NEED, THAT CAN BE HARD FOR SOMEONE WHO IS TRYING TO CHANGE FIELDS, FOR EXAMPLE, TO GET. ONCE UPON A TIME PEOPLE COULD GO ON SABBATICALS AND DO CRAZY THINGS IN OTHER PLACES AND TRY NEW IDEAS. I DON'T HEAR SO MANY OF MY FORMER COLLEAGUES DOING THAT ANYMORE. >> THANK YOU VERY MUCH. [OFF MIC] [APPLAUSE] [OFF MIC] >> IT WAS INITIALLY G-1, THEY WILL BE IN D-2, FUREMAN, MAMOUNAS AND KEHNE. A GROUP IS OUT TO THE RIGHT, THE CONTRACTOR WORKING WITH US TO BUILD A BETTER WEBSITE FOR NINDS. SO IF YOU GIVE THEM INPUT THAT WOULD BE GREAT. OKAY, WELL, THANKS A LOT AND WE'LL SEE YOU AT THE BREAKOUTS AND THE RECEPTION. THANK YOU.