>> THE FIRST TALK TODAY, THIS IS WHAT YOU'VE ALL BEEN WAITING FOR, WE'RE TALKING ABOUT FUTURE AND THINGS THAT COULD BE DONE, AND THE FIRST PRESENTATION IS DAVID BRIANSA AND DON CHE. AND THEY'RE GOING TO TALK ABOUT TECHNOLOGY AND NOT SO MUCH TECHNOLOGY AS A TREATMENT, BUT TECHNOLOGY AS IT CAN BE UTILIZED IN CLINICAL TRIALS. SO THANK, DAVID AND JOHN. >> GOOD MORNING, EVERYBODY. I DON'T KNOW ABOUT YOU ALL BUT I'M CERTAINLY LEARNING A LOT BEING HERE AND IT'S GREAT TO HEAR EVERYBODY'S PERSPECTIVES AND EXPERIENCES DAVE AND I WERE CHARGED TO DISCUSS REHAB CLINICAL TRIALS AND TECHNOLOGY AND TECHNOLOGY, PRIMARY AS A FACILITATOR OF THE REHAB EXPERIENCE. SO IN A RECEIPT RECENT ARTICLE, MANY OF YOU KNOW, CLINICAL TRIALS ARE THE MOST PUBLICLY VISIBLE COMPONENT OF THE BIOMEDICAL RESEARCH ENTERPRISE FROM THE POTENTIAL HUMAN APPLICATION OF NOVEL LABORATORY FINDINGS. THE JENRATION OF ROBUST EVIDENCE ABOUT TREATMENT OR PREVENTIVE INTERVENTION IN ROUTINE CLINICAL CARE, AND SO I HIGHLIGHT THIS LAST COMPONENT. GENERATION OF ROBUST EVIDENCE ABOUT THE TREATMENT OF PREVENTATIVE INTERCEPTION AND ROUTINE CLINICAL CARE. SO I THOUGHT ABOUT THIS TRAJECTORIES TO GO FROM IDEAS TO TRANSLATION. IT OCCURRED TO ME, ALL OF US HAVE DRANK THE SAME KOOL-AID AND THAT KOOL-AID HAS LED US TO WHAT SOME PEOPLE WOULD SAY, THE EVOLUTION OF STATE, WE CAN CHANGE THE WORLD AND THAT'S OKAY. I LIKE HAVING THAT DELUGAL STATE. AND I LIKE, YOU KNOW, IF YOU AIM AT NOTHING, YOU HIT IT ALL THE TIME. YOU MIGHT AS WELL AIM HIGH. BUT IN ORDER TO CHANGE THE WORLD, YOU NEED MORE THAN JUST A BRIGHT IDEA THIS. BRIGHT IDEA HAS TO BE TRANSLATED TO CLINICAL PRACTICE. AND WHILE THE STATEMENT OF CHANGING THE WORLD SEEMS A LITTLE BIT HIGH FLUNG, I THINK WE CAN OPERATIONALLIZE THAT. THIS PATHWAY TOWARDS TRANSLATION, STARTS WITH A PRECLINICAL MECHANISTIC APPROACH TO A PROBLEM. AND I DO MEDICAL DEVICES THIS. TRANSLATION'S GOING TO BE MORE SPECIFICALLY MEDICAL-DEVICE RELATED. BUT EVENTUALLY, T1. THE FIRST HUMAN TRIAL, PHASE 1. AND PHASE 2 AND IN ORDER TO GET FURTHER FOR THE MEDICAL DEVICE WORLD, WE HAVE TO SATISFY THIS ENTITY CALLED THE FDA AND TYPICALLY, ACADEMIC PRACTICE IS A TEASE OR T1. IN ORDER TO GO BEYOND THIS. WE HAD TO PARTNER WITH A COMPANY. A COMMERCIAL ENTITY IN PARTICULAR, WHO WILL THEN DO THE T2 WITH A LARGE PHASE III TRIAL, AND THIS IS A PIVOTAL TRIAL AND UNDER INVESTIGATION OF THE FDA. AND IF YOU'RE SUCCESSFUL USER GO TO T-3, WHICH IS CLINICAL PRACTICE. HOWEVER, THERE'S ALSO A LACK OF TRANSLATION, CALLED GETTING INTO THE WORLD BECOMING THE STANDARD OF CARE. SO WHAT I MEAN BY CHANGING THE WORLD, TOO DEVELOP SOMETHING NOVEL, AND NOT ONLY IS IT EFFECTIVE, BUT PEOPLE ARE ACTUALLY USING IT. AND IT BECOMES STANDARD OF CARE. NOW, OF COURSE, YOU KNOW THAT INFORMATION FROM EACH OF THESE NODE CAN BE USED IN THE PRIOR NODE TO REFINE THE PROCESS FORWARD. IF A COMPANY HAS NO INTEREST IN THIS. YOU'RE DEAD. DOESN'T GO BEYOND T1. THAT'S WHAT WE CALL THE VALLEY OF DEATH. BUT I FURTHER SUGGEST THOUGH, THERE IS A [INDISCERNIBLE]. EVEN IF YOU GET FDA APPROVAL, AS LONG AS THE WORLD DOES NOT SAY, I'M GOING TO USE, NO MATTER HOW MUCH EVIDENCE THERE IS, IT'S NOT GOING TO CHANGE THE WORLD AND IN A VERY FAIR DOCKS CALLOWAY, ESPECIALLY IN THE REHAB WORLD, THERE ARE PLENTY OF THINGS FOR A STANDARD OF CARE. IF YOU PRESENT SOMETHING SOMETHING WITH EVIDENCE, THEY MAY NOT ACCEPT T. SO IT'S A PHENOM NAN THAT'S INTERESTING. BUT ALSO PRETTY SAD. BUT THAT'S THE REALITY WE HAVE TO EXPERIENCE. HOW LONG DOES IT TAKE FROM IDEA TO ACTUAL IMPLEMENTATION? WELL, ACTUALLY, ONE OF MY IDEAS, WAS THAT I DEVELOPED A MINERALLY INVASIVE [INDISCERNIBLE] STIMULATION TREATMENT FORTUNATE TREATMENT OF SHOULDER PAIN. THAT IDEA CAME IN 1995. I GOT A GLANCE FROM CHASE WESTERN RESERVE UNIVERSITY FOR $10,000. I GOT A GRANT. IN 1995 IN 1996, I IMPLEMENTED MY FIST PATIENT. SINCE THAT TIME, I'VE DONE TWO CASE REPORTS, FOUR CASE SERIES AND RANDOMIZED TRIALS. IN 2013, THE PIVOTAL TRIAL WAS. >> Quint: [INDISCERNIBLE]. WE HAVE FDA CLEARANCE JUST THIS QUARTER. THAT'S 20 YEARS. BY THE WAY, IN 1995, MY FIRST YEAR OUT OF RESIDENCY. I'M AN OLD GUY NOW. BUT THIS IS REALITY. I HOPE THAT IT'S NOT. BUT THIS IS KIND OF WHAT WE MEAN BY TRANSLATION. AND RIGHT NOW, WE'RE ONLY AT T3. WE STILL HAVE A CLIFF WE HAVE TO NAVIGATE. AND THAT CLIFF IS PRETTY DEEP. ACTUALLY. TRYING TO CONVINCE A GROUP OF REHAB COMMUNITY PUTTING HIP MALLY INVASIVE PROCEDURES, IT'S NOT THAT EASY. SO THE INDUSTRY WILL DO MUCH OF THAT, BUT WE ALSO WANTED TO HELP THIS PROCESS, AND SO AND RICH WILSON, A K12 TRAINEE, I BELIEVE UNDER JOHN WHITE IT RECEIVED HIS FIRST RO1 GRANT AND WAS A MULTIMODAL RANDOMIZED CONTROL QUESTION. WHICH ASK, IF YOU COMELECTRIC STIMULATION THERAPY, IS THE EFFECTS SUE PEER YOUR TO ONE ALONE. THAT'S NOT A QUESTION THAT INDUSTRY IS INTERESTED OF. SO THAT AS A PAIR LIME, LET'S GO TO THE TOPIC HERE TODAY. SO WHAT IS TECHNOLOGY? >> I DON'T WANT TO DEFINE TECHNOLOGY. THE USE OF SCIENCE INDUSTRY ENGINEERING TO INVENT USEFUL THING OR TO SOLVE PROBLEMS. A MACHINE. A PIECE OF EQUIPMENT, MILITARY METHOD, THAT IS CREATED BY TECHNOLOGY. SO FOR US, MARYELEN GAVE US THESE GUARD RAILS. SHE WANTED US TO TALK ABOUT TECHNOLOGY IN THE CLINICAL TRIAL AND UNDERVENTIONS, TECHNOLOGY INTERVENTIONS TO BE EVALUATED OUTSIDE OF THOSE GUARD RAILS. I THINK WE CAN ALL AGREE THAT SOMETIMES THOSE BORDERS ARE A LITTLE BIT FUZZY AND WE CAN'T REALLY MUTUALLY EXCLUSIVE TREATMENT OF THESE THINGS W. THAT CAVEAT, WE CAN TALK ABOUT CLINICAL TRIALS. GENERATION OF ROBUST EVIDENCE ABOUT TREATMENT OR PREVENTATIVE INTERVENTION, ROUTINE CLINICAL CARE. AND WE DISCUSS A LOT WITH EFFICACY AND EFFECTIVENESS. AND THERE ARE ROLES IN BOTH TYPE OF TRIALS. A REMINDER, EFFICACY IS REALLY ABOUT INDEMONSTRATING CAUSE AND EFFECT. AND IT IS A FOCUS OPESCAPES, AND A LOT OF TIMES, IS A COMPARISON GROUP PLACEBO WITH A DOUBLE BLIND SUGGEST POSSIBLE. AND INTERESTING CONTENTS, AS WE DISCUSSED MULTIPLE TIMES HERE, AND THE QUESTION IS CAN WE USE TECHNOLOGY TO ENSURE CONTENT VALIDITY WITH THE INTERCEPTIONS THAT WE HAVE. AS WELL AS MONITOR WHAT THEY ARE ACTUALLY RECEIVING. YOU KNOW, A LOT WORK ON DOSE AND I THINK WE ALL CAN AGREE THAT IN ATYPICAL REHABILITATION SETTING, PEOPLE DO NOT RECEIVE NOWHERE NEAR THE DOSE REQUIRED TO REALLY INVOKE ACTIVITY NEUROPLASTICITY. CAN WE USE TECHNOLOGY TO NOT ONLY ENSURE THE DOSE, BUT ALSO MONITOR THAT DOSE. AND THE ISSUE OF ADHERENS OF COMPLIANCE, DEPENDING ON WHICH VERSION YOU WOULD LIKE TO USE. ESPECIALLY IN THE EFFICACY TRIALS WHERE YOU HAVE TO ENSURE -- INSURE, IF ONE OF OUR CLINICAL TRIALS OF ELECTRICAL SIM LAYING, IT WAS AN EFFICACY TRIAL. WE SENT OUT PHYSICAL THERAPISTS, ONCE A WEEK TO THEIR HOMES TO MAKE SURE USING THE ELECTRIC STIMULATION SYSTEM L. IN REAL LIFE, THERE'S NO WAY IT COULD HAPPEN. CAN WE USE TECHNOLOGY TO DO THAT AND NOT SEND OUT A VERY, VERY EXPENSIVE CLINICIAN. AND OF COURSE, THERE'S OUTCOMES, AND IN THE EFFICACY TRIALS, IT CAN BE AT THE LEVEL OF MECHANISTIC PHYSIOLOGY, IMPAIRMENT ACTIVITIES AND REALLY, MEASURING CAPACITY IN AN IDEALIZED ENVIRONMENT. IN THOSE SITUATIONS, CAN WE HAVE TECHNOLOGY TO OBJECTIVELY MEASURE INSTEAD OF RELYING ON OBJECTIVE TYPE OF MEASURES. LET'S GO TO EFFECTIVENESS TRIAL. HERE, -- OFTENTIMES, THE CONTROL GROUP IS REALLY A COMPARISON OF A STANDARD OF CARE. NOW, CONTENT SYSTEM ALSO IMPORTANT, ALTHOUGH A LITTLE BIT DIFFERENT. PERHAPS IT'S MORE, WHAT ARE THEY ACTUALLY RECEIVING? AND PERHAPS MONITOR THAT, WITHOUT TRYING TO ENSURE FIDELITY. IT'S AN EFFECTIVENESS TRIAL. QUESTION OF THOSE. CAN A DOSE BE DELIVERED THE WAY WE SAID WE'RE GOING TO DELIVER, AND WHAT DID THEY RECEIVE? ADHERE EPPS. WHAT DO THEY ACTUALLY D. A LITTLE ASK, DO I REALLY CARE. BUT ALL OF THESE QUESTIONS ARE AMENABLE TO TECHNOLOGY ASSESSMENT. AND OF COURSE, OUTCOMES. OUTCOMES AND EFFECTIVENESS TRIALS ARE ESPECIALLY IMPORTANT. IN MANY WAYS, WE'RE INTERESTED IN PERFORMANCE. OUTSIDE THE LABORATORY. SO HOW DO WE MONITOR. DO IT IN SUCH A WAY, WITHOUT HAVING AN EFFECT WHERE THE PATIENTS, THE PARTICIPATES ARE KNOWING THEY ARE BEING MONITORED. THOSE ARE DEVICE WE HAVE TO ADDRESS. BUT I ALSO WANT TO TALK A LITTLE BIT ABOUT IN THIS STATEMENT, CALLED ROUTINE CLINICAL CARE. IF OUR JOB IS TO GENERATE ROBUST EVIDENT ABOUT TREATMENTS OF TREATMENT IN CLINICAL CARE. WHAT DO WE MEAN BY THAT? DO WE NEED TO BE CONCERN ABOUT THE HEALTHCARE ENVIRONMENT THAT WE ARE OPERATING IN. SO THE BIG ELEPHANT IN THE ROOM. SO THREE YEARS AGO, I WENT TO THE DARK SIDE, I SPENT THE FIRST 20 YEARS OF MY LIFE DOING PRIMARY RESEARCH. YES, I TOOK CARE OF PATIENTS, BUT I DIDN'T HAVE TO MANAGE ANYBODY. AND A YEAR AND A HALF AGO, I WAS ALSO THEN PROMOTED, THE DUBIOUS HONOR, TOO LEAD OUR SERVICE LINE. SO I ACTUALLY MANAGED, A NEUROLOGIST, PODIATRIST, PAPER ANESTHESIOLOGIST, MY WORLD WAS OPEN TO -- MY EYES WERE OPENED TO A WORLD I'VE NEVER SEEN BEFORE. THE RELATIONSHIP WITH THE WORLD OUT THERE, WITH RESPECT TO MEDICAL ECONOMICS AND HEALTHCARE DELIVERY. AND SO, THE BIG ELEPHANT IN THE ROOM, EVEN HERE, REALLY SHOULD BE HEALTHCARE REFORM, IF WE ARE AT ALL INTERESTED IN ROUTINE CLINICAL CARE. AND PERHAPS ONE SIDE SADDAM PART ABOUT THIS. HEALTHCARE REFORM IS NOW HAPPENING 2002 AN OUNCE OF EVIDENCE, IT'S JUST BEING DONE. WHERE IN THE WORLD WERE WE AT THE NIH WHEN IT WAS HAPPENING? THERE'S THOUGH EVIDENCE THAT WHAT'S BEING PROPOSE SIDE BETTER THAN WHAT'S HAPPENING NOW WE DEAL WITH ROUTINE CLINICAL CARE. I THOUGHT I'D SHARE WITH YOU, THINGS I'VE LEARNED AND THE IMPLICATION IT HAS OCLINICAL TRIALS AND HOW TECHNOLOGY MIGHT HAVE A ROLE. SO THIS IS THE HEALTHCARE SYSTEM AS WE HAVE IT NOW. WE HAVE A POPULATION, AS YOU SEE UNDERNEATH, AND THIS POPULATION WANTS TO HAVE GOOD HEALTH. SO THEY SEEK AND INTERACT WITH OUR HEALTHCARE SYSTEM AND WE ARE HAPPY TO DELIVER HEALTHCARE THAT INCLUDES ACUTE HOSPITALIZATION, SURGERY, OUT PATIENT AS WELL AS INPATIENT. PRIMARY CARE, AS WELL AS MEDICINE. SUPPOSEDLY WHEN, POPULATION DOES THIS, THEY GET GOOD HEALTH IN RETURN. AT LEAST THAT'S THE IDEA. OF COURSE, ALL OF THIS COST MONEY, AND WHILE WE DO HAVE THIRD PARTY PAYERS, BUT AT THE END OF THE DAY, WE PAY FOR T. WE ARE THE SOURCE OF THE MONEY. AND THEN AND THEN, THE HEALTHCARE SYSTEM IS HAPPY TO RECEIVE REVENUE FOR THE SERVICES THEY OFFER, WHICH WE CALL THESE THINGS. BUT OF COURSE, THAT REVENUE IS MORE THAN WHAT IT ACTUALLY COST. AND SO THE HOSPITALS HAVE A COST STRUCTURE. THEN IF YOU SUBTRACT THE EXPENSES FROM THE REVENUE. THAT, OF COURSE IS THE OPERATING INCOME. WHEN YOU ADD UP ALL THE OPERATING INCOME, THAT BECOMES A SYSTEM OPERATING ECONOMIC THAT THEY CAN WORK W. AND WE CALL THIS FEE FOR SERVICE. PERHAPS YOU CAN DEBATE WHETHER THIS HAS WORKED OR NOT. I THINK WE CAN ALL AGREE, THIS IS VERY, VERY EXPENSIVE AND THERE ARE OTHER COUNTRIES WHO SPEND LESS MONEY THAN WE D. WHO HAVE BETTER HEALTH THAN US. SO HOW IS THAT HEALTHCARE REFORM GOING AND WHAT'S THE GENERAL IDEA? IN THAT WORLD, THE HEALTH SYSTEM NOW INCLUDES THE POPULATION. DOES THAT INCLUDE HEALTHCARE SYSTEM THERE'S A MOTIVATION NOT TO PROVIDE THESE SERVICES AND THE HOT TICKET ITEMS LIKE IN-PATIENT -- IS A NO-NO. ALL OF THESE THINGS WE THINK ARE BIG, WOULD NEED TO BE REDUCED. HOSPITALIZATION, PROCEDURES, POST CHARACTERS PRIMARY CARE. AND INCREASE AS WELL AS REPRESENTATIVE CARE. AND HERE'S WHAT'S INTERESTING. SUDDENLY, HOME CARE BECOMES IMPORTANT. THERE'S NO OVERHEAD. THE IDEA THAT WE, AS HEALTHCARE PROFESSIONALS, DELIVERING CARE IN THE HOME, IS A VERY BIG PART OF WHAT THIS NEW WORLD LOOKS LIKE. AND WE NEED TO BE AWARE OF. BY THE WAY, THE SYSTEM CAN REALLY CHEAT BY DELIVERING NO HAIR, RIGHT? THE MONEY YOU GET NEXT YEAR IS GOING TO BE FUNCTION OF THE HEALTH OF THE POPULATION THIS YEAR. WE CALL THAT VALUE BASE PURCHASING. AND THE HOSPITAL WANTS TO MAKE A LOT REVENUE. SO THEY BUY UP A LOT OF HOSPITALS. WE CALL THAT CONSOLIDATION. SO AT THE END OF THE DAY, WE CREATE VALUE. QUALITY OF EXPERIENCE OUTCOME. DENOMINATOR. DOLLARS. THE QUALITY AND EXPERIENCE HAPPENING NOW. OUTCOME SYSTEM NOT. THIS IS WHERE WE HAVE THE OPPORTUNITY AND REHABILITATION. THE OUTCOMES ARE GOING TO BE IMPORTANT. I BELIEVE, ULTIMATELY, IT'S GOING TO BE FUNCTIONAL IN SOCIETAL PARTICIPATION, WHICH MEANS PERFORMANCE MORE THAN CAPACITY. WE HAVE AN OPPORTUNITY. SO THOSE ARE THE THINGS RESOURCES, TECHNOLOGY CAN ADMIT GAY GATE THESE THINGS. PERFORMANCE, ESPECIALLY, SO THIS MAY RESULT IN REDUCED COST, ININCREASED EFFECTIVENESS AND CLINICAL RELEVANCE AND TECHNOLOGIES TO FACILITATE DATA COLLECTION OF STORE ANNUAL, ENSURE FIDELITY. PROVIDE MONITOR, PRESCRIBE DOSING, INSURE MONITOR COMPLIANCE AND MEASURING OUTCOMES, ESPECIALLY PERFORMANCE. >> THE USE OF TECHNOLOGY, BOTH IN PRACTICE AND RESEARCH. MY ROLE HERE IS TO GET DOWN TO THE NUTS AND BOLTS OF TECHNOLOGY ITSELF. SO JUST BRIEFLY, YOU KNOW, TOO SUMMARIZE THE USES OF TECHNOLOGY, FIRST, THE TECHNOLOGY, YOU KNOW, WAS INTRODUCED AND IT WAS USED BY RESEARCHERS. TO COLLECT DATA AND THE TECHNOLOGY WAS PRIMARY THE SUBSTITUTE FOR TRADITIONAL PAPERWORK PAPER FORMS. NOW THAT'S DONE ELECTRONICALLY AND I HEARD YESTERDAY, YOU KNOW, THERE'S TOOLS AVAILABLE TO DO THIS RED HAT, AND CUSTOM TOOLS. BUT ALSO, YOU KNOW, TECHNOLOGY SUICIDED BY PARTICIPANTS OF THE TRIALS THEMSELVES. EITHER TO ENTER DATA THEMSELVES. OF COURSE, THERE IS TECHNOLOGY THAT KIND OF CROSS THAT IS GUARD RAIL AND BECOMES PART OF THE INTERVENTION ITSELF. AND WHEN WHEN THAT HAPPENS, SOMETIMES IT'S IN THE FORM, I USED TERM ADAPTIVE PROTOCOL. A PROTOCOL THAT CHANGE, ACCORDING TO REAL-TIME EVALUATION OF THE VARIABLES. SO ONE OF THE ADVANTAGE OF DOING TECHNOLOGY OF THE OLD STANDARD FORMS, BETTER DATA ACCURACY, MONITORING OF TREATMENT FIDELITY, AS WE HAVE MENTIONED, AND I MAY JUST PAUSE HERE FOR A SECOND AND SAY, YOU KNOW, MOST OF WHAT I'M PRESENTING, WE TALKED ABOUT YESTERDAY, TECHNOLOGY IS NOW A VERY COMMON THREAT IN OUR TRIALS AND IT'S HARD TO SEPARATE A TRIAL FROM TECHNOLOGY. SO MUCH OF WHAT I'M SAYING HERE IS BY WAIVE SUMMARY WHAT WE HAVE ALREADY COVERED THROUGH THE OTHER SESSIONS. BETTER COMPLIANCE ONE BENEFITS OF USING TECHNOLOGY. AND THAT MAY BE BECAUSE THE PARTICIPANTS CAN USE THE DEVICES. EASILY. MORE EASILY THAN PAPER DIARIES AND TOOLS LIKE THAT. AND OF COURSE, THE IT'S REMOTE MONITORING INSURES BETTER COMPLIANCE AS WELL. AND WANT ABILITY TO ADAPT THE INTERVENTION WITHOUT ANY DELAY, IS SOMETHING CAN YOU DO WITH TECHNOLOGY, THAT YOU COULDN'T DO WITH FORM. SO IF YOU WANT TO RESPOND TO HOW THE PARTICIPANTS ARE PERFORMING IN THE FIELD, CAN YOU DO THAT WHEN YOU'RE USING TECHNOLOGY TO GET INSTANT FEEDBACK. OR IN SOME CASE, THAT MAY BE IT'S AS AN ADAPTATION. FOR EXAMPLE, IF YOU HAVE A PROTOCOL WHERE YOU'RE LOOKING AT A PARTICIPANT ADHERENS TO AN EXERCISE PROTOCOL. WELL, THERE MAY BE SOME WAY OF AUTOMATICALLY SENDING A REMINDER TO THAT PERSON IF THEY'RE NOT ADHERING. ANOTHER BENEFIT IS SIMPLIFIED DATA PROCESSING. YOU KNOW, IF THE DATA IS ELECTRONIC TO BEGIN WITH, IT'S EASY TO GENERATE REPORTS AND DUMP THAT INTO YOUR STATISTICS PROGRAM, AND ANALYZE THE DATA AND AS JANE MENTIONED, YOU KNOW, TECHNOLOGY IS PARTICULARLY USEFUL IN EFFICACY TRIALS SOME OF THE DISADVANTAGES AND CONSIDERINGS OF TECHNOLOGY. COST. IT'S NOT FREE. YOU NEED A DIFFERENT SET OF EXPERTISE, YOU NEED THE EQUIPMENT AND HAVE TO MAINTAIN THAT EQUIPMENT. THAT ALL COSTS MONEY. SECURITY IS PRETTY STANDARD NOW. IT USED TO BE A, YOU KNOW, A CONCERN, BUT THE PROTOCOLS ARE IMPLEMENTED IN ALMOST ALL MOBILE DEVICES NOW, COMPUTER DEVICES, CRYPTIC DATA DURING TRANSMISSION, WHEN YOU STORE IT AND THAT'S NOT REALLY A PROBLEM ANYMORE. WHAT IS A MORE INTERESTING PROBLEM THOUGH IS PRIVACY. AND ONCE YOU COLLECT DATA REMOTELY, YOU'RE GETTING INTO A PERSON'S LIFE, IN A WAYS YOU HAVEN'T GOTTEN INTO THEIR LIVES BEFORE. SO YOU HAVE TO CONTRACT PRIVACY OCCURRENCE OF THAT ACTIVITY. THAT'S ESPECIALLY CONCERNING WHEN YOU DO THINGS LIKE LOCATION TRACKING. ACCESSIBILITY OF THE DEVICES IS A CONCERN. AND SO IF YOUR PARTICIPATE GROUP IS AN OLDER GROUP, YOU NEED TO BE CONCERNED WITH VISION. IF PEOPLE HAVE DEXTERITY, I NEED TO TAKE T TAKE THAT INTO ACCOUNT. IF YOU'RE USING MOBILE DEVICES, MAYBE YOU NEED TO USE A BIGGER DEVICE, UPON BIGGER SCREEN, LARGER PRINT. AND MAYBE YOU NEED TO DESIGN YOUR APPS IN SUCH A WAY THAT THEY CAN BE USED BY SOMEBODY WITH A DEXTERITY IMPAIRMENT IF THAT'S PART OF YOUR POOL. AND THERE'S ENVIRONMENTAL HAZARDS OF THE EQUIPMENT ITSELF. THEY GET DROPPED IN TITS. THEY GO OUT IN THE RAIN AND OF COURSE, ANYTHING WITH A BATTERY IS, HAS SOME SUSPECT RELIABILITY. BATTERIES RUN OUT OF ENERGY AND THEY FAIL AND DON'T GET CHARGED. THE BATTERY IS ALWAYS A CONCERN. I PUT THESE ON THE SCREEN. HOPE HOPEFULLY, YOU'LL HAVE SOMETHING IN MIND. IF NOT, WE CAN TALK ABOUT THIS. SING I'LL OPEN THIS INAND SEE WHAT YOU'RE THINKING AND SEE WHAT'S INTERESTING TO YOU. I THINK I'D LIKE TO THROW SOMETHING ELSE INTO THE DISCUSSION. AND THIS IS A CONCERN I V EXPRESSED IN THE CONFERENCE IN MAY, AFTER A TECHNOLOGY TALK. A CONCERN THAT I HAVE THAT I SEE IN THE LITERATURE, THE SCIENTIFIC RESEARCH LITERATURE, THAT WHEN THERE'S AN INTERVENTION, THAT INVOLVE ACE HIGH-TECH INTERVENTION, AND A HIGH-TECH OUTCOME MEASURE, EVEN WHEN THOSE OUTCOMES ARE NOT AS GOOD OR IF THEY'RE EQUIVALENT TO THE THING THAT IS CAN BE ACHIEVED WITH A LOW TECH INTERVENTION, THEY GET AN AMAZING AMOUNT OF HYPE, WE HAD THIS NARROW PLASTICITY CONFERENCE IN THE PHYSICAL THERAPY WORLD, AND THAT SAME CONTENT WAS DISCUSSED AT THAT MEETING. AND I THINK SOMEWHAT PURPOSEFUL FOR ME. I HAVE RECEIVED BETTER OUTCOMES AND I DIDN'T USE YOU AN OUTCOME MEASURE BIOMARKER. BUT MY OUTCOMES WERE BETTER. MY WAS MORE TRACT CAL. DIDN'T USE A VERY EXPENSIVE PIECE OF EQUIPMENT THAT REQUIRED EXTENSIVE TRAINING TO BE ABLE TO USE AND SET UP AND YET, CAN I ACTUAL, MY INTERVENTION DID NOT GET THE SAME PRESS THAT THIS INTERCEPTION D. SO I 19MEAN, I HAVE A CONCERN, IT'S NOT BRIGHT AND SHINY. SOMETIMES THE THINGS WE DO IN REHAB ARE NOT BRIGHT AND SHINY, AND I'M CONCERNED WHEN THE TECH GRABS ATTENTION. I'M NOT A TECHNO PHOBE. I GOT THE THIRD LOCOMAT DESERVED TO THE UNITED STATES. I FEEL I GAVE BERKELEY BIONICS THE USE OF EXOSKELETON. I GET CONCERN WHEN I SEE THAT GOT ALL THE TAKEN AND STUFF THAT PEOPLE HAVE TO HAVE ACCESS TO, DOESN'T GET THE TENACIN. >> THAT'S FRUSTRATING AND I SHARE YOUR FRUSTRATION. >> (INAUDIBLE). >> COULD YOU SPEAK INTO THE MIC A LITTLE. YEAH, SO I THINK WHEN YOU THINK ABOUT USING TECHNOLOGY, YOU HAVE TO BE PRACTICAL. YOU HAVE TO, YOU KNOW, CONSIDER, REALLY IS A PAPER FORM THE BETTER WAY TO GO FOR IN PARTICULAR TOOL. THERE WAS A STUDY PUBLISHED LASTED WEEK, CAME OUT OF PITTSBURGH AND IT WAS A WEIGHT LOS STUDY, A RANDOMIZED TRIAL, WHERE PEOPLE WERE RANDOMIZED INTO A GROUP, PUT INTO A WEIGHT LOSS PROGRAM, WITH A FIT BIT AND ONE WITHOUT IT. YOU PROBABLY SAW THIS. IN GOT A LOT OF PRESS. AND THE FIT GROUP PERFORMED STATISTICICALLY WORSE. SO TECHNOLOGY CAN SOMETIMES HINDER. I THINK THE PEOPLE TOOK 2,000 STEPS AND SAID, I DESERVE A CUP CAKE. YOU KNOW. SO YEAH. TECHNOLOGY IS NOT ALWAYS THE ANSWER, BUT IT DOES GET PROCESS. MY FIST TRIAL, WHERE THERE WAS A WOMAN WHO WAS 98% ADHERENT TO OUR EXERCISE PROTOCOL AND GAINED 15 POUNDS OVER THE TWO YEAR INTERCEPTION AND WE WERE LIKE, WHAT? THERE WAS NO DIETARY INTERVENTION. SHE DECIDED SHE COULD HAVE A MCDONALD'S APPLE PIE EVERY DAY, BECAUSE SHE WAS IN THE INTERVENTION. IT ACTUALLY DOES HAPPEN. SO NO. BUT MY QUESTION TO YOU ALL IS I'D LIKE YOU ALL TO TALK A LITTLE BIT ABOUT THE ISSUE OF THE DIGITAL DIVIDE, AND THE FACT THAT THERE ARE STILL PLACES I'M FINDING OUT IN CENTRAL PENNSYLVANIA, THAT THEY'RE USING DIAL-UP. YOU KNOW, AND WHERE THEY DON'T HAVE, YOU KNOW, SMART PHONE, AND WHERE THEY DON'T HAVE, WE CAN'T GUARANTEE THEY HAVE INTERNET SERVICE THAT DOES EXIST, AND OLDER ADULTS ARE LESS COMFORTABLE WITH TECHNOLOGY. SO -- >> LET ME JUST RESPOND TO EDDIE FIRST. I AGREE WITH YOU. ESPECIALLY, BECAUSE I COME FROM A CENTER, WHERE WE DEVELOP FULLY IMPLANTABLE DEVICES THAT COST $7,000 PER POUND. AND YES THOSE THINGS GET A LOT MORE PRESS. THE PRESS IS ONLY INTERESTED IN WHAT WHAT'S GOING TO CELL, NOT WHAT'S REALLY HELPING SOCIETY. SO I BEIRUT THIS UP AGAIN. AT THE END OF THE DAY, AT THE END OF THE DAY, IF YOU HAVE AN APPROACH THAT GIVE US YOU THE SAME EFFECT TREATMENT WISE. BUT MINE COSTS FIVE TIMES MORE, GUESS WHAT THE SYSTEM IS GOING TO PURCHASE? ACTUALLY, JUSTICE WILL HAPPEN, EDDIE. YOU'RE NOT REQUESTING TO BE ABLE TO AFFORD, UNLESS THE OUTCOME IS THAT MUCH BETTER. RIGHT AND SO. PEOPLE VERY MUCH OPINIONS ABOUT HEALTHCARE REFORM. I THINK IT'S GOING TO BE GOOD. IT'S GOING TO FORCE US TO ADDRESS THIS EQUATION. NOW, THE QUESTION ABOUT THE DIVIDE. IS ACTUALLY A QUESTION OF DISPARITY. WE CONNECT OUR INTERVENTIONS TO VIDEO GAME.<$ TAB TO FACILITATE, TOO ENGAGE OUR LIMBIC SYSTEM TO PARTICIPATE IN THIS MOTOR RELEARNING PROCESS MY OLDER STROKE PATIENTS DON'T CARE FOR T. THAT'S JUST NOT PART OF THEIR WORLD. OUR YOUNGER ONCE, OUR C.P. KIDS TAKES LOVE T. THEY'RE REALLY INTO THAT SO THINGS GOING TO BE TRUE FOR ALMOST EVERY OTHER DIABETES OF MENTION OF RESOURCE AND A QUESTION WE HAVE TO ADDRESS FROM A DISPARITY PERSPECTIVE. I DON'T HAVE A SOLUTION, BUT SIMPLY ACKNOWLEDGING, THAT PROBLEM EXISTS. WHEN WE RELIED ON THE INTERNET, IT WAS MORE OF A PROBLEM. BUT THERE AREN'T MANY PLACES WITHOUT CELL PHONE ACCESS AND SO NOW, WE'RE MOVE TO THE MOBILE DEVICES AS OUR BASIC COMMUNICATION TOOL AND I THINK THAT PROBLEM IS DIMINISHED. SO I WONDER, IF THERE'S A COUPLE POLICY-RELATED ISSUES. SO AS PEOPLE WHO HAVE APPLIED TO PECORI, THEY ARE WRITING UP METHODOLOGY STANDARD THAT IS THEY EXPECT PEOPLE TO ADHERE TO, IF THEY'RE PLAYING FOR GRANTS. AND, I GUESS I JUST WONDERED WHETHER THERE ARE CERTAIN DEMARCATED WHERE NIH MIGHT WANT TO GO ON RECORD IN A MORE PUSHY WAY. A LOT OF PEOPLE FILLED THEM OUT THE NIGHT BEFORE. HYPOTHETICALLY, IF I INTEND TO COLLECT DATA ON WHAT PEOPLE ARE DOING AT HOME. SHOULD WE BE SAYING YOU NEED A STRATEGY THAT IS MORE RELIABLE THAN PAPER TIRES. JUST AN EXAMPLE I CAN'T REMEMBER THE NAME OF THIS CERTIFICATE THAT YOU CAN GET, BUT PEOPLE WHO DO LIKE DRUG AND ALCOHOL RESEARCH OR THINGS LIKE THAT, CAN GET THIS SO COULD YOU GET A CERTIFICATE OF CONFIDENTIALITY AROUND GPS DATA EXAMPLE OR SOMETHING LIKE THAT SUCH THAT YOU CAN TELL PARTICIPANTS THAT YOUR RECORDS CONDITION BE SUBPOENAED. >> YOU CAN GET A CERTIFICATE OF CONFIDENTIALITY. IT'S THROUGH HHS. I'M LOOKING FOR POLICY PEOPLE. BUT IT'S FOR RESEARCH AND WE GET CERTIFICATES FOR TRAUMATIC BRAIN INJURIES. MY HAPPENING, IT'S AVAILABLE THROUGH A RESEARCH PROJECT. I HAVEN'T DEALT WITH ONE FOR A LONG TIME AND WE WOULD LINK YOU WITH PEOPLE AT THE INSTITUTE, WHO COULD HELP YOU WHEN I SAY YOU, I MEAN YOUR INSTITUTION. >> I SAT ON ONE OF THE BOARDS FOR ABOUT THREE YEARS, AND THE ISSUE OF DIGITAL DATA AND DIGITAL SECURITY CAME UP WE DID NOT ALLOW ANY WAIVERS OF CONFIDENTIALITY BECAUSE THE ISSUE WAS, YOU COULDN'T PROTECT THE DATA UNLESS YOU CAN SAY THAT YOU COULD PROTECT THE DATA THE ISSUE IS NOT WHETHER YOU CAN GET THE CONTACTOR WAIVER. IT'S WHETHER OR NOT YOU CAN SHOW THAT YOU CAN SHOW THAT YOU CAN PROTECT THE DATA. THERE ARE STANDARDS FOR SECURITY OF DATA, WHICH WE ALL KNOW ARE GARBAGE AND OTHER HACKERS CAN GO PAST THEM, YOU HAVE TO AT LEAST SHOW THAT YOU'VE GONE THE DISTANCE TO ENSURE, WHATEVER YOU'RE DOING WITH TECHNOLOGY],YOU'VE DONE YOUR BEST TO AVOID HAVING ITIN' DISAND THE OTHER PEOPLE WITH CERTIFICATES OF CONFIDENTIALITY, JOHN, IS IT'S REALLY INTENDED AN--THE INTENT OF CERTIFICATE OF CONFIDENTIALITY IS A LEGAL PROTECTION FOR THE PERSON INVOLVED. THERE TO PROTECT TEMTHEM FROM HARM. AND SO, ESSENTIALLY, WHAT YOU WOULD BE ARGUING FOR A GPS, IT'S POTENTIALLY POSSIBLE THAT IN TRACKING SOMEONE'S LOCATION, THEY CAN BE TIDE TO SOMETHING NEVER AIRUOUS THAT THEY WEREN'T INVOLVED N BUT IF A CRIME HAPPENED IN THE SAME LOCATION, THEY CAN BE TRACKED USING GPS THAT YOU COLLECTED DURING YOUR STUDY. THEY CAN BE CONNECTED TO SOMETHING THEY WOULD NOT OF COURSE OTHERWISE. BUT YOU WOULD HAVE TO MAKE A PRETTY BIG REACH TO DO THAT. THE REASON THEY DO IT FOR SUBSTANCE USE, FOR GENETIC TEST SUGGEST BECAUSE IT COULD CAUSE A HARM. POLICY WISE, THE DIGITAL PROTECTION IS GOING TO BE THE BIGGER HURDLE. THE CONFIDENTIALITY TO PROTECT THE PERSON FROM LEGAL HARM. WE MAKE A VIRTUAL SENSE AROUND WHERE WE TRACK THEM. >> SO I JUST WANT TO GO BACK TO TALKING ABOUT THE TECHNOLOGY USE USED FOR FOR LOW-INCOME PATIENTS AND COMMUNITIES THERE ARE SOME FACILITATORS AND? BARRIERS, SO YOU KNOW, I DID WORK IN RURAL ALABAMA WITH BREAST CANCER SCREENING FOR MINORITY WOMEN IN RURAL COUNTIES AND YOU WOULD BE SURPRISED THAT EVERY WOMAN HAS A SMART PHONE. YOU KNOW, EVEN IF THEY'RE LOW INCOME, THE SMART PHONE IS VERY, VERY IMPORTANT. THE SAME IF YOU DO GLOBAL HEALTH IN DEVELOPING COUNTRIES. FOUND ARE JUST EVERYWHERE, AND MY COLLEAGUES USE THEM TO SCREEN FOR CERVICAL CANCERS, TAKING PHOTOS AND TRANSMITTERS. SO THE TECHNOLOGY CAN FOR CERTAINLY TO USE T. BUT THE PROBLEM WE HAVE, AGAIN, I'M GOING TO COMMENT TO THE EPIROLEMENT OF MINORITIES AND CLINICAL TRIALS. IF WE START DOING THE MONITORING. THEY DON'T LIKE THAT. IF YOU'D LIKE, YOU KNOW, WHEN YOU'RE INVADING, YOU'RE TAKING IT ON THIS CONSPIRACY THEORY, YOU'RE LOOKING FOR OTHER THINGS, AND JUST BEING THAT. SO WE HAVE HAVEN'T DONE WORK YET OF HOW TO OVERCOME THIS BARRIER AND DEVELOPING TRUST. IF, IF WE ARE REALLY TRYING TO TRACK MOBILITY AND ALL THE AGE, WHATEVER WITHIN YOUR HOME. OR TRYING TO GET TO KNOW YOUR VITAL STATUS THROUGH TECHNOLOGY, TOO KNOW IF YOUR BLOOD PRESSURE OR YOUR DIABETES OR IF YOU COMPLIED WITH YOUR MEDICINE. WE'RE USING TECHNOLOGY TO DO THAT. HOW CAN WE, GIVEN THE ASSURANCE THAT THAT'S ALL WHAT YOU'RE GETTING BECAUSE THEY'RE GOING TO THINK YOU'RE GETTING EVEN THEIR DEBT. A LOT OF THINGS. SO WE HAVEN'T DONE THIS KIND OF WORK HERE AND IT WOULD BE GREAT THAT, PEOPLE IF THEY DO TECHNOLOGY, THEY CAN HELP US TO KNOW MORE TO MAKE THEM INFORMED. >> I'M GOING TO CHANGE TOPIC A LITTLE BIT AND I JUST WANT TO MAKE TWO POINTS. IS THE FOLLOW-UP TO EDDIE'S COMMENTS. PERHAPS WE SHOULD BE THINKING B WHEN WE SEE APPLICATIONS OR WHERE WE'RE DOING PROJECTS WITH TECHNOLOGY, THAT THE BAR FOR THE DIFFERENCES THAT WE'RE LOOKING FOR, IS GOING TO HAVE TO BE A LOT BIGGER THAN THE BAR FOR A LOW-COST, LOW EXPENSE AND THAT MIGHT BE A TRIGGER POINT, AS DANIEL MAPPED OUT YESTERDAY, IN HIS STUDY. THERE ARE TRIGGER POINTS WHETHER HE GOES FORWARD OR TO THE KNOT AND IF THE GAINS FROM A PARTICULAR DEVICE AREN'T BIGGER THAN OR SUBSTANTIALLY BIGGER THAN LOW COST, THEN WHAT'S THE POINT OF GOING FORWARD WITH IT MY SECOND POINT WOULD BE. WE HAVE SPENT A LOT TIME MONITORING PEOPLE WITH ACCELEROMETERS. AND OH, WE HAVE THIS COOL TECHNOLOGY AND THEREFORE, WE'RE DONE. AND IT TOOK IS FOUR YEARS TO FIGURE OUT WHAT SIGNAL FROM THESE THINGS THAT WE SHOULD TAKE, THAT MEANT ANYTHING. WE KIND OF ENTER THIS DANGEROUS PERIOD, NOW WE CAN MEASURE EVERYTHING UNDER THE SUN. OF ALL THE 150 THINGS, THAT WE MEASURE, HOW DO WE DO THE WORK THAT IS NOT BRIGHT AND SHINY IS NOT LABOR INTENSIVE AND THE THIS IS SOMETHING THAT'S PERSON. >> ALL RIGHT. SO AN ISSUE REALITIED TO THAT IS THE VALIDITY OF THESE DEVICES, TOO. FOR PEOPLE FOR PEOPLE WHO HAVE VERY SLOW GAIT, THEY'LL SMART WATCH. OR THE CELL PHONE, OR THE FIT BIT, THEY DON'T COUNT VERY WELL IF YOU'RE MOVING VERY SLOWLY AND IRREGULATEDLY. THE TYPE OF DEVICE IS SOMETHING THAT'S ON THE LEG COUNTING STEPS. >> SO I JUST WANT PHONIC A COUPLE COMMENTS. THERE IS A SLIPPERY SLOPE. YOU JUST BROUGHT IT UP, DAVID AND THAT IS THE VALIDATION. WE NEED TO BE CAREFUL, USING TECHNOLOGY FOR TECHNOLOGY SAKE. IT'S AUTOMATICALLY THOUGHT OF AS BETTER THAN DIETARY RECORD KEEPOG PAPER ATTACHED ON THE PRIVILEDGE. AND PHYSICAL ACTIVITY. WE CONGRESS, CAN I COUNT STEPS, IT'S GOT TO BE BETTER, I CAN PROVIDAN APP WHERE THEY CAN LOG WHAT THEY EAT. HAD T HAS TO BE BETTER. THERE'S A DANGER IN ADOPTING IT BECAUSE IT'S TECHNOLOGICALLY ADVANCED, IT'S GOT TO BE BETTER THAN WHAT WE'VE BEEN DOING. SO THAT'S SORT OF POINT NUMBER ONE. THE ONE YOU JUST MENTIONED ABOUT VALIDATION IS CRITICAL. BECAUSE THE DEVICE INDUSTRY IS PRODUCING PRODUCT AT SUCH A RAPID RATE, AND PUTTING IT OUT IN THE MARKETPLACE W A PRIMARY GOAL OF MAKING MONEY. WHETHER OR NOT IT'S SCIENTIFICALLY RIGOROUS TO USE THAT DEVICE AND WE NOW HAVE THESE NEW RIGOR STANDARDS TO ADHERE TO IT AT NIH AND I WOULD SAY, WEARABLE TECHNOLOGY, DOESN'T MEET THE MARK OF SCIENTIFIC RIGOR. THERE'S NO EVIDENCE BASE. THERE'S NO VALIDATION WE CAN GO TO. SO WE HAVE TO BE VERY CAREFUL. BUT I WOULD SAY, YOU KNOW, IN YOUR MODEL, JOHN, T4. HOW ARE WE GOING TO DO T4 WELL? WELL, SOME GROUPS ARE TRYING TO DO T4 WELL. HE COULD SPEAK TO IT. IN A COMMUNITY BASED A NATIONAL POPULATION OF COMMUNITY BASED PEOPLE, WITH O.A. THEY'RE USING TECHNOLOGY TO MONITOR THESE PEOPLE. NOT NECESSARILY THE STEP COUNT ON THE IPHONE. BECAUSE THAT'S NOT VERY ACCURATE ACCURATE. BUT FOR MEDICATION MANAGEMENT, FOR COMORBIDDITY DATA, HOSPITAL SITUATIONS, YOU KNOW, FOR DOCTORS VISITS AND SO FORTH, THEY'RE GETTING DATA IN REAL TIME ALL THE TIME, FROM THEMEDS OF PEOPLE, WALKING AMONGST US WITH EOA BECAUSE OF TECHNOLOGY. THE, THING I THOUGHT OF, AS PEOPLE WERE SPEAKING, YOU KNOW, THE FOX FOUNDATION, THEY HAVE GOT A NATIONAL EFFORT RIGHT NOW, A CLOSED-BASED SYSTEM TO COLLECT ALL KINDS OF INTERESTING DATA FROM COMMUNITY DWELLING PEOPLE WITH PARKINSON'S. NOW, HOW MUCH OF IT IS GOING TO BE USEFUL IN THE END, WE DON'T KNOW. BUT WE, I THINK, AS RESEARCHERS, NEED TO TRY OUR BEST TO BE ON THE FRONT END BUTT PROBLEM S WE'RE NOT ON THE FRONT END. THE DEVICE INDUSTRY IS ON THE FRONT END, WHETHER IT'S USEFUL OR NOT AND WE'RE RUNNING BEHIND, SAYING WHICH DEVICE WE SHOULD USE. IF I DON'T KNOW PUT A DEVICE IN MY GRANT. PEOPLE ARE GOING TO SAY, WHY AREN'T YOU USING A DEVICE, SO WE JUST NEED TO BE CAREFUL. >> AND I THINK THE OTHER THING THAT I THINK IS DEE, IS MOST OF THESE DEVICES WERE DEVELOPED FOR THE HEALTHY POPULATION. SO I THINK YOUR POINT ABOUT THE ACCELEROMETERS. I THINK IF WE'RE GOING TO VALIDATE THESE AND HAVE STANDARDS AND THAT'S A NEED THERE'S GOT TO BE WAYS OF WHAT THE STANDARDS NEEDS ARE. AND IT'S GOING TO GO BEYOND THAT. IF YOU'VE SEEN SOME OF THE STUFF COMING OUT WITH THE BIOTATTOOS. HE'S GOT THE [INDISCERNIBLE] FOR LOW BACK PAIN. [LAUGHTER] NO, BUT THERE ARE WEARABLE SENSE ORS THAT ARE TOO TATTOO LIKE, AND THEY WILL STALEST ON, FOR MULTIPLE WEEKS AND THEY CAN TRACK HEART RATE AND A WHOLE BUMP OF THINGS B. THEY ARE NOT A WATCH. THEY ARE LIKE, SOMETHING THAT GOES ON YOUR SKIN AND LOOKS KIND OF LIKE A TATTOO. THEY CAN BE ARTED UP, TOO. BUT I THINK THE TECHNOLOGY IS GOING TO ADVANCE FASTER THAN YOU CAN VALIDATE IT FOR EACH POPULATION. IF THERE ARE STANDARDS THAT CAN BE DONE, YOU KNOW, THERE ARE AGENCIES WE CAN WORK WITH TO TALK ABOUT THAT. >> LET ME COMMENT ON THAT. I THINK THE NOVELTY EFFECT, WHICH IS PRETTY COMMON WHEN SOMETHING NEW IS INTRODUCED, ACTUALLY WE AND I'M GOING TO SAY WE. THE NIH IS VERY VULNERABLE TO THAT. WHEN THE SMRICAME OUT, HAVE YOU TO DO IT IN ORDER TO GET ANYTHING FUNDED. I AM SITTING IN A STUDY SECTION. I SAID, WHAT DO THOSE COLORS MEAN? AND EVERYBODY WAS LIKE, DOESN'T MATTER. IT'S COOL. AND I THINK THOSE QUESTIONS ARE VALID. JUST BECAUSE IT'S NEW AND FANCY, DOESN'T MEAN WE SHOULD USE T. SO I THINK THE QUESTION OF VALIDATION IS CRITICAL. TECHNOLOGY AFFECTS BEHAVIOR F. YOU ARE LOOKING. THAT'S GOING TO EFFECT ON HOW SOMEBODY BEHAVES. REMINDS ME OF THE QUANTUM MIXTURE MENOMINI. YOU THROW -- IF YOU DON'T CHECK TO SEE WHICH SLIPS THE ELECTRON WENT INTO, YOU GET AN EMBARRASS PATTERN, BUT AS AS SOON AS CHECK TO SEE WHICH ONE OF THOSE TWO CHANNEL ITS WENT INTO, SUDDENLY T BEHAVES LIKE A PARTICLE. AND OF COURSE, THAT'S KIND MUCH A ROUND ABOUT WAY OF SAYING HAWTHORNE EFFECT. IF YOU'RE BEING LOOKED AT, YOU'RE GOING TO BEHAVE DIFFERENTLY. AND THE QUESTION IS DO WE REALLY HAVE AN UNDERSTANDING, ESPECIALLY UPON IF IT'S WEARABLE WHERE YOU ACTUAL VERY CLOSELY TO DO THE TASK OF WEARING. AND A LOT QUESTIONS LIKE IS THAT. THAT I THINK WE DO NEED TO ADDRESS BEFORE WE ADOPT THESE TECHNOLOGIST. >> SO I JUST WANT TO MAKE ONE POINT AND THAT IS, THAT I'M CONCERN WHEN WE TALK ABOUT ADHERENS AND COMPLIANCE AND MONITORING WHERE WE'RE NOT ENGAGING THE PATIENT OR THE PERSON. AND I THINK WE COULD TURN THIS AROUND AND HAVE THEM, I MEAN, TECHNOLOGY DOES, YOU KNOW, PEOPLE LIKE GETTING FEEDBACK. SO WE COULD USE THIS TO ACTUALLY GIVE THEM RESPONSIBILITY FOR REPORTING. GIVING THEM CHOICE IN THE MATTER. YOU KNOW, WHEN DO YOU WANT TO -- INSTEAD OF US MONITORING THEM IN A CONTROLLING WAY, WE COULD REALLY TURN IT AROUND. AND ACTUALLY LEVERAGE THE FACT THAT PEOPLE ENJOY, YOU KNOW, THE TECHNOLOGY AND THE FEEDBACK AND GIVE THEM CHOICE IN REPORTING THEIR DATA. WE HAVE TO BE CAREFUL WITH THIS, WE'RE MONITORING YOU, AND WHAT MONA SAID. I THINK WE CAN TURN AROUND AND REALLY BENEFIT FROM THAT PERSPECTIVE. >> I AGREE WITH THAT AND I THINK WE NEED TO BE THINKING. THE TECHNOLOGY, BEYOND INTERVENTIONS ARE WEARING BUT JUST THE TECHNOLOGY ITSELF, GIVES US OPPORTUNITIES TO FIND THAT LOW-HANGING FRUIT, DO LARGE, SIMPLE TRIALS. THINK ABOUT, WHAT ARE THE QUESTIONS IN IN FACT TO WHAT WE TALKED ABOUT GETTING THE COMMUNITY INVOLVED, WHAT ARE THE QUESTIONS INVOLVED AND WHAT'S THE SIMPLEST WAY WE CAN APPROACH THAT TO GET AN ANSWER. IT WON'T BE AS SEXY OR INNOVATIVE BUT IT CAN CHANGE PRACTICE IF WE CAN ANSWER SOME OF THE QUESTIONS. SOMETIMES WE DON'T KNOW WHAT THE QUESTIONS. WHEN WE DO, WE CAN OVER COMPLICATE THEM. SO WE CAN GET THROUGH STUDY SECTIONS. A LONG TIME AGO, THE HEART DISEASE WAS STUDIED PIE TENSE OF THOUSANDS OF PATIENTS AND HE LOOKS AT US IN THE STATES AND THINKS WE'RE CRAZY. I THINK TECHNOLOGY THAT WE HAVE NOW, ALLOWS US TO DO THIS IN LARGE POPULATIONS. YESTERDAY I MENTIONED THE TYPE 1 DIABETES. THESE ARE PATIENT COMMUNITIES. YOU CAN LAUNCH A STUDY WITH THEM, PROVIDING CONCEPT AND GET ANSWERS BACK IN TWO WEEKS AND IT CAN BE A TWO-WEEK TRIAL OF COMPLIANCE OR WHATEVER. SO I THINK IT'S GREAT OPPORTUNITIES FOR US. ONE THING ABOUT SMRE. THERE'S A GREAT PAPER ABOUT A DEAD SOLOMON WHEN, THEY DID AN SMRION, AND SOME OF THE [INDISCERNIBLE] LIT UP. SO I UNDERSTAND WHAT YOU'RE SAYING FOR THOSE WHO DON'T DO IMAGEOG A DALEY BASIS, THERE WAS A PAPER OUT THREE MONTHS AGO, THAT ARGUED, THAT 40,000 PAPERS CAN BE IN JEOPARDY, BECAUSE OF THE STATISTICAL ALGORITHMS THAT OF COURSE USED WERE WRONG, AND INSTEAD OF WORKING AT A 5% ERROR RATE, THEY WERE PROBABLY WORKING AT A 30% ERROR RATE. SO THAT HAS CAUSED A LOT OF CONSTERNATION AND HAS A LOT OF PEOPLE WORKING. THE KIND OF COMMENT I WANT TO MAKE ABOUT THE TECHNOLOGY. MARCOS MENTIONED THE MICHAEL J. FOX INITIATIVE AIR, TERRIFIC INITIATIVE. IT WILL BE WORTH FOLLOWING FOR MANY PEOPLE. I THINK ALONG WITH TECHNOLOGY, THE MORE DATA YOU'VE GOT, THE BETTER BECAUSE IT'S GENERALIZABLE AND THAT YOU'RE GOING TO GET A BIG ENOUGH SIGNAL TO DOMINATE ALL THAT NOISE AND WHEN YOU GIVE ALL THIS DATA TO A BUNCH OF REALLY GIFTED STATISTICIANS, AND PEOPLE WHO CAN DO MACHINE LEARNING, OUT WILL MONTH MAGICAL ANSWER. AND I AM VERY PESSIMISTIC ABOUT THAT. I THINK THOSE KIND OF STUDIES RUN INTO PROBLEMS. IF YOU DON'T DO THE CAP OPERATION. SO -- [PLEASE STAND BY] I JUST WANTED TO GO BACK TO THE ISSUE OF THE -- OUR LAY DONORS ARE ALWAYS INTERESTED IN TALKING TO THEM,A THAN TO TALKING TO ME. BUT I THINK THE ISSUE ABOUT COST IS A LITTLE BIT MORE COMPLICATED BECAUSE TYPICALLY, THE COST OF TECHNOLOGY COMES DOWN OVER TIME AND ALSO UNTIL WE GET INTO THE PHASABILITY OF RESEARCH. WE DON'T KNOW THE COST-BENEFIT TRADEOFF. SO MAYBE, A MORE QUOTE, UNQUOTE, PASSIVE DEVICE THAT KIND OF LEADS YOU THROUGH SOME ROUTINE IS MORE SUSTAINABLE THAN ONE THAT REQUIRED MORE EFFORT AND WE WOULD HAVE TO KEEP GIVING COACHING INGREDIENTS OVER TIME, TOO KEEP THE ONE GOING. SO I JUST I TOTALLY AGREE, WE NEED TO MAKE SURE THAT THE BENEFIT WE'RE GETTING IS WORTH THE COST. BUT I DON'T THINK THAT'S AN EARLY QUESTION TO ANSWER OR AN EASY QUESTION TO ANSWER. SO WE WERE TAKING THOSE SAMPLES, JUST IN THE CASE THAT, SOMETHING'S GOING TO WORK OUT. WE HAVE THE PARADIGMS AND SO THE RISK WE HAVE NOW, WE HAVE ALL THE SAMPLES IN THE FREEZERS. MUCH LIKE THE CONFIDENTIALITY PROBLEMS WITH THE DATA YOU'RE YOU'RE COLLECTING. THAT'S THE LEGACY WE HAVE. IT HASN'T QUITE PANNED OUT. AND ANOTHER PIECE OF THAT IS, LET'S NOT 94 GET ABOUT, BECAUSE SOMETHING HAPPENS TECHNOLOGY, AND YOU GUYS ARE ALL RESEARCH NERDS. THE BURDEN YOU'RE PUTTING ON YOUR THERAPISTS, AND THE PATIENTS ABOUT THROWING IN THERE, MAKES IT SO MUCH EASIER. IN A WAY, YOU'RE CREATATION WHOLE EXTRA LAYER TO EVERY STUDY THAT WASN'T THERE BEFORE. NOW WE'RE GOING TO GO ON TO SPEAKING OF GENOMICS. >> SO ONE OF THE THINGS ABOUT BEING AT NIH. MAYBE ONE CAN MAKE COMMENTS ON NIH. FOR ME, ONE OF THE HARDEST THINGS ABOUT WRITING A GRANT IS SEPARATING WHAT GOES IN SIGNIFICANCE FROM WHAT GOES IN INNOVATION. AND ONE CAN ASK ONE'S SELF, WHETHER THE PUSH FOR INNOVATION, WHICH IS VERY NICE ISN'T CAUSING US TO PUSH HARDER AND FASTER THAN REALLY, WE SHOULD. WHEN I GET TO THE PART ABOUT OMECS, JUST CLOSE YOUR EAR. WHEN WE GET TO THE PART ABOUT HIRESLATION IMAGING, CLOSE YOUR EARS. SO INTERINDIVIDUAL RESPONSE HETEROGENEITY. WE UNINTENTIONALLY, PUT THIS AT BACK END OF OUR MEETING BECAUSE WE HEARD ABOUT VERY PRACTICAL AND INNOVATIVE APPROACHES OF CLINICAL TRIALS AND WHERE THE FUTURE MIGHT LEAD US AND I THINK THIS TOPIC. SUMMER, SOMETHING I STUDY EVERY DAY, BUT THIS IS A TOPIC, I THINK, REGARDLESS OF WHERE YOU ARE ON THAT SPECULUM, FROM T1 ALL WAIT TO THE RIGHT, THIS NEEDS TO BE CONSIDERED. SO WHAT DO WE MEAN BY INTERINDIVIDUAL RESPONSE HETEROGENEITY F. YOU HAVE AN INTERVENTION TRIAL OF ANY SORT. WHEN IT'S ALL OVER W YOU'RE INTERESTED IN STATISTICICALLY AND CLINICALLY, WHETHER OR NOT THE MEAN CHANGED. AND SO WE ALL FOCUS ON THE MEAN. WE ALL WANT TO KNOW, DID WE REACH THE STATISTICAL SIGNIFICANCE, THRESHOLD AND DID WE REACH SOME CLINICALLY MEANINGFUL THRESHOLD ON THE MEAN. WHAT WE ALWAYS IGNORE, THE EXTREMES AND SO WE WANT TO TALK ABOUT THAT. I THINK IT'S SOMETHING THAT GETS OVERLOOKED. SO THE VALUE OF STUDYING T WE'LL TOUCH ON THAT. WHAT CAN WE LEARN FROM IT. WHAT ARE THE SOURCES OF T. STRATEGIES, BY WHICH WE CAN BEGIN TO UNDERSTAND IT A BIT BETTER. AND CAN WE USE IT TO OUR ADVANTAGE. HOW CAN WE DO THAT? WHERE CAN WE MAXIMIZE THE YIELD OF THE TRIAL. WE'LL TALK ABOUT ADVANCES IN TECHNOLOGY THAT, CAN BE APPLIED IN THIS SPACE AS WELL. AND RESOURCE SHARING IS WHERE WE'LL SORT OF END UP. ESSENTIAL LEAKER THE INTERVENTION IS THE SAME. THE PHENO TYPE IS THE INDIVIDUAL SYSTEM LARGE LITE SAME. AND THE GOAL IS TO TAKE PEOPLE WHO ARE CORRESPOND AND INCLUDE THEIR CARDIORESPIRATORY FIT. SO 192 INDIVIDUALS, NICE GENDER MIX. FAIRLY LARGE POPULATION. ALL HAVE UNDERGONE AN INTERVENTION. THEY HAVE ALL BEEN TESTED FOR CARDIORESPIRATORY FITNESS, BEFORE AND AFTER. IF WE WERE TO PUBLISH THIS, WE WOULD BE VERY TO BE ABLE TO SAY, WE HAVE A MEAN CHANGE THAT'S MEANINGFUL. IF WE LOOK AT MEAN DOWN THE MIDDLE, BUT WE IGNORE THE FOLKS TO THE LEFT AND TO THE RIGHT AND THOSE TO THE RIGHT, WHAT WE WOULD CALL THE HIGH OR THE EXTREME RESPONDERS AND THOSE TO THE LEFT WHAT WE WOULD CALL THE HIGHER LOW RESPONDER. SO WHY IS IT THAT IF YOU DELIVER THE SAME TREATMENT, YOU GET THIS WIDE ARRAY OF RESPONSIVENESS. AND I THINK WE NEED TO UNDERSTAND WHAT THAT IS. THIS IS A COMPILATION DATA FROM A COUPLE OF STUDY THAT IS PUBLISHED SEPARATELY, BUTT INTERVENTION IS THE SAME. >> CAN I ASK YOU A QUESTION ABOUT THAT? >> YEAH. >> IS THE PERCENT CHANGE? WOULD YOU GET THE SAME AMOUNT OF GENEITY IF [INDISCERNIBLE] RAS 4. >> YES. WE HAVE DONE IT BOTH WAYS. SAME DEAL. NOW, LET'S GO TIE SECOND EXAMPLE. SO WE NOW VAN EXAMPLE. I STUDY MUSCLE ATROPHY, INCLUDING RELATIVELY HEALTHY AGES BUT ALSO IN FOLKS WHO ARE SUFFERING FROM A CRIME DISEASE, ACCOMPANYING THEIR AGING. THESE ARE HEALTHY FOLKS. RELATIVELY HEALTHY, OTHER THAN THE FACT THAT THEY HAVE LOST A SIGNIFICANT AMOUNT OF MUSCLE MASS. SO AN INTERVENTION TO IMPROVE MUSCLE MASS AND SAT THE MUSCLE FIBER LEVELS, TYPE 2 FIRES, WITH AGING, SAME THING. HIGH INTENSITY INTERVENTION. VERY PROUD OF THE FACT THAT OUR MEAN IS A NICE ROBUST PERCENT INCREASE THIS HETEROGENEITY IS MEANINGFUL. AND ONCE YOU DEFINE THESE CLUSTERS OF INDIVIDUALS. WE DO THIS BY MEANS OF CLUSTER ANALYSIS. NOW, WE CAN REALLY INTERROGATE THIS GROUP. WHY ARE THEY NONRESPONDERS? WHAT CAN WE DO DIFFERENT? WHAT CAN WE DO TO UNDERSTAND, THESE ARE A MIX OF PEOPLE THERE'S A SLIGHT AGE MIX BUT THEY HAVE A COMMON PHENO TYPE. THAT'S THE BACKDROP AS TO WHY WE'RE TALKING ABOUT IN. ALL RIGHT. SO WHAT'S THE VALUE. SO IN AN INTERVENTION TRIAL WHOA FOCUS, ON THE MEAN CHANGE. WE WANT TON, DOES OUR PRIMARY OUTCOME HAVE A MEAN CHANGE AND IS IT STATISTICALLY SIGNIFICANT. WE WANT TO KNOW IF OUR SECONDARY OUTCOMES UNDER WENT SOME STATISTICALLY SIGNIFICANT CHANGE. BUT WE OVERLOOKED THAT VARIANCE. SO WE THINK, AT LEAST I DO, THIS CAN REVEAL REALLY IMPORTANT PREDICTORS OF RESPONSIVENESS AND I THINK MORE IMPORTANTLY, CAN CHANGE OUR APPROACH. SO I'M NOT READY TO GIVE UP ON THE LOW RESPONSE. WE GAVE THEM THE WRONG TREATMENT. SO IF WE CAN FIGURITY WHO THEY ARE AND WE CAN PHENO TYPE THEM, WE CAN ANY BACK WITH A MORE PRECISE TREATMENT. AND SO THAT'S REALLY THE GOAL. THIS IS, YOU KNOW, WE HAVE ALL HEARD THE TERM PERSONALIZED MEDICINE, WHICH HAS NOW BECOME PRECISION MEDICINE AND OF COURSE, IT ALSO APPLIES TO REHABILITATION MEDICINE AND THIS IS HOW YOU GET THERE. YOU WANT TO KNOW WHAT MAKES PEOPLE TICK. SO THIS IS COMPLICATED. THIS IS MY GRAPHIC OF EVERYTHING THAT I CAN COME UP WITH THAT INFLUENCES RESPONSIVE HETEROGENEITY. ALL OF US TRUE TO REEXECUTE A PARTICULAR DISEASE STATE AND STUDYSIDE IT AND TRY TO IMPROVE ON IT AND HOPEFULLY, WE'RE COLLECTING A LOT OF DATA THAT ALLOW US TO GO BACKWARD, AND FIGURE OUT, WHY DID SOME PEOPLE GET THE OUTCOME AND SOME OTHERS DID NOT. LOTS OF INPUTS THAT CAN INFLUENCE THAT. WE JUST TALKED ABOUT TECHNOLOGY I HAVE A CIRCUMSCRIBED INTERVENTION. BUT WHAT ABOUT OVER HERE? WHAT ARE THEY DOING IN THEIR FREE TIME? DOES THAT MATTER? DO WE HAVE TO MONITOR THAT. THAT CAN HAVE AN INFLUENCE. SO THIS IS A COMPLEX SLIDE. I'M GOING TO REVISIT THIS AND GIVE YOU ONE EXAMPLE OF HOW WE HAVE USED THAT TO BREAK IT DOWN AND TRY TO FIGURE THIS OUT. ADDITIONAL SOURCE THAT IS DIDN'T MAKE THEIR WAY ONTO THAT MODEL. BECAUSE THAT MODEL DIDN'T HAVE ANY ROOM ON IT. THESE ARE IMPORTANT FOR FOLKS IN THIS ROOM. IN MEDICAL REHABILITATION. SPECIFICS OF THE CONDITION. ARE WE TALKING ABOUT A TRAUMATIC INJURY, IF WE ARE, WHAT'S THE SITE OF INJURY? WHAT'S THE DIAGNOSIS? ARE WE TAKING ALL COMERS OR DOES IT MATTER, THE SEVERITY AND THE SITE OF INJURY. DOES IT MATTER HOW LONG AGO THE INJURY OCCURRED. WE HAVE ALL EXPERIENCED THIS WHERE YOU'RE DEALING WITH A POPULATION THAT'S DIFFICULT TO RECRUIT ANY WAY. AND SO THE TRIAL BECOMES MORE OPEN THAN WE: SO YOU END UP WITH PEOPLE ACROSS A WIDE RANGE OF HOW LONG WAS THEIR S.S.C.I. TWO YEARS AGO, 20 YEARS AGO. THEY'RE ALL THROWN IN THE SAME DATA S. CRIME DISEASE. WHAT STAGE ARE THEY IN DANIEL DOES PARKINSON'S RESEARCH. STAGE OF DISEASE REALLY, REALLY MATTERS. SOMETIMES YOU PICK UP A PAPER, AND READ ABOUT SOME INTERVENTION IN PARKINSON'S DISEASE, AND POSTSURGICAL REHABILITATION, WHICH I THINK IS A GREAT OPPORTUNITY FOR THE FIELD, AND I DON'T THINK IT'S SOMETHING THAT IS WIDELY STUDIED. YOU CAN APPLY THIS TO ORGAN STRANDS PLANT AND SO FORTH. YOU CAN GO DOWN THE LIST. WHAT'S THE MODE OF SURGERY? WHAT STRUCTURES ARE AFFECT BY THE THAT. A PERFECT EXAMPLE, WE HAVE A REHABILITATION CLINICAL TRIAL IN [INDISCERNIBLE] ARCHO PLASTY, I'VE GOT THREE SURGEONS ON THE TEAM. THOSE THREE SURGEONS, WHEN THEY GO DO HIPS. ONE OF THEM DOES A POSTERIOR APPROACH, ONE DOES ANTERIOR APPROACH. I'M NOT GOING TO CHANGE THAT. I JUST WANT A PIECE OF THE MUSCLE, I WANT A BLOOD SIMPLE, GET THEM IN THE INTERVENTION OR THE USUAL CARE GROUP. BUT IN THE END. ARE WE GOING TO HAVE A BIAS IN OUR DATA? HOW MANY OF THEM ARE POSTERIOR, HIM ANTERIOR. SO THESE DETAILS, I THINK CAN BECOME REALLY IMPORTANT. HOPEFULLY, THAT WON'T IMPACT IN OUR MODEL. BUT CERTAINLY, IT COULD. SO DR. CHAP, PRODUCE THIS PAPER. I THINK IT'S A REALLY, REALLY INTERESTING STUDY. FROM THE CASE WESTERN GROUP. I JUST WANT TO HIGHLIGHT T. THIS IS HOT HOT OFF THE PRESS AND IT'S A GREAT EXAMPLE OF A STUDY THAT LEADS TO AN IMPORTANT CLINICALLY RELEVANT OUTCOME, BUT ALSO, REVEALS SOME INTERINDIVIDUAL RESPONSE. SO TWO DIFFERENT INTERCEPTIONS. POST STROKE SO I'M JUST GOING TO SUMMARIZE IT AS BEST I CAN. SO THE INTERVENTIONS, 24 TREATMENT, CONTRALATERALLY CONTROLLED ELECTRICAL STIMULATION, VERSUS MORE OF A STANDARD NEUROMUSCULAR ELECTRICAL SIM LAYING MODEL. THIS IS TO IMPROVE THE HAND DEXTERITY. I DON'T KNOW A LOT ABOUT IT, SO I COPIED THIS FROM THE PAPER. SO MANY OF YOU PROBABLY DO. SO ESSENTIAL LEAKER IT'S A MOTOR CONTROL TASK, OPENING AND CLOSING THE -- ESSENTIALLY, IT'S A MOTOR CONTROL TASK. GRASP AN OBJECT. MOVE IT OVER A SMALL PARTITION AND DROP THE OBJECT. HOW MANY TIMES CAN YOU DO T. SO VERY FUNCTIONALLY RELEVANT TASK. THAT'S THE PRIMARY OUTCOME. THEY DID A REALLY NICE JOB OF USING A MINIMUMIZATION MODEL, RATHER THAN A RANDOMIZATION MODEL. BUT THE IDEA WAS, WE DON'T WANT ANY BIAS, SO LET'S MAKE SURE THEY'RE EQUALLY DISTRIBUTED ACROSS THE TWO TREATMENT ARMS. SO THAT'S THE MODEL. SO WHAT HAPPENED. THE HYPOTHESIS WAS THAT THIS CONTRALATERAL STIMULATION MODEL IN WHICH THEY TY TRADED THE INTENSITY OF THE STIMULATION, BASED UPON, WHEN THE CONTRALATERAL EXDANCERS WOULD BE STIMULATED ENOUGH, THAT THE CONTRALATERAL HAND WOULD OPEN UP. IS THAT ACCURATE? >> [SPEAKING AWAY FROM MICROPHONE] >> RIGHT. BUT MONITORING WHAT'S HAPPENING ON THE CONTRALATERAL SIDE IS SORT OF A MEASURE OF WHAT IS NEEDED, PERHAPS. YEAH. THIS IS THE PARADIGM, THE CONTRALATERAL HEMISPHERE. SO WE HAVE THE UPAFFECTED LIMB THAT SENDS INFORMATION INTO A STIMULATOR AND PROVIDE ACE PATTERN STIMULATION TO THE EFFECTIVE LIMB. SO AS THE UNAFFECTED LIMB OPENS, THE EFFECTIVE HAND ALSO OPENS AND THEN WE DO TASK PRACTICE WITH THIS APPROACH. SO A VERY FUNCTIONING REL VAN. IN CONTRAST TO THE CONTROL GROUP, CYCLIC SIM LAYING, WHERE IT OPENS AND CLOSES WITHOUT ANY COGNITIVE ENGAGEMENT. THOSE ARE THE TWO MODELS. THIS GENERALLED IN THE CONTRALATERAL SIDE IS BETTER OTHIS BB T-TEST AND PARTICULARLY BETTER OVER TIME. THER VENTION, IS THE FIRST 12 WEEKS, AND AFTER THE FIRST 12 WEEKS, THEY CONTINUED TO IMPROVE. NOW, WHAT I PUT IN AROUND HERE WAS A BOX. THERE'S SOME VARIABILITY. WE ALL HAVE VARIABILITY IN OUR DATA SO EXPLORING THE VARIABILITY IS WHERE THIS THING, I THINK GETS INTERESTING. SO HERE ARE THE DATA FOR THAT SAME OUTCOME MEASURE. I WANT TO POINT OUT TWO THINGS. I SORT OF ENCIRCLED DATA SETS THAT I THINK ARE REALLY IMPORTANT. THE FIRST ONE IN BLUE IS THE INTERINDIVIDUAL RESPONSE HETEROGENEITY. 19 RESPONDERS, REALLY, REALLY ROBUST RESPONDERS, AND THOSE THAT ARE IN THE MIDDLE. REALLY NICE SPREAD OF DATA, THE SAME KIND OF DATA WE ALL GET. OVER HERE, FOR THOSE INDIVIDUALS WHO ARE MORE THAN TWO YEARS POST-STROKE, THEY HAVE GOT NO BENEFIT. SO THAT 6 MONTH TO TWO-YEAR WINDOW, THERE'S SOME NICE RESPONSE HETEROGENEITY, THAT CAN BE PROBED AND STUDIED. AND THE OTHER THING THIS REVEALS, AFTER TWO YEARS, VERY LITTLE BENEFIT. SO WHEN YOU BREAK IT DOWN EVEN FURTHER. THIS, TOO ME IS JUST A VERY NICE SORT OF PROGRESSION TO UNDERSTAND THESE DIFFERENCES. WHAT IS THE IMPACT OF DURATION SINCE THE STROKE? WELL, THE PREVIOUS SLIDE JUST SAID, THE REAL MALUPONNUAL TODAY PhD IS THIS FIRST -- MALLEABLE PERIOD IS THE FIRST 16 MONTHS. POST STROKE. THESE ARE THE INDIVIDUALS THAT ARE LESS THAN TWO YEARS POST STROKE. NOW, GRADE BEING THE SEVERITY, A MODERATE VERSUS SEVERE STATE OF THE IMPAIRMENT, IF YOU LOOK AT THE MOD SERGEANT THOSE THAT HAVE MODERATE IMPAIRMENT, DO MUCH BETTER WITH THE INTERVENTION, THAN THOSE THAT ARE QUITE SEVERE. SO THE INTERVENTION IMPACT, THOSE WHO ARE MODERATE IN IMPAIRMENT, AND IT IMPACTED THOSE WHO ARE LESS THAN TWO YEARS POST STROKE. THAT'S THE GROUP THAT BENEFITED THE MOST. SO IT WAS A VERY NICE, I THINK, BREAKDOWN. BUT MY QUESTION IS, WHAT ARE SOME OTHER POTENTIAL PREDICTORS. THESE ARE THE TWO THAT CAME OUT QUITE NICELY. SOME OTHER STRATEGIES IF WE WANT TO EXPLORE HETEROGENEITY. CLUSTERING REQUIRES PRETTY MUCH LARGER DATAA SET. IT ALLOW US TO -- PROBE OUTCOME MEASURES AND YOU CAN PROBE ALL THE OTHER POTENTIAL INFLUENCE FACTORS THAT MIGHT RESULT IN THE CATEGORIZATION, WE CAN LOOK FOR COMMON PHENO CHARACTERISTICS. LARGE PROBING THROUGH SOME TECHNOLOGIST, WHETHER THEY BE VERY TARGETED. MAYBE SOMETHING VERY SPECIFIC, A CLUSTER OF GENIUS OR SIGNALING PATHWAYS OR WHATEVER IT MIGHT BE. MAYBE IT'S BEHAVIOR OR ENVIRONMENTAL PHENO TYPING. MAYBE WE WANT TO SEE IF THEIR FREE LIVING TIME IS SPENT MUCH DIFFERENTLY NOT THAT OTHER GROUPS. THESE ARE WAYS THAT WE CAN REALLY PROBE IT. AND SPECIFICS OF THE CONDITION. SO IT REALLY BOILS DOWN TO HOW HOMEOGENEOUS IS THE GROUP ON THE FRONT END. SO BACK TO OUR EXAMPLE, THIS IS THE EXAMPLE OF ATROPHY FOLKS. LOSS MUSCLE AT THE LEVEL OF CELLULAR MEASURE OF TYPE 2 MUSCLE FIBER SIDE. I JUST WANT TO BREAK THIS DOWN FUR. YOU'VE GOT THREE CLUSTERS THE. SO GOING BACK HERE, JUST A QUICK SNIPPET OF PROBING THIS. SYSTEM THOSE THINGS THAT ARE IN RED, ASTERISK THAT ARE RED, WE DID OUR LEVEL ON THE FRONT END TO CONTROL THOSE THINGS. THOSE THINGS IN APPLY ARE THE THINGS WE PROBED AFTERWARDS. SO I'LL JUST QUICKLY BREAK IT DOWN FUR. THESE ARE THE CONTROLS, SO THE DATA ARE A COMBINATION OF TWO DIFFERENT, YOU KNOW, BY DEFINITION, EFFICACY TRIALS. WE TRY TO CONTROL EVERYTHING. WE SUPERVISE ALL OF THE EXPOSURE TO THE INTERVENTION THAT'S GOING TO INCREASE MUSCLE SIZE AND TRY TO CONTROL EVERY VARIABLE. WHAT'S CONTROLLED. OUR PRIORITY, ALL THESE VARIABLES AS BEST WE CAN. COMORBIDITY IS VERY STRICT. SO WE TRY TO GET A COMMON PHENO TYPE. HOW MUCH TRAINING HISTORY, PARTICULARLY IN THIS CHARACTERS IT'S STRENGTH TRAINING. WHAT'S THEIR HISTORY? THEY'RE NOT ALLOWED TO HAVE A HISTORY. SO THEY'RE ALL LUMPED AT NAIVE AND NOT BEEN EXPOSED TO THIS BEFORE. AND I DON'T HAVE AGE UP HERE, BUT THESE ARE 60 TO 75 YEARS OLD, AVERAGE AGE, 65, 66. MODALITY AND PRESCRIPTION DOSE, NOT ALLOWED TO VARY. EVERYBODY'S THE SAME. THIS REALLY, REALLY MATTERS TO IS IF WE WANT TO STUDY THESE OUTCOMES. SO WE'RE VERY STRICT ABOUT T. WE HAVE IRB-APPROVED CRITERIA FOR ADMINISTRATIVE WITHDRAWL OF INDIVIDUALS WHO DON'T MEET THESE STANDARDS. SO IN THE END, WE GET ABOUT 90% ADHERENS TO ATTEND ALL THE SESSIONS, AND WITHIN THE SESSIONS, WE'RE VERY STRICT ABOUT MAKING SURE THAT EVERYBODY IS EXPOSED TO THE SAME INTENSITY AND SAME DOSE AND SO FORTH. SO NONE OF THOSE ARE DIFFERENT. THEN YOU GO DOWN. OKAY. WHAT ARE THE POTENTIAL THINGS. GENDER. NOT REALLY. BOTH GENDERS ARE WELL REPRESENT IN ALL THREE CLUSTERS. THAT'S NOT IT. DISEASE STAGE. NOT REALLY. ONE OF THE THINGS WE THOUGHT OF IS WELL, MAYBE THE PEOPLE WITH MORE MUSCLE ATROPHY, IT COULD HAVE GONE EITHER WAY. EITHER MUCH MORE MORE OR LESS RESPONSIVE. BUT AT THE END OF THE DAY, THEY ALL START WITH THE SAME MUSCLE MASS, THE SAME CELLULAR CHARACTERISTICS AS WELL. AT THE FIBER LEVEL. SO THAT'S NOT IT. DIET. OBVIOUSLY, ACRITICAL INFLUENCE ON ALL OF OUR BODY. THIS IS IN PARTICULAR, MUSCLE MASS. SO WE BREAK THE DIET DOWN IN EXCRUCIATING DETAIL. WE'LL GET THERE. SO WE DIG INTO THE MUSCLE A BIT. WHAT WE FIND IS, TRANSCRIPT ONLY PROFILE IS FAR AND AWAY, DIFFERENT. THOUSANDS OF GENIUS, DIFFERENTLY EXPRESSED. AND THE INTERESTING THING IS MEN AND WOMEN IN THE RESPONSE CLUSTERS ARE CLUSTERING TOGETHER IN THEIR TRANSPORT. AND THIS IS PRIOR TO TREATMENT. HOW MANY STEM CELLS THEY HAVE, HOW EASILY THEY'RE TURNED O. MUCH DIFFERENT. GENE EXPRESSION SIGNALING SAME THING. SO LOTS OF THINGS HAPPENING THAT WE FOUND TO BE DIFFERENTIALLY REGULATED WITH THE PROBE -- THIS IS JUST AN EXAMPLE. OF DOING A TRANSCRIPTO WIDE. THESE ARE THE EXTREME RESPONDERS. WHAT PATHWAYS AND NETWORKS POP OUT ON THEM? THIS IS UPON BASELINE TISSUE. THIS IS PRIOR TO ANY TREATMENT. THE NONRESPONDERS. WHAT THEY UPREGULATING IN THEIR MUSCLE. DISEASE-RELATED GENE NETWORKS AND SO FORTH. SO SUPPOSEDLY, THEY'RE HEALTHY B YOU WALKING IN WITH A PROFILE THATS, I'M NOT GOING TO RESPOND VERY WELL. WE ARE THINKING, IF WE HAVE A DATA SET WE CAN RELY ON. TO POWER A TRIAL ON THIS. THIS WE ALL POWER THE TRIAL ON THE EFFECT SIZE ASSOCIATED WITH THE MEANS CHANGE OR POWER THE TRIAL BASED ON PREDICTED NUMBER OF NONRESPOND EAND EXTREME RESPONDERS AND SO FORTH. IN THE END, IF YOU'RE GOING TO PROBE THIS AND TRY TO BECOME MORE PRECISE, YOU'RE GOING TO HAVE TO HAVE A SUFFICIENT NUMBER OF PEOPLE WHO CATEGORIZE. AND JUST WAYS OF MAXIMIZING YIELD FOR SURE. REGENERATIVE, THE HOT BUZZ WORD. CREATE TISSUE THAT IS REPLY, REPAIR, RESTORE, AND WHAT I WANT TO SAY TO YOU, WE OUGHT TO BE USING THAT TERMINOLOGY HERE. WHY ARE WE USING THAT TERMINOLOGY HERE? SOUNDS LIKE REHAB TO ME. IF YOU'RE GOING TO REPAIR SOMETHING, REPLACE SOMETHING, RESTORE A FUNCTION. DUE TO A CHRONIC DISEASE, A CONGENITAL DEFECT AGING AND SO FORTH. YOU'RE ALL DOING REGENERATIVE REHABILITATION. YOU'RE JUST NOT STUDYING THE MOLECULAR MECHANISM, BUT YOU'RE DOING REGENERATIVE REHABILITATION. I THINK WE HAVE A HUGE OPPORTUNITY TO PURSUE THAT. SHAUN, YOU WANT TO TALK ABOUT THESE VARIOUS APPROACHES. WE WENT THROUGH AND JUST SAID, WE COULD IDENTIFY A LOT OF APPROACHES THAT COULD BE PROBED OR USED. MY FIRST PLEA, AND WE HAVE HEARD THIS, WHY ARE WE BANKING ALL THESE SPECIMENS AND NOT USING THEM? L WE SHOULD BE USING THEM F. YOU'RE GOING TO BE DOING A LARGE SCALE TRIAL, AND NOT BANK -- BANKING, THAT'S A LOST OPPORTUNITY. INTERDISCIPLINARY TIMES IS AN IMPORTANT NEXT DIRECTION. MANY GROUPS ARE ALREADY DOING T BUT THAT'S HOW YOU GET IT DONE. LOTS OF THINGS THAT CAN BE COLLECTED CAN BE VERY USEFUL. LOTS OF APPROACHES TO STUDY THOSE BIOSPECIMENS, AND INCLUDING EVEN THOUGH WE JUST HEARD SPECIAL DISCUSSION ABOUT IMAGING, THERE'S A LOT OF REALLY, REALLY HIGH FIDELITY IMAGING THAT CAN TELL US A LOT. WE HAVE ONE SET, AND I'LL QUICKLY JUST GO TO THAT. AND I'LL JUMP BACK INTO THAT IN JUST A SECOND. THERE ARE SO MANY DIFFERENT WAYS WE COULD UNDERSTAND HETEROGENEITY. ONE THING, I WAS REALLY EXCITE REQUESTED MARCUS ASKED ME IF I COULD HELP WITH THIS TALK, WAS REALLY WITH REGARDS TO IMAGING, AND YOU KNOW, SO BECAUSE WE HAVE T AND WE HAVE IT AVAILABLE, IN VARIOUS DIFFERENT WAYS AND WHILE IT'S BEEN BROUGHT UP ABOUT SOME OF THE LIMITATIONS OF T I'M WONDERING I DON'T KNOW IF WE CAN GET THE LIGHTS OFF TO SHOW THIS. L YOU'VE GOT THOSE OFF. ON THE OTHER SIDE. IF EVERYBODY LOOKS AT THE SIDE WALL, IT'S ACTUALLY NICE. ALL RIGHT. I DON'T KNOW IF THE LASER POINTER WILL WORK. MAYBE IT WILL. OH, WELL. NO. IT DOESN'T. SO WHAT WE HAVE BEEN, SORRY. THE FOLKS IN THE BACK CAN'T SEE T. OH. YOU CAN. GREAT. WONDERFUL. MANY GROUPS, ARE STUDYING LOOKING AT, FOR EXAMPLE, LIGHT ATTRACT MILLIONING, WITH PATIENTS WHO HAVE STROKE, OTHER NEUROLOGICAL DISORDER AND IT IS TECHNOLOGY HAS EINVOLVED -- OKAY. GREAT. NOW IT WORKS AND TECHNOLOGY HAS COME TO THE POINT WHERE WE COULD DO A LOT MORE SOPHISTICATED IMAGES AND THIS IS NOT FUNCTIONAL IMAGING, THIS IS STRUCTURAL IMAGING TAKING CARE OF PATIENTS WITH STROKE, WORKING WITH PATIENTS, FOR EXAMPLE, TRY TO IMPROVE THEIR MOTOR IMPAIRMENTS. WE HAVE THE OPPORTUNITY HERE, IN THINKING ABOUT PATIENTS OVER A LONGITUDINAL PERIOD OF TIME. THIS IS A PATIENT, GOING ANYWHERE FROM THE FIRST MONTH, ALL THE WAY OUT TO TWO YEARS. AND THE STROKE IS THIS GREEN AREA, AND THESE ARE THE CORTICOSPINAL TRACT OUTTOPS LATERAL SIDE AND CONTRALATERAL SIDE. WHAT HAPPEN SYSTEM ONE OF THE MOST IMPORTANT TRACKS, WHICH IS MOTOR CONTROL, OF THE LIMBS, ARMS AND LEGS A THICKNESS OF THE CORTICOSPINAL TRACTS, AND YOU CAN CAN SEE OVER TIME, IT LOOKS TO US AND IT LOOKS LIKE THE THICKNESS OF THE CST HAS INCREASED, AND THIS IS A PATIENT WHO'S MOTOR IMPAIRMENT DID IMPROVE OVER TIME. YOU HAVE THE OPPORTUNITY TO CHART THE COURSE OF PATIENTS OF WHAT'S HAPPENING STRUCTURALLY WENT BRAIN. I JUST WANTED TO ADD THIS INTO MARCUS' TALK OVERALL, THIS IS ANOTHER WAY WE COULD PROBE OR INTELLIGENT THE HETEROGENEITY N TERMS OF RESPONSIVE'S RESPONSE TO THERAPY -- IN TERMS OF THE PATIENT'S RESPONSE TO THERAPY. >> OKAY. THANK YOU. EVERYBODY STILL AWAKE? I FOUND THAT TO BE ABSOLUTELY FASCINATING WHEN SHAUN WAS FIRST SHOWING ME THOSE IMAGES I'VE ALREADY GIVEN WATT THUNDER BECAUSE THE ANIMATION HAS ALREADY HAPPENED, UNFORTUNATELY. BUT ALL RIGHT. SO WE TALKED A LITTLE BIT ABOUT APPROACHES AND WHAT I WANT TO SAY IS THIS IS A PLEA. FIGHTING OPPORTUNITIES, AS WELL AS PEER REVIEW: WE, I THINK AS A COMMUNITY, NEED TO DO A BETTER JOB OF EMBRACING EXPLORATORY AIM. WHY DO WE GET CRITICIZED FOR HAVING EXPLORATORY AIM? PRIMARY OUTCOMES ARE FANTASTIC. CLINICALLY RELEVANT TRIAL DESIGNS ARE REALLY, REALLY IMPORTANT. BUT WHY NOT HAVE SOME EXPLORATORY OPPORTUNITY? AND I THINK INVESTIGATORS SHOULD EMBRACE THE CHANCE, YOU KNOW, YOUR EXPLORATORY AIM MIGHT BE CUT FROM THE GROUND. MIGHT BE OTHER WAYS OF SUPPORTING THAT EXPLORATORY APPROACH AND JUST AS A QUICK PLUG, YOU KNOW, THROUGH REACT, YOU COULD SAY HEY, GOT AN OPPORTUNITY HERE. NEED SOME MONEY. WANT TO GET? BLOOD SPECIMENS. I'VE GOT A FUNDED TRIAL. STUDY SESSION CUT THAT AIM. SO AGAIN, WE'LL VISIT THAT AGAIN, WE'RE ABOUT TO WRAP UP. LAST COUPLE OF THINGS TO SAY. RESOURCE SHARING THIS IS A REALLY, REALLY HOT ITEM AT NIH AND SHOULD BE. WE SHOULD ALL DO A JOB OF MAXIMIZING YIELD FROM THE TRIALS. CENTRALIZE PUBLIC DATA WAREHOUSING. I THINK THE GAME IS GOING TO STEP UP MORE N TERMS OF ACCESS AND SPECIMENS, WE NEED FUNDING MECHANISMS TO TAP INTO AND I LIKE THIS TERM, GARY CAME UP W. RESEARCH MATCH FOR SPECIMENS AND DATA WE HAVE RESEARCH MATCH WHERE WE -- FOR PARTICIPATES. WE SHOULD HAVE RESEARCH MATCH TO LOG IN AND SEE WHAT I'M LOOKING FOR. AND IS IT OUT THERE. THAT'S AN OVERVIEW. YOU HEARD ABOUT THE ONE ABOUT REACTING. WHEN WE TALK ABOUT REGENERATIVE REHABILITATION. I JUST WANT TO POINT YOU TO THE GROUP IN PITTSBURGH AND THAT'S THE FOCUS OF THEIR BTC WITHIN THE NETWORK. BUT ALL OF THESE SITES WOULD BE MORE THAN HAPPY TO WORK WITH YOU, OVERSEAS PECKS OF INTEGRATING SORT OF TECHNOLOGY AND BEGINNING TO EXPLORE THIS HETEROGENEITY. SO I'LL STOP WITH THAT. >> NICHD. IF YOU HAVE FUNDING THROUGH NICHD AND YOU ARE REQUIRED TO SHARE YOUR DATA, WE DO HAVE A MECHANISM BY WHICH TO DO THAT. IT'S CALLED DASH O.NHDIC WEBSITE. AND THERE ARE MONEYS AVAILABLE, TOO SUPPORT YOU GETTING YOUR DATA INTO DASH. WE HAVE ONE REHABILITATION TRIAL UPLOADED WE'RE WORKING ON A COUPLE OF MORE. WE ALSO DID, WE'LL BE THINKING ABOUT, AND WE DID THIS PAST YEAR, SOME -- BECAUSE YOU CAN CREATE THE DATABASES. NOBODY GIVE US OUT MONEY TO ANALYZE THEM, NOBODY USES THEM. LED THE INITIATIVE TO DO AN INITIATIVE ON TRAUMATIC BRAIN INJURY TO LET OUR PEOPLE DO 21'S TO, ANALYZE EXISTING DATA IT WAS INCREDIBLELY WELL-RECEIVED AND THING HAS SHOWN US THE NEED FOR MECHANISMS. WE'RE GOING TO GET SOME VERY INTERESTING QUESTIONS ANSWERED. SO I THINK IF THOSE ARE THE TYPES OF THINGS YOU POINTED OUT, THAT MAY HELP DRIVE SOME OF THESE HYPOTHESIS, I THINK THAT'S A POTENTIAL AS WE MOVE FORWARD AND THERE'S LARGER AMOUNTS OF DATA AVAILABLE. >> BACK TO YOUR COMMENT ABOUT RESEARCH MATCH. I THINK NINDS CO-FUNDS, WITH THE NATIONAL, IT'S THE NATIONAL NEUROAIDS TISSUE CONSORTIUM, DOING JUST THAT, WHERE THEY HAVE BANK OF BRAIN TISSUE FROM AIDS PATIENTS AND RESEARCHER CONSIST APPLY. AND THEY CAN RESEARCH WHAT PIECES OF BRAIN TISSUE ARE AVAILABLE, IF THEIR RESEARCH IS APPROVED, YOU KNOW, BY A SEPARATE COMMITTEE, THEN THOSE ARE AUTHORIZED TO BE SENT TO THOSE RESEARCHERS FROM THE FOUR BRAIN BANKS, AND AGAIN, LEVERAGING TECHNOLOGY TO OPEN UP ACCESS TO OUR INCREDIBLE RESEARCH RICH RESOURCES. ONE OF THE REQUIREMENTS IS THAT ONCE THEY HAVE DONE WHATEVER THEIR RESEARCH S THEY'RE REQUIRED TO POST THE PAPER. IF THEY ARE NOT POSTING THEIR PAPERS THEY'RE NOT GOING TO BE GET ANYMORE TISSUE, EITHER. >> WE MAY BE TALKING ABOUT THE SAME THING. THE BEST EXAMPLE OF IN RESOURCE SHARING IS CNIX, A NATIONAL EFFORT IT'S IN HIV AND AID. HIV UPON IT'S A DATABASE OF CLINICAL DATA SO EMR DATA, BIOSPESMAN BANKING, AND THEY HAVE THOUSANDS UPON THOUSANDS OF PATIENTS IN THE DATABASE AND IT'S THE SAME THING. YOU CAN APPLY TO THE DATABASE. YOU KNOW, SAY WHAT YOUR QUESTION IS, HOW YOUR QUESTION CAN BE ADDRESSED. AND IF YOU'RE APPROVED. YOU GET THE DATA, YOU GET THE SPECIMENS. THEY HAVE SALIVA, BLOOD, ET CETERA. THAT'S MANAGED BY MIKE SAGA AT UAB. IT'S AN 8-SITE NETWORK AMONG C CFAR. CENTERS FOR AID RESEARCH. >> I WANTED TO ASK A COUPLE OF METH LOGIC QUESTIONS IN THE AREA. AND THE QUESTIONS WILL SOUND SORT OF STRAIGHTFORWARD. I THINK THE ANSWERS ARE PROBABLY NOT SO STRAIGHTFORWARD. ONE HAS TO DO WITH THE CART AND HORSE PHASE. ONE STRATEGY IS LET ME CROSS MY FINGERS AND HOPE I GET A MEMBERSHIP EFFECT, AND START LOOKING FOR HETEROJEN 8ITY F. YOU DON'T GET A MAIN EFFECTS. YOU ARE GOING TO HAVE A HARD TIME LOOKING FOR THE HETEROGENEITY. THE OTHER APPROACH IS TO TRY TO THINK VERY APRIORI UP FRONT, ABOUT WHO SHOULD BE RESPONDERS AND PICK A VERY NARROW POPULATION AND THEN SORT OF BEGIN TO MOVE OUT AND I STRUGGLE WITH THIS BECAUSE THEY'RE BOTH IN PRINCIPAL, REASONABLE STRATEGIES BUT AT EARLIER PHASES OF RESEARCH. WHEN STUDIES TEND TO BE SMALL, THEN YOUR ABILITY TO LOOK FOR THE SUBGROUPS IS MORE CHALLENGED. THE OTHER QUESTION I HAVE IS THAT, YOU KNOW, YOU KEEP REFERRING TO CLUSTERS IT LOOKED LIKE A CONTINUOUS DISTRIBUTION. SO IF WE FIND MULTI MODAL RESPONSES TO TREATMENT, VERSUS A CONTINUOUS DIMENSION OF LESS TO MORE, DOES THAT SEND US IN DIFFERENT DIRECTIONS, FROM TERMS OF WHERE TO LOOK. BECAUSE I THINK I'VE SEEN BOTH KINDS OF PATTERNS, JUST TO GIVE YOU AN EXAMPLE OF THIS METHOD LOGIC. I'VE DONE A COUPLE OF STUDIES ON ZOLPIDIMOR. AMBIEP, WHICH WAKES PEOPLE UP FROM A CERTAINLY AT A STATIVE STATE. BUT IT WAYS UP ABOUT 5% OF THE PEOPLE. YOU CAN IMAGINE IF I HAD STARTED WITH A GROUP, RCT. I WOULD HAVE SAID, DRUG DOES SNOT WORK, END OF STORY. I HAVE A DIFFICULT PROBLEM FINDING THE 5%. SO THAT KIND OF ILLUSTRATES. IF THE RESPONDING GROUP IS SMALL. NOT IF YOU'RE SCREWED IF YOU GO IN THE MAIN AFFECT. >> ALL GOOD COMMENTS. I WANT TO COMMENT ON THE CLUSTERING. I THINK FROM A PRACTICAL STANDPOINT CLEARLY, YOU DID SEE THAT THOSE CHANGE WERE RELATIVELY, YOU KNOW, YOU COULD ARGUE WHERE THE CUT POINTS SHOULD BE. I THINK IT'S AN IMPORTANT DISTINCTION. I THINK IT'S ALL ABOUT YOUR QUESTION. IN OUR CASE, WE HAVE A GROUP OF WHAT WE CALL THE MODEST RESPONDER, RIGHT THERE IN THE MIDDLE. WHICH REPRESENTS THE MEAN, ASK THEY'RE NOT THAT INTERESTING TO US. WHAT IS INTERESTING IS THE TWO EXTREME. EVEN THOUGH YOU CAN ARGUE, IT LOOKS SORT OF CONTINUOUS. WE CUT THE MIDDLE OUT. AND WE WANT TON, WHAT IS SO DIFFERENCE ABOUT THESE TWO GROUPS, THE HIGH AND LOW RESPONDERS. SO I JUST WANT TO PUT THIS BACK UP AS YOU WERE TALKING BECAUSE 5,000 DIFFERENTIAL LESION PRESSED GENES IN BASELINE MUSCLE, BETWEEN THESE TWO COHORTS OF PEOPLE. 5,000 GENIUS. THAT'S INCREDIBLE. ARE WE TALKING ABOUT THE SAME SPECIES HERE? BUT. LOOK HERE. NOW YOU CAN COMPARE THE DIFFERENCE BETWEEN EXTREME AND MODS, IT DROPS DOWN 182. VERY SUBTLE DIFFERENCES IN BASELINE MUSCLE. THE DIFFERENCE BETWEEN MODS AND NODS. ONLY 28. SO THIS GROUP IS UNIQUELY EXPRESSING 617 TRANSPORTS. IN OTHER WORDS, THAT MEANS THEY'RE DIFFERENTIALLY, EXPRESSINGS 617 GENIUS, DIFFERENT FROM BOTH OF THESE. AND YOU KNOW, YOU CAN FOLLOW IT. BUT THE POINT IS THIS IS WHERE WE SPEND A LOT OUR ATTENTION TO, FIGURE OUT WHAT, MAKES EACH OF THOSE GROUPS SO UNIQUE. >> SO ONE OF THE ISSUES WITH THESE ARE GREAT, EXPLORATORY, EMBRACING EXPLORATORY AIMS, AND WHAT I A SEE IN THE LITERATURE THAT MAKES MY BLOOD RISE IS THAT IT'S QUICKLY TRANSMITTED INTO AS IF WE FOUND GOD'S TRUTH IN THESE. AND SO I THINK WE NEED TO CONTINUE TO BE VERY EXPLORATORY, WE CAN'T, YOU KNOW, PRE-IDENTIFY EVERYTHING WE LEARN FROM THESE THINGS. BUT THING THINKING OF THIS WHITE PAPER, ENCOURAGING NEEDS IN A TEST SET AND IN A CONFIRMATORY DATA SET, SO THAT, YOU KNOW, WE HAVE A LITTLE MORE CONFIDENCE AND HOPEFULLY, LATER, KEEP PUSHING THOSE AS POSSIBLE TO BE TESTED TO REALLY ANSWER THOSE QUESTIONS YOU CAN TAKE THESE BIG HEART STUDIES AND FINALLYO AND GEMINI DO A LOT BETTER THAN ARIES AND FATHERES. WE HAVE TO BE CAREFUL WITH THE REST OF THE COMMUNITY WHEN, WE REPORT THIS, ESPECIALLY WITH GENETIC FACTORS, TOO THAT IT DOESN'T QUICKLY TRANSMIT TO AN ENTREPRENEUR WHO NOW WANTS TO CREATE AGENETIC TEST, THAT THEN THEY CAN SELL TO SAY, YOU SHOULD TAKE THESE VITAMINS, AND NOT THAT VITAMIN OR THIS TEST AND NOT THAT EXERCISE. >> I AGREE. I WILL JUST SAY THAT WE'RE USING IT, IN OUR HANDS WE'RE USING THIS AS A PROBING TECHNIQUE TO IDENTIFY PEOPLE WHO NEED TO BE TREATED DIFFERENTLY AND PERHAPS, UNDERSTAND WHY SO WE HAVE IDENTIFIED THIS GROUP THAT DIDN'T DO VERY L. WHY. DID WE INDUCE CHRONIC INFLAMMATION IN THAT GROUP? WE CAN PROBE FOR THAT. ARE THEY COMING IN THE DOOR DIFFERENTLY IN SOME WAY, AND CLEARLY WHAT, WE'RE SHOWING IS THAT YOU KNOW, WE THINK THE PRIMARY PREDICTOR OF THIS IS HOW INFLAMED THE MUSCLE IS OFF THE STREET. THAT GETS TO MORE OF THE PRESAY. BUT I DON'T WANT YOU TO GET TOO CAUGHT UP ON ALL OF THE MOLECULAR THINGS, BECAUSE IF YOU GO BACK TO THAT REMEMBER MODEL. THERE'S A LOT OF POTENTIAL DETERMINENANTS OF HETEROGENEITY, BEHAVIORAL, ENVIRONMENTAL DETERMINANTS. LOTS OF THINGS GO INTO THIS. I'VE GIVEN YOU AN OVERVIEW OF WHAT WE FOUND IN ONE SET OF DATA THAT SUGGESTED TO US, WHAT THEIR DIFFERENCES WERE, SORT OF CELLULARLY, MOLECULARLY. BUT ALL YOU HAVE DATA TRIALS, WHAT CAN YOU IDENTIFY THAT PREDICTED THEIR HETEROGENEOUS RESPONSE. GARY. >> TWO POINTS. STATISTICIANS ARE ACTUALLY ON THE HOOK. THERE'S THERE'S A LOT OF ARC UNTIL SIS PROGRAM THAT IS PEOPLE CAN TAKE OFF THE SHELF, ESPECIALLY FOR GENETICS. BUT UNFORTUNATELY, THE METHODOLOGIES, YOU KNOW, P VALUES OF 10 TO THE - -6 AND 10 TO THE - - -6. RISK FACTORS WE THINK ARE IMPORTANT LIKE SMOKING. YOU KNOW, IN THE NEIGHBORHOOD OF 2. SO THEY'RE LOST IN THOSE ANALYSES AND IDEA THAT A SINGLE SNIP IS GOING TO TELL US EVERYTHING WE NEED TO KNOW IS ALSO FLAWED. THAT DOESN'T MEAN WE SHOULDN'T BE EXPLORING THEM. BUT ONE GREAT GRATE USE OF EXISTING DATA IS VALIDATION. IT ENABLES YOU TO MOVE QUICKLY IN WHAT YOU MIGHT HAVE DISCOVERED IN YOUR EXPLORATORY TO SEE IF IT HOLDS UP. IN MULTIPLE DATABASES, WHERE THE CRITERIA IS DIFFERENT. NOT EVERY MEASUREMENT IS GOING TO BE THERE. AND YOU MAY HAVE TO DO SOME SEARCHING AROUND FOR THE DATABASE. WHICH IS ALSO THE IDEA OF THE RESEARCH MATCH FOR BIOMARKER SAMPLES AND THINGS THAT CAN YOU ACTUALLY GO TO, WITHOUT HAVING TO DO ANOTHER STUDY. I THINK IT WOULD HAVE A GREAT SPEED KIND OF THING. WE TALK ABOUT INNOVATION. WE'RE VERY SLOW IN MANY WAYS, TOO MOVE FORWARD. THEN WE LAMENT THE FACT THAT IT TOOK 20 YEARS I MEAN, I FINISH AID STUDY THAT START THED IN 1996. WE DIDN'T FIND A WHOLE A LOT BECAUSE IT WAS NO LONGER AN INTERESTING QUESTION BY THE TIME WE GOT DONE. >> ANY OTHER QUESTIONS? >> SO THIS IS REALLY INTERESTING WORK. I THINK THE GENETICS, DO LIKELY EXPLAIN A PORTION OF THIS RESPONSE. AND THE OTHER PIECE, I GUESS IS A REMINDER, THAT I FIND IMPORTANT TO EMPHASIZE IS THAT WHEN LOOKING AT HETEROGENEITY AND INDIVIDUALIZED TREATMENT RESPONSES, THAT OTHER DISCIPLINES HAVE ALREADY LOOKED AT BEYOND THE GENETICS, HAVE LOOKED AT THE ENVIRONMENTAL BEHAVIORAL SOCIAL FACTORS THAT MIGHT CONTRIBUTE TO SOME OF THESE RESPONSES. WHILE WE THINK THAT REHAB OR ISMAIL DOSE OF PHYSICAL ACTIVITY IS GOING TO HAVE A HUGE EFFECT. REHAB, TWO TIMES A WEEK, PROBABLY 30 MINUTES A SESSION, DEPENDING ON THE TYPE OF INTERVENTION, THERE'S PROBABLY A LOT OF OTHER FACTORS THAT YOU'VE ACACKNOWLEDGED UP HERE THAT, OTHER DISCIPLINES HAVE BEEN LOOKING AT IN VERY COMPLEX MODELING, LIKE PSYCHOLOGY. WE'RE LOOKING AT ENJOYING ARTHROPLASTY, TAKING MODELS IN PSYCHOLOGY, TOO LOOKING AT HOW WE INDIVIDUALIZE TREATMENTS OX THING COMPLEMENT THAT IS NICELY. WE SHOULD BE CONSIDERING NOT RE-INVENTING THE WHEEL AND LOOKING TO DISCIPLINE THAT IS HAVE DEVELOPED SOME OF THESE ALGORITHMS AND APPLY THEM TO OUR REHAB QUESTION. >>S IN NOT UNIQUE TO A PHYSICAL INTERVENTION. YOU LOOKING AT EVERY AVAILABLE MEDICATION IT'S IN EVERY FORM OF CARE AND MEDICINE. SOME, IT'S BEEN EXPLORED PRETTY L IN OTHERS. NOT SO WELL. TO THE GOING THANKSGIVING THAT WE CAN CAPITALIZE ON THE DATA WE HAVE COLLECTED AND BECOME MORE PRECISE WITH HOW WE INTERSEEN ON PEOPLE, I THINK WE CAN LEARN A LOT FROM WHAT WE'RE DOING. AND SO THIS SORT OF SPEAKS TO WHAT GARY TALKED ABOUT IN THE OPENING SESSION, IN TERMS OF TRIAL DESIGN. HOW ADAPTIVE CAN WE GET. HOW PRECISE CAN WE GET. DO WE HAVE TO, THE DOSE RESPONSE QUESTION IS REALLY IMPORTANT. >> WE HAD A PAR. GENOMIC UNDER PIPINGS -- WE MADE ONE AWARD, WHICH NCMRR IS COFUNDING. I WAS A LITTLE SURPRISED THAT THE RESPONSE WASN'T AS GOOD AS I THOUGHT. ROBUST. I DON'T KNOW THAT IF THE TIMING, THIS WASN'T RIGHT. I DON'T KNOW IF IT'S A REVIEW ISSUE, AS YOU MENTIONED COULD BE. THESE TYPES OF TOPICS ARE ALSO WELL HIGHLIGHTED IN THE REHAB RESEARCH PLAN THAT WAS RECENTLY RELEASED. SO I AGREE, THIS IS IMPORTANT. AND THERE IS AN INTEREST AT NIH IN LOOKING AT SOME OF THESE. SO I ENCOURAGE TO YOU THINK ABOUT THAT. >> I'M REALLY SAD THEY MISS THAT WOULD. >> YEAH, CAN YOU GUYS TALK A LITTLE BIT ABOUT HOW WE DEAL WITH THE FACT THAT THESE FACTORS INTERACT WITH EACH OTHER? AND SOMETIMES IT'S THE INTERACTION OF THE FACTORS THAT CAUSES SOMEONE TO BE A NONRESPONDER AND NOT, YOU KNOW, WE'RE ALWAYS SORT OF LOOKING FOR THE HOLY GRAIL. IT'S THE CLINICAL PEOPLE TRACT OR SOMETHING IN THEIR GENOMICS. CAN YOU TALK ABOUT THAT A LITTLE BIT? YOUR THOUGHTS. >> I'M HEARING FROM SEVERAL TEAM. THERE'S A JEN ONLYICS, END ALL, BE ALL. I THINK IT'S A PIECE OF THE PUZZLE. IT'S NOT THE END ALL, BE ALL. IT'S GOING TO ACCOUNT FOR SOME OF THE VARIANCE. THERE'S THE POSSIBILITY IN THE FIGHTER THAT, WE'RE GOING TO BE ABLE TO DO THESE EXPLORATORY ANALYSIS, EMBEDDED IN OUR GRANTS THAT WE BE ALLOWED TO COLLECT MULTI MODAL INFORMATION. SO IT'S GOING TO BE IMPORTANT TO COLLECT TISSUE AS WELL AS IMAGING, TO BE ABLE TO DO THOSE ANALYSES AND LOOK FOR INTERACTIONS. THE ENVIRONMENTAL AND BEHAVIORAL PIECE IS CRITICAL. IT MAY BE, IN SOME MODELS. IN SOME POPULATIONS, THERE MAY BE ENVIRONMENTAL INFLUENCES THAT ARE THE PRIMARY DETERMINENANTS OF HEY ROW GENEITY. THEY HAVE NOTHING TO DO WITH PROFILING OR STEM CELL ACTIVITY. BUT ALL OF IT NEED TO BE EXPLORED OKAY . WE'LL END, TAKE A BREAK. COME BACK AND START OUR WHITE PAPER PLANNING. SO THANK YOU. THAT IS REALLY FANTASTIC, AND I THANK YOU ALL VERY MUCH. SO WHAT MARCUS INDICATE THAT IS WE'RE GOING TO DO IS TO GO, AS IS ON THE AGENDA SESSION, BY SESSION, AND THE SPEAKERS FROM THE SESSION WILL GIVE THEIR TOP THREE, AND YOU CAN DEVELOP THEM FURTHER FOR THE WHITE PAPER. MARCUS. I'VE LEARNED A LOT. HOPEFULLY, WE HAVE ALL LEARNED A LOT FROM ONE ANOTHER. THIS HAS BEEN A REALLY, REALLY FRUITFUL MEETING THERE IS NO SINGLE BEST APPROACH TO DEVELOPPATION WHITE PAPER OUTLINE. I THINK THE NEXT TWO BLOCKS OF TIME, CAN PROBABLY BE CONDENSED TO ONE. A LITTLE FATIGUE IS SETTING IN. WE ALL NEED AN INTERVENTION AT THIS POINT. TO BE AS PATIENT AS WE K. JUST REVISIT THE DIFFERENT SESSION AREAS. WE K IN A FREE-FLOWING WORKING DOCUMENT, IDENTIFY WHAT WE THINK THE HIGH PRIORITY AREAS ARE AND OR THE OPPORTUNITIES FOR THE FUTURE. IN EACH OF THESE DOMAINS. SO IF THAT SOUNDS OKAY TO EVERYBODY. I CAN'T THINK OF A BETTER APPROACH. IF HAVE YOU ONE, I'D BE OPEN TO HEAR IT. OTHERWISE, THING MIGHT BE THE BEST APPROACH. SO LET'S START WITH THE FIRST SESSION, WHICH IS REALLY ALL ABOUT DESIGN CONSIDERATIONS. I'M GOING TO GET A MICROPHONE TO GARY AND ALSO, EDDIE, HERE YOU CAN HAVE THIS ONE. SO CAN YOU BOTH HAVE ONE. AND JUST SORT OF LEAD US THROUGH THIS. I THINK WHAT WE TRIED TO CONVEY IS THAT THERE IS A BANQUET OF DESIGNS THAT WE CAN CHOOSE FROM AND IT ISN'T ABOUT THE DESIGN ITSELF. THE DESIGN IS EXCEEDINGLY IMPORTANT WANT BUT THE QUESTION IS WHAT DRIVES IT AND WE ALL HAVE TO MAKE COMPROMISES ALONG THE WAY IN TERMS OF THE KINDS OF QUESTIONS WE'RE ASKING, THE RESOURCES WE HAVE TO ANSWER THEM AND THEN THE DESIGNS THAT WE CAN APPLY THAT ARE GOING TO GETS WHERE WE'RE GOING. BUT THERE IS NO SINGLE, PERFECT ANSWER. THERE'S ALWAYS A COMPROMISE AND STRENGTHS AND WEAKNESSES TO EVERY APPROACH. >> ONE OF THE IMPORTANT THINGS RELATED TO DESIGN, IS EFFICIENT INVESTIGATOR TO BE ABLE TO COMMUNICATE WHAT PHASE OR STAGE OF RESEARCH THEY'RE AT. AND I THINK JOHN MADE SOME REALLY GOOD POINTS YESTERDAY ABOUT HOW SOMETIMES IT'S HARD TO FIGURE THAT OUT. WHERE YOU ARE IN YOUR STAGE, MAY DEPEND ON THE TYPE OF WORK THAT YOU DO. NONETHELESS, I THINK THERE ARE SOME OVER ARCHING SIMILARITIES AMONG THE DIFFERENT TYPES OF RESEARCH THAT COULD SERVE TO HELP THE REVIEWERS, READERS, EVEN HELP THE INVESTIGATOR ONE HAVE A WHERE THEY ARE, IN THEIR STAGE OF RESEARCH. BY DISCUSSING THE PRIOR LITERATURE AND THAT TYPE OF THING AND WHERE IT'S INTENDED TO GO. SO I THINK, I DOUBLE THERE'S VALUE IN TRYING TO IDENTIFY WHAT STAGE YOU ARE. SO THAT YOU ARE CLEAR ABOUT WHAT'S COME BEFORE AND WHAT WILL COME NEXT MUCH SO THAT'S ONE THOUGHT. THE OTHER THOUGHT THAT I BELIEVE WAS REALLY IMPORTANT ABOUT THE CONTENT THAT WE DISCUSSED IN THAT FIRST SESSION IS THIS ISSUE OF MULTI MODAL INTERVENTION. AND THE NEED TO UNDERSTAND THE INFLUENCE OF A SINGLE INTERCEPTION AND POSSIBLE INTERACTIONS WITH INTERVENTIONS, AND HOW THOSE INTERACTIONS, AS DARCY POINTED OUT, MIGHT BE DIFFERENT FOR INDIVIDUALS AND MAY ACCOUNT FOR THE DIFFERENCES WE SEE IN RESPONSES. >> SO IF I CAN JUST ADD SOMETHING. I'M TRY I'M TRYING TO THINK OF A POTENTIAL STRUCTURE IN AND ONE APPROACH MIGHT BE HERE ARE THE DIFFERENT DESIGN OPTIONS. HERE'S HOW, YOU KNOW, YOU MIGHT WANT TO TACKLE A MULTI MODAL, VERSUS A UNIMODAL INTERCEPTION. HOW CAN YOU TAKE ADVANTAGE OF THESE DIFFERENT DESIGN OPTIONS, IN EACH OF THOSE TWO MODDYS BECAUSE IN THIS FIELD, YOU KNOW, MANY PEOPLE GRAPPLE WITH THE MULTI MODAL AND SOME PEOPLE ARE TRYING TO SIMPLIFY IT TO ONE. >> SOMEBODY MENTIONED YESTERDAY, IT'S THE KITCHEN SINK APPROACH, VERSUS THE, YOU KNOW, WHAT IS IT YOU WANT TO TIME-OUT. DO YOU WANT TO FIND OUT WHAT PART OF THE KITCHEN WORKS? ARE YOU OKAY? IF YOU JUST FIND OUT IF IT WORKS. AND I THINK WE COULD ACTUALLY TAKE AN ARRAY OF DESIGNS AND DESIGN TYPES AND THEN LOOKING AT VARIOUS TYPES OF DESIGNS AS SORT OF JUST A FRAMEWORK FOR THIS. THE OTHER THING I HAVE TO TAKE. I THINK JOHN WHITE MENTIONED A GREAT POINT, WHICH IS, ARE YOU GOING TO BE VERY HOMOGENEOUS, WHICH IS WHAT WEIGH OFTEN SAY WE WANT IN PHASE 2 TRIALS THAT, WE WANT TO HAVE A HOMO SCREENUOUS POPULATION, WITH FEW COMORBIDDITIES AND BE ABLE TO FOCUS IN ON THOSE, DOES IT WORK IN THIS THING AND GENERALIZE IT FROM THERE. BUT IF YOU DON'T KNOW WHETHER YOU'RE WORKING WITH T IS IT GOING TO WORK WITH EVERYBODY OR A SUBGROUP. WHAT DO YOU PUT IN THAT HOMOGENEOUS POPULATION. EVEN THOUGHTS ABOUT THAT, AS IT RELATES TO DESIGN, WOULD PROBABLY BE VERY USEFUL. WHAT IS THE PRIORITY. WHERE SHOULD WE PUT OUR ATTENTION? I CENTER A MED QUESTION. NOT ABOUT THIS SESSION, BUT THE OVER ALL STRUCTURE, WHICH IT FEELS TO ME LIKE WE SAW A LOT GOOD IDEAS THAT ONE COULD SORT OF PUT AS A MENU. BUT THERE WAS A MORE AMBITIOUS, BUT MORE DANGEROUS APPROACH, WHICH IS TO SAY F YOU'RE IN THIS PHASE, SHOULD DO THIS, YOU KNOW, TOO PROVIDE ACTUALLY MORE DECISION GUIDANCE. I MEAN, IN PRINCIPLE, I THINK THAT WILL BE MORE USEFUL. A LOT PEOPLE KNOW THERE ARE MANY DESIGNS OUT. MANY OPTIONS, BLAH, BLAH, BLAH, BUT IT'S PICKING THE RIGHT ONE THAT'S THE HARD PART. WE HAVE NOT HAD A LOT OF DISCUSSION ABOUT WHAT WOULD BE THE RIGHT THING TO DO WHEN. SO YOU KNOW, COULD WE WRITE ABOUT THAT AND REACH ANY KIND OF CONSENSUS. CAN WE CLARIFY HOW MUCH WE'RE LAYING OUT A BUNCH OF OPTIONS TO CHOOSE FROM AND HOW MUCH WE'RE REALLY TRYING TO GUIDE PEOPLE MORE FORCEFULLY, IF YOU AM -- IF YOU WILL. ONE THING THAT WOULD BE VERY VALUABLE WOULD BE TO CREATE A WORKBOOK FOR PEOPLE. IT'S LIKE, SO, I'M GOING TO DO A CLINICAL TRIAL NOW. WE HAVE DISCUSSED HOW PEOPLE, OUR RESEARCH, ALL OF A SUDDEN, COME TO THAT, AND THEY DON'T REALIZE, THEY DON'T APPRECIATE WHAT WE'RE GETTING INTO. A LOT OF ISSUES THAT YOU FOLKS HAVE RAISED ABOUT HOW TO DO A CLINICAL TRIAL RIGHT. PEOPLE MAY NOT CATCH TO IT. AND I'D ALSO LIKE TO EXPRESS A HIGHER GOAL. SOME OF THE THINGS YOU'RE TALKING B THAT THE REHAB FIELD IS DOG, THAT A LOT NIH CLINICAL TRIALS DESIGN COULD BENEFIT F. A LOT OF OTHER TRIALS ARE NOT AS SIMPLE AS THEY THINK. AND YOU GUYS ARE ALL OVER. AND IF SOME AIR, I DON'T KNOW IF THEY WOULD READ YOUR ARTICLE. BUT IT WOULD BE GREAT IF YOU ACTUALLY THOUGHT OF THIS AS A WAY TO MAKE CLINICAL TRIALS BETTER AND MORE VALID. >> ANOTHER POINT THAT I THINK IS REALLY CRITICAL TO BE ADDRESS ADDRESSED IN THE RIGHT PAPER WOULD BE THIS ISSUE OF THE CONTROL GROUP, AND THE COMMENT DANIEL MADE YESTERDAY, I THINK WAS REALLY UPON KEY TO THAT; THAT WHOLE FIGURING OUT WHAT THE ACTIVE INGREED IANT AND I THINK IT WOULD BE VALUABLE TO COME TO A CONSENSUS TO GIVE GUIDANCE ABOUT WHETHER OR NOT DOING NOTHING OR YOU KNOW, SOMETHING SIMILAR TO NOTHING, IS A REASONABLE COMPARISON GROUP, ESPECIALLY IN THE EARLY STAGES, WHERE YOU'RE TRYING TO STILL WORK OUT MECHANISMS. RELATED TO THAT MATTER S WHAT ARE WE TO DO WITH OUR PERENNIAL PROBLEM OF THE USUAL AND CUSTOMARY CARE THAT WE'RE PROVIDING TO PATIENTS THAT, WE'RE MANDATED TO PROVIDE PHYSICAL AND AUTOPSIESAL THERAPY, PARENTS WITH STROKE, IN THE EARLY TIME PERIODS AFTER ON SET, CAN WE PROVIDE ANY GUIDANCE FOR A STUDY DESIGN IN THIS RESPECT. I THINK WHEN WE'RE STUDYING PARTIALS WITH STROKE AND TBIIN THE CHRONIC PHASES. IT ISN'T USUAL AND CRUST MARRY FOR THEM TO GET ANYTHING. SO IS IT QUOTE, UNQUOTE, WRONG OR BAD TO COMPARE TO NOTHING. IF THAT'S THAT'S REALLY WHAT THE STANDARD OF CARE IS. I THINK IT'S GOING TO BE IMPORTANT TO POINT OUT AND DISTINGUISH USUAL FROM STANDARD. BECAUSE THERE IS NO STANDARD. IN A LOT THESE CASES I THINK USUAL CARE AND A PART OF THIS PAPER NEEDS TO SPEAK TO THE FACT THAT USUAL CARE IS ALL OVER THE PLACE. WHETHER WE COME TO A CONSENSUS ABOUT WHETHER WE WANT TO CONTROL THAT, THAT'S SOMETHING THAT'S GOING TO, WE'RE GOING TO HAVE TO THINK ABOUT. BUT AT LEAST CAPTURING IT AND QUANITYIFYING IT AS BEST WE CAN, TO MAKE SURE THAT THE VARIABILITY ACROSS PEOPLE WHO ARE IN USUAL CARE, IS AT LEAST CAPTURED IN A WAY THAT WE CAN STUDY IT. I THINK THAT'S THE REALITY. I THINK IF YOU WANT TO START TO CONTROL USUAL CARE, YOU NOW HAVE A TWO - ARM INTERVENTION TRIAL YOU DON'T HAVE A CONTROL. >> SO LET ME JUST MAKE A RECOMMENDATION ALSO USUAL CARE, VARY FROM REGION TO REGION. THIRD ISSUE FOR ME, IS THINK OF PLACEBO PART OF IT IS DRIVEN BY THE FDA BUT I THINK FROM THE NIH PERSPECTIVE, IN TERPS WHATEVER MAKES SENSE AT THE LEVEL OF EFFICACY, AND THE LEVEL OF EFFECTIVENESS, ALMOST A PERMISSION NOT TO DO PLACEBO. I THINK IS SOMETHING, IF IT COMES OUT IN A WHITE PAPER LIKE THIS. MAY HAVE SOME INFLUENCE ON THE STUDY SESSIONS, WHO I THINK BY DEFAULT, TEND TO GO TO PLACEBO AND I'M NOT SHOW SURE IT IS IN THE BEST BENEFIT OUR COMMUNITY. >> I THINK THEY'LL GO TO PLACEBO BECAUSE THEY WORRY ABOUT MASKING. IN TERMS OF THE OBJECTIVE RESPONSE OR THE EXAMINER AND WE TALKED ABOUT, YOU KNOW, CAN YOU HAVE MAST OBSERVER. CAN YOU GO TO A USUAL CARE APPROACH. IF IT'S AN OBJECTIVE OUTCOME THAT CAN'T BE SO INFLUENCED BY THE SUBJECT'S KNOWLEDGE OF WHETHER THEY GOT THE COOL NEW TREATMENT, THEN AT THE TOTALITY OF A DESIGN, THAT MAY BE ACCEPTABLE WITHOUT A PLACEBO. BUT THAT'S THE CONFOUNDING OF THOSE TWO IDEAS. AND I THINK WE HAVE TO BE VERY CAREFUL. BECAUSE YOU GET, YOU KNOW, IN MILD TBE. USUAL CARE AND YOU GIVE AN INTERVENTION AND MOST OF THE OUTCOMES ARE SUBJECTIVE. AND WHEN THE PATIENT HASN'T BEEN GIVEN ANYTHING TO THINK THEY HAVE GOTTEN SOMETHING DIFFERENT. IT'S HARD TO BELIEVE THAT'S NOT IMPACKING THEIR RESPONSIVENESS TO THEIR DEPRESSION OR HOW THEY'RE FEELING ABOUT LIFE. SO I THINK WE HAVE TO BE VERY CAREFUL. WHEN WE MAKE THOSE POINTS IN THIS PAPER TO, COUNTER IT, INTO GOOD PRACTICE AND DESIGN. >> [SPEAKING AWAY FROM MICROPHONE] >> SO I WAS GOING TO SAY, EDDIE, I DON'T THINK WE CAN PREACH A CONSENSUS, BUT I THINK FOLLOWING UP ON WHAT MARCUS SAID, IF WE TIE THAI IT INTO THE OTHER SECTIONS WE WILL BE TALKING ABOUT THIS IDEA OF MEASURING WHAT WAS GIVEN, MEASURING THE SPECIFIC INGREDIANTS OF WHAT WAS GIVEN. HOW MUCH OF THEM, ET CETERA. WE COULD HAVE A STATEMENT. A CONSENSUS STATEMENT THAT, REGARDLESS OF THE DESIGN YOU CHOOSE, AND WHAT CONTROLS YOU CHOOSE AND THAT YOU NEED JUSTIFICATION FOR THEM, YOU NEED TO BE MEASURING EVERYTHING IN ALL GROUPS. SOMETHING LIKE THAT. I THINK THERE'S A PLACE TO THAT. BUT THERE'S A PLACE ALSO NOT FOR THAT. IF YOU BELIEVE USUAL CARE ISN'T WORKING AND THAT'S WHY YOU'RE TRYING AN INTERVENTION, WHO CARES WHAT THEY'RE DOING AND WHY MEASURE IT? BECAUSE YOU'VE ALREADY DECIDE IT'S NOT GOOD ENOUGH. AND THEREFORE, HAVE YOU THIS GREAT NEW INTERVENTION. IF PERHAPS, YOU HAVE PENICILLIN IN YOUR NEW INTERVENTION. DOESN'T MATTER WHAT THEY'RE DOING AND YOU WON'T NEED A LOT OF SUBJECT. SO I WANT TO BALANCE THESE STATEMENT WITH THE ALTERNATIVE, SO PEOPLE STAY ON THE END AND DON'T GET STUCK INTO RUTS. >> AND TO ADD TO THE USUAL CARE SITUATION, WHEN PEOPLE ARE BEING WATCHED, THEY DO THINGS DIFFERENTLY. AND SO WE'RE CONCERN, AND THIS TOPIC CAME UP, BY NOT STANDARDIZING THE USUAL CARE GROUPS, EVEN THOUGH THERE'S HETEROGENEITY IN WHAT PEOPLE DO, PEOPLE ARE GOING TO REGRESS TOWARDS THE INTERVENTION GROUP AND WE'RE NOT GOING TO BE ABLE TO DETECT AND EFFECT. JUST BECAUSE THEY'RE BEING WATCHED. SO I LIKE THE APPROACH OF KIND OF A BALANCE BETWEEN THE TWO, AND I DON'T THINK WE CAN ABSOLUTELY DEFINE T. I WAS GOING TO SUGGEST. I LIKE RALPH IDEA. HOW CAN WE DO CLINICAL TRIALS ON A BIGGER LEVEL KIND OF, COULD WE SET THE STANDARD. BUT I WONDER IF THAT IS KIND OF A LARGE ENDEAVOR TO DO WELL. AND SHOULD WE FOCUS MORE ON, BECAUSE THERE'S LOTS OF INFORMATION ON HOW TO RUN CLINICAL TRIALS OUT THERE. SHOULD WE REALLY FOCUS THIS PAPER ON SPECIFICALLY, WHAT ARE THE NUANCES OF REHAB, COMPARED TO PHARMACEUTICAL TRIALS FOR WHICH THERE'S A TON OF GUIDELINES AND REALLY HAVE IT EMPHASIZE THOSE UNIQUE ASPECTS WHATEVER WE'RE DOING, THAT -- AND HAVE T BECAUSE EVEN WITH THAT, THERE'S A TON OF CONCEPTS AND IDEAS THAT WE HAVE TO COVER. INSUFFICIENT DETAIL FOR IT TO BE HELPFUL. >> I WAS SAYING GOALS. [LAUGHING] >> SO TWO COMMENTS. THE FIRST COMMENT IS TO JENNIFER'S COMMENT. YOU MENTIONED NUANCES OF REHAB. I JUST THINK REHAB IS DIFFERENT FROM MEDICATION. I DON'T THINK IT'S SLIGHTLY DIFFERENT. AND I THINK WE ARE NOT PENALIZED, BUT I THINK WE TRY TO KIND OF FIT OURSELVES INTO THE MEDICATION MODEL AND I THINK IT'S MUCH MORE THAN A SUBTLE NUANCE. AND AIR, NOT THAT THE DEVIL NEEDS ANY ADVOCATING FOR. BUT JUST TO ADVOCATE ON BEHALF OF THE DEVIL, I'M ALSO VERY CONCERN ABOUT THE PLACEBO COMMENT. I UNDERSTAND WHERE JOHNNY'S COMING F. BUT IT MAY BE, DISEASE-SPECIFIC BUT IN PARKINSON'S DISEASE. ANYTHING YOU DO, WHICH ISN'T BETTER THAN 20% IS REALLY DISCOUNTED. BECAUSE YOU ACCEPT AN ENORMOUS PLACEBO EFFECT, WHICH HAS DOMINATED STUDIES OVER 20, 30 YEARS, SO I THINK THE PARTS OF PLACEBO NEED TO BE WRITTEN IN A VERY CAREFUL AND NUANCED WAY. I THINK WE SHOULD DISCUSS THE CONCEPTS BEHIND PLACEBO. I THINK WE HAVE TO GIVE UP ON LITERAL PLAZEB OH, WHICH MEANS GUARANTEED TO BE INERT AND FULLY PLAUSIBLE. FOR MOST OF THE INTERVENTION WEES DO, THERE IS NO WAY CAN YOU PROVIDE A PLACEBO. YOU CAN THINK ABOUT HOW TO COUNTER EXPECTATIONS. HOW TO MANAGE THE BIAS. BUT IT'S NOT GOING TO BE WITH A THING THAT LOOKS LIKE THE DEVICE AND EVERYBODY FALSE FOR. BECAUSE WE CAN'T DO THAT FOR MOST OF OUR INTERVENTIONS. YOU WANT TO KEEP THE CONTROL GROUP AS POOR AS POSSIBLE SO CAN YOU SEE T. ARE YOU TRYING TO DO SOMETHING BETTER THAN YOU CAN DO, RELATIVELY, INEXPENSIVELY. AND REALLY, THERE ARE TWO DIFFERENT QUESTIONS. THEY HAVE OUTCOME COMPONENTING TO IT. BUT IN DRUG TRIALS, THE PLACEBO CAN USED BECAUSE WE CAN GET THE ANSWER THE QUICKEST. YOU ACTUALLY HARM FEWER PATIENTS FROM A SOCIETAL PITCH. THERE ARE FEWER EVENT THAT IS OCCUR. YOU CAN SEE A BIGGER DIFFERENCE AGAINST A PLACEBO AND THAT'S THE CONCEPT. IT DOESN'T QUITE APPLY HERE AND MAYBE WE CAN MAKE THAT POINT. >> WE'RE GOING TO HAVE TO BE VERY CLOSE TO THE TIME NOW. I WANT TO MAKE ONE PLUG FOR ALSO INCLUDATION SMALL TOPIC YOU BROUGHT UP, GARY IN YOUR TALK, WHICH WAS REGRESSION TO THE MEAN. OFTEN, IT IS ACTUALLY REGRESSION TO THE MEAN, THAT'S DRIVING THE CONTROL GROUPS' RESPONSE, RATHER THAN A PLACEBO RESPONSE, AND THIS IS ACROSS A LOT OF NIH TRIALS, WHERE THEY USE A DISEASE CUT POINT, WHICH IS ALSO THEIR OUTCOME. SO I THINK WE COULD, BY HAVING THAT IN THIS PAPER AND HIGHLIGHTING, I THINK WOULD BE A GREAT POINT. I WAS JUST TALKING ABOUT CONTROL AND PLACEBO. THERE IS SOME INFORMATION IN THE LITERATURE ABOUT EVEN THE CONTROLLED GROUP GETTING IMPROVED. LIKE THE PLACEBO. SO I KNOW THERE'S A DIFFERENCE BETWEEN THE PLACEBO AND THE CONTROL. BUT THING ALSO NEEDS TO BE MENTIONED. EVEN IF WE DO NOTHING WITH THE CONTROL. AND WHAT IS THE DIPS BETWEEN THE EFFECT OF THE PLACEBO AND CONTROLLED. SO THOSE TWO GROUPS, NEED TO BE DISCUSS. >> SO WE'RE GOING TO GO ON TO SESSION 2. KEN IS NOT HERE. STEPHANIE S. YOU'RE THE LEADER. THERE WAS A DIVERSE TOPIC SORT OF, YOU KNOW, YOU COVERED A WIDE RANGE OF TOPICS. SO LET'S ALL SORT OF THINK ABOUT WHAT DO WE WANT TO TEASE OUT THAT HAVE SESSION. >> I DO WANT TO POINT OUT ONE THING THAT'S A BIG DEAL WITH REGARD TO USUAL AND CUSTOMARY CARE AND CONTROL. THAT'S SOMETHING WE HAVEN'T TALKED MUCH ABOUT THIS AND I REALIZE, EVERYONE HERE IS PRIMARY ADULT REACHERS. BUT REHAB DOESN'T ONLY APPLY TO ADULTS. IT APPLIES TO CHILDREN AND CLINICAL TRIALS APPLY TO CHILDREN. WHEN YOU TALK ABOUT CONTROL GROUPS, AND YOU TALK ABOUT USUAL AND CUSTOMARY CARE, YOU BRING A WHOLE OTHER LEVEL OF COMPLEXITY INTO T WHEN YOU BRING IN THE DEVELOPMENTAL SPECTRUM OF CHANGE AND SO I WOULD HATE FOR THIS TO NOT -- FOR US NOT TO ADDRESS THAT, IN SOME CAPACITY, AS WE MOVE FORWARD WITH THIS DISCUSSION AND WITH THIS PAPER. BECAUSE THERE'S A HUGE POPULATION OUT THERE. REHAB OCCURS IN CHILDREN AND AS WELL. BUT WITH REGARD TO OUR TOPIC, I THINK WE DID COVER A WHOLE A LOT OF A BROAD RANGE OF AREAS AND I THINK WE ALL KIND OF AGREED THAT FIDELITY WAS ONE OF A MAJOR THING. UNDERSTANDING AND DO YO DO YOU MEANING WHAT WE DO WITH ALL OF OUR INTERVENTIONS, WAS PROBABLY, I THINK, AN AGREEMENT ACROSS THE BOARD. WE NEED GREATER STANDARDIZATION OF DOCUMENTING WHAT IS BEING DONE, IN VARIOUS INTERVENTIONS. AND THEN, YOU KNOW, I GO BACK AND FORTH WITH THE ANY WE TALKED ABOUT THE CLINICAL TRIALS AND PRAGMATIC TRIALS. IT GOES BACK TO THE QUESTION. I AGREE TWO PROBLEMS THAT. THING YOU REALLY HAVE TO LET YOUR RESEARCH QUESTION AND WHAT I WANT TO DO KNOW, CHRIST TYPES THAT YOU PREFER. THE OTHER THING, I DO THINK WE NEED TO CONTINUE TO BUILD IN AND WE PRINTOUT THIS IN. OTHER DISCIPLINES. IMPLEMENTATION SCIENCE BEING ONE. BUT DEFINITELY, BEHAVIORAL PSYCHOLOGY AND THINGS OF THAT NATURE. THEY HAVE HUGE ASPECTS TO BUILD INTO WHAT WE'RE DOING. AND IN MY MIND, THOSE ARE THE TOP AREAS THAT KIND OF CAME OUT FROM OUR SESSION. I THINK IT GETS TO RIGOR AND REPRODUCIBILITY IN A VERY KEY WAY WITH RESPECT TO POLICY. I THINK WHEN THERE'S AN INTERVENTION DESIGNED, AND YOU WANT TO SHOW FIDELITY AND DOCUMENTATION AND REPRODUCE IT AT MULTIPLE SITES OR IN AN VALIDATION STUDY, IF WE DON'T HAVE THE INVESTIGATORS' BROCHURE ON WHAT YOU'RE DELIVERING, IT'S GOING TO BE VERY DIFFICULT TO DO. IF YOU DON'T HAVE THE SAME FIDELITY CHECKS. THE STUDY WAS A PERFECT EXAMPLE. IT'S DIFFICULT TO EXECUTE MULTI SITE TRIAL OF A SPECIFIC INTERVENTION IN A SPECIFIC POPULATION, SAME WITH JOHN'S TRIALS AND SOME OF THOSE ARE INCREDIBLY WELL-CHARACTERIZED. BUT IN ORDER FOR IT TO BE REPRODUCIBLE, IT HAS TO HAVE THAT, AND I THINK THAT IS A KEY CONSIDERATION, ESPECIALLY AS YOU THINK ABOUT LARGER SCALE TRIALS. >> THERE IS SOME HELP AND MAYBE IT'S VALUABLE AS TO WHAT THINGS WE MIGHT PUT INTO, PEPPED EXISTING STUDIES. THERE'S A RELATIVELY NEW CHECK LIST THAT MANY WHO PUB ACCOMPLISHED THE REHAB WORLD MAY BE FAMILIAR WITH, WHICH IS THE TIDYER CHECK LIST, WHICH IS INTENDED TO PROVIDE AUTHORS, A GUIDE TO BE REALLY TRANSPARENT ABOUT EVERY ASPECT OF THE INTERCEPTION, SO THAT IT CAN FACILITATE REPLICATION OF THE TRIAL. AND IT MAY BE SOMETHING THAT, FOR EXAMPLE, COULD GO INTO AN APPENDIX OF HELP FOR PEOPLE WHO ARE DEVELOPING TRIALS. >> OKAY. WHEN I WAS AT NIDA. SO I'M CONCERN ABOUT WHAT IS FIDELITY AND WHAT IS THE TREATMENT AND BELIEVING FROM METHODOLOGY TO THE TREATMENT. DURING THE CLINICAL TRIALS, SENDING PHYSICAL THERAPISTS OUT, BEEPING THEM ALL THE TIME ON THEIR CELL PHONE, ET CETERA, ET CETERA. ALL THAT BECOMES PART OF THE TREATMENT. YET. THE TREATMENT THAT WE'RE LOOKING AT, MAY BE 10 MINUTES OF EXERCISE A DAY, WHATEVER. BUT ALL OF THIS ACCESS TO FIDELITY OR KEEPING FIDELITY IS REALLY DRIVING THE EFFECT. SO I HAVE A -- I HAVE TWO OCCURRENCE. ONE IS THAT ALL THE FIDELITY INFORMATION IS NOT CAPTURED AS PART OF THE DATA AND IS NOT STUDIED. AND THAT ALL OF THE CONCERN ABOUT GETTING THE DATA AND HAVING FOLLOW-UP OVERWHELMS THE TREATMENT THAT YOU'RE TRYING TO STUDY. >> I'LL JUST COMMENT I'M GOING TO EXPLAIN SOMETHING, THAT IS A FRUSTRATION WE HAVE ALL EXPERIENCED, WRITE OR READ META ANALYSIS. AND SO ONE OF THE THINGS THAT COULD COME OUT OF THIS, WOULD BE DISCUSSED IN THIS PAPER. IS THIS IDEA OF REPRODUCIBILITY, RIGOR, TRANSPARENCY, FIDELITY. THINGS THAT WOULD BENEFIT THE FIELD MORE BROADLY, NOT NECESSARILY FOR THE NEXT TRIAL, BUT TO INTERPRET ALL THESE TRIALS. AND PICK UP ANY META ANALYSIS. WE FOUND 6,724 PAPERS. BY THE TIME THEY WENT THROUGH THEIR QUALITY CHECKS WE'RE GOING TO NOW DISCUSS 12 PAPERS. WE DON'T EXPECT PEEP TO STANDARDIZE THE TREATMENTS AND WHAT NOT. OBVIOUSLY, THEY'RE ASKING DIFFERENT QUESTIONS. BUT IF WE CAN EXPRESS SORT OF WHAT NIH IS DESIRING, AND WHAT WE THINK IS APPROPRIATE FOR THE FIELD, TOO BE FULLY EXPLICIT ABOUT IN THE DESIGN PHASE AND SO FORTH. THING WILL GO A LONG WAY TO HELP EVERYBODY INTERPRET THE RESULTS OF THE STUDIES AS THEY COME OUT. >> I AGREE IN AN IDEAL WORLD. I THINK WE HAVE TO BE A LITTLE BIT CAREFUL. WE KEEP ADVOCATING TO COLLECT MORE AND MORE AND THERE IS A COST TO COLLECTING MORE AND MORE. IT'S EVERYONE ALONG THE LINE, THERE'S A COST TO T. EVEN IN TERPS OF MAKING IT INTO A REPOSITORY, BECAUSE OF THE TIME, MONEY AND EFFORT THAT GOES INTO T. ASK WHAT YOU NEED AND REALLY REFINE THAT AND MAKE THAT PART OF THE EARLIER PHASE WORK. TO FIGURE OUT WHAT IS THE MINIMAL YOU NEED TO MEASURE. I THINK WE WEE DO COLLECT TOO MUCH DATA. THEY'RE HAVING TO RECORD THINGS THAT THEY DON'T NORMALLY RECORD IN EVERY DAY THERAPY. BUT IT IS CHANGING SORT OF THE WAY THAT THEY'RE INTERVENING. WE'RE ASKING THEM TO RECORD ALL OF THESE THINGS. HOWEVER, YOU WOULD ARGUE, THAT'S WHAT WE WANT. WE ACTUALLY WANT THEM TO BE DELIVERING THINGS, CONSISTENTLY SO I GO BACK AND FORTH. CAN I JUST MAKE A SIDE COMMENT, THOUGH, THAT I REALLY THINK WE NEED TO ADVOCATE, THAT PEOPLE WHO ARE INTERESTED IN DOING CLINICAL TRIALS HAVE A BIOSTATISTICIANS FROM THEIR TEAM, FROM THE VERY BEGINNING TO HELP THEM MAKE DESIGN DECISIONS. I SEE A LOT OF REHAB SCIENTISTS WHO ARE FLYING BLIND, AND FLYING BLIND AND BRINGING IN A STAT STATION LATER. JUST THIS ISSUE OF MONITORING, AND ADHERENS AND WHAT EFFECT COULD THAT HAVE AND SO O. SEEMS TO ME, PRACTICALALLY EVERYTHING THAT COMES UP DEPEND ON THE QUESTION OR THE STATE. SEEMS TO ME THAT IF YOU WANT TO SHOW, THAT THESE INGREDIENTS GIVE YOU A DIFFERENT AFFECT, THOSE INGREDIENTS YOU NEED TO MEASURE, PEOPLE REALLY GET DIFFERENT INGREDIENTS REGULAR OLD PT'S WHO DON'T SAY BEEPING CUING SYSTEM HAVE TO DELIVER THE TRIAL. YOU MIGHT GET A DIFFERENT ANSWER, AND THAT'S THE UNDERSTANDING THAT THAT EARLIER DESIGN WAS FILLING. I HAVE ONE MORE THING I WANT TO ADD. EVEN THOUGH I GET THAT IT'S HARD TO DO MEASURE A LOT OF DIFFERENT THINGS ABOUT FIDELITY. MAYBE WE NEED TO TALK ABOUT SOME KEY THINGS THAT WE NEED TO BE IDENTIFYING ACROSS TREATMENTS THAT COULD BE COMMON. SO FOR EXAMPLE, DOSAGE IS A HUGE ISSUE NOW. 20 YEARS AGO, NOBODY THOUGHT ABOUT DOSAGE AND REHABILITATION. I MEAN, THAT WAS A NONEXISTENT TERM. IF YOU SAID THE TERM DOSAGE, PHOBE EVEN KNEW WHAT YOU MEANT. IN TERPS OF REHABILITATION. SO MAYBE WE NEED TO THINK ABOUT SOME KEY TERMS. THAT WE THINK EVERYONE NEEDS TO DOCUMENT. I LOVE THAT IDEA. A SIMPLE GROCERY OF TERMS AND PROCESSES AND CONCEPTS THAT ARE GOING TO BE THREADED THROUGHOUT THIS DOCUMENT AND THAT'S AT THE TOP. THAT'S ONE OF THEM FOR SURE. >> I JUST WANTED TO MENTION, I SAW THE PUZZLED LOOKS WHEN DEAN PLACED IN FRONT OF YOU, ANOTHER PIECE OF PAPER AND THIS IS JUST A LEDGABLE VERSION OF THE NOTES THAT WE HAVE SPRAWLED UP ON THE BOARD. SO IF IT'S USEFUL, GREAT. IF NOT, L JUST LEAVE T. I ALSO WOULD LIKE TO MENTION, THE SLIDES WILL BE AVAILABLE IF YOU WANT COPY ON THE SLIDES SEE IDA AND THEY WILL BE POSTED FOR THE SPEAKERS. OTHER COMMENTS? THE NEXT COUPLE OF SESSIONS ARE REALLY, AS WE TALKED ABOUT YESTERDAY, THE NUTS AND BOLTS OF MAKING THESE TRIALS SUCCESSFUL. AND I THINK WE HAVE TO ADDRESS SOME OF THESE TOPIC IN THE WHITE PAPER. CARLA, LET'S JUST TAKE ABOUT THE CRITICAL STAFF. >> SO WE, OUR SESSION WAS PROBABLY THE BRIEFEST, BUT WE TALKED ABOUT CRITICAL STAFF, IN TERMS OF THEIR ROLES AND RESPONSIBILITIES, THEIR TRAINING AND THE BUDGET IMPLICATIONS, AND THOSE ISSUES ARE RELATED TO THE SIZE OF THE TRIAL YOU'RE RUNNING. IF IT'S A SINGLE SITE, IF IT'S A MULTI SITE, IF IT'S WHAT PHASE IT IS. SO I THINK THAT WAS THE ISSUE FOR PEOPLE THAT, YOU KNOW, DON'T KNOW HOW TO THINK ABOUT THESE ISSUES. THAT'S A NICE FORMULA. THE OTHER ISSUE RELATED TO, SO THERE WERE A COUPLE THINGS THAT CAME OUT, LIKE, BLINDING OF AN ASSESSOR, AND HAVING THIS PERSON BE INDEPENDENT FROM THE SITE, IF IT'S A MULTI SITE. SO THAT'S SOMETHING THAT PEOPLE MIGHT NOT THINK ABOUT OFF THE BAT THAT WOULD HELP. AND THEN THE FIDELITY ISSUE -- WELL, WHO DELIVERS THE INTERVENTION, THE ISSUE OF WHAT AGAIN, WHAT IS YOUR QUESTION. WHAT EFFICACY IS IT, EFFECTIVENESS. SO WHERE DO YOU SEE THIS INTERVENTION OPERATING, KIND OF. YOU HAVE TO THINK ABOUT THAT AHEAD OF TIME. IN ORDER TO TERM WHO WOULD BE DELIVERING THE INTERVENTION. AND THEN UPON FINALLY, THE FIDELITY ISSUE. I LOOKING AT FIDELITY DIFFERENTLY. I PUT IT INTO A MODEL, SORT OF JOHN'S MODEL WHERE I'VE GOT SOME INGREED IANTS, I HAVE A PUNITIVE MECHANISM THAT, I THINK IS OPERATING ON SOME TARGET AND SO I LOOKING AT FIDELITY AS CHECKING THAT MY HYPOTHESIS OF THE MECHANISM THAT IS OPERATING FROM MY INGREDIENT, THAT IS IN MY INTERVENTION, IS OPERATING THE WAY I THINK IT IS OPERATING. AS OPPOSED TO MONITORING DOSE. I MEAN, I THINK THAT'S IMPORTANT, BUT FIDELITY WOULD BE TRYING TO GET AT THE HOW YOUR INGREDIENT, THAT YOU ARE PUTTING INTO YOUR INTERVENTION, IS, WHETHER IT'S OPERATING IN THE WAY IN WHICH YOU THINK IT'S OPERATING AND I THINK THAT'S MAYBE A MORE SUBTLE WAY OF LOOKING AT FIDELITY, BUT I THINK IT'S IMPORTANT ONE, SOMEBODY TRYING TO THINK ABOUT, YOU KNOW, AN INTERVENTION SHOULD BE GUIDED BUT I THINK THOSE ARE THE MAIN POINTS CARLA MADE. I THINK IT'S IMPORTANT TO REMEMBER WHEN, YOU'RE CHOOSING THE RESEARCH CAREDNATOR FOR YOUR TRIAL. YOU'RE LOOKING TO SEE WHAT YOU ARE EXPECTING THAT PERSON TO D. IT DOESN'T ALWAYS HAVE TO BE A NURSE. WE'RE RIGHT NOW, LOOKING AT TRYING TO HIRE A NURSE WE ACTUALLY WANT TO DO THE MUSCLE BIOPSIES. SO WE'RE LOOKING AT A NURSE PRACTITIONER AND THAT NURSE PRACTITIONER WILL COORDINATE THIS TRIAL. SO I THINK IT'S IMPORTANT. NEED TO REALIZE HOW CRITICAL THAT IS. THE OTHER POINT WAS TRAINING AND CONTINUOUS TRAINING. ESPECIALLY ON THE REGULATORY SIDE. AND THE CERTIFICATIONS THAT I SPOKE OF WITH THE GROUP THAT IS I'M CERTIFIED W THE GOOD THING ABOUT THEM, SUHAVE TO RECERTIFY EVERY TWO TO THREE YEARS, AND YOU KNOW THAT WHEN THAT'S HAPPENING, FOR YOUR CLINICAL TEAM, THAT THEY'RE STAYING UP TO DATE ON THINGS. I THINK THAT'S ALSO SOMETHING AND IT WOULD BE NICE IF PARTICULAR BE AN INCENTIVE WITHIN THE DEPUTY F THEY GOT CERTIFIED, THERE WAS SOME KIND OF A BUMP-UP. IT WASN'T FOR ME. I PAID FOR IT MYSELF. BUT YOU DON'T REALLY GET A LOT PEOPLE TO BE MOTIVATE MOTIVATED TO PUT OUT THE MONEY TO DO THAT. SO THAT'S ANOTHER IMPORTANT PART. AND THE FINAL THING IS STANDARDIZATION OF PROCESSES. WHATEVER THOSE PROCESSES ARE FOR RUNNING THAT CLINICAL TRIAL, ESPECIALLY WHEN IT'S A MULTI CENTERED TRIAL. WE NEED TO TRY AND STANDARDIZE OUR DO YOU MEANS AND THINGS AND CHECKING OUR DATA, AND ONE OF THE THINGS I DIDN'T MENTION, BECAUSE I DIDN'T HAVE THE TIME. I CANNOT UNDERESTIMATE THE VALUE OF A CLINICAL TRIAL'S MANAGEMENT SYSTEM. A CTMS, ESPECIALLY WHEN YOU'RE DOING A MULTI CENTER STUDY ALLOWS YOU TO HAVE, AS A COORDINATORRING CENTER, STANDARDIZED DOCUMENTS OF EACH VISIT IT'S WEB BASED AND EVERYBODY GOES IN AND THEY GET IT AND THEY'RE FILLING OUT THE SAME THING AND THEY TURN IT IN. SO I DID NOT BRING THAT UP. I DO KNOW, WE HAVE TALKED ABOUT DATA AND DATA BEING CONSISTENT. I THINK THAT'S SOMETHING WE SHOULD MENTION. THE INTERVENTIONIST IS PART OF THE CRITICAL TEAM. SO WHETHER IT'S ACRITICAL THERAPIST OR SOMEBODY ELSE. WHOEVER'S DELIVERING T. WE TALKED ABOUT, HOW CRITICAL IT IS, THAT ALL OF THOSE PEOPLE, DO IT THE SAME WAY. AND IF IT IS AN EFFORT-BASED INTERVENTION, OR A DOSE-BASED INTERVENTION, THAT CAN VARY, BASED ON THE PARTICIPANT'S LEVEL OF EFFORT. YOU'VE GOT TO HAVE PEOPLE WHO UNDERSTAND HOW TO BE CONSISTENT SO THING, FOR PEOPLE IN THIS FIELD, THAT IS A HUGE CHALLENGE. THAT'S FAR AND AWAY, DIFFERENT THAN HAVING PEOPLE RESPONSIBLE FOR MONITORING WHETHER OR NOT THE PILLS WERE CONSUMED. AND I JUST THINK IT'S ABSOLUTELY CRITICAL. IF YOU START SEEING TRENDS IN THE MULTI SITE DATA SET, WHERE ONE SITE IS NOT AS DEMANDING OR RIGOROUS THAN THE OTHER. YOU'VE GOT A REAL PROBLEM. SO FOR ME BEING THAT'S CRITICAL FROM THE RESEARCH TEAM LEVEL AS WELL. I WOULD ALSO SAY, THESE CREDENTIALS IN WAYS OF MAKING SURE THAT AGAIN, I'M THINKING MULTI SITE. MAKE SURE YOU HAVE PEOPLE WITH THE SAME SKILL SET AT EACH SITE, OPERATING IN THAT FUNCTION, IS REALLY, REALLY IMPORTANT. >> THANK YOU FOR RAISE THAT. WHEN YOU HAVE A MULTI SITE TRIAL AND YOU'VE GOT PEOPLE DELIVERING AN INTERVENTION AND IT'S OVER A 4 TO 5 YEAR PERIOD, THIS HAS TO BE A MECHANISM FOR THEM TO TALK TO EACH OTHER. AND COORDINATE ACROSS SITES BECAUSE OF THE DIFFERENCES IN SITES AND THE NUMBER OF PEOPLE BE RECRUITED AT THOSE SITES. AND WE EVEN DEVELOPED A SORT OF GENERATE DISCUSSIONS, ASK RESPONSES, AND THAT WORKED VERY L. HAVE YOU TO CONSIDER THIS IS GOING ON FOR A NUMBER OF YEARS. THAT'S GREAT. YOU HELD UP YOUR SIGN, SO WE'RE JUMPING TO THE NEXT ONE. THE BIG ELEPHANT IN THE ROOM. RECRUITMENT AND RETENTION. SO WE TALKED A LOT ABOUT THE COMPLEXITY OF TRIALS INFLUENCING THE RECRUITMENT RETENTION AND THAT COULD BE CERTAINLY, A THEME THAT I THINK WE NEED TO, THAT WE COULD SEND A MANY TO MAKE SURE WE'RE REALLY EVALUATING, WHETHER OR NOT ADDING ALL THESE OUTCOME MEASURE SYSTEM REALLY THE RIGHT STRATEGY. WE ALSO, THERE WAS A NICE EMPHASIS ON KIND OF THE WORK THAT MONA'S DONE N TERMS OF RECUTEMENT OF MINORITIES AND POPULATIONS TARGET THOSE SUBGROUPS, WHICH I THINK SHOULD BE A BIG EMPHASIS AS WELL, BECAUSE OF THE UNIQUE AND SUCCESSFUL STRATEGIES THAT YOU'VE UNDERTAKEN AND HOW TO IMPROVE RECRUITMENT AND ALSO RETENTION. SO THERE MIGHT BE, I MEAN, THE WAY WE LAID OUT THE PRESENTATION, THE DISCUSSION THAT FOLLOWED AN OPPORTUNITY OF TABLE OF SORTS TO KIND OF IDENTIFY SOME OF THE BARRIERS OR THE FACILITATES WAS IDENTIFIED, THE GLASS HALF FOOL, DEPENDING ON HOW WE WANT TO FRAME IT AND THEN POTENTIAL SOLUTIONS. THERE WAS TALK ABOUT POTENTIALLY PUTTING AN APPENDIX TOGETHER OF WHAT ARE SOME STRATEGIES THAT OF COURSE SUCCESSFUL IN DIFFERENT CLINICAL SETTINGS. DIFFERENT CLINICAL STUDIES. JUST IDEA GENERATING. I LIKE THE IDEA OF A FORUM, IT WOULDN'T WORK FOR THE PAPER. BUT I KIND OF LIKE THE IDEA OF CREATING A FORUM WHERE PEOPLE WHO ARE INVOLVED IN CLINICAL TRIALS COULD SAY, I TRIED THIS STRATEGY AND IT WORKED. HE WAS THE CHALLENGE WITH THIS PARTICULAR STRATEGY, INSTEAD OF ALL OF US TRYING TO E-INVENT THE WHEEL INDEPENDENTLY. IN ANY DEVELOPMENT OF A CLINICAL TRIAL. YOU PUT THE DESIGN. YOU NEED, YOU KNOW, TOO DESIGN A RECRUITMENT PLAN. THE OTHER THINGS, WE HAD ONLY ONE SLIDE OBUT IS VERY IMPORTANT IS MONITORING. THE IMPORTANCE OF MONITORING RECRUITMENT, WITH TIMELINE AND HAVING SHORTED AND LONG GOALS BECAUSE IT REALLY IMPACTS THE MANAGEMENT OF THE STUDY SO YOU CAN HAVE A CLINIC WITH NURSES AND STAFF. AND ONE WEEK WITH EVERYBODY AND ONE WEEK WITH EVERYBODY COMING IN. SO THE MONITORING WOULD BE VERY, VERY IMPORTANT. THE OTHER THING IS TO HAVE RESOURCES FOR RECRUITMENT AND BECAUSE A LOT OF TIMES, THEY'RE NOT GOING TO BE, THEY USUALLY DON'T DO THAT. AND THEN HOW YOU CAN ENGAGE THE HEALTH PROVIDERS AND THE DIFFERENT SECTORS. IN YOUR RECRUITMENT. YOU NEED TO ENGAGE YOUR HEALTH PROVIDERS, YOU NEED TO ENGAGE THE SYSTEM AND YOU NEED TO GENERAL THE COMMUNITY, ALL GREAT COMMENTS. I THINK IN FACT, WHAT YOU JUST SAID ABOUT THE UP-FRONT RECRUITMENT PLAN WITH EXPERTS, WHETHER OR NOT THERE'S A CORE -- IT'S JUST AS THE UP FRONT MEETINGS WITH GARY CUTTER. I WAS ACTUALLY SURPRISED, DARCY WHEN, YOU SAID YOU'RE SEEING A LOT OF APPLICATIONS WITHOUT ACUTER PHENO TYPE. I MEAN, I DON'T KNOW. THE PANELS I'VE BEEN OTHAT'S JUST, THEY WERE NEVER GOING TO MAKE IT TO THE TABLE. SO IT'S INTERESTING. BUT THEY'RE COMING IN. SO THAT MEANS, THERE'S A NEED. THERE'S A SERVICE WE CAN PROVIDE. AND SO I THINK BUT WHAT IS DIFFERENT, I THINK MANY OF US, WE PROBABLY ALL WOULD OBVIOUSLY AGREE, THAT YOU NEED THE BIOSTATISTICIANS UP FRONT. BUT WHAT I'VE LEARNED YOU NEED THE RECRUITMENT FRONT. WE DID IN THE MOST RECENT APPLICATION WE HAVE TOGETHER, BUT IN THE PAST, NO. I THINK THAT WOULD BE A GREAT SERVICE IN THIS WHITE PAPER. YOU KNOW. EVEN PROVIDING RESOURCE F THEY DON'T EXIST AT THEIR INSTITUTION FOR THAT. THESE ARE CRITICAL DESIGN OPPORTUNITIES, TOO REALLY ENHANCE THEIR APPLICATIONS. >> WITH ALL DUE RESPECT TO GARY, MIGHT BE MORE IMPORTANT BECAUSE IF IF THERE ARE NO PEOPLE, THERE ARE NO DATA. >> LATIN COOPER, NHLBIHAS OUT, BOTH FOR MULTI SITE TRIALS AND NOW, VERY RECENTLY FOR SINGLE SITE TRIALS SPECIAL ANNOUNCEMENTS AND PARTICULARLY, FOR THE MULTI SITE TRIALS, THERE ARE A LOT THESE ISSUE THAT IS HAVE TO BE DEALT WITH UP FRONT, BEFORE THEY CAN COME IN BECAUSE WE JUST HAD TOO MANY DIFFICULTIES TOO MANY DIFFICULT EXPERIENCES, AS I'M SURE A LOT OF THE IC'S HAVE. SO ANYTHING COMING TO NHBIAND I THINK IT'S GOING TO BE NIH-WIDE OVER A SHORT PERIOD OF TIME. YOU'RE GOING TO SEE THESE THINGS HAVE TO BE ADDRESSED. I KNOW YOU'RE AWARE THAT NIH HAS ISSUED THE NO MORE SMALL CRAPPY TRIALS AND THAT'S NICE. ALL THE INSTITUTES WILL ACCEPT CLINICAL TRIALS. NO MATTER WHAT PHASE. ONE PERSON, ONE INTERVENTION. CONTROL GRIPE, SURE THING. BUT SO ANY CLINICAL TRIAL FOR NIH WILL HAVE TO HAVE ALL OF THESE PLANS. WE'LL HAVE TO HAVE A PROTOCOL. WE'LL HAVE TO COME IN INTERAN FOA. THAT'S GOING INTO EFFECT NEXT FALL. AND THE ANNOUNCEMENTS ARE IN THE BACK IF YOU WANT TO READ THEM. THOSE LINES WILL CERTAINLY BE A SERVICE. >> I ACTUALLY WANTED TO FOLLOW THAT. BECAUSE I THINK A SECTION OF THIS SHOULD INFORM A SUMMARY, ALL THE CHANGES AT NIH. I MEAN, WE HAVE TO HAVE THAT IN THERE PROBABLY UP 41. LET THEM KNOW, MAYBE BEFORE THE HEAROG MPR WHAT THE RESULTS OF THE TRIAL WERE, WHAT THEIR ENTERVENTION WAS AND THE IMPACT THAT THAT WORK HAD BECAUSE THEY VOLUNTEER AND I THINK IT HELPS THAT NETWORKING OF HAVING THEM THINK ABOUT CLINICAL TRIALS AGAIN, BECAUSE THEY HAD THAT EXPERIENCE. SO I'D LOVE TO SEE THAT EMPHASIZED. AND RELATED TO THAT, TOO. MAYBE THAT CAN BE ONE OF THE RECOMMENDATIONS TO NIH THAT IT BECOMES MORE OF A MANDATE THAT THE CLOSE-OUT, WHICH HAPPENS IN SOME CASES, MORE RIGOROUSLY THAN OTHERS, INCLUDING THAT PIECE. THAT'S A REQUIREMENT. >> I DON'T REMEMBER HOW THE DEFINITION OF CLINICAL TRIAL IS WRITTEN. BUT I'M DEALING WITH THE FACT, ONE OF THE POINTS I WAS GOING TO MAKE ABOUT OUR SESSION, MAY OR MAY NOT CONFLICT WITH THAT DEFINITION. YOU KNOW, WE TALKED ABOUT HOW IT'S NOT A NICE LINEAR SEQUENCE, SO I WAS GOING TO ADVOCATE FOR A SORT OF PURPOSE-BASED CLASSIFICATION. A WHAT DOES THIS THING LOOK LIKE. BUT I DON'T REMEMBER, IN OTHER WORDS, I WILL NEED TO DO SOME RANDOMIZED DESIGN TO JUST FIGURE PROOF OF CONCEPT. AM I NOW GOING TO BE HELD TO A WHOLE BUNCH OF STANDARDS ABOUT CLINICAL TRIALS. I WOULD ADVOCATE, THINKING ABOUT THE FRAMING, WHEN THOSE REQUIREMENTS KICK IN AROUND WHAT THAT STUDY IS DESIGNED TO W. BUT I DON'T KNOW WHETHER THE HORSE IS ALREADY OUT OF THE PARTNERSHIP, IN TERMS OF BEING ABLE TO ADVOCATE THAT PERSPECTIVE. >> I JUST WANT TO FOLLOW UP. THE FEEDBACK NEEDS TO GO TO THE PATIENTS AND TO THE PROVIDERS THAT REFER THE PATIENTS. I THINK I BROUGHT THIS YESTERDAY AND IT'S NOT ONLY FOR RECRUITMENT PIECE, BUT FOR THE WHOLE THING. WE NEED TO LOOK AT THE ETHICAL AND LEGAL ISSUES. PROBABLY, WEN'T DISCUSSED THIS, IN THIS WORKSHOP, BUT THERE ARE EXPERTS ALSO. IN DOING THAT, BECAUSE PROBABLY, THIS IS GOING TO BE PART OF THE REVIEWS THAT YOU WOULD HAVE TO HAVE SOMEBODY LOOKING AT LEGAL ISSUES. AND SOCIETAL IMPLICATIONS FOR CLINICAL TRIALS. SO FOR DATA COLLECTION, I THINK SOME OF THE MANIES WE WERE TRYING TO PUT FORTH IS TO KEEP THE DATA CHECKED LEAN, AND AS CLOSE TO THE SOURCE AS YOU CAN. DEPENDING ON WHAT TOOLS THAT YOU HAVE AVAILABLE TO IT. AND FOR LEAN, I THINK WE HEARD IT ACROSS MANY SESSIONS, AND I THINK THAT WILL BE INTERESTING HOW TO ORGANIZE THIS. TOPICS ARE RELEVANT TO MANY OF THE OTHER SECTIONS, TOO. AND USING A WEB-BASED SYSTEM ALLOWS REAL TIME DATA SPREE AND FEEDBACK. IT HELPS LIKE, AS YOU WERE SAYING, A CLINICAL TRIAL MANAGEMENT SYSTEM. IT ASK HELP WITH SCHEDULING, WHICH WILL DECREASE MISSING DATA SO QUALITY BY DESIGN, IT'S NOT JUST THE DESIGN OF YOUR PROTOCOL AND IMPLEMENTATION, BUT YOUR DATA SYSTEM CAN HELP TO, TRY TO MINIMIZE MISSING DATA AND INCREASE THE QUALITY. THE INTERVENTIONIST, AS WELL AS THOSE MEASURING OUTCOMES. I THINK WHEN WE TALK ABOUT, YOU KNOW, THE SHINY OBJECT IT IS, ELECTRONIC HEALTH RECORD IS ONE FOR GOOD REASON. WE DON'T WANT PEOPLE TO HAVE TO KEY IN MULTIPLE SOURCE. WHEN WE DO USE THOSE, BE THOUGHTFUL ABOUT THE MOTIVATION FOR WHAT'S BEING PUT INTO THE ELECTRONIC HEALTH RECORD AND IS IT REALLY CONSISTENT WITH WHAT YOUR NEEDS ARE AS A RESEARCHER. WERE YOU REALLY GET THE DATA THINK YOU ARE, WHEN YOU PULL THINS OUT OF EHR. I THINK ANOTHER PLEA IS FOR RESEARCHERS TO START WORKING TOGETHER TO STANDARDIZE THE WAY THEY COLLECT THEIR DATAA, TO THE EXTENT POSSIBLE. CLEARLY, THERE'S INSTRUMENTS OUT THERE, THAT OF COURSE VALIDATED, BUT THERE'S ALSO THINGS LIKE CONCOMITANT MEDS. DEMENTIA GRAPHIC. CAN WE PUSH TOWARDS THE FDA MODEL OF AN SDM. SO THAT WE CAN SHARE DATA MORE SEAMSLY, AFTER TRIALS AND WE HAVE LESS START UP. BECAUSE TEASE INSTRUMENTS HAVE ALREADY BEEN USED, VALIDATED, YOU KNOW WHAT TO NAME THE VARIABLE. YOU KNOW WHAT THE RANGES ARE. AND YOU JUST DON'T HAVE TO KEEP REINVENTING THE WHEEL. ING BE A GREAT RESOURCE, NEW INVESTIGATOR COULD COME TO YOU ALL FOR, YOU KNOW, WHERE'S THE LIBRARY OF DATA FORMS OUT OF THE BOX THAT I CAN USE AND THAT GETS A LITTLE TO THE NEXT SESSION FOR ADVERSE EVENT REPORTING. IT SHOULD BE ONE AND IT SHOULD EASILY LEAD THE INVESTIGATORS TO GO. SAY AN A.E. -- IT MEETS THE PROTOCOL DEFINITION. SO WE HAVE TO BE USING OUR TECHNOLOGY TO MAKE OUR TRIALS MORE EFFICIENT. >> IF FLOWER NO COMMENTS, WE CAN MOVE AHEAD. I THINK THAT'S A GREAT SUMMARY. CAN YOU WRITE THAT UP TONIGHT? >> NO PROBLEM. NO MATTER WHAT KIND OF STUDY YOU'RE DOING, YOU NEED TO HAVE A PLAN BEFORE YOU START, OF HOW YOU'RE MONITORING SAFETY AND WHETHER THAT'S WITH AN INDEPENDENT DATA AND SAFETY MONITORING BOARD, A SAFETY COMMITTEE, A MEDICAL MONITOR OR EVEN THE P.I. DOING T. YOU NEED TO WRITE IT DOWN AND IT SHOULD BE ASSESSED BY SOMEBODY ELSE TO MAKE SURE THAT YOU'RE COVERING THE WELFARE OF THE SUBJECTS. IT'S GOT TO BE CLEAR WHAT EACH PARTY'S ROLE S WHAT IS THE INVESTIGATOR'S ROLE IN EVALUATION, IN TRI ANNUALING UPEXPECTED ACCEPTS THAT HAPPEN, WHILE IN A STUDY. SO THAT AT THE END OF THE DAY, THE PARTICIPANT SAFETY IS COVERED. AND THEN WITH TECHNOLOGY, WE CAN ACTUALLY PROGRAM THOSE RULES IN, SO WHEN THERE'S A REPORT OF AN EVENT, IT CAN AUTOMATICALLY GO TO THE PEOPLE WHO NEED TO KNOW AND REPORTS CAN BE GENERATED AND SO THAT WE MAKE SURE THE IRB'S ARE INFORMED PROPERLY. YOU'VE BROUGHT UP A GOOD POINT THOUGH. WE HAVE GOT ISSUES WITH IRB'S, ASKING FOR DIFFER THINGS AND CAN WE PUSH TO STREAMLINE THAT ACROSS OUR STATES AND INSTITUTIONS. CENTRAL IRB IS ONE METHOD. BUT THERE MAY BE OTHERS AS WELL. WE DID A LOT OF DISCUSSION ON INSURING THE INDEPENDENCE OF THE DSMB AND EXACTLY, I THINK YOU BROUGHT IT UP, JOHN, -- WHEN A DMB IS NEEDED AND WE HAVE SOME INPUT FROM SOME OF OUR NIH COLLEAGUESED IN AUDIENCE AS WELL. BUT IT REALLY DEPENDS ON YOUR TRIAL. IT DEPEND ON THE RISK OF THE SUBJECT, THE MULTI CENTER NATURE, PERHAPS AND THE STAPLE OF YOUR TRIAL. AND WHETHER DSNB SHOULD BE MASTER. GARY AND I FEEL STONNINGLY, THEY SHOULD BE UNMAST, ALTHOUGH THE REPORT MAY APPEAR MAST, JUST TO PREVENT IT FROM BEING SEEN ON AN AIRPLANE. BUT THERE ARE, OPINIONS IN THE FIELD. SO WE SHOULD COME TO CONSENSUS ON THAT. GARY AND I HAVE OUR BIAS. CERTAINLY, I WOULD AGREE WITH THE POSITION TAKEN BY BY YOU AND GAR. CAN YOU TELL ME WHAT THE ARGUMENTS ARE FOR KEEPING THE DMV MAS, I JUST DON'T GET IT. >> I'LL TELL WHAT YOU I'VE HEARD AND GARY HAS, THINGS I'M SURE HE'S HEARD. YOU CAN OVERCOME VARIABILITY. YOU CAN'T OVERCOME BIAS. AND SO THAT'S THE CONCERN PEOPLE HAVE WHEN WE GET INTO THE INDEPENDENT STATISTICIANS. COULD THEY BE INTRODUCING BIAS, IF YOU IF HAVE YOU SOMEONE VIVID IN A TRIAL, MAKING DECISION THAT IS IMPACT THE DESIGN WHEN THE STUDY IS ON GOING. SO THAT'S THE ARGUMENT PEOPLE MAKE FOR A STATISTICIANS. IF THEY HAVE KNOWLEDGE OF THE OUTCOMES, WILL THAT INFLUENCE WITH A NOD AIR, WINK OR OTHER. THE INVESTIGATORS IN CONDUCTING THE TRIAL. SO DSMB. SAME THING IF THEY HAVE KNOWLEDGE OF THE OUTCOME. WILL THAT SOMEHOW, CLOUD THEIR DECISION MAKING SUCH THAT THEY MAKE A RECOMMENDATION FOR A DESIGN CHANGE, MID TRIAL, BUT THEY ALREADY KNOW THE OUTCOMES, THEY HAVE SEEN THE TRENDS AND THAT MIGHT PUSH THEM TO SAY, YOU KNOW, YOU NEED TO ENROLL MORE OF THESE PATIENTS, AND THE DSMB MAY-RECOMMEND THIS. AND TO ME, THAT'S NOT A GOOD ARGUMENT BECAUSE IF, THE DSMB SHOULDN'T REALLY BE TALKING ABOUT DESIGN CHANGES. THAT SHOULD COME FROM THE INVESTIGATORS. AND IF THERE IS SOMETHING LIKE THAT, COMING UP IN THE TRIAL, YOU CAN ALWAYS CONVENE ANOTHER INDEPENDENT GROUP TO LOOKING AT THAT ISSUE. SO I DON'T BUY THAT ARGUMENT. AND I THINK THE HARM DONE IN NOT UNMASKING THEM, FAR OUT OUTWEIGHS ANY RISK OF THEIR BIAS OF DISCUSSION OR PROVIDING RECOMMENDATIONS TO INVITE V. IRB IS DEPENDING ON THEM, AS THE LAST RESORT. THEY ARE A SEE THE TOTALITY OF THE DATA AND TO EFFECTIVELY WEIGH THE POTENTIAL RISKS AND BENEFITS OF ANY KIND OF EFFICACY SIGNAL WITH A SAFETY SIGNAL, YOU'VE GOT TON WHAT YOU'RE LOOKING AT AND NOT HAVE IT LOOK HAPHAZARD TO YOU. I'M NOT AS FUNDAMENTAL ABOUT THE BLINDED STATISTICIANS. WE HAVE TO REAL I'D AND LOOKING AT WHAT'S AFFORDABLE. THERE ARE A LOT OF NICE THINGS THAT CAN HAPPEN. BUT WE CERTAINLY, YOU KNOW THE BLINDED AND UNBLINDED STATISTICIANS WAS REALLY DIRECTED AT DRUG COMPANIES THAT COMPANIES THEMSELVES WOULDN'T HAVE ACCESS TO THE ONGOING DATA TO MAKE A DECISION, MID COURSE, THAT MAY EFFECT THE INABILITY ABILITY TO SEAT DATA THAT IS, THEY JUST CANCEL S. AND IT'S CARRIED OVER INTO MANY OTHER THINGS. IN AN IDEAL WORLD, YOU MIGHT SAY THAT IS OKAY. YOU CAN AFFORD THAT THERE'S AN ADVANTAGE WITH THE STATISTICIANS, BEING INVOLVED WITH THE PROCESS AND COLLECTION. THEY HAVE A GREATER KNOWLEDGE OF WHAT THOSE TABLES ACTUALLY SHOW AND WHAT THEY MEAN AND WHERE THERE ARE POTENTIAL GLITCH IN THE DATA THEMSELVES THAT, MAKE YOU INTERPRET THEM SLIGHTLY DIFFERENT THAN YOU WOULD HAVE, IF YOU HAVE AN UNINVOLVED PERSON WHEN, ALL OF A SUDDEN, GOT A FAST FILE AND GENERATE SOME TABLES. THE BLINDING, COME FROM CLINICIANS. YOU WANT TO HAVE AN UNBIAS ASSESSMENT OF THIS AND YOU DON'T OVER REACT. AND I THINK THERE IS A DANGER, BUT I THINK YOU SHOULD ALWAYS GET EXPERIENCED PEOPLE ON DSMB'S, WHO DON'T STOP THE STUDY BECAUSE THEY SAW TWO OF SOMETHING. AND IT DOES HELP WHEN YOU'VE SEEN TWO OF SOMETHING, TOO FIND OUT WHAT GROUP THAT'S IN. BECAUSE YOU CAN OFTEN BE SURPRISED AS TO WHAT GROUP IT'S IN. BUT IT'S THE SINGLE EVENT, THAT IS SO UNUSUAL AND IN THE TREATMENT GROUP THAT CAUSE TO YOU LOOK SO MUCH. I THINK YOU SHOULDN'T BE BEELINED F. YOU MAY WAIT TOO LONG, WAITING 6 MONTHS AND THERE COULD BE OTHER CASES YOU COULD HAVE REPRESENTED. WE'RE GOING TO WRITE A WHITE PAPER WITH SUGGESTIONS, RECOMMENDATIONS, FUTURE DIRECTIONS, THINGS THAT HOPEFULLY, THE READER CAN LEADERSHIP FROM AND WHEN IT COMES TO DSMB'S, IF IT'S NIH-SUPPORTED, JOE AND I TALKED YESTERDAY DURING A BREAK. THE DSMB IS NOT REPORT TO THE INVESTIGATOR. THEY'RE AT THE, YOU KNOW, THE NIH IS BASICALLY, A POINTING THE DSMB MUCH THE DSNB IS REPORT TO THE NIH I THINK THE OTHER POINT WE WANT TO MAKE IN OUR SECTION S THE NEWER INVESTIGATORS NEED TO GO TO THE INSTITUTE RESOURCES FOR GUIDANCE, AS TO WHAT THAT INSTITUTE WOULD LIKE TO HAVE, BASED ON THE RISKS OF THEIR STUDY, AND TAKE ADVANTAGE OF THE POTENTIAL DSMB'S THAT MIGHT SIT IN THEIR CONTRIBUTE TA'S OR OTHER CTSA'S THAT HAVE THE EXPERIENCE GARY'S TALK B. TO HELP THEM ALONG IN THE PROCESS. >> IF IT'S SOMEBODY WHO SAYS, I WANT TO DO A TRIAL. THIS PAPER SAY RESOURCE FOR ME TO HELP ME, HAVE SOME GUIDANCE, WHAT WOULD BE MOST HELPFUL IS SORT OF A FLOW DIAGRAM. THIS IS GOING TO REQUIRE DSMV. THIS WILL PROBABLY REQUIRE AN INDEPENDENT STUDY MONT. ULTIMATELY, IT WILL BE THE NIH'S DECISION BUT IF YOU FOLLOW DOWN THIS PATH. YOU GO ON THIS BRANCH OF THE TREE. HERE'S THE SUGGESTED APPROACH TO WRITING A DSMB PLAN. IF YOU HAVEN'T BUDGETED FOR FOR T YOU'LL STILL PAY FOR T. THAT'S WHY WE NEED TO DO ORIGINAL ANALYSIS BEST. GIVE THEM A DECISION TREE, BASED ON THE NIH GUIDELINES, THAT WILL HELP THEM UNDERSTAND WHAT TO BUDGET FOR, HOW TO LAB AND SO FORTH. TELL BE IN THIS PAPER. BECAUSE I AGREE WITH THEM, BECAUSE OF THE INDEPENDENT STATISTICIANS. I THINK WE GET THESE GUIDELINE AND THEY BECOME LAW. IT'S A GUIDELINE, THEREFORE, YOU'VE GOT TO HAVE AN INDEPENDENT UPON STATISTICIANS. AND FOR SOME TRIALS, YOU REALLY DON'T NEED IT. WHEN YOU WAIVE THE COST. I'M GOING TO MAKE A RADICAL COMMENT, AND I THINK WE NEED A PREAMBLE TO THIS DOCUMENT THAT SAYS, YOU KNOW, WHY DO YOU WANT TO DO A CLINICAL TRIAL. PEOPLE ARE GOING INTO THIS NOT BECAUSE -- I MORNING I HEAR PEOPLE TALKING. I WANT TO DO A CLINICAL TRIAL. I WANT TO SEE IF MY INTERCEPTION WORKS AND THAT'S AS FAR AS THEY HAVE GONE. OR THAT, YOU KNOW, I WANT TO DO A CLINICAL TRIAL BECAUSE IT'S MORE MONEY. AND THESE ARE RIDICULOUS IDEAS. WE NEED TO MAKE SURE PEOPLE HAVE THOUGHT THROUGH THE RATIONAL FOR WHY THEY WANT TO DO A CLINICAL TRIAL BEFORE THEY GET ALL THE FORMULA THERE HAS TO BE A BETTER RATIONAL TO THAT. >> COMING FROM CALIFORNIA, I WAS HOPING YOU'D BE A LITTLE BIT MORE RADICAL. THE I THOUGHT THE PREAMBLE WAS YOU DO NOT WANT TO DO A CLINICAL TRIAL. LET GO TO OUTCOME STAGES OF DEVELOPMENT. DANIEL AND JOHN. >> ONE QUESTION I HAVE, IS WHETHER THERE'S ANY PART OF THIS WHERE IT'S APPROPRIATE TO SPEAK TO NIH. BECAUSE IT SEEMS TO ME, THERE ARE A COUPLE OF THINGS, THAT WE DISCUSSED HERE, HAVE INTERNAL IMPLICATIONS. ONE I SEE IT AS PRETTY PROBLEMATIC AND PRETTY AMBIGUOUS FOR EARLY PHASES OF RESEARCH. LET ME GIVE YOU AN EXAMPLE FROM MY WORK. DIA STUDY WHERE I EXPOSED A BUNCH OF UNCONSCIOUS PATIENTS TO A SINGLE DOSE OF AMBIEN OR PLACEBO MY SPECIFICALLY WAS TO FIND OUT WHAT THE RESPONSE RATE WAS IF I WAS GOING TO DO ANY KIND OF FISHING STUDY. I NEEDED TO KNOW IS THIS ONE IN A MILLION PEOPLE, TWO PEOPLE, WHATEVER. I DIDN'T DO ANY FOLLOW-UP, CLINICAL INTERVENTION. I HAVE NO IDEA WHAT BENEFIT THAT HAD. OTHER THAN THE THREE, FOUR HOURS THEY STUDIED THEM. BUT I COULD READ THE DEFINITIONS. PATIENCE WERE RANDOM USED. THEY WERE PROSPECTIVELY ASSIGNED, AN INTERVENTION, ACCORDING TO THIS DEFANGED, AND I LOOKED AT A HEALTH-RELATED OUTCOME CONSCIOUSNESS. SO I DON'T THINK WE WANT THAT STUDY TO BE CLASSIFIED AS A CLINICAL TRIAL. I DON'T THINK I WANT TO MAIL EVERYBODY AT THE END OF THE TRIAL AND HAVE TO DESCRIBE THE BENEFIT OF THAT INTERVENTION, BECAUSE I DON'T KNOW THE ANSWER TO THAT QUESTION. ET CETERA. ET CETERA. SO THAT'S WHY I WOULD GO BACK TO, I THINK WE WANT TO DEFINE, WHEN THERE ARE MANDATED RULES ABOUT WHAT YOU NEED TO DO FOR CLINICAL TRIALS, I THINK WE NEED TO DEFINE CLINICAL TRIALS, BASED ON THE SOMETHING LIKE THE GOAL OF THE TRIAL, NOT THERE WAS RANDOMIZATION OR SOMETHING. >> [SPEAKING AWAY FROM MICROPHONE] >> IT WAS DONE UNDER THE DIME OF AN AGENCIES THAT MAKE UP A DUMB RULE. SO THE OTHER THING THAT I'M THINKING B YOU KNOW, I RAISED EARLIER WHETHER WE WOULD HAVE ACTUAL RECOMMENDATIONS IF YOU'RE DOING THIS, YOU SHOULD USE THIS DESIGN, AND CLEAR, SOMETHING HIGHLY PRESCRIPTIVE DOESN'T SEEM REALISTIC WANT. BUT I AM WONDERING, AS I'VE HEARD THE OTHER GROUPS. IT WAS VERY OFTEN SAID, IT DROPPED WHAT STAGE OR IT DEPENDS ON WHAT YOU'RE TRYING TO DO. I WONDER IF IT WOULD BE USEFUL FOR US TO CREATE A SORT OF MACRO TABLE PROOF OF CONCEPT, SORT OF EARLY EXPLORATION. DEFINITIVE EFFICACY. DEFINITIVE EFFECTIVENESS AND ASK EACH OF US WHEN WE'RE SAYING YOU SHOULD THINK ABOUT A BUMP OF DESIGNS. CAN WE SAY IF IT'S MORE LIKE THIS. USE THIS. MORE LIKE THAT, USE THAT. SUCH THAT THERE WILL BE A SIMILAR FRAMEWORK. THAT WAS ASKING US, ALL TO THINK ABOUT, CAN YOU SAY ANYTHING SYSTEMATIC ABOUT HOW YOUR TOPIC WOULD VARY, DEPENDING ON THAT SORT OF MATURATION OR TRAJECTORY TRAJECTORY. I THINK I WOULD LIKE TO HAVE US TALK ABOUT PURPOSES OF PHASES OF RESEARCH, AND HOW THE OUTCOME MEASURES MIGHT AND SHOULD CHANGE, DEPENDING ON THAT. I'D ALSO LIKE TO RAISE THE TOPIC OF HOW YOU GO ON TO STUDY THE LARGER FUNCTIONAL IMPACT OF YOUR FOCUSED INTERVENTION, BUT I DON'T EXACTLY KNOW WHAT TO SAY THERE. I THINK IT'S AN UNANSWERED QUESTION TO A LARGE EBBING TENT IN MY OWN MIND. THAT MAY BE A SECOND QUESTION ABOUT NIH, WHETHER THAT WOULD BE SOME KIND OF TOPIC FOR A METHODOLOGY WORKSHOP OR SOMETHING TO, THINK ABOUT HOW WE AGGREGATE THE REHAB INTERVENTIONS WE HAVE TO STUDY, INTO BUNDLE THAT IS CAN HAVE A BIG ENOUGH IMPACT WE CAN PICK UP WITH MORE MACRO MEASURES, WHAT THAT WOULD LOOK LIKE, SOMETHING LIKE THAT. >> I WOULD LIKE TO JUST ADD SOMETHING. FOLLOWING IN WHAT VAUGHN JUST TALKED ABOUT, I THINK BECAUSE THIS IS AN INTENDED FOR A MEDICAL REHABILITATION CLINICAL TRIALS AUDIENCE, CERTAINLY COULD BE USED MORE BROADLY, BUT THE BENT IS TO HELP INVESTIGATORS IN THE FIELD. WE NEED TO TALK ABOUT WHAT IS SIMILAR AND WHAT IS DIFFERENT ABOUT THE PHASING OF CLINICAL TRIALS IN THIS SPACE VERSUS THE PHASE 1-2-3 DRUG TRIAL. AND THAT'S AN PENITENTIARY DIFFERENCE THAT SHOULD COME FORWARD. AND I MEAN, ONE QUESTION, I DON'T KNOW IF I AGREE WITH THIS, DANIEL. BUT WOULD WE WANT TO GO ON RECORD AS SAYING, DON'T ONSERVICE OVER TRYING TO TRYING TO DECIDE YOU ARE PHASE 2 OR PHASE III. IN OTHER WORDS IS WHAT WE'RE ASKING, HERE ARE THE PHASES, TRY TO SLOT YOURSELF IN, OR ARE WE SUGGESTINGA AN ALTERNATIVE WAY OF FRAME WHAT YOU ARE ABOUT, AT ANY GIVEN POINT IN TIME. I DO THINK THE WAY NIH IS STRUCTURED. I HAVE A PHASE 2 STUDY AND IN THE PHASE 2 STUDY, IT IS DEFINITELY NOT A PHASE III STUDY. AND I THINK THAT WOULD BE A DIFFERENT KIND OF PAPER SO I AGREE WITH EVERYTHING YOU'RE SAYING BUT PARTLY WHAT YOU'RE SAY SUGGEST A VISION OF THE FUTURE. AS OPPOSED TO A PAPER WHICH HELPS PEOPLE NAVIGATE THE CONCERN SYSTEM. I THINK WE NEED TO BE VERY CLEAR WHAT WE'RE TRYING TO D. ARE WE HELPING PEOPLE NAVIGATE THE RULES IF THEY ARE. I ADOPT TO MAKE AN OFF THE RECORD COMMENT. >> NOTHING'S OFF THE RECORD. YOU HAVE A MICROPHONE AND IT IS BEING RECORDED. >> OKAY. WELL, THAT'S FINE. [LAUGHTER] >> A CERTIFICATE OF CONFIDENTIALITY. >> SO ANY TIME YOU'RE BOARD AND YOU HAVE ABOUT 15 MINUTES, GO AND READ ALL THE DIFFERENT EXAMPLES WHICH ARE PROVIDE OF WHAT MAY OR MAY NOT BE A CLINICAL TRIAL. THEN SEE IF YOU GET THEM ALL CORRECT. AND I GUARANTEE, THAT NOT EVERYBODY IN THIS ROOM WILL GET THEM ALL CORRECT. AND THEN WHEN YOU'RE WRONG, YOU ASK YOURSELF WHY WAS I WRONG AND THEN TRY TO FIGURE THAT OUT. BECAUSE IT IS HARD AND LET ME ACTUAL, THERE'S VERY BRIGHT PEOPLE IN THIS ROOM, WHO HAVE FAILED THAT TEST. >> SO I WANT TO FOLLOW UP ON THAT. MANY THINGS I WANT TO SAY. AND VERY GOOD POINT. WE HAD AN INTERNAL DISCUSSION ON CLINICAL TRIAL DEFINITION. SO I'LL COME BACK TO THAT. AND I REALLY WANT TO EMPHASIZE WHAT CAROL LEE SAID. THIS IS SOMETHING I'VE BEEN DOING AND WALK PEOPLE, AND IN MOST OF THE CASES, I'VE TALKED PEOPLE OUT OF DOING CLINICAL TRIAL. AND IT'S SIMPLY BECAUSE, [INDISCERNIBLE] INVERTEBRATE ANIMAL FROM MARCHINGY TO HUMANS, MANY OF THE THINGS WE TALK ABOUT HERE S INTERVENTION AND BEHAVIORAL INTERVENTION, FOR NEUROLOGY, IT'S A BRAIN BASE AND WE SIMPLY DON'T KNOW ENOUGH ABOUT HOW BRAINWORKS. SO THIS ONE WAY TRANSLATION, FROM PHASE 1, IT WOULD INN WORK FOR US. IN A PERFECT WAY. UNLESS WE STARTED TO LOOKING AT SORT OF BASIC SCIENCE QUESTION IN HUMAN. WE CANNOT WAIT FOR THE ANIMAL MODEL TO BE DEVELOPED THAT YOU THESE TRANSTRANSLATIONS. SO GO BACK TO THE CLINICAL TRIAL DEFINITION. THE RESULT I SEE IS COMING, IS GOING TO KILL US BECAUSE WE DON'T HAVE ENOUGH TO DO THESE PERFECT TRIALS, AS WE TALK ABOUT. AND THE DEFINITION WILL NOT ALLOWED US TO DO IN MY VIEW, IT'S NOT LIKE PHASE 1, 2, 3. IT'S A PHASE -1. PHASE -1. USING HUMAN SUBJECT AS CLINICAL RESEARCH TO ASK SOME FUNDAMENTAL QUESTIONS. WE CAN DO THAT. MANY OF THE STUDIES WE FUNDED. WE ASK WRITE IF A WAY THAT ASK SPECIFIC QUESTIONS -- TO ALLOW US TO ANSWER SPECIFIC QUESTIONS. THEY SAY, COMPARE GROUP A VERSUS B. I'VE BEEN TALKING WITH MANY OF YOUR STUDENTS, GREAT FOR CONDUCTING ALL THESE TRIALS, AND ASK THEIR EXPERIENCE. DO YOU WANT TO REPEAT THAT TRACK OR DO YOU WANT TO MAKE A CAREER OUT OF IT. IT'S WONDERFUL IF WE'RE ASKING SOME MEANINGFUL QUESTION FOR, YOU KNOW, DECADES OF RESEARCH. AND THE RENEWABLE FASHION. SO WHAT I THINK I'M TRYING TO GET, IT WILL BE USEFUL TO HEAR FROM YOU, WHETHER THE NEW DEFINITION WOULD ACTUALLY HURT THIS COMMUNITY. OR -- >> , I GUESS TO SOMEWHAT ADD TO T. SO I AGREE THAT THERE ARE STRUGGLES WITHIN THE DEFINITION, AND THING HAS BEEN DISCUSSED. THROUGHOUT, AND ESPECIALLY IF WE'RE LOOKING AT THESE NEW POLICIES WITH RESPECT TO HOW WE'RE GOING TO FUND AND GOVERN CLINICAL TRIALS. THERE'S NO QUESTION. THIS IS MOVING. SO I THINK IF YOU COULD MAKE SOME PERFORMS OF HOW THE TRIAL'S POLICY AND THE DEFINITION ITSELF FOR DEVELOPING FIELDS THAT, HAVE NOT DONE DEFINITIVE TRIALS. FOR EXAMPLE, NHLBE. CARDIAC CARE, THERE OF COURSE -- THEY HAVE BIOMARKERS. -- BUT THEY HAVE THOSE THINGS. ESTABLISHED. MANY OF THE FIELDS THAT REHAB DEALS WITH DO NOT HAVE THOSE. SO ONE MAY BE, THERE NEEDS TO BE AN IDENTIFICATION THAT WE NEED BETTER THINGS LIKE THAT. BECAUSE DOING TRIALS, BECOMES SLIGHTLY -- I DON'T WANT TO SAY EASIER. PLAUSIBLE. ABSOLUTELY MORE PLAUSIBLE. BUT THE SECOND PIECE IS IN DESIGNING THOSE TRIALS F YOU DON'T HAVE THOSE THINGS, HOW DOES THAT DEFINITION LIMIT YOUR ABILITY TO DO THE THING YOU NEED TO DO OR NOT? IT MAY NOT. SOME OF YOU HAVE DONE SUCCESSFUL TRIALS. AND SUCCESSFUL, THE OTHER PROBLEM THAT I OF COURSE STRUGGLING WITH, IS SUCCESSFUL DOES NOT MEAN THAT WHAT YOU THOUGHT WAS GOING TO WORK, WORKED. SUCCESSFUL MEANS THAT YOU ARE ABLE TO EITHER PROVE OR DISPROVE THE NULL HYPOTHESIS. IN A SUFFICIENTLY, POWDER EXPERIMENT, THAT ALLOWS YOU TO MAKE ILLOGICAL AND ACCURATE CONCLUSION. SO I THINK THAT'S GOING TO BE A CHANNEL AND THE WAY THAT THE DEFINITION IS WRITTEN AND WAY THE POLICY IS WHIP, MAY BE A CHANNEL. SO IF THAT'S SOMETHING YOU ALL WANT TO BRING UP, I THINK IT WOULD BE USEFUL. >> I THINK THE REASON THIS IS BEING BROUGHT TO OUR TENACIN FROM THE PUBLIC IS A, THEY WANT TO SHOW THAT NIH IS REALLY DOING TRANSLATION RESEARCH. YOU FOLKS ARE NOT JUST DOING MORE AND MORE OF THE SAME BASIC STUDIES BUT YOU'RE CALLING THE QUESTION. WHERE PEOPLE CAN ACTUALLY ACCESS T. THAT'S THE POSITIVE. THE NEGATIVE IS, THEY WANT CLINICAL TRIALS TO BE DONE RIGHT BECAUSE THERE'S A FEAR WE COULD BE HURTING OR ABUSING PATIENTS. HAVE THE TWO FORCES DRIVING THIS. I THINK WE HAVE TO BE CAREFUL WE DON'T TRY TO DO TOO MUCH IN ONE PAPER F. ANYTHING ANYBODY'S INTERESTED IN WRITATION SHORT POSITION PAPER, JUST ABOUT WILL CLINICAL TRIAL. I WOULD BE VERY HAPPY TO WORK WITH PEOPLE. MY OWN FEELING IS REALLY, THE ROCK OF NIH AND WHY NIH IS GREAT AND IS THE CROWN JEWEL OF THE WORLD IN TERMS OF HEALTH-RELATED RESEARCH, IS THE INVESTIGATOR-INITIATED RO1. AND THING THIS POLICY HAS A POTENTIAL TO MAKE THAT VERY DIFFICULT. IT'S CERTAINLY MAKING IT MUCH HARDER FOR ME TO WRITE A STAND-ALONE, RO1. WHAT CLINICAL TRIALS HAVE, WHICH IS TERRIFIC S A METHODOLOGY. WHEN YOU BRING MUCH OF THAT METHODOLOGY TO YOUR SCIENCE, IT IS GOOD. SO RANDOMIZATION IS GOOD. IT JUST DOESN'T MEAN YOU'VE GOT A CLINICAL TRIAL. HAVING HEALTH-RELATED OUTCOME MEASURES IS GOOD. DOESN'T MEAN YOU HAVE A CLINICAL TRIAL. AND BECAUSE YOU INTERVENE, DOESN'T MEAN YOU HAVE A CLINICAL TRIAL AND I WOULD VERY MUCH LIKE MAYBE SOMETHING LIKE INTERVENTION. AND THAT CAN YOU DO THOSE KINDS OF STUDIES, USING THE DESIGN, WHICH CLINICAL TRIALS HAVE NOW GOTTEN VERY, VERY GOOD AND TO BORROW FROM THAT IS TERRIFIC. BUT THINGS, I THINK THIS IS A TRULY DESPERATELY BAD IDEA. AND ACTUALLY, I DON'T KNOW WHO WAS BEHIND T. BUT UNTIL THEY CAN ALL THE QUESTIONS, AS TO THIS IS A CLINICAL TRIAL. THEY SHOULD NOT ROLL DOCUMENT. >> COULD I ADD AND CHANGE THE TOPIC A LITTLE BIT. ONE OF THE THINGS I THINK WILL BE REALLY IMPORTANT TO SAY, IN THIS SECTION ABOUT OUTCOMES AND STAGES OF DEVELOPMENT IS IT'S OKAY TO GO BACKWARDS. AND IT'S OKAY TO NOT GO FORWARD AND BOOMED THE IDEA. I THINK WE FORGET. WE THINK THIS IS RELATED TO ALL TREATMENT IT IS ARE -- HAVE PEOPLE THAT ARE NONRESPONSIVE. THERE ARE A TON OF PHARMACEUTICAL ALENENE THAT IS GET TOSSED OUT. AND WE THINK ALL REHAB REHABILITATION EFFORTS GO FROM ONE PHASE TO THE NEXT PHASE TO THE NEXT PHASE. IF WE CAN GIVE PERMISSION TO PEOPLE, TOO THROW OUT INTERVENTION THAT IS DON'T SHOW PROMISE EMERGENT, THING WILL BE A REAL BENEFIT TO THE FIELD. WE'RE NOT INVENTING NEW DRUG THAT IS START OVER HERE AND MOVE FROM LEFT TO RIGHT. THERE'S A LOT OF STUFF THAT'S IN PRACTICE, THAT WE DON'T UNDERSTAND AT ALL. WHERE WE WANT TO DO TEASE BACK UP STEPS AND SORT OF TRY TO EXPLORE MECHANISMS. SO I THINK IT WOULD BE MORE HELPFUL TO REALLY FRAME IT AROUND WHAT KINDS OF QUESTIONS DO YOU NEED TO ASK AND WHAT KINDS OF OUTCOMES, WHAT KINDS OF DESIGNS. >> OKAY. WE'RE GOING TO TAKE A SHORT BREAK. WE HAVE TWO MORE SESSIONS TO DISCUSS. SO WE'LL TAKE A SHORT BREAK. 10 MINUTES AND THEN WRAP UP. A VERY STRICT TREATMENT I THINK YOU NEED TO BE CAREFUL ABOUT THE WORD, EFFECTIVENESS THOUGH. A LOT PEOPLE THINK THAT'S PROMOTING INSURANCE RECORDS AND I WOULD NOT USE BILLING RECORDS TO TRY AND GET AT THAT CONCEPT. IT MIGHT BE USEFUL AS A START BUT IT JUST DOESN'T DISCREDIT RESOLUTION YOU NEED TO REALLY UNDERSTAND WHAT'S BEING DONE. [PLEASE STAND BY] RESEARCH PERSON RECEIVE. WHAT THOSE TRIAL VS. LACKED, BEFORE S ANY KIND OF EXPLORATION WHY. AND DOWN AT THE LEVEL OF LOOKING AT BIOMARKERS, FOR EXAMPLE, TREATMENT FOR RESPONSES. >> RIGHT AND I AGREE. AND THE CAUTIONARY, EVEN WHEN WE LOOKING AT BIOMARKERS AND THINK WE UNDERSTAND, WE STILL MAY NOT. IT LOOKS BETTER. IT LOOKS MORE REAL. BUT AGAIN, IT MAY BE JUST A FALSE PATH THAT WE GO DOWN. BUT IT'S AN IMPORTANT ONE. BECAUSE IT CAN LEAD TO DISCOVERYIES. >> SO, I REALLY LIKE THE POINT THAT YOU GUYS BOTH MADE AND THING SHOULD GO IN THE PAPER OF YOU MAY WANT TO BE POWERING YOUR TRIAL DIFFERENTLY. AND DEPENDING ON THE PHASE, AND THE QUESTIONS THAT YOU'RE ASKING. MAYBE WHAT YOU'RE POWERING IS THE ABILITY TO EXPLORE THESE ISSUES, NOT THE ABILITY TO HAVE A MAIN EFFECT. SO I THINK THAT WOULD ABE REALLY IMPORTANT POINT TO MAKE TO PEOPLE WHO DON'T THINK ABOUT CLINICAL TRIALS. AND I THINK YOU CAN ARGUE THAT PRETTY NICELY, AS LONG AS YOU HAVE YOUR AIMS LAID OUT CAREFULLY TO A REVIEW COMMITTEE I'M WONDER IF YOU CAN SHOULD MAKE A STATEMENT, THAT MAY NEOTAB GOOD STRATEGY. YOU'LL BE MISSING GENERAL RELIABILITY. I THINK WE HAVE TO WRITE THAT IN, HAVE YOU TO FIND THE BALANCE. BUT I DO THINK, YOU KNOW, THERE'S A LOT OF WAYS YOU COULD APPROACH IT AIM 1 CAN STILL BE THE MAIN EFFECT YOU'RE TALKING B. ANYWAY, I THINK WE'LL ADDRESS IT, IN A WAY, THAT HOPEFULLY, EVERYBODY CAN COMMENT ON IT AND SEE WHAT DIRECTION IT TAKES. IT WON'T DOMINATE THE PAPER, BUT I THINK IT SHOULD BE IN THERE. >> I THINK WE DO NEED TO CONSIDER AND THAT IS THE DIFFERENCE BETWEEN AN INVESTIGATION AND A TRIAL. SO IF YOU'RE LOOKING TO DISCOVER A BIOMARKER, THAT IS NOT A TRIAL. THAT IS AN INVESTIGATION. SO SOME OF THE THINGS, THE QUESTION THAT IS YOU'RE ASKING, MAY NOT BE A TRIAL. NOW, YOU MAY WANT TO INCLUDE THINGS IN A TRIAL FOR FOLLOW-UP INVESTIGATION. BUT IN INVESTIGATION, WOULD BE, LET'S SAY WITH THIS NEW THING BEING CREATED, INVESTIGATION WOULD BE AN RO1. AND YOU WOULD BE LOOKING TO DISCOVER A BIOMARKER OR A MECHANISM OR A PATH OLDEST AGES OR WHATEVER. THAT'S AN INVESTIGAION. A TRIAL HAS A SPECIFIC PURPOSE. AND I THINK THAT NEEDS TO BE CLEAR. SO YOU CAN INCLUDE SOMETHING IN A TRIAL THAT WILL LEAD TO INVESTIGATION. OR CAN YOU JUST BE DOING A TRIAL. I THINK THERE'S A KEY QUESTION AND THAT'S SOMETHING THAT NEEDS TO BE CLEAR TO PEOPLE. >> I AGREE. I THINK THERE ARE THREE WAYS TO GO. ONE IS YOU KNOW, EMBRACE EXPLORATORY AIMS. YOU STILL HAVE YOUR MAIN TRIAL. THAT'S ONE OPTION. SECOND OPTION IS, THAT'S NOT EMBRACED, BUT AT LEAST EMBRACE SOME AMOUNT OF DATA COLLECTION THAT DOESN'T HAVE A HYPOTHESES ASSOCIATED WITH T. SO THE FIELD CAN MAXIMIZE THE YIELD FROM YOUR TRIAL. AND THEN THE THIRD -- RIGHT. THAT'S RIGHT. THE THIRD IS SUPPLEMENTAL FUNDING OPPORTUNITIES, LIKE RO1'S, R21'S AND OTHER WAY THAT IS PEOPLE CAN ESSENTIALLY DO THE FOLLOW-UPS, NEEDED THESE CLINICAL TRIALS ARE DAUNTING AND EXPENSIVE. WHEN IT'S ALL OVER WITH, NIH WANTS RESOURCES THAT WE CAN ALL BENEFIT FROM AND WHETHER IT'S TO STUDY RESPONSE HETEROGENEITY OR NOT, WE SHOULD COME OUT IN YEAR 5 OF A TRIAL, WITH REALLY VALUABLE RESOURCES. DOESN'T MEAN, EVERYONE'S GOING TO TAP THOSE RESOURCE THAT IS WANTS TO DO HETEROGENEITY. MAYBE THAT'S PART OF THE RECOMMENDATION, TOO. GENERATE THE RESOURCES TO MAXIMIZE TRIAL YIELD. YOU CAN'T ANY BACKWARDS. I'M GOING TO EXERCISE MY POWER, AND THANK YOU TO MARCUS AND GAR AND HE CHRISTINE A WHO JUST PUT EVERYTHING TOGETHER. AND MADE IT WORK. THANK YOU MUCH AND THANK YOU ALL, FOR YOUR LIVELY DISCUSSION AND HAVE A GREAT TRIP BACK. AND AS I SAID, THE SLIDES POSTED. IF YOU DON'T WANT YOUR SLIDES SHARED OR YOU NEED TO DO SOMETHING THEY WILL BE POSTED FOR LIKE A MONTH IF YOU WANT TO MAKE CHANGES AND I DISTRIBUTED THESE NOTES THAT ARE HERE, IF THAT WOULD BE OF ANY USE TO YOU, IN GUIDING DISCUSSIONS. I'M AVAILABLE FOR ANY DISCUSSION LEADING UP. >> CAN YOU SPECIFY WHERE THEY'RE POSTED IN. >> IDA'S GOING TO MAKE THEM AVAILABLE ON THE WEBSITE WHERE YOU REGISTERED. WHERE IT IS NOW AVAILABLE, IT WILL BE THERE FOR ABOUT A MONTH. DO YOU ALL NEED A SHARING SITE OR SOMETHING WHERE YOU CAN COLLABORATE OR ARE YOU GOING TO DO THAT OUTSIDE. GO I'VE BEEN THINKING ABOUT THIS, ACTUALLY, BUT WE HAVE OF COURSE, WE ALL HAVE ACCESS TO DROP BOX AND THINGS LIKE THAT. WE CAN JUST USE DROP BOX. >> GOOD. OKAY.