WELCOME TO THE LONG AWAITED LAUNCH OF THE PATIENT CENTERED OUT COME TOOLS HERE AT THE NATIONAL INSTITUTES OF HEALTH. I AM SUSANA SERRATE-STEIN. I AM THE DIRECTOR OF THE DIVISION OF AT NIAMS, BUT I AM ALSO THE PROJECT OFFICER FOR THE PATIENT-REPORTED OUT COME MEASUREMENT INFORMATION SYSTEM, OTHERWISE KNOWN AS PROMIS. AND IT IS MY PLEASURE TO INTRODUCE THE PATIENT CENTERED OUT COME TOOLS TO THE NIH HERE TODAY. I WOULD LIKE TO TAKE A MOMENT TO THANK ALL THE CENTER DIRECTORS WHO ARE HERE. THE NIH LEADERSHIP HAS ALL ALONG SINCE THE BEGINNING OF MANY OF THESE PROJECTS BACK IN THE 2002 TO 2004 PERIOD SHOWN TREMENDOUS LEADERSHIP, SUPPORT AND COMMIT PIMENTS TO THESE TOOLS WHICH ARE ESSENTIAL TO MEASURE PATIENT-REPORTED OUTCOMES IN CHRONIC DISEASES. THEY WILL INTRODUCE TO YOU THE COMPONENTS OF THE -- MANY OF THE COMPONENTS THAT COMPRISE WHAT WE ARE NOW CALLING PATIENT-CENTERED OUTCOME TOOLS. I WOULD ALSO LIKE TO THANK THE TEAM AT NORTHWESTERN UNIVERSITY, DR. RICHARD GERSHON, KAREN COOK -- AND MONICA FOR COMING ALL THE WAY FROM CHICAGO TO MAKE THE PRESENTATION TODAY, AND ESPECIALLY I WOULD LIKE TO THANK THEM FOR OFFERING THE TRAINING OPPORTUNITY THIS SESSION THIS AFTERNOON. I KNOW A LOT OF PEOPLE HAVE REGISTERED FOR THAT, SO WE ARE EXCITED THAT THERE IS SO MUCH INTEREST ABOUT LEARNING HOW TO USE THESE TOOLS HERE AT NIH. WE WOULD LIKE -- I WOULD LIKE ALSO TO THANK THE TEAM AT NINDS WHO WORK TO IMPLEMENT AT THE ASSESSMENT CENTER, WHICH IS THE REAL GOAL, TO BRING PROMIS, NEURO-QOL AND THE TOOLBOX HERE TO THE CLINICAL CENTER AND THE INTRAMURAL RESEARCH PROGRAM AT NIH. ALSO DR. MARGARET, WHO I KNOW CANNOT BE HERE TODAY, AND -- FOR TAKING THE LEAD IN TESTING THE ASSESSMENT CENTER AND THE DIFFERENT TOOLS HERE AT NIH BEFORE LAUNCHING. I WOULD LIKE TO THANK ALSO THE ONE PERSON WHO HAD THE VISION AND THE COMMITMENT TO BRING THE ASSESSMENT CENTER AND THE TOOLS TO CAMPUS, AND THAT IS THE CHIEF SCIENCE OFFICERS OF THE PROMIS INITIATIVE, DR. GIN WITTER. JIM WITTER. A FEW YEARS BACK HE REALIZED THE IMPORTANCE OF THE TOOLS AND THE RELEVANCE AND POTENTIAL IMPACT INTRODUCING THE TOOLS HERE AT NIH TO UNDERSTAND PATIENT-REPORTED OUTCOMES IN DISEASES IN THIS VERY UNIQUE ENVIRONMENT AND VERY UNIQUE GROUP OF PATIENTS. IT TOOK HIM A COUPLE OF YEARS TO CONVINCE US ALL AND TO MOVE THIS FORWARD, AND I'M VERY GRATEFUL FOR HIS COMMITMENT AND HIS PERSEVERANCE IN BRINGING THESE TOOLS TO CAMPUS. I WOULD LIKE TO THANK YOU ALL FOR BEING HERE, I KNOW THE PEOPLE WHO ARE OBSERVING THE PRESENTATION VIA VIDEOCAST, AND I THINK THIS IS GOING TO BE SAVED POR FOR FUTURE VIEWING, SO IF IF YOU CANNOT BE HERE TODAY, PLEASE WATCH THERE. BRIEFLY, THE SESSION THIS MORNING WILL BEGIN WITH THE INTRODUCTION OF THE DIFFERENT TOOLS BY THE NIH INSTITUTE AND CENTERS DIRECTOR AFTER A BRIEF INTRODUCTION BY DR. ANDERSON, WHO'S DIRECTOR OF THE COMMON FUND. THIS WILL BE POLLED BY A PRESENTATION BY DR. RICHARD GERSHON, DIRECTOR OF THE PROPTIOPROMISASSESSMENT CENTER. WE WILL HEAR FROM INTRAMURAL SCIENTISTS WHO HAVE PILOT-TESTED SOME OF THE TOOLS, AND THIS WILL BE FOLLOWED BY DR. ROTHROCK AND DR. YANG FANN. WE ASK THAT YOU PLEASE HOLD YOUR QUESTIONS UNTIL THE END OF THE SESSION SO THE SPEAKERS WILL CONVENE HERE AND THERE WILL BE TIME FOR YOU TO ASK THE QUESTIONS USING THE MICROPHONES THAT WE HAVE IN THE AISLE. OUR HOPE IS THAT YOU WILL LEAVE THE MEETING TODAY FEELING EXCITED ABOUT USING PROMIS, NEURO-QOL AND THE TOOLBOX, AND ENERGIZED ABOUT THIS GROWING FIELD OF PERSON CENTERED OUTCOME TOOLS. SO WITHOUT FURTHER ADO, I WOULD LIKE TO INTRODUCE DR. JIM ANDERSON, DIRECTOR OF THE DIVISION OF PROGRAM PLANNING THAT HOUSES THE COMMON FUND WHO HAS PROVIDED SUPPORT FOR PROMIS. >> THANK YOU. [APPLAUSE] >> THANK YOU FOR YOUR INTEREST IT IN PROMIS. I WANT TO THANK THE TEAM THAT DELIVERED ON PROMIS. THIS IS A FANTASTIC COMMON FUND PROJECT. IT ORIGINATED IN 2003, AS PART OF THE ROAD MAP PLANNING PROCESS. THE NEED WAS IDENTIFIED TO HAVE A VALID APPROACH OR METHOD TO STUDY PATIENT SYMPTOMS AND OTHER OUTCOMES, PARTICULARLY IN CHRONIC DISEASES, CONDITIONS WHERE AN X-RAY OR A SODIUM LEVEL ISN'T GOING OH TELL YOU THE STORY. SO THIS WAS A RESEARCH TOOL, AND I'VE BEEN HERE A LITTLE OVER THREE YEARS, I REMEMBER THE FIRST TIME I MET WITH THE PROMIS TEAM. I HAD READ UP A LITTLE BIT ON THE PROGRAM BUT WAS VERY INFORMED. I WAS READY TO ASK WHAT IS COMPUTER ADAPTED TESTING? I REMEMBER THIS AHA MOMENT DURING THEIR PRESENTATION WHEN I SAID, OH, YOU CAN ACTUALLY REPEATEDLY IN A STANDARDIZED WAY MEASURE SYMPTOMS FOR YOUR RESEARCH STUDIES? AND I JUST EMBRACED THIS AS A REAL COMMON FUND PROJECT. WHY? BECAUSE IT'S IMPORTANT FOR MULTIPLE I.C.s, BECAUSE IT'S IDENTIFIED AN OBSTACLE FOR PROGRESS IN RESEARCH ACROSS A WIDE NUMBER OF AREAS, AND MORE IMPORTANTLY, BECAUSE IT IDENTIFIED A DELIVERABLE, WHICH IF IT WERE PRODUCED, WOULD REALLY TRANSFORM THE WAY THE SCIENCE COULD BE DONE. AND THAT'S WHAT THEY'VE DONE. THESE ARE REMARKABLE SET OF TOOLS. SO REMARKABLE AND SO SUCCESSFUL, THEY'RE BEING WIDELY ADAPTED, THEY'RE NOW BEING INTRODUCED FOR USE BY ETHICS, THE DEPARTMENT OF DEFENSE, CMS, THIS WILL ALSO ENTER CLINICAL PRACTICE TOO AS A VALID WAY OF QUANTIFYING SYMPTOMS AND OTHER OUTCOMES. SO AGAIN, I WANT TO THANK THE TEAM. I HOPE THAT FOLKS HERE IN THE CLINICAL CENTER WILL ADOPT AND ALSO ADAPT THE METHODS THAT PROMIS HAS DELIVERED, AND SINCE I REPRESENT THE COMMON FUND EVERYWHERE I GO, I WANT TO SAY HAD IS NOW 10 YEARS, AND WE WILL BE HAVING A CELEBRATION IN NATCHER ON JUNE 19TH RECOGNIZING THE SUCCESSES OF THE COMMON FUND OVER THE LAST DECADE AND THE PROCESS THAT THE COMMON FUND USES TO DO NEW TYPES OF SCIENCE. AND PROMIS WILL BE REPRESENTED THAT DAY. SO THANK YOU, PLEASE ADOPT AND ADAPT. [APPLAUSE] [INAUDIBLE] >> BECAUSE I'M NOT SURE EVERYBODY ON THE WEB HEARD, WHAT SUSANA JUST SAID, I'M STEVE KATZ, DIRECTOR OF THE NATIONAL INSTITUTES OF ARTHRITIS AND MUSCULOSKELETAL SKIN DISEASES. ALONG WITH JOSIE BRIGGS AND HER PREDECESSOR, STEVE STRAWS, WE REALLY HAVE CO-CHAIRED THIS FROM A DIRECTORS' STANDPOINT HAD IT PROMIS INITIATIVE. THANKS, EVERYONE, FOR JOINING TODAY. I DO WANT TO GIVE A SPECIAL THANKS TO SUSANA AND JIM WITTER WHO REALLY HAVE BEEN MEETING WITH A LARGE GROUP OF PEOPLE REPRESENTED FROM VARIOUS INSTITUTES. LET ME JUST GO INTO A LITTLE MORE DETAIL THAN JIM WENT INTO BECAUSE I ACTUALLY WAS HERE IN 2003, WHEN ALL THIS STARTED, AND ONE-THIRD OF THE SEGMENT OF THE ROAD MAP FOR MEDICAL RESEARCH THAT WAS PUT TOGETHER AMONGST ALL OF THE INSTITUTE DIRECTORS, ONE-THIRD OF IT WAS GEARED TOWARDS CLINICAL RESEARCH, AND THIS SEGMENT, ONE-THIRD, THE CTSAs ACTUALLY CAME OUT OF THAT, EXTRA MONEY FOR THE GCRCs, BUT ONE-THIRD OF THAT WAS PARTICULARLY FOCUSED ON CLINICAL RESEARCH, AND A SEGMENT OF THAT WAS FOCUSED ON GETTING PATIENTS' VOICES HEARD IN TERMS OF OUTCOMES. THERE WERE MANY INSTRUMENTS THAT WERE AVAILABLE, THEY WERE CUMBERSOME AND BURDENSOME, AND THEY WERE NOT REALLY TRANSLATABLE FROM ONE AREA TO THE OTHER. SO WITH THE HELP OF REALLY LARRY FRIEDMAN WAS SO CRITICALLY INVOLVED, LARRY FRIEDMAN FROM NHLBI, THAT WE PUT TOGETHER THIS INITIATIVE, AND THE ACRONYM IS GREAT, ISN'T IT? IT REALLY REFLECTS WHAT THIS IS ALL ABOUT. PATIENT, REPORTED, OUTCOMES, MEASUREMENT, INFORMATION, SYSTEM. THAT'S WHAT IT'S ABOUT. AND BASICALLY IT COVERS A BROAD RANGE OF PATIENT-REPORTED OUTCOMES IN A NUMBER OF AREAS, PAIN, FATIGUE, FUNCTIONING, EMOTIONAL DISVEST, SOCIAL ROLE PARTICIPATION AND MINIMIZES PATIENT BURDEN WHEN ANSWERING QUESTIONS SO THAT YOU DON'T HAVE TO GO THROUGH A THOUSAND QUESTIONS TO GET THE ANSWER. AND THAT'S WHAT COMPUTER ADAPTIVE TESTING IS ALL ABOUT, THAT JIM HAS TALKED ABOUT. PROMIS MEASURES CAN BE USED AS PRIMARY AND SECONDARY END POINTS IN CLINICAL STUDIES FOR THE EFFECTIVENESS OF TREATMENT, IT CAN BE USED TO DESIGN TREATMENT PLANS AND MANAGE CHRONIC DISEASE. AND IN DOING ALL OF THIS, WE DIDN'T DO THIS OUT OF THE LANDSCAPE OF HAVING THE FDA AT THE TABLE. THE F CADA AT THE TABLE. JIM WITTER WAS AT THE FDA WHEN ALL THIS WAS STARTED AND IT WAS TO OUR BENEFIT THAT HE CAME OVER SO HE COULD ENCOURAGE ITS USE HERE NOT ONLY ACROSS THE NIH EXTRAMURAL PROGRAMS, BUT THE PURPOSE OF TODAY'S MEETING IS REALLY TO EDUCATE OUR INTRAMURAL PROGRAM AS WELL AS SOME OF OUR EXTRAMURAL PROGRAM OF STAFF AS TO WHAT PROMIS IS ALL ABOUT. PROMIS ACTUALLY ALLOWS FOR APPLES TO APPLES COMPARISONS OF RESULTS ACROSS STUDIES, SOMETHING NOT CURRENTLY POSSIBLE WITH OTHER MEASUREMENT SYSTEMS. IT ALSO INCLUDES, FOR EXAMPLE, NOT ONLY ADULTS, BUT ALSO CHILDREN, VERY YOUNG CHILDREN. IT ALLOWS FOR A PARENT AS SURROGATES FOR CHILDREN, AND IT ALLOWS FOR USE WITH ALL INDIVIDUALS REGARDLESS OF LITERACY, LANGUAGE, PHYSICAL FUNCTION, OR LIFE COURSE. WHAT ARE THE FUTURE DIRECTIONS? WELL, WE'VE SEEN REAL UPTICK IN THE UTILIZATION OF THE PROMIS DATABASE THAT IS HOUSED AT NORTHWESTERN. WE'VE SEEN A REAL UPTICK IN STUDIES THAT ARE BEING PERFORMED IN TERMS OF CLINICAL AND INTERVENTION STUDIES IN CLINICAL MEDICINE, AND WE HOPE THAT IT WILWILLCONTINUE TO BE USED, AND AS JIM SAID, ADOPTED AND ADAPTED. SO IN CLOSING, I WOULD NOTE THAT IN 10 YEARS SINCE PROPS STARTED, AND THIS IS ONE OF THE FIRST ACTUALLY OF THE COMMON FUND PROJECTS, AND I THINK THE VERY FIRST THAT HAS ACTUALLY BEEN TRANSFORMED TO A POINT WHERE THE INSTITUTE IS SUPPORTING IT RATHER THAN THE COMMON FUND, OR THE COMMON FUND TO A LESSER EXTENT, WE WOULD CONSIDER THAT A SUCCESS BECAUSE THERE IS ENTHUSIASM TO TAKING IT UP. PERHAPS THE MOST TANGIBLE EVIDENCE OF THIS IS THE ESTABLISHMENT OF THE PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE, THE PCORI, THAT I KNOW MY COLLEAGUE AND FRIEND, JOSIE BRIGGS, IS GOING TO BE TALKING ABOUT, HOW IT'S BEING ADAPTED THERE AND OTHER THINGS. SO THANK YOU VERY MUCH FOR BEING HERE. [APPLAUSE] >> I'M JOSIE BRIGGS, DIRECTOR OF THE NATIONAL CENTER FOR COMPLEMENTARY AND ALTERNATIVE MEDICINE, AND LIKE STEVE, I WAS PART OF THE WORKING GROUP ESTABLISHED BY DR. ZERHUNI TO, QUOTE, TRANSFORM CLINICAL RESEARCH. AND THAT WAS CHAIRED BY MY PREDECESSOR, STEVE S IT TRAUSS, AND STEVE KATZ, AND WE CALLED IT "THE STEVES." IT WAS VERY CLEAR TO STEVE STRAUSS THAT THE FIELD OF MEASURING THE VALUE OF COMPLEMENTARY HEALTH PRACTICES WAS VERY CENTERED ON THEIR IMPACT ON SYMPTOMS. PATIENTS TURN TO A VARIETY OF HEALTH PRACTICES, WHETHER IT'S MEDITATION OR DIETARY SUPPLEMENTS OR CHIROPRACTORS BECAUSE OF SYMPTOMS, AND THE POTENTIAL VALUE OF THESE APPROACHES LIES IN THEIR ABILITY TO CONTRIBUTE TO SYMPTOM MANAGEMENT. SO STEVE UNDERSTOOD VERY CLEARLY AND FOCUSED IN ON THE NEED FOR MORE RIGOROUS AND CONSISTENT METHODS FOR MEASURING SYMPTOMS. I WAS INTRIGUED BY THIS WHOLE INITIATIVE, AT THAT TIME I WAS RUNNING -- OVERSEEING A NUMBER OF LARGE CLINICAL TRIALS IN KIDNEY DISEASE, AND I WAS SEEING THE CONSERVATISM OF CLINICAL INVESTIGATORS WHO WOULD PUT IN FRONT OF PATIENTS AT EACH VISIT A STACK OF PAPER QUESTIONNAIRES, TWO HOURS OF FILLING IN FORMS, I DIDN'T THINK WE WERE REALLY CAPTURING ELEMENTS OF QUALITY OF LIFE OR IMPACT OF OUR INTERVENTIONS ON HOW THEY WERE LIVING. BUT I DON'T HINGE ANY OF US ACTUALLY COULD SEE THE FUTURE A AT THAT TIME, AND THAT'S ONE OF THE INTERESTING THINGS ABOUT LOOKING BACK AT THE 10 YEARS OF THIS INITIATIVE. I THINK PROMIS HAS BEEN TRANSFORMATIVE IN A WAY WE DEPARTMENT EXPECT. WE DIDN'T REALIZE THE WAY IN WHICH PATIENT-CENTEREDNESS WOULD BECOME A HALLMARK OF CHANGE IN OUR HEALTHCARE SYSTEM. SO I'M VERY PLEASED RIGHT NOW TO BE INTERACTING A LOT WITH THINGS HAPPENING IN PCORI. PCORI HAS USED AS THEIR BRANDING PATIENT-CENTEREDNESS, AND IT'S AN INTERESTING CHANGE THAT'S GOING TO OCCUR IN OUR RESEARCH PROCESSES, AS WE SEE BUILDING MORE INTO RESEARCH STUDIES A CENTEREDNESS ON WHAT PATIENTS WANT AND HOW THEY PERCEIVE TREATMENT. THIS IS VERY IN SYNC AND VERY CENTRAL TO EMPLOYING THE KIND OF MEASURES THAT ARE -- HAVE BEEN DEVELOPED BY PROMIS AND BY METHODOLOGY THAT PROMIS HAS DEVELOPED. ONE OF THE THINGS THAT WAS CERTAINLY AN EDUCATION FOR ME IN OVERSEEING THE EVOLUTION OF PROMIS IS THE WHOLE SCIENCE OF BIOMETRICS, AND THE METHODS THAT PROMIS HAS DEVELOPED FOR VALIDATING INDIVIDUAL ELEMENTS OF UTILIZING COMPUTER ADAPTIVE TESTING AND REALLY LEARNING HOW TO SELECT ITEMS THAT WILL ACTUALLY ACCURATELY AND AS EFFICIENTLY AS POSSIBLE MEASURE THE PATIENTS' EVOLVING SYMPTOMATOLOGY IS AN IMPORTANT PART OF PROMISES CONTRIBUTION, AND SOMETHING I THINK THAT WILL REQUIRE CONTINUED INVESTMENT. PCORI -- THE PCORI LEADERSHIP ARE INTENSELY AWARE OF THESE ISSUES AND HAVE COMMITTED TO UTILIZATION OF MANY PROMIS MEASURES AND TO CONTINUED INVESTMENT IN IT, AND I THINK ALL OF US WHO SEE PATIENT SYMPTOMS AS CENTRAL IN OUR RESEARCH WILL STAY SIMILARLY COMMITTED. IS OH IT'S A WONDERFUL THING THAT'S NOW GOING TO BE HERE ON THIS CAM P PUS AND CAMPUS AND PART OF THE FABRIC OF CLINICAL INVESTIGATION HERE IN OUR OWN HOSPITAL. MCAM IS NOW IN THE PROCESS OF LAUNCHING A PROGRAM THAT'S FOCUSED ON PAIN MANAGEMENT, AND DR. KATHRY CATHERINE BUSHNELL IS SCIENTIFIC DIRECTOR, SHE'S RECRUITED THREE TERRIFIC YOUNG SCIENTISTS AND BUILDING A PAIN PROGRAM, SO MEASURING PAIN OUTCOMES AND ALL THE WAYS IN WHICH PAIN IMPACTS ON QUALITY OF LIFE WILL BE VERY CENTRAL IN THAT PROGRAM, AND SO I AM JUST THRILLED THAT THE PROMIS TOOLS ARE NOW GOING TO BE HERE, READY FOR THE WORK THAT WE ARE INITIATING. SO I THINK THIS SHOULD BE A GREAT DAY. I LOOK FORWARD TO SEEING THE WEBCAST AND THANK EVERYONE HERE FOR THEIR INTEREST IN THIS. [APPLAUSE] [INAUDIBLE] >> FOR THOSE LISTENING ON THE WEBCAST, I'M MARIE BERNARD, RICHARD HODES SENDS HIS REGRETS HE'S NOT ABLE TO BE HERE. WE HAVE BEEN WORKING WITH A COLLABORATIVE GROUP WITH THE NIH TOOLBOX, AND WE'RE REALLY PLEASED THAT THIS IS NOW GOING TO BE PART OF THE NIH ASSESSMENT CENTER. THE TOOLBOX IS RELEASED TO THE EXTRAMURAL COMMUNITY JUST A YEAR AGO RIGHT HERE IN MASUR. IT IS VERY SIMILAR IN SOME WAYS TO PROMIS IN THAT IT ALLOWS US TO COMPARE APPLES TO APPLES WHEN IT COMES TO BEHAVIORAL AND NEUROLOGICAL OUTCOMES. IT PROVIDES A SET OF BRIEF, WELL VALIDATED MEASURES, IT'S A COGNITIVE, MOTOR, SENSORY AND EMOTIONAL FUNCTION IN INDIVIDUALS AGED 3 TO 85. MAJOR INITIATIVES EMPLOYING THE NIH TOOLBOX INCLUDE THE HUMAN COULD NECCONNECTOME PROJECT -- THE IDEA FOR THE TOOLBOX WAS BROUGHT FORWARD IN 2005 TO THE NIH BLUEPRINT FOR NEUROSCIENCE RESEARCH BY NIA, NINDS AND NIMH. IT RECEIVED UNANIMOUS ENTHUSIASTIC AGREEMENT AMONG THE 15 INSTITUTE DIRECTORS AT THAT TIME. THEREFORE A CONTRACT WAS AWARDED IN LATE 2006 TO NORTHWESTERN UNIVERSITY TO LEAD THE DEVELOPMENT OF THE TOOLBOX, AND OVER THE COURSE OF TIME, SOME 250 RESEARCHERS AND MEASUREMENT EXPERTS HAVE BEEN INVOLVED IN ITS DEVELOPMENT. THERE ARE A NUMBER OF GOALS ASSOCIATED WITH THE TOOLBOX. VERY IMPORTANTLY, A PRIMARY GOAL WAS TO RESPOND OH TO THE TO THE RESEARCH COMMUNITY'S NEEDS FOR UNIFORM MEASURES THAT CAN BE USED AS A FORM OF COMMON CURRENCY OR, AGAIN, ALLOWING COMPARISON OF APPLES TO APPLES ACROSS DIVERSE STUDY DESIGNS AND POPULATIONS, ENABLING RESEARCHERS TO SHARE, COMPARE, AND COMBINE RESEARCH DATA. ADDITIONALLY, IT WAS MEANT TO ESTABLISH INSTRUMENTS THAT WOULD ASSESS FUNCTION AND CHANGE IN FUNCTION OVER TIME IN ORDER TO ALLOW MEASUREMENT OF HEALTH SPAN AND AVOID THE PITFALLS USING INSTRUMENTS THAT WERE DESIGNED FOR DIAGNOSING DISEASES. IT PROVIDES STANDARDIZED MEASURES FOR NEUROLOGICAL AND BEHAVIORAL RESEARCH THAT CAN BE USED FOR LARGE SCALE LONGITUDINAL AND EPIDEMIOLOGICAL STUDY, CLINICAL STUDIES, AND AS WE ALL KNOW, WE'RE INVESTING MORE AND MORE AND THOSE SORTS OF STUDIES ARE GETTING MORE EXPENSIVE. THE ADVANTAGE OF THE TOOLS THAT ARE AVAILABLE THROUGH THE NIH TOOLBOX IS THAT THEY WILL ALLOW NEUROLOGICAL AND BEHAVIORAL ASSESSMENT IN THESE SORTS OF STUDIESTUDY WITHOUT SIGNIFICANT INCREASES IN ADMINISTRATION TIME OR O COST, ALLOWING FURTHER ENHANCEMENT OF THESE SORTS OF STU CAN DIS. STUDIES. IN CLOSING, I'D LIKE TO DESCRIBE WAYS IN WHICH YOU CAN HELP OH EXPAND THIS UNIQUE RESOURCE AND MAKE IT EVEN MORE UNIVERSAL. WE HOPE THAT YOU'LL USE IT AND BECOME FAMILIAR WITH THE INSTRUMENTS WITHIN IT, AND APPLY IT TO YOUR RESEARCH AND YOUR PATIENT POPULATION, AND THAT VERY IMPORTANTLY, THAT YOU'LL PROVIDE FEEDBACK SO THAT THERE MAY BE THE FUTURE POSSIBILITY OF THESE INSTRUMENTS BECOMING VALIDATED FOR THINGS RANGING FROM ADHD TO MAJOR DEPRESSIVE DISORDERS TO ALZHEIMER'S DISEASE. AS WELL AS USING THESE FOR THE EXAMINATION OF POTENTIAL NEURAL AND BEHAVIORAL SEQUELAE IN CARDIOVASCULAR DISEASE, MUSCULOSKELETAL DISEASE, RENAL DISEASE, AND DIABETES. SO WE LOOK FORWARD TO YOUR FEEDBACK. [APPLAUSE] >> NEXT WE'LL HEAR FROM DR. STORY LANDIS. >> SO I HAVE TO SAY, I WAS EXTREMELY DUBIOUS ABOUT PROMIS. 10 YEARS AGO. AND DUBIOUS ABOUT THE TOOLBOX. AND EVEN A LITTLE BIT DUBIOUS ABOUT THE PIECE THAT I'M GOING TO TELL YOU NOW, BUT IT'S VERY CLEAR IN RETROSPECT THAT I WAS WRONG THAT THIS WHOLE INITIATIVE WAS INCREDIBLY PR INCREDIBLY PRI SHEPT AND LEAVES INVESTIGATORS POISED IN LOOKING AT PATIENT-CENTERED OUTCOME. I HOPE EXTRAMURAL INVESTIGATORS WILL EMBRACE THESE TOOLS AND A USE THEM. SO THE NEURO-QOL IS FOCUSED ON QUALITY OF LIFE FOR PATIENTS WITH NEUROLOGICAL DISORDERS, AND JUST AS JIM SAID, IT'S NOT IN A POTASSIUM TEST OR A BLOOD PRESSURE, IMPACT OF NEUROLOGICAL DISORDERS ON THE LIFE OF PATIENTS IS MUCH MORE COMPLEX THAN WHAT WE MEASURE IN A ROUTINE EXAMINATION OF THE ONE OF THE COMMON EXAMS FOR MUSCULAR DYSTROPHY, PORE EXAMPLE, I FOR EXAMPLE, IS A SIX MINUTE WALK. IT'S A EUT USEFUL MEASURE BUT IT'S REALLY NOT ENOUGH BECAUSE IT MATTERS WHERE YOU WALK, WHY YOU WALK THERE, AND IT'S IMPORTANT TO KNOW YOU CAN MANAGE YOUR FINANCES, MAINTAIN A HOUSEHOLD, SOCIAL EVENTS, ALL THE KINDS OF ELEMENTS THAT MAKE UP QUALITY OF LIFE. SO HEALTH RELATED QUALITY OF LIFE IS PARTICULARLY IMPORTANT IN CHRONIC DISEASES INCLUDING MANY NEUROLOGICAL DISORDERS WHERE OUR EFFECTIVE TREATMENTS ARE LIMITED AND THE FOCUS OF CARE IS OFTEN ON MINIMIZING THE NEGATIVE IMPACT OF THE DISEASE AND IN FACT ITS TREATMENT. SO THE PIECES OF PROMIS AND TOOLBOX AND NEURO-QOL THAT COMPLEMENT TRADITIONAL ASSESSMENTS ALLOW THE CARETAKER OR THE PATIENT TO REPORT THEIR EXPERIENCE OF THE DISEASE AND HOW THE TREATMENT IS ACTUALLY HELPING THE PATIENT WITH QUALITY OF LIFE. AND ONE OF THE THINGS THAT BECOMES VERY CLEAR TO ME IN THE P PAST SIX MONTHS IS THERE'S HUGE RELEVANCE HERE TO OUR ABILITY TO TRANSLATE AND HAVE ACCEPTED AS A NEW VALID TREATMENT THESE KINDS OF TOOLS. BECAUSE THE FDA IS INTERESTED NOT IN -- FOR NEUROLOGICAL DISEASES, YOU HAVE A A LITTLE LESS TREMOR, THEY'RE REALLY INTERESTED IN WHAT THE QUALITY OF LIFE IS. SO NEURO-QOL WAS DEVELOPED CONSISTENT WITH THE FDA GUIDANCE ON PATIENT-REPORTED OUTCOME INSTRUMENT DEVELOPMENT AND OUR EARLY RESULTS SUGGESTED HAS GOOD QUALITY AND VALIDITY AND IT COULD BECOME AN ACCEPTED IF NOT REQUIRED END POINT IN CLINICAL TRIALS. SO PATIENT FOCUS GROUPS, LARGE SAMPLES, ADVANCED PSYCHOMETRIC METHODS TO DEVELOP A SET OF INSTRUMENTS BOTH IN ENGLISH AND SPANISH THAT CAN BE USED TO ASSESS THE QUALITY OF LIFE OF ADULTS AND CHILDREN WITH COMMON NEUROLOGICAL DISEASE. SO WE'VE ACTUALLY VALIDATED THIS WITH ADULTS SUFFERING FROM EPILEPSY, PARKINSON'S DISEASE, LOU GEHRIG'S DISEASE, DEADLY DISEASE FIVE YEARS FROM DIAGNOSIS OH TO DEATH, STROKE OR MS, A AND CHILDREN WITH EPILEPSY OR MUSCULAR DYSTROPHY. SO IT'S AVAILABLE FOR RESEARCH PURPOSES WITHOUT CHARGE, IT HAS A CORSET OF QUESTIONS THAT CUT ACROSS CHRONIC NEUROLOGICAL DISORDERS, AND SUPPLEMENTAL QUESTIONS THAT ADDRESS ADDITIONAL CONCERNS FOR SPECIFIC DISEASES. IT IS LIKE THE OTHER TOOLS, COMPUTERIZED ADAPTIVE TESTS OR SHORT FORMS FOR SUBDOMAINS AND MOST SCORES CAN BE LINKED TO PROMIS. SO LOOKING FORWARD, WE REALLY HOPE THAT THESE NIH MEASURES TOOLBOX, PROMIS TOOLBOX AND NEURO-QOL WILL BE USED IN ALL OF OUR FUNDED CLINICAL STUDIES, AND EXTRAMURALLY, AND WE HOPE THAT THEY'LL BE ADAPTED BY MANY OTHERS AS WELL. SO I THINK THE TAKE HOME MESSAGE IS PRECISE VALID MEASUREMENT IS POSSIBLE WITHOUT UNDULY BURDENING PATIENTS AND THAT IS GOING TO BECOME PART OF THE REQUIRED REPERTOIRE OF PEOPLE DOING CLINICAL RESEARCH AND PARTICULARLY CLINICAL TRIALS. SO I WANT TO THANK ALL OF YOU FOR BEING HOO HERE, AND HOPE YOU FIND THE DAY AND THE USE OF THESE TOOLS REALLY USEFUL, AND I'M DELIGHTED O TO SEE DAVID GOLDSTEIN HERE IN THE AUDIENCE, ONE OF OUR INVESTIGATORS. THANKS. [APPLAUSE] >> NOW WE'RE GOING TO MOVE TO THE MAIN PORTION OF THE PROGRAM. IT IS MY PLEASURE TO INTRODUCE DR. RICHARD GERSHON. RICHARD IS ASSOCIATE PROFESSOR AT THE MEDICAL SOCIAL SCIENCES AT NORTHWESTERN UNIVERSITY, AND HE HAS BEEN LEADING THE EFFORT IN THE DEVELOPMENT OF THE ASSESSMENT CENTER, ALSO INVOLVED WITH THE DEVELOP. DEVELOPMENT OF NEURO-QOL. HE'S EXCITED TO BE HERE AND TO MAKE HIS PRESENTATION. IT'S A WONDERFUL PRESENTATION, AND HE'LL PROVIDE WITH YOU A WONDERFUL INTERACTION TO THESE NEW TOOLS AND I THINK HE CAN BE AN EXTREMELY PERSUASIVE SPEAKER AND I'M SURE YOU ALL FEEL VERY EXCITED ABOUT USING THE TOOLS AFTER YOU HEAR HIS PRESENTATION. RICHARD? >> I THINK THIS IS OUR THIRD SCHEDULED ATTEMPT. WE WERE FIRST SNOWED OUT BY D.C. WEATHER. WE WERE THEN SNOWED OUT BY CHICAGO WEATHER. I LOOKED OH OUTSIDE TODAY BUT IT SEEMS LIKE WE'RE OKAY. ALL RIGHT. WE'RE GOING TO SPEND A FAIR AMOUNT OF TIME RIGHT NOW TALKING ABOUT A FEW THINGS. WE'RE GOING TO TALK ABOUT THE COMMONALITY BETWEEN THESE THREE SETS OF INSTRUMENTS, WE'RE GOING TO TALK ABOUT WHAT THE RELEVANCE IS IN CLINICAL USE, WE'RE GOING TO TALK ABOUT HOW THEY WORK, WE'RE GOING TO TALK ABOUT HOW THEY MEASURE UP COMPARED TO CURRENT INSTRUMENTS, AND THEN WE'LL SPEND SOME TIME ON WHAT INSTRUMENTS ARE AVAILABLE. I URGE YOU TO GRAB THE -- IF YOU'RE HERE, COPIES OF THE BROCHURES AND LISTS OF INSTRUMENTS THAT WERE OUT ON THE TABLES IN THE LOBBY AND MOST OF THOSE PIECES ARE AVAILABLE ONLINE AS WELL. SO I'VE BEEN ABLE TO REVISE MY SLIDES THREE TIMES FOR THIS PRESENTATION WHICH IS THE DANGER OF BEING RESCHEDULED, BUT I WAS BEGINNING TO THINK ABOUT WHAT IS GOING ON, WHY ARE THESE THREE SETS OF INSTRUMENTS TOGETHER? THEY'RE THE SAME BUT DIFFERENT, AND I THINK THEY EACH OFFER SOMETHING EU MEEK AN UNIQUE AND THEY OVERLAP QUITE A BIT AS WELL. PROMIS, YOU'VE HEARD NICE INTRODUCTIONS TO ALL OF THESE, BUT I WENT BACK TO THE ORIGINAL DOCUMENTATION, THE ORIGINAL OFFERING DOCUMENT TO SAY WHAT WAS THE ORIGINAL INTENT AND WE HEARD FROM THE PEOPLE WHO WERE HAVING THESE IDEAS CLOSE TO A DECADE AGO FOR PROMIS, BUT IT HAS A LARGE BANK OF ITEMS MEASURING PROs. THIS IS A RELATIVELY STRANGE CONCEPT A DECADE AGO. NOW WE TALK ABOUT OH, WE'RE DOING THIS, DOING THAT, WE'VE GOT PCORI WE'RE USING HERE AND THERE BUT PEOPLE WEREN'T THINKING THESE TERMS. UP AT THE TOP, MEET THE NEEDS OF CLINICAL RESEARCHERS ACROSS A WIDE VARIETY OF CHRONIC DISORDERS AND DISEASES. I SHOULD POINT OUT THAT WAS THE ORIGINAL GOAL. THIS IS FOR RESEARCH. I WILL TELL IT YOU TODAY ALL OF THESE INSTRUMENT BATTERIES ARE PROBABLY PRIMARILY THE SCALES HAVE TIPPED AND PEOPLE ARE USING THEM FOR CLINICAL -- DIRECT CLINICAL WORK AND THAT IS RAPIDLY GROWING AND I'LL TALK ABOUT THAT IN A LITTLE BIT. SECONDLY, INNOVATIVE IDEA, CREATE A SYSTEM THAT ALLOWS US TO DO THESE THINGS MORE QUICKLY IN CLINICAL RESEARCH, AND MAKE IT PUBLICLY AVAILABLE. SO THERE'S A COMMON REPOSITORY OF ITEMS. THAT'S DEFINITELY A FOREIGN CONCEPT IN MEASURE DEVELOPMENT. TURNS OUT ANY INSTRUMENT USE, THERE'S A LITTLE PRICETAG ASSOCIATED WITH IT, WHICH HAS REALLY REDUCED THE DESIRE TO USE THESE ITEMS IN RESEARCH AND EVEN FURTHER IN CLINICAL USE. TOOLBOX. I SHOULD KIND OF OVERLAY THESE. DEVELOP AN INTEGRATED SET OF MEASURES. COMMON CURRENCY. STILL KIND OF TRYING TO GET THINGS THAT ARE USED COMMON ACROSS STUDY DESIGN. MAXIMIZE YIELD FROM LARGE EXTENSIVE STUDIES WITH MINIMAL INCREMENT IN SUBJECT BURDEN AND COST. UNDERLYING WE JUST -- ALMOST A CUT AND PASTE, ADDING NOT JUST OUTCOME MEASURES BUT OUTCOME MEASURES AND OBJECTIVE MEASURES. AND NEURO-QOL. ALL THAT SAID AND DONE, A LOT OF US GREW UP IN THE SAME -- DEFINING NEUROLOGICAL DISORDERS, PARTICULARLY ADDING SYMPTOM-SPECIFIC AREAS THAT ARE REALLY ONLY FOUND IN PATIENTS WITH NEUROLOGICAL DISORDERS, CORSET OF QUESTION, CHRONIC NEUROLOGICAL DISEASES, THAT MAY BE SPECIFIC DEFINED BY DISEASE, AGE OR OTHER FACTORS. SO ALL PRETTY SIMILAR-SOUNDING. SO GOING UNDER THE COVERS, THE 10 OR P 15 PAGES OF THESE ORIGINAL RFAs, WHAT WAS THERE, IPT LEK ACTUAL PROPERTY FREE. HIGH UP ON THE LIST. PSYCHOMETICALLY SOUND. TURNS OUT A LOT OF MEASURES USING RESEARCH AND PARTICULARLY CLINICALLY, IF YOU GO BACK AT THE MEASURES WE LEARNED IN RESIDENCY AND MED SCHOOL AND ANYWHERE, TURP TURNS OUT THEY VERY OFTEN WERE INSTRUMENTS THAT WERE NEVER FULLY IMPLEMENTED, NOT RELIABLE, THAT DOESN'T MEAN THEY'RE NOT USED HUNDREDS OF THOUSANDS OF TIMES A DAY BUT THEY WEREN'T QUALITY WE WOULD LIKE, CERTAINLY WOULDN'T PASS THE SMELL TEST DERIVED TODAY. YOU CAN HAVE DIFFERENT SETS OF TESTS ON THE SAME METRIC. BRIEF AND EASY TO USE, WHICH TURNS OUT TO BE THE MAJOR STUMBLING BLOCK/THE MAJOR GOAL OF ALL THIS INSTRUMENT WORK, IS HOW DO YOU GET THESE THINGS SHORT ENOUGH THAT PEOPLE CAN USE THEM. APPLICABLE IN A VARIETY OF SETTINGS. AVAILABLE FOR USE ACROSS THE AGE SPAN, AND ALL OF THEM SHOULD BE GENERALLY TRANSLATABLE, AND I SHOULD SAY THAT ALL OF THESE INSTRUMENTS ARE ALREADY NORM AND VALIDATED IN SPANISH AND MANY OF THEM IN DOZENS OF OTHER LANGUAGES. WE'RE ALSO GOING TO TALK ABOUT LOTS OF DIFFERENT TYPES OF MEASURES. THOSE THAT NEED TO BE GIVEN BY COMPUTER, WE CAN SEE HERE, WE CAN GIVE COMPUTER ADAPTIVE TESTS. IF YOU DON'T KNOW WHAT THAT IS, YOU WILL KNOW BY THE TIME I STEP DOWN FROM HERE. HIGH LEVEL PRECISION, VARIABLE LENGTH, FOUR TO 12 ITEMS, REALLY SEEING MORE LIKE FIVE OR SIX ITEMS DOES THE TRICK. SHORT FORMS CAN BE GIVEN ON COMPUTER OR PAPER. VARIES BY LENGTH, YOU CAN DO A SHORT FORM THAT MATCHES YOUR SAMPLES. SO WE NEED SOMETHING VERY SHORT, WE CAN USE IT FOR A TRIAL, WE CAN GET THE MEAN LEVEL OF SOMETHING AND THAT WORKS, IF I NEED SOMETHING FOR CLINICAL EU I CAN MAKE IT A LITTLE LONG ESHES THEN FULL LENGTH SCALE INSTRUMENT. WE CAN STILL DO VERY LONG BUT YOU CAN SEE WE DON'T NEED THEM. THANK YOU FOR THE PEOPLE AT THE NIH PUTTING THIS TOGETHER. COMMONLY REFERRED TO AS PROMIS 1 2004-2008, CROSS DISEASE EMPHASIS, SELF-REPORT, A LOT OF DOMAINS, A LOT OF AGE RANGES, OVER 40 LANGUAGES AT THE MOMENT, HEALTHY PEOPLE. NIH TOOLBOX, CONTRACT ME CAN NESM RELEASED IN THIS ROOM OCTOBER OF LAST YEAR, PERFORMANCE SELF-REPORT, 45 INSTRUMENTS, 3-85, LET'S DIVE IN. SO THE OTHER PIECE FORTHCOMING ON THIS IS THREE SYSTEMS, ONE RESEARCH RESOURCE, IT'S A SLIDE BILL RILEY PUT TOGETHER, INTEGRATING THESE THREE MEASUREMENT SYSTEMS WITH THE GOAL OF NON-NIH DIRECT SUPPORT, THAT WILL SUPPORT THE INFRASTRUCTURE AND SCIENTIFIC STANDARDS AND FACILITATE DATA HARMONIZATION. MORE ON THAT AS WE LEARN. SO PUBLIC BUT PRIVATE. AND I WAS -- BILL RILEY AND I SPOKE WITH THE HHS INTERAGENCY MEASUREMENT SELECTION TEAM, I DON'T KNOW WHAT THE OFFICIAL NAME IS, TWO WEEKS AGO, AND THE BIGGEST SHOCK I HEARD WAS WE'RE NOT LOOKING AT THESE INSTRUMENTS BECAUSE THEY COST TOO MUCH. I GOT TO PUT RIGHT UP AT THE TOP, THERE IS ZERO DIRECT COST, THERE IS ZERO ROYALTY FOR ANY OF THE MEASUREMENT SYSTEMS THAT I'M PRESENTED TODAY. THERE'S NO PER TEST COST AND DEPENDING ON THE FORMAT, YOU HAVE NO COST TO DO THEM OTHER THAN HOW YOU PUT THEM TOGETHER. THERE ARE REVENUE NEUTRAL FEES FOR TECHNOLOGY, EAN ONLY FOR THOSE PEOPLE WHO WANT TO ACCESS CENTRAL TECHNOLOGY. THE REASON FOR TODAY IS A AMOWNSING THAANNOUNCING WE HAVE A CLINICAL BASED -- INTRAMURAL RESEARCH AND THERE'S NO COST TO USE THAT. THESE ARE OPEN ACCESS TOOLS, RESEARCH CLINICAL USE AND EDUCATION, BUT THERE ARE RESTRICTIONS IT -- THE NIH TOOLBOX COGNITION BATTERY, IF P IF THAT WAS FULLY PUT OUT THERE IN THE PUBLIC DOMAIN, THEN PEOPLE WOULD PRACTICE IT AT HOME AND THE SCORES WOULD HAVE LESS USE TO THEM. COMPUTER ADAPTIVE TESTING OR CATS. FOR THE INSTRUMENTS IN HERE, MOST OF THE INSTRUMENTS THAT WE HAVE, MOST OF THE PRO MEASURES IN PROMIS, NEUROQOL AND TOOLBOX ARE BASED ON USING CAT. WE ALSO USE IT FOR DIE COT MUST TYPE ITEMS, SUCH AS VOCABULARY AND READING IN THE NIH TOOLBOX. WE JUST USE ITEM RESPONSE THEORY ON ITS OWN FOR SCORING, THINGS SUCH A PICTURE SEQUENCE MEMORY AND BALANCE AND SO ON. WE ALL LEARNED CONVENTIONAL TEST THEORY SOMEWHERE ALONG THE WAY, PROBABLY FIRST TIME WE TOOK A TEST IN THE SECOND GRADE, WE ADD UP THE SCORES, WE GOT A GRADE THAT SOUNDED GOOD, AN INDIVIDUAL TAKES AN ASSESSMENT, WE TAKE A LOOK AT THEIR TOTAL SCORE, WE USE THAT FOR COMPARISON PURPOSES, HIGH SCORE, PERSON IS HIGHER ON THAT TRAIT, A LOW SCORE, THE PERSON IS LOWER ON THE TRAIT. MAKES SENSE UNLESS YOU'RE THE PERSON IN EIGHTH GRADE WHO HAD THE REALLY HARD TEACHER AND THEIR 80% WAS REALLY LIKE A 99% IN THE OTHER CLASS. BASE VERY OFTEN, TESTS ARE NOT COMPARABLE. IF WE JUST ADD UP THE SCORES, THAT DOESN'T MEAN THEY HAVE THE SAME DIFFICULTY. ITEM RESPONSE THEORY LOOKS AT LIFE IN A DIFFERENT WAY. EACH INDIVIDUAL ITEM CAN BE USED FOR COMPARISON PURPOSES. IT'S KIND OF LIKE EACH ITEM IS ITS OWN TEST. IF YOU ENDORSE HARDER ITEMS, YOU'RE HIGHER ON THE TRAIT, IF YOU INDUCE LOWER, EAR ITEMS, YOU'RE LOWER ON THE TRAIT. WE CAN AGGREGATE THESE ITEMS TOGETHER -- IF YOU DON'T FOLLOW THAT, GIVE ME A MINUTE. SO THE PROMIS OF NIH INSTRUMENTS AND NOT JUST PROMIS INSTRUMENTS, EXRABBABILITY, PROVIDE THE ABILITY TO COMPARE COMBINE RESULTS FROM MULTIPLE STUDIES, RELIABILITY, REDUCING RESPONSE BURDEN, ALSO IMPROVING MEASUREMENT PRECISION. THE IRONY, TYPICALLY A TEST IS MOST RELIABLE WHEN IT'S LONG. USUALLY IT'S THE ASSESSMENT CREATOR'S SECRET, PUT MORE ITEMS IN FRONT OF A PERSON, YOU GET A MORE RELIABLE TEST. WHAT WE ARE ABLE TO SHOW, THAT'S NOT TRUE. INDEED SOMETIMES MORE ITEMS TELLS YOU LESS. SIMPLIFY ADMINISTRATION BY A COMPUTER BASED ADMINISTRATION, SCORING AND REPORTING. THERE ARE NON-COMPUTER BASED FORMS ON A LOT OF THE ASSESSMENTS THAT I AM PRESENTING TODAY, BUT IF YOU USE THE COMPUTER BASED FORMS, THE LAST KEY STROKE THAT A PATIENT, A SUBJECT TAKES, THE INSTRUMENT IS FULLY SCORED AND DEPENDING WHAT INSTRUMENT IS, NORMATIVE SCORE IS AVAILABLE IMMEDIATELY. LET'S TALK ABOUT RELIABILITY OR PRECISION. THIS IS THE PROMIS PHYSICAL FUNCTIONING INSTRUMENTS, COMPARISON OF DIFFERENT INSTRUMENT TYPES. ORIENTING TO THE SLIDE IS ALWAYS A LITTLE BIT OF A CHALLENGE BUT HERE GOES. YOU HAVE ON THE LEFT HERE STANDARD AIR OR. ERROR. WE TYPICALLY WANT AS LOW AN ERROR AS WE CAN. THIS LINE RIGHT HERE, INDICATES A LEVEL WITH A RELIABILITY OF 95. IF YOU'RE MORE FAMILIAR WITH THAT METRIC. THE LINE UP HERE, 90. WHAT WE'VE LEARNED OVER TIME IS, IS THAT CLASSIC TEST THEORY SAYS AN INSTRUMENT HAS ONE LEVEL OF RELIABILITY NO MATTER WHO YOU GIVE IT TO. BUT THAT CAN'T POSSIBLY BE TRUE. LET'S THINK ABOUT THAT. IF I HAVE A FOURTH GRADE VOCABULARY TEST, IT'S HIGHLY RELIABLE, HOW CAN IT HAVE THE SAME LEVEL REEF LIABILITY IF I GIVE IT TO KINDERGARTNERS? SAME THING APPLIES TO DEPRESSION. SO WE CAN SEE HERE THE DIFFERENT INSTRUMENTS HAVE DIFFERENT -- OKAY. HAVE DIFFERENT RANGES OF RELIABILITY ASSOCIATED WITH THEM. SO THIS IS A PROMIS 20 ITEM SHORT FORM. WE ACTUALLY DON'T -- NOBODY USES THAT ONE BUT YOU CAN SEE IF IT'S THERE. , IT'S HIGHLY RELIABLE TO DOWN PRETTY DARN LOW. WE HAVE THE FS36 PHYSICAL FUNCTION COMPONENT. IT DOES A VERY GOOD JOB, 95 RELIABILITY LEVEL, FOR THIS VERY NARROW RANGE RIGHT HERE. WHAT'S THE DANGER OF USING AN INSTRUMENT WHOSE RELIABILITY IS ONLY STRONG IN A CERTAIN AREA, YOU ACTUALLY CAN'T TRACK PROGRESS, RIGHT? YOU CAN GET AN ACCURATE ASSESSMENT OF YOUR PATIENT TODAY WHEN THEY'RE IN A CLINICAL RANGE, BUT IF THEY TRULY GO THROUGH IMPROVEMENT, THESE INSTRUMENTS, THE INSTRUMENTS THAT DON'T HAVE CONTINUING RELIABILITY OUT TO A HIGHER END, DON'T ACCURATELY IDENTIFY A PATIENT WHO'S IMPROVED. SO YOU CAN DO A TRIAL, YOUR TRIAL SHOWS NO IMPROVEMENT, I'LL SHOW YOU AN EXAMPLE OF THIS IN A MINUTE, AND INDEED, THIS PERSON HAS CONTINUED TO IMPROVE, IT'S THE INSTRUMENTS YOU'RE USING TO ASSESS THAT THAT HASN'T CAUGHT IT. ANOTHER ONE, FATIGUE. YOU CAN SEE THE ITEM REALLY JUST TOUCHES AT THE 90 RELIABILITY LEVEL BUT ONLY FOR PEOPLE SLIGHTLY FATIGUED. THE FOUR ITEM CAT HITS THE 90 LEVEL ALMOST FOR 70 TO 80% OF THE TRAIT RANGE. IF YOU WANT TO USE A 13 ITEM CAT, YOU CAN GET BELOW THE 95 LEVEL, THIS BLUE LINE DOWN HERE IS 90 ITEMS, I ALWAYS SAID YOU DON'T WANT TO GIVE 90 FATIGUE ITEMS BECAUSE YOU'LL DEFINITELY BE TIRED BY THE TIME YOU'RE DONE. TALK ABOUT VALIDITY. THE SCORES ON ALL THESE NIH MEASURES SHOULD CORRELATE WITH ACCEPTED MEASURES OF THE SAME DOMAIN. CONCURRENT VALIDITY, CREATING THESE TESTS IF THEY DON'T RELATE TO GOLD STANDARD TESTS THAT WE USE ALL DAY LONG, THEN THERE'S SOMETHING GOING ON THAT'S WRONG. SO -- WE CAN STILL SEE IT. THIS IS ANXIETY. WE'RE GOING TO ORIENT YOU HERE. THE FIRST THING TO NOTICE IS, SO THIS COMPARES PROMIS ANXIETY VERSUS THE MASK. I'M ALWAYS AFRAID OF LASER POINTERS, I'M GOING TO POINT AT MY OWN EYES. SO THIS IS GOOD, THIS MEANS THE INSTRUMENTS ARE HIGHLY CORRELATED, RIGHT? THEY SCORE ON ONE MEASURE, RELATES TO SCORE ON THE OTHER MEASURE, THAT'S GOOD. WE WANT TO SEE THAT. FOR THOSE OF YOU WHO HAVEN'T LOOKED AT THIS, THIS IS VERY GOOD CORRELATION. BUT THE STORY THAT'S GOING ON THAT'S A LITTLE BIT MORE DISTURBING IS HERE. THE MASK HAS A SIGNIFICANT FLOOR EFFECT. IT DOES NOT DISTINGUISH THE MAJORITY OF PEOPLE IN THIS SAMPLE THAT IT WAS MEASURED WITH. WE DO NOT KNOW, WE CAN'T DISTINGUISH PEOPLE AT ONE END OF THE SCALE. PROMIS, ON THE OTHER HAND, HAS GOT A LITTLE BIT OF A FLOOR EFFECT BUT IS ABLE TO DIFFERENTIATE PEOPLE ACROSS 90 SOME ODD PERCENT OF THE TRAIT RANGE. LOOK AT ONE MORE. DEPRESSION. SIMILARLY, COMPARING PROMIS DEPRESSION TO THE CESC, WHEN WE STARTED THIS PROJECT, WE ACTUALLY HAD TO PAY TO USE THE CESC, NOW YOU CAN GET FOR FREE, BUT LOOK WHAT'S GOING ON. THE CESC AGAIN HAS A STRONG EFFECT, IT DOES NOT COVER PEOPLE IN THIS RANGE, AND ON THE PROMIS DEPRESSION, COVER ACROSS THE ENTIRE RANGE OF THE TRAIT. WHEN PEOPLE EXPERIENCE CLINICAL BENEFIT -- RESPONSIVENESS, I DON'T THINK I CAN DI -- WHAT IS COMPUTER ADAPTIVE TESTING? IT'S SHORTER, TARGETED AND USES A COMPUTERIZED ALGORITHM. WE HAVEN'T FIGURED OUT HOW TO DO IT WITHOUT THE COMPUTER BUT WE'LL WORK ON IT. I SHOWED THE SLIDE A DECADE AGO. THESE SAIMS GROUPTHESE SAME GROUPS ARE UP HERE. NPROs, AT THE BEGINNING OF THESE CONTRACT OR GRANTS WERE PLEADING EDGE IN PROs, THEY WERE NOT BLEEDING EDGE IN THE WORLD. THE ARMY ORIGINALLY PROBABLY THE FIRST GROUP TO USE CATS, THEY GIVE THIS OVER A MILLION TIMES A YEAR PRIMARILY TO HIGH SCHOOL STUDENTS TO DIFFERENTIATE THEIR ABILITY AND WHAT KIND OF ROLES THEY COULD PLAY IN THE ARMED SERVICES. THEY USE CAT TO BE A ABLE TO ZERO IN ON A PERSON'S ABILITY LEVEL AND ALSO SECURITY. IT'S A COMPUTER GENERATED EXAM, WE CAN MAKE CERTAIN NO TWO APPLICANTS SEE THE SAME ITEMS AND BECAUSE OF THAT, RECRUITERS FOR THE ARMY WHO GET COMPENSATION BASED ON HOW MANY PEOPLE THEY BRING IN LOSE THEIR ABILITY TO TELL PEOPLE WHAT THE ANSWERS ARE IN ADVANCE. GRADUATE RECORD EXAM, GIVEN EVERY DAY OF THE YEAR. SIGNIFICANTLY CUTS TEST LENGTH AND AGAIN FOR SECURITY, NATIONAL ASSOCIATION OF SECURITY DEALERS, NATIONAL COUNCIL OF STATE BOARDS OF NURSING, NURSES UP UNTIL EIGHT, NINE YEARS AGO TOOK A 12-HUR EXAM, IT WAS OFFERED TWICE A YEAR, COST ABOUT $2.5 MILLION TO PRODUCE THAT EXAM. FOR THOSE OF YOU WHO TOOK IT THEN, VERSUS NOW, THE EXAM AVERAGES ABOUT 90 MINUTES. IT IS GIVEN EVERY DAY OF THE YEAR, EVERY WEEKDAY OF THE YEAR. IT IS MORE RELIABLE IN ITS HOUR AND A HALF FORMAT THAN IT WAS IN THE 12 HOUR FORMAT. BY THE WAY, NURSING IT WAS QUITE A PROBLEM. CONGRESS ACTUALLY ASKED THE NATIONAL ASSOCIATION OF STATE BOARDS OF NURSING TO FIGURE OUT A WAY TO GIVE TESTS MORE OFTEN. ORIGINALLY THIS WAS BASED UPON WHEN THERE ARE BIG NURSING SHORTAGES ABOUT A DECADE AGO, WE WERE IMPORTING NURSES FROM ALL OVER THE WORLD BECAUSE WE COULDN'T PRODUCE ENOUGH HERE. EVEN WHEN IT GOT PEOPLE HERE, IF YOU WERE BUSY ON AN EMERGENCY ROOM SHIFT THAT DAY, YOU HAD TO WAIT ANOTHER SIX MONTHS TO GET TEST ITED. I SEE A FEW NURSES NODDING THEIR HEADS HERE. SO LET'S TALK ABOUT CAT, I'M GOING TO NEED TWO VOLUNTEERS. YOU CAN STAY IN YOUR SEATS. THIS ALWAYS BOTHERS ME BECAUSE THE FIRST PERSON -- I NEED SOMEBODY WHO CAN COUNT TO 7 OR 8 OR 9. THIS IS NO BETTER THAN ANY M.D. GROUP I GO TO. THANK YOU VERY MUCH. ONE PERSON IN THE ROOM CAN COUNT. AND I NEED SOMEBODY ELSE WHO CAN ANSWER SIMPLER QUESTIONS THAN THAT. THANK YOU. SO I NEED YOU TO PICK A NUMBER BETWEEN ONE AND 100. NO, I DON'T NEED YOU TO TELL ME THE NUMBER. [LAUGHTER] >> PICK ANOTHER NUMBER, DON'T TELL ME. OKAY, GREAT. YOU'RE GOING TO COUNT HOW MANY QUESTIONS IT TAKES ME TO FIGURE OUT THAT HE PICKED 57 -- OH, MO, WHATEVER MUM BER IT'S GOING TO BE. SO IS YOUR NUMBER GREATER THAN 50? IS IT LESS THAN 75? IS IT LESS THAN 67? IS IT GREATER THAN 56? IS IT GREATER THAN 61? IS IT GREATER THAN 64? IS IT 65? IS IT 66? HOW MANY QUESTIONS? EIGHT. I MADE A MISTAKE. I SHOULD BE ABLE TO GET IT IN SEVEN. OKAY. THAT'S EFFECTIVELY HOW A CAT WORKS. A CLASSICAL TEST, IS IT ONE, IS IT TWO, IS IT THREE? WELL, I HAVE A LOT OF TIME, I PROBABLY DON'T HAVE ENOUGH TIME, A PAPER BASED TEST, I'D HAVE TO HAND OUT A SHEET WITH 100 TRUE-FALSE QUESTIONS, BRING THAT SHEET BACK, ADD UP MY SCORES AND FIND OUT I'M AT 66. WHAT DID I DO? THE FIRST QUESTION, IS IT GREATER THAN 50? YES. I JUST CUT MY TEST LENGTH IN HALF. RIGHT? I JUST ELIMINATED 50 QUESTIONS. NEXT QUESTION, LESS THAN 75? YES. I JUST CUT MY TEST LENGTH -- I KNOW IT'S BETWEEN 50 AND 75. I JUST REDUCED MY TEST LENGTH BY 75 QUESTIONS IN TWO QUESTIONS. THAT'S THE BASIS OF COMPUTER ADAPTIVE TESTING, A BASIS OF A LOT OF THE TESTS WE'RE ABOUT TO SHOW. BECAUSE ONCE I KNOW THAT PERSON IS ABOVE 50, I DON'T NEED TO ASK THEM ABOUT THEIR QUESTIONS AND 10, 20, 30 AND 40. ONCE I KNOW A PERSON'S PAIN IS LOW, WHY DO I ASK THEM SEVEN TO 20 MORE TIMES HOW BAD WAS YOUR PAIN THIS MORNING? HOW BAD WAS IT YESTERDAY? AND CONVERSELY, WHEN THE PERSON FIRST SAYS THIS PAIN INTERFERES WITH DAILY ACTIVITIES NEVER, WHY IS IT THAT ANY STANDARDIZED INSTRUMENT, I ASK ANOTHER 7 TO 20 TIMES, HOW BAD WAS IT THIS MORNING, HOW BAD WAS IT YESTERDAY, HOW BAD WHEN YOU DO THINGS? IT'S A WASTE OF PATIENT EFFORT, FRANKLY IT FRUSTRATES PATIENTS AND SUBJECTS AS WELL. SO CAT STARTS WITH AN ITEM BANK. AN ITEM BANK IS A GROUP OF ITEMS THAT COVER THE WHOLE RANGE OF THE TRAIT. SO INDEED, IF I WANT TO ASSESS PAIN, I NEED ITEMS IN THERE ON PEOPLE WITH REALLY SEVERE PAIN, PAIN THAT'S COMPLETELY INTERRUPTED TO THEIR LIVES. I ALSO NEED QUESTIONS IN THERE THAT DEAL WITH ALMOST IN PAIN OR NO PAIN, BOTHERS ME A LITTLE BIT, ET CETERA. AND THEN I HAVE TO CALIBRATE THESE ITEMS. I CALIBRATE ITEMS BY UNDERSTANDING WHERE THEY FALL ON THAT DIFFICULTY RANGE AND HOW WELL THEY DISCRIMINATE. IT TURNS OUT SOME ITEMS DO A BETTER JOB THAN OTHERS AT DETERMINING IF IT WAS OVER 50. IF IT WAS CLEAR-CUT, IT WOULDN'T TAKE SO MUCH EFFORT. SO LET'S TAKE CREDIT TO KAREN COOK HERE. LET'S TAKE PHYSICAL FUNCTIONING. SO WE HAVE AN ITEM BANK, PEOPLE WITH PHYSICAL FUNCTIONING THAT ARE BASICALLY BED-BOUND, THEY HAVE VERY, VERY LITTLE PHYSICAL FUNCTIONING, AND WE HAVE PEOPLE WHO ARE HIGHLY ACTIVE PHYSICALLY. TURNS OUT EVERY TEST THAT'S USED FUNCTIONALLY STOPS RIGHT HERE. I ALWAYS THOUGHT THAT WAS PRETTY GOOD. WE DID A REALLY GOOD JOB OF MEASURING HERE. BUT IT TURNS OUT, HE STARTED WORKING HERE, DR. ROTHROCK WITH ORTHOPEDIC TRAUMA SURGEONS AND THEY SAY WAIT A MINUTE, OUR PATIENTS ARE PHYSICALLY ACTIVE. WHEN WE TREAT THEM, OR THEY'RE SPORTS STARS OR MARATHON RUNNERS. WHEN WE TREAT THEM AND OUR CURRENT INSTRUMENT TAKES THEM TO COMPLETELY CURED RIGHT HERE, THEY DON'T FEEL COMPLETELY CURED. THEY NEED TO KNOW THAT THEY GOT BACK UP TO TO THIS LEVEL. WE'RE WORKING WITH THE DEPARTMENT OF DEFENSE. THEY'RE SAYING OKAY, THAT'S GREAT YOU GOT THEM TO HEAR IT. THEY WANT TO KNOW, WE NEED TO DIFFERENTIATE IF THEY CAN HIKE 50 MILES OR 50 MILES CARRY AGO BACKPACK ON THEIR BACK. STILL WORKING ON THAT ONE. SO WE CAN SET A RULER ACROSS THIS RANGE AND WE CAN FIND ITEMS THAT ACCURATELY PLACE PEOPLE OR THAT ARE LOCATED, CENTERED IN HAD IT RANGE. SO AN ITEM THAT'S APPLICABLE TO SOMEONE'S BED BOUND LEVEL OF PHYSICAL FUNCTIONING IS CLEARLY GOING TO BE DIFFERENT THAN AITEM THAT'S ASSOCIATED WITH SOMEBODY WHO HAS A VERY HIGH LEVEL OF PHYSICAL FUNCTIONING. THIS IS OUR CAT ALGORITHM. SO WE HAVE A PERSON OF UNKNOWN LEVEL OF PHYSICAL FUNCTIONING. OUR CAT ALGORITHM TYPICALLY IS GOING TO LOOK FOR AN ITEM IN THE MIDDLE OF THIS TRAIT RANGE. WHY IS IT DOING THAT? IT'S SIMILAR TO THAT QUESTION I ASKED BEFORE. IS YOUR NUMBER GREATER OR LESS THAN 50. WHY DID I PICK 50? BECAUSE IT'S GOING OH GIVE ME THE GREATEST IDEA, AM I LOOKING HIGH OR AM I LOOKING LOW? IF I KNEW THIS PERSON WAS IN A CLINICAL SETTING, I COULD ACTUALLY START DOWN HERE. BUT FOR OUR PURPOSES RIGHT HERE, WE'RE GOING TO SAY WE DON'T KNOW ANYTHING ABOUT THIS PERSON. I'M GOING TO GIVE THIS PERSON THIS ITEM. TURNS OUT THEY'RE AN ACTIVE SOCCER PLAYER, THEY'RE GOING TO HAVE A HIGHER LEVEL OF PHYSICAL FUNCTIONING, SO OUR CAT ALGORITHMS ARE GOING TO PULL AN ITEM AT THE HIGHER END OF THE SCALE. IT'S GOING TO SKIP DOWN HERE. I ASSURE YOU, ANY OFF THE SHELF PHYSICAL FUNCTIONING MEASURE OF GENERAL ABILITY, THE NEXT QUESTION IS HERE, AND THE NEXT QUESTION IS HERE, AND THE NEXT QUESTION IS HERE. AND THIS PERSON HAS ABSOLUTELY NO IDEA WHY YOU'RE ASKING QUESTIONS ABOUT YOUR LIMITED PHYSICAL FUNCTIONING. SO WE'RE GOING TO CONTINUE TO GIVE ITEMS IN HAD RANGE. THERE WILL BE A TEST ON THIS ALGORITHM AT THE END. WE PERFORM LITTLE MAXIMUM LIKELIHOOD CALCULUS IN THE BACKGROUND. WHAT'S GOING ON? WE DO TWO THINGS AFTER EVERY ITEM IS GIVEN. WE CALCULATE OUR BEST ESTIMATE OF THE PERSON'S ABILITY LEVEL, IN THIS CASE ABILITY BEING PHYSICAL FUNCTIONING, BUT ABILITY COULD ALSO BE THEIR DEPRESSION LEVEL OR THEIR ANXIETY LEVEL OR THEIR VOCABULARY LEVEL. FROM MY PERSPECTIVE, THEY'RE ALMOST ALL THE SAME. A COUPLE MINOR CHANGES. SO WE GET OUR BEST IDEA OF THEIR LEVEL OF FUNCTIONING AND WE'RE ALSO ABLE TO CALCULATE HOW ACCURATE WE BELIEVE WE KNOW THAT. EVERY SINGLE ITEM GIVES US INFORMATION ABOUT THE RELIABILITY OF THEIR MEASUREMENT. WE'RE GOING OH SEEK AND FIND THE ITEM THAT'S GOING IT TO TELL US THE MOST ABOUT THIS PERSON GIVEN WHAT WE KNOW ABOUT THEM. EVEN THIS PERSON RIGHT HERE IS IS NOT THE PERSON WITH THE 50-POUND BACK ON. SO THEY COULD BE AN ACTIVE WEEKEND SOCCER PLAYER BUT THEY'RE NOT ON THE OLYMPIC TEAM. SO WE'RE GOING TO CONTINUE GIVING ITEMS. NEXT, WE'LL TAKE ANOTHER PERSON. WHILE WE PHYSICALLY CAN SEE. WE VISUALLY SEE THAT THEY'RE PROBABLY AT A LOWER LEVEL OF FUNCTIONING, WE CAN GIVE THEM THE SAME TEST, WE GIVE THEM AN ITEM IN THE MIDDLE OF THE TRAIT RANGE. BASED ON THEIR ANSWER TO TO THE FIRST ITEM, WE'RE GOING TO GO LOW ON THE RANGE THIS TIME. WE'RE NOT GOING TO ASK THEM ABOUT HOW MANY MILES THEY RAN THIS WEEK, WE'RE NOT GOING TO ASK THEM QUESTIONS WHICH TO THEM SEEM QUITE SILLY. NOW LET'S LOOK AT BOTH THESE PEOPLE AT THE SAME TIME. SO WE KNEW WHERE THEIR ABILITY WAS, APPROXIMATELY, IN THE FIRST ITEM. WITHIN ONE OR TWO ITEMS, WE HAVE A PRETTY GOOD IDEA OF WHERE THEY'RE LOCATED. FOR EVERY SUCCESSIVE ITEM, WE NARROW DOWN OUR ESTIMATE OF THEIR ABILITY. ESTIMATE OF THEIR FUNCTIONING. ALL TESTS ARE ESTIMATE OF FUNCTIONING. WE ALL KNOW HAD. THIS. HECK, EVEN WHITE BLOOD COUNTS HAVE SOME LEVEL OF ACCURACY ASSOCIATED. IT WAS ONE OF THE THINGS WE CONSIDERED TO BE OBJECTIVE. BUT IN THIS CASE, WE KNOW IF WE CAN QUANTIFY IT. NOW, IF I'M DOING A STUDY, A TRIAL COMPARING THE EFFICACY OF A DRUG, I MIGHT IN TWO OR THREE ITEMS BE DONE BECAUSE IF I AVERAGE 100 PEOPLE WITH THIS LEVEL OF ERROR IN THEM, I'M GOING TO BE ABLE TO VERY ACCURATELY SEND CHANGE. BUT IF I WANT TO USE MY ASSESSMENT OF THIS PERSON TO CHANGE THEIR OWN CLINICAL TREATMENT RIGHT NOW, I MIGHT TAKE THIS ON A LITTLE BIT, GIVE THEM A SLIGHTLY LONGER EXAM, IN THE CASE OF A CAT, WE'RE NOW TALKING ABOUT FOUR OR FIVE ITEMS INSTEAD OF TWO OR THREE, AND MOVE TO GET A HIGHLY RELIABLE MEASURE OF THEIR P FUNCTIONING. SO WHEN DO WE STOP? WE CAN STOP AT A SPECIFIED NUMBER OF ITEMS, SAY ENOUGH IS ENOUGH, OR WE CAN STOP AT A LEVEL OF PRECISION. THAT LEVEL OF PRECISION CAN BE DEFINED BASED ON WA YOUR NEED IS. AGAIN, IT'S A DIFFERENT LEVEL OF NEED P IF YOU'RE JUST TRYING TO COMPARE WITH IT GROUPS, THAN P IF YOU'RE TRYING TO INDIVIDUALLY LOOK AT A CURRENT PATIENT'S LEVEL OF FUNCTIONING. SO WHY BOTHER? I'M GRIMACING BECAUSE -- OKAY. REDUCE THE BURDEN OF RESPONDING. MAKE ROOM FOR MEASURING OTHER DOMAINS. YOU'RE GOING TO SEE TODAY, I'M GOING TO SHOW YOU QUITE LITERALLY ABOUT 200 INSTRUMENTS, DON'T WORRY, WE WON'T GO TOO DEEP. WHEN PEOPLE START GIVING THESE, AND PROMIS STARTED DERIVING DIFFERENT MEASURES, IT TURNED OUT PEOPLE REALLY WANTED TO GIVE PEOPLE SIX OR SEVEN. NOW -- AT MINIMUM. IN THE OLD WAY OF LOOKING AT THINGS, THE OLD PARADIGM, THE STANDARD FATIGUE INSTRUMENT COULD HAVE BEEN 70 ITEMS. WE GAVE SIX OF THOSE, YOU'RE AT 360 ITEMS. YOU HAD A PERSON SITTING IN THAT WAITING ROOM FOR AN HOUR, HECK, THEY WOULD NEVER FINISH THE INSTRUMENTS, THEY'D RARELY VOLUNTEER TO BE IN THE U.S. DI. WE NOW HAVE NUMEROUS CLINICAL USES OF THESE INSTRUMENTS GOING ON. THE AVERAGE PROMIS INSTRUMENT CAN BE GIVEN IN UNDER A MINUTE. WE HAVE SIMILAR PROPORTIONATE LEVEL OF BEING ABLE TO SHORTEN ASSESSMENTS FOR ALL THE INSTRUMENTS WE'RE TALKING ABOUT. SO HERE'S PROMIS ON THE WEBSITE: OVER 100 PEER REVIEWED PUBLICATIONS, THIS IS NOT BLEEDING EDGE, THIS IS REAL, THESE ARE PUBLICATIONS ACROSS THE GAMUT FROM DISEASE-SPECIFIC TO METHOD STUDIES, THINGS LIKE THAT. FROM AN INFORMATICS PERSPECTIVE, WE HAVE ASSESSMENTS CENTER, AS WELL AS -- SIMULATION ENGINE, THIS HAS 40 ADULT MEASURES, 20 PEDIATRIC MEASURES. I ASSURE YOU THERE'S MORE THAN THAT. NEW INSTRUMENTS IN SPECIFIC AREAS, LITERALLY COMING OUT WEEKLY. ALL ITEMS IN SPANISH, MANY LANGUAGES. NON-DISEASE-SPECIFIC. WHY PROs, PROVIDE PATIENT-CENTERED PERSPECTIVE TO UNDERSTAND HOW TREATMENTS AFFECT PATIENT EXPERIENCES, SYMPTOM, FUNCTIONS, PARTICIPATION, QUALITY OF LIFE. HONESTLY TALKING ABOUT PROs 10 YEARS AGO, PEOPLE THOUGHT, OKAY, THAT'S NICE, DON'T NEED IT TO CAN DEAL WITH THAT. WELL, TODAY, CANNOT GET A DRUG APPROVED WITHOUT SHOWING AN IMPROVEMENT IN A PATIENT-REPORTED OUTCOME MEASURE. YOU ARE GOING TO HAVE PCORI, YOU'VE GOT TO SHOW THAT PATIENTS THINK THEY'RE BETTER. YOU CANNOT BIOLOGICALLY TREAT A PERSON, REPAIR THEM AND HAVE THEM FEEL WORSE AT THE EN. THAT IS NOT THE DEFINITION OF SUCCESSFUL TREATMENT TODAY. WE CAN USE IT TO AUGMENT DESIGNS OF PATIENT TREATMENT PLANS AND BETTER MANAGE THAT CHRONIC DISEASE. AT THE TIME -- FACE TO FACE PATIENT TIME DECREASES, IT'S ALL THE MORE IMPORTANT THAT WE CAN RAPIDLY FIND OUT WHAT'S GOING ON WITH THE PATIENT, HAVE THEY CHANGED IN CERTAIN AREAS, FLAG AREAS, AT HOME, IN THE WAITING ROOM, INSERTED INTO ELECTRONIC MEDICAL RECORD, AND THEY CAN BE USED TO HELP AUGMENT A PHYSICIAN IN TREATMENT, PEOPLE WHO USE THESE SORTS OF INSTRUMENTS FIND IT DECREASES, IT DOES NOT INCREASE THE AMOUNT O OF THE CLINICIAN ENCOUNTER. PERSON WHO SAYS QUICKLY -- THAT'S FINE, THAT'S FINE, UNTIL YOU CHART YOUR PROMIS SCORE AND IT'S GOING UP OR DOWN, AND THEY'RE NOT SENSING IT, THEY'RE NOT TELLING YOU, THEY'RE MUCH MORE CONCERNED WITH THEIR WOUND DRESSING AND IT MAY TAKE A VISIT OR TWO FOR A CLINICIAN TO REALIZE NO, THAT'S NOT ALL HEAS GOING ON. THESE MEASURES HELP YOU DO THAT. THE CLINICAL MODEL, WHAT PEOPLE ARE TRAINED IN MEDICAL SCHOOL IS THAT THE PHYSICIAN KNOWS ALL THESE THINGS ABOUT THEIR PATIENTS. WE KNOW THEIR LEVEL OF DEPRESSION, WE KNOW THEIR FATIGUE, WE KNOW THEIR PAIN. SOMEHOW IN OUR 6.5 MINUTE CLINICAL ENCOUNTER, WE KNOW ALL THAT. A, I DON'T THINK WE EVER DID, B, WE DON'T HAVE TIME TO DRILL DOWN, WE DON'T HAVE THAT HALF-HOUR TO CHAT EVERY TIME. THIS HELPS GET OH TO IT. AND PROVIDES PRECISE MEASURE OF THINGS THAT ARE DIFFICULT TO OBTAIN BY OTHER MEASURES. WE CAN'T -- LET'S TALK ABOUT PHYSICAL FUNCTIONING. YES, YOU CAN SEND SOMEBODY FOR A CONSULT, WE CAN MEASURE ALL SORTS OF THINGS USING OBJECTIVE EQUIPMENT, WE COULD TAKE LONG SURVEY, BUT IN THE END, A SIMPLE FOUR ITEM PROMIS PHYSICAL FUNCTIONING MEASURE IS GOING TO GET A GENERALIZED LOOK AT THEIR LEVEL OF PHYSICAL FUNCTIONING WHICH WE LEARNED STUDY AFTER STUDY, CLINICAL POPULATION IS GOING TO CORRELATE IN THE 90s WITH TRADITIONAL MEASURES. IN A FRACTION OF THE TIME. SO WHY PROMIS? MORE PRECISE OUTCOMES, INTERNET ITEM RESPONSE -- RIGOROUS ITEM DEVELOPMENT, SEVERAL HUNDRED SCIENTISTS INVOLVED IN THIS SET VERY HIGH STANDARDS FOR THEMSELVES BEFORE STARTING THEIR WORK, THEY PUBLISHED PATIENTS IN ADVANCE, WHAT LEVEL OF VALIDITY, RELIABILITY IS IMPORTANT TO HAVE, MORE EFFICIENT ITEM ADMINISTRATION USING BANKING, COMPUTER ADAPTIVE TESTING, MAKING ALL THIS PUBLICLY AVAILABLE. GENERIC TOOLS THAT CAN BE ADMINISTERED ACROSS DISEASE GROUPS. AND TOOLS TO PROVIDE A COMMON METRIC. SO THANKS TO BILL RILEY, MANY INSTRUMENTS BUT ONE METRIC. LET'S TAKE THE CASE OF BLOOD PRESSURE. THE DEFINITION BLOOD PRESSURE IS BASED ON MERCURY READINGS. HOW MANY OF YOU ARE USING MERCURY BASED TO MEASURE BLOOD PRESSURE TODAY? WE DON'T, YET WE STILL CITE THE SAME COMMON METRIC ACROSS USING NEW MEASUREMENT TYPE. SO WE CAN USE OUTCOMES SUCH AS PROMIS AND THESE OTHER INSTRUMENTS ACTUALLY AS WELL IN CLINICAL USE IN THE HOSPITAL, IN CLIP CAL PRACTICE, IF CLINICAL RESEARCH BOTH BY THE NIH, INDUSTRY AND FDA, AND IN THINGS LIKE SURVEYS SUCH AS THE CDC CDC. BRIEF HISTORY OF PROMIS. INITIALLY A ROAD MAP INITIATIVE. SO DEVELOP AND EVALUATE PROs, COMPLETE -- CREATE COMPUTER ADAPTIVE TESTING, DISSEMINATE TO CLINICAL RESEARCHERS. UNDER PROMIS 1. FIVE CORDO MAINS, PAIN, FATIGUE, EMOTIONAL DISTRESS, SOCIAL WELL-BEING. PROMIS 2, COMING TO A CLOSE RIGHT NOW. ADDED A LOT OF RESEARCH SITES. ADDITIONAL CENTERS. A LOT OF VALIDATION AND A LOT OF DIFFERENT DISEASES. PROMIS FUNDING HAS BEEN FAIRLY SIGNIFICANT. IT'S BEEN DOLLARS WELL SPENT. I'M NOT A AWARE OF A STUDY THAT DIDN'T SHOW SOME IMPROVEMENT USING MEASURES AFROFFED BY THE NIH. PROMIS DOMAIN FRAMEWORK. THIS IS AVAILABLE IN MINIATURE FORM IN THE LITTLE ONE-PAGE BROCHURE. I WOULD URGE TO YOU GO TO NIHPROMIS.ORG ONLINE AND DO A DEEP DIVE. SO HERE'S THE GENERALIZED FRAMEWORK. WE HAVE MEASURES OF GLOBAL HEALTH, VERY FAST INSTRUMENTS, GET A QUICK SNAPSHOT OF A PERSON'S HEALTH. OTHERWISE WE HAVE NUMEROUS INSTRUMENTS ROUGHLY UNDER THE UMBRELLA OF PHYSICAL HEALTH, MENTAL HEALTH AND SOCIAL HEALTH. THINGS LIKE SYMPTOMS, FUNCTION, ET CETERA. SO LET'S GO IN. PROMIS BASIC PROFILE BANK. PEOPLE VERY OFTEN COME TO US AND SAY I WANT THE PROMIS, I KNOW THERE'S A PERSON WHO'S LOOKING FOR SOMETHING, AND TYPICALLY WE SAY OKAY, IF YOU'RE LOOKING FOR A SHORT SET OF MEASURES, A SHORT SET OF DOMAINS, THAT WILL HELP -- CAN BE HELPFUL IN RESEARCH BUT GENERALLY APPLICABLE TO MANY DISEASES, TAKE SOMETHING FROM THE PROFILE BANKS. NOW, I'VE GOT THE WRONG SET OF SLIDES UP HERE, BUT THOSE SLIDES THAT I HAD UP HERE THAT I SHOULD HAVE HAD WAS PRODUCED BY MEDICARE, CMS, AND WAS PRESENTED AT THE PCORI MEETING A FEW MONTHS AGO, AND IT SHOWED THAT THERE IS NO SUCH THING OR LESS THAN HALF OF THEIR PATIENTS ONLY HAVE A SINGLE DISEASE. SO WE CAN USE A SINGLE DISEASE BASED MEASURE ALL WE WANT AND IT CAN MAKE US FEEL GOOD, BUT THE REALITY IS THAT'S NOT THE PATIENT'S EXPERIENCE. SO WHEN WE'RE MEASURING FATIGUE, P IF WE DO A FATIGUE BASE FOR THIS CONDITION THEY HAVE AND ANOTHER ONE FOR THAT CONDITION THEY HAVE, WE'RE NOT GETTING AT THE PATIENT. WE'RE GETTING AT WHAT LOOKS GOOD IN OUR RESEARCH PAPERS. SO THERE'S REALLY A NECESSITY TO HAVE INSTRUMENTS THAT ARE ACROSS DISEASES BECAUSE IT'S A RARE PATIENT WHEN'S ONLY MEASURING IN THE ONE AREA. I'M FORTUNATE THE MAJORITY OF THE AREA OF RESEARCH OF ASSESSMENT MEASURES TO DATE OUTSIDE OF PROMIS HAVE BEEN, HEY, HOW DOES THIS INSTRUMENT THAT WE CREATED FOR THIS DISEASE WORK? I REMEMBER BEING AT MY FIRST INTERNATIONAL SOCIETY OF QUALITY OF LIFE RESEARCH CONFERENCE. THERE WERE 40 -- I WENT THROUGH THE PROGRAM. THERE WERE 40 DIFFERENT IF FATIGUE MEASURES USED SPONSORED BY THE NIH. 40 DIFFERENT FATIGUE MEASURES. AND THEY WERE FATIGUE FOR THIS, FATIGUE FOR THAT. THE ITEMS IN FATIGUE MEASURES OVERLAP A LOT BUT THEY SAID THIS IS THE FATIGUE FOR BLANK-BLANK DISEASE. THERE WERE 40 OF THEM AT THE TIME. AND I WAS STRUCK AT THE TIME WITH TWO THINGS. ONE IS THEY'RE NOT COMPARABLE. SECONDLY, WHEN WE LOOK AT IT RELATIVE TO CMS, IT'S KIND OF SILLY. IF YOU LOOK AT THE RESEARCH STUDIES, ANYONE WHO'S DONE THIS SAY WE'RE GOING TO OH DO IT IN THIS DISEASE AND WE'RE GOING TO NOT INCLUDE ANYBODY WHO HAS ANY OTHER CONDITION. THEN WE FIND OVER TIME, WHEN WE GET RID OF A HALF OF TWO THIRDS TO PEOPLE WE SEE IN OUR CLINIC BECAUSE THEY HAVE MORE THAN ONE CONDITION, AND SO WHAT PROMIS DOES IS IT DOESN'T MATTER WHICH CONDITION THEY HAVE. WE CAN TAKE THIS INSTRUMENT, USE IT ACROSS CONDITIONS. SO PHYSICAL FUNCTIONING, WE TALKED A LITTLE ABOUT THAT. PAIN INTENSITY. THAT'S GOOD. I THINK KAREN GOT INTO MY SLIDES. I'M NEVER GOING TO LIVE THAT ONE DOWN. FATIGUE AND SLEEP DISTURBANCE, DEPRESSION AND ANXIETY. WHEN PEOPLE COME TO ME, THEY VERY OFTEN SAY -- IT'S VERY HARD TO GET A PATIENT TO SAY THAT THEIR TREATMENT WAS SUCCESSFUL IF THEY HAVE DEPRESSION. IF YOU'RE A SURGEON AND OUTCOME IS DID NOT DO A GOOD JOB, YOU REALLY WANT TO KNOW WHAT THIS PERSON'S HEALTH IS, WHERE THEY'RE AT. IF A PERSON IS VERY DEPRESSED OR VERY ANXIOUS, WHAT YOU DID TO THEM MEDICALLY IS NOT GOING TO MATTER. AND THEN SATISFACTION WITH PARTICIPATION IN SOCIAL ROLES. PROMIS PLUS. THESE ARE ADDITION FALL INSTRUMENTS AVAILABLE IN EACH OF THESE AREAS TO DO A DEEPER DIVE. PAIN BEHAVIOR, SLEEP RELATED IMPAIRMENT, SERIES OF INSTRUMENTS IN SEXUAL FUNCTIONING. MORE OR LESS IMPORTANT. MENTAL HEALTH, WE CAN ACTUALLY DO MORE, DIVE IN DEEPER, ANG GER, APPLIED COGNITION, ALCOHOL INSTRUMENTS, PSYCHOSOCIAL ILLNESS IMPACT, HOW IS THE ILLNESS IMPACTING PSYCHOLOGICAL CONDITIONS, AND THEN IN SOCIAL HEALTH, ABILITY TO PARTICIPATE, SOCIAL SUPPORT, A LOT OF DISEASES, IF A PERSON DOESN'T HAVE ADEQUATE SUPPORT GOING ON, THEY DON'T GET BETTER. OR IF THEY DO GET BETTER, THEIR RECIDIVISM -- THESE ARE INSTRUMENTS AGAIN AVAILABLE IN VERSIONS UNDER A MINUTE. YOU CAN TRACK THIS OVER TIME, YOU CAN USE IT AS A VARIABLE WHEN BEGINNING TO TREAT A PATIENT, YOU CAN USE IT AS AN OUTCOME MEASURE. AND A LOT OF ADDITIONAL MEASURES, I PUT THEM UNDER THE UNOFFICIAL CATEGORIZATION OF EXTRAS, UPPER EXTREMITY VERSUS MOBILITY, THESE ARE SUBSETS OF PHYSICAL FUNCTIONING SCALE, IF INDEED -- IT TURNS OUT THE PHYSICAL FUNCTIONING MEASURE ON ITS OWN DOES A PRETTY GOOD JOB FOR PEOPLE REGARDLESS IF THEY BROKE THEIR LEG OR THEIR ORM, IT'S A GENERALIZED MEASURE BUT WE HAVE MORE TARGETED TREUMENTS FOR THOSE PEOPLE WHO THINK THEY NEED IT. MOBILITY, ASTHMA, GIs, SELF-EFFICACY, DISEASE CONTROL, PEER RELATIONSHIPS, WHATEVER. LOTS AND LOTS OF TARGETED INSTRUMENTS THAT VERY OFTEN CAN TAKE THE REPLACE M OF SOME EXISTING OR PROVIDE ADDITIONAL INSTRUMENTS THAT CAN BE USED. IN PEDIATRIC, THIS IS JUST A SAMPLING. THERE HAVE BEEN SIGNIFICANT EFFORTS PRIMARILY FOCUSED AT UNC OF BREN BRINGING PEDIATRIC EQUIVALENTS TO THE INSTRUMENTS, PROXY MEASURES BY PARENTS OF CHILDREN EIGHTS FUNCTIONING. NEURO-QOL. CONTRACT MECHANISM, WE HAD FIVE PRIMARY RESEARCH SITES. 12 PUBLICATIONS, MUCH SMALLER EFFORT. YOU'RE GOING TO PIEND A COMMON THREAD HERE IS THAT WE CREATED ALL THESE ON A COMMON TECHNOLOGY PLATFORM SO YOU CAN RUN ONE OF THESE, YOU CAN RUN ALL OR ALMOST ALL OF THESE, AND THAT'S WHAT WE'VE SET UP IN THE CLINICAL CENTER NOW. ITEM BANKS, 14 ADULT DOMAINS, EIGHT PEDIATRIC DOMAINS. IF THE DOMAIN IS TH THE SAME AS DOMAIN COVERED BY PROMIS, VERY OFTEN THERE'S SOME SYMPTOM-SPECIFIC ISSUES OR DIFFERENCE. THEY'VE ALL BEEN TRANSLATED, VALIDATED IN SPANISH. IN THIS CASE, AS OPPOSED TO PROMIS, THEY REALLY ARE TARGETED AT NEUROLOGICAL CONDITIONS, AND THERE ARE SPECIFIC SYMPTOMS JUST NOT APPLICABLE TO GENERAL POPULATION. WE CAN'T -- SERP THINGS JUST DON'T GO ACROSS. GENERALIZE MEUR OWE QOL DES MOINES FRAMEWORK, ROUGHLY DIVIDED AMONG PHYSICAL, MENTAL AND SOCIAL HEALTH, A LITTLE DEEPER DIVE HERE. PHYSICAL HEALTH FOR ADULTS, FATIGUE, SLEEP IMPAIRMENTS, ET CETERA, BUT WE GET INTO AREAS MORE APPLICABLE, NOT THINGS THAT ARE GOING TO TYPICALLY COME IN YOUR CROSS STUDY RESEARCH. GREATER CONCERN ABOUT PAIN IN AREAS OF THE BODY. MENTAL HEALTH, ADDING COMMUNICATION END OF LIFE CONCERN, NOT SOMETHING PROBABLY GOING TO ASK OF EVERY PATIENT BUT IF YOU HAVE A SERIOUS DISEASE, THOSE INSTRUMENTS ARE THERE AND YOU SEE SOME OF THE STIGMA, NOW WE'RE LOOKING AT CERTAIN DOMAINS THAT MAY NOT BE APPLICABLE IN CERTAIN DISEASE AREAS. AND IN SOCIAL HEALTH, ABILITY TO PARTICIPATE, INTERACTIONS WITH ADULTS FOR PEDIATRICS, AGAIN, ADDITIONAL AREAS THAT MAY BE USEFUL IN RESEARCH OR CLINICALLY. NIH TOOLBOX. ANTHAT WENT UP BY A DOZEN THIS WEEK WITH THE ELECTRONIC RELEASE IN THE JOURNAL OF INTERNATIONAL NEUROLOGICAL SOCIETY. TOOLBOX IS A LITTLE DIFFERENT. WE'LL GET INTO IT. IT WAS DESIGNED TO HAVE 40 -- FOUR 30-MINUTE DOMAIN BATTERIES. A LOT OF SUPPLEMENTAL INSTRUMENTS, 108 INSTRUMENTS IN ALL. IN THOSE AREA, AGAIN, PARTICULARLY IN EMOTION, THAT MATCHED PROMIS DOMAINS, PROMIS IS UTILIZED, ALL SPANISH, AGAIN, DESIGNED TO BE NON-DISEASE-SPECIFIC, VALIDATED IN MANY DISEASES AND THAT WORK IS ONGOING. COGNITION DOMAIN FRAMEWORK CREATED INSTRUMENTS IN SIX PRIMARY AREAS. I'M GOING TO RUN THROUGH THESE KIND OF QUICKLY. THIS 30 MINUTE IS A DOMAIN BATTERY, REPLACES A THREE HOUR NEUROPSYCH EVALUATION, TOOLBOX ALSO WANTED TO BE SHORT BUT SHORT MEANT A BATTERY IN UNDER 30 MINUTES. THERE ARE NO INSTRUMENTS IN COG MISSION. COGNITION. THEY TAKE TWO TO FIVE MINUTES BUT REPLACE SOME THAT TAKE 20 TO 30 MINUTES, REQUIRE SOME WITH A LARGE DEGREE PERIOD AND THESE DO NOT. WE HAVE EXECUTIVE FUNCTIONING TASKS, THEY'VE VALIDATED AGAINST NINDS EXAMINER, BEHAVIOR QUESTIONNAIRE, WORKING MEMORY, SOME OF THE NUMBER SEQUENCING, PSAT TEST, EPISODIC MEMORY TEST, MEMORIZE THE SEQUENCE, BE SHOWN A SEQUENCE OF EVENTS, SCRAMBLE THEM UP ON THE SCREEN AND RETAKE THEM. LANGUAGE, A COMPUTER ADAPTIVE VOCABULARY TEST, VALIDATED THE PPVT, SELF GUIDED TO GET A HIGHLY ACCURATE MEASURE OF A PERSON'S V VOCABULARY ABILITY, AGAINST PEABODY AND THE WRAT. ORAL READING RECOGNITION, AGAIN VALIDATING AGAINST THE WRAT IT. THREE MINUTE TEST OF ORAL READING RECOGNITION. PROCESSING SPEED, ARE THINGS THE SAME OR NOT. VALIDATING AGAINST THE WECHSLER -- 90-SECOND TEST, HIGHLY RELIABLE. AGAIN, WE CAN DO IT THAT BECAUSE IT'S COMPUTER-ADMINISTERED. MOTOR DOMAIN FRAMEWORK, ANOTHER SET OF OBJECTIVE MEMBERS NOW, NOT SELF-REPORT MEASURES, SO DEXTERITY, THE TEAM UTILIZED TRADITIONAL PEG BOARD. THE GOAL WAS ALWAYS NOT TO CREATE SOMETHING FROM SCRATCH, IF SOMETHING OUT THERE EXISTED THAT WAS INEXPENSIVE AND BRIEF, USE IT. STRENGTH, USING A GRIP STRENGTH INSTRUMENT. THE TOOLBOX VALIDATION -- THIS MANUFACTURER ASKED IF THEY COULD SWITCH THEIR NORMS TO THE NIH TOOLBOX NORMS BECAUSE THEY WERE BETTER THAN THE NORMS THAT THEY PUBLISHED. BALANCE TESTS. AT THE TIME THIS WAS CREATED, THIS TEST REPLACED THE $100,000 PIECE OF EQUIPMENT FOR PERSONS WEARING A GAIT BELT. IT SENDS 15,000 PIECES OF INFORMATION VIA BLUETOOTH ABOUT POSTURAL AND INTERIOR SWAY. GET A HIGHLY ACCURATE SENSE OF BALANCE. LOCOMOTION USING A GATE SPEED TEST, ENDURANCE. THIS TEAM, IT'S INTERESTING, SOMEONE MENTIONED EARLIER, A 4 MINUTE, 6 MINUTE WALK. THIS TEAM SAID WE CAN MAKE IT SHORTER. THEY VALIDATED A 2 MINUTE WALK TEST, CORRELATES IN THE MID 90s WITH THE 6 MINUTE WHICH SAY THAT'S NOT -- WHAT DID WE DO THAT FOR? WELL, 6 MINUTES TO 2 MINUTES IS A LOT WHEN YOU'RE ASSESSING A LOT OF DIFFERENT AREAS. SENSATION DOMAIN. HEARING. WORD AND NOISE TEST AS WELL AS A HEARING THRESHOLD TEST AND HEARING HANDICAPPED INVENTORY, WHICH IS A SELF-REPORT MEASURE. VISUAL SPATIAL TEST, A TEST GIVEN BY COMPUTER SCREEN AND IT'S A COMPUTER ADAPTIVE TEST ZEROING IN ON A PERSON'S VISUAL ACUITY LEVEL. THERE ARE DIFFERENT VERSIONS HERE FOR YOUNG KIDS VERSUS OLDER ADULTS. AND A SUPPLEMENTAL VISION RELATED QUALITY OF LIFE. LOOKING AT THE NEI VFQ, MAKING IT SHORER AND COVERING MORE DOMAIN. ABILITY TO DO HEAD ROTATIONS WHILE WALKING, WHILE DOING OTHER THINGS. THIS PERSON IS WEARING THIS KIND OF FUNNY LOOKING HEADSET ON. TAKES FOUR MINUTES OR SO. BUT IT'S ABLE -- YOU'RE MOVING YOUR HEAD BACK AND FOR, CAN YOU RICK NIERECOGNIZE STIMULI, HOW LONG TO RECOGNIZE STIMULI. ODOR IDENTIFICATION TEST. STANDARD TEST COSTS BETWEEN 12 AND $30. CAME UP WITH A SET OF SCRATCH AND SNIFF CARDS THAT ARE AVAILABLE FROM TWO MANUFACTURERS ON LINE FOR $1.50. SO THERE IS SOME COST SOMEWHERE HERE. AND IT'S A TEST WHERE A PERSON DOES A LITTLE SCRATCH AND NIF AND THEY'RE GIVEN ITEMS TO IDENTIFY, A LOT OF CORRELATION WITH THINGS LIKE PARKINSON'S DISEASE, WHERE THE OLFACTORY SENSE CAN BE USED TO -- AS A MARKER RELATIVE TO A LOT OF NEUROLOGICAL FUNCTIONING, SPATIAL TASTE TEST, PAIN INTERFERENCE MEASURE, ADOPTED SENSATION HERE IN THE NIH TOOLBOX CH CAME FRO. CAME FROM PROMIS INITIALLY. ITS OWN EMOTION DOMAIN FRAMEWORK. THESE INSTRUMENTS ARE -- THERE'S A FAIR AMOUNT OF OVERLAP WITH PROMIS BUT THE EMPHASIS HERE WAS ON HEALTH. HEALTH AND DISEASE TYPICALLY TAKE UP THE SAME TRAIT CONTINUUM, BUT THERE WERE ADDITIONAL ADDED HERE. INSTRUMENTS THAT FURTHER EXPAND WHAT'S AVAILABLE. I SHOULD POINT OUT THAT GETTING ACCURATE SCORES IN ALL OF THESE AREAS, GENERALLY SPEAKING IT TOOK ABOUT 12 MINUTES. SO 12 MINUTES, AND WE'LL BE RELEASING NEW ALGORITHMS FOR BOTH THE TOOLBOX AND PROMIS, WE'RE FINDING THAT WE CAN -- OUR LONGER TESTS WERE BASED ON PEOPLE WHO WERE SO FAR AWAY THAT THE INSTRUMENTS DIDN'T COVER THEM. EVEN THOUGH I SHOWED YOU BEFORE, PROMIS HAS FEWER FLOOR EFFECTS, THERE ARE STILL THOSE PEOPLE AT THE VERY END AND WE CAN ACTUALLY CUT THEIR TEST LENGTH BY QUITE A BIT. ANOTHER RELATED EFFORT PRESETTA STONE, LOOKING AT EXISTING -- HOW DO THESE INSTRUMENTS RELATE TO EXISTING STEUMENTS? I'VE BEEN COLLECTING DATA FOR A DECADE, I HAVE ALL OF HE'S BLANK SCORES, AND IT IF I SWITCH THESE INSTRUMENTS, DO I THROW ALL THAT AWAY? THE ANSWER IS NO, YOU DON'T NEED TO. MAYBE YOU COULD SEE THIS. I CAN BARELY SEE IT. WHAT'S HAPPENED IS, IS THAT THE PROSETTA STONE EFFORT HAS GONE AHEAD AND GIVEN THINGS LIKE THE CESD, AND THE PROMIS MEASURE TO HUNDREDS OR THOUSANDS OF INDIVIDUALS, AND CREATED LOOK-UP TABLES THAT SAY I HAVE A SCORE ON THE CESC, HOW DOES IT RELATE TO A SCORE ON A PROMIS MEASURE? THIS ENABLES YOU TO, IF YOU'VE BEEN USING THE OLD SCORES, YOU CAN BRING THEM FORWARD, YOU CAN TAKE THE PROMIS SCORES, YOU CAN GO BACKWARD, YOU CAN COMPARE TO OTHER STUDIES. THE OTHER THING THAT IT REALLY BRINGS FORWARD IS THERE'S CLEARLY A LOT DEEPER HISTORY OF RESEARCH IN CESC, WHAT ARE OUR CLINICAL CUTOFFS, IF WE CAN MATCH THE SCORES IN THE CESC TO PROMIS, WE CAN SEE TRADITIONALLY, CLIPICALLY WITH THIS SCORE, WE WOULD ACCESS IT THIS TYPE OF TREATMENT, SO WE CAN MIGRATE THAT TYPE OF RESEARCH TO BE USED WITH PROMIS AS WELL. THERE'S A GROWING NUMBER OF INSTRUMENTS, ANY COVERED BY PROSE TMENT TA STONE HAS A TECHNICAL PAPER ASSOCIATED WITH IT AND YOU CAN LOOK UP VARIOUS SCORES. SO CLINICAL USE OF PROMIS IS EXPANDING QUITE RAPIDLY. YOU ARE NOT THE FIRST HERE AT THE CLINICAL CENTER. CLEVELAND CLINIC HAS BEEN UTILIZING THE PROMIS MEASURE, I WANT TO SAY FOR FOUR OR FIVE YEARS NOW, COMPUTER ADAPTIVE TESTING FORMAT. NORTHWESTERN HAS HAD IN ITS CLINIC ASSESSING PATIENTS AS THEY'RE WAITING IN THE WAITING ROOM. UCLA, DOD PAIN TASK FORCE, JOHN HOPKINS, NCCAM, STANFORD, I SHOULD TALK ABOUT EPIC FOR A FEW MINUTES. EPIC HAS HAD PROMIS SHORT FORM AVAILABLE FOR A LITTLE OVER A YEAR, AND A FEW OF US SPENT A DAY WILL A COUPLE OF WEEKS AGO AND I'M HAPPY TO REPORT THAT EPIC, BY THE END OF THIS YEAR, SHOULD HAVE ALL THE PROMIS SHORT FORMS FROM ALL THE INSTRUMENTS AVAILABLE AS WELL AS ROT BASED SCORING. RIGHT NOW IT HAS BLOCK LOOK UP BASED SCORING SCORING, BUT THEY ALSO ADMIT THERE'S NO -- THEY'LL HAVE ALL THE CATs AVAILABLE AS WELL DIRECTLY. SO ANY USER OF EPIC WILL BE ABLE TO JUST PICK FROM A DROPDOWN AND INCLUDE PROMIS SCORES, RESULTS OF PROMIS ASSESSMENT IN CLINICAL ASSESSMENT IN THE CLINICAL WORK FLOW. I BELIEVE ULTIMATELY THE USE OF ALL THESE INSTRUMENTS WILL REALLY BE FORWARDED BETTER FROM INSTRUMENTS ARE INSERTED IN THE NORMAL CLINICAL WORK FLOW, THE NORMAL TECHNOLOGY IN THE EMR YOU'RE USING EVERY DAY. BY USE OF HAVING THE SERVER IN THE CLINICAL CENTER, WE'RE NOT QUITE THERE YET. HERE'S MY ORDERS, WHENEVER I SEE A PATIENT LIKE THIS, I CLICK THIS, THIS AND THIS AND IT THAT WILL COME UP. WE'RE STILL WORKING ON THAT HERE. SASKIN IS A GROUP OUT OF OF INDIA, THEY WORK ON CUSTOM EMRs AROUND THE WORLD, PEOPLE WHO CUSTOMIZE -- THEY HAVE ALSO MADE THE PROMIS CAT AVAILABLE TO INS TUESDAYS WHO WANT TO DO THAT. ORTHOPEDICS IS AN AREA H HAS REALLY TAKEN OFF WITH WITH PROMIS INSTRUMENTS. PART OF THE REASONS IS THAT ORTHOPEDICS HAS HAD A REALLY LONG-STANDING TRADITION OF THE VOICE OF THE PATIENT AND THEY'VE ALSO HAD THE FUNDING TO BE ABLE TO LOOK AT NEW THINGS. THEY HAVE THEIR OWN INSTRUMENTS AND THEY'RE RAPIDLY SWITCHING -- THEY REALIZE THEY'RE SELF-MADE INSTRUMENTS, PROMIS DOES A BETTER JOB THAN THE INSTRUMENT DEVELOPED FOR X, Y OR Z, GROUPS SUCH AS OUT OF HARVARD, UNIVERSITY OF UTAH,, A PILOT PROJECT TO HAVE ANYBODY WHO'S APPLYING FOREFOOT AN FOR FOOT AND ANKLE SURGERY, CERTIFIED, RE-CERTIFIED, YOU'LL HAVE TO HAVE X NUMBER OF YOUR PATIENTS FILLED OUT AND THEY INDICATE THAT THEY SEE A CHANGE. THEY'RE NOT GOING TO CERTIFY BECAUSE YOU CAN ANSWER A MULTIPLE CHOICE TEST, YOUR PATIENTS FEEL THAT YOU HELP THEM. AMERICAN BOARD OF ORTHOPEDIC SURGEONS, THE GIANT GRANDFATHER OF ALL THESE GROUPS, IS UNDERGOING A SIMILAR PILOT RIGHT NOW AS WELL. TELL ME MORE. IT PROVIDES READY ACCESS FOR THE PROMIS INSTRUMENTS, NEUROQOL. MOST OF THE TOOLBOX INSTRUMENTS. SOME OF THE TOOLBOX INSTRUMENTS, WE'RE WAITING TO SEE FEEDBACK FROM YOU WHAT IS NEEDED, WHAT'S THERE, BUT THERE'S A SERVER THAT MEANS THE DATA IS HERE, THE ASSESSMENT CENTER THAT WE RUN FOR EXTRAMURAL RESEARCH LAST YEAR GAVE 600 -- WAS USED OH TO CONDUCT 650 STUDIES AROUND THE WORLD OFF A SINGLE SERVER BASED AT NORTHWESTERN. THAT'S GOOD FOR OUR MANAGEMENT, IT'S NOT SO GOOD IF YOU HAVE AN INSTITUTION, FEDERAL GUIDELINES FOR KEEPING YOUR DATA BEHIND YOUR OWN CLINICAL FIREWALL, WHICH IS WHY WE'RE EXCITED ABOUT EPIC, YOU CAN USE IT HERE, YOUR DATA NEVER LEAVES, THERE'S NO COMMUNICATION BACK TO NORTHWESTERN, IT'S ALL HERE, WE DON'T DO IT. WE ALSO HAVE OFFLINE AND API OPTIONS, API IS APPLICATION PROGRAMMING INTERFACE, IT IS SOMETHING WE CREATED AS A ONE OFF PROJECT FOR FUN ONE DAY AND IT TURNS OUT IT'S BEING RAPIDLY ASKED FOR AND IT'S THE ABILITY TO TAKE ALL OF PROMIS, MEUR QOL, EMOTIONAL HEALTH INSTRUMENTS, A LITTLE LIBRARY WITH ALL THE ALGORITHMS, HANDED TO A GROUP AND SAY YOU'VE GOTTEN EXISTING EMR, LET'S TAKE THIS, PUT IT ON YOUR SERVER, YOU CAN SERVE UP YOUR OWN INSTRUMENTS. YOU DON'T HAVE TO USE OUR WHOLE BIG ASSESSMENT CENTER, YOU CAN JUST DIRECTLY BUILD IT IN TO WA YOU'RE DOING. IN PLACES SUCH AT UNIVERSITY OF UTAH ARE PUTTING IT THERE, EPIC, WE USE THIS TO ACCESS INSTRUMENTS, ET CETERA. VERY EXCITED. EPIC, I THOUGHT THAT WAS GOING TO MAKE MY MONTH FOR SOME REASON, AND A LOT OF THANKS TO MEMBERS OF THE STEERING COMMITTEE IN PARTICULAR, WE A APPROACHED EPIC ABOUT FIVE YEARS AGO. ORIGINALLY THEY SAID WE ARE NEVER GOING TO HAVE PROs IN EPIC. WE'VE GONE TO THAT, NEVER SAY FOREVER. I'VE LEARNED THAT A LONG TIME AGO. RED CAP. CURRENTLY -- RED CAP ADDED CAT ON A PILOT BASES AND THE SHORT FORMS A COUPLE OF YEARS AGO. PRETTY CLUNKY -- I'M SORE EE THE SHORT FORM IS NOT FUNKY, RED CAP RIGHT NOW IS PILOT TESTING THE FULL USE OF THAT API, ALL PROMIS INSTRUMENTS, SHORT FORMS AND CATs, THE OTHER SELF-REPORTED INSTRUMENTS, HOPING WILL BE AVAILABLE TO ANY RED CAP INSTITUTION INTERNALLY, SO AGAIN, DATA -- ONE OF THE 900 SERVERS THAT RED CAP RETURNS AROUND THE WORLD BY THE END OF THE CALENDAR YEAR. THAT MEANS IF YOU HAVE COLLEAGUES IN JAPAN AND CHINA AND THE NETHERLANDS AND EUROPE, THEY WILL HAVE THE CAPABILITY OF RUNNING THOSE INSTRUMENTS INTERNLLY ON THEIR SERVER. 2015 iPAD APPS, MONDAY OF THIS WEEK, THE NATIONAL CHILDREN'S STUDY TRAINED PEOPLE FOR THE VANGUARD STUDY, THE 40 CENTERS, EVENTUALLY 100,000 PARTICIPANTS, RIGHT NOW A 5,000 PERSON STUDY ACROSS 40 AROUND SELECTED COUNTIES IN THE UNITED STATES. THEY TRAIN THEIR FIELD WORKERS ON MONDAY OF HOW TO USE THE IPAD VERSION, SO A TABLET VERSION OF THE NIH TOOLBOX COGNITION INSTRUMENT FOR 3-YEAR-OLDS. A VERY CUTE VIDEO WHICH WE MIGHT ACTUALLY SHOW IN A MINUTE. CONTENT CREDIT. I'M RUNNING VERY EARLY BECAUSE THIS IS ACTUALLY NOT MY RIGHT SET OF SLIDES. LET ME GET ONE SECOND HERE. I WANT TO JUST CATCH UP IN ONE AREA. WE'RE GOING TO DO SOME REALLY QUICK JUMPING HAID HERE. THAT WASN'T TOO PAINFUL. WHEN TALKING TO CLINICIANS AND RESEARCHES, NOW WE'RE SWITCHING MORE TO THE -- PSYCHOMETRICALLY SOUND, GO BACK AND LOOK AT THE INSTRUMENTS YOU'VE USED, YOU WERE TAUGHT, AND SEE CAN YOU FIND EVIDENCE OF RELIABILITY OR VALIDITY OF INSTRUMENTS USED EVERY DAY. I'M NOT GOING TO SIT HERE AND BAD MOUTH, BUT ALMOST A GUARANTEE YOU'RE USING AN INSTRUMENT THAT HAS VERY QUESTIONABLE RELIABILITY. THE BIG ONE EVERYONE USES IN EVERY EMERGENCY ROOM IN THE WORLD IS THE ZERO TO 10-POINT PAIN SCALE. IT'S HIGHLY, HIGHLY UNRELIABLE. IT WAS NEVER, EVER, EVER, EVER, EVER DESIGNED FOR INDIVIDUAL CLINICAL USE. IT IS ALWAYS, ALWAYS, ALWAYS, ALWAYS, ALWAYS USED FOR CLINICAL USE. THE ACCURACY OF ANY SCORE BETWEEN 3 AND 7 ISN'T THERE, AND GUESS WHAT, WE CHANGE PEOPLE'S TREATMENT BASED ON THAT SCORE, WE CHANGE THE DRUG REJ MAN THEY'RREGIMENTHEY'RE ON, THERE IS NO RELIABILITY FOR SCORES IN THE MIDDLE OF THAT RANGE. AND YET IT'S USED ALL THE TIME. SO PROMIS SAID LOOK, IF WE'RE P IN AN OPEN INSTRUMENT, WE'RE GOING TO GIVE IT A VERY HIGH LEVEL, IT'S GOING TO HAVE TO PASS NOT JUST THE SMELL TEST, IT'S GOING TO HAVE TO PROVE THAT IT CAN DO ITS JOB. BRIEF AND EASY TO USE, INTELLECTUAL PROPERTY, I WAS JUST SO -- BILL RILEY AND I WERE SO SHOCKED A COUPLE WEEKS AGO TO HEAR THAT THE PREDOMINANT FEELING ON CAM P PUS WAS -- OR THROUGH HHS WAS THESE INSTRUMENTS HAVE HIGH COSTS ASSOCIATED WITH THEM. THEY DON'T. THEY ARE FREE. COVER THE FULL RANGE OF TRAIT, NO FLOOR, NO CEILING. WE TALKED ABOUT THE PROBLEM WITH NO CEILING. IF YOU HAVE A CEILING, I'M GOING TO SHOW YOU A SLIDE IN A MINUTE HERE, AND AGAIN NO FLOOR. PHYSICAL FUNCTIONING, P A PERSON IS BED BOUND, WE'RE ABOUT AS LOW AS WE CAN GO. ALL THESE NIH SYSTEMS USING A COMMON METRIC, PROBLEM WITH A LOT OF INSTRUMENTS, UNLESS YOU KNOW THE METRIC, I KNOW IF THIS SCORE GOES FROM HERE TO HERE, I KNOW WHAT IT MEANS. THE SCORES ARE DIFFERENT. THE SCORE HERE IS 1 TO 10. THE SCORE HERE IS 100 -- 70 TO 95. YOU DON'T KNOW WHAT IT MEANS. TRYING TO PUT THESE ON THE SAME SCALE. SAME INSTRUMENTS, SAME DISEASES. SAME SCALE REGARDLESS OF INSTRUMENT FORMAT. FOUR OF THESE INSTRUMENTS, IT IS THE CASE WITH PROMIS AND TOOLBOX, MANY OF THESE INSTRUMENTS ARE AVAILABLE IN VARIOUS LENGTHS IF IT'S ON PAPER OR COMPUTER FORMAT, NO MATTER IF YOU USE SINGLE ITEMS, SHORT FORM, LONG FORM OR CAT, THE SCORE IS ON THE SAME METRIC. OBVIOUSLY THE RELIABILITY OF A ONE OR TWO ITEM INSTRUMENT IS GOING TO BE LESS THAN AN EIGHT ITEM INSTRUMENT. BUT THE SCORE IS COMPARABLE. WHICH IS INCREDIBLY HELPFUL. BUT MOST LEGACY MEASURES FAIL TO MAKE THIS GRADE. PSYCHOMETRICALLY SOUND, NOT ALWAYS. BRIEF, EASY TO USE, RARELY. PATIENTS DON'T DO THEM. INTELLECTUAL PROPERTY PRE, NOT ALWAYS. WE HAVE A CHANGING MARKETPLACE IN THERE, BUT THERE'S A LOT OF MARKET CONDITIONS THAT SAY, HEY, WE WANT TO CHARGE FOR THESE THINGS. APPLICABLE IN -- SETTINGS, SOMETIMES. AGAIN, GO BACK TO AN INSTRUMENT THAT YOU'RE USING IN YOUR DISEASE GROUP, YOU'RE PROBABLY GOING TO FIND THE VALIDITY OF INFORMATION IS BASED ON A DIFFERENT DISEASE IN A PARTICULAR SETTING, AND I'M GOING TO SAY 100% OF THE TIME, I'M SURE THERE'S AN EXCEPTION SO I'LL PROTECT MYSELF. 95% OF THE TIME, ONLY FOR PATIENTS IN A NARROW AGE BAND THAT ONLY HAD THIS DISEASE, HAD NO OTHER ESTIMATES, HAD NOTHING ELSE THAT DIDN'T HAVE EMOTIONAL SIDE EFFECTS AND THAT DIDN'T HAVE ANYTHING ELSE GOING ON. FOR THOSE OF YOU, IF I CLEARED THE ROOM OF ANYBODY WHO HAD PATIENTS THAT MET THAT CRITERIA, THE ROOM WOULD BE EMPTY RIGHT NOW. THAT IS JUST NOT REAL PATIENTS. OKAY. AND THE DIFFERENT SUBGROUPS, RARELY. AVAILABLE IN MULTIPLE LANGUAGES, SOMETIMES. RARELY WITH THE SAME MEANING. THAT'S A REALLY GOOD POINT. A LOT OF TRANSLATION WHERE SOMEBODY'S RESEARCH ASSISTANT, SOMEBODY DOWN THE HALL TRANSLATED THE INSTRUMENT. THAT'S NICE. IT LOOKS GOOD. HOPEFULLY IT DOESN'T LOOK LIKE A BUNCH OF INSTRUCTIONS THAT CAME FROM AN ASIAN MANUFACTURING COMPANY THAT SOMEBODY DIDN'T KNOW WHAT TO DO, BUT WHEN WE'RE DONE, THE INSTRUMENT DOESN'T HAVE THE SAME MEANING. THE GOAL OF PROMIS, THE GOAL OF THE TOOLBOX MEASURES IN SPANISH WAS THAT A SCORE MEANS THE SAME THING IF YOU'RE GIVEN AN ENGLISH OR SPANISH OR ANOTHER LANGUAGE, SO THE SCORE IS DIRECTLY COMPARABLE. THAT WORKS FOR -- WE REMOVED ITEMS THAT HAD DIFFERENCES IN LANGUAGE. WE REMOVED LANGUAGES THAT HAD SIGNIFICANT DIFFERENCES BASED ON SUBGROUP. THE ONLY INSTRUMENTS OUT OF EVERYTHING I MENTIONED HERE THAT PERFORMED DIFFERENTLY ARE THE TOOLBOX VEUMENTS. ONE IS VOCABULARY, BECAUSE VOCABULARY IS DIFFERENT DEPENDING ON LANGUAGE, AND OTHER IS ORAL READING, WHICH IS REALLY THE ONE WHERE LANGUAGE MAKES THE CHANGE BECAUSE ANY FOUR OR 5-YEAR-OLD CAN ACCURATELY READ ANY WORD IN SPANISH. IF IT'S GOT THE CORRECT ACCENTS ON IT. MOST 50-YEAR-OLDS CANNOT ACCURATELY READ EVERY WORD IN THE ENGLISH LANGUAGE. I'M GOING TO SAY ALL 50-YEAR-OLDS CAN'T READ IT, BECAUSE WE HAVE SO MUCH EXCEPTIONS OF RULES AND THINGS LIKE THAT. SO -- ALMOST NEVER, NO FLOOR EFFECT SOMETIMES, NO CEILING EFFECTS, NEVER. AVAILABLE ACROSS THE AGE SPAN, RARELY. MOST ARE CREATED FOR A NARROW AGE RANGE. REALLY RARELY, VERY FREU FEW INSTRUMENTS -- SAME SCALE APPLICABLE, AGAIN NEVER, AND SAME INSTRUMENT, DIFFERENT FORMATS, REALLY DOESN'T HAPPEN. SO LET'S TALK ABOUT THE CEILING ISSUE A LITTLE MORE BECAUSE THIS REALLY BOTHERS ME. WORK THAT WAS DONE COMPARING THE PROMIS ANGER CAT QUESTIONNAIRE IN A LONG TEUT NAL LONGITUDINAL STUDY. AT BASELINE, WE'VE GOT BOTH INSTRUMENTS ARE ABLE TO ASSESS WHERE A PATIENT STARTS AT ONE MONTH, WE CAN SEE BOTH INSTRUMENTS HAVE SHOWN THAT THERE'S AN IMPROVEMENT, SOMETHING'S HAPPENED, ARE INTERVENTIONS WORKING, IT'S GREAT, WE FEEL GOOD ABOUT IT. LEGACY MEASURE AFTER THREE MONTHS DEMONSTRATES THERE'S NO IMPROVEMENT, HASN'T CHANGED, I'M THE INSURANCE COMPANY, I SAY LOOK, THERE'S JUST NO REASON TO GIVE HAD IT TREATMENT MOR THIS TREATMENT M ORE THAN ONE MONTH. THAT'S THE PUBLISHED RESEARCH. WAIT A MINUTE. THE PROMIS INSTRUMENT SHOWS THE PROMIS EFFECT IS TWICE AS MUCH AFTER THREE MONTHS THAT IT CAME THERE. WHAT'S GOING ON? PROMIS QUESTIONNAIRE DOES NOT HAVE A CEILING. THIS QUESTIONNAIRE, CEILINGED OUT AND WAS UNABLE TO SHOW IMPROVEMENT BECAUSE THERE WEREN'T ITEMS THAT DEALT WITH THAT IMPROVEMENT LEVEL. THE TREATMENT DID INDEED HAVE AN EFFECT. THE MEASURE WAS NOT ABLE TO CAPTURE IT. OF ALL THE SLIDES I SHOW, THIS ONE BOTHERS ME THE MOST, SUCCESSFUL STUDY, REAL IMPROVEMENT AFTER THREE MONTHS, TRADITIONAL MEASURE, WOULDN'T HAVE CAPTURED IT, WE'RE DONE. THE WORK YOU DID FOR THREE MONTHS WAS SHOWN TO TO BE NOT SUCK SUS FULL, NO BETTER THAN ONE MONTH CUT OFOFF, THAT'S NOT WHAT'S HAPPENING. THERE REALLY WAS IMPROVEMENT THERE. IT'S THE MEASURE THAT WAS CAPPED, NOT THE WORK YOU DID, NOT THE RESEARCH YOU DID, NOT THE WAY THE PATIENTS REALLY FEEL. THE MEASURE SIMPLY WAS DESIGNED FOR FOURTH GRADERS, AND WHEN A KID MADE IT UP TO EIGHTH GRADE, THERE WERE NO EIGHTH GRADE ITEMS, SO THEREFORE THERE'S NOTHING TO SHOW THEY REALLY IMPROVED. HERE'S MY CMS SLIDE. PERCENTAGE OF MEDICARE BENEFICIARIES WERE MULTIPLE CHRONIC CONDITIONS. 32% HAVE TWO OR THREE. BASICALLY 68% OF MEDICARE BENEFICIARIES HAVE MORE THAN ONE CONDITION. HOW IN THE WORLD CAN WE USE INSTRUMENTS THAT HAVE ONLY BEEN PROVEN RELIABLE AND VALID ON A SINGLE DISEASE AT ONE TIME? AND THAT'S WHAT WE ALL DO EVERY SINGLE DAY. WE NEED TO MOVE ON TO INSTRUMENTS AND MEASURES THAT CAN CAPTURE A TRAIT ACROSS MULTIPLE CONDITIONS, OTHERWISE BECAUSE LET'S FACE IT, ALL INSTRUMENTS THAT ARE DISEASE-BASED WERE VALIDATED IN THIS GROUP RIGHT HERE. THEY WERE NOT VALIDATED TO 68% OF REAL PEOPLE THAT ARE OUT THERE. OFTEN PATIENTS WILL -- PHYSICALLY ACTIVE PATIENTS NOW RECOVERED FROM AN ACCIDENT DON'T WANT TO BE CONSIDERED CURED BECAUSE THEIR INSTRUMENT CEILING IS AT THETH PERCENTILE. BASICALLY WE TELL PEOPLE YOUR TREATMENT IS OVER, YOU CAN WALK AROUND THE BLOCK, YOU'RE DONE. I HAVE FROZEN SHOWL D SHOULDER, GOING TO PHYSICAL THERAPY, I DON'T WANT TO BE CONSIDERED CURED WHEN I'M AT THE 50TH PERCENTILE. I'M HOPING MY WHOLE LIFE -- ATHLETES AND OTHER PHYSICALLY ACTIVE PEOPLE WANT TO ACTIVELY DIFFERENTIATE BETWEEN THEIR LEVELS OF FUNCTIONING. EVEN CANCER PATIENTS, FATIGUE -- THEY FEEL FAR AWAY FROM THEIR WANTING TO FEEL NORMAL. LET'S DO ONE MORE DEMONSTRATION OF CAT. THIS IS NOT REALLY APPLICABLE, BUT IT'S EASIER TO SHOW IT. SO MOST CAT TESTING, UNTIL LAST COUPLE OF YEARS, WAS USED FOR SOME KIND OF CERTIFICATION. MEDICAL BOARD CERTIFICATION, IF YOU'RE A SURGEON, YOU'RE A NURSE, WE DIDN'T REALLY CARE ABOUT YOUR LEVEL ALONG THE TRAIT, WE WANT TO KNOW DID YOU PASS OR FAIL THE TEST? SO HOW DO WE DO THAT IN A CAT? TYPICALLY RATHER THAN WORRYING ABOUT THE WHOLE TRAIT RANGE, WE GIVE YOU YOUR FIRST QUESTION AT THE PASS POINT OF THE TEST. IF YOU GET THAT QUESTION CORRECT, WE GIVE YOU A HARDER QUESTION. IF YOU GET THAT ONE WRONG, WE GIVE YOU AN EASIER QUESTION. GET THAT ONE RIGHT, WE GIVE YOU A MORE DIFFICULT QUESTION, YOU PASS IT, WE PASS THE TEST. TURNS OUT IN ALMOST ANY SURGERY BOARD I'VE HELPED DEAL WITH OVER TIME, WE KNOW 90% OF THE PEOPLE, WHETHER THEY'RE GOING TO PASS OR FAIL IN ABOUT 10 OR 12 ITEMS. MOST TESTS DID ON FOR 60 OR 90 ITEMS. FOR TWO REASONS. ONE IS PEOPLE REALLY HATE HAVING GONE TO COLLEGE, GRADUATE SCHOOL, DONE THEIR RESIDENCY, AND BE TOLD EVEN IF THEY PASSED IN A DOZEN QUESTIONS. IT DOESN'T FEEL GOOD, THEY REALLY WANT A LITTLE MORE OF AN EXPERIENCE. PEOPLE WHO FAIL CANNOT UNDERSTAND HOW THE LIFE OF THEM HOW YOU CAN TELL THEM THEY FAIL IN A DOZEN QUESTIONS. AND WE DO HAVE TO GIVE SLIGHTLY LONGER TESTS FOR PEOPLE IN THE MIDDLE. SO THIS IS A MUCH SADDER EXPERIENCE. THE PERSON FAILS AT THE CENTER IT OF THE DISTRIBUTION, WE GIVE THEM AN EASIER ITEM. THEY FAIL THAT ONE, THIS IS NOT GOING WELL. GIVE THEM A LITTLE BIT EASIER ITEM, THANK GOODNESS THEY PASS. THESE TEND TO BE MULTIPLE CHOICE SO THEY SHOULD GET SOME RIGHT. A LITTLE HARDER, THEY FAIL, THEY FAIL THE TEST. NOW, IF THIS PERSON IS UNABLE TO ANSWER QUESTIONS AT THE PASS POINT OR A LITTLE BIT BELOW, THERE'S JUST NO REASON IN THE WORLD TO GIVE THEM QUESTIONS THAT ARE MUCH HIGHER HAN THEIR ABILITY. WE UTILIZE THIS IN HIGH STAKES TESTING TO REDUCE EXPOSURE ON THE TEST. YOU GIVE THE NURSING EXAM EVERY DAY OF THE YEAR, THEY RELEASE 6,000 ITEMS OUT AT A TIME, SO NO TWO PEOPLE SEE THE SAME ITEMS. THEY GO SO FAR TO SAY -- WE ONLY GIVE THEM ITEMS RELATED TO THEIR LEVEL. WE DON'T GIVE THEM ITEMS BELOW IT, WE DON'T GIVE THEM ITEMS ABOVE IT. WE GIVE THEM ENOUGH ITEMS, IT TURNS OUT THERE ARE SOME PEOPLE WHO GET THE FIRST COUPLE WRONG, THEY'RE NERVOUS, TURNS OUT PEOPLE GET THEM WRONG THAT SHOULDN'T ARE REALLY HIGH ABLE PEOPLE THAT THINK THEY'RE SO SMART, THEY DON'T READ THE QUESTION CAREFULLY ENOUGH, BUT WE GIVE THEM A BREAK ALSO. I DO NOT WANT TO FACE A JUDGE IN COURT WITH AN ATTORNEY SAYING THEY FAILED MY CLIENT IN 12 ITEMS. JUDGE, THEY'RE SUCCESSFUL SURGEON, THEY'VE DONE THIS, HEAF DONE THAT. SO FINE, WE DO GIVE THEM 60 QUESTIONS, BUT IF THEY CAN'T PASS 60 QUESTIONS IN THIS RANGE, WE'RE DONE. WE'RE NOT GOING TO GIVE THEM TWO OR 300. IT'S NOT GOING TO MATTER. IT'S FOR YOU, FOR THE FOR PROFIT COMPANY, FOR THE DRUG COMPANY YOU'RE WORKING WITH, YOUR COLLABORATOR? FRANCE, THEY CAN UTILIZE THESE TOOLS AS WELL. WE'VE WORKED WITH GROUPS AT NO CHARGE TO HELP THEM INSTALL ALGORITHMS WITHIN THEIR OWN SERVERS. IF THEY WANT TO USE A TECHNOLOGY PACKAGE THAT WE'RE ACTIVELY SUPPORTING, WE'RE RUNNING ON A COST RECOVERY BASIS, MULTIPLE LANGUAGE IS JUST NOT FOR OUTSIDE THE UNITED STATES. YOU HAVE PLENTY OF PATIENTS HERE WHO SHOULD BE TAKING AN INSTRUMENT IN SPANISH, NATIONAL CHILDREN'S STUDY IS FINDING THEIR GROUPS THAT ONLY SPEAK THAI, GROUPS THAT ONLY SPEAK VIETNAMESE, PARTICULARLY PARENTS. NICE THING IS ONCE A CHILD HAS BEEN ABOUT ONE YEAR OF SCHOOLING IN THE UNITED STATES, THEY DO JUST FINE IN ENGLISH, BUT UNTIL THAT TIME, WE NEED TO TEST THEM IN THEIR NATIVE LANGUAGE. I THINK WE'RE GOING TO RUN THE RISK OF RUNNING EARLY AND HAVE A LITTLE BIT MORE TIME FOR QUESTIONS. >> THANKS, RICHARD, THAT WAS TERRIFIC. RICK BURRZON. I WAS ONE OF THE FOUNDERS OF THAT GROUP ALONG WITH YOUR COLLEAGUE, DAVE SELLA, DONALD PATRICK AND A BUNCH OF OTHER PEOPLE. QUESTION I HAVE IS WHEN WE USED TO PRESENT THIS STUFF TO THE FDA YEARS AGO, BEFORE THEY CHANGED THE NAME AND BECAME PATIENT-REPORTED OUTCOMES, THEY WERE ALWAYS VERY SKEPTICAL OF ACCEPTING THESE KINDS OF OUTCOMES TO EVALUATE ANY KIND OF INTERVENTION BUT MEDICINES IN PARTICULAR. SO I'M WONDERING WHERE THAT DISCUSSION IS. DOES THE FDA -- I KNOW MOST OF THIS IS NIH WORK AND THERE WAS ALWAYS A DIFFERENCE OF OPINION BETWEEN THE AGENCIES AS THERE OFTEN IS HERE IN THE GOVERNMENT. SO I'M CURIOUS TO KNOW WHERE THAT DISCUSSION IS. HAS THE FDA SORT OF ACCEPTED THESE OUTCOME RESULTS AND WILL USE THEM TO -- IN CLINICAL TRIAL WORK SO THAT THE PHARMACEUTICAL COMPANIES WITH WHOM I WORKED A NUMBER OF YEARS AGO CAN ACTUALLY PUT THAT STUFF IN THE LABEL? >> I'M GOING TO ACTUALLY -- NOT MY NORMAL STYLE BUT I'M GOING TO HAND THAT QUESTION UP TO DR. JIM WITTER, WHO HAS BEEN MUCH MORE INVOLVED IN THAT ON A DAY-TO-DAY BASIS. >> THE ANSWER TO YOUR QUESTION IS, YES, WE HAVE BEEN WORKING WITH THE FDA PRIMARILY THROUGH SOMETHING CALLED THE INTERAGENCY WORKING GROUP WHICH IS BETWEEN A WORKING GROUP BETWEEN FDA ACHED NIH. AND NIH. FOR THE LAST NUMBER OF YEARS, WE HAVE BEEN WORKING WITH THEM ABOUT PROMIS AS WELL AS OTHER INSTRUMENTS THAT ARE HERE AT NIH. THE CURRENT STATUS IS THAT IF THE ADULT FATIGUE BANK, WE ARE WORKING WITH THEM IN TWO DISEASES, RHEUMATOID ARTHRITIS AND CHRONIC FATIGUE SYNDROME TO GO THROUGH THEIR QUALIFICATION PROCESS THROUGH ALL THAT RIGOR. THEY HAVE RECENTLY VERY MUCH EMBRACED WHAT WE'RE DOING. THEY HAVE FINE WILLLY SORT OF COME AROUND TO UNDERSTANDING WHAT PROMIS HAS TO OFFER IN PARTICULAR, AND IT'S MOVING FORWARD AT A GOOD PACE AT THIS POINT IN TIME. THE INDUSTRY ALSO IS PICKING UP ON THIS IN THE SENSE THAT WE HAVE AN INDUSTRY WORKING GROUP NOW WITH PROMIS, AND THEY ARE ALSO SORT OF HOPING AND PUSHING THE FDA TO MOVE THIS FORWARD, FOFOR EXACTLY THE REASON THAT YOU MENTIONED. PCORI IS ALSO IN THE SAME TRACT, THEY UNDERSTAND THE NEED FOR THE FDA TO ENGAGE AND ACCEPT THESE MEASURES AND THEY'RE HELPING US WITH THIS EFFORT. SO THE SHORT ANSWER IS YES AND THE LONG ANSWER IS IT'S COMING, WE HOPE SOON THAT THIS WILL BE EMBRACED IN A WIDER VARIETY OF SOME OF THE OTHER INSTRUMENTS AS WELL BUT RIGHT NOW WE'RE FOCUSING ON FATIGUE. [INAUDIBLE] >> OH OKAY, SO IN THE SENSE OF? [INAUDIBLE] >> RIGHT. OKAY. SO THE RHEUMATOID ARTHRITIS INITIATIVE THAT WE HAVE IS ACTUALLY WORKING THROUGH A COUPLE OF THINGS. IT'S WORKING THROUGH THE OMRACT, WHICH IS A THINK TANK THAT'S IN RHEUMATOLOGY TO COME TOGETHER IN HAD EFFORT TO REALLY QUALIFY THOSE THINGS, SO THIS IS AN INTERNATIONAL EFFORT AS WELL NOW THROUGH THAT ORGANIZATION AS WELL AS THROUGH THE CRITICAL PATH INITIATIVE. THERE'S SOMETHING THERE CALLED THE PRO COULD BE SORE SHUM CONSORTIUM. THAT ALSO IS ANOTHER COROLLARY EFFORT THAT WE'VE ACTUALLY BEEN ENGAGED WITH AND WE'RE WORKING WITH THEM IT IN THIS CASE IN RHEUMATOID ARTHRITIS. >> SO IS THIS HAPPENING BY DISEASE OR -- >> THAT'S A GOOD QUESTION, AND WE'VE BEEN ON THAT FOR A WHILE. THE WAY THAT WE'RE DOING THIS IS TO ADDRESS HA SOR THAT SORT OF PIECEMEAL AND TO DEMONSTRATE TO THE FDA THAT THESE INSTRUMENTS DO WHAT WE SAY HE DO I THEY DO IN DEEDS, AND THE IDEA THEN IS THAT ONCE WE HAVE ESTABLISHED THAT, THEY WILL THEN BE VERY ABLE TO RAPIDLY EXPAND THIS INTO OTHER DISEASES AND TO LOOK FOR AND USE ACROSS DISEASES AS RICHARD HAS BEEN SAYING. SO THE FIRST STEPS FIRST, WE'RE TRYING TO CONVINCE THEM THAT THIS WORKS IN THE DEEDS OF INTEREST TO THEM, AND THEN IT WILL EXPAND FROM THAT POINT OUT. >> LOOKS GOOD AND I APPRECIATE THAT. I JUST WANT TO SAY THAT WE'VE BEEN HAVING THOSE DISCUSSIONS WITH THE FDA SINCE I WAS IN INDUSTRY, WHICH WAS WEN FIVE YEARS AGO OR SOMETHING OR LONGER. THE POPULATION IS AGING AND WE REALLY NEED THIS STUFF NOW. THERE ARE LOTS OF -- I MEAN, NEUROLOGY IS AN OBVIOUS EXAMPLE WHERE PEOPLE ARE AGING, THEY'RE DEVELOPING LOTS OF CONDITIONS FOR WHICH THERE AREN'T ANY LABORATORY OR SURROGATE MEASURES. THESE KINDS OF TOOLS ARE REALLY ESSENTIAL, I HAVE TO TELL YOU -- >> AND JUST -- >> AND I'M REALLY PLEASED THAT THE FIRST EFFORT IS ACTUALLY GOING TO BE ALSO LOOKING AT CAT. WE WERE CONCERNED THAT THEY WOULDN'T REALLY BE INTERESTED IN THAT BECAUSE THAT IS VERY PROBLEMATIC FOR THEM FOR A VARIETY OF REASONS BECAUSE OF THEIR REGULATORY STANCE AND THE ISSUES THEY'VE HAD TO DEAL WITH, BUT THESE IT FIRST EFFORTS ARE GOING TO HAVE TO BE WITH CAT TOO, THAT ARE GOING TO BE VERY HIGHLY ENCOURAGING. >> DO THEY NOT HAVE -- >> THEY DO, AND THEY'RE LOOKING FOR A NEW DIRECTOR OF FIELD SO WE'RE ALL SORT OF WONDERING WHO THAT'S GOING TO BE, BUT YES, THEY HAVE NOW SOME GREAT PSYCHOMETRICIANS ON BOARD, WE'VE BEEN WORKING WITH THEM FOR QUITE A WHILE SO WE'RE GETTING THERE. >> THANK YOU. >> THANK YOU. DO YOU HAVE ANOTHER QUESTION? >> SO IN A RY A RELATED VEIN, FROM AN INDUSTRY PECK SPERTIVE IT MAKES SENSE TO GO BY DISEASE BECAUSE THEY HAVE VERY SPECIFIC INDICATIONS, BUT AS YOU POINTED OUT, TWO THIRDS OF MEDICARE PATIENTS, FOR EXAMPLE, HAVE MORE THAN ONE CONDITION, SO MY QUESTION IS, IN THE INSTITUTIONS LIKE THE CLEVELAND CLINIC WHERE IT HAS BEEN ADOPTED FOR ROUX TEEROUTINECLINICAL PRACTICE, ARE THEY ALSO LOOKING DEPARTMENT BY DEPARTMENT OR IS THERE A MORE HOLISTIC APPROACH TO IT AND HOW IS THAT WORKING OUT? >> FROM WHAT I KNOW OF THE CLEVELAND CLINIC EFFORT, THEY ORIGINALLY STARTED IN THEIR NEUROCLINIC AND I BELIEVE IT'S BEING SPREAD OUT TO ALL THEIR CLINICS. THEY FIRST START OUT IN ONE CLINIC, THEN THEY PUT IT IN ALL THEIR NEURAL CLINIC. SO IF YOU'RE SEEING ANYBODY IN ANY NEURAL CLINIC, WHILE YOU'RE SITTING IN THE WAITING ROOM, YOU TAKE ONE OF THESE. YOU'VE GOT TO EMAIL THE DAY BEFORE ASKING TO DO IT AT HOME. I CANNOT TELL YOU WHAT THE SPREAD IS OF MULTI-DISEASE. >> I GUESS THE REAL TEST WILL BE ADOPTION BY THE PRIMARY CARE PROVIDERS, AND DO YOU HAVE ANY SENSE OF THAT HAPPENING? >> I THINK THE WAY, GIVEN THE OVERLOAD ON PRIMARY CARE PROVIDERS TODAY, TO DO THIS, THAT AND THE OTHER THING, THAT IT'S GOING TO TAKE A COMBINATION OF A PCORI EFFORT COMBINED WITH INTEGRATION OF THESE TOOLS REALLY IN THE EMR THAT EACH PRIMARY CARE PROVIDER IS USING. NOW, SO IN ORTHOPEDICS, FOR INSTANCE, FIELD TESTING ACTUALLY STARTED LAST WEEK, A COUPLE WEEKS AGO, AND THERE'S A CLAMORING FOR PEOPLE THAT ARE HEARING IN IS IN FIELD TESTING, PLEASE LET ME USE THIS IN IN MY CLINICAL PRACTICE. BUT I THINK WE'RE JUST AT THE DAWN OF PEOPLE REALLY USING THIS IN CLINICAL WORK. I MEAN, THERE ARE DOZENS OF EXEMPLARS, UNIVERSITY OF OF WASHINGTON HAS IT, MANY OF THEIR CANCER CLINICS FOR PRIMARY PROVIDERS -- WELL, WHAT'S GOING ON THERE, BUT WE'RE AT THE DAWN. THIS IS IT. THESE TOOLS WERE CREATED A DECADE AGO, AND THE RESEARCH IS NEVER GOING TO BE FINISHED BUT IS THERE TO PROVE THAT WORKS AND IT'S JUST NOW PRIMARY CARE PROVIDERS PICKING UP. THE UNFORTUNATE FINDING, LOOKING AT HISTORY OF ASSESSMENT ADOPTION, PEOPLE USE ASSESSMENTS THAT THEY LEARN IN SCHOOL OR THEY LEARN IN THEIR RESIDENCY. AND WE ARE JUST GETTING TO THE POINT TODAY WHERE THESE -- DSM5 INCLUDES MENTION OF PROMIS INSTRUMENTS, THAT'S THE BEGINNING. PEOPLE DON'T SAY OH, LOOK, A NEW THING OUT THERE. LET'S FACE IT, NOBODY HAS TIME CLINICALLY TO SEE WHAT'S THE GREATEST AND THE BEST, IS THIS A FAD, ET CETERA. SO I THINK IT'S GOING TO BE ESCALATED BY MEANINGFUL USE, BY PCORI AND BY BEING READILY AVAILABLE INSIDE OF EMRs, BUT I THINK -- I DON'T WANT TO FULLY PREDICT IN THE FUTURE HOW LONG THAT'S GOING TO BE. >> I HAD A QUESTION. >> YES. >> I WORK OVER AT UCEF FOR -- CONSORTIUM FOR -- I WAS WONDERING IF YOU CAN TELL US A LITTLE BIT ABOUT PASTURE AND HOW THE PROMIS TOOLS ARE BEING ALTERED FOR USE BY THE MILITARY OR IF SOMEONE CAN ADDRESS THAT LATER? >> STAY TUNED UNTIL AFTER THE BREAK. WE WON'T HOLD YOU TOO LONG. >> GREAT. THANK YOU. >> AND ON THAT NOTE, MAYBE WE SHOULD -- >> WELL, I GAVE A PRESENTATION AT NATIONAL CHILDREN'S STU IT DI LAST WEEK AND I I WAS PREEMPTED BY SOMEBODY SHOWING DANCING 3-YEAR-OLDS. WE HAVE 2 1/2 MINUTES. I'M JUST GOING TO SHOW TWO QUICK EXAMPLES, THESE HAPPEN TO BE TOOLBOX TOOLS BUT WE'RE ALSO BRINGING THE PROMIS TOOLS FOR USE IN THE NATIONAL CHILDREN'S STUDY. I'LL CUT THIS OFF BY 10:30. >> WE'RE GOING TO PLAIB TOGETHER PLAY TOGE THER AND YOU'RE GOING IT TO BE TOUCHING LETTERS ON THE SCREEN. TOUCH THE FLOWER ON THE SCREEN. GOOD JOB. NO WE'RCAN YOU PUT YOUR FINGER ON HOME BASE AND WAIT FOR THE NEXT PICTURE? ALL RIGHT. NOW YOU TRY. KEEP YOUR EYE ON THE STAR IN THE MIDDLE SCREEN, REMEMBER TO PUT YOUR FINGER BACK ON HOME BASE AFTER YOU ANSWER. READY? THERE YOU GO. >> SHAPE. >> WHICH ONE? >> IS IT THE SAME OR IS IT THIS ONE? >> THIS ONE. >> OKAY. TOUCH THE ONE YOU THINK IS THE SAME. >> THIS IS THE SAME SHAPE, SO YOU SHOULD CHOOSE THIS BOX. >> THEY'RE BOTH BOATS. PUT YOUR FINGER BACK. EACH ONE IS THE SAME. TOUCH THE BUTTON THAT SHOWS THE SAME PICTURE AS THAT. GOOD JOB. >> THAT'S RIGHT. >> GREAT! >> SO THE ENTIRE COGNITION BATTERY IS BEING USED WITH THE iPAD, THE PROMIS SELF-REPORT BATTERIES FOR OLDER CHILDREN, PROXY MEASURE ALL BEING CONVERTED FOR USE ON THE iPAD. ASSESSMENT CENTER, YOU CAN HAND A PATIENT AN iPAD AND TAKE A PROMIS CAT, THE INSTRUMENT FULLY FORMATS FOR USE ON THE iPAD MAKING BOTH NATIVE. THIS IS THE FUTURE IN THE WAITING ROOM TO ME OR THIS IS THE FUTURE OF CLINICAL USE OF THESE TOOLS. THESE ARE 3-YEAR-OLDS TAKING A TEST THAT WOULD OTHERWISE REQUIRE A PH.D. WITH A LOT OF EXPERIENCE GETTING A SCORE AND THE PEDIATRIC BA TRE BATTERY IS ABOUT 12 MINUTES LONG. LAST YEAR THE HEAD OF SPECIAL ED FOR CHEVY CHASE SCHOOLS SAID WE NOW GET FUNDING FOR TWO CHILDREN WE TESTED A YEAR. TOOLS LIKE THIS THAT CAN BE SHORT, PROMIS INSTRUMENTS, TOOLBOX INSTRUMENTS, NEURO-QOL INSTRUMENTS ARE COMPLETE GAME CHANGERS IN THE ASSESS M OF KIDS AND CERTAINLY OF ADULTS, AND I THINK WE HAVE A BREAK. >> WE WILL RECONVENE AT 10:40 WITH A PRESENTATION FROM DR. COOK. I HAVE THE GREAT PRIVILEGE OF INTRODUCING DR. KARON COOK. SHE'S A RESEARCH ASSOCIATE PROFESSOR ALSO AT NORTHWESTERN AND SHE'S GOING TO BE GIVING US AN OVERVIEW OF THE TOOLS TO ASSESS ADULT AND PEDIATRIC PAIN. THIS IS GOING TO BE, I THINK, QUITE ENTERTAINING. >> THANKS VERY MUCH TO JIM AND ALL OF YOU. I'M REALLY HAPPY TO BE HERE. I WAS THINKING ABOUT THIS ON THE WAY HERE ON THE PLANE, REALIZING HOW MUCH IT WAS BY ACCIDENT THAT I CAME IN TO BEING ASSOCIATED WITH PAIN ASSESSMENT. FIRST I BEGAN TO TAKE IT PERSONALLY THAT PEOPLE THOUGHT OF ME WHEN THEY THOUGHT OF PAIN, BUT REALLY WHEN I THINK ABOUT IT, I CAN'T THINK OF ANY SYMPTOM THAT MORE DRIVES QUALITY OF HEALTHCARE UTILIZATION AND QUALITY OF LIFE. SO IF YOU'RE GOING TO BE ASSOCIATED WITH A SYMPTOM, I THINK IT'S NOT A BAD ONE TO BE ASSOCIATED WITH. WHEN I GOT THE EMAIL FROM PRI AND FROM JIM WITTER ABOUT TALKING ABOUT PAIN AND PROMIS, THEY KEPT TALKING ABOUT THE PROMIS PAIN 6-PACK. THIS IS A TERMINOLOGY THAT I HAD NOT HEARD, SO I DIDN'T REALLY KNOW WHERE THE METAPHOR WAS COMING FROM SO I DID A LITTLE BIT OF GOOGLING AND I FINALLY FOUND OUT WHY THAT TER TERM HAD BEEN USED. APPARENTLY JIM HAS BEEN WORKING OUT, HE'S BEGUN TO THINK IN TERMS OF SUCH BODILY METAPHORS, SO LET'S TAKE ANOTHER CLOSER LOOK AT THOSE ABS. AS IT TURNS OUT, WHAT THEY WERE TALKING ABOUT WAS THE FACT THAT THERE ARE BOTH ADULT AND CHILD MEASURES FOR PAIN. WITHIN THOSE, THERE ARE THREE DIFFERENT KINDS OF MEASURES FOR PAIN. PAIN BEHAVIOR, PAIN INTERFERENCE, AND PAIN QUALITY. THOSE ARE REALLY GREAT SHORTS, JIM. I WANTED TO COMPLIMENT YOU ON THOSE. AS MUCH AS IT'S A SHAME TO MOVE FROM THIS SLIDE, WE'LL GO AHEAD AND COULD DO THAT. SO LET'S START BY TALKING ABOUT PAIN MEASURES IN PEDIATRICIAN DRIK POPULATIONS. THE PROMIS METHODS HAVE BEEN PUBLISHED AND I WON'T GIVE YOU A IT FULL RUNDOWN OF THEM NOW, BUT I DO WANT TO MENTION ONE PORG OF THE METHODS THAT ARE ASSOCIATED WITH DEVELOPING PROMIS ITEMS, WHAT'S CALLED COGNITIVE INTERVIEWS. I KNOW MANY OF YOU ARE FAMILIAR WITH THAT THOSE ARE LIKE, BUT BASICALLY IT'S GETTING RESPONDENTS, ASKING THEM THE QUESTION, FINDING OUT FROM THEIR PERSPECTIVE WHAT THEY THINK THE QUESTION IS ASKING THEM AND HOW THEY CAME UP WITH A PARTICULAR RESPONSE TO MAKE SURE THAT THOSE WHO CAME UP WITH THAT ITEM ARE COMMUNICATING THE SAME THING OR THAT THE RESPONDENT IS HEARING THE SAME QUESTION. SO THESE INTERVIEWS WERE DONE WITH CHILDREN AND ADOLESCENTS, AND THE GOOD NEWS IS THAT WE FOUND KIDS AS YOUNG AS 8 COULD COMPREHEND MOST ITEMS. SO THE DECISION WAS MADE TO DEVELOP MEASURES FOR AGES 8 TO 17 FOR SELF-REPORT MEASURES AND THEN ALSO PROXY MEASURES FOR YOUNGER CHILDREN. SO LET'S START WITH THE PAIN INTERFERENCE FOR PEDIATRICS. PEDIATRICS PAIN INTERFERENCE COMES IN AN 8-ITEM SELF-REPORT AND A PROXY REPORT SHORT FORMS, AND IT MORE THAN ANY OF THE OTHER PAIN MEASURES HAS BEEN TRIED OUT IN MANY DIFFERENT CLINICAL POPULATIONS AND SO THERE'S THIS -- REALLY A VERY LARGE GROWING BODY OF EVIDENCE ABOUT HOW SUCCESSFUL THE PEDIATRIC PAIN INTERFERENCE MEASURE IS. THIS ACTUALLY DOESN'T INCLUDE ALL OF THEM. SOME OF THE OTHER WORK THAT'S NOT YET PUBLISHED ALSO INCLUDES SICKLE CELL DISEASE AND ALSO IT -- PAIN BEHAVIOR IS ALSO GOING TO BE ONE OF THE AREAS THAT THERE'S A PEDIATRIC MEASURE FOR, AND IN FACT, THERE ARE NEWLY DEVELOPED SELF-REPORT AND PROXY REPORT SHORT FORMS FOR PAIN BEHAVIOR, THEY HAVEN'T BEEN PUBLISHED YET, THEY'VE JUST NOW BEEN DEVELOPED. THEY HAVE EIGHT ITEMS EACH. THEY, LIKE ALL THE OTHER PROMIS ITEMS, ARE CALIBRATED TO AN IRT CALIBRATION. I'VE SEEN THE PRELIMINARY WORK ON THIS. THEY FIT WELL, THEY ARE FUNCTIONING EXTREMELY WELL. THEY WERE CALIBRATED WITH A GROUP OF 450 PEDIATRIC SAMPLES AGE 8 TO 17. SO THOSE WILL BE COMING ON BOARD. PAIN FAULT, I'M GOIN FAULT, RIGHT NOW THIS IS A SAY TUNED FOR PEDIATRIC PAIN QUALITY. I CAN TELL YOU THAT WHEN WE FIRST IN PROMIS DID PAIN MEASURES, BOTH ADULT AND PEDIATRIC, IN ADULT, WE ADMINISTERED A NUMBER OF PAIN QUALITY ITEMS IN WAVE ONE, SO VERY EARLY ON, AND UNLIKE PAIN INTERFERENCE OR PAIN BEHAVIOR, WHERE YOU HAVE THIS SINGLE UNI DIDDIMENSIONAL WAY PEOPLE ARE ANSWERING THE ITEM, THIS DID NOT PROVE TO BE THE CASE IN PAIN QUALITY. FOR THOSE OF YOU WHO ARE PAIN CLINICIANS OR CLINICIANS WHO DEAL WITH PATIENT'S PAIN, THIS PROBABLY WON'T SURPRISE YOU BECAUSE THERE ARE, IN FACT, MANY DIFFERENT QUALITIES OF PANE. PAIN. SO NIH SPONSORED A PROJECT, I'LL TELL YOU MORE ABOUT THAT WHEN I GET TO TALKING ABOUT ADULT PAIN, BUT I'LL JUST MENTION THAT AS PART OF THAT, IN A SAMPLE OF 309 CHILDREN, THESE PAIN QUALITY ITEMS WERE ADMINISTERED. I ALSO THINK IT'S INTERESTING TO NOTE, DR. ESSIE MORGAN DEWITT IS ISLY RESPONSIBLE FOR IT. WHEN WE DECIDED TO DO THE ITEMS FOR PAIN QUALITY, AND PAIN QUALITY IS UPMENT, WE CHOSE AS A BASIS THE PEDIATRIC WORDING BECAUSE IT WAS CLEAR AND IT COMMUNICATED WELL, AND I THINK GENERALLY THAT'S WHAT YOU FIND, IS THAT FOR BOTH -- FOR PEDIATRIC MEASURES AND FOR ADULT MEASURES, THE MORE CLEAR YOU CAN BE, THE MORE SIMPLE THE LANGUAGE, THE BETTER. SO NOW WE HAVE THESE ADULT PAIN MEASURES, AND WE HAVE PEDIATRIC MEASURES. AND WHAT'S GOING ON NOW IS AN ATTEMPT TO DO SOMETHING THAT HASN'T BEEN DONE BEFORE, AND THAT'S LINK THE SCORES FROM THE PEDIATRIC TO THE ADULT PAIN MEASURES, AND THIS IS CALLED VERTICAL LINKING. THIS WORK IS ACTUALLY GOING ON RIGHT NOW, DARREN DEWALT AND OTHERS ARE WORKING ON THIS IN COMPLETING PRELIMINARY WORK RIGHT NOW. WHEN THIS IS COMPLETED, THIS WILL BE A VERY EXCITING THING TO HAVE. YOU WILL NOW HAVE A MEASURE THAT GOES ACROSS THE ENTIRE LIFETIME, A AND IN THE SAME WAY THAT IF YOU HAVE HEMOGLOBIN, IT MEANS THE SAME THING IN PEDIATRIC AS IT DOES IN ADULT. YOU MAY HAVE DIFFERENT NORMATIVE VALUES PORE DIFFEREN FOR DIFFERENT AGE GRO UPS, BUT THE METRIC WILL BE THE SAME ACROSS THE ENTIRE LIFESPAN. BY THE WAY, THIS IS NOT JUST HAPPENING IN PAIN, BUT IN MANY OF THE OTHER PROMIS DOMAINS AS WELL. SO WHEN PROMIS BEGAN, IF YOU LOOK AT THE ORIGINAL STATEMENT OF PURPOSE, IT TALKED ABOUT THE USE OF MEASURES IN RESEARCH, IN CLINICAL RESEARCH. BUT NOT -- THERE WAS NOT A LOT IN THERE ABOUT USE OF THEM IN CLINICAL SETTINGS FOR JUST PATIENTS. ONE OF THE THINGS I WAS ABLE TO BE IN PROMIS AFTER THE FIRST YEAR, SO THE LAST -- STARTING IN 2005, SO ONE YEAR INTO THE DEVELOPMENT, SO I'VE BEEN ABLE TO COME TO WATCH THE FEEL AS IT'S PROGRESSED, AND AS PROMIS HAS GONE FROM A GLIMMER IN SOMEONE'S EYE TO WHAT IT IS NOW WITH THIS WHOLE SET OF VALIDATED MEASURES, ONE OF THE THINGS THAT'S HAPPENING OUTSIDE IS MORE AND MORE EMPHASIS OF THE IDEA OF USING PATIENT-REPORTED OUTCOME MEASURES IN CLINICAL SETTINGS. SO AS THAT HAPPENED, THE PEOPLE, THE SCIENTISTS AND THE LEADERS IN PROMIS WERE AWARE OF THIS MOVE AND BEGAN TO PUSH ON ENABLING THESE MEASURES TO BE USED IN CLINICAL SETTINGS. SO LET ME GIVE YOU A COUPLE EXAMPLES FOR THE CLINICAL. AT CINCINNATI CHILDREN'S HOSPITAL, WHEN A PATIENT ARRIVES, HE OR SHE IS GIVEN A TABLET. AND THEN BASED ON THAT PATIENT'S INFORMATION, THIS IS A CHILD WHEN COMES IN TO THE CLINIC, BASED ON THAT CHILD'S INFORMATION, A COMPUTER ALGORITHM SELECTS WIDOW MAINS TO GIVE THIS PATIENT. AND THEY'RE ON A TABLET. THE PATIENT STARTS IN THE WAITING ROOM FILLING THIS OUT, BUT THEY MIGHT GET CALLED INTO THE PHYSICIAN'S OFFICE LATER, SO THEN THEY CAN JUST TAKE THAT TABLET WITH THEM ON INTO THE EXAM ROOM AND FINISH IT THERE. THEN THOSE PROS, THOSE PATIENT-REPORTED OUTCOMES AND REPORTS OF THOSE, ARE AVAILABLE TO THE PROVIDER TO INFORM THE VISIT. ONE OF THE THINGS DR. DEWITT IS DOING AT CINCINNATI CHILDREN'S HOSPITAL IS THEY WERE EXPERIMENTING WITH USING A PAIN INTERFERENCE THRESHOLD TO KEY TO IPT VENGSES. INTERVENTIONS. SO IF A CHILD REACHES, BASED ON THEIR REPORT, SELF-REPORT OF PAIN INTERFERENCE REACHES A PARTICULAR THRESHOLD, IT WILL TRIGGER AT THE NEXT APPOINTMENT A NURSING INTERVENTION FOR THAT CHILD, IN EXPERIENCING HIS OR HER PAIN. MOWNOW, CURRENTLY THE WAY THE SYSTEM IS SET UP, IT'S TRIGGERED FOR THE NEXT VISIT, BUT IN THE FUTURE, IT WILL ACTUALLY BE TRIGGERED IN REALTIME, SO THIS PERSON EXPRESSES THIS LEVEL OF PAIN INTERFERENCE AND RIGHT THEN, IT TRIGGERS A NURSING INTERVENTION AND SO YOU CAN SEE HOW THIS KIND OF TRIGGERED THRESHOLD APPROACH CAN REALLY NOT ONLY INFORM BUT ACTUALLY INITIATE PATIENT CENTERED IPT VENGSES. IPT INTERVENTIONS. USED TO WHEN WHEN HE THOUGHT ABOUT PATIENT-CENTERED OUTCOMES, WE THOUGHT VERY RIGHTLY OF COURSE ABOUT HOW IMPORTANT IT IS TO GET THE PATIENT'S VOICE IN RESEARCH. IT SEEMS LIKE NOW FINALLY WITH THESE CLINICAL APPLICATIONS, WE'RE USING PATIENT-REPORTED OUTCOMES IN PATIENT-RELEVANT WAYS. NOT JUST TO RESEARCH THEM, BUT ACTUALLY TO INFORM HOW WE CAN HELP THEM IN MEDICAL INTERVENTIONS. SO LET'S MOVE ON AND LOOK AT THE ADULT MEASURES FOR A SECOND. STARTING WITH THE LEFT SIDE OF JIM'S SIX PACK, WE'LL START WITH PAIN INTERFERENCE. IN PAIN INTERFERENCE, THERE'S A 4 ITEM, 6 ITEM AND 8 EYE TELL TEM SHORT FORM AND ALSO A CAT ITEM BANK. SO BECAUSE THE PROMIS MEASURES ARE CALIBRATED USING AN ITEM RESPONSE THEORY APPROACH, SHORT FORM SCORES, CAT ITEM BANK SCORES ARE ALL IN THE SAME METRIC SO THEY'RE DIRECTLY COMPARABLE. PAIN INTERFERENCE IS BEING USED IN THE LITERATURE, I'M NOT EVEN GOING TO TRY TO LIST ALL THE POPULATIONS. THE SECOND MEASURE WITHIN THE ADULT PAIN BANKS IS THE PAIN BEHAVIOR MEASURE, WHICH IS A 7 ITEM SHORT FORM AND ALSO HAS A CAT ITEM BANK, AND THEN THERE'S PAIN QUALITY. HERE'S WHERE I'M GOING TO OH TELL YOU A LITTLE MORE ABOUT THE SUPPLEMENT. I MENTIONED BEFORE THAT THE PEDIATRIC GROUP HAD 309 INDIVIDUALS, AND FOR THE SUPPLEMENT, WE INTENTIONALLY SELECTED DISEASE POPULATIONS OR DIAGNOSTIC CATEGORIES RATHER THAT EITHER TYPICALLY WERE ASSOCIATED WITH NEUROPATHIC PAIN OR WERE NOT ASSOCIATED WITH NEUROPATHIC PAIN. PEDIATRIC SAMPLES INCLUDED J.I.A., JUVENILE I IDIOPATHIC ARTHRITIS, JUVENILE ONSET FIBROMYALGIA, AND I THINK SICKLE SELL ANEMIA. CELL ANEMIA. FOR ADULTS, WE COLLECTED 963 RESPONSES. THE NEUROPATHIC DIAGNOSES ASSOCIATED WERE DIABETIC NEUROPATHY, CHEMOTHERAPY-INDUCED NEUROPATHY, AND THEN THE NON-NEUROPATHIC WAS RHEUMATOID ARTHRITIS AND OSTEOARTHRITIS. THIS GAVE US A NICE ABILITY TO DO SOMETHING THAT YOU WOULD WANT A PAIN QUALITY MEASURE TO DO. TO SEE IF RESPONSES AND SCORES AND THRESHOLDS ARE IN PARTICULAR ITEMS COULD HELP YOU DISTINGUISH BETWEEN PEOPLE WHO HAD NON-NEUROPATHIC PAIN AND THOSE WHO HAD NEUROPATHIC PAIN. SO THE FINDINGS ARE PRETTY NEW OFF THE PRESS. WHAT WE FOUND IS THAT FIVE ITEMS, NUMBNESS, STINGING, TINGLING, PINS AND NEEDLES AND ELECTRICAL, DID THE BEST JOB OF EFFICIENTLY AND ECONOMICALLY DISTINGUISHING AMONG THESE DIFFERENT DISEASE CONDITIONS. A CUT POINT WAS IDENTIFIED THAT HAS A SENSITIVITY OF .76, SPECIFICITY OF .7 FLEE AND PERCENT CORRECT CLASSIFICATION OF 74%. SO PRETTY GOOD JOB WITH A FIVE-ITEM SCALE. WE'RE ALSO EXPERIMENTING IN AN EXPLORATORY WAY OF LOOKING AT A SCALE INCLUDING THINGS LIKE ACHINESS, TENDERNESS, SORENESS AND THERE'S ONE LEFT OFF, DEEP PAIN AND ALSO STEADY PAIN. SO THIS IS THE PROMIS 6-PACK, ADULT AND CHILD MAY HAVE YOUR CHILD BEHAVI OR QUALITY AND INTERFERENCE. YOU MAY BE SITTING THERE THINKING THAT'S GREAT, I USE ADULT PAIN INTERFERENCE MEASURES BUT I HAVE MY OWN SCALE. MAYBE YOU USE THE BRIEF PAIN INVENTORY SEVEN ITEM PAIN INTERFERENCE SUBSCALE. RICHARD HAS ALREADY STOLE MY THEN DER ON THISTHUNDER, BUT IF YOU ALREAD Y USE SOMETHING LIKE THE BPI, BODILY PAIN, YOU'D LIKE TO CHANGE BUT YOU DON'T WANT TO LOSE ALL THOSE DATA THAT YOU COLLECTED, THE CONNECTION WITH ALL THOSE DATA THAT YOU'VE COLLECTED IN THE PAST,, THEN WE HAVE AN APP FOR THAT. IT'S PROSETTA STONE. THIS PROJECT WAS USED TO LINK MANY DIFFERENT COMMON MEASURES THAT ARE USED, WE CALL THEM LEGACY MEASURES, TO PROMIS EQUIVALENT OF THE SAME DOMAIN. YOU SEE THERE TAB FOR LINKING TABLES, AND IF YOU GO TO THE LIPGING TABLES AND YOU SCROLL DOWN, YOU'LL SEE A PLACE FOR PROMIS PAIN, AND YOU'LL SEE CROSSWALKS AVAILABLE LINKING TABLES THAT ASSOCIATE A BPI PAIN INTERFERENCE SCALE WITH A PROMIS METRIC AND THE SHORT FORM BODILY PAIN SCALE TO THE PROMIS METRIC. SO LETS IT TAKE A LOOK. THIS IS VERY SIMILAR TO WHAT RICHARD SHOWED YOU FOR DEPRESSION, HERE YOU HAD THE BPI PAIN INTERFERENCE SCORE, LET'S JUST KIND OF SCROLL DOWN AND LOOK A LITTLE BIT. AND YOU CAN SEE RIGHT THERE AT A SCORE OF 31, ON THE BPI, THAT'S ROUGHLY EQUIVALENT TO A 59.9 ON THE PROMIS METRICS, SO A 60, A STANDARD DEVIATION. NOW NOTICE WHAT YOU GET OUT OF THIS. YOU NOT ONLY GET THIS LINK TO THE BPI, BUT WITH THE BPI ITSELF, AND JUST THAT SCORE, YOU REALLY DON'T KNOW HOW IT ASSOCIATES WITH THE GENERAL POPULATION. P BUT THROUGH THE PROMIS LINK, YOU ACTUALLY DO, SO YOU GET THAT ADDED ADVANTAGE OF HAVING AN INTERPRETIVE CONTEXT FOR UNDERSTANDING A PARTICULAR SCORE. THE OTHER THING YOU GET IS THIS OTHER COLUMN HERE WHICH IS A STANDARD ERROR. AS RICHARD SHOWED YOU ON THE SLIDES, WHEN WE THINK ABOUT CLASSICAL APPROACH, WE MEASURE LIABILITY LIKE THE INVERSE OF STANDARD ERROR. WHEN WE DO THAT, WE ASSUME THAT THE MEASURE IS RELIABLE AT THE SAME LEVEL ACROSS THE ENTIRE CONTINUUM. YET, IN FACT, AS RICHARD'S SLIDES SHOW, THAT IS NOT THE CASE. SO YOU GET SEVERAL THINGS OUT OF THIS CROSSWALK. I WANT TO TALK ABOUT ONE PARTICULAR KIND OF USE THAT'S HAPPENING THAT I'M PARTICULARLY EXCITED ABOUT. AND THAT'S THE USE OF ADULT PAIN MEASURES FOR REGISTRIES AND CLINICAL USE BOTH AT THE SAME TIME. SO I'M GOING TO TALK ABOUT WITH IT DIFFERENT INSTANCES OF THIS. ONE IS HAPPENING AT STANFORD UNIVERSITY AND IT'S CALLED HERO, THE OTHER IS HAPPENING WITH THE DEPARTMENT OF DEFENSE, AND THAT'S PASTOR, VERY NICELY ASKED ABOUT PRIOR TO MY COMING UP HERE. SO LET'S LOOK AT HERO. HERO AT STANFORD IS THE HEALTH ELECTRONIC REGISTRY OF OUTCOMES. A VERY CONVENIENT ACRONYM. IT IS BOTH A RESEARCH TOOL AND A CLINICAL TOOL. THE WAY THEY HAVE IT SET UP IS THAT THE STANFORD PAIN CLINIC, WHEN A NEW PATIENT COMES IN, ANY NEW PATIENT, THEY TAKE A COLLECTION OF MEASURES, AND THEY ARE NOT JUST THE PROMIS PAIN MEASURES BUT ALSO PROMIS MEASURES OF PHYSICAL FUNCTION AND FATIGUE AND SOCIAL FUNCTION AND CORRELATES OF PAIN. THIS SLIDE IS BORROWED FROM ONE OF MY COLLEAGUES AT STANFORD WHO WAS SHOWING AN EXAMPLE OF THEIR REPORT MECHANISM. SO HERE'S A PARTICULAR INDIVIDUAL WHO'S COME IN AT A FIRST TIME AT BASELINE. YOU CAN SEE REMEMBER THE 50 IS -- THIS IS DONE IN PERCENTILE SCORE SO 50 IS THE 50TH PERCENTILE, AND YOU CAN SEE HOW THIS PERSON IS ABOVE AVERAGE IN DEPRESSION, ANXIETY, AND PHYSICAL FUNCTION IS REVERSED, SO I WON'T GO INTO THAT, BUT PAIN INTERFERENCE IS VERY HIGH, PAIN BEHAVIOR IS VERY HIGH. SO MEDICATION INTERVENTION HAPPENED AT THE FIRST TIME POINT, AND THEN LET'S LOOK DOWN AT THE THIRD IT TIME POINT AND YOU CAN SEE AT THAT TIME POINT, A HEALTH EDUCATION WAS USED AS AN INTERVENTION, AND YOU SEE THIS ADDITIONAL REDUCTION OF POOR OUTCOMES. HERE YOU'VE GOT A REPORT ON AN INDIVIDUAL PERSON AND IT'S LINKED, IT'S LAID OVER, ACTUAL INTERVENTIONS, YOU GET A SNAPSHOT OF WHAT IS HAPPENING IN THIS PERSON'S LIFE OVER TIME, AND HOW IT'S ASSOCIATED WITH DIFFERENT INTERVENTIONS. PASTOR IS PART OF -- CAME OUT OF THE ARMY MANAGEMENT TASK FORCE. THIS INITIATIVE WAS INITIATED BY GENERAL SCHUMACHER AND THE CHARGE TO PASTOR WAS THAT THEY DEVELOP STANDARDS AND APPROACHES AND RECOMMENDATIONS THAT WOULD OPTIMIZE CARE FOR WARRIORS IN THEIR FAMILIES AND WOULD STANDARDIZE THE DEPARTMENT OF DEFENSE AND THE VA'S APPROACH TO PAIN MANAGEMENT. PAIN MANAGEMENT IS A HUGE ISSUE IN THE DEPARTMENT OF IT DEFENSE AND OF GREAT CONCERN OH THEM AND SO THE ARMY MANAGEMENT TASK FORCE DEVELOPED A REPORT. ONE OF THE THINGS THEY SAID IS THAT WE NEED A SCREENING TOOL AND WE NEED A REGISTRY BECAUSE WE NEED TO KNOW WHERE PATIENTS ARE WHEN THEY COME IN AND WE NEED TO ALSO BE ABLE TO COLLECT A LOT OF DATA SO WE CAN ASK IMPORTANT RESEARCH QUESTIONS THAT WILL HAVE AN IMPACT ON HOW WE TREAT PATIENTS IN THE FUTURE. SO THAT'S WHAT PASTOR IS. THE PAIN ASSESSMENT SCREENING TOOL AND OUTCOMES REGISTRY. SO JANUARY OF THIS YEAR, THIS IS WALTER REID HOSPITAL, JANUARY 1 OF THIS YEAR, STARTED PASTOR'S BEGINNING PILOT. SO NOW WHEN A NEW PATIENT COMES INTO THE WARRIOR TRANSITION UNIT, HE OR SHE BEFORE COMING FOO THEIR CLINIC VISIT RECEIVES AN EMAIL THAT ASKS THEM TO GO ONLINE AND TO COMPLETE A SERIES OF ASSESSMENTS. THOSE ASSESSMENTS INCLUDE PAIN. AGAIN, VERY SIMILAR TO ONES THAT ARE BEING USED AT STANFORD, NOT ACCIDENTALLY BY THE WAY, AND PAIN CORRELATES WHICH IS SOCIAL FUNCTION, FATIGUE, SLEEP, MENTAL HEALTH ISSUES THINGS LIKE SUBSTANCE ABUSE, ALCOHOL USE, TRAUMATIC STRESS SYNDROME, OTHER ISSUES LIKE THAT. SO RIGHT NOW WE HAVE GOT IT IN PLACE, IT'S ACTUALLY WORKING IN A REAL LIVE CLINIC AT WALL IT TEWALTERREED. THE EXCITING THING IS NOW WHEN OUR PARTNERS IN THE MILITARY WHO WERE PART OF THE ARMY PAIN MANAGEMENT TASK FORCE GO TO THEIR LEADERS, THEY CAN SAY, LOOK, THIS IS SOMETHING THAT'S ACTUALLY OPERATING, WE ARE USING THIS TO HELP MANAGE OUR PATIENTS' PAIN IN THE CLINIC SETTING. EXCITING THING ABOUT IT IS I THINK STARTING NEXT MONTH, THIS VERY SIMILAR KIND OF THING, SAME MEASURES WILL BE ROLLED OUT AT BALBOA, NAVY MEDICAL CENTER, IT ALSO WILL BE ROLLED OUT AT THE MATT BEGAN ARMY PAIN CENTER CLINIC, AND IN A VERY EXCITING DEVELOPMENT -- MATTIGAN ARMY MEDICAL CENTER, THEY HAVE THE CENTER FOR THEIR I.T. SOLUTIONS AND SOFTWARE DEVELOPMENT. SO WITH A CONTRACT FROM THE U.S. DEPARTMENT OF DEFENSE IS INTEGRATING USING AN API THE PROMIS MEASURES INTO THEIR ARMY MEDICAL RECORD SYSTEM THAT ALREADY EXISTS, SO IT WILL FLAWLESSLY INTEGRATE WITH WHAT YOU SEE HERE, WHICH IS WHAT THEIR PATIENT DASHBOARD LOOKS LIKE. WITH PLANS LATER ON TO USE IT, AGAIN TO USE THRESHOLDS AND TRIGGER PARTICULAR SPECIFIC CLINICAL POTENTIAL INTERVENTIONS, AND ALSO TO USE IT FOR BIG DATA COLLECTION. FROM THE BEGINNING, THIS WAS TO BE A CLINICAL TOOL, NOT JUST SOMETHING FOR RESEARCHERS. VERY EARLY, THEY IDENTIFIED THAT IN ORDER FOR THAT TO HAPPEN AND BE OF USE IN THE CLINICAL SETTING, THEN YOU NEEDED TO NOT JUST COLLECT DATA, YOU NEEDED TO TO HAVE INFORMATION, AND THE STEP BETWEEN A LOT OF DATA AND REAL USABLE ACTIONABLE INFORMATION IS REPORTING. SO WE SPENT A LOT OF TIME DEVELOPING A GRAPHICAL PAIN REPORT THAT CLINICIANS RECEIVE AND WE HAVE IT NOW DOWN TO TWO, TWO AND A THIRD PAGES, AND I'LL SHOW YOU MOST OF THE PARTS OF IT IT. VERY STRATEGICALLY THE FRONT PAGE OF IT, WHICH I'M GOING TO SHOW YOU FIRST, ARE THE THINGS THAT WILL BE IN THE CLINICIANS' FACE AT THE BEGINNING OF THE VISIT. SO WE HAVE A PAIN MAP. YOU CAN SEE NOT ONLY DOES A PATIENT-REPORTED HIS OR HER PAIN LOCATIONS, WHERE ALL DO I HURT, BUT ALSO WHERE DO I HURT THE MOST TODAY. IN ADDITION, THERE'S A TABLE DOWN BELOW THAT SHOWS NUMBER OF PAIN LOCATIONS OVER TIME, NUMBER OF PAIN LOCATIONS IS A VERY IMPORTANT PREDICTOR OF HOW PATIENTS DO. ALSO MORPHINE EQUIVALENT DOSE OVER TIME, ALL THIS TIED TOGETHER NICELY. WE ALSO CREATED A WARNING SCREEN, SO THAT IF THIS PATIENT HAD EMERGENT ISSUES THAT SHOULD BE DEALT WITH, SO NOTICE THAT FOR THIS PARTICULAR MOCKUP PRETEND PERSON THAT THEY'VE GOT A NEGATIVE SCREEN FOR DEPRESSION, ANXIETY AND PTSD, BUT THEIR ALCOHOL MISUSE/ABUSE SCORE IS UP. AND SO YOU GET A NICE LITTLE -- IT WARNS THIS MIGHT BE AN AREA YOU SHOULD ADDRESS WITH THE PATIENT. SO WE ALSO HAVE AGAIN IN USING PERCENTILE SCORE THE PROMIS OUTCOMES SHOWN OVER TIME, BECAUSE THEY REQUESTED IT, BETTER HEALTH BEING UP, AND SO YOU GET SO MUCH OUT OF THIS ONE LITTLE PICTURE. IT LOOKS LIKE -- IT'S JUST A GRAPH, BUT YOU SEE IT PHYSICAL FUNCTION, SOCIAL FUNCTION, PAINT INTERFERENCE, AND HOW THEY'RE TRACKING TOGETHER OVER TIME. YOU SEE ALSO WHERE THIS PATIENT IS RELATIVE TO THE U.S. GENERAL POPULATION. AND ONE OF OUR GOALS IN THE FUTURE IS TO NOT JUST HAVE NORMS FOR GENERAL POPULATION, BUT WITHIN THE DOD, TO HAVE DOD-SPECIFIC NORMS, WHICH WOULD BE ANOTHER INTERPRETIVE CONTEXT TO HELP UNDERSTAND THE SCORES. WHEN A PATIENT SITS DOWN TO FILL OUT THE PASTOR REPORT, UP WITH OF THE THINGS HE OR SHE IS INVITED TO DO IS TO LIST THREE ACTIVITIES THAT ARE IMPORTANT TO THEM AND ARE CURRENTLY LIMITED BY PAIN. THEN THEY'RE ASKED TO SAY AT WHAT LEVEL ARE THEY ABLE TO PERFORM CURRENTLY. SO WHAT'S NICE ABOUT THIS IS ON THAT VERY FRONT PAGE, THE CLINICIAN KNOWS THREE MOST IMPORTANT THINGS TO THIS PATIENT THAT ARE AFFECTED BY PAIN. AND HOW THAT PATIENT IS DOING ON THEM TODAY, AND HOW THAT TRACKS OVER TIME. SO IT REALLY CENTERS THE CLINICIAN'S ATTENTION ON WHAT THE PATIENT HAS SAID THAT'S IMPORTANT TO THEM AND REALLY PROMOTES THE KIND OF CONVERSATION THAT YOU WOULD WANT TO BE HAPPENING WITHIN THE CLINICAL ENCOUNTER. SO THE IDEA THEN IS WE HAVE A SCREENING TOOL FOR THE PASTOR AND ALSO BECAUSE -- IMAGINE THIS, WE'VE STARTED THE ARMY -- WALTER REED ARMY MEDICAL CENTER, MOVING ON TO MATTIGAN, TO BALBOA, AND MANY, MANY OTHER MEDICAL CENTERS WITHIN THE MILITARY HEALTH SYSTEM THAT WANT TO COME ON BOARD. WHEN YOU START TALKING ABOUT THE SIZE OF MILITARY HEALTH SYSTEM, VERY QUICKLY, YOU HAVE A BIG, BIG REGISTRY OF DATA TO USE TO ASK THE KINDS OF QUESTIONS THAT YOU CAN ONLY ASK WITH LARGER DATASETS BECAUSE YOU NEED TO ASK THINGS AT A SUBGROUP LEVEL. AND WHAT'S ALSO NICE ABOUT THIS IS THESE ITEMS THAT ARE BEING COLLECTED AS PART OF PASTOR ARE BEING COLLECTED AT STANFORD AS PART OF THEIR PAIN REGISTRY. THEY'RE VERY SIMILAR TO THE ONES THAT EPIC HAS BEEN USING. THEY'RE VERY SIMILAR TO ONES THAT MANY OF THE OTHER THINGS RICHARD SHOWS YOU ARE USING, SO YOU CAN IMAGINE NOW, WE DON'T HAVE 40 DIFFERENT MEASURES FOR FATIGUE, BUT WE HAVE SIMILAR DATA COLLECTED ACROSS MANY POPULATIONPOPULATIONS, ACROSS MANY SETTINGS, AND THESE CAN BE COMBINED FOR THE KIND OF LARGE SCALE STUDIES THAT WE REALLY NEED TO USE TO KNOW THINGS NOT JUST HOW DOES THIS MEAN COMPARE TO THAT MEAN, BUT WHICH KIND OF PATIENT GETS BETTER WITH WHICH KIND OF INTERVENTION? SO THIS IS THE FUTURE. BIG DATA WILL ALLOW US TO ASK REALLY BIG QUESTIONS ABOUT PAIN AND PAIN MANAGEMENT AND OTHER OUTCOMES, LIKE WHO GETS BETTER, SPECIFICALLY WHO GETS BETTER WITH WHAT KIND OF TREATMENT. THIS WILL HELP US ANSWER HOW TO BEST MANAGE PAIN, SOMETHING THAT WE CAN ONLY DO WITH BIG DATA. WE HAVE TO END WITH THIS SLIDE, OF COURSE OH. DO YOU HAVE ANY -- I CAN'T ASK YOU IF YOU HAVE ANY QUESTIONS, I'M SORRY. I'M GOING TO HAVE TO ASK YOU AT A LATER TIME IF YOU HAVE QUESTIONS BECAUSE I ONLY HAVE A MINUTE LEFT. [APPLAUSE] >> I'VE GOT TO STOP SENDING KAREN SELFIES. I SAID IT WAS GOING TO BE ENTERTAINING BUT I HAD NO IDEA. A COUPLE YEARS AGO, I THINK IT WAS, I HAD THE PRIVILEGE OF GIVING A PRESENTATION ON PROMIS AND TOOLBOX, AND FOLKS IN THE AUDIENCE CAME UP TO ME AND SAID WHY DO YOU TELL US ABOUT ALL THESE GREAT TOOLS IF WE DON'T HAVE ACCESS TO THEM HERE AT NIH? THAT REALLY WAS THE IMPETUS FOR WHAT WE HAVE HERE TODAY. ONE OF THOSE PEOPLE WAS MARGARET BEVANS, SHE'S A NURSE IN THE CLINICAL CENTER WHO UNFORTUNATELY DEVELOPED THE FLU YESTERDAY, WE FOUND OUT, SO SHE IS NOT ABLE TO BE HERE. BUT WE DO HAVE LESLIE VERLAND HERE, WHO WILL TELL US WHAT HAS BEEN GOING ON IN TERMS OF EARLY ADOPTION OF THESE PATIENT CENTERED TOOLS AND TECHNOLOGY HERE ON CAMPUS, SO THANK YOU, LESLIE, FOR SUBSTITUTING AND WE LOOK FORWARD TO THIS. [APPLAUSE] >> SO GOOD MORNING, EVERYONE. THANK YOU ALL FOR BEING HERE TODAY AND ALLOWING ME TO SHARE OUR EXPERIENCE WITH USING THE MANAGEMENT INFORMATION SYSTEM IN COLLECTING PATIENT CENTERED OUTCOMES HERE AT THE NIH CLINICAL CENTER. SO AS JIM JUST SAID, MARGARET IS OUT WITH THE FLU AND I VERY UP VERY UNFORTUNATELY IS NOT ABLE TO DELIVER THIS PRESENTATION TO YOU TODAY. TO SAY SHE'S DEVASTATED IS QUITE AN UNDERSTATEMENT. SHE'S REALLY BEEN DRIVING THIS PROJECT FOR SEVERAL YEARS TO BRING IT HERE TO THIS PROGRAM INTRAMURALLY FOR US. SO SHE'S VERY PASSIONATE ABOUT THIS WORK THAT WE DO AND SHE'S REALLY WORKED VERY HARD WITH OUR TEAM TO BRING IT HERE. SO WITH YOUR PATIENCE AND UNDERSTANDING, I WILL DO MY BEST TO SERVE AS HER SUBSTITUTE AND NO ONE IS MORE SORRY THAN I AM THAT SHE'S NOT HERE THIS MORNING. SO HERE WE GO. EARLY THIS MORNING YOU'VE HEARD FROM PREVIOUS SPEAKERS THAT PROMIS, THE NIH TOOLBOX, AND THE NEURO-QOL MEASURES OFFER VALID ASSESSMENTS PORE CRITICAL CONCEPTS FOR PATIENTS THAT WE SEE HERE AT THE NIH INTRAMURAL RESEARCH PROGRAM AND THEY'RE ALSO AVAILABLE VIA ELECTRONIC DATA SYSTEM HERE AT THE CLINICAL CENTER SO THE OBJECTIVE OF THIS PRESENTATION IS REALLY FOR ME TO OUTLINE THE PROCESS THAT AS YOU CONTEMPLATE INCORPORATING THIS NEW SYSTEM INTO YOUR RESEARCH PROGRAMS FROM OUR EXPERIENCE OF USING THIS NEW RESOURCE AT THE CLINICAL CENTER IT. ASSUMING WE ALL AGREE FROM THE START THAT THESE CONCEPTS ENHANCE OUR OVERALL UNDERSTANDING OF THE IMPACT OF OUR RESEARCH ON OUR SUBJECTS, I'M GOING TO SHOW YOU HOW WE CAN MOVE FORWARD WITH THIS NEW RESOURCE. AS YOU SEE ON THE SLIDE, THIS MAY DIFFER BASED ON WHAT YOUR PREVIOUS EXPERIENCE WITH PROs ARE IN YOUR STUDIES. SO THERE ARE KIND OF TWO CAMPS OF PEOPLE THAT WE BELIEVE ARE IN THE AUDIENCE. CAMP ONE ARE THOSE PEOPLE WITH PROs EXPERIENCE. TODAY YOU'RE GOING OH TO LEARN HOW TO IMPLEMENT THIS INTO YOUR RESEARCH PROGRAM. THEN THERE ARE FOLKS IN CAMP TWO, AND THOSE THAT -- THOSE ARE PEOPLE THAT DON'T NECESSARILY HAVE THE EXPERIENCE WITH COLLECTING PSM ROs, AND HOPEFULLY YOU'RE HERE TODAY TO LEARN HOW YOU CAN BEGIN TO START THAT PROCESS PROCESS. SO WHEN CONSIDERING THE TRADITIONAL PROCESS OF LEGACY MEASURES AND BEFORE WE HAD THIS OPTION TO USE THE MEASUREMENT INFORMATION SYSTEM, COLLECTING PSM ROs -- FINDING THE BEST ASSESSMENT TOOLS THAT MEASURE THE CONCEPT YOU WANT IN YOUR POPULATION IS QUITE A CHALLENGE. SOMETIMES THERE ARE THE FEES AND DER THERE ARE ALSO CHALLENGES WITH MANAGING THE DATA ONCE YOU HAVE IT, SCORING IT, MAINTAINING THOSE FILES, AND THEN UNDERSTANDING WHAT THAT DATA IS REALLY TELLING YOU ONCE YOU ACTUALLY HAVE IT. SO THESE CAN BE SIGNIFICANT ISSUES, ESPECIALLY WHEN YOU CONSIDER THOSE PEOPLE IN CAMP TWO, THOSE PEOPLE WITHOUT THAT PRO EXPERIENCE LIKE WHERE WOULD YOU EVEN BEGIN. SO WITH THIS IT'S LESS COMPLEX FOR US, IT OFFERS MANY ADVANTAGES TO INTRAMURAL RESEARCHERS THAT WERE NOT PREVIOUSLY AVAILABLE. THIS NEW MEASUREMENT INFORMATION SYSTEM WILL ADDRESS MANY OF THE TRADITIONAL BARRIERS NOTED ON THE PREVIOUS SLIDE, LIKE THERE ARE NO FEES WITH ITS USE, IT'S WEB BASED SO IT OFFERS THE ABILITY TO COLLECT AND STORE THE DATA, IT OFFERS OPTIONS FOR SPECIFIC ASSESSMENT TOOLS, AND YOU CAN CHOOSE EITHER THE WEB BASED OPTION OR A PAPER FLEXIBLE OPTION IF YOU SO CHOOSE. THERE'S ASSISTANCE FOR SCORING, THE SCORING IS DONE, I'LL EXPLAIN THAT IN JUST A MINUTE, AND ALSO GUIDED INTERPRETATION OF WHAT THOSE SCORES MEAN. ADDITIONALLY THERE ARE TOOLS IN VARIOUS LANGUAGES AVAILABLE. SO TO UNDERSTAND THE IMPACT ON RESEARCH, WOAR GOING TO START BY DEVELOPING PARTNERSHIPS HA ENSURE OUR SUCCESS WITH THE SYSTEM. THAT WILL VARY BASED UPON WHAT CAMP YOU'RE IN. SO FOR THOSE IN CAMP ONE, WITH THE PRO EXPERIENCE, YOU COULD START BY PARTNERING WITH STAFF HERE AT THE ASSESSMENT CENTER AT THE NIH, AND THOSE ARE THE PEOPLE FROM THE AC-LITE TEAM. YOU'LL GET MORE INFORMATION ABOUT DEVELOPING A SURVEY WITH AC-LITE AFTER THIS PRESENTATION. FOR THOSE THAT DON'T HAVE PRIOR EXPERIENCE WITH PROs IN YOUR RESEARCH, YOU MIGHT CONSIDER STARTING OFF BY PARTNERING WITH SOMEBODY HERE IN THE CLINICAL CENTER OR IN THE EXTRAMURAL PROGRAM THAT HAS EXPERIENCE WITH PROs TO HELP GUIDE YOU. THEY COULD E HELP YOU SELECT THE MOST APPROPRIATE MEASURES BEFORE MOVING ON TO DEVELOP THAT SURVEY IN THE AC-LITE SYSTEM. SO TO ILLUSTRATE THE OPTIONS, CONSIDERING THCONSIDER THE FOLLOWING EXAMP LE. BASED ON OUR PREVIOUS RESEARCH STUDIES IN CAMP ONE, SO WITH PRO EXPERIENCE. THIS IS ACTUALLY BASED ON OUR STUDY, SO WE EXAMINE HEALTH RELATED QUALITY OF LIFE, SYMPTOMS AND CAREGIVER BURDEN. THIS NEW MEASUREMENT INFORMATION SYSTEM OFFERS COMPARABLE MEASURES WITH THE CONCEPT THAT WE EXPLORE FOR SOME OF THE CONCEPTS BUT NOT ALL, SO FOR HEALTH RELATED QUALITY OF LIFE, THERE'S THE PROMIS GLOBAL HEALTH. AN IMPORTANT FEATURE IS IT PERMITS YOU TO COMPARE DATA YOU COMPLETED PREVIOUSLY OR OFFERS STANDARDIZED COMPARISON TABLES FOR SOME OF THE OTHER COMMONLY USED MEASURES. THERE ARE ALSO PROMIS MEASURES FOR THE COMMON SYMPTOMS THAT WE MEASURE IN OUR STUDIES, SUCH AS PAIN, FATIGUE OR SLEEP DISTURBANCE, BUT WHAT THERE WASN'T ONE FOR FOR OUR PARTICULAR STUDY WAS FOR CAREGIVER BURDEN. SO WE WERE ABLE TO CREATE A CUSTOM MEASURE AND INCORPORATE THAT INTO OUR SURVEY. SO IF YOU'RE IN CAMP TWO, WITHOUT PRIOR PRO EXPERIENCE, YOU COULD START BY REVIEWING THE MEASURES THAT -- TAKING A STEP BACK AND REVIEWING THE MEASURES THAT ARE AVAILABLE IN THIS SYSTEM, SO LOOKING AT WHAT ARE THE PROMIS MEASURES AND THE CONCEPTS, THE NEURO-QOL AND -- TOOLBOX TO DETERMINE THE RELEVANCE OF THOSE MEASURES TO YOUR SPECIFIC POPULATION BEFORE YOU BEGIN BUILDING YOUR SURVEY. SO THIS IS JUST A SCREEN SHOT AND IT'S A VERY BRIEF LISTING OF THE PROMIS MEASURES AVAILABLE. AS YOU SEE AT THE BOTTOM, THERE'S A VARIETY OF DIFFERENT DOMAINS THAT ARE AVAILABLE, AND THERE ARE MEASURES FOR ADULT PEDIATRIC PATIENTS AND ALSO PATIENT PROXY. YOU'LL SEE THAT SOME ARE SHORT FORMS AND SOME ARE ITEM BANKS SO THIS IS YOUR CAT FORM. THESE ARE A BRIEF LISTING OF THE NEURO-QOL MEASURES THAT ARE AVAILABLE. THE DOMAINS, YOU SEE YOU HAVE BOTH ADULT AND PEDIATRIC MEASURES, AND A VARIETY OF FORMATS, EITHER IN CAT FORM OR IN A SHORT FORM BANK. FINALLY THERE'S THE NIH TOOLBOX. AGAIN, YOU HAVE MANY DIFFERENT CONCEPTS AND DIFFERENT OPTIONS PORE YOU TO CHOOSE FROM. REGARDLESS OF WHAT MEASURE YOU CHOOSE, WHETHER IT'S THE NIH PROMIS, THE NEURO-QOL OR THE NIH TOOLBOX, OR P YOU'RE ADDING CUSTOM MEASURES THAT YOU CAN REALLY CUSTOMIZE THIS APPROACH TO MEET YOUR NEEDS OF YOUR POPULATION. THE NEXT I WANT TO SHOW YOU IS AN EXAMPLE OF A STU IT DI WHERE SOMEONE IN CAMP TWO, SO A RESEARCHERS THAT HAS NO PRO EXPERIENCE, AND THEY ACTUALLY PARTNERED WITH DR. BEVANS TO DO PRO -- INCORPORATE PRO INTO THEIR CLINICAL RESEARCH STUDY. SO THIS IS A REAL LIFE EXAMPLE. THE PROTOCOL IS DR. DESMOND'S PROTOCOL AND HE WAS LOOKING AT THE EFFICACY OF THIS NEW AGENT AND SEVERE APLASTIC ANEMIA PATIENTS. HE WANTED TO INCORPORATE PRO DATA INTO HIS BIOMEDICAL -- INTO HIS RESEARCH FOR THIS DRUG. AND HE ADDED THAT, WANTED TO ADD THAT AS A SECONDARY OUTCOME. AND THAT WAS -- HE WANTED TO LOOK AT HEALTH RELATED QUALITY OF LIFE, SO HE PARTNERED WITH DR. BEVANS TO EXAMINE HEALTH RELATED QUALITY OF LIFE, IT CAME UP THAT THAT WAS AN IMPORTANT FEATURE, AND SELECTED THE PROMIS GLOBAL HEALTH. THEY ALSO REALIZED THAT THERE ARE SOME MEASURES THAT THEY PROBABLY SHOULD CHU GIVEN THIS POPULATION, AND THE FACT THAT MEASURES WERE SELECTED FOR THIS POPULATION BUT WERE NOT INCLUDED IN THE MEASUREMENT INFORMATION SYSTEM. THERE WERE SPECIFIC SYMPTOMS THEY WANTED TO LOOK AT AND THERE WERE PROMIS MEASURES AVAILABLE. THIS IS COMPLETELY CUSTOMIZABLE DEVELOPING AC-LITE IN ALL OF THESE MEASURES. THIS IS A REAL LIFE EXAMPLE OF WHAT DR. BEVANS DOES AT THIS CENTER. SO NOW LET'S CONSIDER AN EXAMPLE OF WHEN THE RESEARCHERS IN CAMP ONE, SO THIS IS A RESEARCHER THAT HAS EXPERIENCE WITH PROs AND USES THEM AS PRIMARY OUTCOMES IN THEIR RESEARCH STUDIES. TO EXPLORE BURDEN AND STRESS IN CANCER CAREGIVERS. WE REFER TO IT AS PROMIS TO EXPLORE BASIC. THIS REALLY IS TESTING THE PLATFORM OF AC-LITE HERE AT THE NIH CLINICAL CENTER. SO AS PREVIOUSLY MENTIONED, THERE WERE A VARIETY OF MEASURES AVAILABLE IN THE SYSTEM TO LOOK AT THE CONCEPTS THAT WERE REALLY IMPORTANT TO US IN THIS STUDY. THERE WAS NOT ONE FOR THE CAREGIVER REACTION, THE CAREGIVER BURDEN, ASSESSING CAREGIVER BURDEN, SO THAT HAD TO BE CREATED AS A CUSTOM MEASURE. BUT WHAT'S INTERESTING IS THAT THE NIH THAT WE HAD PREVIOUSLY USED AT UCLA PERCEIVED STRESS SCALE AND SOME OF OUR OTHER WORK, AND THE NIH TOOLBOX HAS INCORPORATED THIS SAME SCALE INTO THE MEASURES THAT THEY OFFER. SO THE REST OF THE TALK WILL REALLY FOCUS ON THE CONCRETE EXAMPLES OF HOW WE DEVELOP THE STUDY AND HOW WE'RE USING IT HERE IN THE CLINICAL CENTER NOW. SO THIS IS AN EXAMPLE OF PARTICIPANT REGISTRATION SCREEN THAT WE USE IN THE PROMIS STUDY SO THIS IS WHAT IT WOULD LOOK LIKE IF YOU BUILT YOUR SURVEY WITH THE AC-LITE SYSTEM HERE AT THE NIH. AND PLEASE NOTE THAT PATIENTS CAN SELF-REGISTER OR THEY CAN -- OR THE TEAM CAN SELECT TO PREREGISTER THEIR SUBJECTS AND WE'VE ACTUALLY OBJECTED FOR THE PREREGISTRATION OPTION. SO IT DOESN'T TAKE VERY MUCH TIME OR EFFORT TO DO THIS. YOU JUST FILL IN THE STUDY ARM, THE SITE, AND THEN YOU ASSIGN THEM A USER NAME, THE PASSWORD, AND THEN YOU CAN IDENTIFY IF THEY WERE CONSENTED, SO THIS CAN ALSO BE HELPFUL FOR IRB TRACKING AND REPORTING. ADDITIONALLY, HE WOULD FILL OUT THE NEXT TAB HERE, YOU'D FILL OUT THE AGE OF YOUR PATIENT, AND THIS HELPS YOU WITH YOUR ENROLLMENT INCLUSION REPORTING TO THE IRP. SO ONCE A SUBJECT HAS EITHER REGISTER ITED THEMSELVES OR YOU'VE REGISTERED THEM, THIS IS THE SCREEN THEY WOULD COME TO TO BEGIN THEIR SURVEY. THEY WOULD LOG IN WITH THE ASSIGNED USER NAME AND PASSWORD TO BEGIN. THIS IS A SCREEN SHOT, AN EXAMPLE OF WHAT THE PATIENT WOULD SEE USING A PROMIS ITEM SO YOU'LL SEE THAT THERE'S ONLY ONE ITEM PER PAGE, IT AVOIDS THAT CONFUSION OF HAVING A SURVEY WITH MULTIPLE ITEMS TO ANSWER ON ONE PAGE. THIS IS AN EXAMPLE OF WHAT AN NIH TOOLBOX ITEM LOOKS LIKE. SO YOU'LL SEE THAT IT LOOKS A LITTLE DIFFERENT THAN THE PROMIS MEASURE, BUT IT IS AT THE SAME -- THE SAME IN THAT IT'S ONE QUESTION PER PAGE AND THAT THIS AUTOMATICALLY ADVANCES FOR THE PATIENT ONCE THEY'VE ANSWERED. I ALSO WANTED TO SHOW YOU AN EXAMPLE OF A CUSTOM ITEM SO YOU'LL SEE IT'S REALLY THAT THEY'RE NOT GOING TO GET THREE DIFFERENT LOOKING KINDS OF SURVEYS IF YOU'RE SELECTING DIFFERENT MEASURES, SO THIS IS AN EXAMPLE OF A CUSTOM ITEM AND IT'S ONE OF THE ITEMS FROM OUR HEALTH PROMOTING LIFESTYLES, A MEASURE THAT WE USED. SO ALSO WHAT'S IMPORTANT TO SEE FOR YOU ALL AS YOU CONSIDER OPTIONS FOR YOUR STUDIES, WHAT IT LOOKS LIKE FOR YOUR RESEARCH TEAMS AND YOUR ADMINISTRATORS, SO WHEN YOU'RE CREATING -- YOU GO TO THIS OPTION. WHEN YOU LOOK AT THE DATA, YOU HAVE A VARIETY OF OPTIONS AND YOU LOOK AT REGISTRATION DATA, YOU CAN LOOK AT THEIR ASSESSMENT SCORES, AT THE CONSENT REGISTRATION DATA OR YOU CAN EVEN LOOK AT A TABLE -- THIS IS ALL THE DATA IF YOU HAVE CUSTOM MEASURES AND A VARIETY OF NIH TOOLBOX OR PROMIS MEASURES OR NEURO-QOL. ALSO HERE, YOU CAN LOOK AT THE -- THIS IS JUST A BRIEF SUMMARY AT THE TOP OF ALL THOSE ITEMS. SO WE THOUGHT IT WOULD BE HELPFUL FOR YOU TO SEE REALLY WHAT YOU'RE GOING TO BE GETTING WHEN YOU DO HAVE SOME DATA, SO THIS IS DATA FROM ONE OF THE SUBJECTS OR SOME OF THE SUBJECTS WE'VE COLLECTED ON THIS STUDY. SO THIS IS ONE CASE YOU'LL SEE HERE, THESE ARE THE SCORED ITEMS FROM THE NIH TOOLBOX, THE PROMIS MEASURES AND NEURO-QOL MEASURES THAT WE'RE USING IN OUR STUDY. THEN IF I CLICK AGAIN, I'M JUST GOING TO HIGHLIGHT HERE, ANXIETY. SO YOU CAN SEE THAT ANXIETY WAS COMPUTER ADAPTIVE TEST FROM THE ITEM BANK. THE LAW SCORE WAS 8 FOR THIS SUBJECT, AND THEIR FINAL SCORE WAS 56.49, AND AS YOU'VE HEARD THIS MORNING, EVERYTHING IS BASED ON THAT STANDARDIZED SCALE WHERE 50 IS WHERE THE GENERAL POPULATION SITS. AND IN THIS CASE, WE CAN SEE THAT THE SUBJECT IS MAYBE JUST A LITTLE MORE ANXIOUS THAN THE GENERAL POPULATION. THIS IS THE SCORE YOU WOULD USE IN YOUR ANALYSIS. ALTHOUGH THERE ARE MANY ADVANTAGES TO HAVING THIS NEW INFORMATION SYSTEM, WE WANTED TO SHARE THAT IT'S NOT ALL ROSES AND GREAT. THERE ARE STILL SOME THINGS WE HAVE TO WORK ON THAT NEED YOUR CONSIDERATION AS YOU BEGIN TO USE IT. THERE ARE SOME ISSUES THAT WE'VE NOTICED WITH COMPATIBILITY, SO SOME BROWSERS WORK BETTER THAN OTHERS. IT'S ALSO NOT -- IT'S NOT BEEN FUNCTIONING EXACTLY PERFECTLY, SO WE DON'T RECOMMEND IT FOR USE IN A MOBILE MEDIA, SO IF A PATIENT ONLY HAS A SMARTPHONE TO ACCESS THIS SYSTEM, IT'S PROBABLY NOT THE BEST OPTION AT THIS TIME BUT HOPEFULLY WE'LL BE ADDRESSING THOSE IN THE FUTURE. THERE ARE SOME ISSUES WITH CREATING MANUAL TOOLS OR CUSTOM TOOLS, SO YOU JUST HAVE TO DEVELOP THOSE IN ADVANCE. THEY HAVE TO BE TRANSLATED OVER INTO THE AC-LITE SYSTEM AND THEN IT LEAS SOME TESTING THAT HAS TO GO TO ENSURE THAT WHAT YOU REALLY ARE PUTTING IN IS WHAT YOU WANT THE PATIENTS TO SEE AND WHAT YOU'RE SUPPOSED TO BE DOING. THERE'S NO AUTOMATIC SCORING OF THE CUSTOM INSTRUMENTS. SO THE NICE FEATURE IS WHEN YOU'RE USING THAT NIH PROMIS, THE TOOLBOX, I'M SORRY, THE PROMIS, THE NIH TOOLBOX, NOR QO NEURO-QOL, AND ALSO THE RESPONSE DATA, I HAD A PATIENT THAT WE ENROLLED TELL ME AFTERWARDS, SHE SAID YOU KNOW, I THINK I MIGHT HAVE ANSWERED ONE OF THOSE QUESTIONS INCORRECTLY. I MEANT TO SAY THIS, BUT I REALLY SAID THIS. SO UNFORTUNATELY WE CAN'T REALLY GO BACK AND EDIT THAT OR ISOLATE THAT OTHER THAN MAKING HAND NOTATIONS IN OUR DATA. AN ADDITIONAL HINGE THAT WE THING THAT WE NOTICED IS THERE'S NOT A WAY TO TRACK WHEN OUR SUBJECTS ARE COMPLETING THEIR SURVEYS, SO WE'LL HAVE TO GO IN AND EXPORT DATA TO SEE IF THEY'VE COMPLETED, BUT THERE'S NOT THAT AUTOMATIC NOTIFICATION THAT A SUBJECT HAS COMPLETED IT. RIGHT NOW THE USE OF THIS SYSTEM IS PRETTY MUCH LIMITED TO RESEARCH CARE BECAUSE WE DON'T HAVE IT CONNECTED TO THE PATIENT'S MEDICAL RECORD WHICH LIMITS OUR APPLICATION TO CLINICAL CARE A AT THIS MOMENT. SO AS SOME OF YOU MIGHT BE CONSIDERING, THESE MEASURES OR THIS SYSTEM AS A VALUE TO, THIS COULD BE A FEATURE IN THE FUTURE, ONCE THE CONSIDERATIONS ARE ADDRESSED FOR HOW WE CAN INCORPORATE THAT DATA INTO THE ELECTRONIC MEDICAL RECORD. BUT I HOPE YOU ALL SEE THERE'S DEFINITELY A FUTURE POTENTIAL FOR IT USED IN CLINICAL PRACTICE, BUT WE'RE JUST NOT THERE YET. SO THIS IS OBVIOUSLY AN OPTION FOR YOUR RESEARCH STUDIES, AND WE WILL HOPEFULLY BE WORKING TOWARDS THAT IN THE FUTURE. I WANT TO THANK YOU ALL FOR YOUR TIME AND LETTING ME SHARE WITH YOU THE TEAM'S EXPERIENCE WITH THE RESEARCH APPLICATION OF THIS NEW SYSTEM, AND I'LL BE GRAD TO GLAD TO TAKE ANY QUESTIONS THAT YOU MIGHT HAVE AND I'LL DO MY BEST TO ANSWER ON BEHALF OF OUR TEAM. THANK YOU. [APPLAUSE] I UNDERSTAND WE'LL TAKE QUESTIONS AT THE END. THANK YOU. >> GREAT. THANK YOU, LESLIE. FOR THE REST OF THE MORNING, WHAT WE WANTED TO DO WAS TO GIVE THOSE OF YOU THAT WILL NOT BE IN THE TRAINING THIS AFTERNOON A SENSE OF WHAT YOU'VE NOW HEARD AT AC-LITE, SO WE HAVE THE GOOD FORTUNE TO HAVE NAN ROTHROCK HERE FROM NORTHWESTERN, AND WE ALSO HAVE YANG FANN, DIRECTOR OF THE INTRAMURAL T. AND BIOINFORMATICS PROGRAM HERE AT NINDS. I HOPE YOU HAVE SIGNED UP FOR THE IN DEPTH TRAINING THIS AFTERNOON, WHICH IS GOING TO BE IN BUILDING 12A. ROOM B51. SO WITHOUT ANY FURTHER DELAY, NAN AND YANG. >> THANKS, JIM. I'M HAPPY TO BE HERE TO REPRESENT THE ASSESSMENT CENTER TEAM. I WON THE COIN TOSS SO I'M GOING TO BE TALKING AND YANG WILL BE AVAILABLE TO ANSWER ALL OF YOUR HARD QUESTIONS WHEN WE GET TO THE END HERE. SO THAT'S OUR PLAN. I WANT TO TALK ABOUT ASSESSMENT CENTER, IT'S A CLIP CALORIE SECH DATA COLLECTION TOOL. THAT WAS ACTUALLY ONE OF THE AIMS IN THE PROMIS RFA. THERE WAS THIS AWARENESS THAT IF WE CREATE ALL THESE GREAT MEASURES AND DON'T ACTUALLY CREATE A WAY FOR PEOPLE TO ADMINISTER THOSE MEASURES TO TO PATIENTS OR STUDY PARTICIPANTS, IT'S NOT GOING TO BE PARTICULARLY EFFECTIVE. SO WE DEVELOPED ASSESSMENT CENTER THAT DEVELOPS BELLS AND WHISTLES TO ALLOW YOU TO USE THESE NEW TOOLS. IT FUNCTIONS BY ALLOWING YOU TO CREATE A STUDY-SPECIFIC URL THAT A PARTICIPANT CAN ACCESS IN THE CLINIC OR REMOTELY, AND THE LIBRARY AS YOU'VE SEEN INCLUDES EVERYTHING FROM PROMIS, NEURO-QOL AND THE NIH TOOLBOX. YOU CAN ALSO HAVE RESEARCHER COMPLETED INSTRUMENTS, AND SO THAT'S HELPFUL FOR SOME CHART REVIEW DATA OR THINGS LIKE THAT. AS LESLIE INDICATED, YOU CAN CREATE CUSTOM SHORT FORMS SO IF YOU WANT TO ADD AN INSTRUMENT THAT'S NOT PART OF THE LIBRARY, YOU CAN DO THAT. YOU CAN HAVE SINGLE TIME POINT OR MULTIPLE TIME POINT STUDIES. OTHER FEATURES WERE DEVELOPED TO REALLY ENHANCE THE RESEARCH APPLICABILITY OF THIS SOFTWARE SO IF YOU HAVE MULTIPLE PARTICIPANTS WHO SHOULD BE RECEIVING DIFFERENT INSTRUMENTS, YOU CAN SET THAT UP. YOU CAN ALSO DO A RIDICULOUS AMOUNT OF RANDOMIZATION. YOU CAN RANDOMIZE ITEMS WITHIN AN INSTRUMENT, YOU CAN RANDOMIZE INSTRUMENTS, YOU CAN RANDOMIZE GROUPS OF INSTRUMENTS, SO IF YOU HAVE ANY CONCERNS ABOUT ORDER EFFECTS, THIS IS A GOOD TOOL FOR YOU BECAUSE YOU CAN RANDOMIZE UNTIL THE CATS COME HOME. WE HAVE SOME COOL TRACKING FEATURES SO YOU CAN MONITOR HOW YOU'RE DOING TOWARDS REACHING YOUR A ACCRUAL GOAL. ONE HINGE THAT I THINK IS EXTREMELY HELPFUL IS DATA IS AVAILABLE AT ANY TIME BY SOMEBODY WITH PERMISSION TO EXPORT DATA ON YOUR I'M. AS YOU SAW ON A SCREEN SHOT LESLIE SHOWED, THE DATA IS SEPARATED INTO DIFFERENT FILES SO IF YOU HAVE CONTACT INFORMATION, YOU CAN KEEP THAT SEPARATE FROM YOUR PATIENT-REPORTED OUTCOME DATA SO THAT'S HELPFUL IN EASING THE TASK OF DE-IDENTIFYING DATA PRIOR TO ANALYSES. THERE'S ALSO AN AUTOMATED DATA DICTIONARY WHICH HAS MADE OUR TEAM REALLY POPULAR WITH STATISTICIANS BECAUSE YOU CAN SEE EXACTLY WHAT THE ITEM INFECTIONS DISEASE WAS, EXACTLY WHAT THE ITEM WAS AND WHAT EVERY RESPONSE WAS. SO THAT HELPS CLARIFY WHAT WAS ACTUALLY INCLUDED. AS I SAID, THE INSTRUMENT LIBRARY HAS EVERYTHING THAT WE'VE HEARD ABOUT TODAY. THE TOOLBOX A LITTLE UNIQUE IN THAT YOU DO HAVE TO GET PERMISSION TO ACCESS IT BECAUSE OF CONCERNS ABOUT SECURITY. THE OTHER THING I WANT TO HIGHLIGHT, IF YOU HAVE NO INTEREST IN USING ASSESSMENT CENTER AS A DATA COLLECTION TOOL, IT'S WORTH EXPLORING AT LEAST TO PULL DOWN A PDF FOR VARIOUS INSTRUMENTS. SO IF YOU WANTED TO LOOK AT A PROMIS SHORT FORM, COW GO INTO ASSESSMENT CENTER AND PULL DOWN A PDF. THE FUNDING FOR ASSESSMENT CENTER WAS PROVIDED BY NIH THROUGH GRANTS, NEURO-QOL, PROMIS AND TOOLBOX. THAT COVERS HARDWARE SOFTWARE MAINTENANCE, NEW FEATURE DEVELOPMENT, A NEW HELP DESK AVAILABLE DURING BUSINESS HOURS IN CHICAGO. I DO WANT TO NOTE WE ARE TRANSITIONING AND THAT THERE ARE GOING TO BE SUPPORT AND HOSTING FEES FOR SOME USERS IN SOME INSTANCES. I'M MAKING THIS AS VAGUE AS POSSIBLE. AS WE SHIFT AWAY FROM NIH FUNDING TO A SUSTAINABILITY MODEL. SO IT'S A WEB BASED TOOL, YOU OH COULD RIGHT NOW GO ONLINE AND GO TO WWW.ASSESSMENTCENTER.NET. ANYONE CAN REGISTER. YOU CAN SEE IN THE UPPER RIGHT CORNER HERE THIS REGISTER NEW USER LINK, YOU COULD REGISTER TODAY AND POKE AROUND. ONE THING ABOUT ASSESSMENT CENTER IS ALL OF THIS DATA THAT YOU COLLECT WITHIN THE APPLICATION IS STORED ON A SERVER OWNED BY NORTHWESTERN UNIVERSITY IN CHICAGO. SO THIS PRESENTED A PROBLEM FOR THE NIH CLINICAL CENTER BECAUSE WE LEARNED THAT THE CLINICAL CENTER HERE IS KIND OF LIKE VEGAS, WHAT HAPPENS HERE HAS TO STAY HERE. THEY COULDN'T HAVE DATA STORED IN CHICAGO, IT NEEDS TO BE HERE LOCALLY ON AN NIH SERVER. SO THIS PRESENTED A CHALLENGE. THE SOLUTION THAT WE CAME UP WITH WAS WHAT YOU'VE HEARD REFERRED TO AS AC-LITE. IT SOUNDS TO ME LIKE A SOFT ROCK STATION AND THAT IS NOT WHAT IT IS. SO I'M GOING TO TRY TO EXPLAIN THE LINK BETWEEN ASSESSMENT CENTER IT AND AC-LITE USING CLIP ART. I WILL SAY I AM TRAINED AS A HEALTH PSYCHOLOGIST, NOT AS ANYTHING RELATED TO INFORMATICS, SO FORGIVE ME, MONICA, MY COLLEAGUE, WHO IS PROJECT MANAGER FOR THIS ATTEMPT AT EXPLAINING HOW THIS WORKS. SO WE START WITH AN NIH PERSON WHO IS AT THEIR TYPICAL WORK STATION THEY ARE HOUSED ON A SERVER AT NORTHWESTERN, AND BEHIND A DEEP, DEEP FIREWALL ON THE NIH SERVER IS AC-LITE. AS AN NIH STAFF PERSON POINTS THEIR BROWSER TO ASSESSMENT CENTER. THEY LOG IN, THEY DO ALL OF THEIR STUDY SETUP SO THEY PICK WHICH INSTRUMENTS THEY WANT TO INCLUDE, THEY DECIDE HOW MANY TIME POINTS, THEY CONFIGURE THE STUDY EXACTLY AS THEY WANT, THEY LAUNCH IT FOR DATA COLLECTION SO IT'S READY TO GO. THEY CAN'T USE ASSESSMENT CENTER AS THE DATA COLLECTION PLATFORM ACCORDING TO NIH POLICY. SO WHAT THEY THEN -- THE PERSON THEN DOES IS DIRECT THEIR BROWSER TO THE AC-LITE SYSTEM, AND THE FIRST THING YOU NEED TO DO IS GET THROUGH THIS FIREWALL INTO THE SYSTEM, SO THEY HAVE TO LOG ON USING THEIR NIH LOG-IN. ONCE THEY DO THAT, THEY CAN GET TO AC-LITE. AC-LITE, THEN, CALLS UP ASSESSMENT CENTER AND SAYS, HEY, THERE'S THIS AWESOME STUDY THAT THIS PERSON JUST SET UP, I'D LIKE TO PULL IT IN TO AC-LITE. ASSESS THEMENASSESSMENT CENTER SAYS SURE, PUSHES ALL THAT OVER, SO NOW IT'S AVAILABLE ON AC-LITE. AND THEN THIS INVESTIGATOR WOULD USE AC-LITE AS THE DATA COLLECTION PLATFORM TO COLLECT DATA AND THEN ALSO WHERE THEY EXTRACT DATA. NOW THE GENIUS OF THIS SYSTEM, AND I ATTRIBUTE THIS TO MONICA AND HER TEAM, IS THAT WHAT THIS MEANS IS THAT AS NEW INSTRUMENTS ARE DEVELOPED, AS WE HEARD ABOUT ADAPTIONS AND NEW SHORT FORMS, NEW DOMAINS ARE DEVELOPED AND LOADED INTO ASSESSMENT CENTER, THEY'RE AUTOMATICALLY DEVELOPED HERE. WE DON'T HAVE TO GO THROUGH THE PROCESS OF PUSHING THINGS OVER, SO THEY'LL BE PART OF THE LIBRARY AND YOU CAN USE THEM. SO IT'S GOOD FOR SUSTAINABILITY THAT YOU'LL ALWAYS HAVE ACCESS TO THE NEW INSTRUMENTS AS THEY JOIN THE LIBRARY. SO YOU SAW THIS IN LESLIE'S SLIDE. I'M GOING TO JUST SHOW YOU WHAT PARTICIPANTS WOULD SEE IF THEY WERE COMPLETING AN ASSESSMENT IN AC-LITE. THERE'S A WELCOME PAGE THAT'S CUSTOMIZABLE. THIS IS A SUPER BORING ONE THAT I MADE, ASSUMING THIS WAS AN ORTHOPEDIC OUTCOME STUDY. YOU HAVE THE ABILITY TO LOG IN IF YOU ALREADY HAVE A LOG-IN, IF YOU'RE PREREGISTERED OR RETURNING TO A SECOND OR THIRD ASSESSMENT OR YOU CAN START FROM SCRATCH. THE INTERFACE, VERY SIMPLE. THIS IS AN EXAMPLE OF A PROMIS MOBILITY ITEM. THIS IS ANOTHER PROMIS MOBILITY ITEM. THIS IS A PROMIS UPPER EXTREMITY ITEM. AS IS THIS. ARE YOU ABLE TO HOLD A PLATE FULL OF FOOD. THEN WE HAVE SOME OF THE PAINT INTERFERENCE ITEMS THAT KARON MENTIONED EARLIER THIS MORNING. SO FOR THE PARTICIPANT, IT JUST GOES FROM ONE INSTRUMENT TO THE NEXT, THEY DON'T KNOW, THEY'RE ON A SECOND INSTRUMENT, IT WOULD BE SEAMLESS WITH ANY CUSTOM ITEMS YOU WOULD WRITE OR INSTRUMENTS YOU WOULD ADD MANUALLY THAT AREN'T PART OF THE LIBRARY. THEY GET A COMPLETION MESSAGE AND THEY'RE DONE. SO HOW WOULD YOU START DOING THAT? WHAT YOU WOULD DO HERE IS YOU WOULD NAVIGATE TO THIS WEBSITE, AC.NINDS.NIH.GOV AND YOU WOULD LOG IN WITH YOUR EXISTING NIH USER NAME AND PASSWORD. THIS IS THE NIH ASSESSMENT CENTER HOME PAGE. YOU CAN SEE IT HAS SOME DESCRIPTIVE INFORMATION AND THEN FIVE LINKS HERE, ONE OF WHICH WILL DIRECT YOU TO ASSESSMENT CENTER WHERE YOU DO ALL YOUR STUDY SETUP, AND THE SECOND LINK, AC LIGHT, WHERE YOU DO YOUR DATA COLLECTION AND EXTRACT YOUR DATA. SO THIS IS ASSESSMENT CENTER. WHEN YOU ARE SETTING UP YOUR STUDY. ASSESSMENT CENTER IS ORGANIZED BY TABS, THERE ARE FIVE TABS UP THERE AT THE TOP. STUDIES, INSTRUMENTS SETUP PREVIEW AND ADMINISTRATION. I AM ON THE STUDIES PAGE AND YOU CAN SEE I'VE BEEN BUSY, I HAVE SEVERAL STUDIES LISTED HERE, EACH ONE OF THOSE HAS A DIFFERENT SET OF INSTRUMENTS. THIS IS ALSO WHERE YOU CAN IDENTIFY WHO'S ON YOUR STUDY TEAM. IF IF I WANTED TO ADD KARON COOK, I WOULD LIKE FOR THE NAME, ADD HER AND NOW SHE HAS THE ABILITY TO EXTRACT DATA, ADD PARTICIPANTS OR WHATNOT. SO THE SECOND TAB IS INSTRUMENTS. THIS IS WHERE YOU WOULD PULL IN THE INSTRUMENTS YOU WOULD WANT TO INCLUDE IN YOUR STUDY SO YOU CAN ADD INSTRUMENTS FROM THE LIBRARY AS I DID HERE. I HAVE THE PROMIS MOBILITY CAT, UPPER EXTREMITY CAT AND PAIN INTERFERENCE CAT. THIS IS ALSO WHERE YOU WOULD CREATE ITEMS SO IF I WANTED TO CREATE THE BPI, WHICH IS NOT IN THE LIBRARY, YOU COULD DO THAT. ALTHOUGH AFTER SEEING KARON'S SLIDES, WHY WOULD I WANT TO CREATE THE BPI. ONSETUP, THIS IS AN AREA WHERE YOU WOULD SET YOUR WEBSITE UP FOR DATA COLLECTION. YOU CAN SET UP WHEN YOU WANT ACCRUAL TO END OR WHEN YOU REACH A SPECIFIC TARGET SAMPLE SIZE, YOU CAN STOP. THIS IS WHERE YOU WOULD CUSTOMIZE THAT WELCOME PAGE AND YOU CAN ADD IMAGES AND MODIFY THE TEXT WITH FONT COLOR AND WHATNOT. THIS IS ALSO WHERE IF YOU NEEDED TO INCLUDE AN ONLINE CONSENT FOM, YOU WOULD TO FORM, YOU WOULD DO THAT, OR MULTIPLE TIME POINTS, YOU WOULD DO THAT HERE AS WELL. SO NOW ALL YOUR STUDIES ARE DONE, YOU'RE READY TO GO, YOU GO BACK TO THE ASSESSMENT HOME PAGE AND YOU ACCESS AC-LITE OF THE THE FIRST THING YOU DO IS YOU PULL YOUR STUDY FROM ASSESSMENT CENTER INTO AC-LITE ON THIS DOWNLOAD STUDIES PAGE. IT REQUIRES LOGGING IN, THAT POPULATES A LIST OF STUDIES YOU HAVE IN ASSESSMENT CENTER. YOU CHECK WHAT YOU WANT AND YOU BRING IT OVER. THIS NOW IS MEU STUDY LIST IN AC LIGHT. YOU CAN SEE THERE ARE A NUMBER OF STUDIES THAT HAVE BEEN PULLED OVER. I LOOK AT THE ONE I WANT, I HAVE THE ABILITY TO EITHER GO TO AN ADMINISTRATION AREA WHERE I CAN SEE -- I'LL SHOW YOU THAT IN A SECOND, HOW MANY PEOPLE PARTICIPATED, OR I CAN COLLECT -- IF YOU HAVE BIONIC EYES, YOU CAN READ, IT SAYS START. SO I HAVE A PARTICIPANT RIGHT IN FRONT OF ME AND I WANT HER TO START DATA COLLECTION, I WOULD CLICK START AND SHE'D BE READY TO GO. IF I CLICKED ADMINISTRATION, I SEE THIS PAGE THAT YOU SAW THAT LESLIE SHOWED, IT SHOWS A QUICK UPDATE ON YOUR ACCRUAL. I HAVE A MASSIVE STUDY GOAL OF 25 PARTICIPANTS AND 13 PEOPLE HAVE ACCESSED THE STUDY BUT NOBODY'S STARTED IT. SO NOT A VERY EFFECTIVE RECRUITMENT YET. THEN DOWN HERE IS WHERE I WOULD EXPORT MY DATA. AGAIN ANOTHER SAMPLE DATA EXPORT. THESE ARE CSV FILES SO THEY'RE READY TO GO IF YOU WANT TO OPEN THEM IN EXCEL, THERE ARE NICE TABLES THAT DESCRIBE FOR YOU EXACTLY WHAT EACH COLUMN MEANS, SO WE CAPTURE LOTS OF INFORMATION, INCLUDING HOW MANY SECONDS A PARTICIPANT TOOK TO ANSWER AN ITEM, A TIME AND DATE STAMP OF WHEN THEY PROVIDED THAT RESPONSE, AND SO YOU CAN GET A LOT OF INFORMATION FROM THESE FIVE DIFFERENT EXPORTS. THIS IS A LOT TO LEARN AND YOU HAVE A LOT OF RESOURCES TO HELP YOU LEARN THE TECHNOLOGY. TO LEARN AC-LITE, THERE IS A USER MANUAL AND IT'S ON THAT HOME PAGE UNDER INSTRUCTION, YOU CAN GO THERE AND DOWNLOAD THAT MANUAL AND READ MORE ABOUT AC-LITE. THERE'S ALSO AN AC-LITE SUPPORT DESK THAT'S AVAILABLE BY EMAIL ADDRESS AND ALSO BY PHONE DURING NORMAL BUSINESS HOURS, NOT AT 2:00 IN THE MORNING WHEN YOU'RE HAVING TROUBLE, THEY'LL RETURN YOUR CALL IN THE MORNING. TO LEARN ASSESSMENT CENTER, WE HAVE A LOT OF RESOURCES ON THE ASSESSMENT CENTER HOME PAGE, INCLUDING SOME ONLINE VIDEO TUTORIALS, SO IF YOU'RE WITHIN DERING HOW DO I SET UP MY OWN SHORT FORM THAT'S NOT IN THE LIBRARY, I CAN'T FIGURE OUT HOW TO DO THAT, THERE'S A QUICK VIDEO YOU CAN WATCH. IF YOU PREFER TO READ, YOU CAN PULL DOWN THE USER MANUAL AND THAT SAME INFORMATION IS PRESENTED IN TEXT FORM. AND IF YOU WANT TO LEARN MORE ABOUT WHAT TO EVEN INCLUDE IN YOUR STUDY, YOU SHOULD GO TO THE STUDY-SPECIFIC WEBSITE. ONE OF THE THINGS WE FOUND IS PEOPLE OFTEN GO TO ASSESSMENT CENTER NOT KNOWING WHAT INSTRUMENTS THEY WANT TO USE AND ARE PUZZLED BECAUSE THERE AREN'T INSTRUMENT SELECTION GUIDES WITHIN ASSESSMENT CENTER. THAT'S BECAUSE ASSESSMENT CENTER IS A DATA COLLECTION TOOL, NOT AN INSTRUMENT DECISION AID. SO I WOULD SUGGEST IF YOU ARE NOT SURE WHAT TO USE, GO TO THESE REALLY RICH WEBSITES FOR PROMIS, FOR NEURO-QOL AND TOOLBOX TO LEARN MORE ABOUT THOSE MEASUREMENT INITIATIVES. I WILL PUT IN A PLUG AS WELL ON THE PROMIS WEBSITE, YOU COULD START FOLLOWING PROMIS ON TWITTER AND LEARN ABOUT NEW DEVELOPMENTS FROM OUR COLLEAGUES AT PROMIS NIH. AND THERE'S ALSO A NEWSLETTER YOU CAN SIGN UP FOR, WHICH IS A NICE WAY OH TO JUST BE KEPT UP TO DATE ABOUT WHAT'S HAPPENING WITHIN THAT INITIATIVE. THE OTHER WEBSITES ALSO HAVE REALLY GREAT INSTRUCTIONAL VIDEOS THAT I WOULD RECOMMEND WATCHING AS WELL. THERE'S ALSO A SERIES THAT KARON COOK DID EXPLAINING IRT AND CAT IF YOU WANT A REFRESHER OR YOU WANT TO SHARE WITH A COLLEAGUE MORE INFORMATION ABOUT THAT. SO LOTS OF ADDITIONAL TOOLS FOR SELF GUIDED LEARNING. SO I'D LIKE TO END A LITTLE EARLY AND JUST ACKNOWLEDGE THE FUNDING WE'VE HAD FOR THIS WORK. ONE OF THE REALLY FORTUNATE THINGS THAT HAPPENS WAS ALL THREE OF THESE INITIATIVES HAD P.I.s AT NORTHWESTERN UNIVERSITY, AND SO IT ALLOWS FOR A LOT OF SYNERGY AND FOR THE ABILITY FOR US TO HAVE THIS ONE DATA COLLECTION PLATFORM THAT CAN HOUSE ALL THREE, IT'S ALLOWED US TO HAVE LOTS OF OPPORTUNITIES TO DISCUSS DIFFERENCES BETWEEN MEASUREMENT SYSTEMS, WAYS TO UTILIZE THEM INDEPENDENTLY OR TOGETHER, SO THAT'S BEEN A REAL PLUS FOR THIS WORK. HERE AT THE NIH, THERE'S BEEN A GREAT TEAM OF FOLKS THAT HAVE HELPED MAKE AC-LITE HAPPEN AND WE'RE VERY GRATEFUL FOR THEIR ASSISTANCE. THEY'LL BE LEADING THE AFTERNOON PORTION ON LEARNING MORE ABOUT HANDS-ON HOW TO USE AC-LIGHT, AND THEN AT NORTHWESTERN UNIVERSITY, WE'VE GOT A REALLY GREAT ARE FOR MA TICS TEAM OF MANAGER, SOFTWARE DEVELOPERS, CUSTOMER SUPPORT FOLKS, SO I'D LIKE TO ACKNOWLEDGE ALL OF THEIR WORK AS WELL. SO I AM GOING TO WRAP UP HERE, AND WE WILL TURN IT OVER TO YANG, DO YOU HAVE ANYTHING YOU WOULD LIKE TO ADD? NO? OKAY. SO I COVERED EVERYTHING FOR YANG TOO. SO I THINK WHAT WE'LL DO NOW IS WE'LL HAVE THE PANEL COME UP AND DO Q & A. [APPLAUSE] >> SO IT'S NICE TO FINISH EARLY HERE, AND WE HAVE PLENTY OF TIME FOR QUESTIONS. I DON'T THINK WE'LL BE ABLE TO TAKE QUESTIONS FROM THE OUTSIDE, SO IF THOSE OF YOU THAT ARE VIEWING DO HAVE QUESTIONS, FEEL FREE TO SEND THEM TO MYSELF AND WE'LL GET THEM ADDRESSED BUT I THINK WE'RE JUST GOING TO BE ABLE TO TAKE QUESTIONS FROM THE AUDIENCE. SO FIRST ONE? >> MY NAME IS TED. I HAVE A VERY NAIVE QUESTION ABOUT SCALE, AND THIS QUESTION IS FOR DR. COOK, MAYBE RICHARD. SO DIFFERENT ITEMS HAVE DIFFERENT SCALE, LIKE YOU HAVE ZERO RANGE FROM 1 TO 10, 0 -- BUT FOR OTHER, LIKE FOR ANXIETY, FROM RICHARD'S SLIDE, YOU HAVE FIVE SCAIM AN SCALE AND DEPRESSION HAS FOUR. SO MY QUESTION IS -- STUDIES SHOW THAT MOST PEOPLE CAN TELL THE DIFFERENCE BETWEEN 1 TO 7, MAXIMALLY 9, THEN -- SO BASICALLY WHAT'S THE BASIC FOR THOSE NUMEROUS SKILLS IN ORDER TO MAKE SURE THEY'VE REACHED THE -- THEY ARE OPTIMIZED TO REACH THE BEST SENSITIVITY, SPECIFICITY. >> IT'S AN EXCELLENT QUESTION. EARLY ON, WHEN PROMIS WAS IDENTIFYING RESPONSE -- YOU'RE TALKING ABOUT THE RESPONSE SCALE. RESPONSE OPTIONS. WE HAD A GREAT DEAL OF DEBATE ABOUT THIS, ABOUT HOW MANY. PART OF WHAT WAS DISCUSSED IS IS THE RESEARCH THAT YOU MENTIONED THAT SHOWS THE PEOPLE TYPICALLY CAN ONLY DISCRIMINATE MAYBE FIVE TO SEVEN CATEGORIES. SO IN PROMIS, I BELIEVE MOST OF THE SKILLS ARE FIVE RESPONSE OPTIONS. WE ALSO TRIED WITHIN A DOMAIN AND ACTUALLY SOMEWHAT ACROSS DOMAINS TO MAKE THOSE -- TO LIMIT THE NUMBERS OF RESPONSZS, DIFFERENT RESPONSE -- IN TERMS OF THE SEMANTIC LABEL THAT'S -- TO EACH OF THOSE. THE ONLY TIME WHERE THERE'S ZERO TO 10, I THINK, IS WITH THE PAIN INTENSITY ITEM THAT'S USED IN THE GENERAL FORM. THAT'S THE ONLY TIME ZERO TO 10. I AGREE WITH YOU, IN FACT, DAVID AND I HAVE DONE SOME -- WE ACTUALLY HAVE A PUBLICATION ABOUT SHOWING THAT PEOPLE CAN'T DISCRIMINATE THAT MANY LEVELS. HOWEVER, ZERO TO 10 SCALE HAS GOT, AS YOU PROBABLY KNOW, WITHIN THE PAIN COMMUNITY, A HUGE HISTORY, AND WE WERE REALLY ENCOURAGED TO USE THAT FOR PAIN INTENSITY. NOW, THERE IS A GROUP WITHIN PROMIS THAT'S EXPERIMENTING WITH THE USE OF A SO-CALLED MINI BANK FOR PAIN INTENSITY THAT INCLUDES SEVERAL FIVE RESPONSE CATEGORY ITEMS. NOW IN TERMS OF THE MATHEMATICS OF IT, THE GREAT RESPONSE MODEL, KIND OF ITEM RESPONSE THEORY MODEL WE USE HANDLES DIFFERENT ITEMS WITH DIFFERENT CATEGORIES WITH NO PROBLEM, SO IT DOESN'T INTERFERE WITH THE MEASUREMENT MESH MEASUR EMENT PROPERTIES, WHICH IT WOULD IF IT WAS CLASSICALLY DONE, BUT IT HANDLES IT NICELY. >> NEXT QUESTION. >> I'M DR. SANTOS FROM KINGS COLLEGE, LONDON. WHAT I WANTED, TWO THINGS I WANT WANTED TO FIND OUT, DO YOU ACTUALLY THINK CHILDREN AGED FOUR HAS THE SAME CAPACITY TO LOOK AT THINGS IN TERMS OF DIE MENTION OF FIVE OPTIONS COMPARED TO AN 85-YEAR-OLD SIMPLY BECAUSE DEVELOPMENTALLY FOR CHILDREN AGED BELOW EIGHT, THEY CAN DISTINGUISH THREE OPTIONS EASILY EASILY. BUT ANYTHING MORE THAN THAT BECOMES DIFFICULT. THERE IS DEVELOPMENTAL SCIENCE BEHIND IT IN TERMS OF -- IT CAN ACTUALLY MANAGE. I MEAN, HOW IS THAT BEING TAKEN INTO ACCOUNT WHEN DEVELOPING IT, ESPECIALLY NOW THAT YOU'RE TRYING TO DEVELOP THE LINK BETWEEN THE CHILD AND THE ADULT MODEL? THAT'S ONE. THE SECOND ONE IS BASICALLY TO DO WITH CONFIDENTIALITY OF PATIENT INFORMATION SINCE THE SYSTEM SEEMS TO GENERATE AN EMAIL TO THE PATIENT ASKING THEM TO GO ONLINE AND THEN ANSWER QUESTIONS, HOW IS THAT BEING TAKEN INTO ACCOUNT? BY DEFINITION, THE EMAILS CAN BE AN IDENTIFIER THAT CAN TRACK WHO IS RESPONDING. >> SO I'LL PROBABLY NOT ANSWER THE PEDIATRIC QUESTION. I'M REPRESENTING THE PEDIATRIC FOLKS SO I PROBABLY WON'T DO AS GOOD A JOB AS THEY WOULD. I THINK YOU'RE RIGHT, AND I KNOW THAT IN TERMS OF A LOWER DEVELOPMENTAL LEVEL, PEOPLE DON'T DISCRIMINATE AS MANY CATEGORIES, AND I KNOW THAT -- I DON'T KNOW THE SPECIFICS ABOUT THE PEDIATRIC ITEM PARAMETERS, BUT I KNOW THAT FOR SOME PROMIS ITEMS, WHEN THERE IS NOT THAT LEVEL OF DISCRIMINATION, SOMETIMES THOSE CATEGORIES HAVE BEEN OH COLLAPSED S COLLAPSED SO THEY'RE S CORED AS THEY'RE REALLY THREE CATEGORIES INSTEAD OF FIVE OR FOUR CATEGORIES INSTEAD OF FIVE. THAT MAY BE HOW IT'S HANDLED. IF YOU EMAIL ME LATER, I'LL BE HAPPY TO FIND OUT A MORE SPECIFIC EXAMPLE. AS FAR AS SECURITY, I'M GOING TO TURN IT OVER TO MY COLLEAGUE, DR. GERSHON. >> I WILL EAPS THE FIRST QUESTION. BOTH PROMIS AND NIH TOOLBOX FOR SELF-REPORT DON'T RECOMMEND BELOW EIGHT FOR EMOTIONAL HEALTH ITEM. BOTH BANKS OFFER SELF-REPORT STARTING -- I'M SORRY, PROCTOR REPORTS STARTING YOUNGER THAN THAT -- >> FIVE. >> FIVE FOR PROMIS AND -- AND THREE FOR -- WITH NIH TOOLBOX, THERE'S A BRACKETED AGE SPOT IN THE MIDDLE THROUGH ABOUT AGE 12, A RECOMMENDATION YOU MAY WANT TO DO BOTH, THEN ABOVE THAT, CHILD SELF-REPORT. BUT THIS IS IN AGREEMENT, NONE OF THESE EFFORTS HAVE ATTEMPTED TO TACKLE SELF-REPORT IN KIDS YOUNGER THAN AGE 8. >> AS FOR SECURITY DATA, SO THE FIRST THING IS THE SYSTEM DOES NOT EMAIL A PARTICIPANT. THE SYSTEM WOULD ALLOW A RESEARCHER TO MAKE A DECISION ABOUT CONTACTING A PARTICIPANT. THE SECOND THING IS THE RESEARCHER HAS THE ABILITY TO INCLUDE PROTECTED HEALTH INFORMATION LIKE. MAIL ADDRESS OR NAME IN THEIR STUDY OR TO NOT INCLUDE IT, SO IF THERE ARE SECURITY CONCERNS, THEY COULD OPT TO NOT INCLUDE THAT I IN THE DATA COLLECTION PLATFORM. THEN ANOTHER THING WE DO IS RESEARCHER DOES HAVE THE ABILITY TO EMAIL A PARTICIPANT AND SAY YOU'RE DUE FOR AN ASSESSMENT, NAVIGATE TO THIS WEBSITE, BUT THE PARTICIPANT HAS TO LOG IN WITH THEIR OWN USER NAME AND PASSWORD IN ORDER TO GET IN TO COMPLETE THE DATA COLLECTION. SO IN THAT WAY, THE EMAIL ISN'T TIED WITH THE ACTUAL PATIENT-REPORTED DATA OUTCOME THAT'S BEING COLLECTED AT THE ASSESSMENT CENTER. >> THE REASON I'M ASKING, IN THE U.K., THIS WOULD BE A PROBLEM. BECAUSE THE FACT THAT THERE'S AN EMAIL BEING SEPTEMBER T SENT TO THE PATIEN T IS BY NATURE DE-IDENTIFIED -- >> IF THAT WAS A DEAL BREAK E I WOULD SAY DON'T EMAIL PATIENTS. IF YOU'RE INTERACTING WITH PATIENTS FACE TO FACE, YOU CAN PROVIDE THEM WITH THE URL, THEIR LOG IN AND PASSWORD AND HAVE THEM ACCESS IT INDEPENDENTLY WITHOUT AN EMAIL PROMPT FROM YOU. >> I SHOULD PUT OUT THERE, NUMEROUS INSTANCES, INDIVIDUAL INSTITUTIONS THAT I'VE INTEGRATED THE PROMIS INSTRUMENTS WITH ELECTRONIC HEALTH RECORD, VERY OFTEN THE EMAIL PROMPT IS YOU HAVE A MESSAGE IN YOUR -- YOU HAVE A COMMUNICATION THAT WE'D LIKE YOU TO IT ADDRESS AND BY THE WAY, IT MIGHT BE THAT THEIR PRESCRIPTION IS READY, CONFIRM THEIR APPOINTMENT, ET CETERA. ONCE THEY LOG IN -- >> AND THAT'S THE EXPERIENCE THAT WE HAD HERE. WE WITWE WERE HAVING CONCERNS ABOUT EMAILING, THE LINK IS QUITE LONG, SO WE ALSO GIVE THAT TO THEM IN PAPER FORMAT AND THE CODE, THEIR USER NAME AND ACCESS CODES ARE NEVER IN THE SAME CORRESPONDENCE, SO BASICALLY IT'S A LINK TO THE SURVEY, AND WE DID CHECK WITH OUR SAFETY OFFICE, OUR SECURITY OFFICE AND THEY ADVISED THAT JUST THE EMAIL WITH THE LINK WAS ACCEPTABLE BECAUSE IT DIDN'T HAVE PROTECTED HEALTH INFORMATION. BUT WE DO HAVE THE SECURE MESSAGING SYSTEMS AS WELL TO EMAIL OUR PATIENTS. >> THANK YOU. >> HI, ERIN KENT FROM THE NATIONAL CANCER INSTITUTE. I'M CURIOUS ABOUT THE -- WHEN YOU SHOWED THE EQUIVALENCY SCORES BETWEEN THE LEGACY MEASURES AND SOME OF THE PROMIS ITEMS, FOR EXAMPLE, IF NO DIFFERENCES WERE TAKEN INTO ACCOUNT, SO IF YOU'RE INTERESTED IN SORT OF SEEING WHAT AN HISTORICAL SURVEY HAS GIVEN YOU IN THE PAST, SAY, WITH THE SF36, HOW THAT WOULD MAP ON TO SOME OF THE PROMIS ITEMS, BUT MODE DIFFERENCES, PAPER AND PENCIL VERSUS CAT HAVE BEEN TAKEN INTO ACCOUNT. >> LOTS OF ANSWERS. PROMIS HAS FUNDED NUMEROUS STUDIES THAT LOOK AT MODE. EVEN MODE WITHIN PROMIS. BECAUSE WITHIN PROMIS, WE HAVE EVERYTHING FROM PAPER AND PENCIL FORMS FILLED OUT ON PAPER TO SHORT FORMS FILLED OUT ON COMPUTER VERSUS CAT. THE NICE THING WITH ALL MODE STUDIES THAT I'M AWARE OF, WHEN IT COMES TO SELF-REPORT, MODE ENDS UP BEING A NON-ISSUE EXCEPT WHEN YOU TAKE INTO CONSIDERATION A PERSON LIVE GIVING. SO A PLACE WHERE YOU HAVE A RESEARCH ASSISTANT ASKING QUESTIONS, THAT'S WHERE YOU GET INCREDIBLE VARIATION. REGARDLESS OF THE INSTRUMENT TYPE. BECAUSE THERE'S EITHER A DEMAND CHARACTERISTIC THERE OR NOT WANTING TO GIVE SOME KIND OF ANSWER IN FRONT OF PEOPLE, BUT ALL THE MODE-TIME STUDIES WITHIN PROMIS OR ACROSS PROMIS HAVE GENERALLY BEEN NEGATIVE. THEY DO -- EVEN FOR ALL KINDS OF COMPUTER-BASED TESTING, THE ONLY OPLACE THAT'S EVERY FOUND A DIFFERENCE IN LONG READING PASSAGES ON PAPER VERSUS COMPUTER, AND IT TURNS OUT YES, THERE'S A DIFFERENCE, IF YOU HAVE THE ENTIRE PASSAGE AVAILABLE TO YOU IN FRONT YOU HAVVERSUSJUST LOOKING AT A WINDOW, ALTHOUGH THAT'S RAPIDLY CHANGING BECAUSE MORE PEOPLE ARE PRIMARILY LEARNING HOW TO READ ON A SMALL SCREEN SO I THINK THOSE DIFFERENCES WILL DISAPPEAR OVER TIME. >> THE SPECIFIC PROMIS STUDY LOOKED AT MODE EFFECTS, AND THERE WAS A STUDY WITHIN PROMIS THAT LOOKED AT THIS -- LIKE A TABLET OR I THINK LOOKED AT LIKE A SMARTPHONE SCREEN AND LOOKED AT PAPER AND PENCIL FORM. THE MODE EFFECTS -- >> HI. SO THIS QUESTION IS DIRECTED AT EITHER RICHARD OR NAN OR ANYONE ELSE WHO MIGHT BE ABLE TO ANSWER IT ACTUALLY. THERE ARE HUNDREDS IF NOT THOUSANDS OF WHAT WE USED TO CALL HRQL, HEALTH RELATED QUALITY OF LIFE, NOW CALLED PATIENT-REPORTED OUTCOMES TOOLS OUT THERE, AND EVERYBODY HAS A FAVORITE THAT'S BEEN INVOLVED IN CLINICAL RESEARCH. AND THAT HAS ACTUALLY DRIVEN THE EXPANSION OF THE NUMBER OF TOOLS OVER THE YEARS. THE QUESTION IS, I'M NOT EXACTLY CLEAR BUT IT SOUNDED LIKE IF YOU HAD A FAVORITE TOOL AND YOU DON'T WANT TO USE ONE OF THE TOOLS THAT -- YOU DON'T WANT TO USE ONE OF THE SCALES THAT'S INCLUDED IN THESE TOOLBOXES, THAT ONE COULD USE THE TOOL THAT ONE LIKES. IF I LIKE THE EROTC OR THE FACT OR WHATEVER, CANCER, A NUMBER OF AIDS-SPECIFIC TOOLS, SO FORTH, THAT I COULD USE THAT TOOL IF I DIDN'T WANT TO USE THE QUESTIONS THAT GET AT THOSE ISSUES THAT ARE AVAILABLE TO ME IN THIS -- IN THESE TOOLBOXES. IS THAT CORRECT? AND WHAT MIGHT BE DONE TO SORT OF GET THE WORD OUT TO CLINICAL RESEARCHERS, PARTICULARLY THOSE IN CLINICAL MEDICINE THAT MAY NOT BE AWARE OF THIS PROJECT OR MAY NOT BE AS FAMILIAR WITH IT OR MAY NOT UNDERSTAND STA IT TIS TICS ANSTATISTICSAN JUST WANTS THE TOOL TH EY'VE ALWAYS BEEN USING TO CAPTURE THAT KIND OF INFORMATION? >> SO I'LL START AND OTHERS CAN CHIME IN. THE FIRST PART IS YES, YOU CAN USE AN EXISTING LEGACY TOOL AND PROSETTA STONE ARE LINKING TABLES WITH SOME WIDELY USED LEGACY TOOLS WITH THE PROMIS METRIC. I THINK PART OF THE RATIONALE IS WE'RE THINKING THAT WE'RE IN A TRANSITION PERIOD. THE FIRST PROMIS INSTRUMENTS CAME OUT IN 2007. WE'RE RIGHT ON THE CUSP OF ANOTHER HUGE WAVE OF ADDITIONAL PROMIS INSTRUMENTS BEING RELEASED, AND SO THEY DON'T HAVE THAT VERY RICH LEGACY OF LOTS OF PUBLICATIONS DEMONSTRATING THEIR USEFULNESS IN A VARIETY OF CLINICAL POPULATIONS. >> THESE ARE ACTUAL INSTRUMENTS, THOUGH, THEY'RE SCALES THAT GET PARTICULAR CONCEPTS OR FACETS OR THAT SORT OF THING. THEY'RE NOT ACTUAL -- YOU DON'T HAVE AN HIV-SPECIFIC TOOL. >> ARE YOU ASKING RELATIVE TO ASSESSMENT CENTERS, PUTTING IT IN THERE? >> I'M ASKING IF I HAVE A SPECIFIC AREA THAT I WANT TO MEASURE AS A CLINICIAN TO TRACK THE PROGRESS OF MY PATIENT OVER TIME, WHETHER IN CLINICAL PRACTICE OR IN RESEARCH, THAT SOME OF THESE CONCEPTS ARE LIKE, I'M SURE, ARE COVERED WITHIN PROMIS OR ANY OF THE OTHER TOOLBOXES, BUT I MAY NOT KNOW THAT AS SOMEONE WHO HAS BEEN DOING RESEARCH IN THE FIELD FOR A NUMBER OF YEARS. THIS IS NEW INFORMATION. SO HOW IS THAT -- IS THERE SOMETHING THAT IS IN THE ASSESSMENT MECHANISM THAT YOU'VE JUST DESCRIBED TODAY THAT SAYS FYI, THERE'S A NEW WAY OF GOING ABOUT DOING THIS, ALTHOUGH IT'S A DATA CAPTURE INSTRUMENT SO IT MIGHT NOT BE THERE, BUT THERE MIGHT BE A WAY TO SORT OF ALERT PEOPLE THAT WANT CONTINUE AND HAVE BEEN DOING RESEARCH IN PARTICULAR, THEY DON'T NECESSARILY KNOW WHAT'S IN THE TOOL BUT THEY KNOW THEY LIKE TO USE THIS TOOL AND IT HELPS THEM TRACK OF PROGRESS OF THEIR PATIENT ON THEIR THERAPY AND SO FORTH. SO I'M ASKING A COUPLE QUESTIONS BUT HOW WOULD I AS A CLINICIAN KNOW THERE'S A NEW WAY TO GO ABOUT DOING THIS? HOW IS THAT MESSAGE GETTING OUT THERE? AND IF I DON'T WANT TO DO THAT, CAN I STILL USE MY LEGACY TOOL, HOW DO I KNOW IF THAT TOOL IS ONE OF THOSE TOOLS THAT'S BEEN MAPPED TO -- INTO PROMIS OR ONE OF THE OTHER TOOLBOXES? >> SO I THINK WHAT YOU'RE TALKING ABOUT IS OUTREACH, HOW CAN WE LET THE CLINICAL AND RESEARCH WORLD ABOUT THESE TOOLS. WE'RE DOING THAT THROUGH EVENTS LIKE THIS. THERE IS GRANTS THAT WILL BE FUNDED, WE HOPE REALLY SOON, THAT AIMS TO DO MORE OF THIS OUTREACH WORK TO IDENTIFY WHAT ARE THE TARGET CLINICAL AREAS, HOW CAN WE GET INFORMATION TO CLINICIANS AND CLINICAL RESEARCHERS ABOUT THESE NEW TOOLS AND HOW THEY COMPARE TO LEGACY TOOLS AND SHOW THEM DATA ABOUT WHY THEY WOULD WANT TO SWITCH. SO DATA LIKE WHAT RICHARD PRESENTED SHOWING YOU'RE DOING A CLINICAL TRIAL, YOU'RE NOT GOING TO CAPTURE CHANGE IF YOU CONTINUE TO USE THE SF36. I THINK IF WE CAN DEMONSTRATE THAT, IT WILL HELP. WE HAVE LOTS OF OUTREACH EFFORTS WITHIN THE PROMIS NETWORK. I THINK JIM WITTER HAS BEEN CHAMPIONING OUTREACH WITH PROMIS AT CLINICAL MEETINGS, IN WEBCASTS AND ALL KINDS OF THINGS, SO I THINK WE'RE TRYING TO DO THAT, BUT IT'S A HUGE AUDIENCE AND THERE'S A LOT OF INERTIA WITH MEASUREMENT SELECTION. >> I SHOULD POINT OUT THE INSTITUTE OF MEDICINE HAS AN ACTIVE EFFORT RIGHT NOW TO IDENTIFY INSTRUMENTS THAT WOULD BE AVAILABLE ACROSS SYSTEMS, PCORI HAS NUMEROUS EFFORTS IN THAT REGARD AS WELL. EPIC IS LOOKING FOR SOMEONE, PROMIS NETWORK, IDENTIFY WHICH INSTRUMENTS BY DISEASE SHOULD BE RECOMMENDED TO PEOPLE, THEY HAVE A DEFAULT INSTRUMENT AREA THAT IT'S ACTUALLY FULL CLINICAL MANAGEMENT AREA THAT -- I FORGET WHAT THEY CALL IT, BUT WITH DISEASE X, DIAGNOSIS OF X, HERE'S WHAT WE'D RECOMMEND FOR CLINICAL WORK FLOW, HERE IS INSTRUMENTS WE'D LIKE TO SUGGEST. I THINK WE'RE REALLY AT THE EARLY SIDE. IT'S CLEARLY A CRITICAL OPPORTUNITY AND I THINK AS WE MENTIONED A COUPLE DIFFERENT WAYS, WITHIN ASSESSMENT CENTER, IF YOU HAVE A SCALE OR ANOTHER INSTRUMENT, THERE ARE WAYS OF ADDING IT IN SO THE PATIENT DOESN'T PERCEIVE THAT THEY'RE BOUNCING BETWEEN SYSTEMS TO BE ABLE TO ANSWER SURVEYS. >> RIGHT, IT'S JUST SOMETHING TO THINK ABOUT AS THIS EVOLUTION TAKES PLACE. IT'S HARD ENOUGH GETTING INSIDE INTO THE FIELD, THE FAULT OF LIFE FIELD HAS BEEN RAPIDLY CHANGING EVER SINCE IT WAS FIRST NAMED, SO IT'S A MOVING TARGET AND TO THE EXTENT THAT THIS KIND OF OUTREACH AND MEETINGS LIKE THIS TAKE PLACE, I THINK THAT'S TO THE BEN P FIT OF OF ALL PEOPLE WHO TREAT AND RESEARCH PATIENTS. >> IF I COULD JUST ADD, ONE OF THE HATS THAT I WEAR AS CHAIR OF THE OUTREACH COMMITTEE, WE DISCUSS THIS ALL THE TIME, HOW DO WE GET THE MESSAGE OUT, WE TRIED ALL AVAILABLE VENUES AS YOU MIGHT IMAGINE, QUITTER CHATS, NEWSLETTER, EMAIL BLAST OF OVER 4,000 RECIPIENTS. WE HAVE VENUES LIKE THIS. WE DO -- AND THE NETWORK IS AS YOU SAW IS LARGE, SO ALL OF US HERE, PEOPLE THAT YOU DON'T SEE, WE REGULARLY DO SORT OF OUTREACH AS YOU'RE SAYING TO GET THE WORD OUT BECAUSE I THINK WHAT'S GOING TO HAPPEN GIVEN TIME IS WHEN THESE MEASURES START TO BE USED, ESPECIALLY IN CLINICAL SETTINGS AND RESEARCH SETTINGS, PEOPLE WILL START TO SEE HOW THEY COMPARE WITH WHAT THEY'RE FAMILIAR WITH, AND THEY'LL GO, O THIS IS NOT ONLY AS GOOD AS, HOPEFULLY BETTER THAN WHAT IT IS FOR REASONS WE'VE HEARD TODAY. THOSE SORTS OF THINGS. SO IT'S A MAJOR ER EFFORT THAT WE'RE TRYING TO DO. MY REQUEST TO YOU WOULD BE TO HELP US DO THAT. >> RIGHT. I THINK DOING THESE KINDS OF FORUMS AT THE LARGE CLINICAL ASSOCIATION MEETINGS, NEUROLOGY, CANCER, AIDS, I THINK THAT WOULD BE VERY HELPFUL BECAUSE YOU'RE PROBABLY DOING THAT. >AND PCORI HAS PICKED THIS UP AS WELL. THERE IS CURRENTLY A PFA SPECIFICALLY DEVOTED TO PROMIS. AND THAT'S A -- WHAT WE HOPE IS AN INITIAL INTEREST ON THEIR PART, WE'RE HOPING THEY PUT OUT AN RFI AT THE SAME TIME TO REALLY GLEAN FROM THE COMMUNITY WHAT SORTS OF WAYS CAN THEY HELP TO FACILITATE PROMIS IN ALL SORTS OF POPULATIONS, VENUES, THAT WE HOP WILL B HOPE TO BE LEADING TO A MORE SUBSTANTIAL EFFORT ON THEIR PART TO FUND PROMIS RELATED RESEARCH AND USE AND PARTICULARLY IN SORT OF CER HAD CER-TYPESETTINGS. SO THE FUTURE LOOKS VERY BRIGHT AND VERY PROMISING, NO PUN INTENDED. >> THIS STUFF NEEDS TO BE TAUGHT IN MEDICAL SCHOOL, BECAUSE IT ISN'T. >> I'M JOSH. I WAS JUST WONDERING WHETHER THERE ARE ANY EFFORTS TO DEVELOP TOOLS TO HELP P BEHAVIORS SUCH AS EXERCISE OR DIET, WHICH IS ANOTHER AREA OF -- >> THERE'S AN EFFORT IN PEDS FOR A MEASURE TO SELF-REPORT PHYSICAL ACTIVITIES. WITHIN ADULTS, THERE IS A MEASURE OF SMOKING AS WELL AS ALCOHOL USE, BUT OTHERWISE I DON'T BELIEVE THERE'S ANYTHING HAPPENING IN HEALTH BEHAVIORS. >> DID WE HAVE A QUESTION BEFORE ABOUT PASTOR? THAT'S BEEN ADDRESSED ALREADY? OKAY. GREAT. JUST TO REMIND YOU, WE DON'T HAVE CAPABILITIES FOR QUESTIONS FROM THE OUTSIDE, BUT IF YOU DO, THOSE THAT ARE WATCHING, FEEL FREE TO EMAIL ME. MY EMAIL ADDRESS IS JAMES.WITTER@NIH.GOV, AND I WILL MAKE SURE THAT WE GET YOUR QUESTION ANSWERED. ONE OF THE THINGS, AND I DO HAVE A QUESTION FOR YANG. I'D JUST LIKE TO POINT OUT ONE OF MY HOPES IS THAT THIS WILL BE THE BEGINNING OF FACILITATING A COLLABORATIVE SORTS OF RESEARCH BETWEEN INTRAMURAL AND EXTRAMURAL RESEARCHERS LIKE WE'VE BEEN DOING IN OTHER VENUES, BUT I HAVE A QUESTION FOR YANG. WHEN MIGHT WE EXPECT TO SEE THESE TOOLS AVAILABLE IN -- >> SO WE NEED TO HAVE MORE RESEARCHERS JOIN INTO THE START OF USING AC-LITE IN CONJUNCTION WITH WORKING WITH THE --. THE DATA IS EXPORTABLE, AND WE HAVE A STUDY THAT WE HAVE IDENTIFIED, SO THAT CANNIN CAN BE DONE -- TO MAKE IT WORTHWHILE, BECAUSE JUST -- I ASKED TO SPEND HER EFFORTS DOING ALL THIS DATA LINKAGE -- SO WE'LL -- THIS DATA TO THE -- THAT WILL BE MUCH EASIER. THE -- ON THE OTHER HAND IS A LITTLE TRICKIER, BECAUSE YOU HAD TO PROBABLY USE AN API DIAGNOSE -- AND CHRISTINE HAS OVERWHELMED TASK AND PROJECT TO BE ABLE TO IMPLEMENT OUR -- BUT IT'S A POSSIBILITY, I CAN SEE -- CLINICAL CENTER -- >> OKAY. IF THERE'S NO FURTHER QUESTIONS, JUST TO REMIND YOU THEN THAT THERE WILL BE MORE EXTENSIVE HANDS-ON TRAINING THIS AFTERNOON, IT'S GOING TO BE IN BUILDING 12A, SPECIFICALLY IN ROOM B51. THAT STARTS WITH REGISTRATION AT 1:00 AND IT GOES UNTIL ABOUT 4:30, SO THOSE OF YOU THAT ARE REALLY GOING TO BE DOING SORT OF THE HANDS-ON, IT'S REALLY EXCELLENT TRAINING, I THINK YOU'RE GOING TO IT LEARN A LOT. I THINK WE'RE SIGNED UP ALREADY, BUT IF YOU HAVEN'T SIGNED UP, FEEL FREE TO CHECK IN AND I THINK THERE MIGHT BE SOME SPACES. WE'LL SEE. BUT THANK YOU VERY MUCH FOR ATTENDING, AND ENJOY THE REST OF YOUR DAY. [APPLAUSE]