THE OFFICE OF BEHAVIOR AND SOCIAL SCIENCES RESEARCH IS PLEASED TO HOST THE BEHAVIORAL AND SOCIAL SCIENCE LECTURE SERIES. TODAY WE'RE VERY HAPPY TO HAVE DR. HOLLY JIMISON WITH US TO DELIVER HER LECTURE, NEW CHALLENGES FOR BIG DATA, HOW MONITORING BEHAVIORS IN THE HOME AND ENVIRONMENT CAN DRIVE THE DISCOVERY OF IMPORTANT BEHAVIORAL MARKERS AND INFLUENCE CARE. DR. JIMISON RECEIVED HER DOCTORATE IN MEDICAL INFORMATION SCIENCES AT STANFORD UNIVERSITY. SHE'S CURRENTLY AN ASSOCIATE PROFESSOR OF MEDICAL IN FOE MATTICS AND CLINICAL EPIDEMIOLOGY AT OREGON HEALTH AND SCIENCE UNIVERSITY, A FELLOW OF THE AMERICAN COLLEGE OF MEDICAL INFOMATICS, A PAST PRESIDENT OF THE ORIGIN CHAPTER AND SERVES ON THE COUNCIL FOR OREGON CENTER FOR AGING AND TECHNOLOGY. SHE'S SERVED AT HEALTH SCIENCES SINCE JANUARY 2012. DR. JIMISON HAS BUILT ACADEMIC AND INDUSTRY EXPERIENCE IN THE DESIGN AND EVALUATION OF MEDICAL TECHNOLOGIES, INCLUDING THE AUTOMATED ANALYSIS OF AMBULATORY CONTROL SIGNALS AT THE ICU. SHE FOUNDED AT THE INFORMED PATIENT DECISIONS GROUP WHICH FOCUSES ON THE USE OF COMPUTER TOOLS TO EMPOWER PATIENTS TO BE MORE ACTIVE AND INFORM PARTICIPANTS IN THEIR MEDICAL CARE. DR. JIMISON'S CURRENT RESEARCH WORKS TO DEVELOP NEW TECHNIQUES FOR USING CENTERS TO MONITOR HEALTH BEHAVIORS OF HOLDER ADULTS IN THEIR HOMES. AND THEN DELIVERING HEALTH COACHING INTERVENTIONS REMOTELY BY COACHING FAMILY MEMBERS AND INFORMAL CARE GIVERS. THE ULTIMATE GOAL OF THIS PROJECT IS TO DEMONSTRATE THE FEASIBILITY OF A SCALABLE APPROACH TO DELIVERING HEALTH INTERVENTIONS SUCH AS COGNITIVE EXERCISE, SLEEP MANAGEMENT AND SOCIALIZATION IN THE HOME. AT BSSR DR. JIMSONS USES THIS INITIATIVE IN BIG DATA FOR BEHAVIORAL AND SOCIAL SCIENCES WITH THE AIM OF DEVELOPING A PROCESS FOR MULTIDISCIPLINARY TEAMS OF RESEARCHERS TO WORK TOGETHER TO DISCOVER NEW CLINICAL BEHAVIORAL METRICS THAT WOULD ENABLE EARLY DETECTION OF HEALTH PROBLEMS AND PROVIDE FEEDBACK FOR TAILORED HEALTH INTERVENTIONS. WITHOUT FURTHER ADIEU, PLEASE JOIN ME IN WELCOMING DR. JIMISON. [APPLAUSE] >> THANKS. I'M REALLY HAPPY TO BE HERE AND APPRECIATE ALL OF YOU COMING OUT WITH OUR HALF INCH OF SNOW. I KNOW PEOPLE HAVE HAD A SNOW DAY AND MY RELATIVES DON'T QUITE APPRECIATE, THEY'VE JUST HAD MINUS 54 DEGREE WEATHER AND I'M NERVOUS ABOUT THEM BUT ANYWAY, WE'LL GET GOING. THERE'S A CONTEXT FOR THIS TALK IN THAT BIG DATA IS VERY FASHIONABLE RIGHT NOW. EVERYBODY'S DOING IT, THE FEDERAL LAST MARCH WHEN THE FEDERAL GOVERNMENT BASICALLY PUT A LOT OF MONEY EFFORT INTO THIS AND BIG PLAN IN ALMOST EVERY AGENCY ACROSS THE BOARD ESPECIALLY NIH. I COUNTED ABOUT 25 DIFFERENT EFFORTS HERE. ALSO MOST COMPANIES ARE USING BIG DATA TECHNIQUES IN ANALYZING THEIR PURCHASING DATA, WHAT THE CUSTER WANTS, CUSTOMER PURCHASES, INTERACTIONS ON COMPUTERS. AND THEN IN ADDITION, PEOPLE ARE DEVELOPING PURCHASES AND INTUITION BASED ON THEIR PARTICULAR EXPERIENCE. AND HERE AT NIH, MOST OF THE WORK AND BIG DATA IS GOING ON IN GENOMIC, PROTOOH PICKS AND MOLECULAR BIOLOGY. IN BUSINESS IT'S MORE ABOUT WEB CLICK SEARCH TERMS, PURCHASING DATA. AND WHAT I WANT TO FOCUS ON TODAY HOW OUR WORK ESPECIALLY AT OBSSR IS SUBSTANTIALLY DIFFERENT AND CAN LEND NEW APPROACHES TO LOOKING AT BIG DATA BECAUSE WE HAVE VERY CHALLENGING PROBLEMS IN THE AREA OF MONITORING BEHAVIORS IN THE HOME AND ENVIRONMENT. THIS MEANS WE NEED TO BROADEN OUR RANGE OF TECHNIQUE AT EVERY LEVEL OF ANALYSIS FROM THE SAMPLING TO THE INFERENCE OF PATIENT ACTIVITY. THIS IS ALL AGAINST A BACK DROP OF CHANGING HEALTHCARE LANDSCAPE. WE'RE MOVING FROM LOOKING AT EPISODIC REACTIONS TO PROACTIVE AND PREVENTIVE APPROACHES THAT FOCUS MORE ON WELL BEING AND DISEASE. IT MOVES AWAY FROM SOLELY HOSPITAL AND CLINIC CARE TOWARDS PATIENT CENTERED CARE IN THE HOME. IN ADDITION, MEDICAL TRAINING HAS BEEN EXPERIENCE BASED, AND WITH TECHNOLOGY AND THE LARGE NUMBER OF CLINICAL TRIALS SUPPORTED BY NIH, WE NOW HAVE MORE EVIDENCE BASED AND DECISION SUPPORT TOOLS TO HELP INFORM CLINICAL PRACTICE, DATA SYSTEMS ARE IMPROVING. WE'RE MORE LIKELY TO HAVE INOPERABLE DATA AVAILABLE ON PATIENTS ANYWHERE ANY TIME. AND ALSO, FROM THE PATIENT'S POINT OF VIEW, PEOPLE ARE BECOMING MORE EMPOWERED. MORE PROACTIVE AND INFORMED PARTICIPANTS IN THEIR OWN MEDICAL CARE. SO THIS MEANS THERE'S MUCH MORE OF AN EMPHASIS IN THE FUTURE ON HEALTH INTERVENTIONS THAT WILL TAKE PLACE IN THE HOME. WE KNOW THAT MOST OF OUR MONEY, HEALTHCARE DOLLARS ARE BEING SPENT ON CHRONIC ILLNESSES AND CONDITION ASSOCIATED WITH AGING. THESE ISSUES REALLY REQUIRE A FOCUS ON HEALTH MANAGEMENT IN THE HOME, NOT JUST WHAT HAPPENS IN THE CLINIC IN AN OFFICE VISIT. ALSO, MOST IMPORTANT LIFE-STYLE ISSUES LIKE OBESITY, PHYSICAL EXERCISE AND SMOKING CONTRIBUTES TO FUTURE HEALTHCARE PROBLEMS. AGAIN THESE AND MOST EFFECTIVELY TREATED AT THE POINT OF CARE OUTSIDE OF THE CLINIC VISIT. IT CAN HELP TO GIVE A PRESCRIPTION FOR SOME SMOKING, MEDICATIONS THAT WILL HELP QUIT SMOKING. BUT MOST OF THE BEHAVIORS THAT WE'RE INTERESTED IN CAN BENEFIT FROM REAL TIME MONITORING JUST IN TIME INTERVENTION, SOCIAL SUPPORT, COACHING, THINGS THAT ARE NOT NORMALLY ASSOCIATED WITH WHAT HAPPENS IN THE CLINIC VISIT. NOW TECHNOLOGY, FOR THIS TYPE OF MONITORING, BOTH FOR SENSES, COMMUNICATIONS, DECISION SUPPORT, ALL ENABLE TAILORED INTERVENTIONS FOR THE HOME. AND THESE WE KNOW ARE MUCH MORE EFFECTIVE THAN GENERIC MATERIALS. NOW THIS KIND OF SENSING, THOUGH, DOES LEAD TO BIG DATA. AND THE OPPORTUNITIES THAT WE HAVE IN THIS AREA ARE THAT WE NEED THE LOW COST APPROACHES IN GENERAL FOR MEDICAL CARE. THIS KIND OF TECHNOLOGY CAN HELP UNDERSTANDING AND IMPROVING HEALTH BEHAVIOR CHANGE ALSO BECOMES CREDIT CREE IMPORTANT. THIS REQUIRES NEW METHODS OF CONTINUOUS MONITORING AND A MEANINGFUL INTERPRETATION OF BEHAVIORS. THE OPPORTUNITIES FOR RESEARCH IN THIS AREA ARE GREAT. WE DO HAVE THE POTENTIAL NOW TO COLLECT MASSIVE AMOUNTS OF DATA FROM A VARIETY OF SOURCES. AND THE UNOBTRUSIVE AND CONTINUOUS MONITORING BEHAVIORS IS LARGELY UNEXPLORED AND VERY RIPE FOR NEW DISCOVERIES. SO MY POINT HERE, AND I'LL SAY RIGHT FOR DISCOVERY PROBABLY FIVE TIMES THROUGHOUT THE TALK, IT'S A WEALTH OF NEW INFORMATION THAT CAN REALLY HAVE THE POTENTIAL TO INFLUENCE CARE IN GREAT WAYS. HOWEVER, THERE ARE CHALLENGES. WE'VE GOT NOW WITH THIS TYPE OF SENSING, A VARIETY OF LARGE DIVERSE COMPLEX TYPES OF DATA THAT ARE COLLECTED OVER TIME, CONTINUOUSLY. SOME ARE SAMPLED MORE FREQUENTLY THAN OTHERS. IT'S NOT OBVIOUS HOW TO SAMPLE MOST INTELLIGENTLY, HOW TO FILTER, HOW TO GET RID OF THE NOISE, HOW TO INTERPRET BIASES, HOW TO MODEL. THESE DATA SETS ARE IMPORTANT AND THE ANALYSIS IS IMPORTANT IN REAL TIME SO THAT YOU CAN INTERVENE IN REAL TIME TO IMPROVE HEALTH BEHAVIOR CHANGE. NOW, FOR SCALABLE SCALABLE APPROACHES WE ARE USING LOW COST UNOBTRUSIVE SENSORS. NOW THESE ARE FRAUGHT WITH NOISE, POTENTIAL CONTEXT BIASES AND ALL THIS REQUIRES ALGORITHMS. THE SMARTS IS NOT NECESSARILY GOING TO BE IN THE DESIGN OF THE CENTERS BUT RATHER IN THE ALGORITHMS THAT ARE USED TO INTERPRET THE DATA. AND THAT CAN BE VERY SCALABLE. BUT REQUIRES ADVANCES IN MACHINE LEARNING, DATA MINING. FUSION, WHAT I MEAN BY FUSION ALGORITHMS IS HOW DO YOU COMBINE DATA FROM DIFFERENT SENSORS. THE DATA SOURCES MAY HAVE DIFFERENT NOISES ASSOCIATED WITH THEM, DIFFERENT TYPES OF RELIABILITY UNDER DIFFERENT CONDITIONS. AND IT'S QUITE OFTEN YOU JUST DON'T WANT TO AVERAGE DATA FROM SENSORS BUT HAVE AN INTELLIGENT ALGORITHM FOR FUSION. SO QUITE A BIT OF RESEARCH IS IMPORTANT IN THAT AREA AS WELL, AS WELL AS MODELING AND DATA VISUALIZATION. WE'LL GET TO SOME EXAMPLES SOON. JUST A QUICK LOOK AS A REMINDER OF WHAT BIG BUSINESS IS DOING THAT WILL BE SOMEWHAT DIFFERENT FROM THE APPROACHES FOR MONITORING BEHAVIORS. BUT THERE IS QUITE A BIT OF OVERLAP. AS SHOWN IN THE CARTOON ON THE RIGHT, BUSINESSES WILL TAKE DATA FROM DIFFERENT SOURCES AND MAKE AN INTERPRETATION AS THEY TAILOR THEIR INTERACTION WITH A CUSTOMER, IN THIS CASE THE PASSPORT OFFICIAL SAYING HE'S ONLY 23.5% WELCOME HERE BASED ON HIS ENZYME PURCHASES AND LOCATION HISTORY. SO COMPANIES AND MANY BUSINESSES ARE LOOKING AT GPS PURCHASING BEHAVIORS FOR SOCIAL NETWORKS OF HUGE AMOUNT OF DATA, FOR EXAMPLE, FACEBOOK WITH MORE THAN 40 BILLION PHOTOS. AND IT'S EASY FOR YAHOO AND GOOGLE TO LOOK AT YOUR SEARCHING BEHAVIORS. AS AN EXAMPLE, YOU'RE PROBABLY FAMILIAR WITH THIS. WE'VE SEEN THAT GOOGLE STEPS FROM SEARCHES ON FLU-RELATED SEARCH TERMS, COMPARING TO IN YELLOW, CDC'S REPORT AND ANALYSIS FOR FLU PROJECTIONS. QUITE OFTEN GOOGLE CAN ANTICIPATE EVEN BEFORE CDC. THIS IS WITH VERY SIMPLE APPROACHES. SIMILARLY, DATA FROM, COLLECTED FROM PHARMACIES OR ANY STORES THAT ARE SELLING OVER-THE-COUNTER MEDICATIONS CAN ALSO PREDICT FLU EPIDEMICS, FLU OUTBREAKS. OF COURSE THEY ARE SEASONAL, BUT THIS IS JUST AN EXAMPLE OF HOW A SINGLE TYPE OF DATA CAN BE VERY HELPFUL IN A HEALTHCARE CONTEXT. NOW, WHAT WE'VE SEEN FOR HOME MONITORING IN THE PAST IS MAINLY BEEN AROUND ISSUES OF DISEASE MANAGEMENT. THE TYPICAL CONDITIONS WHERE DISEASE MANAGEMENT WHERE ONE USES IN THE HOME, DIABETES, AS MA, CARDIAC FAILURE. THERE'S WEIGHT WATCHERS, WEIGHT MANAGEMENT, SMOKING CESSATION. THEY'RE ALREADY OUT THERE GROWING NOT NECESSARILY THROUGH EFFORTS OF THE HEALTHCARE COMMUNITY BUT BY COMPANIES. IN SOME CASES HEALTH INSURANCE GROUPS WILL PROMOTE CERTAIN INTERVENTIONS LIKE THIS THAT ARE WEB BASED OR PHONE BASE. IN OTHER AREAS FOR SAFETY OF OLDER ADULTS WHERE YOU PUT SERPSES -- SENSORS IN THE HOME FOR MONITORING FOR FALL DETECTION OR ACTIVITIES IN THE HOME. NOW THERE'S A WIDE VARIETY OF THING THAT CAN BE MEASURED AS PEOPLE GO THROUGH THEIR DALLY LIVES. THE REAL TRICK IS TO INTERPRET THE DATA, FUSE IT, UNDERSTAND HOW IT RELATES TO CARE AND INTERACT IN REAL TIME HOPEFULLY WITH DECISIONS SUPPORT. THERE'S ALSO THE OPPORTUNITY TO LOOK AT POPULATION STATISTICS AND EPIDEMIOLOGY. AND I'M GOING TO DO RIGHT NOW THOUGH IS TO GIVE SOME EXAMPLES FROM WORK THAT HAS BEEN GOING ON FOR THE PAST, OH, NEARLY EIGHT YEARS IN OREGON HEALTH AND SCIENCE UNIVERSITY AND UNDER A LARGER UMBRELLA UNDER THE OREGON CENTER FOR AGING AND TECHNOLOGY WHERE WE FOCUS ON HOME MONITORING APPROACHES FOR OLDER ADULTS TO ENABLE THEM TO MAINTAIN THEIR FUNCTIONALITY AND INDEPENDENCE AND BE ABLE TO AGE IN PLACE. I'LL BE DESCRIBING WORK FROM BOTH JUST MONITORING AND ALSO FROM MY PROJECT IN HEALTH COACHING, REMOTE HEALTH COACHING WHERE WE'RE LOOKING AT ACTIVITY MONITORING IN THE HOME, COGNITIVE MONITORING, MOTOR SPEED, SLEEP, SOCIALIZATION, PHYSICAL EXERCISE, MEDICATION MANAGEMENT AND MEASURES OF INTERVENTIONS FOR DEPRESSION. NOW THIS IS A LARGE PROJECT AND HAS RANGED OVER THE YEARS FROM THE ALZHEIMER'S ASSOCIATION IN FUNDING PROJECTS IN EVERY DAY TECHNOLOGY THROUGH ALZHEIMER'S CARE. NATIONAL INSTITUTE ON AGING HAS SUPPORTED ORCATECH IN GENERAL WHICH IS A CONSORTIUM OF INSTITUTIONS, SURORGANIZE SERVICE ORIONS. WE HAVE A COMMON PLOT FORM AND TRY TO FACILITATE RESEARCHING THIS AREA BY GIVING RESEARCH GROUPS ACCESS TO A LARGE COLLECTION OF PATIENTS. GENERALLY OUR HOMES FOR OLDER ADULTS ARE SET UP WITH MOTION SENSORS IN THE HOME AS YOU'LL SEE ON THE UPPER LEFT. WE RESTRICT THE FIELD OF VIEW ON THE SECOND MOTION SENSOR AND TOOK THREE IN A ROW WHENEVER THERE'S A HALLWAY AND WE CAN MEASURE WALKING SPEED. WALKING SPEED IS A GREAT INDICATOR OF EARLY COGNITIVE DECLINE. AS YOU'LL SEE MOTOR SPEED IN GENERAL IS AN IMPORTANT INDICATOR AND IS OFTEN UNDER APPRECIATED. EACH SUBJECT HAS A COMPUTER IN THE HOME. WE HAVE A SWEET OF NINE COMPUTER GAMES THAT WE USE FOR COGNITIVE MONITORING AND INTERVENTION. IN SOME CASES WE'LL HAVE, NOW FOR THE SENSORS I'VE MENTIONED SO FAR, THEY'RE IN THE HOMES OF ABOUT 200 OLDER ADULTS LIVING INDEPENDENTLY. AND THEN THERE'S A SUBSET IN WHAT WE'RE CALLING A VIRTUAL LIVING LAB. PEOPLE THAT ARE PRECONCEPT TO TRY OUT NEW TECHNOLOGIES LIKE THE IMMEDIATE INDICATION DISPENSER IN THE LOWER RIGHT WHICH IS BLUETOOTH NEIGHBORHOOD -- ENABLED AND WE USE OUR CONTEXT ALGORITHMS TO PROVIDE REMINDING UNDERSTANDING WHERE THEY ARE IN THE HOME, WHETHER THEY'RE SLEEPING, WHETHER THEY'RE ON THE PHONE, WHEN IS THE OPTIMAL TIME TO REMIND. WE IN SOME CASES HAVE PUT IN CONTACT SWITCHES WITH SIMILAR DOORS, REFRIGERATORS, MICROWAVES TO UNDERSTAND EATING BEHAVIORS WHICH IS SO OFTEN AT RISK FOR NOT HAVING ENOUGH, NOT EATING ON SCHEDULE OR LOSING TRACK OF THEIR EATING. THIS MAY ALSO BE TRUE FOR CANCER PATIENTS RECOVERING. IN ANY EVENT, USING THIS TYPE OF DATA, THIS IS FROM AN ARTICLE BY -- WHERE WE CAN TAKE, WITHOUT KNOWING THE LAYOUT OF THE HOME, JUST BY LOOKING AT THE MOTION SENSORS FIRING, WE CAN INFER THE LAYOUT OF THE APARTMENT, INFER ACTIVITIES OF DAILY LIVING. BUT ESPECIALLY IF YOU HAVE SOME INPUT FROM OTHER SENSES AND CAN USE SENSOR FUSION AS WELL. ANOTHER TYPE OF VISUAL, THIS IS AN EXAMPLE OF HOW TO VISUALIZE A VERY SIMPLE APPROACH TO DATA. IN THIS CASE, WE'RE ONLY LOOKING AT THE FIRING OF THOSE SENSORS, THE MOTION SENSORS IN EACH ROOM. AND WE'RE LOOKING AT SLEEP BEHAVIORS AND DEVELOPING HYPOTHESES FOR HOW BEST TO CHARACTERIZE SLEEP IN THIS. SO IN THIS CASE THERE'S ONE PATIENT OVER A YEAR'S TIME WHERE EACH CIRCLE REPRESENTS A DIFFERENT DAY AS YOU GO OUT FROM THE MIDDLE. MIDNIGHT IS AT THE TOP. NOON AT THE BOTTOM. AS YOU GO AROUND THE DAY, YOU'LL SEE THAT THE GREEN DOTS REPRESENT MOTION IN THE BEDROOM. AND THIS PERSON IS IN THE KITCHEN FOR BREAKFAST, LUNCH, A MORE DISBURSED ROOM. I'VE BEEN WORKING ON A SLEEP PROJECT AS WE VAN INTERVENTION TO RESPOND TO QUALITY OF SLEEP. FOR THIS CASE, THIS PARTICULAR PERSON IN THE HOME HAD VERY RESTFUL SLEEP THE FIRST SIX MONTHS. IT BECAME MORE DISRUPTIVE AS THE TIME WENT ON. THIS WOULD BE A GOOD TIME TO SUGGEST A SLEEP INTERVENTION AND ALSO MONITOR AS YOU IMPROVE SLEEP HYGIENE, MUSIC THERAPY, LIGHT THERAPY, WHATEVER IS INDICATED AS WE TRY TO ASSESS AND TAILOR INTERVENTION. THIS GIVES US INSIGHTS SO WE KNOW HOW BEST TO CHARACTERIZE SLEEP. WE KNOW BY LOOKING AT ANOTHER EXAMPLE, THAT WE HAVE TO UNDERSTAND WHAT'S NORMAL FOR AN INDIVIDUAL. IN THIS CASE, THE PERSON IS, THIS IS A DIFFERENT SUBJECT. AGAIN OVER A YEAR'S TIME HAS FAIRLY DISRUPTED SLEEP THROUGHOUT. BUT YOU'LL NOTICE IT'S ALWAYS OUT OF THE APARTMENT FOR BREAKFAST LUNCH AND DINNER. THIS IS SOMEONE WHO LIVES IN A RESIDENTIAL FACILITY WHERE THE MEALS ARE SOMEWHERE ELSE. WE CAN INFER SOCIALIZATION IMPROVEMENT FROM THIS. THE OTHER ONE HARDLY HAD ANY TIME AWAY FROM THE APARTMENT. SO IT'S IMPORTANT TO CHARACTERIZE NORMAL ACTIVITY SCHEDULES FOR EACH INDIVIDUAL AND LOOK FOR MEANINGFUL DIFFERENCES. NOW I TALKED A LITTLE EARLIER ABOUT MEASURING WALKING SPEED. I'M GOING TO SHOW YOU ANOTHER EXAMPLE OF DATA VISUALIZATION AGAIN FOR A VERY SIMPLE SET OF BEHAVIOR DATA IN THE HOME. THESE ARE FROM THESE MOTION SENSORS THAT HAVE THE NARROW FIELD OF VIEW SO WE CAN MONITOR THE POINTS AS THEY PASS BACK AND FORTH IN THE HALLWAY. AND MEASURE GATE VELOCITY. SO THIS IS ONE SUBJECT'S TIME FROM, LET'S SEE, DECEMBER IN 2007 UNTIL DECEMBER 2010. WHERE THE SUBJECT HAD A STROKE IN THE MIDDLE. NOW THE WARM COLORS REPRESENT THE SPEAKS OF THE DISTRIBUTION. SO ON THE VERTICAL AXIS YOU'RE GETTING VELOCITY. THEY'RE FASTER, THE HIGHER UP, AND THEN THIS PERSON SLOWS DOWN AFTER A STROKE AND RECOVERS. BUT YOU'LL NOTICE A LITTLE EARLIER AROUND AUGUST 2008, THERE APPEARS TO HAVE BEEN ANOTHER INCIDENT THAT THIS SUBJECT HAD TO RECOVER FROM. THIS KIND OF PLOT, WE DIDN'T KNOW ANYTHING ABOUT THAT. BUT THIS TYPE OF DATA VISUALIZATION ALLOWS US TO DEVELOP HYPOTHESES, DEVELOP ALGORITHMS THAT WOULD BE ABLE TO PICK UP CHANGES LIKE THIS AND INVESTIGATE. ANOTHER EXAMPLE, YOU'LL NOTICE THE OTHER ONE, THE PERSON WHO IS STABLE AFTER RECOVERY, THIS IS SOMEBODY WITH MAL COGNIZANT IMPAIRMENT WHO IS DECREASING SLOWLY OVER TIME. MOTOR SPEED. NOW, IF WE COULD DETECT THESE TYPES OF CHANGES, LOOKING AT WITHIN SUBJECT PERFORMANCE, IT CAN BE VERY VALUABLE INPUT. AND IN THESE KINDS OF DATA PLOTS, THE DATA VISUALIZATION LEADS US TO DEVELOP ALGORITHMS TO DO APPROPRIATE DETECTION AS WE LOOK AT THIS TYPE OF DATA FOR LARGE NUMBER OF SUBJECTS. NOW, ANOTHER SOURCE OF BIG DATA IS WHAT WE USE TO MEASURE SOCIALIZATION. AGAIN, SOCIALIZATION ISN'T SOMETHING THAT'S REALLY HANDLED BY THE MEDICAL CARE TEAM AND YET WE HAVE SEEN THAT IT DIRECTLY INFLUENCES QUALITY OF LIFE AND CLINICAL OUTCOMES ESPECIALLY IN THE COGNITIVE AREA. SO AS I'M WORKING ON COGNITIVE HEALTH COACHING, I'M SURE TO PUT IN SOCIALIZATION INTERVENTIONS. WE WITH OUR SUBJECTS GIVE EVERYONE A WEB CAM AND SITE CAMERA, WEEK CAM AND SITE ON THE COMPUTER. AND THEN ALSO ENROLL FAMILY MEMBERS WITH THE SAME TOOLS AND LOOK FOR INTERACTIONS AND ACTUALLY PUT ON THEIR WEEKLY ACTION PLAN FOR HEALTHCARE, SOCIALIZATION EXERCISES. SLEEP IS ANOTHER AREA OF GREAT INTEREST TO US. NOW YOU SAW THAT FROM THE OTHER DATA, WE COULD MEASURE SLEEP QUALITY JUST BY LOOKING AT THE MOTION SENSOR FIRING. HOWEVER, I'LL GIVE YOU AN EXAMPLE OF WHAT CAN BE DONE EVEN MORE IN-DEPTH MEASURES OF SLEEP. BY COMPARING WHAT'S DONE TYPICALLY WHEN PEOPLE HAVE SLEEP PROBLEMS, THEY GO TO A SLEEP LAB FOR A COUPLE THOUSAND DOLLARS IN THE UNITED STATES AND GET ALL WIRED UP AND REST IN AN UNNATURAL ENVIRONMENT FOR A ONE TIME EXPOSURE. IT WOULD BE MUCH MORE IMPORTANT TO DO THIS AT HOME WITH YOUR OWN BED. WHAT WE'VE USED IN THIS EXAMPLE, WE HAVE PH.D. STUDENT ZACK BEADY USED LOAD SENSORS UNDER EACH BED POST TO MEASURE FORCES. HE MEASURES CENTER OF MASS CHANGES UP AND DOWN THE BODY. AND WITH THIS, CAN GET A MEASURE OF RESPIRATION RATE AND LOOK AT APNEA DETECTION. AND WHAT YOU'LL SEE HERE IN GREEN IS HIS ESTIMATES AND AS THEY COMPARE TO THE BLUE CONTROLS STANDARD MEASURES. SO THAT WAS ONE AREA. THIS TYPE OF MEASUREMENT COULD BE MADE EVERY NIGHT UNOBTRUSIVELY AND MONITOR SLEEP A KNOW THROUGHOUT TIME, DO EARLY DETECTION OF IT, MAKE SURE IT'S TREATED AND THEN LOOK TO SEE WHETHER THE INTERVENTION CONTINUES POSITIVE AIRWAY APPRECIATE ARE HELPFUL. ACTUALLY GET FEEDBACK IN THE HOME. THIS WOULD BE QUITE NEW. I PUT THIS UP, IT'S A POPULATION PLOT, BUT AGAIN, AS AN EXAMPLE OF A VERY SIMPLE MEASURE, JUST SIMPLE COMPUTER USE, WE'RE COMPARING THE BLUE OLDER ADULTS THAT ARE COGNITIVELY IN TACT. THE COMPUTER USE ON AVERAGE IS MAINTAINED THROUGHOUT TIME WHEREAS PEOPLE WITH MILD COGNITIVE IMPAIRMENT AS MEASURED WITH STANDARD TECHNIQUES ARE DECLINING. THESE ARE SIMPLE MEASURES. WE ACTUALLY LOOK AT LINGUIST IC COMPLEXITY FROM WHAT PEOPLE TYPE. THAT'S A MEASURE OF COGNITIVE PERFORMANCE. MOTOR SPEED CAN BE INFERRED FROM THE SPEED OF WHICH THEY TYPE PASSWORDS AND LOG-INS. THERE'S A WEALTH OF INFORMATION THAT COMES FROM COMPUTER ACTIONS, INCLUDING PEOPLE'S INTERACTIONS WITH GAMES, COMPUTER GAMES. IN THIS CASE -- FUNDED US TO DEVELOP NINE COMPUTER GAMES THAT HAD IMBEDDED MEASURES OF COGNITIVE PERFORMANCE. THESE ONE THING TO GET THEM TO PLAY AND KEEP THEM INTERESTED BUT YOU CAN HAVE SIMPLE MODELS THAT TEASE APART THE VARIOUS COGNITIVE TASKS THAT CAN GIVE ONGOING MEASURES OF MEMORY DIVIDED ATTENTION, PLANNING VERBAL FLUENCY. FOR INSTANCE YOU'LL SEE THE GAMES HERE. THERE ARE A COUPLE OF WORD GAMES THAT MEASURE VERBAL FLUENCY. THERE'S A TWO DIMENSIONAL BLACK JACK GAME THAT MEASURES DIVIDED ATTENTION. AGAIN IT'S NOT SO MUCH ABOUT HOW YOU ACTUALLY PLAY BLACK JACK BUT CAN YOU DO ROWS AND COLUMNS AT THE SAME TIME. A LOT OF THE GAME PERFORMANCE, WE'RE USING SUBJECTS AS THEIR OWN CONTROLS. THIS ALLOWS US TO DO TREND DETECTION, EARLY DETECTION, HAVE MUCH MORE SENSITIVE MEASURES THAT ARE NOT AS CULTURALLY BY ELSED AS YOU MIGHT SEE WITH STANDARD NEUROPSYCHOLOGICAL TESTS THAT ARE VERY DEPENDENT ON A PERSON'S LANGUAGE. TEST TAKING ABILITIES, CULTURE, EDUCATION, ETCETERA. THERE'S A STANDARD CONCENTRATION GAME IF YOU MOVE UP TO THE UPPER RIGHT THAT MEASURES MEMORY, AND I'LL SHOW A QUICK MODEL HOW WE INTERPRET NOT THE CONCENTRATION GAME SCORES BUT OUR IMBEDDED MEASURES OF MEMORY. BUT FIRST I'LL START WITH ONE IN THE UPPER RIGHT AND GIVE YOU A BETTER IDEA OF WHAT THAT LOOKS LIKE. WE CALL IT SCAVENGER HUNT. IT'S PATTERNED AFTER THE STANDARD TRAIL MAKING TEST, WHICH IS TRAILS A, TRAILS B, WHICH LOOKS AT THE DIFFERENCE IN HOW SOMEONE CAN GO FROM ONE DOT TO ANOTHER IN NUMBERED ORDERS ON A PIECE OF PAPER AS YOU JUST GO FROM ONE TO 23, COMPARED TO AN ALTERNATING SET OF SWITCHES WHERE. THIS IS A TYPICAL NEUROLOGICAL TEST USING A VARIETY OF NEUROLOGICAL CONDITIONS THAT PEOPLE DIAGNOSE, TREAT AND MONITOR. NOW WHAT WE'VE DONE HERE IS ATTEMPT TO MAKE A FUN GAME OUT OF IT WHERE PEOPLE AS FAST AS THEY CAN, TAKE THE SEQUENCE AT TOP AND THEN CLICK ON THE TARGETS IN ORDER. AND WE HAVE SOME SIMPLE SEQUENCES, SOME MEANINGFUL WORDS THAT ARE EASY TO REMEMBER, AND THEN SOME MORE COMPLICATED SO WE CAN GET MUCH MORE DYNAMIC RANGE AND REPEATED MEASURES THROUGHOUT TIME. WHEREAS THE STANDARD TRAIL MAKING TELLS CAN BE, IS TYPICAL AND ONLY ADMINISTERED ONCE A YEAR, AND IS ADVISED NOT MORE THAN EVERY SIX MONTHS TO GET VERY SPORADIC MEASUREMENTS. NOW ONE OF THE HALL MAKES HOW FAR DEMENTIA AS PEOPLE WHO MOVE TOWARD COGNITIVE IMPAIRMENT IS VARIABLITY SCORES INCREASES. SO IT'S IMPORTANT TO MEASURE THE INDIVIDUAL TREND AND THE VARIABILITY WHICH YOU CAN ONLY DO IF YOU'RE MONITORING FREQUENTLY AND HOPEFULLY IN THE HOME AND ACTUAL ENVIRONMENT. NOW, WHEN YOU'RE LOOKING AT THE BIG DATA ASPECTS OF THIS, IT'S NOT SUFFICIENT JUST TO THROW A STANDARD STATISTICS AT IT. OR TO JUST SIMPLY DATA MINE WHEN YOU KNOW THERE'S A STRUCTURAL TYPE OF MODEL THAT COULD GIVE YOU MORE INFORMATION. SO IN THIS CASE WE BROKE APART RECALLING THE NEXT TARGET WITH SEARCHING FOR THE NEXT TARGET, WHICH DEPENDS ON THE NUMBER OF TARGETS AND THE NUMBER OF DISTRACTERS, AND JUST THE SPEED OF MOTION. WITH THAT KIND OF MODEL, WE WERE ABLE TO COMPARE STANDARD -- MAKING TESTS AND THE CIRCLES REPRESENT TRAIL MAKING A WHERE THEY'RE JUST GOING ONE, TWO, THREE, FOUR, FIVE, SIX, SEVEN, TO THE INFORMATION THAT WE GET FROM THE GAME, WHICH WE CALLED OUR ESTIMATE OF A TRAIL MAKING SCORE. SO WE CAN PREDICT WHAT PEOPLE WOULD HAVE DONE ON A COMPUTER WITH TRAIL MAKING SIMPLY BY THEIR PERFORMANCE IN A GAME. YOU MAY NOT THINK THAT A .78 CORRELATION IS THAT GOOD BUT IT'S COMPARABLE TO REPEATED TRAIL MAKING SETS. LIKE PSYCHOLOGICAL TEST THIS IS A VERY STRONG TYPE OF CORRELATION. NO NO, WHAT I MEAN TO SAY, IT'S ONE, IT'S A GROUP OF PATIENTS EACH DOT REPRESENTS A PATIENT. THE CIRCLE IS THEIR PERFORMANCE ON TRAILS A. THE PLUS WHICH IS HARDER TO DO IS PERFORMANCE ON TRAILS B WHERE THEY HAVE TO DO SWITCHING, ADDITIONAL COGNITIVE LOAD SLOWS THEM DOWN. BUT THE PREDICTION HOLDS. THANKS FOR ASKING, AND PLEASE DO INTERRUPT WHEN YOU HAVE QUESTIONS. NOW, LOOKING AT ONE OF THE OTHER GAMES. I'M JUST GIVING TWO AS AN EXAMPLE. THIS WAS THE MEMORY GAME, AND THIS MIGHT BE A LITTLE CONFUSING. I SHOW AN ANALOGUE CLOCK VERSUS A DIGITAL CLOCK JUST MAKING IT, WE INCREASE DIFFICULTY ON THESE GAMES. ALL OF OUR GAMES ARE ADAPTIVE, SO IN THIS CASE MORE CARDS MAKES IT MORE DIFFICULT. MOVING FROM A SIMPLE CARD TO SOMETHING AS ABSTRACT AS DOMINOS OR THIS DIGITAL VERSUS ANALOGUE CLOCK, IS MUCH MORE CHALLENGING. NOW, IN MODELS OF PERFORMANCE, WHAT WE WANT TO DO IS ESTIMATE THE SURVIVAL OF THE MEMORY. WE CALL THAT THE MEMORY WORKING -- THIS IS EXPLAINED ON THE ORCATECH WEBSITE. I WON'T GO THROUGH THE DETAILS BUT THE GRAPH OF THE RIGHT SHOW TYPICAL SURVIVAL CHARACTERISTICS FOR OUR ESTIMATES FROM THE GAME AND HOW IT FITS SURVIVAL FUNCTION AND HOW YOU CAN USE JUST TWO PARAMETERS TO COMPLETELY CHARACTERIZE AN INDIVIDUAL'S PERFORMANCE WITH REGARD TO THEIR WORKING MEMORY. SO THIS IS ONE AREA WHERE WE USE BIG DATA THAT WE'RE ANALYZING IT WITH STRUCTURED MODELING APPROACHES. WE'VE ALREADY GONE THROUGH THE FILTERING AND EVERYTHING ELSE. BUT THERE'S MANY MORE TYPES OF DATA COMING IN AND SOME OF THE QUESTIONS ARE HOW DO YOU FUSE THIS. SAY WE'RE GETTING MOTOR SPEED FROM A VARIETY OF THE COMPUTER GAMES. WE'RE ALSO GETTING MOTOR SPEED FROM THE WALKING BEHAVIOR. WE WANT TO COLLECT DATA AND TRIANGULATE ON PATIENT STATE AND PATIENT CAPABILITY. SO THESE ARE THE OTHER SOURCES OF VERY COMPLEX DATA THAT ARE COMING IN AND BASICALLY OVERWHELMING US. BUT THE IMPORTANT THING HERE IS THAT WE'RE TRYING TO TAKE THIS DATA, ANALYZE IT IN A MEANINGFUL WAY AND CLOSE THE LOOP SO WE INFLUENCE HEALTH BEHAVIOR IN THE HOME. SO I'M JUST GOING TO DO A QUICK DESCRIPTION OF HOW THIS TYPE OF DATA IS USED IN OUR INTERVENTIONS. WE HAVE A PLATFORM THAT HIPS THE COACH INTERACT WITH A LARGE NUMBER OF CLIENTS. IN THIS CASE OLDER ADULTS. AND WE HAVE MULTIPLE MODULES. SAY WE'RE TRYING TO DO COGNITIVE COACHING WITH PHYSICAL EXERCISE, SLEEP MANAGEMENT, SOCIALIZATION. ALL OF THESE ISSUES DIRECTLY AFFECT COGNITIVE HEALTH AS WELL AS PRACTICE ON COMPUTER GAMES. THE THING THAT OLDER ADULTS ARE ABLE TO LEARN NEW TASKS AND DEVELOP IMPROVEMENT IN COGNITIVE PERFORMANCE ON SINGLE TASKS, WE DON'T KNOW MUCH ABOUT HOW THIS LASTS OVER TIME. BUT IT'S BEEN OUR FEELING THAT IT'S IMPORTANT, IN ORDER TO HAVE THIS EFFECT GENERALIZED, THAT WE GO AT IT AND INTERVENE FROM A VARIETY OF ANGLES. ADDRESS SLEEP ISSUES, ADDRESS SOCIALIZATION. OF COURSE PHYSICAL EXERCISE IS PROBABLY THE NUMBER ONE THING YOU CAN TO HELP COGNITIVE PERFORMANCE. SO I WON'T GO INTO THE DETAILS HERE AS FAR AS WHAT THE OVERALL ARCHITECTURE OF OUR HEALTH COACHING PLATFORM, BUT BASICALLY JUST WANTED TO POINT OUT THAT WE HAVE A FAIRLY SOPHISTICATED USER MODEL THAT IS BASED ON OUR INITIAL ASSESSMENT AND ONGOING MONITORING ASSESSMENTS OF THE PATIENTS. THIS INFLUENCES THE AUTOMATED MESSAGES THAT ARE GENERATED WHEN A COACH COMES INTO INTERACT. SO THE COACH SEES ONE INTERFACE TO THE SYSTEM AND THE PATIENT SEES THEIR OWN. AND WE'RE ALSO WORKING ON A FAMILY INTERFACE BECAUSE ONE OF THE UNTAPPED RESOURCES WE HAVE IN THIS TYPE OF WORK IS BRINGING FAMILY MEMBERS AND INFORMAL CARE GIVERS INTO THE CARE TEAM IN A MEANINGFUL WAY, PROVIDING THEM WITH INFORMATION, HELPING THEM PARTICIPATE FOR THEIR LOVED ONES. NOW SO THE COACH INTERFACE ALLOWS THEM TO SEND OUT MESSAGES VERY QUICKLY. THEY'RE ABLE TO EDIT THEM BUT WE BREAK THEM APART INTO GREETING, FEEDBACK, PLANS FOR THE NEXT WEEK, COMPLEMENTARY CLOSURE AND TRY TO MAKE IT LOOK AS TAILORED AS POSSIBLE. WE HAVE FOUND IN THIS WORK THAT ONE OF THE CHALLENGES IS KEEPING PEOPLE ENGAGED OVER A LONG PERIOD OF TIME. MOST OF THE COGNITIVE HEALTH COACHING OR COGNITIVE INTERVENTIONS IN THE LITERATURE ARE SHORT TERM FOR TWO WEEKS, MAYBE A MONTH. BUT AS WE KNOW, RIGHT WEIGHT MANAGEMENT OR SMOKING, IT'S A LIFE-STYLE. IT'S SOMETHING THAT NEEDS TO BE SUSTAINED AND VARIETY IS VERY IMPORTANT. IT HAS TO BE FUN. THE GAMES CAN'T BE ONEROUS. IT'S MUCH MORE HELPFUL IF THEY'RE ENGAGING. WE KNOW THAT WE NEED TO TAILOR AND WE CAN DO THAT BASED ON MONITORING FEEDBACK FROM THE SENSORS AND FROM THIS BIG DATA. IT'S IMPORTANT THAT THE ALGORITHMS INCORPORATE WHAT WE ALREADY KNOW ABOUT HEALTH BEHAVIOR CHANGE. WE KNOW A LOT ABOUT MOTIVATIONAL INTERVIEWING, UNDERSTANDING BARRIERS AND MOTIVATION. INCORPORATING A PERSON'S READINESS TO CHANGE AS WE GENERATE MESSAGES. SO THE IDEA HERE IS TO CREATE A SCALABLE APPROACH THAT USES THE BIG DATA INFERENCES TO INTERVENE IN THE HOME. WE'VE SEEN THESE MODULES. I'M JUST GOING TO QUICKLY SHOW WHAT SOME WEB PAGES LOOK LIKE FOR PHYSICAL EXERCISE. WE STARTED WITH YOUTUBE VIDEOS LIKE THIS WHERE WE CAN MONITOR HOW LONG THEY USE THE SYSTEM, AND IF THERE'S MOTION. NOW WE'RE USING A CONNECT CAMERA TO HELP WITH GIVING IMMEDIATE FED BACK ON HOW WELL THEY'RE DOING THE MOTIONS. WE CAN MEASURE ANGLES, IT USES, IT HAS 3D INTERPRETATION. SO WE HAVE HAD OUR PHYSICAL THERAPIST DEVELOP SPECIFIC EXERCISES, CHAIR EXERCISES AT THIS POINT FOR PEOPLE IN THE HOMES AND WE'RE TESTING OUR ABILITY TO INTERPRETED THE EXERCISE AND GIVE AUTOMATED FEEDBACK. THE SLEEP MODULE INCLUDES ISSUES LIKE SLEEP HYGIENE, LIKE MUSIC THERAPY, MANY OPPORTUNITIES FOR INTERVENING FOR SLEEP. SOCIALIZATION IS ANOTHER ONE THAT I'VE ALREADY MENTIONED. AND THIS IS WHAT THE PARTICIPANT CURRENTLY SEES, WHICH INCLUDES A FEATURE STORY WEEKLY GOALS AND ACTION PLAN ON THE RIGHT. BUT THE IMPLICATIONS, GOING BACK TO BIG DATA, ONCE YOU UNDERSTAND HOW THE DATA INFORMS A HOME HEALTH INTERVENTION, THE IMPLICATIONS NOW, YOU SAW THIS OVERWHELMING AMOUNT OF DATA. AND WE'RE NOT USING ALL OF THIS, WE'RE JUST AT THE BEGINNING STEPS. AND I THINK A LOT OF PEOPLE WORKING IN THIS AREA ARE OVERWHELMED WITH THEIR DATA AS WELL. AND WHAT WE'RE PROMOTING AT LBSSR IS THE USE OF MULTIDISCIPLINARY TEAMS TO ATTACK THESE PROBLEMS. BECAUSE THIS REQUIRES NEW TECHNIQUES THAT DON'T COME FROM JUST ONE DISCIPLINE. WE'VE SEEN THE MATHEMATICAL PSYCHOLOGISTS ARE EXPERTS IN MEASUREMENT THEORY, PERCEPTUAL PSYCHOLOGISTS ARE ABLE TO MAKE HUGE STRIDES IN INTERACTING WITH DATA THAT IS DRAWN FROM HUMAN PERFORMANCE. SOMETHING THAT MOST ENGINEERS AND SCIENTISTS ARE NOT AWARE OF, WHEREAS ALL THE DATA MINING TECHNIQUES AND THERE ARE MANY LESSONS TO BE LEARNED FROM OTHER APPLICATIONS. SO WE NEED MORE SOPHISTICATED DATA MODELS TO INFER PATIENTS STATE AND MAKE WHAT OUR GROUP IS STARTING TO CALL BEHAVIORAL MARKERS. THAT ARE CLINICALLY RELEVANT. THIS IS IN CONTRAST TO, YOU'RE PROBABLY MORE FAMILIAR WITH BIO MARKERS WITH THE GENETICS OR STANDARD CLINICAL PROCESSES. WHAT I'M TRYING TO PROMOTE HERE IS THIS NEW AREA OF LOOKING AT BEHAVIORAL MARKERS IS GOING TO BE MORE AND MORE IMPORTANT. BUT IT DOES REQUIRE SOPHISTICATED USER MODELS AND UNDERSTANDING SECURITY AS WE GO INO THE HOME AND ESPECIALLY WITH SLEEP BEHAVIORS MOVEMENT THROUGHOUT THE WORLD. AND AGAIN, ANOTHER PLEAD INCLUDES ELDERS THAT CAN HELP FAMILY MEMBERS AND INTERESTED LOVED ONES PARTICIPATE IN THE CARE. BUT AGAIN, SO MANY CHALLENGES. THE CONTEXT IS HARD TO MEASURE, THE MODELS NEED TO BE CLARIFIED. THERE'S A VARIETY OF SENSORS OUT THERE AND THE INFORMATION ASSOCIATED WITH THE SENSORS. THE FILM POINT ON THIS SLIDE IS SOMETHING THAT I HAVEN'T PREPARED YET AND I WANT TO EMPHASIZE WHEN THEY ARE CONTINUOUSLY MONITORING THE HOME, YOU'RE LIKELY TO OVERALERT. IT'S ALMOST REQUIRED A NEW SCIENCE IN ITSELF, HOW WHEN YOU HAVE EVENTS IN A LARGE LARGE AMOUNT OF DATA AND YOU'RE TRYING TO DETECT INFREQUENT EVENTS, HOW DO YOU AVOID THE FALSE POSITIVES WHILE MAINTAINING SENSITIVITY. SO YOU PICK UP THE TRUE POSITIVES. NOW, ALONG WITH THE CHALLENGES A RECENT GARTNER REPORT THAT IS DISCUSSING ALL THE HYPE. WE'RE OVERWHELMED WITH BIG DATA. PEOPLE WANT TO KNOW WHEN ARE WE DEALING WITH SOMETHING NEW. SO THERE'S THE TROUGH, THE HYPE AT THE PEAK AND THE TROUGH AND WONDERING WELL WHAT'S GOING TO BE SUSTAINED OVER TIME. AND WHAT I THINK IS DIFFERENT ABOUT OUR AREA WHEN YOU'RE LOOKING AT BEHAVIORAL DATA IS IT'S TRULY UNEXPLORED. PEOPLE HAVEN'T SEEN THE SLEEP DATA REPORT, THEY HAVEN'T SEEN THIS WALKING DATA. IT'S SO IMPORTANT FOR EARLY DIAGNOSIS AND INTERVENTIONS. SO IT ALSO ALLOWS PATIENTS TO USE THEIR OWN CONTROLS, MAKING IT MORE SENSITIVE, LESS BIASED AND CONTRIBUTES TO PERSONALIZED PRECISION MEDICINE THAT'S BEEN PROMOTED HERE AT NIH. AND IN ADDITION, THIS TYPE OF DATA ALLOWS US TO MEASURE VARIABILITY, WHICH IS SOMETHING WE'VE SEEN AS SOMETIMES MORE ROBUST INDICATOR OF CLINICAL PROBLEMS THAN THE ACTUAL MEASURE ITSELF. SO WE NEED NEW DATA VISUALIZATION APPROACHES. THIS IS AN EXAMPLE FROM DEB ROY'S TALK. HE'S PLOTTING, HE'S LOOKING AT HIS SON'S, THIS IS VIDEO DATA OVER TIME LOOKING AT HIS SON'S LANGUAGE LEARNING AND THIS IS A PLOT OF WHEN HIS SON HAS SPOKEN THE WORD WATER SHOWING THERE IN THE APARTMENT UNDER WHAT CONTEXT. THERE'S ONE WAY TO VISUALIZE THE DATA. AND ANOTHER APPROACH YOU COME FROM THE DARPA PROJECT ON A TOPOLOGICAL DATA NOW. THIS IS SOMETHING THAT'S BEEN USED MORE ROUTINELY IN COMPUTER VISION. IT'S AN APPROACH WHERE YOU TAKE, I CAN'T PRETEND TO BE AN EXPERT ON THIS BECAUSE I'M NOT SURE WHERE, WHAT THIS PARTICULAR PLOT REPRESENTS. BUT TOPOLOGICAL ANALYSIS HAS BEEN USED TO TAKE PLOT DATA AND INFER MULTIDIMENSIONAL STRUCTURE FROM IT. FOR EXAMPLE HUMANS ARE VERY GOOD AT LOOKING AT A PHOTOGRAPH AND INFERING 3D STRUCTURE AND THESE APPROACHES HAVE BEEN USED IN COMPUTER VISION BUT COULD ALSO APPLY FOR A LOT OF OTHER DATA TYPES WHEN YOU NEED TO UNDERSTAND THE MULTIDIMENSIONALITY OF A PROBLEM. SO I'M GOING TO WIND UP JUST TALKING ABOUT THAT WE NEED TO UNDERSTAND WHAT MODELING IS APPROACH FOR WHAT KIND OF APPLICATION. AND MODELS RANGE FROM PURE STATISTICAL MODELING AND LIKE THE MODEL I TRIED TO DESCRIBE SOME EXAMPLES. WE ALSO CONSIDER DECISION MODELS, TIME SERIES MODELS GRAPHICAL MODELS SIMULATION MODELS. BUT MOST PEOPLE ARE COMING AT THESE PROBLEMS WITH A CERTAIN METHOD OR TOOL THAT THEY'VE BEEN TRAINED IN. AND JUST TRYING TO SEARCH FOR AN APPLICATION. WHERE WHAT WE REALLY NEED IS TO BE ASKING THE RIGHT QUESTIONS AND USING THE RIGHT APPROACH. I LOVE JOHN -- QUOTE THAT AN AWE APPROXIMATE ANSWER TO THE RIGHT PROBLEM IS WORTH A GREAT DEAL MORE THAN AN EXAM ANSWER TO AN APPROXIMATE PROBLEM. WE ARE ENCOURAGED AS SCIENTISTS WHEN TO PUBLISH, TO MEASURE THINGS THAT ARE EASY TO MEASURE. THEY MAY NOT BE AS RELEVANT. FOR EXAMPLE MEASURING PATIENT UTILITY IS VERY DIFFICULT, VERY A CROSS MIX AND YET INCREDIBLY IMPORTANT COMPARED TO TRYING TO REFINE A PROBABILITY THAT'S BEEN OVERSTUDIED IN A PARTICULAR DECISION PROBLEMS. SO GOING BACK TO OUR LIST OF MODELS, WHEN DO YOU USE WHICH MODEL, WHAT VARIABLES SHOULD WE BE INCLUDING, WHAT FEATURES ARE IMPORTANT, HOW TO GET STARTED AT OBSSR AND WE WOULD LIKE YOUR HELP AND INPUT. WE'RE LOOKING AT DEVELOPING A BIG DATA TRAINING THAT WOULD ENCOURAGE MULTIDISCIPLINARY TEAMS, TRANSDISCIPLINARY TEAMS TO WORK TOGETHER TO SOLVE THESE TYPES OF PROBLEMS THAT ARE FOR A STARTING POINT SPECIFIC TO THIS TYPE OF DATA COMING IN FROM THE HOME ENVIRONMENT THAT IS FRAUGHT WITH NEW CHALLENGES. THESE ARE A LIST OF CURRENT SKILL SETS THAT WE ARE INCLUDING, BUT IN ADDITION, WE WANT TO MAKE SURE WE'RE JUST NOT TALKING ABOUT THE TECHNOLOGY BUT ALSO THE CLINICAL HEALTH RELEVANCE, GETTING A TEAM THAT INCLUDES EXPERTS IN THE DOMAIN AND ALSO FACILITATING THE COMMUNICATION AMONG TEAM MEMBERS. AND EXPERTISE REQUIRED FOR IRBs TO EVEN REVIEW THESE TYPES OF INTERVENTIONS. AND SO TO FINISH OFF, JUST REVIEWING WHAT I FELT WERE THE MOST IMPORTANT MESSAGES HERE IS THAT WE'RE SHIFTING OUT OF HOSPITAL TOWARD THE HOME. WE NEED BEHAVIORAL METRICS, WHAT I'M CALLING BEHAVIORAL MARKERS. WE HAVE AN OPPORTUNITY TO LOOK AT UNOBTRUSIVE AND CONTINUOUS MONITORING FROM LOW COST CENTERS BUT OF COURSE IT NATURALLY MEANS A LOT OF NEW APPROACHES ARE REQUIRED TO INTERPRET THIS. AND VARIETY OF CHALLENGES, INCLUDING FUSION ALGORITHMS, VISUALIZATION. AND WOULD APPRECIATE YOUR INPUT AND HELP AND FEEDBACK IF YOU'D LIKE TO TALK MORE, HERE'S MY E-MAIL. I'M AT NIH FOR ANOTHER YEAR AND PEOPLE IN OBSSR ARE VERY SUPPORTIVE OF WORK IN THIS AREA. WE'D LIKE TO TALK TO YOU MORE AND I WOULD LIKE TO TAKE QUESTIONS NOW. THANK YOU. [APPLAUSE] >> [INDISCERNIBLE] >> IN ONE CASE, THAT'S ONE PATIENT WELL NOT ONE PATIENT, ONE USER PLAYING THE GAME THROUGHOUT TIME AND YOU CAN DO IT FOR A PARTICULAR GAIN AND THEN GET THE NEXT ASSUMPTION ON THE NEXT GAIN. YOU'RE TAKING WHAT YOU KNOW ABOUT SURVIVAL CHARACTERISTICS. WHAT YOU WANT TO UNDERSTAND IS THE SURVIVAL OF THE PARTICULAR MEMORY. AND IT HAPPENS TO BE AN EXCELLENT FIT FOR A WIDE VARIETY OF PERFORMANCES. AND WHAT HAPPENS IS WITH VERY LITTLE DATA YOU GET A VERY CHOPPY CURVE BUT YOU WANT TO APPROXIMATE IT WITH A KNOWN DISTRIBUTION. BUT IT'S NOT REALLY LOSING MUCH BECAUSE THEN YOU'RE ACTUALLY CHARACTERIZING THE PERFORMANCE ON THAT GAME, THE MEMORY PERFORMANCE ON THAT GAME WITH TWO PARAMETERS. ONE BEING WHAT WE'RE CALLING THE MEMORY BUFFER LINK AND THEN YOU CAN PLOT THAT MEMORY BUFFER LINK FOR THAT INDIVIDUAL EVERY SINGLE TIME WE PLAY THE GAME. THEN THERE'S ANOTHER QUESTION HOW DO YOU SUMMARIZE THAT KIND OF INFORMATION, TREND IT. SO THIS CHARACTERIZATION, THE ARRIVAL FUNCTIONS FOR CHARACTERIZING THE COMPUTER INTERACTIONS ON THAT MEMORY GAME WAS JUST ONE STEP IN KIND OF SMOOTHING AND UNDERSTANDING THE INFERENCE ABOUT PERFORMANCE. >> [INDISCERNIBLE] >> I SHOULD HAVE MENTIONED THAT. THAT'S ONE OF THE BIG CHALLENGES RIGHT NOW. IF YOU DON'T HAVE ANYBODY, ONE APPROACH WE'VE TAKEN WHERE IF WE HAVE PEOPLE -- DIFFERENTIATES THEM THAT WAY. SOME PEOPLE HAVE TRIED, INCLUDING US, WE'VE LOOKED AT GATE DIFFERENCES TO SEE WHO IS WALKING DOWN THE HALL, MAKE SOME INFERENCES ABOUT WHO IS WHO. BUT THIS IS A REALLY UNSOLVED ISSUE RIGHT NOW. SO THAT, YES. WE CAN INFER THIS SOMETIME. WE DEFINITELY KNOW WHEN THERE'S TWO PEOPLE OR MORE. BECAUSE THERE'S INCONSISTENCIES. IF YOU'RE GETTING TRIGGERS IN MULTIPLE ROOMS. AND WE ALSO KNOW WHEN PEOPLE, WE HAVE CONTACT SWITCHES ON THE FRONT DOOR, YOU KNOW WHEN PEOPLE ARE GOING IN AND OUT. WE PUT A LOT OF WORK INTO THIS. AND AS A STARTING POINT, WE'VE CONCENTRATED ON SINGLE PEOPLE LIVING ALONE IN THEIR HOMES DURING TIMES WHEN THROUGHS NO VISITORS AND NO INCONSISTENCIES LIKE THAT. ONE OF THE BEAUTIES OF MEASURING OVER TIME IS YOU CAN AFFORD TO THROW AWAY DATA AND CONCENTRATE ON MEANINGFUL REPRESENTATIVE PERIODS WHERE YOU CAN MAKE WHAT WAS THINKING OF AS A CLINICAL MEASUREMENT. WE'RE ALSO TRYING TO MEASURE SOCIALIZATION. WE INTENTIONALLY WANT TO KNOW WHEN THERE'S MULTIPLE PEOPLE THERE AND IT'S ACTUALLY PRESUMING AN IMPROVEMENT IN SOCIALIZATION. OUR METRIC OF SOCIALIZATION RIGHT NOW IS TIME ON THE PHONE, AMOUNT OF E-MAIL, SKYPE, VIDEO AND CHAT. TIME OUT OF THE APARTMENT WHICH WE PRESUME TO BE WITH OTHERS. MOSTLY WE'RE NOT MAKING DIRECT MEASUREMENTS. FOR EXAMPLE IF WE'RE ASSESSING EATING BEHAVIOR, WE'RE LOOKING AT SIGNALS THAT REFLECT AT WHAT TYPICALLY HAPPENS WHEN SOMEBODY EATS BUT WE DON'T SEE THE FOOD IN THE MOUTH. WITH THE MEDICATION DISPENSER WE KNOW THE DISPENSER'S OPEN. WE DON'T SEE PILL GOING INTO THE MOUTH. BUT THIS IS A BIG IMPROVEMENT, AND WHEN I MENTIONED THE ALERTING, I WANT TO SAY ONE OF THE CONTRIBUTIONS BY HAVING FAMILY MEMBERS AND INFORMAL CARE GIVERS INVOLVED, EVEN SOCIALIZATION STRUCTURE WHERE YOU KNOW NEIGHBORS, YOU CAN HAVE SOFT ALERTS. SO IF SOMETHING LOOKS SUSPICIOUS OR UNUSUAL, MIGHT BE A PROBLEM. YOU DON'T FLOOD THE PHYSICIAN WITH INFORMATION BUT YOU START WITH THE FAMILY INFORMAL CARE GIVERS MOVE TO THE COACH, MOVE TO THE NURSE. I'VE SEEN SO MANY PEOPLE DEVELOP HOME BASE SYSTEMS AND THEY WANT TO GIVE ALL THE DATA RIGHT TO THE PHYSICIAN. INCLUDING SOME WORK I'VE DONE IN THE PAST THAT'S A RECIPE FOR FAILURE. SO LESSON LEARNED. >> [INDISCERNIBLE] >> I DEFINITELY THINK SO. THERE'S BEEN SOME INTERESTING -- IT DEPENDS ON YOUR RESEARCH QUESTION, HOW DO YOU REALLY NEED TO IDENTIFY EXACTLY WHO IS WHO, THERE'S SOME WORK BEING DONE IN SWITZERLAND AND MOTION SENSING STREET CLEARANCE TRYING TO DETECT VIOLENCE. AND IT'S HARD TO TELL HAPPY REVELERS. WHEN YOU HAVE A TASK IN MIND IN THIS CASE TO ALERT SECURITY OR WHATEVER, YOU CAN DO MORE ABOUT THE ACCURACY THAT'S GENERATED FOR ALERT. MONITORING KIDS THEY HAVE SOME GOOD EXAMPLE HERE AT GEORGIA TECH WITH ADHD AND AUTOMATED VIDEO. WE CAN FOLLOW VERY WELL, AS YOU KNOW EVEN THE SIGHT CAMERAS, THE WEB CAMS CAN FOLLOW PLACES. THERE'S A LOT OF FACE RECOGNITION WORK THAT'S BEING DONE TO VARYING DEGREES OF SUCCESS. BUT THEN IT'S A QUESTION OF TRAINING. IF YOU REALLY WANT TO FOLLOW A PARTICULAR GROUP IN DEPTH, YOU HAVE TO TRAIN ON THEM AND YOU MIGHT HAVE LESS STRINGENT REQUIREMENTS FOR RANDOM PEOPLE COMING IN AND OUT OF THE SCENE. SO DEPENDING ON THE TASK, CERTAINLY SO MUCH CAN BE AUTOMATED AND PERHAPS DONE BETTER. SO IT WOULD BE INTERESTING TO COLLECT WHAT TYPICAL IMPORTANT TASKS MIGHT BE AND START THINKING ABOUT THAT. >> [INDISCERNIBLE] >> WHAT I MEANT TO SAY WAS NOISY UNINTERPRETABLE DATA THAT MIGHT, YOU KNOW, WHEN YOU HAVE PLENTY OF DATA AND YOU'RE ABLE TO PICK AND CHOOSE, NOT NECESSARILY AVERAGE IN TIMES WHEN THERE'S MOSTLY PEOPLE IN THE HOMES, FOR EXAMPLE WHEN YOU'RE TRYING TO GET WALKING SPEED ON ONE PERSON. JUST IGNORE THE TIME THAT'S CONFUSING WHEN YOU DON'T KNOW WHO IS WHO. BECAUSE YOU HAVE PLENTY OF OTHER DATA. >> [INDISCERNIBLE] >> YES. THIS IS A REALLY IMPORTANT QUESTION BECAUSE THIS MEANS THAT SOLVING THESE PROBLEMS REQUIRES MODELING IT RATIVELY, I PUT UP A COUPLE SAMPLE PLOTS WHERE WE DIDN'T HAVE A HYPOTHESES YET. IT WAS FIRST LOOK AT WHAT WALKING SPEED DATA MIGHT BE. WHO KNEW IT WOULD BE SO VALUABLE. JUST MOTION SENSOR. ONLY LOOKING AT MOTION SENSORS. STILL VERY INFORMATIVE DATA. EVEN BEFORE WE START ADDING IN OTHER THINGS. SO WHAT I'VE SEEN ANYWAY SO FAR IS WE HAVE TO WORK INCREMENTALLY UNDERSTAND ONE PIECE OF DATA, CLEAN IT UP AND BE FAMILIAR WITH THE CONTEXT MODEL AND START TO BUILD ON THAT. BUT LEARN ITERATIVELY WHAT'S IMPORTANT. BUT ALWAYS INFORMED BY A CLINICAL CONTEXT OF WHAT'S MEANINGFUL. >> IT WAS IN THEIR HOMES, EACH INDIVIDUAL'S HOMES. >> [INDISCERNIBLE] >> WHEN I SAID VIRTUAL LIVING LAB, THE REASON WE CALL THIS A LIVING LAB IS THERE ARE PEOPLE IN THEIR OWN HOMES THAT HAVE BEEN CONSENT TO PARTICIPATE AND TRY OUT A VARIETY OF NEW SENSORS. BUT THEY ALL KNOW EACH LAYOUT IS DIFFERENT. YOU DO BRING UP A POINT THAT WE HAVEN'T, YOU KNOW, OUR DATA COLLECTED THINGS ORCATECH HAS. I PERSONALLY DON'T HAVE THE DATA OUTSIDE THE HOME. SO IT IS CONFINED TO WHATEVER THE LAYOUT OF THEIR HOME IS. GO AHEAD. >> [INDISCERNIBLE] >> YES, RIGHT. A COLLEAGUE OF MINE IN ORCATECH USES CELL PHONES TO AS A MEASURE OF SOCIALIZATION. AND BECAUSE THIS IS A REAL BIG ISSUE FOR OLDER ADULTS WHO ARE SOMETIMES AFRAID TO GO OUT AND ALSO FOR PEOPLE WITH IMPENDING DEMENTIA NEED ASSISTANCE IN WAY FINDING AND THERE ARE WANDERING ISSUES THAT PEOPLE HAVE PROBLEMS. ABSOLUTELY THE CELL PHONE IS GOING TO BE THE MEDIUM USED. ONE PROBLEM WE HAVE IS USING THE CELL PHONE IN THE HOUSE, PEOPLE NEED TO CHARGE IT. THEY DON'T, WOMEN QUITE OFTEN DON'T HAVE POCKETS LIKE I HAVE NO POCKETS TO PUT MY MIKE PHONE THING IN. AND IF WE HAVE BIG HEAVY WATCHES IDENTIFYING THEM WITH A GPS LOCATION, FRAIL ELDERLY WOMEN AREN'T GOING TO WEAR IT. SO WE'VE FOUND THE PHONE IN THE HOME NOT TO BE QUITE THE USEFUL DEVICE THAT IT IS ONCE YOU'RE OUT OF THE HOME, IT STAYS WITH YOU ON AND VERY USEFUL. ONE OF THE THINGS THAT'S IMPORTANT IS TO TAKE ADVANTAGE OF EXISTING TECHNOLOGY THAT PEOPLE ALREADY HAVE AND ARE FAMILIAR WITH. YOU DON'T WANT TO PUT EXTRA BURDEN ON, WE TALKED ABOUT WORK FOR CLINICIANS. WELL PEOPLE IN THE HOME HAVE THEIR OWN WORK FLOW. YOU WANT TO ADAPT TO THAT. GOOD QUESTION. >> [INDISCERNIBLE] >> WHAT I'VE DONE IS STANDARD UTILITY THEORY AND STANDARD DECISION ANALYSIS. FOR EXAMPLE WITH MEDICATION REMINDING, WE CHARACTERIZE EACH MEDICATION WITH HOW IMPORTANT IT IS. VITAMINS ARE, I FORGET, MAYBE WE'LL REMIND BUT MIGHT JUST BE AN E-MAIL. OVER TIME. BUT IF THEY'RE MISSING THEIR -- MEDICINE OR SOMETHING LIKE THAT. SO WE HAVE A RAMPING FUNCTIONING. WE HAVE A MODEL OF HOW LIKELY THEY ARE TO TAKE THE MEDICATION THAT WHEN IT LOOKS, IT RAMPS UP AS TIME GOES ON, THEY'RE NOT GOING TO DO IT, THEY'RE NOT GOING TO DO IT. IT WAS A THRESHOLD OF WHERE THE MOST IMPORTANT MEDICATION OF THEIR TYPICAL SEVEN MEDS HITS AND YOU REMIND THEM AT THAT TIME BUT THEN THIS STRENGTH, WE'VE TESTED THE STRENGTH OF THE REMINDER, THE ANNOYANCE IT HAD IN LAB STUDIES OF DISTRACTION TASKS AS THEY'RE TRYING TO REMIND AND TAKE THEIR MEDICATIONS APPROPRIATELY. AND SOMETIMES IT HAS TO BE TAILORED TO THE INDIVIDUAL. SO WE HAVE DONE A LOT OF WORK IN THAT AREA, BUT WITH FALL DETECTION, THAT'S BEEN ONE WHERE WE CAN'T REALLY GET A DATABASE OF TRUE FALLS VERY EASILY. IT'S A VERY VERY RARE EVENT, CRITICAL, ABSOLUTELY CRITICAL TO DETECT, BUT SO MANY TIMES PEOPLE ARE SITTING DOWN AND IT LOOKS LIKE A FALL. IF YOU'RE JUST USING A -- IF YOU HAVE A VIDEO CAMERA THAT'S REQUIRING THAT THEY'RE ACTUALLY HITTING THE FLOOR LAYING DOWN BUT YOU CAN'T TELL THE DIFFERENCE BETWEEN LAYING DOWN FALLING DOWN. A LOT OF WORK HAS GONE INTO FALL DETECTION. IT STILL HAS SO MANY FALSE ALARMS. IT PROBABLY IS A TRADE OFF. YOU HAVE YOUR ROC CURVE AND YOU'RE PICKING A POINT TO THRESHOLD. AND YOU CAN HAVE DIFFERENT THRESH HOLDS FOR CALLING THE NEIGHBOR OR WHATEVER. BUT EVEN THAT, AS YOU KNOW -- IF YOU HAVE TO MAKE FALSE POSITIVE, ARE PEOPLE JUST FORGET IT. EVEN A PHONE CALL, IF IT'S A LEARNING EVERY DAY, YOU START WITH AND IT'S NEVER TRUE, THE NEIGHBOR WILL KIND OF TUNE OUT AS WE ALL WOULD. IT'S A REAL CHALLENGE. >> [INDISCERNIBLE] >> AS YOU SAID, THESE ARE GAMES THAT APPEAL TO OLDER ADULTS WHO ARE AT RISK OF COGNITIVE DECLINE, CAN'T BE LEARNING. IF THEY GO ON VACATION THEY FORGET TO LEARN HOW TO PLAY THE GAME. FOR YOUNGER PEOPLE OR PEOPLE WHO ARE AVID GAMERS, YES YOU CAN AS LONG AS YOU HAVE ACCESS TO THAT UNDERLYING DATA. WE HAVE TO GET GOOD TIMING INFORMATION, PUT THE GAME ON THE LOCAL COMPUTERS SO THAT WE GET ACCURATE TIMES. BECAUSE ACCURATE TIMES IS PRETTY MUCH WHAT IT'S ALL ABOUT AND UNDERSTANDING THE COGNITIVE LOAD. BUT EACH GAME REQUIRED A GOOD DEAL OF STUDY. WE DEVELOPED NINE OF THEM. BUT THEY WERE CHOSEN ALSO TO CORRESPOND WITH STANDARD NEUROPSYCHOLOGICAL TESTS SO THERE WOULD BE SOME KIND OF CORRESPONDENCE. MOTOR SPEED CAN BE MEASURED ON ALMOST ANY VIDEO GAME. BUT HOW YOU GET A GOOD MEASURE OF WORKING MEMORY WOULD HAVE TO BE WELL STUDIED AND ONE OTHER THING I FOUND IS THAT PEOPLE HAVE SUCH PERSONAL PREFERENCES FOR GAMES. SOME JUST TURN THEM OFF AND SOME THEY ARE COMPELLED TO PLAY. WE FOUND FREE CELL WAS THE MOST ADDICTIVE OF ALL OF OUR GAMES AND IT'S SOMETHING I SHOULDN'T CLAIM WE INVENTED. WE SIMPLY REPLICATED IT IN A WAY THAT WE COULD EXTRACT EACH MOVEMENT AND MAKE AN ESTIMATE OF WHETHER THEY WERE MOVING TOWARD SOLUTION OR AWAY AND HOW QUICKLY THEY GOT THE USES. THIS WOULD BE A GOOD MEASURE OF PLANNING. THAT WAS SUCH A NATURAL BEAUTIFUL GAME, PEOPLE ALREADY LOVED IT. THEY HAD VERY GRADUAL DIFFICULTIES TO ADAPT IT AND CHALLENGE ANYBODY. EVEN THE MOST NOVICE PEOPLE COULD PLAY A VERY SIMPLE GAME. BUT IT TAKES ANY OF THE VIDEO GAMES THAT YOU PLAY AND THEY GET MORE AND MORE COMPLEX. AS YOU HAVE PATHS WITHIN A GAME THAT'S SO COMPLICATED IT BECOMES MUCH MORE DIFFICULT TO DISENTANGLE AND EXTRACT THESE PURE MEASURES THAT WE WANT. BUT YES, THIS IS ONLY GOING TO WORK FOR A LITTLE WHILE. . >> [INDISCERNIBLE] >> YES. THAT IS SUCH AN IMPORTANT AREA. I WOULD LIKE TO MAKE THAT PART OF WHAT WE'RE CALLING BIG DATA TRAINING BECAUSE EVERY RESEARCHER'S GOING TO BE USING DIFFERENT SENSORS AND WE NEED META DATA ABOUT THOSE SENSORS AND WE HAVE INOPERABILITY AT A MEDIUM LEVEL BETWEEN THE CLINICAL METRIC AND THE VARIETY OF SENSORS THAT EITHER MEASURE BELOW THE LOW LEVEL OR SEMI I HIGH LEVEL. WE STARTED DOING WORK WITH -- AND DIANE COOK AT WASHINGTON STATE. PEOPLE THAT HAVE THIS TYPE OF HALL MONITORING DATA, IT NOT IDENTICAL TO WHAT'S BEING COLLECTED, BUT WITH SIMILAR CLINICAL GOALS. SO THAT'S A HUGE NEW AREA, AND WHEN WE REPRESENT DATA ABOUT A SENSOR, IT HAS TO INCLUDE THE META DATA ABOUT IT THAT DESCRIBES IT AS A LIABILITY SAMPLING RATE. ALL THE DETAILS THAT ARE REQUIRED TO DENY FUSE IT OR UNDERSTAND HOW COMPARABLE IT IS FOR SOMEBODY ELSE TO SENSOR. SO THANK YOU ALL FOR YOUR ATTENTION. I THINK WE'LL LEAVE IT THERE. I ENJOYED THE DISCUSSION. THANKS.