WELCOME BACK. I'M CATHERINE LAWS, DIRECTOR OF COMMUNICATIONS. ON BEHALF OF THE TRANS-NIH PLANNING COMMITTEE FOR THIS WORKSHOP I WANT TO WELCOME YOU TO DAY 2 OF METHODSOLOGICAL APPROACHES FOR WHOLE PERSON RESEARCH. WE HAD WONDERFUL PRESENTATIONS AND DISCUSSIONS YESTERDAY AND I LOOK FORWARD TO WHAT WE SEE AND HEAR ON DAY 2. I'LL TURN THINGS OVER TO DR. RANJAN GUPTA AND DR. MIYA, FROM THE FOGARTY INTERNATIONAL CENTER, DR. MIYA WHITAKER, OFFICE OF RESEARCH ON WOMEN'S HEALTH, WILL SERVE AS MODERATORS FOR SESSION 3, COVERING HOW TO INVESTIGATE THE IMPACT OF MULTI-COMPONENT INTERVENTIONS ON THERAPEUTIC SYSTEMS ON A -- OR ON A SINGLE OUTCOME. DR. GUPTA IS GOING TO BEGIN BY INTRODUCING OUR SPEAKERS. DR. GUPTA? >> THANK YOU. GOOD MORNING. WE HAVE A FULL AGENDA THIS MORNING WITH SIX SPEAKERS SO I WILL BE GIVE BRIEF INTRODUCTIONS, PLEASE REFER TO THE SPEAKERS' DETAILED BIOS AND ABSTRACTS ON THE WEBSITE. DR. LYNNE SHINTO, PREFER, DEPARTMENTS OF NEUROLOGY AND BE OBSTETRICS AND GYNECOLOGY AT OREGON HEALTH AND SCIENCE UNIVERSITY, PRINCIPAL INVESTIGATOR ON STUDIES THAT EVALUATE NATURAL THERAPIES FOR BRAIN AND WHOLE PERSON WELLNESS. SECOND SPEAKER DR. LYNDA POWELL, PROFESSOR OF PREVENTIVE MEDICINE AND PHARMACOLOGY, CHAIR OF DEPARTMENT OF PREVENTIVE MEDICINE RUSH UNIVERSITY MEDICAL CENTER, PRINCIPAL INVESTIGATOR OF A MULTI-SITE CLINICAL TRIAL ON LIFESTYLE INTERVENTION. >> RECORDING IN PROGRESS. >> OUR THIRD SPEAKER IS DR. LINDA COLLINS, PROFESSOR EVER SOCIAL AND BEHAVIORALLAL SCIENCE, BIOSTATISTICS, SCHOOL OF GLOBAL PUBLIC HEALTH, NEW YORK UNIVERSITY. WORKING TO DEVELOP METHODS FOR DECISION MAKING BASED ON OPTIMIZATION, COLLABORATOR ON RESEARCH TO APPLY MULTI-PHASE OPTIMIZATION STRATEGY, MOST, TO VARIOUS HELPS PROBLEMS SUCH AS HIV PREVENTION, SMOKE CESSATION, WEIGHT LOSS. FOURTH IS DR. LILIANE WINDSOR, ASSOCIATE DEAN FOR RESEARCH, ASSOCIATE PROFESSOR OF SOCIAL WORK AT UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN, RESEARCH FOCUSES ON CRITICAL CONSCIOUSNESS THEORY, MULTI-LEVEL INTERVENTION TO PROMOTE HEALTH EQUITY IN THE TREATMENT OF SUBSTANCE USE DISORDER, HIV PREVENTION, CRIMINAL JUSTICE, MARGINALIZED COMMUNITIES. FIFTH SPEAKER IS DR. MARK JENSEN, UNIVERSITY OF WASHINGTON, UNDERSTANDING EFFECTS OF PSYCHOLOGIC YAM PAIN INTERVENTIONS, HE'S EDITOR IN CHIEF OF THE JOURNAL OF PAIN. LAST BUT NOT LEAST DR. NADJA CECH, DISTINGUISHED PROFESSOR OF CHEMISTRY, UNIVERSITY OF NORTH CAROLINA GREENSBORO, DEVELOPING NOVEL APPROACHES TO SOLVING CHALLENGING PROBLEMS IN NATURAL PRODUCTS AND INTEGRATIVE MEDICINE RESEARCH. WITH THAT, THANK YOU AGAIN. DR. SHINTO, PLEASE START YOUR PRESENTATION >> THANK YOU. I'M VERY HAPPY TO BE HERE. TODAY I'M GOING TO BE TALKING ABOUT METHODS FOR DESIGNING MULTI-COMPONENT INTERVENTIONS BASED ON NATUROPATHY. I'LL DO A BRIEF OVERVIEW, FOCUSING ON INTERVENTION DESIGN USING EXAMPLES FROM TWO PILOT STUDIES, THE FIRST STUDY IS NATUROPATHIC MEDICINE IN MULTIPLE SCLEROSIS, I'M GOING TO TALK ABOUT METHODS FOR MODELING A SYSTEM OF MEDICINE. SECOND PILOT STUDY IS MEALS, MINDFULNESS AND MOVING FORWARD, M3 STUDY, FIRST EPISODE PSYCHOSIS, AND I'M GOING TO TALK ABOUT HOW WE MODELED ELEMENTS INTO A COMMUNITY CARE SYSTEM TO BE TESTED IN CLINICAL TRIALS. I'LL JUST BRIEFLY GO OVER STRENGTHS AND LIMITATIONS OF THESE MODELS. NATUROPATHIC MEDICINE. THIS IS A SCHEMATIC I PUT TOGETHER TO KIND OF HIGHLIGHT SOME THINGS ABOUT NATUROPATHIC CLINICAL PRACTICE, THREE NDs, 1, 2, 3, EACH TREATING A PATIENT. LET'S SAY IT'S MULTIPLE SCLEROSIS STAISHTS. -- PATIENTS. EACH CIRCLE WITH COLOR IS INTERVENTIONS THAT ARE BEING DELIVERED. THERE ARE MULTIPLE THERAPIES DELIVERED OVER TIME WITH LOTS OF COLORS. EACH NATUROPAST DOES NOT DELIVER THE SAME THERAPIES. ND1 HAS FOUR VISITS WITH DIFFERENT TREATMENTS AT EACH TIME. SO THIS IS HIGHLIGHTING THAT IN NATUROPATHIC MEDICINE THE MULTI-COMPONENT INTERVENTIONS ARE INDIVIDUALIZED AND CHANGE OVER TIME. THE OTHER THING I WANT TO JUST BRIEFLY TOUCH ON IS HOW DO NATUROPATHS GAUGE CLINICAL EFFECTIVENESS. THIS IS PATIENT 1, 2, 3. THIS PATIENT CHIEF COMPLAINT WE ALL KNOW ABOUT THE PRIMARY CONCERN, OH, I WANT LESS M.S. SYMPTOMS. SECOND DOESN'T EVEN CARE ABOUT M.S., WANTS WEIGHT LOSS. THE THIRD IS INTERESTED IN SLOWING DISEASE PROGRESSION. I THINK WE ALL KNOW IF WE SEE PATIENTS OVERLAID ON THE CHIEF COMPLAINT IS EVERYBODY WANTS WHOLE HEALTH. A LOT OF PEOPLE COME IN AND JUST DON'T FEEL WELL AND WANT TO FEEL BETTER. I'M JUST CALLING THAT WHOLE HEALTH, WHICH IS OVERLAID FOR MOST OF OUR PATIENTS. SO, HOW DO NATUROPATHS GAUGE TREATMENT EFFECTIVENESS? PATIENT REPORT, PHYSICAL EXAM, LABS. GIVEN THE VARIABILITY ON TREATMENTS AND MEASURE OF EFFECTIVENESS IN CLINIC, HOW CAN WE MODEL A COMPLEX SYSTEM THAT IS VALID REPRESENTATION OF A SYSTEM AND WILL THIS MODEL BE REPRODUCIBLE AND GENERALIZABLE? WILL THE MODEL REDUCE BIAS? I'M GOING TO TALK ABOUT THE NATUROPATHIC STUDY IN MS. WHAT DOES BEST PRACTICE MEAN? THESE ARE DEEMED TO BE THE MOST VALUABLE OR BENEFICIAL FOR PEOPLE WITH M.S. AND HOW DID WE MEASURE THAT OR FIND THAT OUT? SO, THE DESIGN OF THE BEST PRACTICE NATUROPATHIC TREATMENT MODEL FOR US TO ADDRESS MODEL VALIDITY IS THE MODEL, A REPRESENTATION OF THIS SYSTEM. WE FIRST SURVEYED NATUROPATHS IN PRACTICE TO IDENTIFY TREATMENTS THEY REPORTED AS VALUABLE FOR M.S. AND THEN WE CONVENED A DELPHI PANEL OF M.S. CLINICAL EXPERTS TO DESIGN THE TREATMENT MODEL TO BE TESTED IN CLINICAL TRIAL. I JUST WANT TO SHOW, THIS IS A SURVEY DATA FROM THE SURVEY OF NATUROPATHS. THREE TREATMENTS THAT NATUROPATH CONSIDERED MOST VALUABLE FOR M.S., DIET, ESSENTIAL FATTY ACIDS, VITAMINS AND MINERALS, A MAILED SURVEY OF A THOUSAND PEOPLE, 42% RESPONSE RATE. I ALSO WANT TO POINT OUT WE DID ASK ABOUT WHAT -- HOW NATUROPATHS GAUGE TREATMENT EFFECTIVENESS. 88% PATIENT REPORT, 60% SAID THERAPIES WERE VERY EFFECTIVE FOR IMPROVING QUALITY OF LIFE. SO THE DELPHI PANEL, THE DELPHI PANEL USED ND SURVEY RESULTS, CLINICAL EXPERTISE AND SCIENTIFIC INFORMATION. THEY RECEIVED WRITTEN INSTRUCTION AND SURVEY DATA, ONE MONTH PRIOR TO THE PANEL MEETING, SO THEY WERE PREPARED. IT CONVENED IN PORTLAND AT OHSU, A DAY LONG, OBJECTIVE WAS TO COME UP CAN TREATMENT INTERVENTION FOR CLINICAL TRIAL. THIS IS THE PANEL. I JUST WANT TO SHOW YOU THE THOUGHT PROCESS THAT WENT INTO THE DECISIONS FOR THE INTERVENTION. AND THIS IS THE INTERVENTION THAT WAS TESTED IN TRIAL. WE HAD DIET, IT WAS INDIVIDUALIZED TO FOUR LEVELS. DIETARY ADVICE TO DOING ELIMINATION/CHALLENGE DIET. FIVE NUTRITIONAL SUPPLEMENTS. B-12 INTRAMUSCULAR ONCE A WEEK AND COUNSELING. THIS WAS A RANDOMIZED PILOT TRIAL, RANDOMIZED TO THREE GROUPS, USUAL CARE, NATUROPATHIC PLUS USUAL CARE, EDUCATION VISITS WITH M.S. NURSE PLUS USUAL CARE. M.D. AND NURSE VISITS WERE NEXT FOR FREQUENCY, 45 PARTICIPANTS, 15 PER GROUP, THIS WAS A PILOT. ALL PARTICIPANTS HAD A DEFINITE DIAGNOSIS OF RELAPSING REMITTING M.S. SIX MONTHS TREATMENT INTERVENTION. BLINDED ASSESSMENTS, BLINDED DATA ANALYSIS, AND THE PRIMARY OUTCOME WAS SF-36 QUALITY OF LIFE INFORMED FROM THE SURVEY. STRENGTHS OF MODEL INCLUDED LIMITED INDIVIDUALIZATION, WAS MULTI-COMPONENT, WE HAVE MODEL VALIDITY INFORMED BY NDs, REPRODUCIBLE INTERVENTION, RANDOMIZED. LIMITS ARE IT LIMITS BREADTH OF NATUROPATHIC CARE, ARE THESE THE BEST TREATMENTS FOR M.S. IT WAS A COLLECT DELPHI PANEL, SO WE NEVER KNOW IF THESE ARE THE BEST TREATMENTS. IT WAS NOT DOUBLE BLIND PLACEBO CONTROLLED, THERE WAS ALWAYS A QUESTION DID WE CHOOSE THE RIGHT CONTROL GROUP WITH THE M.S. NURSE INTERVENTION. I'M GOING TO SWITCH GEARS AND TALK ABOUT THE M3 STUDY. THE QUESTION HERE WAS CAN WE FIND ELEMENTS OF NATUROPATHIC MEDICINE. A LOT OF NATUROPATHS DO DIET, EXERCISE COUNSELING ON STRESS REDUCTION, THIS LIFESTYLE MANAGEMENT, LIFESTYLE COUNSELING IN THEIR PRACTICE. SO, THIS IS WHAT WE USE FOR M3. ELEMENTS OF NATUROPATHIC MEDICINE INTEGRATED INTO EXISTING SYSTEM, IN THIS CASE THE DESIGN OF THE INTERVENTION WAS INFORMED BY THE SYSTEM, EASA, WHICH I'LL TALK ABOUT IN THE NEXT SLIDE SO WE CAN STREAMLINE INTEGRATION. SO, WHAT IS EASA? IT IS THE EARLY ASSESSMENT SUPPORT ALLIANCE, THESE ARE COMMUNITY SPECIALTY CARE CLINICS THAT SCREEN AND TREAT PEOPLE WITH EARLY SIGNS OF PSYCHOSIS IN TEENAGERS AND YOUNG ADULTS, IN THE STATE OF OREGON, FUNDED BY THE STATE, PARTICIPANTS FROM 15 TO 25 YEARS, ONCE ENROLLED IN. EASA PROGRAM THEY STAY IN FOR TWO YEARS. SO, I ALSO WANTED TO SHOW YOU THE EASA TEAM MODEL OR CLINIC DYNAMIC HERE. THIS IS ITS OWN SYSTEM. THERE'S A PROGRAM DIRECTOR, TEAM LEADER, PSYCHIATRIC PRESCRIBER, SOCIAL WORKER, EMPLOYMENT SPECIALIST, NURSE COUNSELOR, SOME HAVE OCCUPATIONAL THERAPISTS. OUR INTEREST WAS TO BRIDGE M3 WITH THIS EXISTING CLINIC IN THE PORTLAND METRO AREA. SO WE HELD A STAKEHOLDER MEETING TO GET FEEDBACK, AND TO BUILD BRIDGES TO FIND OUT WHAT EASA'S NEEDS WERE. HERE ARE PARTICIPANTS. YOU SEE A LOT OF LEADERSHIP, YOUNG PEOPLE WHO ARE CLIENTS. WE HAD OCCUPATIONAL, OUR RESEARCH TEAM AT OHSU, NEUROLOGY, PSYCHOLOGY, NATIONAL UNIVERSITY OF NATURAL MEDICINE WAS INVOLVED. SO WHAT DID WE HEAR BACK? LIKE MINDFULNESS AND YOGA ACTIVITIES, LIKED MORE ACTIVE DIET AND NUTRITION ACTIVITIES, MORE DOING, LESS LECTURING. WANTED EXERCISE TRAINING THAT COULD DO AT HOME OR IN A ROOM RATHER THAN GOING OUT. EXPAND STUDY PARTNER TO A FRIEND, NOT JUST PARENTS. WEEKEND LESS TIME BECAUSE A LOT OF CLIENTS HAVE SCHOOL OR WORK ON WEEKDAYS. PLACE NEEDS TO BE EASY TO GET TO. TRANSPORTATION CAN BE CHALLENGING WITH PARTICIPANTS IN THE PROGRAM. SHARE DATA COLLECTED. SO THIS WAS A REQUEST FROM THEEASA TEAM LEADERS, TAKING BEHAVIORAL ASSESSMENTS, SAFETY LABS, AND THE TEAM LEADERS WANTED US TO SHARE THAT DATA AND WE DID. SO THE INTERVENTION DESIGN WAS MINDFULNESS PRACTICE, ACTIVE PHYSICAL ACTIVITY, ACTIVE DIET AND NUTRITION, FACILITATED GROUP DISCUSSION AT THE CLOSE OF EACH SESSION. THE GREEN IS JUST HIGHLIGHTING THE PARTS OF THE DESIGN OF THE INTERVENTION THAT WAS INFORMED BY THE STAKEHOLDER MEETING. SO OF COURSE THE INTERVENTION, IT WAS A SIX-WEEK PROGRAM, ONCE A WEEK, ON SATURDAYS, FOR 4 HOURS. AND WE INCLUDED FOR STUDY PARTNERS FAMILY OR A FRIEND. OUTCOMES AT BASELINE 6 WEEKS AND 6 WEEKS POST-THERAPY AT 12 WEEKS, COLLECTED QUANTITATIVE AND QUALITATIVE MEASURES. BECAUSE THIS WAS A PILOT, THE REAL QUESTION WAS WOULD PEOPLE ACTUALLY DO THIS FOR 6 WEEKS. ADHERENCE WAS THE PRIMARY OUTCOME, A PRIORI ATTENDING FOUR OUT OF SIX SESSIONS. >> THREE MINUTES. >> THANK YOU. SO, WE DID FIND THAT THE INTERVENTION HAD HIGH ATTENDANCE. IT WAS A PILOT 15 OUT OF 17 OR 88% ATTENDED. WE TOOK QUALITATIVE MEASURES TO BETTER UNDERSTAND PROGRAM ELEMENTS THAT CONTRIBUTED TO HEALTHY BEHAVIOR CHANGES AND ALSO BARRIERS TO CHANGES, 6 WEEKS POST-PROGRAM. WHAT WE FOUND FROM QUALITATIVE INTERVIEWING WAS THAT ALL COMPONENTS OF THE M3 STUDY WERE PERCEIVED AT BENEFICIAL, ENJOYED NON-MEDICALIZED APPROACH FOCUSED ON LIFESTYLE, DESTIGMATIZING. BARRIERS HARD TO SUSTAIN HEALTHY DIET, COST OF HEALTHY FOOD, GYM MEMBERSHIP, AND WE ASKED ABOUT BEHAVIOR CHANGE SUSTAINABILITY AND MANY WANTED BOOSTER SESSIONS AFTER THE 6 WEEKS. I JUST WANT TO READ A QUOTE FROM ONE OF THE PARTICIPANTS ABOUT MINDFULNESS, 6 WEEKS POST-PROGRAM, IT HELPS ME WHEN I GO SHOPPING, WHEN I GET OVERWHELMED. I JUST USE IT TO HELP ME GET BACK IN THE MOMENT AND NOT FREAK OUT AND WIG OUT AND STUFF. STAKEHOLDER INFORMED, REPRODUCIBLE WITHIN A SYSTEM, QUALITATIVE ANALYSIS INFORMS ON INTERVENTION COMPONENTS, THIS WAS NOT A DOUBLE BLIND PRACTICE CONTROLLED STUDY, WE'VE NOT ESTABLISHED TREATMENT FIDELITY, NON-RANDOMIZED, CONVENIENCE CONTROL, AND, AGAIN, FOR FUTURE TRIALS THERE'S A QUESTION ABOUT WHAT IS A GOOD CONTROL GROUP. I WANT TO THANK AND JUST ACKNOWLEDGE PEOPLE INVOLVED IN THE NATUROPATHIC M.S. STUDY, M3 STUDY, REFERENCES ON SYSTEM MODELS OF NATUROPATHIC MEDICINE. THANK YOU. >> THANK YOU, DR. SHINTO. I BELIEVE WE'LL MOVE NOW TO DR. POWELL. DR. POWELL, WOULD YOU LIKE TO SHARE YOUR SLIDES PLEASE. YOU NEED TO UNMUTE, DR. POWELL. >> SUCCESS. >> THE SLIDES LOOK GOOD. >> GOOD MORNING, EVERYBODY. LET'S SEE. I WANT TO ADVANCE. ALL RIGHT. I WANT TO TALK FOR A MOMENT ABOUT A MOVEMENT THAT'S HAPPENING IN APPLIED BEHAVIORAL MEDICINE. THE MOVEMENT IS TOWARD PROGRESSIVE TRANSLATIONAL SCIENCE. THE BEHIND -- THE QUESTION, HOW COME TREATMENTS DON'T GET INCORPORATED INTO CLINICAL PRACTICE? THE MOST GERMANE REASON FOR OUR DISCUSSION THIS MORNING IS THAT WE NEED TO BE MORE CRITICAL ABOUT OUR SCIENCE. PROGRESSIVE TRANSLATIONAL SCIENCE MOVEMENT IS ABOUT SPENDING MORE TIME OBTAINING MULTIPLE PERSPECTIVES, FOCUS MORE ON CLINICAL SIGNIFICANCE THAN WE HAVE IN THE PAST, USE A VARIETY OF METHODS, AND MOST IMPORTANTLY TO PROGRESS SCIENCE AND STICK WITH THINGS THROUGH FAILURE. I WANT TO SHOW HOW BASIC PRINCIPLES WERE APPLIED TO THE STORY OF ELM, A LIFESTYLE PROGRAM. IT FOCUSING ON DIET, PHYSICAL ACTIVITY, AND SOME MEASURE OF STRESS, TARGET OF STRESS. METABOLIC SYNDROME IS HAVING THREE OUT OF FIVE CARDIOMETABOLIC RISK FACTORS, INCREASED IN THE LAST TEN YEARS, TEN YEARS AGO ONE-QUARTER OF AMERICAN POPULATION HAD THE METABOLIC SYNDROME. CURRENTLY IT'S ONE-THIRD. AND THE WAY WE TREAT IT IN CLINICAL PRACTICE TO FOCUS ONLY ON TREATING INDIVIDUAL COMPONENTS WITH (INAUDIBLE). THE QUESTION THAT WAS IMPORTANT TO ME TEN YEARS AGO, TEN YEARS AGO, WAS THIS PROBLEM OF THE CHECK MARK. WE DO BEHAVIORAL INTERVENTIONS AND THEY WORK REALLY WELL IN THE SHORT TERM, BUT THEY ARE JUST NOT SUSTAINED OVER TIME. HOW DO WE GET TO SUSTAINABILITY, DO SOMETHING ABOUT METABOLIC SYNDROME AND GET TO SUSTAINABILITY? THE FIRST EFFORT WE ENGAGED IN WAS TO PULL TOGETHER A MULTI-DISCIPLINARY TEAM OF CLINICIANS, RESEARCHERS, APPLIED BEHAVIORAL SCIENTISTS AND RESEARCHERS AND TALKED FOR A YEAR IN A SMALL GROUP. ONE OF THE THINGS THAT'S MOST IMPORTANT THAT CAME OUT OF THAT WE WOULD LISTEN TO STORIES FROM CLINICIANS ABOUT SOME OF THEIR PATIENTS EATING, BINGE EATING IN RESPONSE TO PROBLEMS WITH THEIR TEENAGER. WE NEEDED TO INCLUDE STRESS AS A COMPONENT WAS THE CONSENSUS, IMPORTANT AT THAT MOMENT IN TIME, TEN YEARS AGO, WHEN WE TALKED ABOUT BEHAVIORS FOR METABOLIC SYNDROME. WE ONLY TALKED ABOUT DIET AND PHYSICAL ACTIVITY. THIS WAS SOMETHING THAT WAS PRETTY EXCITING, WE CAME UP WITH INTERVENTION. HERE IS A PICTURE OF WHAT IT LOOKS LIKE, OR LOOKED LIKE TEN YEARS AGO. IT'S COMPLEX. YOU CAN SEE WE HAVE THREE COMPONENTS, IT'S A HABIT-BASED INTERVENTION TRYING TO HELP PEOPLE CHANGE THEIR HABITS IN A WAY THAT ARE SUSTAINED OVER TIME. YOU CAN SEE THERE ARE NINE HABITS HERE SO IT'S A COMPLEX INTERVENTION. THE THING THAT'S REALLY IMPORTANT ABOUT THIS IS WE STARTED WITH OUR PRIMARY OUTCOME. WE'RE TRYING TO DEAL WITH THE METABOLIC SYNDROME. WE'RE TRYING TO GET A SUSTAINED IMPROVEMENT IN THE METABOLIC SYNDROME. SO UP FRONT WE SAID THE MAIN THING WE'RE GOING TO BE LOOKING AT IS WHETHER OR NOT WE CAN GET REMISSION OF THE METABOLIC SYNDROME IN 50% OF OUR PEOPLE. HOW DID WE GET TO CLINICALLY SIGNIFICANT CUT POINT OF 50%? THAT'S THE RECORD WE HAVE WITH DRUGS. 50% OF PEOPLE DO NOT TAKE DRUGS AS PRESCRIBED. SO IF WE COULD DO AS WELL AS WHAT THE DRUGS WERE DOING, WE'D BE HAPPY. SO, THE VERY FIRST STUDY WE DID WAS PROOF OF CONCEPT STUDY, WHY DID WE DO THIS? THIS IS A QUASI-EXPERIMENTAL STUDY, DOES NOT HAVE A STATISTICAL ANALYSIS IN IT, IT'S A PROOF OF CONCEPT WHERE WE'RE LOOKING AT OUTCOME. WE'VE GOT COMPLEX INTERVENTION AND THE QUESTION IS DOES IT HAVE ANY KIND OF CLINICAL SIGNAL AT ALL? SO IF WE HAD 26 PEOPLE WE FOLLOWED FOR 2 1/2 YEARS, GOAL TO GET TO 50% SUSTAINED REMISSION OF THE METABOLIC SYNDROME. YOU CAN SEE FROM THE PICTURE WE GOT TO 61%, AT 6 MONTHS, END OF INTENSIVE PHASE. BY THE TIME WE GOT TO 2.5 YEARS WE HUNG IN THERE WITH ABOUT A 54% REMISSION IN THE PEOPLE THAT WERE IN THE STUDY. SO THAT WAS THE GOOD NEWS. NEXT STEP, LET'S LOOK AT PEOPLE WHO BY THE TIME WE GET TO 2 1/2 YEARS SOME WERE IN REMISSION OF METABOLIC SYNDROME, SOME NOT. WE KNOW THAT STANDARD OF CONVENTIONAL RISK FACTORS, DIET AND PHYSICAL ACTIVITY, DISCRIMINATED. WE HAD A MEASURE OF DEPRESSION THAT WAS SUPPOSED TO EVALUATE STRESS COMPONENT, LO AND BEHOLD WE FOUND NOBODY WAS DEPRESSED. SO THEREFORE WE HAD A CEILING EFFECT. IN OTHER WORDS, THE STATUS OF WHETHER YOU WERE IN REMISSION OR NOT AT 2.5 YEARS WAS NOT A FUNCTION OF DEPRESSION. OUR PATIENTS WERE NOT DEPRESSED. SO THAT HELPED US UNDERSTAND WE HAVE TO DUMP THE STRESS COMPONENT OR FIND A BETTER MEASUREMENT. WHY DON'T WE ASK THE PATIENTS, ASK THE PARTICIPANTS THEMSELVES, WHAT THEY THINK IS MOST IMPORTANT IN SUSTAINING CHANGE. WE GOT ALL OF THE 26 PEOPLE BACK, ONCE AGAIN. AND WE ASKED THEM WHAT WAS MOST IMPORTANT IN HELPING YOU SUSTAIN CHANGE? AND THE MOST IMPORTANT CHARACTERISTIC WAS NON-REACTIVE, A MINDFULNESS HABIT. THEY WERE SAYING TO HOLD BACK FROM EMOTIONAL REACTIONS TO FOOD, OPPORTUNISTIC FOOD, EMOTIONAL REACTIONS THAT WOULD TRIGGER EATING, AND EMOTIONAL REACTIONS IN TERMS OF INTERPERSONAL INTERACTIONS, THAT'S MOST IMPORTANT. SO OKAY, MAYBE WE'VE GOT TO BE MORE PRECISE ABOUT OUR MINDFULNESS COMPONENT. SO WE WENT TO THE NEXT STEP AND WE SAID, OKAY, LET'S TINKER A BIT. LET'S SEE IF WE HAVE A NEW GROUP OF PEOPLE, AND PUT THEM IN A SMALL GROUP AND WORK WITH THEM OVER THE COURSE OF SIX MONTHS, WHETHER OR NOT THIS MINDFULNESS THING IS GOING TO COME OUT AGAIN. ESSENTIALLY WE DID A HUMAN-CENTERED DESIGN WHERE WE WATCHED PEOPLE EXERCISE TOGETHER, PREPARE FOOD TOGETHER, AND EAT TOGETHER OVER THE COURSE OF 12 WEEKS. SO THIS IS A PICTURE OF OUR GROUP. AND ONE OF THE THINGS MOST POWERFUL ABOUT THIS IS FIRST FIRST OF ALL THEY ATE FAST, THEY WOULD EMPTY THE PLATE QUICKLY. THE OTHER THING THAT WAS REALLY IMPORTANT, THEY WERE VERY JUDGMENTAL. I DON'T LIKE VEGETABLES. OH, I REALLY LIKE HAMBURGERS. SO WATCHING THEM FOR A LONG PERIOD OF TIME REALLY GOT US TO THE POINT WHERE WE THOUGHT, OKAY, MAYBE WE SHOULD HAVE NOT JUST EMOTIONAL REACTIVITY AS A COMPONENT BUT ALSO THIS OTHER COMPONENT THAT'S PART OF MINDFULNESS, SENSORY AWARENESS. IF WE COULD GET PEOPLE TO FOCUS ON THE TASTE OF FAT, TASTE OF SALT, FOCUS ON COLOR OF VEGETABLES, WE MIGHT HAVE A BETTER SHOT AT IMPROVEMENT. SO THAT GOT US TO THE POINT WHETHER WE COULD NOW SIMPLIFY PROGRESSIVE TRANSLATIONAL SCIENCE, IT'S ALL ABOUT SIMPLIFICATION, WITH EACH EFFORT, EACH STUDY THAT YOU DO, YOU PUSH TOWARDS SIMPLIFICATION. NOW WE HAVE TWO CONVENTIONAL HABITS WHICH ARE STANDARD. WE HAD THE MINDFULNESS COMPONENT. THAT'S SIMPLIFIED. WE'VE GOT TWO PIECES OF MINDFULNESS, BOTH OF WHICH HAVE BEEN SUPPORTED IN THE LITERATURE AS BEING RELATED TO CARDIOMETABOLIC ABNORMALITIES, AND WE HAVE EVERYTHING QUANTIFIED. SO THAT WE'RE READY NOW, WE HAVE QUANTIFICATION OF METABOLIC SYNDROME, AND WE HAVE QUANTIFICATION OF MINDFUL AWARENESS, SO WE'RE READY NOW WITH THIS HYPOTHESIZED PATHWAY TO GET TO THE PLACE WE'RE AT RIGHT NOW WHICH IS AN EFFICACY TRIAL, ASKING THE QUESTION OF HOW MINDFULNESS, HOW THE COMPONENT OF MINDFULNESS CONNECTS TO SUSTAINED REMISSION OF METABOLIC SYNDROME. THE DESIGN OF THIS EFFICACY TRIAL IS WE HAVE AN INTENSIVE INTERVENTION THAT GOES ON FOR SIX MONTHS. FOCUSING ON DEVELOPMENT OF FOUR HABITS TWO CONVENTIONAL AND TWO MINDFULNESS AND A MAINTENANCE PERIOD FOR 4 MONTHS WHERE WE HAVE SMALL NUMBER OF MAINTENANCE SESSIONS WHERE WE HOPE TO SUSTAIN THE BEHAVIOR THAT WE SAW AT THE END. OVER THIS TIME WE'RE ASSESSING HABITS AND THESE FOUR ASSESSMENT PERIODS AND WE'RE ASSESSING METABOLIC SYNDROME AGAIN AT 24 MONTHS BECAUSE THAT IS THE PRIMARY OUTCOME. THE WAY WE ASSESS HABITS IS USE OF ECOLOGICAL MOMENTARY ASSESSMENT, WE WANT TO GET A REALLY PRECISE MEASURE OF HOW FREQUENTLY PEOPLE ARE ENGAGING IN THESE MINDFULNESS HABITS AS THEY GO THROUGHOUT THE DAY SO THE PROTOCOL IS TO ASSESS AND REASSESS PEOPLE EVERY 4 HOURS, AND ASK THE QUESTION, DID YOU USE THIS MINDFULNESS BEHAVIOR SINCE WE LAST ASKED THE QUESTION? SO IT GIVES US A MUCH MORE UPDATED MEASURE OF WHAT IS AND IS NOT BEING DONE THROUGHOUT THE DAY, FOR 7 DAYS TO GET REPRESENTATIVENESS. THIS IS A SITUATION TO ANSWER THE QUESTION I THINK IS A BURNING QUESTION, COMPONENTS QUESTION. DOES MINDFULNESS PREDICT 24 MONTHS SUSTAINED REMISSION OF METABOLIC SYSTEM, ALL HAVE METABOLIC SYSTEM AT BASELINE. DO THE TARGETS INTERACT? SO WE'RE NOW IN A POSITION TO ANSWER THAT QUESTION BECAUSE WE CAN LOOK AT PREDICTIVE VARIABLE, DOWN HERE, BASED ON WHERE YOU ARE IN TERMS OF HABITS, SIX MONTHS, 15 MONTHS. WE'LL HAVE PEOPLE WHO ARE -- YEAH, THEY HAVE AUTOMATIC HABITS 6 MONTHS, SUSTAINED AT 15 MONTHS. SOME DIDN'T CHANGE AT 6 MONTHS OR 15 MONTHS. WE ALSO HAVE INTRIGUING COUPLE SUBGROUPS, I CHANGED HERE BUT I LOST IT BY 24 MONTHS. OR A DELAYED EFFECT, I DIDN'T CHANGE HERE BUT BY THE TIME I GOT TO 15 MONTHS, SO IT'S A DELAYED EFFECT. ONE OF THE THINGS THAT'S REALLY IMPORTANT FOR US TO START DOING IS STOP JUST LOOKING AT ONE MOMENT IN TIME TO EVALUATE OUR TREATMENT EFFECTIVENESS, SEE HOW THE CHANGES EVOLVE AS WE GO FORWARD. THE ANALYSIS HERE IS NOT ANYTHING SEXY OR CUTE. IT'S JUST A MIXED EFFECTS LOGISTIC REGRESSION WHERE WE'RE LOOKING AT MAIN EFFECTS AND INTERACTION BETWEEN MINDFULNESS AND CONVENTIONAL HABITS. >> TWO MINUTES. >> IN SUMMARY, IN SUMMARY, I AM TRIPPING TO MAKE THE -- TRYING TO MAKE THE ARGUMENT TO DETERMINE THE IMPACT OF A MULTI-COMPONENT INTERVENTION IT IS HELPFUL. I BROUGHT FOUR FRIENDS TO MAKE THE CASE FOR ME. HERE ARE MY FOUR FRIENDS. NUMBER ONE, DONALD CAMPBELL. GET MULTIPLE PERSPECTIVES, DON CAMPBELL SAYS TO THE EXTENT THAT EXPERIMENTS REVEAL NATURE TO US, IT'S STILL A CLOUDED WINDOWPANE. WE'RE NOT GOING TO GET THERE IF WE TALK TO RESEARCHERS. TALKING TO CLINICIANS, TO PATIENTS THEMSELVES, USING VARIETY OF DIFFERENT PERSPECTIVES, I THINK, GETS US WHERE WE WANT TO BE WHICH IS TO HAVE AN IMPACT ON HEALTH. MY SECOND FRIEND IS PAUL, ONE OF OUR MOST EMINENT PHILOSOPHERS OF SCIENCE. THIS FOCUSES ON WHAT WE SEE IN THE MOVEMENT AS EXCESSIVE RELIANCE ON RCT RATHER THAN USING MULTIPLE METHODS TO GET TO WHERE WE WANT TO GO. AND TALKING ABOUT INFANTICIDE, IF YOUR TREATMENT IS BORN IN HOSTILE ENVIRONMENT, PROTECT EARLY DEVELOPMENT AND NOT CONTRIBUTE TO PREMATURE DEATH, CONTRIBUTE TO PREMATURE DEATH WHEN WE JUMP QUICKLY TO RANDOMIZED DESIGN. MY THIRD FRIEND IS SIR RONALD FISCHER WHO DEVELOPED THE HYPOTHESIS TESTING AND P-VALUE METHODS THAT WE USE ALL THE TIME. HE'S SAYING, LOOK, IT'S JUST A TOOL. DON'T USE IT AS THE ONLY WAY YOU'RE GOING TO USE -- DETERMINE WHETHER SOMETHING WORKS OR NOT. IN HIS WORDS, NO SCIENTIST HAS (INDISCERNIBLE) LEVEL TO REJECT THE NULL HYPOTHESIS, GIVES HIS MIND TO EACH PARTICULAR CASE IN LIGHT OF EVIDENCE AND IDEAS. WHEN HE TALKS ABOUT IN LIGHT OF EVIDENCE AND IDEAS, HE'S TALKING ABOUT CLINICAL SIGNIFICANCE, P-VALUES ARE JUST ONE TOOL AND PERHAPS ACCORDING TO BRAD, THE LOWEST RUNG ON THE CAUSAL LADDER. MY FINAL FRIEND IS KARL POPPER, THE MOST INFLUENTIAL PHILOSOPHER OF SCIENCE THAT WE HAVE. HE SAYS THAT, MY WORDS, STOP ONE FELL SWOOP EVALUATIONS. ONE EVALUATION IS NOT GOING TO GET US WHETHER WE NEED TO GET IF WE'RE TRYING TO IMPROVE HEALTH. HE SAYS, STICK YOUR NECK OUT WITH A QUANTIFIED HYPOTHESIS, THE WRONG VIEW OF SCIENCE PORTRAYS ITSELF IN CRAVING TO BE RIGHT. SO PART OF THIS MOVEMENT IS FOCUSING ON WELCOMING FAILURE AS AN OPPORTUNITY FOR GROWTH AND PROGRESSION. AND THAT'S IT. IF YOU DON'T WANT TO LISTEN TO ME, LISTEN TO MY FRIENDS. >> THANK YOU, DR. POWELL. DR. COLLINS, WOULD YOU LIKE TO BEGIN THE SHARE ON YOUR SLIDES? >> OKAY. IT'S GREAT TO BE HERE. I'LL BE TALKING TODAY ABOUT -- >> DR. COLLINS, WE'RE NOT SEEING YOUR SLIDES. >> OH, I'M SORRY. HOW'S THAT? >> NOW WE HAVE THEM. THANK YOU. THAT'S IT. >> ALL RIGHTY. I'M GOING TO TALK TODAY ABOUT ACHIEVING INTERVENTION EASE, USING MULTI-PHASE OPTIMIZATION STRATEGY, OR MOST. I'M GOING TO EXPLAIN IN A MOMENT, INTERVENTION EASE. I'D LIKE TO SUGGEST TODAY THAT TO ACHIEVE GREATER PUBLIC HEALTH IMPACT WE SHOULD BE FROM THE VERY BEGINNING OF INTERVENTION DEVELOPMENT CONSIDERING AFFORDABILITY, SCALABILITY, EFFICIENCY ALONG WITH EFFECTIVENESS. I'M GOING TO DEFINE THE TERMS IN A MOMENT. I ALSO WANT TO SUGGEST THAT WE SHOULD BE OPTIMIZING INTERVENTIONS TO ACHIEVE INTERVENTION EASE. THIS IS A SCHEMATIC OF THE PACKAGE APPROACH WHICH EVERYONE HERE IS PRETTY ACCUSTOMED TO. THERE'S SOME NUMBER OF INTERVENTION COMPONENTS, HERE I HAVE FIVE, OF COURSE THERE COULD BE ANY NUMBER OF INTERVENTION COMPONENTS. THEY ARE ASSEMBLED IN A TREATMENT PACKAGE. THAT INTERVENTION AS A PACKAGE IS EVALUATED TYPICALLY BY MEANS OF RCT, COULD HAVE A THIRD ARM, BUSINESS AS USUAL AND HAS BEEN FOR MORE THAN 30 YEARS OF INTERVENTION SCIENCE. NOW, THIS IS A REALLY GREAT WAY, A GREAT APPROACH FOR ASSESSING PERFORMANCE OF AN INTERVENTION AS A PACKAGE. BUT IT'S NOT A VERY GOOD WAY TO LOOK AT EFFECTIVENESS OF THE INDIVIDUAL COMPONENTS OF INTERVENTIONS. AND OF COURSE UNLESS WE LOOK AT INDIVIDUAL COMPONENTS OF INTERVENTIONS WE REALLY CAN'T CONSIDER MATTERS LIKE AFFORDABILITY, SCALABILITY, EFFICIENCY, BECAUSE WE DON'T KNOW WHAT IS THE RIGHT COMBINATION OF COMPONENTS TO GIVE US MOST AFFORDABILITY, SCALABILITY AND BEST EFFICIENCY. LET'S REVIEW EACH IN TURN. I'M GOING TO DEFINE AFFORDABILITY AS THE EXTENT TO WHICH THE INTERVENTION IS EFFECTIVE WITHOUT EXCEEDING WHATEVER BUDGETARY CONSTRAINTS MAY BE OPERATING. MAYBE IT'S KNOWN THAT A PARTICULAR COMMUNITY WILL SPEND UP TO A PARTICULAR INSURANCE ORGANIZATION OR SOMETHING LIKE THAT IS WILLING TO PAY UP TO $500 PER PERSON FOR A PARTICULAR INTERVENTION, AND THAT WOULD BE A BUDGET OR UPPER LIMIT ON IMPLEMENTATION COST. THERE'S LITTLE INCENTIVE IN THE U.S., DIFFERENT IN OTHER COUNTRIES, BUT LITTLE INCENTIVE IN THE U.S. FOR ACADEMIC INTERVENTION SCIENTISTS TO THINK ABOUT HOW MUCH THEIR INTERVENTION WILL COST TO IMPLEMENT, AND WHO WOULD BE WILLING TO PAY. AND IN FACT AT LEAST IN MY EXPERIENCE IN TALKING WITH INTERVENTION SCIENTISTS MAYBE DON'T KNOW WHO WOULD PAY FOR THEIR INTERVENTION IF IT GOES TO SCALE, AND EXACTLY WHERE THE MONEY WOULD COME FROM. OF COURSE, IF INTERVENTION IS DEVELOPED AND FOUND TO BE EFFECTIVE, BUT IT'S TOO EXPENSIVE, THE NEXT STEP IS SOMEHOW WE HAVE TO MAKE IT LESS EXPENSIVE, WE HAVE TO ALTER INTERVENTION, TYPICALLY TO ACHIEVE THAT, THE OBVIOUS WAY IS TO REMOVE SOME COMPONENTS OF THE INTERVENTION, BUT THIS IS RISKY BECAUSE IF YOU EVALUATED THE INTERVENTIONS IT'S BEEN DEVELOPED AND EVALUATED USING CLASSICAL TREATMENT PACKAGE APPROACH YOU DON'T KNOW WHICH COMPONENTS TO REMOVE, WHICH WOULD BE THE BEST TO REMOVE, TO END UP WITH INTERVENTION THAT GIVES YOU THE BEST PREDICTED OUTCOME WITHIN A PARTICULAR BUDGET. SO THE CONSEQUENCES OF THIS KIND OF THINKING ARE THAT MANY INTERVENTIONS WILL NEVER BE IMPLEMENTED BECAUSE THEY ARE TOO EXPENSIVE OR THEY END UP BEING IMPLEMENTED BUT AD HOC MODIFICATIONS HAVE BEEN MADE TO THE INTERVENTION THAT UNDERMINE ITS EFFECTIVENESS. NOW, LET'S THINK ABOUT SCALABILITY, WHICH OF COURSE IS VERY CLOSELY RELATED TO SUSTAINABILITY. EXTENT TO WHICH INTERVENTION CAN BE IMPLEMENTED WIDELY WITH FIDELITY, I MEAN IMPLEMENTED IN ANY INTENDED SETTING, EXACTLY THE WAY IT WAS EVALUATED. WITH NO NEED FOR AD HOC MODIFICATIONS. PREVAILING LOGIC TODAY IN INTERVENTION DEVELOPMENT AND EVALUATION IS FIRST ESTABLISH EFFECTIVENESS. AND THEN START WORRYING ABOUT SCALABILITY. WHAT IF THE INTERVENTION THAT HAS BEEN DEMONSTRATED TO BE EFFECTIVE IS NOT SCALABLE? THIS COULD HAPPEN FOR A NUMBER OF REASONS. IT COULD BE TOO COMPLICATED TO IMPLEMENT IN COMMUNITY SETTING, MAY REQUIRE MORE STAFF ATTENTION THAN CAN BE COMPARED IN A PARTICULAR SETTING. MAY NOT BE COST EFFECTIVE AND MAY NOT BE OF MUCH INTEREST. ALTERATIONS ARE NEEDED. AGAIN, SAME ISSUE. IT'S RISKY TO MAKE ALTERATIONS AN INTERVENTION IS DEVELOPED USING CLASSICAL TREATMENT APPROACH BECAUSE YOU DON'T KNOW WHICH COMPONENTS TO REMOVE AND WHICH TO KEEP. AND THE CONSEQUENCES ARE THE SAME AS BEFORE, MANY INTERVENTIONS END UP NEVER BEING IMPLEMENTED EVEN THOUGH THEY HAVE BEEN DEMONSTRATED TO BE EFFECTIVE, OR IF IMPLEMENTED WITH UNKNOWN UNDERMINED EFFECTIVENESS BECAUSE OF AD HOC MODIFICATIONS. I WANT TO TALK ABOUT EEFFICIENCY. THIS IS THE EXTENT TO WHICH INTERVENTION AVOIDS WASTING TIME, MONEY, OR OTHER VALUABLE RESOURCES. AND THE WAY TO ACHIEVE AN EFFICIENT INTERVENTION IS TO DEVELOP AN INTERVENTION THAT IS MADE UP COMPLETELY OF COMPONENTS THAT ARE HAVING DETECTABLE EFFECT ON OUTCOME OR ARE EFFECT OF ANOTHER COMPONENT, NO DEAD WOOD. LET'S SAY YOU DEVELOP AND INTERVENTION, USING CLASSICAL TREATMENT PACKAGE APPROACH. OF COURSE YOU'RE AN ACADEMIC INTERVENTION SCIENTIST, YOU WANT TO GET A SIGNIFICANT EFFECT IN RCT. THE WAY THE REWARDS STRUCTURE IS, THIS IS IN PARTICULAR MANY JUNIOR PEOPLE FEEL WILL MAKE OUR BREAK THEIR CAREER, GETTING THAT SIGNIFICANT EFFECT IN AN RCT. OF COURSE, THERE'S NO INCENTIVE TO CONSIDER MATTERS OF COST OR SCALABILITY WHILE YOU'RE DEVELOPING THE INTERVENTION. WE NEVER TALK ABOUT THIS, BUT THERE IS AN IMPLICIT CONVENTION THAT INACTIVE COMPONENTS ARE OKAY, AS LONG AS -- IN A TREATMENT PACKAGE AS LONG AS THE TREATMENT PACKAGE OVERALL SHOWS A SIGNIFICANT EFFECT IN THE RCT. I SAY THAT BECAUSE I NEVER EVER HEAR ANYONE TALK ABOUT ANY KIND OF IMPERATIVE TO REMOVE INACTIVE COMPONENTS FROM INTERVENTIONS AND I'M NOT SEEING ANY SERIOUS ATTEMPTS TO DO THAT OUTSIDE OF INTERVENTION ORGANIZATION. THE CONCLUSION, INCLUDE MANY COMPONENTS, DON'T HOLD YOURSELF BACK, INCLUDE AS MANY AS SEEMS REASONABLE TO YOU BECAUSE YOU WANT TO GET AS LARGE EFFECT AS I CAN TO GET THE SIGNIFICANT RCT. SO AS A CONSEQUENCE, MANY INTERVENTIONS, I WOULD SAY, ARE LIKELY -- WE DON'T KNOW FOR SURE BUT LIKE LIE TO INCLUDE COUNTER PRODUCTIVE COMPONENTS. SO LET ME SUM UP, MANY ARE EXPENSIVE OR OTHERWISE IMPRACTICAL, SO THEREFORE THEY NEVER SEE THE LIGHT OF DAY, NEVER GET DISTRIBUTED BROADLY, IMPLEMENTED BROADLY. THE FACT THAT NIH HAS SPENT A LOT OF MONEY TO DEVELOP INTERVENTIONS IS SOMETHING HAS IN A NUMBER OF IMPORTANT PAPERS, ANOTHER POINT I WANT TO MAKE MANY INTERVENTIONS HAVE BEEN SUBJECT TO AD HOC ALTERATIONS THAT UNDERMINE EFFECTIVENESS, WE DON'T KNOW HOW MUCH. AND INTERVENTIONS LIKELY INCLUDE USELESS OR EVEN COUNTERPRODUCTIVE COMPONENTS. I WANT TO POINT OUT THAT NO OTHER CONSUMER PRODUCT IS DEVELOPED THIS WAY. SOFTWARE IS NOT DEVELOPED USING ANYTHING LIKE THE TREATMENT PACKAGE APPROACH. AND THIS IS IMPORTANT, BECAUSE OF THIS APPROACH WE DON'T KNOW WHICH COMPONENTS WORK AND WHICH ONES DON'T, WHICH IS MAKING IT DIFFICULT TO DEVELOP A (INDISCERNIBLE). I'D LIKE TO INTRODUCE THE IDEA OF INTERVENTION EASE, STRATEGIC BALANCE OF EFFECTIVENESS, AFFORDABILITY, SCALABILITY, EFFICIENCY. IT MAKES THAT NICE ACRONYM, EAST. EASE IS ACHIEVED BY BALANCING. IF WE'RE HONEST WE WILL ADMIT YOU CAN PROBABLY DEVELOP A MORE EFFECTIVE INTERVENTION IF YOU DON'T WORRY ABOUT AFFORDABILITY OR SCALABILITY IN PARTICULAR. SO WHAT I'M SAYING IS THAT WE NEED TO BALANCE EFFECTIVENESS AGAINST THESE OTHER -- THESE OTHER CONSIDERATIONS. IT'S EASY TO SAY BUT HOW DO YOU DO IT? I THINK THE ANSWER TO THIS IS INTERVENTION OPTIMIZATION, USING MULTI-PHASE OPTIMIZATION STRATEGY WHICH I'M GOING TO EXPLAIN NOW. FIRST I WANT TO TALK ABOUT OPTIMIZATION, OFFER A DEFINITION BECAUSE I THINK IT'S VERY IMPORTANT TO DEFINE THIS TERM. IT'S THE PROCESS OF IDENTIFYING STRATEGIC BALANCE OF EFFECTIVENESS AGAINST AFFORDABILITY, SCALABILITY, EFFICIENCY, HOW WE'RE DEFINING OPTIMIZATION, A WORD THAT CAN BE USED IN DIFFERENT WAYS BUT I THINK IT'S ALWAYS IMPORTANT TO HAVE A PRECISE DEFINITION OF IT. OKAY. POINT ONE, OPTIMIZE SITUATION IS NOT ABOUT FINDING ABSOLUTE BEST BUT BEST YOU CAN ACTUALLY USE. THE ABSOLUTE BEST WOULD BE MOST EFFECTIVE IN SOME ABSOLUTE SENSE BUT OPTIMIZE MEANS MOST EFFECTIVE SUBJECT OF REALISTIC CONSTRAINTS. THE SECOND POINT I WANT TO MAKE THERE MAY NOT BE, PROBABLY WON'T BE, ONE SINGLE OPTIMAL INTERVENTION. WHAT'S OPTIMAL IS LIKELY TO VARY ACROSS SETTINGS AND TIME. THIS IS A SCHEMATIC OF THE MULTI-PHASE OPTIMIZATION STRATEGY, THREE PHASES. PREPARATION, YOU LAY THE GROUND WORK, ACTIVITIES YOU'RE FAMILIAR WITH ALREADY EXCEPT FOR THE LAST ONE. I WANT TO POINT OUT THAT OPTIMIZATION CRITERIUM IS YOUR DEFINITION OF INTERVENTION EASE. FOR EXAMPLE YOU MIGHT DECIDE YOU WANT TO DEVELOP INTERVENTION THAT GIVES YOU BEST EXPECTED OUTCOME FOR IMPLEMENTATION COST OF LESS THAN $300, THAT'S AN EXAMPLE, OR BEST EXPECTED OUTCOME ACHIEVED IN LESS THAN ONE HOUR OF PROGRAMMING. >> TWO MINUTES. >> OKAY. >> NEXT PHASE IS OPTIMIZATION, IN WHICH YOU BUILD AN OPTIMIZED INTERVENTION, ONE OF THESE OPTIMIZATION TRIAL DESIGNS. I THINK (INDISCERNIBLE) IS GOING TO TALK ABOUT THAT LATER. ALL THESE DESIGNS ASSESS PERFORMANCE OF INDIVIDUAL COMPONENTS. FINALLY ONCE THE INTERVENTION HAS BEEN OPTIMIZED YOU CAN EVALUATE IT IN AN RCT. I WAS GOING TO GIVE A QUICK EXAMPLE OF INTERVENTION OPTIMIZATION BUT I THINK I'M RUNNING OUT OF TIME. SO I WON'T DO THAT, JUST TO SAY THAT LITERATURE IS THIS FAIRLY LARGE STUDY FUNDED BY NCI, AND WE DID ACTUALLY THREE DIFFERENT OPTIMIZATION TRIALS IN THE FIELD. I WANT TO POINT OUT THAT THERE ARE EXPERIMENTAL DESIGNS THAT ARE VERY EFFICIENT FOR LOOKING AT THE EFFECTS OF INDIVIDUAL COMPONENTS, IN CASE YOU THINK THIS SEEMS LIKE AN OVERWHELMING ASK. I WANT TO SUGGEST TO ACHIEVE GREATER HEALTH IMPACT WE CONSIDER AFFORDABILITY, SCALABILITY, EFFICIENCY ALONG WITH EFFECTIVENESS FROM THE OUTSET AND OPTIMIZE. I'VE BEEN GIVE AN SHORT TIME TO EXPLAIN SOMETHING I DO A FIVE-DAY TRAINING IN, I'M SURE YOU HAVE A NUMBER OF QUESTIONS. IF YOU'RE INTERESTED, LOOK AT ONE OF THESE BOOKS, I WOULD START WITH THE ONE ON THE LEFT, OFFERS AN INTRODUCTION, I CAN SUGGEST YOU READ WITHOUT FEAR OF CONFLICT OF INTEREST, YOU CAN DOWNLOAD PDF VERSIONS FOR FREE FROM THE UNIVERSITY LIBRARY. I'M STARTING THE INTERVENTION OPTIMIZATION INITIATIVE AT NYU, THIS WEBSITE IS JUST GETTING STARTED, MORE INFORMATION IN THE COMING YEAR. >> THANK YOU, DR. COLLINS. READY FOR DR. WINDSOR TO SHARE YOUR SCREEN. YOUR SLIDES LOOK GOOD. >> YOU NEED TO UNMUTE, DR. WINDSOR. >> THANK YOU. THANKS, EVERYONE, FOR THE OPPORTUNITY TO BE TALKING ABOUT THIS VERY EXCITING MOMENT IN SCIENCE WHERE WE'RE TALKING ABOUT DIFFERENT METHODS, HOW TO MAKE IT BETTER. I'D LIKE TO START BY ACKNOWLEDGING MY PARTNERS, MY FUNDERS, RESEARCH TEAM, ALL THE CONTRIBUTIONS THEY HAVE DONE TO THIS WORK OVER THE PAST TWO YEARS. I ALSO WANTED TO USE HUMOR TO CONTEXTUALIZE THIS DISCUSSION BECAUSE HUMOR DOES THAT BETTER THAN WORDS. I'LL READ IT TO YOU. TWO SCIENTISTS TALKING, ONE SAYS TO THE OTHER I SPENT AGES TRYING TO MAKE MY INVENTION SMALL ENOUGH TO FIT INTO A TROUSER POCKET BEFORE I REALIZED THAT THE SOLUTION LAY IN DOING THINGS THE OTHER WAY AROUND. AND HE'S WEARING A PAIR OF PANTS THAT HAS THIS HUGE POCKET CARRYING A HUGE MACHINE. IN MANY WAYS THIS IS A FITTING METAPHOR FOR WHAT WE'RE TALKING ABOUT TODAY BECAUSE IN MANY WAYS WE'VE BEEN TRYING TO FIT THE WORLD TO THE SCIENCE, WHEN WE REALLY SHOULD BE DOING IT THE OTHER WAY AROUND, AND REALLY TRYING TO FIGURE OUT HOW CAN WE FIT THE SCIENCE TO MEET THE NEEDS OF THE WORLD. ABOUT 11 YEARS AGO AS I STARTED AS ASSISTANT PROFESSOR, I WAS REALLY CONNECTED TO COMMUNITY AND REALLY WANTED TO CONDUCT RESEARCH WITH REAL WORLD IMPLICATIONS. THERE WAS THAT WHOLE DEBATE ABOUT DISSEMINATION OF INTERVENTIONS AND HOW MANY OF THEM END UP COLLECTING DUST IN LIBRARIES, I WANTED TO FIND SOMETHING THAT HAD REAL WORLD APPLICATION AND SIGNIFICANCE. SO I STARTED A PROGRAM OF COMMUNITY-BASED PARTICIPATORY RESEARCH WHERE I WANTED TO WORK WITH COMMUNITY AS PARTNERS, AND WE STARTED BY REALLY LOOKING AT WHAT ARE THE SOCIAL PROBLEMS HERE THAT WE NEED TO FOCUS. AND ONE OF THE ART PIECES OF DOING THIS KIND OF WORK THAT PEOPLE OFTEN DON'T REALIZE IS THE IMPORTANCE OF TAKING THE INTERESTS OF SO MANY DIFFERENT STAKEHOLDERS AND ISSUES AND IN ORDER TO MAKE YOU WORK YOU REALLY HAVE TO FIND THAT SWEET SPOT AT THE CENTER WHERE YOU ARE COMBINING THE NEEDS OF ALL OF THESE DIFFERENT THINGS, SO WE NEEDED TO FIND SOCIAL PROBLEM, IN A WORLD THAT HAS MULTIPLE SOCIAL PROBLEMS THAT ARE VERY COMPLEX AND VERY MESSY. WE NEEDED TO FIND A POPULATION IN A WORLD WHERE YOU HAVE MORE THAN ONE POPULATION THAT ARE INTERCONNECTED AND IMPACTING ONE ANOTHER. WE NEEDED TO FIND THE ISSUE THAT FUNDERS CARED ABOUT AND WERE WILLING TO FUND BECAUSE WITHOUT FUNDING YOU CAN'T WORK. WE NEEDED TO MAKE SURE IN OUR BOARD, IN OUR TEAM, WE HAD THE RIGHT EXPERTISE AND FOR US IT WAS REALLY IMPORTANT THAT THOSE EXPERTISE WERE ABLE TO MERGE THE SCIENTIFIC KNOWLEDGE WITH THE COMMUNITY KNOWLEDGE THAT WE WANTED TO BRING IN THE SWEET SPOT. AND WITH THAT WE ALSO LOOKED AT THE STATE OF INTERVENTION SCIENCE AT THE TIME AND INTERVENTION MODELS AVAILABLE AND TRY TO PUT SOMETHING TOGETHER. THIS IS HOW COMMUNITY WISE WAS BORN. AND FROM THAT MERGING OF INTERESTS, WE WANTED TO DO A MULTI-LEVEL INTERVENTION THAT CAN MULTIPLE OUTCOMES, AND AT THE INDIVIDUAL LEVEL OUTCOME WAS TO REDUCE ALCOHOL AND SUBSTANCE DISEASE, AT THE MESO LEVEL IT WAS ABOUT IMPROVING RELATIONSHIPS BETWEEN PEOPLE, ORGANIZATIONS, AND PEOPLE AND ORGANIZATIONS. ENED'S ALSO INCREASE COMMUNITY CAPACITY TO CREATE COMMUNITY CHANGE AND PROMOTE EQUITY IN THAT COMMUNITY. WHEN I WOULD PRESENT THIS AND TALK TO PEOPLE ABOUT THIS IN THE SCIENTIFIC WORLD, PEOPLE WOULD LOOK AT ME AND SAY, YOU'RE CRAZY. THIS IS JUST TOO MUCH, IT'S TOO BIG, IT'S TOO COMPLEX. YOU CAN'T STUDY THAT. I WOULD SAY HOW CAN I NOT STUDY THAT BECAUSE THIS IS WHAT THE REAL WORLD HAS? THERE HAS TO BE A WAY. AND THE INTERVENTION ITSELF ON TOP OF HAVING THIS MULTIPLE COMPLEX OUTCOMES, IT WAS A MULTI-COMPONENT INTERVENTION. IT INCLUDED THESE CORE COMPONENTS THAT HAD TO CREATE THE FOUNDATION TO MAKE THE INTERVENTION WORK. THIS INTERVENTION ENDED UP BEING A GROUP INTERVENTION, THAT INCLUDED 15 SESSIONS, AND THE IDEA WAS TO PROMOTE CRITICAL THINKING TO EVALUATE WHAT THE CONDITIONS IN THE COMMUNITY WERE DOING KIND OF A POWER ANALYSIS, CRITICAL DIALOGUES, USING QUALITY OF LIFE APPROACH, INDIVIDUAL LEVELS, SMART GOAL DEVELOPMENT, CAPACITY BUILDING PROJECTS THAT PARTICIPANTS WOULD CREATE TO CHANGE AND PROMOTE HEALTH AT THEIR COMMUNITIES. LAST QUESTION, DO WE HAVE TO USE A LICENSED FACILITATOR OR WOULD A PEER BE JUST AS EFFECTIVE? WHAT I HAD IS WHAT MANY COLLEAGUES HAVE ALREADY TALKED ABOUT, TRADITIONAL SCIENTIFIC INTERVENTION PROCESS WHERE YOU START WITH THE EPIDEMIOLOGY, GO INTO THIS LINEAR FASHION TO DISSEMINATION, AND YOUR INTERVENTION HAS TO SURVIVE THE VALLEY OF DEATH WHERE YOU MIGHT RUN OUT OF FINDING OR FIND OUT IT'S NOT EFFICACIOUS AND WHAT DO YOU DO AND FINALLY NOT HAVE THE BUY-IN FROM COMMUNITIES AND IMPLEMENTATION AT COMMUNITY LEVEL. I REALLY WAS SEEKING AND SEARCHING FOR SOMETHING THAT COULD GIVE ME THAT SOLUTION. THIS IS WHERE I MET LINDA COLLINS AND HER WORK, AND I WAS SO BLOWN AWAY BECAUSE I FELT THAT MOST WAS GIVING ME A VEHICLE TO BE ABLE TO DO JUST THAT, IT WAS MOLDING THE SCIENCE TO SET THE REAL WORLD, AND SO I GOT TOGETHER WITH MY PARTNERS AND WE STARTED DOING THIS. WE HAD COMPONENTS, WENT INTO OPTIMIZATION TRIAL, FACTORIAL DESIGN TO ASSESS WHAT WAS THE MOST EFFICIENT COMBINATION OF COMPONENTS THAT WOULD REDUCE SUBSTANCE USE, AND DOING THAT WAS COMMUNITY PARTICIPATION FROM THE GET-GO. WE INCLUDED TRAINING FOR THAT BOARD THAT INCLUDED RESEARCHERS AND CLINICIANS AND PEOPLE THAT WERE STRUGGLING WITH INCARCERATION, SUBSTANCE USE ISSUES, HOW CAN WE TRAIN AND MAKE SURE WE'RE USING THE SAME KNOWLEDGE AND POWER SHARING AND ALL OF THOSE THINGS IN A MORE QUALITATIVE WAY TO MAKE SURE THEIR VOICES ARE REFLECTED IN THE INTERVENTION BUT ALSO IN THE SCIENCE THAT WE WERE USING TO EVALUATE THAT INTERVENTION. I'M NOT GOING INTO SPECIFIC DETAILS OF THESE DIFFERENT PROJECTS. MOST OF THIS IS PUBLISHED. SO YOU GUYS CAN GET THOSE DETAILS FROM THE LITERATURE. A COUPLE THINGS I WANT TO HIGHLIGHT, ONE IS IMPORTANCE TO HAVE A PARADIGM SHIFT BECAUSE THE I WAY WE'RE USED TO DO THINGS IS INVEST MORE TOWARDS THE END AND SEE IT DIE AT DISSEMINATION TIME. THIS KIND OF MODEL REALLY NEEDS A LEAP OF FAITH AND INVESTMENT EARLY ON WHEN WE'RE STILL TRYING TO DEVELOP THAT INTERVENTION, WE DON'T KNOW WHAT THAT'S GOING TO LOOK LIKE. SCREENING TRIAL REQUIRES A LARGE AMOUNT OF INVESTMENT BECAUSE IT'S A LARGE PROJECT. AND IT HAS TO HAPPEN EARLY ON IN THE PROCESS BEFORE YOU HAVE A FINALIZED MANUAL AND FINALIZED INTERVENTION THAT YOU BELIEVE IS EFFECTIVE. OUR INTERVENTION COMMUNITY WISE TRIAL FOUND TO BE EXPECTED TO BE EFFECTIVE, WE ARE HOPEFUL TO BE PUBLISHING THIS IN THE -- IT'S UNDER REVIEW NOW AND HOPEFUL THIS YEAR IT WILL BE OUT. NEXT STEP IS GOING TO BE MOVING TOWARDS IMPLEMENTATION TO SEE IF IT WORKS WITH DIFFERENT COMMUNITIES AND SO FORTH. SORRY, I WAS KIND OF UNSURE IF I HAD A CONCLUSION SLIDE. THIS IS MY LAST ONE. WHAT I WANTED TO KIND OF BRING UP ARE SOME LIMITATIONS OF THE APPROACH IN ADDITION TO STRENGTHS. I THINK IT'S A REALLY EXCITING APPROACH THAT ALLOWS US TO BE MORE MINDFUL AND CREATE MUCH MORE EFFECTIVE INTERVENTIONS. THE OTHER THING TOO IS THE IDEA OF ONLY FOCUSING ON HAVING EFFECTIVE INTERVENTIONS BOTH CLINICALLY AND STATISTICALLY TO MAKE INTERVENTIONS BE MORE POTENT ON THE LONG RUN. BUT THERE ARE LIMITATIONS OF COURSE, ONE ISSUE FOR ME IN THE SESSION IS ABOUT LOOKING AT MULTI-LEVEL INTERVENTIONS WITH SINGLE OUTCOMES. SELECTING THAT SINGLE OUTCOME WAS EXTREMELY HARD AND WE DON'T HAVE A GOOD WAY TO MEASURE MULTI-LEVEL INTERVENTIONS THAT ARE SEEKING TO CREATE CHANGE AT DIFFERENT LEVELS. OUR METHODOLOGIES UNFORTUNATELY ONLY ALLOW YOU TO STUDY THE COMMUNITY LEVEL WITH COMMUNITY LEVEL DESIGNS, OR YOU HAVE TO DO INDIVIDUAL LEVEL VARIABLES, COLLECTING DATA AT THE INDIVIDUAL LEVEL REFLECTING CHANGES AT THE COMMUNITY LEVEL. AND ALL OF THIS IS COMPLEX IN SO MANY WAYS. SO, I WOULD LIKE TO PUT OUT A CALL FOR PEOPLE TO REALLY BE CREATIVE AND START THINKING ABOUT HOW CAN WE CHANGE OUR METHODOLOGY, HOW CAN WE CHANGE SCIENCE TO MAKE IT MORE ACCEPTING OF THE MESSINESS THAT RESEARCH HAS WHEN IT'S BEING CONDUCTED IN THE REAL WORLD, HOW CAN WE TRY TO MEET IN THE MIDDLE SO THAT WE CAN CREATE INTERVENTIONS AND APPROACHES THAT ARE MUCH MORE USER FRIENDLY, MUCH MORE EFFECTIVE, MUCH MORE EFFICIENT, AND MOST GIVES US ONE WAY OF DOING THAT. WE'VE LEARNED FROM MANY OTHERS THAT HAVE TALKED ABOUT THAT, BUT I THINK THERE'S A LONG ROAD AHEAD. I WANT TO THANK YOU FOR PUTTING TOGETHER THIS WORKSHOP TODAY. I'LL STOP HERE. >> THANK YOU SO MUCH, DR. WINDSOR. AND WE'LL STOP YOUR SLIDE SHARE AND OUR NEXT SPEAKER IS WELCOME TO BRING UP THEIR SLIDES. I BELIEVE THAT IS DR. JENSEN. >> VERY GOOD. CAN YOU SEE THEM? >> THEY ARE COMING UP. >> EXCELLENT. >> WE HAVE THEM, THANK YOU. >> WELL, I'M GOING TO SEE IF I CAN -- THERE WE GO. I'LL SPEND MY TIME I THINK PRESENTING JUST TWO ANALYTICAL APPROACHES THAT CAN BE USED TO HELP UNDERSTAND THE COMPONENTS WITHIN MULTI-COMPONENT TREATMENTS THAT ARE MOST EFFECTIVE. I THINK IT SPEAKS TO THE ISSUES RAISED SO FAR THIS MORNING PRETTY WELL. I'LL START WITH MAKING JUST THE POINT MULTI-DIMENSIONAL PROBLEMS REQUIRE MULTI-COMPONENT TREATMENTS, THE WAY TO GO, BUT IT'S BEEN MENTIONED SO FAR WE DON'T ALWAYS KNOW WHICH COMPONENTS ARE NECESSARY, WHICH COMPONENTS ARE MOST EFFECTIVE. I'LL MAKE THE PITCH MECHANISM ANALYSES CAN ADDRESS THIS, AND TO PICK THE COMPONENTS THAT MIGHT BE MOST USEFUL THAT DR. WINDSOR WAS SPEAKING TO. I'M GOING TO FOCUS TO TWO ANALYTIC APPROACHES WE USE IN OUR RESEARCH. I'D LIKE TO ACKNOWLEDGE THE WORK THAT WE'RE DOING TO UNDERSTAND THE MECHANISMS UNDERLYING PAIN TREATMENTS IS A TEAM EFFORT AND TWO LEADERS IN THAT ARE MELISSA DAY AND JOHN BURNS WHO WORK CLOSELY WITH ME ON THESE PROJECTS. TO MAKE THE POINT CHRONIC PAIN IS A MULTI-DIMENSIONAL PROBLEM, BIOLOGICAL, PSYCHOLOGICAL, SOCIAL AND ENVIRONMENTAL COMPONENTS. IF WE WANT TO TREAT CHRONIC PAIN AND USE SINGLE TREATMENT, FOR EXAMPLE A MEDICATION THAT FOCUSES ON A PIECE OF BIOLOGY, WE'RE UNLIKELY TO HAVE A LARGER EFFECT. WE LEARNED THAT DECADES AGO AND DEVELOPED MULTI-COMPONENT TREATMENTS TO ADDRESS THE MULTI-COMPONENT ISSUES. AND THIS IS NOW THE GOLD STANDARD OF PAIN TREATMENT. AND THAT'S GOOD. THAT'S WHAT WE DO. IT CONTAINS MANY COMPONENTS INCLUDING EXERCISE, COGNITIVE THERAPY, EDUCATION, COPING SKILLS, TRAINING, FAMILY THERAPY, AND MEDICATION WITHDRAWAL. AS WE LEARN ABOUT EFFECTIVE TREATMENTS WE'RE ADDING THEM, MINDFULNESS, HYPNOSIS, YOGA, TAI CHI, ET CETERA, AS EFFICACY. THE QUESTION IS, HAS ALREADY BEEN TALKED ABOUT, WHICH COMPONENTS SHOULD BE INCLUDED TO CREATE THE MOST EFFECTIVE ONES, WHICH ARE THE MOST EFFECTIVE AND WHICH ONES SHOULD BE EXCLUDED. THE DEAD WOOD THAT DR. COLLINS WAS TALKING ABOUT. AND HOW DO THEY WORK TOGETHER TO ENHANCE EFFICACY. THESE ARE CRITICAL QUESTIONS MOVING FORWARD. I'M GOING TO MAKE THE PITCH THAT WE SHOULD CONSIDER MECHANISTIC RESEARCH TO HELP UNDERSTAND THIS BECAUSE AS WE UNDERSTAND THE MECHANISMS OF MULTI-COMPONENT TREATMENTS WE CAN IDENTIFY COMPONENTS THERE WILL MOST LIKELY IMPACT BOTH PIECES. AND AS YOU PROBABLY KNOW, NIH DEFINES MECHANISTIC STUDIES AS THOSE DESIGNED TO UNDERSTAND BEHAVIORAL OR BIOLOGICAL PROCESS THAT UNDERLINE MECHANISM OF ACTION OF OUR TREATMENTS. AND NCCIH HAS A PROGRAM OF MECHANISTIC CLINICAL TRIALS INCLUDING RESEARCH STUDIES IN WHICH INTERVENTIONS THAT WE ALREADY KNOW ARE EFFECTIVE ARE THEN STUDIED TO UNDERSTAND THEIR MECHANISMS. THIS IS A CRITICAL PIECE OF WHAT WE NEED TO DO. MECHANISTIC CLINICAL TRIALS FOCUS ON THE WHY OF TREATMENTS, HOW DO THEY WORK, AND NOT ON WHETHER THEY WORK. THE PRIMARY OUTCOME IN MECHANISTIC RESEARCH ARE THE MECHANISM VARIABLES, IT'S ALL ABOUT THE WHY. SO LET ME JUMP INTO THE TWO TYPES OF ANALYSES WE'RE USING. FIRST IS CLASSIC MEDIATION. WE TAKE SOME INTERVENTION THAT WE KNOW IS EFFECTIVE, FOR EXAMPLE THAT CPT HAS EFFECT UPON PAIN. THE QUESTION NOT IS IT EFFECTIVE BUT HOW. WE WANT TO IDENTIFY THE MECHANISMS THAT UNDERLIE THESE TREATMENTS. WE IDENTIFIED POTENTIAL MECHANISM VARIABLES, THESE AGAIN ARE PRIMARY OUTCOMES IN MECHANISTIC STUDIES. THEN WE WANT TO UNDERSTAND HOW CHANGES IN THOSE MECHANISMS, SELF-EFFICACY IS POTENTIAL, HOW CHANGES INFLUENCE IN THIS EXAMPLE PAIN. TO PULL AN EXAMPLE OF THE INTERVENTION DR. SHINTO SPOKE OF, WE HAVE THIS TREATMENT CONDITION THAT WE ANTICIPATE WILL AFFECT QUALITY OF LIFE, BUT THE QUESTION IS WHICH OF THE COMPONENTS ARE MOST IMPORTANT OR ARE THEY EQUALLY IMPORTANT. IS IT EXERCISE? IS IT MINDFULNESS PRACTICE? OR IS IT THE FOOD INTAKE? OR DO ALL THEY PLAY A ROLE? AND THE WAY YOU DO THIS IS BY MEASURING THE QUALITY OF LIFE IN THIS CASE, BEFORE AND AFTER TREATMENT, WE MEASURE AMOUNT OF FOOD INTAKE, AMOUNT OF EXERCISE, AND AMOUNT OF TIME SPENT DOING MINDFULNESS, DURING TREATMENT, AND THEN YOU DETERMINE WHETHER THE TREATMENT IMPACTED THE MECHANISM VARIABLES, THE A COEFFICIENT, AND IF CHANGES IN THOSE MECHANISM VARIABLES INFLUENCE IN THIS CASE QUALITY OF LIFE. IT'S THAT A TIMES B TERM THAT YOU LOOK AT FOR STATISTICAL SIGNIFICANCE AND MEANINGFULNESS TO SEE IF THAT -- WHICH FACTOR IS MOST IMPORTANT IN THE TREATMENT. YOU CAN RULE OUT POTENTIAL MEDIATOR AND THEN CONSIDER TAKING OUT THE TREATMENT COMPONENT THAT TARGETS THAT. SO THAT'S ANOTHER STRENGTH. PRIMARY WEAKNESS, THE ASSOCIATE BETWEEN POTENTIAL MEDIATOR AND YOUR CLINICAL OUTCOME, WHATEVER THAT IS, IS CORRELATIONAL AND SO IT'S NOT NECESSARILY A CAUSAL ANALYSIS. SO THIS IS A MOST USEFUL APPROACH FOR RULING OUT POTENTIAL CAUSAL FACTORS. IT'S LIKE HOW YOU CUT OUT PIECES AND YOU'RE LEFT WITH FACTORS MOST LIKELY TO BE MEDIATORS. AN EXAMPLE WAS A STUDY PUBLISHED THIS YEAR, RANDOMIZED CONTROLLED TRIAL OF 173 INDIVIDUALS ON A PAIN, RANDOMLY ASSIGNED. WE STUDIED 17 MECHANISM VARIABLES WITH MEDIATION ANALYSES, AND SIGNIFICANT EFFECTS ONLY EMERGED FOR TWO. WE WERE ABLE TO CARVE OUT MANY FACTORS THAT WERE LESS LIKELY TO BE IMPACT OUTCOME. AND SO THIS HELPED US MOVE FORWARD I THINK. AND WE NOW -- WE CAN NOW TARGET TO MAKE THESE TREATMENTS MORE EFFECTIVE. SWITCHING TO ANOTHER ANALYTICAL APPROACH, SIMILAR BUT DIFFERENT, WE ALSO USE THIS APPROACH TO UNDERSTAND MECHANISMS, IT'S A CROSS-LAGGED ANALYSIS. YOU MEASURE THE THING YOUR TREATMENT IMPACTS PAIN, IN THIS CASE PAIN AND YOUR POTENTIAL MECHANISM VARIABLE, AGAIN PRIMARY OUTCOME AT BASELINE. AND ALSO MULTIPLE TIMES DURING TREATMENT. AT LEAST MID-TREATMENT, WE DO THESE MEASURES DAILY, AND POST-TREATMENT. IN ANALYSES YOU LOOK AT EXTENT TO WHICH CHANGE IN YOUR PRIMARY OUTCOME, THE MEDIATOR, FROM ONE TIME POINT TO THE NEXT, THEN PREDICTS LATER CHANGE IN THE THING THAT YOU'RE TRYING TO IMPACT WHICH IN THIS CASE IS PAIN. AND YOU CAN ALSO LOOK AT THE EXTENT TO WHICH CHANGE IN PAIN MIGHT AFFECT YOUR MECHANISM VARIABLE. YOU CAN LOOK AT BOTH OF THOSE EFFECTS. AND SO AGAIN LOOK AT THE EXAMPLE FROM THE 3M STUDY THAT DR. SHINTO PRESENTED EARLIER WHERE YOU'RE TARGETING EXERCISE, TRAINING, MINDFULNESS, FOOD INTAKE, THREE Ms, TO IMPACT QUALITY OF LIFE. IF YOU WERE MEANT TO MEASURE THOSE AT BASELINE, MID-TREATMENT AND POST-TREATMENT YOU COULD LOOK THE EXTENT TO WHICH HOW MUCH CHANGES IN EXERCISE, FOR EXAMPLE, IMPACT LATER QUALITY OF LIFE, OR CHANGES IN MINDFULNESS IMPACT LATER QUALITY OF LIFE. UNDERSTANDING THE ROLE EACH PLAYS IN TREATMENT. AND THE STRENGTH WITH THIS STUDY, THIS APPROACH, HAVE YOU TEMPORAL PRECEDENCE. CAUSAL CONCLUSIONS CAN BE STRONGER BECAUSE YOU KNOW THAT IF YOU'RE MEASURING EXERCISE CHANGES FIRST, AND THEN QUALITY OF LIFE, YOU CAN'T IMPACT EXERCISE EARLIER. THIS IS A REAL STRENGTH IN THIS APPROACH. YOU CAN TRACK HOW THINGS CHANGE OVER TIME AND START TO UNDERSTAND HOW LONG DOES IT TAKE YOUR TREATMENT TO START TO IMPACT THE MECHANISM VARIABLES YOUR PRIMARY OUTCOME HERE. YOU CAN ALSO ADDRESS TEMPORAL QUESTIONS, THAT IS ONCE YOUR MECHANISM VARIABLE CHANGES, ONCE PEOPLE START EXERCISING FOR EXAMPLE, HOW LONG DOES IT TAKE FOR THAT TO HAVE AN IMPACT ON THE THINGS THAT YOU WANT TO IMPACT? THAT'S ANOTHER STRENGTH IN THIS DESIGN. PRIMARY WEAKNESS THIS APPROACH REQUIRES YOU MEASURE ALL THESE VARIABLES MULTIPLE TIMES, AND IF YOU HAVE MULTIPLE POTENTIAL MEDIATORS, AND YOU WANT TO MEASURE THEM OFTEN, THAT CAN RESULT IN MISSING DATA WHICH CAN COMPLICATE YOUR ANALYSES. ALSO, IF YOU MISS THE TIME POINTS, IF EXERCISE TAKES TWO WEEKS TO IMPACT AN OUTCOME AND YOU JUST MEASURE IT AFTER ONE WEEK AND TRY TO LOOK AT ITS EFFECT ON YOUR CLINICAL VARIABLE, YOU MIGHT GET A NULL EFFECT WHEN IN FACT THE MECHANISM IS IMPORTANT, JUST TAKES LONGER TO HAVE THE EFFECT. WHEREAS WITH MEDIATION ANALYSES YOU HAVE A GREATER RISK FOR IDENTIFYING MEDIATOR AS SIGNIFICANT MIGHT NOT BE CAUSAL WITH CROSS-LAGGED ANALYSES YOU HAVE THE POTENTIAL WEAKNESS OF NOT IDENTIFYING A VARIABLE THAT'S IMPORTANT WHEN IN FACT IT IS. SO, I RECOMMEND THAT IN DOING MECHANISM RESEARCH YOU CONSIDER BOTH APPROACHES AND TAKE ADVANTAGE OF BOTH STRENGTHS AND WEAKNESSES. EXAMPLE, WE PUBLISHED LAST YEAR, RANDOMIZED CONTROLLED TRIAL OF 290 INDIVIDUALS WITH CHRONIC PAIN, RANDOMLY ASSIGNED. WE STUDIED TWO POTENTIAL MECHANISMS HERE, CHANGES IN CATASTROPHIZING BELIEFS AND PAIN SELF-EFFICACY, AND WE FOUND THAT BOTH TREATMENTS HAD DECREASEED CATASTROPHIZING, INCREASING PAIN SELF-EFFICACY BELIEFS, AND THAT CHANGES IN CATASTROPHIZING AND CHANGES IN SELF-EFFICACY BELIEFS THAT OCCURRED DURING TREATMENT THEN WERE ASSOCIATED WITH IMPROVEMENTS, LATER IMPROVEMENTS IN PAIN OR DECREASES IN PAIN IN THIS CASE. WE ALSO FOUND THAT MID TO POST TREATMENT -- PRE-TO MID-TREATMENT CHANGES IN CAMPAIGN IMPACTED BELIEVES, MUTUALLY CAUSAL. CONSISTENT WITH THE IDEA THAT WE FUNCTION AS COMPLEX NETS, IT'S NOT ONE THING ONLY AFFECTING ANOTHER. NOW WE'VE IDENTIFIED THAT PAIN CATASTROPHIZING AND SELF-EFFICACY ARE THINGS WE NEED TO TARGET AND FIND TREATMENTS THAT IMPACT THOSE THE MOST SO WE CAN GO BACK TO THE MEDIATION ANALYSES BECAUSE THAT A COEFFICIENT IS CAUSAL. INSTEAD OF ONLY LOOKING AT HOW THESE TREATMENTS AFFECT CLINICAL OUTCOMES WE CAN BUILD A DATABASE TO LOOK AT HOW THESE IMPACT POTENTIAL MEDIATORS INTO MECHANISM VARIABLES. >> TWO AND A HALF MINUTES. >> TO SUMMARIZE MULTI-COMPONENT TREATMENTS ARE HOW WE TREAT COMPLEX PROBLEMS INCLUDING CHRONIC PAIN, IT'S HOW IT'S DONE IN THE REAL WORLD. WE STILL KNOW VERY LITTLE REGARDING THE SPECIFIC MECHANISMS THAT UNDERLIE THESE MULTI-COMPONENT TREATMENTS, AND A GREATER UNDERSTANDING OF THESE MECHANISMS I THINK IS CRITICAL, VERY IMPORTANT TO INFORM ADAPTATIONS TO MAXIMIZE EFFECT EFFICACY OR OPTIMIZE THEM AS DR. COLLINS TALKED ABOUT, AND ASSESSING CLINICAL OUTCOMES AND MEDIATORS MULTIPLE TIMES IS IMPORTANT TO DO THIS. DR. POWELL HAS THOSE DATA IN HER STUDY AND SO, YOU KNOW, I WOULD -- I'M LOOKING FORWARD TO SEEING HOW ANALYSES SHE DESCRIBED PLUS MEDIATION ANALYSES AND CROSS-LAGGED COULD INFORM HOUR MINDFUL INSIDE IMPACTS METABOLIC FUNCTION. THE TWO APPROACHES HAVE DIFFERENT STRENGTHS AND WEAKNESSES, IT'S WORTH CONSIDERING USING BOTH. PARTICULARLY USEFUL FOR ELIMINATING POSSIBLE MECHANISMS, TAKING OUT DEAD WOOD SO WE CAN MAXIMIZE AND OPTIMIZE OUR INTERVENTIONS. AND AS WE COLLECT THESE DATA, WE'LL BE ABLE TO LEARN MORE ABOUT WHICH COMPONENTS TO INCLUDE FOR THE MOST EFFECTIVE TREATMENTS, AND THOSE ARE THE THINGS WE WANT TO TARGET MOST. THAT SHOULD BE IT. SO THANK YOU VERY MUCH FOR YOUR INVITATION TO PARTICIPATE AND FOR LISTENING TO THIS. >> THANK YOU, DR. JENSEN. DR. CECH, PLEASE GO AHEAD AND START YOUR SHARE. >> HELLO, EVERYONE. >> GOOD AFTERNOON. >> GOOD AFTERNOON! IT'S ALMOST AFTERNOON. I GUESS IT IS. ALL RIGHT. WELL, IT'S WONDERFUL TO BE HERE AS PART OF THIS WORKSHOP, I WANT TO THANK THE ORGANIZERS FOR INVITING ME TO PARTICIPATE AND I REALLY ENJOYED LISTENING TO SOME REALLY INTERESTING TALKS. I'M COMING TO YOU AS A CHEMIST, SOMEONE WHO DOES BASIC SCIENCE RESEARCH. SO IT'S SORT OF GOING TO BE DEPARTURE IN A DIFFERENT DIRECTION HERE BUT THERE'S SOME NICE OVERLAPS BECAUSE THE WORK WE DO IS LOOKING AT WHAT HAPPENS WHEN WE STUDY BOTANICAL DIETARY SUPPLEMENTS PRIOR TO CONDUCTING CLINICAL TRIALS AND THINKING HOW TO SELECT PRODUCTS THAT ARE GOOD CHOICES FOR CLINICAL TRIALS REALIZING BOTANICAL SUPPLEMENTS ARE COMPLEX, DIFFERENT COMPLEXITY, I'LL TRY TO TAKE IT AT A PACE FOLLOWABLE FOR THOSE NOT IN THIS FIELD ALREADY. SO, THE CASE STUDY I THOUGHT I WOULD SHARE TODAY, WE'VE DONE A NUMBER OF PROJECTS AROUND THIS TOPIC, BUT I WANTED TO HIGHLIGHT SOME WORK WE'RE DOING AS PART OF THE CENTER OF EXCELLENCE FOR NATURAL PRODUCT DRUG INTERACTION RESEARCH FUNDED BY NATIONAL INSTITUTES OF COMPLEMENTARY AND INTEGRATIVE HEALTH. AND THAT IS A PROJECT THAT INVOLVES THREE UNIVERSITIES THAT ARE COLLABORATING AS WELL AS NATIONAL INSTITUTES OF HEALTH, COOPERATIVE AGREEMENTS, AND OUR MISSION IS TO CONDUCT CLINICAL STUDIES THAT GIVE INSIGHT INTO POTENTIAL INTERACTIONS THAT CAN OCCUR WHEN INDIVIDUALS CONSUMED DIETARY SUPPLEMENTS SUCH AS BOTANICAL MEDICINES AT THE SAME TIME AS CONVENTIONAL MEDICATIONS. I'LL JUST SHARE AN EXAMPLE STORY FROM THAT OF ONE STUDY WE'VE DONE AS PART OF THAT PROJECT. SO ONE OF THE DIETARY SUPPLEMENTS WE LOOKED AT IS GREEN TEA, WHICH IS ONE OF THE MOST COMMONLY CONSUMED DIETARY SUPPLEMENT PLANTS IN THE WORLD. THERE ARE SOME REPORTS THERE ARE ISSUES WITH CONSUMPTION OF GREEN TEA AND POSSIBLY ALTERING MA METABOLISM OR PHARMACEUTICAL DRUGS. MY COLLABORATORS DID A CLINICAL STUDY ON THIS, WASHINGTON STATE UNIVERSITY, AND THEY TOOK 16 HEALTHY VOLUNTEERS AND THESE VOLUNTEERS CONSUMED GREEN TEA WHILE CONSUMING RALOXIFENE, USED FOR TREATING BONE LOSS IN POST MENOPAUSAL WOMEN. AND THE REASON WE CHOSE RALOXIFENE IS BECAUSE OF GOOD EXAMPLE OF PHARMACEUTICAL METABOLIZED IN THE LIVER VIA GLUCORONIDATION, A GOOD TEST CASE FOR LOOKING AT HOW METABOLISM COULD BE ALTERED BY GREEN TEA. WHAT YOU SEE IN THIS PLOT BELOW IS SOME DATA SHOWING CHANGES IN THE LEVELS OF RALOXIFENE, AND WE OBSERVED 30% DECREASE IN AMOUNT OF RALOXIFENE PATIENTS WERE -- THAT PARTICIPANTS WERE BEING EXPOSED TO AS A CONSEQUENCE OF CO-CONSUMING GREEN TEA. THIS IS JUST AN EXAMPLE OF HOW THERE CAN BE A CLINICALLY RELEVANT INTERACTION AND THAT CAN OCCUR AND CERTAINLY SOMETHING THAT WAS WORTH CONSIDERING WHEN WE HAVE, FOR EXAMPLE, A REASON WHY WE ENCOURAGE PATIENTS TO TELL THEIR DOCTORS AND NATUROPATHS WHAT THEY MIGHT BE CONSUMING INCLUDING DIETARY SUPPLEMENTS, IT'S IMPORTANT TO HAVE THAT INFORMATION FOR POSSIBLE INTERACTIONS. I'M GOING TO STEP BACK AND THINK ABOUT WHAT WE HAD TO DO BEFORE THIS STUDY, SELECT THE RIGHT GREEN TEA STUDY MATERIAL. AND THAT'S, AS YOU MIGHT IMAGINE, A TRICKY QUESTION. THERE ARE MANY PEOPLE WHO CONSUME GREEN TEA, THEY ARE NOT ALL CONSUMING THE SAME GREEN TEA. THIS IS JUST A PICTURE OF A SUPERMARKET SHELF WHERE YOU MIGHT GO AND LOOK AT THE SHELF AND TRY TO DECIDE WHAT GREEN TEA TO TAKE AND THERE'S ALL THESE DIFFERENT GREEN TEA PRODUCTS, SOME ARE SOLD OUT TODAY SO YOU'RE NOT GOING TO PICK THOSE ONES, THERE MIGHT BE SOME THAT YOU LIKE THE LABEL OR YOU LIKE THE PRODUCT. BUT THOSE ARE NOT OBVIOUSLY THE REASONS THAT WE WANT TO SELECT THEM FOR CLINICAL STUDY AND SO WE REALLY THOUGHT CAREFULLY BECAUSE THERE'S BEEN CASE WHERE IS CLINICAL STUDIES OF DIETARY SUPPLEMENTS HAVE BEEN CRITICIZED AFTER THE FACT BECAUSE PEOPLE SAY, WELL, YOU CHOSE THE WRONG MATERIAL, YOU SHOULD HAVE STUDIED THIS INSTEAD, AND OF COURSE IF WE'RE GOING TO DO CLINICAL STUDY WE CAN ONLY STUDY ONE MATERIAL. WE HAVE TO CHOOSE SOMETHING. WE WANTED TO HAVE A RATIONALE FOR WHY WE WERE CHOOSING WHAT WE WERE CHOOSING. AND SO THE RATIONALE WAS THAT WE DECIDED TO SELECT A BUNCH OF PRODUCTS BASED ON CONSUMER REPORTS, AND ALSO BASED ON AMAZON SALES REPORTS THAT SEEM TO BE VERY WIDELY USED BY THE PUBLIC AND THEN TO COMPARE THEM TO EACH OTHER AND TO TRY TO DECIDE WHICH WOULD BE A REASONABLE CHOICE FOR DOING THE STUDY. AND THE WAY WE COMPARED THEM WAS USING MASS SPECTROMETRY METABOLOMICS, EXTRACTING SAMPLES, IN REPLICATES, ANALYZE ARE CHROMATOGRAPHY AND MASS SPECTROMETRY FOR MOLECULES PRESENT IN EXTRACTS. WE INCLUDED THREE STANDARDS FROM NIST, NIST MAKES STANDARDS OF BOTANICAL SUPPLEMENTS, WE INCLUDED THOSE TO HAVE A COMPARISON. NIST DOES A GOOD JOB SELECTING MATERIALS VERY REPRESENTATIVE OF CUSTOMER USAGE. BUT WE COULDN'T USE THE NIST MATERIALS FOR THE CLINICAL STUDY BECAUSE THEY ARE NOT APPROVED FOR HUMAN CONSUMPTION. SO WE HAD TO -- OUR GOAL WAS TO PICK SOMETHING THAT WAS SIMILAR TO THE NIST MATERIALS BUT EVEN THOUGH WE COULDN'T USE THE NIST MATERIALS THEMSELVES. I USE THIS TERM UNTARGETED METABOLOMICS, I'LL DEFINE WHAT I MEAN. IF WE HAVE A LEAF OF TEA OR PLANT, IT WILL HAVE MULTIPLE MOLECULES IN IT. AND WITH METABOLOMICS WE MEASURE USING TECHNIQUES SUCH AS THOSE I MENTIONED PREVIOUSLY, MASS SPECTROMETRY, CHROMATOGRAPHY, LOOK AT DIFFERENCES IN ABUNDANCE OF THOSE MOLECULES AND THEIR PRESENCE AND ABSENCE AND TRY TO LOOK AT A HOLISTIC PROFILE OF ALL OF THE SMALL MOLECULES, SECONDARY METABOLITES PRESENT IN THAT SAMPLE. WE'RE NOT JUST TARGETING A SINGLE MOLECULE THAT MIGHT BE INTERESTING. WE'RE TRYING TO CAPTURE AS MUCH AS WE CAN OF WHAT IS THERE. AND THEN TO COMPARE DIFFERENCE SAMPLES FOR CHEMICAL PROFILE IN A HOLISTIC FASHION. SO, WHAT DO DATA LOOK LIKE? PROBABLY MANY HAVE SEEN CHROMATOGRAM IN YOUR HISTORY, EVEN IF YOU DON'T TYPICALLY DO CHEMISTRY RESEARCH, CHROMATOGRAM IS A PLOT OF SIGNAL, Y-AXIS IS SIGNALS SUCH AS INTENSITY OF SIGNAL DETECTED BY MASS SPECTRO METER AS A FUNCTION OF TIME. THE MOLECULE IS PRESENT IN THE SAMPLE. YOU CAN LOOK AT THIS CHROMATOGRAM, WHAT'S INTERESTING IS WHEN YOU COMPARE CHROMATOGRAMS FROM MULTIPLE SAMPLES AND START TO TALK ABOUT HOW SIMILAR OR DIFFERENT THEY ARE BASED ON CHROMATOGRAMS. IF OUR GOAL IS COMPARE DIFFERENT TEA PRODUCTS WE CAN COMPARE CHROMATOGRAMS AND LOOK VISUALLY BUT IT'S TRICKY TO DRAW CONCLUSIONS WILL ABOUT SIMILARITY AND DIFFERENCE, SO WHAT WE DO IS WE WILL TRANSFORM THAT INTO WHAT WE CALL PRINCIPAL COMPONENTS PLOTS WHERE EACH DOT REPRESENTS A SAMPLE AND RECOMMENDS CHROMATOGRAPH DATA. YOU CAN SEE THIS BLUE IS DIFFERENT FROM THE GREEN CHROMATOGRAM AND THEY ARE SHOWING UP IN DIFFERENT POINTS IN THREE DIMENSIONAL SPACE. YOU CAN SEE THREE ANALYSES SHOWN WITH TRIANGLES OR WITH THE STARS HERE. SO HOW DO WE APPLY THAT? THE REASON I GAVE THAT ANALYSIS, I WANT TO SHOW YOU NOW WHAT THE DATA LOOK LIKE FOR THE GREEN TEA ANALYSIS WE DID. SO THIS IS THE PRINCIPAL COMPONENT AND ANALYSIS PLOT OF 37 SAMPLES OF GREEN TEA, EACH DOT REPRESENTS A DIFFERENT SAMPLE OF GREEN TEA. AGAIN HOW CLOSE THEY ARE TO EACH OTHER SHOWS CHEMICAL SIMILARITY. RIGHT AWAY WE SEE A COUPLE THINGS. ONE, WE SEE CLUSTERING OF DATA OF SAMPLES ALL IN BLUE, THE SUPPLEMENTS WE LOOKED AT, ENCAPSULATED GREEN TEA SUPPLEMENTS AS OPPOSED TO POWDERED TEA OR LOOSE TEA LEAVES THAT WOULD BE USED TO MAKE A TEA. AND THEN THERE'S A SAMPLE OVER HERE NOT GREEN TEA, THAT'S INCLUDED JUST TO SHOW THERE'S AN OUTLIER IN THE DATASET. WE HAVE RED DOTS, THE NIST STANDARDS. THE RED DOTS ARE OVERLAYING WITH COMMERCIAL PRODUCTS, AND THAT'S ACTUALLY GOOD SIGN, THIS IS THE NIST STANDARD FOR GREEN TEA LEAVES AND THESE ARE TWO NIST STANDARD SUPPLEMENTS. IN THE END WHAT WE DID IS CHOOSING TO DO THE STUDY WITH TEA LEAVES, BUT ONE THING WE LEARNED FROM CHEMICAL ANALYSIS IS THAT'S VERY LIKELY TO GIVE US DIFFERENT RESULTS THAN WE GET IF WE DID THE STUDY WITH THE SUPPLEMENTS THAT ARE CHEMICALLY DIFFERENT. WE KNOW WE'RE MAKING A CHOICE THAT'S GOING TO LIMIT APPLICABILITY OF OUR RESULTS, WE HAVE A JUSTIFICATION FOR DOING THAT. AND THEN WE WOULD HAVE TO GO ONE WAY OR THE OTHER. WE DECIDE TO GO WITH TEA LEAVES BECAUSE THEY ARE REPRESENTATIVE OF PUBLIC USE AND CHOSE A SAMPLE SIMILAR TO NIST STANDARD AND DID CLINICAL STUDY WITH THAT. WHEN PEOPLE ASK WHY YOU SELECTED THE TEA SAMPLE WE CAN SHARE THESE DATA AND WE HAVE THEM AVAILABLE ON PUBLICLY ACCESSIBLE REPOSITORIES WHERE PEOPLE CAN ACCESS THEM. NEXT QUESTION, YOU'VE GOT THIS CHEMICAL PROFILE OF ALL THESE THINGS THAT ARE IN THIS GREEN TEA EXTRACT. WHICH OF THOSE IS RESPONSIBLE FOR WHATEVER BIOLOGICAL EFFECTS YOU'RE SEEING? OF COURSE THE EFFECTING COULD BE VARIABLE, WE WERE INTERESTED IN LOOKING AT INHIBITION OF METABOLISM, SO WHICH COMPONENTS OF THE EXTRACT SHOWN LABELED IN THIS CHROMATOGRAM MIGHT BE UPON FOR THAT ACTIVITY, THE GROUP IS DEVELOPING METHODS FOR CORRELATING CHEMICAL COMPOSITION WITH BIOLOGICAL ACTIVITY TO PREDICT WHAT THE ACTIVE COMPOUNDS OF THOSE MIXTURES MIGHT BE. THIS COULD BE USED IN MULTIPLE CONTEXTS, WE HAVE USED ANYTIME MULTIPLE CONTEXTS BUT I WANT TO CONTINUE WITH THE GREEN TEA EXAMPLE OF THIS. WE CALL THIS TECHNIQUE BIOCHEMO METRICS, ALLOWS US TO INCORPORATE CHEMICAL AND BIOLOGICAL DATA AND PREDICT MARKERS OF A PARTICULAR BIOLOGICAL ACTIVITY. SO, WE ACTUALLY FRACTIONATE EXTRACTS FIRST SO WE CAN GET SOME SEPARATION OF WHAT COMPONENTS ARE PRESENT IN THOSE EXTRACTS, SO MAYBE YOU COULD IMAGINE THERE WOULD BE A THOUSAND MOLECULES IN THE ORIGINAL, WE CREATE A SERIES OF FRACTIONS, EACH HAS A SUBSET OF THOSE MOLECULES. MAYBE A HUNDRED OF THOSE. AND THEN WE ANALYZE EACH ONE OF THOSE FRACTIONS AND ALSO TEST THEM FOR BIOLOGICAL ACTIVITY. CLEARLY WHEN WE'RE TALKING ABOUT GENERATING A SERIES OF FRACTIONS IT'S NOT REALLY FEASIBLE TO DO THIS IN A CLINICAL STUDY CONTEXT ANYMORE. SO WHAT WE DO IS USE IN VITRO ASSAY WHICH IS ONE OF THE CHALLENGES OF THESE STUDIES, CHOOSING THE RIGHT IN VITRO ASSAYS. BECAUSE WE WERE INTERESTED IN THIS CASE IN IMPACTS ON METABOLISM WE CHOSE AN ASSAY WE WERE LOOKING AT ENZYME INHIBITION, GLUCORONIDATION IN VITRO, AND THEN WE INCORPORATE THE DATA FROM THE CHEMICAL ANALYSIS AND FROM THE BIOLOGICAL EVALUATION AND USE MULTIVARIATE STATISTICS TO PREDICT WHICH OF THE COMPOUND MIGHT BE ACTIVE AND THIS PART IS ACTUALLY THE PART THAT WE'VE THOUGHT ABOUT A LOT, AND THAT'S SORT OF DESCRIBING -- DESCRIBED IN RECENT PAPERS FROM OUR LAB. HOW DO WE INTERPRET THE DATA THAT OF? THIS IS AN EXAMPLE OF THE TYPE OF DATA WE MIGHT SEE. THIS IS WHAT WE CALL SELECTIVITY RATIO PLOT, SELECTIVITY RATIO IS A METRIC THAT CORRELATES WITH CHEMICAL COMPONENTS WITH BIOLOGICAL AFFECT. EACH LINE REPRESENTS A DIFFERENT ION THAT WAS DETECTED BY MASS SPECTROMETER, HOW THE ION CORRELATED WITH THE BIOLOGICAL EFFECT. YOU SEE A BUNCH OF THESE FEATURES, THESE IONS, THAT ARE GOING IN A STRONGLY NEGATIVE DIRECTION WHICH INDICATES THAT THEY ARE ASSOCIATED WITH THE INHIBITION OF THE ACTIVITY OF THE ENZYME WHICH IS WHAT WE WERE INTERESTED IN IN THIS CASE. AND IT TURNS OUT THAT ALL OF THOSE IONS ARE ASSOCIATED WITH VERY SAME MOLECULE, ECG. WE'VE BEEN ABLE TO TAKE A MIXTURE OF EXTRACT FROM GREEN TEA THAT CONTAINS MANY, MANY MOLECULES AND ZERO IN ON ONE OF THE MOLECULES AS BEING STRONGLY ASSOCIATED WITH THE ENZYME INHIBITORY ACTIVITY. ANYTIME THAT ONE DOES AN EXPERIMENT LIKE THIS IT'S NECESSARY TO DO SOME SORT OF FOLLOW-UP VALIDATION, BECAUSE ALL OF TO THOSE RESULTS ARE CORRELATIVE. WE CAN'T ESTABLISH CAUSALITY SO WHAT WE DO IS TESTING WITH THE PURE ISOLATED MOLECULES AND LOOK AT THEIR ACTIVITY ONCE WE'VE PREDICTED THEM TO BE RESPONSIBLE FOR THE ACTIVITY OF THE MIXTURE, AND IN THIS CASE WHAT YOU SEE IS THAT INDEED ECG AND ALSO THIS RELATED COMPOUND EGCG, BOTH FROM GREEN TEA, HAD EFFECTS ON INHIBITION OF ENZYMES, OBSERVED AS CONCENTRATIONS THAT ARE CLINICALLY RELEVANT BASED ON THE DOSAGE THAT PATIENTS RECEIVE WHEN THEY CONSUME GREEN TEA. SO, OVERALL I JUST WILL SHARE A COUPLE CONCLUDING POINTS HERE. ONE IS THAT WE'VE BEEN USING MASS SPEC AND METABOLOMICS TO CONSIDER IDENTITY AND QUALITY AND VARIABILITY OF NATURAL PRODUCTS BEFORE SELECTING THEM FOR CONDUCTING CLINICAL STUDIES, IN VITRO STUDIES SO WE STUDY THE RIGHT THING AND NOT CONTAMINATED AND COMPARABLE TO WHAT IS BEING USED BY THE PUBLIC OR CLINICAL USE. SECOND PIECE THERE'S THIS FOCUS ON VIGOR AND REPRODUCIBILITY, INTEREST IN BEING ABLE TO UNDERSTAND WHAT'S GOING ON IN THE STUDIES AND REPEAT THEM. TO DO THAT WE REALLY NEED TO KNOW WHICH CONSTITUENTS OF THE COMPLEX MIXTURES THAT WE'RE STUDYING ARE RESPONSIBLE FOR ACTIVITY, AND SO WE'VE BEEN REALLY TRYING TO SHIFT FROM OLD SCHOOL WAY OF LOOKING AT THINGS TO SHAPE WE'RE -- SAY WE'RE LOOK TO LOOK FOR MARKER COMPOUNDS AND MOVE TOWARD A COMPOUND THAT SAYS THESE ARE CONSTITUENTS IMPORTANT FOR A PARTICULAR BIOLOGIC ACTIVITY. I'VE HIGHLIGHTED DIFFERENT PEAKS TO DEMONSTRATE THAT MAYBE GREEN ONES RESPONSIBLE FOR ONE, RED MIGHT BE RESPONSIBLE FOR A DIFFERENT ACTIVITY AND WE NEED TO DISTINGUISH IN THAT MIXTURE. AND JUST A QUICK REFERENCE, I'LL MENTION THAT WE HAVE PUBLISHED SORT OF THE STRATEGIES I'M DESCRIBING HERE AND COMPARED WITH OTHER APPROACHES, IN THIS PAPER, NATURAL PRODUCTS REPORTS. IF ANYONE IS CONSIDERING A STUDY ON BOTANICAL DIETARY SUPPLEMENT, IT MAY BE WORTH LOOKING AT THIS. I'M HAPPY TO DISCUSS IT, AS APPROACH FOR CONSIDERING STUDY MATERIAL BEFORE CONDUCTING THE STUDY BECAUSE ONCE THE STUDY MATERIAL IS SELECTED IT CHANGES THE TRAJECTORY FOR THE STUDY. AND I'LL ALSO MENTION OUR GROUP HAS DONE A LOT OF WORK ON THE TOPIC -- GO AHEAD, KATHERINE. >> 30 SECONDS. >> PERFECT. I HAVE TWO SLIDES. THE GROUP HAS DONE RESEARCH ON THE TOPIC OF SYNERGY, AND IMPORTANCE OF HAVING MULTIPLE COMPONENTS IN A MIXTURE. I ONLY HAVE 30 SECONDS LEFT, I DIDN'T INCLUDE THAT BUT IF YOU'RE INTERESTED IN HEARING ABOUT THAT WORK, I'M GIVING A TALK AS PART OF THE 25-YEAR ANNIVERSARY CELEBRATION OF THE OFFICE OF DIETARY SUPPLEMENTS ON THAT TOPIC, FREE AND OPEN, OR REACH OUT TO ME LATER I'D BE HAPPY TO TALK ABOUT IT. I WILL FINISH BY ACKNOWLEDGING THE PEOPLE WHO DO THE ACTUAL WORK AND WONDERFUL COLLABORATORS ON THE PROJECTS AND OF COURSE THE FUNDERS AT THE NATIONAL CENTER FOR COMPLEMENTARY AND INTEGRATIVE HEALTH AND OFFICE OF DIETARY SUPPLEMENTS. >> THANK YOU. NOW I'M PLEASED TO WELCOME DR. MIYA WHITAKER AND DR. RANJAN GUPTA TO BRING US INTO THE PANEL DISCUSSION. I'LL BRING UP THE QUESTIONS AND DR. WHITAKER UNFORTUNATELY HER CAMERA HAS DECIDED THAT TODAY IS THE DAY IT'S GOING TO TAKE A VACATION. DR. WHITAKER UNFORTUNATELY DOES NOT HAVE VIDEO TODAY BUT WE'RE PLEASED SHE'S AVAILABLE AND WAS ABLE TO JOIN US ON AUDIO. DR. WHITAKER, WOULD YOU LIKE TO KICK OFF THE DISCUSSION SESSION? >> THANK YOU, KATHERINE. GOOD AFTERNOON, ESTEEMED COLLEAGUES, PRESENTERS, ATTENDEES. WE'RE NOW GOING TO TRANSITION INTO A RICH DISCUSSION OF CONTENT PRESENTED ON ASSESSMENT OF IMPACT OF MULTI-COMPONENT INTERVENTION. WHAT METHODOLOGIES CAN BE USED IN PRE-CLINICAL MODELS FOR STUDYING THERAPEUTIC SYSTEMS AND WHAT ARE ADVANTAGES, PROS, LIMITATIONS OR CONS OF THOSE METHODOLOGIES? LET'S START WITH DR. SHINTO. >> HI, OKAY. SO, I THINK I DESCRIBED A COUPLE MODELS WE USED TO -- I WOULD SAY I DON'T THINK MY METHODS COULD BE USED IN PRE-CLINICAL MODELS. I'M ASSUMING THAT'S LIKE ANIMAL STUDIES. SO, THIS WOULD BE HUMAN CLINICAL TRIAL RESEARCH. SOME MODELS I DISCUSSED IN DESIGNING MULTI-COMPONENT INTERVENTIONS WAS FOR THE NEUROPATHIC STUDIES, USING A DELPHI PANEL TO INFORM ON WHAT THEY THOUGHT USING THE SURVEY DATA WOULD BE, IMPORTANT THERAPIES, FOR MULTIPLE SCLEROSIS. FOR THE M3 STUDY WE, LIKE LYNDA POWELL'S TALK, WE HAD MULTIPLE PERSPECTIVES. WE SPOKE WITH THE SYSTEM WE'RE WORKING WITH, EASA, GOT INPUT FROM THE PROVIDERS THERE, INPUT FROM THE YOUNG PEOPLE WHO ARE PART OF EASA AND RESEARCH TEAM HAD A SKELETON OF THINGS WE WERE INTERESTED IN IN CLINICAL TRIAL INTERVENTION. SO THOSE ARE THE METHODS I DESCRIBED. I THINK THOSE ARE SOME METHODS THAT MIGHT BE USEFUL TO INFORM ON BUILDING AN INTERVENTION. >> THANK YOU, DR. SHINTO. HOW DID DR. POWELL, DO YOU HAVE SOME THOUGHTS TO SHARE ON THAT QUESTION? >> I WAS MUTED. WE SHOULD MAKE MORE USE OF PROOF OF CONCEPT STUDY. QUASI-EXPERIMENTAL STUDY WITH TREATMENT HAD BEEN ONLY ARM, WITH OUTCOME CLINICALLY SIGNIFICANT TARGET. WHY DO I SAY THAT? BECAUSE WE DON'T USE IT VERY MUCH. WHEN YOU LOOK AT DRUG DEVELOPMENT THEY USE IT ALL THE TIME. SMALL STUDIES OF ACCESSIBLE PEOPLE, MAKE IT EASY TO FAIL AND HAVE SOME KIND OF APPROACH TO WHERE WE CAN TRY SOMETHING, IT DOESN'T WORK, TRYING SOMETHING ELSE, TRY SOMETHING TOGETHER BUT KEEP TRYING. WHEN WE FOCUS ON RANDOMIZED DESIGNS WE CAN'T DO THAT WORK. I'M A REAL ADVOCATE OF PROOF OF CONCEPT DESIGNS. >> THANK YOU. DR. COLLINS, DO YOU HAVE ANYTHING TO SHARE ON THAT? >> YEAH, I COMPLETELY AGREE WITH WHAT LYNDA POWELL SAID, HOW IMPORTANT IT IS TO DO SMALL STUDIES EARLY ON AND TAKE ADVANTAGE OF OPPORTUNITIES. LYNDA HAS GIVEN A NUMBER OF PRESENTATIONS OVER THE YEARS AND TALKS ABOUT HOW IMPORTANT IT IS TO FAIL. I COMPLETELY AGREE WITH THAT. SOMETIMES ESPECIALLY EARLY IN PEOPLE'S CAREERS, A FAILURE JUST SEEMS OVERWHELMING, BUT WE CAN LEARN SO MUCH BY FAILURE AND DARING TO FAIL THAT'S REALLY IMPORTANT. I KNOW I EMPHASIZED KIND OF TIGHTLY CONTROLLED EXPERIMENTAL DESIGNS IN MY TALK, BUT IN THE PREPARATION PHASE OF MOST THERE ACTUALLY IS A MENTION MADE OF PILOT STUDIES AND OTHER KINDS OF KIND OF PRE-EXPERIMENTATION OR EVEN -- MAYBE EVEN NOT EXPERIMENTAL STUDIES AT ALL, WE CAN CERTAINLY ALSO LEARN A LOT FROM SECONDARY DATA ANALYSIS, ALL OF THAT KIND OF INFORMATION CAN FEED INTO AN OPTIMIZATION TRIAL. BUT ONE POINT I'D LIKE TO MAKE IS THAT LYNDA TALKED ABOUT HOW IMPORTANT IT IS TO BE ABLE TO ACCUMULATE KNOWLEDGE AND I JUST WANT TO EMPHASIZE AGAIN THAT IN OTHER FIELDS, OTHER THAN BEHAVIORAL SCIENCE AND INTERVENTION SCIENCE THEY DO A BETTER JOB OF BEING SYSTEMATIC ABOUT GATHERING KNOWLEDGE. I THINK WE CAN LEARN, I'M NOT SAYING WE'RE NOT ENGINEERING OR CHEMISTRY BUT WE CAN LEARN A LOT FROM THOSE OTHER FIELDS IN TERMS OF HOW TO ACCUMULATE KNOWLEDGE SO THAT WE DEVELOP A COHERENT KNOWLEDGE BASE THAT LATER SCIENTISTS CAN BUILD ON INSTEAD OF REINVENTING, INSTEAD OF BEHAVIORAL SCIENTISTS OR INTERVENTION SCIENTISTS REINVENTING THE WHEEL, UNFORTUNATELY I THINK WHAT'S HAPPENING A LOT NOW. >> THANK YOU FOR YOUR RESPONSE. DR. WINDSOR, A DIFFERENT QUESTION. CAN YOU TALK ABOUT THE EASA PARTICIPANTS, IF THEY ARE ONLY IN THE PROGRAM FOR TWO YEARS WHAT HAPPENS TO THEM, THEY ARE AT RISK AFTER THE TIME FRAME? MY APOLOGIES IF I -- >> THAT QUESTION MIGHT BE FOR DR. SHINTO ACTUALLY. >> YES. >> MY APOLOGIES. DR. SHINTO? >> I DID THE EASA STUDY. THIS IS A STATE-FUNDED -- THESE ARE STATE-FUNDED CLINICS, AND SO WHAT HAPPENS AFTER TWO YEARS IS IT'S NOT SO BLACK AND WHITE. THEY ARE NOT IN FOR JUST TWO YEARS, SOME IN FOR A LITTLE LONGER. THE GOAL IS STABILIZATION. SO, AGAIN, THE GOAL IS NOT JUST TREAT AND DECREASE SYMPTOMS. IT'S ABOUT STABILIZATION, IF THE YOUNG PERSON IS A TEENAGER, STABILIZATION IN SCHOOL, EDUCATION COORDINATORS. IF A YOUNG PERSON IS IN WORK STABILIZATION IN THE WORK ENVIRONMENT. SOME OF THAT WILL DEPEND ON HOW THE YOUNG PERSON IS DOING AT THE TWO-YEAR MARK AND IF THEY NEED MORE TIME THEY CAN STAY IN MORE TIME. THE OTHER THING I DIDN'T ADD EASA HAS EASA YOUTH LEADERS, AND THESE ARE EASA CLIENTS WHO HAVE FINISHED TWO-YEAR MARK OR THREE-YEAR MARK AND WHO ARE STILL KIND OF IN TOUCH WITH EASA AND AS MENTORS, THE SYSTEM ITSELF HAS PEER MENTORS FOR THE PARTICIPANTS WHO HAVE JUST STARTED THE PROGRAM. AND SO SOME OF THE PARTICIPANTS MAY HAVE AN EASA YOUTH LEADER MENTOR TO HELP THEM THROUGH TRANSITION BUT THERE IS CHECK ON THAT TRANSITIONAL TWO-YEAR PERIOD WITHIN THE EASA SYSTEM. I JUST WANTED TO COMMENT ON THIS PROOF OF CONCEPT BECAUSE I'M TOTALLY IN LINE WITH WHAT LINDA COLLINS AND LYNDA POWELL HAVE SAID. BOTH NATUROPATHIC STUDY AND THIS M 3 STUDIES WITH YOUNG PEOPLE WHO HAVE FIRST EPISODE PSYCHOSIS, THESE ARE BOTH PROOF OF CONCEPT STUDIES. YOU NOTICE THE DESIGNS WERE NOT PERFECT. THEY WEREN'T CONTROLLED FOR EVERYTHING. BUT WHAT WE FOUND OUT, I WISH WE HAD A PUBLICATION FOR ALL THE THINGS THAT DIDN'T WORK OUT, BECAUSE ALL THE THINGS THAT DIDN'T WORK OUT IN PROOF OF CONCEPT STUDY REALLY INFORM ON YOUR NEXT STEPS, AND I'LL USE M3 AS AN EXAMPLE. SO WE ACTUALLY WANTED TO LOOK AT RESILIENCE IN M3. WE WANTED TO LOOK AT QUALITY OF LIFE. WE WEREN'T ABLE TO DO THAT. WE REALIZED IN THIS POPULATION GIVING A RESILIENCE MEASURE, WE WERE CEASE TO USE CD RISK BUT WE LEARNED WITHIN A WEEK OF SURVEYING THAT THE SURVEY BURDEN HAD TO BE VERY LOW, SURVEYS HAD TO BE SHORT, WE COULDN'T GET A EVERY BLOOD DRAW. THERE WERE ALL THESE THINGS THAT WE LEARNED THAT WE COULD ADAPT TO A NEXT PHASE. WHAT'S NOT PUBLISHED AND WHAT'S NOT SPOKEN OF ALL THE TIME WE ALWAYS WANT TO TALK ABOUT OUR SUCCESSES, BUT I WOULD SAY THAT 50% OR MORE ARE THINGS THAT DON'T WORK OUT AND IT'S REALLY THE THINGS THAT DON'T WORK OUT THAT INFORM A STUDY SOMETIMES. ESPECIALLY PROOF OF CONCEPT STUDIES. >> THANK YOU. WELL ARTICULATED AND I AGREE. DR. JENSEN, A QUESTION FOR YOU. SO GIVEN YOUR WORK WITH MULTI-COMPONENT PAIN INTERVENTIONS WHAT ARE SOME STRATEGIES THAT YOU'VE USED TO ATTUNE INTERVENTIONS OR ASSESSMENT APPROACHES TO HEALTH OF WOMEN, FOR EXAMPLE, OR CULTURALLY CENTERING ON INTERVENTION AND ASSESSMENT APPROACHES IN MARGINALIZED POPULATIONS? >> WE'RE JUST NOW PUTTING TOGETHER A PROJECT TO WORK WITH OUR LOCAL TRIBAL COMMUNITY TO DEVELOP A CHRONIC PAIN INTERVENTION SPECIFIC TO AND CULTURALLY APPROPRIATE FOR THAT COMMUNITY. AND I THINK THAT GOES BACK TO THE ISSUES ABOUT THE IMPORTANCE OF PUTTING IN THE TIME AND EFFORT AT THE BEGINNING, THAT DOES TAKE A LOT OF TIME AND EFFORT. AND I THINK WE SPEND TOO MUCH OF OUR LIMITED RESOURCES FUNDING TO FUND THESE LARGE RCT TRIALS THAT TURN OUT TO BE VERY EXPENSIVE, AND THAT WE NEED TO PROBABLY PUSH SOME OF THOSE RESOURCES TO HELP US DEVELOP ONES THAT ARE MORE APPROPRIATE FOR INDIVIDUAL COMMUNITIES. WHEN WE LOOKED AT INDIVIDUAL COMMUNITIES AS A MODERATOR OF OUTCOME WE DON'T SEE A LARGE EFFECT BETWEEN MEN AND WOMEN BUT NEED TO PUT MORE TIME AND EFFORT INTO DEVELOPING INTERVENTIONS TO BE MORE SPECIFIC AND APPROPRIATE TO THE INDIVIDUAL COMMUNITIES. >> THANK YOU. NEXT QUESTION FOR DR. WINDSOR. WHAT ARE SOME OF THE FINDINGS IN THE PROCESS TO HELP MAKE YOUR INTERVENTION MORE POTENT? ALSO IF YOU COULD TOO PROVIDE SOME OF YOUR INSIGHTS ON CULTURAL CENTERING AND GENDER SENSITIVE APPROACHES TO MULTI-COMPONENT INTERVENTIONS. >> THANK YOU FOR THE QUESTION. IT GIVES ME THE OPPORTUNITY TO TALK ABOUT SECTORIAL DESIGNS. WE HAD FOUR COMPONENTS, EXPERIMENT WITH SUBSTANCE USE OUTCOME FOUND TWO COMPONENTS WERE THE MOST PROMISING FOR INCLUSION BECAUSE INTRODUCING SUBSTANCE USE WAS SIGNIFICANTLY LOWER FOR THE PEOPLE WHO RECEIVED THOSE COMPONENTS VERSUS PEOPLE THAT DID NOT. THEN ON TOP OF THAT WE FOUND PEER FACILITATORS WERE ABLE TO DELIVER COMPONENTS AS EFFECTIVELY AS LICENSED FACILITATOR. THAT ALLOWED US TO GET RID OF THE DEAD WOOD. IN FACT, THE COMPONENT THAT WE REMOVED HAD AN OPPOSITE DIRECTION EFFECT THAT WAS KNOT STATISTICALLY SIGNIFICANT. WE'RE LOOKING AT BOTH CLINICAL AND STATISTICAL SIGNIFICANCE. BY DOING THAT WE'RE ABLE TO MAXIMIZE THE EFFICACY THAT THIS COMBINED INTERVENTION IS GOING TO HAVE IN OUR FUTURE RANDOMIZED CLINICAL TRIAL. IN THE OTHER PIECE THAT'S BRILLIANT IN THE MOST FRAMEWORK, THE ACCUMULATION OF DATE, DATA FACTS OVER TIME. SO IN MY DATA, FOR EXAMPLE, WE OPTIMIZED WITH THAT PARTICULAR POPULATION TO FIND THAT MOST EFFICIENT AND EFFECTIVE COMPONENTS THAT COULD BE DELIVERED TO REDUCE SUBSTANCE USE FOR $200 OR LESS PER PERSON, THAT WAS OUR OPTIMIZATION CRITERION. NOW WE HAVE THAT DATA. IF THERE WAS A COMMUNITY, FOR EXAMPLE, THAT COULD USE MORE FUNDING TO COVER THE COSTS OF THAT INTERVENTION, WE COULD ACTUALLY RUN THE ANALYSIS TO SEE IF OPTIMIZED INTERVENTION WOULD BE DIFFERENT USING THE SAME DATASET. AS MORE SCIENTISTS START ENGAGING IN THIS APPROACH, AND USING FACTORIAL DESIGNS TO IDENTIFY INDIVIDUAL IMPACT OF INDIVIDUAL COMPONENTS AND EFFECT OF THOSE INDIVIDUAL COMPONENTS, WE CAN ACTUALLY BE POTENTIALLY LINKING THAT DATA TO IDENTIFY COMPONENTS THAT MIGHT BE MORE PROMISING FOR DIFFERENT POPULATIONS VERSUS OTHER POPULATIONS THAT WOULD HELP AS WELL IN TERMS OF MAKING THINGS MORE CULTURALLY TARGETED. THE OTHER THING, IMPORTANCE OF INCLUSION OF COMMUNITIES, AS YOU ARE DOING THIS WORK EARLY ON, AND REALLY THINKING ABOUT COMMUNITY CULTURE AS YOU'RE DEVELOPING THESE INTERVENTIONS, AND THEN YOU CAN TEST IT IN OTHER PLACES WHEN YOU'RE IN THE IMPLEMENTATION PHASE. I'LL STOP THERE. >> NEXT QUESTION FOR DR. CECH, COULD METHODS YOU DESCRIBED FOR IDENTIFYING COMPONENTS OF BOTANICAL AND VALIDATING BE USED IN CLINICAL STUDIES OF MULTI-COMPONENT INTERVENTIONS VALIDATING WHICH COMPONENTS WITH ACTIVE? >> I THINK IT'S A GREAT QUESTION THAT I THINK ABOUT A FAIR AMOUNT. IT'S EXCITING TO BE IN PANEL WITH FOLKS DOING CLINICAL STUDIES, NICE TO COLLABORATE WITH PEOPLE DOING CLINICAL STUDIES RECENTLY, AND AT LEAST DOING THAT PRE-CLINICAL WORK TO HELP SELECT THE RIGHT STUDY MATERIALS. I THINK DR. WEBER'S POINT IS TO THE QUESTION OF WHETHER THERE COULD BE MORE INTEGRATION THERE, WHERE APPROACHES WE'RE USING ARE HELPING TO DETERMINE WHICH COMPONENTS OF MIXTURES THAT ARE BEING USED IN CLINICAL STUDIES ARE THE ACTIVE ONES. THE ANSWER MIGHT BE YES BUT ONE OF THE MAIN CHALLENGES WOULD BE THAT WHILE IT'S POSSIBLE TO GET APPROVAL TO TEST BOTANICAL EXTRACT IT'S NOT SO POSSIBLE FOR US TO GET THE APPROVAL TO PUT NECESSARILY FRACTIONS OF EXTRACT INTO A CLINICAL STUDY. THAT TENDS TO BE THE WAY WE DISTINGUISH. IN PRE-CLINICAL AND IN VIVO ANIMAL WORK WILL BE THE WAY TO GO BUT I WOULD LOVE TO BRAINSTORM IDEAS BUT IMPORTANT TO USE PRE-CLINICAL WORK TO INFORM THE RIGHT STUDY MATERIAL ONCE WE DO THE CLINICAL STUDY, AND THAT'S WHERE WE'RE AT RIGHT NOW. >> YOU HAVE A READOUT OF DIFFERENT COMPONENTS IMPACTING A PARTICULAR PATHWAY, FOR EXAMPLE, AND SO THINKING OF SORT OF THE MECHANISTIC WORK THAT DR. JENSEN IS DOING AND HOW YOU COULD POTENTIALLY LOOK AT MECHANISTIC EFFECT OF MULTI-COMPONENTS OF THESE THROUGH THE CLINICAL SPACE WONDERING IF THE WAY YOU ANALYZE DATA TO BE USE TO ANALYZE MECHANISTIC RESEARCH OUTCOMES OR MAYBE IT'S BIOLOGICAL MEASURES OR SOMETHING ELSE THAT WE LOOK AT AND SORT OF HOW YOU ARE USING PRINCIPAL COMPONENT ANALYSIS AND OTHER APPROACHES WHETHER THAT MIGHT BE A WAY TO SORT OF IDENTIFY -- YESTERDAY WE TALKED ABOUT DIFFERENT GENETICS, AND EPIGENETIC OUTCOMES WHERE YOU HAVE JUST A LOT OF DATA. AND YOU SORT OF FOUND LIKE ALL OF THESE SPECIFIC METABOLITES SORT OF CLUSTERED TOGETHER, ALL HAD A LARGE EFFECT WHEN YOU LOOKED AT OUTCOMES, WE'RE SEEING THAT ON THE EPIGENETIC SIDE THAT SOMETIMES WHEN YOU'RE LOOKING AT OMICS WORK, MULTIPLE THINGS LIGHT UP, AND CERTAIN PATHWAYS OR OTHER THINGS LIKE THAT THAT REALLY SORT OF IN SOME WAYS VALIDATE ONE ANOTHER. WHAT I THOUGHT WAS INTERESTING, WHAT YOU DO, YOU DID THAT NOT ONLY TO IDENTIFY WHICH CONSTITUENT BUT THEN THE NEXT STEP OF CONFIRMING THAT THAT PARTICULAR COMPONENT HAD AN EFFECT AND COULD WE USE THAT TYPE OF APPROACH IN CLINICAL RESEARCH IN ANY WAY. >> YEAH, I THINK NOW I SEE WHAT YOU'RE GETTING AT, WENDY, MAYBE AN EVEN BETTER QUESTION THAN THE ONE I THOUGHT YOU WERE ASKING. YEAH, I THINK A LOT OF THESE STATISTICAL APPROACHES ARE APPLICABLE IN OTHER CONTEXT, TRYING TO CORRELATE THIS WITH THIS, AND THIS IS COMPLEX AND THIS IS ALSO COMPLEX. SO, YOU KNOW, ON A FUNDAMENTAL LEVEL, I THINK THAT SOME OF THOSE SAME METHODOLOGIES COULD APPLICABLE TO AT LEAST MAKE PREDICTION AND SOMEHOW VALIDATE IT. I GUESS WITH SOME OF THE OTHER PRESENTERS TODAY, LINDA TALKED ABOUT -- LINDA COLLINS TALKED ABOUT IMPORTANCE OF THE PILOT STUDY, AND LYNDA POWELL I THINK. THAT IDEA THAT YOU COULD DO THE PILOT STUDY AND LOOK AT THE POSSIBLE DIFFERENT FACTORS AND THEN DO THE VALIDATION ONCE YOU HAVE A LIST OF -- ONCE YOU DETERMINED WHAT SEEMS TO BE THE MOST LIKELY ONE. >> THANK YOU. >> I THOUGHT WENDY BROUGHT UP AND INTERESTING POINT. I'M ALSO CURIOUS ABOUT IN THE PRE-CLINICAL WORLD, NOT THE WAY YOU'RE ANALYZING, WHAT IS THE SAMPLE SIZE AND THAT YOU ARE LOOKING AT TO DO THE PRINCIPAL ANALYSIS? THE REASON I'M ASKING, I WONDER IN CLINICAL STUDIES, IN THE CLINICAL EXAMPLES THAT WERE DISCUSSED EARLIER IN THE SESSION BY THE SPEAKERS, I WONDER, YOU KNOW, HOW DIFFICULT IT IS FOR THEM TO DO SOMETHING SIMILAR IN THE HUMAN SUBJECT, WOULD THAT BE TOO EXPENSEIVE, NOT FEASIBLE, OR MAYBE THEY COULD DO IT. I'M CURIOUS TO THE THOUGHTS FROM OTHERS IN THE SESSION AS WELL. >> YEAH, DR. CHEN, THAT'S WHAT I WAS THINKING ALSO WHEN I WAS FIRST ANSWERING THE QUESTION IS THAT THE CHALLENGE OF SCALING PRE-CLINICAL TO CLINICAL, IT GETS MORE EXPENSIVE FROM IN VITRO TO ANIMALS TO HUMANS. IT'S HARDER TO HAVE AS LARGE OF A SAMPLE SIZE. WITH PART OF THE POWER OF THESE MULTIVARIATE STATISTICAL APPROACHES THAT SIMPLIFY THE DATA ANALYSIS TO FIND GROUPS THAT ARE RELATED AS OPPOSED TO LOOKING ONE BY ONE YOU'LL HEAR STATISTICIANS SOMETIMES SAY, WELL, YOU HAVE TO HAVE AS MANY BIOLOGICAL OBSERVATIONS AS HAVE YOU CONSTITUENTS IN YOUR SAMPLES, IF YOU HAVE A THOUSAND COMPOUNDS YOU HAVE TO TEST A THOUSAND TIMES BIOLOGICALLY. BUT THAT'S NOT REALLY HOW MULTIVARIATE STATISTICS WORKS. WE CAN SIMPLIFY. ON YOU SAMPLE SETS ARE MORE LINE 20 BIOLOGICAL MEASUREMENTS AND WE'RE STILL ABLE TO MAKE INTERESTING PREDICTIONS FROM 20 MEASUREMENTS. THERE POSSIBLY ARE WAYS TO DO THIS IN MORE OF A CLINICAL SAMPLE TYPE OF CONTEXT. >> THANK YOU. >> SO FOR THE NEXT WE HAVE A QUESTION FOR THE ENTIRE PANEL. CAN EACH ONE OF YOU TAKE A MINUTE TO TALK ABOUT WHETHER THERE'S A NEED FOR PROOF OF CONCEPT STUDIES OF INTERVENTIONS THAT ARE SIMILAR TO EFFECTIVE INTERVENTIONS THAT HAVE BEEN STUDIED? HOW MANY FEASIBILITY STUDIES DO YOU THINK ARE NEEDED FOR SIMILAR INTERVENTIONS. >> I CAN START WITH THAT ONE. THE ESSENCE OF A PROOF OF CONCEPT STUDY IS THE QUESTION OF A PLAUSIBLE CLINICAL SIGNAL. IT'S NOT SAYING THERE IS A CLINICAL SIGNAL THERE BUT SAYING IS IT PLAUSIBLE, IT'S A DECISION-MAKING TOOL. WHAT WE AGREE ON IS RE WE HAVE A NUMBER OF SOPHISTICATED METHODS BUT WHAT WE NEED IS A BETTER UNDERSTANDING OF THE PROCESS AS WE MOVE FORWARD. AND ASK A PROGRESSION OF QUESTIONS THAT LEAD TO PROGRESSION OF STUDIES. AND SO THE ISSUE ABOUT FEASIBILITY STUDIES, HOW MANY FEASIBILITY STUDIES DO WE NEED TO DO, IT'S FEASIBILITY OF WHAT? IS IT FEASIBILITIES OF THE TRIAL PROTOCOL? WHICH MEANS WE WOULD NEED TO DO A NEW ONE IF THERE WAS A NEW TRIAL PROTOCOL. IF WE'RE THINKING ABOUT DOING A PROOF OF CONCEPT AND IT'S ALREADY BEEN DONE, ON A PARTICULAR OUTCOME, AND WE WANT TO SHIFT, WE NEW - THINK TREATMENT IS GOOD ENOUGH AND WANT TO APPLY DIFFERENT OUTCOME YOU ABSOLUTELY DOO PROOF OF CONCEPT TO GET AT ISSUE OF PLAUSIBLE CLINICAL SIGNAL. I THINK THESE METHODS, DISCUSSION OF THESE METHODS AS I'M LISTENING IS FASCINATING. I'M LEARNING SO MUCH MORE ABOUT THE VARIETY OF DIFFERENT METHODS THAT WE HAVE FROM JUST LISTENING TO THE OTHER PANEL MEMBERS. I THINK THE ESSENCE OF OUR PROBLEM IS FIGURING OUT WHAT THE SET OF QUESTIONS ARE WE WANT TO ASK AND THEN LINKING THE METHODS THAT EXIST ALREADY TO AN APPROPRIATE -- THE BEST ANSWER TO THE QUESTION THAT WE'RE POSING. >> GO AHEAD. >> I COMPLETELY AGREE WITH WHAT LINDA SAID AND IF I CAN SUM OF, IF THE QUESTION IS CLEARLY DEFINED, THEN YOU HAVE -- THAT IS TO SAY THE QUESTION YOU'RE INVESTIGATING EMPIRICALLY IS CLEARLY DEFINED THAT HELPS YOU FIGURE OUT HOW MUCH EXPERIMENTATION NEEDS TO BE DONE. AND SO IT'S REALLY HARD TO I THINK GET ANY KIND OF BLANKET ANSWER TO THE QUESTION THAT YOU JUST POSED, ALTHOUGH IT'S VERY INTERESTING. WHENEVER YOU START TO TALK ABOUT EXPERIMENTATION, EXPERIMENTAL DESIGN, HOW MANY SUBJECTS YOU NEED, HOW MUCH EXPERIMENTATION YOU NEED TO DO, IT'S REALLY HARD TO TALK ABOUT THAT IN THE ABSTRACT. YOU HAVE TO HAVE A CLEARLY STATED SCIENTIFIC QUESTION. THEN YOU CAN START TO TALK ABOUT THAT OTHER STUFF. BUT THAT ALL FOLLOWS FROM A CLEARLY STATED SCIENTIFIC QUESTION. >> TO SAY THAT SIMPLY, THE QUESTION DRIVES THE METHOD RATHER THAN THE METHOD DRIVING THE QUESTION. >> YES, ABSOLUTELY. >> TO GO ALONG WITH THAT WE HAVE A QUESTION ABOUT DATA, TIMES OF DAFT THAT CAN BE CAPTURED IN PRE-CLINICAL AND HUMAN SUBJECT STUDIES. HUMAN BEINGS AT THE CELLULAR LEVEL AND BEYOND ARE VERY COMPLICATED SO HOW DO YOU FRAME YOUR RESEARCH QUESTIONS IN THE WAY THAT ALLOWS FOR YOU TO GET MAXIMIZE DATA CAPTURED BUT ALSO HAVE THE -- CREATE INCREASED POTENTIAL TO BE ABLE TO ANSWER YOUR QUESTION FULLY? ANY OF THE PANELISTS CAN KIND OF DIVE IN. >> I'M NOT SURE I UNDERSTAND THE QUESTION. TRY IT AGAIN. >> OKAY, SURE. SORT OF LINKING TWO QUESTIONS. WE HAD A QUESTION EARLIER ABOUT THE TIMES OF DATA THAT COULD BE CAPTURED IN CLINICAL AND HUMAN STUDIES. AND SO THAT WAS THE INITIAL QUESTION BUT I THOUGHT EARLIER ABOUT DR. WINDSOR AND JENSEN'S TALK ABOUT HOW COMPLEX HUMAN BEINGS ARE, THINKING IN TERMS OF EXPERIMENTAL DESIGN AND LOOKING AT QUESTIONS HOW DO YOU KIND OF WEIGH ALL OF THAT WHEN IT COMES TO LOOKING AT THE TIMES OF -- TYPES OF DATA YOU'LL COLLECT. >> I CAN JUMP IN BECAUSE THESE ARE THE QUESTIONS WE ALWAYS ADDRESS WHEN STARTING TO DESIGN A STUDY. AND WE ALWAYS WANT TO ASSESS EVERYTHING. AND WE CAN'T. SO, WHAT WE END UP DOING IS GOING BACK TO THE THEORETICAL MODEL, WHAT DOES THE MODEL TEST US WE NEED TO ASSESS? USUALLY IT'S BIOLOGICAL, SOCIAL, ENVIRONMENTAL, ISSUES. GEE WHAT DO WE KNOW ABOUT CRITICAL ISSUES AND MEASURES VALID AND RELIABLE, HAVE THE FEWEST ITEMS. WE ASK WHAT ADDITIONAL COOL IDEAS ARE OUT THERE THAT REALLY HAVEN'T BEEN TESTED YET. WE ALWAYS INCLUDE THOSE BECAUSE EVERY STUDY I THINK IS CRITICAL, YOU KNOW, TEST OUR HYPOTHESES AND HAVE A DISCOVERY PIECE. AND SOMETIMES WE GO SO FAR AS TO TELL PARTICIPANTS, YOU KNOW, YOU'RE DONE BUT HERE'S THIS OTHER PIECE THAT'S OPTIONAL THAT'S GOING TO GIVE US NEW IDEAS, SO THAT'S A STRATEGY. IF YOU HAVE TIME AND INTEREST, PLEASE ANSWER THESE ADDITIONAL QUESTIONS, WE COLLECT MORE DATA FOR IDENTIFYING FUTURE POTENTIAL ISSUES. IT'S ALWAYS THE SECONDARY DISCOVERY ANALYSES THAT END UP BEING MOST EXCITING, ALTHOUGH WE PUBLISH THE PRIMARY PAPER FIRST. >> I ALSO WANTED TO JUMP IN HERE AND HIGHLIGHT THE IMPORTANCE OF THINKING ABOUT THIS AS A PROCESS AND NOT AS A SINGLE STUDY. AND FOR ME VERY MUCH AS MUCH ART AS IT IS A SCIENCE BECAUSE YOU REALLY HAVE TO LOOK AT WHAT IS THE PROBLEM YOU HAVE AT HAND, AND THEN WHAT DO FUNDERS WANT, WHAT DOES YOUR COMMUNITY WANT, WHAT DOES SCIENCE SAY, WHAT ARE MEASURES AVAILABLE AND PUT ALL THIS TOGETHER IN THAT POT TO TRY TO COME UP WITH A QUESTION, RIGHT? AND THEN AS YOU DO THAT STUDY YOU HAVE YOUR FINDINGS THAT ARE GOING TO END UP LEADING TO NEW QUESTIONS AND HOPEFULLY THOSE NEW QUESTIONS ARE BEING INFORMED BY THE FINDINGS YOU HAVE AND NEW THINGS YOU DIDN'T EXPECT THAT WERE GOING TO HAPPEN THAT END UP LEADING YOU TO WHERE YOU NEED. I THINK THAT ALL OF THESE THINGS TOGETHER ALSO HELP INFORM HOW MANY PILOT AND INITIAL STUDIES YOU HAVE BEFORE YOU ARE READY SO ONE EXAMPLE IN MY STUDY WE THOUGHT WE WERE GOING TO HAVE TO DO AN EFFICACY TRIAL BEFORE WE DID THE IMPLEMENTATION TRIAL. BUT AS WE SUBMITTED THAT FOR REVIEW PART OF THE ARGUE; THE CONCERN WE GOT BACK, YOU DON'T NEED TO DO AN EFFICACY TRIAL, JUST JUMP AHEAD TO THE IMPLEMENTATION BECAUSE YOU HAVE EVIDENCE TO MAKE THE CASE. IT'S REALLY ABOUT LOOKING AT WHAT HAVE YOU AT ANY GIVEN TIME AND LETTING THAT INFORM YOUR DECISIONS. >> MAY I COMMENT? THE THEME EMERGING RESEARCH IS ITERATIVE AND DISCOVERY DRIVEN OR AT LEAST SHOULD BE. THAT'S HOW WE WERE TRAINED AT SCIENTISTS. FOR THOSE WHO ARE NIH PROGRAM OFFICIALS, I GUESS I WOULD LIKE YOU TO ASK WHETHER THE WAY WE AS SCIENTISTS ARE ASKED TO WRITE GRANT PROPOSALS IS CONSISTENT WITH THAT. THAT IDEA THAT SCIENCE SHOULD BE ITERATIVE AND DISCOVERY DRIVEN. >> THANK YOU, DR. COLLINS. DR. GUPTA, GO AHEAD. >> THANK YOU. I HAD A VERY FOR DR. CECH. I LIKED THE STUDY, BOTANICALS. AS GO THROUGH VERIFICATION PROCESS, EVEN INITIALLY GREEN TEA TAKEN ORALLY AS PEOPLE DRINK IT BUT WHEN YOU GO TO THAT LEVEL AND ADMINISTER WHETHER PRE-CLINICAL OR CLINICAL, DOES YOUR MODE OF ADMINISTRATION CHANGE FROM BEING ORAL ADMINISTRATION TO SOMETHING ELSE AND IF THAT DOES HOW DOES THAT IMPACT IN THE END TO WHAT PEOPLE NORMALLY USE VERSUS OUTCOMES IN YOUR KIND OF STUDY? THANK YOU. >> THANK YOU, DR. GUPTA. YOU'RE ASKING IF THE METHOD OF ADMINISTRATION CHANGES? >> RIGHT. BECAUSE IT'S NO LONGER A TEA EXTRACT, AND THEN IF YOU CHANGE THE MODE OF ADMINISTRATION DOES THAT CHANGE THE RESULT OUTCOME? >> SURE. WELL, I THINK THAT YOUR QUESTION WHETHER YOU MIGHT CHANGE THE OUTCOME BY CHANGING MODE OF ADMINISTRATION IS EXCELLENT, THE ANSWER IS YOU COULD. THAT'S AN INSIGHTFUL THOUGHT. THE WORK THAT WE DID WITH GREEN TEA, ALL OF THE BIOLOGICAL EVALUATION THAT WAS BEING DONE WAS DONE IN VITRO. SO WE'RE USING ESSENTIALLY EXTRACT OR SUBFRACTIONS OF EXTRACTS DRIED AND ADDED TO AN ENZYME MIXTURE OR IN SOME CASES A CELL-BASED ASSAY. SO I THINK IN THOSE CASES MODE OF ADMINISTRATION IS NOT THE QUESTION. IF WE'RE DOING PRE-CLINICAL STUDY, FEEDING TO ANIMALS OR SOMETHING LIKE THAT, I THINK IT IS IMPORTANT CERTAINLY TO MAKE SURE THE MODE OF ADMINISTRATION IS CONSISTENT WITH THE EXTRACTS WHICH CAN BE DONE. IF ANOTHER THING WE THINK ABOUT IS WHETHER WHAT WE'RE DOING IS RELEVANT TO SOMEONE ACTUALLY CONSUMING THE TEA, I THINK THIS IS PART OF WHAT YOU'RE GETTING TO WE HAVE TO THINK CAREFULLY WHEN DESIGNING OUR PRE-CLINICAL STUDIES TO MAKE SURE WE HAVEN'T GONE AND VEERED OFF INTO THE DIRECTION NO LONGER RELEVANT BY ACTUAL USE BY PATIENTS OR POPULATION THAT'S USING THIS BOTANICAL MEDICINE SO THAT'S A CONVERSATION THAT WE ALWAYS HAVE AND SOMETIMES THERE ARE SHORT CUTS IN THE LAB WOULD MAKE IT EASIER BUT IF IT MAKES IT LESS RELEVANT WE TRY NOT TO DO THAT. >> I ASKED BECAUSE WE RAN INTO SIMILAR ISSUES STUDYING NATURAL PRODUCTS. THANK YOU. >> I'M SURE. THOSE ARE TOPICS NO ONE TALKS ABOUT BUT THE ONES THAT END UP MAKING OR BREAKING A STUDY, RIGHT? >> DR. CHEN, GO AHEAD. >> YES, THANK YOU. I WANT TO GET BACK TO WHAT DR. COLLINS WAS ASKING REGARDING NIH FUNDING RELATED TO STUDIES, CLINICAL TRIALS. THIS IS WHY WE'RE HAVING THIS WORKSHOP, TRYING TO EXPLORE METHODOLOGIES AND CHALLENGE ISSUES SO THAT WE CAN OPTIMIZE OUR FUNDING INITIATIVE EFFORTS TO SUPPORT THIS AREA OF RESEARCH, WHETHER WE HAVE -- WENDY CAN GIVE MORE COMMENTS ABOUT THE SUITE OF FOAs BUT I WANT TO POINT OUT WE'RE WORKING TOWARDS THE OPT OPTIMAL WAY TO SUPPORT THIS AREA OF RESEARCH. >> THANK YOU. NEXT QUESTION FOR DR. COLLINS, THANK YOU FOR THE FRAMEWORKS DISCUSSION AND PUSH TOWARD PARADIGM SHIFT. REGARDING AFFORDABILITY ASPECT, ASIDE FROM TIME, MONEY, STAFFING, ET CETERA, HOW MUCH OF A PRIORITY WOULD YOU ASSIGN TO PATIENT OR CLIENT BURDEN SIDE EFFECTS PRIOR TO COST, ET CETERA. >> YEAH, ABSOLUTELY, ALL THAT CAN BE INCORPORATED INTO DECISION MAKING, AN AVE AREA OF RESEARCH IN MY LAB IS WAYS TO IMPROVE DECISION MAKING BASED ON RESULTS OF AN OPTIMIZATION TRIAL. YES, DEFINITELY REALLY ANY CONSIDERATION CAN BE INCLUDED IN THE DECISIONMAKING BUT PARTICIPANT BURDEN CAN BE INCLUDED. >> THE AMOUNT OF TIME, A LIMIT ON TIME, INTERVENTION, THAT IS RELATED TO PARTICIPANT BURDEN. THERE'S A LOT OF DIFFERENT WAYS THAT THOSE CONSIDERATIONS CAN BE INCLUDED IN DECISION MAKING. DOES THAT ANSWER THE QUESTION? >> VERY MUCH SO. THANK YOU. THAT WAS GREAT. >> GETTING BACK TO NIH PERSPECTIVE I WANT TO POINT OUT THIS SESSION HAS BEEN EXCITING FOR ME BECAUSE I'VE BEEN INVOLVED FOR A LONG TIME WITH NIH INITIATIVES THAT ARE COMPLETELY CONSISTENT WITH WHAT EVERYBODY IS TALKING ABOUT HERE. SO OVER TEN YEARS AGO WE HAD A COMMON FUND PROJECT, SCIENCE OF BEHAVIOR CHANGE, WHICH WAS DIRECTED AT TRYING TO UNDERSTAND TARGETS AND MECHANISMS BY WHICH DIFFERENT INTERVENTIONS MIGHT WORK. IN MANY CASES WE FOCUSED ON TOPICS THAT MANY SPEAKERS HAVE TALKED ABOUT LIKE STRESS, INTERPERSONAL RELATIONSHIPS AS POSSIBLE MEDIATORS. ALSO I WOULD LIKE TO POINT PEOPLE TO THE NIH STAGE MODEL WHICH IS ONE VERSION OF AN ITERATIVE MODEL FOR MOVING FROM BASIC SCIENCE TO IMPLEMENTATION WHILE CONSIDERING SCALABILITY, AFFORDABILITY, FIDELITY, ALL THOSE THINGS FROM THE VERY IMPINGE. -- BEGINNING. THAT IS AN ACTIVE AREA WE'RE INTERESTED IN HAVING MORE RESEARCH DONE WITH THOSE METHODS. >> THANK YOU, DR. SIMMONS. CAN MEDIATION ANALYSIS OR OTHER METHODS IDENTIFY WHICH COMPONENTS ARE EFFECTIVE FOR A GIVEN HAVE HAD AND -- INDIVIDUAL AND COULD YOU COMMENT ON MEDIATION ANALYSES, HOW THEY ARE POWERED AND LIMITING TYPE 2 ERROR. >> FIRST QUESTION WITH RESPECT TO MEDIATION, MEDIATION ANALYSES IS FOCUSED ON MECHANISMS. IT'S A MECHANISTIC ANALYSIS. SO IF, FOR EXAMPLE, YOU IDENTIFY AMOUNT OF EXERCISE IS THE MOST IMPORTANT PIECE, THEN YOU WOULD GO BACK AND HAVE TO LOOK AT WHICH OF THE COMPONENTS ARE MOST LIKELY TO AFFECT THAT MECHANISM. SO IT DOESN'T SPECIFICALLY TEST THE COMPONENTS. IT TESTS MECHANISMS THAT UNDERLIE THOSE AND YOU CAN USE THAT INFORMATION TO GUIDE WHICH COMPONENTS YOU MIGHT THEN USE. I THINK YOU HAVE TO GO TO SOME ANALYSES DR. COLLINS TALKS ABOUT OR REFERS TO, REFERRED TO IN HER TALK, ABOUT LOOKING AT EFFECTS OF SPECIFIC COMPONENTS. POWERING MEDIATION ANALYSES IS A CHALLENGE, AND I'M NOT A STATISTICIAN, IN THE LAST APPLICATION HAS DONE IT SO IT'S POSSIBLE. YOU CAN'T GO TO G POWER TO DO IT. IT'S POSSIBLE TO POWER THEM AND IF YOU HAVE QUESTIONS I'LL SHARE WHAT THE BIOSTATISTICIAN DID. THERE IS NO INFORMATION ABOUT THE CLINICAL MEANINGFUL EFFECT SIZE OF MEDIATION. THAT'S WORK THAT NEEDS TO BE DONE. AS WE HAVE SOME INFORMATION ABOUT CLINICAL MEANINGFUL CHANGES, CLINICAL OUTCOMES, WE DON'T HAVE DATA TO HELP GUIDE US WHAT IS A CLINICALLY MEANINGFUL MEDIATION EFFECT. THE TYPE 1 ISSUE, THAT GETS TO ONE OF MY PET PEEVES, WHAT I REFERRED TO AS TYRANNY OF THE P-VALUE, IF WE ARE REQUIRED TO MAKE A DETERMINATION AT ONE STUDY THAT MUST MEET THIS ONE P-VALUE, WE TRY TO CONTROL FOR IT, I THINK WE'RE GOING TO NOT EVERY DISCOVER ANYTHING. MY HOPE IS OVER TIME THAT INSTEAD OF LOOKING AT ONE STUDY TO ANSWER A QUESTION, THAT WE THINK IN TERMS OF USING STUDIES THAT ARE DESIGNED TO CREATE REALLY EXCELLENT ESTIMATES OF EFFECT SIZES. AND THAT WE THEN BUILD A BODY OF RESEARCH WHERE WE LOOK AT EFFECT SIZES AS THE TREATMENT HAS THIS EFFECT SIZE, THAT HAS THAT EFFECT SIZE RATHER THAN THIS IS STATISTICALLY SIGNIFICANT AND THIS IS NOT. I'M HOPING THE FIELD CAN MOVE AWAY FROM WORRYING ABOUT TYPE I AND TYPE II ERRORS SO MUCH, INCLUDING REVIEW PANELS AND MOVE TOWARDS LOOKING AT DESIGN STUDIES THAT GIVE US A GOOD UNDERSTANDING OF THE EFFECT SIZES OF INTERVENTIONS WITH THE IDEA THAT WE'RE GOING TO BUILD THE KNOWLEDGE BASE ON MULTIPLE STUDIES AND GET A SOLID ESTIMATE OF THOSE. A LOT OF QUESTIONS, A LOT OF ANSWERS. I HOPE SOME OF THAT WAS HELPFUL. >> YES, VERY MUCH SO. DR. WEBER? >> YEAH, I WAS GOING TO ASK SORT OF THE CHALLENGE THAT OFTEN COMES UP IN PEER REVIEW IS THE RIGOR AND BEING POWERED AND ABLE TO TEST CLEAR HYPOTHESES VERSUS HAVING MORE EXPLORATORY ANALYSES AND THAT SORT OF THING. I'M CURIOUS BROADLY TO THE GROUP BUT ALSO SPECIFICALLY TO MARK ABOUT HOW CAN THESE METHODS BE ENHANCED OR DO WE NEED NEW METHODS THAT CAN SORT OF DEMONSTRATE THERE IS REALLY STRONG RIGOR HERE AND THESE METHODS ARE REALLY HYPOTHESIS TESTING AND MAYBE WE NEED TO BETTER DISTINGUISH WHEN WE'RE, YOU KNOW, WHEN PROJECTS ARE HYPOTHESIS GENERATING VERSUS HYPOTHESIS TESTING. >> AS A PERSON WHO WRITES GRANTS FOR PANELS YOU CAN TELL YOU WHAT I DO TO ADDRESS WHAT OFTEN COMES BACK, NOT FRANKLY WHAT I WANT TO DO. WHAT I DO IS I STUDY SINGLE PRIMARY OUTCOME, SINGLE PRIMARY ANALYSIS, AND I SAY I HAVE THE POWER TO TEST THAT AT WHATEVER P-VALUE I CHOOSE AND THAT I SAY IN ADDITION TO TESTING THAT I PLAN TO DO THESE PLANNED SECONDARY ANALYSES. AND HONESTLY IT'S ALL THE ANALYSES THAT ARE OF INTEREST TO ME, NOT THE ONE. I DO THAT BECAUSE THAT'S REQUIRED TO GET A SCORE THAT'S FUNDABLE. I WOULD LIKE TO SEE US MOVE TOWARDS SAYING THAT WE HAVE THESE BROAD QUESTIONS THAT ARE CRITICAL FOR US TO ANSWER, AND I'M GOING TO DESIGN A RIGOROUS STUDY THAT WILL GIVE US EXCELLENT ESTIMATES OF THE EFFECT SIZES ASSOCIATED WITH THESE QUESTIONS, MULTIPLE QUESTIONS, AND THAT WE'LL PUBLISH THOSE EFFECT SIZES THAT AS OTHER PEOPLE DO THAT, DO THIS BODY OF RESEARCH, WE WILL GET GOOD ESTIMATES AGAIN AND AGAIN AND WE WILL UNDERSTAND OUR EFFECT SIZES WHETHER THEY ARE CLINICALLY MEANINGFUL AND, OKAY, P VALUE IS FINE BUT THAT BECOME A SECONDARY THINK, HOPING THE FIELD WILL MOVE AWAY AND REVIEWERS, HAVEN'T SEEN THAT BUT THOSE ARE MY THOUGHTS. >> I WANT TO COMMENT AS WELL BECAUSE IN MY EXPERIENCE THERE'S BEEN A MAJOR FOCUS ON THE RIGOR AND WHAT DOES RIGOR MEAN. USUALLY RIGOR MEANS RANDOMIZED CLINICAL TRIALS AND POWERED FOR ONE SINGLE OUTCOME AND THIS GOES BACK TO THE PARADIGM SHIFT THAT I MENTIONED. AND THE IDEA OF FITTING SCIENCE TO THE WORLD VERSUS WORLD TO THE SCIENCE. AND THIS PARADIGM SHIFT I HAVE NOT SEEN AT A LARGE SCALE YET. I THINK THAT NIH HAS MADE GREAT PROGRESS OVER THE PAST TEN YEARS, TWENTY YEARS, IN GETTING TOWARDS THAT BUT THERE'S NOT BEEN A MAINSTREAM PARADIGM SHIFT TO REFLECT THE MESSY REAL WORLD RESEARCH THAT NEEDS TO HAPPEN AND ALSO IN EVALUATING THE QUALITATIVE OR NON-AS RIGOROUS FROM THE PERSPECTIVE OF AN RCT OR POSITIVE APPROACH TO VALUE THESE DIFFERENT METHODS IN WAYS THAT ALLOW THEM TO DO WHAT THEY NEED TO DO. >> THANK YOU, DR. WINDSOR. ALL OF OUR PANELISTS BROUGHT FORWARD SOME REALLY GREAT DISCUSSION POINTS THAT WE WILL CONTINUE IN THE PANEL DISCUSSION IN A FEW MOMENTS. OR LATER IN THE DAY. THANK YOU ALL FOR YOUR WONDERFUL PRESENTATIONS. IT LED US TO A RICH DISCUSSION OF SUCH IMPORTANT CONTENT. WE'RE GOING TO TRANSITION INTO OUR NEXT AREA FOR TODAY. THANK YOU SO MUCH. >> THANK YOU TO THE PARTICIPANTS AND SPEAKERS AND PANELISTS. AT THIS POINT I'M HAPPY TO ANNOUNCE WE'LL TAKE A BREAK. AND THIS IS A SHORT BREAK, WE HAVE 8 MINUTES, WE NEED TO COME BACK AT 2 P.M. TO DIVE INTO SESSION 4 AND KEEP THIS DAY GOING. SO IF YOU WOULD ALL -- IF PANELISTS AND SPEAKERS WOULD MUTE AND STOP YOUR VIDEO WE'LL BRING UP A BRIEF SLIDE TO SHARE DURING THE BREAK AND THEN WE'LL SEE YOU BACK HERE AT2 P.M. PROMPTLY. THANK YOU SO MUCH. WE'RE STARTING SESSION 4, HOW TO EXAMINE MULTI-COMPONENT INTERVENTIONS ON MULTI-SYSTEM OR MULTI-ORGAN OUTCOMES. THIS IS MODERATED BY DR. BRYAN AND DR. KIM. DR. BRYAN WILL INTRODUCE SESSION SPEAKERS. >> THANK YOU. IT'S MY GREAT PLEASURE TO INTRODUCE THE SPEAKERS. THEY ARE BY DESIGN VERY BRIEF IN THE INTEREST OF TIME. PLEASE REMEMBER THAT THE FULL BIOS ARE AVAILABLE IN THE BOOKLET ONLINE, PROGRAM BOOK. FIRST DR. ROB KNIGHT, UNIVERSITY OF CALIFORNIA AT SAN DIEGO. NEXT DR. NICHOLAS SCORK, DR. INBAL SHANI, DR. HAMMOND, AND DR. BUTTE FROM UNIVERSITY OF CALIFORNIA, SAN FRANCISCO. JOIN ME IN A WARM WELCOME TO THESE SPEAKERS. DR. KNIGHT? >> HI. LET ME JUST START SHARING MY SCREEN AND FIRST THANK YOU FOR ORGANIZING THIS WONDERFUL MEETING AND INVITING ME TO SPEAK. I'VE LEARNED A LOT AND AM EXCITED TO SEE THE REST OF THE TALKS. I'M GOING TO TALK ABOUT THE MICROBIOME AND METABOLOME IN GENES THROUGHOUT THE BODY, FOCUSED ON DIET AND OTHER THINGS. IT IS COMPLEX. THINK ABOUT OUR BODIES, WE HAVE 30 TRILLION HUMAN CELLS, 39 TRILLION MICROBIAL CELLS. HUMAN GENE CATALOG OF 20,000 HUMAN GENES, A ASTONISHING THE HUMAN MICROBIOME PROJECT REVEAL ALSO 2 TO 20 MILLION MICROBIAL GENES, WE'RE JUST 1% HUMAN BY THAT MEASURE. WHAT'S AMAZING WHEN WE FOCUS ON THE HOO OF -- ON THE HUMAN SIDE WE'RE IGNORING 99%. THE MICROBIOME IS LINKED TO DISEASES AND TREATMENT RESPONSE TO THE BODY AND BRAIN. IN THE GUT IT'S A READOUT OF EXPOSURES, DELIVERY MODE TO DIET, EXERCISE TO AGING, TO DRUGS, BUT ADDITIONALLY GUT MICROBIOME AND THOSE ELSEWHERE COMMUNICATE. FOR EXAMPLE MICROBES IN THE GUT COMMUNICATE WITH THE BRAIN THROUGH THE IMMUNE SYSTEMT RELEASE OF SMALL MOLECULES AND DIRECT SIGNAL THROUGH THE VAGUS NERVE. SOME DISTAL ORGANS THROUGHOUT THE BODY ARE STARTING TO BECOME MORE WELL CHARACTERIZED. IN ADDITION, THE MICROBIOME AFFECTS WHETHER PARTICULAR DRUGS ARE SAFE OR EFFECTIVE FOR AN INDIVIDUAL TRUE FROM PAIN KILLERS TO HEART MEDICATIONS, EVEN TO THE LATEST ANTI-CANCER AGENTS LIKE CHECKPOINT INHIBITORS AND MAY AFFECT THE IMPACT OF MOST MOLECULES WE INGEST, DRUGS OR FOOD. FOR NUTRITION, THERE ARE GOOD PLEA OF PREDICTIVE MODELS TELLING YOU WHAT TO EAT BASED ON YOUR MICROBIOME AND GLYCEMIC INDEX, EFFECTS ON AGING, CARDIOVASCULAR HEALTH AND THINGS WE CAN'T MEASURE. WE'VE SHOWN PROFOUND TRANSFORMATION UPON FOOD MOLECULES. THIS IS DATA COMBINING MASS SPEC DATA, MOLECULAR NETWORK, THE POINT IS MOLECULE FOUND IN THE AMERICAN GUT PROJECT STOOL OR IN FOOD IN GREEN OR BOTH LOCATIONS IN BLUE. YOU CAN SEE ALMOST ALL THE GREEN STUFF HAS BEEN TURNED TO BROWN STUFF, HARD ANY OVERLAP BETWEEN WHAT COMES IN AND WHAT GOES OUT. WE TOOK THIS INTO 3D IN THE CONTEXT OF GERM FREE VERSUS CONVENTIONALLY RAISED MICE, WE FOUND MOLECULES THROUGHOUT THE BODY THERE WAS COMPLETE TRANSFORMATION WHERE UP TO 50% OF MOLECULES IN DISTAL ORGANS INCLUDING BRAIN CAN BE DIFFERENT DEPENDING WHETHER OR NOT YOU HAVE THE MICROBIOME. WE PUBLISHED IN "NATURE" A COUPLE YEARS AGO, BASIS OF MY PIONEER AWARD FOLLOWING UP THIS KIND OF FATE WITH RESPECT TO INFECTIOUS DISEASE. AN IMPORTANT POINT ABOUT THE MICROBIOME WE'RE LOSING COMPLEXITY THROUGH INDUSTRIALIZATION INCLUDING DIET. WHEN WE LOOK AT TRADITIONAL PEOPLE, HUNTER GATHERERS, WE SEE STRONG SEASONAL CYCLING WHICH IS ATTENUATED IN MODERN POPULATIONS, WHEN WE COMPARE POPULATIONS TO OTHER POPULATIONS AROUND THE WORLD WHAT WE SEE IS AUTOMATICALLY POPULATIONS SORT OUT IN TERMS OF MICROBIOME ALONG PRINCIPAL AXIS FROM LEAST TO MOST INDUSTRIALIZED, LOSS OF ENTIRE MAJOR GROUPS OF TAXA INCLUDING PHYLA AND REPLACEMENT WITH MUCH SIMPLER COMMUNITY, MORE INDUSTRIALIZED SOCIETIES LIKE THE U.S., EUROPE, AUSTRALIA. SO IT'S ALMOST LIKE WE'RE TAKING OUT ECOSYSTEMS AND REPLACING. MY COLLEAGUE AT RUTGERS HAS WRITTEN A WONDERFUL BOOK, MISSING MICROBES, NOT ONLY ANTIBIOTICS BUT ALSO ALL KINDS OF THINGS LIKE LOW FIBER DIET LEADING TO ATTENUATION OF OUR MICROBIOME DIVERSITY. OVER THE 20th CENTURY AS ONE INFECTIOUS DISEASE AFTER ANOTHER CAUSED BY SINGLE MICROBES IS BROUGHT UNDER CONTROL, WE SEE EXPLOSION OF CHRONIC DISEASES WHERE PUBLISHED IN THE NEW ENGLAND JOURNAL IN 2002 NONE OF THOSE CHRONIC DISEASES ARE LINKED TO MICROBIOME BUT TODAY WE KNOW FROM EXPERIMENTS LIKE THE ONES SHOWN AND POPULATION STUDIES IN HUMANS THAT ALL OF THESE DISEASES ARE LINKED TO CHANGES IN THE MICROBIOME IN HUMANS AND ANALOGS CURED BY MANIPULATION OF THE MICROBIOME IN MICE. IMPORTANT INTERVENTION IS DIET, OVER THE PAST DECADE LARGELY THROUGH NIH SUPPORTED RESEARCH WE FOUND LONG-TERM BUT NOT SHORT-TERM DIET HAS A KEY ROLE. WE PUT THEM ON PHYLO GENETIC TREE AND COMPARED COMMUNITIES IN TERMS OF DIVERSITY SO IDENTICAL COMMUNITIES, ON THE TREE ON THE LEFT, UNRELATED TREES HAVE A SEPARATION ON THE TREE LIKE ON THE RIGHT. WE TAKE THESE DISTANCES, SUMMARIZE IN A DISTANCE MATRIX, CLOSE TO PCA, HIERARCHICAL CLUSTERING AND LOOKING AT GENOTYPES, HIGH FAT VERSUS LOW FAT DIET IN DIFFERENT COLORS HERE, ALL POINTS SEPARATE BY COLOR, ALL THE DIET, GENOTYPING UNIMPORTANT. THIS IS THE RESULT OF PKA ON DISTANCES. WHEN WE GET TO HUMANS, LOOK AT LONG-TERM DIET, WE SEE OVER THE LONG TERM PROFOUND EFFECT OF DIET IN TERMS OF MICROBIOME IN HUMANS SPECIFIC MICROBES CORRELATE WITH HIGH PROTEIN OR HIGH CARBOHYDRATE DIETS, HOWEVER WHEN WE TRY TO DO A DIET INTERVENTION SAME PAPER WHAT YOU CAN SEE IS MULTIPLE TIME POINTS FROM EACH PERSON ON THE LEFT, ALL TIME POINTS CLUSTER SO THERE'S NOT MUCH CHANGE IN AN INDIVIDUAL BUT TAKE STATISTICALLY SIGNIFICANT BUT SMALL EFFECT SIZE DIFFERENT BETWEEN FIRST TIME POINT AND SUBSEQUENT TIME POINTS. YOU MIGHT BE FAMILIAR WITH PAPERS LIKE THIS ONE THAT SUGGESTS THAT DIET CAN RAPIDLY REPRODUCE -- REPRODUCE BELLY ALTER THE GUT MICROBIOME, BUT THIS IS THE MAGNITUDE OF EFFECT, VEGAN OR MEAT/CHEESE/EGGS. WHEN WE PROCESS THE DATA WITH OUR TECHNIQUES WE CAN SEE ON THE LEFT SUBJECT LOOKING DIFFERENT ON THE MEAT DIET IN RED. HERE IS THE SECOND SUBJECT WHERE WE SEE THE SAME THING. WHEN WE POP EVERYONE TOGETHER YOU SEE NO CONSISTENCY IN THE MEAT PART VERSUS PLANT DIET, SO BASICALLY ON THE SAME DIET INTERVENTION DIFFERENT PEOPLE CHANGE IN COMPLETELY DIFFERENT DIRECTIONS, EFFECT SIZE IS SMALL COMPARED TO STANDING VARIATION BETWEEN PEOPLE UNLESS YOU GET THEM TO STICK TO THE DIET FOR MONTHS. IN CASE YOU'RE WONDERING IF YOU CAN GET EFFECT SIZE WITH FECAL TRANSPLANTATION I'M SHOWING A MAP OF THE WHOLE BODY FROM HUMAN MICROBIOME PROJECT, STOOL SAMPLES FROM C. DIFF PATIENTS, ORANGE SYMBOLS, YOU CAN SEE THEIR STOOL IS DIFFERENT FROM HEALTHY STOOL IN TERMS OF MICROBIOME. THEY WILL GET STOOL TRANSPLANT FROM ONE DONOR, ALL FOUR, ANIMATED ONE DAY AT A TIME, ALL FOUR IN A COUPLE DAYS MOVE INTO THE HEALTHY STOOL STATE, WHERE SYMPTOMS VANISH. THIS IS AN ININTERVENTION CHANGING THE MICROBIOME, CHANGING FROM THE COMMUNITY FROM ONE PERSON TO ANOTHER. WE STILL HAVE TO ACHIEVE THROUGH DRUGS OR DIET. IT DOESN'T HANDLE COMPOSITIONAL DATA. KEVIN, IN MY LAB, EXTENDED A COMPOSITIONAL TENSOR FACTORIZATION TEASING OUT SUBTLE SIGNALS THAT I DON'T HAVE TIME TO GO INTO BUT WE CAN SEE IMPACTS OF FACTORS LIKE DELIVERY MODE, EVEN IN ADULTS AND RELATE THEM TO DISEASES LIKE INFLAMMATORY BOWEL DISEASE. WE ALSO USE CLASSIFIERS AND REGRESSION MODELS, AND LIKE OTHER PROJECTS WE'VE SEEN WE CAN ALSO TELL YOUR AGE FROM YOUR MICROBIOME TO WITHIN ABOUT THREE YEARS USING SKIN MICROBIOME TO WITHIN ABOUT TEN YEARS USING THE GUT. LIKE OTHER PROJECTS, INTERESTED IN WHAT LEADS TO RESIDUALS AND IF DIET BASED OR OTHER INTERVENTIONS CAN ADDRESS DIFFERENCES. I'LL TALK ABOUT THE PROJECT, ALZHEIMER'S GUT MICROBIOME PROJECT STUDYING COMPLEXITY LINKING MICROBIOME DIET AND ALZHEIMER'S DISEASE. IT LEVERAGES DEVELOPMENT OF AMERICAN GUT PROJECT, KITS FOR METABOLOME MICROBIOME AND BEING ABLE TO DO BLOOD COLLECTION WITH FINGER PRICK FOR METABOLOMICS AND SEROLOGY. WE'RE LEVERAGING CLINICAL TRIALS ADDING MULTIPLE KINDS OF METABOLOMICS, METAGENOMIC READOUT TO INTEGRATE THIS INFORMATION LONGITUDINALLY WHICH AS YOU'VE SEEN IN THIS TALK AND IN OTHERS, LONGITUDINAL STUDIES CRITICAL TO GETTING TOWARDS CAUSALITY ESPECIALLY WHEN IT'S DEFINED INTERVENTION THAT'S INVOLVED. SO LED BY MYSELF AND OTHERS. WE HAVE A LONG LIST OF COLLABORATORS WHO I WON'T READ OUT INDIVIDUALLY. IT'S BEEN A PRIVILEGE TO WORK WITH THEM AS WELL AS MANY PEOPLE IN MY LAB WORKING ON THIS AND OTHER PROJECTS WHICH INVOLVES OVER A THOUSAND COLLABORATORS. AND THANKS FOR THE OPPORTUNITY TO PARTICIPATE. REALLY LOOKING FORWARD TO OTHER TALKS AND DISCUSSION. THANK YOU. >> THANK YOU, DR. KNIGHT. NOW WE'RE READY FOR DR. SCHORK. PLEASE SHARE YOUR SLIDES. >> I'M GOING TO QUOTE FROM JOHN CLEESE, NOW FOR SOMETHING COMPLETELY DIFFERENT. I'M GOING TO FOCUS ON N-of-1, AGGREGATED N-of-1 STUDIES FOR EXPLORING MULTI-COMPONENT INTERVENTION EFFECTS ON MULTIPLE OUTCOMES. I DO HAVE ONE DISCLOSURE, I'M A FOUNDER OF A COMPANY TRYING TO EXPLOIT TECHNIQUES I'LL BE DESCRIBING IN COMMERCIAL SETTINGS. SO WHAT IS AN N-of-1 TRIAL? IN THEORY BASIC AND SIMPLE. LET'S SAY YOU ARE DON'T KNOW HOW TO TREAT SOMEONE'S HYPERTENSION, MEASURE BLOOD PRESSURE AT BASELINE, GIVE THEM, SAY, A DRUG, BETA BLOCKER, WASH THEM OFF THE DRUG, GIVE THEM A DIFFERENT DRUG, DIURETIC, SOME NUMBER OF TIMES, COLLECTING BLOOD PRESSURES THROUGHOUT THE ENTIRE PROCESS. AND THEN ONE MIGHT BE ABLE TO READ OFF WHICH DRUG APPEARED TO HAVE AN EFFECT, MORE PRONOUNCED EFFECT ON THE INDIVIDUAL'S BLOOD PRESSURE. YOU COULD IMAGINE COLLECTING OTHER VARIABLES, SAY SLEEP QUALITY ON THE INDIVIDUAL, OR THEIR WEIGHT OR SOME OTHER VARIABLES THAT MIGHT BE NEGATIVELY OR POSITIVELY COORELATED AND OTHERS THAT AREN'T CORRELATED AT ALL. THESE ARE ENABLED BY EMERGING DEVICES AND TECHNIQUES FOR OBTAINING MICROSAMPLED LIKE DRIED BLOOD SPOTS BUT HERE THE IMPORTANT THING IS THAT THE PATIENTS ACT AS THEIR OWN CONTROLS. THESE STAND IN DISTINCTION TO LARGE POPULATION RCTs IN THE OTHER TALKS. THEY CAN TAKE ADVANTAGE OF PREVAILING STATISTICAL TECHNOLOGIES USED, YOU CAN RANDOMIZE ORDER IN WHICH INTERVENTIONS ARE PROVIDED, USING BLINDING. IF YOU DID A HUNDRED N-of-1 TRIALS AND IDENTIFIED UNEQUIVOCAL RESPONDERS AND NON-RESPONDERS TO PARTICULAR TREATMENTS YOU COULD ASK QUESTIONS ABOUT LET'S SAY RESPONDERS HAVE IN COMMON THAT THE NON-RESPONDERS DID NOT. I'VE DISCUSSED THEM WITH THE FDA OVER THE YEARS FOR EXAMPLE IN THIS CONFERENCE REFLECTED IN THE BOTTOM RIGHT HERE. LET'S SAY WE HAVE RESPONSE, DOESN'T NECESSARILY HAVE TO BE IN CONTINUOUS TIME BUT I'M GOING TO MAKE THE ASSUMPTION WE HAVE CONTINUOUS TIME, BEAR WITH ME. TREATMENT RESPONSE VARIABLE, IF WE CAN INDICATE WHICH INTERVENTION, MAYBE PLACEBO, ACTIVE COMPOUND, THERE'S OBVIOUSLY BIG DIFFERENCES IN RESPONSES DURING INTERVENTION TIME. NORMALLY WHAT ONE WOULD DO IS DO A SIMPLE T TEST BETWEEN THE MEASURES COLLECTED WHILE ON PLACEBO AND SAY MEASURES COLLECTED WHILE ON THE ACTIVE COMPOUND. WHERE THINGS GET INTERESTING, IF IN FACT THERE ARE SOME NUMBER OF UNDERLYING FACTORS ASSOCIATED WITH INTERVENTION, MAYBE THERE ARE FOUR NUTRIENTS INCREASED IN ABUNDANCE WHEN THE INTERVENTION IS PROVIDED, THE PERSON MAY HAVE NATURAL EXPOSURE WHEN THEY ARE NOT ON, SAY, INTERVENTION WHICH JUST ADDS MORE AMOUNTS. THERE MAY BE MULTIPLE COMPONENTS WHOSE EFFECTS ON RESPONSE WE WANT TO WORK OUT. IT'S IMPORTANT TO REALLY UNDERSTAND WHAT YOU HAVE IS A BIG DOSE EFFECT WHEN THE TREATMENT IS PROVIDED VERSUS, SAY, PLACEBO. SO IF THERE WERE FOUR UNDERLYING FACTORS HERE IN RED, WHOSE VARIATIONS I'VE SHOWN HERE OVER THE COURSE OF THE TRIAL WE COULD ASK CERTAIN QUESTIONS ABOUT THEM. IMPORTANTLY WE KNOW THERE'S VARIATION WITHIN THE TREATMENT PERIODS WHILE THEY ARE ON PLACEBO OR ACTIVE COMPOUND. THIS IS CRUCIAL TO ANALYZING THE DATA IN APPROPRIATE WAYS. WE MAY HAVE THE NUTRIENTS MEASURED IN THEIR BASAL STATES, ON PLACEBO, AND THEN WHAT THE RESPONSE MIGHT BE WHEN THEY ARE ACTUALLY GIVEN AS PART OF THE TREATMENT. IT'S IMPORTANT TO UNDERSTAND THAT THERE MAY BE RELATIONSHIPS BETWEEN THE LEVELS OF THOSE INGREDIENTS OR FACTORS WITHIN THE TREATMENT PERIODS. SO HERE THERE'S A CORRELATION BETWEEN THE FACTOR LEVELS AND TREATMENT RESPONSE DURING THE TREATMENT PERIODS. THAT COULD BE IN DISTINCTION TO, SAY, ONE OF THOSE NUTRIENTS NOT HAVING AN EFFECT SO YOU WOULD NOT SEE A RELATIONSHIP BETWEEN THE LEVELS, LET'S SAY, OF THE NUTRIENT THAT OCCUR NATURALLY DURING THE TIMES WHEN THEY ARE ON AND OFF THE INTERVENTION. AND ONE COULD THEN EVALUATE HOW ONE COULD ANALYZE DATA, TAKING INTO ACCOUNT THERE MAY BE NATURAL VARIATION IN THE COMPONENTS OF THE INTERVENTION WHILE ON PLACEBO AND SAY ACTIVE. IF ONE ANALYZES DATA IN NAIVE WAY ALL THE NUTRIENTS SAY IN THAT EXAMPLE I GAVE WOULD BE SURROGATES FOR WHEN THEY ARE ON OR OFF THE ACTIVE TREATMENT. IF YOU ANALYZE DATA THAT WAY, YOU WOULD HAVE REALLY GOOD POWER TO EFFECT EFFECTS, A COUPLE WOULD BE BOCK US IF THEY HAD NO -- BOGUS IF HAD NO EFFECT ON OUTCOME. SAY USE A DUMMY VARIABLE FOR PLACEBO OR THE ACTIVE INGREDIENTS, DO THAT, LOOK AT RELATIONSHIPS BETWEEN THE LEVELS OF FACTORS AND OUTCOME AND HERE YOU'D SEE YOU WOULD HAVE NO POWER TO DEFECT TWO FACTORS THAT AREN'T CAUSALLY RELATED. YOU CAN DO THIS SEQUENTIALLY. MAKE DECISIONS IN REAL TIME ABOUT WHEN YOU HAVE ENOUGH DATA TO IMPLICATE A FACTOR AS ASSOCIATED WITH A RESPONSE. YOU CAN USE SEQUENTIAL PROBABILITY RATIO TESTS FOR THIS, NOT GOING INTO TECHNICAL DETAILS BUT THERE ARE ISSUES, STATISTICAL ISSUES IN USING THIS IN, SAY, MULTIPLE REGRESSION CONTEXTS WHERE YOU MAY HAVE MULTIPLE COMPONENTS THAT YOU'VE MEASURED OVER TIME AND WANT TO KNOW WHICH ONES ARE ACTUALLY THE ACTIVE INGREDIENTS INFLUENCING RESPONSE. THERE ARE WAYS OF DEALING WITH THIS. USING BOOTSTRAP TECHNIQUE, I'LL TELL YOU THAT IF YOU DO THAT THE BOOTSTRAP APPROACH WORKS. I HAD A PAPER COMING OUT. IT'S POSSIBLE TO IMPLEMENT THE SORT OF STUDY DESIGN THAT I'VE DESCRIBED IN REAL TIME SO YOU'RE NOT WASTING MONEY, SAY, IF YOUR DRUG IS FUTILE, IT HAS NO RESPONSE OR HAVE YOU ENOUGH DATA AT ANY ONE POINT IN TIME TO MAKE A DECISION THAT THINGS WORK. AN IMPORTANT THING THAT CAME UP IN DR. JENSEN'S TALK EARLIER WAS THAT WE'RE TRYING TO IDENTIFY CAUSAL RELATIONSHIPS BETWEEN VARIABLES, NOT JUST ASSOCIATIONS. HENCE THE NEED FOR CLINICAL TRIALS SUPPOSED TO GIVE CAUSALITY. THERE'S LIKELY A TEMPORAL COMPONENT IN THE RELATIONSHIPS BETWEEN, SAY, COMPONENTS OR FACTORS IN A MULTI-COMPONENT TRIAL AND A RESPONSE. SUCH THAT FACTORS COLLECTED AT TIME POINT 1 MIGHT INFLUENCE OUTCOMES AT TIME POINT 2 GIVEN THERE MIGHT BE A TIME LAG WITH RESPECT TO THEIR PHYSIOLOGIC EFFECTS. WE'VE LOOKED AT DIFFERENT TECHNIQUES FOR DOING THIS USING COHORTS LIKE THIS ONE WHERE WE IDENTIFIED CAUSAL RELATIONSHIPS USING THE TIME LAG APPROACH. WHAT WE DID IS SIMULATE ENVIRONMENTS, SYSTEMS OF VARIABLES, FACTORS ASSOCIATED WITH MULTI-COMPONENT INTERVENTION, AND MULTIPLE OUTCOMES, AND YOU DON'T HAVE TO GO THROUGH DETAILS, JUST KNOW SOME HAD A TEMPORAL RELATIONSHIP, OTHERS DIDN'T. COMPLICATED SYSTEM HERE. WE JUST LOOKED AT THE POWER OF DETECTING THE INFLUENCES OF THESE DIFFERENT FACTORS, ON OUTCOMES, BY, A, DOING SIMPLE NAIVE TESTS, JUST ASSOCIATING THE FACTORS WITH THE RESPONSES AND THEN ACCOUNTING FOR THE FACT MANY OF THOSE FACTORS AND OUTCOMES MIGHT BE RELATED IN TEMPORAL WAYS, FOR EXAMPLE. HERE IS JUST A PICTURE OF WHAT THAT MIGHT LOOK LIKE, NOISY DATA, LESS NOISY DATA, ET CETERA, ET CETERA. AND THEN WE EVALUATED THE POWER OF THE DETECTING OF THESE EFFECTS. WHAT THESE PLOTS SHOW WHEN YOU USE NAIVE APPROACHES, SAY NO CONDITIONING, NO ACCOUNTING FOR THE FACT THAT THE INDEPENDENT VARIABLES, FACTORS IN THE MULTI-COMPONENT FACTOR ARE THEMSELVES RELATED TO THE OUTCOMES. YOU HAVE CERTAIN POWER. WHEN YOU CONDITION OR ACCOMMODATE THE FACT THAT YOU'RE STUDYING A SYSTEM OF VARIABLES YOU HAVE. MUCH GREATER POWER TO DEFECT VARIABLE ALSO. I WANT TO DIVE IN. WE HAD A FEMALE WITH SERIOUS SLEEP PHASE DISORDER BEING TREATED WITH MULTIPLE DRUGS, ONE SLEEP AID, ONE ANTI-DEPRESSANT, ET CETERA, ET CETERA. POLYPHARMACY. SHE WAS VERY UNHAPPY. WE WEANED HER OFF ALL DRUGS, PUT HER BACK ON THE THREE DRUGS IN DIFFERENT DOSAGES, DIFFERENT COMBINATION, MEASURED SLEEP QUALITY, ANXIETY LEVELS, DEPRESSION. PUNCH LINE, WHEN ON THE INTERVENTIONS VERSUS OFF INTERVENTIONS, SHE HAD POOR SLEEP. WHEN SHE WAS OFF ALL THE INTERVENTIONS SHE HAD BETTER QUALITY SLEEP, IN FACT BORN OUT WITH OTHER ANALYSES LOOKING AT OTHER SLEEP PHENOTYPES. THIS WASN'T GOING TO MAKE THE DRUG COMPANIES HAPPY BUT THIS IS IN FACT WHAT WE FOUND WITH HER. NOW I WILL SAY THERE'S A CAVEAT HERE. EVEN THOUGH HER SLEEP WAS IMPROVED WHEN OFF ALL MEDICATIONS, SHE STILL HAD DEPRESSIVE EPISODES AND OTHER THINGS THAT DESERVED ATTENTION. OKAY. THE SECOND EXAMPLE I'M GOING TO GIVE IS A SERIES OF N-of-1 STUDIES USING DIGITAL THERAPEUTIC. MANY OF YOU MAY NOT KNOW IT'S POSSIBLE TO BE PRESCRIBED AN APP FROM YOUR PETITION RATHER T HAN -- PHYSICIAN RATHER THAN A PILL TO TREAT YOUR DISORDER, THE FDA MAKE THAT POSSIBLE, THAT YOUR INSURANCE CAN REIMBURSE, ET CETERA. I'M GOING TO TALK ABOUT ONE, STRESS RELIEF APP THROUGH A COMPANY CALLED STOP BREATHE AND THINK, WE'RE GIVEN DATA ON MILLIONS OF USERS OF THIS APP AND BASICALLY WHAT YOU DO IS YOU GET THE APP, STATE SPECIFICALLY WHAT YOUR MOOD IS, CHOOSE ABOUT 80 MOODS, THEN YOU ARE GIVEN A MENU OF MEDITATIONS THAT COME WITH CONTENT AND WHAT NOT TO CHOOSE FROM. THEN AFTER YOU'RE DONE WITH THE MEDITATION, YOU ENDORSE YOUR MOOD AFTERWARDS. THERE'S MANY MEDITATIONS AND MOODS IN THIS. HERE IS ONE EXAMPLE, A PERSON'S PROFILE IN USING THIS APP. AND ANYTHING ABOVE ZERO MEANS THEY ARE IN POSITIVE EMOTIONS, BELOW ZERO IS NEGATIVE. THIS IS PRE-EMOTION EMOTION, AND POST-MEDITATION EMOTION. YOU CAN SEE OVERALL THIS PERSON HAD A POSE RESPONSE. SO THEY STARTED OUT LOW, LOW MOOD, AND IMPROVED MOOD AFTER MEDITATIONS SUGGESTING THE APP WAS HAVING A POSITIVE EFFECT. THIS PERSON WAS IN TROUGH HERE, ACTUALLY ELECTION, TRUMP WON, FIVE OR SIX YEARS AGO, THIS PERSON WAS NOT A REPUBLICAN. HERE IS ANOTHER PROFILE OF A PERSON WHERE IT APPEARS THE APP WORKED. WE LOOKED AT THE LONG-TERM EFFECTS OF APP, IMPROVED BASAL MOOD OF SOMEONE. AND FOUND EVIDENCE THAT WAS INDEED THE CASE. MORE INTERESTING IN CONTEXT OF MULTI-COMPONENT INTERVENTIONS WE TRIED TO IDENTIFY FROM THE MILLIONS OF DATA POINTS THAT WE HAVE THOSE MEDITATIONS THAT TENDED TO MOVE PEOPLE FROM ONE MOOD STATE TO ANOTHER AND FOUND STRONG STATISTICAL EVIDENCE, FOR EXAMPLE, THAT IF SOMEONE CAME IN IN A DEPRESSED AND TIRED STATE TO GET OUT OF THAT PARTICULAR STATE YOU MIGHT WANT TO USE CERTAIN EMOTIONS, MEDITATIONS, HERE IN THE BLUE. WHAT YOU WOULDN'T WANT TO USE IS, SAY, THESE MEDITATIONS. NOW-- >> HAVE YOU TWO MINUTES LEFT. >> TWO MORE SLIDES. TURNS OUT THAT THE MEDITATIONS THAT PEOPLE CHOSE FROM ALSO HAD INDIVIDUAL EFFECTS ON EACH INDIVIDUAL HERE, 16 INDIVIDUALS, THESE COLOR CODINGS JUST CORRESPOND TO MEDITATIONS THAT HAD POSITIVE OR NEGATIVE EFFECTS ON SOMEONE'S MOOD. AND HERE ARE THE DIFFERENT MEDITATIONS. YOU CAN SEE SOME MEDITATIONS HAD A MARGINALLY OR SIGNIFICANT EFFECT ON SOMEBODY'S MOOD BUT THERE WERE DIFFERENT COLLECTIONS OF MEDITATIONS THAT APPEARED TO WORK OR NOT WORK FOR INDIVIDUALS. THOUGH THERE WERE SOME MEDITATIONS WITH COMMON EFFECTS. WE'RE ABLE TO TEASE OUT WHICH COMPONENTS OF THIS INTERVENTION APPEARED TO WORK FROM MOVING PEOPLE FROM ONE MOOD STATE TO ANOTHER AND WHICH ONES APPEARED TO WORK BEST FOR PARTICULAR INDIVIDUALS. SO TO WRAP UP, PERSONALIZED OR INDIVIDUALIZED MEDICINE FROM MY ANGLE WILL REQUIRE NOVEL AND MORE APPROPRIATE WAYS OF TESTING INTERVENTIONS ESPECIALLY THOSE WITH MULTI-COMPONENTS, N-of-1 AND AGGREGATED N-of-1 CLINICAL TRIALS HAD POTENTIAL TO ADDRESS NEEDS AND I TALKED ABOUT THIS KIND OF THERAPEUTIC ENTER VEHICLES MONITORING WHERE YOU ACTUALLY HAVE THE OPERATIVE LEVELS OF THE DIFFERENT FACTORS MEASURED OVER TIME. I KNOW THAT MIGHT BE A CHALLENGE BUT DO BELIEVE IT'S POSSIBLE WITH ALL SORTS OF EMERGING TECHNOLOGIES. RELEVANT STUDIES HAVE POTENTIAL TO SHED LIGHT A HUMAN ORGANISMAL TRIALS AND WILL HELP INDIVIDUAL PATIENTS. IF YOU KNOW WHAT WORKED FOR THAT PATIENT IN N-of-1 TRIALS YOU HAVE SOMETHING TO BENEFIT THEM GOING FORWARD. BETTER MONITORING DEVICES AND METHODS ARE ON THE HORIZON. I WILL STOP THERE AND THANKS FOR LISTENING. >> THANK YOU SO MUCH. WE'RE READY NOW FOR DR. NAHUN-SHANI TO SHARE HER SCREEN. THANK YOU. >> CAN YOU SEE MY SCREEN? >> YES. >> IT'S GOOD TO BE HERE. IN THE NEXT 15 MINUTES I'LL SHOW YOU A FRAMEWORK I'M WORKING ON TO HELP INVESTIGATORS DECIDE WHICH DESIGN TO USE WHICH EXPERIMENTAL DESIGN TO USE. THE PROCESS OF DEVELOPING A MULTI-COMPONENT INTERVENTION. NOW, THE DESIGN I'M GOING TO FOCUS ON, TYPICALLY USED WITH SINGLE OUTCOME IN MIND BUT ALL OF THEM CAN BE AND SHOULD BE EXTENDED TO ACCOMMODATE MULTIPLE OUTCOMES. THIS IS THE FRAMEWORK. BASICALLY INCLUDES FIVE KEY QUESTIONS THAT AS INVESTIGATORS WE CAN ASK OURSELVES IN THE PROCESS OF DEVELOPING AN INTERVENTION, AND THIS FRAMEWORK FOCUSES ON FOUR TYPES OF APPROACHES. THE FIRST ONE IS OMRT. SECOND IS SEQUENTIAL RANDOMIZED TRIAL, SMART. AND OTHER DESIGNS PRETTY MUCH MEANS STANDARD RANDOMIZED TRIALS SUCH AS A RANDOMIZED CONTROLLED TRIAL. I'LL TALK ABOUT EACH OF THE QUESTIONS AND SHOW WHERE THE ANSWER TO EACH QUESTION CAN BE NO AND YES SO YOU CAN SEE DIFFERENT ANSWERS CAN LEAD US TO CHOOSE A DIFFERENT DESIGN. LINDA MENTIONED THIS. A COMPONENT IS ANY ASPECT OF INTERVENTION THAT WE CAN REASONABLY SEPARATE OUT FOR INVESTIGATION. SOMETIMES WE'RE NOT INTERESTED IN DEVELOPS MULTI-COMPONENT INTERVENTION. WHEN WE WANT TO DEVELOP INTERVENTION WITH SINGLE COMPONENT OR WHEN WE WANT TO DEVELOP INTERVENTION THAT INCLUDES MULTIPLE ELEMENTS THAT CANNOT BE DISENTANGLED FOR INVESTIGATION. WE ALSO WANT TO DO IS COMPARE THE EFFECTIVENESS OF INTERVENTION TO SUITABLE ALTERNATIVE WHERE WE WILL CONSIDER A RANDOMIZED CONTROLLED TRIAL. MOST OFTEN WE'RE INTERESTED IN DEVELOPING INTERVENTION THAT INCLUDES MULTIPLE COMPONENTS FOR EXAMPLE THIS IS BASED ON WORK EVER BONNIE SPRING, SHE WANTED TO DEVELOP INTERVENTION THAT USES TECHNOLOGY TO SUPPORT WEIGHT LOSS AND IN THE PROCESS OF DEVELOPING INTERVENTION SHE REALIZED SHE HAS MULTIPLE COMPONENTS, FIVE COMPONENTS IN MIND, THAT CAN BE DISENTANGLED FOR INVESTIGATION. IN THIS CASE WE WOULD ANSWER YES TO QUESTION NUMBER ONE, TRANSITION TO QUESTION NUMBER TWO. WE NEED TO ASK DO WE HAVE SCIENTIFIC QUESTIONS ABOUT THE SELECTION OF COMPONENTS? SOMETIMES WE DON'T. SOMETIMES WE HAVE EMPIRICAL EVIDENCE TO PUT TOGETHER HIGH QUALITY INTERVENTION PACKAGE TO PUT TOGETHER COMPONENTS IN EFFECTIVE WAY BUT WHAT WE WANT TO DO IS CONFINE EFFECTIVENESS, COMPARED TO A SUITABLE CONTROL. CONSIDER RANDOMIZED CONTROLLED TRIAL. WE DO HAVE QUESTIONS ABOUT SELECTION OF COMPONENTS. SHOULD WE INCLUDE TEXT MESSAGES? I WOULD SAY YES TO TWO AND TRANSITION TO NUMBER TREE. WE CAN ASK DO WE HAVE SCIENTIFIC QUESTIONS ABOUT MORE THAN A SINGLE COMPONENT? SOMETIMES WE DON'T. WHETHER OR NOT TEXT MESSAGES SHOULD BE INCLUDED IN INTERVENTION PACKAGE, IN THIS CASE WHAT I SHOULD DO IS CONSIDER A TWO ARM TRIAL, COMPARING PACKAGED, INTERVENTION PACKAGE WITH TEXT MESSAGES TO THE PACKAGE WITHOUT TEXT MESSAGES. BUT SOMETIMES WE HAVE SCIENTIFIC QUESTIONS ABOUT MORE THAN A SINGLE COMPONENT, FOR EXAMPLE WE HAVE QUESTIONS ABOUT TWO COMPONENTS. AND IN THIS CASE WE WOULD ANSWER YES TO NUMBER THREE AND TRANSITION TO QUESTION NUMBER FOUR. HERE WE NEED TO ASK OURSELVES A VERY INTERESTING QUESTION. DO WE HAVE SCIENTIFIC QUESTIONS ABOUT INTERVENTION TIMING? WHAT DOES IT MEAN? QUESTIONS ABOUT WHEN TO INTERVENE, WHEN TO DELIVER A COMPONENT AND WHICH SHOULD BE OFFERED AT DIFFERENT POINTS IN TIME. SOMETIMES WE DON'T HAVE QUESTIONS ABOUT TIMING. FOR EXAMPLE HERE WE DON'T HAVE QUESTIONS ABOUT WHEN BUT HOW TO INTERVENE AT A SPECIFIC POINT IN TIME. IN THIS CASE THIS FRAMEWORK RECOMMENDS WE SHOULD CONSIDER WHEN WE WANT TO CONSIDER FACTORIAL DESIGN. LEVELS CROSS WITH LEVELS OF OTHER FACTORS TO FORM A DESIGN WITH MULTIPLE EXPERIMENTAL CONDITIONS. THIS IS A VERY SIMPLE EXAMPLE, SO THAT WE HAVE TWO QUESTIONS ABOUT TWO COMPONENTS. TEXT MESSAGES AND MEAL REPLACEMENT. IN THIS CASE WE CAN CONSIDER TWO FACTORS, ONE FOR EACH COMPONENT, EACH CAN HAVE FOR SIMPLICITY AND CROSS, GIVING US A DESIGN WITH FOUR EXPERIMENTAL CONDITIONS. NOW AGAIN THIS IS A VERY SIMPLE EXAMPLE OF A FACTORIAL DESIGN, A LOT OF WORK BY LINDA COLLINS AND OTHERS SHOWING HOW EFFICIENT DESIGNS ARE ANSWERING QUESTIONS ABOUT SELECTION OF COMPONENTS INTRODUCED AT A SINGLE POINT IN TIME. LET'S ASSUME THAT I WANT TO DEVELOP INTERVENTION THAT USES TECHNOLOGY TO SUPPORT WEIGHT LOSS, THIS TIME WHAT I WANT TO DO IS START WITH MINIMAL TECHNOLOGY-BASED SUPPORT. WEEK TWO IDENTIFY PARTICIPANTS NOT LOSING SUFFICIENT AMOUNT OF WEIGHT, CLASSIFY THEM AS NON-RESPONDERS AND PROVIDE MORE SUPPORT TO NON-RESPONDERS. RESPONDERS CAN CONTINUE WITH THE SAME INITIAL INTERVENTION. SUPPOSE THAT'S THE INTERVENTION I WANT TO DEVELOP. LET'S ASSUME I HAVE TWO SCIENTIFIC QUESTIONS. FIRST ONE IS WHICH COMPONENT TO INCLUDE AT WEEK ZERO, AT THE BEGINNING, TEXT MESSAGES OR PHONE COACHING? SECOND QUESTION IS WHAT TO DO AT WEEK TWO FOR NON-RESPONDERS? SHOULD I PROVIDE BODY TRAINING OR PHONE COACHING AT WEEK TWO FOR NON-RESPONDERS? THESE ARE QUESTIONS WHEN TO INTERVENE, WHICH TO DELIVER AT DIFFERENT POINTS IN TIME. IN THIS CASE I WOULD ANSWER YES TO QUESTION FOUR AND TRANSITION TO NUMBER FIVE. WE NEED TO ASK OURSELVES ARE THE COMPONENTS AND INTERVENTION WE'RE TRYING TO DEVELOP DESIGNED TO ADDRESS CONDITIONS THAT CHANGE SLOWLY OR RAPIDLY? BY SLOWLY I MEAN OVER A FEW WEEKS OR A FEW MONTHS. THIS EXAMPLE YOU CAN SEE WE'RE TRYING TO ADDRESS CONDITIONS THAT UNFOLD OVER TWO WEEKS. WE'RE TRYING TO ADDRESS EARLY SIGNS OF NON-RESPONSE, INEFFICIENT -- INSUFFICIENT WEIGHT LOSS, CONDITIONS THAT CHANGE SLOWLY. NOTICE THAT WE'RE TRYING TO ADDRESS THESE CONDITIONS USING ADAPTATION. SO WHAT IS ADAPTATION? IT IS THE USE OF ONGOING INFORMATION ABOUT THE PARTICIPANT TO DECIDE WHETHER AND HOW TO INTERVENE. HERE WE'RE USING INFORMATION ABOUT EARLY RESPONSE TO DECIDE WHETHER WE SHOULD DELIVER MORE SUPPORT OR CONTINUE. INTERVENTIONS THAT USE ADAPTATION CHANGE SLOWLY, WE CALL THEM ADAPTIVE INTERVENTIONS. SOME INTERESTING TO KEEP IN MIND, THIS RELATES TO MULTIPLE OUTCOMES, IS THAT THESE INTERVENTIONS ARE DESIGNED TO ADDRESS OR ACHIEVE TWO TYPES OF OUTCOMES. A DISTAL AND PROXIMAL OUTCOME. DISTAL IS LONG TERM, END GAME. HERE IT'S 5% WEIGHT LOSS. TO ACHIEVE THIS DISTAL OUTCOME, WITHOUT INTERVENTIONS DESIGNED TO ADDRESS PROXIMAL OUTCOMES THESE ARE SHORT-TERM GOALS, ADAPTATION IS INTENDED TO ACHIEVE. ONE OF THE PROXIMAL OUTCOMES IS ONGOING WEIGHT LOSS. IF THE PARTICIPANT IS SHOWING EARLY RESPONSE OF RESPONSE, NOT LOSING SUFFICIENT WEIGHT, WE WANT TO DELIVER MORE SUPPORT. WE WANT THE PERSON TO LOSE WEIGHT IN THE INTERVENTION, ACHIEVING THE DISTAL OUTCOME, 5%, WE CAN THINK ABOUT PROXIMAL OUTCOMES AS MECHANISMS OF CHANGE OR MEDIATORS, PATHWAYS, THROUGH WHICH ADAPTATION CAN HELP US ACHIEVE DISTAL OUTCOMES. WE HAVE TWO QUESTIONS, WHICH AT WEEK ZERO AND WEEK TWO, CONSIDER SEQUENTIAL A SMART. IT'S A RANDOMIZED TRIAL BUT SPECIAL BECAUSE IT INCLUDES MULTIPLE STAGES, SEQUENTIAL RANDOMIZATION. IN THE SMART, EACH STAGE OF RANDOMIZATION CORRESPONDS TO POINT IN TIME IN WHICH WE DON'T KNOW, HAVE SCIENTIFIC QUESTIONS, ABOUT THE SELECTION AND ADAPTATION OF INTERVENTION COMPONENTS. SO, FOR EXAMPLE, THIS IS A SIMPLE EXAMPLE OF A SMART. I WANT TO ANSWER TWO QUESTIONS, WHICH COMPONENT TO OFFER AT WEEK ZERO, TEXT OR PHONE COACHING, WHICH AT WEEK TWO FOR NON-RESPONDERS, BODY TRAINING OR PHONE COACHING. WHAT I CAN DO IS FIRST RANDOMIZE PARTICIPANTS AND WEEK ZERO TO THE TWO COMPONENTS THAT ARE UNDER CONSIDERATION. AND THEN AT WEEK TWO I WOULD IDENTIFY RESPONDENTS AND RERANDOMIZE THEM, SUBSEQUENT COMPONENTS FOR NON-RESPONDERS. RESPONDERS CAN CONTINUE WITH THE SAME INITIAL INTERVENTION. WHAT'S IMPORTANT TO KEEP IN MIND IS THE CONNECTION BETWEEN THE TWO STAGES OF RANDOMIZATION AT WEEK ZERO AND TWO AND TWO SCIENTIFIC QUESTIONS ABOUT THE SELECTION OF COMPONENTS AT WEEK ZERO AND WEEK TWO. THERE'S BEEN A LOT OF WORK SHOWING HOW SMART DESIGNS CAN BE EFFICIENT IN HELPING US ANSWER QUESTIONS ABOUT THE SELECTION AND ADAPTATION OF COMPONENTS DESIGNED TO ADDRESS CONDITIONS THAT CHANGE RELATIVELY SLOWLY OVER TIME. OKAY. NOW I'M SHOWING EXAMPLE ANSWER TO QUESTION NUMBER FIVE. THIS IS A SIMPLIFIED DESCRIPTION OF INTERVENTION, SMOKING CESSATION. THIS INTERVENTION IS BASED ON EVIDENCE SHOWING THAT WHEN SMOKERS ATTEMPT TO QUIT AND EXPERIENCE STRESS, THE STRESS EPISODE IS LIKELY TO LEAD TO A LAPSE, LIKELY TO LEAD TO FULL RE RELAPSE. PARTICIPANTS ARE WEARING A COLLECTION OF SENSORS THAT MONITOR PHYSIOLOGY. THERE'S ANOTHER THAT SAYS TO CLASSIFY EVERY MINUTE AS STRESS OR NO STRESS. IF STRESS IS DETECTED, THE PHONE LIGHTS UP OR RECOMMENDS THE PERSON ENGAGE IN STRESS REGULATION ACTIVITY, ONE OF THREE ON THE DEVICE. >> THREE MINUTES LEFT. >> THANK YOU. NOTICE HERE THAT WE HAVE ADAPTATION HERE BECAUSE WE'RE USING ONGOING INFORMATION ABOUT THE PARTICIPANT'S STRESS TO DECIDE WHETHER OR NOT TO DELIVER INTERVENTION. BUT I WANT YOU TO NOT ADAPTATION HAPPENS MUCH MORE RAPIDLY, EVERY MINUTE. WHY? BECAUSE EVERY MINUTE THE PERSON CAN TRANSITION FROM EXPERIENCE NOTHING STRESS TO EXPERIENCE STRESS PUTTING THEM AT RISK. WE ADAPT MORE FREQUENTLY, MORE FREQUENTLY THAN PREVIOUS EXAMPLE. INTERVENTIONS THAT USE ADAPTATION TO ADDRESS CONDITIONS THAT CHANGE RAPIDLY, THEY ARE CALLED JUST-IN-TIME INTERVENTIONS, DESIGNED TO ADDRESS DISTAL OUTCOME, TO ACHIEVE ADAPTATION IS DESIGNED TO ACHIEVE PROXIMAL OUTCOMES REDUCING PROBLEM BUILT OF RELAPSE IN THE NEXT TWO YEARS, BY DOING THAT WE'RE MORE LIKELY TO ACHIEVE SMOKING CESSATION. LET'S ASSUME WE DON'T KNOW WHETHER OR NOT WE SHOULD DELIVER A PROMPT. THIS RECOMMENDS WE CONSIDER RANDOMIZED TRIAL. MICRORANDOMIZED TRIALS, MRTs, INCLUDES SEQUENTIAL RANDOMIZATION BUT THE SAME PARTICIPANT CAN GET RANDOMIZED HUNDREDS OF THOUSANDS OF TIMES IN THE COURSE OF TRIAL AS OPPOSED TO A SMART. WHY DO WE NEED TO RANDOMIZE RAPIDLY? WE'RE TRYING TO ANSWER QUESTIONS ABOUT SELECTION ADAPTATION, TRYING TO ADDRESS CONDITIONS THAT CHANGE RAPIDLY. AT EVERY GIVEN MINUTE WHEN PEOPLE EXPERIENCE STRESS, I DON'T KNOW IF I SHOULD DELIVER THE PROMPT, I CAN RANDOMIZE PEOPLE TO PROMPT OR NO PROMPT, AT EVERY GIVEN MINUTE THEY EXPERIENCE STRESS. NOW YOU CAN SEE WITH AN MRT WE CAN HAVE HUNDREDS AND THOUSANDS OF MICRO RANDOMIZATION IN THE COURSE OF A TRIAL. ALL RIGHT. MRTs CAN HELP US ANSWER SCIENTIFIC QUESTIONS ABOUT THE SELECTION AND ADAPTATION OF COMPONENTS THAT ARE INTRODUCED RAPIDLY. CAN HELP US ANSWER QUESTIONS ABOUT THE CONSTRUCTION OF JUST-IN-TIME ADAPTIVE INTERVENTIONS. ALL RIGHT. THIS IS A SCIENTIFIC FRAMEWORK. AS I SAID, FOCUSES ON FOUR APPROACHES, CAN BE EXTENDED TO MULTIPLE OUTCOMES AND FRAMEWORK IS LIMITED TO FOUR EXPERIMENTAL, AND THERE ARE OTHER DESIGNS THAT CAN BE USEFUL IN DEVELOPING MULTI-COMPONENT INTERVENTION, IMPORTANT IN HELPING INVESTIGATORS CHOOSE DESIGN, IT CONNECTS TO WHAT LYND AND COLLINS AND LINDA POWELL SAID IN THE PREVIOUS SESSION THIS FRAMEWORK FORCES US TO DECIDE WHICH TYPE OF INTERVENTION WE WANT TO DEVELOP AND WHETHER THE SCIENTIFIC QUESTIONS THAT PREVENT US FROM DEVELOPING THIS INTERVENTION AND MATCH THE DESIGN TO THE SCIENTIFIC QUESTIONS BECAUSE SCIENTIFIC QUESTIONS SHOULD NOT BE EXPERIMENTAL DESIGN AND NOT BE THE OTHER WAY AROUND. THANK YOU. >> THANK YOU. WE'RE READY FOR DR. ROSS HAMMOND TO SHARE YOUR SLIDES. >> ABSOLUTELY. I'M PLEASED TO BE HERE. CAN YOU SEE MY SLIDES OKAY? I ASSUME SO. >> YES. >> I'M GOING TO TALK ABOUT SOMETHING A LITTLE BIT DIFFERENT. I'M GOING TO TALK ABOUT USING COMPLEX SYSTEM SCIENCE, A SET OF TOOLS, FROM A VARIETY OF QUANTITATIVE AND QUALITATIVE PERSPECTIVES, FOR THE PURPOSE OF DESIGNING MULTI-FACETED MULTI-OUTCOME INTERVENTIONS ABOVE THE SKIN PRIMARILY IN USING SOCIAL AND BEHAVIORAL SCIENCE AS OPPOSED TO BIOLOGICAL SCIENCE, SPECIFICALLY TO PREVENT CHILDHOOD OBESITY AND GIVEN THE TIME CONSTRAINT I'M GOING TO GIVE AN OVERVIEW OF A PARTICULAR STUDY WITHOUT GOING INTO TECHNICAL DETAILS BUT I'M HAPPY TO BRING THOSE OUT IN THE QUESTION-AND-ANSWER SESSION. THE CHALLENGE THAT MOTIVATES THIS WORK THAT I'M TALKING ABOUT IS THAT OBESITY IS A COMPLEX PROBLEM. IF WE'RE TRYING TO PREVENT OBESITY WE HAVE TO TAKE INTO ACCOUNT COMPLEXITY. THIS IS A DIAGRAM SHOWING MULTIPLE PATHWAYS AND DETERMINANTS. THIS IS A DIAGRAM WHICH WAS DRAWN, COMMISSIONED BY THE FORESITE GROUP IN THE U.K., EACH OF THESE BOXES IS A MEASURABLE FACTOR, EACH ARROW ARROW REPRESENTATIVES EVIDENCE-BASED CAUSAL RELATIONSHIP. THERE'S LOTS OF HETEROGENEITY IN THE CONTEXT EXPERIENCED BY INDIVIDUALS, STARTING POINTS FOR CHANGE, THERE'S ADAPTATION OVER TIME, POTENTIALLY LONG TIME PERIODS WHERE YOU HAVE TO CONSIDER SUSTAINABILITY OF INTERVENTIONS, AND WE KNOW THAT FOR MANY OBESITY INTERVENTIONS THE IMPLEMENTATION PROCESS ITSELF IS MAKE OR BREAK, KNOWING WHAT TO CHANGE IS NOT ENOUGH, WE NEED TO KNOW HOW TO CHANGE IT EFFECTIVELY AND SUSTAINABLY. THESE CHALLENGES CREATE PROBLEMS BOTH FOR MANY CONVENTIONAL ANALYSIS TOOLS SUCH AS RANDOMIZED CONTROLS TRIED OF TRIALS AND STATISTICAL TOOLS AND CREATE PROBLEMS FOR INTERVENTIONS. AND THE FIELD HAS LED -- STARTED TO REALLY ACCELERATE ITS USE OF TOOLS FROM COMPLEX SYSTEMS WHICH WERE FIRST PROPOSED A DECADE AGO BUT HAVE NOW SHOWN UP IN MANY HIGH PROFILE PLACES INCLUDING IN THE SECRETARY'S ADVISORY COMMITTEE FOR HEALTHY PEOPLE 2030 REPORT IN A PIECE ON COMPLEX SYSTEMS, SCIENCE AND MODELING THAT I CO-AUTHORED WITH BRUCE LEE WHO WILL DO A WRAP-UP TODAY, AND SEVERAL COLLEAGUES. WE'VE COME TO RECOGNIZE THREE KEY THEMES IN HOW TO MAKE OBESITY PREVENTION WORK. THE FIRST OF THESE IS THIS IS A SYSTEMS PROBLEM. WE NEED SURROUND SOUND APPROACHES, MULTIPLE COMPONENT INTERVENTIONS THAT ADDRESS MANY PIECES OF THE PROBLEM WITH MULTIPLE LEVELS AND SECTORS AND MULTIPLE STRATEGIES, AS SHOWN IN THE NICE QUOTE AT THE TOP FROM THE NATIONAL ACADEMY OF SCIENCES RECORD ACCELERATING PROGRESS IN OBESITY PREVENTION. THAT'S HARD TO DO. THE FIELD HAS LEARNED THAT ONE OF THE PLACES WHERE YOU CAN DO THIS EFFECTIVELY IS ACTUALLY AT THE COMMUNITY LEVEL. WHICH IS WHAT THE SECOND PAPER ON THE LIST HERE IS ABOUT. WHY THE COMMUNITY LEVEL? WELL, COMMUNITY LEVEL IS HIGH ENOUGH LEVEL OF SCALE THAT WE CAN ACTUALLY CHANGE PHYSICAL ENVIRONMENTS AND ENVIRONMENTAL DRIVERS, MACRODRIVERS OF OBESITY, BUT A LOW ENOUGH LEVEL OF SCALE THAT WE HAVE A CHANCE TO COORDINATE POLICY AND ACTION ACROSS MANY SECTORS AND USE MULTIPLE STRATEGIES EFFECTIVELY. THE THIRD THING WE'VE LEARNED IS THAT WHAT WORKS IN ONE PLACE MAY NOT WORK IN ANOTHER PLACE, THAT WE NEED TO TAILOR TO CONTEXT BECAUSE OF THE HUGE HETEROGENEITY ACROSS INDIVIDUAL SETTINGS AND STARTING POINTS FOR CHANGE. WE'VE LEARNED THERE'S NO SINGLE SOLUTION OR APPROACH THAT FITS ALL INDIVIDUALS, CIRCUMSTANCES OR CONTEXT, PART OF A NEW MOVEMENT OF WORK THAT I'M TRYING TO ADVANCE ALONG WITH MANY OTHERS THAT'S BEING CALLED PRECISION PREVENTION, ALL ABOUT HOW DO WE USE TOOLS FROM PRECISION MEDICINE AND TECHNIQUES FROM UNDERLYING SCIENCE TO THINK ABOUT PREVENTING CHRONIC DISEASE EFFECTIVELY BY TAILORING TO CONTEXT IN ADDITION TO INDIVIDUAL BIOLOGY SETTING UP A STUDY WHICH I WANT TO TALK ABOUT I SERVED AS P.I. ON OVER FIVE YEARS FUNDED BY NIH, COMPACT, CHILDHOOD OBESITY MODELING FOR PREVENTION AND COMMUNITY TRANSFORMATION, WHAT WORKS, FOR WHOM, WHY. WHOLE OF COMMUNITY CHILDHOOD OBESITY PREVENTION. AND ITS FOCUS IS VERY MUCH UPSTREAM ON IMPLEMENTATION, NOT SO MUCH ON HOW INDIVIDUAL BEHAVIOR CHANGES RESULT IN CHANGE IN BODY MASS IN INDEX, BUT MORE ON HOW DO YOU ACTUALLY CREATE LASTING CHANGE IN POLICY, PRACTICE, ENVIRONMENT UPSTREAM. AND THE GOAL OF COMPACT WAS TO DEVELOP A GENERAL THEORY OF HOW YOU CAN DO THIS KIND OF IMPLEMENTATION IN A COMMUNITY. IT STARTED WITH A STUDY OF PREVIOUS INTERVENTIONS OF THIS TYPE OF WHICH THERE'S QUITE A FEW, AND TRIED TO LOOK AT THOSE BOTH SUCCESSFUL AND UNSUCCESSFUL AND FIGURE OUT WHICH OF THE MANY THINGS THEY DID WAS MOST CRITICAL TO THEIR SUCCESS. THIS IS AN EXAMPLE OF SUCH A STUDY WHICH LOOKED AT RATHER FAMOUS OLD COMMUNITY OBESITY INTERVENTION IN MASSACHUSETTS WHICH WORKED WELL AND THIS WORK DETERMINED THE THING SHOWN ON THE BOTTOM IN BROWN, LABELED THE SHAPEUP SUMMERVILLE TASK FORCE WAS CRITICAL TO THE SUCCESS OF THIS INTERVENTION, THIS WAS A GROUP OF COMMUNITY STAKEHOLDERS WHO FORMED A COALITION WHICH DROVE -- DESIGNED AND DROVE FORWARD THE INTERVENTION. ON FURTHER RESEARCH INTO THIS SPACE WE FOUND THAT ACTUALLY ALMOST ALL OF THE SUCCESSFUL WHOLE OF COMMUNITY INTERVENTIONS HAD ONE OF THESE COALITIONS. AND VERY FEW OF THE UNSUCCESSFUL ONES DID. WE FOCUSED OUR ATTENTION ON THIS ASPECT OF INTERVENTIONS. AND WHAT WE DID IS DEVELOPED AND MATHEMATICIZING, THE WAY YOU SUSTAIN AND DEVELOP WORKABLE COMMUNITY LEVEL OBESITY PREVENTION INTERVENTION IS BRING TOGETHER THE RIGHT PEOPLE TO HAVE THE RIGHT CONVERSATIONS, TO REACH THE RIGHT GROUPS, TO DIFFUSE THOSE THINGS THE COMMUNITY NEEDS TO CREATE PRECONDITIONS FOR SUCCESSFUL CHANGE. AND WHAT DOES IT MEAN TO REACH THE RIGHT PEOPLE AND TO HAVE THE RIGHT CONVERSATIONS AND REACH THE RIGHT GROUPS? THAT'S WHERE THE SCIENCE COMES IN. WE USED A VARIETY OF TOOLS TO OPERATIONALIZE THIS THEORY TO DEVELOP MEASUREMENT TOOLS TO DEVELOP COMPUTATIONAL PREDICTIVE MODELS THAT COULD HELP US USE NETWORK ANALYSIS TO PICK THE RIGHT PEOPLE TO ENGAGE WITH AND THE RIGHT KIND OF ACTIVITIES TO DO WITH THEM, SO THAT WE COULD ACTUALLY TRANSFORM COMMUNITIES AND DRAMATICALLY INCREASE SUCCESSFUL IMPLEMENTATION OF ACTIONS TO CHANGE THEIR ENVIRONMENT AND THUS OBESITY. WE'VE TESTED THIS WORK IN A VARIETY OF COMMUNITIES NOW, BOTH IN THE U.S. AND GLOBALLY, AND I CAN'T GO INTO MANY OF THE DETAILS BUT THE KEY TO GO ALIGN WITH THE THEME OF THE SESSION THERE WERE MULTIPLE INTERVENTION ELEMENTS, MULTIPLE METHODOLOGIES, MULTIPLE OUTCOMES THAT WE'RE TRYING TO CHANGE AMONG THEM THE IDEAS OF KNOWLEDGE AND ENGAGEMENT WHICH ARE MULTI-FACTORIAL THEMES WE LEARNED AND DEVELOP MEASUREMENT TOOLS TO STUDY ALONG WITH NETWORK STRUCTURE AND WE BECAME SOPHISTICATED IN HOW WE THOUGHT ABOUT THESE VARIOUS ASPECTS OF HOW INDIVIDUAL CHARACTERISTICS ARE DISTRIBUTED ACROSS MATHEMATICAL NETWORK STRUCTURE IN A COMMUNITY. AND WE USED A VARIETY OF TOOLS TOGETHER, INCLUDING SYSTEM SCIENCE TOOLS CALLED GROUP MODEL BUILDING WHICH YOU'RE HEAL ABOUT IN THE PANEL LATER THIS SESSION, OF TOOL CALLED SYSTEMS MAPPING WHICH IS A QUALITATIVE TOOL TO MANAGE COMPLEXITY VISUALLY AS I SHOWED EARLIER WITH THE SHAPE UP SUMMERVILLE MAP IN ADDITION TO MODELS OF TESTABLE PREDICTIONS OF HOW DIFFUSION WILL OCCUR AND SOCIAL NETWORK ANALYSIS TO COLLECT DATA ABOUT STRUCTURE OF COMMUNITY NETWORKS. THESE THINGS WORK TOGETHER TO DESIGN AND FIELD INTERVENTIONS WHICH WERE SHOWN TO HAVE QUITE A LOT OF SUCCESS. AND WE'RE ABLE TO TEST THESE MODELS, USING VARIETY OF DATASETS THAT BOTH EXISTED PREVIOUSLY AND THAT WE WERE ABLE TO COLLECT WITH THESE NEW CUSTOMIZED TOOLS, TO SEE WHAT THIS KIND OF APPROACH COULD ACTUALLY DO IN THE FIELD, AND THEN THIS ENDED UP BEING AN APPROACH THAT WAS RECOMMENDED IN THE RECENT "LANCET" COMMISSION REPORT I HELPED WRITE ABOUT A YEAR AND A HALF AGO THAT FOCUSED ON THIS IDEA OF THE IMPORTANCE OF LOCAL CONTEXT AND TAILORING INTERVENTION ELEMENTS TO THOSE CONTEXTS, AND ON THIS APPROACH OF IDENTIFYING BOTTOM-UP CHANGE AGENTS THAT COULD CATALYZE COMMUNITY ACTION BY FOCUSING BOTH ON THE NETWORK PROPERTIES OF THE INDIVIDUAL SETTING AND DISTRIBUTION ACROSS THOSE NETWORKS OF INDIVIDUAL, SOCIAL, BEHAVIORAL PROPERTIES. THIS STUDY AGAIN IS CALLED COMPACT, CHILDHOOD OBESITY MODELING, THERE ARE A NUMBER OF PAPERS I CAN POINT THOSE INTERESTED IN METHODOLOGY TO BUT TAKEAWAY LESSONS FOR THIS GROUP BOTH ARE THAT THIS KIND OF MULTI-INTERVENTION, MULTI-EFFECT STUDY CAN BE DONE IN SOCIAL SCIENCE SETTING AS WELL AS BIOLOGICAL SETTING, ABOUT PREVENTION AS WELL AS TREATMENT, AND THAT THE USE OF COMPLEX SYSTEMS TOOLS INCLUDING THE ONES MENTIONED HERE CAN BE AN EFFECTIVE WAY BOTH TO DESIGN SUCH A STUDY AND DO ANALYSIS TO DEMONSTRATE THAT THE POWER OF THE APPROACH AND MAKE TESTABLE PREDICTIONS ABOUT CAUSALITY THAT YIELD INSIGHT TO WHAT WORKS AND WHAT DOESN'T. I'M CLOSE TO THE END OF MY PROJECTED TEN MINUTES, SO I WILL STOP HERE AND HAPPY TO FIELD LOTS OF QUESTIONS DURING THE DISCUSSION LATER. >> THANK YOU SO MUCH, DR. HAMMOND. NOW WE'RE READY FOR DR. BUTTE TO PLEASE GO AHEAD AND SHARE YOUR SLIDES. THANK YOU. >> GREAT. THANKS FOR HAVING ME. AND I KNOW WE'RE WRAPPING UP, THIS IS DAY 2 OF THE MEETING SO I'LL TRY TO STAY ON TIME HERE. I THINK THIS SESSION HAS BEEN ABOUT PROPOSING ALTERNATIVE METHODOLOGIES AND SOURCES OF DATA FOR WHOLE PERSON RESEARCH. I'M GOING TO TAKE PERSPECTIVE OF HEALTH SYSTEM BRINGING CLINICAL DATA OVER THE NEXT 15 MINUTES OR SO. FIRST, MY CONFLICTS OF INTEREST, I COULD SPEND AN HOUR ON THESE, I WOULDN'T BLAME YOU IF YOU DIDN'T BELIEVE ANOTHER WORD I SAID OVER THE NEXT 15 MINUTES. LET ME INTRODUCE UNIVERSITY OF CALIFORNIA. WE'RE ENORMOUS, TEN CAMPUSES, THREE NATIONAL LABS INCLUDING LAWRENCE LIVERMORE, 200,000 EMPLOYEES, QUARTER MILLION STUDENTS PER YEAR, DECENT AMOUNT OF FUNDING, $2 BILLION IN NIH, 10% OF RESEARCH PRODUCTIVITY COMES FROM THE UNIVERSITY OF CALIFORNIA. IF YOU TAKE THE U.C. SYSTEM, WHAT I'M TRYING TO INTRODUCE YOU TO IS THE NEW UMBRELLA OVER HEALTH ENTERPRISE, UC HEALTH, SIX MEDICAL SCHOOLS, 14 HEALTH PROFESSIONALS, NURSING, PHARMACY, VETERINARY, DENTAL. IF THEY DON'T STAY WHAT WE TEACH HAS INFLUENCE AS THEY GO OUT. WE MAKE 13 BILLION SEEN PATIENTS A YEAR, NUMBER 10 IN THE COUNTRY, 5,000 DOCTORS GET A PAYCHECK FROM US, 100,000 WRITE ORDERS ON OUR PATIENTS, 12,000 NURSES, AND UCSF AND UCLA ARE IN THE UNIVERSITY TOP TEN, WE MENTIONED THOSE IN WHEN WE'RE IN THE TOP TEN, OR FIVE COMPREHENSIVE CANCER CENTERS, IRB RELIANCE. THIS UMBRELLA OVER U.C. NOW MEANS THAT WE REALLY ARE TRYING TO ACT AS ONE HEALTH ENTERPRISE. NOWHERE ELSE DO THESE SIX MEDICAL SCHOOLS, SIX ACADEMIC HEALTH SYSTEMS, ALL WORK TOGETHER ESPECIALLY SHARING DATA AT THIS LEVEL. THIS IS A UNIQUE OPPORTUNITY FOR OBVIOUSLY IMPROVING PRACTICE OF MEDICINE BUT ALSO OBVIOUSLY FOR RESEARCH. AND THE TYPE OF DATA I'M SPECIFICALLY TALKING ABOUT IS ELECTRONIC HEALTH RECORD DATA. EHRs ARE SO OLD THEY ARE NEW AGAIN. IN FACT, THEY ARE EXPENSIVE. WE'RE SPENDING BILLIONS PUTTING THEM N IN. SIREN SPENT A BILLION DOLLARS. MASS GENERAL AND OTHERS IN BOSTON SPENT $1.2 BILLION, AND KAISER PERMANENTE SPENT $4 BILLION, THE BUDGET OF ENTIRE INSTITUTES OF NIH. THEY ALL BROUGHT THE SAME SYSTEM, EPIC. PRICE TAG FROM $1 BILLION, TO $4 BILLION. WE'RE PAYING DOCTORS TO TYPE IN STUFF, SO THE NARRATIVE IS IT WILL BE A NATIONAL TRAGEDY IF WE DON'T USE THIS DATA TO IMPROVE THE PRACTICE OF MEDICINE AND FOR RESEARCH INTO THAT. OF COURSE WE HAVE TO DO IT SAFELY, RESPONSIBLY, RESPECTFULLY. BUT WHAT A TRAGEDY AFTER SPENDING HUNDREDS OF BILLIONS WE DON'T USE THE DATA. WE HAVEN'T SPENT BILLIONS ON OURS, PROBABLY MANY MILLIONS. WE HAVE A CENTRAL DATA WAREHOUSE THAT TAKES IN DATA FROM OUR SIX ACADEMIC HEALTH SYSTEMS AND PUTS IT INTO THE CENTRAL DATA WAREHOUSE. YOU SEE THE LOGOS. RIVERSIDE IS TINY, BRAND NEW MEDICAL SCHOOL, DON'T HAVE A HOSPITAL SYSTEM. THE OTHERS ARE ENORMOUS. WE WERE HAPPY WITH THIS DATA MONTHLY AND NOW BECAUSE OF COVID WE MOVED MUCH OF THE DATA DAILY EVERY NIGHT, FROM 6 TO 9 IN 9 9 -- 6 TO 9 IN THE MORNING PACIFIC TIME. WE'RE USING ANOTHER OMOP, A VENDOR NEUTRAL FORMAT, SO PATIENTS GO HERE, LAB TESTS THERE, WE INSTALLED IN 2012, COMING UP ON TEN YEARS OF LONGITUDINAL DATA ON MANY PATIENTS. THIS IS STRUCTURED DATA, WE'RE GETTING UNSTRUCTURED. WE CROSSED 7 MILLION PATIENTS, MODERN DATA. EVERY PULSE, EVERY VITAL SIGN, EVERY DOSE OF DRUG FOR 7 MILLION PATIENTS, MORE THAN HALF A BILLION PROCEDURES, QUARTER MILLION ENCOUNTERS, 798 MILLION MEDICATIONS, 720 MILLION DIAGNOSIS CODES, PAIN SCORES, ALL THOSE NUMBERS ARE IN THIS DATABASE. IT'S EVERYTHING FROM TYLENOL TO CAR T CELLS, TIE TYLENOL IS THE CHEAPEST, CAR T CELLS MEANS EXTRACT BLOOD, FIND CANCER CELLS. WE'RE GETTING BETTER AT TEXT, ANOTHER DISCUSSION. AND WE CAN RUN WHAT'S CALLED DEATH INDEX FOR THE STATE OF CALIFORNIA, EVERYONE HAS TO KEEP TRACK OF WHO IS DYING BECAUSE OF SOCIAL SECURITY. WE HAVE THAT CONTRACT TO MIX CLOSE. WE HAVE CLAIMS, TAKE CARE OF EMPLOYEES, PUNCHLINE IS SAFE, RESPECTFUL, RESPONSIBLE USE OF DATA. NOW THAT WE'VE EARNED OUR KEEP WE CAN USE THIS FOR RESEARCH AS WELL. THIS IS A LOCATION OF THE 6.3 MILLION PATIENTS IN THIS PART OF THE COUNTRY, NORTHERN CALIFORNIA, UCLA, U.C. DAVIS AND SOUTHERN CALIFORNIA UCLA, SAN FRANCISCO DAVIS IN THE IN ORDER, LA, IRVINE, SAN DIEGO IN THE SOUTH, RIVERSIDE A TINY DOT. YOU SEE AGE, RACE, GENDER, ETHNICITY, MOST TERTIARY CARE, PRIMARY CARE, AND DEPRIVATION, I'M NOT SURE HOW MANY TALKED ABOUT SOCIAL DETERMINANTS OF HEALTH. CALCULATING THIS FROM WHERE PEOPLE LIVE IS JUST THE TIP OF THE ICEBERG. WE HAVE TO GO THROUGH NOTES, SOCIAL THINGS, A LOT, RIGHT? WE'VE GOT TO START WITH FIGURING OUT SOME ASPECTS OF SOCIAL DETERMINANTS OF HEALTH HERE. JUST TO PROVE THIS ISN'T JUST CLAIMS DATA, HERE ARE 3.1 MILLION HEMOGLOBIN A1c MEASURES, A PATIENT WITH DIABETES GETS MEASURED THREE OR FOUR TIMES A YEAR. WE USE THIS TO IMPROVE PRACTICE OF MEDICINE. WHAT WE CAN DO IS TAKE ANY DISEASE, WHETHER WE SEE THEM AS SPECIALISTS AS PRIMARY CARE, ALL THE 40,000 PATIENTS WITH DIABETES, OLD SLIDE, NOW UP TO 46,000 PATIENTS THAT WE TREAT FOR TYPE 2 DIABETES. YOU CAN SEE THE NUMBER. I COPY AND PASTED THE WRONG SLIDE BUT WE UPDATE MONTHLY. WHO TAKES CARE OF THESE PATIENT, ENDOCRINOLOGIST, PRIMARY CARE, IS IT INCREASING OR DECREASING? AT THE BOTTOM EYE HEALTH, KIDNEY HEALTH, COMPARE THIS TO BENCHMARKS. I WOULD ARGUE NOWHERE ELSE IN AMERICA DO WE HAVE FIVE OR SIX ACADEMIC MEDICAL CENTERS ON A SINGLE DASHBOARD UPDATED REGULARLY. LET'S GET INTO SOCIAL DETERMINANTS. I WANT US TO THINK ABOUT THIS AS INPUT FOR MODELING. WE CAN USE THE ADDRESS, WE START WITH IDENTIFIABLE DATABASE, GEO CODE EVERY LATITUDE AND LONGITUDE OF EVERY PATIENT. FIGURE OUT WHAT IS IT LIKE IN THEIR CENSUS TRACT, NOT JUST ZIP CODE BUT CENSUS TRACT. AND THIS IS A MAP OF CALIFORNIA, BLUE MEANS LEAST DISADVANTAGED NEIGHBORHOODS. RURAL AREAS ON THE RIGHT MOST DISADVANTAGED. IF YOU LOOK ACROSS THE COUNTRY, EVEN STATES ARE MUCH MORE DISADVANTAGED THAN CALIFORNIA I WOULD ARGUE. THIS IS OUR NEIGHBORHOOD OF OUR SIX ACADEMIC HEALTH CENTERS, SPAT SMATTERING OF BLUE AND RED, ALONG THE COAST MORE FAVORABLE NEIGHBORHOODS. WE CAN MIX THIS UP WITH PHYSIOLOGIC AND LAB TEST DATA, WE'VE DONE THIS FOR TYPE 2 DIABETES. WE TAKE CARE OF PATIENTS FROM AN AREA OF ONE TO AREA OF TEN, LEAST DISADVANTAGED TO MOST DISADVANTAGED, OBVIOUSLY WE'RE ON THE COAST SO WE TAKE CARE OF MORE PRIVILEGED PATIENTS, WE ADMIT THAT. WE CAN PUT THAT AS INPUT INTO MODELING. WHAT WE'VE LEARNED IS REGARDLESS CONTROLLING FOR YOUR AGE, SEX, RACE, AND ETHNICITY, CONTROLLING FOR THOSE, IT STILL MAKES A DIFFERENCE WHERE YOU HAPPEN TO LIVE. IF YOU LIVE IN THE LEAST DISADVANTAGED AREAS ON THE LEFT YOUR HEMOGLOBIN A1c COULD BE A HALF POINT LOWER THAN AT THE MOST DISADVANTAGED AREAS ON THE RIGHT. I SAY HALF A POINT, PHARMA COMPANIES START A NEW DRUG THEY ARE TRYING TO GET PATIENTS BETTER BY HALF A POINT. IT MAKES A DIFFERENCE WHERE YOU LIVE IN CALIFORNIA. LIKE STARTING OR ENDING A DRUG FOR DIABETES. OF COURSE ALL THE REST OF THE MODELS AND THINGS AS WELL. ALL OF THIS REAL WORLD DATA, WE PUT OUT A PAPER LAST YEAR IN JCI. YOU CAN SEE THE LINK. CLINICAL DATA IS A CHUNK OF THAT DATA, WE SURFACE FOR PATIENTS, FOR RESEARCHERS, OPERATIONAL CARE, MAKE SURE WE'RE IMPROVING PARAGRAPH MEDICINE AND NOW MAKING IT AVAILABLE FOR RESEARCH. HERE ARE 21 USES WE MODELED OR DISPLAYED AT THE TIME. EVERYTHING FROM POST-APPROVAL SAFETY, CLINICAL TRIAL DESIGN, IMAGINE DESIGNING A TRIAL KNOWING HOW MANY PATIENTS ARE COMING IN IN THE NEXT 90 DAYS. THERE'S A LINK WHERE YOU CAN FIND THE PAPER. I THINK IT'S OPEN ACCESS. YOU HAVE TO WORK AT U.C. HEALTH. WE'RE ALWAYS RECRUITING. IF -- LITERALLY THE SAME QUERY, THAT WORKS IN OUR DATABASE, SAME SEQUEL QUERY ON ONE CAMPUS RUNS ON OUR CAMPUS CENTRALLY, FIVE OR SIX TIMES THE ANSWER. THIS EXECUTES ON THE CLOUD. WE HAVE ONE VERSION, THE BOOK ENDS, FULLY IDENTIFIABLE ON THE LEFT, COMPLETELY LEGALLY DE-IDENTIFIED ON THE RIGHT. EVERYTHING IN BETWEEN IS WHAT WE CALL HIPAA LIMITED DATASET. YOU CAN UPLOAD BUT YOU CAN'T DOWNLOAD, YOU HAVE TO SIGN FOR THAT. WE WOULD SAY THIS IS SAFE RESPECTFUL REGULATED USE OF CLINICAL DATA. TO PROVE THIS IS EXISTS, THIS IS MICROSOFT AZURE, THEIR CLOUD. AND DATABRICKS, IF YOU KNOW JUPYTER NOTEBOOKS, THIS IS UC-BERKELEY. IF YOU WANT TO TRAIN OR BUILD HAVE YOU 100 ARROWS POINTING AT 165 MILLION AMBULATORY VISITS TO TRAIN YOUR A.I. ON HERE. SO THIS EXISTS, THIS IS REALLY. ALL THIS IS TO ENABLE U.C. RESERVERS AND PATIENTS TO GO BEYOND AND ENABLE RESEARCHERS IN A SAFE, RESPECTFUL, FAIR AND EQUITABLE WAY BECAUSE THAT'S THE BEAUTY, IT'S WHAT I LOVE ABOUT THE UNIVERSITY OF CALIFORNIA, DIVERSITY OF THE WORLD IS IN CALIFORNIA, AND IT'S TREATED BY THE UNIVERSITY OF CALIFORNIA. >> TWO MINUTES. >> PEOPLE MAKE THIS HAPPEN ACROSS THE CAMPUSES, THESE FOLKS CAME TOGETHER DURING COVID. I THANK ZUCKERBERG, BAKER FOUNDATION, BLESSED WITH 20 NIH GRANTS FROM 11 INSTITUTES. IN RED THEY NEED MORE MONEY BUT I LOVE THE ONES IN ORANGE. FDA, MANY OTHER FOUNDATIONS, I THANK MY ADMIN, TECH STAFF, MENTORS AND MY FAMILY FOR LETTING ME DO ALL OF THIS WORK. LET ME STOP THERE. THANK YOU VERY MUCH. >> THANK YOU SO MUCH. AND YOU WERE PERFECT ON YOUR TIME. I WAS JUST ABOUT TO GIVE YOU THE TWO-MINUTE WARNING. AT THIS POINT WE'RE READY TO MOVE INTO THE PANEL DISCUSSION SESSION WHICH IS GOING TO BE MODERATED BY DR. KIM FROM NCCIH. TO KICK THINGS OFF, FOR DR. KIM, I'M GOING TO SHARE JUST THE QUESTIONS. WE'LL SHARE JUST THE QUESTION SLIDE WHILE DR. KIM KICKS THINGS OFF AND ONCE DISCUSSION STARTS WE'LL BRING THAT DOWN. >> THANKS, CATHERINE. WHAT A SPECTACULAR TALK. I WAS AMAZED BY THE VARIETY OF APPLICATIONS TO STUDY IMPACTS OF THE MULTI-COMPONENT INTERVENTIONS ON MULTI-SYSTEM OUTCOME. I REALLY APPRECIATE ALL OUR SESSION SPEAKERS. I'D LIKE TO INVITE ALL THE SESSION SPEAKERS BACK ON THE VIRTUAL STAGE TO TAKE SOME QUESTIONS AND ANSWER THOSE COMING FROM NIH VIDEOCAST AS WELL AS PANELISTS. LET'S START WITH THIS GENERAL QUESTION. I REALIZE MANY OF THE SPEAKERS ARE NOT ACTUALLY RELATED TO THE QUESTIONS SO I'D LIKE TO START WITH DR. KNIGHT. I KNOW THAT MY FIRST QUESTION IS LIKE SO IT'S ABOUT AROUND THE HOW THE TRANSLATABLE TO USING I'M SUBJECTS, YOU KIND OF TOUCHED UPON THAT ANIMAL MODEL TO USE THE MICROBIOME AND METABOLOMIC SIGNAL WITH MULTI-COMPONENT INTERVENTION OUTCOME SO CAN YOU MAKE SOME COMMENT ON WHAT WOULD BE THE BEST PRACTICE TO USE THE PRE-CLINICAL MODEL TO GET BETTER THE RESEARCH IN THE CLINICAL STUDY SO THERE MIGHT BE SOME ADVANTAGE AND DISADVANTAGE AND ALSO LIMITATION TO USE THE PRE-CLINICAL MODEL TO MOVING TOWARD THE CLINICAL STUDY. >> YEAH, ABSOLUTELY. ALL OF THE METHODOLOGIES I SHOWED ARE APPLICABLE TO BOTH PRE-CLINICAL MODELS AND HUMAN SUBJECTS WITH EXCEPTION OF THE METABOLOMICS AND MICROBIOME STUDIES WHICH REQUIRE YOU TO DISSECT THE INDIVIDUAL INVOLVED SO THOSE APPROPRIATE FOR HUMAN RESEARCH SUBJECTS OBVIOUSLY. WE HAVE HAD A LOT OF SUCCESS GENERALIZING MICROBIOME FINDINGS BETWEEN HUMANS AND ANIMALS SO, FOR EXAMPLE, INFLAMMATORY BOWEL DISEASE WE'RE ABLE TO TRAIN A MODEL ON HUMAN SUBJECTS OR DOGS AND APPLY TO OTHER SPECIES LOOKING AT THE KEY MICROBES INVOLVED THERE, USING RANDOM (INDISCERNIBLE) TO IDENTIFY THOSE MICROBES. WE'RE WORKING ON UNDERSTANDING HOW YOU TRANSLATE TEMPORAL AND SPATIAL SCALES ESPECIALLY BETWEEN MOUSE MODELS AND HUMANS AND WE WOULD BE INTERESTED IN WORKING WITH PEOPLE ON THOSE ISSUES. IN OTHER WORDS IF DOING CIRCADIAN STUDY IN A MOUSE CAN YOU TRANSLATE TO A HUMAN OR DO YOU NEED TO ADJUST THE TIME SCALE OF THE INTERVENTION. ONE KEY PROBLEM AS I SHOWED YOU IN HUMANS IF YOU DO A DIETARY INTERVENTION RESPONSE TIME IS SLOW, YOU WILL NOT SEE A CHANGE IN A WEEK EVEN ON EXTREME INTERVENTION, COMPARED TO DIFFERENCES BETWEEN PEOPLE BUT IF DO YOU IT IN MICE YOU WILL GET A CHANGE AFTER ONE DAY, SO UNDERSTANDING THOSE DIFFERENCES AND TIME SCALE BETWEEN SPECIES WE THINK IS A MAJOR CHALLENGE FOR INCREASING GENERALIZABILITY, TRUE FOR CIRCADIAN AND SLEEP STUDIES THAT I DIDN'T SPEAK OUT IN THAT TALK. >> THANKS VERY MUCH. TRANSMIT ANIMAL MODEL TO HUMANS, A LOT OF CHALLENGING. I SAW THE HAND WAVE. DR. SHINTO? >> HI. I HAD A QUESTION FOR ATUL. I WANT TO COMMENT, LIKE WOW! VERY IMPRESSIVE. AND I'M A CALIFORNIAN, MY MOM GOES TO UCSF, ONE OF THOSE DATA POINTS. MY QUESTION IS UCSF, UCLA, UCSD HAVE VERY LARGE INTEGRATIVE MEDICINE CENTERS. AND I'M SURE THEY ARE COLLECTING DATA ON YOUR POINT. I WAS WONDERING IF IT'S POSSIBLE TO PARSE, TO PULL THAT INFORMATION AND LOOK AT SORT OF COMPARATIVE DATA, WHATEVER THE QUESTION WOULD BE, LOOKING AT INTEGRATIVE MEDICINE USE UNIVERSE US -- AND OUTCOMES THAT WAY. >> YEAH, IT'S A GREAT QUESTION. ABSOLUTELY THE FIRST ANSWER, SHORT ANSWER YES, ABSOLUTELY. EACH CAMPUS THOUGH, THE DEVIL IS IN THE DETAILS. FOR EXAMPLE, WHEN WE PRESCRIBE A DRUG OR WHEN WE ORDER A DRUG, IT'S VERY EASY TO LINK TO NATIONAL -- LET'S SAY STANDARDIZED VOCABULARY, LIKE RxNORM IS A CODE NUMBER, NDC, THEY ARE THE SAME THEY HAVE THE SAME RxNORM COLD OR NDC. WE DON'T HAVE AS FAR AS I KNOW OUR DATA LINKED TO STANDARDIZED NOMENCLATURE OR ARE THERE STANDARDIZED LISTS WE CAN HARMONIZE DATA ELEMENTS. WE CAN ORDER SOME OF THESE THINGS, FOR EXAMPLE THERAPEUTIC USE OF MARIJUANA, IS THERE AN NDC? WHICH VENDOR, WHICH MANUFACTURER? THINKS LIKE THAT ARE HARDER BUT WE MIGHT HAVE THEM. SOMETHING WILL BE ORDERED, SOMETHING DISPENSED BUT WE MIGHT NOT HARMONIZE IT. WE SHOULD PUSH FOR NATIONAL STANDARDS TO BUILD LARGER DATASETS. ANOTHER WHOLE SET, ANOTHER WAY TO ANSWER THE QUESTION, A BUNDLE OF THESE ARE CAPTURED IN THE NOTES, CLINICAL NOTES. WE NEED TO GET INTO THE NOTES. WE'VE HAD AN EFFORT FOR SIX YEARS AT UCSF UNDERSTANDING THE NOTES. WE HAVE 105 MILLION NOTES AT UCSF, CLINICAL NOTES. WE'VE GOTTEN THEM ALL COMPUTATIONALLY REDACTED, IDENTIFIERS STRIPPED, CERTIFICATED BY A THIRD PARTY, THE LARGEST CORPUS OF DE-IDENTIFIED CLINICAL NOTES PROBABLY IN THE WORLD. WAY LARGER THAN MIMIC, IF YOU KNOW WHAT THAT IS IN BOSTON AREA. WE'RE NOW PUTTING ALL SORTS OF COMPUTATIONAL FOLKS ON THAT CORPUS. FOR EXAMPLE, DEEP LEARNING ON TEXT IS EXPLODING AS A FIELD. DEEP LEARNING METHODOLOGIES IN GENERAL TEXTS, GPT 3, TRAINING A LOT OF CONVERSATIONAL TEXT ON THE INTERNET, WE'RE TRYING TO BUILD SOMETHING SIMILAR WITH 100 MILLION NOTES AT UCSF. NOW THAT THIS METHODOLOGY IS PROVEN WE'RE READY TO SCALE ACROSS THE ENTIRE UNIVERSITY OF CALIFORNIA, ESTIMATING MORE THAN HALF A BILLION CLINICAL NOTES THERE. IT'S A VERY LONG ANSWER OF OF COURSE WE KNOW IT'S IN THE NOTES BUT WE CAN ORDER THINGS BUT IF A PATIENT COMES AND TELLS US THEY ARE USING THOSE THINGS IT'S GOING TO BE DOCUMENTED AND WE'VE GOT TO GET IT OUT OF THERE. THE CREW QUICK ANSWERS, HUNGER, OTHER THERAPEUTIC USES, WE CAN GET TO THOSE IN THE NOTES AND SEE THE WORDS. HOMELESSNESS, THINGS LIKE THAT, WE CAN GET TO. WE REALLY WANT TO GET TO BETTER UNDERSTANDING OF WHAT IS HAPPENING WITH THE PATIENTS THROUGH THE NOTES. SORRY FOR THE LONG ANSWER. >> THANK YOU. >> DO YOU HAVE A QUESTION FOR DR. BUTTE? >> I DID. >> DID HE COVER? >> YEAH, PULLING FROM THE DIGITAL THERAPEUTICS THEY WILL BE PRESCRIBED EVENTUALLY, IN THE ELECTRONIC HEALTH RECORD TRACKING BACK TO BEHAVIORAL OR OTHER INTERVENTIONS THAT MAY BE DELIVERED THROUGH THE DIGITAL THERAPEUTICS. >> YOU ARE ASKING INCREDIBLY GOOD QUESTIONS. WE'RE PRESCRIBING. WE HAVE 20 APPROVED BY THE FDA. WE HAVE A GRAD STUDENT WORKING ON THIS, WHERE ARE YOU DOCUMENTING, ORDERING THIS APP, RIGHT? AND REAL WORLD EVIDENCE, IS IT WORKING, NOT WORKING, POST APPROVAL, THIS IS EXTREMELY TIMELY QUESTION WE'RE WORKING ON. WE DON'T HAVE FUNDING IF YOU'RE INTERESTED. WE'RE JUST DOING IT. YOU'RE RIGHT. FASCINATING QUESTION. IT'S GOING TO EXPLODE. WE'RE NOT SURE HOW WE'RE ORDERING THESE THINGS YET. WHAT'S THE FORMULARY LOOKING LIKE, WHICH PAYER COVERS IT? THOSE ARE ALL SEEMINGLY TINY DETAILS THAT MAKE A DIFFERENCE IN THE USE OF THESE THINGS. >> I HAVE A QUESTION COMING IN FOR THE DR. NAHUM-SHANI, WHETHER THE DESIGN USE DESCRIBES SMART AND MRT CAN BE USED WITH MULTIPLE OUTCOMES OR DO YOU HAVE TO START AT THE PRIMARY OUTCOME? >> YEAH, IT'S A REALLY GOOD QUESTION. THE ANSWER IS YES THEY CAN BE USED. BUT TYPICALLY THEY USE THE SINGLE OUTCOME. THE METHOD THAT WE CURRENTLY HAVE FOR ANALYZING DATA FROM SMART, ALSO FOR SAMPLE SIZE FOR TRIALS, THEY ARE SUITABLE FOR PRIMARY OUTCOME. BUT THERE ARE VARIOUS WAYS TO THINK ABOUT INCORPORATING MULTIPLE OUTCOMES IN THE ANALYSIS OF THE DESIGN. SO, FOR EXAMPLE, YOU CAN THINK ABOUT IF YOU HAVE MULTIPLE OUTCOMES IN MIND THINK ABOUT COMPOSITE OUTCOME. THERE ARE NO FREE MEALS. WHEN YOU USE COMPOSITE OUTCOME YOU MAKE ASSUMPTIONS. AND IN MANY CASES WHAT HAPPENS IS THAT THE EFFECTS, SOME SIGNIFICANT EFFECTS CAN BE WASHED OUT IN A COMPOSITE OUTCOME, SOMETHING TO TAKE INTO ACCOUNT. YOU CAN INCORPORATE WEIGHTS WHEN YOU CREATE COMPOSITE OUTCOME BUT THAT REQUIRES YOU TO ACTUALLY PRIORITIZE YOUR OUTCOMES AND COME UP WITH A WEIGHT, INFORMED BY EXISTING EMPIRICAL EVIDENCE, PRACTICAL CONSIDERATIONS. WHAT I'M TRYING TO SAY THERE ARE SOLUTIONS BUT EACH SOLUTION HAS ADVANTAGES AND DISADVANTAGES. SOMETIMES WHAT PEOPLE DO, THEY HAVE MULTIPLE OUTCOMES, THEY FOCUS ON MULTIPLE TESTING. BUT WITH MULTIPLE TESTING THERE ARE DISADVANTAGES YOU HAVE TO CORRECT, YOUR SAMPLE SIZE, QUALITY OF THE SAMPLE SIZE MIGHT BE HIGHER COMPARED TO A SINGLE OUTCOME. I THINK AT THIS POINT GIVEN THE STATE OF SCIENCE, TO CHOOSE YOUR BATTLE. CHOOSE YOUR DISTAL OUTCOME. THINK ABOUT ULTIMATE GOAL YOU'RE TRYING TO ACHIEVE, FOCUS ON SIZE OF THE TRIAL, PLAN THE TRIAL FOR THIS OUTCOME, AND THEN HAVE OTHER TIMES OF OUTCOMES THAT ARE PRIORITY BE YOUR SECONDARY ANALYSIS OR EXPLORATORY ANALYSIS. >> ALL RIGHT. THANK YOU. THAT'S A GREAT ANSWER. I HAVE A QUESTION FROM THE PANELIST FOR THE DR. BUTTE. CAN YOU SPEAK TO NEEDS OF THE DATA CURATION IN THE DATABASE, WHAT LEVELS OF EFFORTS ARE NEEDED TO CLEAN THE DATA UP SO YOU CAN BE EFFECTIVELY USED. >> GREAT QUESTION. I'LL TRY TO ANSWER BRIEFLY. >> GO AHEAD. >> YEAH, OKAY. SO, WE OFTEN FEEL LIKE ELECTRONIC HEALTH RECORD IS MESSY TODAY. PERHAPS YOU HEARD PEOPLE SAY THAT. I LIKE TO SAY MEDICINE IS A MESSY WORLD. ELECTRONIC HEALTH RECORDS CAPTURE IT ACCURATELY. IT IS -- WE'LL HAVE COMPLICATED THINGS, CONTRADICTION IN THE MEDICAL RECORD, THAT'S HOW MEDICINE IS PRACTICED. IT'S THE LEGAL RECORD NOW HAVE. IT'S NOT GOING TO GET BETTER BY ITSELF. DOCTORS FEEL OVERWORKED. BUT WE CAN FIX TRIVIAL THINGS. WE HAVE AT UCSF INCREDIBLE COHORT OF PATIENTS THAT ARE INCREDIBLY SHORT AND WIDE, SOMEBODY PUTS HEIGHT WHERE WEIGHT SHOULD GO AND WEIGHT WHERE HEIGHT CAN GO. WE CAN FIX THAT, OR USE COMPUTATIONAL TOOLS TO EXTRACT THINGS. FOR EXAMPLE, WHEN WE DO ECHOES WE CAN READ EJECTION FRACTURES OUT OF EVERY NOTE IN THE U.C. SYSTEM, ANY ECHO, WE KNOW EJECTION FRACTION. THERE'S WAYS TO CURATE. COMMON DRUGS, LAB TESTS, AND COVID HAPPENED. AND EACH CAMPUS CAN DO ECMO, ONE OF THE MOST INVASIVE THINGS TO A PATIENT WITH COVID, BUT WE HA NOT HARMONIZED ECMO SETTINGS, THAT WAS WAY LATER ON THE LIST OF THINGS TO DO. OF COURSE THAT CAME RIGHT FRONT AND CENTER. UCLA HELPED WITH THAT WORK, BORROWING FROM WHAT THEY TAUGHT US WERE ABLE TO GET DATA ELEMENTS HARMONIZED BECAUSE OF COVID, WE CAN CHOOSE WHAT TO WORK ON BASED ON PRIORITIES, PROJECTS, THE NEEDS HERE. BUT THE SHORT ANSWER IS IT NEVER ENDS. HOW MUCH WORK YOU COULD PUT INTO CURATING. IT'S ALL ABOUT STRATEGY TO CURATE EFFECTIVELY AND EFFICIENCY I SUPPOSE. >> ALL RIGHT. GREAT ANSWER. THANK YOU. I SAW A RAISED DIGITAL HAND. >> THANKS. I HAVE A QUESTION FOR DR. SCHORK AND NAHUM-SHANI. RANDOMIZED TRIAL DESIGN IN STUDIES FOR HYPOTHESIS-GENERATING VERSIONS OR HYPOTHESIS TESTING, I WONDER GIVEN THE SIMILARITIES BETWEEN N-of-1 DESIGN AND MICRO RANDOMIZED TRIAL DESIGN IN THE FUTURE WOULD YOU EXPECT TO SEE MORE SIMILAR APPLICATIONS OF THESE KIND OF DESIGNS FOR, SAY, BEHAVIOR OR IN-HOUSE INTERVENTIONS OR MAYBE CONVERGENCE? >> YEAH, THIS IS NICHOLAS, I'LL GO FIRST. THERE ARE QUESTIONS ON SIMILAR LINES TO THE ONE YOU'RE PRESENTING. I THINK N-of-1, MORE IMPORTANTLY AGGREGATED N-of-1 TRIALS CAN BE USED TO ADDRESS JUST ABOUT ANY INTERVENTION. WE DISCUSSED THE CONTEXT THE TRIALS ARE PURSUED THAT COULD IMPACT OUTCOMES. THERE'S NO REASON WHY, SAY, A SERIES OF N-of-1 TRIALS COULDN'T BE DONE WHERE I AM NOW, A POOR RURAL COMMUNITY IN SOUTHERN CALIFORNIA, BORREGO SPRINGS VERSUS COASTAL COMMUNITY WHERE THERE'S NO WEALTH. NO REASON N OF 1 TRIALS COULDN'T BE CONDUCTED IN BOTH TO SEE WHO RESPONDED AND MAYBE ATTRIBUTE DIFFERENCES TO THE SOCIAL SETTING OR FACTOR THOSE IN IN A LARGER AND BROADER META-ANALYSIS OF THOSE N-of-1 TRIALS. WHERE PEOPLE GET CONFUSED, IT'S GREAT FOR PEOPLE WITH A RARE GENETIC DISORDER WHERE THERE DIDN'T A LARGER DENOMINATOR BUT WHEN YOU AGGREGATE, AS LONG AS YOU GENERATE ENOUGH DATA ON ANY ONE INDIVIDUAL TO STATE WITH SOME CONFIDENCE WHETHER THEY ARE AN UNEQUIVOCALLY RESPONDER OR NON-RESPONDER, THEN YOU CAN AGGREGATE THE TRIALS. IN MOST TRADITIONAL CLINICAL TRIALS, NOT ENOUGH DATA IS COLLECTED ON ANY ONE INDIVIDUAL TYPICALLY TO MAKE UNEQUIVOCAL CLAIMS ABOUT RESPONSE AND MAYBE INTERVENTIONS BEING RESPONSIBLE FOR THAT PARTICULAR RESPONSE. I THINK ANY SETTING IS RIGHT, INCLUDING PRE-CLINICAL STUDIES. I'M JUST ADD THIS AND THEN BE QUIET. BUT, YOU KNOW, NOW THERE IS THE ABILITY TO CREATE KIND OF INDUCE THE PLURIPOTENT STEM CELLS AND DERIVED CELLS FROM iPSs, FOR WHICH PEOPLE CAN DO WHAT AMOUNTS TO N-of-1 TRIALS IN AN EX VIVO SETTING. SO IF I HAD A MINI-ME IN THE DISH FOR, SAY, MY LIVER OR KIDNEY, I COULD TRY OUT DIFFERENT DRUGS, SEE WHICH ONES APPEAR TO INFLUENCE PHENOTYPES THAT MIGHT BE RELEVANT IN THAT DRUG WAS GIVEN TO ME IN MY BODY SO YOU COULD SCREEN. THIS IS DONE NOW BELIEVE IT OR NOT FOR A LOT OF CANCER SETTINGS WHERE YOU CAN MAKE TUMOR ORGANOIDS. PRE-CLINICAL AND CLOSE TO CLINICAL SETTINGS ARE ALSO RIPE FOR N-of-1 DESIGNS BUT I SHOULD LET DR. SHANI SPEAK. >> I GUESS TRADITIONALLY, THE KEY DIFFERENCE MIGHT BE BETWEEN MRTs AND N-of-1s IN THE SCIENTIFIC QUESTIONS THE DESIGNS TO TRYING TO ANSWER. TRADITIONALLY, N-of-1 STUDIES ARE TRYING TO ANSWER A QUESTION, SIMILAR TO SET OF RANDOMIZED TRIALS, COMPARISON BETWEEN DRUG A OR B, COMPARISON BETWEEN PACKAGE OR B, SELECTION BETWEEN COMPONENTS, C, D, E, RIGHT? AND SUPER USEFUL WHEN THE SAME PERSON CAN SERVE EASY HIS OWN CONTROL AND WHEN THERE ARE ISSUES, CHALLENGES WITH RECRUITING LARGE NUMBER OF PARTICIPANTS TO THE TRIAL, THE SAME PERSON CAN SERVE AS HIS OWN CONTROL, ALSO BASED ON EXAMPLES, IF THERE ARE NO CARRYOVER EFFECTS, OR IF THE WASHUP PERIOD BETWEEN TREATMENT OF CELLS ARE SUFFICIENTLY LONG SO EFFECT OF PRIOR INTERVENTION EPISODE, TREATMENT EPISODE WOULD DISSIPATE BY THE NEXT EPISODE, RIGHT? SO THE DIFFERENCE HERE IS THAT MRTs ARE DEVELOPED TO ANSWER QUESTIONS ABOUT SEQUENCES AND TREATMENTS, AND BECAUSE OF THAT, NOT BECAUSE OF THAT, BUT PART OF IT IS MANY MRTs ARE MOTIVATED BY SCIENTIFIC QUESTIONS ABOUT CARRYOVER EFFECTS. CARRYOVER EFFECTS ARE OF SCIENTIFIC QUESTIONS, SCIENTIFIC INTERESTS TO PEOPLE DESIGNING MRTs, FOR EXAMPLE WE WANT TO KNOW IF THE EFFECT OF THE DELIVERING SINGLE POINT IN TIME IS ACTUALLY INFLUENCED BY PRIOR COMPONENT YOU RECEIVED. ANOTHER THING IS THAT MANY MRTs ARE MOTIVATED, PEOPLE WANT TO KNOW WHETHER EFFECT OF COMPONENT CAN VARY OVER TIME INSTEAD OF ASSUMING IT WAS CONSTANT. THE DIFFERENCE IS IN THE KEY SCIENTIFIC QUESTIONS, HOW THE TWO CAN RUN. >> THANK YOU. I SAW DR. CHEN, DO YOU HAVE A QUESTION? >> YES, I HAVE MAYBE A QUESTION ON THE EFFECT ISSUE. I THINK DR. NAHUM-SHANI CLARIFIED, THE WASHOUT PERIOD NEEDS TO BE SHORT ENOUGH TO ALLOW FOR THE N-of-1 DESIGN. AND THE QUESTION I WAS WONDERING, WHEN WE STUDY MULTIPLE COMPONENT INTERVENTIONS OR JUST PARTICULAR INTERVENTION TYPICALLY HAVE TWO TYPES OF MECHANISTIC QUESTIONS. ONE IS VERY FAST RESPONSE, MECHANISTIC EFFECT OF INTERVENTION, USING MEDICATION AS EXAMPLE, MEDICATING WHAT'S HAPPENING TO YOUR BRAIN, YOUR BODY, VERSUS A LONG-TERM EFFECT, TYPICALLY REQUIRING FOUR TO EIGHT WEEKS OF TRAINING WE LOOK AT BEFORE THE TRAINING AND EIGHT WEEKS AFTER TRAINING, A LONG-TERM CUMULATIVE PHYSIOLOGIC EFFECT. I WANT TO GET CLARIFICATION MAYBE WOULD THE N-of-1 TYPE OF DESIGN BE HELPFUL OR USEFUL FOR THIS KIND OF FAST SORT OF ONLINE MECHANISTIC EFFECT ANALYSIS, BECAUSE YOU WOULD THINK THAT EFFECT IS QUICK AND HOPEFULLY IN A WAY NOT MEDITATING OR WHATEVER THE ACTIVITY GOES BACK TO BASELINE OF SOME SORT. >> ABSOLUTELY. N-of-1 TRIALS OF THE TYPE I DESCRIBE WITH CROSSOVERS AREN'T APPROPRIATE FOR ALL DRUGS. SEVERE CONDITION, YOU WON'T HAVE LUXURY OF TRYING DIFFERENT DRUGS. THERE ARE SITUATIONS N-of-1 TRIALS WITH CROSSOVER PERIODS OR WASHOUT PERIODS OR WHAT NOT ARE NOT APPROPRIATE IN ALL SITTINGS. HOWEVER I DID MENTION ONE COULD DO THE SORT OF REALTIME TESTS TO MAKE DECISIONS ABOUT WHAT SUBSET OF FACTORS BUILT INTO YOUR THERAPEUTIC CONSTRUCT ARE ACTUALLY INFLUENCING THE RESPONSE USING KIND OF SEQUENTIAL REGRESSION PROCEDURES. THE PROBLEM THERE, IT CAME UP IN PREVIOUS TALKS, YOU NEED SOME WAY OF QUANTIFYING THE CONTRIBUTION OF THE DIFFERENT FACTORS THAT ARE BUILT INTO YOUR THERAPEUTIC. AGAIN, IF IT'S A NUTRITIONAL INTERVENTION AND THERE ARE FIVE NUTRIENTS, NIACIN AND RIBOFLAVIN INCREASED IN DOSE, YOU WANT TO KNOW WHICH ONE OF THOSE IS RESPONSIBLE FOR THE PATIENT'S RESPONSE, WHATEVER THAT MIGHT BE, YOU NEED TO MEASURE THOSE OVER TIME, MAYBE DIFFERENT DOSES OR AGAIN ASSUMING SOME EFFECT AND MEASURE RESPONSES, NOT NECESSARILY TRIVIAL BUT IN THE CHAT I DID MENTION THERE IS PRECEDENT FOR DOING THIS. THERE ARE THESE STUDIES USED ROUTINELY IN PHARMA CALLED THERAPEUTIC DRUG MONITORING STUDIES FOR DRUGS THAT HAVE A SMALL THERAPEUTIC WINDOW LIKE CARBOMAZAPEN WHERE YOU HAVE TO GET DOSING RIGHT. THEY MEASURE PLASMA LEVELS, CORRELATE PLASMA LEVELS WITH THE RESPONSE, NOT JUST MERE ADMINISTRATION OF THE DRUG, SO IT'S MORE QUANTITATIVE THAN JUST A QUALITATIVE YES/NO WHETHER YOU'RE ON THE DRUG. AND THOSE SORTS OF STUDIES CAN AVAIL THEMSELVES TO REALTIME ASSESSMENT TO MAKE DECISIONS AFTER A NUMBER OF OBSERVATIONS WHETHER THE DRUG IS SHOWING PROMISE OR NOT OR WHETHER YOU SHOULD CALL IT QUITS. >> THANK YOU FOR THE CLARIFICATION. CAN I HAVE DR. HAMMOND FOR TWO QUESTIONS? DR. HAMMOND? >> YEP, HAPPY TO ANSWER QUESTIONS. >> TWO QUESTIONS FOR YOU. LET'S GO WITH THE FIRST QUESTION. HOW DO YOU RECOMMEND THAT THE PEOPLE START TO VIEW MULTI-COMPONENT INTERVENTION WITH METHODS YOU DESCRIBED, I THINK OBESITY PREVENTION STUDY, AT COMMUNITY LEVEL. >> SURE. SO IF YOU'RE SPECIFICALLY INTERESTED IN BUILDING MULTI-COMPONENT INTERVENTION THERE'S A COMPACT OF TOOLS THAT WORK TOGETHER THAT PRESCRIBE A RECIPE THAT YOU CAN FOLLOW WHERE YOU START BY IDENTIFYING THE MOST IMPORTANT ASPECTS OF THE SOCIAL CONTEXT AND MEASURE THEM. AND THEN YOU USE THAT INFORMATION TO SELECT NODES IN THE NETWORK, MEMBERS OF THE COMMUNITY, YOU GAUGE SPECIFIC SETS OF PROCESSES WHICH CATALYZE CHANGE IN THAT COMMUNITY BY LEVERAGING RESOURCES THAT ARE ALREADY PRESENT IN THE COMMUNITY. THAT'S DESCRIBED IN THE PAPER. THAT'S WHAT THE PROCESS LOOKS LIKE. MORE BROADLY, TOOLS I DESCRIBED FOR MANY KINDS OF MULTI-COMPONENT INTERVENTIONS NOT JUST OBESITY PREVENTION, SIMULATION MODEL, COMPUTATIONAL SIMULATION MODELINGS I DESCRIBED, AGENT-BASED TECHNIQUE, IS A VERY DIFFERENT WAY OF DOING SCIENCE FROM RANDOMIZED CONTROLLED TRIAL OR ANIMAL MODEL OR ANY OTHER THINGS WE'VE HEARD TALKED ABOUT THAT EMBODIES HYPOTHESIZED MECHANISM AND USES TO EXTRAPOLATE BEYOND WHERE DATA CAN BE COLLECTED. IT ALLOWS YOU TO HOOK AT COUNTER-FACTUALS AND THINK ABOUT THE MOST IMPORTANT DATA TO COLLECT, ALL ADVANTAGES THAT MODE OF INQUIRY OFFERS AND THATS THAT BROAD APPLICABILITY TO THINGS WE TALKED ABOUT HERE. >> I WANT TO FOLLOW UP. WE PROBABLY HAVE PEOPLE ONLINE THAT HAVE NEVER DONE THIS WORK BEFORE. SO HOW WOULD YOU RECOMMEND THEY FIND PEOPLE WHO CAN DO THAT KIND OF MODELING OR THAT MIGHT HAVE THAT KIND OF DATA. I THINK WITH PEOPLE THAT ARE REALLY INTERESTED IN FIGURING OUT HOW THEY COULD DO THIS SIMULATION TO FIGURE OUT WHAT COMPONENTS TO INCLUDE, WHAT LEVELS, YOU DON'T KNOW WHERE TO START, WHO MIGHT HAVE THAT DATA? >> I ASSUME THAT QUESTION WAS DIRECTED TO ME. LESS ABOUT HAVING THE RIGHT DATA, MORE ABOUT HAVING THE RIGHT TECHNICAL SKILLS TO DO THIS KIND OF MODELING, AND COMPLEX TRAINING AND COMPLEX METHODS IS PART OF THE ACCREDITATION PROCESS FOR SCHOOLS OF PUBLIC HEALTH IN THE U.S., CREATING A NEW CROP OF PEOPLE WHO KNOW HOW TO DO THEM, SLOWLY WITH SOME LAG. THERE ARE A NUMBER OF CLEARINGHOUSE PLACES CREATED THAT CONNECT PEOPLE WITH TOPICAL DOMAIN EXPERTISE, PEOPLE WITH MODELING SKILLS, A NUMBER FUNDED BY NIH THAT I HELPED SET UP THAT I WOULD BE HAPPY TO SEND LINKS FOR THOSE INTERESTED IN THIS KIND OF CONNECTION AND NIH AND CDC USED TO RUN A SUMMER TRAINING INSTITUTE FOR PEOPLE TO GET LIKE A ONE-WEEK EXPOSURE, TO WORK WITH AN EXPERT. THAT'S BEEN DISCONTINUED BUT I TEACH A SIMILAR SUMMER SCHOOL AT WASHINGTON UNIVERSITY IN ST. LOUIS EVERY YEAR AN ONE AT UNIVERSITY MICHIGAN THAT I HELP WITH, QUITE GOOD, AIM AT PRACTITIONERS, PEOPLE OF OTHER FIELDS WHO WANT TO LEARN ABOUT THESE TOOLS, NOT PEOPLE TRYING TO BECOME MODELERS. >> THANK YOU. >> THE SECOND QUESTION IS ABOUT CONTEXT BEING IMPORTANT FOR DEVELOPMENT AND IMPLEMENTATION IN THE COMPACT STUDY, ANY ADVICE TO THE RESEARCHERS? >> THERE'S A HUGE NUMBER OF EXAMPLES IN CHRONIC DISEASE PREVENTION, THINGS THAT WORK IN MASSACHUSETTS WON'T WORK IN OREGON OR THINGS THAT WORK IN RURAL SETTING WON'T WORK IN URBAN SETTING. AND THAT VARIES FROM TOPIC TO TOPIC. WE'RE DEFINING CONTEXT SPECIFICALLY THE SOCIAL NETWORK STRUCTURE OF A COMMUNITY, WHO TALKS TO WHOM, WHERE POCKETS OF KNOWLEDGE RESIDE, AND SO THE IDEA IS THAT TO UNDERSTAND WHAT IT MEANS TO TRANSLATE SUCCESSFUL INTERVENTION FROM ONE CONTEXT TO ANOTHER YOU NEED ABILITY TO MEASURE ASPECTS OF THE CONTEXT WHICH WE HAVE DESIGNED SOME CUSTOM TOOLS TO DO NETWORK SCIENCE TOOLS AND SURVEY TOOLS THAT UNDERSTAND HOW INDIVIDUAL CHARACTERISTICS ARE DISTRIBUTED ACROSS THAT COMMUNITY NETWORK. AND B, YOU NEED A SIMULATION MODEL TO TELL YOU HERE'S THE COMMUNITY I OBSERVED, STRUCTURED LIKE THIS, THIS IS OUTCOME, NOW I WANT TO MOVE THAT INTERVENTION TO NEW COMMUNITY STRUCTURED DIFFERENTLY, HOW WILL I NEED TO CHANGE, A QUESTION THAT CAN BE ANSWERED. WE'VE STUDIED DIVERSE SETTINGS AROUND THE U.S. AND INTERNATIONALLY TO GET HETEROGENEITY IN THESE THINGS THAT VARY COMMUNITY TO COMMUNITY. >> THANK YOU VERY MUCH. INTERESTING QUESTION TO ALL THE SPEAKERS, (INDISCERNIBLE) ANY VOLUNTEER TO START WITH? >> I'LL START. I'D ARGUE INCLUDING MICROBES WILL TELL YOU A LOT OF PIECES OF THE SYSTEM YOU OTHERWISE MISS AND WHAT WE'RE ESPECIALLY FINDING, COMBINATION OF LOOKING AT MICROBES AND THE METABOLISM BY HOST AND MICROBE TOGETHER AS ESPECIALLY USEFUL TO MECHANISTIC HIGH HYPOTHESES INCLUDING PROSPECTIVE CLINICAL TRIALS, LIKE THE 2002 STUDY, WHERE YOU HAVE PLASMA FROM 2002, THAT COMES TO THE PRESENT, ESPECIALLY USEFUL FOR GOING BACK AND FORTH BETWEEN THE ANIMAL AND HUMAN. GLIZ >> I AGREE. YOU JUST NEED THE RIGHT MEASUREMENTS, THAT COMES WITH A BIG COST. IF YOU'RE GOING TO MEASURE LIKE IN THE CASE I GAVE PLASMA LEVELS OF NUTRIENTS OR DRUGS THAT ARE PART OF A COMPOSITE YOU HAVE TO DO THAT, THAT COSTS. IT COSTS IN TERMS OF CONVENIENCE ON THE PARTICIPANTS IN YOUR STUDY DOING DRIED BLOOD SPOTS OR DRAWING BLOOD. WITH THE RIGHT KIND OF MODELING YOU CAN WORK OUT WHAT INFLUENCES WHAT TO COME UP WITH SOME KIND OF RESPONSE RATHER THAN JUST FINDING ASSOCIATIONS WHICH IS THE HOLY GRAIL OF THESE SORTS OF STUDIES TO REALLY GET AT CAUSES AND NOT JUST ASSOCIATIONS. DOABLE, JUST GOING TO COST MONEY. >> OKAY. >> I'LL TAKE A CRACK TOO. IT WOULD DEPEND HOW YOU DEFINE WHAT THE SYSTEM BOUNDARY IS. I THINK THAT THERE'S, AS SOMEONE WHO DOES COMPLEX SYSTEMS ANALYSIS FOR A LIVING, THERE'S OFTENTIMES A MENTAL MODEL WHERE IF YOU DO AWAY WITH REDUCTION REDUCTIONIST APPROACHES YOU CAN STUDY AN ENTIRE SYSTEM EVEN WITH BROAD BOUNDARIES AND THAT RARELY WORKS WELL, THERE'S A MIDDLE GROUND THAT'S OFTEN MOST PROFITABLE AND SO I THINK BOTH EXAMPLES THAT MY COLLEAGUES GAVE INVOLVE TIGHTLY BOUNDED PIECE OF BROADER SYSTEM THAT CREATES DISEASE AS WE SEE IT AND THOSE PIECES BOTH HAVE LOTS OF COMPLEXITY AND ARE RICH AND INTERESTING, THERE'S A LOT TO LEARN BUT TRYING TO UNDERSTAND THE ENTIRE SYSTEM THAT PRODUCES OBESITY ALL AT ONCE IS VERY CHALLENGING AND IT'S BEST DONE BY BUILDING UP PIECES OF KNOWLEDGE IN THE MODULAR WAY CONNECTED TOGETHER TO CREATE A PATCHWORK RATHER THAN STARTING WITH ENTIRE COMPLEXITY THAT I SHOWED IN THE INITIAL DIAGRAM TRYING TO DO IT ALL AT ONCE. >> I THINK AS I WAS MENTIONING IN THE TALK AS WELL, LET'S NOT FORGET ALL THE OTHER HARDER TO MEASURE THINGS ABOUT OUR PATIENTS. AND OUR PARTICIPANTS. WE CAN DO ALL THE MOLECULAR TESTING BUT IF THEY WANT AFFORD THE MEDICATIONS WE PRESCRIBE OR ARE NOT EATING PROPERLY, THAT SOME OF THOSE THINGS ARE HARDEST TO CAPTURE IN ANY STRUCTURED WAY. I WANT TO POINT OUT CHANGING NATURE OF SOCIETY ITSELF. HOW LET'S SAY A SIXTH GRADER TODAY IN SCHOOL TODAY IN TERMS OF EXERCISE, TV, SITTING AT HOME IS VERY DIFFERENT THAN A SIXTH GRADER FROM 30 YEARS AGO. SO SOCIETY ITSELF IS A MOVING TARGET. GOOD LUCK MODELING THAT, RIGHT? I MEAN, SO ALL OF THOSE PLAY INTO HOW THE PEOPLE WE SEE ARE INFLUENCED BY DISEASE AND THEIR CONDITIONS. >> THAT'S A GREAT POINT. >> IT'S A FANTASTIC GOAL TO FOCUS ON BUT TO MAKE SURE PARTICIPANTS ARE ENGAGED, SOMEHOW WE'RE KIND OF IN THIS TOPIC, GOT LOST A LITTLE BIT, SUPER CRITICAL BECAUSE WE CAN DEVELOP THE MOST AMAZING COMPONENT THAT COULD TARGET AND MAKE A DIFFERENCE IN THE PERSON AS A WHOLE BUT IF THE PERSON IS NOT ACTUALLY USING, NOT JUST USING BUT ENGAGING, PUTTING EFFORT, EMOTIONALLY, COGNITIVELY, PHYSICALLY IN THE COMPONENTS THAT OTHER PEOPLE CAN ACHIEVE GOALS WE ASPIRE TO ACHIEVE. >> THANK YOU FOR THE GREAT POINTS. I HAVE ANOTHER QUESTION FOR DR. SCHORK. YOU PRESENTED N-of-1 STUDIES THAT EXEMPT OUTCOMES ONE AT A TIME. HAVE YOU FUND HOW TO GET GLOBAL OUTCOME INCLUDING OUTCOMES THAN MAYBE OF INTEREST TO THE PATIENT? >> YES, ABSOLUTELY. I DIDN'T WANT TO GIVE AN OVERLY COMPLICATED TALK BUT YOU CAN HAVE MULTIPLE DEPENDENT VARIABLES, MULTIPLE OUTCOMES WHEN YOU DO ANALYSES LIKE I DESCRIBED. IT IS TRUE THAT STATISTICS GET MORE COMPLEX PRESERVING ALPHA, TYPE 1 ERROR RATES BUT IT'S DOABLE. I WOULD ARGUE THOSE STUDIES ARE GOING BECOME THE RULE RATHER THAN EXCEPTION WITH ALL THE DATA COLLECTION AND MONITORING DEVICES THAT ARE EMERGING. IT IS POSSIBLE. I'LL GIVE YOU ONE EXAMPLE, IT BEARS ON THE COMMENT MADE JUST PRIOR ABOUT PATIENT ENGAGEMENT. WE DID A SERIES OF N-of-1 TRIALS ON DIFFERENT ANTI-HYPERTENSIVE, REGISTERED IN clinicaltrials.gov, STOPPED A LITTLE WHILE AGO. BUT WE HAD ONE PATIENT WHERE WE WERE MEASURING MUST BE THINGS. NOT ONLY THEIR BLOOD PRESSURE BUT THEIR WEIGHT, MOOD, OTHER SORTS OF THINGS, WHILE THEY ARE ON, YOU KNOW, EACH OF TWO DIFFERENT ANTI-HYPER TENSIVES WITH WASHOUT PERIODS, BLOOD PRESSURE DROPPED IN INTERVENTION PERIOD SUGGESTING IT LOOKED LIKE IT WORKED BETTER BUT WHEN WE LOOKED AT WEIGHT IT DROPPED BY 10 POUNDS DURING THAT TIME INTERVAL. SO WHAT WE LEARNED WAS THE PATIENT, BECAUSE OF THEIR ENGAGEMENT AND PURSUING N-of-1 STUDY, SEEMED TO BE MORE CONSCIOUS OF HEALTH, CHANGED BEHAVIOR, NORMALIZED BLOOD PRESSURE. WHAT WE LEARNED FROM THAT STUDY IS NOT WHICH OF THE TWO DRUGS WORKED BUT THAT ACTUALLY THIS PERSON JUST THROUGH ENGAGEMENT WAS GOING TO TAKE HEALTH AND BEHAVIOR CHANGES INTO ACCOUNT AND ENDED UP NO LONGER NEEDING TO BE TREATED FOR THEIR HYPERTENSION. SO, YES, THERE ARE GOOD EXAMPLES OF WHERE YOU CAN MEASURE MULTIPLE OUTCOMES MAYBE TO GREAT BENEFIT TO THE PATIENT, SOMETHING I WANTED TO EMPHASIZE ABOUT N-of-1. THE TOP-DOWN APPROACH TO CLINICAL TRIALS WHERE YOU LOOK AT THE COMMUNITY AND WORK YOUR WAY DOWN ARE ONE APPROACH AND THEY HAVE MERIT AND THEY ARE FANTASTIC. THE OTHER IS TO START WITH INDIVIDUAL PATIENTS, SEE WHAT WORKS FOR THEM, AND BEGIN TO AGGREGATE THE INFORMATION THAT YOU HAVE. IN DOING THIS BOTTOM-UP APPROACH YOU HAVE ABILITY TO GAIN INSIGHT TO INDIVIDUAL PATIENTS TO BENEFIT THEM GOING FORWARD AS OPPOSED TO SAYING WE DON'T KNOW WHO IN THE COMMUNITY MIGHT BENEFIT BUT THE COMMUNITY AS A WHOLE IS BENEFITING BUT HERE IT'S PATIENT BASED. >> THANK YOU FOR THE COMMENTS. WE HAVE NO MORE QUESTIONS COMING IN. >> IF THERE'S ANYBODY IN THE VIDEOCAST AUDIENCE THAT HAS A LIST-MINUTE QUESTION FOR THIS SESSION NOW WOULD BE THE TIME TO DO THAT. AND I THINK IF THERE'S ANYBODY ELSE IN THE ROOM, I SEE ANOTHER HAND RAISED. >> THANK YOU. I'M LISTENING TO THE COMMENT, ANSOUNDS LIKE IT'S THE PRACTICE OF PRECISION MEDICINE, NOT A STUDY. THESE N-of-1 STUDIES BEING AN AGGREGATED EXPERIENCE OF DIRECTPRECISION MEDICINE. >> I THINK THAT'S ABSOLUTELY TRUE. I'M NOT AN M.D. MY SIGNIFICANT OTHER WHO SEES PATIENTS MIGHT BE ABLE TO RESPOND. THE LEGACY WAY OF TREATING PATIENTS WAS MORE AD HOC N-of-1s, YOU DON'T KNOW WHAT ANTI-HYPERTENSIVE YOU'LL BENEFIT FROM SO I'LL START YOU ON A DIURETIC AND SEND YOU HOME. KEEP TRACK OF HOW OFTEN YOU GO TO THE BATHROOM OR WHAT SIDE EFFECTS THERE MIGHT BE AND COME BACK IN SIX MONTHS AND WE'LL MEASURE YOUR BLOOD PRESSURE AND IF IT'S FOUND YOUR BLOOD PRESSURE IS NOT NORMALIZED YOU'LL SWITCH TO A BETA BLOCKER. SOMETHING LIKE THAT. YOU HAVE THESE KIND -- EACH PATIENT WAS THEIR OWN LIVED EXPERIMENT BUT THERE WAS BEING PURSUED IN AN AD HOC MANNER. YOU'RE RIGHT, N-of-1 TRIALS ARE SAYING WHAT IS IT THAT WE CAN USE TO BENEFIT THIS PATIENT GOING FORWARD AND HOW CAN WE MAKE OBJECTIVE CLAIMS ABOUT THAT? THERE ARE LOGISTICAL AND EXPENSE ISSUES, YOU CAN'T EQUIP PEOPLE WITH MEASUREMENT DEVICES AND SEND THEM ON THEIR WAY BUT IT IS A PLACE TO START. I CAME TO N-of-1 TRIALS AS AS A GENETICIST. THERE WEREN'T ENOUGH PATIENTS TO DO TRADITIONAL RCT AND HAD TO COLLECT DATA ON INDIVIDUAL PATIENTS. AS PRECISION MEDICINE IDEA GREW WE THOUGHT, WELL, THESE MAKE PERFECT SENSE TO ACTUALLY VET DRUGS OF WHICH NOW THERE ARE MANY, ASOs, CAR T CELLS, AN ASO MADE TO BIND TO MY GENOME MAY NOT WORK FOR YOU BECAUSE MY SEQUENCE IS DIFFERENT. THE ACTUAL CONSTRUCTS ARE TAILORED TO THE INDIVIDUALS. RAISING QUESTIONS ABOUT HOW YOU VET THEIR USE. >> ANY QUESTIONS? IF YOU HAVE QUESTIONS USE THE DIGITAL HANDS. >> ANY LAST THOUGHTS FROM OUR SPEAKERS FROM SESSION 4? >> SOUNDS LIKE IT'S DRAWING THIS TO A CLOSE. IS THERE A FINAL COMMENT BEFORE ROUNDTABLE 2? I'LL TURN IT OVER TO WENDY AND CRAIG. >> I WOULD LIKE TO ADD THANK YOU FOR VERY RICH DISCUSSION, VERY DIFFICULT QUESTION AND VERY COMPLEX BUT I THINK WE HAVE THE GREAT QUESTION FROM THE AUDIENCE AS WELL AS THE PEOPLE. THANK YOU TO SPEAKERS GIVING GREAT DISCUSSION. THANK YOU SO MUCH. >> THANK YOU. NOW WE'LL MOVE TO ROUNDTABLE 2, I'LL GO AHEAD AND SHARE SOME SLIDES ON BEHALF OF MODERATORS. I'M HAPPY TO INTRODUCE DR. CRAIG HOPP, DEPUTY DIRECTY, AND DR. WENDY WEBER, A BRANCH CHIEF WITHIN OUR DIVISION OF EXTRAMURAL RESEARCH, LEADS BRANCH ON CLINICAL RESEARCH AND COMPLEMENTARY AND INTEGRATIVE HEALTH. SO I'M GOING TO TURN THINGS OVER TO DR. HOPP AND DR. WEBER TO OUTLINE THEIR KEY ISSUES FOR ROUNDTABLE 2. >> WE WILL TAG TEAM. I HAVE SO MANY QUESTIONS, IN ADDITION TO THE ONES WE HAVE A PRIORI LISTED HERE. IF YOU WERE ON THE MEETING YESTERDAY THESE WILL LOOK FAMILIAR BECAUSE THEY ARE VERY REMINISCENT OF THE QUESTIONS WE HAD FROM ROUNDTABLE 1. FOR THIS ROUNDTABLE WE'RE THINKING ABOUT TALKS WE HEARD TOGETHER, APPLYING THESE QUESTIONS TO THE ISSUES OF MULTI-COMPONENT INTERVENTIONS IMPACTING ON SINGLE OR INTERCONNECTED MUST BE SYSTEMS. I THINK WE'RE GOING TO HAVE A COUPLE OF PANELISTS THAT ARE GOING TO BE SPEAKING FIRST, I WILL START BY INTRODUCING DR. SCOTT MIST, ASSOCIATE PROFESSOR AT OREGON HEALTH AND SCIENCE UNIVERSITY, ALSO A LICENSED ACUPUNCTURIST, PAIN RESEARCHER, AND SYSTEMS SCIENTIST. I WILL TURN IT OVER TO DR. MIST. FEEL FREE TO SHARE YOUR SLIDES AND GIVE YOUR BRIEF PRESENTATION. >> FANTASTIC. THANK YOU. LET ME GET THIS GOING HERE. THANK YOU FOR INVITING ME. I HAVE ENJOYED LISTENING TO THIS AND A NUMBER OF THE FOLKS HAVE TEED UP FOR -- TEED ME UP FOR MY DISCUSSION HERE. JUST AS A BACKGROUND I'M IN OHSU, OREGON HEALTH AND SCIENCE UNIVERSITY, IN THE ANESTHESIOLOGY DEPARTMENT. AND I'VE GOT A BACKGROUND IN SYSTEM SCIENCES. AND THIS IS GOING TO BE MY PLUG FOR SYSTEM SCIENCES. YOU MAY NOT BE ABLE TO SEE ALL OF THESE THINGS BUT IT'S KIND OF A LIST OF THE DIFFERENT THINGS THAT -- DIFFERENT TOOLS WITHIN SYSTEMS SCIENCE FOR MODELING COMPLEX SYSTEMS. SYSTEM SCIENCE IS ABOUT BOTH METHODOLOGY DESIGN ANALYSIS OF COMPLEX SYSTEMS AND THE WAY WE THINK OF COMPLEXITY IS THAT MOST GENERALLY IT'S A SYSTEM THAT HAS A FEEDBACK LOOP IN IT. ALL THROUGHOUT THIS DISCUSSION WE'VE BEEN TALKING ABOUT HOW YOU DEAL WITH FEEDBACK LOOPS, HOW YOU MAKE AN INTERVENTION, AND THAT CHANGES SOME ASPECT WHICH THEN FEEDS BACK INTO THE ORIGINAL SYSTEM, AND YOU HAVE THESE COMPOUND RELATIONSHIPS GOING ON. THE DIFFICULTY IS THAT ONCE YOU START ADDING IN MORE THAN ONE FEEDBACK LOOP, YOU CAN GET REALLY COMPLEX BEHAVIORS. AND AT THAT POINT WE DON'T HAVE GOOD METHODOLOGIES IN OUR STANDARD STATISTICS TO TEASE OUT THESE FEEDBACK LOOPS. I WISH I HAD DR. HAMMOND'S OBESITY SLIDE BECAUSE THAT WAS A GREAT EXAMPLE OF A SYSTEM WITH MULTIPLE FEEDBACKS AND MAKING A CHANGE IN ONE MAY OR MAY NOT MAKE A CHANGE IN THE OVERALL SYSTEM, SYSTEM BEING THE INDIVIDUAL. ONE OF THE THINGS I WANT TO PULL FROM THE SYSTEMS SCIENCE FIELD THAT HASN'T BEEN TALKED ABOUT HERE IS RECONSTRUCTABILITIABLE SIS, DISCRETE MULTIVARIATE MODELING, A METHODOLOGY THAT IS A MORE GENERAL EXTENSION OF THE LOG LINEAR METHODS. ABLE TO HANDLE SET THEORETIC MODELS AND INFORMATION MODELS SUCH AS WE'VE BEEN TALKING ABOUT. IT'S ABLE TO HANDLE DIRECTED AND NEUTRAL SYSTEMS. MOSTLY WE'VE BEEN TALKING ABOUT DIRECTED SYSTEMS WITH INDEPENDENT AND DEPENDENT VARIABLES, ABLE TO HANDLE TIME DEPENDENT SYSTEMS. WHERE IT OVERLAPS WITH LOG LINEAR METHODS, IT IS 100% OVERLAP. IT'S THE EXACT SAME SYSTEM. THE UNIQUENESS THAT I THINK IS REALLY APPROPRIATE FOR THIS GROUP IS THAT THE SYSTEMS THAT IT CONSIDERS IS LARGER, A LOT LARGER SET OF MODELS THAN WE NORMALLY CONSIDER. WHEN LOOKING AT JUST THREE SETS OF VARIABLES, YOU CAN HAVE A TOTAL OF NINE SPECIFIC STRUCTURES WITHIN THE LOG LINEAR METHODS, AND WHEN YOU GET UP TO SIX, WHICH IS A PRETTY COMMON ONE, YOU GET UP TO OVER SEVEN MILLION STRUCTURES THAT YOU CAN ARRANGE THOSE IN. AND WHEN WE THINK ABOUT SIX VARIABLES WITHIN MOST OF STATISTICS, THE NUMBER EVER MODELS THAT WE'RE CONSIDERING ARE ACTUALLY A GOOD DEAL SMALLER, WE THINK OF INTERACTIONS OF A SMALL GROUP OF THESE VARIABLES, AND OFTENTIMES THE MODELS THAT WE'RE INVESTIGATING ARE IN THE FEW HUNDREDS OR OFTENTIMES IN THE TENS OF SYSTEMS. USING THE DISCRETE MULTIVARIATE MODELING, YOU HAVE THE OPPORTUNITY TO EXAMINE A NUMBER OF NON-LINEAR SYSTEMS, AND THIS METHODOLOGY GIVES YOU A GOOD VISUALIZATION OF WHAT THAT NON-LINEAR SYSTEM COULD BE. GOING TO DUAL DIAGNOSIS, STUDIES DID NOT INCLUDE DIAGNOSTIC SYSTEM OF THE WHOLE SYSTEM. I'M GOING TO TALK ABOUT ACUPUNCTURE BECAUSE THAT'S MY FIELD BUT IT APPLIES TO ALL THE OTHERS AS WELL. WHEN YOU DON'T INCLUDE THE DIAGNOSTIC SYSTEM OF THE WHOLE SYSTEM, THE RESULTS ARE UNINTERPRETABLE TO THE FIELD THAT YOU'RE STUDYING. SAY, CHINESE MEDICINE. SO WHEN YOU DO THIS DUAL DIAGNOSTIC PROCESS YOU WANT TO MAKE SURE YOU DO THE DIAGNOSIS FROM YOUR ALLOPATHIC WORLD, AS WELL AS THE WHOLE SYSTEMS. WE DID SOME FIBROMYALGIA RESEARCH WHERE WE USED THE FIBROMYALGIA CRITERIA TO DIAGNOSE A GROUP OF FOLKS WITH FIBROMYALGIA AND USED TRADITIONAL CHINESE MEDICINE DIAGNOSIS TO GUIDE OUR TREATMENTS. SO THAT THE TREATMENTS MATCHED WHAT THE CHINESE MEDICINE SIDE OF THINGS SAID AND THE RESULTS WERE INTERPRETABLE FROM A WESTERN MEDICINE POINT OF VIEW. YOU CAN DO THIS WITH A SINGLE DIAGNOSIS FROM BOTH SYSTEMS OR YOU CAN DO IT WITH MULTIPLE DIAGNOSES FROM THE WHOLE SYSTEM. SO LIKE WITH THE FIBROMYALGIA STUDY WE USED SIX DIFFERENT DIAGNOSES, WITH DIFFERENT TREATMENTS, TRYING TO ANSWER THE QUESTION IF YOU WENT TO A CHINESE MEDICINE DOCTOR WOULD THE FIBROMYALGIA CHANGE OR YOU CAN NARROW DOWN TO A SINGLE ONE AND GET REALLY SPECIFIC WITHIN THAT AREA AND THAT ALLOWS YOU TO USE A SINGLE TREATMENT PROTOCOL AND HAVE IT BE VALID IN BOTH. YOU CAN DO THIS FROM BOTH DIRECTIONS. THERE'S SOME EXCITING WORK THAT'S HAPPENING IN KOREA RIGHT NOW WHERE THEY ARE USING CHINESE MEDICINE DIAGNOSES TO SELECT ALLOPATHIC TREATMENTS IN THE FIELDS OF DEPRESSION AND IN SOME OF THE HEART MEDICATIONS. THEY ARE FINDING WHEN YOU DO THE TRADITIONAL CHINESE MEDICINE DIAGNOSIS, YOU'RE ABLE TO MAKE THE ALLOPATHIC MEDICINE MORE EFFECTIVE BY GIVING THE RIGHT PHARMACEUTICAL TO THE RIGHT PATIENT. AND AS I SAID THIS MAKES IT MORE RELEVANT TO BOTH FIELDS. THE PROBLEM WITH THIS METHODOLOGY IS THAT IT PLACES A GREATER IMPORTANCE ON RELIABILITY OF THE DIAGNOSIS. AND AS WE ALL KNOW IN ALL OF MEDICINE, DIAGNOSIS HAS A LOT OF VARIABILITY. A LOT OF WORK HAS TO GO INTO THAT SIDE OF THINGS. >> I THINK WE'RE READY FOR YOU TO WRAP UP, DR. MIST. >> I'M DONE. >> WONDERFUL. >> THANK YOU SO MUCH. >> I'LL TURN IT OVER TO MY COLLEAGUE TO INTRODUCE THE NEXT PANELIST. >> GREAT. I HAVE ONE SLIDE TO GO OVER QUICKLY BEFORE I INTRODUCE OUR NEXT PANELIST. GOOD AFTERNOON. THANK YOU FOR STICKING WITH US. WE'RE LOOKING FORWARD TO OUR NEXT DISCUSSION. I WANTED TO REINTRODUCE HERE AGAIN THE NATIONAL INSTITUTE ON MINORITY HEALTH AND HEALTH DISPARITIES RESEARCH FRAMEWORK. THIS ISSUE OF SOCIAL DETERMINANTS EVER HEALTH -- OF HEALTH HAS COME UP. IT'S RELEVANT TO THE NEXT SPEAKER, SOMETHING WE WANT TO TALK ABOUT IS AS WE'RE DOING THESE STUDIES AND AS WE'RE LOOKING AT THIS KIND OF RESEARCH THERE ARE MULTIPLE LEVELS OF INFLUENCE AS WELL AS DIFFERENT DOMAINS OF INFLUENCE THAT ARE IMPORTANT TO LOOK AT IN WE'RE TRYING TO LOOK REALLY AT THE WHOLE PERSON. AND SO I'LL GO AHEAD NOW AND WE CAN SKIP TO THE NEXT SLIDE AS I INTRODUCE OUR NEXT PANELIST, DR. IRENE HEADEN, ASSISTANT PROFESSOR OF BLACK HEALTH IN DEPARTMENT OF COMMUNITY HEALTH AND PREVENTION, DREXEL UNIVERSITY. INTERESTED IN SOCIAL AND STRUCTURAL DETERMINANTS OF RACIAL AND ETHNIC INEQUALITIES IN ADVERSE PREGNANCY OUTCOMES AND RESEARCH FOCUSES ON IDENTIFYING NEIGHBORHOOD AND COMMUNITIY FACTORS TO IMPROVE BLACK MATERNAL OUTCOMES. WELCOME. >> EVERY TIME YOU PRACTICE, IT DOESN'T MATTER. ALL RIGHT. IS EVERYONE SEEING MY SLIDES AND CAN HEAR ME? >> YES. >> ALL RIGHT. AS WENDY MENTIONED I'M PICKING UP A THREAD FROM THE CONVERSATION WE STARTED YESTERDAY IN ROUND TABLE 1 AND COMING UP OVER DAY 1 AND DAY 2, THE METHODOLOGICAL CONSIDERATIONS INCORPORATING SOCIAL AND STRUCTURAL DETERMINANTS OF HEALTH INTO WHOLE PERSON RESEARCH. FIRST I WANT TO SAY THE DRIVE TO GENERATE ACTIONABLE EVIDENCE ON THE SOCIAL DETERMINANTS OF HEALTH HAS STEADILY EVOLVED. TO NOW IT'S LIKE COMMONPLACE TO UNDERSTAND THESE FACTORS AFFECT HEALTH. THE W.H.O. DEFINES THEM CONDITIONS PEOPLE ARE BORN, WORK, GROW, LIVE, AGE, WIDER SET OF FORCES AND SYSTEMS SHAPING CONDITIONS OF LIFE. ON THE RIGHT YOU CAN SEE A FEW DOMAINS ACROSS WHICH YOU SEE MEASURES TO OPERATIONALIZE THIS CONSTRUCT. AS ACCELERATION OF RESEARCH HAS EVOLVED WE'VE RETURNED TO A POINT IN THE TRAJECTORY WHERE WE THERE'S BEEN DISTINCTIONS ABOUT WHAT IS AND IS NOT STRUCTURAL DETERMINANTS OF HEALTH. I PUT A SEED FOR FOLLOW-UP, AN ARTICLE WHERE THEY ARE DELVING INTO THE DISTINCTION BETWEEN SOCIAL DETERMINANT OF SET, NEEDS FOCUSED INTERVENTION AND RISK FACTOR AND HOW CAN DISTINCTIONS BETTER IMPROVE RIGOR AND VALIDITY IN THE AREA. IN THIS VEIN THERE HAVE BEEN DISCUSSIONS ABOUT HOW FURTHER DISTINCTIONS ABOUT THE STRUCTURAL DETERMINANTS OF HEALTH WITHIN OR IN RELATION TO SOCIAL DETERMINANTS OF HEALTH CAN BE A KEY ELEMENT FOR ADVANCING HEALTH EQUITY RESEARCH. STRUCTURAL DETERMINANTS OF HEALTH ARE DEFINED AS INCLUDING POLICIES, PRACTICES, CULTURAL NORMS AND INSTITUTIONS THAT DEFINE DISTRIBUTION OF SOCIAL FACTORS AS WAS DEFINED IN THE PAPER THAT YOU CAN SEE CITED ON THE RIGHT. AND WHERE THIS BECOMES PARTICULARLY RELEVANT IN DISTINGUISHING SOCIAL VERSUS STRUCTURAL IS AS WE KNOW THERE'S CERTAIN IDENTITY STRUCTURES THAT ARE -- TYPES FOR WHICH STRUCTURES CONSTRAIN OR EXPAND THE ACCESS TO SOCIAL FACTORS. IMPORTANTLY, IT'S CAPTURED IN THE THREAD, AS A PUBLIC HEALTH AND SOCIETAL CRISIS TO BE ADDRESSED ACROSS HEALTH INEQUITIES INTERVENTIONS. SOME OF THE KEY ELEMENTS IN TERMS OF TAKING THIS DISTINCTION AND DEFINITION INTO OPERATIONALIZING HOW WE LOOK AT THOSE IN AND ACROSS STUDIES HAS A FEW KEY CONSIDERATIONS THAT HAVE DIFFERENT WEIGHTS, RETURNING TO HUERISTICS THAT WENDY BROUGHT UP AT THE BEGINNING. ORGANIZED AROUND THESE CONSIDERATIONS OF LEVEL OF INFLUENCE, WHICH YOU CAN SEE CAN BE INDIVIDUAL TO POPULATION HEALTH, TIME WHICH OF COURSE EVOLVED IN DIFFERENT WAYS OF SOCIAL FACTORS BEING EMBODIED, AND TAKING TO STRUCTURAL LEVEL TIME CAN DEFINE TRAJECTORY WHERE PROCESSES BECOME REINFORCEABLE AND INDEPENDENTLY REINFORCING OR CAN BE KIND OF INTERRUPTIBLE OR MODIFIABLE AT DIFFERENT TIME SCALES. THERE'S THE ISSUE OF MULTI-DIMENSIONALITY, LOOKING ACROSS DOMAINS OF INFLUENCE AND DIMENSIONS THAT CAN BE OPERATIONALIZED IN TERMS OF EDUCATION, UNDERSTANDING SOCIAL DETERMINANTS, HOUSING SYSTEM, NEIGHBORHOOD ENVIRONMENTS AND INTERCONNECTEDNESS. THE FACT THESE SOCIAL AND STRUCTURAL DETERMINANTS DON'T OPERATE IN A VACUUM OR IN INDEPENDENT FORMS. SO IN THE NEXT SLIDE I'M GOING TO TALK ABOUT -- I'VE CHOSEN EXAMPLES HOW DO I OPERATIONALIZE TO IMPLEMENT SOCIAL AND STRUCTURAL DETERMINANTS IN HEALTH DISPARITIES RESEARCH, LOOKING ACROSS THE RESEARCH PROCESS, FROM STUDY DESIGN EMBEDDING MY WORK IN LONGITUDINAL COHORT STUDIES SO WE CAN SEE RATHER THAN PICKING A POINT IN TIME TO UNDERSTAND HOW NEIGHBORHOOD DEPRIVATION, FOR EXAMPLE, IS EMBEDDED TO HOW IT ACCUMULATES WITHIN INDIVIDUALS AND POPULATIONS AT DIFFERENT SOCIAL IDENTITIES OVER TIME TO AFFECT HEALTH OUTCOMES. MY WORK LOOKING AT NEIGHBORHOOD FACTORS, FIRST THIS LOOK AT INTEGRATED DATABASES THAT AGGREGATE ACROSS MULTIPLE DIMENSIONS OF THE NEIGHBORHOOD ENVIRONMENT. PREVIOUSLY WAS MENTIONED NEIGHBORHOOD ADVERSITY INDEX, IN ADDITION THERE'S MEASURES OF SERVICE ENVIRONMENT, FOOD ENVIRONMENT THAT CAN ALL BE LAYERED TO ACTUALLY CREATE A MORE COMPREHENSIVE TYPOLOGY OF NEIGHBORHOOD CONTEXT. I IN THIS RESPECT APPLIED A LATENT CLASS ANALYSIS ONE OF THE CLUSTERING TECHNIQUES THAT WE'VE SEEN USED ACROSS DIFFERENT PARTS OF THE SYSTEM LOOKING IT'S A WHOLE PERSON RESEARCH TO ACTUALLY GO FURTHER AND IDENTIFY TYPOLOGIES OF NEIGHBORHOOD, IF WE START AT A CERTAIN NEIGHBORHOOD DEPRIVATION INDEX LEVEL ARE THERE QUALITIES THAT BECOME RELEVANT WHEN WE LOOK AT EFFECTIVENESS OF A MINDFULNESS INTERVENTION IN IMPROVING METABOLIC HEALTH IN PREGNANCY. FINALLY BECAUSE OF THIS MULTI-DIMENSIONALITY ASPECT A LOT OF TOOLS USED IN MULTI-LEVEL MODELS HAVE BEEN LOOKING AT SYSTEM SCIENCE METHODS WHICH MY COLLEAGUE SAID IN THE PREVIOUS PRESENTATION AND MENTIONED IN SESSION 4 TO UNDERSTAND INTERCONNECTEDNESS AND DYNAMIC NATURE OF HOW NEIGHBORHOODS REPRODUCE INEQUITIES FOR POPULATIONS WITHIN AND MOVING THROUGH AS HEALTH EVOLVES OVER TIME. I THINK I WANTS TO PICK UP A THREAD THAT HAS BEEN MENTIONED ACROSS DIFFERENT PRESENTATIONS BUT FROM MY WORK IN OPERATIONALIZING THE SOCIAL AND STRUCTURAL DETERMINANTS OF HEALTH PARTICULARLY IMPORTANT, ASPECT OF HAVING METHODS AND DEVELOPING METHODS TO EXAMINE RESEARCH PROCESS ITSELF. THESE ARE INTEGRAL TO MAINTAINING RIGOROUS SCIENCE IN THIS AND OTHER ASPECTS OF RESEARCH BUT I THINK THIS IS CAPTURED IN THIS QUOTE, THE FACT THAT DATA NEVER SPEAK FOR THEMSELVES, IT IS THE QUESTIONS WE POSE AND THESE THAT WE FAIL TO ASK AS WELL AS THEORIES, CONCEPTS, IDEAS THAT BRING A NARRATIVE AND MEANING TO MARGINAL DISTRIBUTIONS, CORRELATIONS, REGRESSION COEFFICIENTS AND STATISTICS OF ALL KINDS. IN THIS CASE HE WAS TALKING ABOUT MARGINAL DISTRIBUTION IN THE STATISTICAL SENSE BUT WHEN WE TRANSLATE TO MARGINALIZED COMMUNITIES THE EXTENT, ABILITY TO HAVE SELF-AWARENESS ABOUT SOCIETAL CONTEXTS SHAPING SCIENCE, IMPLICATIONS ARE GREATER FOR HOW WE STUDY AND HOW THAT'S DISTRIBUTED TO NOT REPRODUCE SOCIAL DISTRIBUTIONS OF HEALTH THAT WE'RE SO HARD WORKING TO TRY TO DISMANTLE. WITH THAT I WILL SAY THANK YOU. THERE'S MANY COLLABORATORS AND FUNDING SOURCES THAT HAVE MADE MY BODY OF WORK POSSIBLE AND I WILL SEND AND HOPEFULLY OPEN UP TO A REALLY ROBUST PLATFORM FOR CONVERSATION. >> WONDERFUL. THANK YOU SO MUCH, DR. HEADEN. I'LL KICK OFF WITH THE FIRST TWO QUESTIONS AND OPEN THIS CONVERSATION UP TO PANELISTS BUT ALSO TO REALLY ALL OF OUR SPEAKERS OVER THE LAST TWO DAYS. PLEASE RAISE YOUR HAND TO JOIN THE CONVERSATION RELATED TO THE QUESTIONS. YOU TEED US UP WELL, IN TERMS OF THIS FIRST TWO QUESTIONS THAT WE HAVE HERE. YOU KNOW, I THINK, YOU KNOW, WE SO OFTEN FOCUS ON THE COMPONENTS OF THE INTERVENTION AND IF WE'RE NOT LOOKING AT MULTI-LEVEL INTERVENTIONS WE OFTEN DON'T TAKE INTO CONSIDERATION SORT OF THOSE SOCIAL DETERMINANTS OF HEALTH. AND SO WHAT MAYBE ADVICE DO YOU HAVE OR, YOU KNOW, I'LL TURN IT TO YOU TO KICK THIS OFF. HOW DO WE NEED TO THINK ABOUT THIS IF WE'RE THINKING ABOUT MULTI-COMPONENT INTERVENTIONS, AND WHAT MAY BE MULTI-CONNECTED OUTCOMES THAT WE'RE MISSING IF WE'RE, FOR EXAMPLE, APPROXIMATING A HOST OF SOCIAL DETERMINANTS OF HEALTH WITH THE SINGLE VARIABLE LIKE RACE OR ETHNICITY, WHICH IS WHAT OFTEN UNFORTUNATELY HAPPENS. >> THERE HAVE BEEN THREADS THAT HIGHLIGHT IMPORTANCE OF INCLUDING SOCIAL -- I WOULD ARGUE NEW -- NOT NEW, RETURNING TO THE EMPHASIZE ELEMENTS OF STRUCTURAL DETERMINANTS OF HEALTH, YOU CAN DO ALL THE RESEARCH FROM MOLECULAR TO BIOLOGICAL TO CLINICAL, BUT IF THAT DOESN'T ACTUALLY TRANSLATE TO THE POPULATIONS AGAIN WHICH HAVE A DISPROPORTIONATE IMPACT WE'RE MISSING THE POINT OF THE WHOLE PERSON RESEARCH, TAKING THE WHOLE PEOPLE OUT OF THE POPULATION THAT THEY ARE EMBEDDED IN, WE TRY TO TRANSLATE WE END UP WITH THESE GAPS AGAIN THAT PERPETUATE, WE CAN HAVE THE MOST WELL OPTIMIZED SYSTEM BUT WHEN IT DOESN'T TRANSLATE TO ACTUAL REAL WORLD CONTEXT I THINK THAT WAS ONE OF THE THEMES HAVING REAL WORLD DATA, THEN WE ACTUALLY ARE KIND OF NOT FULLY LEVERAGING THE PROMISE OF WHAT IT MEANS TO FULLY PUSH FORWARD A WHOLE PERSON RESEARCH AGENDA. I THINK IN TERMS OF -- THERE'S BEEN A LOT OF TOOLS OVER THE COURSE OF THE LAST FEW DAYS THAT ARE ACTUALLY -- I'M VERY EXCITED IN TERMS OF HOW THEY INTERACT CAN OPERATIONALIZING THE SOCIAL AND STRUCTURAL DETERMINANTS OF HEALTH. I WAS VERY CAPTIVATED BY DR. COLLINS, THE MOST FRAMEWORK. AGAIN, THAT COULD BE AN OPTIMIZATION CRITERIA, WHERE IT'S LIKE YOU'RE IN A CERTAIN POPULATION, YOU KNOW EITHER THROUGH ENGAGING WITH THAT COMMUNITY, HISTORICAL PROCESSES THAT YOU'RE AWARE OF, THAT CERTAIN STRUCTURAL BARRIERS ARE GOING TO BE THERE. YOU CAN SAY HOW DO WE OPTIMIZE NOT ONLY WHAT IS DONE BUT HOW THIS CAN BE SCALABLE WITHIN THE CONTEXT OF THE STRUCTURAL DETERMINANTS AND HISTORY OF STRUCTURAL RACISM, SEXISM, OTHER TIMES OF THOSE IS STRUCTURAL CONSTRAINTS. BEFORE OPENING TO OTHER PANELISTS THERE'S A WEALTH OF KNOWLEDGE IN THIS ROOM I'M SURE, ONE IMPORTANT COMPONENT IS THAT OUR OWN -- THAT LAST POINT, POSITIONALITY, THE EXTENT TO HOW DOES THAT TRANSLATE TO WHAT WE DO AND DON'T KNOW ABOUT THE COMMUNITIES, WE'RE WORKING IN AND WITH, HOPEFULLY MORE WITH RATHER THAN IN. HOW DOES THAT TRANSLATE TO PRACTICALLY OPERATIONALIZING SOCIAL DETERMINANTS OF HEALTH, WE HAVE TO WORK WITH COMMUNITIES, THE ONES LIVING AND MOVING THROUGH AND CREATING - -- CREATIVE RESPONSES OVER TIME SO WE HAVE TO, WORK WITHIN COMMUNITIES. THAT GOES TO THAT QUESTION YOU'RE ASKING ABOUT HOW DO WE EXPAND BEYOND JUST RACE AND ETHNICITY AS KIND OF THAT ONE DETERMINE NATURE OF -- SOCIAL DETERMINANT OF HEALTH WHICH I HOPE THAT'S NOT WHAT PEOPLE ARE DOING BECAUSE WHEN YOU GO DEEPER THERE ARE OTHER INTERSECTIONAL WAYS IN WHICH STRUCTURAL FACTORS ARE CONSTRAINING AGENCY OF POPULATION ACROSS THIS CONTINUUM SO I THINK WE NEED TO GO TO COMMUNITIES AND SAY WHAT ARE STRUCTURAL DIMENSIONS, DYNAMICS WHICH COMMUNITY POPULATIONS ARE EMBEDDED SO WE CAN MORE ROBUSTLY CREATE SOCIAL AND STRUCTURAL DYNAMICS, MEASURES, EXCUSE ME, THAT WILL BE EFFECTIVE IN IMPLEMENTATION. >> THANK YOU. WE'VE GOT A FEW HANDS RAISED. DR. WINDSOR? >> THIS IS AN EXCITING QUESTION. ONE THING I DIDN'T SHARE FOR THE CRITICAL CONSCIOUSNESS, DIALOGUE COMPONENT OF THE INTERVENTION, PRECISELY DESIGNED TO ADDRESS RACISM, CLASS ITCH, SEXISM AS DIRECT INFLUENCERS, IF YOU WILL, OF THE OUTCOMES WE WANT TO CHANGE WHICH INCLUDES SUBSTANCE MISUSE, RELATIONSHIPS WITH OTHERS, AND THE COMMUNITY. THE OTHER COMPONENT THAT WE HAVE IN THE INTERVENTION IS CAPACITY TO DO PROJECT WHICH IS DESIGNED TO UTILIZE TO ENGAGE PARTICIPANTS IN COMMUNITY ORGANIZING AND COMMUNITY ACTIVITIES THAT THEY IDENTIFY IN THE COMMUNITIES. METHODOLOGICAL LIMITATION WHEN WE'RE OPTIMIZING FOR SUBSTANCE USE, MISUSE, DON'T KNOW IF WE OPTIMIZE FOR CAPACITY BUILDING PROJECT, OUTCOME AT THE COMMUNITY LEVEL, WE WOULD END UP WITH THE SAME OPTIMIZED INTERVENTION, ANOTHER OPTIMIZATION STUDY FOR THAT PARTICULAR PROCESS BUT FACTORIAL DESIGN ALLOWS YOU TO SEPARATE COMPONENTS THAT GET NOT JUST TO INDIVIDUAL LEVEL, FACTORS THAT AFFECT HEALTH BUT ALSO COMMUNITY AND STRUCTURAL LEVELS THAT IMPACT HEALTH. >> DR. MIST? >> YES. I WANT TO ADD IN I IT WAS HOOD ET AL. IN 2016 THAT SAID MEDICINE CAN CHANGE 20% OF CHANGEABLE OUTCOMES FOR A PERSON. WE REALLY NEED TO BE THINKING MULTI-STRUCTURAL RESEARCH IN BRINGING ALL THESE OTHER THINGS IN BECAUSE WE'RE NOT EVEN AFFECTING THE LARGEST PERCENTAGE OF WHAT CAN BE CHANGED. IT'S NOT IN MEDICINE. WE NEED TO FIND WAYS OF BRINGING MEDICINE AND STRUCTURE AND THIS DISCUSSION OF COMMUNITIES INTO OUR RESEARCH BECAUSE THAT'S WHERE WE'RE GOING TO CHANGE THE MOST LIVES. >> I'LL TURN THINGS OVER TO CRAIG TO GET US TO TALK ABOUT GAPS AND OPPORTUNITIES BUT WE'LL COME BACK TO POTENTIALLY ADDITIONAL QUESTIONS REGARDING THINGS WE NEED TO MINE, DATA COLLECTION, BIAS. OVER TO CRAIG. >> YEAH, I WOULD SUMMARIZE THESE QUESTIONS AS WHAT CAN WE DO, WHAT CAN'T WE DO, HOW DO WE BRIDGE THE GAP? AND THERE'S LOTS OF DISCUSSION ABOUT THE DIFFERENT METHODOLOGIES AND WHAT THEY ARE USED FOR, WHAT I HEARD A LOT WAS THAT YOU HAVE TO ENGAGE IN TRADEOFFS, YOU CAN OPTIMIZE USING THIS METHOD BUT IT WON'T WORK FOR THE OTHER THING, WITH THAT METHOD. I'M ASKING FOR ANY SPEAKERS OVER THE LAST TWO DAYS, WHAT METHODOLOGIES CAN WE APPLY, WHAT GAPS DO WE HAVE? AND A THIRD QUESTION, MOST INTERESTED IN, I'LL ASK THAT FIRST. WHERE DO WE NEED MORE? WHERE ARE THERE FIELDS IN TECHNOLOGIES, INTERSECTIONS WITH THE INHIBITIONS WE TALKED ABOUT IN THE LAST TWO DAYS WHERE THEY ARE RIPE FOR INNOVATION? I'LL START WITH THAT QUESTION AND ASK IF ANY OF THE SPEAKERS HAVE ANY THOUGHTS, BASED ON WHAT WE'VE HEARD OVER THE LAST TWO DAYS, WHERE CAN A LITTLE BIT OF INVESTMENT FROM NCCIH HELP TO MOVE THE NEEDLE? DO I HAVE TO CALL ON PEOPLE? >> I THINK TRIAL DESIGN FOR EXAMPLE IS ONE AREA WITH A NEED FOR INNOVATION. I THINK IN TERMS OF CROSSOVER WITH OTHER FIELDS, THIS WAS NOT SOMETHING DISCUSSED OVER THE LAST TWO DAYS BUT COULD HAVE BEEN A TOPIC. THERE ARE KIND OF WAYS OF TESTING PLATFORM INTERVENTIONS AND WHAT I MEAN BY THAT IS A LOT OF THE DISCUSSION OVER THE LAST COUPLE DAYS HAS BEEN AMONG OTHER THINGS TO FIND A SET OF DISTINGUISHING FEATURES FOR PATIENTS THAT MIGHT BENEFIT FROM ONE TREATMENT OR INTERVENTION TYPE VERSUS ANOTHER. SO IF ONE HAD PROFILES OF A GROUP OF PATIENTS AND YOU COULD LUMP THEM INTO CATEGORIES, YOU'RE LIKELY TO RESPOND TO THIS TREATMENT VERSUS THAT ONE, THOSE SORTS OF TRIALS ARE BEING PURSUED IN, SAY, CANCER CONTEXTS ALL THE TIME UNDER THE KIND OF GUISE OF BUCKET AND BASKET AND UMBRELLA TRIALS. AND THESE SORTS OF TRIALS MIGHT HAVE UTILITY IF THERE ARE MULTIPLE INTERVENTIONS THAT MIGHT BE MORE OR LESS APPROPRIATE FOR ONE POPULATION OR ANOTHER. SO NOT ONLY INNOVATION IN THAT SPACE WOULD BE IMPORTANT TO KIND OF HELP DEVELOP AND NURTURE BUT ALSO JUST BRINGING THOSE SORTS OF IDEAS TO THE TABLE, IF PEOPLE STUDYING COMPLEMENTARY MEDICINES AREN'T AWARE OF THOSE SORTS OF TRIAL DESIGNS. I CAN BE WAY MORE SPECIFIC BUT I'LL LET OTHERS CHIME IN >> DR. MIST? >> I TOTALLY AGREE ABOUT STUDY DESIGN. I THINK THAT'S AN IMPORTANT AREA OF INNOVATION THAT, YOU KNOW, AS YOU LOOK THROUGH THE STUDY DESIGNS THEY ARE SLOWLY EVOLVING BUT IT'S BEEN SLOW. I ALSO THINK THAT METHODOLOGICALLY WE -- IF YOU TAKE A LOOK AT THE NUMBER OF POSSIBLE MODELS THAT ARE OUT THERE, LINEAR MODELS IS SUCH A SMALL GROUP OF THEM, AND YET MOST OF THE STATS I READ, MOST OF THE DESIGNS WE LOOK AT ARE MAKING ASSUMPTIONS OF LINEARITY. WE NEED TO FOCUS ON HOW DO WE DESIGN OUR STUDIES AROUND NON-LINEARITY, HOW DO WE THINK ABOUT GROUPING THAT IS IN A NON-LINEAR FASHION OR A FUZZY FASHION, WHERE SOMEONE CAN HAVE A PARTIAL GROUP MEMBERSHIP, AND, YOU KNOW, THINKING IN TERMS OF SIMPLE BEHAVIORS CAUSING COMPLEX OUTCOMES. A LOT OF THAT MODELING WE WERE TALKING ABOUT, I THINK THOSE ARE ALL AREAS OF GROWTH THAT WE COULD BE MOVING TOWARDS. >> DR. HEADEN, YOUR SQUARE MOVED. DO YOU HAVE A COMMENT TO MAKE? >> IT'S BLENDING INTO THE COLOR SCHEME. ONE OF THE THINGS I HEEDED OVER THE LAST FEW DAYS THAT I GET THIS QUESTION OF INNOVATION OF METHODS LEARNING FROM DIFFERENT KIND OF DISCIPLINES AND ALL OF THAT REALLY WHAT I THINK WHERE THE INNOVATION COULD BE LEVERAGED CREATING INFRASTRUCTURES THAT ROUTINELY ALLOW US TO BRIDGE ACROSS DIFFERENT DISCIPLINES. I LEARNED PSYCHOLOGY AND INTEGRATING THAT CLASS ANALYSIS, NOW TAKING INITIATIVES IN TERMS OF PULLING INTO COMPLEX SCIENCE WORK BUT THAT'S ALL LIKE TANGENTIAL TO MY ACADEMIC HOME, EPIDEMIOLOGY. I LOVE IT BUT IT'S LINEAR. WE LOVE OUR LINEAR MODELS AND ALL OF THAT. AND, YOU KNOW, I THINK ARE THERE WAYS FROM AN EARLY CAREER STANDPOINT WE CAN ENCOURAGE THIS INNOVATION AROUND CROSS-DISCIPLINARY METHODOLOGICAL APPLICATIONS IN THE LAST FEW DAYS SOME THINGS APPLIED TO MOLECULAR, N-of-1, QUESTIONS THAT CONTINUE TO RATTLE IN MY BRAIN IN TERMS OF BUILDING RESEARCH AGENDA AROUND SOCIAL DETERMINANTS OF HEALTH, CONTINUUM OF WHOLE PERSON RESEARCH BUT ALSO FUNDING STRUCTURES, THE WHOLE POINT OF THESE LAST FEW DAYS THAT CAN SUPPORT THESE CROSS-DISCIPLINARY APPLICATIONS AND METHODOLOGICAL INTERVENTIONS, METHODOLOGICAL UNDERSTANDINGS AND KNOWLEDGE, THAT'S WHERE INNOVATION CAN BE LEVERAGED. >> THANKS. DR. WINDSOR, LOVE TO HEAR YOUR PERSPECTIVES ON THIS. >> I'M GOING TO BUILD ON WHAT HAS BEEN SAID. I AGREE. TO TALK MOTHER ABOUT -- MORE ABOUT STRUCTURAL CHANGES, INVESTMENT IN PERHAPS FUNDING FOR METHODOLOGISTS TO TRY TO DEVELOP SOLUTIONS TO SOME CHALLENGES THAT HAVE BEEN POSED HERE WOULD BE AN IMPORTANT STRATEGY. CREATING MORE SPACES WHERE WE CAN CONTINUE OVER THE PAST COUPLE DAYS IS REALLY IMPORTANT AND CAN GO A LONG WAY. I THINK THAT IT'S REALLY IMPORTANT TO TRY TO DO A PARADIGM SHIFT BECAUSE IF WE'RE NOT -- IF DIALOGUE AND PARADIGM IS NOT REACHING PEOPLE WHO ARE STILL THINKING MORE FROM A TRADITIONAL SCIENTIFIC PERSPECTIVE, THAT'S JUST LOOKING AT INDIVIDUAL LEVEL OUTCOMES AND RCTs, WE'RE GOING TO END UP IN TROUBLE WITH THE INNOVATION. THOSE ARE OTHER AREAS THAT CAN BE IMPROVED. >> I GUESS I'M GOING TO GO OFF SCRIPT TO FOLLOW UP ON THAT. WE'VE HEARD A LOT OVER THE LAST FEW DAYS ABOUT TRADITIONAL NIH PARADIGMS FOR RESEARCH. AND NEEDING TO HAVE ONE MEASURE, ONE INTERVENTION. I THINK THAT'S MAYBE ON SOME LEVEL TIED TO PERCEPTION, THAT'S WHAT RIGOROUS RESEARCH LOOKS LIKE. I WONDER IF YOU CAN MAYBE ALSO COMMENT ON HOW -- IN WHAT WAYS CAN WE CHANGE THAT DIALOGUE? TALKING ABOUT RIGOR OF THE RESEARCH, THE LAST FEW DAYS, HOW DO WE EMPHASIZE HOW THIS RESEARCH IS STILL RIGOROUS, EVEN THOUGH WE'RE DOING IT IN THE VERY NON-TRADITIONAL WAY? >> I THINK THAT ONE IS WITH REVIEWERS AND SCIENTIFIC COMMUNITY, NIH CAN TAKE LEADERSHIP EMPHASIZING OTHER PER PERSPECTIVES NEED TO BE EQUALLY VALUED AND SHIFT THE FOCUS. >> THANK YOU. DR. HAMMOND? >> YEAH, I REALLY AGREE WITH ALL THE COMMENTS BY MY COLLEAGUES, PLEASED TO SEE SYSTEM SCIENCE APPEAR MUCH LATER ALSO BECAUSE I REALLY DO THINK IT'S A SET OF TOOLS THAT ARE HELPFUL TO THINGS WE'RE TALKING ABOUT. TWO POINTS I WANTED TO MAKE NOW, ONE, THAT I THINK NONE OF THE TOOLS WE'VE HEARD ABOUT TODAY ARE TOTALLY NEW. THEY HAVE BEEN USED IN OTHER PLACES, OTHER FIELDS. THERE ARE LESSONS TO BE LEARNED HOW THEY PERMEATED TO THOSE FIELDS, WHAT WENT WELL, WHAT DIDN'T GO WELL, HOW THE PARADIGM SHIFT AND THE WAY PEOPLE LOOK AT THE FIELD MIGHT BE ACHIEVED. IN POPULATION HEALTH, THINGS UNKNOWN 20 YEARS AGO, INFECTIOUS DISEASE, CHRONIC DISEASE, NOW INCREASINGLY HEALTH DISPARITIES RESEARCH, THERE'S LESSONS ABOUT HOW THAT HAPPENED. AND SPECIFIC ACTIONS FOR EXAMPLE OBSSR HELPED FOSTER PROPAGATION OF TOOLS TO GET THEM TO PASS REVIEW PANELS. A LOT OF WORK WENT INTO THAT. THE OTHER POINT I WANTED TO MAKE, I THINK TOO OFTEN IN HEALTH SCIENCES AND I MYSELF AM GUILTY WE DRAW THIS SHARP DISTINCTION BETWEEN THE BIOLOGICAL AND SOCIAL AND BEHAVIORAL. REALLY THE PROBLEMS WE CARE ABOUT ALMOST ALWAYS TRANSCEND THOSE LEVELS OF SCALE AND INVOLVE MECHANISMS THAT SPECIFICALLY CROSS OVER INTO LEVELS OF SCALE SO I THINK SOME OF THE PROGRESS WE NEED TO MAKE IS BREAKING UP SOME OF THESE ARTIFICIAL BOUNDARIES BETWEEN FIELDS AND GROUPS OF PEOPLE WHO AREN'T USED TO TALKING TO EACH OTHER. >> I COULDN'T AGREE MORE. THANK YOU FOR THAT PERSPECTIVE. >> I'M ECHOES WHAT OTHERS SAID BUT I WANT TO JUST MENTION THAT I THINK SOME OF THESE COMPLEX INTERVENTIONS ESPECIALLY LIKE THE M3 STUDY OR LILLY STUDY WORKING WITH OTHER SYSTEMS TRYING TO WORK WITH AND ENGAGE ANOTHER GROUP TO KIND OF DEVELOP A METHOD. THAT'S CLUNKY. WE WERE ACTUALLY FUNDED BY A FOUNDATION BUT I FEEL THAT WHERE NIH MIGHT BE ABLE TO HELP IS IF THERE ARE -- IF THERE'S MONEY TO FUND FEASIBILITY STUDIES, ARE THESE STUDIES THAT MIGHT HAVE POTENTIAL FOR HIGH IMPACT? IF WE ARE ABLE TO IMPLEMENT THE LIFESTYLE PROGRAM ACROSS THE STATE OF OREGON, THAT WOULD HAVE HIGH IMPACT IN THIS GROUP. SO, YOU KNOW, IT'S RISKY ON NIH SIDE BUT IF IT PANS OUT IT COULD BE PHENOMENAL. AND I THINK IT DOESN'T NEED TO BE A LOT OF MONEY TO START OUT BECAUSE YOU'RE WORKING OUT MAYBE QUALITATIVE METHODS, THESE OTHER METHODS THAT HAVE BEEN DISCUSSED. BUT THERE ARE THESE LEARNING POINTS AT THE VERY BEGINNING WHERE YOU I THINK ARE IN THIS TRIAL WHERE SOME THINGS DON'T WORK OUT BUT SOME THINGS DO. AND TO HAVE SOME FUNDING TO BE ABLE TO START THAT PROCESS I THINK WOULD BE A PARADIGM SHIFT. AND I THINK THE OTHER THING, I REALLY WANT TO EMPHASIZE THIS, IS REALLY STUDY SECTION. PEOPLE WHO REVIEW THESE TYPES OF APPLICATIONS REALLY NEED TO UNDERSTAND, YOU KNOW, BOTH, YOU KNOW, WE HAVE TO PAY ATTENTION TO RIGOR BUT THERE IS A BIG PICTURE HERE. THE REAL WORLD IS VERY COMPLICATED. AND YOU NEED REVIEWERS WHO REALLY UNDERSTAND BOTH THOSE WORLDS VERY WELL AND CAN BALANCE THAT IN REVIEW. THOSE ARE MY MAIN COMMENTS. >> THANK YOU FOR THAT. ONE THING THAT NCCIH HAS DONE IN THE PAST -- WE WANT TO MAKE SURE TRIALS WE FUND, THAT WE LEARN SOMETHING WHETHER THEY COME UP POSITIVE OR NEGATIVE IN THE TRADITIONAL SENSE. THAT'S ANOTHER ASPECT. I'M ASKING AS A NAIVE NON-CLINICIAN IF WE'RE DOING THESE SORT OF STUDIES AND WE'RE INTRODUCING LOTS OF VARIABLES AND IT DOESN'T WORK OUT, I MEAN, IS THERE STILL THINGS WE CAN LEARN FROM THOSE STUDIES THAT WILL INFORM THE NEXT STUDY I GUESS IS THE QUESTION I HAVE. WE'VE FUNNED LARGE TRIALS, THEY DIDN'T WORK OUT, WE WERE LEFT SCRATCHING OUR HEAD AS TO WHERE WE GO NEXT. THAT'S AN IMPORTANT ASPECT THAT I WANT TO ADD HERE TO MAKE SURE THAT WHAT WE'RE INTERESTED IN ENGAGING IN THIS AREA, HAVING THIS MEETING, BUT WE ALSO WANT TO MAKE SURE WE'RE LEARNING ALONG THE WAY AS WE'RE DOING THESE TYPES OF STUDIES, MAKE SURE WE LEARN FROM OUR MISTAKES AS WELL AS SUCCESSES. DO YOU WANT TO FOLLOW UP ON THAT, DR. SHINTO? >> MAYBE STUDIES IN THE PAST, THOSE WERE LARGER TRIALS, THEY WERE EXPENSIVE STUDIES. WHAT I'M TALKING ABOUT HERE FOR FEASIBILITY IS ON A MUCH SMALLER SCALE. THERE IS A RISK BUT SMALLER RISK FINANCIALLY. >> THANK YOU. THANK YOU. >> I WANT TO ADD TO THIS THAT AN IMPORTANT ASPECT I THINK NCCIH, HOW METHODS DISSEMINATION, THERE'S BEEN A LOT OF CONVERSATION ABOUT DEVELOPING NEW METHODS AND I AGREE IT'S IMPORTANT BUT IN OTHER WORDS METHODS WE DISCUSSED TODAY ARE NOT NEW BUT THEY HAVEN'T BEEN WELL DISSEMINATED TO SCIENTISTS IN A WAY THEY CAN USE THEM THEMSELVES. RIGHT? SO FOCUSING ON TRYING TO GET METHODOLOGISTS TO PUT THEIR METHODS OUT THERE IN A WAY THAT OTHERS CAN USE THEM WITHOUT THEIR HELP, I THINK CAN REALLY HELP ADVANCE THE FIELD. AND MORE SPECIFICALLY TALKING ABOUT MAKING CODE, SOFTWARE, DOCUMENTATION AVAILABLE, INCENTIVIZING SO METHODS WOULD BE AVAILABLE TO THE BROADER COMMUNITY. >> I'M HEARING THE METHODS ARE THERE BUT HAVE NOT BEEN APPLIED. A FOLLOW-UP QUESTION IS IT YOUR OPINION, GAPS AND CHALLENGES, IS IT MORE ABOUT BRINGING THE RIGHT PEOPLE TOGETHER OR MORE ABOUT DEVELOPING NEW TECHNOLOGIES TO ANSWER THE QUESTIONS WE HAVE, IN YOUR OPINION ARE TECHNOLOGIES THERE WE JUST HAVE TO BRING THE RIGHT PEOPLE TOGETHER? >> I THINK IT'S BOTH. BUT IT'S NOT JUST ABOUT -- WELL, IT IS ABOUT BRINGING PEOPLE TOGETHER. SOME METHODS ARE OUT THERE BUT I DON'T THINK THERE ARE -- THEY HAVE BEEN APPLIED OR THERE ARE ENOUGH ILLUSTRATIONS OF THE APPLICATION IN VARIOUS SETTINGS OR SCENARIOS THAT CAN BE HELPFUL TO THE BROADER COMMUNITY, FOR EXAMPLE IN DEVELOPING MULTI-COMPONENT INTERVENTIONS, MULTIPLE OUTCOMES, RIGHT? FOR OTHER PROBLEMS, IT'S CLEAR WE NEED TO DEVELOP THE METHODS. I THINK TO BE ABLE TO GET THE METHODS IN HANDS OF SCIENTISTS THERE'S GOT TO BE -- IT'S ABOUT BRINGING PEOPLE TOGETHER, A CONVERSATION BETWEEN METHODOLOGISTS AND SCIENTISTS, METHODS CANNOT BE DEVELOPED IN ISOLATION IF YOU WANT THEM TO BE USEFUL. THIS CONVERSATION COULD LEAD TO USEFUL TOOLS THAT SCIENTISTS CAN USE ON THEIR OWN WITHOUT NEEDING METHODOLOGISTS TO GUIDE THEM. >> THANK YOU. DR. JENSEN, I'D LOVE TO HEAR YOUR PER PERSPECTIVES. >> REVIEW HANDLES -- PANELS ARE CONSERVATIVE. I'VE BEEN ON THEM. I'VE SUBMITTED GRANTS. I'VE HEARD COOL IDEAS IN THIS WORK SOM.SHOP. WE WORRY IS TO GET THEM OUT THERE AND GET PEOPLE TO TRY THEM AND FAIL AND LEARN, IT MAY BE THAT THERE MAY BE SPECIFIC RFAs COMING OUT THAT SAY, FOR EXAMPLE, THIS STUDY HAS TO HAVE LESS THAN 20 SUBJECTS, N-of-1 SERIES, WILL BE DESIGNED TO ANSWER SPECIFIC QUESTIONS, TERRIBLY EXCITING BUT DIFFICULT TO GET THROUGH STUDY SECTION. IF AN RFA SAID YOU HAD TO DO THIS. I THINK, THOSE STRATEGIES WOULD GET OUT THERE. BUT THERE ARE MANY IDEAS. TO ME IT'S ABOUT GETTING STRUCTURE IN THE RFA SO INVESTIGATORS HAVE TO WRITE IT IN A WAY THAT MOVE IN THIS DIRECTION AND REVIEWERS, THAT'S WHAT THEY HAVE TO REVIEW. AND NOT GO WITH THE OLD, YOU KNOW, 300-SUBJECT RCT, WHICH IS OKAY BUT WE MUST MOVE BEYOND THAT OR WE WON'T GET WHERE WE NEED TO GET. >> I'VE WRITTEN MANY AND YOU'D BE AMAZED HOW CREATIVE BOTH AUTHORS AND REVIEWERS CAN GET IN INTERPRETING RFA. I DIGRESS. WHAT ELSE CAN YOU ADD? >> YES, I AGREE BOTH WITH YOU AND DR. JENSEN ABOUT THE NEED FOR SPECIFIC FUNDING IN THIS AREA. GOING BACK TO THE METHODOLOGY THING, I AGREE WITH THE COMMENTS MADE ABOUT HOW A LOT OF WHAT WAS DISCUSSED HAS BEEN ABOUT BEFORE BUT IT'S REALLY ABOUT INTEGRATION AND THIS CAME UP IN THE INTERDISCIPLINARY FIELD. WHAT I MEAN BY DESIGN IS NOT JUST N-of-1 CLINICAL TRIALS WHICH HAVE BEEN IN LITERATURE FOR 50 YEARS. IT'S WHAT MEASURES AND HOW OFTEN AND THAT HAS TO BE DESIGNED WITH DIFFERENT STAKE HOLDERS AT THE STABLE WHO KNOW ENOUGH ABOUT THE COMPLEXITIES OF THE MEASUREMENTS THAT NEED THAT NEED TO BE MADE. THEY NEED TO BE AT THE TABLE TO GIVE INSIGHT, HOW MANY MEASURES YOU NEED TO ESTABLISH A BASELINE, THESE THINGS. ONE OFFSHOOT, THIS IS A WHOLE PERSON RESEARCH, THE PANELISTS AND EVEN THE AUDIENCE MEMBERS SHOULD ASK WHAT DOES IT MAKE SENSE TO MEASURE TO GET AT A WHOLE PERSON PHENOTYPE, WOULD IT BE BLOOD PRESSURE, SOME WAY OF QUANTIFYING THE SLEEP QUALITY OF SOMEONE? WEIGHT? MOOD? WHAT WOULD IT BE? HOW COMPLEX WOULD THE MEASURES BE BEFORE WE CAPTURE THE WHOLE PERSON KIND OF HEALTH IN SOME ALL-ENCOMPASSING WAY IN A TRIAL, I THINK THOSE SORTS OF METHODOLOGIES MIXED WITH THE MATH ARE WHERE THE FIELD SHOULD BE GOING, IN MY ESTIMATION. >> APPRECIATE THAT. DR. GUPTA. >> THANK YOU. I WANT TO EXTEND SLIDE QUESTIONS IN THE PERSPECTIVE OF RELEVANCE TO OUR FOGARTY INTERNATIONAL CENTER. WE HEARD VERY EXCITING DISCUSSIONS. WHOLE PERSON RESEARCH, WE HEARD HOW MUCH THERE'S VARIABILITY, GENETICS, EPIGENETICS, MICROBIOME. NOT ONLY WITHIN THE UNITED STATES BUT GLOBAL CONTEXT, SO MANY POPULATIONS, AND SO ON. THE SECOND THING IS METHODOLOGY. WE HEARD CHINESE MEDICINE, YOGA, OTHER THINGS WHICH HAVE CULTURAL CONTEXT, SOME FROM OTHER PARTS OF THE WORLD. WHAT IS THE ROLE OF GLOBAL RESEARCH COOPERATION IN THIS CONTEXT? WE HEAR A LOT IN INFECTIOUS DISEASES, HUMAN GENOME PROJECT WOULDN'T HAVE HAPPENED WITHOUT THE GLOBAL CONSORTIA. SO WHAT ABOUT WHOLE PERSON RESEARCH, WHERE DOES GLOBAL COOPERATION FIT INTO THIS? THE QUESTION TO ALL THE SPEAKERS FROM YESTERDAY AND TODAY. AND TO YOU. THANK YOU. >> THANKS. YEAH, I WILL ATTEMPT TO ANSWER THAT ONE. THIS IS WELL OUTSIDE MY FIELD BUT I'LL BE HAPPY TO ENTERTAIN THOUGHTS ON TO WHAT EXTENT DO WE NEED TO INCLUDE A GLOBAL PER PERSPECTIVE, TALKING ABOUT THE VERY DIFFICULT QUESTIONS, SOCIAL DETERMINANTS OF HEALTH IN THE UNITED STATES, BUT I'M HAPPY TO HEAR WHAT PEOPLE THINK. DR. WINDSOR? >> I'M DOING COVID-RELATED RESEARCH AND THERE'S NOTHING LIKE THE PANDEMIC TO SHOW HOW INTERCONNECTED AND SIMILAR A LOT OF STRUCTURES ARE, SOCIAL DETERMINANTS OF HEALTH IMPACT NOT JUST IN THE U.S. BUT IT'S GLOBAL. IT'S CRITICALLY RELEVANT, NEEDS TO BE INCLUDED IN THIS CONVERSATION, IN THE DEVELOPMENT OF THESE METHODS AND FOR THE FUNDING. >> I AGREE. YOU REALIZE WE'RE ALL ALIKE DESPITE OBVIOUS DIFFERENCES. DR. HEALEY? >> BUILDING ON WHAT DR. WINDSOR SAID WHEN THINKING ABOUT STRUCTURAL DETERMINANTS OF HEALTH, POSITIONING -- THESE CROSS-NATIONAL SETTINGS TO UNDERSTAND HISTORICAL EVOLUTIONS OF STRUCTURAL OPPRESSION CAN GIVE OPENINGS FOR UNDERSTANDING DIFFERENCES WE MAY BE CONSTRAINED TO SEE IN OUR POPULATIONS, LOOKING AT BLACK MATERNAL HEALTH, A MODEL THAT I'VE LOOKED AT NATIONALLY, BRAZIL. YOU HAVE SIMILAR HISTORIES, LEGACIES OF SLAVERY, EVOLUTION OF RACIAL CATEGORIES, SHIFTS IN HOW THAT WAS ACTUALLY ENCAPSULATED IN STRUCTURAL POLL SILLS. YOU STILL HAVE SAME PATTERNS BUT CAN SEE WHERE A LOT OF TIMES IN THE EARLY WORK, YOU CAN'T INTERVENE, IT'S INTERVENING ON RACISM AND PUSHING THE GLOBAL CONTEXT CAN ALLOW ALLOW US TO SEE WHEN WE BEING CONSTRAINED, WHEN WE JUST LOOK AT ONE NATIONAL CONTEXT. IN THAT CASE LIKE FOR ME EXTENDING MY WORK GLOBALLY AND FINDING WAYS TO DO THAT IS KIND OF THE $100,000 QUESTION, MAYBE MORE. THAT'S THE CRUX, HOW DO WE FIND WAYS TO LEVERAGE GLOBAL RESEARCH TO UNDERSTAND BOTH OUTCOMES AND THESE ARE EVOLUTIONS OF OUTCOMES AND PEOPLE EMBEDDED IN DIFFERENT PLACES BUT AN INCREASINGLY VOCAL SOCIETY, THINGS NO LONGER ARE INTERDEPENDENT, THINGS ARE CO-DEPENDENT AND INTERRELATED GOING FORWARD. >> WENDY? YOU MIGHT GET THE LAST COMMENT. >> I WANT TO COME BACK TO THE QUESTION AROUND DO WE NEED A METRIC THAT COULD BE DEVELOPED TO LOOK AT THIS WHOLE PERSON HEALTH, REFLECTING TO YESTERDAY'S TALK FROM DR. MOFFIT, HAVING A SINGLE SERUM MEASURE THAT PREDICTS, I DON'T KNOW IF IT'S THE SAME MEASURE. IS PATIENT AGE A MEASURE OF HEALTH? I DON'T KNOW IT CAPTURES EVERYTHING WE WANT IT TO POTENTIALLY CAPTURE. BUT I'M CURIOUS AS TO WHETHER THAT MIGHT BE SOMETHING THAT YOU THINK THE FIELD POTENTIALLY WOULD NEED OR IF WE'RE GOING INTO THIS TO SPACE AND WANT TO LOOK ACROSS DIFFERENT OUTCOMES DOES CREATING SOME SORT OF COMPOSITE OR WAY OF HAVING SORT OF SUMMARY MEASURE OF SOME SORT WOULD THAT BE PARTICULARLY HELPFUL? >> DR. HEADEN? >> NOT WANTING TO RECREATE BIASES THAT IMPEDE US ACROSS POPULATION, DIFFERENT POPULATIONS DON'T THINK ABOUT THEIR HEALTH, CONVENIENT FOR US AS SCIENTISTS WE ALSO NEED TO THINK ABOUT HAVING FLEXIBILITY IN TERMS OF WHAT DOES THAT MAYBE LOOK LIKE FOR QUESTIONS THAT WE'RE ANSWERING. THERE MAY BE A PLACE IN TIME WHEN IT'S USEFUL TO HAVE. I WAS INTRIGUED BY THE AGING MEASURE AS WELL AS THE EPIGENETIC PROFILES AS A PRECURSOR FOR UNDERSTANDING PRE-CLINICAL DISEASE, ALSO FOR A LOT OF BLACK POPULATIONS, OUTCOMES CAN COME ON THE SO ADVERSE MATERNAL HEALTH, CHRONIC DISEASE, BUT HAVING PRE-CLINICAL LIKE AGING MEASURES CAN BE USEFUL AND HOW CAN WE WEATHER FORECAST -- HOW CAN WE WORK WITH COMMUNITIES TO UNDERSTAND HEALTH SO WE'RE NOT CREATING RESEARCH WITH NO TRANSLATION. WE TRY TO IMPLEMENT INTERVENTIONS AND, YOU KNOW, NATIONAL LEVEL PROGRAMS IN THIS REGARD SO IT CAN BE, YOU KNOW, DEPENDENT ON THE QUESTION WE WANT TO ANSWER, RIGHT. >> THIS IS A WONDERFUL CONVERSATION. I WISH WE ARE ANOTHER HOUR. DR. WINDSOR? >> I WAS GOING TO SAY THAT WE ALSO HAVE OUTCOMES THAT WE KNOW IMPACT SEVERAL THINGS AT THE SAME TIME, INDIVIDUAL LEVEL AND NEXT LEVEL. LIKE POVERTY, FOR EXAMPLE, ACCESS TO FOOD. WE COULD DO A LOT BY THINKING FROM THOSE PARTICULAR OUTCOMES AS WELL. >> ONE LAST QUICK COMMENT ABOUT FLEXIBILITY. NOT JUST IN TERMS OF OUTCOMES BUT ALSO ON THE EXPERIMENTAL DESIGNS AND METHODS THAT WE'RE USING, GOING BACK TO THIS IDEA OF HAVING, YOU KNOW, FOCUSING ON SINGLE METHODOLOGICAL APPROACH, YOU HAVE TO USE THIS APPROACH TO APPLY FOR THIS FUNDING MECHANISM BUT I THINK THAT YOU MIGHT WANT TO TAKE A MORE FLEXIBLE APPROACH BECAUSE SOME REALLY NOVEL SCIENTIFIC QUESTIONS AND REALLY NOVEL INTERVENTIONS CAN WE DEVELOP BY USING SOMETIMES STANDARD EXPERIMENTAL APPROACHES AND RELATIVELY STANDARD METHODOLOGIES, RCTs CAN BE USED TO ANSWER NOVEL QUESTIONS, RIGHT? THE CONCEPT OF FLEXIBILITY SHOULD BE CONSIDERED IN RELATION TO THE SCIENTIFIC QUESTIONS THAT WE HAVE IN MIND. THESE SHOULD BE EMPHASIZED. NOVELTY SHOULD BE EMPHASIZED. METHODS AND OUTCOMES WE'RE FOCUSING ON AND HOW WE INTEGRATE WE SHOULD FOLLOW THE SCIENTIFIC QUESTIONS. >> WELL SAID. THANK YOU. I DON'T SEE HANDS UP AND WE HAPPEN TO BE RIGHT AT 4:59 SO MAYBE I WILL END US AND TURN IT BACK OVER TO CATHERINE WHO WILLs TAKE US TO THE LAST TALK. I THANK THE PANELISTS, I APPRECIATE IT. WE'LL HAVE LOTS TO THINK ABOUT IN THE DAYS AND WEEKS AHEAD. THANK YOU AGAIN. CATHERINE? >> THANK YOU, CRAIG AND WENDY AND EVERYBODY, PANELISTS AND EVERYBODY WHO JOINED THE DISCUSSION. DR. WEBER WILL TAKE US INTO THE FINAL CLOSING TALK FOR TODAY. WENDY, WE INVITE YOU TO INTRODUCE DR. BRUCE LEE. >> WONDERFUL. WHAT AN AMAZING CONVERSATION. WE'VE GIVEN DR. LEE A BIG CHALLENGE THIS AFTERNOON, SYNTHESIZING WHAT WE'VE TALKED ABOUT OVER THE LAST TWO DAYS AND FORWARD THINKING FOR US. WHERE DO WE GO FROM HERE. IT IS MY PLEASURE TO INTRODUCE DR. BRUCE LEE, PROFESSOR OF HEALTH POLICY AND MANAGEMENT, GRADUATE SCHOOL OF PUBLIC HEALTH AND HEALTH POLICY, EXECUTIVE DIRECTOR OF THE CENTER FOR ADVANCED TECHNOLOGY AND COMMUNICATION IN HEALTH, CITY UNIVERSITY OF NEW YORK. HE'S ALSO THE EXECUTIVE DIRECTOR OF PUBLIC HEALTH COMPUTATIONAL AND OPERATIONS RESEARCH, FOUNDER AND CHIEF EXECUTIVE OFFICER OF -- I WON'T SAY THIS RIGHT. SILLICO. SYSTEMS MODELER, WRITER, JOURNALIST, DR. LEE HAS OVER TWO DECADES OF EXPERIENCE IN DEVELOPING MATHEMATICAL AND COMPUTATIONAL MODELS. IT'S OUR PLEASURE TO WELCOME YOU, DR. LEE, AND THANK YOU FOR DOING THIS SYNTHESIS FOR US. >> THANK YOU, DR. WEBER. THIS HAS BEEN A GREAT CONVERSATION THESE PAST COUPLE DAYS. I'VE BEEN SITTING TRYING TO ABSORB EVERYTHING. I WON'T BE ABLE TO SUMMARIZE EVERYTHING. I WANTED TO GO OVER THOUGHTS IN TERMS OF SOME TAKEHOMES FOR THE PAST TWO DAYS. AND SOME POSSIBLE WAYS FORWARD. WITH THAT I WILL SHARE MY SLIDES. OKAY. CAN EVERYONE SEE MY SLIDES? >> YES. >> OKAY. SO WHAT ARE TAKEHOME POINTS FROM THIS WORKSHOP? AGAIN, LOTS OF GREAT INSIGHTS. IT'S IMPOSSIBLE TO SUMMARIZE EVERYTHING. IT WOULD TAKE ABOUT, OH, MAYBE TEN HOURS WHICH WAS ABOUT HOW LONG THE WORKSHOP WAS. THERE ARE MANY PEOPLE WHO AWARE OF WHOLE PERSON HEALTH BUT HERE IS THE DEFINITION. YOU'RE LOOKING AT THE WHOLE PERSON, THE WHOLE COMPLEX SYSTEM THAT CONSISTS OF PEOPLE BUT ALSO EVERYTHING AROUND PEOPLE. SO, THIS IS A DIAGRAM PULLED FROM THE NCCIH WEBSITE WHICH DESCRIBES WHAT WE MEAN BY WHOLE PERSON HEALTH AND SHOWS THE DIFFERENT SCALES. THIS WAS AN ARTICLE WRITTEN BY DR. LANGEVIN TALKING ABOUT THE INITIATIVE, NIH CENTER'S FIVE-YEAR PLAN TO FOCUS ON RESEARCH ON WHOLE PERSON HEALTH. WE STARTED OFF WITH THAT INTRODUCTION TO SET THE PLAYING FIELD. ALSO MENTIONED DURING HER TALK SHE MENTIONED ON YOU MANY MAJOR HEALTH PROBLEMS HAVE BEEN ON THE HORIZON SINCE THE '80s. WE LOOK AT THEM SEPARATELY. SHOULD WE DEAL WITH OBESITY? WITH MENTAL HEALTH CRISIS? OPIOID CRISIS? THESE ARE ALL PROBLEMS OBVIOUSLY. BUT IT'S VERY INTERESTING IF YOU LOOK AT TEMPORALLY THEY ALL SEEM TO TAKE THE SAME TRAJECTORY. OFF HERE AND THERE IN TERMS OF YEARS BUT SEEM TO BE PICKING UP SINCE THE 1980s SUGGESTING THEY ARE CONNECTED, THEY ARE NOT REALLY SEPARATE ENTITIES BUT MAYBE THERE ARE SOME ISSUES WITH THE SYSTEMS INVOLVED. MAYBE THEY ARE BROKEN SYSTEMS WE NEED TO ADDRESS. WE WON'T BE ABLE TO ADDRESS THOSE THINGS UNTIL WE UNDERSTAND THAT EVERYTHING IS CONNECTED. IF WE'RE FOCUSED ON BODY PART OR JUST DISEASE WE MAY NOT SEE THE BIGGER PICTURE. THIS IS A SEEM -- THEME ECHOED OVER THE PAST COUPLE DAYS. CORRELATION DOES NOT EQUAL CAUSATION. WE'LL SEE THESE GENERAL CORRELATIONS SO I POINT OUT THESE COFFEE STUDIES, IS IT GOOD, BAD, DEPENDS WHICH STUDY YOU LOOK AT. IT'S BEEN ASSOCIATED WITH CERTAIN TYPES OF CANCER, OTHER CASES MAYBE IT CAN PREVENT CANCER OR IT'S ASSOCIATED WITH LONGER LIFE, THE PROBLEM IS YOU'RE LOOKING REALLY AT SUPERFICIAL LAYER OF DEEPER COMPLEX SYSTEM THAT MIGHT AFFECT ALL THESE TYPES OF OUTCOMES. I FREQUENTLY POINT OUT THIS GRAPH WHICH SHOWS THE CORRELATION BETWEEN THE NUMBER OF PIRATES AND GLOBAL WARMING. IF YOU WERE TO CORRELATE THAT YOU WOULD SEE AS THE NUMBER OF PIRATES HAVE GONE DOWN OVER THE YEARS, GLOBAL TEMPERATURES HAVE GONE UP. DOES THAT MEAN WE NEED TO TRAIN MORE PIRATES? IS THAT THE SOLUTION TO THE CLIMATE CRISIS? OF COURSE NOT. SO THAT'S WHERE WE CAN GET INTO DANGER WHEN WE HAVE -- WE RELY ON PURE ASSOCIATION OF CORRELATIONS. PEOPLE ARE AND LIVE WITHIN COMPLEX SYSTEMS. THESE ARE MULTI-LEVEL SYSTEMS. I THINK DR. HAMMOND JUST RAISED A GREAT POINT DURING THE PAST SESSION, WE THINK ABOUT LIKE BELOW THE SKIN, WITHIN THE BODY, EVEN WITHIN THE BODY, WITHIN BODY PARTS. WE FORGET THE OTHER SCALES LIKE WE'RE INDIVIDUALS, INDIVIDUAL BEHAVIORS, AFFECTED BY OUR SOCIAL CONSTRUCTS, NETWORK, ENVIRONMENT, ECONOMICS, THESE ARE VERY COMPLEX. WE NEED TO THINK ABOUT ALL THESE THINGS TOGETHER. THE DANGERS OF NOT USING A SYSTEMS APPROACH ARE MULTIPLE. YOU MAY LEAD TO BAND-AIDS INSTEAD OF REAL SOLUTIONS, COVER UP A PROBLEM AND THE PROBLEM LIKE A GIGANTIC WEATHER BALLOON WILL POP UP SOMEWHERE ELSE. MANY TIMES YOU'RE GOING TO OVERLOOK THE SECONDARY AND TERTIARY EFFECTS AND SOME MAY BE BAD LIKE THEY MIGHT BE UNINTENDED CONSEQUENCES BUT SOME MAY BE GOOD LIKE IF YOU APPLY SOME TYPE OF INTERVENTION, IF YOU APPLY SOME TYPE MUCH BEHAVIORAL HEALTH INTERVENTION, YOU MIGHT HAVE REVERBERATING EFFECTS. IF YOU'RE APPLYING MULTI-COMPONENT INTERVENTION YOU MIGHT BE ADDRESSING MANY THINGS AND IF YOU DON'T USE SYSTEMS APPROACH MAY BE GROSSLY UNDERESTIMATING POSITIVE IMPACT. YOU CAN SPEND TIME, EFFORT, RESOURCES TRYING DIFFERENT THINGS, IF YOU DON'T HAVE AN UNDERSTANDING OF THE ENTIRE SYSTEM YOU MIGHT BE TRYING THE WRONG THINGS. THESE ARE REASONS WHY YOU NEED TO USE A SYSTEMS APPROACH. THE QUESTIONS COME UP FREQUENTLY. ARE THESE COMPLEX SYSTEMS THAT DIFFICULT TO UNDERSTAND? PEOPLE SAY, OH, IT'S TOO COMPLEX. I'VE SEEN PEOPLE BE SHOWN THAT DIAGRAM, BYTES, FACTORS, THAT'S OVERWHELMING. YOU ARE HUMAN. SYSTEMS ARE COMPLEX. AND YOU NEED HELP. NOT IN GENERAL BUT YOU NEED HELP IN TERMS OF UNDERSTANDING THESE SYSTEMS. THERE IS HELP BECAUSE THINGS HAVE CHANGED. ONE OF THE CHALLENGES IS BIOMEDICAL AND HEALTH RESEARCH OCCURRED IN SILOS, WE DON'T WORK LIKE THIS BUT THIS IS AN EXAMPLE OF THE CUBICLE APPROACH TO MEDICAL RESEARCH, WITHIN YOUR AREA. WE NEED TO BREAK DOWN THE SILOS AND THIS THEME WAS ECHOED SEVERAL TIMES THROUGHOUT NOT JUST TODAY BUT YESTERDAY AS WELL. WE'VE GOT THIS EXPLOSION OF DATA. YOU'VE HEARD TALKS ABOUT HOW WE HAVE A LOT MORE DATA THESE DAYS THAN BEFORE. BUT WE HAVE TO BE CAREFUL BECAUSE IT CANTON LIKE, OKAY, I HAVE MORE DATA, MY DATA IS BIGGER THAN YOUR DATA. HAVE YOU TEN THOUSAND OR A ONE HUNDRED THOUSAND, WHATEVER COMES FROM THAT THEREFORE IS CANON, IT'S NOT JUST HAVING THE DATA BUT FIRST OF ALL YOU HAVE TO DETERMINE IS THIS DATA USEFUL? IS THIS INFORMATION USEFUL? THIS IS THE EXAMPLE OF SIGNS, GARBAGE ONLY, NO TRASH, EVERY ELSE WILL BE TOAD, T-O-A-D. DOES THIS NEED TO BE CLEANED, CORRECTED? WE HAD A NUMBER OF TALKS, GREAT METHODS ON HOW TO ORGANIZE DATA, CLEAN DATA, DRAW INSIGHT. IT'S IMPORTANT. THIS SLIDE EMPHASIZES MANY CRUCIAL STEPS GOING FROM DATA SOURCES TO ANALYSIS. SESSION 1 COVERED VERY INNOVATIVE METHODS TAKING DATA, SORTING THROUGH AND COLLAPSING AND GET READY FOR ANALYSIS. INFORMATION CAN GO DIFFERENT DIRECTIONS. THE WAY YOU ANALYZE AND ORGANIZE CAN LEAD TO PROPER CONCLUSIONS OR TO IMPROPER CONCLUSIONS. I GIVE THIS EXAMPLE. RUMOR HAS IT SOMEONE ELSE NAMED BRUCE LEE, IF YOU LOOK AT THERE'S ALL THESE MARTIAL ARTS FILMS YOU ASSUME BRUCE LEE IS A MARTIAL ARTIST OR EVERYONE OF ASIAN DESCENT KNOWS KUNG FU. YOU COULD DRAW WRONG CONCLUSIONS, THERE ARE WORSE THINGS THAN BEING KNOWN FOR KUNG FU BUT YOU CAN GET A LOT OF BIAS AND A LOT OF MISUNDERSTANDING THAT COMES FROM ANALYZING, LEADING TO DIVERSITY, EQUITY, AND INCLUSION, WORSENING HEALTH DISPARITIES, THAT'S A THEME THAT RAN THROUGHOUT THE TALKS, IT'S IMPORTANT TO SEE WHAT YOU'RE DOING AND HOW YOU ANALYZE, ARE WE INTRODUCING NEW BIASES, MAKING THINGS WORSE. BIG DATA IS NOT NECESSARILY BETTER DATA. WE'RE AT A KEY INFLECTION POINT AS YOU'VE SEEN IN THE PAST COUPLE DAYS. WE HAVE A LOT MORE DATA NOW. WE HAVE ALL THESE DIFFERENT DEVICES, TECHNOLOGIES, WEARABLES. WE HAVE DIFFERENT APPROACHES, A.I., MACHINE LEARNING, WHAT YOU HEARD OVER THE PAST COUPLE DAYS. WHAT DIRECTION ARE WE GOING, THE RIGHT DIRECTION WHERE WE ARE ACTUAL EYE USING THIS TO UNDERSTAND COMPLEX SYSTEMS OR THE WRONG DIRECTION WHERE WE BASICALLY WORSEN OR DEEPEN BIASES OR DEEPEN MISINFORMATION. IT'S ALL IN HOW YOU USE THESE THINGS, IT'S IMPORTANT TO KEEP THIS IN MIND. ONE THING THAT HAS COME UP A LOT OF STANDARD RESEARCH METHODS, THERE'S GREAT STUFF ABOUT STANDARD METHODS, IT'S HELPED US BRING INSIGHT OVER HISTORY, BUT THERE CAN BE OVERSIMPLIFICATION. THIS IS AN EXAMPLE. THIS IS THE MATRIX, THE RED PILL OR BLUE PILL? NOT EVERYTHING IS A PILL. THERE'S NO MAGICAL PILL THAT WILL MAKE EVERYTHING GO AWAY. MANY OF THESE EFFECTIVE INTERVENTIONS AND MEASURES ARE COMPLEX, THEY ADDRESS DIFFERENT ASPECTS OF MENTAL HEALTH, MENTAL WELL BEING AND PHYSICAL HEALTH AND HOW THEY ARE CONNECTED. CAN WE USE THE PILL PARADIGM? MAYBE NOT SO MUCH. IT'S HELPFUL TO UNDERSTAND THE IMPLICATIONS OF EVERYTHING. ONE THING THAT CAME UP STARTED WITH SESSION 2 AND SESSION 3, WE'VE GOT TO RETHINK HOW WE DO CLINICAL STUDIES. THERE'S TRADITIONAL TRIAL, RCT, ENROLL AS MANY AS POSSIBLE, CHOOSE ONE OUTCOME AND FOCUS AND IS IT SIGNIFICANT. MANY HAVE MULTIPLE COMPONENTS. HOW DO YOU CONNECT THE COMPONENT WITH THE RESULT? THERE'S MANY FACTORS AFFECTING THESE DIFFERENT OUTCOMES. AND THERE'S MULTIPLE OUTCOMES. THERE'S MULTIPLE DIFFERENT PROCESSES INVOLVED. DO YOU FOCUS ON OUTCOME, JUST LIVE OR DIE? DO YOU FOCUS ON HEART DISEASE, NO HEART DISEASE? MAYBE THAT'S AN OVERSIMPLIFICATION. THAT'S COME UP, WE TALKED ABOUT EARLIER TODAY MULTI-COMPONENT INTERVENTION, YESTERDAY ABOUT HOW DO YOU MEASURE HEALTH OUTCOMES AND THROUGHOUT THE DAY ALL THESE DIFFERENT TYPES OF COMPLEX FACTORS. THAT'S IMPORTANT. ONE THING THAT'S MENTIONED IS DIFFERENCE BETWEEN TOP-DOWN AND BOTTOM-UP APPROACHES. TOP-DOWN YOU HAVE DATA, ALL THIS INFORMATION, OKAY, I'M GOING TO LOOK DOWN, I'VE GOT 10,000-FOOT LEVEL, LOOKING FOR PATTERNS. THAT'S GREAT WHEN YOU'RE AT THE DISCOVERY PHASE. WHEN YOU'RE TRYING TO FIGURE OUT IN GENERAL IS SOMETHING GOING ON. HOW BIG IS THIS PROBLEM? WHAT ARE THE FACTORS THAT I SHOULD CONSIDER? THAT ALONE IS NOT ENOUGH. YOU NEED A BOTTOMS UP APPROACH, A SITUATION WHERE YOU'RE REBUILDING THE SYSTEMS, TRYING TO BUILD THE COMPLEX SYSTEMS, PIECE BY PIECE, PART BY PART. REALLY UNDERSTANDING THESE MECHANISMS. THAT'S ONE OF THE THINGS WE'RE FOCUSED ON, DURING THIS LAST SESSION WHERE WE TALKED ABOUT SYSTEMS APPROACHES AND REBUILDING THE DIFFERENT TYPES OF SYSTEMS. SO ONE OF THE THINGS THAT CAME UP WAS MODELING. WHENEVER I TELL PEOPLE THAT I'M A MODELER, THEY LOOK AT MY CLOTHES, MY COMPLETE LACK OF FASHION SENSE. NO, NO, YOU'RE NOT A MODELER. NOT A CAT WALK MODELER, I'M THE TYPE THAT USES A COMPUTER. WE BUILD COMPUTER MODELS. A FEW EXAMPLES, YOU'VE SEEN EXAMPLES EARLIER TODAY. THIS IS JUST TO SHOW HOW YOU CAN HAVE SOME COMPLEX INFORMATION AND INSIGHT EMERGE FROM WHAT INITIALLY DOESN'T SEEM COMPLEX. TO UNDERSTAND AND ADDRESS THESE SYSTEM, THIS IS AN EXAMPLE WHERE WE BUILD A MODEL OF DIFFERENT CITIES TO UNDERSTAND IMPACT OF WARNING LABELS, YOU PUT THEM IN STORES, ON PRODUCTS. WE BUILT SIMULATION MODELS OF BALTIMORE, SAN FRANCISCO, PHILADELPHIA, AND WE FOUND THAT THE IMPACT OF WARNING LABELS VARIED ACROSS THOSE DIFFERENT CITIES BECAUSE CITIES ARE DIFFERENT. CITIES HAVE DIFFERENT COMPOSITIONS OF PEOPLE. THEY HAVE DIFFERENT DISTRIBUTIONS OF BODY MASS INDEX. STORES LOCATED IN DIFFERENT AREAS, ENVIRONMENT IS DIFFERENT. PEOPLE ARE DIFFERENT. THAT SHOULDN'T BE SURPRISING. WE'VE SEEN SITUATIONS LIKE USING COMPUTER MODEL OF VIRTUAL INFANT, TOOK FORMULA FEEDING RECOMMENDATIONS ESTABLISHED AND APPLIED TO THE VIRTUAL INFANT AND WE FOUND THAT EVEN WHEN YOU FOLLOW THESE RECOMMENDATIONS YOU HAVE INFANTS THAT HAVE BMIs THAT GO ABOVE RANGE THAT YOU CONSIDER NORMAL. YOU'RE FOLLOWING THE RECOMMENDATIONS BUT THEY ARE STILL RUNNING ASTRAIGHT. THAT'S BECAUSE INFANTS ARE DIFFERENT. TEN INFANTS, DIFFERENT BODY COMPOSITION, METABOLISM, THINGS LIKE THAT. WE BUILD A MODEL OF THE D.C. METRO AREA AND WE SAID, OKAY, WHO DO YOU WANT TO VACCINATE FIRST? WE FOUND ACTUALLY VACCINATING THE LOWEST INCOME NEIGHBORHOODS HELPS EVERYONE ELSE BECAUSE IF YOU THINK LOW INCOME NEIGHBORHOODS, PEOPLE ARE MORE DENSITY PACKED TOGETHER, MORE LIKELY TO TRAVEL TO HIGH INCOME NEIGHBORHOODS TO WORK VERSUS HIGH INCOME TO LOWER INCOME NEIGHBORHOODS TO WORK. IF YOU WANT TO PROTECT THE POPULATION, PROTECT THE MOST VULNERABLE MER. THESE ARE EXAMPLES OF HOW IF YOU UNDERSTAND THE COMPLEX SYSTEMS INVOLVED, YOU CAN ACTUALLY COME UP WITH INSIGHTS THAT ARE MAYBE NOT AS OBVIOUS. YOU SAY WE ALL KNOW FROM A MORAL STANDPOINT ETHICAL STANDPOINT AND FAIRNESS STANDPOINT YOU WANT TO TRY TO REDUCE HEALTH INEQUALITY, INEQUITIES. AND YOU WANT TO PROTECT THE MOST DISADVANTAGED PEOPLE. YOU WANT TO HELP THEM. THERE'S ACTUALLY A UTILITARIAN BENEFIT TO EVERYONE FOR THAT TO HAPPEN. YOU CAN ONLY SEE THAT WHEN YOU SEE COMPLEX SYSTEMS. UNDERSTAND COMPLEX SYSTEMS THAT WE'RE ALL CONNECTED. THOSE ARE QUICK EXAMPLES. ONE THING TO REMEMBER SYSTEMS APPROACHES ARE ITERATIVE. THIS IS SOMETHING THAT'S EMERGED THROUGHOUT THE WORKSHOP AS WELL. SOMETHING TO PULL THINGS TOGETHER. WE HEARD OF METHODS, APPROACHES, IT'S NOT ONE OF THE SITUATIONS WHERE, OKAY, THAT METHOD IS THE METHOD. IT'S BETTER THAN EVERYONE ELSE. THIS IS NOT LIKE AN EPISODE OF THE BACHELOR WITH ONE WINNER. THESE WORK TOGETHER AND THEY CAN HELP UNCOVER DIFFERENT PARTS OF THE SYSTEM. CONCEPTUALIZATION, YOU HAVE A SITUATION YOU CAN START OFF AND MAP THE SYSTEM AND SAY, OKAY, HOW DOES EVERYTHING FIT TOGETHER? IN THE LAST SITUATION MAPPING WAS MENTIONED. BUILD A MODEL. ONE OF THE THINGS THAT CAME UP, THINGS ARE COMPLEX, WE HAVE TO GET TO THAT POINT. NO, CAN YOU DO IT OFF THE BAT. IT DOESN'T TO BE A PERFECT MAP OR MODEL. THE MODEL HELPS SEE HOW EVERYTHING FITS TOGETHER. CAN YOU -- YOU CAN RUN TO PLAN CLINICAL STUDIES, COHORT STUDIES, ALL THESE OTHER THINGS. THAT IN TURN CAN ALLOW YOU TO THEN, OKAY, SET UP THESE TRIALS OR TEST DIFFERENT INTERVENTIONS, COLLECT DATA, FEEDBACK INTO THIS LOOP, MORE DATA, MORE INFORMATION, USE APPROACHES TO COLLAPSE AND ANALYZE, ORGANIZE DATA, UPDATE THE MAP. UPDATE THE MODEL. KEEP GOING AROUND IN A CIRCLE. THINK OF A CHURN, CHURNING OUT MORE INSIGHT, GETTING A BETTER UNDERSTANDING OF COMPLEX SYSTEM. SIMILAR TO HOW WE THINK. WHEN YOU'RE BORN, YOU DON'T KNOW ANYTHING. THIS IS MY THUMB, I THINK I'LL SUCK IT. YOU TEST AND ADD TO YOUR MENTAL MODEL. I TOUCHED THE STOVE. I SHOULDN'T DO THAT AGAIN. SO YOU'RE GETTING A BETTER UNDERSTAND OF THE COMPLEX SYSTEM AROUND YOU. WE CAN DO THE SAME THING WHEN IT COMES TO ALL THESE TYPES OF APPROACHES. WE DON'T HAVE TO HAVE IMMEDIATE ANSWER OR WE CAN CONNECT THESE THINGS AND MOVE TOWARDS A BETTER UNDERSTANDING OF THE SYSTEM. HOW DO WE CATALYZE? NOT ACTUAL MORTAR AND GLUE BUT BREAKING DOWN SILOS, FINDING PEOPLE, APPROACHES, ORGANIZATIONS, WHAT HAVE YOU, THAT CAN SERVE AS THE BRIDGES BETWEEN DIFFERENT SILOS. BETWEEN DIFFERENT DISCIPLINES. WE NEED PEOPLE WHO GO DEEP, SOMEONE WHO KNOWS CERTAIN BODY PART BETTER THAN ANYONE. THAT'S IMPORTANT. WE ALSO NEED CONNECTORS, THE GLUE, PEOPLE WHO CAN SAY LET'S GRAB SOME OF THIS, SOME OF THIS, PULL YOU GUYS TOGETHER AND THOSE THINGS. WHETHER THAT'S FUNDING MECHANISMS OR PEOPLE, EXPERTS, METHODS, ALL THOSE TYPES OF THINGS TO BRING EVERYTHING TOGETHER TO HELP BETTER UNDERSTAND WHOLE PERSON HEALTH. HOW DO WE CREATE THE NEXT GENERATION OR RECRUIT PEOPLE INCLINED TO THAT DIRECTION AS CONNECTORS. CONNECTORS ARE JUST AS IMPORTANT AS PEOPLE WHO GO VERY DEEP. WE NEED BOTH. WITH THAT I'M OUT OF TIME SO I'LL GO THROUGH A QUICK SUMMARY. WHOLE PERSON RESEARCH IS NEEDED. UNLESS YOU ARE A LEGO PERSON AND CAN FALL INTO PIECES QUICKLY YOU ARE A COMPLETE HUMAN BEING SO WE NEED TO UNDERSTAND HOW YOU FIT TOGETHER BUT HOW YOU FIT WITH OTHER PEOPLE, INTERACT WITH THE ENVIRONMENT. THIS IS A MULTI-SCALE SYSTEM THAT YOU ARE IN AND YOU ARE YOURSELF. THERE'S THIS EXPLOSION OF DATA. WE HAVE TO BE CAREFUL. WE CAN'T DO THE SAME OLD SAME OLD AS IN THE PAST. WE'VE GOT MORE DATA. WE CAN DO MORE. DATA IN ITSELF IS NEITHER GOOD NOR BAD. IT'S NEUTRAL. IT'S WHAT YOU DO WITH IT. THERE'S A MAJOR RISK OF BIAS AND LACK OF DIVERSITY AND INCLUSION. WE HAVE TO KEEP IN MIND ALWAYS HAS TO BE A NUMBER ONE PRIORITY BECAUSE THE GOAL IS TRUTH. THE GOAL IS AN UNDERSTANDING OF TRUTH AND NOT INTRODUCING MORE BIAS AND MORE INEQUITY. INTERVENTIONS ARE SUPER COMPLEX. BEING VERY REDUCTIONIST AND TRYING TO FOCUS ON ONE AT A TIME LOSES TRACK OF THE ENTIRE SYSTEM. THERE'S A NEED FOR MORE ITERATIVE SYSTEMS APPROACHES AS I MENTIONED, AND PULLING EVERYTHING TOGETHER AND WE NEED HYBRIDS, WE'VE ALL MET PEOPLE WHO ARE I LIKE DIFFERENT THINGS, AND PEOPLE WHO WANT TO THINK ABOUT ONE THING. WITH THAT I WANT TO ACKNOWLEDGE SOME OF THE WORK I MENTIONED AND GIVE CREDIT TO MY TEAM TO PUT TOGETHER STUDIES AND HELP WITH SLIDES, HAPPY TO OPEN THE FLOOR TO DISCUSSION, QUESTIONS. ANYTHING THAT ANYONE WANTS TO TALK ABOUT. >> WE WERE GOING TO INVITE DR. DR. LANGEVIN TO MAKE CLOSING REMARKS BUT, SO WOULD YOU LIKE TO JOIN US AND BRING UP YOUR -- >> THANK YOU, CATHERINE. I WANT TO MAKE SURE IF THERE ARE COMMENTS THAT SPEAKERS WANT TO MAKE FIRST THAT THEY HAVE A CHANCE. IT'S BEEN SUCH AN INSPIRING SUMMARY. I CANNOT THANK YOU ENOUGH FOR THE WONDERFUL OVERVIEW OF THE LAST TWO DAYS AND BLENDING YOUR THOUGHTS AND WORK, SO INSPIRING. ARE THERE ANY HANDS RAISED? >> NOT SEEING ANY AMONGST THE PANELISTS AND SPEAKERS. DON'T BE BASHFUL IF YOU HAVE A THOUGHT TO JUMP IN. >> LET ME THANK REALLY -- I THINK OVERALL I CAN SAY THAT THIS WORKSHOP MET AND EXCEEDED OUR EXPECTATIONS. WE EMBARKED IN THIS ADVENTURE AND WITH SOME TREPIDATION, REALIZING IT WAS NOT GOING TO BE A SIMPLE THING TO PUT A WORKSHOP LIKE THIS TOGETHER. I JUST WANT TO THANK SO MUCH DR. WEBER AND DR. CHEN FOR THIS JUST MASSIVE, YOU KNOW, UNDERTAKING THAT THEY DID. THEY CAME OUT SO BEAUTIFULLY. AND ALL OF THE PLANNING COMMITTEE MEMBERS, SUCH A GROUP EFFORT. AND I KEPT GETTING REPORTS AS TO HOW THE WORKSHOP WAS TAKING SHAPE. IT WAS JUST REFLECTED, THE COMPLEX OF THE SUBJECT MATTER BUT NEVERTHELESS MANAGED TO FALL INTO A WELL-ORGANIZED FRAMEWORK THAT MADE SENSE. AND SO IT WAS JUST REALLY CAME OUT BEAUTIFULLY. I REALLY ALSO WANT TO MAKE SURE TO THANK CATHERINE LAW, AND OUR WONDERFUL COMMUNICATIONS TEAM AND I.T. TEAM. MAKE SURE THESE TWO DAYS WENT BY WITHOUT A GLITCH THAT AT LEAST I WAS AWARE OF. SO THANK YOU. THAT WAS REALLY -- AND FINALLY ALL OF OUR WONDERFUL SPEAKERS. I THINK I CAN SAY FOR SURE THAT PEOPLE REALLY ROSE TO THE CHALLENGE HERE. AND THERE WAS A LOT OF -- SO MUCH OF -- FIRST OF ALL ENGAGING PRESENTATIONS AND ENGAGEMENT AS A GROUP. WHAT SEEMS TO BE EMERGING IN A WAY IS -- MANY PEOPLE COMMENTED, THAT THEY HAD SORT OF, YOU KNOW, LEARNED AND HEARD SOME THINGS FOR THE FIRST TIME. I THINK THAT BECAUSE WE PURPOSELY BY PUTTING THIS WORKSHOP TOGETHER PULLED PEOPLE FROM MANY AREAS, I THINK THERE'S BEEN HOPEFULLY CROSS-REALIZATION AS A RESULT OF THE WORKSHOP AND SO I HOPE THIS HELPS GROW A SCIENTIFIC COMMUNITY AROUND THE THEME OF WHOLE PERSON RESEARCH. AS DR. LEE SO NICELY TALKED ABOUT, GOING BACK TO THE ORIGINAL SETTING THE STAGE, THESE TWO ARROWS THAT WERE POINTING IN DIFFERENT DIRECTIONS, TO ANALYSIS AND TOWARDS SYNTHESIS, AND HOW I THINK THAT FROM WHERE I SAT AND LOOKED, THERE WERE ELEMENTS OF INTEGRATION AND SYNTHESIS IN EVERY SINGLE ONE OF THE TALKS WE HEARD AND ALL OF THESE DISCUSSIONS AND EVERYBODY WAS KIND OF TACKLING THEIR PIECE BUT I THINK WHAT EVERYBODY HAD IN COMMON IS THAT THE DETERMINATION TO TACKLE SOMETHING THAT'S COMPLEX AND NOT EASY AND REQUIRES A LOT OF EFFORT AND BUT DEFINITELY WORTH IT. IT'S REALLY PUSHING THE FRONTIER OF SCIENCE IN AN INTEGRATIVE DIRECTION. AND THERE WAS ALSO A COMMENT THAT I APPRECIATED, I THINK FROM DR. HAMMOND EARLY ON TODAY ABOUT MAKING SURE WE PUT BOUNDARIES AROUND QUESTIONS EVEN THOUGH WE ARE MOVING THE ARROW TOWARDS INTEGRATION, YOU CAN'T JUST INTEGRATE THE WHOLE SYSTEM, IT'S IMPOSSIBLE. YOU HAVE TO CHUNK IT AND GO STEP BY STEP BUT EVENTUALLY TO TRY TO COMPLETE THE PICTURE AS MUCH AS WE CAN. I THINK THAT'S WHAT WE CAN DO AS A COMMUNITY. SO, YOU KNOW, I CAN'T WAIT TO REALLY -- OUR JOB NOW IS TO KIND OF INTEGRATE EVERYTHING THAT HAPPENED AT THIS WORKSHOP AND WE'RE GOING TO BE PRODUCING A SUMMARY WHICH WE WILL PUBLISH ON OUR WEBSITE AND ALSO THE VIDEO CAYS -- VIDEOCAST WILL BE AVAILABLE IF YOU WANT TO GO BACK AND I KNOW I CERTAINLY WILL. THIS WAS REALLY A WHIRLWIND AND I REALLY CAN'T WAIT TO SEE WHAT HAPPENS NEXT. SO I WANT TO THANK EVERYONE FOR HANGING ON WITH US FOR THOSE TWO DAYS, IT WAS FASCINATING AND WISH YOU A GOOD REST OF THE DAY WHEREVER ARE YOU. HAVE A GOOD NIGHT. >> THANK YOU. A HUGE THANK YOU TO WENDY AND EVERYBODY ON THE PLANNING TEAM, THE MANY SPEAKERS, PANELISTS, EVERYBODY WHO HELPED BRING THIS ALL TOGETHER BEHIND THE SCENES AND ON THE ZOOM. WENDY AND WEN, ANY FINAL THOUGHTS THAT YOU HAVE BEFORE I DRAW THIS TO A CLOSE AND THANK THE VIEWERS FOR JOINING US? >> I THINK A COUPLE NOTES, ONE THING THAT I THINK WEN AND I TALKED ABOUT THANKING DR. LANGEVIN FOR THE NEW DIRECTION, SENIOR STAFF GIVING FEEDBACK AND PUSHING IN THE FIELDS AND INSTITUTES AND CENTERS AND PARTNERS WHO OFFERED SPEAKERS AND HELPED MODERATE AND DIFFERENT ELEMENTS, YOU MENTIONED THE AMAZING TEAM THAT CATHERINE HAS AND THE CONTRACTORS THAT WE WORKED WITH. WE HAVE A NEW STAFF MEMBER WHO ALL OF THE SPEAKERS KNOW INTIMATELY NOW, DIANE, WHO JUMPED IN ON HER FIRST DAY TO REALLY MAKE THIS HAPPEN. SO WERE THERE OTHERS YOU WANTED TO MENTION, SPECIFICALLY, WEN? >> YEAH, JUST TO SUPPLEMENT WHAT DR. WEBER HAS MENTIONED, WE ALSO WANT TO THANK OUR TERRIFIC OUTREACH TEAM, INCLUDING MANY OF THE STAFF IN THE COMMUNICATION OFFICE, AS WELL AS WITHIN THE DIVISION OF EXTRAMURAL RESEARCH, IN PARTICULAR ANITA McCURRY WILLIAMS FOR HELPING US WORKING WITH DIANE JOSS AND SERA NADA TO REACH OUT TO VERY BROAD AUDIENCE HERE AND CERTAINLY THE NCCIH I.T. TEAM AND NIH VIDEOCAST TEAM FOR THEIR IMPECCABLE TECHNICAL SUPPORT FOR THE VIDEOCAST OF THE VIRTUAL WORKSHOP AND THANK YOU TO THE SPEAKERS AND PANELISTS FOR YOUR AMAZING PRESENTATIONS AND TIMELY SUBMISSION OF ALL OUR MATERIALS, PRIOR TO THE WORKSHOP. AND I WANTED TO THANK DR. BRUCE LEE FOR YOUR TERRIFIC SYNTHESIS OF THE WORKSHOP. WITH GREAT INSIGHT A LOT OF THOUGHTS FOR US TO THINK ABOUT INTERNALLY HOW TO MOVE FORWARD. FINALLY I WANT TO THANK ALL THE VIEWERS FOR JOINING US AND SO MANY COMMENTS, GREAT COMMENTS, SUGGESTIONS, AND QUESTIONS, FOR SOME QUESTIONS HOPE TO REACH OUT TO THE INDIVIDUAL SPEAKERS WHO HAVE NOT HAVE A CHANCE TO ANSWER YOU AND OTHERS CERTAINLY WE WILL TAKE THEM SERIOUSLY AS WE CONTINUE OUR INTERNAL EFFORT. LASTLY I WANT TO THANK DR. DAVID SHURTLEFF FOR SUPPORT GUIDING DR. WEBER AND MYSELF, AS WELL AS DR. LANGEVIN TO PLAN THE WORKSHOP AND I'LL TURN IT BACK TO YOU. >> THANK YOU, WEN AND WENDY. I'M GOING TO REMIND VIEWERS THIS HAS BEEN RECORDED. IT WILL BE AVAILABLE ARCHIVED ON THE NIH VIDEOCAST WEBSITE IN THE PAST EVENTS SECTION IN A FEW DAYS TIME MORE THAN LIKELY. NOW I HAVE THE WONDERFUL TASK OF BID OF YOU GOOD EVENING AND BRINGING THIS TO A CLOSE. AND JUST IT'S BEEN A VERY RICH COUPLE DAYS. IT'S GOING TO TAKE SOME TIME TO PROCESS EVERYTHING WE'VE HEARD. BUT I HAVE ALL FAITH THAT OUR PROGRAM AND SCIENTIFIC TEAM IS JUST GOING TO EMBRACE WHAT WE'VE HEARD AND TAKE IT TO THE NEXT STEPS. THANK YOU ALL SO VERY MUCH FOR JOINING US. AND PARTICULARLY DR. LEE FOR BRINGING THE SESSION TO A CLOSE TODAY WITH SOME WONDERFUL THOUGHTS. THANK YOU SO MUCH, EVERYBODY. HAVE A GREAT EVENING.