>> GOOD MORNING. I WOULD LIKE TO WELCOME YOU ALL TO THE MIND THE GAP SERIES ORGANIZED BY THE OFFICE OF PREVENTION. OUR MIND THE GAP GAP SESSION IS EXPLORING RESEARCH AND CLINICAL PRACTICE. WE HOPE WITH THIS SERIES, WE ENGAGE THE COMMUNITY IN THOUGHT PROVOKING DISCUSSIONS, TO CHALLENGE WHAT WE THINK WE KNOW AND TO THINK CRITICALLY ABOUT OUR ROLE IN TODAY'S RESEARCH ENVIRONMENT. WE'RE WEBCAST TODAY SO WHEN YOU ASK QUESTIONS WE ASK THAT YOU PLEASE ASK THEM AT 1 OF THE MICROPHONES SO THAT THE FOLKS ON WEBCAST CAN HEAR THE QUESTIONS. FOR THOSE OF YOU VIEWING OUR WEBCAST, PLEASE FEEL FREE TO SUBMIT YOUR QUESTIONS USING TWITTER USING # NIMHTG, AND FOLLOW US AT NIH PREVENTS. WEE ALSO HAPPEN TO--WE'RE ALSO HAPPY ACCEPT QUESTIONS AT MAIL.NIH.GOV. I WOULD BE REMISS IF I DIDN'T THANK OUR CO-SPONSOR FOR TODAY'S LECTURE THE NATIONAL COUNCIL ON ALCOHOL OOH BUSES ANDICALLYISM. AND WITH THAT I WOULD LIKE TO TURN THE MIC OVER FOR ASSOCIATE DIRECTOR OF PREVENTION AND DIRECTOR OF OFFICE DISEASE PREVENTION DR. DAVID MUREON NEURONY. --MURRAY. >> THANK YOU. I'M HAPPY TO BE ABLE TO INTRODUCE OUR SPEAKER TODAY. LINDA COLLINS. DR. COLLINS IS PROFESSOR OF HUMAN DEVELOPMENT AND STUDIES AT PENN STATE UNIVERSITY, DIRECTOR OF STATISTICS AT PENN STATE UNIVERSITY. SHE'S DIRECTOR OF THE METHODOLOGY CENTER AT PENN STATE WHICH IS A WELL KNOWN GROUP THAT FOCUSES ON PREVENTION AND TREATMENT METHODOLOGY, DEVELOPMENT, SHE DIRECTS A TRAINING PROGRAM IN PREVENTION AND METHODOLOGY AND HER RESEARCH INTERESTS NEAR ANALYSIS OF LONGITUDESINAL DATA, PARTICULARLY LATENT CLASS APPROACHES, HAS A RECENT BOOK ON LATENT CLASS ANALYSIS AND SHE'S GOING TO TALK WITH US TODAY ABOUT METHODS THAT CAN BE USEFUL IN DESIGNING TRIALS DRAWN ON EXPERIENCES FROM ENGINEERING WHICH IS A DIFFERENT FIELD ALTOGETHER FROM MOST OF WHAT WE DO. I'M OFTEN ASKED, WHY DO WE ALWAYS HAVE TO DO RANDOMIZED CLINICAL TRIALS. WHY DO SWREE TO DO THESE STUDIES THE--WE HAVE TO DO THESE STUDIES THE SAME WAY ALL THE TIME. AREN'T THERE OTHER METHODS WE CAN USE? SO I'M PLEASE THAD DR. COLLINS WILL TALK ABOUT THE METHODS THAT HAVE BEEN DEVELOPED RECENTLY THAT LED US TO A BETTER JOB WITH THE CLINICAL TRIALS THAT WE DO. DR. COLLINS. >> THANK YOU. >> THANK YOU. OOSE REALLY GREAT TO BE--IT'S REALLY GREAT TO BE HERE AND I'D LIKE TO SAY I HOPE WE HAVE A LOT OF DISCUSSION SO FEEL FREE TO INTERRUPT ME WITH QUESTIONS OR COMMENTS AT ANY POINT. OKAY THE TITLES OF MY TALK TODAY IS RAISING THE BAR, OPTIMIZING AND BEHAVIORIAL INTERVENTIONS FOR IMPACT. FIRST LET'S ESTABLISH WHAT GAP WE'RE MINDING AS IF DIFFERENT 1S COULD BE MINDING, I'D LIKE TO GET A FEW DEFINITIONS SO WE'RE ALL ON THE SAME PAGE ABOUT A FEW THINGS. I'LL TALK ABOUT WHAT BUSINESS IS USUAL IN DEVELOPING BEHAVIORIAL INTERVENTIONS AND THEN I'LL TALK ABOUT HOW 1 CAN ENGINEER THESE USING THE OPTIMIZATION STRATEGY OR MOST, DISCUSS THE QUESTIONS AS OPTIMIZATION REPLACE THE EVALUATION, WE'LL TALK ABOUT 2 FUNDAMENTAL PRINCIPLES THAT UNDERLIE MOST, OKAY, SO THEN HOW DO YOU DO THIS? I'LL TALK ABOUT 2 EXAMPLES ARE THAT IN APPLICATION OF MOST. AND THEN I'LL ASK YOU TO IMAGINE A FEW THINGS WITH ME. OKAY, SO WHAT GAP ARE ME MIND SOMETHING WELL EVERY YEAR IN THE UNITED STATES--WELL, EVERY YEAR IN THE UNITED STATES, CIGARETTES CAUSE 443,000 DEATHS AND 193 BILLION DOLLARS IN HEALTHCARE LOSS AND PRODUCTIVITY AND ANOTHER 400,000 DEATHS ARE ATTRIBUTABLE TO POOR DIET AND INACTIVITY. EVERY YEAR WORLD WIDE CARDIOVASCULAR DISEASES, CANCERS CHRONIC RESPIRATORY DISEASE AND CAUSE 35 MILLION STEPS. ALL OF THESE HAVE IMPORTANT BEHAVIORIAM RISK FACTORS--BEHAVIORIAL RISK FACTORS, THE WORLD HEALTH ORGANIZATION IS EILLUMINATE IN A MINUTED THAT RISK FACTORS WOULD PREVENT UP TO 80% OF HEART DISEASE, STROKES, TYPE 2 DIABETES AND OVERONE-THIRD OF CANCERS. --OVER 1-THIRD OF CANCERS. SO BEHAVIORIAL RISK FARTORS ARE--FACTORS ARE VERY IMPORTANT. SO I'M TALKING ABOUT BEHAVIORS LIKE ALCOHOL, TOBACCO AND DRUG USE. PHYSICAL ACTIVITY AND SEDENTARY BEHAVIOR, EATING POORLY, RISKY SEXUAL BEHAVIOR, NONCOMPLIANCE OF CANCER DISEASE REGIMENS AND MANY, MANY, OTHER KINDS OF BEHAVIORS. ALL OTHER BEHAVIORS CAN BE MODIFIED BY MEANS OF BEHAVIORIAL INTERVENTIONS SO BEHAIRIAL INTERVENTIONS ARE POTENTIALLY VERY IMPORTANT. SO THE GAP FOR MINDING TODAY, IS THE GAP BETWEEN THE IMPORTANCE OF THE IMPACT OF THESE BEHAVIORIAL FACTORS ON PUBLIC HEALTH AND THE EFFECTIVENESS OF TODAY'S BEHAVIORIAL INTERVENTIONS THAT MODIFY THESE FACTORS AND YOU CAN PROBABLY GUESS THAT I BELIEVE THAT BEHAVIORIALG INTERVENTIONS--BEHAVIORIAL INTERVENTIONS COULD HAVE A MUCH, MUCH GREATER IMPACT ON PUBLIC HEALTH HEALTH THAN THEY--HEALTH THAN THEY DO TODAY. IF YOU ONLY REMEMBER 1 THING FROM OUR TIME TOGETHER TODAY, I HOPE IT'S THIS: THAT BEHAVIORIAL INTERVENTIONS CAN BE ENGINEERED TO MEET SPECIFIC CRITERIA. IF YOU DO THIS IT WILL SET THE BAR FOR ACCEPTABLE AND BEHAVIORIAL INTERVENTION. AND THEN ONCE WE SET THE BAR WE CAN CLOSE THIS GAP WE'RE TALKING ABOUT TODAY. A FEW DEFINITIONS TO MAKE SURE WE'RE ALL ON THE SAME PAGE, LET'S START BY DEFINING A BEHAVIORIAL INTERVENTION. IT'S A PROGRAM AIMED AT MODIFYING BEHAVIOR, THE PURPOSE OF TREATING OR PREVENTING DISEASE, PROMOTING HEALTH AND/OR ENHANCING WELL BEING. THAT'S THE DEFINITION OF BEHAVIORIAL INTERVENTION I'M GOING TO USE TODAY. SO THERE'S LOTS AND LOTS OF EXAMPLES, THE FEW ARE CLINIC SMOKING CESSATION INTERVENTION, WEIGHT LOSS OR WEIGHT MANAGEMENT PROGRAMS AND SCHOOL BASED DRUG ABUSE PREVENTIONS, THOSE ARE JUST MANY, MANY KINDS OF BEHAVIORIAL INTERVENTIONS. NOW, ACCORDING TO THIS DEFINITION, MOST BEHAVIORIAL INTERVENTIONS ARE PACKAGES THAT ARE MADE UP OF MANY COMPONENTS AND IF THERE ARE ANY PHYSICIANS IN THE ROOM, I THAN PHYSICIANS SOMETIME USE THE WORD INTERVENTION THE WAY I'M GOING TO BE USING THE WORD COMPONENT TODAY. SO TELL BE GOOD TO KEEP THAT IN MIND. SO, MY DEFINITION OF INTERVENTION COMPONENT, A COMPONENTS THAT MAKE UP BEHAVIORIAL INTERVENTION. ANY ASPECT OF AN INTERVENTION THAT CAN BE SEPARATED OUT FOR STUDY. SO WHAT IS A COMPONENT IS KIND OF IN THE EYE OF THE BEHOLDENER MANY WAYS. SO IT COULD BE--BEHOLDER IN MANY WAYS. SO A COMPONENT COULD BE A PART OF INTERVENTION CONTENT. SO MAYBE YOU HAVE A DRUG ABUSE PREVENTION CURRICULUM AND PART OF THE CURRICULUM IS DEVOTED TO ESTABLISHING SOCIAL NORMS, APPROPRIATE SOCIAL NORMS, PART OF IT IS DEVOTED TO RESISTANCE TRAINING, MAYBE THERE'S A COMPONENT AIMED AT PARENTS EACH OF THOSE COULD BE CONSIDERED A SEPARATE COMPONENT OF THE INTEREVICTION, A COMPONENT--INTERVENTION. A COMPONENT CAN ALSO BE A FEATURE THAT PROMOTES COMPLIANCE OR INTERVENTION AND IF IT INVOLVES TAKING MEDICATION, YOU COULD USE A MEN'S CAP TO RECORD WHEN PEOPLE ARE ACTUALLY TAKING MEDICATION OR AT LEAST WITH THEY OPEN THE BOTTLE. ALSO A COMPONENT CAN BE AIMED AND IMPROVING THE FIDELITY OF INTERVENTION. SO FOR EXAMPLE, AN AID HUNDRED NUMBER THAT PROGRAM DELIVERY STAFF CAN CALL IF THEY HAVE QUESTIONS ABOUT HOW TO DELIVER THE INTERVENTION. THESE ARE ALL EXAMPLES OF INTERVENTIONS COMPONENTS. SO INTERVENTION COMPONENTS CAN IMPACT THE EFFECTIVENESS OF AN INTERVENTION, EFFICIENCY, OR ITS ECONOMY. REALLY ALMOST ANY ASPECT OF AN INTERVENTION. SOME COMPONENTS OF BEHAVIORIAL INTERVENTION MAY BE PHARMACEUTICAL. I'LL BE SHOWING AN EXAMPLE LATER OF A SMOKING IS CESSATION INTERVENTION WHERE MIC O TEEN REPLACEMENT--NICOTINE REPLACEMENT AND A COMPONENT AND COMPONENT CANS BE DEFINED AT ANY LEVEL, MANY INTERVENTIONS ARE MULTILEVEL. SO IT CAN BE DEFINED AS THE INDIVIDUAL LEVEL, FAMILY LEVEL, SCHOOL LEVEL, ET CETERA. BUSINESS AS USUAL IN BEHAVIORIAL INTERVENTION, HOW ARE INTERVENTIONS MOSTLY DEVELOPED TODAY? TYPICALLY, THE INTERVENTION COMPONENTS ARE CHOSEN BASED ON SCIENTIFIC THEORY, CLINICAL EXPERIENCE, SOMETIMES, ANALYSIS ON DATA THAT HAVE BEEN CONDUCTED, DATA ANALYSIS CONDUCTED PRIOR, THERE'S LOTS AND LOTS OF WAYS THAT BEHAVIORIAL INTERVENTION COMPONENTS ARE SELECTED BUT TYPICALLY, THEY'RE SELECTED A PRIORI AND THEY'RE COMBINED INTO A TREATMENT PACKAGE. AND THEN, THAT TREATMENT PACKAGE IS EVALUATED BY MEANS OF A STANDARD RAPPED ORDER OF MICRONSIZE RANDOMIZED CONTROL TRIAL AND BY RANDOMIZED CONTROL TRIAL I'M TALKING ABOUT AN EXPERIMENT THAT IS INTENDED TO EVALUATE INTERVENTION OF ANY KIND. SO TYPICALLY THERE'S RANDOM ASSIGNMENT TO A TREATMENT GROUP, AND A CONTROL GROUP WHICH IS STANDARD OF CARE OR SOME OTHER CONSTITUTABLE CONTROL GROUP. TYPICALLY, THERE'S TWO EXPERIMENTAL CONDITIONS USUALLY CALLED ARMS IN RCT AND SOMETIMES THERE ARE THREE, IT'S RARE BUT IT DOES HAPPEN SOMETIMES THERE ARE MORE THAN THAT. THIS IS WHAT I'M GOING TO CALL THE TREATMENT PACKAGE APPROACH. IT'S BUSINESS AS USUAL IN DEVELOPMENT OF BEHAVIORIAL INTERVENTION. SO THIS IS KIND OF A SCHEMEATIC OF THE TREATMENT PACKAGE APPROACH. SO YOU HAVE A SELECTION OF A COMPONENT, IT COULD BE ANY NUMBER OF INTERVENTION COMPONENTS AND THEY'RE COMBINED INTO AN INTERVENTION AND THAT SERKS VALUATED BY A STANDARD RCT AS I SAID BEFORE. NOW, WHAT'S WRONG WITH EVALUATING A TREATMENT PACKAGE BY MEANS OF AN RCT? THE ANSWER IS THERE'S ABSOLUTELY NOTHING WRONG WITH THAT. THAT'S HOW YOU SHOULD EVALUATE A TREATMENT PACKAGE. THAT'S NOT WHERE MY ISSUE IS. THE RCT IS DESIGNED TO TELL US WHETHER A TREATMENT PACKAGE PERFORMS BETTER THAN A CONTROL OR A COMPARISON, OR WHETHER ONE TREATMENT PACKAGE PERFORMERS BETTER THAN ANOTHER. , MY ISSUE IS WITH WHAT HAPPENS RIGHT HERE, BETWEEN COMPONENTS AND PUTTING THEM TOGETHER FOR THE BEHAVIORIAL INTERVENTION. THE RCT DOES NOT TELL US SOME IMPORTANT THINGS. SUPPOSE YOU DO AN RCT AND YOU GET A SIGNIFICANT EFFECT. IT DOES NOT TELL YOU WHICH COMPONENTS ARE MAKING POSITIVE CONTRIBUTIONS TO THE OVERALL EFFECT. YOU DON'T KNOW WHETHER THE EFFECT IS DRIVEN, BY ONE COMPONENT OR WHETHER ALL OF THE COMPONENTS ARE CONTRIBUTING TO THE OVERALL EFFECT. YOU DON'T KNOW WHETHER THE COMPONENT'S CONTRIBUTION OFFSETTED--OFFSETS ITSELF COST. NOW THERE ARE SOME INTERVENTIONS WHERE THE COMPONENTS COST THE SAME, INTERNET INTERVENTIONS, TYPICALLY THAT'S THE CASE. THERE ARE OTHERS WHERE COMPONENTS ARE EXPENSIVE AND SOME ARE INEXPENSIVE AND MAYBE IT WOULD BE GOOD TO KNOW WHETHER THOSE COMPONENTS ARE HAVING A CORRESPONDINGLY POWERFUL EFFECT. YOU DON'T KNOW WHETHER ALL THE COMPONENTS ARE NEEDED, YOU DON'T KNOW WHETHER YOU'RE WASTING TIME, MONEY, MATERIALS, AND IN GENERAL YOU DON'T KNOW WHAT TO DO BASED ON RCT TO MAKE THE INTERVENTION BETTER, TO MAKE IT MORE EFFECTIVE AND MORE EFFICIENT AND MORE SCALABLE. IF YOU'RE RCT IT'S NOT SIGNIFICANT, THAT DOES NOT TELL YOU SOME IMPORTANT THINGS, TOO, IT DOESN'T TELL YOU WHETHER ANY OF THE COMPONENTS ARE WORTH RETAINING. IT COULD BE THAT YOU HAD A NONSIGNIFICANT EFFECT OVERALL BUT ONE OR TWO OF THE COMPONENTS IS WORTH RETAINING. YOU DON'T KNOW WHETHER ONE COMPONENT HAD A NEGATIVE EFFECT THAT OFFSET THE POSITIVE EFFECT OF SOME OF THE OTHERS AND THERE ARE SOME EXAMPLES OF THIS, NOT THERE HAVEN'T BEEN A LOT OF INTERVENTION COMPONENTS SO FAR BUT SOME META-ANALYSIS LIKE THE ONE I CITED HERE SUGGESTED CAN BE SOME NEGATIVE EFFECTS OF AN INTERVENTION COMPONENT. AND IN GENERAL, YOU DON'T KNOW SPECIFICALLY WHAT WENT WRONG AND HOW DO TO BETTER THE NEXT TIME. VERY OFTEN IT'S BACK TO THE DRAWING BOARD BUT IT'S UNCLEAR WHAT THE NEXT STEP SHOULD BE. THERE ARE TREATMENT PACKAGES THAT ARE ENCOURAGED, ONE IS STUFFING THE BEHAVIORIAL INTERVENTION TO WITH COMPONENTS TO GET A SIGNIFICANT EFFECT. IF IT'S IMPORTANT TO GET A SIGNIFICANT EFFECT IT'S FORTUNATE EVALUATE INTERVENTIONS AND GET AN EFFECT IN AN RCT AND YOU'RE NOT WORRY BODY--WORRIED ABOUT ECONOMY, THERE'S DISINCENTIVE TO ADD MORE COMPONENTS TO THE INTERVENTION AND ALL TAUGHT TO INCREASE THE POWER, YOU NEED TO INCREASE EFFECT SIZE SO THAT SEEMS LIKE A REASONABLE THING TO DO. THE ISSUE I HAVE WITH THAT IS, THINK OF ALL THE BEHAVIORIAL INTERVENTIONS THAT ARE SUPPORTED BY NIH, ALL THE SUCCESSFUL ONES, HOW MANY OF THOSE ACTUALLY GO TO SCALE IN ANY MEANINGFUL WAY. I THINK ONE REASON WHY THEY DO NOT IS BECAUSE THEY'RE TOO BIG. THEY'RE IMPRACTICAL FOR ACTUAL IMPLEMENTATION AND COMMUNITY SETTINGS. THE TREATMENT APPROACH ARE DOWN PLAYED IN EFFICIENCY AND EFFECTIVENESS AND I NEVER HEAR PEOPLE TALK ABOUT TIME EFFECTIVENESS BUT IT'S INTERESTING BECAUSE MORE AND MORE WHEN YOU TALK TO PEOPLE ABOUT SCALABILITY IN SCHOOL SETTINGS, IN HEALTHCARE DELIVERY SETTINGS, TIME IS AT A PREMIUM AND IT'S JUST NOT PRACTICAL TO IMPLEMENT AN INTERVENTION THAT TAKES A LONG TIME. AND ALSO, I BELIEVE WE'RE FOCUSING TOO MUCH ON OBTAINING STATISTICAL SIGNIFICANCE. I DON'T OF OF I WANT TO EMPHASIZE THAT I'M NOT SAYING THAT STATISTICAL SIGNIFICANCE IS UNIMPORTANT, BUT WE'RE NOT PAYING ENOUGH ATTENTION TO MEETING CLINICALLY MEANINGFUL CRITERIA AND SETTING A BAR THAT CAN BE RAISED AND THAT'S A THEME THROUGH MY TALK TODAY, SETTING A BAR, CLEAR BAR, SO WE CAN THEN KEEP RAISING THE BAR. SO WHAT'S THE ALTERNATIVE? WHAT'S THE ALTERNATIVE TO THE EFFECTIVE APPROACH? WELL, WHEN ENGINEERS BUILD PRODUCTS THEY TAKE A DIFFERENT APPROACH. ENGINEERS DO NOT USE THE TREATMENT PACKAGE APPROACH. THEY TAKE AN APPROACH THAT'S SYSTEMATIC, THAT'S EFFICIENT THAT, HOWESS RESEARCH PRODUCTS AND FOCUSED ON THE OPTIMIZING THE PRODUCT, NOT ON OBTAINING STATISTICAL SIGNIFICANCE BUT ON ON OPTICAL OPTIMIZING THE PRODUCT. SO A FEW YEARS AGO, MY COLLABORATORS AND I STARTED WONDERING, CAN WE BOREY--BORROW IDEAS FROM ENGINEERING? CAN WE BUYER O IDEAS FROM ENGINEERING AND APPLY THEM IN BEHAVIORIAL INTERVENTION AND THEN BUILD OPTIMIZED BEHAVIORIAL INTERVENTION AND THAT'S WHAT I WILL TALK ABOUT TODAY. OKAY, SO ENGINEERING BEHAVIORIAL INTERVENTION, THE MULTIPHASE OPTIMIZATION STRATEGY, MOST, WHICH IS HOW I'LL ABBREVIATE THIS IS AN ENGINEERING INSPIRED FRAMEWORK FOR DEVELOPMENT OPTIMIZATION AND EVALUATION, SO WE'RE STILL EVALUATING HERE, BEHAVIORIAL INTERVENTION. AND I WANT TO EMPHASIZE THAT THE FRAMEWORK IS NOT AN OFF THE SHELF PROCEDURE. SO BECAUSE IT'S A FRAMEWORK, IT CAN BE VERY DIFFERENT IN DIFFERENT IMPLEMENTATIONS. BUT USING MOST, CAN YOU ENGINEER BEHAVIORIAL INTERVENTION TO MEET A SPECIFIC OPTIMIZATION CRITERION OF YOUR CHOOSING. SO AGAIN, HERE'S THE SCHEMEATIC OF THE TREATMENT PACKAGE APPROACH I SHOWED YOU. A MINUTE AGO AND SO I'M SUGGESTING REMOVE TAG PART AND REPLACING IT WITH THIS. SO IN BETWEEN DEBEHAVIORIAL INTERVENTION OR PUTTING TOGETHER THE COMPONENTS OF BEHAVIORIAL INTERVENTION THERE'S IMPERICALLY BASED OPTIMIZATION. I WILL TALK MORE ABOUT WHAT I MEAN BY THAT BUT ESSENTIALLY YOU CONDUCT AN EXPERIMENT TO LOOK AT THE INDIVIDUAL EFFECT OF THE INTERVENTION COMPONENT. AFTER YOU'VE DONE THAT, YOU PUT THE COMPONENTS TOGETHER IN A WAY THAT MEETS THE OPTIMIZATION CRITERION THAT YOU SELECTED. I'M GOING TO TALK ABOUT OPTIMIZATION CRITERIA SOME MORE IN JUST A MOMENT. BUT NOTICE THAT HERE, THE BOX IS TRANSPARENT, I CHOSE THAT DELIBERATELY BECAUSE WE KNOW WHAT'S GOING ON INSIDE THE BOX AFTER WE'VE DONE THIS STUDY, WE KNOW THAT IT'S MADE UP OF COMPONENTS THAT ARE OPERATING THE WAY WE WANT THEM TO. AND THEN THERE'S EVALUATION BY STANDARD RCT. SO OPTIMIZATION, LET'S THINK ABOUT THAT FOR A MOMENT. I USE THIS DEFINITION WHICH COMES FROM THE CONCISE OXFORD DICTIONARY OF MATHEMATICS. THE PROCESS OF FINDING THE BEST POSSIBLE SOLUTION, SUBJECT TO GIVEN CONSTRAINTS. AND THE SUBJECT TO GIVEN CONSTRAINTS PART IS REALLY IMPORTANT. OPTIMIZE, DOES NOT MEAN BEST IN ABSOLUTE OR IDEAL SENSE. VERY OFTEN WHEN WE USE THE WORD OPTIMIZE CO LOCKIALLY, THAT'S WHAT WE MEAN, WHEN WE SAY OPTIMIZE, WE MEAN BEST BUT THE TERMS OF THAT IN ENGINEERING IS A LITTLE BIT DIFFERENT. I LIKE IT'S BECAUSE IT'S REALISTIC AND IT INCLUDES CONSTRAINTS. THERE ARE ALWAYS CONSTRAINTS OPERATING AND I LIKE THE IDEA OF USING OPTIMIZING AND HOW WELL, WHAT'S THE BEST WE CAN DO GIVEN A REALISTIC SET OF STRENGTHS. SO OPTIMIZATION INVOLVES THE CLEARLY STATED OPTIMIZATION CRITERION. THIS IS YOUR DEFINITION OF BEST POSSIBLE GIVEN CONSTRAINTS. IT'S A CLEAR GOAL YOU WANT TO ACHIEVE AND ONCE--ONCE THAT GOAL IS ACHIEVED, ONCE YOU'VE MET AN OPTIMIZATION CRITERION, YOU SET A BAR, AND NOW SOMEONE ELSE CAN COME ALONG AND RAISE THE BAR OR YOU CAN RAISE THE BAR YOURSELF IN YOUR NEXT ROUND OF RESEARCH. SO, LET'S TALK ABOUT A FEW EXAMPLES OF OPTICAL IMAGES IMAGES--OPTIMIZATION CRITERIA. SO MAYBE YOU ARE SIMPLY LOOKING FOR AN EFFICIENT INTERVENTION WITH NO DEAD WOOD. YOU WANT AN INTERVENTION IN WHICH THERE ARE NO INACTIVE COMPONENTS. SO CONSIDER FOR EXAMPLE, A FULL BASE OF THESE PREVENTION PROGRAMS AND SUPPOSE YOU JUST WANT TO BE CONFIDENT THAT EVERY COMPONE SENT NECESSARY SO THAT YOU FEEL THAT YOU'RE NOT WASTING TIME AND MONEY. WELL, YOU CAN ACHIEVE THIS, BY CONDUCTING THE OPTIMIZATION EXPERIMENT THAT I TALKED ABOUT A MOMENT AGO AND WE'LL GO INTO MORE DETAIL ABOUT THAT AND BASED ON THAT SELECTING,--SELECTING ONLY ACTIVE INTERVENTIONS COMPONENTS. IT'S POSSIBLE TO USE A MORE SOPHISTICATED OPTICAL IMAGES IMAGES--OPTIMIZATION CRITERIA, AND THE MOST EFFECTIVE INTERVENTION DELIVERED FOR OR LESS OR EQUAL TO SOME DOLLAR AMOUNT. SO HERE THINK ABOUT GOING TO SCALE WITH AN INTERVENTION. WE KNOW THAT VERY EXPENSIVE INTERVENTIONS ARE NOT LIKELY TO GO TO SCALE. SO ONE WAY TO APPROACH THIS IS TO FIND OUT HOW MUCH IS ASSOCIATE WILLING TO PAY, HOW MUCH ARE INSURERS WILLING TO PAY, HOW MUCH OR SCHOOL DISTRICTS WILLING TO PAY, WHOEVER IS FOOTING THE BILL, WHAT'S A REALISTIC AMOUNT OF MONEY AND THEN ENGINEERS, THE BEST INTERVENTION YOU CAN OBTAIN FOR THAT--THAT WILL NOT EXCEED THAT AMOUNT OF MONEY TO IMPLEMENT. SO CONSIDER A MOKING CESSATION INTERVENTION AND SUPPOSE AN INSURERS SAY THAT THEY'LL PAY FOR A SMOKING INTERVENTION THAT COSTS NO MORE THAN $5000 A PERSON TO DELIVERY. --$500 A PERSON TO DELIVER. SO YOU CAN ACHIEVE THIS BY CONDUCTING IMPERICAL RESEARCH THAT LOOKS AT COMPONENTS AND SELECTING I SET THAT EFFECTS THE MOST EFFECTIVE INTERVENTION YOU CAN GET FOR LESS THAN $500 A PERSON. NOW LET'S ASSUME YOU HAVE DATA ON EACH COMPONENT AND YOU CAN COLLECT THAT WHILE IT'S DOING THE EXPERIMENT.(M„ SUPPOSE YOU'RE INTERESTED IN THE MOST EFFECTIVE INTERVENTION THAT CAN BE DELIVERED IN LESS THAN OR EQUAL TO SOME AMOUNT OF TIME. SO, MAYBE YOU'RE TRYING TO DELIVER AN HIV PREVENTION INTERVENTION TO MSN IN A CLINIC SETTING AND THERE'S NOT A LOT OF TIME SO THE INTERVENTION IS GOING TO BE DELIVERED BY NURSES OR PHYSICIANS WHO HAVE ONLY A FEW MINUTES WITH A PATIENT. SO, LET'S SAY THEY GIVE YOU SIX MINUTES. THAT'S ALL WE HAVE. WELL, CAN YOU ENGINEER AN INTERVENTION THAT'S SIX MINUTES OR LESS. SO THESE ARE EXAMPLES OF OPTIMIZATION CRITERIA. COST EFFECTIVENESS COULD BE AN OMENT MYSELFATION--OPTIMIZATION CRITERIA, BUT I DON'T HAVE A LOT EXPERIENCE WITH THAT BUT IT IS POSSIBLE. YOU COULD COMBINE CRITERIA SO YOU HAVE A CRITERION BASED ON A COMBINATION OF COST AND TIME. AND CANCER TREATMENTS, VERY OFTEN THERE ARE COGNITIVE ISSUES AND YOU POETIC--POTENTIALLY OPTIMIZE A REGIMEN SO IT DIDN'T EXCEED A COGNITIVE LEVEL OF BURDEN. SO ANYTHING THAT'S RELEVANT THIS ANY PARTICULAR SITUATION CAN BE STIPULATED IN THE OPTIMIZATION CRITERION. OKAY, SO--YES? >> [INAUDIBLE QUESTION FROM AUDIENCE ] >> SO WHEN ENGINEERING SAYS A SINGLE CRITERION, THE FIRST EXAMPLE THAT YOU GAVE FOR A CERTAIN LEVEL OF EFFECTIVENESS FOR A CERTAIN DOLLAR AMOUNT, CAN YOU ALSO ENGINEER IT FOR VARYING LEVELS OF EFFECTIVENESS AND VARYING DOLLAR LEVELS AND THINKING OF A DECISION MAKER OR SOMEBODY WHO DIRECTS A PROGRAM OR A STATE OFFICE. SO YOU SAY I WANT IT LEVEL OF EFFECTIVENESS FOR THIS AMOUNT OF MONEY SO CAN THERE BE A RANGE TO BE BUILT IN THERE. >> SO I'LL TALK ABOUT THIS MORE LATER AND IF I DON'T COVER IT TO YOUR SATISFACTION, PLEASE RAISE THE QUESTION AGAIN, BUT, I WOULD ENVISION THAT, AFTER, YOU CONDUCT THE EXPERIMENT THAT YOU'RE PLANNING ON WHICH YOU'RE PLANNING TO BASE THE OPTIMIZATION, YOU COULD EVEN POST THAT--POST THE RESULTS ON THE INTERNET, YOU WOULDN'T BE POSTING INDIVIDUAL SUBJECT DATA, IT WOULD BE THE RESULTS OF ANALYSIS OF VARIANTS ALONG WITH ANY INFORMATION ABOUT COST THAT YOU MIGHT HAVE, COLLECTED ALTHOUGH PEOPLE MAY KNOW WHAT WHAT'S BETTER WITH THE COSTS IN THEIR OWN LOCALE AND IT WOULD BE POSSIBLE FOR PEOPLE TO OPTIMIZE THEM TO A DIFFERENT OPTIMIZATION CRITERIA. YEAH. THAT WOULD BE KIND OF A NEW LEVEL OF TRANSPARENCY IN INTERVENTION SCIENCE THAT I THINK WOULD BE REALLY HEALTHY. >> HI, RICK, WITH THE NATIONAL INSTITUTE ON MINORITY HEALTH AND HEALTH DISPARITIES, I JUST HAVE TWO QUICK QUESTIONS ABOUT YOUR SUPPOSITIONS, ONE IS, IF YOU'RE GOING TO MAKE A SELECTION OF OF AN INTERVENTION AND YOU'RE GOING TO ENGINEER IT BASED ON WHAT WORKS, I DON'T UNDERSTAND HOW YOU GOT TO THE POINT OF KNOWING WHAT WORKS? >> I HAVEN'T GOTTEN TO THAT YET. I WILL. >> OH, OKAY, THAT'S GREAT. AND THEN THE OTHER QUESTION I HAD WAS IT SOUNDED LIKE IN THE EXAMPLE THAT YOU USED, YOU'RE BASING ON A WILLINGNESS TO PAY MODEL WHICH IS OFFERED BY THE SPONSOR, BUT, THERE'S A--KIND OF A CONFLICT THERE, ISN'T THERE? IF YOU'RE ASKING THE SPONSOR TO BASE A SEDITION ON WILLINGNESS TO PAY, THAT DOESN'T NECESSARILY MEAN THAT THE INTERVENTION IS GOING TO WORK, IT JUST MEANS THAT YOU'RE ASKING THE SPONSOR TO BASE IT ON RESOURCES AND THAT BASING IT ON RESOURCES THAT HAVEN'T NECESSARILY BEEN PROVEN EFFECTIVE OR EVEN EFFICIENT DOESN'T SOUND LIKE--I DON'T KNOW HOW YOU GOT TO THAT--USING THAT CRITERIA. >> YEAH, OKAY. I'M NOT SURE I UNDERSTOOD THE LAST QUESTION, AND LET ME GO OVER SOMETHING THAT MIGHT ANSWER YOUR QUESTION YOU ABOUT IF IT DOESN'T, I WILL GO BACK TO YOU. >> THAL IS WHERE YOU GET THE INFORMATION ABOUT WHAT WORKS AND YOU CONDUCT AN EXPERIMENT HERE, AND I WILL TALK ABOUT THAT IN A MOMENT AND AFTER YOU PUT TOGETHER THE INTERVENTION, YOU THEN WOULD EVALUATE IT BY MEANS OF AN RCT, SO THAT WOULD TELL YOU WHETHER IT'S A STATISTICALLY SIGNIFICANT EFFECT? IS THAT WHAT YOU MEAN BY WORK? SO YOU'RE SORT OF RANDOMLY SELECTING THE OPTIMIZATION INTERVENTION? >> NO YOU'RE CONDUCTING--THIS MAY BE CLEARER IN A MOMENT. BUT YOU CONDUCT THE RANDOMIZED EXPERIMENT HERE AND BASEDDED ON THE RESULTS OF THE--BASED ON THE RESULTS OF THE EXPERIMENT, YOU WILL GET COMPONENTS ON THE INDIVIDUAL DATA. >> SO YOU START WITH ALL OF THEM IN THE FIRST PLACE? >> OH YES, YES, YES. YOU START WITH ALL OF THEM AND YOU LOOK AT MEANS BASED ON EXPERIMENT AND THEN BASED ON EXPERIMENT YOU GET IT FOR EACH COMPONENTS, AND COMPONENTS INTERACT WITH EACH OTHER AND THEN BASED ON THAT INFORMATION, CAN FLIP THEM TOGETHER TO MEET THE OPTIMIZATION CRITERIA. >> OKAY. >> OKAY, I'M GOING TO GIVE A COUPLE OF EXAMPLES OF THIS, IF IT DOESN'T CLEAR IT UP FOR YOU, WE CAN COME BACK TO IT A LITTLE LATER. >> ANY OTHER QUESTIONS BEFORE I GO ON, THOSE ARE GREAT QUESTIONS. >> THANKS. YOU KNOW I WAS CURIOUS ABOUT HOW YOU DEAL--I'M UP HERE. >> OH ACCIDENT--OH, OKAY. >> I'M CURIOUS ABOUT EFFICIENCY. SO WE WANT TO HAVE EFFICIENCY AND IT SEEMS LIKE WHEN YOU DEVELOP AN INTERVENTION, YOU'RE DOING IT IN THE MOST EFFECTIVE WAY, IT'S ALMOST ANALOGOUS TO THE PHARMACEUTICAL INDUSTRY, SO BUILDING A BILL IS LIKE BUILDING A NEW BRIDGE, AND AFTER THAT, YOU JUCT COLLECT TOLES WHEN PEOPLE COME ACROSS, BUILDING A NEW PILL IS LIKE THAT, YOU REACH A BIG MARKET AND YOU CAN SELL A PILL AT A MUCH LOWER RATE. ISN'T THAT SO WITH BEHAVIORIAL INTERVENTION? SO IF YOU'RE DO THANKSGIVING BY COST, YOU'RE OPTIMIZING BY COST, YOU PUT ALL THIS EFFORT INTO GETTING IT RIGHT, WHEN IT GOES OUT TO CLINICAL PRACTICE WE WOULD HOPE IT WOULD BE MUCH MORE EFFICIENT THAN THAT? >> YEAH, THAT'S AN INTERESTING POINT ABOUT WHETHER INTERVENTIONS GET MORE EFFICIENT WHEN THEY GO OUT INTO CLINICAL PRACTICE. I'M NOT SURE THAT'S TRUE, ACTUALLY BUT I'D BE INTERESTED TO HEAR WHAT OTHER PEOPLE THINK OR HAVE MORE ON THE GROUND EXPERIENCE OF BEHAVIORIAL INTERVENTIONS THAN I DO. I SORT OF WIDELY THOUGHT THAT MOST INTERVENTIONS BECOME LESS EFFECTIVE WHEN THEY GO OUT IN THE FIELD SO THAT'S THAT EFFICACY EFFECTIVENESS, DIFFERENTIAL. AND I GUESS, I WOULD CALL THAT LESS EFFICIENT IF THEY'RE BEING LESS EFFECTIVE. >> I USED TO WORK IN STUDIES WHERE THEY DID COST EFFECTIVENESS IN BEHAVIORIAL INTERVENTIONSVENTIONS AND WHAT HAPPENS IS WHEN YOU COST OUT THE BEHAVIORIAL INTERVENTION IT'S MUCH MORE EXPENSE JUST BECAUSE YOU HAVE STAFF TIME. >> OH, I SEE. SO YOU'RE SAYING THE COST ESTIMATES MAY NOT BE RIGHT. YES, THAT IS PROBABLY TRUE. AND I GUESS I'VE A COUPLE OF COMMENTS TO MAKE ABOUT THAT. ONE IS IF YOU'RE THINKING IN RELATIVE TERMS, AT LEAST YOU WOULD--YOU WOULD PROBABLY KNOW IT IS RELATIVE COST OF THE COMPONENT. BUT YOU'RE PROBABLY RIGHT THAT DATA THAT YOU COLLECT AS YOU GO, MIGHT OR MIGHT NOT BE VALID ABOUT THE COST. SO, THERE MIGHT BE OTHER WAYS TO DO IT FOR EXAMPLE, TALKING TO PEOPLE IN THE SETTINGS IN WHICH IT'S GOING TO BE DELIVERED AND COLLECTING COST DATA THAT WAY. THEY MIGHT HAVE TO HAVE A MUCH BETTER WAY OF COSTING IT OUT. THAT'S A REALLY GOOD POINT. THANKS, BOB. >> OKAY, THERE'S OPTIMIZATION--DOES OPTIMIZATION REPLACE EVALUATION? I WANT TO MAKE A POINT HERE THAT OPTIMIZATION AND EVALUATION ARE DIFFERENT. THEY ANSWER DIFFERENT QUESTIONS. SO, WE'RE TYPICALLY USED TO THINKING IN TERMS OF EVALUATION. THIS IS A QUESTION THAT THE RCT IS MEANT TO ADDRESS. IS THE INTERVENTION EFFECTS STATISTICALLY SIGNIFICANT. WE DEFINITELY WANT INTERVENTION BUT HAVE A DEMONSTRATED SPECIFICALLY SIGNIFICANT EFFECT OPTIMIZATION IS A DIFFERENT QUESTION. THE QUESTIONS THAT OPTIMIZATIONS POSES IS, IS THE INTERVENTION THE BEST POSSIBLE GIVEN CONSTRAINT AND SO, I PRESENTED THIS TWO BY TWO TABLE SO CAN YOU SEE THAT THERE IS ACTUALLY POSSIBLE THAT YOU COULD HAVE AN INTERVENTION THAT FALLS IN ANY OF THESE FOUR CELLS. SO FOR EXAMPLE, MAYBE THE INTERVENTIONS EFFECT IS NOT STATISTICALLY SIGNIFICANT BUT YOU ALSO HAVE NOT OPTIMIZED THE INTERVENTION, SO MAYBE YOU WANT TO TRY OPTIMIZING THE INTERVENTION, CAN YOU EVEN USE THE EFFECT SIZE AS A CRITERION AND WE CAN TALK ABOUT THAT MORE HOW YOU WOULD DO THAT IF YOU ARE INTERESTED LATER. MAYBE THE EFFECT IS NOT STATISTICALLY SIGNIFICANT, BUT YOU HAVE OPTIMIZED THE INTERVENTION, SO IT'S POSSIBLE TO HAVE AN OPTIMIZED INTERVENTION THAT STILL DOES NOT DEMONSTRATE A STATISTICALLY SIGNIFICANT EFFECT AND THAT TO ME SUGGESTS YOU MIGHT MEET AN INTERLY INTERVENTION STRATEGY. THIS IS WHERE I THINK WE ARE WITH MOST OF OUR EVIDENCE BASED ABOUT HAIRIAL INTERVENTIONS TODAY. --THEY'VE SHOWN A STATISTICALLY SIGNIFICANT EFFECT BUT NOT OPTIMIZED SO THEY COULD BE MADE BETTER, MORE IESKTIVE, MORE EFFICIENT, MORE COST OOSKTIVE AND A LOT OF THINGS THAT COULD BE DONE. THIS IS WHAT WE'RE AIMING FOR, BEHAVIORIAL INTERVENTIONS THAT BOTH HAVE BEEN OPTIMIZED AND DEMONSTRATED TO HAVE A STATISTICALLY SIGNIFICANT EFFECT. SO THAT'S THE DIFFERENCE BETWEEN OPTIMIZATION AND EVALUATION. TWO FUNDAMENTAL PRINCIPLES. THERE ARE TWO FUNDAMENTAL PRINCIPLES THAT UNDERLIE THE MOST. THE FIRST IS THE RESOURCE MANAGEMENT PRINCIPLE, IT SAYS THAT, WE WANT TO CONDUCT OUR RESEARCH IN SUCH A WAY THAT WE GET THE MOST SCIENTIFIC INFORMATION THAT WE CAN FOR THE AMOUNT OF MONEY THAT WE HAVE TO CONDUCT RESEARCH, SO THE IDEA HERE IS, YOU IDENTIFY CLEARLY, THIS IS WHAT I NEED TO FIND OUT, THESE ARE THE RESOURCES I HAVE OR CAN REASONABLY OBTAIN AND NOW, HOW CAN I MANAGE MY RESOURCES STRATEGICALLY TO GET THE MOST INFORMATION AND THE MOST RELEVANT INFORMATION GIVEN THE RESOURCES THAT I HAVE. AND YOU'LL SEE, AS I GO ON, THAT THAT'S SOMETIMES SUGGESTS, SOMEWHAT DIFFERENT APPROACHES TO EXPERIMENTAL DESIGNS THAN WHAT WE MIGHT BE ACCUSTOMED TOO. THE SECOND IS THE CONTINUOUS OPTIMIZATION PRINCIPLE. NO INTERVENTION IS EVER JUST PERMANENTLY OPTIMIZED. WHEN YOU THINK ABOUT CONSUMER PRODUCTS, OVER THE YEARS, THEY MOSTLY GET BETTER, RIGHT? SO THE CARS OF TODAY ARE TYPICALLY MORE FUEL EFFICIENT THAN CARS WERE 20 YEARS AGO. MOST OF THE TIME IF YOUR VACUUM CLEANER BREAK US AND GO AND GET A NEW ONE, IT WILL HAVE MORE FEATURES, MORE ERGNATIONAL LIBRARY OF MEDICINIC AND SO ON. SO ENGINEERS ARE ALWAYS REVISING PRODUCTS TO MAKE THEM BETTER AND I BELIEVE WE SHOULD BE DOING THE SAME THING WITH OUR BEHAVIORIAL INTERVENTIONS. YOU CAN ALSO MAKE AN INTERVENTION BETTER. YOU CAN MAKE IT MORE EFFECTIVE, YOU CAN TRY AND KEEP THE OOSKTIVENESS ABOUT WHERE IT IS BUT MAKE IT CHEAPER OR MORE EFFICIENT. THERE'S ALWAYS THINGS YOU SHOULD DO. THERE'S A TENDENCY NOW WHEN WE BRING AN INTERVENTION TO A RANDOMIZED CLINICAL TRIAL AND IT SHOWS A SIGNIFICANT EFFECT, WE DON'T WANT TO MESS WITH IT, LIKE THAT'S THE INTERVINGZ, THAT'S THE WAY IT IS AND I BELIEVE WE SHOULD ALWAYS BE WORKING ON REVISING BEHAVIORIAL INTERVENTIONS, MAKING THEM MORE MODERN AND MORE EFFECTIVE. SO THOSE ARE THE TWO FUNDAMENTAL PRINCIPLES AND YOU'LL SEE THEM IN ACTION AS I G. SO HOW DO YOU DO THIS? SO LET ME SHOW YOU A COUPLE OF IN-PROGRESS EXAMPLES THAT HAVE BEEN FUNDED BY NIH. THE FIRST I THINK YOU WILL RECOGNIZE, BONNIE SPRING, SHE'S AN FANTASTIC INTERVENTION SCIENTISTS, I FEEL LUCK TOW WORK WITH HER. THE OBJECTIVE IS TO DEVELOP A HIGHLY EFFECTIVE WEIGHT REDUCTION ISHT VENTION AIMED AT ADULTS CALLED OPTICAL IMAGES-IN. IT'S FUNDED BY NIDDK, IT WAS FUNDED QUITE RECENTLY. HERE'S A THEORETICAL MODEL. THERE'S A CORE EDUCATION PROGRAM THAT ACTUALLY HAS BEEN DEMONSTRATED TO WORK ALREADY, MADE UP OF EDUCATION, GOAL-SETTING, SKILL-BUILDING AND SOME TECHNOLOGY ALTHOUGH TOOLS. WE ARE LOOKING AT COMPONENTS THAT ARE ULTIMATELY GOING TO EFFECT ADHERENCE TO THE INTERVENTION AND HOPE. AND THOSE COMPONENTS ARE PHONE COACHING, THIS IS A NUMBER OF SESSIONS, 12 VERSES 24 SESSIONS WE'RE LOOKING AT. TEXT MESSAGES, WHETHER THERE'S COMMUNICATION WITH THE PRIMARY CARE PHYSICIAN, EVERY--SEVEN GOING TO HAVE A BUDDY. ALL THE PEOPLE TAKING PART IN THE INTERVENTION IS GOING TO HAVE A NAMED BUDDY BUT THE QUESTION HERE IS WHETHER THEY SHOULD HAVE BUDDY TRAINING OR NOT, THE BUDDY SHOULD BE BE TRAINED AND WHETHER MEAL REPLACEMENTS ARE PROVIDED OR NOT. THESE FACTORS OR COMPONENTS ARE AIMED AT SELF-EFFICACY, SELF-REGULATION, ACCOUNTABILITY AND SUPPORTIVE AND FACILITATION AND WE BELIEVE THESE WILL ULTIMATELY LEAD TO WEIGHT LOSS. YES? >> [INAUDIBLE QUESTION FROM AUDIENCE ] >> YEAH. >> YEAH THE QUESTION WAS CAN YOU EXPLAIN SUPPORTIVE ACCOUNTABILITY? IT'S A NEAT CONCEPT, THE IDEA IS THAT THERE'S SOMEONE YOU REPORT TO AND IT'S TYPICALLY--TYPICALLY THE COACH, BUT THE BUDDY SERVES THIS ROLE A LITTLE BIT, TOO, SO IT'S SOMEONE THAT YOU GO TO AND YOU TELL THEM WHAT YOUR WEIGHT IS AND OF COURSE YOU ARE MAINTAINING OR LOSING WEIGHT. SO THERE IS ACCOUNTABILITY THERE BUT THE PERSON IS ALWAYS SUPPORTIVE BUT IT'S NOT A PUNITIVE ARRANGEMENT, THE PERSON DOESN'T MAKE YOU PAY A FINE OR YELL AT YOU OR SOMETHING IF YOU ACTUALLY ARE LOSING WEIGHT. INSTEAD THEY'RE SUPPORTIVE AND THEY KEEP TRYING TO GET YOU TO DO AS WELL OR BETTER. THAT'S THE IDEA. OKAY SO THIS IS MOST AS IMPLEMENTED IN OPTICAL IMAGES-IN. THESE ARE THE FIVE COMPONENTS I SHOWED YOU A MINUTE AGO. WE DO A COMPONENT SCREENING EXPERIMENT THAT I WILL DESCRIBE IN JUST A MOMENT. THE OPTIMIZATION CRITERION IS THE MOST EFFECTIVE INTERVENTION WE CAN GET THAT WILL--WE ESTIMATE WILL COST $500 OR LESS TO DELIVER PER PERSON. ONCE WE OPTIMIZE THE INTERVENTION IS GET THE OPTIMIZED TREATMENT PACKAGE, EVENTUALLY TELL BE EVALUATED IN THE RCT. OKAY. SO NOW, I WANT TO TALK ABOUT THIS, THE COMPONENTS SCREENING EXPERIMENT BECAUSE THAT IS--YES? >> LINDA, SINCE THIS IS A REAL EXAMPLE IN A STUDY THAT'S UNDERWAY, CAN YOU JUST FILL US IN ON THE MECHANISM THAT WAS USED TO FUND IT? IS IT AN RO-1? IS IT-- >> IT'S AN RO-1 FROM NIDDK. >> DOES THE RO-1 GO THROUGH THE ENTIRE SCHEMA HERE OR WHERE--OR WHAT PART OF THIS DOES IT COVER? >> IT COVERS UP TO THE OPTIMIZATION, IT DOES NOT INCLUDE THE RCT. YEAH, THE PLAN IS TO APPLY FOR FUNDING TO DO THE RCT. OTHER QUESTIONS BEFORE I GO ON? >> OKAY, SO COMPONENT SCREENING EXPERIMENT, THIS IS IMPORTANT FOR OPTIMIZATION, THE IDEA HERE IS TO SCREEN INTERVENTION COMPONENTSENTIOUS FICIENTLY. WE'RE TALKING ABOUT WEEDING OUT THE COMPONENTS THAT ARE UNDERPERFORMING OR NOT PERFORMING, WE WANT TO GET A SENSE OF THE MAGNITUDE OF EACH COMPONENTS EFFECT AND WE WANT TO EXAMINE WHETHER THE EFFECTS OF A COMPONENT MAY BE AUGMENTED OR REDUCE INDEED THE PRESENCE OF ANOTHER COMPONENT, IN OTHER WORDS WHETHER THERE'S INTERACTIONS BETWEEN THE COMPONE SPENT THIS THERAPIST IS INFORMATION WE USE TO OPTIMIZE THE INTERVENTION. SO ANY EXPERIMENTAL DESIGN IS A POSSIBILITY BUT THE CHOICE OF DESIGN IS REALLY CRITICAL. THE ONLY ENVIRONMENT IN MOST IS THAT THIS EXPERIMENTAL DESIGN BE SELECTED BASED ON THE RESOURCE MANAGEMENT PRINCIPLE. SO IT HAS TO BE A DESIGN THAT GIVES YOU THE MOST INFORMATION YOU GET GIVEN THE AMOUNT OF MONEY YOU HAVE TO SPEND ON IT. SO THE IDEA IS TO SELECT A DESIGN. YOU WANT TO USUALLY CONSIDER SEVERAL AND EXAMINE THE SPECIFIC INFORMATION YOU WOULD GET FROM EACH ONE AND WHETHER OF COURSE IT'S THE INFORMATION YOU WANT AND THEN WHAT EACH DESIGN COSTS AND THE COSTS USUALLY REVOLVE AROUND THE NUMBER OF SUBJECTS, THAT'S A BIG DRIVING FORCE IN COST AND A NUMBER OF EXPERIMENTAL CONDITIONS AND I WANT TO POINT OUT, THE STARTING POINT IS THE RESOURCES YOU HAVE OR CAN REASONABLY GET. SO SOMETIMES PEOPLE ASK ME, ISN'T MOST PROHIBITIVELY EXPENSIVE? WELL, IT DOESN'T. BUT IT ISN'T BY DEFINITION BECAUSE THE IDEA HERE IS TO TAKE A REASONABLE AMOUNT OF RESOURCES AND GET THE MOST SCIENTIFIC INFORMATION THAT YOU CAN OUT OF THOSE RESOURCES. SO I'M GOING TO GO THROUGH A FEW EXPERIMENTAL DESIGN POSSIBILITIES HERE. SO ONE POSSIBILITY WOULD BE TO CONDUCT AN EXPERIMENT ON EACH COMPONENT SO THIS WOULD BE ON THIS CASE, FIVE SEPARATE EXPERIMENTS, KIND OF LIKE FIVE LITTLE RCTs. ANOTHER APPROACH IN THIS DESIGN IS USED A LOT, THIS IS OFTEN CALLED A COMPARATIVE TREATMENT EXPERIMENT SO IT'S LIKE RUNNING FIVE INDIVIDUAL EXPERIMENTS EXCEPT THERE'S ONE SHARED CONTROL GROUP. SO YOU WOULD COMPARE 24 COACHING--THERE WOULD BE A CONDITION WITH 24 COACHING SESSIONS ALL THE OTHERS ARE SET TO LOW, COMPARE THAT TO A CONTROL GROUP, TEXT MESSAGES ARE ON ALL THE OTHER COMPONENTS ARE SET TO LOW OR OFF, COMPARE THAT TO THE CONTROL GROUP AND SO ON. SO THERE WOULD BE IN THIS CASE, EXPERIMENTAL CONDITIONS. SO IF YOU COMPARE THESE TO DESIGN POSSIBILITIES, YOU SEE THAT THERE'S SOME DIG DIFFERENCES IN COST. 2800 SUBJECTS TO GET THE POWER THAT YOU WANT, VERSES ABOUT 1700 HERE IN THE COMPARATIVE TREATMENT EXPERIMENT THIS INVOLVES FEWER EXPERIMENTAL CONDITIONS. JOHN DO HAVE YOU A QUESTION? >> YEAH, I'M GETTING CURIOUS FOR ANY ONE COMPONENT HOW WOULD YOU TREAT A PHENOTYPIC INTERACTION TERM? I. E. ONE CONDITION WILL WORK BEST FOR RED HEDS,--HEADS, THE OTHER FOR BLONDS. >> THAT'S A GREAT QUESTION AND THE ISSUES ARE ARE THE STANDARD IN AN RCT. SO WHEN I TALK ABOUT INTERACTIONS HERE, I'M TALKING ABOUT INTERACTIONS BETWEEN THE COMPONENTS WE'RE LOOKING AT BUT OF COURSE, IT OFTEN IS ALSO IMPORTANT TO LOOK AT INTERACTIONS BETWEEN CHARACTERISTICS OF THE INDIVIDUAL AND PERFORMANCE OF THE INTERVENTION COMPONENTS. THERE'S A COUPLE DIFFERENT WAYS CAN YOU HANDLE THAT. ONE IS CAN DO EXPLORATORY ANALYSIS THAT EXAMINE THOSE INTERACTIONs, NOW SOMETIMES THOSE ARBITRATE WELL POWERED LIKE MANY EXPLORATORY ANALYSIS BUT THEY CAN BE INFORMATIVE AND AN ADVANTAGE OF--I EVAPORATE GOTTEN TO THE WINNING DESIGN, STUDY YET BUT THE ADVANTAGE TO DESIGNS THAT ENABLE YOU TO LOOK AT THE EFFECTS OF INDIVIDUAL ISHT VENTION COMPONENTS IS YOU CAN LOOK AT INTERABSS FOR THE INDIVIDUAL AND THE INDIVIDUAL INTERVENTIONS SO THAT CAN BE VERY REVEALING. IF IT YOU KNOW A PRIORI THAT THERE ARE GOING TO BE IMPORTANT INTERACTIONS BETWEEN HAIR COLOR, AS YOU SAID AND SOME INTERVENTION COMPONENTS, CAN YOU BUILD THAT INTO THE DESIGN. THAT CAN BE EXPENSIVE, BUT IT DEFINITELY WOULD GIVE YOU THE MOST POWER. DOES THAT ANSWER YOUR QUESTION. OKAY THE THIRD POSSIBILITY IS FACTORIAL EXPERIMENTS, AND FOR THOSE WHO HAVEN'T THOUGHT ABOUT IT, THERE'S 101 FACTORIAL EXPERIMENT. SO SUPPOSE YOU HAD A TWO X TWO FACTORIAL DESIGN, ONE COMPONENT CAN BE EITHER OFF OR ON COMPONENT A. SECOND CAN BE OFF OR ON AND THERE'S FOUR EXPERIMENTAL CONDITIONS, ONE WHERE THEY'RE BOTH OFF, ONE WHERE THEY'RE BOTH ON AND TWO CONDITIONS WHERE ONE IS OFF AND ONE IS ON. WE'RE INTERESTED HERE PRIMARILY IN MAIN EFFECTS ALTHOUGH INTERACTIONS ARE IMPORTANT, TOO, IF YOU WANT TO LOOK AT THE MAIN EFFECT OF A, YOU WOULD COMPARE THE AVERAGE OF THESE TWO CONDITIONS TO THE AVERAGE OF THOSE TWO CONDITIONS, THE AVERAGE OF THE CONDITIONS WHERE COMPONENT A IS OFF, THE AVERAGE CONDITIONS WERE COME PONENT A IS ON, SIMILARLY IF YOU'RE INTEREST INDEED LOOKING AT THE MAIN EFFECTIVE COMPONENT B, IT WOULD BE THE AVERAGE OF THE TWO CONDITIONS, WOULD BE AS OFF, COMPARED TO THE AVERAGE OF THE TWO CONDITIONS WHERE B IS ON. SO SOMETHING VERY IMPORTANT HERE, I WANT TO POINT OUT, AND A FACTORIAL EXPERIMENT, ALL OF THE SUBJECTS ARE INVOLVED IN ALL THE EFFECT ESTIMATES, SO YOU NOTICE HERE, THERE'S A MAIN EFFECT OF A THAT INVOLVED ALL SUBJECTS IN ALL FOUR CONDITIONS. MAIN EFFECT WOULD BE ALSO INVOLVED, ALL SUBJECTS, ALL FOUR CONDITIONS THERE,'S JUST RESUFLE. THERE'S JUST RESHUFFLE. FACTORIAL EXPERIMENTS CAN HAVE MORE THAN TWO FACTORS OF COURSE AND I'M GOING TOXIC EFFECTS SHOW YOU SOME IN A MINUTE. AND THERE'S ALSO MORE THAN TWO LEVELS PER FACTOR. THOSE DESIGNS CAN BE EXPENSIVE AND I USUALLY DON'T RECOMMEND THEM FOR COMPONENT SCREENING EXPERIMENTS, WE CAN TALK ABOUT THAT MORE IF YOU'RE INTERESTED. SO THE OPTICAL IMAGES-IN STUDY IF WE CHOSE TO USE THAT STUDY IF WE USE THAT DESIGN WOULD BE A TWO BY TWO BY TWO BY TWO, FACTORIAL EXPERIMENT. IT WOULD HAVE 32 EXPERIMENT CONDITIONS. HOW COULD THAT EVER BE POWERED? SOME OF YOU MAY BE THINKING? I WILL EXPLAIN THAT IN JUST A MOMENT. BUT, I DO WANT TO POINT OUT, AND I'LL EXPLAIN MORE ABOUT THIS, THAT IT WOULD TAKE AN N-560 TO POWER THAT EXPERIMENT EVEN THOUGH INDIVIDUAL EXPERIMENTS WOULD TAKE 2800 SUBJECTS AND THE COMPARATIVE TREATMENT EXPERIMENT WOULD TAKE ALMOST 1700 SUBJECTS SO AT LEAST IN TERMS OF SUBJECTS, THIS FACTORIAL EXPERIMENT IS LOOKING FOR EFFICIENT. 32 EXPERIMENTAL CONDITIONS THOUGH, WE FELT WE COULD HANDLE 16 CONDITIONS IN THE FIELD, THAT WAS ABOUT IT. SO IS THERE A POSSIBILITY HERE? THERE IS A FOURTH TYPE OF EXPERIMENTAL DESIGN, IT'S ACTUALLY A SPECIAL CASE OF THE FACTORIAL EXPERIMENT, CALLED A FRACTIONAL FACTORIAL EXPERIMENT. IN A FRACTIONAL FRACTORRIAL EXPERIMENT, A FRACTION OF THE EXPERIMENTAL CONDITIONS ARE RUN. IT COULD BE HALF OF THE CONDITIONS, QUARTER OF THE CONDITIONS AND SO ON. FRACTIONAL FACTORIAL EXPERIMENTS ARE POWERED THE SAME WAY AS ORDINARY EXPERIMENTS BUT, THEY INVOLVE FEWER EXPERIMENTAL CONDITIONS, BUT THERE'S AN IMPORTANT TRADE OFF THAT WE'LL DISCUSS. BUT NOW, LET'S LOOK AT, LET'S COMPARE ALL FOUR OF THESE POSSIBILITIES, THE TWO FACTORIAL EXPERIMENTS REQUIRE THE FEWEST SUBJECTS BY FAR, AND ALTHOUGH 16 CONDITIONS IS A LOT, IT'S HALF OF 32 EXPERIMENTAL CONDITIONS SO WE DECIDED WE WOULD GO WITH FRACTIONAL FACTORIAL EXPERIMENT. NOTICE THAT WITH INDIVIDUAL EXPERIMENTS AND THE COMPARATIVE TREATMENT APPROACH, YOU CANNOT EXAMINE INTERACTION. YOU CAN EXAMINE ALL INTERACTIONS WITH THE FULL FACTORIAL EXPERIMENT, WITH THE FRACTIONAL FACTORIAL EXPERIMENT YOU CAN LOOK AT COLLECTING INTERACTIONS. IS THAT YOU DENISE? >> YES, IT IS. I'M JUST WONDERING WHEN YOU'RE TALKING ABOUT POWER DURING THIS WHOLE THING, WHAT OUTCOME YOU'RE TALKING ABOUT? THIS IS A WEIGHT LOSS EFFECTIVELY STUDY BUT ARE YOU TALKING AT WEIGHT LOSS OR INTERMEDIATE OUTCOME. >> IT'S WEIGHT LOSS HERE. >> AND THE END THAT YOU'RE TALKING ABOUT IS THE NOTAL NUMBER OF PARTICIPANTS. >> TOTAL NUMBER OF PARTICIPANTS. >> IN THE WHOLE SUBJECT. >> IN THE WHOLE SUBJECT. >> WHOLE EXPERIMENT. >> NOT PER CONDITION. >> DEFINITELY NOT, IT'S THE ACROSS ALL CONDITIONS. >> OKAY, SO THIS IS DESIGN FOR OPTICAL IMAGES IN. EVERYONE GETS THE CORE INTERVENTION ON THE SLIDE I SHOWED YOU BEFORE AND THEN THESE FACTORS ARE VARIED. NUMBER OF PHONE COACHING SESSIONS, 12 OR 24, COMMUNICATION WITH A PRIMARY CARE PHYSICIAN, YES OR NO AND SO ON! SO THE OVERALL N AS I SAID IS 560. THAT'S ABOUT 35 PER CONDITION. SO HOW CAN THAT BE SUFFICIENTLY POWERED? LET'S LOOK AT THE MAIN EFFECT OF NUMBER OF PHONE COACHING SESSIONS. YOU WOULD GET THAT BY COMPARING THE MEAN OF CONDITIONS ONE THROUGH EIGHT VERSES THE MEAN OF CONDITIONS NINE-16 IT'S BASED ON ALL 560 SUBJECTS. NOW LET'S LOOK AT THE MAIN EFFECT OF PC P COMICATION, THE MEAN OF ONE-FOUR AND NINE-12 AND COMPARED TO FIVE-EIGHT AND 13-16, AGAIN BASED ON A TOTAL N-550 AND IT'S THE SAME FOR ALL FIVE MAIN EFFECTS AND IT'S ACTUALLY THE SAME FOR ANY INTERACTION YOU LOOK AT AS WELL. THIS IS WHY, FISHER INVENTED THE FACTORIAL EXPERIMENT AND THE ANALYSIS OF VARIANT. THIS IS WHY. BECAUSE FACTORIAL EXPERIMENTS ARE VERY, VERY ECONOMICAL IN TERMS OF USE OF EXPERIMENTAL SUBJECTS. AND AS I HOPE I SHOWED YOU, OR CONVINCED YOU YOU A MINUTE AGO, MUCH MORE ECONOMICAL THAN DOING INDIVIDUAL EXPERIMENTS ON THE INDIVIDUAL FACTORS. MUCH, MUCH MORE ECONOMICAL AT LEAST IN EMERGINGS OF SUBJECTS. --IN TERMS OF SUBJECTS. BUT THERE'S NO FREE LUNCHES IN STATISTICS, EVER. SO THERE ARE SOME TRADE-OFFS WHEN YOU CHOOSE TO USE A FACTORIAL EXPERIMENT, FRACTIONAL FACTORIAL EXPERIMENT AS OPPOSE TO FACTORIAL EXPERIMENT. SO WHAT DO YOU GAIN? WELL, BAYOUSING A FRACTIONAL FACTORIAL EXPERIMENT, IN OPTICAL IMAGES-IN. WE'RE ABLE TO REDUCE THE NUMBER OF EXPERIMENTAL CONDITIONS BY HALF, 32 TO 16 OR PUT ANOTHER WAY, IF WE DECIDED THAT WE COULD ONLY HANDLE 16 EXPERIMENTAL CONDITIONS, WE COULD HAVE LOOKED AT FOUR FACTORS OF THE COMPLETE FACTORIAL. SO WITH A FRACTIONAL FACTORIAL, WE WERE ABLE TO LOOK AT FIVE WITH 16 EXPERIMENTAL CONDITIONS. BUT THIS IS WHAT WE GIVE UP. ONCE YOU START PULLING EXPERIMENTAL CONDITIONS OUT OF A FACTORIAL EXPERIMENT THEN CERTAIN EFFECTS GET BUNDLED TOGETHER AND YOU CAN NO LONGER ESTIMATE THEM INDIVIDUALLY. CERTAIN EFFECTS ARE BUNDLED IN CERTAIN OTHER EFFECTS. NOW THE GOOD THING HERE IS THAT FRACTIONAL--ALL THE FRACTIONAL FACTORIAL DESIGNS THAT EXIST, WERE FIGURED OUT BY STATISTICIANS A LONG TIME AGO AND IT'S VERY EASY TO LOOK UP WHAT EFFECTS ARE BUNDLED WITH WHAT AND WE KNOW IN THE DESIGN WE CHOSE EXACTLY WHICH EFFECTS ARE BUNDLED WITH WHICH OTHER EFFECTS. AND THE DESIGN WE CHOSE WITH OPT-IN, IS BUNDLE WIDE A FOUR WAY INTERRATION AND EACH TWO WAY IS BUNDLED WIDE A THREE WAY INDACTION AND WE KNOW WHICH ONE IT IS. SO THE LOGIC HERE IS, OUR THEORETICAL MODEL DOES NOT PREDICT ANY LARGE THREE WAY AND FOUR WAY IRPT ACTION. --INTERACTION AND I'D BE INTERESTED IF ANYONE KNOWS OF A THEORETICAL MODEL THAT PREDICTS LARGE INTERACTION, WE FELT PRETTY SAFE BUNDLING THE MAIN EFFECT WITH FOUR WAY INTERACTION. SO IF YOU CAN ASSUME, IF YOU FEEL COMPORTABLE ASSUMING THAT THOSE HIGHER ORDER INTERACTIONS ARE NEGLIGIBLE IN SIZE, THEY DON'T HAVE TO BE ZERO, THEY JUST HAVE TO BE NEGLIGIBLE IN SIZE, THEN THE IDEA IS THAT THE MAIN EFFECT THAT YOU ESTIMATE IS GOING TO BE DUE PRIMARILY TO THE MAIN EFFECT NOT TO THE FOUR WAY INTERACTION. SO YOU ESTIMATING A BUNDLED EFFECT, THE MAIN EFFECT AND A SPECIFIC MAIN EFFECT AND THE SPECIFIC FOUR WAY INTERACTION BUT THE FOUR WAY IRPT ACTION IS NEGLIGIBLE IN SIZE AND SO, YOU'RE AX SOUPING THE ESTIMATE--ASSUMING THE ESTIMATE YOU GET IS PRIMARILY DUE TO THE MAIN EFFECT. SO THERE ARE--IT'S A TRADE TRADE --TRADE OFF. SO IN REALITY, THE HIGHER ORDER EFFECTS CAN BE NEGLIGIBLE OR THEY CAN BE REALLY LARGE, RIGHT? THOSE ARE THE ONLY TWO ALTERNATIVES. IT CAN BE NEGLIGIBLE OR IT CAN BE LARGE, LARGE ENOUGH TO WORRY ABOUT AND YOU CHOOSE A FACTORIAL DESIGN OR A FRACTIONAL FACTARRIAL DESIGN IF THE ORDER ORDER OF EFFECTS ARE REALLY ARE NEGLIGIBLE, FRACTIONAL FACTORIAL DESIGN IS THE RIGHT CHOICE BECAUSE IT MOVES SCIENCE FORWARD FASTER. IF YOU CHOSE A COMPLETE FACTORIAL THEN YOU WOULD WASTED RESOURCES YOU COULD HAVE LOOKED AT ADDITIONAL FACTORS. IF THE HIGHER ORDER EFFECTS ARE REALLY LARGE, THOUGH, THERE IS A POSSIBILITY OF SOME INCORRECT DECISION, OF COURSE THERE'S ALWAYS THE POSSIBILITY OF SOME INCORRECT ONES, BUT THERE IS AN ADDITIONAL POSSIBILITY OF INCORRECT DECISIONS ABOUT COMPONENT SELECTION. SO THERE'S THE TRADE OFF, HERE, SOMETIMES MAXIMIZING EFFICIENCY CALLS FOR TAKING CALCULATED RISKS. PART OF THE RESOURCE MANAGEMENT PRINCIPLE AND THERE ARE OPPORTUNITY COSTS SEESHTED WITH A--ASSOCIATE WIDE A LESS RISKY OPTION. YOU MIGHT BE THINKING WELL, THEN WHY DON'T YOU ALWAYS DO A FACTORIAL, IF YOU CAN ONLY HANDLE 16 CONDIPPINGSS JUST DO A SMALLER COMPLETE FACTORIAL. WELL, YOU CERTAINLY COULD DO THAT AND THAT MIGHT BE THE RIGHT CHOICE MANY TIMES BUT I JUST WANT TO POINT OUT THERE ARE OPPORTUNITY COSTS THERE. THERE ARE COMPONENTS YOU WOULDED NOT BE INVESTIGATING IF YOU CHOSE THAT THAT GROUP. SO THIS IS THE RESOURCE MANAGEMENT PRINCIPLE IN ACTION. I ARE A CERTAIN AMOUNT OF RESOURCES. I KNOW I CAN CONDUCT 16 EXPERIMENTAL CONDITIONS, I HAVE FIVE COMPONENTS I WANT TO LOOK AT, WHAT DO I DO? DO I GIVE UP ONE COMPONENT OR DO I--AND CONDUCT A COMPLETE FACTORIAL, OR DO I CONDUCT A FRACTIONAL FACTORIAL AND MAKE THOSE ASSUMPTIONS ABOUT HIGHER ORDER, INTERACTIONS BUT THAT GIVES ME THE ABILITY TO LOOK AT FIVE FACTORS. SO IT'S A TRADE OFF. I WANT TO POINT OUT THOUGH THAT YOU DON'T ESCAPE THAT BUNDLING WHEN YOU USE A DESIGN LIKE THAT. A LET OF PEOPLE LIKE THIS, BUNDLES FOR EXAMPLE, THE MAIN EFFECT OF COACHING WOULD BE BUNDLE WIDE ALL OF THE INTERACTIONS INVOLVING COACHING, WHICH IS MORE ALIAISON AS THINGS THAN THERE IS A FRACTIONAL FACTORIAL FOUNDATION. SO I'M SORRY, THE STATISTICS WORDS THAT I TALK ABOUT SO THERE'S MORE OF THIS PUNNEDLING IN THE COMPARATIVE TREATMENT EXPERIMENT. SO ONCE YOU TAKE A COMPLETE FACTORIAL AND YOU START MOVING CONDITIONS WHICH IS WHAT HAVE YOU HERE, THIS IS THE SUBSET OF THE CONDITIONS FROM--COMPLETE FACTORIAL EXPERIMENT, YOU WILL HAVE EFFECTS AND ITS INEVITABLE AND WHEN YOU USE A FRACTIONAL FACTORIAL DESIGN, YOU KNOW WHICH ONES THEY ARE. SO LET'S SAY WE CONDUCTED THE EXPERIMENT WHICH WE HAVEN'T STARTED DOING YET BUT SUPPOSE YOU HAD THE DATA, YOU CONDUCT ANALYSIS OF VARIANTS THAT WOULD GIVE OF ESTIMATES OF THE EFFECTS OF EACH OF THE COMPONENTS AND YOU WOULD USE THIS INFORMATION TO SELECT COMPONENTS TO INCLUDE IN THE INTERVENTION. YOU WOULD DISCARD COMPONENTS THAT ARE NOT PERFORMING ADEQUATELY. YOU HAVE AN ESTIMATE OF THE EFFECT, SIZE OF EACH COMPONE AND HE WANT YOU CAN--COMPONENT AND YOU CAN COMBINE THAT WITH COST OR JUST BASE YOUR DECISION ON THAT AND SELECT COMPONENTS THAT WILL MAKE UP THE OPTIMIZED INTERVENTION. OF COURSE KEEPING THE OPTIMIZATION CRITERION FRONT AND CENTER. SO IT WAS JUST FUND THD PAST FALL, WE'RE FINALIZING THE INTERVENTION PROTOCOLS AND THE MANUEL OF OPERATIONS AND WE EXPECT TO START THE EXPERIMENT THAT I SHOWED YOU LATE SUMMER, EARLY FALL. SO THERE'S NO DATA YET. SO AFTER WE OPTIMIZE AND EVALUATED THE INTERVENTION, WE WILL HAVE SET A BAR FOR WEIGHT LOSS INTERVENTION THAT WILL HAVE BEEN OPTIMIZED TO A SPECIFIC CRITERION, WE'LL SAY IT'S MOST EFFECTIVE WE COULD DO THAT CAN BE DELIVERED FOR $500 A PERSON, SO OUR WORK WILL ESTABLISH WHICH COMPONENTS WORK, TELL BE KNOWN WHICH COMPONENTS WORK SO FUTURE WORK BY US, BY OTHERS CAN BUILD ON THIS AND IT COULD--YOU COULD DEVELOP AN INTERVENTION THAT'S EQUALLY EFFECTIVE BUT COST LESS FOR EXAMPLE, THAT COULD BE THE OPTIMIZATION KRIST TERRION OR MOREEFFECTIVE. KEEP THE DOLLARS BUT BUMP UP THE EFFECTIVENESS. BUT THERE'S A BAR THERE SO IT'S CLEAR, MY NEXT INTERVENTION OR YOUR NEXT INTERVENTION IS GOING TO BE BETTER THAN THIS, IN THESE VERY SPECIFIC WAYS. OR SOMEONE MIGHT SAY I DON'T LIKE YOUR OPERATING GLOBALLY MYSELFATION CRITERION, BUT THAT'S STILL ENABLES TO YOU COMPARE THE PERFORMANCE OF THE TWO INTERVENTIONS. THIS IS A CONTINUOUS OPTIMIZATION, JUST KEEP OPTIMIZING THE INTERVENTION, KEEP RAISING THAT BAR. >> UNDER FUTURE WORK, THE THIRD BULLET HERE'S A MORE MEANINGFUL OPTIMIZATION CRITERION, IS THAT SOMETHING THAT COULD BE APPLIED TO THE STUDY THAT YOU'VE JUST DONE? >> POTENTIALLY IF YOU HAVE THE DATA, RIGHT? >> YEAH. >> SO IF YOU COULD DO IT, BECAUSE YOU WOULD HAVE THE DATA BUT SOMEONE ELSE COULD DO IT IF THEY HAD ACCESS TO YOUR DAT ASO YOU'RE NOT--YOU'RE NOT LIMITED EVEN THOUGH YOU'VE DONE THE OPTIMIZATION EXPERIMENT, YOU'RE NOT LIMITED TO THE OPTIMIZATION CRITERIA YOU STARTED WITH, YOU COULD POST HOC APPLY OTHERS? >> YOU COULD, DEFINITELY, YES. THAT'S WHY I'M ENVISIONING THAT YOU COULD PUT THE RESULTS OF THE ANALYSIS OF VARIANTS UP ON LINE AND SOMEONE ELSE COULD USE THAT AND THEY COULD OPTIMIZE THE WAY. >> DENIES? >> YES DEKNEES--DENISE FROM NHLBI. >> SEE WE HAVE FUNDED STUDIES ARE THAT HAVE HAD INTERVENTION BUT THERE'S A LOT OF PARTICIPANT IRPT ACTION WITH THE PROGRAM DELIVERER. SO THERE'S A LOT OF SORT OF PATIENT ORIENTED THIS MORNING AND PERSPECTIVE SO THAT FOR EXAMPLE IN THIS INSTANCE, YOU MIGHT--IT'S A RANDOMIZED TRIAL THAT YOU MIGHT SAY, HERE'S THE ENTIRE PROGRAM, WOULD YOU AS AN INDIVIDUAL PARTICIPANT, LIKE TO HAVE A BUDDY OR WOULD YOU LIKE TO HAVE 12 OR 24 PHONE CALLS OR WHATEVER IT IS. SO TO PROGRAM THE WAY IT'S DELIVERED IS THE WAY IT WOULD BE DELIVERED PERHAPS IN CLINICAL PRACTICE WITH NEGOTIATION IN THE PATIENT CENTERED WAY. THE WAY YOU'RE DOING IT IS EXTREMELY INTERESTING, BUT IT'S EVERYBODY GETS THE FINAL PACKAGE, THERE'S NO PARTICIPANT NEGOTIATION, RIGHT? >> RIGHT, BUT YOU COULD BUILD THAT IN SO I WOULD BE SAYING THAT IN COMPARISON NEXT ROOM, WHAT YOU END UP WITH HERE WITH ALL THE PIECES OF THE PROGRAM, IT DELIVERED IN AN INDIVIDUALIZED PATIENT-CENTERED WAY. IS THAT SOMETHING, THAT THIS ISSUE IS SOMETHING AT ALL YOU DISCUSSED OR THOUGHT OF? >> NO, WE'VE KIND OF HAD OUR HEADS WRAPPED UP WITH THIS BUT THAT'S INTERESTING. BUT DENISE, YOU COULD CERTAINLY DO IT THE WAY YOU'RE SUGGESTING, THAT IS WHEN YOU GOT ON THE RCT POINT THERE COULD BE ANOTHER ARM THAT ALLOWED PEOPLE TO SELECT FROM SORT OF A CAFETERIA, AND WHAT COMPONENTS THEY WANTED AND SO YOU ALSO COULD DO SCREENING EXPERIMENT IN WHICH DEPENDING ON EXACTLY WHAT THE RESEARCH QUESTION IS, YOU COULD COMPARE SAY, ASSIGNING A COMPONENT AND ALLOWING PEOPLE TO SEE IT AND WHAT THE EFFECT OF THAT WAS FOR AN INDIVIDUAL COMPONENT. JUST DEPENDING ON WHAT YOUR INTERESTS ARE. SO SO WITH THE OPTIMIZATION PRINCIPLE, I'M ASSUMING YOU DON'T NEED TO HAVE DONE AN OPTIMIZED EXPERIMENT ON A PREVIOUS INTERVENTION. YOU COULD TAKE THE CONTINUOUS OPTIMIZATION PRINCIPLE AND APPLY THAT TO SOME OTHER INTERVENTION THAT HAS BEEN TESTED IN THE RCT WAY AND IMPLEMENTED AND SAY PERHAPPED SCALED UP BUT YOU COULD STILL USE THIS PRINCIPLE TO IMPROVE IT WITH A NEW OPTIMIZATION METHOD. >> YES, ABSOLUTELY, EVERYTHING I'M TALKING ABOUT TODAY WOULD APPLY TO AN EXISTING INTERVENTION, IF YOU WANTED TO BREAK IT DOWN INTO COMPONENTS, YES. >> ISN'T THE CLINICAL EFFECT MUCH SMALLER IN THE FACTORIAL DESIGN, OF YOU REQUIRING MANY MORE SUBJECTS IN ORDER TO SHOW SIGNIFICANT EFFECT AS OPPOSED TO TESTING INDIVIDUAL COMPONENTS AND SEEING THEIR TOTAL EFFECTS SIZE, GIVEN THAT THE INCREMENTAL ADDITIONAL CLINICAL EFFECT OF AN INTERVENTION WHEN EVERYBODY'S ON TYPICALLY MAYBE SEVEN OTHER COMPONENTS OF AN INTERVENTION, IT'S GOT TO BE MUCH SMALLER AND HARDER TO SHOW AN EFFECT. >> RIGHT. YEAH, SO, THIS IS TRUE THAT THE EFFECTIVE AN INDIVIDUAL COMPONENT AND ALL THE COMPONENTS ARE WORKING IN THE RIGHT DIRECTION, THAT'S TRUE. BUT THE--THIS EXAMPLE WAS POWERED FOR--I FORGET WHAT EFFECT SIZE WE USE BUT MAYBE .3 IN THAT NEIGHBORHOOD. SO THAT'S 310thS OF A STANDARD DEVIATION SO IT IS POSSIBLE TO POWER EXPERIMENTS PROPERTILY EVEN THOUGH-OF APPROPRIATELY EVEN THOUGH YOU WOULD EXPECT THAT, AGAIN IF ALL THE COMPONENTS WERE HAVING A POSITIVE EFFECT AND PULLING IN THE RIGHT DIRECTION AND HAVING A DETECTABLE EFFECT THEN YES THE TOTAL WOULD LIKELY--WOULD OF COURSE HAVE TO BE LARGER THAN ANY INDIVIDUAL COMPONENT. I ALSO WANT TO MENTION THAT WHEN YOU'RE POWERING THESE STUDIES, TYPICALLY WHEN WE POWER AN EXPERIMENT THAT'S MORE FOR SPECIFIC DISCOVERY, THE FIRST THOUGHT IS WELL, I NEED TO GO TO THE LITERATURE AND LOOK AT WHAT EFFECT SIZE DO I EXPECT, IS THERE PART LITERATURE ON THIS, AND SO ON, AND THAT'S CERTAINLY A GOOD THING TO DO IN THIS CONTEXT, TOO, BUT YOU TYPICALLY HAVE A MINIMUM EFFECT SIZE THAT IS A CUT OFF FOR INCLUSION. AND SO YOU ONLY HAVE TO POWER THE EXPERIMENT FOR THAT INMUM EFFECT SIZE, SO IF YOU'RE SAYING FOR EXAMPLE, I'M INTERESTED ONLY IN INTERVENTION COMPONENTS THAT DEMON TRAIT THE STANDARD DEVIATION THEN YOU POWER THREE/10thS OF THE STUDY FOR STANDARD DEVIATION, YOU SEE WHAT I MEAN? >> [INDISCERNIBLE] >> YOU COULD FACTOR IN COST RIGHT AWAY, YES. YES, THAT WOULD BE A GOOD IDEA. >> SECOND EXAMPLE THIS IS I CLINIC BASED SMOKING CESSATION STUDY NYES? >> COULD I ASK A QUESTION ABOUT THE PREVIOUS EXAMPLE, SO AT THE END OF OF THAT OPTIMIZATION EXPERIMENT, YOU WILL HAVE ESTIMATES EFFECTS AND COMPONENTS FOR TWO WAY INTERACTIONS, PRESUMABLY AND THEN YOU'RE GOING TO SELECT THE COMPONENTS THAT YOU'RE GOING TO PUT IN THE TREATMENT PACKAGE. PRESUMABLY, YOU COULD EFFECT THE EASTIVE THAT TREATMENT PACKAGE ONCE PUT TOGETHER GIVEN THE PILOT DATA YOU HAVE? DO YOU DO THAT AND IS THAT USED IN FOLLOWING THE RCT TO EVALUATE THAT PACKAGE? >> YOU COULD DEFINITELY DO THAT. SO YOU WOULD HAVE, IT WOULD BE BASED ON A LITTLE TINY RCT THAT'S BASED ON 70 PEOPLE BECAUSE THERE'S 35 CONDITIONS HERE, BUT THAT WOULD GIVE YOU A GREAT ESTIMATE OF THE EFFECT SIZE GOING ON INTO THE RCT. NOW FRACTIONAL FACTORIAL DESIGN BECAUSE THEY DON'T INCLUDE ALL OF THE CONDITIONS, THE WINNING COMBINATION MIGHT NOT BE REPRESENTED IN THE DESIGN. SO THAT'S ONE THING TO CONSIDER, AND YOU WOULD HAVE--BUT EVEN IF YOU DON'T HAVE THAT, HAVE YOU THE INDIVIDUAL COMPONENT EFFECT AND THE INTERACADEMIESS AND THAT WOULD GIVE YOU A ROUGHER AND THE EXPERIMENT THAT PEOPLE TYPICALLY DO GOING INTO AN EXPERIMENT. A CLINIC BASED SMOKING CESSATION STUDY, TWO MORE PEOPLE I'M FORTUNATE TO BE WORKING WITH, TIM BAKER AND MIKE FIROIAE AT THE UNIVERSITY OF WISCONSIN. THIS IS P50 FUNDED BY NCI. P50 IS A CENTERED GRANT. THE SO THE STUDIES GO TO THE DOCTOR, AND A USE RECORDS IN THE SYSTEM AND THEY'RE KIND OF FLAGGED AS A SMOKER AND SO THAT'S HOW WE KNEW TO INVITE THEM INTO THE STUDY, WHEN THEY CHECKED INTO TO GO TO THE DOCTOR, THEY WERE TOLD ABOUT THE EXPERIMENT AS I MENTIONED MUCH EARLIER, THIS EXPERIMENT INVOLVES BOTH BEHAVIORIAL AND PHARMACEUTICAL COMPONENTS AND OF COURSE, NICOTINE REPLACEMENT IS VERY, VERY, COMMON AND CESSATION WORK. AND WE'RE DEVELOPING TWO DIFFERENT INTERVENTIONS USING TWO DIFFERENT CRITERIA. ONE OPTIMIZATION CRITERIA IS ABNORMALITIES SINNANCE OPTIMIZED AND THAT'S NO INACTIVE COMPONENTS. AND THEN COST OPTIMIZED IS MOST EFFECTIVE THAT CAN BE ACHIEVED FOR LESS THAN THAN OR EQUAL TO $500 PER PERSON AND IT'S JUST A CO INCIDENCE THAT BOTH OF THE ONES I'M SHOWING YOU, $500 PER PERSON SEEMS TO BE THE CUT OFF. SO BAKER AND FIORE'S USES A SMOKING CESSATION PROCESS, WHICH IS TWO WEEKS, SIX MONTHS AFTER THE QUIT DAY AND I WILL TALKING ABOUT AN EXPERIMENT THAT IS PRIMARILY IN THESE THREE PHASES. SO THESE ARE THE COMPONENTS THAT WE'RE EXAMINING. PRECESSATION USE OF THE NICOTINE PATCH, PRECESSIONATION OF NICOTINE GUM AND PATCH. AND EVERYBODY GETS NICOTINE REPLACEMENT AND EVERYONE IN THIS STUDYST DID, TOO, THAT GOT IT ON THE QUIT DAY, SO HERE, THE QUESTION IS ABOUT PRECESSATION USE OF NICOTINE REPLACEMENT PRECESSATION IN PERSON COUNSELING, CESSATION IN-PERSON COUNSELING MINIMAL OR INTENSIVE. CESSATION PHONE COUNSELING, MIN MEDICARE AND MEDICAID MAL--MINIMAL OR INTENSIVE AND THEN MAINTENANCE MEDICATION DURATION WHICH IS COULD BE SHORT OR LONG. SO THOSE SIX COMPONENTS WERE EXAMINE INDEED A SINGLE SCREENING EXPERIMENT--EXAMINED IN A SINGLE SCREENING EXPERIMENT, IN ADDITION THERE'S ANOTHER EXPERIMENT I'M NOT GOING TO SHOW YOU THAT EXAMINES FIVE COMPONENTS AIMED AT MAINTENANCE PRIMARILY AND WE'RE EXAMINING, 11 COMPONENTS AIMED AT PRECESSATION AND MAINTENANCE, THERE WAS ALSO A THIRD SCREENING EXPERIMENT, EXAMINING FOUR COMPONENTS SO MOST IMPLEMENTED IN THE CESSATION STUDY, SO HERE ARE SIX COMPONENTS WE DID A COMPONENT SCREENING EXPERIMENT OPTIMIZED WAS ONE CRITERION AND ANOTHER CRITERION, OPTIMIZED AND WE TALKED ABOUT THIS A LITTLE BUT JUST TO NOTE WE'RE ONLY DOING A SCREENING EXPERIMENT THAT WE'RE USED IN GOAL OPTIMIZATION, YOU DON'T NEED TO DO AN EXPERIMENT AGAIN. AND OUR FUNDING GOES UP TO HERE. WE CURRENTLY HAVE APPLIED FOR FUNDING TO DO THE RCT AND SOME OTHER STUFF. FIVE EXPERIMENTS WOULD HAVE BEEN OVER 3000 SUBJECTS, COMPARATIVE STUDY WOULD HAVE BEEN ABOUT 1800 SUBJECTS AND FACTORIAL EXPERIMENTS WOULD BE ABOUT 512 EXPERIMENTS AND THAT'S ACROSS ALL EXPERIMENTAL CONDITIONS. WE CHOSE THE FRACTIONAL FACTORIAL DESIGN ACQUIRING 32 CONDITIONS SO THIS IS THE DESIGN, I KNOW IT'S HARD TO SEE, LIKE THIS, AND IN THIS DESIGN, THE MAIN EFFECTS ARE BUNDLED AND TWO WAY ARE BOBBED--BUNDLED WITH THE FOUR WAY INTERACTION. ABOUT IMPLEMENTATION, SOME OF YOU MAY BE WONDERING HOW DO YOU CONDUCT, THIS WAS THREE FACTORIAL EXPERIMENTS SIMULTANEOUSLY, I SHOWED YOU ONE OF THEM, REAL WORLD HEALTH CARRY SETTINGS WITH A TOTAL OF THIS IS 80 EXPERIMENTAL CONDITIONS IN THE FIELD SIMULTANEOUSLY, VERY CAREFULLY, THAT'S THE ANSWER, IT TAKES CAREFUL ORGANIZATION, AND COMPUTERS AND ENOUGH WELL-TRAINED STAFF BUT IT CAN BE DONE. IT CAN BE DONE. WE'VE PULLED IT OFF WITH VERY,f‡ VERY FEW SNAFUs, AND THAT'S A TRIBUTE TO MIKE AND HIS ORGANIZATION. STAT US OF THE MOKING, AND SUBJECTS HAVE BEEN RECRUITED AND ASSIGNED, FOR RECRUITING PREPARATION AND DATA COLLECTION. WE'RE CLOSE TO BE BEING DONE WITH DATA AND ALTOGETHER WE'RE HAVE COMPONENTS AND 11 IN THE AND FOUR COMPONENTS IN MOTIVATION, THREE SEPARATE COMPONENTS AS I SAID, 80 CONDITIONS AND 1700 SUBJECTS. ALL TOGETHER IN THE THREE WEEK SPAN. AFTER WE EVALUATED THIS INTERVENTION, WE WILL HAVE SET A BAR FOR CLINIC SMOKING CESSATION INTERVENTION AND FUTURE WORK BY US OR OTHERS AND THAT INTEND TO DEVELOP OR INTERVENTION THAT IS AS EFFECTIVE BUT COSTS LESS. MAYBE THAT IS STILL COSTS FIVE HELPED DOLLARS BUT IS MORE EFFECTIVE. MAYBE RAISING THE COST AND GOING FOR A MORE EFFECTIVE INTERVENTION. OR AS EFFECTIVE BUT LESS BURDENSOME, FOR EXAMPLE, OR ANY OTHER OPTIMIZATION CRITERION THAT YOU CAN THINK OF. AND AGAIN THIS IS MORE ABOUT THE CONTINUOUS OPTIMIZATION. OKAY, SO, I TALKED ABOUT A FEW DEFINITIONS ABOUT BEHAVIORIAL INTERVENTIONS AND COMPONENTS. I DESCRIBED WHAT'S BUSINESS AS USUAL AND CONTRASTED IT WITH THE MULTI OPTIMIZATION STRATEGY AND WE TALK ABOUT THE DIFFERENCE BETWEEN OPTIMIZATION AND EVALUATION BOTH ARE IMPORTANT, WE TALKED ABOUT THE RESOURCE MANAGEMENT PRINCIPLE AND THE CONTINUOUS OPTIMIZATION PRINCIPLE AND I SHOWED YOU TWO EXAMPLES OF THE WORK IN THIS FIELD SO NOW I'D LIKE YOU TO IMAGINE A FEW THINGS. IMAGINE THESE OPTIMIZED INTERVENTIONS, INTERNET DELIVERED INTERVENTION TO PREVENT DRUG ABUSE AND NTAA ATHLETES OPTIMIZED FOR EFFECTIVENESS THIS, IS CURRENTLY FUBBEDDED BY NIDA, I CAN TALK ABOUT THAT MORE. MORE IF YOU'RE INTERESTED? OPTIMIZED TO BE THE MOST IESKTIVE IT TAKES UNDER SIX MINUTES TO DELIVER. INTERVENTION TO INCREASE COMPLIANCE WITH A CANCER TREATMENT REGIMEN, OPTIMIZED TO BE THE MOST EFFECTIVE WITH THE LEAST COGNITIVE BURDEN. A WORKPLACE, BASED INTERVENTION TO REDUCE SITTING TIME OPTIMIZED TO BE THE MOST EFFECTIVE WHILE BEING LEAST DISRUPTIVE TO PRODUCTIVITY AND MAINTAINING MODERATE PHYSICAL ACTIVITY, OPTIMIZED TO BE THE MOST EFFECTIVE THAT CAN BE DELIVERED IN A SINGLE ONE HOUR SESSION. OPTIMIZED TO BE THE MOST EFFECTIVE FOR LESS THAN $300 A PATIENT. AND MANY, MANY MORE, INFINITE POSSIBILITIES OF COURSE IN THE NUMBER. NOW I'D LIKE TO ASK TO YOU IMAGINE A STATE-OF-THE-ART MOST WIDELY IMP ELEMENTED SO SUPPOSE THAT EVERY EVIDENCE BASED INTERVENTION IS OPTIMIZED WE KNOW IT'S POSSIBLE GIVEN CONSTRAINTS. INTERVENTIONS ARE ENGINEERED WITH WITH IMPORTANT OUTCOME CRITERIA AND ECONOMY AND SCALABILITY IN MIND. AND EVERY INTERVENTION THAT'S SUCCESSFUL IS IMMEDIATELY SCALABLE. IT'S SCALABLE BECAUSE IMPORTANT CONSTRAINTS THAT COME UP IN IMPLEMENTATION HAVE BEEN INCLUDED IN THE OPTIMIZATION OF THE CRITERION. IT'S KNOWN WHICH COMPONENTS WORK, WE KNOW WHICH COMPONENTS WORK, WE KNOW WHICH IS INSIDE THE BOX. CO HERENT BASE THEREFORE OF SCIENTIFIC KNOWLEDGE IS ACCUMULATED. WE KNOW WHAT WORKS AND WHAT DOESN'T AND PEOPLE CAN START TO BUILD ON THAT; ONCE A COMPONENT SCREENING EXPERIMENT AS BEEN CONDUCTED DENOVA IS MADE PUBLIC. SHOULDN'T BE ANY HUMAN SUBJECTS ISSUE WIDE THIS BECAUSE IT WOULD JUST BE RESULTS WITH THE VARIANTS, AND SO OTHERS WHO WOULD LIKE TO OPTIMIZE USING DIFFERENT CRITERION IF THEY WANT TO. MANY FEWER RCTs SHOW UP AFTER THE RESULTS. IF AFTER THE OPTIMIZATION EXPERIMENT, THE SCREENING EXPERIMENT, IF YOU REALLY HAVE NOTHING, AT THAT POINT, YOU WOULD NOT GO TO AN RCT. YOU WOULD GO BACK TO THE DRAWING BOARD. AND WOULD KNOW WHAT HADN'T WORKED. SO SUPPOSE THE RESOURCE MANAGEMENT PRINCIPLE IS USED AND EVERY NIH DOLLAR YIELDS THE MOST SCIENTIFIC INFORMATION ATTAINABLE BECAUSE THE DESIGN HAS BEEN CHOSEN WITH THAT IN MIND. IMAGINE THAT EVERY TIME A BEHAVIORIAL INTERVENTION IS OPTIMIZED A CLEAR BAR IS SET FOR EFFECTIVENESS, EFFICIENCIES AND ECONOMY. AND ANY NEW INTERVENTION HAS TO DEMONSTRATE IT'S INCREMENTALLY FOR A PROCEEDING ONE AND SPECIFICALLY IN WHAT WAY AND IN THIS WAY, THE BAR WOULD BE RAISED WITH EACH NEW EVIDENCE BASED INTERVENTION, SO THERE WOULD BE INCREMENTAL PROGRESS OVER TIME AND INTERVENTIONS WOULD STEADY ILLEGALSLY GAIN IN HEALTH IMPACT OVER TIME. LET'S GO BACK TO THE GAP. EVERY YEAR 443,000 DEATHS DUE TO SMOKING, 400,000 DEATHS DUE TO DIET AND INBEINGIVITY,. --INACTIVITIES AND 35 MILLION DEATHS WORLD WIDE FROM NONCOMMUNICABLE DISEASES, AND SO MUCH MORE DUE TO MORBIDITY AND MORTALITY DUE TO INDIRECTLY DUE TO BEHAVIORIAL FACTORS. IMAGINE CLOSING THIS GAP. I BELIEVE IT'S POSSIBLE. BEHAVIORIAL ENGINEERING CAN HELP TO SET A BAR FOR EFFECTIVENESS AND SCALABILITY AND RAISING THE BAR AND GRADUALLY CLOSE THE GAP. THANK YOU. [ APPLAUSE ] >> THANK YOU LINDA AND VERY PROVOCATIVE TALK ABOUT THE POWER OF FACTORIAL DESIGNS IN ACTION. I'M EXCITED TO SEE THE EXAMPLES AND LOOK FORWARD TO THE RECUTS PRETTY SOON, AT LEAST FROM THE PROJECT. IT SEEMS TO ME THAT ONE OF THE KEY ISSUES IN THE OPTIMIZATION STRATEGY DEPENDS IS THE SELECTION OF YOUR COMPONENTS FOR TESTING. SO I WAS PARTICULARLY IN THE SMOKING INTERVENTION, I WAS CURIOUS THAT THEY ONLY CHOSE NICOTINE REPLACEMENT AND DON'T INCLUDE BRUPRIONE, AND OTHERS AND I'M WONDERING WHAT WILL HAPPEN, THAT'S A CLEAR DECISION FOR PRACTICES, HOW WILL WE WEIGH IN THOSE STANDARD MEDICAL PRACTICES AGAIN THIS NRT SYSTEM AND SIMILARLY WHEN YOU HAVE AN INTERVENTION THAT'S COMPLETELY ODES THE RANGE OF WHAT'S TESTED, WHAT'S THE NEXT STEP, AND HOW DO YOU--HOW DO YOU DECIDE WHICH ONES TO INCLUDE AND NOT INCLUDE? >> THAT'S A GREAT QUESTION AND OF COURSE, VERY OFTEN CAN'T INCLUDE ALL OF THE COMPONENTS THAT YOU MIGHT BE INTERESTED IN, SO, THERE'S A LOT MORE TO MOST THAN I'VE BEEN ABLE TO SHOW YOU IN THIS TALK. ONE OF THE STAGES OF--STEPS OF THIS INVOLVES DEVELOPING A THEORETICAL MODEL OF THE PROCESS THAT YOU'RE TRYING TO INTERVENE ON AND THEN ENABLES YOU TO SHOW WHERE THE COMPONENT IS EXPECTED TO INTERVENE AND THAT IS--THAT'S WHERE THE COMPONENTS COME FROM OR AT LEAST HOW THE DECISION, A LARGE PART OF HOW THE DECISION IS MADE TO INCLUDE VARIOUS COMPONENTS. IN AN INTERVENTION THAT ALREADY HAS A NUMBER OF COMPONENTS AND YOU WANT TO LOOK AT SOME OF THE COMPONENTS BUT CAN'T LOOK AT ALL OF THEM, AND I OFTEN RECOMMEND THAT PEOPLE TAKE THE COMPONENTS THEY'RE NOT VARYING AND JUST GIVE THOSE TO EVERYBODY. SO EVERYONE CAN GET KIND OF A CORE INTERVENTION, THAT'S ACTUALLY WHAT WE'RE DOING IN THE OPTICAL OPT-IN STUDY. BUT OUR STUDY IS MORE ABOUT COMPLIANCE RIGHT HERE THAN IT IS ABOUT THE ACTUAL INTERVENTION ARE, BUT YOU CAN TAKE ANY COMPONENTS YOU DON'T WANT TO VARY AND JUST EVERYBODY GETS THOSE AND THEN THEY VARY ANOTHER ROUND. >> YEAH, ANOTHER OBSERVATION IS THAT--AS YOUR SAMPLE SIZE DIMINISHES BY SOMETHING LIKE 80%, THAT IS A REMARKABLE EXAMPLE OF HOW WE COULD SPREAD OUR NOW SCARCER RESOURCES, SO THANK YOU. >> LIF YOU KNOW IF YOU'RE DOING A FACTORIAL EXPERIMENT, ONCE YOU HAVE THE SUBJECT, TO ADD ANOTHER FACTOR, VERY OFTEN DOESN'T REAR ANY ADDITIONAL SUBJECTS, NOW IT WOULD REQUIRE ADDITIONAL EXPERIMENTAL CONDITIONS I'M NOT SAYING IT'S A DECISION THAT SHOULD BE MADE LIGHTLY, BUT I THINK IT'S--IT'S SOMETHING IMPORTANT TO CONSIDER AND AS MORE AND MORE PEOPLE ARE EXPERIMENTING WITH INTERNET DELIVERED INTERVENTION, VERY OFTEN, THE LOGISTICS OF DELIVERING A LOT OF INTERVENTION COMPONENTS IS GREATLY REDUCED MAINLY BECOMES A PROGRAMMING ISSUE AND ONCE THE PROGRAMMING IS DONE, IT'S NOT DIFFICULT TO CONDUCT THAT EXPERIMENT WITH MANY. >> THANK YOU FOR A GREAT TALK. A COUPLE OF QUESTIONS. THE--I'M INTERESTED IN HOW THE TWO STUDIES THAT YOU DESCRIBED IN DETAIL AND YOU SAID BOTH ARE FUNDED AS RO-1s. >> THE WISCONSIN STUDY IS A P50. >> BUT IT'S ONE OF THE PROJECTS WITHIN THE PFIRST. SNSHES YES. >> SO IT'S LIKE AN RO-1. SO, I'M INTERESTED IN WHAT THE EXPERIENCE HAS BEEN IN TERMS OF HOW MUCH TIME DOING THE--GOING THROUGH THE PROCESS YOU DESCRIBED TAKES RELATIVE TO THE FOLLOW ON STUDY WHICH MIGHT BE ANOTHER RO-1 TO DO THE RCT TO EVALUATE THE PACKAGE, SO DOES IT--IS THE--THE FIRST STEP THE OPTIMIZATION STEP, A LOT SHORTER, THE SAME LENGTH, SO I'M INTERESTED IN THAT AND THE OTHER IS, WHAT'S THE OUTCOME, THAT YOU'RE MEASURING, IS IT GOING TO BE THE SAME AS THE OUTCOME IN THE RCT THAT'S GOING TO FOLLOW OR IS IT AN IMMEDIATE OUTCOME OR SOMETHING ELSE? >> YEAH, THOSE ARE ACTUALLY RELATED QUESTIONS, I THINK. SO ABOUT HOW LONG IT TAKES, AS CAN YOU IMAGINE, IT'S HIGHLY VARIABLE, DEPENDING ON THE CONTEXT AND WHAT AREA WE'RE TALKING ABOUT. SO THE OPTICAL OPT-IN STUDY AND THE WISCONSIN STUDY, THESE ARE BOTH CLINIC BASED STUDIES AND SO PEOPLE HAVE TO TRICKLE IN AND SO THAT'S SORT OF LIMITING FACTOR, IS THE SUBJECT TRICKLING IN. >> [INDISCERNIBLE]--BUT RATHER RELATIVE TO THE FOLLOW UP OF THE RCT? LESS TIME OR MORE TIME? >> SO IT DEPENDS ON WHETHER YOU WANT TO OPTIMIZE ON THE ACTUAL OUTCOME WHICH IN THE CASE OF SMOKING CESSATION IS PRETTY EASY TO DO AND IN WEIGHT LOSS IS ALSO DOABLE AND DRUG ABUSE PREVENTION PROBABLY NOT DOABLE. SO IN DRUG ABUSE PREVENTION, TYPICALLY, THE INTERVENTION IS DELIVERED IN FIFTH OR SIXTH GRADE AND THEN THE END OUTCOME IS MORE LIKE 10-11th GRADE AND YOU CAN'T WAIT AROUND FOUR OR FIVE YEARS FOR THAT. SO THE BEST EVIDENCE BASED BEHAVIORIAL INTERVENTION ARE BASED ON MEDIATION MODEL SO THE INTERVENTION COMPONENTS DON'T OPERATE DIRECTLY ON THE OUTCOME OF INTEREST, INSTEAD THEY OPERATE ON MEDIATORS AND IMMEDIATEIATORS ARE HYPOTHESIZED TO HAVE THE--MEDIATORS ARE HYPOTHESIZED TO HAVE THE OUTCOME OF INTEREST, SO MY RECOMMENDATION, TO SHORTEN THE PROCESS, IN CASES LIKE THAT WOULD BE TO OPTIMIZE USING MEASURES OF THE MEDIATOR, AS SHORT-TERM OUTCOMES, BECAUSE THAT IS WHAT THE INTERVENTION IS INTENDED TO DO, IT'S INTENDED TO HAVE AN EFFECT ON IMMEDIATEIATORS AND THEN THE BATON IS HANDED TO THE MEDIATORS WHICH IN TURN EFFECT THE OUTCOME BUT IF DO YOU IT THAT WAY, THEN YOU HAVE SHORTER TERM OUTCOMES YOU USE FOR OPTIMIZATION AND THEN WHEN YOU DO THE RCT LATER, YOU WOULD OF COURSE USE THE LONGER TERM, THE LONGER TERM OUTCOME. YEAH. JUST A MINUTE THERE'S SOMEONE UP THERE. >> YEAH, SO FIRST OF ALL, I FOUND THIS TALK VERY PERSUASIVE, I THOUGHT IT WOULD REALLY, REALLY GOOD SO CONGRATULATIONS ON THAT. MOST OF US-- >> THAT'S WHAT I WAS GOING FOR. >> [LAUGHTER] >> AND I KNOW YOU'RE A GOOD EXPERIMENT ALEXANDERRISTS I'D LIKE TO PUSH I A BIT BECAUSE I THINK THIS WAS VERY PERSUASIVE BUT IF I LOOK AT WHAT WE'RE DOING HERE, I SEE US GOING IN A DIFFERENT DIRECTION. SO WHAT YOU HEAR A LOT ABOUT NOW IS USING BIG DATABASES, PLANNING ON THERE BEING A LOT OF NATURAL VARIATION IN THE WAY HEALTHCARE PRACTICE, LOTS OF THINGS HAPPENING ALL OVER THE PLACE AND THE ARGUMENT'S BEING MADE THAT IF YOU JUST GET BIG ELECTRONIC DATABASES, ELECTRONIC MEDICAL RECORDS WITH HUNDREDS OF MILLIONS OF CASES AND WE PLAY ON NATURAL VARIATION AND USE THESE LINEAR PROGRAMMING APPROACHING TO OPTIMIZE, WE'LL GET THE SAME ANSWER AND, OR MAYBE EVEN A BETTER ANSWER AND I WAS CURIOUS AS AN EXPERIMENTALIST WHERE DO YOU GET NERVOUS ALONG THAT CONTINUUM? >> WELL I FEEL THAT--FIRST OF ALL I DON'T SEE IT AS EITHER OR THING BECAUSE BIG DATA OUT THERE I BELIEVE CAN BE TREMENDOUSLY HELPFUL IN FORMING THE KINDS OF EXPERIMENTS I'M TALKING ABOUT DOING HERE. I'M NOT SURE THOUGH, HOW HELPFUL BIG DATA CAN BE IN TELLING US WHETHER A PARTICULAR INTERVENTION STRATEGY IN THE FORM OF A-COMPONE THAT HE WENT'S INTENDED TO MOVE A PARTICULAR MEDIATOR WILL WORK. I THINK THAT THEY CAN--I THINK THOSE ANALYSIS CAN BE TREMENDOUSLY HELPFUL IN HELPING TO--HELPING US TO SEE WHAT THE MEDIATORS ARE AND WHAT WE SHOULD BE MOVING BUT I'M MOT DISHUR THEY CAN REALLY--NOT SURE THEY CAN REALLY TELL US, FOR EXAMPLE A PARTICULAR SET OF LESSONS IN A SCHOOL BASED INTERVENTION IS GOING TO PERSUADE THAT FEWER KIDS ARE USING DRUGS THAN REALLY ARE, SO I SEE THEM AS BEING DIFFERENT, SOMEWHAT DIFFERENT NOW I COLLEAGUES SUSAN MURPHY AND HER COLLABORATORS, HAVE BEEN WORKING ON WAYS OF USING MACHINE LEARNING TO ANALYZE THE DATA FROM CERTAIN KINDS OF RANDOMIZED EXPERIMENTS TO GET SORT OF EVEN MORE INFORMATION OUT OF THE RANDOMIZED EXPERIMENT AND THAT I THINK IS AN AREA THAT HAS A LOT OF POTENTIAL, SO I CAN FEEL LIKE THINGS ARE KIND OF DOVETAILING IN THAT WAY. DOES THAT ANSWER YOUR QUESTION? >> YEAH, SO I THINK THAT WAS A REALLY GOOD ANSWER. I THINK YOU NEED TO KEEP MAKING THIS POINT. THAT IN FACT YOU MIGHT HAVE GREATER EFFICIENCY, USING FACTORIAL DESIGNS EVEN WITH SMALLER FRACTIONS DOWN, AND HAVE FRACTIONS BUT DOWN TO THE ONE/16th OR SO. YOU ACTUALLY MIGHT BE ABLE TO GET ANSWERS TO QUESTIONS, ACTUALLY IN A MORE EFFICIENT WAY THAN A LOT OF--A LOT OF THE DIRECTIONS THAT THESE ARE GOING. BUT I LIKE THE IDEA OF THE TWO APPROACHES OF DOVETAILING. >> CERTAINLY EXPERIMENTATION GIVES YOU A LESS THAN AMBIG OWOUS ANSWER, THIS WORK IN AN EXPERIMENT CAN BE A GREAT WAY TO ANSWER THAT QUESTION. >> AGAIN, I AGREE WITH EVERYBODY. VERY PERSUASIVE SET OF POINTS AND IN THE SPIRIT OF IMLEVELLATION AND FEASIBLE, HOW YOU IMPLEMENT THE OPTIMIZATION STAT STRAGY IN OUR SCIENCE-OF STRATEGY IN OUR SCIENCE, BUT A PRACTICAL QUESTION WITH RESPECT TO THE RO-1S AND WHAT WAS THE PROCESS OF GETTING THROUGH THE REVIEW COMMITTEES AND I GUESS, BEING ARTICULATE AND CONVINCING WITH THEM ABOUT THE VIRTUES OF THE OPTIMIZATION STRATEGY. >> YEAH, SO THE--BIGGEST OBSTACLE, VERY OFTEN IS A LOT OF PEOPLE WHO HAVE BEEN TRAINED PRIMARILY IN THE RCT HAVE SOME FUNDAMENTAL DIFFICULTIES WITH THE FACTORIAL EXPERIMENT. THEY SORT OF DON'T BELIEVE THAT IT'S NOT A MULTIARM RCT. SO IF SOMEONE IS PROPOSING AN EXPERIMENT WITH 16 EXPERIMENTAL CONDITIONS AND YOU SEE IT AS A 16 ARM RCT. YOU WOULD JUST SIMPLY NOT BELIEVE THAT IT COULD EVER BE POWER POWERED. SO WE'VE RUN INTO THEY A FEW TIMES AND WE'VE GOTTEN BETTER FOR HELPING PERSUADE THE REVIEW COMMITTEE AND IT'S HARD TO DO IT IN 12 PAGES, SOMETIMES BUT YOU KNOW I'VE BEEN WORKING ON CONCISE WAYS TO GET THAT. AND BUT IF ANYONE'S INTERESTED IN LOOKING AT MY WEB SITE, I HAVE SOME INFORMATION THERE FOR PEOPLE WHO ARE WRITING GRANT PROPOSALS USING MOST OR USING FACTORIAL EXPERIMENTS AND SOME WORDS I'VE USED THAT HAS--SEEMS TO BE CLEAR AND HAS PERSUADED THE PAST AND PEOPLE ARE WELCOME TO TAKE THEM AND ADAPT IT TO THEIR OR FOR THEIR OWN NEEDS. THE OPTICAL IMAGES IN EXPERIMENT WAS VERY WELL RECEIVED BY THE COMMITTEE. THEY REALLY SEEM TO RESONATE TO THE INNOVATIVE ASPECT OF IT, SO YOU KNOW, I FEEL LIKE WE'RE STARTING TO HAVE MORE SUCCESS. I'VE ALSO BEEN TRYING TO PUBLISH ARTICLES THAT COVER THIS MATERIAL AND TRY AND CLARIFY SOME ISSUES ABOUT--ABOUT MOST AND ABOUT FACTORIAL EXPERIMENTS AND OF COURSE I'M LEARNING AS I GO, YOU KNOW MOST IS AN OPEN SEARCH AREA AND NOT EVERY QUESTION HAS BEEN ANSWERED WITH RESPECT TO MOST, EITHER. RIGHT NOW, NCI, NIDA, NIDDK, AND NIMH ARE ALL FUNDING ONE MOST EXPERIMENT THEY KNOW OF, AND NIDA IS FUNDING TWO. SO THINGS ARE MAYBE STARTING TO PICK UP STEAM, I HOPE. OH, I'M SORRY, JOHN? >> SO JUST AS A FOLLOW ON ON TO THAT QUESTION, NOT TO BE TOO REVIEWISH BUT THERE'S NO PROVE OF CONCEPT RIGHT? NO PRELIMINARY DATA TO SUGGEST THIS POINT? >> YEAH, THAT'S RIGHT? >> SO A SPECIAL MECHANISM WOULD BE BETTER WAY TO GO BECAUSE IT CAN BE SHOT DOWN TECHNICALLY. >> YEAH, THAT'S TRUE. >> THAT'S NOT COMPLETELY TRUE, THERE'S A NICE PAPER OUT IN ADDICTION SHOWING A DESIGN VERY SIMILAR TO THIS, FRACTIONAL, A FACTORIAL DESIGN BY DAVE GUSTAFSON, FROM WISCONSIN PUBLISHED JUST THIS MONTH. SO NOT EXACTLY THE SAME BUT COULD BE USED TO HELP JUSTIFY IT. >> ACTUALLY THERE ARE A COUPLE OF--THERE'S NOT A PROOF OF CONCEPT OF MOST FROM START TO FINISH, IT'S TRUE, BUT THERE ARE A COUPLE OF EXAMPLES BESIDES THAT ONE OF FACTORIAL, LARGE FACTORIAL EXPERIMENTS, THERE'S VIC STRUCKER PUBLISHED AN ARTICLE THAT I WAS AN AUTHOR ON AND HOST OF OTHERS AND 2008 AND THEN THERE WAS ANOTHER ONE JUST LAST YEAR THAT WAS ALSO SOME THE SMOKING AREA, ANOTHER FRACTIONAL FACTORIAL DESIGN. SO YOU KNOW--BOTH DATABASE, BOTH REPORTING INVOLVED SO THERE'S STARTING TO BE SOME. SO THAT'S A FAIR CRITICISM, NOW BECAUSE IT'S RIGHT, THERE'S NOT--THERE'S NOT A LOT OF PROOF OF CONCEPT AT THIS POINT. >> JUST CURIOUS WHAT THE STUDY SECTIONS WERE THAT REVIEWED THE COUPLE THAT YOU'RE BEEN INVOLVED IN. >> IF YOU REMEMBER? >> NO, BECAUSE IT'S MY FIRST NIDDK THERE WAS AN ACRONYM FOR THE STUDY SECTION BUT I KNOW I'LL GET IT WRONG BUT I BETTER JUST NOT SAY IT. FOR NIDA, AND THE NCI STUDY WAS IT SORT OF--IT WAS SORT OF A SET OF WEIRD CIRCUMSTANCES. IT ACTUALLY, SO MIKE AND SIN HAD HAD A TEACHER AND THE TEACHER MECHANISM WAS STOPPED BY NIDA AND THEY WERE ENCOURAGED TO PUT IN AN APPLICATION FOR A P50 AND SO WE PUT IN AN APPLICATION FOR A P50 THAT WAS KIND OF CENTERED ON MOST TO DO THE EXPERIMENTS AND IT GOT A GOOD BUT NOT FUNDABLE FOR 34 AND WE ACTUALLY HAD WRITTEN REVISION AND WE WERE DAYS AWAY FROM PRESENTING IT AND THEY SAID THERE WERE CHANGES SO NATURE TELEVISION, BUT THOUGHT OF AS REAL HOPE FOR DOING SOMETHING DIFFERENT. ONE OF THE QUESTIONS THAT CAME UP AT THE PHYSICAL ACTIVITY WORKSHOP THAT THE OFFICE OF DISEASE PREVENTION SPONNORRED IN DECEMBER--SPONSOR INDEED DECEMBER WAS HOW DO WE FIND WAYS FOR NIH TO SUPPORT INTERVENTION DEVELOP AND IT'S NOT AN EASY SELL TO THE INTERVENTIONS AND THEY'LL HAVE TO HAVE THE PEOPLE TO APPRECIATE WHAT'S BEING PROPOSED THIS, IS AN EXCELLENT WAY OF DOING WHAT I THINK OF AS THE INTERVENTION--THE INTERVENTION DEVELOPMENT ACTIVITY READY TN LAUNCH THE FULL BLOWN RCT. >> YEAH, AND I WANT TO STRESS BY THE WAY THAT IF IT WASN'T CLEAR, THAT THESE EXPERIMENTS ARE NOT PRIVATE STUDIES, THEY'RE FULLY POWERED EXPERIMENTS. >> AND THAT'S WHAT THE REVIEWERS WILL LIKE WITH THE RO-1 APPLICATIONS BECAUSE THERE WILL BE A REVIEWED OUTCOME. >> YES. >> IT SEEMS TO ME THAT THE EXAMPLES AND THE OTHER ONES I THOUGHT OF ARE FOCUSED ON PRACTICE PRACTICAL AREAS OR IMPLEMENTATION AREAS LOOSELY BROADLY DEFINED. DO YOU SEE A ROLE FOR THE MOST APPROACH IN MORE FUNDAMENTAL INTERVENTION DEVELOPMENT FOR SORT OF BASELINE THEORY DRIVEN-- >> TYPE ONE? NYEAH. >> YEAH, DEFINITELY. IF YOU WERE DEVELOPING AN INTERVENTION FROM SCRATCH, AND ALL YOU REALLY HAD WAS RESULT OF LABORATORY STUDIES OR PERHAPS A META-ANALYSIS OR SOMETHING LIKE THAT, I THINK MOST WOULD BE A GREAT WAY JUST TO GET STARTED. DEFINITELY. YEAH, I SEE IT AS A GOOD WAY TO DO BOTH TYPE ONE AND TYPE TWO. YEAH WHEN IT COMES TO IMPLEMENTATION, VERY OFTEN PEOPLE DO IMPLEMENTATION STUDIES AND THEY'RE ALWAYS--ALWAYS OBSERVATIONAL, BUT I BELIEVE, YOU KNOW IT'S POTENTIALLY POSSIBLE TO EXPERIMENT YOU KNOW POTENTIAL--[INDISCERNIBLE] >> DENISE AGAINST FROM NHLBI, I THOUGHT I WOULD COMMENT THAT IF IT YOU WANT EVIDENCE THAT FACTORIAL DESIGNS ARE ACCEPTABLE TO TRIALISTS, YOU COULD CITE THE ACCORD TRIAL. >> OH, OKAY. >> WHICH IS 10,251 PEOPLE ADULTS WITH TYPE TWO DIABETES THAT USE THE TWO BY TWO FACTORIAL DESIGN LOOKING AT EVENTS AS OUTCOME. IF THAT'S NOT ACCEPTED BY THE FIELD, I DON'T KNOW WHAT IS, IT WASN'T BEHAVIORIAL THOUGH, IT WAS IN THAT THEY HAD TO COMPLY, ADHERE TO THE DRUG TREATMENT TRIAL BASICALLY AND DRUG STRATEGY, TREATMENT TRIAL. BUT-- >> A LOT OF MONEY SPENT ON THAT FACTORIAL DESIGN STUDY. >> WELL, THANK YOU DENISE, THAT'S REALLY HELPFUL. THIS HAS ME THINKING THAT I SHOULD HAVE A SECTION ON MY WEB SITE ON THIS VERY TOPIC. >> ANY MORE QUESTIONS? >> IF WE HAVE NO OTHER QUESTIONS THEN I THINK WE THANK DR. COLLINS FOR A GREAT PRESENTATION. THANK YOU FOR ALL FOR THE DISCUSSION. [ APPLAUSE ]