GOOD EVENING. I AM CHUCK NATANSON, AN INTENSIVIST, NOT A STATISTICIAN. I THINK THE REASON THAT I GIVE THIS LECTURE IS I'VE WRITTEN QUITE A FEW META-ANALYSIS, SOME QUITE CONTROVERSIAL. AND WHAT I AM GOING TO DO TODAY IS, FIRST, PRESENT SOME BACKGROUND INFORMATION OF HOW YOU APPROACH AND WRITE META-ANALYSIS, BUT THAT'S NOT GOING TO BE THE MAJOR FOCUS. I'M GOING TO DESCRIBE SOME OF THE META-ANALYSIS THAT I'VE DONE WITH THE IDEA OF TRYING TO BRING TO LIFE THE TERMS OF I SQUARE, METAREGRESSION, FIXED RANDOM FORGETS MODELS, SO THE TERMS -- THEY HAVE REAL MEANING. IN THE END WHEN YOU READ A PAPER AND IT'S A META-ANALYSIS YOU'LL UNDERSTAND THOSE TERMS, HOPEFULLY I CAN GET THAT ACROSS. LET ME AGAIN WITH THE FIRST HALF OF THE TALK WHICH IS A DISCUSSION OF WHAT ARE THE MAJOR PARTS OF A META-ANALYSIS AND HOW YOU DO IT. SO THE TERM META-ANALYSIS WAS FIRST COINED IN 1976, AND YOU CAN SAY, WELL, WHY IN 1976 DID THEY FIRST TRY TO DEFINE META-ANALYSIS? WELL, IN SOME RESPECTS A META-ANALYSIS IS DEPENDENT ON A RANDOMIZED CONTROL TRIAL. AND THE RANDOMIZATION RANDOMIZED CONTROL TRIAL WAS INVENTED IN THE 1930s BY A MAN NAMED FISHER. IT TOOK THAT NUMBER OF YEARS IN ORDER TO COMBINE THEM. AND THE TERM AS DEFINEDITATE STATISTICAL ANALYSIS OF A LARGE COLLECTION OF RESULTS FOR THE PURPOSE OF INTEGRATING THEIR RESPECTIVE FINDINGS. THERE IS REALLY TWO TYPES OF META-ANALYSIS, AND THE SECOND ONE IS THE ONE I DO MORE OF. THE FIRST ONE, I THINK THAT'S DONE LESS TODAY, IS TO DETERMINE IF THERE IS A SIMILAR TREATMENT EFFECT EXISTS FOR A THERAPY, AN INDEPENDENT STUDY, TO ESTIMATE A NET EFFECT POR THIS THERAPY WITH THE IDEA THAT YOU DID A LARGE NUMBER OF RANDOMIZED CONTROL TRIALS AND THEY WERE UNDER POWERED. AND WHEN YOU COMBINE YOU'LL FIND A SIGNIFICANT EFFECT. AND ONE CAN -- ONE CAN SAY IS, CAN YOU USE THIS TO APPROVE A DRUG BY THE FOOD AND DRUG ADMINISTRATION. AND MY STATISTICIAN THAT I WORKED WITH FOR MANY YEARS SORT OF SAID THIS. HE BELIEVED THAT A PROPERLY DONE -- PROPER LY DONE, THAT'S THE CORRECT TERMS. WE'LL GO OVER THAT. META-ANALYSIS, MAY WELL BE EQUIVALENT TO A MULTI CENTER RANDOMIZED CONTROL TRIAL. IN THE END HE SAID HE WOULD BELIEVE MORE A MUST CENTER RANDOMIZED CONTROL TRIAL. THE SECOND PURPOSE META-ANALYSIS WHICH IS WHAT I USE IT FOR, IF YOU FIND TREATMENT EFFECTS DIFFER SUBSTANTIALLY, TRY TO EXAMINE FACTORS THAT MAY EXPLAIN THESE DURING EFFECTS. SOME TRIALS MAY HAVE SHOWED SOMETHING WAS BENEFICIAL, OTHERS SHOWED THEY WERE HARMFUL. GLOW MIGHT THE METAREGRESSION OVER CERTAIN FACTORS. YOU CAN DETERMINE WHAT WAS THE CAUSE. THIS CAN GIVE YOU GREAT INSIGHT INTO WHAT WAS THE HISTORY OF THIS THERAPY, AND WHY IN SOME INSTANCES IT WAS BENEFICIAL, WHY IT WAS HARMFUL. I FIND THIS TYPE OF META-ANALYSIS MUCH MORE INTERESTING. SO I'M GOING TO GO THROUGH THE FOUR OR FIVE STEPS OF DOING A META-ANALYSIS. THE FIRST ONE IS FORMULATING THE QUESTION. YOU HAVE TO HAVE AN INTERESTING QUESTION OR ONE THAT MAKES SENSE PAUSE THE VALIDITY IMPORTANCE OF THE STUDY ARE CONTINGENT ON THIS STEP. POORLY CONCEIVED RESEARCH HYPOTHESIS WILL USUALLY LEAD TO ANALYSIS OF DUBIOUS VALUE. THE SECOND THING IS YOU HAVE TO DEFINE PROSPECTIVELY. META-ANALYSIS SHOULD BE DONE PROSPECTIVELY. THE ONLY THING DIFFERENT FROM A ADMINISTRATIONIZED CONTROL TRIAL IS THE PATIENTS HAVE ALREADY BEEN INVOLVED. YOU HAVE TO WRITE OUT AHEAD OF TIME YOUR INCLUSION CRITERIA, SHOULD BE PROSPECTIVE, SYSTEMATIC, EXPLICIT. AND IDEALLY, YOU WANT TO COMBINE RANDOMIZED CONTROL TRIALS WITH A SIMILAR DIAGNOSIS, OUTCOME, PATIENT CHARACTERISTICS AND TREATMENT GROUPS. SO THERE IS TWO ISSUES HERE WITH HOW YOU COMBINE DATA AND WHETHER YOU SHOULD ONLY COMBINE RANDOMIZED CONTROL TRIALS OR TRY TO ANALYZE OBSERVATIONAL STUDIES. THE ADVANTAGE OF IMCLUDING ALL THE AVAILABLE STUDIES REGARDLESS OF THE SIZE, DESIGN, OR QUALITY RESULTS OF ANALYSIS IS THAT IT'S BROADLY REPRESENTATIVE OF THE CLINICAL EXPERIENCE. BUT BECAUSE OF THIS, IT MAY COMPROMISE THE ACCURACY. ALTERNATIVELY, EXCLUSION OF POORLY DONE STUDIES MAY INCREASE THE STATISTICAL VALTY, BUT LIMIT THE ABILITY TO GENERALIZE RESULTS. IF YOU ONLY USE VERY CONSISTENT RANDOMIZED CONTROL TRIALS, YOU MAY NOT GET THE FULL SPECTRUM OF WHAT HAS HAPPENED. WHAT I ACTUALLY LIKE TO DO IS INCLUDE THEM BOTH. OBSERVATIONAL STUDIES AND RANDOMIZED CONTROL TRIALS AND ANALYZE THEM BOTH AND SEE IF THEY HAVE SIMILAR FINDINGS OR DIFFERENT FINDINGS SO YOU GET THE FULL CLINICAL EXPERIENCE AT THE SAME TIME YOU GET THE DATA WHICH HAS THE MOST CONSTITUENTAL -- STATIS TICAL VALIDITY. META-ANALYSIS, AFTER YOU HAVE YOUR QUESTION, USUALLY BEGINS WITH A SEARCH. MOST OF THE TIME YOU HAVE TO GO TO A PROFESSIONAL THAT ARE AVAILABLE IN LIBRARIES, AND YOU SEARCH DATABASES, SUCH AS MED LINE, CURRENT CONTENTS, BEST EVIDENCE, COCHRAN, HEALTHS STAR. YOU HAVE CERTAIN TERMS. THEN YOU GET A LARGE NUMBER OF PAPERS USUALLY AND YOU SEARCH THE TITLE ABSTRACTS TO FIGURE OUT WHICH OF THE STUDIES ARE PERTINENT TO WHAT THE QUESTION YOU ARE ASKING. AND THEN YOU GO THROUGH THOSE STUDIES WHICH ARE PERTINENT, WHICH THEY FIT THE DESIGN OR THE QUESTION THAT YOU'RE ASKING, AND HAVE THE APPROPRIATE DATA. YOU GO THROUGH THE FULL TEXT OF THOSE ARTICLES, AND YOU REVIEW THEM THOROUGHLY. AND YOU GO LIEU THE REFERENCE LIST TO SEE IF THERE IS ADDITIONAL ARTICLES, AND ONCE THE STUDY IS SELECTED FOR INCLUSION, THE DATA SHOULD BE EXTRACTED FROM THAT STUDY BY MORE THAN ONE REVIEWER INTO STRUCTURED FORUMS WHICH HAVE YOUR PROSPECTIVE ANALYTICAL PLAN IN THEM OF WHAT THINGS YOU'RE GOING TO ANALYZE. FOURTH STEP, YOU HAVE TO FIND A COMMON MEASURE THAT YOU'RE GOING TO USE FOR YOUR TREATMENT EFFECT. I'LL GO OVER THAT. YOU WANT TO INCLUDE AS MANY STUDIES AS YOU CAN. YOU WANT TO DEFINE YOUR OUTCOME AS LOOSEY AS YOU CAN OR SPECIFIC IF YOU WANT TO NARROW IT DOWN. THE NEXT ISSUE IS SHOULD YOU DO A FIXED VERSUS RANDOM EFFECTS MODEL TO COMBINE THE DATA. AND THIS IS IMPORTANT WHEN YOU LOOK AT PAPERS. THEY'RE JUST TWO WAYS OF ANALYZING THE DATA AND COMBINING THEM THAT HAVE DIFFERENT SOURCES OF VARIABILITY. A FIXED MODEL IN GENERAL REALLY IS TO TRY TO TELL YOU OR MAKE CONCLUSIONS ABOUT THE STUDIES. A RANDOM EFFECTS MODEL IS A LARGER SOURCE OF VARIABILITY -- TRYING TO MAKE CONCLUSIONS ABOUT THE PATIENTS. AND IN GENERAL, I THINK YOU SHOULD DO BOTH A FIXED MODEL AND -- MOST O F THE TIMES THEY WON'T AGREE. THERE IS CERTAIN SITUATIONS WHERE A FIXED IS MORE ACCURATE THAN A RANDOM EFFECTS MODEL. NOW, OTHER IMPORTANT THING TO KNOW, NOT ONLY WHETHER THEY USE A FIXED OR RANDOM EFFECTS MODEL TO ANALYZE THEIR COMBINED DATA, THE Q STATISTIC OR I SQUARED CALCULATION. NOW, I DON'T KNOW ACTUALLY HOW TO CALCULATE AN I SQUARED BUT I KNOW HOW TO INTERPRET IT. IT GOES FROM 0 TO 100%. AND WHEN YOU READ A STUDY AND YOU LOOK AT THE I SQUARED, IF IT'S BELOW 20 OR 30%, YOU FEEL COMFORTABLE COMBINING THE DATA. WHAT IS THE I SQUARED MEAN, 0- 0-20%? WHAT IDEALLY YOU LIKE IT TO BE, 0%. WHEN IT'S 0%, THE FIXED AND RANDOM EFFECTS MODEL ARE IDENTICAL. NOW, WHAT IS IT TELL YOU? IT TELLS YOU THAT THE VARIABILITY WITHIN THE STUDIES CAN EXPLAIN THE VARIABILITY ACROSS THE STUDIES. SO WHEN YOUR I SQUARED IS SMALL, IT MEANS THE VARIABILITY WITHIN A STUDY EXPLAINS THE VARIABILITY ACROSS THE STUDIES. WHEN THE I SQUARED IS 30% OR GREATER, THAT MEANS THAT THE VARIABILITY WITHIN THE STUDIES CAN'T EXPLAIN THE VARIABILITY ACROSS THE STUDIES. AND WHAT AM I SAYING? WELL, I THINK YOU CAN SEE WHERE ONE SAYS WHEN YOU HAVE AN I SQUARED GREATER THAN 30% -- I'LL GIVE YOU THIS EXAMPLE. YOU'LL HAVE A HIGH I SQUARED. SOME STUDIES SHOW SEGHARM, SOME SIGNIFICANT BENEFIT. WOULD IT MAKE SENSE TO COMBINE THEM AND SAY THERE IS NO EFFECT? IT MAKES A LOT MORE SENSE TO DO SOMETHING CALLED A METAREGRESSION. TRY TO FIGURE OUT WHAT FACTOR IN THOSE STUDIES EXPLAINS WHY SOME STUDIES ARE BENEFICIAL, SOME ARE HARMFUL. YOU REALLY -- WHEN YOU HAVE APPLES AND ORANGES YOU SHOULDN'T COMBINE STUDIES. NOW, THAT ISN'T COMPLETELY TRUE. WHAT I WAS TALKING ABOUT IS A QUALITATIVE INTERACTION, THE TERM USES, SOME STUDIES ARE ON ONE SIDE BEING BENEFICIAL, ON THE OTHER SIDE, BEING HARMFUL. YOU SHOULD NEVER COMBINE THOSE KIND OF STUDIES HAVE YOU SHOULDN'T BELIEVE A META-ANALYSIS WHICH HAS A HIGH I SQUARED. BUT IF SOME STUDIES ARE ONLY SLIGHTLY HARMFUL, AND OTHER STUDIES ARE REALLY HARMFUL IT'S PROBABLY OKAY TO COMBINE THEM AND THEN TRY TO INTERPRET WHY THE STUDIES ENGENERAL YOU'RE MAKING A CONCLUSION THAT THE STUDIES ARE HARMFUL, IT'S JUST YOU CAN'T DETERMINE WHAT THE LEVEL OF HARM IS, WHICH USUALLY IS NOT THAT IMPORTANT OR THE LEVEL OF BENEFIT. THE NEXT ISSUE IS WHEN YOU DO A META-ANALYSIS, YOU'VE GOT TO EXAMINE FOR BIAS. WHEN YOU HAVE A BENEFICIAL OR HARMFUL EFFECT, THERE SHOULD BE A NORMAL DISTRIBUTION OF THE STUDIES EFFECT SIZES AROUND THAT EFFECT. AND I'LL GO OVER WHAT META-REGRESSION IS. FUNNEL PLOT, WHAT THE I SQUARED IS, AND HOPEFULLY THOSE TERMS WILL BECOME VERY EASY, THEN, TO UNDERSTAND AS OPPOSEED TO THE DRY WAY OF PRESENTING IT NOW. THE STIFF STEP IN REPORTING RESULTS -- FIFTH STEP, THEY'VE GOTTEN TOGETHER AND MADE SORT OF RULES THAT THEY PUBLISHED IN PAPERS, AND TO IMPROVE THE QUALITY OF REPORTING FOR META-ANALYSIS THEY HAVE CHECKLISTS AND FLOW CHARTS THEY'VE PUBLISHED, CALLED THE QUORUM STATEMENT. THEY PROVIDE GUIDELINES FOR REPORTING SEARCHES, STUDY SELECTION, VALTY ASSESSMENT, DATA ABSTRACTION, STUDY CHARACTERISTICS AND DATA SYNTHESIS. WHEN YOU TRY TO PUBLISH A META-ANALYSIS THEY ASK YOU TO GIVE THOSE CHECKLISTS AND SHOW HOW YOU DO IT AND TRY TO IMPROVE THE OVERALL QUALITY, THAT YOU FOLLOWED THE PROSPECTIVE METHODOLOGY. SO NOW I'M GOING TO CHANGE THE FOCUS. NOW LET'S DO A NUMBER OF META-ANALYSIS THAT I PUBLISHED IN PAPERS AND WE'LL GO OVER WHAT AN I SQUARED -- WHAT META-REGRESSION IS. I'M A SEPSIS RESEARCHER. I TAKE CARE OF PATIENTS WITH SEPTIC SHOCK. THE FIRST THING IS THE META-ANALYSIS THAT I PUBLISHED ON COLUMN TRIALS IN SEPSIS -- CLINICAL TRIALS IN SENSE THE. THIS IS A BACTERIAL INFECTION, THE MANAGEMENT IS USUALLY EARLY RECOGNITION. YOU TRY TO HAVE A HIGH SUSPICION. YOU WANT TO GIVE THE RIGHT ANTIBIOTICS RIGHT AWAY. YOU WANT TO DO -- THESE PEOPLE DEVELOP HYPOTENSION. YOU WANT TO GIVE FLUIDS. THEIR BLOOD PRESSURE IS LOW. YOU WANT TO GIVE AGENTS WHICH WILL CAUSE -- IF THEY'RE HYPOTENSION IS SEVERE YOU WANT TO GIVE AGENTS WHICH WILL CAUSE THE BLOOD VESSELS TO RESTRICT SO THE BLOOD PRESSURES MAINTAIN. IF THERE IS A SITE OF INFECTION, YOU WANT TO REMOVE IT. AND TO SHOW YOU SOMETIMES YOU DON'T NEED ANIMALS, THIS -- DON'T NEED ANALYSIS, THIS IS -- I'LL BE SHOWING YOU THIS, THIS IS A CLASSIC CHRISTMAS TREE. THIS IS THE NO EFFECT LINE. THESE ARE THE ODDS RATIO OF SURVIVAL. HERE WOULD BE EARLY VERSES LATE ANTIBIOTICS. THESE ARE 95% CONFIDENCE INTERVALS. IF THE ODDS RATIO RANDS HERE AND IT DOESN'T TOUCH THE NO EFFECT LINE THERE IS A SIGNIFICANT BENEFICIAL EFFECT. THESE ARE ALL HOW META-ANALYSIS ARE PRESENTED. AND THIS MEANS YOU'RE ODDS OF SURVIVING ARE TEN TIMES GREATER. IF YOU HAVE APPROPRIATE ANTIBIOTICS. VERSES INAPPROPRIATE ANTIBIOTICS. THAT THE CULTURES SHOW THAT THE ANTIBIOTICS YOU STARTED WERE NOT EFFECTIVE. YOU CAN SEE HERE YOU DON'T NEED A META-ANALYSIS THAT NO MATTER WHAT YEAR THE STUDY WAS DONE, AND THE AUTHOR, THAT ALL THE STUDIES SHOWED THE SAME EFFECT. AND THAT THERE IS A BENEFIT OF GIVING THE APPROPRIATE ANTIBIOTICS. THIS IS THE BENEFIT OF GIVING ANTIBIOTICS EARLY VERS LATE. YOU CAN SEE THE ODDS RATIOS OF SURVIVAL HERE. YOU DON'T NEED ANIMALS NO MATTER WHAT THE -- NEED AN ANALYZE NO MATTER WHAT. YOU FOUND AN BENEFICIAL EFFECT WHEN YOU GAVE EARLY AND APPROPRIATE ANTIBIOTICS. NOW LET ME SHOW YOU A META-ANALYSIS WHERE YOU DID NEED TO DO ANIMALS TO FIGURE IT OUT -- TO DO AN ANALYSIS TO FIGURE IT OUT. IN THE 1980s, PEOPLE HAD THE IDEA THAT SEPTIC SHOCK WAS CAUSED BY THE BACTERIA STIMULATING INFLAMMATORY. THAT YOU HAD AN INFECTION. THE BACTERIA RELEASED PATHOGENS AND TOXINS, ENDOTOXIN, YOUR WHOLE SYSTEM WAS STIMULATED TO RELEASE INFLAMMATORY MEDIATORS, AND IT'S THESE -- THE HOST'S MEDIATORS THAT CAUSED THE SHOCK AND INJURY. AND IT LED TO MULTI ORGAN FAILURE AND DEATH. NOW, IN THE 1980s, THERE WERE HUGE NUMBER OF COUNCIL TRIALS. THE BIO TECH INDUSTRY WAS EMERGING AT THIS TIME. MULTIPLE BIO TECH COMPANIES FOLDED AND THEY COULDN'T CONFIRM THIS HYPOTHESIS. THEY CAPITOL FIND A BENEFICIAL EFFECT. AND SEPSIS WAS CONSIDERED THE BERMUDA TRIANGLE OF THE BIO TECH INDUSTRY. THE FIRST QUESTION THAT WE ASKED, IF YOU LOOKED AT ALL THESE ANTI-INFLAMMATORY AGENTS TOGETHER AND YOU DID A META-ANALYSIS, COULD THAT HELP US FIGURE OUT WHY THESE CLINICAL TRIALS FAILED? WHO I'M REGRESSING THIS META-ANALYSIS AGAINST IS THE SIZE OF THE CLINICAL TRIALS FROM 0 TO 2000 PATIENTS. AND AGAIN HERE IS OUR INFECT LINE. THIS IS THE ODDS RATIO. YOU HAVE AN ANTIINFLAMMATORY INVESTS CONTROL POPULATION, TREATED ALL THE NORMAL TREATMENTS BUT THEY DIDN'T GET AN ANTIINFLAMMATORY AGENT. THIS IS YOUR 95% INTERVAL. IF YOU HAVE THE AGENT YOU'RE TWO TIMES MORE LIKELY TO SURVIVE, 4 TIMES, 8 TIMES MORE LIKELY. AND HERE YOU'RE MORE LIKELY TO DIE. THEY SAID WERE TRIALS OF AIN'T TUMOR NECROSIS FACTOR ANTIBODIES, RELEASED BY WHITE CELLS. A PRO INFLAMMATORY CYTOKINE WHICH CAUSES INFLAMMATION. AND ONCE YOU -- WHAT YOU CAN SEE HERE, THESE ARE ALL THE TRIALS. THIS WAS 2000 PATIENTS. THERE WAS NO BENEFIT. THE ODDS RATIO CROSSES. THERE WAS NO BENEFIT HERE. THE 95% CROSSES THE NO EFFECT LINE. YOU CAN SEE IN A TRIAL OF A THOUSAND, YOU FIND THE SAME THING. HERE, YOU FIND, AGAIN, THE SAME THING. AND AGAIN, IN THIS TRIAL OF ABOUT 500 YOU FIND THE SAME THING. WHEN YOU GET BELOW 500 PATIENTS, THESE SMALL TRIALS, YOU REALLY FIND NOW IT'S ON BOTH SIDES OF THE NO EFFECT LINE AND THERE IS NO EFFECT, BUT THERE SEEMS TO BE A SMALL TREND THAT'S VERY SIMILAR WHEN YOU HAVE TRIALS OVER 500 PATIENTS, AT LEAST FOR THIS ANTITNF ANTIBODY. SO LET'S LOOK AT THE NEXT TREATMENT. A LARGE TRIAL, SMALL BENEFICIAL TREND. SHOWN IN WHITE I'VE TURNED THE ANTIBODIES, BLUE, MAO THE NEW TREATMENT ARE WHITE. YOU CAN SEE THE SAME BENEFICIAL TREND. WHEN YOU GET TO SMALLER TRIALS THE EFFECT IS MORE VARIABLE. AND IF I LOOK AT IL-1RA I FOUND SAME TREND. AND PATH RECEPTOR AN TAGNIST, ANOTHER INFLAMMATORY AGENT, THE SAME TREND. ANTI-PROSTA ENGLANDEN, WE FOUND THE SAME TREND. KENTUCKY ENTERTAIN, THE SAME TREND -- KI THIS. AN, THE SAME TREND. YOU DON'T REALLY NEED ANALYSIS. YOU SEE THE 20712 TRIALS SHOWED THE SAME BENEFICIAL TREND. WHAT IT SUGGESTED IS THAT THE MISTAKE WE MADE, THIS ISN'T SUCH A BENEFICIAL TREATMENT. IT'S A VERY MINIMAL BENEFICIAL TREATMENT. ONLY HAS A 1 OR 2% INCREASE IN SURVIVAL. AND WHEN YOU GET TO TRIALS BELOW 500 YOU SEE THIS VARIABILITY BECAUSE NOW LONGER YOU HAVE LARGE ENOUGH SAMPLE SIZE TO CAPTURE THE TREND. IT WASN'T THAT THEY FAILED, THEY OVERISN'TED THE TREATMENT EFFECT, AND THIS THERAPY REALLY MOST PEOPLE COME TO BELIEVE IS NOT VERY EFFECTIVE. IT'S MINUTE MALE EFFECTIVE. -- MINUTE MALE EFFECTIVE. BUT THE QUESTION WE ASKED, THE NEXT ONE, IS WHY DID ALL THESE AIN'T INFLAMMATORY AGENTS -- WHAT WAS COMMON ABOUT THEM THAT THEY WOULD HAVE THE SAME EFFECT? WE LOOKED AT A LOT OF DIFFERENT THINGS. THE ONE THING THAT WE FOUND WHICH WAS VERY SIMILAR OVERALL THESE CLINICAL TRIALS WAS THE CONTROL MORTALITY RATE. THIS IS UNUSUAL. THEY ALL HAD A MORTALITY RATE IN THE 30s. SO IT WAS VERY BENEFICIAL. I MEAN IT WAS VERY -- ABOUT 30%. AND SO THE QUESTION WE ASKED, SO YOU HAVE THE SMALL EFFECT OF WITHOUT A 2% BENEFIT -- ABOUT A 2% BENEFIT DONE AT THE SAME CONTROL MORTALITY RATE AT 30%. ALL THESE TRIALS STUDIED ANIMAL MODELS. AND IN THE ANIMAL MODELS, ALL THESE ANTI-INFLAMMATORY BENEFITS WERE VERY BENEFICIAL. 95 ANIMAL STUDIES WERE DONE. WE ASKED WHAT WAS THE CONTROL MORTALITY RATE THAT THEY DID IN THE ANIMAL MODELS? WAS IT THE SAME AS THE CONTROL MORTALITY RATE DONE IN THESE HUMAN CLINICAL TRIALS? SO IN THE NEXT SLIDE I'LL SHOW YOU DATA FROM THE 95 ANIMAL STUDIES THAT WERE DONE, REFERENCED IN THESE CLINICAL TRIALS. I'LL SHOW YOU THE CONTROL MORTALITY RATE AND THE TREATMENT EFFECT AND I'LL DO A META-REGRESSION. SO NOW HERE AGAIN I'VE JUST TURNED THE CHRISTMAS TREE ON ITS SIDE. THIS IS THE NO EFFECT LIEN. THIS IS BENEFIT, TEN TIMES MORE -- INCREASE IN THE ODD RATIO OF RIFLE, 100, 1,000. HERE IS MORE HARMFUL. AND EACH ONE OF THESE, THE SIZE OF THESE CIRCLES IS PROPORTIONAL TO THE NUMBER OF ANIMALS IN THESE CLINICAL TRIALS. THE FIRST THING YOU NOTICED WHEN YOU LOOK AT THIS, ALMOST NOBODY DID A CLINICAL TRIAL BELOW A MORTALITY RATE OF 50%. THIS IS THE ODDS RATIO OF DYING. ONE IS A 50% MORTALITY IN THE CONTROL GROUP. THIS IS 100%. THIS IS LESS THAN A 1%. YOU SEE AT A 50% MORTALITY, THERE IS ALMOST NO STUDIES. THERE IS ONLY THESE TWO. AND THEY HAD VERY LOW CONTROL MORTALITY RATES, AND THEY WERE HARMFUL. WHEN I WENT TO COMPANIES AND I GAVE THIS TALK, THE SCIENTISTS THERE WOULD SAY WE KNEW THAT. ANY TIME WE HAD -- WOE USED A LOW MORTALITY RATE MODEL OF INFECTION, THE AGENTS WERE HARMFUL. WE THREW THE DATA OUT. SO FIRST THING. NEXT THING, WHAT YOU SEE IS DESPITE ALL THESE STUDIES BEING ABOVE A 50% MORTALITY RATE, THERE IS STILL A RELATIONSHIP. AND IT'S HIGHLY SIGNIFICANT. AND THIS IS BENEFICIAL. THIS IS HARMFUL. BUT AS YOU GO, EVEN WITH THIS CONSTRAINED RANGE, FROM A HIGH MORE TALLRY RATE IN THE CONTROL GROUP, THERE IS STILL A SIGNIFICANT RELATIONSHIP AND AS YOU GO DOWN, THE TREATMENT EFFECT GETS SMALLER. SO NOW LET'S GO BACK, WE'VE DONE THIS META-REGRESSION SHOWING MORTALITY RATE IS IMPORTANT. LET'S BLOT THE HUMAN DATA ON THIS GRAPH. WHERE WOULD IT FIT? IT WAS DONE AT A 30 TO 40% MORTALITY. IT FITS PRETTY GOOD. EXACTLY WHERE YOU'D EXPECT IT. THE TREATMENT EFFECT IS MUCH LOWER CONTROL MORTALITY, MUCH LOWER AS THE ANIMAL DATA WOULD SUGGEST IT. THE NEXT QUESTION YOU WOULD ASK, ALL THIS IS RETRO EXPECTATIVE DATA. CAN I CUE AN EXPERIMENTAL PROSPECTIVELY AND TEST THIS HYPOTHESIS THAT THE CONTROL -- THAT THE TREATMENT EFFECTS IN THE HUMAN CLINICAL TRIALS WERE VERY SMALL. THE CONTROL MORTALITY RATE WAS VERY SIMILAR AT 30%. ONLY HIGH CONTROL MORTALITY RATES OF 70, 80% THAT ANTI INFLAMMATORIES ARE BENEFICIAL. WE TOOK ANIMAL MODELS AND GAVE THEM DIFFERENT DOSES OF BACTERIA AND TOXINS, TO PRODUCE DIFFERENT MORTALITY RATES, TO TEST IF OUR HYPOTHESIS PROSPECTIVELY, AT LEAST IN ANIMAL MODELS, WAS TRUE. AS YOU SEE HERE, WE HAD DIFFERENT SIZED STUDIES AGAIN. THIS IS THE NO EFFECT LINE, BENEFIT, HARM, THE CONTROL ODDS OF DYING. AND WHAT YOU SEE HERE, IF YOU'RE ABOVE A 50% MORTALITY, THINGS LOOK PRETTY GOOD. IF YOU'RE BELOW A 50% MORTALITY, HERE, THINGS DON'T LOOK SO GOOD. AGAIN, THERE IS A SIGNIFICANT RELATIONSHIP WITH THESE AIN'T INFLAMMATORY AGENTS, IN THESE MODELS, AND USING DIFFERENT TYPES OF TREATMENTS AND DIFFERENT TYPES OF BACTERIAL CHALLENGES. IF WE DID IT IN A CHARGE ANIMAL MODEL WE FOUND THE SAME THING. IF YOU PLOT THE HUMAN CLINICAL TRIALS IT'S EXACTLY WHERE YOU EXPECT. IN SUMMARY, USING META-ANALYSIS AND META-REGRESSION TECHNIQUES WE WERE ABLE TO FIGURE OUT, WE THINK, THAT THE TREATMENT EFFECTS WERE VERY SMALL IN THESE CLINICAL TRIALS DONE IN HUMANS, BUT STATISTICALLY SIGNIFICANT. BENEFICIAL AT HIGH RISK OF DEATH BECAUSE WHEN YOU HAVE A VERY, VERY SEVERE INFECTION WITH A VERY SEVERE INFLAMMATORY RESPONSE, THAT INHIBITING INFLAMMATION OVERALL HAS A BENEFICIAL EFFECT. BUT IF YOU HAVE -- IT'S INEFFECTIVE OR HARMFUL WHEN RISK IS LOW. WHEN THE INFLAMMATORY RESPONSE IS WORKING WELL AND YOU'RE GOING TO SURVIVE IT DOESN'T -- IT'S NOT BENEFICIAL TO INHIBIT IT. IT WILL INCREASE MORTALITY BECAUSE YOU'RE BLOCKING THE INFLAMMATORY RESPONSE. THIS IS A VERY DIFFICULT TREATMENT TO GIVE. AND VERY DIFFICULT TREATMENT TO USE CLINICALLY. SO I'M GOING TO DO ANOTHER METAREGRESSION. CORTICOSTEROIDS, THOSE ARE STEROIDS TESTED IN SEPTIC SHOT FOR OVER 50 YEARS. I'M NOW GOING TO DO A META-REGRESSION TO FIGURE OUT THE EFFECT OF THE STEROIDS IN SEPSIS OVER THE LAST 50 YEARS. THEY HAVE BEEN INVESTIGATED SINCE THE 1960s, IN THE EARLY 1990s THEY WERE SHOWN TO BE IN EFFECTS THAT WERE HARMFUL BUT THERE BECAME A RENEWED INTEREST AFTER THE 1990s IN LOWER DOSES. IN SEPTIC SHOCK. SO NOW, AGAIN, HERE IS MY NO EFFECT LINE. THIS IS PEN, THIS IS HARM. I'M GOING TO LOOK AT THE ODDS RATIO SURVIVAL. NOW I'M GOING TO DO A META-REGRESSION ACROSS THE YEAR THE STUDY WAS ESTABLISHED. THAT'S THE FACTOR WHICH IS INFORMATIVE. BEFORE I LOOK AT CONTROL MORTALITY, NOW I'M GOING TO LOOK AT THE YEAR PUBLISHED. THESE WERE THE STUDIES PLEASURED FROM 1963 TO 1998. THEY USED VERY HIGH DOSES AND SHORT COURSES. FOCUS ON THIS ONE STUDY. SO REMEMBER, I TOLD YOU YOU WANT TO LOOK AT THE I SQUARED. WHEN YOU DO A META-ANALYSIS. AND HERE IT'S 70%. SO YOU WOULD SAY I'M NOT GOING TO COMBINE THIS DATA BECAUSE THIS STUDY IS SHOWING BENEFIT AND THIS STUDY IS SHOWING HARM. AND YOU SAY, OKAY, WHAT IS IT ABOUT THIS STUDY? BECAUSE YOU CAN'T JUST EXCLUDE A STUDY BASED ON A STATISTICAL ANALYSIS. YOU HAVE TO HAVE A METHODOLOGICAL REASON. THIS WAS A SINGLE CENTER UNBLINDED STUDY DONE BY A SINGLE SURGEON OVER TEN YEARS AND HE WAS RUNNING AN OBSERVATIONAL STUDY AND A RANDOMIZED CONTROL TRIAL AT THE SAME TIME. I'M SORRY TO SAY WE ACTUALLY CHANGED OUR CARE AND STARTED DOING THIS TREATMENT BASED ON THIS STUDY AT THE NIH. WE STARTED USING HIGH DOSAGE STEROIDS. IF YOU EXCLUDE THIS STUDY, WHICH WAS DOPE VERY DIFFERENTLY. THESE WERE MOSTLY SMUT CENTERED RANDOMIZED CONTROL TRIALS, THE I SQUARED GOES TO 0. THE VARIABILITY GOES AWAY. AND YOU FIND, ACTUALLY, THAT HIGH DOSE STEROIDS SHORT COURSES, ACTUALLY IS HARMFUL. SO I HOPE YOU GOT THE RESCUED OF WHAT THE I SQUARED TELLS YOU. AGAIN, META-REGRESSION BY YEARS. SO SHORT COURSES OF VERY HIGH DOSE OF STEROIDS, HAVE IS A HUGE DOSE. EQUIVALENT TO 24 HOUSE MILLIGRAMS OF HYDRO CORTISONE WORSENED SURVIVAL. TEN YEARS LATER, PEOPLE STARTED STUDYING THEM AGAIN. AND YOU FIND NOW THAT THERE IS A VERY DIFFERENT APPEARANCE THAN THIS ONE. AND THIS IS A SIGNIFICANTLY DIFFERENT EFFECT NOW. AND THE FIRST THING, WHAT IS THE I SQUARED. SO STRESS DOSED STEROIDS TAPERED OVER 67 DAYS, GIVING OVER -- 6 DAYS IMPROVED SURVIVAL. WHAT'S THE I SQUARED? IT'S 25%. SO BORDER LINE. BUT YOU STILL ARE WORRIED ABOUT IT. IT'S -- YOU LIKE THE I SQUARED TO BE 0 BUT IT MAY BE ACCEPTABLE. BUT IT TURNS OUT IT'S CAUSED PIE THIS TRIAL. IF YOU REMOVE IT THE I SQUARED GOES TO 0. WHETHER YOU REMOVE IT OR NOT THERE IS STILL BENEFIT. SO YOU NEED TO DO SOME INVESTIGATING HERE. SO SINCE STEROIDS ARE ANTIINFLAMMATORY, WE WENT BACK TO OUR CONTROL MORTALITY AS THE METAREGRESSION. SO WHY IS THE CORTICO STUDY DONE PER THE 11 OTHER TRIALS? REMEMBER, IT COULD BE THE COURTCUS IS RIGHT, THE OTHER TEN TRIALS ARE WRONG OR THEY COULD BOTH BE RIGHT BUT THEY'RE SHOWING DIFFERENT EFFECTS SO YOU WANT TO FIGURE THIS OUT. SO WHEN WE GRAFT THIS AGAINST THE CONTROL ODDS OF DEATH, WE FOUND THIS SAME RELATIONSHIP AGAIN. WHICH ONE WAS THE COURTICUS? IT TURNS OUT IT'S NOT UNEXPECTED WHAT THEY FOUND. WELL, THIS IS A LARGE MULTICENTER RANDOMIZED CONTROL TRIAL. AND THESE ARE RELATIVELY SMALL TRIALS. 40 OR 50 PATIENTS. THIS ONE, MUST CENTER, RANDOMIZED CONTROL TRIAL YOU BELIEVE AND IT SUGGESTS TO YOU THAT AT LEAST AT LOW MORE TALLRY RATES, THERE IS NO BENEFIT OF STEROIDS IN NOT SUCH SICK PATIENTS. IN VERY SICK PATIENTS THIS DATA SUGGESTS, WELL, MAYBE IT'S BENEFICIAL. WELL, THESE ARE SMALL STUDIES. SO NOW WE NEED TO LOOK AT THE ISSUE OF PUBLICATION BIAS, BECAUSE IF YOU HAVE A SMALL STUDY, PEOPLE TEND TO ONLY PUBLISH THEM IF THEY'RE BENEFICIAL. IF THEY'RE NOT, THEY CALL IT THE DRAWER EFFECT. THEY TEND TO NOT PUBLISH THEM. WE NEED TO DO NOW, A FUNERAL PLOT. THAT'S WHAT I DISCUSSED IN THE BEGINNING. WHAT IS A FUNNEL PLOT? LET'S GO THROUGH. THAT WILL TELL YOU IF THERE IS A PUBLICATION BIAS. HERE IS THE LOG ODDS RATIO OF DEATH. I'M SORRY THIS IS REVERSED. CONTROL IS HERE AND STEROIDS ARE HERE. I SHOULD HAVE REVERSED THIS. I HAVEN'T DONE IT YET. LET'S NOW LOOK AT THE TREATMENT EFFECT. SO THIS SAYS THAT STEER ROADS IN THIS META-ANALYSIS OF THESE SMALL -- OF THESE LOWER DOSES WAS BENEFICIAL. HERE IS THE TREATMENT EFFECT. THERE IS A SMALL BENEFICIAL EFFECT. OFTEN ON OLIVE OIL. SO WHAT YOU'RE GOING TO LOOK AT, WHAT'S THE DISTRIBUTION OF THE STUDIES AROUND THIS EFFECT SIZE? THEY SHOULD BE EQUALLY DISTRIBUTED ON BOTH SIDES OF THIS EFFECT SIZE. WHEN HE LOOK AT IT ANOTHER WHAT YOU FIND IS THERE IS THESE BUNCH OF SMALL STUDIES THAT WAY OVER HERE. NOW, THESE ARE THE STUDIES THAT ARE 40 OR 50 PATIENTS. THIS IS DRIVING THE BENEFICIAL EFFECT. AND YOU CAN ACTUALLY PLOT -- WHY IT'S CALLED A FUNNEL PLOT. WHERE YOU EXPECT STUDIES BASED ON THE TREATMENT EFFECT TO RESIDE. THAT'S WHAT THIS RED LINE SHOWS. SO THE STUDIES SHOULD BE WITHIN -- IN ORDER FOR THIS EFFECT TO NOT BE BIAS, TO BE NORMALLY DISTRIBUTED, ALL THESE DOTS SHOULD BE EQUALLY DISTRIBUTED IN HERE, IN THIS FUNNEL. WHAT YOU'LL FIND IS ONE, TWO, THREE, FOUR -- OF THESE SMALL STUDIES ARE PREJUDICING THE DATA SAYING IT'S BENEFIT. YOU WOULD HAVE EXPECTED THERE TO BE EQUAL NUMBER OF STUDIES OVER HERE. IF THIS WAS TRULY AN EFFECT. WHAT YOU BELIEVE IS, THESE ARE THE STUDIES THAT ARE SITTING IN SOMEBODY'S DRAWER. THEY WERE NOT BENEFICIAL. THEY WERE SMALL, DIDN'T THINK ANYBODY WOULD PUBLISH IT. THAT'S WHAT PUBLISHINGS BIAS IS. THAT'S WHAT A FUNNEL PLOT IS. THAT'S HOW YOU DETERMINE IT AND WHY YOUDITE. THE SMALL STUDIES ON THE WOULD THEEM, LARGE ON THE TOP. YOU CAN SEE THAT THERE IS A PREPONDERANCE OF SMALL STUDIES OVER HERE AND NOT OVER HERE. AND THAT'S THE PUBLICATION BIAS. SO DO CORTICOSTEROIDS HAVE ANY EFFECTS THAT ARE CONSISTENT, DIFFERENT FROM SURVIVAL? IT TURNS OUT REVERSAL OF SHOCK OR REVERSING THE BAD BLOOD PRESSURE OCCURS EARLIER IF YOU GIVE STEROIDS. THIS IS A VERY CONSISTENT EFFECT. I SQUARED IS 0. ALL THE STUDIES FOUND THE SAME THING. SO THIS EFFECT YOU BELIEVE. LET'S SUMMARIZE THIS, WHAT WE LEARNED OVER FIVE YEARS OF DATA. DESPITE FIVE YEARS OF RESEARCH, WE STILL DON'T KNOW IF STEROIDS ARE BENEFICIAL OR FOOT. WE ONLY KNOW THEY REVERSE SHOCK. EFFECTS DEPENDS ON DOSE. HIGH DOSES ARE HARMFUL. THEY SEEM TO BE MORE BENEFICIAL IN LOW DOSES IN VERY SECURE PEOPLE. HIGH DOSES CORTICOSTEROIDS INCREASE MORTALITY, LOW DOSE IMPROVE SURVIVAL IN SEVERELY LAILL PATIENTS. LOW DOSE STEROIDS DECREASE VASO DEPRESSOR REQUIREMENTS AND ENHANCE OR REVERSE SHOCK. BUT AT PRESENT WE CAN'T REALLY KNOW WHAT'S GOING ON BECAUSE THE BENEFICIAL EFFECT OF LOW DOSE STEROID ARE BASED ON SMALL TRIALS WITH ONLY 40 PATIENTS, 40-4 FOURTH OF JULY THEY'RE CONFOUNDED BY PUBLICATION BIAS. THE LARGEST TRIAL, WE ACTUALLY CAN BELIEVE, AND IT TELLS US IN NOT SUCH SICK PATIENTS THAT CORTICOSTEROIDS ARE NOT WOMEN. AND AT -- ARE NOT BENEFICIAL. AND AT PRESENT, UNTIL A LARGE MULTI CENTER TRIAL TELLS US WHETHER OR NOT IN VERY SICK PATIENTS CORTICOSTEROIDS ARE BENEFICIAL, YOU REALLY DON'T KNOW TO USE THEM OR NOT. YOU HAVE NO DATA EXCEPT THAT IT WILL REVERSE SHOCK. SO UNTIL NEW DATA ARE AVAILABLE YOU HAVE TO USE CLINICAL JUDGMENT AND WHAT IS THE RISK BENEFIT TO ADMINISTRATION LOW DOSE STEROIDS FOR SEPTIC SHOCK. IT HAS TO BE INDIVIDUALIZED. IF YOU THINK SOMEBODY IS VERY SICK AND THEY'RE GOING TO DIE, IT WILL REVERSE SHOCK. STEROIDS CAN INCREASE THE RISK OF INFECTION AND IMMUNOSUPPRESS YOU. YOU HAVE TO DECIDE THE RISK. SO WHAT I'M TRYING TO SAY, BY ANALYZING THE DATA, LOOKING AND DOING BETTER REGRESSION, META-ANALYSIS, LOOKING AT PUBLICATION BIAS, META-REGRESSION, WE'RE ABLE TO FIND THIS DATA FIGURE OUT WHAT IT SHOWS. SO LET'S LOOK NOW ANOTHER SET OF STUDIES AND THIS IS SORT OF A SAD DAY -- THING IN THE MEDICAL INDUSTRIAL COMPLEX, WHAT HAPPENED WITH INTENSIVE INSULIN THERAPY. SO THEY HAD -- SOME PEOPLE IN BELGIUM, AN THESOLOGIST HAD THE IDEA WE SHOULD -- DIABETICS, IF YOU KEEP THEM UNDER VERY CLOSE CONTROLS, THERE IS A BELIEF THEY GET LESS VASCULAR DAMAGE. LET'S TAKE SOMEBODY WITH SHOCK OR SEVERELY ILL AND KEEP THEM UNDER VERY TIGHT CONTROL HAVE. LET'S SEE IF THAT HAS A BENEFIT, LIKE THE BENEFIT YOU SEE OVER 20 TO 30 YEARS WHEN YOU TREAT SOMEBODY WITH DIABETES. TIST PUBLISHED IN THE NEW ENGLAND JOURNAL, AND SHOWED A BENEFIT FROM KEEPING THE GLUCOSE BELOW 100 IN PEOPLE WITH SHOCK. THIS CLINICAL TRIAL WAS ENDORSED SINGLE, UNBLINDED TRIAL, LIKE THE STEROID TRIAL I TOLD YOU. SINGLE, UNBLINDED TRIAL. CAUSED THE HOSPITALS TO ADOPT TIGHT GLUCOSE CONTROL. THE AMERICAN COLLEGE OF ENDOCRINOLOGY, THE VOLUNTEER HOSPITAL ASSOCIATIONS, THE INSTITUTE FOR HEALTHCARE IMPROVEMENT, THE SURVIVALING SEPSIS CAMPAIGN. EVERY ONE BASED ON THIS SINGLE CENTER UNBLINDED TRIAL ADOPTED THIS THERAPY. SO LET'S LET'S LOOK AT THE STUDY MORE CAREFULLY. WHO DID THEY STUDY WITH THIS TIGHT GLUCOSE CONTROL USING HIGH DOSES OF INSULIN. IN MAJORITY OF PATIENTS, 63%, WERE CARDIAC SURGERY. THEY STUDIED MOSTLY PEOPLE WITH CORONARY DISEASE, HAVING -- PLACING CABBAGES OR PLACING VEINS IN THE CORONARIES TO INCREASE THE FLOW. AND THESE PEOPLE USUALLY GO HOME IN ONE, TWO OR THREE DAYS. THIS IS AN OPERATION WITH A VERY LOW MORTALITY RATE. 1 OR 2%. WHO DID THEY FIND THEY BENEFITED? IT WAS MOSTLY THE CARDIAC SURGERY. THIS WAS THE DIFFERENCE IN DEATHS. YOU CAN SEE THE DIFFERENCE WAS MOSTLY IN CARDIAC SURGERY AND THORACIC SURGERY THAT MADE UP THIS BENEFICIAL EFFECT, KEEPING THE GLUCOSE TIGHTLY CONTROLLED, BETWEEN 80 AND 90 IN PATIENTS WITH CARDIAC SURGERY. SO IT WAS A SINGLE CENTER, UNBLINDED STUDY THE THING FUNNY, IT HAS A RELATIVELY HIGH MORE TALLRY RATE. IN THE STATE OF MASSACHUSETTS THERE ARE 11 INTENSIVE CARE UNITS. IF ANY HAD THIS HIGH A MORTALITY RATE, THEY WOULD ALL BE SHUT DOWN. THIS IS MORE THAN DOUBLE THE MORTALITY RATE FOR THIS PROCEDURE. THE OTHER THING THAT WAS FUNNY ABOUT THIS STUDY, THEY WANTED TO GIVE INSULIN. WHAT THEY DID WAS THEY TOOK THE PATIENTS POST OPERATIVELY, AND AS SOON -- THESE ARE PEOPLE GOING HOME IN TWO OR THREE DAYS. THEY GAVE THEM HIGH GLUCOSE LOADS. THESE ARE ENORMOUS DOSES TO GIVE THEM INSULIN. THIS REALLY WASN'T A CONTROL. THIS WAS REALLY THE CONTROL -- HIGH DOSES OF INSULIN AND HIGH TOASTS OF GLUCOSE CAN GIVE YOU INFECTIONS. AND IT'S NOT ROUTINE CARE. WHAT I'M SAYING, IF YOU HAD PEOPLE WITH CORONARY -- HEART DISEASE AND YOU GAVE THEM SOMETHING TO RAISE THEIR BLOOD PRESSURE, WHICH WOULD INCREASE THEIR RISK OF A HEART ATTACK AND GAVE THEM SOMETHING TO DECREASE THEIR BLOOD PRESSURE AND YOU FOUND THAT THAT WAS BENEFICIAL, YOU WOULDN'T CALL IT BENEFICIAL. THAT'S WHAT THEY DID. THEY TOOK PEOPLE, MADE THE GROW DOES HIGH AND TREATED THEM WITH INSULIN, SOME OF THEM. SO ONCE THIS STUDY WAS ESTABLISHED AND MOST OFF THESE THINGS TAKE ABOUT TEN YEARS TO REVERSE, YOU CAN SEE HERE NOW THAT THERE ARE MULTIPLE OTHER STUDIES DONE. AND NOBODY COULD REPRODUCE THIS EFFECT. THIS IS THE STUDY I SHOWED YOU. NOBODY COULD REPRODUCE THIS EFFECT. IN 27 TRIALS, TIGHT. >> DOES CONTROL. THIS ACCEPTABLE. AND OVERALL, THERE WAS NO BENEFIT. SO NOW WHAT I'M GOING TO SHOW YOU AND THE PURPOSE OF THIS IS TO DISCUSS SENSITIVITY ANALYSIS. WHEN YOU DO A META-ANALYSIS, YOU WANT TO DO A SENSITIVITY ANALYSIS. YOU DON'T CARE IF THERE IS SEGP VALUE. YOU WANT TO SEE IF THE EFFECT SIZE IS CONSISTENT. YOU WANT TO MAKE SURE IT'S NOT DUE TO ONE PARTICULAR TYPE OF CRITICALLY ILL PATIENT. I'LL SHOW YOU WHAT A SENSITIVITY ANALYSIS IS. THE PURPOSE IS TO SHOW EFFECT IS CONSISTENT OVER THE DIFFERENT TYPES OF TRIALS YOU DID. IF YOU LOOK AT PATIENTS THAT HAD GLUCOSES OF LESS THAN 110, THEY DIDN'T HAVE SO TIGHT -- RATHER THESE ARE THE TIGHT CONTROL. THE EFFECT SIZE SAY NO BENEFIT. IF YOU LOOK AT THE LESS MODERATE CONTROL, 150, THERE IS STILL NO BENEFIT. THE YOU LOOK AT SURGICAL ICUs, STILL MOW BENEFIT. IN MEDICAL ICUs, STILL NO BENEFIT. MED SUMMERICAL, STILL NO BENEFIT. THIS WAS NOT WOMEN NO MATTER WHERE YOU LOOKED -- BENEFICIAL NO MATTER WHERE YOU LOOKED. AND REALLY, THEY COULDN'T REPRODUCE THIS ORIGINAL STUDY EFFECT. SO WHAT WAS THE PROBLEM OR WHY WAS THERE A HARMFUL EFFECT? ONCE YOU FOUND -- WHAT YOU FOUND, IN THOSE STUDIES THAT REPORTED IT, WHETHER THE GLUCOSE WAS LESS THAN A TEN, YOU GOT HYPOGLYCEMIA. IF YOU KEPT IT LESS THAN 150 ACTION YOU STILL GOT HYPOGLYCEMIA. THIS IS SUGARS LESS THAN 40. IF YOU USE SURGICAL ICUs, LESS THAN 40. MEDICAL ICUs, IT WAS HYPOGLYCEMIA. MED SURGICAL ICUs. SO OVERALL THE I SQUARED IS 0. WHAT HAPPENED IN THESE STUDIES, THERE WAS A RISK OF HYPOGLYCEMIA WHICH INCREASED OR MADE THE OUTCOME WORSE. IN AUSTRALIA, THEY DECIDED TO DO A VERY LARGE STUDY AND TO COMPARE GLUCOSE CONTROL IN THE STUDY OF 6,000 PATIENTS IN MEDICAL ICUs. THERE WOULD BE A SINGLE STUDY DONE IN AUSTRALIA, NEW ZEALAND AND CANADA. AND THIS STUDY FOUND THAT TIGHT GLUCOSE CONTROL INCREASED 7-8 FOLD RISK OF HYPOGLYCEMIA, INDEPENDENT OF TARGET GLUCOSE. THIS PAPER TEN YEARS LATER WAS PUBLISHED. THEY DO 6,000 PATIENTS AND TEST THIS TIGHT GLUCOSE CONTROL. THEY ENROLLED 6,000 PATIENTS. 3,000 INTENSIVE, 3,000 CONVENTIONAL. THESE WERE SURGICAL PATIENTS. THIS MEANS 31% WERE VERY SICK, 21% HAD SEPSIS. MOST OF THEM WERE ON MECHANICAL VENTILATION. THEY FOUND TIGHT GLUCOSE CONTROL SIGNIFICANTLY INCREASED MORTALITY. JUST LIKE HIGH DOSE STEROIDS. SAME PROBLEM. IN 6,000 CRITICALLY ILL PATIENTS, INCREASED MORTALITY RISK IN 90 DAYS. ON THE BASIS OF THIS WE DO NOT RECOMMEND USE OF LOWER TARGET LESS THAN 110 IN CRITICALLY ILL PATIENTS. SO I HAVE BEEN DOING META-ANALYSIS FOR A LONG TIME, STUDIES IN THE FIELD OF SEPSIS. WHAT WENT WRONG WITH THESE STEROIDS AND WHAT WANT WRONG WITH THE TIGHT GLUCOSE CONTROL. IT'S QUITE A PATTERN. IF I LOOK AT ALL THE TRIALS THAT WERE DONE, THE META-ANALYSIS, THERE WAS ONE SIGNIFICANT BENEFICIAL EFFECT. THEN THE SUBSEQUENT TRIALS IS WHAT I'M GOING TO SHOW YOU IN THE NEXT ANALYSIS. AND TO TRY TO EXPLAIN WHAT WENT WRONG. SO IN SEPSIS THEY TRIED ENDOTEXAN -- ANTI ENDOTOXIN. THEY TOOK FIREMAN AND INJECTED THEM WITH ENDOTOXIN. THEY PUBLISHED A PAPER IN THE NEW ENGLAND JOURNAL THAT SAID THIS WAS SIGNIFICANTLY BENEFICIAL. WELL, SO THIS IS AS I'VE SHOWN YOU MUCH BENEFIT, HARM, THE NO EFFECT LINE. YOU CAN SEE ALL THE SUBSEQUENT PEOPLE THAT TRIED TO PRODUCE THIS RESULT COULDN'T REPRODUCE IT. THEY ALSO TRIED A MONOCLONAL ANTIBODY AGAINST ENDOTOXIN. AGAIN PUBLISHED IN THE NEW ENGLAND JOURNAL. SIGNIFICANT EFFECT. YOU CAN SEE SUBSEQUENT TRIALS COULDN'T REPRODUCE THIS EFFECT. WHEN YOU COMBINE ALL THE PLU ONES, P VALUE OF .6 THAT COULDN'T REDUCE THE RED ONE SIGNIFICANTLY BENEFICIAL. HERE .59. HI DOSE CORTICO STORY ROUTES. THIS WAS THAT STUDY BY THE SURGEON. ALL THE SUBSEQUENT STUDIES COULDN'T REPRODUCE IT. SUBSEQUENT STUDIES SHOWED HARM. ACTIVATED PROTEIN C, THIS WAS AN ANTICOMPANYING ALENT THAT THEY USED IN SEPTIC PATIENTS, PUBLISHED AGAIN IN THE NEW ENGLAND JOURNAL. SUBSEQUENT STUDIES COULDN'T REPRODUCE THIS EFFECT. P VALUE .5. INTENSIVE INSULIN, PUBLISHED IN THE NEW ENGLAND JOURNAL, AGAIN, BENEFICIAL EFFECT. I SHOWED YOU, SUBSEQUENTLY COULDN'T REPRODUCE IT. .32 IF YOU PUT THESE ALL TOGETHER. IF YOU LOOK AT THE SHIFT ARE THE BENEFICIAL TO THE LAST STUDY, THERE IS A SIGNIFICANT SHIFT. WHAT DO I THINK WENT WRONG? MOST PEOPLE THINK WHEN YOU RANDOMIZED, MAGIC OCCURRED. SUDDENRY YOU HAVE DATA VERY INTERPRETABLE, VERY IMPORTANT. BUT ALL RANDOMIZATION DOES IS GET RID OF ONE FORM OF BIAS, SELECTION BIAS. WHAT REALLY IS IMPORTANT IN SCIENCE IS REPRODUCIBILITY. RANDOMIZED CONTROL TRIAL, MINIMIZE BIAS, DOES NOT ELIMINATE THE NEED FOR REPRODUCIBILITY. I THINK THAT'S WHAT HAPPENED IN THESE CLINICAL TRIALS. IT'S THE ESSENTIAL CONDITION OF SCIENTIFIC EVIDENCE, IF YOUR FINDINGS ARE REPRODUCIBLE IN OTHER PEOPLE'S HANDS. SO I HOPE THAT WITH MY EXAMPLE OF SEPSIS IN CLINICAL TRIALS AND A LITTLE BIT OF WHAT I GAVE YOU BEFORE, YOU KNOW NOW WHAT A META-REGRESSION IS. WHAT PUBLICATION BIAS AND FUNNEL PLOT IS, WHY YOU DO SENSITIVITY ANALYSIS, WHAT THE I SQUARED MEANS, AND HOW YOU INTERPRET AN I SQUARED, HOW IT HELPS YOU ANALYZE THE STUDY. I NOW WILL OPEN THIS UP FOR QUESTIONS IF THERE ARE ANY ABOUT WHAT I JUST PRESENTED. THANK YOU. [APPLAUSE] YOU NEED TO SPEAK LOUD. [INAUDIBLE QUESTION] >> DO YOU WANT TO USE THE MICROPHONE? >> NO. MOST OF THEM USE HYDRO COURTIZEN. THAT IS A WEAKNESS IN THE STUDY. WE DIDN'T FIND THAT -- WHEN WE LOOKED WITHIN THE DIFFERENT GROUPS WE DIDN'T FIND THERE WAS VARIABILITY THAT THE I SQUARED WAS ACCEPTABLE, SO IT DIDN'T SEEM TO BE A CAUSE OF VARIABILITY. WE THOUGHT IT'S OKAY TO ARCH OVER THOSE STUDIES. THAT'S WHY YOU DO AN I SQUARED. AT LEAST STATISTICALLY WE COULDN'T FIND A DIFFERENCE. YOU MAY WANT TO DO SENSITIVITY ANALYSIS, AND LOOK AT THE DIFFERENT MANUFACTURERS AND SEE THAT THE EFFECT IS THE SAME USING A SENSITIVITY ANALYSIS, SO THAT IT ISN'T JUST ONE TYPE OF MANUFACTURER OF SISTER ROUTES CAUSING THE PROBLEM. >> [INAUDIBLE QUESTION] >> SURE. MAY WANT TO DO THAT WITH SENSITIVITY ANALYSIS. THAT'S WHY YOU DO SENSITIVITY ANALYSIS, WHEN YOU DO THE I SQUARED, IT'S OKAY TO PUT IT TOGETHER, AND WHEN YOU LOOKED AT THE STUDIES BY THE YEAR, THE META-REGRESSION, THAT HELPED YOU UNDERSTAND WHAT