>> GOOD MORNING. THANK YOU FOR COMING TO TODAY'S MEDICINE MIND THE GAP LECTURE SERIES, FEATURING DR. SCOTT RICHARDSON. MIND THE GAP IS A LECTURE SERIES THAT EXPLORES ISSUES AT THE INTERSECTION OF RESEARCH, EVIDENCE AND CLINICAL PRACTICE. AREAS IN WHICH CONVENTIONAL WISDOM MAY BE CONTRADICTED BY RECENT EVIDENCE. FROM THE ROLE OF ADVOCACY ORGANIZATIONS IN MEDICAL RESEARCH AND POLICY TO OFF LABEL DRUG USE TO THE EFFECTIVENESS OF CONTINUING MEDICAL EDUCATION. THE SEMINAR SERIES AIMS TO ENGAGE THE NIH COMMUNITY IN THOUGHT PROVOKING DISCUSSIONS TO CHALLENGE WHAT WE THINK WE KNOW AND TO THINK CRITICALLY ABOUT OUR ROLE IN TODAY'S RESEARCH ENVIRONMENT. TODAY DR. RICHARDSON WILL DISCUSS THE VARIOUS FA FACETS OF EVIDENCE BASED MEDICINE, WHAT IT IS WHY WE NEED IT AND HOW TO PRACTICE IT. HOW CLINICIANS CAN LEARN HOW TO PRACTICE EVEN AFTER FORMAL TRAINING. HE WILL EXAMINE WHAT IT WILL TAKE TO INCORPORATE EVIDENCE BASED EVIDENCE INTO THE CORRECT CLAW OF MEDICAL SCHOOLS INDICATING HOW HIS TEACHING INSTITUTIONS ARE DOING THIS. DR. RICHARDSON IS AN ACADEMIC GENERAL INTERNIST AT THE GEORGIA HEALTH SCIENCES UNIVERSITY, THE UNIVERSITY OF GEORGIA MEDICAL PARTNERSHIP CAMPUS IN ATENTHS GAGE WHERE HE IS PROFESSOR OF MEDICINE AND CAMPUS ASSOCIATE DEAN FOR CURRICULUM. HIS PRINCIPAL SCHOLARLY INTERESTS ARE IN CLINICAL EPIDEMIOLOGY, EVIDENCE BASE HEALTHCARE AND MEDICAL EDUCATION. HE'S COAUTHOR OF THE BOOK EVIDENCE BASED MEDICINE, HOW TO PRACTICE AND TEACH IT. DR. RICHARDSON CONTINUES TO WORK ON THE CHALLENGES OF INTEGRATING EVIDENCE AND TO CLINICAL DECISIONS, PARTICULARLY IN EVIDENCE BASED CLINICAL DIAGNOSIS. HE ALSO IS WORKING TO INCORPORATE EVIDENCE INTO THE NEW CURRICULUM AT HIS INSTITUTION AND IN MEDICAL EDUCATION AT ALL WHRESTLES. PLEASE JOIN ME IN WELCOMING DR. SCOTT RICHARDSON. [APPLAUSE] >> THANK YOU AND GOOD MORNING. LET'S SEE IF THIS WORKS. SURE. YOU'LL SEE ABOUT ME A COUPLE THINGS. FIRST OF ALL I CAN'T READ MINDS. I'VE WORKED VERY HARD OVER THE YEARS TO LEARN TO DO THAT BUT IT HASN'T WORKED. I'VE EVEN GO SO FAR TO TURN MY HAIR GRAY AND LOSE A LOT OF IT BUT SO NOTHING GETS IN. SO IF YOU HAVE QUESTIONS I WOULD BE DELIGHTED TO HEAR FROM YOU BUT YOU'LL HAVE TO FLAG ME DOWN AND SAY I HAVE A QUESTION. I HAVE LESS THAN TWO HOURS OF MATERIAL SO I'M HOPING WE'LL FILL WHATEVER TIME WITH DIALOGUE AND QUESTIONS FROM THE AUDIENCE. THE OTHER THING IS I TEND TO MOVE AROUND A BIT SO YOU'LL HAVE TO GET MY ATTENTION BY ACTUALLY MOVING YOUR HAND OR SOMETHING SO I CAN SEE YOU. SO I HOPE THIS WON'T DISTURB YOU. YOU'LL SEE I USE VISUAL METAPHOR FROM POPULAR CULTURE, AS I UNDERSTAND IT, OVER MY LATE TEENAGER DAUGHTER HAS ASSURED ME I DON'T KNOW ANYTHING ABOUT POPULAR CULTURE. IT'S OKAY. WHAT WE'LL TACKLE ARE THESE THREE TOPICS IN THIS ORDER, BUT AS YOU'LL SEE, WE CAN MOVE BACK AND FORTH. SO IF YOUR QUESTION COMES OUT OF SYNC IT'S ALL RIGHT JUST GO AHEAD AND ASK IT. I WANT TO EXAMINE WHAT WE MEAN BY EVIDENCE BASED MEDICINE OR PRACTICE OR HEALTHCARE AND THINK ABOUT WHY WE NEED IT. AND WHAT IT IS REALLY THAT WE'RE NOT DOING MAYBE IN SOME CERTAIN CIRCUMSTANCES, AND WHAT WE COULD BE DOING IF WE WERE TO BE USING IT. AND SO TO THE THIS, WHAT I'D LIKE TO START OFF IS IF EVERYONE COULD RAISE THEIR RIGHT HAND. ALL RIGHT. NOW KEEP IT UP IN THE AIR IF YOU WOULD LIKE YOUR HEALTH SYSTEM TO ROUTINELY PROVIDE THE POLY VALENT PNEUMOCOCCAL VACCINE TO HIGH RISK ELDERLY PERSONS TO PREVENT PNEUMONIA AND DEATH. KEEP IT UP. ALL RIGHT. SO THAT'S MOST OF YOU IN THE ROOM. I CAN'T SEE THE WEBCAST FOLKS BUT I'M GUESSING A LOT OF THEM WOULD HAVE THEIR HANDS UP AS WELL. AND I'LL PICK UP THE THREAD OF THAT STORY IN JUST A LITTLE BIT AS WE EXAMINE THIS. I NOW FACE THE PROBLEM I'M ALSO DEALING WITH NOVICE CLINICIANS. AND WE WANT TO HELP OUR FIRST AND SECOND YEAR STUDENTS BEFORE THEY HEAD OFF TO THE WARDS TO UNDERSTAND THE SCIENCE THAT GUYS CLINICAL DE-- THAT GUIDES CLINICAL DECISION-MAKING. THIS RAISES THE ADDITIONAL QUESTION WHAT WE WANT TO DO AS A HEALTH SYSTEM BUT ALSO WHAT E WE SHOULD DO EDUCATIONALLY TO HELP PEOPLE UNDERSTAND HOW THESE DECISIONS GET MADE. LET'S START WITH WHAT WE MEAN. I THINK OF EVIDENCE BASED HEALTHCARE AS THE BROADEST OF THE TERMS. EVIDENCE BASED PRACTICE IS THE TERM THAT HAS TO DO WITH THE CARE OF INDIVIDUAL PERSONS WHERE AS HEALTHCARE MIGHT BE THE CARE OF ALL PEOPLES IN POPULATIONS OR POPULATION SUBGROUPS. AND OF COURSE THERE'S EVIDENCE BASED EDUCATION, EVIDENCE BASED LIBRARIANSHIP. I WAS SENT A REPRINT THERE'S EVIDENCE BASED BUSINESS MANAGEMENT. SO THE IDEAS OF MAKING WELL INFORMED DECISIONS ARE STARTING TO APPEAL TO PEOPLE BEYOND HEALTHCARE, AND THEY ARE HOLDING UP HEALTHCARE AS THE EXAMPLE. INTERESTING. SO WHAT DO I MEAN EVIDENCE BASED. IT REALLY HAS TO DO WITH THIS IDEA THAT WE CONSCIENTIOUSLY EXPLICITLY AND THOUGHTFULLY USE CURRENT BEST EVIDENCE FROM CLINICAL CARE RESEARCH IN OUR WORK. WHETHER OUR WORK IS ACTUAL CLINICAL PRACTICE OR IN BROADER ISSUES OF HEALTHCARE OR EDUCATION OR ELSEWHERE. AND IT'S A RECOGNITION OF THE FOLLOWING. 150 TO 200 YEARS AGO, THE KNOWLEDGE THAT WAS USED FOR CLINICAL DECISIONS WAS PRIMARILY OF ONE SORT. CLINICAL EXPERTISE. SO IT'S HARD LESSONS FROM DOING IT OVER AND OVER. AND NOW IN THIS CENTURY, WE THINK IT'S MORE COMPLICATED. THERE'S STILL CLINICAL EXPERTISE, AND I WANT TO BE CLEAR. I'M NOT ARGUING AGAINST THE USE OF CLINICAL EXPERTISE. YOU DON'T WANT AN INEXPERT HEALTHCARE PROVIDER, YOU WANT SOMEBODY WHO IS GOOD AT THEIR JOB. WHAT WE'RE IN A SENSE ARGUING FOR IS THE ADDITION OF CONSIDERATION OF ALL THESE OTHER THINGS. THE KNOWLEDGE FROM THE STUDIES OF THE HUMAN BIOLOGY. THAT'S WHY YOU EXIST, RIGHT AT NIH. THERE'S ALSO KNOWLEDGE OF STUDIES OF HEALTH SYSTEMS AND HOW TRANSLATIONAL RESEARCH CAN INFLUENCE, BUT IN ADDITION, THERE'S THE RECOGNITION IN THE LAST CENTURY AND NOW THIS CENTURY OF PATIENT'S PERSPECTIVES HOW THEY CAN AND SHOE INFLUENCE THINKING. THEREFORE THEIR CIRCUMSTANCES, THE HEALTH SYSTEMS, OUR ETHICS. WHAT WE'RE TALKING ABOUT IS ADDING THIS ONE TO THE MIX, NOT SUBSTITUTING. OKAY. AND THAT'S THE FIRST POINT OF CONFUSION. A LOT OF PEOPLE SAY HOW COME YOU'RE AGAINST CLINICAL JUDGMENT. I PREEMPTIVELY SAY I'M NOT. I THINK WHAT WE'RE TALKING ABOUT IS ADDING CONSIDERATIONS OF RESULTS FROM CLINICAL CARE RESEARCH TO 69 OTHER KNOWLEDGE THAT WE USE AND MAKE JUDGMENTS ABOUT. YOU ASK PEOPLE LIKE THESE RESEARCHERS DO. THEY DID A SURVEY OF AMERICANS IN A FEW THOUSAND WERE ASSEMBLED AND THEY SAID SHOULD YOU BELIEVE THAT HEALTHCARE SERVICES YOU RECEIVE SHOULD BE BASED ON THE CURRENT, THE BEST AND MOST RECENT RESEARCH AVAILABLE. AND IT PROBABLY WON'T SURPRISE YOU THAT MOST SAID YES. SOME SAID NO, AND I'M NOT SURE WHAT THESE PEOPLE WERE WORRIED ABOUT, BUT THAT'S WHAT WE HAVE. IT SEEMS LIKE A NO BRAINER. IN FAWCT, A LOT OF PEOPLE WHEN I START IN ON THIS CONVERSATION SAY REALLY IT'S NOT ALREADY THAT WAY. SO WHY IS IT THAT WE NEED IT. LET'S GO A LITTLE BIT MORE DEEPLY. HERE ARE SOME OF THE ISSUES. THERE'S AN ENORMOUS AMOUNT OF CLINICAL CARE RESEARCH THAT'S BEEN PUBLISHED. AND ALTHOUGH PEOPLE WILL COMPLAIN ABOUT THIS, I SEE THIS MOSTLY AS SUCCESS. SUCCESS IF NO OTHER INSTITUTION, IT'S SUCCESSIVELY NIH AND THE ENORMOUS PUBLIC AND PRIVATE INVESTMENT IN RESEARCH. WE HAVE A LOT OF NEW KNOWLEDGE. NOW WE HAVE LOTS AND LOTS OF IT, THOUGH. SO MUCH SO THAT NOT ALL OF IT IS NECESSARILY ABLE TO BE UNDERSTOOD BY AN INDIVIDUAL. IN FACT, IT CANNOT BE DONE OVER 25,000 BIOMEDICAL PERIODICALS EVERY YEAR. NOBODY CAN ACTUALLY EVEN READ THEM TO SAY ANYTHING OF UNDERSTANDING AND HOW TO USE THEM. SO WE HAVE A LOT OF RESEARCH AND IMPORTANTLY ALSO ITS NATURE HAS CHANGED DRAMATICALLY. WHETHER YOU THINK ABOUT INDIVIDUAL RANDOMIZED TRIALS OR SYSTEMATIC REVIEWS IN CLINICAL CARE EVIDENCE OR GENE-WIDE ASSOCIATION STUDIES, S AND AND P RESEARCH. LET'S AN ENORMOUS TYPE OF RESEARCH THAT WASN'T THERE 60 YEARS AGO. THE TRADITION OF LEARNING AND KEEPING UP TO DATE THAT WE HAD 60 YEARS AGO CAN BE EXPECTED TO NO LONGER APPLY. 60 TO 100 YEARS AGO WE HAD WHAT YOU MIGHT THINK OF THE PASSIVE DIFFUSION MODEL. WHICH IS TO SAY NEW MODULE AS IT WAS SLOWLY MADE THAT WAS PUBLISHED IN BIOMEDICAL JOURNALS AND TALKED ABOUT THE FEW WHO ARE EXPERT IN IT. AND THEN THAT POSSIBLY DEFUSED THROUGH THE KNOWLEDGE WORKERS WHO NEEDED TO KNOW THAT. WHAT WE'VE LEARNED OVER THE YEARS AS I'LL SHOW YOU THAT MODEL DOESN'T WORK SO WELL NOW. PARTICULARLY WHEN THERE'S SO MANY VOICES AND SO MANY OTHER THINGS TO CONSIDER. HERE'S OF THE PROBLEM. ONLY A TINY FRACTION OF THIS IS SUFFICIENTLY VALID, IMPORTANT ENOUGH AND DIRECTLY APPLICABLE TO CARE. WE'LL TALK ABOUT APPLICABILITY A BIT MORE BUT YOU'VE HEARD OF THIS IF YOU'VE INVESTED AS MUCH AS YOU HAVE IN TRANSLATIONAL RESEARCH. YOU KNOW THAT NOT EVERY KNOWLEDGE BIT THAT WE YIELD FROM BIOLOGY RESEARCH IS ALWAYS DIRECTLY APPLICABLE. AND YOU CAN THINK OF A LOT OF WHAT'S IN THE RESEARCH AS COMMUNICATION FROM ONE GROUP OF SCIENTISTS TO OTHERS, ABOUT FURTHERING SCIENCE. NOT SO MUCH FROM SCIENCES TO CLINICIANS ABOUT MAJOR ADVANCES IN PATIENT CARE. SO YOU'VE GOT THIS PROBLEM, AN ENORMOUS AMOUNT. IT'S DIFFERENT THAN IT USED TO BE, ONLY A FRACTION OF IT IS READY TO HIT THE STREETS. WE ALSO NEED IT OFTEN. NOW IF YOU ASK DOCTORS HOW OFTEN THEY NEED IT, THEY'LL SAY THAT THEY NEED NEW INFORMATION TWO OR THREE TIMES A WEEK AND THAT THEY USUALLY GET IT FROM LOOKING AT TEXT BOOKS OR FROM JOURNAL ARTICLES. BUT IF YOU FOLLOW THEM AROUND AND SAY DO YOU NEED SOME NOW, DO YOU NEED SOME NOW, DO YOU NEED SOME NOW, IT TURNS OUT THAT THERE'S BEEN A WIDE NUMBER OF STUDIES AND THEY FOUND A WIDE RANGE BUT THE AVERAGE IS SOMETHING LIKE TWO BITS OF INFORMATION FOR EVERY THREE PATIENTS TEAM. AND WHAT WE TEND TO DO IS WE'RE SO BUSY THAT WE EITHER DON'T PURSUE THE QUESTIONS OR WE ASK EXPERTS. AND WHY IS THAT? WELL, THE MAIN REASON IS OUR USUAL SOURCES OF INFORMATION DON'T WORK SO WELL. THE USUAL PATTIV MASSIVE DIFFUSION MODEL DOESN'T WORK. WHAT DO I MEAN BY THAT. WE ALREADY TALKED ABOUT THE OVERWHELMING NUMBER IN SCOPE, OKAY. IMAGINE YOU NEED A HALF CUP OF WATER AND YOU NEED TO GO TO NIAGRA FALLS AND IN GETTING THERE GET YOUR HALF A CUP AND GET OUT. TOO MUCH FOR MOST OF US. BUT IN ADDITION MANY OF THE FORMS OF THE USUAL SOURCES HAVE BEEN SHOWN TO BE INEFFECTIVE. IF YOU'VE HEARD FROM SOME OF THE PEOPLE I THINK YOU'VE HEARD THEM FROM, THEN THE STANDARD LECTURE NOT WHAT I'M DOING HERE, HAS NOT CHANGED MUCH IN TERMS OF KNOWLEDGE AND CLINICAL PERFORMANCE, UNLESS IT MOTIVATES YOU TO GO OFF AND LEARN AND DO SOMETHING DIFFERENTLY. FOR THE MAIN THING THE LARGE GROUP PRESENTATION CAN DO IS TO MOTIVATE YOU, TO GO OFF AND LEARN. AND THAT'S USEFUL. BUT BY ITSELF IT OFTEN DOESN'T CHANGE PRACTICE. AND THERE'S BEEN OVER 100 TRIALS OF CME, SOME SHOWING CERTAIN THINGS EFFECTIVE, SOME INEFFECTIVE BUT THE TRADITIONAL FORMS DON'T WORK VERY WELL. THAT'S WHAT WE REQUIRE. SO AS TIME ACCUMULATES AND WE HAVE ALL THESE QUESTIONS THAT EITHER DON'T GET ANSWERED OR GET POORLY ANSWERED, OUR IGNORANCE OF CURRENT BEST CARE GROWS AND OUR CLINICAL COMPETENCE BEGINS TO SHRIVEL. YOU MAY SAY WELL WHAT'S THE EVIDENCE OF THAT ASSERTION OR DO I LEAVE OUT TOO LITTLE TIME. HOW MUCH TIME DO YOU HAVE, DO DOCTORS HAVE TO LEARN NEW THINGS WHEN PEOPLE HAVE DONE SURVEYS, THEY ESTIMATE AMONG BRITISH GENERAL PRACTITIONERS, THEY HAVE HALF HOUR PER WEEK. AND THEN IF YOU TURN TO INDIVIDUAL PRACTITIONERS AND SAY DID YOU GET YOUR HALF HOUR LAST WEEK, ONLY HALF OR SO HAVE. WHICH TELLS YOU MAYBE THEY'RE NOT GETTING IT VERY FREQUENTLY. AND IMPORTANTLY, IT'S HALF HOUR SEPARATE FROM WORK. SO HERE'S THE THING. GOING BACK TO THAT WATER ANALOGY. THEY NEED THE KNOWLEDGE AT THE POINT OF CARE. THEY'RE THIRSTY THEN BUT THEY DON'T GET IT THEN, THEY GET IT LATER. LIKE SEVERAL DAYS LATER, OKAY. IF THIS WERE WATER, WHAT WOULD WE DO? WE'D BUILD A PLUMBING SYSTEM BEFORE BOTTLING WHERE WE'D GO TO A TAP AND OUT WOULD COME PURIFIED WATER. WHY ISN'T KNOWLEDGE LIKE THAT? WHY ISN'T KNOWLEDGE, CURRENT BEST KNOWLEDGE DELIVERED TO THE POINT OF DECISION-MAKING WHILE WORKING IN A WAY THAT IT'S RIGHT THERE WHEN WE NEED IT RATHER THAN QUITE A BIT LATER. YES, SIR? >> [INDISCERNIBLE] >> SO THERE ARE PRODUCTS THAT ARE NOW STARTING TO ATTEMPT TO SOLVE THIS PROBLEM. AND UP TO DATE IS ONE. THERE ARE SEVERAL OTHER POINT OF CARE RESOURCES. BUT YOU CAN SEE WHY IT'S ONLY BEEN IN THE LAST FEW YEARS THAT PEOPLE HAVE SAID OH, WE'VE GOT TO DO THE WAY WE DID KNOWLEDGE, DELIVER KNOWLEDGE TO THE KNOWLEDGE WORKERS. SO THERE HAS BEEN RESEARCH ON THIS BUSINESS ABOUT TIME AND ITS ASSOCIATION WITH OUR QUALITY OF CARE. THERE HAVE BEEN A NUMBER OF STUDIES, 62 EVALUATIONS OVER TIME THAT WERE SYSTEMATICALLY REVIEWED IN 2005. THERE MAY BE OTHER NEWER EVIDENCE AS WELL. FROM JOHN INOD'S THEY'VE LEARNED THERE ARE SEVERAL SIGNALS FOR UPDATES EVERY SO OFTEN. SOMETIMES A YEAR AFTER A SYSTEMATIC REVIEW IS PUBLISHED SO I RECKON BY NOW THAT'S QUITE OUT OF DATE. BUT I WOULD BE SURPRISED IF IT CHOSE SOMETHING DIFFERENT. BECAUSE THERE WAS A STRONG AND KEPT NEGATIVE ASSOCIATION BETWEEN THE YEAR SINCE GRADUATION FROM MEDICAL SCHOOL AND OUR KNOWLEDGE AS MEASURED BY TESTS ARE RECOMMENDATIONS FOR DIAGNOSIS, SCREENING AND PREVENTION. AND OUR TREATMENT. DECISIONS. IT TURNS OUT THAT OBSOLESENCE IS EQUAL OPPORTUNITY SO THAT WHEN YOU EXAMINE THESE DATA ACROSS SUBGROUPS, YOU HAVE INTERNATIONAL SCHOOLS, TOP OF THE CLASS, BOTTOM OF THE CLASS. THERE'S A DEPRESSING SIMILARITY OF THE DOWNWARD FLOAT. AND THE EDITORIAL IS TO A COMPANY THAT SAYS LOOK IT SHOULD BE NO SURPRISE. WE NEED ACTIVE MESSAGE TO DELIVER KNOWLEDGE, NOT PASSIVE KNOWLEDGE. AND WE CAN START TO CUSTOMIZE PATIENT'S CARE FROM EVIDENCE BASED STANDARDS RAFT THAN FROM OLD TEXTBOOKS. GOING BACK TO THE WATER ANALOGY IN A MOMENT, RIGHT HERE. OUT OF DATE. THINK ABOUT THIS. THERE'S A THREE YEAR OR SO TIME CYCLE FROM THE TIME A TEXTBOOK CHAPTER IS WRITTEN TO THE TIME IT'S PUBLISHED WHERE SOME EXCEPTIONS LIKE THE ELECTRONIC TEXTS THAT ARE STARTING TO COME OUT. IF THEREFORE WATER, WE WOULDN'T TOLERATE IT. IF YOU WENT OVER TO THE SPIGOT AND OUT CAME YEAR OLD WATER AFTER 30 YEARS OF SITTING IN SOME TANK, WHAT WE WOULD DO? WE'D RAISED A STINK AS THEY SAY AND THE GOVERNMENT WOULD INTERVENE AND WE WOULD DO SOMETHING ABOUT IT. WHY ISN'T THAT SAME SOCIETAL OUTCRY HAPPENING ABOUT THE OUT OF DATE KNOWLEDGE THAT WE'RE DELIVERING TO OUR HEALTHCARE THRSTLEPROFESSIONALS. OFTEN IT'S WRONG. WHAT'S THAT ABOUT. LET'S TAKE A LOOK AT THIS DAWVMENT YOU MAY HAVE SEEN THIS EARLIER BUT LET'S LOOK AGAIN. THIS IS DATA FROM ELIEN ELLIOTT ANTMAN PUBLISHED IN 92 WHERE A NUMBER OF TREATMENT TRIALS FOR TREATMENT OF MYOCARDIAL INFARCTION AND THIS IS JUST ONE OF THEM, THROMBOLYTICS AND I KNOW WE HAVE NEWER TRAIMENTS BUT TREATMENT S BUT WORK WITH B ME ON THIS. THE THIRD COLUMN ARE THE NUMBER OF PATIENTS WHO HAVE CUMULATIVELY BEEN RANDOMIZED TO THEM. IN THE EARLY 70 THERE WERE TEN TRIALS WITH JUST OVER 2500 PATIENTS SO THAT IN 1990 THERE HAS BEEN 70 TRIALS OF OVER 48,000 PATIENTS. IN THE MIDDLE HERE, THIS IS A CUMULATIVE A SYSTEMATIC REVIEW WITH CUMULATIVE META-ANALYSIS DISPLAY OF THE EFFECT SIZE WHICH IS TO SAY MORTALITY REDUCTION FROM THROMBOLYTIC THERAPY. HERE'S THE LINE OF NO DIFFERENCE WHERE THE RATIO IS ONE. AND THIS SIDE FAVORS THE TREATMENT, THE MORTALITY'S LOWER AND THIS SIDE FAVORS THE CONTROL. AND I THINK YOU SEE THAT AFTER 23 PATIENTS, THERE SEEMS TO BE A REDUCTION BUT THERE WAS A WIDE CONFIDENCE INTERVAL. THAT BLOB IS THE ESTIMATE OF THE EFFECT SIZE AND THAT LINE IS THE CONFIDENCE INTERVAL. AND BY THE TIME YOU GET TO THE TENTH TRIAL, BY THEN STATISTICALLY SIGNIFICANT SHOWING A REDUCTION IN MORTALITY. AND THAT WAS ESTIMATED AROUND 1973. AND FOR THE NEXT MORE THAN A DECADE, 60 MORE TRIALS WITH NEARLY 46 THOUSAN,000 PATIENTS ENTERED INTO THOSE TRIALS. OUR NET EFFECT HAS MOVED A LITTLE BUT NOT A QUALITATIVE BELIEF THAT THIS TREATMENT SAVES LIVES. NOW ON THE OTHER SIDE, THOUGH, LET'S HAVE A GOOD LOOK AT THE RECOMMENDATIONS OF TEXTBOOKS AND EXPERTS. AS MANIFESTED BY WHAT THEY SAID OF ROUGHLY THE SAME YEARS THEY CHECKED SOURCES EITHER REVIEW ARTICLES OR PRACTICE GUIDELINES OR TEXTBOOK CHAPTERS. AND COUNTED UP, IT WAS NOT MENTIONED EXPERIMENTAL, ETCETERA. AND YOU'LL SEE THAT EVEN THOUGH IT WAS BEING USED HERE, IT WAS NOT MENTIONED IN MOST PLACES. IN 1969, IT BEGAN TO APPEAR AS MENTIONED. BUT AS EXPERIMENTAL AND IT WASN'T UNTIL THIS POINT, SO ROUGHLY 1987, 1990 THAT IT WAS RECOMMENDED AS ROUTINE CARE THAT WOULD REDUCE MORTALITY. THE THIS DEMONSTRATED, THIS ARTICLE DEMONSTRATED THAT EXPERTS OFTEN LAG IN THEIR RECOMMENDATION OF LIFE SAVING THERAPY. AND THE LACK CAN B LAG CAN BE MEASURED IN YEARS, SOMETIMES DECADES. DURING THAT TIME THERE ARE MILLIONS OF PEOPLE NOT GETTING THIS LIFE SAVING THERAPY. IN THE SAME ARTICLE, THERE WERE TREATMENTS THAT ACTUALLY CAUSED MORE HARM, AND SO THERE WERE MILLIONS OF PATIENTS RECEIVING THOSE POTENTIALLY HARMFUL THEOREMS. THEOREMS -- HARMFUL THERAPIES. NOT JUST IN THIS COUNTRY, ALL OVER THE WORLD. SO YOU HAVE TO START REALLY ASKING OURSELVES AS A SOCIETY, WHY IS THAT. WHAT IS IT THAT WE'RE DOING OR NOT DOING IN DELIVERING KNOWLEDGE TO A KNOWLEDGE INTENSIVE LINE OF WORK IN SUCH A WAY THAT RESISTS FAR BEHIND WHAT WE ALREADY KNOW. WHY CAN EXPERTS BE WRONG. NOW I HAVE TO SAY I HAVE -- OOPS, OH YES. LET ME PICK UP THE STORY. WHY SHOULD WE PICK UP THE PNEUMOCOCCAL VACCINE. EARLY O OBSERVATIONAL STUDIES SHOW THE USE OF EARLY INFECTION. THE BY LAWLG POSSIBILITY. THE VASCULAR NIGHT AGAINST THE STRAINS. YOU REDUCE IT. DEFINITELY SURROGATE OUTCOMES. USUAL MEASURE FOR INSTANCE THAT ITS USE IS ASSOCIATED WITH INCREASE IN ANTIBODY TITERS AND THERE ARE LITTLE O LOTS OF ANALOGIES. THERE ARE BELIEFS WHO WANTS TO BE AGAINST PREVENTION. NO ONE WANTS TO REALLY COME OUT AND SAY I'M AGAINST IT. AND OF COURSE THERE'S ALSO THESE OTHER THINGS THAT ARE REALLY MORE SYSTEM ISSUES. LOW UNIT COSTS. EACH ONE DOESN'T COST THAT MUCH SO PEOPLE THINK IT'S CHEAP. EVEN IF THE AGGREGATE COST IS ENORMOUS. IT HAS THE PROPERTY OF BEING MEASURABLE SO THAT PEOPLE COULD SAY OH YES CHECK WE DID IT, WE'RE DOING PREVENTIVE SERVICES. AND EXPERTS RECOMMEND IT, PRACTICE GUIDELINES SURE DO, THE MANUFACTURERS WEREN'T GOING TO FIGHT US. AND THESE EXTREMIST OR WE DON'T KNOW THEM BY THAT. OR THESE GROUPS RECOMMENDED, THE LAST ONE GIVES YOU DEMERITS IF YOU DON'T. SO YOU MIGHT QUITE REASONABLY ASK WHAT'S THE EVIDENCE THAT SHOWS WE SHOULD DO THIS. THERE AT THE TIME I LOOKED LAST SUMMER, THERE HAS BEEN THREE SYSTEMATIC REVIEWS FOR, OF 13 TRIALS. I DON'T KNOW IF THEY ARE NEW TRIALS SINCE BUT IN ANY CASE, THERE MIGHT BE. THREE OF THOSE TRIALS WERE IN HEALTHY PERSONS OTHERWISE LOW RISK BUT IN HIGH ATTACK RATE SITUATIONS. ARMY RECRUITS, SOUTH AFRICAN HIGHLANDERS, THE NEW ZEALAND HIGHLANDERS OF SOUTH AFRICAN GOLD MINERS. YOU TAKE A LOT OF PEOPLE PUT THEM TOGETHER IN CLOSE QUARTERS AND THE INFECTION SWEEPS THROUGH THE CAMP AND YOU CAN SHOW IN THAT GROUP YOU CAN REDUCE MORTALITY. BUT IN THE GROUP OF THE OLDER AND THE HIGH RISK PATIENTS IN WHOM IT'S RECOMMENDED, THERE HAS BEEN TEN TRIALS WITH OVER THIS,000 SUBJECTS IN AL -- 24,000 SUBJECTS IN ALL PARTS OF THE WORLD. TOTAL MORTALITY, PNEUMONIA MORTALITY, BAC AND SO FORTH. I'M NOT SURE IF WE SHOULD GIVE THIS STUFF BUT IT DOES STRIKE ME AS REMARKABLE THAT MOST RELEVANT VALUATIONS RECOMMEND THAT AGE GROUP GET IT. BUT HAS IT SHOWN A DIFFERENCE? NOT SURE ABOUT THAT. THE THIS PROVIDES US WITH WHAT THING SCIENTISTS CALLED COGNITIVE DISSONANCE. YET WE'RE MAKING A DECISION THAT SEEMS UNINFORMED BY THE INFORMATION. IT SHOULD MAKE US SOMEWHAT UNCOMFORTABLE AND MOVE US TO ACTION. I HAVE COMPASSION FOR EXPERTS. EXPERTS ARE ASKED TO DO THE IMPOSSIBLE. THEY ARE ASKED TO OPINE EARLY. WHEN WE HAVE A NEW THING THAT COMES OUT WE TURN TO EXPERTS AND SAY WELL WHAT DO YOU THINK. THEY'VE OFTEN JUST SEEN THE DATA JUST LIKE WE HAVE. WE ALSO DON'T LIKE OUR EXPERTS WISHY WASHY, WE WANT THEM BACK DECISIVE, TO BE DE FINNIVE. DEFENSIVE. THEY'RE ASKED FOR OPINIONS TO BE FORMED EARLY. THEY HAVE OVERLOAD, SOMETIMES TOO MUCH INFORMATION. THEY MAY ALSO BE INVOLVED IN THE RESEARCH AND MAY OR MAY NOT BE FREE OF THEIR OWN BIASES OR CONFLICTS OF INTEREST. AND OF COURSE, PERTS BEIN EXPERTS BEING HUMAN WE HAVE ALL THE HUMAN FRAILTIES. IN ADDITION, IT'S A RECOGNIZED PHENOMENA ABOUT THE PRELIMINARY EVIDENCE, IT SHOULD WORK WITH DEFINITIVE EVIDENCE. SEE, WE DON'T REALLY MAKE A HABIT OF DOING TRIALS OF THINGS WE ALREADY BELIEVE ARE STUPID. IF WE DON'T THINK IT'S GOING TO WORK, THAT'S NOT WHAT WE DO TRIALS. AND WE TAKE OUR BEST IDEAS. WHERE DO THOSE BEST IDEAS COME FROM. ANIMAL STUDIES EXPERL OBSERVATIONAL -- AND EARLY OBSERVATIONAL STUDIES. WE TAKE THE THING THAT SHOULD WORK AND TRY THEM OUT. THAT MAKES SENSE FROM THE POINT OF VIEW WHEN YOU'RE ABOUT TO EMBARK ON A TRIAL BUT IT MEANS PEOPLE ALREADY HAVE FORMED OPINIONS ABOUT WHETHER IT SHOULD WORK. AND SO THOSE FORMED OPINIONS ARE HARD TO UNLEARN. AND WE OFTEN CONFUSE THESE TWO. IT SHOULD WORK WITH IT DOES WORK. AND ONLY THE EXPERTS THAT I'VE TALKED TO FOCUS ON HOW SOMETHING WORKS RATHER THAN ON HOW WELL IT WORKS. SO THEY'LL TELL ME ABOUT HOW THE ANT BODY GLOMS ON TO THE RECEPTORS, INTERFERES ABOUT THE LIGAND AND SO FORTH AND I WILL SAY WELL HOW ARE PATIENTS, ARE THEY BETTER OFF, DO THEY HAVE FEWER EXACERBATIONS AND THIS SORT OF THING. UP UNTIL RECENTLY MANY EXPERTS HAVE NOT USED EXPLICIT SYSTEMATIC METHODS TO GATHER THE EVIDENCE THAT MAKES SENSE OF THEM. SO, PERTS ARE PUT IN A JAM BUT IT DOES MEAN WE CAN'T ALWAYS RELY ON THEM TO KNOW THE TRUTH OF THE MATTER. BUT IF WE DID, IF WE DID HAVE GOOD QUALITY INFORMATION, IT MIGHT MAKE A DIFFERENCE. THERE HAVE BEEN A SYSTEMATIC REVIEW OF THE TREATMENT OF CARDIOVASCULAR DISEASE FROM 2007 AND THEY FOUND THAT PATIENTS WHO GOT THE ACTUAL TREATMENT BASED ON TRIALS AT THE OPTIMAL DOSES FOR THE RIGHT DURATION DID BETTER THAN THOSE WHO DIDN'T GET THE TREATMENT. AND THAT THE IMPROVEMENT WAS PROPORTIONAL TO THE NUMBER OF TREATMENTS THEY GOT. WELL INFORMED DECISIONS MIGHT MAKE A DIFFERENCE. THAT'S PART OF THE CONSTRUCT HERE. WHY ARE WE DOING THIS? IT'S BECAUSE WE CAN'T MAKE WELL INFORMED DECISIONS WITHOUT INFORMATION AND WE'VE LEARNED THE HARD WAY THAT NOT ALL INFORMATION IS CREATED EQUAL, AND SOMETIMES MISINFORMATION IS WORSE. I ONLY GOT TO GO BACK A LITTLE BIT TO THE WHI TRIAL BUT ALSO TO THE PAST STUDY, THERE HAVE BEEN A NUMBER OF TRIALS MANY OF THEM SPONSORED BY THIS INSTITUTION WHERE WHEN WE FINALLY PUT IT TO THE TEST, WE LEARN THE TRUTH IS DIFFERENT THAN THE ORIGINAL EARLY STUDIES. AND THERE IS EVIDENCE THAT BETTER INFORMATION CAN LEAD TO BETTER INFORMED DECISIONS AND BETTER OUTCOME. SO NOW WE FACE MORE CHOICE. DPO WDO WE TAKE THE BLUE PILL. FORGET THIS, LEAVE THE AUDITORIUM THIS MORNING AND GO BACK TO BUSINESS AS USUAL. OR DO WE TAKE THE RED PILL AND REMEMBER AND TRY AND MAKE A DIFFERENCE AND TRY AND MAYBE TEACH OURSELVES OR TEACH OUR COLLEAGUES, OUR JUNIOR COLLEAGUES IN PARTICULAR HOW TO MAKE A DIFFERENCE. SO FEEL FREE TO WALK OUT AT ANY POINT BUT I'M GOING TO ASSUME IF YOU STAY YOU WANT TO HEAR A LITTLE BIT MORE ABOUT TEACHING. SO I WOULD BE DELIGHTED TO ANSWER QUESTIONS AS WE GO. BUT LET'S THINK ABOUT THAT. HOW ARE WE GOING TO GET THIS TAUGHT? AND THE FIRST PART IS TO UNDERSTAND WHAT THE PROCESS IS, HOW IT GETS DONE AND SOME OF THE PRINCIPLES. SO IN THIS LIGHT I'VE TRIED TO COMBINE TWO KIND OF THINGS. ONE IN WHITE, THE PROCESS OF EVIDENCE BASED PRACTICES IN WHICH A PATIENT DILEMMA OR IF IT'S EVIDENCE BASED HEALTHCARE A POPULATION HEALTHCARE DILEMMA. IT COULD BE INDIVIDUAL PATIENTS, SUBGROUPS OR WHOLE POPULATIONS. WE HAVE TO ASK ANSWERABLE QUESTIONS THAT ARISE FROM THAT DILEMMA, THEN WE ACQUIRE EVIDENCE THAT'S EXISTING ALREADY ABOUT THE ANSWERS TO THAT. APPRAISE THAT EVIDENCE FOR ITS CREDIBILITY OR VALIDITY, IMPORTANCE AND APPLICABILITY. AND THEN APPLY IT, NOW WE WERE GOING WITH THE A'S I THINK YOU CAN SEE BUT APPLY REALLY HERE MEANS INTEGRATE THIS NEW KNOWLEDGE WITH OUR EXISTING KNOWLEDGE OF THE SEVERAL TIMES AROUND THE HUMAN BIOLOGY, THE PATIENT PREFERENCES, THE HEALTH SYSTEMS AND SO FORTH. AND THEN WE ACT ON THAT AND ASSESS THE IMPACT OF OUR ACTIONS. AND THIS IS MEANT TO SHOW, SORRY ABOUT THE LITTLE DARK, MEANT TO SHOW THAT SINCE PATIENTS WILL VARY, THOSE ACTIONS MAY VARY. LET'S LOOK AT THESE TWO PRINCIPLES HERE, AND THESE ARE THE TWO SORT OF BIGGIES. THIS ONE HAS A LOT OF DIFFERENT NAMES BUT I THINK THE SIMPLEST IDEA AND THE LEAST JUDGMENTAL IS THAT EVIDENCE VARIES IN TERMS OF ITS ROBUSTNESS. ITS RISK FOR BIAS, ITS RISK FOR ERROR. AND SO SOME THING ARE MORE CREDIBLE THAN OTHERS, AND A LOT OF THIS GOES BACK TO SIMPLE IDEAS OF FAIRNESS. YOU KNOW, IF YOU'RE, IF YOU WATCH TWO BOY THROW SNOW BALLS AND ONE OF THEM THROWS A BALL AND HITS A TREE AND SAYS SEE THAT TREE I HIT THAT'S WHAT I MEANT TO HIT. THE OTHER ONE SAYS SEE THAT ONE OVER THERE WITH THE CROOKED NOTCH COMING OUT OF IT I'M GOING TO HIT THAT ONE. THROWS IT AND HITS THAT ONE. WHICH ONE IS MOST CREDIBLE. MOST HOW FAR OUR SENSES OF FAIRNESS TELL US THE SECOND ONE IS MORE CREDIBLE. AND THAT'S THE MOTION OF PRESPECIFIED HYPOTHESES IN RESEARCH. AND MANY OF OUR IDEAS ABOUT FAIRNESS AND CREDIBILITY COME AT THE ROOT FROM OUR UNDERSTANDING OF FAIRNESS. WHAT'S A FAIR TEST. WHAT WOULD BE FALSIFIABLE EITHER WAY THAT THIS COULD WIN, THAT COULD WIN OR WE COULD TURN OUT THAT THEY'RE EQUIVALENT. THIS IDEA OF FAIRNESS, THOUGH, HAS SOME FORMALIZATION FROM MANY METHODS. SO DEE EPIDEMIOLOGY, BOTH CLASSICAL AND CLINICAL, HEALTH ECONOMICS. THERE ARE LOTS OF FORMAL WAYS THAT WE STATE CREDIBILITY, VALIDITY AND SO FORTH. BUT YET AT THEIR ROOT THEY HAVE TO DO WITH FAIRNESS. THEN THERE'S IMPORTANCE WHICH HAS MOSTLY TO DO WITH HOW WELL SOMETHING WORKS OR HOW SAFE IT IS. THE SIZE OF THE DIFFERENCE NOT ITSISTIC CULL DIFFERENCE PRIMARILY BUT THE CLINICAL IMPACT. EVIDENCE CAN VARY IN ITS RISK FOR BIAS AND ERROR. SOME PEOPLE SAY THERE'S A HIERARCHY, A SINGLE HIERARCHY, I'M NOT SURE THERE'S A SINGLE HIERARCHY FOR EVIDENCE BECAUSE THERE ARE LOTS OF DIFFERENT QUESTIONS AND THEREFORE THE TYPES OF EVIDENCE YOU NEED FOR THOSE ARE DIFFERENT. AND THEREFORE YOU MIGHT HAVE MULTIPLE HIERARCHIES DEPENDING ON YOUR TYPES OF QUESTIONS. BUT ANYWAY, THE IDEA IS THIS ONE, THEY VARY AND WE HAVE TO BE ABLE TO SORT IT OUT. THE SECOND PRINCIPLE IS THIS. WHEN WE'RE APPLYING EVIDENCE, EVIDENCE ALONE DOES NOT DECIDE. WE HAVE TO COMBINE WITH OTHER KNOWLEDGE AND VALUES AND MORE ON THAT IN BIT BUT THIS BASIC NOTION IS, IT'S INFORMING DECISIONS, NOT PRESCRIBING DECISIONS THAT WE'RE TALKING ABOUT. SO HOW ARE WE GOING TO GET STARTED IN TEACHING. THIS IS DUMB LEDORE IN HIS REFLECTION. WE HAVE TO FIGURE OUT OUR OWN SKILLS. HOW DO I GET THAT DONE. THERE ARE SOME COURSES TO TAKE. THERE ARE PARENTS AND OTHERS WENT TO COLORADO TO TAKE A COURSE THERE. THEIR COURSES IN THIS COUNTRY AND ELSEWHERE. I'M HOPELESSLY BIASED IN FAVOR OF TAKING A COURSE BECAUSE PRIMARILY WE'RE ALL SO BUSY, WHERE ELSE DO YOU GET THE TIME TO CONCENTRATE ON BUILDING A SCARE THAT'S SO UNFAMILIAR TO YOU IF YOU'RE TRYING TO DO IT IN BITS AND PIECES. BUT IT CAN BE LEARNED WITHOUT TAKING A COURSE. THERE ARE SEVERAL BOOKS AND I'M HOPELESSLY BIASED IN FAVOR OF SEVERAL OF THEM NOT THE ONES I'VE KNOWN TO BE COAUTHOR ON BUT I'VE CONTRIBUTED TO SEVERAL. I HOPE YOU DUST OFF YOUR OWN BASIC SKILLS AND START DOING IT. THAT WILL HELP YOU IN A FEW OTHER AREAS. ANOTHER THING IS TO MAKE SURE YOU HAVE ACCESS TO GOOD EVIDENCE RESOURCES. YOU'VE MENTIONED ONE UP TO DATE BUT THERE ARE LOTS OF OTHERS NOW STARTING TO COME TO MARKET. AND ONE OF THEM IS NONE OTHER THAN PEDESTRIA PED IMMEDIATE IF YOU -- PED MED AND KNOWING THE INFORMATION FROM THOSE RESOURCES. NOT JUST HAVING IT. THE BOOK ON THE SHELF DOESN'T SHARE ITS KNOWLEDGE IT'S ONLY WHEN IT'S IN YOUR HAND SCOPE IS WHEN IT IS WORKING. SAME WITH THE ELECTRONIC RESOURCES. AS WE BUILD OUR OWN SKILL AND ACTUALLY DO IT AND PRACTICE PRACTICE PRACTICE, ANOTHER THING TO THINK ABOUT IS TO START BUILDING SOME ALLIES. AND THEN THEY COME IN ALL SHAPES AND SIZES. OF COURSE YOU WOULD THINK OF YOUR OWN COLLEAGUES IN YOUR OWN PRACTICE BUT YOU MIGHT ALSO THINK WOW, THERE'S A LIBRARIAN AND I NEED HIS OR HER HEALTH AND I HAVE GOT A CLINICAL PHARMACIST AND ALL THESE OTHER PLAYERS. BUT THERE MAY BE OTHER ALLIES OF COMMUNITY GROUPS, HEALTHCARE JOURNALISTS AND ALL SORTS OF PEOPLE ARE NOW SAYING THIS IDEA SPEAKS TO US AND WE WANT TO WORK WITH PEOPLE WHO WANT TO DO THINGS. I HAVE A LAPEL PEN THAT I WEAR ON MY WHITE COAT, AND IT'S IN THE SHAPE OF A MILLIO YIN AND YANG SYMBOL. THOSE WERE MEANT TO SHOW YOU PARADOXICAL IDEAS THAT TURN OUT NOT TO BE PARADOXICAL. ON ONE SIDE IT SAYS TRUST ME I'M A DOCTOR AND THE OTHER SIDE IT SAYS ASK ME ABOUT THE EVIDENCE BECAUSE I ACTUALLY DON'T THINK THEY'RE PAR DOCKIC. DOCKIC. I WANT TO TRUST THEM ABOUT THE EVIDENCE OR WE WILL LOOK IT UP TOGETHER AND I WILL HELP THEM INTERPRET IT. IT'S NOT A MAYO PARADOXIC STATEMENT. I DON'T SEE IT AS A CHALLENGE BECAUSE I WANT TO FEEL WE'RE MAKING WELL INFORMED DECISIONS ABOUT THEIR CARE. IT'S NOT ALWAYS EASY OR FAST. WE COME BACK TO SOLVING THAT WITH PRODUCTS LIKE WHAT YOU MENTIONED BUT THE IDEA IS IT'S PART OF OUR CONTRACT TO MAKE WELL INFORMED DECISIONS WITH OUR PATIENTS IF YOU USE CURRENT INFORMATION. WHAT I MEAN BOY THREE MODES DOWN HERE. I'VE MET A LOT OF TEACHERS IN EVIDENCE BASED MEDICINE, A LOT OF WONDERFUL PEOPLE, GIFTED TEACHERS AND I'VE TRIED TO LEARN FROM EACH AND EVERY ONE OF THEM. WHAT I STARTED DOING SEVERAL YEARS AGO IS STARTED WATCHING HOW THEY TEACH AND TRYING TO CATEGORIZE IT IN A WAYS THAT MADE AS SOON AS TO ME. AS I BEGAN TO DO IT, I NOTICED ALTHOUGH I'VE MET THOUSANDS AND THERE WERE THOUSANDS OF DIFFERENT INDIVIDUALS' STYLES THAT THERE WERE THREE MAIN METHODS OR THREE MAIN MODES IN WHICH THEY TAUGHT. AND SO I TAUGHT TO MYSELF WELL, WHAT ARE THOSE. THE FIRST ONE IS ROLE MODELING. AND THIS IS ANYONE WHO ACTUALLY PRACTICES OR ACTUALLY DOES HEALTHCARE DECISIONS AT THE POPULATION LEVEL WHO USES EVIDENCE AS PART OF THEIR DECISION-MAKING. AND ARE SEEN BY LEARNERS TO DO THIS. SO LEARNERS SEE THAT EVIDENCE AND ITS USE THAT'S CONSCIENTIOUS AND JUDICIOUS IS PART OF GOOD PATIENT CARE OR FILL IN POPULATION HEALTH OR PREVENTIVE CARE OR WHATEVER IT IS THAT YOU DO. THIS IDEA OF BEING THAT EVIDENCE IS NOT SEPARATE FROM IT, IT'S RIGHT THERE WITH EVERYTHING ELSE. IN ADDITION, THEY SEE WHO YOU DO IT WITH, WHAT YOU ACTUALLY DO, WHEN YOU DO IT, WHERE YOU DO IT, HOW YOU DO IT. IN OTHER WORDS, THEY BEGIN TO SEE THROUGH THOSE PRAGMATICS THAT IT IS ACTUALLY POSSIBLE TO DO. THAT IT'S NOT AN IMPOSSIBLE PIPE DREAM. IF YOU TEACH BY DOING, THEY WILL LEARN BY DOING. THEY CAN'T LEARN BY DOING IF YOU DON'T TEACH BY DOING. SO THIS IS THE FIRST ONE. AND I ACTUALLY WAGER IF WE DO LOTS OF THIS WHEN WE HAVE CERTAIN MORAL LEGITIMACY WHEN WE LEARN TO DO THE SAME. IF WE DON'T DO THIS, WE COME ACROSS A LITTLE BIT INDIFFERENT. WE'RE BEING SIGNALED. OKAY. THE NEXT ONE IS WEAVING OR MIXING OR SOME OTHER TERM THAT IS WHEN YOU'RE TEACHING ABOUT CLINICAL PRACTICE OR POPULATION HEALTH, WHATEVER YOU DO, THAT YOU INTEGRATE EVIDENCE FROM CLINICAL CARE RESEARCH ALONG THIS OTHER KNOWLEDGE YOU'VE USED IN HOW YOU TEACH. SO THAT LEARNERS SEE USING THAT KNOWLEDGE FROM CLINICAL CARE RESEARCH, THAT EVIDENCE AS PART OF GOOD CLINICAL LEARNING, AND BY SHOWING EXPLICITLY HOW YOU MAKE DECISIONS AND WEIGH THIS, YOU CAN TEACH WHAT I THINK OF WHAT'S BEEN ADDRESSED. HERE IT IS SEPARATE THE OIL AND THE VINEGAR. LET ME GIVE YOU AN EXAMPLE. AS A GENERAL INTERNIST I'M OFTEN ASKED TO TEACH IN THE CURRICULUM AT MY NEW SCHOOL THE STUFF THAT NOBODY ELSE WANTS TO TEACH. AND PERHAPS AS A GENERAL INTERNIST I SHOULD KNOW BETTER BUT I'M NOT AFRAID OF THESE. I'M QUITE IN FACT FASCINATED BY THE PROBLEMS OF THE RECOGNITION AND TREATMENT OF FLUID PROBLEMS IN OUR PATIENTS. A LOT OF IT COMES DOWN TO THE COMBINATION OF THE PHYSIOLOGY AND APPLIED PHYSIOLOGY OF FLUID MOVEMENT ACROSS ALL THESE SPACES IN OUR BODIES WAS THE EVIDENCE ABOUT THE DIFFERENT SIGNS FOR FLUID DISORDERS AND HOW ACCURATE THEY ARE. AND THE EVIDENCE ABOUT THE TREATMENTS FOR THOSE FLUID DISORDERS. SO I TAKE THE VINEGAR, YOU'LL PARDON ME I HOPE OF THE PHYSIOLOGY, MIX IT WITH THE OIL OF THE EVIDENCE FROM CLINICAL CARE RESEARCH. ADD SOME SPICE WITH SOME CASES, THAT SORT OF THING. AND WE TEACH THEM TOGETHER. NOW REMEMBER, IF IT'S WIRED TOGETHER IT'S GOING TO BE FIRED TOGETHER. WHAT HAPPENS NOW IS THAT PHYSIOLOGY IS TAUGHT IN THE FIRST YEAR. WHEN DO THEY LEARN THE EVIDENCE ABOUT THE ACCURACY OF THESE FINDINGS IF THEY LEARN IT AT ALL, IT'S IN THE THIRD YEAR. THEY CAN'T REMEMBER THOSE THINGS THAT WAY. IF YOU TEACH THEM TOGETHER, THEY WILL LEARN THEM TOGETHER. NOW, THEY STILL HAVE MEMORY PROBLEMS. BUT AT LEAST YOU CAN HOPE THAT THEY WILL WIRE IT TOGETHER IN THEIR MEMORIES AND ACCESS IT THAT WAY AS THE SCIENTISTS THAT TAUGHT US THAT WE DO. THE THIRD ONE IS WHAT MOST PEOPLE THINK OF THAT I MEAN WHEN WE SAY TEACHING EVIDENCE BASE PRACTICE, AND THAT IS TARGETING SPECIFIC SKILLS IN EVIDENCE BASED PRACTICE. AND YOU KNOW, WHETHER IT'S ASKING QUESTIONS, IT'S ANYTHING ABOUT THAT CYCLE. SO I'D LIKE TO HAVE EVERYONE RAISE THEIR LEFT HAND AND THEM KEEP IT UP IF YOU'VE BEEN TO SOME SESSION WHERE YOU THOUGHT THIS WAS HAPPENING, WHERE YOU WERE LEARNING SOMETHING ABOUT EVIDENCE BASED PRACTICE. OKAY. YOU CAN LOWER IT DOWN IF YOU HAVEN'T. STILL THAT'S A LOT OF YOU WHERE YOU'VE HAD SOMETHING THAT YOU WERE LEARNING THAT YOU IDENTIFIED THIS. AND AGAIN WHAT I'D SAY IS, IF THE OTHER TWO ARE HAPPENING, IF YOU SEE PEOPLE ROLE MODELING, IF YOU SEE PEOPLE WEAVING IT, THEN YOU MAY BE CURIOUS WELL HOW DID YOU KNOW THAT. I'LL TELL YOU A STORY. I OCCASIONALLY TEACH ON IN PATIENT ROUNDS AND THEY OCCASIONALLY HEARD THAT I'M INTERESTED IN EVIDENCE BASED MEDICINE. SOME GROUPS SAY DR. RICHARDSON SAY WE'VE HEARD YOU LIKE THAT STUFF BUT WE DON'T WANT TO LEARN THAT THIS MONTH, WE WANT TO LEARN CLINICAL MEDICINE. I SAY OKAY, FINE. AND I THINK THEIR FIRST SHOCKED. AND WE DO IT ALL MONTH LONG, WITH A HE DON'T CALL IT THAT. WE'RE JUST TRYING TO MAKE SENSE OF OUR PATIENT'S TEST RESULTS, WHICH OF THESE ARE MORE RELIABLE. THIS CLINICAL FINDING. HOW DO YOU GATHER IT. WELL HOW ACCURATE IS IT. THIS TREATMENT THEY SEEM TO BE HAVING AN ADVERSE EFFECT. ANALYSIS AIS THAT A COMMON ONE, LESS COMMON ONE. ALL THESE THING ARE WAYS TO BRING EVIDENCE INTO CLINICAL CARE RESEARCH IN EVERY DAY DECISION-MAKING WITHOUT NECESSARILY A LABEL. ABOUT HALFWAY THROUGH THE MONTH THE BRIGHTER TEAMS HAVE FIGURED ME OUT DISME SAY WAI OUT AND SAY WE'RE TEA CHING US MEDICINE USING CLINICAL EVIDENCE AND NOW WE WANT TO KNOW HOW YOU'RE DOING THAT. WHEN YOU SAID THIS TEST RESULT WAS MORE RELIABLE THAN THAT ONE, YOU SEEM TO PULL OUT A TABLE OF NUMBERS AND THOSE NUMBERS SPOKE TO YOU IN A WAY THEY DON'T SPEAK TO US. WHAT DO THEY TELL YOU THEY DON'T TELL ME. HOW CAN I LEARN TO LEARN THAT WAY. THEN YOU HAVE IT, THEY WANT IT. AND USUALLY THEY ARE DELIGHTED TO LEARN IT IF YOU SHOW ITS DIRECT RELEVANCE. BECAUSE WHY? MOST PEOPLE ARRIVED TO ME HATING STATISTICS ALREADY. IT'S NOT NEUTRAL, THEY'RE IN THE GROUND. I MEAN A HOLE. WILL RODGERS SAID IF YOU FIND YOURSELF IN A HOLE THE FIRST WILL STOP DI DIGGING. IT'S OFTEN NOT THAT MUCH ABOUT STATISTICS BUT MORE IN DETERMINING THE EFFECTS SIDES AND THAT THING. MY POINT IS IF THEY SEE IT AS FOREIGN TO THEIR CAREERS THEY TYPICALLY WON'T LEARN IT. IF THEY SEE IT DIRECTLY RELEVANT TO THEIR CAREERS AND YOU CAN SHOW THEM HOW IT FITS TO THE DECISIONS THEY MAKE THIS WILL LEARN IT PARTICULARLY IF IT MAKES THEM FASTER. I HOPE TO SHOW YOU SOME OF THOSE. YOU HAVE TO START WITH THEIR LEARNING NEEDS AND THEN TOWARDS WHAT YOU WANT TO TEACH THEM. THIS IS MY REMINDER. I SHOWED YOU THAT KIND OF ROUND CIRCLE. YOU CAN THINK OF THAT AS A PIE. A LOT OF PEOPLE THINK THEY HAVE TO TEACH THE WHOLE THING ALL THE TIME. OTHERS DON'T THINK YOU DO. I THINK OFTEN YOU CAN TEACH SMALLER SLICES AND I'LL AIL ADMINISTRATE WITH JUST TWO -- ILLUSTRATE WITH TWO BECAUSE THEY ARE COMMON AND I DON'T WANT YOU TO GET BOGGED DOWN WITH THE DETAILS. ONE OF THE TRADITIONS OR -- PROBABILISTIC TRADITION OR EVIDENCE BASE THERE'S PROBABILITY ZERO FROM ONE IF YOU'RE A PUREST FROM ZERO TO 100% IF YOU'RE A PERCENTAGIST, I MAY HAVE JUST MEAD THAT WOR MADE THAT WORD UP. THE POINT IS OUR UNCERTAINTY ABOUT A DIAGNOSE WHETHER IT'S PREJUDICE OR NOT. AND THAT THIS THRESHOLD APPROACH IDENTIFIES THAT THERE IS A PROBABILITY ABOVE ZERO BUT NOT VERY HIGH WHICH WE'RE READY TO ACCEPT THAT THAT DIAGNOSIS IS NOT THERE EVEN IF IT REALLY IS THERE AND BE WILLING TO TAKE THAT AIR RATE BECAUS ERROR RATE BECAUSE TH E COST FROM TESTING OR THE HARM FROM TESTING IS SIGNIFICANT. SIMILARLY THERE'S A LEISURE HOLD AT THIS END -- A THROASH HOL THRESHOLD AT THIS END. YOU MAY HAVE HEARD THE STORY OF HERMEHERPES INSELFITIS -- WHEN I WAS IN R1, TREATMENT FOR HER PEST INFEINSELF LIIVES WAS -- AND THE DIAGNOSTIC TEST WAS BRAIN BIOPSY. WE DIDN'T WANT TO GIVE ERROR C IF WE DIDN'T HAVE TO SO WE HAD A PRETTY HIGH THRESHOLD FOR LOOKING FOR IT AND A HIGH THRESHOLD FOR TREATING FOR IT. NOW THE DIAGNOSTIC TEST MR AND THE TREATMENT INVA INVENTORY ES -- OR SOME OF THE NEWER FIRST COUSINS. WE'RE MUCH HAPPIER TESTING EARLIER AT THE A LOWER THRESHOLD SO IT HAS COME DOWN OVER TIME AND THE TREATMENT THRESHOLD HAS ALSO COME DOWN BECAUSE WE'RE WILLING TO GIVE PSYCH VEER WHICH IS MUCH FAVOR THAN ERROR C EVEN IF WE'RE NOT ENTIRELY SURE THEY HAVE IT YET. SO THOSE CONCEPTS YOU PROBABLY HAVE A FEEL FOR EVEN IF YOU'RE NOT SURE WHERE THE NUMBERS COME FROM. AGAIN I DON'T WANT TO GET BOGGED DOWN. BUT ONE OF THE THING THAT'S IMPORTANT AND I'VE WORKED HARD TO DO IS A LOT OF PEOPLE PORTRAY THIS PROBABILISTIC APPROACH WHEN THEY'RE TEACHING EVIDENCE BASED MEDICINE AS THE WAY TO DO DIAGNOSTIC REASONING. BUT STUDENTS HAVE ALREADY LEARNED FIVE OTHERS. THEY'VE LEARNED THE DESCRIPTIVE TRA DEXESTRADITIONS, THE CRITERIA BASE FOR RHEUMATIC CONDITIONS. THEY'VE LEARNED A LOT THAT A BROKEN BONE HERE IS DIFFERENT FROM A BROKEN BONE HERE SO THERE'S AN AN ANATOMIC DECISION AND BIO PSYCHOSOCIAL. WE HAVE TO ADD PAIN, NOT SUBSTITUTE. THEY DON'T WANT YOU TO COME AND TAKE AWAY WHAT THEY ALREADY KNOW. THEY SPENT ALL THIS HARD WORK LEARNING THAT AND THEY'LL DEFEND THEMSELVES AGAINST YOU TAKING IT. BUT I DON'T THINK WE HAVE TO. ADDING EVIDENCE IN THIS CONSIDERATION DOESN'T REQUIRE US TO SUBTRACT THE OTHER KNOWLEDGE. IN FACT, THEY'RE PROBABLY BETTER DOCTORS IF THEY CAN FLAWLESSLY DID WE GO BACKWARD. NO WE WENT FORWARD. THEY WOULD BE BETTER IF THEY I DON'TED ALL SIX. SOMETIMES WE ARE DOING ANALYTIC THINKING AND SOMETIMES WE'RE DOING PATTERN RECOGNITION. SO PUTTING EVIDENCE BASE THE DIAGNOSIS IN THE CONTEXT OF ACTUAL CLINICAL DIAGNOSIS IS EASIER FOR MOST PEOPLE TO UNDERSTANDING AND ACCEPT THOUGHTN'T COMPLETELY WIPING THE SLATE CLEAN AND STARTING OVER FOR THEM. SO THAT'S A MODERATELY LARGE SIZE SLICE. NOW LET'S TACKLE A SMALLER EXAMPLE. AND IT'S FROM WHAT I THINK OF AS THE ART OF APPLICABILITY NOW. IF WE KNOW ANYTHING ABOUT BIOLOGICAL RESEARCH AND THAT'S THAT HUMANS VARY. AND THEY VARY IN MANY IMPORTANT WAYS AND NOW WE'RE STARTING HOW THEY VARY. GENETICS, EPI JETS AND LOT GENETICS AND LO TS OF EXPLANATIONS FOR IT. A BIG PART IS THAT THEY VARY IN THEIR BIOLOGY, THEIR PSYCHOLOGY, THEIR SOCIOLOGY THAT DEFINES THEIR CIRCUMSTANCES AS BROADLY AS YOU CAN THINK OF IT. AND THE WAY THEY VALUE AND CHOOSE BASED ON THEIR CIRCUMSTANCES. AND THOSE VARIATIONS CAN BE REPRESENTED. SOME OF THOSE VARIATIONS HAVE TO DO WITH THE REPRESENTATION OF THEIR BASELINE RISK OF BAT OUTCOMES FROM A -- BAD OUTCOMES FROM A CONDITION. AND SCALED WITHOUT UNITS FROM LOW TO HIGH. SO IT COULD BE BY LOGICAL DETERMINANTS, SOCIAL DETERMINANCES OR ALL OF THESE DETERMINANTS OF HEALTH. THERE ARE ALSO SOME OF THESE SAME THINGS BUT MAYBE DIFFERENT THINGS THAT COULD DETERMINE THEIR RESPONSIVENESS TO A TREATMENT. AND SO THOSE SAME OR OTHERS COULD DETERMINE THEIR VULNERABILITY TO ADVERSE EFFECTS. ALL OF THESE SCALED WITHOUT UNITS FROM LOW TO HIGH EXCEPT THIS ONE FROM HIGH TO LOW. YOU'LL SEE WHY NOW BECAUSE YOU CAN CREATE A TUBE THAT REPRESENTS VARYING DEGREES FROM LOW TO HIGH OF RESPONSIBILITY. ASK YOURSELF IF YOU WERE THE DESIGNER OF A TRAIL OR TREATMENT WHICH GROUP DO YOU WANT IN THIS TRIAL IN YOU WANT TO MAXIMIZE THE CHANCE YOU'D FIND BENEFIT. A LOT OF YOU WOULD ALREADY GO WELL WE WANT PEOPLE WITH HIGH BASELINE RISK SO THAT PRETTY FREQUENCY, PRETTY SOON WE WOULD START SEEING PEOPLE WITH A BAD OUTCOME IF THE TREATMENT DIDN'T WORK. WE'D ALSO WANT PEOPLE WHO ARE VERY RESPONSIVE AND VERY LOW CHANCE OF HAVING AN ADVERSE EFFECT. AND SO SURPRISE SURPRISE, MOST SO-CALLED EFFICACY TRIALS ARE UP IN THIS CORNER. AND WHEN WE INVENTORY TRY TO BROADEN THE TRIALS TO SEE DO THEY WORK IN OTHER GROUPS OF PEOPLE, THOSE TRIALS ARE OFTEN MORE LIKE THIS WHERE WE TAKE HIGH AND MEDIUM RISK AND WE TAKE HIGH AND MEDIUM RESPONSIVENESS, LOW AND MEDIUM VULNERABLE THEY AND WE ACCEPT THEM INTO OUR TRIALS. CLINICAL PRACTICE IS QUITE A BIT MORE LIKE THAT, OKAY. NOW WE'RE NOT REALLY THINKING OF DOING IT IN THE ABSOLUTELY LOW BASE RISK OR IMPOSSIBLE RESPONSIVENESS OR ENORMOUSLY HIGH VULNERABLE THEY BUT MOST OF THE TIME WE'RE TRYING TO MAKE JUDGMENTS ABOUT THAT SIZE FROM INFORMATION GENERATED FROM HERE. AND THIS ISSUE OF APPLICABILITY CAN BE DEALT WITH OFTEN AT THIS CONCEPTUAL SEMI QUANTITATIVE GRAPH CUL LEVEL WITHOUT NECESSARILY GETTING INTO THE ARITHMETIC OF HOW THIS GETS DONE. SO IN PEOPLE UNDER EXTEND THE WORDS AND THE CONCEPTS BEFORE THEY UNDERSTAND THE FORMULA AND THE NUMBERS, THAT'S OKAY. PEOPLE, THIS IS ALL THEY'RE GOING TO GET BUT MANY CAN GO ON TO THE NUMBERS IF THEY'VE UNDERSTOOD THE CONCEPTS. MOST OF JUDGE APPLICABILITY OF RESULTS GO DOWN TO SAY ARE WE AT THEM OF STUDY, THIS ONE OR THIS ONE. OKAY. SO THAT'S ANOTHER SMALL SLICE THAT YOU CAN TURN AND THEN TURN TO TEACH AS YOU BEGIN TO PREPARE YOUR LEARNERS FOR THEIR WORK IN EVIDENCE BASED PRACTICE OR HEALTHCARE. OF COURSE AS YOU CAN PROBABLY IMAGINE, REALLY REALLY RECOMMENDING THAT YOU COMBINE THEM. YOU LEARN EACH OF THESE MODES. ROLE MODELING, WEAVING AND TARGETING SPECIFIC SKILLS. AND THEN TEACH IN THE NATURAL CONTEXT THAT YOU FIND YOURSELF, WHETHER IT'S CLASSROOM OR THE CLINICAL SETTINGS. AND YOU ADJUST THESE TO FIT SO THAT YOU'RE TEACHING WHAT I DON'T REMEMBER LEARNERS NEED BASED ON YOUR STRENGTHS AS WELL. DO YOU HAVE QUESTIONS ABOUT THIS BEFORE WE MOVE ON? THAT'S PURSUE THE PROBLEM NOW HOW DO YOU PREPARE NOW AS CLINICIANS, BASICALLY SENIOR UNDER GRADUATES FOR LEARNING AND PRACTICING THIS WAY. THAT'S THE JAM I FOUND MYSELF IN 2009 WHEN I WAS ASKED TO BE THE ASSOCIATE DEAN FOR CURRICULUM. IN A WAY IT'S PEOPLE BASICALLY SAYING OKAY YOU'VE SAID SOME THINGS ABOUT EDUCATION, NOW IS THE TIME FOR YOU TO SHOW US PUTTING YOUR MONEY WHERE YOUR MOUTH IS. HOW ARE YOU REALLY GOING TO DO THIS. AND I SHOULD TELL YOU THAT WHAT WE'VE UNDERTAKEN IS NOT JUST ABOUT INTEGRATING EVIDENCE INTO THE KNOWLEDGE, BUT INTEGRATING. MORE BROADLY THAN THAT AND MUCH DEEPER THAN THAT. AND I WANT TO GIVE YOU THAT CONTEXT BEFORE WE TALK ABOUT INTEGRATING EVIDENCE. SO HERE'S WHAT WE'RE GOING TO TACKLE IS SHOWING AN EXAMPLE. I DON'T THINK IT'S THE EXAMPLE. IT'S A WORKED EXAMPLE FROM OUR WORK IN ATHENS. THIS IS OUR BUILDING, A 160 YEAR OLD BILLING ON TH BILL BUILDING ON THE RIV ER. THIS IS WHERE THE WATER FROM THE RIVER GOES UNDER THE BUILD&IT'S STIL-- ANDON RANY RAINY DAYS IT GOES UNDER THE BUILDING. ATHENS IS A WONDERFUL DOWN, A GREAT UNIVERSITY. I'M TOLD THERE'S A FOOTBALL TEAM THERE. SO WE HAD TO GET OUR STUDENTS' ATTENTION. WE WERE HIRED PRETTY MUCH IN 2009 AND IN ONE YEAR WE WERE TO OPEN OUR DOORS SO IN 2010 WE HAD TO HAVE A FIRST YEAR WORKING CURRICULUM. AND WE'VE WORKED VERY HARD TO GET THIS DONE AND YET IT'S STILL A WORK IN PROGRESS. WE'VE JUST FINISHED OUR SECOND YEAR. AND WE DOING SOME SOMETIME BECAUSE IT WAS 2010 TO REFLECT ON THE PRIOR HUNDRED YEARS AS TO WHAT WE HAD LEARNED FROM MEDICAL EDUCATION. IF YOU WERE OPENING A MEDICAL IN 1910, YOU'D HAVE HAD ACCESS TO -- REPORT AND HIS HOLDING UP OF JOHNS HOPKINS AS THE MODEL. IN WHICH NOW THEY'VE BECOME ALMOST ENSHRINED THE TWO YEARS OF BASIC SCIENCE AND TWO YEARS OF CLINICAL ROTATION. MOST PEOPLE THINK OF THEM AS THE PERSON WHO MADE SURE WE HAVE RESEARCH UNIVERSITIES ASSOCIATED WITH MEDICAL SCHOOLS AND THAT'S TRUE. IN ADDITION, HE WAS A BIG PROMOMENT OPROPONENT OF SUPERVISED OR GRADED PROVISION. YOU COULDN'T JUST LECTURE STUDENTS OR PUSH THEM OUT WITH A DEGREE, THEY ACTUALLY HAD TO LEARN THROUGH DOING. AND THERE ARE OTHER THINGS ATTRIBUTED TO HIM AS WELL. AT THE TIME, THOUGH, THE SHAPE OF THE MEDICAL SCHOOLS REFLECTED THESE SEVERAL BELIEVES THAT MANY PEOPLE STILL HOLD. THOSE ARE FOUNDATIONAL POLICE THAT HAVE BEEN GREAT IN MEDICAL EDUCATION. ONE OF THEM IS FOUNDATIONAL SUBJECTS CAN BE TAUGHT EARLY AND ONLY ONCE. LET'S TAKE ANATOMY. WE TEACH ANATOMY IN THE FIRST YEAR IN MANY PLACES. WHY DO WE ONLY TEACH IT ONCE. WHY IS IT THAT WE OVERLOAD OUR STUDENTS IN THE FIRST YEAR BEFORE THEY UNDERSTAND WHY THEY NEED IT AND NEVER COME BACK AFTER THEY'VE GONE THROUGH THE CLINICAL ROTATIONS AND GO HEY I WANT TO BECOME A CHEST SURGEON. NOW I SHOULD KNOW A LOT MORE ABOUT CHEST ANATOMY THAN I DID WHEN I WENT THROUGH ANATOMY. WHY DO WE TEACH IT ONLY ONCE. THERE WERE, IT WAS GOOD TO INCULPATE GENERAL MENTAL FACULTIES. BUT THE STUDIES OF DISCIPLINES WAS PRIMARILY THAT IT WOULD STRETCH AND STRENGTHEN YOUR MIND AND THESE ARE PEOPLE WHO MADE SURE WE STUDIED LATIN AND PHILOSOPHY TO STRETCH AND STRENGTHEN THEIR MIND. AND THAT ALSO THAT TO USE KNOWLEDGE, STUDENTS HAD TO MASTER ITS METHODS OF QUEER TO USE A BETA BLOCKER YOU HAD TO DO STUDIES ON BETA BLOCKERS AND LIE LIGANDS. WE KNOW THAT'S NOT SO. THE INCORPORATIO NOTION IS IT WOULD CHANGE SLOWLY AND STUDENTS WOULD BE ABLE TO INTEGRATE THIS KNOWLEDGE ALL ON THEIR OWN. THIS IS AKIN TO THINKING THAT YOU COULD GO INTO A DOUGHNUT SHOP AND SAY I'D LIKE TO CHOCOLATE GLAZED DOUGHNUT AND SOMEONE HANDS YOU A CUP OF FLOWER A TEASPOON AVENU OF BAKING SODA AND SOME COCA AND SAY THERE'S YOUR DOUGHNUT. THAT'S WHAT WE'RE DOING NOW IN MEDICAL SCHOOLS ACROSS THE COUNTRY. DOES THAT MEAN SOMEHOW THAT I'M DOING -- OH SOMEBODY HAS THEIR CELL PHONE? OKAY. AS LONG AS WE'RE NOT BEING SIGNALED TO EVACUATE OR SOMETHING. IF THAT'S TRUE, YOU LET ME KNOW. THE BIGGEST PROBLEMS THIS IDEA THAT TRANSFER WILL HAPPEN AUTOMATICALLY. NOW WHAT DO I MEAN BY THESE LAST TWO WORDS? INTEGRATION, MOST PEOPLE TALK ABOUT HORIZONTAL VERSUS VERTICAL BUT I THINK INTEGRATION AS THE DELIBERATE PUTTING TOGETHER OF ANY KNOWLEDGE, ANYTHING THAT WOULD GUIDE DECISIONS. AND THAT COULD BE HORIZONTAL, DISCIPLINES THAT ARE TAUGHT IN THE SAME YEAR OR VERTICAL BUT CLINICAL AND THE SCIENCE. AND TRANSFERS THIS IDEA OF USING KNOWLEDGE ACQUIRED FROM ONE CONTEXT TO SOLVE DIFFERENT PROBLEMS IN OTHER CONTEXT. AND THIS IS WHAT THE LEARNING SIEBSZ SAY ISCIENCES SAY IS THE HOLY GRAIL . THEY WANT PEOPLE TO LEARN ON CONTEXT AND APPLY IN ANOTHER. MOST ON THE LEARNING SIERNS STUDIESCIENCEHAVE SHOWN WE APPLY BEST IN T HE CONTEXT WE LEARN IT. SO FOR INSTANCE, DIVERS HAVE BEEN CHECKED ON THINGS AS MUNDANE AS SPELLING WORDS AND THEY'VE CHECKED THEM ON LAND AND IN SEA. AND THE ONES THEY LEARN DOWN IN THE DEEP, THEY SPELL BETTER DOWN IN THE DEEP THAN THEY DO ON LAND. AND THE REVERSE, THE ONES THEY LEARN ON LAND, THEY SPELL BETTER ON LAND. THE KNOWLEDGE WE KNOW IN THE CLASSROOM SETTING, WE DON'T NECESSARILY KNOW WHEN WE'RE TAKING CARE OF PATIENTS. IT DOESN'T TRANSFER AUTOMATICALLY. AGAIN, THERE'S THIS IDEA THAT THESE THINGS WILL MASSIVELY AND EASILY AND AUTOMATICALLY HAPPEN WHEREAS THE LEARNING SCIENTISTS THE LAST THREE DECADES HAVE TAUGHT US IT TAKES A LOT MORE ACTIVE TRANSPORT. NOW WE HAVE FUNCTIONAL MRI AND NOW WE HAVE PET SCANS AND OTHER IMAGES AND BETTER WAYS OF MEASURING COGNITIVE TASKS, THEY'VE BEEN ABLE TO LABEL THE GLUCOSE THAT WAS SHOWING EXACTLY HOW MUCH ENERGY IT TAKES TO LEARN OR RELEARN THINGS. IT'S FASCINATING THAT IF YOU ALREADY HAVE A BELIEF YOU USE LESS ENERGY THAN IF SOMETHING IS COUNTERTO YOUR BELIEF. WHY IS THAT? IT HAS TO DO WITH NEUROCOMMITMENT. WHEN WE LAY DOWN A MEMORY TRACE WE'RE PUTTING TISSUE ON FROM ONE CELL TO THE NEXT. DENY DRIEDZDENDRITES, MOLECULES. WE'RE COMMITTING TO CERTAIN TRACES. IF WE THEN FIND COUNTERVAILING ARGUMENTS THAT MAKE US CHANGE OUR MIND WE HAVE TO DECOMMIT THAT AND RECOMMIT SOMETHING ELSE. AND THAT LITERALLY IS A BIOLOGIC CONTRIBUTORRER TO RESISTANCE TO CHANGE. NOT DO NOT WE PSYCHOLOGICALLY WANT TO DEFEBLED THE KNOWLEDGE WE SPEND SO MUCH TIME GAINING BUT WE HAVE TO RECOMMIT NEURONS. THAT'S HARD FOR US, ALL OF US. NOT JUST DOCTOR. HERE'S THE FIRST YEAR OF CLASS, THEY ARRIVED. VERY BRIGHT, VERY EAGER AND WE SAID WE WANT TO HELP YOU LEARN BY ENGAGING MORE IN ACTIVE LEARNING. THERE WILL BE CONTEXT FOR THE CONTENT YOU DO THAT WE WILL HAVE YOU LEARN IN TEAMS. LIKE YOU'LL PRACTICE IN TEAMS LATER. WE WANT YOU TO BUILD COHERENT EXPLANATORY NARRATIVES, NOT JUST MEMORIZE A LOT OF FACTS THAT ARE UNRELIGHTED BUT THEY ARE EXPLANATIVE NARRATIVES. THINK ABOUT THIS, IF YOU WERE TO WALK UP TO A BUNCH OF KIDS ON THE STREET NOW AND SAY WHO DOES PROFESSOR NATHAN TEACH. OF THEM WOULD KNOW, WHY BECAUSE THEY WOULD HAVE EITHER SEEN THE MOVIE OR READ THE BOOK. WITHOUT NEEDING TO LEARN, JUST HEARING THE STORY, JUST BEING ENGAGED BY THE STORY OF THE PEOPLE THEY'VE COME TO CARE ABOUT, THEY LEARN A LOT OF THINGS. LIKE WHO PROFESSOR SNAPE IS AND WHAT HE FEATURES. IT' AND WHAT HE TEACHES. IT'S EASIER TO UNDERSTAND PHENOMENA WHEN ALL THE PIECES FIT TOGETHER AND YOU SEE HOW IT ALL WORKS AND YOU CAN SEE THE PIECES AND THEIR RELATIONSHIP TO OTHERS SO THAT YOU CAN BEGIN TO NAME THEM AND TELL THEIR STORY. OTHERWISE IT'S A LOT OF FACTS TO BE MEMORIZED SEPARATELY. WE WANTED TO USE COGNITIVE SCAFFOLDING, MORE ON THAT IF YOU'D LIKE. OF COURSE THIS IDEA. AND THIS IDEA, THE IDEA TO BECOME EXPERT IN ANYTHING, THE STUDIES THAT THE EXPERTISER WOULD SUGGEST AS IT PROBABLY KNOW IT TAKES OVER TEN YEARS AND OVER 10,000 HOURS OF DELIBERATE PRACTICE TO GET GOOD AT SOMETHING. AND WE WANTED TO INTEGRATE ACROSS MANY LEVELS THE ISSUE ABOUT ACTIVE VERSUS PASSIVE. I'M HERE AT THE NIH I FIGURE YOU CAN HANDLE A BIOLOGIC UNDERSTANDING. PASSIVE LEARNING IS THE ACT OF FILLING LEARNING MANY OF US EXPERIENCED IN UNDERGRADUATE AND PRECLINICAL LEARNINGS SITUATIONS WHERE SYSTOLIC IS THE ACTIVE LEARNING. BUT THEY DO DIFFERENT THINGS. HERE YOU CAN ACQUIRE FACTS AND YOU CAN IDENTIFY WHAT YOU DON'T KNOW. HERE YOU USE THE FACTS SO YOU HAVE TO ORGANIZE THEM TO USE THEM AND BY APPLYING AND TRANSFER, YOU LAY DOWN, YOU COMMIT NEW NEURONS TO HOW WE USE THIS KNOWLEDGE. SO INSTEAD OF BEING INERT IN OUR KNOWLEDGE CUPBOARDS IN OUR BRAIN, WE GET PRACTICE OPENING THE CUPBOARD AND POINT IT OUT AND USING IT REGULARLY ENOUGH SO THAT NOW WE CAN TRANSFER MORE EASILY, MORE READILY. YES? >> [INDISCERNIBLE] >> THEY DO, THEY DO. SO THE QUESTION HAPPENS TO DO WITH HOW THEY ARE DOING ON THE TESTING. AND I WANT TO SHOW A SAMPLE WEEK AND I'LL GET BACK TO YOUR QUESTION ABOUT HOW OUR STUDENTS LEARN THIS WAY. OH YES. THANK YOU. I SHOULD HAVE EBB ARED -- I SHOULD HAVE REMEMBERED. WILL YOU ASK IT AGAIN. >> I THINK THIS IS A REALLY GOOD IDEA. IT'S ON THE JOB LINING BUT I WORLD HOW THE STUDENTWONDER HOW THE STUDEN TS DO IN TESTING BECAUSE IN THE END THEY ARE WORRIED ABOUT THEIR SCHOOLS AND PLACEMENT. >> YES THEY DO. THANK YOU FOR YOUR QUESTION. HERE'S OUR SAMPLE WEEK. IN THE FIRST YEAR IT LOOKS LIKE THIS. IN THE SECOND YEAR IT'S VERY SIMILAR EXCEPT THE SMALL GROUP LEARNING CASE BASE LEARNING HAPPENS IN THIS TIME BLOCK. THEY START IN THE FIRST YEAR WITH SMALL GROUP AND THEY HAVE ENLARGED GROUPS WHICH YOU MIGHT CALL LECTURERS, HUMAN LABS, CLINICAL SKILLS AND SO FORTH. OVER HERE IT SAYS WEEKLY ASSESSMENT. WHAT THIS REPRESENTS IS A CHANCE FOR STUDENTS TO CHECK THEIR KNOWLEDGE BEFORE THEY LEAVE THE WEEK. NOW IT TAKES THE FERMEN FERMENT OF 30 THE FORM OF 30 MULTIPLE CHOICES EACH WEEK. SCIENTISTS TELLS US IF YOU HAVE REGULAR CHECKS OF YOUR KNOWLEDGE ALTERNATING CYCLES MORE OF LEARNING AND RELEARNING YOU DO DO BETTER ON FINAL EXAMS THAN TESTS. AT THE END OF TWO YEARS OUR STUDENTS HAVE EXPERIENCED WHETHER IT'S WEEKLY ASSESSMENTS OR FINAL EXAMS OVER 3 THYROID MULTIPL -- OVER30 PRACTICE QUESTIONS. THERE'S A CERTAIN AMOUNT OF FACILITY THAT COMES WITH DOING IT OVER AND OVER AND LEARNING TO APPLY YOUR KNOWLEDGE TO ASSESSMENTS ON A REGULAR BASIS. PEOPLE SAY ARE YOU TEACHING TO THE TESTS. NO THE PRIMARY REASON THIS TEST ENHANCE LEARNING, WE'RE HELPING THEM CHECK THEIR KNOWLEDGE. A SIDE EFFECT OF THIS IS THAT BY THE TIME THEY HIT THEIR STEP ONE, THEY'VE DONE A LOT OF PRACTICE FOR THIS KIND OF TESTING. AND YES THEY'RE NERVOUS LIKE EVERY OTHER MEDICAL STUDENT ALL AROUND THE COUNTRY IS NERVOUS RIGHT ABOUT NOW BECAUSE OUR SECOND YEARS HAVE JUST FINISHED EFFECTIVE HERE AND THE NEXT MONTH THAT WAS THEY HAVE TO TAKE STEP ONE. I HAVE YET TO MEET A CLASS OF MEDICAL STUDENTS WHO ARE NOT NERVOUS AT THIS TIME. ARE THEY MORE NERVOUS, I RECKON NOT AS A GROUP. BUT AS AN INDIVIDUAL, SURE BECAUSE HUMANS VARY BUT I THINK THEY ARE MORE PREPARED AS GROUPS AS WELL. LET'S LOOK AT THE OTHER STUFF. WE'VE GOT A THING CALLED ESSENTIAL OF DOCTORING OR ESSENTIALS OF CLINICAL MEDICINE AND THAT'S WHERE WE DO SOME LEARNING ABOUT EVIDENCE BASED PRACTICE, A THING CALLED COMMUNITY HOUSE AND A THINGS CALLED CLINICAL SKILLS. MORE ON THOSE IN MOMENT. I WANT TO SHOW AS A EXAWZ AT THICOMPOSITEWEEK THERE'S A DIFFERENT THEME . WHAT'S AN EXAMPLE. ONE WEEK IN THE FALL THEY START ON MONDAY MORNING WITH A CASE OF A YOUNG MAN, A SENIOR ON THE FOOTBALL TEAM WHO HAS AN INJURY TO HIS SHOULDER. AND AFTER THAT INJURY FEELS PAIN AND HE CAN'T MOVE HIS SHOULDER. CAN'T MOVE HIS ARM THEY REALIZE TO SOLVE THE PROBLEM THEY HAVE TO LEARN THE ANATOMY OF THE SHOULDER AND THE BRACHIAL PLEXUS. THAT'S EIGHT TO TEN. THE VERY FIRST LARGE GROUP IS AN ANATOMY PRESENTATION ABOUT THIS COMPLICATED INSTRUCTURE OF THE BRACHIAL PLEXUS. THE NEXT ONE IS A HISTOLOGY PRESENTATION ABOUT THE HISTOLOGY OF NERVES. THE NEXT DAY THEY LEARN ABOUT ACTION POTENTIALS AND THE PHYSIOLOGY OF NERVES. AND THEIR ANATOMY LAB IS THE ANATOMY OF THE SHOULDER AND THE BRACHIAL PLEXUS. AND CLINICAL SKILLS IS THE EXAMINATION OF THE SHOULDER. SO THAT WHAT WE'RE TRYING TO DO IS INTEGRATE NOT JUST BY ADJACENT WHERE THESE THINGS ARE ADJACENT BUT THE LEARNING OBJECTIVE AND ALL THESE THINGS ARE DIFFERENT PARTS ON THE ELEPHANT. THEY SEE ALL THESE PIECES NEXT TO EACH OTHER AND THEY INFORM EACH OTHER. BECAUSE WE'RE TALKING ABOUT VARIOUS CLINICAL DECISIONS IT ALLOWS US IN THIS SORT OF SESSION TO DEAL WITH THE EVIDENCE ABOUT THAT KIND OF DECISION. WHILE WE'RE AT IT, THE CASES ARE FAIRLY BROAD. WE'VE DECIDED IT'S NOT JUST THE STANDARD THING. WE'RE GOING TO INCLUDE FROM GENES TO SOCIETY ALL THE DETERMINANTS OF HEALTH WE CAN THROW IN THERE. THE AUTHENTIC CLINICAL PREDICTMENTS AND THE UNSOLVE OVER THREE DAYS. SO IN SOME SCHOOLS PROBLEM BASE LEARNING IS ONE CASE YOU GET EVERYTHING ON THE FIRST DAY AND THEN YOU HAVE TO DO FOLLOW UP. IN OUR SITUATION THEY GET A PART OF IT, THEY DISCUSS IT, THEY GET ANOTHER PART, DISCUSS IT AND SO FORTH. AND MORE PARTS HERE AND MORE PARTS THERE. SO THERE'S A SUSTAINED MYSTERY UNTIL FRIDAY. AND THAT GIVES US A CHANCE TO LEARN BETWEEN MONDAY AND WEDNESDAY AND BETWEEN WEDNESDAY AND FRIDAY NEW STUFF THAT RELATES TO THE DECISIONS THAT ARE BEING MADE. AND WE COACH THEM ON HOW TO FIND KNOWLEDGE. SO WE HAVE A LIBRARIAN WHO IS ON OUR FACULTY. SHE TEACHES SEARCHING AND SHE STEVENS ASKINTEACHES US ASKING QUESTIONS, SHE TEACHES US WITH APPRAISAL. WE HAVE DECISIONS ON DETERMINED HEALTH, BIOLOGIC AND SOCIOLOGIC. WE THINK ON THE NOTIONS ON THE SIDE THIS IS FROM A PRESENTATION I GAVE RECENTLY TO THE COLLEGE OF ED. THERE'S A BUNCH OF REASONS HERE EDUCATIONAL SCIENCE BUT THESE BOIL DONE TO THE INTERIMMEDIATE STEP, OUR SURROGATE MARKER OF IMPROVING INTEGRATION ON THE WAY TO THE LONGER TERM OUTCOME OF IMPROVING TRANSFER. SO THAT'S HOW AND WHY WE TEACH SCIENCE IN THE CASES AND THAT PROVIDES THEN SOME OF THE CONTEXT. AND WE'VE ORGANIZED AROUND LOTS OF DOCTORING TASKS, DECISIONS, PROBLEM SOLVING CONVERSATIONS. A LOT OF TIMES WE SAY MY DOCTOR DOESN'T KNOW HOW TO TALK TO ME AND I WONDERED DID THEY EVER LEARN, OKAY. SO WE HAD SESSIONS ON NOT ONLY THE EASY ONES, THE FAMILIAR ONES HOW TO BREAK THE BAD NEWS BUT EVEN TALKING ABOUT A DIAGNOSE, TALKING ABOUT THE ADVERSE EFFECTS OF TREATMENT. VERY SPECIFIC. SO THEN AGAIN YOU SAY WELL WOW, WE'VE JUST TOLD THEM HERE'S AN ADVERSE EFFECT OF TREATMENT. WHY NOT HAVE THE EVIDENCE ABOUT THE FREQUENCY OF HOW OFTEN THAT ADVERSE EFFECT APPEAR RIGHT THEN SO THAT WHEN THEY NEED TO UNITE IS THE TIME WE BRING THE KNOWLEDGE. SO THAT'S WHAT WE MEAN BY INTEGRATION THAT IT'S KNOWLEDGE AND TIME FOR THEIR LEARNING. AND IT COMES FROM ANY DISCIPLINE. WE'VE DECIDED NOT TO SPEND SO MUCH TIME THINKING ABOUT THE ROOT DISCIPLINE THAT THIS COMES FROM ANATOMY. BECAUSE WE DON'T DO IN LANGUAGE, RIGHT? YOU AREN'T SAYING, YOU AREN'T TOLD THAT IN ORDER TO USE THE WORD QUIZ, YOU HAVE TO LEARN ITS ETYMOLOGY. YOU HAVE TO USE THE WORD AND EMPLOYEE IT. THE WORD PATTERN DOES IT REALLY MATTER WHETHER IT'S LATIN OR GREEK THAT IT CAME FROM AND ITS HISTORY OVER TIME? WE SPENT A LOT OF TIME IN MEDICAL EDUCATION DECLAIMING THE ORIGIN OF KNOWLEDGE RATHER THAN EMPHASIZE HOW THAT KNOWLEDGE GETS USED IN EVERY DAY PRACTICE. A TRADITION WE SHOULD PROBABLY GIVE UP BECAUSE OUR STUDENTS RECOGNIZE IT PRIMARILY AS AN ELABORATE TWO YEAR LONG HAZING RITUAL THAT THEY THEN PRACTICE A KIND OF INTRAELECTUA INTELLECTUAL BULIMIA THAT THEY DON'T ABSORB AND THEY MOVE ON TO THE CLINICAL YEARS. SO THEN IF YOU THINK OF IT THIS I WITH A, YOU COULD ORGANIZE YOURSELF AROUND THESE TYPES OF DECISIONS. AND THE TYPES OF EVIDENCE YOU NEED. IF YOU'RE GOING TO TALK ABOUT RISK OR HARM, THANK YOU ABOUT PROGNOSIS, THANK YOU ABOUT TREATMENT OR DIAGNOSIS. AND EACH OF THESE HAS SUBGROUPS, OKAY. SUBTYPES AND SUBCACATEGORIZATION. THOSE APPEAR IN A LOT OF OUR CASES. IF WE THINK ABOUT THE SKILLS WE TALKED ABOUT ASKING ACQUIRING APPRAISING APPLYING AND THEN ACTING AND ASSESSING. AND THEN WE ADD THE THIRD DIMENSION OF THE CONTEXT EITHER SINGLE PATIENTS GROUPS OF PATIENTS, WHOLE POPULATIONS OR WHAT I THINK OF AS FUTURE PATIENTS. THIS GOES FOR LIFE LONG LEARNING. IF YOU MAP OUT THAT THREE DIMENSIONAL GRID, THEN YOU CAN BEGIN TO SAY WELL, ACTUALLY THERE'S LOTS OF OPPORTUNITY AS LOUNGLONG AS WE DO THE SLICING, RATHER THAN TRY TO TEACH THE WHOLE THING, WE CAN REINFORCE IN THE CASES IN THE SCIENCE MODULE THE RISK FROM SOMETHING. SO OUR VERY FIRST WEEK, OUR VERY FIRST CASE, I HOW THE RISING FIRST YEARS AREN'T LISTENING. HERE'S A PATIENT WITH SICKLE DISEASE, THEY DON'T KNOW THAT, THEY JUST SEE A YOUNG CHILD WITH PAIN. WE LEARN ABOUT THE GENE ASSOCIATED AND WE LEARN ABOUT THE RISK OF TRANSMISSIONS AND SO FORTH. AND SO EARLY ON THERE ARE QUANTITATIVE NOTIONS FROM GENETICS THAT ARE COMING IN SO WE BEGIN UNDERSTANDING RISKS. THE QUANTITATIVE EXPRESSION OF RISK AND HOW TO TALK ABOUT IT WITH OUR PATIENTS IN THE FIRST WEEK AS THEY ARE LEARNING THE SCIENCE OF GENE PRODUCT AND RED CELL MAKE UP AND SO FORTH. AND EVEN HEMOGLOBIN I BEEN HEMOGLOBIN COOPERATIVITY. IS IT COOPERATIVITY. I MAY HAVE LOST TRACK OF THAT BIT OF BIOCHEMISTRY. IN ANY CASE WE TRY TO LEAVE IT ALL TOGETHER. FOR THE STUFF THAT NEEDS EXPLICIT TEACHING, WE'VE DONE SOME WORK IN COMMUNITY HEALTH IN THE DEFENSES OF CLINICAL MEDICINE AND IN THE CLINICAL SKILL PARTS. I WANT TO TURN TO THOSE. IN THE FIRST TWO YEARS WE TALK ABOUT THESE SKILLS WITH THESE HEADINGS IN THOSE AREAS AND THEY LEARN TO APPRAISE EVIDENCE IN THESE TYPES SO THAT THEY CAN FIND IT AND USE IT AND MAKE SENSE OF IT. IN ADDITION, IN THE COMMUNITY HEALTH PROJECTS WE'VE TAKEN THIS IDEA OF LEARNING THROUGH DOING. INSTEAD OF JUST A FEW LETTERS ON COMMUNITY HEALTH. IN THE FIRST SEMESTER THEY WORK WITH CLINICAL PARTNERS IN THE COMMUNITY AND THEY DO A NEEDS ASSESSMENT. IN THE SECOND SEMESTER THEY DO A BRIEF INTERVENTION TARGETED AT THOSE NEEDS. FIRST YEAR MEDICAL STUDENTS WORKING WITH THE UNIVERSITY HEALTH, THAT FIRST YEAR WE'VE GONE THROUGH DEPARTMENT OPERATION, DEPRESSION CARE, THIS YOU MAY KNOW THAT ATHENS HOME TO A LARGE GREATIVE COMMUNITY, MANY OF THEM MUSICIANS AND OTHER TYPES BUT A LOT OF THEM, WHAT'S THE PLIGHT TER POLITE TERM UNDER INSURED. WE'VE PROVIDED CARE THERE. THERE'S ASTHMA, HEAD START AND THIS I THINK WAS GERIATRICS, I CAN'T QUITE REMEMBER WHAT THAT IS, NOW THAT I SEE IT. THE IDEA BEING THAT YES THEY HAVE SOME INTRODUCTORY IDEAS BUT THEN THEY ALSO LEARN THROUGH DOING. AND THAT LEAVES IN CLINICAL SKILLS, WE HAVE THEM AS I MENTIONED LEARN THE EXAMINATION, HOW TO DO IT BUT ALSO LEARN STUFF ABOUT HOW ACCURATE IT IS. OKAY. IT INCLUDES THINGS LIKE THE DELIVERING THE MESSAGES WE WERE TALKING ABOUT. ARE YOU FAMILIAR WITH THIS RATIONAL CLINICAL EXA SERIES IN THE JOURNAL OF AMERICAN MEDICAL ASSOCIATION. DAVID I SAW AND -- ARE AMONG MY MANY HEROES AND HERE'S WHY. THEY STUCK WITH SINCE 1992 THE DELIBERATE COLLECTION AND UPDATING OF WHAT IS KNOWN ABOUT THE ACCURATE SEE OF BITS AND PIECES OF THE CLINICAL EXAM. SO HISTORY OR PHYSICAL FOR THE DIAGNOSIS OF CERTAIN CONDITIONS. IT TURNS OUT FOR INSTANCE THERE ARE 43 DIFFERENT MANEUVERS FOR THE DIAGNOSIS OF CHRONIC OBSTRUCTIVE PULMONARY DISEASE DESCRIBED OVER THE YEARS. THREE OF THEM HAVE SUFFICIENT ACCURACY AND REPRODUCIBILITY THAT WE NOW TEACH OUR STUDENTS IN PHASE THREE BECAUSE THE OTHERS EVEN IN EXPERIENCED HANDS AREN'T REPRODUCIBLE ENOUGH OR ACCURATE ENOUGH TO BE WORTH THEIR TIME LEARNING. NOW WHEN WE WERE TALKING EARLIER ABOUT SPEED HERE'S AN EXAMPLE. WHY DO WE TEACH EPI NOOM ES BATTLE AND PIECES OF PHYSICAL EXAM HAVE A RATIO OF ONE MAKE NO DISTINCTION BETWEEN THOSE WITH AND WITHOUT DISEASE. WHY DO WE MAKE OUR STUDENTS LEARN THEM. WHY DO WE INCEST ON TESTING THEM IN CLINICAL SKILLS AND WHY DO WE ASSESS ON DEMONSTRATING THEM AT THE BEDSIDE. IS IT BECAUSE WE'RE TEACHERS OF HISTORY? IS IT BECAUSE WE'RE RELATED TO THOSE PEOPLE WHOSE NAMES ARE ON THERE? I WOULD LIKE TO KNOW WHY. I WANT TO KNOW WHY WE CAN'T CONCENTRATION OUR LEARNERS ATTENTION ON THE FEW FINDING THAT MATTER AND REALLY HELP US. NOT ONLY KNOW HOW TO GATHER THEM BUT HOW TO INTERPRET THEM AND HOW TO USE THEM IN CLINICAL MATTERS IN A WAY THAT SPEEDS THEM UP AS WELL AS MAKES THEM MORE ACCURATE AT DIAGNOSE. MAKE THEM MORE ACCURATE AT DIAGNOSIS AND SO FORTH. THE BITS OF HISTORY PHYSICAL, THERE ARE ALSO OTHER TESTS IN THAT SERIES. SO THAT'S HOW WE PUT ALL THOSE PIECES TOGETHER. INTO OUR CURRICULUM. AND ONE OF THE THINGS -- YES? >> [INDISCERNIBLE] >> THIS MAKES A LOT OF SINCE. I ALWAYS WONDERED IN A CURRICULUM LIKE THIS HOW DO YOU TEACH SOMETHING THAT MAY NOT BE AMENABLE TO INTERMITTENT TWO OR THREE HOUR SESSIONS LIKE IMMUNOLOGY OR GENETICS WHERE THERE'S A LARGE AMOUNT OF INFORMATION AND SORT OF THAT NEEDS TO BE TAUGHT SORT OF OVER A ROUGHLY SHORT PERIOD OF TIME BECAUSE A LOT OF IT MAY BE NEW. MAY NOT WORK EACH TWO HOURS OF IMMUNOLOGY NOW AND TWO HOURS TWO MONTHS FROM NOW. >> THANK YOU FOR YOUR QUESTION. I THINK IT HAS SEVERAL PARTS. SO HOW DO WE DEAL WITH THE LARGE GROUPS. SO DO WE HAVE LARGE GROUPS AND WE CLEARLY DO. AND HOW DO WE TEACH NEW KNOWLEDGE ALONG WITH OLD KNOWLEDGE, IF YOU WILL. AND YOU KNOW, IF YOU TAKE THE IDEA THAT WE SHOULD BE TEACHING OUR BEST KNOWLEDGE, WHETHER IT'S NEW OR OLD, OUR MOST RELIABLE, OUR MOST UNDERSTOOD, WE TRY TO HAVE THAT SELECTION GOING THERE. AND THEN ISSUE ABOUT PACING IS INTERESTING. I FOUND OVER THE TWO YEARS THAT AT FIRST MANY OF OUR SCIENTISTS HELD THE VIEW THAT YOU'RE TO HOLD THAT YOU HAVE TO TEACH ALL THESE THINGS TOGETHER. AND I THINK KNOW SOME OF THEM ARE STARTING TO CHANGE THAT VIEW. THAT IF THEY SEE THAT IF THEY EXPERIENCE A CASE IN WHICH THAT KNOWLEDGE IS DIRECTLY RELEVANT AND THEN THEY HEAR THAT KNOWLEDGE IN THE LARGE GROUP AND THEY SEE THAT CASE USE THAT KNOWLEDGE AND SO FORTH THEN THEY CAN BUILD THAT KNOWLEDGE MORE LONGITUDINALLY. NOW I DON'T MEAN ONCE NOW AND THEN ONCE A YEAR FROM NOW. WE TRY TO GET SOME SEQUENTIAL OUT OF IT BUT IT ISN'T NECESSARILY SO THAT THE ONLY WAY TO LEARN THE NEXT HOUR OF GENETICS IS IMMEDIATELY FOLLOWING THE FIRST HOUR. IN FACT, MANY OF OUR SCIENTISTS ARE COMING TO UNDERSTAND THAT IF THEY GO AWAY AND REALLY LEARN THOSE THING WE JUST ASKED THEM IN THE FIRST HOUR THAT A FEW DAYS LATER OR THE NEXT WEEK THEY CAN PICK UP THE THREAD AND MOVE FORWARD. PARTICULARLY IF THERE'S A CASE THAT MAKES THAT COME ALIVE. SO I TAKE YOUR POINT YET WHAT I SEE IN OUR SCIENCES IS THEY'RE SEEING OH, WE SHOULD TEACH OF THE UNDERSTANDING OF MY MATERIAL WHICH ISN'T NECESSARILY THE SPEED OF OUR SPEECH OR THE SPEED OF OUR MOUSE CLICKS OR HOW FAST THE SLIDES CAN GO BY. I RECENTLY ATTENDED A LECTURE WHEREIN HALF HOUR SOMEONE HAD 96 SLIDES. TO MY WAY OF THINKING THAT'S REALLY NOT ABOUT LEARNING, THAT'S A KIND OF SHOCK AND AWE CAMPAIGN. [LAUGHTER] YES, SPEAK LOUDLY, WE WILL HEAR YOU. WE'LL USE THE MICROPHONE FOR THE WEBCAST. I GUESS FOLLOWING THIS GENTLEMAN'S QUESTION, I REALLY I MIGHT WANT TO HEAR THE EVIDENCE THAT THAT IS THE CASE. I'M THINKING ABOUT A RECENTLY TAUGHT EKG TO A FIRST YEAR STUDENTS AND THEN IN THIS MODULE BASE SECOND YEAR STUDENTS. MOST OF THE TIME WAS SPENT REMINDING PEOPLE WHAT THEY HAD PREVIOUSLY LEARNED. SO YOU KNOW, I MEAN THERE MUST BE A REASON WHY IN COLLEGE WE TAKE A SEMESTER OF CHEMISTRY AND NOT AN OCCASIONAL LECTURE IN CHEMISTRY. I HAVE TO ADMIT THAT I'M A SKEPTICAL THAT SOMEONE COULD TAKE A LARGE BODY OF SCIENTIFIC KNOWLEDGE AND LEARN THAT AS EFFECTIVELY METHODICALLY AS THEY WOULD WITHOUT THE CONSTANT REINFORCEMENT OF REMINDING WHAT WE WERE TALKING ABOUT YESTERDAY. >> WELL THANK YOU FOR YOUR COMMENT. AND I ACTUALLY, I SUPPOSE I DID NOT BRING THE SLIDE THAT SHOWS YOU EXACTLY HOW COALESCED THESE REALLY ARE. FOR INSTANCE WHEN WE'RE IN MUSCULOSKELETAL MODULE IT'S SEVEN OR EIGHT WEEKS OF LEARNING A WHOLE LOT ABOUT MUSCULAR ANATOMY AND BONE ANATOMY AND THOSE PIECES COME FAIRLY FAST AND FURIOUSLY. AT THE SAME TIME THEY'RE LEARNING ABOUT BONE PHYSIOLOGY AND MUSCLE PHYSIOLOGY AND BONE AND MUSCLE ENERGETICS AND METABOLISM, DO YOU SEE WHAT I MEAN. SO IT'S NOT AS SEPARATE AS I MUST HAVE SOMEHOW COMMUNICATED TO YOU. IT'S NOT BY WEEKS OR YEARS IT'S REALLY MORE THAT IT'S NOT NECESSARILY ALL THE SAME WEEK BECAUSE THE PIECES THAT GO TOGETHER FOR THOSE TOPICS GO TOGETHER. BUT THE NEXT WEEK WE PICK UP QUITE A BIT MORE ANATOMY AND GENETICS AND SO FORTH. SO THERE IS A REGIONALIZATION WITHOUT NECESSARILY A TOTAL FRAGMENTATION IF THAT HELPS YOU. YES, MA'AM. >> BUT IS THERE ANY [INDISCERNIBLE] >> I WOULD BE DELIGHTED TO STAND HERE ONE DAY AND TELL YOU WE HAVE THAT EVIDENCE. I THINK WHAT WE HAVE IS IN MEDICAL EDUCATION A NUMBER OF STUDIES ABOUT PROBLEM BASED LEARNING THAT EITHER SHOWED NO DIFFERENCE IN THEIR KNOWLEDGE OR SOME MODEST IMPROVEMENT IN THEIR KNOWLEDGE FOR REGULAR FORM OF PROBLEM BASED LEARNING. THIS IS QUITE A BIT DIFFERENT FROM REGULAR PROBLEM BASED LEARNING AS I HOPE YOU'LL SEE QUITE A BIT MODIFIED. AND WE'VE DURAN WHA DONE IS IMPORTED A LOT OF IDEAS ABOUT GENERAL LEARNING SCIENCE ABOUT HOW TO GET THIS DONE. FROM THE GENERAL LEARNING SCIENCE PIECES OF THIS HAVE BEEN TESTED AND SHOWN THAT THEY LEAD TO BETTER KNOWLEDGE. BUT I DON'T KNOW THAT WE'VE DONE THIS EXPERIMENT WHERE ALL THESE PIECES PUT TOGETHER HAVE BEEN DEMONSTRATED TO DO BETTER. I HAD HOPED TO RANDOMIZE STUDENTS TO ONE CAMPUS OR ANOTHER. BUT THAT GOT ROUNDLY REJECTED, I'LL SAY. HAD WE BEEN ABLE TO RANDOMIZE WE MIGHT HAVE BEEN ABLE TO CONTROL FOR ALL THOSE WONDERFUL THINGS WE CALL CONFOUNDING VARIABLES WHICH MEANS THAT THREW LIKELIHOOD OF SUCCESS -- THEIR LIKELIHOOD OF SUCCESS IN THE MORE TRADITIONAL CURRICULUM IN AGUSTA VERSUS OUR CURRICULUM. BUT NONETHELESS THE MARYBD MEASURES OF OUR CURRICULUM OF OUR STUDENTS SAY THEY HAD THE SAME ENTERING M CAT SCORES THE SAME ENTERING GPA'S A SIMILAR DISTRIBUTION OF BACKGROUND AS THOSE IN AWE GONZALE --AWE AWE GUST GUSAWEAGUSTA. I THINK PERHAPS THE SKILL MEMBERS WILL BE DIFFERENT. THE ABILITY USE EVIDENCE IN THEIR EVERY DAY DECISION-MAKING MAY BE DIFFERENT. BUT EVEN PASSING THE BOARDS AT A SIMILAR RATE WILL BE STILL AN ACHIEVEMENT OF THIS. SHOWS IT CAN WORK THIS WAY. I HAVE LOTS OF PEOPLE WHO ARE SURE THAT I CAN'T WORK THIS WAY. PRIMARILY FOR THE REASON THAT IT'S NOT THE WAY THEY LEARNED. IF WE, THAT'S THE KIND OF CREATIONIST VIEW OF EDUCATION THAT WE SHOULD DO IT THE WAY THEY DID IT. YOU HAD A QUESTION. >> YES, PROFESSOR I STUDIED AT UNIVERSITY OF MISSOURI AND THEY USED A PROBLEM BASED LEARNING STRUCTURE THAT I THINK IS VERY SIMILAR TO YOURS WHERE IT'S EACH WEEK IS BASED ON A THEME AND THERE WAS SOME EVIDENCE COLLECTED. THERE WAS A BIG BUMP IN THE STEP ONE SCORES IN THE YEARS FOLLOWING THE ADOPTION OF THAT CURRICULUM. I THINK THERE IS SOME EVIDENCE THAT SHOWS THAT THIS WORKS. I WOULD ALSO LIKE TO MENTION THAT THIS 5 AT MISSOURI THIS CURRICULUM WAS CREATED NOT BY MDs BUT BY ED Ds AND PH.D.'S IN EDUCATION AND DIFFERENT BIASES HADN'T HAD A MEDICAL EDUCATION SO FREE OF DEVICES YOU HAVE SOMEHOW FORTUITOUSLY ESCAPED THAT EVERYTHING HAS TO BE TAUGHT THE WAY OF THE IN THE 20'S CENTURY. >> WELL, I APPRECIATE YOU BRINGING THAT UP. I HOPE EVERYONE HEARD. TWO POINTS IN RESPONSE. THERE ARE OTHER STORIES, RIGHT. SO WE HAVE ONE OF OUR PEOPLE THAT TEACHES WITH US CAME FROM UTMB IN GA VALUE GALVESTON THEY CAME UP WITH A GROUP OF SMALL CASES IF YOU ACCENT BEFORE OR AFTER STUDY EVIDENCE IS USEFUL, THEY HAD FOR THE SEVERAL YEARS BEFORE ACTS AND THEY HAD FOR THE SEVERAL YEARS AFTER ALMOST 1.3X IN TERMS OF THEIR BOARD SCORES SO QUITE A BIG JUMP AND IT WAS SUSTAINED. IT MAY INTEREST YOU THAT I USE SOME OF THESE SAME IDEAS IN MY PRIOR LOCATION WHEN I OVERHAULED A MEDICINE CLERKSHIP AS A CLERKSHIP DIRECTOR AT ONE POINT. WE WENT FROM FOUR HOURS IN A ROW OF SUBSPECIALTY LECTURES. CAN YOU IMAGINE BY THE SAME PERSON IN THE AFTERNOON. AND DID I LEAVE OUT THAT YOU HAD TO GO WHETHER YOU HAD BEEN ON CALL THE NIGHT BEFORE. OKAY. YOU CAN IMAGINE HOW LETHAL THAT WAS. AND OUR STUDENTS SCORED AT THAT OPEN SUBSTITUTION AINSTITUTION AS THEY ENT ERED MEDICAL SCHOOL AT THE 50% TILE -- BUT USING MANY OF THESE NOTIONS, WE OVERHAULED THE CLERKSHIP, USED MORE ACTIVE NORMS OF LEARNING, DID SOME -- FORMS OF LEARNING, DID SOME OF THE SAME IDEAS. AND FOR THE NEXT TWO YEARS, THEIR SHELF EXAM SCORES WENT TO THE 77TH PERCENTILE. I WAS CHECKING THE SELF EXAM SCORES PRIMARILY BECAUSE I WAS WORRIED THAT IT MIGHT GO DOWN. OKAY. I DIDN'T ROLE EXPECT IT T REALLY DIDN'T EX PECT IT TO GO UP BUT IT DID GO UP AND THOSE DATA ARE PROBABLY NOT GOING TO BE PUBLISHED FOR THE FOLLOWING REASON. ONE OF THE COOLEST THING WAS WE HAD NO FAILURES IN THAT CLERKSHIP FOR ACADEMIC REASONS WHEREAS IN THE PRIOR YEAR THERE WERE SEVERAL PER YEAR WHO COULDN'T PASS THE SHELF EXAM. NOBODY FLUNKED FOR ACADEMIC REASONS. AND IT TRANSFERRED THEIR STEP TWO SCORES WERE HIGHER THAN THAT SCHOOL HAD SEEN INCLUDING ONE STUDENT WHO SCORED 2 POINTS SHAI OF THE HIGHEST POSSIBLE SCORE. THAT HAPPENED BEFORE. AND AGAIN, NO FAILURES AMONG FIRST TIME TAKERS. NOBODY REALLY WANTS TO OWN UP TO THAT. MY VIEW OF THIS IS THAT THE EVIDENCE OUT THERE SHOWS WHILE INCOMPLETE AND NOT NECESSARILY RANDOMIZED EVIDENCE, SHOWS THE SAME OR BETTER OUT COME IN TERMS OF STANDARDIZED TEST MEASUREMENTS OF KNOWLEDGE. THAT IS INCOMPLETE AND I DO LOOK FORWARD TO BETTER MORE ELABORATE STUDIES THAT CONTROL MORE CONFOUNDING. IN THE MEANTIME, IF YOU USE SOMETHING THAT WOULD HELP EVEN THE WEAKEST PEOPLE IN THE CLASS SUCCEED WHERE THEY HADN'T SUCCEEDED BEFORE, WHY WOULDN'T WE DO IT? PARTICULARLY WHEN YOU'RE STARTING FROM SCRATCH AND YOU'RE GIVEN THE OPPORTUNITY, THE TABULAR RAZA. THAT MAY NOT BE PRONOUNCED CORRECTLY. IN OTHER WORDS THAT WHITE TABLET STARTING FROM SCRATCH AND NOT HAVING TO REINVENT IT. YOU HAVE A QUESTION. >> I DO. I AM A DOCTOR -- I WANT TO TALK ABOUT WHETHER THEY HAVE SUCCESSFULLY APPLIED THIS MODEL TO OTHER MEDICAL EDUCATION SUCH AS DENTISTRY, NURSING ETCETERA. >> I'M AWARE OF SEVERAL BUT NOT IN A DENY CONTEMPORANEOUSITY. BY THE WAY I SHOULD SAY THERE'S AN EVIDENCE BASED DENTISTRY IN THE UNITED KINGDOM. I'M NOT SURE ABOUT DENTAL SCHOOL EDUCATION DOING THIS THE WHOLE PIECE THAT WE HAVE HERE. BUT I THINK THERE PROBABLY ARE. I DID SOME YEARS BACK, THIS IS GOING TO BE ABOUT FOUR OR FIVE YEARS AGO NOW WHEN I WAS AT OXFORD AT THE EDM COURSE THERE MEET SOME PEOPLE FROM ONE OF THE SCHOOLS IN THE NETHE NETHERLANDS IN DENTISTRY AND THEY WERE TALKING ABOUT HOW THEY OVERHAULED THE CURRICULUM BUT I DO NOT HAVE ENOUGH DETAIL TO BE ABLE TO ANSWER YOUR QUESTION FULLY. >> THE PROBLEM [INDISCERNIBLE] INCORPORATE WITH THE MEDICAL SCHOOL [INDISCERNIBLE] YOU HAVE TO LEARN SOME TECHNICAL SKILLS AND SCWTIO EXAMINATION AND WHATNOT [INDISCERNIBLE] >> WELL, PEOPLE HAVE ASKED ME HOW I SUCCEEDED IN THE THINGS THAT I'VE DONE AND USUALLY MY ANSWER IS SOMETHING LIKE THE COMBINATION OF GOOD LUCK, OPPORTUNISM AND SUFFICIENT NAIVETY TO BELIEVE IT MIGHT STILL BE POSSIBLE DESPITE ALL THE WISER PEOPLE WHO TELL ME IT IS NOT POSSIBLE. I CAN'T HELP BUT WONDER. I WONDER WHOSE TRIED IT AND I WONDER HOW IT SUCCEEDED. I SUSPECT THERE ARE SCHOOLS OUT THERE TRYING SOME OF THESE IDEAS. I DO THINK IT'S TRUE THAT EACH PROFESSION IS LIKELY TO HAVE DIFFERENT SHAPES OF THESE IDEAS. IT'S PERHAPS THE DIFFERENCE BETWEEN A METHOD AND A PERSONAL STYLE. YOU MIGHT HAVE THE SAME METHOD BUT WITH WITH DIFFERENT SHAPES TO FIT THE PROFESSION. BUT I WONDER HOW MANY OF THESE IDEAS. THERE HAVE TO BE WHY LEARN ABOUT THE ANATOMY OF TOOTH ENAMEL. IS OTHER BECAUSE IT'S THERE OR BECAUSE YOU HAVE TO MAKE DECISIONS DIAGNOSTICALLY AND THERAPEUTICALLY ABOUT IT. TO THE EXTENT THE DECISIONS CAN HELP US ORGANIZE WHAT PEOPLE LEARN SO THAT IT PULLS, THAT THE KNOWLEDGE IS GUIDING THAT DECISION IS BROUGHT TO BEAR THEN I THINK THERE'S A GOOD CHANCE IT COULD AFFECT. I JUST DON'T KNOW ENOUGH ABOUT DENY DISTREDEN TICE TREEDENDENDENY DENTIST RY. >> WHENEVER I TALK 2350 TO PHYSICIANS ONE OF THE BIGGEST OBSTACLES THE PHYSICIANS SPEND SEVERAL YEARS IN RECOVERING -- I'M CURIOUS IF YOU HAVE ANY ANECDOTAL EXPERIENCE OR MORE THAN THAT TO COMMENT ON HOW THIS CURRICULUM WORKS BETTER AT -- >> THANK YOU FOR YOUR QUESTION. I'M NOT SURE WE HAVE ENOUGH OUT COME DATA TO TELL YOU ABOUT HOW WELL IT WORKS BETTER OR DOES IT WORK BETTER. I CAN TELL YOU WE HAVE TAKEN -- >> [INDISCERNIBLE] >> I FEEL THE SAME WAY OFTEN ABOUT STATISTICS IN EPIDEMIOLOGY LIKE PEOPLE HATE IT BEFORE I GET TO THEM. WHY. BUT YES, SO WHY, WHY DO HE THIS HATE THE KIDNEY. BUT THE THING IS A LOT OF PEOPLE DON'T UNDERSTAND THE ENORMOUS FUNCTIONAL BEAUTY OF LIKE THE PROXIMAL TUBULE CELLS. THEY'RE AMAZING CELLS. THEY DO THESE AMAZING THINGS AND THEY'RE COMPLICATED. AND BUT THEY'RE VERY FUNCTION ORIENTED, RIGHT, SO THERE'S A FORM KIND OF FUNCTION DUTY BUT IT'S THOUGHTS ALWAYS PORTRAYED THAT WAY, IS IT NOW. I WOULD SAY THAT THE EXPERIENCE OF MANY STUDENTS IS THAT RENAL PHYSIOLOGY, CARDIOVASCULAR PHYSIOLOGY AND PULMONARY PHYSIOLOGY ARE SOME OF OUR MOST DIFFICULT AREAS OF PHYSIOLOGY AND THE CORRESPONDING AREAS OF PATHOPHYSIOLOGY AND PATHOLOGY ARE ALSO QUITE DIFFICULT. BUT WE'VE DONE IT. IT REMAINS TO BE SEEN HOW WELL WE'VE DONE. I WAS ACTUALLY ASKED TO GIVE A NON-EXPERT VIEW OF MANAGING PAGES OF PATIENTS WITH CHRONIC PATIENT DISEASE, WHAT EVERY DOCTOR NEEDS TO KNOW. LET ME GO BACK TO MY EXPERIENCE AS A CLERKSHIP DIRECTOR. ONE OF THE BIG CHALLENGES I FACED AT THAT TIME WAS OVERLOAD, CURRICULUM OVERLOAD BECAUSE I HAVE THESE SPECIALISTS SAYING THAT THEY NEEDED TO KNOW EVERYTHING -- PROLIFERATIVE AND SO FORTH. MY THIRD YEAR STUDENTS COULDN'T ASSORT ACUTE KIDNEY INJURY FROM CHRONIC KIDNEY DISEASE. I WAS FACED WITH THE PROBLEM THEY WERE TRYING TO TEACH THE ENTIRE DISCIPLINE OF INTERNAL MEDICINE AND NEVADA RECAL NEPHROLOGY IN TH AT ONE FOUR HOUR SESSION. IT'S MY ONLY CHANCE THEY WOULD SAY AND I'VE GOT TO GET IT IN THERE. WELL THAT'S NOT REALLY GOING TO HAPPEN NOW IS IT. YOU CAN'T GET IT ALL IN THERE. I DON'T KNOW WHAT ALL IS BUT AFTER THE FIRST HOUR OF FOUR HOURS THEY HAVE SHUT YOU DOWN IF NOT SOONER. SO AS A CONSEQUENCE, BY TRYING TO EACH EVERYTHING, AS FAR AS I COULD TELL THEY LEARN CLOSE TO NOTHING. SO I THOUGHT WHAT IF WE BUILT THE OTHER WAY. COMPARED TO LEARNING NOTHING, WHAT IF THEY LEARNED ONE THING AND THEN A SECOND THING. AND SO FORTH. SO AFTER HOURS OF NEGOTIATION, I FINALLY POINTED OUT FOR EVERY HUNDRED STUDENTS AT THAT MEDICAL SCHOOL WHO CAME, TENS OF 15 WERE HEADED INTO INTERNAL MEDICINE. LET ME SAY THAT ANOTHER WAY. 85 TO 90 WERE NOT. WE NEEDED TO ORIENT OUR CLERKSHIP TO TEACH NOT INTERNAL MEDICINE FOR INTERNISTS BUT WHAT EVERY DOCTOR NEEDED TO KNOW ABOUT INTERNAL MEDICINE EVEN IF YOU WERE A SURGEON OR PEDIATRICIAN. AND IF WE DO THAT, IT WASN'T THAT HARD, IT WASN'T THAT LONG EVENTUALLY THEY SAID YES, YOU'RE RIGHT. ACUTE KIDNEY INJURY THAT'S IMPORTANT CHRONIC KIDNEY GAS THAT'S IMPORTANT AND MAYBE ONE OR TWO OTHER THING AND WE FOCUSED IT. AND AGAIN THE SHELF COURSE WENT UP. I THOUGHT THEY MIGHT BE DOWN BECAUSE WE WERE TEACHING FEWER FACTS. THE NUMBER OF FACTS FLOATING IN THE AIR WERE FEWER. THEY WERE A LOT MORE COHERENT AND I ACTUALLY THINK THAT INSTEAD OF THE STANDARD 5% OR LESS RETENTION AND TRANSFER THAT MOST PEOPLE GET OUT OF MOST LECTURES. MOST OF THE LEARNING SCIENCES TELL US THAT THE RETENSION AND TRANSFER FROM ACTIVE LEARNING 25 TO 50%. IT'S ACTUAL NOT A HUNDRED. BUT WHICH YEAR ARE YOU PUTTING YOUR NICKEL ON. PARTICULARLY IF IT HELPS THE PEOPLE AT THE BOTTOM OF THE CLASS. THE PEOPLE AT THE TOP, THEY CAN LEARN ANYTHING. YOU COULD PUT THEM DOWN IN THE CENTER OF ARMED CONFLICT IN KOSOVO OR SOMETHING AND THEY'RE STILL LEARNING THERE LIKE WHOA I DIDN'T KNOW ANY OF THIS. WOULD YOU LOOK AT THIS. THEY'RE GOING TO BE OKAY. WHAT I'M MUCH MORE INTERESTED IS THE WHOLE RANGE OF OUR LEARNERS MAKING SURE IT'S POSSIBLE FOR ALL OF THEM. AND WHEN WE DID THAT IN NEPHROLOGY PARTLY BECAUSE OF THIS APPROACH. IF YOU LOOK AT THE OVERVIEW MANY SCHOOLS IN THE FIRST YEAR YOU HAVE LEARNING IN THE FIRST YEAR YOU HAVE LEARNING FUNCTION AND THE SECOND YEAR LEARNING ABOUT HUMAN DISEASE AND IN THE THIRD YEAR YOU GET TO CLINICAL SCIENCES. AND WHAT WE'VE TRIED TO INTRODUCE THROUGH THE CASE BASE LEARNING AND OUR APPROACH IS THAT IN BOTH YEARS THEY'RE LEARNING PATHOPHYSIOLOGY. IN POET YEARS. BUT IN THIS YEAR YOU'RE LEARNING THE NORMAL BY COMPARING AND CONTRASTING IT WITH THE ABNORMAL. IN THE WEEK THEY'RE LEARNING CARDIAC OUTPUT. THEY HAVE A PATIENT WITH HEART FAILURE BUT IT'S NOT PRIMARILY TO LEARN THE DISEASE THAT CAUSES THE HEART FAILURE, PRIMARILY TO LEARN THE SYNDROME. AND THEY LEARN IN THE FIRST YEAR WHAT THE -- SYSTEM NORMALLY DOES BY COMPARING OR CONTRASTING THE NORMAL TO AN ABNORMAL. AND WHY WHEN A PATIENT WITH HEART FAILURE LIES DOWN AT NIGHT DO THEIR LUNGS FILL WITH FLUID AND THEY HAVE TROUBLE BREATHING, WHY DOESN'T THAT HAPPEN TO EVERYONE IN THIS ROOM. HOW DO WE STAY OUT OF HARD FAILURE IS THE KIND OF, AND WHAT IS THE NORMAL. BUT AGAIN BY COMPARING AND CONTRASTING IT WITH THE ABNORMAL. IN THE SECOND YEAR WE REMIND THEM OF THE PATHOPHYSIOLOGY AND THEN FOCUS ON THE DISEASES THAT CAUSE IT AND THE VARIATION. IN THE FIRST YEAR THE HEART FAILURE PATIENT HAS A PROBLEM WITH MUSCLE PUMPING AND THE SECOND YEAR WE INTRODUCE VALVULAR DISEASE WITH THE PRESSURE OR VOLUME LOADS THAT CAUSE IT AND SO FORTH SO WE CONNECT THEM. INTERSTITIAL LUNG DISEASE WE JUST RECENTLY DID THIS. THERE'S ALMOST LIKE I'VE GOT YOUR NUMBER ON THE INTERSTITIAL LUNG DISEASE. THEY CAN'T DO IT JUST BY MEMORY ALONE BUT WE HAD A WEEK THAT WAS ORGANIZED AROUND IT. 10 OR 11 PATH PATHY PHASE LOGIC AROUND IT -- BY ORGANIZING AND CONNECTING THROUGH THE COHERENT NARRATIVE PATHOPHYSIOLOGY WHICH INCLUDES AGAIN FROM SOCIETY. IN EFFECT THEY SPENT TWO YEARS STUDYING PATHOPHYSIOLOGY CONNECTING ACROSS ALL THESE SCIENCES. AND WHY NOT INCLUDE WHEN YOU SAY OKAY, WELL, YOU KNOW THAT HEART FAILURE IS ASSOCIATED WITH EXCESS ACTIVATION OF ANGIOTENSIN WHEN WE TALK ABOUT ITS TREATMENT. THIS IS WHERE THEY WORK. THEY INTERFERE WITH THIS ACTIVATION OR SOMETHING LIKE THAT. AND THEN IT'S NOT TOO HARD TO SAY WELL HOW WELL DO THOSE TREATMENTS WORK. SEE HOW THOSE PIECES DO FIT TOGETHER. REMAINS TO BE SEEN, THOUGH. I SUSPECT LIKE MANY THINGS, IT WILL BE THE ROUGHLY THE SAME OR BETTER. BUT I SHARE YOUR PAIN ABOUT NEPHROLOGY. I'M NOT A NEVAD NEPHROLOGIST BUT BECAUSE I'M THE CLEAN UP TEACHER, SO I ENDED UP TEACHING A FAIR AMOUNT OF THE KIDNEY PHYSIOLOGY. I'M SURE THERE WERE OTHER QUESTIONS AND THEN I IGNORED THEM SO I APOLOGIZE. YES, MA'AM. >> I JUST WANT TO KNOW, ARE YOU DOING ANYTHING TO [INDISCERNIBLE] >> I'M SORRY THAT MEDICINE LOST YOU BUT PERHAPS PSYCHOLOGY HAS THE BETTER TO WHATEVER YOUR FIELD NOW IS. LET ME SAY THAT I THINK THE TECTONIC PLATES ARE MOVING IN THAT DIRECTION EVEN IF I DON'T DO IT BUT I'M TRYING TO PLAY A ROLE IN MY LOCAL SITUATION. FIRST THE TECTONIC PLATES. HAVE YOU HEARD ALREADY ABOUT DEVELOPMENTAL, THE DREYFUS AND DREYFUS MODEL DEVELOPMENT MODEL OF CONFIDENCE. HAS PEOPLE ALREADY TALKED YOU THROUGH THAT? I'LL TELL YOU WHAT. TWO MINUTES ON THAT. THE IDEA IS PROFESSIONALS DON'T COME OUT OF THE CAN FULLY FORMED, RIGHT. YOU AREN'T BORN AN EXCELLENT TEACHER OR EXCELLENT DOCTOR OR EXCELLENT SCIENTIST. YOU HAVE TO GROW TO BE THAT WAY. THAT GROWTH IS NOT NECESSARILY PRETTY DETERMINED SIMPLY BY GOING TO SCHOOL. THAT GROWTH HAS TO DO WITH THE DEVELOPMENT OF PARTICULAR COMPETENCIES. AND IF YOU TAKE A DEVELOPMENTAL STANCE IE YOU START SOMEWHERE A KNOW INOVICE, YOU BECOME EXPERTS, WHAT ARE THE MILE STONS YOU BEGI STONES YOU BEG IN TO RECOGNIZE AS THEY MOVE TOWARD COMPETENCE AND WHAT ARE THE EXPERIENCES IN COACHING AND FEEDBACK THAT YOU HAVE TO GET TO REACH THAT STAGE. IF YOU TAKE THAT GENERAL IDEA THAT IS WHAT A LOT OF PEOPLE MEAN THE PLUS SIDE OF THAT IS WHAT PEOPLE CALL COMPETENCY BASED EDUCATION. NOW THIS HAS BEEN KNOWN FOR A WHILE. THIS IS NOT NECESSARILY NEW IDEA. IN FACT, THE ACG OF CREDITING BODY THAT ACCREDITS GRADUALING EDUCATION SOME YEARS BACK INSISTED THE PRESIDENCY, AND YOU MAY HAVE TO LIVE WITH SOME OF THIS IN YOUR WORK, RESIDENCELY ADOPTS COURT COMPETENCY MODEL OF ACCEPTANCE OF READINESS TO BE CERTIFIED. THEY IDENTIFIED SIX CORE COMPETENCIES. MEDICAL KNOWLEDGE, SURE THAT SOUNDS RIGHT. PATIENT CARE SURE, WE'RE FOR THAT. COMMUNICATION AND INTERPERSONNEL SKILLS, OKAY, YES THAT'S GOOD. PROFESSIONALISM, WE'RE ALL FOR THAT. TWO THAT ARE A LITTLE LESS CLEAR. PRACTICE BASE LEARNING AND IMPROVEMENT AND SYSTEMS BASE PRACTICE. FLUSH BUT THEN WHEWHEN YOU LOOK UNDER THE HOOD PRACTICE BASE LEARNING AND IM3R506789S WHAT YOU AND I MAY CALL SYSTEMS BASE PRACTICE WHICH HAS A LOT TO DO WITH WORKING DELIBERATELY EFFECTIVELY WITH TEAMS TO PROVIDE HEALTH CARE. IF YOU START LOOKING AT THE CANADIAN SYSTEM, THEY HAVE A DIFFERENT SET OF COMPETENT SEES BUT IN THE LAST DECADE OR TWO NOW AS AT THE GRADUATE LEVEL THEY HAVE MOVED IN THE DIRECTION OF COMPETENCY BASED EDUCATION. WELL AS YOU CAN EMERGENCY. THE FORCE FIELD OF RESIDENCY GOT THE ATTENTION OF A LOT OF MEDICAL SCHOOLS AND SURPRISE SURPRISE. MEDICAL SCHOOLS ARE NOW GOING IN THE DIRECTION OF COMPETENCY BASED EDUCATION AND I MAY NOT SURPRISE YOU THAT THEY'RE ROUGHLY THE SAME SIX. A YEAR OR TWO BACK THE HOWARD HUGHES MEDICAL INSTITUTE AND. >> IS THERE SUCH A THING FOR COMPETENCY BASED EDUCATION FOR UNDERGRADUATE AND THEY RELEASED A REPORT. TWO PARTS OF IT. ONE IS THE SCIENCE PREPARATION AND ANOTHER IS THE HUMANISTIC THAT HAS JUST RECENTLY BEEN PUBLISHED. THOSE ARE THE TECTONIC FORCES THAT WILL EVENTUALLY GET DONE WHAT YOU'RE TALKING ABOUT. WHAT WE'RE DOING ON A LOCAL LEAVE AS WE'RE DOING THIS ON A COMPETENCY BASED EDUCATION WAY WE'RE ALSO TALKING WITH THE PEOPLE RIGHT ACROSS THE STREET IN THE COLLEGE OF ED AND DOWN THE ROAD IN THE BIO MEDICAL HEALTH SCIENCES INSTITUTE TO SEE IF THEY'D LIKE TO ALIGN THEIR EDUCATION IN A WAY THAT PREPARES UGA WITH GRADUATING SENIORS FOR SUCCESS IN OUR MODEL. I THINK THEY'RE WAITING TO SEE IF OUR MODEL SUCCEEDS. BUT ASSUMING IT DOES, ALONG THE LINES OF WHAT WE WERE TALKING ABOUT BEFORE ASSUMING IT DOES THEN I THINK THEY MIGHT BE WILLING TO HAVE AGO AT IT BECAUSE THEY WERE QUITE INTERESTED. NOW REMEMBER THIS IS THREATENING TO A LOT OF TRADITIONAL DEPARTMENTS BECAUSE I'M IN A DEPARTMENT OF BIOCHEMISTRY OR PHYSIOLOGY OR ANATOMY. CELL BIOLOGY. WAIT A MINUTE. WAIT A MINUTE. YOU'RE GOING TO HAVE A KEERS AND IT'S NOT GOING TO BE CALLED BIOCHEMISTRY OR CELL BIOLOGY AND I HAVE TO TEACH THIS BUT I WON'T GET CREDIT. WE HAVE TO FIND A WAY TO SOLVE THAT AND THAT SOUNDS PLAUSIBLE BUT THEY WON'T GIVE IT UP FOR NOTHING. I THINK WHAT THEY NEED TO GIVE IT UP FOR IS MORE INTEGRATIVE UNDERGRADUATE PREMED PREPARATION. THERE'S A LOTS OF PERSUASION LEFT AHEAD. HARD BARGAINING I'M TOLD IS THE WAY THEY SAY IT HERE IN WASHINGTON WHEN THEY TALK ABOUT THESE INTERNATIONAL COMPLAB RATIONS AND SO FORTH. HARD BARGAINING LIES AHEAD. WHAT OTHER QUESTIONS DO YOU HAVE? YES. >> MY QUESTION IS PART OF THE LAST QUESTION. HOW CAN WE APPLY THESE PRINCIPLES TO CONTINUING MEDICAL EDUCATION. >> THANK YOU FOR ASKING. LET'S NOT FORGET THIS LINE BACK HERE. MOST OF WHAT IS DONE IN THE NAME OF CONTINUING EDUCATION AS YOU KNOW IS A REPEAT OF PRECLINICAL MEDICAL EDUCATION. HOUR AFTER HOUR OF COLLECT SURE. THE RANDOMIZE TRIALS SHOW THAT THERE'S NO CHANGE IN CLINICIANS KNOWLEDGE OR THEIR DECISION-MAKING AFTER, ON THE BASIS OF THE STANDARD BIOLECTURE. IN FACT IT'S A VERY CONVERSATIONISCREATIONISTVIEW YOU LEARN AN ATOMY ONCE AND AFTER THAT YOU DON'T HAVE TO LEARN ANYTHING NEW OR UNLEARN. STUDIES THAT HAVE SHOWN EFFECTIVE THINGS HAVE SHOWN THAT THERE ARE PEOPLE THAT HAVE FIGURED OUT WAYS TO HELP PEOPLE UNLEARN SOME OLD IDEAS AND LEARN NEW IDEAS THROUGH MORE ACTIVE LEARNING. WHETHER IT'S WORK SHOPS A THING CALLED ACADEMIC DETAILING, SO INSTEAD OF SOMEONE SELLING YOU A DRUG PRODUCT SOMEONE COME OUT AND SELLS YOU A NEW IDEA BY SHOWING YOU HOW TO MAKE YOU A BETTER FASTER DOCTOR OR EARTH MECHANISMS TO EN-- OR OTHER MECHANISMS TO ENGAGE PEOPLE IN SKILL BUILDING. THERE ARE EFFECTIVE CME. IN FACT THERE'S A MOVEMENT AMONG CME CALLED EVIDENCE BASE CONSUMING EDUCATION IE LET'S USE STUDIES OF EFFECTIVENESS OF THEY METHODS TO 2K3W50EU GUIDE OUR EFFORTS. THERE'S ANOTHER MOVEMENT UNDER WAY OF PEOPLE TRYING TO INFORM PROCESSES OF CONTINUING EDUCATION. WHAT LAGS BEHIND? STATE APPROVAL, THE SORT OF OCCASION. THERE'S A LOT OF ADMINISTRATIVE PROCESSES THAT REGULATE CME THAT AREN'T YET THERE. WE HAVE ANOTHER HUGE BARRIER AND THAT ONE OF THE MAJOR FORMS OF FINANCIAL SUPPORT FOR CONTINUING EDUCATION IS INDUSTRY. I'M NOT HERE TO START A RUCKUS BUT THERE'S LOTS OF PROBLEMS WITH THAT SUPPORT. AND ONE OF THE CHALLENGES IS WHO IS GOING TO PAY FOR CONTINUING EDUCATION, OKAY. IF YOU GO BACK TO THAT PART WATER MALG, IANALOGY, IF WE AREN'T PAID FOR CURRENT AND BEST KNOWLEDGE, IF WE GO AND IT'S 30 YEAR OLD WATER FILLED WITH GOO AND WE USE THAT FOR OUR LIVING AND EATING WE GET SICK SAME THING IF OUR KNOWLEDGE IS THAT CURRENT BEST KNOWLEDGE WE ALL IN SOME WAY PAY FOR IT, EVENTUALLY IT KIND OF COMES DOWN TO HOW ARE WE GOING TO CHAIR THE EXPENSE OF CONTINUING PROFESSIONAL DEVELOPMENT FOR KNOWLEDGE WORKERS IN HEALTHCARE. I DON'T KNOW THAT WE HAVE THAT SORTED OUT. BUT MEANWHILE WE'RE TRYING TO HELP OUR STUDENTS BUILD THE SKILLS BECAUSE ONE OF THE ESSENTIAL SKILLS IS BEING ABLE TO RECOGNIZE WHEN YOU DON'T KNOW SOMETHING TURN THAT HOLE IN YOUR HEAD INSTEAD OF HIDING IT. WE'RE SCHOOLED TO HIDE WHEN WE DON'T KNOW SOMETHING, RIGHT? YOU'RE NOT SURE THIS IS TRUE, I CAN TELL BY THE LOOK ON YOUR FACE, IT'S ALL RIGHT. ASK YOURSELF WHEN A TEACHER ASKS YOU A QUESTION, WHAT DID YOU DO IF YOU DON'T KNOW THE ANSWER? YOU RAISE YOUR HAND. YOU RAISE YOUR HAND AND SAY I DON'T KNOW. WELL BLESS YOU BUT MOST PEOPLE DO NOT DO THAT. MOST PEOPLE LOOK DOWN OR THEY LOOK AWAY TRY TO BECOME INVISIBLE. VERY FEW PEOPLE GO OH, OH, I DON'T KNOW THIS BUT BOY AM I READY TO LEARN TODAY. [LAUGHTER] IT'S LAUGHABLE BECAUSE WE LEARN FROM AS FAR DOWN AS KINDERGARTENNEN TO HIDKINDERGARTENTO HIDE WHEN WE DON'T KNOW. THIS IS ADAPTIVE SITUATIONS WHERE KNOWING ALREADY WHAT'S VALUED AND NOT KNOWING ALREADY IS TAKEN AS A SIGN OF WEAKNESS. BUT IT'S MAL ADAPTIVE IN HEALTHCARE WORK BECAUSE IF YOU DON'T KNOW AND YOU HIDE THAT YOU DON'T KNOW, WHO PAYS THAT PRICE? YOUR PATIENTS OR YOUR POPULATION. WE CANNOT AFFORD THAT. SO WHAT I WANT TO KNOW IS WHAT ARE WE GOING TO DO. I THINK WE'RE GOING TO MAYBE MAKE EVIDENCE BASED PRACTICE POSSIBLE. OH YEAH THERE'S SOME PEOPLE WORKING ON THAT BECAUSE WHEN I STARTED PEOPLE WOULD LAUGH AT ME BECAUSE I THOUGHT WE SHOULD LOOK UP STUFF TO KNOW WHAT TO DO FOR OUR PATIENTS. WE'RE GOING TO TRY TO GET IT INTO THE CURRICULA OF MEDICAL SCHOOLS. OH YEAH THERE ARE PEOPLE WORKING ON THAT. WE'RE GOING TO SPREAD IT TO ANY FIELD WHO WANTS TO. YEAH, THERE ARE PEOPLE WORKING ON THAT. WE'RE GOING TO OVERCOME THE BARRIERS TIGHT. YES, THERE ARE PEOPLE WORKING ON THAT. BUT EACH IN OUR OWN WAY WE CAN TAKE UP THIS IDEA IF WE'RE GOING TO MAKE WELL INFORMED DECISIONS ABOUT WHATEVER MY WORK IS AND WE'RE GOING TO TEACH OUR YOUNG YOUNGLINKS TO MAKE WELL INFORMED DECISIONS AFTER YOU'RE GONE HOW IS THAT GOING TO HAPPEN. EACH OF US COULD MAKE IT HAPPEN IN OUR OWN ENVIRONMENT BUT IT DOES TAKE A FAIR AMOUNT OF WILLINGNESS AS GARRISON KEILLOR SAYS TO STAND UP AND SAY WHAT NEEDS TO BE SAID EVEN IF YOU'RE SHAI. W>> WHAT ROLE DO ELECTRONIC HEALTH RECORDS PLAY INTO YOUR CURRICULUM. >> INTO OUR CURRICULUM OR INTO THE FUTURE. I LOOK FORWARD TO THE FUTURE DAY WHEN IT HELPS US. RIGHT NOW IF YOU'RE IN A CLINICAL ENVIRONMENT, IF MOST SPAITIONSITUATIONS AND MOST ELECTRONIC HEALTH RECORDS IN YOU WANT TO LOOK UP SOMETHING YOU HAVE TO SIGN OUT OF THAT RECORD BECAUSE OF ALL THE HIP PA AND ALL THE PATIENT'S CONFIDENTIALITY REQUIREMENTS THAT YOU CAN'T HAVE IT OPEN WHEN YOU HAVE OTHER THINGS OPEN OR YOU HAVE TO GO SOMEWHERE ELSE OR USE SOME OTHER DEVICE TO GET KNOWLEDGE. THIS SEEMS TO ME FIXABLE. I'M NOT THE PROGRAMMER THAT'S GOING TO MAKE THIS HAPPEN. I DON'T PROGRAM AT ALL BUT SOMEONE SHOULD BE ABLE TO FIGURE OUT HOW TO HAVE KNOWLEDGE RESOURCES WOVEN RIGHT INTO THE PATIENT CARE FLOW RESOURCES. SO WHEN YOU NEED HELP WORKING WITH SOMEONE WITH -- OR WHEN YOU WANT TO CHOOSE TO USE A RESOURCE ABOUT THE ADVERSE IF HE EQUALS OF A MEDICINE YOU DON'T HAVE TO LEAVE WHAT YOU'RE DOING AND GO SOMEWHERE ELSE. AGAIN THAT WATER ANALOGY RIGHT NOW WHEN YOU WANTED TO GO GET INFORMATION, IT'S THE EQUIVALENT OF YOU'VE GOT TO STOP WHAT YOU'RE DOING AND GO DOWN OVER HERE TO THE RIVER BANK AND TAKE YOUR BUCKET AND THEN GO BOIL IT YOURSELF TO GET RID OF THE IMPURITIES. AND ADD THE IODINE AND WAIT A FEW DAYS AND THEN YOU HAVE KNOWLEDGE THAT'S FOR YOUR BIAS. WHYMENT IWHY ISN'T IT LIKE THE TAP. WHY DON'T THE ELECTRONIC HEALTH RECORDS FIND A WAY TO GET KNOWLEDGE RESOURCES IN, IN FOCUSED AMOUNTS IN JUST THE RIGHT LEVEL THAT REPRESENT CURRENT BEST KNOWLEDGE. AND NOT JUST THE GUIDE LIES THAT THEY WANT YOU -- GUIDELINES THEY WANT YOU TO FOLLOW BUT CURRENT BEST KNOWLEDGE THAT ALLOWS YOU TO INDIVIDUALIZE CARE AND PERSONALIZE IT. IT'S GOING TO HAPPEN. SOMEBODY NEEDS TO FIGURE THIS OUT. THERE WAS ANOTHER HAND UP BUT I'VE MOVED AROUND SO I'VE LOST TRACK OF WHERE IT WAS. OKAY. WELL I TELL YOU WHAT, LET'S SKIP AHEAD TO THE HELPED. HERE'S THEM IMITATING ME. I TAKE IT AS A COMPLEMENT. THEY DID THIS TO ENTERTAIN THEMSELVES BUT IT ALSO ENTERTAINS ME. I'VE ALREADY ASKED YOU FOR QUESTIONS. THIS IS ANOTHER SHOT OF OUR BUILDING. I WANT TO THANK YOU. [APPLAUSE] >> I'M GOING TO CLOSE THIS OUT. DR. RICHARDSON YOU DID EXACTLY WHAT WE WANTED YOU TO DO. I THINK TO A PERSON WE ENJOYED THE PRESENTATION AND THERE'S A GOOD CHANCE THAT WE'LL REMEMBER SOME OF WHAT YOU'VE SAID. I WANT TO THANK ALL OF YOU FOR PARTICIPATING NIAAA AND NIC'S DIVISION OF CANCER PREVENTION AND THOSE WHO SPONSORED THIS ACTIVITY. WE LOOK FORWARD TO YOU AGAIN AND DR. RICHARDSON SAFE TRAVEL BACK TO GEORGIA. >> THANK YOU.