>> OKAY. HELLO, EVERYBODY. UH, IT'S MY GREAT PLEASURE TO INTRODUCE MICHALE FEE TO YOU TODAY. HE STARTED HIS CAREER IN PHYSICS, AND IN FACT DID HIS Ph.D. WITH STEVEN CHOO WHOM YOU MAY KNOW AS THE ENERGY SECRETARY OF THE U.S. 4E DID A POST DOC IN BELATE WHERE HE MINGLED WITH PEOPLE LIKE DAVID TANK AND BEN KLINEFELT INCLUDING WINFRED DING AND THEN BECAME INTERESTED IN NEUROBIOLOGY. OVER THE YEARS, HE'S BEGUN TO FOCUS ON THE NEUROBIOLOGY OF SONG GENERATION IN SONGBIRDS AND HAS SOME VERY, A VERY INTERESTING STORY THAT IS VERY DIFFERENT FROM THE SORT OF, UH, GENERATION OF WRITTEN MOTOR BEHAVIOR AND OTHER SYSTEMS. HE'S GOING TO ARGUE AND PROVIDE EVIDENCE THAT THERE'S SEQUENTIAL ACTIVATION OF INDIVIDUAL ELEMENTS RATHER THAN A RHYTHMIC BEHAVIOR. TODAY HE'S GOING TO TELL US ABOUT NEW WORK LOOKING AT THE EQUIVALENT OF THE BIRD'S BASAL GANGLIA AND HE'S -- THE TITLE OF THE TALK IS A NEW MODEL OF BASAL GANGLIA INSPIRED BY THE SONGBIRD. MICHALE. >> SO IT'S A REAL PLEASURE TO BE HERE. THIS IS MY FIRST VISIT TO GIVE A¨ TALK AT THE NIH AND IT'S MY HONOR TO BE HERE. THANK YOU FOR THE INVITATION. SO, UM, SO MY LAB IS WORKING ON TRYING TO UNDERSTAND HOW THE BRAIN GENERATES AND LEARNS COMPLEX SEQUENTIAL BEHAVIORS. MOST OF THE BEHAVIORS THAT WE THINK OF IS UNIQUELY HUMAN SUCH AS mySPEECH, LANGUAGE, PLAYING MUSIC, HITTING A TENNIS SERVE, THESE ARE ALL COMPLEX SEQUENTIAL BEHAVIORS THAT ARE LEARNED BY AN ANIMAL BY LOTS OF PRACTICE. THE SURTH RI IN THE SONGBIRD IS A FANTASTIC MODEL SYSTEM FOR¨w3 UNDERSTANDI UNDERSTANDING THESE BEHAVIORS ARE GENERATED IN THE VERTEBRATE BRAIN. THERE ARE LOTS OF HOMOLOGOUSES BETWEEN BRAIN CIRCUITS IN THE BIRD ANDxD THOSE IN MAMMALS. AND I WILL TRY TO CONVINCE YOU THAT WE'VE MADE PROGRESS IN UNDERSTANDING THE BASIC FUNCTION OF THESE BRAIN CIRCUITS. I'M GOING TO START BY INTRODUCING THE SONGBIRD AS A MODEL SYSTEM FOR UNDERSTANDING HOW THE BRAIN GENERATES AND LEARNS COMPLEX SEQUENTIAL¨q BEHAVIORS. I'LL DESCRIBE SOME OF THE WORK THAT WE'VE DONE IN OUR LAB AND OTHER LABS IN TERMS OF UNDERSTANDING HOW THE SEQUENTIAL BEHAVIOR OF THE SONG]I IS GENERATED AND SOME MORE RECENT WORK ON UNDERSTANDING THE ROLE OF THE BASAL GANGLIA IN VOCAL LEARNING, AND THEN I'M GOING TO, UH -- ONE OF THE KEY FINDINGS THAT HAS COME OUT OF THIS WORK IS THE NOTION THAT, THAT ONE OF THE IMPORTANT SIGNALS GOING TO THE BASAL GANGLIA DURING LEARNING IS A [INDISCERNIBLE] COPY OF MOTOR ACTIONS. AFTER DESCRIBING TO YOU THAT, THOSE RESULTS IN THAT MODEL, I'M GOING¨ TO GO WAY OUT ON A LIMB HERE, ESPECIALLY RIGHT HERE IN FRONT OF OKAHIDA, AND I'M GOING TO TRY TO ARGUE THAT THIS NOTION THAT THE BASAL GANGLIA RECEIVES AND E FRENS COPY OF MOTOR ACTIONS MIGHT BE IMPORTANT FOR MOTOR LEARNING IN THE OCULAR MOTO$o¨ SYSTEM. I'M REALLY CURIOUS TO HEAR WHAT OKAHIDA HAS TO SAY ABOUT THESE IDEAS. SO LET'S START WITH THE SONGBIRD. THE BIRD THAT WE STUDY IS THE ZEBRA FINCH, SORT OF THE LAB RAT OF THE SONGBIRD FIELD. THEY BREED WELL IN CAPTIVITY, THEY SING A LOT, AND THEY LEARN THEIR SONGS FROM THEIR PARENTS, AND OVER THE COURSE OF A COUPLE OF MONTHS OF LEARNING. THE SONG OF THE ADULT ZEBRA FINCH LOOKS LIKE THIS, HAS A VERY CHARACTERISTIC STRUCTURE. THIS IS A SONG SPECK TROE GRAM THIS IS FREQUENCY VERSUS TIME. BRIGHT COLORS INDICATE A LOT OF POWER OR A LOT OF FREQUENCY AT THAT TIME. YOU CAN SEE THE SONG HAS A MOTIF THAT REPEATS OVER AND OVER AGAIN. THIS IS A SONG MOTIF HERE, HERE, AND HERE. MOTIFS LAST ABOUT A SECOND IN AND THEIR COMPOSED OF THREE TO SEVEN SMALLER UNITS AND THEN WITHIN SYLLABLE ARE FAST FLUCTUATIONS IN THE ACUE TICK STRUCTURE. THIS IS AN EXAMPLE OF A ZEBRA FINCH¨ SONG. YOU CAN HEAR THAT MOTIF STRUCTURE THAT REPEATS OVER AND OVER AGAIN. OKAY. [BIRDS SINGING]. THIS IS MY FAVORITE ONE. OKAY. SORRY. FLEW AWAY. OKAY. SO, NOW, ONE OF THE REASONS WHY THE SONGBIRD IS SUCH AN INTERESTING MODEL SYSTEM IS BECAUSE NOT ONLY CAN WE UNDERSTAND HOW THE BRAIN GENERATES COMPLEX SEQUENTIAL BEHAVIORS LIKE THAT, BUT WE CAN ALSO UNDERSTAND HOW THESE BEHAVIORS ARE LEARNED. SONGBIRDS LARN THEY VOCALIZATIONS BY IMITATING THEIR PARENTS. THEY DO THIS THROUGH A VERY CHARACTERISTIC SERIES OF STAGES. A FEW WEEKS AFTER THEY'RE HATCHED, THEY BEGIN LISTENING AND THEY BEGIN LISTENING TO THEIR PARENTS, AND MOST SONGBIRD SPECIES IT'S [INDISCERNIBLE] BEHAVIOR, SO IT'S THE MALES THAT SING. THEY LISTEN TO THEIR FATHER SING DURING THIS PERIOD OF TIME AND THEY STORE A MEMORY OR TEMPLATE IN THE AUD AUDITORY SYSTEM SOMEHOW OF THIS SONG VOCALIZATION THAT THEY WILL THEN TRY TO IMITATE. AROUND THIS TIME THANK BEGIN TO BABBLE, AND BY LISTENING TO THEMSELVES BABBLING, THEY SOMEHOW COMPARE THEIR VOCALIZATIONS TO THAT TEMPLATE AND GRADUALLY CONVERGE, MODIFYING THEIR VOCALIZATIONS, CONVERGE ON TO THEIR ADULT SONG. THIS PRODUCES A FAIRLY CHARACTERISTIC PATTERN OF VOCAL LIE SKAIGSS DURING THE LEARNING PROCESS. THE VERY EARLIEST VOCALIZATIONS ARE CALLED SUB SONG AND THEY SOUND LIKE THIS -- SO THIS IS KIN TO HUMAN BABBLING [BIRD CHIRPS] AROUND THIS TIME, THE BIRD HEARD THE TOOTER SONG. SO THESE ARE RECORDINGS THAT WERE DONE BY [INDISCERNIBLE] IN HIS LAB AT CITY COLLEGE IN NEW YORK. SO THE BIRD HEARS THIS TOOTER SONG, BEGINS TO BABBLE. [BIRD BABBLING]. OVER THE COURSE OF A FEW WEEKS, H HEsPRACTICES THIS SONG THOUSANDS OF TIMES A DAY AND EVENTUALLY YOU CAN START HEARING SOME STEREO TYPICAL STRUCTURE APPEARING IN THEb [BIRD BABBLING]. A FEW WEEKS LATER THAT AVAILABILITY GRADUALLY DECREASES UNTIL HE HAS WHAT CAN BE A VERY GOOD COPY OF THE TOOTER SONG. [BIRD SINGING] PRETTY GOOD. SO THERE ARE SEVERAL THINGS THAT HAPPEN DURING THIS LEARNING PROCESS. THERE'S A GRADUAL DECREASE IN THE VARIABILITY OF THE VOCALIZATION THAT THE BIRD LEARNS, AND A GRADUAL INCREASE IN SIMILARITY OF THE SONG TO THE TOOTER. HOW DO WE THINK ABOUT THIS LEARNING PROCESS? THE BASIC FRAMEWORK THAT MY LAB AND MANY LABS IN THE FIELD THINK ABOUT HOW TO BIRD DOES THIS IS THE NOTION OF REINFORCEMENT LEARNING. IDEA THAT THE BIRD LEARNED ITS SONG BY REINFORCEMENT LEARNING WAS FIRST INTRODUCED BY DOYA AND SINOUSKI. THE NOTION IS THAT YOU HAVE A SONG MOTOR SYSTEM THAT PRODUCES A VOCALIZATION. THE IDEA IS THAT THERE IS SOME SOURCE OF VARIABILITY THAT DRIVES EXPLORATION, MOTOR EXPLORATION, IN THE MOTOR PATHWAY, AND THAT PRODUCES A VARIABLE SONG. THE IDEA IS THAT THE BIRD THEN COMPARED WHAT HE HEARS HIMSELF SING TO THIS AUDITORY MEMORY, AND PRODUCES WHAT'S CALLED AN ERROR OR REINFORCEMENT SIGNAL THAT THEN FEEDS BACK INTO THE MOTOR PATHWAY THE TO MODIFY THAT MOTOR PROGRAM. THAT'S THE BASIC FRAMEWORK. THERE ARE OTHER WAYS THATok PEOPLE THINK ABOUT THIS, BUT THIS IS THE APPROACH THAT WE HAVE KIND OF PARADIGM THAT WE'VE ADOPTED. NOW, BEFORE I GO INTO THE DESCRIPTION OF THE CIRCUIT ELEMENTS THAT IMPLEMENT THIS OR THAT WE HYPOTHESIZE IMPLEMENT THE VARIOUS PARTS OFi] THIS, UM, THIS LEARNING PROGRAM, LET ME JUST GIVE YOU SOME BACKGROUND INFORMATION ABOUT THE MOTOR PATHWAY AND HOW WE THINK THE MOTOR PATHWAY PRODUCES THIS VOCALIZATION. SO THE BRAIN AREAS IN THE BIRD THAT ARE INVOLVED IN PRODUCING THE SONG HAVE BEEN KNOWN FOR A NUMBER OF YEARS, LARGELY FROM THE WORK OF NOTOBOM'S LAB. THE VOCAL ORGAN DOWN HERE HAS ABOUT SEVEN MUSCLES ON EACH SIDE THAT CONTROL THE SOUND THAT THE VOCAL ORGAN PRODUCES. THOSE MUSCLES ARE INTERVAT BID A COUPLE OF THOUSANDS MOTOR NEURONS IN THE STEM. W FROM A CHAIN THOSE RECEIVE INPUTxD OF FOUR BRAIN NEWICALLY YEAH; RA AND HBC. RA AND HBC ARE BOTH IN THE PART OF THE BIRD'S BRAIN CALL THE PALIUM THAT IS APPROXIMATELY ANALOGOUS TO CORTEX IN MAMMALS. THERE'S A LOT OF DEBATE OVER EXACTLY HOW AVIAN PALI MUSHGS AND MAMMALIAN CORTEX ARE RELATED TO EACH OTHER, BUT FOR NOW, THE WAY THAT MANY PEOPLE THINK ABOUT THIS AND THE IDEA THAT WAS PUT FORWARD IS THAT RA IS LIKE LAYER FIVE OF NEOCORTEX. IT HAS THIS PART OF THE PALIUM BACK HERE HAS ALL OF THE DESCENDING PROJECTIONS THAT GO TO THE BRAIN STEM AND HBC MIGHT PERHAPS BE LIKE LAYER TWO OR THREE. THERE ARE ALSO PROJECTIONS TO HBC FROM THE THALAMUS. [LOW AUDIO] IS INVOLVED AND IS NECESSARY FOR SINGING AS WELL. IF YOU LESION ANY OF THESE AREAS T BIRD CAN'T SING ANYMORE. WE SET OUT TO TRY TO UNDERSTAND HOW THIS CIRCUIT ACTUALLY WORKS. OUR LAB AND SEVERAL OTHER LABS HAVE RECORDED FROM NEURONS IN THESE BRAIN AREAS TRO DEVELOP HYPOTHESESES ABOUT HOW THIS WORKS. IF YOU RECORD FROM NEURONS IN HCC THAT PROJECT TO THIS NUCLEUS RA AND IF YOU RECORD THEM USING ANTADROMIC -- HERE'S WHAT YOU FIND. IT'S DIFFICULT TO SEE BUT THIS SHOWS THREE SUCCESS SI REPETITIONS OF THE MOTIF OF THIS BIRD AND ONCE DURING EACH REPETITION THIS NEURON GENERATES A SINGLE BURST OF SPIKES THAT SHOWS THE RAW VOLTAGE TRAITS. THE BURSTS LAST ABOUT SIX MILLISECONDS LONG AND THE FIRING RATE IS AROUND [LOW AUDIO]. THESE ARE ALL LINED UP USING THE SONG AS A TIMING MARKER. THEY'RE RECORDEDED SEQUENTIALLY, ONE¨ AFTER THE OTHER, NOT IN THIS ORDER. IF YOU SEE THAT EACH ONE OF THESE NEURONS GENERATES A SINGLE BURST [LOW AUDIO]. EACH NEURON WE RECORDED IN THE BIRD BURST AT A DIFFERENT TIME IN A SONG. THERE'S NO OBVIOUS RELATIONSHIP BETWEEN THE TIME AT WHICH THESE NEURONS BURST AND THE STRUCTURE OF THE SONG. THEY APPEAR TO BE ALMOST RANDOMLY SCATTERED THROUGHOUT THE SONG MOTIF. ANYQUESTIO QUESTIONS? [LOW AUDIO]. PLEASE INTERRUPT IF YOU HAVE ANY QUESTIONS. I'M NOT SHOW YOU A LOT OF DATA, BUT LET ME JUMP RIGHT TO THE HYPOTHESIS THAT THESE RECORDINGS IN THESE SONG CONTROL AREAS LED US TO ABOUT HOW SONG PRODUCTION WORKS IN THE BIRD. AT EVERY MOMENT IN TIME YOU HAVE IS A POPULATION OF THESE NEURONS THAT ACTIVE AT THAT TIME. THERE ARE ABOUT 20,000 OF THESE NEURONS AND WE THINK THERE SHOULD BE ABOUT TWO HUNDRED OF THOSE NEURONS COACTIVE AT EACH TIME STEP. THE IDEA IS THAT THOSE NEURONS ARE SPARSELY ACTIVITY [LOW AUDIO] AND FORM A SPARSE SEQUENCE THAT FILLES THE ENTIRE DURATION OF THE SONG MOTIF. [LOW AUDIO] CONVERGING THEN ON TOp PRODUCE A PARTICULAR PATTERN OF MUSCLE CONTRACTION, PARTICULAR PATTERN OF TRACTION IN EACH MUSCLE APPROPRIATE TO PRODUCE THAT SONG¨— VOCALIZATION. THE KEY IDEA HERE IS THAT THE SONG PRODUCTION IS ESSENTIALLY BROKEN UP INTO THIN SLICES IN TIME. THE IDEA IS THAT THAT GROUP OF HBC NEURONS ACTIVATE A SINGLE BURST IN THESE RA NEURONS THAT LAST ABOUT EIGHT MILLISECONDS AND THADZ THAT PRODUCES CONTRACT TILE STATE OF THE MUSCLES AT THATT PRODUCES A BRIEF CONTRACT TILE STATE OF THE MUSCLES AT THAT¨q NOW, UM, WE'VE DONE A NUMBER OF EXPERIMENTSi] TO TRY TO UNDERSTAND THIS MOTOR CIRCUIT A LITTLE BIT AT A LITTLE BIT GREATER DETAIL. ACTUALLY, I WANT TO POINT OUT ONE MORE THING BEFORE I GO ON. IN THIS PARTICULAR MODEL, YOU SEE THAT THE PATTERN OF THE ACTIVITY PRODUCED IN THE OUTPUT IS CONTROL BID THE PATTERN OF CONNECTIONS FROM HBC TO RA. SO WHICH PARTICULAR SET OF NEURONS IS BEING GENERATE BID THAT GROUP OF [INDISCERNIBLE] NEURONS IS CONTROLLED BY THIS PATTERN OF [INDISCERNIBLE]. IT'S BELIEVED THAT THE [LOW AUDIO] -- SO WE'VE ALSO WONDERED EXACTLY HOW THIS SPARSE SEQUENCE OF ACTIVITY IN HBC GETS GENERATED. ONE OF THE EXPERIMENTS THAT WE'VE DONE ISf‡ TRY TO UNDERSTAND WHERE THE TIMING, WHERE THE BIOFI SICS ARE THAT CONTROL THE TIMING OF THAT SPARSE SEQUENCE. WHAT CONTROLS HOW FAST THAT SEQUENCE –rGOES? ARE THOSE BUSTS IN HBC BEING GENERATED BY CIRCUIT TRI WITHIN IT OR ARE THEY BEING DRIVEN BY THAT INPUT UP INTO HBC. ONE OF THE WAYS WE THOUGHT WE COULD ADDRESS THIS QUESTION WOULD BE TO LOCALLY CHANGE THE BIOPHYSICAL DYNAMICS IN DIFFERENT PARTS OF THE CIRCUIT. I'LL SHOW YOU RESULTS FROM COOLING LATERAL COOLING OF HBC. I BUILT A REFRIGERATOR HERE. WHEN YOU PASS CURRENT THROUGH IT„/7 ONE WAY, IT COOLS THIS PLATE, IF YOU PASS IT THE OTHER WAY, IT HEATS THAT PLATE SO YOU CAN CONTROL THE TEMPERATURE IN HBC. THERE IT SHOWS TWO BILATERAL COIN PATHS [LOW AUDIO] -- YOU CAN SEE WHEN YOU COOL HBC, THE SONG SLOWS DOWN ALMOST PERFECTLY [LOW AUDIO]. SO EVERY TIME SCALE FROM THE NOTES WITHIN THOSE SIYLLABLES TO THE DURATION OF THE MOTIF ALL SCALE UNIFORMLY BY COOLING HBC. WE ALSO WONDERED WHAT ACTUALLY GENERATES THAT SEQUENCE OFmy BURST IN HBC. THERE ARE SEVERAL DIFFERENT MODELS THAT HAVE BEEN SUGGESTED. SOMEONE A CHAIN-LIKE MODEL WHERE THESE NEURONS ACTIVATE THOSE NEURONS AND SO ON. OTHER MODELS HAVE SUGGESTED INCLUDING THE IDEA THAT THERE ARE OSCILLATORS IN HBC THAT CONTROL THE TIMING OF THEf‡ INTERVALS. WE WERE ABLE TO RECORD INTERCELLULARLY IN HBC NEURONS TRO PROJECT THE [INDISCERNIBLE] IN FREELY-BEHAVING BIRDS. MIKE LONG AND I BUILT A INTERCELLULAR MICRODRIVE THAT HOLDS A SHARP GLASS [INDISCERNIBLE] [LOW AUDIO]. IF YOU RECORD FROM NEURONS DURING SINGING, YOU FIND THERE IS SO SLOW [LOW AUDIO] THAT MIGHT CORRESPOND TO OSCILLATORS OR SLOW MEMBRANE POTENTIAL DYNAMICS CONTROLLING THE [LOW AUDIO] -- EACH BURST IS PRECEDED BY A -- SO OUR RESULTS LARGELY WILL ALLOW OS LATORY MODELS OF DYNAMICS IN HBC AND SUPPORT A CHAIN-LIKE MODEL. ACTIVITY PROPAGATES LIKE A WAVE THROUGH AvJR+S NEURONS. NOW, LET'S COME BACK TO LEARNING. ARE THERE ANY QUESTIONS ABOUT THAT BEFORE I TURN TO THE MAIN TOPIC? SO, UM, ONE OF THE REALLY FASCINATING THINGS ABOUT THE SONGBIRD IS THAT VOCAL LEARNING IS CONTROLLED BY A CIRCUIT SEPARATE FROM THE MOTOR PATHWAY. SO THERE IS ANOTHER SET OF BRAIN AREAS, THIS NUCLEUS [INDISCERNIBLE] WHICH IS ALSO IN THIS CORTEX. THERE IS A AREA CALLED AREA X. IT WAS NAMED BEFORE–r DAVID PERCAL SHOWED THAT IT WAS PART OF THE BIRD'S BASAL GANGLIA AND IT HAS [INDISCERNIBLE] AS IS OFTEN THE CASE IN LOWER VERTEBRAS. THE PAL TALL OUTPUTS AND DLM PROJECTS BACK [LOW AUDIO]. IF YOU LESION THOSE BRAIN AREAS IN ADULT BIRD,t( THERE IS ESSENTIALLY NO AFFECT ON THE VOCALIZATION. THE FACT THESE AREAS ARE IMPORTANT FOR LEARNING WAS SHOWN BY OTHERS IN THE 80s AND EARLY 90s. THIS IS WHAT WE'RE GOING SPEND MOST OF OUR TIME¨— TODAY TALKING ABOUT. THIS LEARNING PATHWAY. ONE OF THE ORIGINAL IDEAS FOR HOW THIS WORKS IS THAT EL MIN SOMEHOW GUIDES -- OUR LAB PRODUCED THE FIRST DIRECT EVIDENCE OF AN INSTRUCTIVE SIGNAL BEING TRANSMITTED FROM EL MIN TO RA. I WANT TO TAKE AN ASIDE AND SHOW YOU ANOTHER VERY IMPORTANT FUNCTION OF THIS EL MIN PROJECTION, AND THAT IS THE IDEA THAT THIS BRAIN CIRCUIT GENERATES VARIABILITY DURING LEARNING. AND THE PICTURE THAT I'M GOING TO TRY TO SHOW YOU SOME EVIDENCE FOR IS THE NOTION THAT THERE ARE ACTUALLY TWO MOTOR PATHWAYS IN THE SONGBIRD. THERE IS A, THIS, UM, EL MIN PATHWAY THAT DRIVES RANDOMNESS AND VARIABILITY AND EXPLORE FLAGS THE SONG THAT'S CHARACTERISTIC OF SUB SONG, AND THAT THIS HBC CIRCUIT IS A SEPARATE MOTORWAY THAT DRIVES STEREO TIPIC AND -- BY VIRTUE OF THESE SPARSE SEQUENTIAL CHANGE. THE IDEA IS THAT THE BIRD STARTS OUT WITH THIS CIRCUIT BEING HIGHLY ACTIVE, GENERATING SUB SONG, PLASTIC SONG IS PRODUCED BY A COMBINATION OF SEQUENTIALzV ACTIVITY BEING DRIVEN BY HBC PLUS NOISE AND EXPLORATION BEING DIVEN BY EL MIN AND AS MATURING INTO AN ADULT THIS CIRCUIT TAKES OVER AND THE EL MIN PATHWAY HAS VERY¨ LITTLE -- THE FIRST THING WE CAN DO IS TAKE A YOUNG BIRD THAT'S IN THIS MRS. TICK SONG STAGE WHERE YOU HAVE A COMBINATION OF STEREO TYPES AND VARIABLE STRUCTURE AND WE CAN LOOK AT THE SONG BEFORE AND AFTER WE'VE ACTIVATED EL MIN. HERE'S WHAT THE SONG¨ OF A TYPICAL SONGBIRD LOOKS LIKE. YOU CAN IDENTIFY SOME STRUCTURE THAT'S RELATIVELY IDENTIFIABLE EACH TIME THE BIRD SINGS YOU SEE THAT'S WHAT IT'S IT BUT IT'S HIGHLY VARIABLE. AFTER YOU INACTIVATE EL MIN HOWEVER THE VOCAL PAT CERTAIN MUCH MORE STEREO TYPE. IS THE, IN TACT BIRD [BIRD CHIRPS]. OKAY. HERE'S WHAT IT SOUNDS LIKE AFTER EL MIN HAS BEEN ACTIVATED. [BIRD CHIRPS]. AFTER EL MIN INACTIVATION THIS IS MORE STEREO TYPED ALMOST AS MUCH AS AN ADULT SONG. HERE'S ANOTHER EXAMPLE OF SLIGHTLY OLDER BIRD WHERE YOU CAN SEE, IDENTIFY THESE SYLLABLES MORE CLEARLY. THIS IS BE EL MIN INTACT AND THIS IS WITH IT INACTIVATED. IF YOU ZOOM IN, YOU'LL SEE WITH EL MIN INTACT THERE'S A LOT OF PITCH VARIATION AND WHEN INACTIVATED THAT PITCH VARIATION GOES AWAY AND THE ACOUSTIC STRUCTURE BECOMES MUCH MORE STEREO TYPED. RED SHOW PITCH TRACES AFTER EL MIN INACTIVATION. EL MIN IS NECESSARY FOR THE BABBLING, THE SUB SONG I SHOWED YOU A YOUNG BIRD AT ABOUT 45 DAYS OF AGE PRODUCES NORMAL SUB SONG. IF YOU INACTIVATE EL MIN BY THAT SAME TECHNIQUE I SHOWED YOU -- SORRY I DIDN'T TELL–rw3 YOU WHAT I WAS DOING THERE. THAT WAS WITH AN INJECTION OF MUSIMOL INTO EL MIN. AFTER INACTIVATING EL okMIN, THE VOCAL PATTERN BECOMES MUCH MORE [LOW AUDIO]. OKAY. SO, WHAT I'M TRYING TO ARGUE IS THAT EL MIN LESIONS ABOLISH VOCAL VARIABILITY TO PLASTIC SONG AND GIVEN IN THE EARLIEST BABBLING VOCALIZATIONS. OTHER LABS [INDISCERNIBLE] HAVE SHOWN THAT LESIONS OF EL MIN ALSO ABOLISH VARIABILITY IN ADULT SONGS. THE IDEA IS THAT EL ¨MIN NEURONS, WHAT WE'VE SHOWN IS THAT THEY GENERATE HIGHLY AVAILABILITY FIRING PATTERNS DURING SINGING IN THE JUVENILE AND ALSO BEEN FOUND IN THEÖ ADULT BIRDS AND WE BELIEVE IT INJECTS THAT VARIABILITY INTO THE MOTOR PATHWAY BY DRIVING RA NEURONS VIA [LOW AUDIO] INPUT INTO THE MOTOR PATHWAY. WE WOULD ARGUE THAT EL MIN SERVES AN ESSENTIAL PATHWAY IN LEARNING THAT'S USED FOR REINFORCEMENT LEARNING. ON THE FLIP SIDE, WE'VE ALSO SHOWN THAT HBC IS NOT NECESSARY FOR BABBLING. SO IF YOU TAKE A YOUNG BIRD AND YOU LESION¨ HBC, THERE IS ESSENTIALLY NO AFFECT ON BABBLING. THIS IS AROUND 45 DAYS OF AGE. IF YOU LESION HBC, YOU HAVE ESSENTIALLY NORMAL BABBLING AFTERWARD. [BIRD CHIRPING].xDa THE SECOND ONE WAS AFTER HBC LESION. IF YOU TAKE A SLIGHTLYd8¨— OLDER BIRD THAT'S PRODUCING THIS PLASTIC SONG, A COMBINATION OF STERP OWE TYPED VOCAL ELEMENTS AND RANDOM VOCAL ELEMENTS AND YOU¨ LESION HBC THIS, BIRD REVERTS TO PRODUCING SUB SONG. [BIRDmy CHIRPING]. AND AFTER HBC LESION. ] BIRD CHIRPING. ONE OF THEmy MOST REMARKABLE THING WE FOUND IS THAT EVEN IF YOU TAKE AN ADULT BIRD THAT'S PRODUCING A NORMAL STEREO TYPED ADULT SONG -- [BIRD CHIRPING] -- AND YOU LESION HBC --] BIRD CHIRPING] -- THE BIRD REVERTS TO SUB SONG. SO ALL OF THOSE THINGS, ALL THOSE EXPERIMENTS SUPPORT THE NOTION THAT THERE ARE SEPARATE CIRCUITS; THERE'S EL MIN¨ CIRCUITS THAT DRIVES VARIABILITY AND HBC CIRCUITS THAT DRIVES STEREO TIPIC. IF YOU TAKE A BIRD WHERE BOTH¨ OF THESE ARE CONTRIBUTING THESE EQUALLY AND ACTIVATE EL MIN -- YES.v: NO. THIS SHOWS YOU AN EXAMPLE. IF YOU GO TO A SLIGHTLY YOUNGER BIRD THAN THIS, WHEN YOU INACTIVATE EL MIN, THEY'RE ESSENTIALLY MUTE. IF YOU GO JESUS TO THE AGE AT AROUND 45 TO 47 DAYS THEY ENTER Ö PHASE WHERE WHEN YOU A INACTIVATE EL MIN THEY PRODUCE THIS HIGHLY STEREO TYPE VOCALIZATION. WHEN YOU INACTIVATE EL MIN, THE BIRD PRODUCES A VERY IMMATURE SONG, IT'S JUST STEREO TYPED. IT'S NOT AS THOUGH THERE'S A HIDDEN ADULT-LIKE SONG BURIED UNDER THIS VARIABILITY. THIS STRUCTURE GRADUALLY DEVELOP [LOW AUDIO]. OKAY.DEVELOPS [LOW AUDIO]. OKAY. NOW LET ME COME BACK TO THIS QUESTION. SO I SHOWED YOU EARLIER THAT IF YOU COOL HBC, YOU SLOW DOWN THE TEMPORAL STRUCTURE OF THE ADULT SONG. SO, IF YOU ASK THE SAME QUESTION ABOUT EL MIN WHICH KIND OF SITS IN AN ANALOGOUS POSITION IN THIS CIRCUIT, IF EL MIN INVOLVEDv: IN PRODUCING THE -- THE QUESTION IS, WHETHER THE SAME EXPERIMENT, IF WE COOL HBC, WE SLOW DOWN ADULT SONG F WE COOL DOWN EL MIN, WILL IT ALSO SLOW DOWN YOU BE SUB SONG. EL MIN AND HBC ARE BOTH CONTAINED IN THIS PART OF THE NETOPALIUM THAT MIGHT BE LIKE LAYER TWO/THREE. IF THIS¨ IS LIKE LAYER FIVE T INPUTS ARE COMING FROM THE NETOPALIUM. I WOULD SAY THIS IS¨ DEBATABLE POINT, BUT THE TRUTH IS THAT THESE TWO -- WHETHER IT'S LIKE LAYER TWO/THREE OR NOT IS DEBATABLE, BUT BOTH OF THESE STRUCTURES ARE IN THIS REGION OF THE PALIUM. THEY'RE BOTH IN THE SAME SIMILAR STRUCTURE, AND SO THERE'S SOME REASON TO THINK THERE MAY BE SOME ANALOGY BETWEEN THESE TWO IN TERMS OF PRODUCING THE DYNAMIC INVOLVED IN THE MOTOR BEHAVIOR. HOW DO WE ANSWER THAT QUESTION? FIRST, WANT TO SHOW YOU HOW YOU CAN CHARACTERIZE THE TEMPORAL BEHAVIOR OF SUB SONG. IF YOU RECORD SUB SONG AND YOU IDENTIFY SYLLABLES -- THEY HAVE A FAIRLY RAPID ONSET AND OFFSET. IF YOU SET A THRESHOLD ON AMPLITUDE YOU CAN IDENTIFY THEM. IF YOU PLOT THE DISTRIBUTION OF DURATION, THEY ARE ARE EXPONENTIALLY DISTRIBUTED. YOU CAN SEE THAT DISTRIBUTION IS A STRAIGHT LINE. THERE'S A SINGLE TIME SCALE THAT CHARACTERIZES THE DURATION OF SUB SONG SYLLABLES AND THIS SHOWS THE DISTRIBUTION OF DURATIONS FOR 19 BIRDS. IF YOU MEASURE THE SLOPE OF THAT LINE, YOU END UP SEEING THE TYPICAL TIME CONSTANT OF SUB SONG SYLLABLES IS ABOUT # 0-09 MILLISECONDS. WE SET OUT TO RECORD SUB SONG IN BIRDS WHERE WE COULD CONTROL THE TEMPERATURE. WE TOOK THE SAME LITTLE BRAIN REFRIGERATOR SAW DERED A GOLD SPIKE ON TO IT AND SHOWED THATyM SLOWLY INTO EL MIN AND RECORDED SUB SONG IN THOSE BIRDS WHILE WE WERE CONTROLLING THE TEMPERATURE YOU CAN SEE THAT IF UH YOU LOOK AT THE DISTRIBUTION OF SYLLABLE DURATIONS IN THE CONTROL BIRD VERSUS THE BIRD WHERE EL MIN IS COOLED, THERE IS AN ACCESS OF LONG SYLLABLEsDURATIONS. QUANTIFY THAT BY MEASURING THE SLOPE OF THIS LINE AND WHAT UH YOU SEE IS THAT WHEN YOU COOL EL MIN, YOU'VE STRETCHED OUT THE SUB SONG SYLLABLES,i]d8 IF YOU COOL HBC, YOU HAVEN'T. SO WHAT THAT SUGGESTS IS THAT THE DYNAMIC THAT CONTROL THE TEMPORAL STRUCTURE OF THE SONG IN ADULTS IS CONTROLLED BY THIS HBC CIRCUIT THAT GENERATES A¨–r STEREO TYPED SEQUENTIAL PATTERN OF THE ACTIVITY, BUT IN JUVENILES IT'S CONTROLLED BY EL MI-N WHICH GENERATES HIGHLY VARIABLE PATTERNS. NOW, ONE MORE IMPORTANT QUESTION. EL MIN IS PART OF A CORTICAL BASAL GANGLIA LOOP. WE WANTED TO KNOW WHICH PARTS OF THE CIRCUIT ARE ACTUALLY INVOLVED IN THE GENERATION OF THIS BABBLING BEHAVIOR. SO JESSE GOLDBERG¨ IN THE LAB CARRIED OUT EXPERIMENT IS WHERE WE RECORDED SUB SONG BIRDS AND LESIONED SUB SONG X. WHEN YOU LESION X YOU FIND THERE IS ESSENTIALLY NO EFFECT ON THE BABBLING BEHAVIOR. THIS SHOWS A SUB SONG IN AN ATTACKED BIRD AND THE SAME BIRD AFTER THE LESION OF X. I JUST WANT TO POINT OUT THAT THIS IS GOING TO BE A VERY IMPORTANT CONCEPT FOR WHAT I'M GOING TO TELL YOU LATER. THE LESIONS OF THE BASAL GANGLIA HAVE LITTLE OR NO EFFECT ON EITHER ADULT SONG OR JUVENILE SONG. OKAY. THE MOTOR BEHAVIOR, ITSELF, IS ESSENTIALLY UNAFFECTED BY LESIONS IN THE BASAL GANGLIA, BUT THE LESION [LOW AUDIO] THE BIRD CAN'T LEARN ANYMORE. SO I WANT TO]I LEAVE YOU WITH THIS IDEA THAT THE DYNAMICS UNDERLYING THE GENERATION OF THE BEHAVIOR BOTH SUB SONG AND ADULT SONG ARE BEING GENERATED, PERHAPS, BY THESE CORTICAL CIRCUITS AND NOT BY THE BASAL GANGLIA, ITSELF. WE HYPOTHESIZED THAT THE VOCAL SEQUENCES IN ADULTS ARE GENERATE LOCAL CIRCUIT S DYNAMICS R WITHIN HBC, I'M FAIRLY CONFIDENT ABOUT THAT STATEMENT. WE HYPOTHESIZED THAT THE EXPLORATORY BABBLING Tv: VOCAL AVAILABILITY IS GENERATED BY LOCAL CIRCUIT DYNAMICS BETWEEN EL MIN. I WOULD SAY THAT'S LESS STRONGLY SUPPORTED, BUT WHAT I CAN SAY FOR SURE IS THAT THE BASAL GANGLIA ARE NOT INVOLVED IN THAT BEHAVIOR. SO KEEP THAT IN MIND. [LOW AUDIO]. OKAY. SO NOW LET'S TURN TO ANOTHER IMPORTANT QUESTION WHICH IS: HOW DOES CONTROL OF THE SONG GET TRANSFERRED FROM THE EL MIN TO THIS SEQUENTIAL CIRCUIT? NOW WE'RE GOING START GETTING INTO THE QUESTION, HOW THE LEARNING WORKS IN THE CIRCUIT, AND I'M GOING TO SHOW YOU EVIDENCE THAT THERE IS AN INSTRUCTIVE SIGNAL THAT GETS TRANSMITTED FROM EL MIN TO RA TO DRIVE VOCAL LEARNING, AND THAT IDEA, THE IDEA THAT THERE'S AN INSTRUCTIVE SIGNAL GOES BACK SARAH WHO WAS THE FIRST TO SHOW THAT EL MIN LESIONS LOCK VOCAL LEARNING. IT SEEMS LIKE A REASONABLE HYPOTHESIS, AND HERE'S HOW WE WERE ABLE TO DEMONSTRATE THAT THAT'S ACTUALLY THE CASE. SO THIS SHOWS A PROGRESSION OF SONG LEARNING FROM IN A SINGLE BIRD, AS HE IMTATDS THE TUTOR SONG. THIS IS A SINGLE SYLLABLE OF THE SONG THAT THE BIRD HEARD. THIS IS OVER A MONTH. THE FIRST THING YOU NOTICE IS THAT THE VOCAL PROCESS IS EXTREMELY SLOW. IT'S ABOUT 606 HERTZ HERE. STARTING EARLY THE PITCH GRADUALLY CONVERGES TO A GOOD COPY OF THE TUTOR'S SONG, BUT THAT TAKES ALMOST A MONTH FOR THE BIRD TO CONVERGE ON THAT. SO HOW DO YOU STUDY LEARNING WHERE THE PROCESS IS SLOW? OUR LAB AND ANOTHER LAB HAVE DEVELOPED AN APPROACH THAT ALLOWS US TO GET VERY STRONG EXPERIMENTAL CONTROL ON THE LEARNING PROCESS. THE IDEA IS, SO THIS IS THE EXPERIMENTAL SETUP THAT WE DEVELOPED. THE IDEA IS THAT WE MONITOR THE BIRD VOCALIZATION, IN THIS KAGS WITH A LITTLE MICROPHONE ON THE BIRD'S HEAD. WE RECORD THAT -- ACTUALLY WE PUT THAT SIGNAL INTO A DIGITAL SIGNAL PROCESSOR THAT CAN–r) MEASUREMENTS ON THAT ACOUSTIC SIGNAL IN REALTIME AND FEEDBACK ACOUSTIC SIGNALS INTO THE BIRD'S EAR AND WE PUT THOSE INTO THE aT SPEAKER RIGHT INTO THE AIR SACK WHICH IS CONNECTED TO THE INSIDE OF THE BIRD'S EAR DRUM. KIND OF LIKE IN-HEAD MICROPHONES. ALLOWS US TO DISTURB THE THIS BIRD'S SONG WITHOUT DISTURBING OUR ABILITY TO ACTUALLY RECORD THE SONG AT THE SAME TIME. SO HERE'S THE EXPERIMENT THAT WE DID. WE -- THE BIRD SINGS, WE'RE RECORDING THAT, AND WE'RE MEASURING THE PITCH OF THIS HARMONIC STACK AS THE BIRD IS SINGING [IT. WE SET A THRESHOLD THAT WHEN THE PITCH GOES ABOVE THAT, WE PLAY NOISE TO THE BIRD. WHAT WE'RE TRYING TO DO IS TO MAKE THE BIRD THINK THAT HE MADE AN ERROR. IF THE PITCH DOESN'T GO ABOVE THAT HARMONIC STACK, THAT THRESHOLD, RATHER, WE DON'T PLAY THAT [LOW AUDIO]. SO WHAT WE WERE HOPING w:D HAPPEN IS EXACTLY WHAT HAPPENED. THE BIRD ADJUST HIS PITCH TO AVOID THAT THRESHOLD. THIS SHOWS PITCH TRAJECTORY AT THE BEGINNING OF THE LEARNING ¨ PROCESS. AFTER TWO HOURS AND AFTER FOUR HOURS YOU CAN SEE HE'S ESSENTIALLY COMPLETELY AVOIDING THE THRESHOLD. SO HIS PITCH NO LONGER GOES ABOVE THAT THRESHOLD. ACTUALLY, UH, MICHAEL'S LAB SHOWED THIS SAME THING, ACTUALLY BEFORE WE DID, THAT IF YOU PLAY [LOW AUDIO]. THIS SHOWS THE PITCH GOING DOWN, YOU CAN ALSO MAKE THE PITCH GO UP. AND SO WE DID EXPERIMENTS CONTROLLING THIS PITCH SO YOU CAN CHANGE THIS THRESHOLD DAY AFTER DAY TO PUSH THE PITCH UP AND DOWN AND UP AND DOWN OVER THE¨ COURSE OF MANY DAYS, AND THIS GIVES YOU EXPERIMENTAL CONTROL OVER THAT LEARNING PROCESS ON AN HOURS TIME SCALE. OVER-9 THE COURSE OF ONE DAY, THE PITCH GOES UP. WE RESET THE FRESHHOLD T PITCH GOES UP AGAIN AND SO ON. WE CAN BEGIN ASKING, BEGIN MANIPULATING THE CIRCUIT ON THIS TIMESCALE AND ASK HOW [LOW AUDIO]? SO, YOU CAN SEE THAT YOU CAN GET PITCH CHANGES IN THE COURSE OF THE DAY OF –rkOABOUT 15-20 HERTZ. HERE'S THE CONCEPTUAL PICTURE. THE IDEA IS THAT YOU HAVE THE MOTOR SYSTEM PRODUCING A TRAJECTORY IN SOME PARAMETER SPACE. OKAY. IN THIS CASE, TWO PARAMETER NPS THE PLANE OF THE BOARD SHOWS THE TRAJECTORY OF THE MOTOR SYSTEM AT ONE PARTICULAR TIME SLOT. OKAY. THE IDEA IS THAT EL MIN IS INJECTING VARIATIONS AROUND THAT. THE AFP IS DRIVING¨ EXPLORATORY VARIATIONS AROUND THAT. SO WHAT WE'VE DONE IS WE'VE IMPOSED AN AIR GRADIENT WHERE THESE THINGS HERE PRODUCE AN ERROR AND THESE DON'T. SO THE SONG SOUNDS BETTER IN THIS DIRECTION [INDISCERNIBLE]. IN RESPONSE TO THAT AIR ¨— GRADIENT, THE BIRD MOVES HIS VOCALIZATION. RIGHT? THOSE PITCHES MOVE IN THE DIRECTION THAT PRODUCES FEWER ERRORS. NOW WE WANT TO ASK THE QUESTION: HOW DOES THAT ACTUALLY HAPPEN? I WANT TO SUGGEST TO YOU TWO POSSIBLE MODELS. ONE POSSIBILITY IS THAT THE ANTERIOR FOREBRAIN PATHWAY STARTS GENERATING EXPLORATORY VARIATION IN THE DIRECTION THAT PRODUCED FEWER ERRORS AND PRODUCING FEWER VARIATIONS IN THE DIRECTION THAT MAKES MORE ERRORS. THIS IS WHAT WE WOULD CALL AFP-DRIVEN BIAS, AND IT'S IN THE DIRECTION OF REDUCED ERROR. THE OTHER POSSIBILITY, THOUGH, IS THAT THE MOTOR PATHWAY EXHIBITS PLASTICITY SO THAT THE MOTOR PATHWAY, ITSELF, HAS PRODUCED A CHANGE IN ITS TRAJECTORY. DOES THAT MAKE SENSE? SO HOW CAN WE DISTINGUISH BETWEEN THESE TWO MODELS? THIS MODEL SAYS THAT THE AFP IS PRODUCING BIASSED VARIABILITY; THIS MODEL SAYS THERE'S NO BIASSED VARIABILITY T PLASTICITY GOES DIRECTLY INTO THE MOTOR PATHWAY AND WE CAN DISTINGUISH BETWEEN THESE 2 SIMPLY BY DRIVEING THIS LEARNING PROCESS AND THEN INACTIVATING THE AFP AND ASKING WHERE THE PITCH IS WHERE IT WAS INITIALLY OR IF IT'S MOVED. SO WE DIDS7„ EXACTLY THAT EXPERIMENTS. WE DRIVED THIS LEARNING DAY AFTER DAY UP AND DOWN. WE CAN -- AT THE END OF THAT LEARNING PROCESS, INACTIVATE EL M-IN AND ASK WHAT HAPPENED TO THE PITCH. WE DO THIS ON EVERY OTHER DAY AND ON THE INTERVENING DAYS WE PUT IN VEHICLE. HERE'S THE TECHNIQUE WE USED. WE DEVELOPED A SMALL MICRODIALYSIS TUBEkO THAT -- AND A DRUG RESERVOIR SO WE CAN PUT THE DRUG INTO THAT RESERVOIR AND IT INFUSES AND ACTIVATES EL MIN FOR HOWEVER LONG WE WANT. WE CAN WASH IT OUT AND DO IT AGAIN DAY AFTER DAY. HERE'S THE RESULT. IF YOU DRIVE THIS LEARN PROGRESS SESZ USING THE DISTORTED AUDITORY FEEDBACK, DISRUPTIVE AUDITORY FEEDBACK, AND AT THE END OF THE DAY YOU ACTIVATE EL MIN YOU SEE THE PITCHok REVERTDS BACK ROUGHLY TO WHAT IT WAS. YOU CAN SEE THAT THE PITCH GOES -- IF YOU'VE BEEN PUCHING THE PITCH UP, THE PITCH GOES BACK DOWN. IFxD YOU'VE BEEN PUSHING THE BICH DOWN, THE PITCH GOES BACK UP. IF YOU PUT IN VEHICLE, THERE'S NO AFFECT. SO THE ANSWER IS THAT IN RESPONSE TO ERROR GRADIENT, WHEN SOME MOTOR EXPLORATIONS PRODUCE MORE ERROR AND OTHERS PRODUCE LESS ERROR, THAT VARIABILITY BECOMES BIASSED SO THAT THE BIRD PRODUCES MORE VARIABILITY IN THE DIRECTION [LOW AUDIO]. SO THAT HAS LEFT US WITH A PICTURE THAT THE WAY THE LEARNING WORKS IS THAT THE MOTOR PATHWAY DOES A FAIRLY STEREO TYPEDi] TRAJECTORY, AFP DRIVES VARIABILITY AROUND IT IN THE PRESENCE OF A ERROR GRADIENT, A FP STARTS PRODUCES BIASSED VARIABILITY, AND WE'VE SEWN THAT THE BIASSED VARIABILITY OVER THE COURSE OF ABOUT A DAY GETS TRANSLATE INTO PLASTICITY IN THE MOTOR PATHWAY OF THAT SAME AMOUNT BY THE NEXT DAY, 24 HOURS LATER.¨ AND THE IDEA IS THAT DAY AFTER DAY THAT BIASSED VARIABILITY KIND OF PULLS THE MOTOR PATHWAY INTO THE DIRECTION OF IMPROVE VOCAL PERFORMANCE. SOsTHESE, THIS NOTION OF BIAS VARIABILITY AND THE LATER CON SOL VEGAS OF THAT BIAS INTO THE MOTOR PATHWAY HAS NOW BEEN SEEN IN THE ZEBRA FINCH WHICH IS WHERE WE OBSERVED IT AND IN OTHERu! FINCHES BY MICHAEL'S LAB. SO I'M GOING TO SHOW YOU A KIND OF A FRAMEWORK ARK GENERAL PICTURE FOR HOW WE THINK THIS WORKS IN THE BIRD. SO THE IDEA IS THAT THE HBC REACHES THE MOTORWAY GENERATES THE MOTOR SEQUENCE -- HOW MIGHT THAT AVAILABILITY IN EL MIN BECOME BIASSED IN THE DIRECTION OF IMPROVED OUTCOME? SO THE IDEA IS VERY SIMPLE. EVERY AXON GOES FROM EL MIN TO MOTOR PATHWAY¨ SENDS A COLLATERAL TO THE BASAL [LOW AUDIO]. [LOW AUDIO]. IFt( AREA X ALSO HAD AN EVALUATION SIGNAL THAT SAID THAT VARIATION SOUNDED GOOD OR BAD, X COULD FIGURE OUT WHICH PATTERNS OF ACTIVITY IN EL MIN LED TO BETTER SONG PERFORMANCE AND WHICH LED TO WORSE SONG PERFORMANCE. WE SET OUT TO LOOK FOR A SIGNAL THATHtr8 MIGHT CARRY SUCH INFORMATION, AND IN FACT, ONE SEEMED LIKELY POSSIBILITY TO US IS THAT SUCH A SIGNAL MIGHT BE CARRIED BY [INDISCERNIBLE] INPUT INTO THE BASAL GANGLIA. AREA X RECEIVES A VERY LARGE [INDISCERNIBLE] FROM THE [LOW AUDIO]. YOU ALL OF COURSE KNOW ABOUT THE POTENTIAL, THE POSSIBLEW ROLE OF [INDISCERNIBLE] MIDBRAIN NEURONS IN -- -- PRIMATES HAVE BEEN SHOWN TO GENERATE A BURST OF ACTIVITY IN RESPONSE TO AN UNEXPECTED REWARD, AND THEY SHOW A DIPPING ACTIVITY IN RESPONSE TO A REWARD THAT WAS EXPECTED BUT WAS NOT RECEIVED. SO THIS IS EXACT THINK KIND OF SIGNAL THAT MIGHT BE USEFUL IN A CIRCUIT LIKE THIS TO DRIVE LEARNING. SO WE WANTED VERY MUCH TO RECORD FROM BTA NEURONS AND SINGING BIRDS. THE¨ NEURONS ARE HARD TO RECORD FROM IN THE SONGBIRD, AND JUST AT THE TIME THE POST DOC WORKING ON THIS PROJECT HAD GOTTEN CLOSE TO PULLING ALL OF [LOW AUDIO], DAVID PUBLISHED A VERY NICE OBSERVATION. ACTUALLY THIS WAS AN OBSERVATION THAT WAS AT THE END OF A LONG PAPER ABOUT SOMETHING ELSE, AND WHAT HE FOUND IS THAT THERE IS ACTUALLY A CORTICAL1E TO BTA, AND IT'S RIGHT NEXT TO THIS AREA, RA, AND WE WERE VERY GOOD AT RECORDING NEURONS IN RA AND WE THOUGHT, OH, WELL MAYBE WE CAN STUDY THIS AREA AND SEE WHERE WHETHER IT MIGHT CARRY A REINFORCEMENT OR ERROR-RELATED SIGNAL TO [LOW AUDIO]. SO THEv: FIRST THING WE DID WAS ASK WHETHER THIS AREA IS EVEN NECESSARY FOR VOCAL LEARNING. THIS SHOWS A RETROGRADE LABEL FROM INJECTIONS OF CHOLERA TOXIN' INTO BTA. WE LABELLED THIS NICE BUNCH OF NEURONS UNDER RA. DAVID DIDN'T EVEN NAME THIS AREA SO WE VERY INEXPERTLY NAMED IT THE VENTRAL PART OF THE INTERMEDIATE AR KOE PAL YUM AFTER TALKING TO SEVERAL PEOPLE, AIV. WE LESIONED AIV AND ASKED WHETHER BIRDS COULD STILL LEARN THEIR SONG. THIS IS A XIE TOE TOXIC LESION WITH MMA. WE TUTORED BIRDS IN THEIR HOME CAGE, TOOK THEM OUT UNTIL THEY BEGAN TO SING AND THEN WE LESIONED AIV WITH THIS INJECTION OF MMA AND THEN WE THEN KEPT THE BIRDS UNTIL 90¨—xD DAYS AND ASKED HOW WELL DID THEY IMITATE THEIR FATHER'S SONG. THIS SHOWS AN EXAMPLE OF A TUTOR SONG. HERE'S¨ THE SONG THAT THIS BIRD SHOULD HAVE LEARNED. [BIRD CHIRPING]. HERE'S WHAT THE BIRD ACTUALLY PRODUCED AT 90 DAYS OF AGE. [BIRD CHIRPING]. OKAY. YOU CAN SEE THAT THE BIRD DOESN'T EVEN HAVE VERY MUCH OF A MOTIF STRUCTURE, LET ALONE A GOOD IMITATION OF THE FATHER'S SONG. HERE'S ANOTHER BIRD. THE TUTOR'S SONG. [BIRD CHIRPING]. HERE'S THE SONG THAT THIS JUVENILE BIRD PRODUCED AS AN ADULT AFTER BEING LESIONED. [BIRD CHIRPING]. YOU CAN SEE THERE'S VERY POOR IMITATION. THERE ARE STILL PRELIMINARY RESULTS AND WE'RE WORKING VERY A7" HARD TO DO THIS IN A LARGER RESULT LOOKS LIKE IT'S HOLDING UP VERY WELL. SO IT SEEMS THAT THIS AREA IS NECESSARY FOR VOCALI, BUT DOES IT CARRY ANY SIGNALS THAT MIGHT BE RELATED TO VOCAL ERRORS? SO WE SET OUT TO RECORD FROM NEURONS IN THIS SPIRITUALLY RECORDED [INDISCERNIBLE] THAT PROJECT TWO THE VENTRAL TAGMENTAL AREA AND HERE WHAT%q WE FOUND. THESE NEURONS ARE VERY WEAKLY RESPONSIVE DURING SINGING. THEY FIRE AT LOW RATES, A FEW HERT HERTZ, BUT WHEN YOU FIRE NOISE TO THE BIRD THAT WE USED IN AN EXPERIMENT, THESE BIRDS PRODUCE A ROBUST RESPONSE TO THAT. YOU COULD SEE [INDISCERNIBLE] A LARGE BIRS T LATENCY IS TYPICALLY AROUND 25 MILLISECONDS. ABOUT 50% OF THE NEURONS THAT WE RECORDED EXHIBIT Au!hT BURST RESPONSE LIKE THIS TO A NOISE PLAYED DURING SINGING. THE LATENCY IS ABOUT 55 MILLISECONDS AND THE [LOW AUDIO]. OKAY. SO, UM, SO WE HAVE A LOT TO DO, STILL, ON THIS OBSERVATION THAT THIS INPUT TO THE SONG SYSTEM MIGHT BE CARRYING AN ERROR-RELATED SIGNAL THAT DRIVES MOTOR LEARNING, DRIVES VOCAL LEARNING. WE'RE CHASING THAT INPUT BACK UP INTO AUDITORY AREAS. WE ARE TRYING TO MANIPULATE THAT DESCENDING INPUT INTO THE VENTRAL TECHMENTAL AREA AND DIRECTLY OTHER NEURONS WITH TOOLS TO SEE IF WE CAN DRIVE LEARNING IN THE SONG SYSTEM. [LOW AUDIO]. IT'S VERY EXCITING FINDING BECAUSE IT REPRESENTS A MECHANISM BY WHICH POTENTIALLY CORTICAL AREAS THAT ARE EVALUATING OUR BEHAVIOR COULD FEEDBACK INTO REWARD-RELATED SYSTEMS TO SHAPE OUR FUTURE BEHAVIORS. OKAY. SO THIS IS AN EXAMPLE WHERE, UM, YOU HAVE AN EVALUATION OF A COMPLEX BEHAVIOR, NOT JUICE DROPS DRIVINGv: LEARNING BUT SELF-EVALUATION. OKAY. SO I'M GOING THE PUT ALL OF THIS TOGETHER AND TRY TO PRESENT TO SIMPLE NOTION OF HOW THE YOU AV WHOLE THING MIGHT WORK. IT'S OUR WORKING HYPOTHESIS. I HOPE THAT SOME PART OF IT IS ACTUALLY RIGHT, BUT HERE'S THE WAY WE THINK ABOUT IT. AGAIN, HBC DRIVES SOME PATTERN OF SEQUENTIAL ACTIVITY NRA THAT DRIVES SOME MOTOR [LOW AUDIO]. EL MIN DRIVES SOME VARIATION AROUND THAT STEREO TYPED TRAJECTORY. THAT PRODUCES A VOCAL OUTPUT WHICH IS THEN EVALUATED BY AUDITORY BRAIN AREAS AND TRANSMITTED THROUGH AIV TO BTA TO THE BASAL GANGLIA. THE NOTION, NOW, IS THAT AREA X IS IN A POSITION TO EVALUATE THE VARIATIONS THAT ARE GENERATED BY EL MIN, AND DO THAT EVALUATION IN A TEMPORALLY-SPECIFIC WAY. I HAVEN'T TOLD YOU YET ABOUT THE PROJECTIONS FROM HVC TO THE STRIATUM, BUT THESE PROJECTIONS HERE ARE ALSO SPARSE. THEY LOOK VERY MUCH LIKE THE PROJECTIONS FROM HVC TO –rRA. -- EL MIN FLUCTUATION AT A PARTICULAR TIME IN THE SONG AND SEE WHETHER THEY LEAD TO AN IMPROVED OUTCOMEf‡ TRANSMITTED TO X [LOW AUDIO]. THEN, WHAT DOES IT DO WITH THAT INFORMATION? IT TAKES THAT INFORMATION AND SENDS IT BACK TO EL –r¨—MIN TO BIAS EL MIN ACTIVITY IN THE DIRECTION OF IMPROVED VOCAL PERFORMANCE WHICH THEN BIASES ACTIVITY IN THE MOTOR PATHWAY, WHICHxD THEN DRIVES PLASTICITY IN THE [LOW AUDIO].w3 OKAY. THAT'S THE BIG PICTURE. NOW, AS USUAL, I KEPT WALKING OUT ON THAT SHAKY LIMB AND DEVELOP -- I COULDN'T HELP MYSELF THINKING OF HOW THAT KIND OF PROCESS, HOW THAT KIND OF LEARNING PROCESS MIGHT ACTUALLY BE [INDISCERNIBLE] AT THE LEVEL OF BASAL GANGLIA CIRCUITRY. I'M GOING TO GO FURTHER OUT ON THAT LIMB OF SUGGESTING A SPECIFIC CIRCUIT MODEL THAT MIGHT IMPLEMENT LEARNING IN THAT CIRCUIT, THEN I'M GOING TO GO FURTHER OUT ON THAT LIMB AND SUGGEST HOW THAT MIGHT BE APPLIED TO MOTOR LEARNING [LOW AUDIO]. EL MIN ACTIVITY IS HIGHLY VARIABLEok DURING LEARNING. HBC NEURONS GENERATE VERY STEREO TYPEDyM SEQUENTIAL ACTIVITY.E IF YOU RECORD FROM MEDIUM SPYNY NEURONS IN JUVENILE BIRDS THEYd8 DO SPARSELY. THEY ARE ACTIVE AT DIFFERENT TIMES IN THE SONG. THAT PROVIDES A SUB STRAIGHT FOR THE BASALu! GANGLIA FOR THE STRIATUM IN THE SONGBIRD TO EVALUATE EL MIN ACTIVITY AT SPECIFIC MOMENTS IN THE SONG. WHY IMPORTANT? BECAUSE THE FLUCTUATION OF AN EL MIN AT ONE TIME MIGHT MAKE THE SONG BETTER, BUT AT A DIFFERENT TIME MIGHT MAKE IT WORSE. HERE'S HOW THE EVALUATION MIGHT HAPPEN. I'M GOING SUGGEST THE SIMPLE MODEL BY WHICH MEDIUM SPYNY NEURONS IN AREA X COULD BIAS EL MIN ACTIVITY IN A TEMPORALLY-SPECIFIC WAY. SO THE IDEA IS THAT EL MIN NEURONS PRODUCT TO MANY DIFFERENT MEDIAL SPYNY NEURONS AND EACH OF THOSE GETS AN INPUT FROM A SMALL SUBSET OF HBC NEURONS ACTIVITY AT A PARTICULAR TIME AND THAT'S HOW THEY PRODUCE THEIR VERY TEMPORALLY-SPARSED PATTERNS OF ACTIVITY. HERE'S THE IDEA. IMAGINE THAT WE HAVE A PARTICULAR EL MIN NEURON THAT WHEN IT'S ACTIVITY AT TIME POINTS TWO AND FOUR MAKES THE SONG BETTER, BUT WHEN IT'S ACTIVITY AT OTHER TIMES, HAS NO AFFECT OR MAKES THE SONG WORSE. EFFECT OR MAKES THE SONG WORSE. WE CAN IMAGINE THAT WE CAN BIAS THAT EL MIN NEURON AT TIME POINTS TWO AND FOUR MOREg# OFTEN SIMPLY BY STRENGTHENING THOSE INPUTS FROM THAT HBC NEWONES TWO AND FOUR SO THAT NOW WE'RE GOING TO ACTIVATE MEDIUM SPYNY NEURON TWO AND FOUR AT THE TIME POINTS IN THE SONG AND THAT WILL INHIBIT THE PAL TALL NEURONS WHICH WILL RELEASE THE¨ [INDISCERNIBLE] NEURONS FROM INHIBITION WHICH WILL FEEDBACK AND MAKE THOSE EL MI NEURONS MORE ACTIVE AT TIME POINTS TWO AND FOUR. YOU CAN SEE THAT SIMPLY BY TURNING ON THOSE TWO SYNAPSES WE'VE MADE THE OUTPUT BETTER. [LOW AUDIO]. HOW DO WE LEARN TO TURN THOSE TWO SYNAPSES ON? HOW DO WE LEARN TO MAKE THOSE SYNAPSES STRONGER? I'VE HYPOTHESIZED A VERY SIMPLE TRIPLET LEARNING RULE THAT, UM, WORKSR>„ LIKE THIS. THE IDEA IS THAT WE'RE GOING TO REMEMBER WHENEVER AN EL MIN NEURON AND AN HBC NEURON WERE COACTIVE. THERE'S OUR HBC NEWON ACTIVE. THERE'S OUR EL MIN NEURON ON BY CHANCE. THERE'S AN ELIGIBILITY TRACE THAT REMEMBERS THAT THOSE TWO THINGS WERE ON AT THE SAME TIME. WHY DO WE NEED TO REMEMBER IT? OUR REWARD SIGNAL DOESN'T COME UNTIL 25 OR 50 OR 100 MILLISECONDS LATER, BUT WHEN THAT REWARD SIGNAL COMES BACK, THEN WE'RE GOING STRENGTHEN JUST THE HBC INPUT, NOT TELL MIN. OKAY. WHY THE HBC INPUT? BECAUSE IT IS THE ONE THAT CAN ACTIVATE THE MEDIUM SPYNY NEURON ELIABLY N AT THAT TIME POINT,¨ THE FUTURE, AND THAT WILL THEN FEEDBACK IN BIAS OUR EL MIN NEURON. OKAY. SO WE'VE NOW DETECTED WHETHER EL MIN ACTIVITY AT THAT TIME LEADS TO A REWARD AND IF IT DOES WE STRENGTHEN THAT SYNAPSE WHICH WILL NOW ALLOW HBC TO BIAS EL MIN THROUGH THIS DIRECT PATHWAY, BE ACTIVE AT THE RIGHT TIME. NOTICE T KEY FEATURE OF THIS MODEL IS THAT MEDIUM SPYNY NEURONS HAVE TWO INPUTS; A CONTEXT INPUT FROM HBC. WHY CONTEXT? BECAUSE IT'S TIME IN THE SONG. WHAT TIME IS IT IN THE SONG RIGHT NOW? AND IT HAS AN E FRENS COPY INPUT FROM EL MIN THAT SAYS, WHAT DID I JUST¨ DO? AND THEN VPA SAYS, WELL, WHATEVER YOU JUST DID WAS GOOD OR WHATEVER YOU JUST DID WAS BAD. WE USE THOSE THREE INPUTS TO FIGURE OUT WHICH MEDIUM SPYNY NEURONS TO ACTIVATE AND -- WHICH IS GENERATED IN THEsCORTEXES. OKAY. ALL RIGHT. SO HOW MIGHT THIS IDEA -- YES. I NEED TO WRAP IT UP. OKAY.–r SO I'M JUST GOING TO SUGGEST A VERY SIMPLE IDEA, AND THAT IS THAT PERHAPS A SIMILAR PROCESS MIGHT OCCUR IN THE BASAL GANGLIA IN OTHER MOTOR BEHAVIORS. SO, UM, SO THE PICTURE OF HOW THE BASAL GANGLIA, STRIATUM AND [INDISCERNIBLE] DRIVE SECOD LEARNING IS THAT YOU CONNECT -- LET'S SAY A PARTICULAR TARGET WAS LIT THAT ACTIVATES [INDISCERNIBLE] FROM INHIBITION AND WHEN YOU LEARN THAT CONNECTION, YOU CAN [LOW AUDIO]. IF YOU REWARD IT IN A PARTICULAR DIRECTION BUT NOT OTHERS, YOU CAN STRENGTHEN A PARTICULAR SYNAPTIC INPUT AND MAKEu! THAT TARGET ILLUMINATION ACTIVATE THAT PATHWAY AND DRIVE [LOW AUDIO]. OKAY.–r SO IMAGINE THAT WE IMPLEMENT THE SAME KIND OF LEARNING RULE WHERE YOU HAVE A SENSORY STIMLAUS THAT INDICATES A PARTICULAR TARGET CAME ON FOLLOW BID ACTIVITY IN LET'S SAY SOMEWHERE IN THE BRAIN, LET'S SAY THIS IS A FRONTAL EYE FIELD NEURON THAT ACTIVATES ACTIVITY IN THE [INDISCERNIBLE]. NOW WE HAVE OUR E FRENS COPY INPUT. [INDISCERNIBLE]. NOW WE'VE MET ALL OF THE CONDITIONS TO DRIVE PLASTICITY IN OUR CORTICOTRY YEA TALL INPUT AND NOW INw3 LATER TRIALS, WE ACTIVATE OUR [INDISCERNIBLE]. AGAIN, WE HAVE CONTEXT INPUT WHICH TELLS US WHICH SENSORY CONTEXT WE'RE IN. WE HAVE AN E FRENS COPY OF A MOTOR ACTION TAKEN OUTSIDE OF THE BASAL GANGLIA, AND THIS LEARNING RULE DRIVESPLASTITY AND [LOW AUDIO]. THERE'S ANOTHER POTENTIAL IMPLEMENTATION INVOLVED E FRERNS COPIES FROM [INDISCERNIBLE] BACK THROUGH THE THALAMUS BACK INTO THE STRIATUM. [LOW AUDIO]. LET ME STOP THERE. THE IDEA IS THAT IN ORDER TO LEa&3 THE RELATION BETWEEN ACTION AND REWARD T BASAL GANGLIA NEED TO RECEIVE INFORMATION NOT JUST ABOUT THE SITUATION OR THE CONTEXT THAT THE ANIMAL'S IN, BUT ALSO ABOUT THE ACTIONS THAT THEQ%„ ANIMAL TOOK. IN THE SONGBIRD BASAL GANGLIA, [INDISCERNIBLE] X DEFINITELY RECEIVES A E FRENS COPY FROM EL MIN AND I HYPOTHESIZE THAT'S A GENERAL PRINCIPLE OF BASAL GANGLIA -- CAN BE USED TO GATE KOE STRIATAL MRAS THE TISTY FROM -- I WANT TO THANK THE MEMBERS OF MY LAB AND THE NIH FOR FUNDING. THANK YOU. [APPLAUSE] >> [LOW AUDIO]. >> YES, YES. >> [LOW AUDIO]. >> UM, LET'S SEE. THE WAY THAT I THINK ABOUT IT IS THAT THAT THAT EXPONENTIAL DISTRIBUTION -- SO THE WAY I THINK ABOUT IT IS THAT THERE MAY BE ONE PART OF EL MIN BUSY GENERATING [LOW AUDIO] AND ANOTHER THAT CAUSES THE PRESSURE TO DROP AND CAUSE [LOW AUDIO] TERMINATIONS. SO YOU COULD JUST IMAGINE THAT THERE ARE TWO INDEPENDENT PROCESSES; ONE ON AND ONE OFF.–r¨—-9 >> I THINK VI TO THINK ABOUT [INDISCERNIBLE] MORE. [LAUGHTER] I HAVE ARE GENERAL QUESTION. >> YES. >> SO YOU'RE TRYING TO RELATE THIS LEARNING TO REINFORCEMENT LEARNING. >> YES. >> AND IT'S A GENERAL FEELING ABOUT REINFORCE M LEARNING IS THE LEARNING IS BASED ON REWARD OR NO REWARD OR IF YOU GET THE REWARD ACTION YOU DID [INDISCERNIBLE], BUT IN YOUR CASE, THE SONG LEARNING, THERE IS TUTOR OR THE GOAL ALREADY GIVEN TO THE ANIMAL. SO IN THIS SENSE, IT DOESN'T SEEM TO BE REALLY REWARD OR NO REWARD, IT'S MORE LIKE TO REACH PARTICULAR GOAL. SO IN THE CORTICAL AREA YOU DESCRIBED PROJECTS TO THE ˇ¨ BTA, ENCODING SOME NOISE OR ERROR. IN THIS CASE, ERROR IS MORE LIKE PREDICTION ERROR, NOT REWARD PREDICTION ERROR, SOME EVENT PREDICTION ERROR. I WONDER IF DOPAMINE SYSTEMd8 IN THE BIRD IS INVOLVED IN MORE GENERAL PREDICTION ERROR RATHER THAN REWARD PREDICTION ERROR. >> I THINK THE RELATIONSHIP BETWEEN THE KINDS OF SIGNALS THAT WOULD CARRY INFORMATION ABOUT A SELF-EVALUATION OF YOUR PERFORMANCE OF A MOTOR ACTION AND REWARD SIGNAL [LOW AUDIO] IT'S A REALLY INTERESTING QUESTION. I DON'T KNOW THE ANSWER TO THAT QUESTION. THERE ARE SEVERAL WAYS THAT YOU MIGHT THINK ABOUT IT. THE WAY THAT I HAVE BEEN TENDING TO THINK ABOUT IT IS THAT I TAKE THE RESULTS FROM THE PRIMATE WORK AND I THOUGHT TO MYSELF, OKAY, THAT IS A MECHANISM THAT POTENTIALLY COULD BE USED BY THE SONG SYSTEM TO LEARN THE LONG IF THE ACTIVITY OF THOSE BTA NEURONS INDICATED NOT AN EXTRINSIC REWARD BUT THE RESULTS OF AN INTERNAL EVALUATION. SO I IMAGINE THAT THE AUDITORY SYSTEM IS COMPARING WHAT THE BIRD JUST SANG WITH THE TEMPLATE AND PRODUCING AN ONLINE SIGNAL THAT SAYS, ¨OKAY, THAT WAS BETTER THAN THE LAST TIME I SANG IT, OR THAT WAS WORSE THAN THE LAST TIME I SANG IT. THAT COULD BE THOUGHT OF AS A REWARD PREDICTION ERROR. YOUR PROTECTION IS HOW WELL DID DO IT LAST TIME? THAT'S MY PROTECTION. BETTER THEN MY SYSTEM INDICATES THAT. IF I DID WORSE, THEN IT INDICATES THAT. SO, VO’UM, I DON'T HAVE THE ANSWER TO HOW THOSE THINGS ARE RELATED TO EACH OTHER, BUT I THINK THAT YOU COULD IMAGINE THAT THAT SYSTEM IS CO-OPTIVE BY A COURT ¨ COSELF-EVALUATION SYSTEM THAT USES THAT SAME WATWAY TO THE BASAL GANGLIA TO DRIVE GOAL-DIRECTED LEARNING WHERE THE GOAL IS SOME INTERNAL EVALUATION. >>v: [LOW AUDIO]. >> YES, SO, UM, SO I THINK THE QUESTION OF HOW -- YOU'RE TALKING ABOUT CORTICOSTRIATAL LTP AND LTE, I PRESUME. SO -- OH, TO RA, WELL, OKAY.f‡ok THERE'S MUCH LESS KNOWN ABOUT LTP AND LTE IN SONGBIRD BRAIN AREAS. I THINK DAVID HAS DONE SOME NICE WORK SHOWING––K THAT THERE IS LP AND LTE IN AREA X, BUT I THINK THERE'S MUCH LESS THAN IN MAMMALS. I DON'T KNOW WHAT HAPPENED IFb YOU BLOCK THEM IN X, I WOULD THINK THE BIRD COU COULDN'T¨f‡ LEARN ANYMORE. THERE ARE TRANSGENIC BIRDS AVAILABLE, BUT THERE'S A BIG GAP BETWEEN PRODUCING A TRANSGENIC BIRD AND HAVING THE KIND OF TOOLS AND DRIVER LINES. THE ANSWER IS YES, IT'S HAPPENING, BUT IT WILL BE THERE EVENTUALLY. [APPLAUSE]