>>I WANT TO SAY THANK YOU TO EVERYBODY THAT HAS JOINED TODAY FOR THIS VERY -- WHAT I THINK IS GOING TO BE VERY EXCITING AND FRUITFUL WORKSHOP THAT BRAIN BEHAVIOR QUANTIFICATION AND SYNCHRONIZATION WORKSHOP. IT WILL BE DIFFERENT, VERY HEAVY ON DISCUSSION, SO WE'RE REALLY EXCITED ABOUT HAVING EVERYBODY GIVE THEIR QUESTIONS AND HAVING A VERY FRUITFUL DISCUSSION. WITH THAT I'M GOING TO TURN IT OVER TO OUR FIRST SPEAKER, DR. HOLLY LISANBY. >> WELCOME, EVERYONE, TO THE BRAIN INITIATIVE SUPPORTED WORKSHOP ON BRAIN BEHAVIOR QUANTIFICATION AND SYNCHRONIZATION. THANK YOU FOR JOINING US FOR WHAT WILL BE AN EXCITING AND FUN NEXT TWO DAYS. SO, MY ROLE IS TO SET THE STAGE AND DESCRIBE THE GOALS OF THE WORKSHOP. WHY IS BRAIN INITIATIVE SUPPORTING A WORKSHOP ON BRAIN BEHAVIOR QUANTIFICATION AND SYNCHRONIZATION? THE RATIONALE IS OUTLINED HERE. BASICALLY BEHAVIOR IS THE PRIMARY OUTPUT OF THE BRAIN. SO, TO REALLY UNDERSTAND THE NEURAL ORIGINS OF BEHAVIOR IT'S ESSENTIAL TO GOAL OF USING SCIENCE TO SUPPORT BRAIN HEALTH. BUT THIS IS A COMPLEX CHALLENGE BECAUSE UNDERSTANDING BEHAVIOR REQUIRES A DETAILED AND MULTI-DIMENSIONAL ANALYSIS OF A BROAD RANGE OF BEHAVIORS IN THE CONTEXT OF ENVIRONMENT. AND INTERACTION WITH THE ENVIRONMENT. NOW, OUR TOOLS FOR QUANTIFYING THE BRAIN SIDE OF THE EQUATION ARE QUITE ADVANCED AND ARE GETTING MORE ADVANCED THROUGH BRAIN INITIATIVE INVESTMENTS. WE'VE GOT GREAT TOOLS NOW FOR QUANTIFYING NEURAL ACTIVITY WITH VERY HIGH TEMPORAL AND SPATIAL RESOLUTION. BUT ON THE BEHAVIOR SIDE, TYPICALLY WE'RE MEASURING BEHAVIOR AT LOWER RESOLUTION. AND THIS CAN MAKE THE DISCOVERY OF THE CAUSAL LINKAGES BETWEEN BRAIN AND BEHAVIOR CHALLENGING. INDEED, THE TOOLS FOR MEASURING THE FULL RICHNESS OF SPECIES-APPROPRIATE BEHAVIORS IN ENVIRONMENTS AND SYNCHRONIZING THESE WITH SIMULTANEOUSLY RECORDED NEURAL ACTIVITY ARE PRESENTLY LACKING. THIS RESEARCH GAP WAS NOT LOST ON THE BRAIN 2.0 RECORD WHICH SPECIFICALLY CALLED FOR MORE SOPHISTICATED METHODS. THUS OUR TOPIC FOR TODAY'S WORKSHOP, BRAIN BEHAVIOR QUANTIFICATION AND SINK SYNCHRONIZATION. I'M GOING TO MAP OUT A CONCEPTUAL TERRAIN, IN BLACK FONT WHAT WE HAVE BUT COULD BE IMPROVED, PURPLE, WHAT WE DON'T HAVE, RESEARCH GAPS IN RED. WE HAVE HIGH RESOLUTION TOOLS. IN THE LABORATORY SETTING WE HAVE TOOLS FOR STUDYING BEHAVIOR BUT WE'RE STUDYING BEHAVIORS NOT NATURALISTIC AND THE MEASUREMENTS OFTEN FAIL TO CAPTURE THE FULL RANGE OF BEHAVIOR AND OFTEN FAIL TO CAPTURE THE ENVIRONMENT THE SUBJECT'S INTERACTING WITH. WE HAVE TOOLS CURRENTLY FOR LINKING NEURAL ACTIVITY RECORDED SIMULTANEOUSLY WITH BEHAVIOR IN THE LAB, BUT AS YOU'LL SEE TYPICALLY THESE TOOLS REQUIRE BULKY EQUIPMENT, LIMITS MOVEMENT OF SUBJECT, AND WHEN WE'RE TRYING TO MODEL THESE DATA THAT ARE MISMATCHED IN TERMS OF TEMPORAL RESOLUTION BETWEEN THE NEURAL SIDE AND BEHAVIOR SIDE, THIS POSES CHALLENGES FOR MODELING AS WELL. NOW, WE DO ALREADY HAVE TOOLS THAT ALLOW US TO TAKE EXPERIMENTS INTO NATURALISTIC SETTINGS AND CAPTURE BEHAVIORS THAT ARE ECOLOGICALLY VALID IN THE REAL WORLD BUT CURRENT TOOLS TYPICALLY HAVE ONLY A FEW CHANNELS OF RECORDING AND LIMITS IN TERMS OF RANGE OF MODALITIES THAT CAN BE SIMULTANEOUSLY SAMPLED. NOW, IN RED HERE IS A BIG RESEARCH GAP. WE'RE PRESENTLY LACKING PRACTICAL SYSTEMS THAT ALLOW US TO BRING NEURAL CIRCUIT RECORDING INTO THE REAL WORLD SETTING SO WE CAN CAPTURE WHAT THE BRAIN IS DOING WHEN IT'S GENERATING THESE BEHAVIORS IN THE REAL WORLD. NOW, I'VE ILLUSTRATED HERE AS A CLOUD THIS THING THAT'S DIFFICULT TO GET A HANDLE ON, INTERNAL SUBJECTIVE STATES. WE CURRENTLY LACK DIRECT OBJECTIVES MEASURES OF THESE AND OFTEN ARE LIMITED TO SELF-REPORT OF THE SUBJECT TELLING US WHAT THEY ARE EXPERIENCING OR PROXY BEHAVIORS WE THINK ARE RELATED TO INTERNAL STATE. AN IMPORTANT GAP BECAUSE MANY INTERNAL STATES ARE TOPICS OF SPECIFIC CLINICAL INTEREST, SUCH AS INTENTION, CRAVING, MOOD, THOUGHT, HUNGER, PERCEPTION, PAIN, REWARD, SUICIDAL INTENT, THINGS WE WOULD LIKE TO BE ABLE TO OBJECTIVELY STUDY, AND IN ABSENCE OF OBJECTIVES MEASURES OF THESE WE'RE CURRENTLY LACKING RELIABLE WAY OF TRAINING DECODERS THAT COULD USE NEURAL ACTIVITY TO INFORM US ABOUT THESE INTERNAL STATES. AND WITHOUT THESE, THIS HAS BEEN AN IMPEDIMENT FOR DEVELOPMENT OF INTERVENTIONS AS WELL. HERE'S SOME EXAMPLES OF STATE-OF-THE-ART TECHNOLOGIES THAT ARE CURRENTLY BEING USED IN RESEARCH. THIS, FOR EXAMPLE, IS A WEARABLE RECORDING OF IEEG IN SUBJECTS NAVIGATING A SPACE, AND YOU CAN SEE THAT AS THE SUBJECT IS WALKING AROUND THIS ROOM, THE CAMERAS ON THE CEILING AND TRACKER ON HIS HEAD IS ABLE TO TRACK WHERE IN SPACE HE'S LOCATED, AND TO DO THIS SIMULTANEOUSLY WITH BRAIN RECORDINGS ON THE RIGHT PART OF THE SLIDE. THIS SETUP WAS ABLE TO DEMONSTRATE WHEN THE SUBJECT HIMSELF WAS NAVIGATING THE ROOM, A SPECIFIC NEURAL SIGNATURE IS SEEN WHICH IS DIFFERENT THAN WHEN HE'S WATCHING SOMEONE ELSE NAVIGATE, ILLUSTRATING IMPORTANCE OF ENVIRONMENT, WHAT'S GOING ON AROUND THE SUBJECT. AS POWERFUL AS THIS TOOL IS, IT'S STILL LACKING IN SOME WAYS. IT'S ONLY MEASURING ONE ASPECT OF BEHAVIORAL RESPONSE, HAS LIMITED CAPTURE OF ENVIRONMENT AND SENSORY CUES, LIMITED TO THE LABORATORY ENVIRONMENT, AND LOOK AT THE BACKPACK HE'S WEARING. IT'S BULKY EQUIPMENT, AND HEAD-MOUNTED GEAR, IT ONLY TRACKS WHERE HE IS IN SPACE, NOT HIS ACTUAL BODY MOVEMENTS OR POSTURES. WE ALREADY HAVE SOFTWARE PLATFORMS SUCH AS DANCE ILLUSTRATED HERE WHICH ARE ABLE TO USE DEEP LEARNING TO DERIVE MEASURES OF NATURALISTIC BEHAVIOR REPERTOIRE WITHOUT HAVING TO PUT MARKERS ON THE SUBJECT. THIS IS A VERY EXCITING OPPORTUNITY TO DISCOVER THE NEURAL BASIS OF THESE NATURALISTIC BEHAVIORS, BUT STILL IT'S LACKING THE FULL RANGE OF BEHAVIORAL RESPONSE AND LACKS CAPTURE OF THE ENVIRONMENT AND SENSORY CUES THAT MAY INFLUENCE BEHAVIOR BEING MEASURED. NOW, LET'S GO BACK TO THE CONCEPTUAL MAP, WHAT WOULD IT MEAN FOR NEUROSCIENCE IF WE IS COULD HAVE SENSORS TO CAPTURE FULL RANGE OF BEHAVIOR IN THE ENVIRONMENT AND IN THE LAB, AND LINK THESE AT THE SAME TEMPORAL RESOLUTION WITH SIMULTANEOUSLY RECORDED NEURAL ACTIVITY AND DEPLOY NEXT GENERATION SENSORS IN REAL WORLD SETTINGS TO RECORD NATURALISTIC BEHAVIORS AND ENVIRONMENTS? WHAT IF WE COULD DEVELOP PORTABLE TOOLS FOR NEURAL RECORDING, IN REAL TIME, IN AMBULATORY SETTINGS TO LINK THESE NATURALISTICALLY RECORDED BEHAVIORS WITH WHAT THE BRAIN IS DOING AT THE TIME THAT IT'S GENERATING THOSE BEHAVIORS IN THOSE NATURAL ENVIRONMENTS. WE MIGHT HAVE THE POTENTIAL THEN TO USE THESE HIGH DIMENSIONALITY DATA STREAMS TO HAVE MORE ACCURACY IN OUR ABILITY TO INFER INTERNAL STATES FROM THESE QUANTIFIED BEHAVIORS. IF THIS COULD GIVE US A MORE OBJECTIVE MEANS OF INFERRING INTERNAL STATES OF INTEREST, LIKE MOOD DISORDER OR SUICIDAL INTENT, THIS THEN GIVES US THE OPPORTUNITY TO TAG NEURONAL ACTIVITY SIMULTANEOUSLY RECORDED DURING INTERNAL STATES OF INTEREST TO BEGIN TO BE ABLE TO DECODE THESE INTERNAL STATES FROM THE NEURAL ACTIVITY. IF WE HAD THAT, THEN WE MIGHT BE ABLE TO BUILD CLOSED LOOP SYSTEMS THAT WOULD BE ABLE TO MODULATE NEURAL ACTIVITY WITH BEHAVIOR ACTUALLY IN THE LOOP. NOW, WHY DO WE THINK THE TIME IS RIGHT TO HAVE THIS DISCUSSION? WE'RE CURRENTLY EXPERIENCING EXPLOSION OF NEW AND ADVANCED STATIONARY AND WEARABLE TECHNOLOGIES, PLATFORMS TO INTEGRATE MULTI-MODAL DATA, TO USE ADVANCED TOOLS OF DATA SCIENCE TO MAKE SENSE OF THESE DATA, AND TO PACKAGE THIS IN A WAY THAT ALLOWS DATA VISUALIZATION AND ANALYSIS. SO THERE'S A LOT OF NEW DEVELOPMENTS THAT HAVE NOT YET BEEN EXPLOITED FOR NEUROSCIENCE APPLICATIONS. THIS GIVES POTENTIAL TO USE NEXT GENERATION SENSORS SUCH AS FOR SELF-DRIVING CARS TO HELP US MEET UNMET NEEDS IN BRAIN HEALTH. NOW, TALKING ABOUT DATA, WE KNOW THAT INTERNAL STATES CAN GENERATE A KALEIDOSCOPE OF DIFFERENT TYPES OF BEHAVIORS THAT CAN BE MEASURED USING A VARIETY OF ARRAYS OF SENSORS LIKE VIDEOGRAPHY, THERMAL SENSORS, PIEZOSENSORS, GENERATING HIGH DIMENSIONALITY DATA, AN OPPORTUNITY FOR DATA DRIVEN MODELING, THEORY BUILDING AND TESTING. THIS APPROACH COULD BE SYNERGISTIC WITH THE BRAIN INITIATIVE'S INVESTMENTS IN INFORMATICS, IN THE DEVELOPMENT OF NEW DATA ARCHIVES, DATA STANDARDS, DATA INTEGRATION APPROACHES, SOFTWARE DEVELOPMENT, SECONDARY DATA ANALYSIS. SPEAKING OF INFORMATICS, WE HAVE AN OPPORTUNITY TO LEVERAGE EXISTING MULTIMODAL DATA ARCHIVES SUCH AS NEURODATA WITHOUT BORDERS, AND THE DISTRIBUTOR ARCHIVES FOR NERVE PHYSIOLOGY DATA INTEGRATION WHICH ALREADY ACCEPTS BEHAVIORAL DATA STREAMS. WE HAVE THE OPPORTUNITY TO BUILD ON ALREADY EXISTING DATA STANDARDS THAT BRAIN INITIATIVE HAS INVESTED IN FOR CATALOGING BEHAVIORAL DATA AND STANDARDIZING THE WAIT WE DEFINE BEHAVIORAL TASKS, SUCH AS BEHAVIORAL TASKS DESCRIPTION LANGUAGE. AND IT'S AN OPPORTUNITY TO EXPAND ON BRAIN INFORMATICS PORTFOLIO BY DEVELOPING DATA ARCHIVES STANDARDS AND SOFTWARE TOOLS THAT ARE CURRENTLY LACKING, SPECIFICALLY FOR CROSS-MODAL DATA FUSION, THAT INCORPORATION INFORMATION ABOUT THE ENVIRONMENT AS WELL. AND WE HAVE AN OPPORTUNITY TO GENERATE A COMMON DATA ECOSYSTEM ACROSS THESE DATA STREAMS WITH PATHWAYS FOR DISSEMINATION. SO HERE'S AN EXAMPLE OF PROOF OF CONCEPT OF WHAT A BRAIN BEHAVIOR QUANTIFICATION SYNCHRONIZATION PROJECT MIGHT LOOK LIKE. WE'LL HEAR FROM DR. PROVENZA LATER IN THE WORKSHOP. THE GOAL WAS TO TRAIN AN ADAPTIVE DEEP BRAIN STIMULATION SYSTEM FOR TREATMENT OF OBSESSIVE COMPULSIVE DISORDER, STARTING WITH SIMULTANEOUS NEURAL RECORDINGS WHILE DIFFERENT BEHAVIORS ARE BEING SENSED BOTH IN THE CLINIC AS WELL AS AT HOME, AND TO BE ABLE TO HAVE THIS HIGH TEMPORAL RESOLUTION LINKAGE BETWEEN WHAT THE BRAIN IS DOING WHEN IT'S GENERATING THE BEHAVIORS IN THESE DIFFERENT CONTEXTS. AND TO REALLY PUT BEHAVIOR IN THE LOOP TO TRAIN THIS ADAPTIVE DBS SYSTEM, HERE'S THE EXAMPLE OF DATA STREAMS THAT CAN BE CAPTURED WITH THIS APPROACH. WE'VE GOT COMPUTER VISION TO CAPTURE MICROMOVEMENTS OF THE FACE, FOR AFFECT, WE'VE GOT AUTONOMIC RECORDING, EEG, YOU CAN AMPLITUDE AND SEE HOW BEHAVIORS RESPOND. WITH THIS ENRICHED MULTIMODAL DATA STREAM IT GIVES US THE OPPORTUNITY TO BEGIN TO INFER INTERNAL STATES FROM THIS QUANTIFIED BEHAVIOR AND USE THAT INFORMATION TO TRAIN ADAPTED DBS SYSTEMS THAT WOULD BE ABLE TO MODULATE STATE AND DISRUPT THE MALADAPTIVE BEHAVIORS WHEN AND WHERE THEY OCCUR. THIS BRINGS US TO THE CONCEPT OF BRAIN BEHAVIOR QUANTIFICATION AND SYNCHRONIZATION. WE PUBLISHED THIS CONCEPT, THE GOALS ARE TO DEVELOP HIGH RESOLUTION TOOLS AND PLATFORMS, TO PRECISELY QUANTIFY BEHAVIOR, TO SYNCHRONIZE WITH SIMULTANEOUSLY RECORDED BRAIN ACTIVITY. ANOTHER GOAL IS TO USE THIS DATA AND THESE TOOLS TO BUILD NEW CONCEPTUAL AND COMPUTATIONAL MODELS OF BEHAVIORAL SYSTEMS, AND USE THOSE MODELS TO TEST CAUSAL RELATIONSHIPS IN BRAIN AND BEHAVIOR. ANOTHER GOAL TO ESTABLISH CROSS-DISCIPLINARY CONSORTIUM TO DEVELOP AND DISSEMINATE TOOLS, ONTOLOGIES, RESEARCH DESIGNS, IMPORTANTLY ETHICAL FRAMEWORKS THAT COULD HAVE THE POTENTIAL OF TRANSFORMING HOW MECHANISTIC BRAIN BEHAVIOR RESEARCH IS CONDUCTED AND ESTABLISHMENT OF CONSORTIA IS PART OF THE GOAL FOR OUR WORKSHOP TODAY. OUR ENVISION IMPACT OF THIS CONCEPT COULD BE THE DIGS DISSEMINATION, HOW WE UNDERSTAND HOW THE BRAIN GENERATES BEHAVIOR, DEVELOPMENT OF NEW PARADIGMS TO ESTABLISH CAUSAL RELATIONSHIPS BETWEEN NEURAL ACTIVITY AND BEHAVIOR, ULTIMATELY THIS COULD ENABLE THE DEVELOPMENT OF THERAPEUTIC INTERVENTIONS FOR COMPLEX NEURAL BEHAVIORAL DISORDERS THAT CURRENTLY LACK TREATMENTS, AND WE WOULD LIKE TO SEE A ROBUST ETHICAL FRAMEWORK THAT ADDRESSES THE IMPLICATIONS OF USING THESE TOOLS TO CAPTURE AND RECORD A PERSON'S EXPERIENCE. NOW, THERE ARE TWO CASE EXAMPLES I'M GOING TO WALK THROUGH AND INTRODUCE THE FORMAT OF THE WORKSHOP. FIRST USE CASE EXAMPLE WHEN THE BEHAVIOR IS THE TARGET OF STUDY, AND THIS IS THE MOST OBVIOUS USE CASE SITUATION, BUT WE MIGHT NEED NEW TOOLS TO DISCOVER THE NEURAL BASIS OF THE BEHAVIOR WHICH MIGHT BE CHALLENGE BE TO STUDY FOR A NUMBER OF REASONS, MIGHT BE THAT THE BEHAVIOR OCCURS IN REAL WORLD SETTING OUTSIDE THE LAB OR COULD BE BEHAVIOR OCCURS TOO INFREQUENTLY AND REQUIRES AMBULATORY MONITORING TO CAPTURE OR THE FREQUENCY AT WHICH THE BEHAVIOR OCCURS OVER EXTENDED PERIODS OF TIME IS ITSELF SALIENT SUCH AS IN THE CASE OF SEIZURES OR BINGE EATING. EXAMPLE OF APPLYING COMPUTER VISION TO CAPTURE AND QUANTIFY NATURALISTIC BEHAVIOR, IN THIS CASE SPONTANEOUS FLY BEHAVIOR, BUT THE CONCEPT WOULD BE TO DO THIS ACROSS DIFFERENT SPECIES IN DIFFERENT SETTINGS. THE POTENTIAL IMPACT COULD BE ADVANCING OUR UNDERSTANDING OF HOW THE BRAIN GENERATES THESE COMPLEX BEHAVIORS IN THE CONTEXT OF ENVIRONMENT, PARTICULAR COMPLEX ENVIRONMENTS, AND ALSO ENABLING THE DEVELOPMENT OF INTERVENTIONS WHETHER CLOSED LOOP OR OTHERWISE, WHEN THE BEHAVIOR ITSELF IS THE TARGETED THERAPY. WHAT ABOUT A DIFFERENT USE CASE SCENARIO, IF IT'S NOT THE BEHAVIOR THAT'S THE TARGET BUT THE INTERNAL STATE THAT'S THE TARGET OF STUDY? IN THIS CASE, BEHAVIORS ARE BEING USED AS MARKERS OF INTERNAL SUBJECTIVE EXPERIENCE THAT CANNOT BE MEASURED DIRECTLY LIKE A THOUGHT PROCESS, AFFECTIVE STATE OR PERCEPTION. HERE THE SUBJECTIVE EXPERIENCE FOR WHICH THE OBJECTIVELY MEASURED BEHAVIORS SERVE AS A MARKER IS THE INTENDED TARGET OF THE STUDY, NOT THE BEHAVIORS THEMSELVES. HERE'S EXAMPLE OF USING COMPUTER VISION AGAIN, TO QUANTIFY DYNAMIC MICROMOVEMENTS OF THE FACE OVER TIME AND USE THESE TO INFER AFFECTIVE -- INTERNAL AFFECTIVE STATE AND THIS COULD BE LINKED WITH SIMULTANEOUS NEURAL RECORDING AND STIMULATION AS YOU'LL SEE FROM SOME OF OUR SPEAKERS IN THE WORKSHOP. THE POTENTIAL IMPACT HERE COULD BE TO ADVANCE OUR UNDERSTANDING OF HOW THE BRAIN GENERATES SUBJECTIVE EXPERIENCES AND COULD ENABLE THE DEVELOPMENT OF INTERVENTIONS WHEN THE SUBJECTIVE EXPERIENCE IS THE THERAPEUTIC TARGET. NOW, THE GOALS OF THE WORKSHOP ARE TO CONVENE A DIVERSE PANEL OF EXPERTS FROM A VARIETY OF FIELDS THAT MAYBE DON'T OFTEN TALK TO EACH OTHER. WE'VE GOT NEUROSCIENTIST, DATA SCIENTISTS, SENSORY DEVELOPMENTS, BEHAVIORAL EXPERTS, ENGINEERS, WE'VE ASKED THEM TO ENGAGE IN CROSS-DISCIPLINARY DIALOGUE. WE'VE ASKED THEM TO HELP US HIGHLIGHT WHERE OPPORTUNITIES ARE, WHERE RESEARCH COULD MAKE THE MOST IMPACT. WE'VE ASKED THEM TO IDENTIFY WHERE THE NEEDS ARE, WHERE NEW TECHNOLOGY NEEDS TO BE DEVELOPED. AND WE HOPE THIS WILL BE THE START OF THE FORMATION OF CONSORTIA TO STIMULATE PROGRESS IN DEVELOPING NEW BEHAVIOR ONTOLOGY, TECHNOLOGY, INFORMATICS, COMPUTATIONAL TOOLS AND ETHICAL FRAMEWORKS. SO, I'M GOING TO GIVE YOU A BIRDSEYE VIEW, THE PANEL IS SPEAKERS AND DISCUSSANTS WITH A MODERATOR, AND IN THIS FIRST PANEL FORMATTED LIKE A CONCEPTUAL DATA BLITZ, CONCEPTUAL BLITZ, WE'VE ASKED SPEAKERS AND DISCUSSANTS TO THINK THROUGH HOW DO WE FRAME OUR RESEARCH QUESTIONS TO STUDY BEHAVIOR AS A COMPLEX DYNAMIC SYSTEM, IN INTERACTION WITH A COMPLEX ENVIRONMENT. OUR SECOND PANEL WILL BE ON SENSING BEHAVIOR IN ITS ENVIRONMENTS, THE DIMENSIONS AND DYNAMICS. AGAIN WE'VE GOT FOUR SPEAKERS, THREE DISCUSSANTS. WE'VE ASKED THEM TO THINK ABOUT HOW DO WE CAPTURE THE INFORMATION NEEDED FROM INDIVIDUAL ORGANISMS, SOCIAL GROUPS, AND PHYSICAL ENVIRONMENTS, THAT WE NEED TO CAPTURE THE COMPLEXITY OF BEHAVIOR, PARTICULARLY IN REAL WORLD SETTINGS. AND THEN THE THIRD PANEL WILL BE ON WHAT ARE THE COMPUTATIONAL OPPORTUNITIES AND CHALLENGES FOR MODELING BRAIN BEHAVIOR ENVIRONMENT, AS A COMPLEX DYNAMIC SYSTEM. MOVING, CLOSING OUT DAY 1, WE'VE ASKED CAROL ROMMELFANGER AND JUSTIN BAKER TO PROVIDE SYNTHESIS AND CLOSING REMARKS FROM THE THREE PANELS ACROSS DAY ONE, TO HIGHLIGHT WHERE THE OPPORTUNITIES AND CHALLENGES MELEE. DAY 2 IS GOING TO START WITH THREE REALLY EXCITING TALKS, WHICH WILL INTEGRATE THE CONCEPTS FROM DAY 1 TO ADVANCE NEUROSCIENCE, AND WE HAVE MALA MURTHY, NANTHIA SUTHANA AND NITA FARAHANY. AFTER THE THREE TALKS WE'LL HAVE CONCURRENT MODERATED BREAKOUT SESSIONS, THE FIRST ONE IS ON SHALL IT - I'LL BE MODERATING ON HUMAN AND CLINICAL RESEARCH WITH OUR BREAKOUT CO-LEADS, AND THEN THE SIMULTANEOUS BREAKOUT WILL BE MODERATED WITH DISCUSSANTS. IF YOU CAN'T DECIDE, THEY ARE BOTH RECORDED, BOTH WILL BE STREAMED AFTER THE MEETING. SO, THEN OF COURSE WE'RE REALLY EXCITED TO BE WELCOMING DR. JOHN NGAI, TO GIVE CLOSING REMARKS AND SYNTHESIS AFTER WE HAVE THE REPORT OUT FROM THE GROUPS AND DISCUSSION. IT WILL BE A REALLY EXCITING TWO DAYS AND WE'LL BE LOOKING FORWARD TO HEARING FROM OUR BREAKOUT GROUPS, WHAT ARE THE HIGH PRIORITY AREAS OF RESEARCH TO PURSUE, WHAT ARE THE TOOLS NEEDED TO PURSUE THEM, AND WHAT ARE THE BARRIERS THAT WE NEED TO ADDRESS. THERE'S SO MANY PEOPLE TO THANK HERE, AND WE'VE LISTED THE PLANNING COMMITTEE AND MEMBERS WHO HELPED TO ORGANIZE THIS MEETING ON THIS SLIDE HERE. THIS IS A COLLABORATION ACROSS MULTIPLE NIH INSTITUTES AND OFFICES AS WELL AS NSF. TO SET US UP FOR PANEL 1, WHICH IS GOING TO BE ON BEHAVIOR AS A COMPLEX SYSTEM WITHIN A COMPLEX SYSTEM, WE'LL HAVE FOUR EIGHT-MINUTE TALKS FOLLOWED BY DISCUSSION, EACH DISCUSSANT WILL INTRODUCE AN IDEA OR QUESTION, GENERATED BY THE TALKS, AND LEAD DISCUSSION FOR NINE MINUTES. THEN THEY WILL HAND OFF TO THE NEXT DISCUSSANT. WORKSHOP PARTICIPANTS ARE INVITED TO USE THE CHAT TO POSE QUESTIONS FOR THE DISCUSSION, AND OTHER ATTENDEES PLEASE USE THE Q&A BOX OR LIVE FEEDBACK FUNCTIONS IN ZOOM OR VIDEOCAST. WITHOUT FURTHER ADO I'D LIKE TO INTRODUCE DR. JANINE SIMMONS, OUR MODERATOR FOR PANEL 1. >> THANKS, HOLLY. I'M EXCITED TO BE LAUNCHING OUR PANEL 1. I'M SUPER EXCITED. THE PANEL 1 AS HOLLY SAID IS ENTITLED BEHAVIOR AS A COMPLEX SYSTEM WITHIN A COMPLEX SYSTEM. AND QUESTIONS ARE RESEARCH QUESTIONS TO STUDY BEHAVIOR AS A COMPLEX SYSTEM. SPEAKERS ARE IAIN COUZIN, JEFFREY KOEN AND LENA TING, DISCUSSANTS MOLLY CUMMINGS, TIMOTHY WRIGHT AND BENCE …LVECZKY. >> I HAVE TO APOLOGIZE. MY FAMILY HAD COVID, I THOUGHT I WAS GOING TO ESCAPE BUT HAD REBOUND SYMPTOMS AND TESTED POSITIVE. EXCUSE THE VOICE, COUGHING, FATIGUE, SO ON. I COULDN'T MISS THIS AMAZING EVENT. YES, I WAS INTRIGUED BY BEHAVIOR AS A COMPLEX SYSTEM WITHIN A COMPLEX SYSTEM BECAUSE I THINK THE NEUROSCIENCE OF COURSE IS DEVELOPING INCREDIBLE TOOLS WE JUST HEARD ABOUT THAT SPAN ACROSS THESE VAST SPATIAL AND ALSO TEMPORAL SCALES. I LOVE THE PIECE BY BASSETT AND MORROW, EXTENDS TO EVOLUTIONARY SCALES. BEHAVIOR DOESN'T MAKE SENSE IF WE CONSIDER HOW THESE FEEDBACK ON EACH OTHER. MULTI-SCALE IMAGING IS SO CRITICAL. ANIMAL BEHAVIOR, ONE OF THE WONDERFUL CHALLENGES IS THAT IT INHERENTLY SPANS SCALES, ANIMALS NEED TO RESPOND TO A RISK OR THREAT, THEY HAVE TO RESPOND IN TERMS OF FOOD RESOURCES, OVER YEARS IN TERMS OF WITH WHOM THEY CHOOSE TO MATE. AND THERE ARE MANY FEEDBACK PROCESSES ACROSS THE SCALES. THEY CAN'T BE DECOUPLED IN A SIMPLE WAY. WHO YOU CHOOSE TO MATE WITH IMPACTS MANY OTHER ASPECTS OF YOUR LIFE. AND WE'VE ALREADY SEEN THIS. I DIDN'T SEE THE PREVIOUS TALKS, JUST COINCIDENCE, THIS WONDERFUL WORK FROM TIMOTHY DUNN, CHARACTERIZING A BODY OF WORK TO TRY TO DEVELOP QUANTITATIVE TOOLS TO STUDY AND TO DESCRIBE STEREOTYPICAL BEHAVIORS. THIS IS TRANSFORMATIVE TO THE WHOLE FIELD OF BEHAVIOR. IN FACT, I THINK BEHAVIOR IS UNDERGOING A RENAISSANCE. WHEN I WAS A POSTDOC AT OXFORD UNIVERSITY, IT WAS THOUGHT BEHAVIOR WAS REDUNDANT, WE DID IT IN THE 70s AND '80s, NOW WE'RE REALIZING MECHANISM IS FUNDAMENTAL TO UNDERSTANDING HOW THE BRAIN PROCESSES INFORMATION. BUT YET WE'RE STILL ONLY AT THE VERY BEGINNING OF OUR STUDIES. I THINK THAT'S THE EXCITING THING. SO, INSTEAD OF TALKING MY ABOUT MY WORK, I'LL MENTION SOME BUT WANTED TO REALLY TALK ABOUT WHAT I'VE BEEN THINKING ABOUT LATE LAST NIGHT, PREPARING MY SLIDES, ABOUT WHAT ARE MAJOR CHALLENGES AND GAPS IN THE STUDY OF BEHAVIOR. SOME OF THIS WILL REFLECT WHAT YOU JUST HEARD BECAUSE I THINK WE'RE INDEPENDENTLY THINKING THE SAME TYPES OF THINGS. SO, WE'RE GETTING PRETTY GOOD AT DOING POSTURE ANALYSIS AND DESCRIBING STEREOTYPICAL PATTERNS. WHAT WE'RE NOT GOOD AT DOING, AND I WOULD LOVE TO HEAR MORE FROM OTHER PARTICIPANTS, HOW ANIMALS TAKE IN HIGH DIMENSIONAL SENSORY INFORMATION, MULTI-MODAL, THE BRAIN IS THIS ENTITY THAT ALLOWS ANIMALS TO MAKE SENSE OF THIS, TO MAKE DECISIONS, BASED ON MEMORY, PHYSIOLOGIC STATE, GENETICS, AS WAS MENTIONED BEFORE BEHAVIOR CAN ACTUALLY GIVE INSIGHT INTO THESE INTERNAL STATES. BUT WE STILL REALLY LACK THIS FORMAL WAY, INFORMATIONAL THEORETIC TOOLS, TO ALLOW US TO CAUSALITY. CORRELATION, YES, BUT CAUSALITY IS STILL CHALLENGING. ONE OF THE REASONS, I'LL COME BACK TO THIS, WE HAVE THESE RECURSIVE FEEDBACKS ACROSS MULTIPLE TIME SCALES. SO I THINK THAT'S A VERY IMPORTANT ASPECT THAT WILL IMPROVE ALL OF OUR RESEARCH. ANOTHER ASPECT I'VE BECOME VERY INTERESTED IN, I THINK, WEAK LINK, OFTEN MOVEMENT IS CONSIDERED THE OUTCOME OF THE DECISION, ANIMAL DECIDES WHAT IT WANTS TO DO, MOVES TO GET TO LOCATION IT WANTS. BUT THE MOVEMENT, THERE'S A DYNAMIC FEEDBACK BETWEEN MOTION SENSING AND DECISION MAKING AND OTHERS BUT THIS IS YET A VERY SMALL FIELD IN BODY DECISION MAKING, IT'S VERY IMPORTANT TO CONSIDER THE BRAIN AS EMBODIED IN THE ORGANISM TO UNDERSTAND SPATIAL TEMPORAL COMPUTATION. TIMOTHY IS IN THE AUDIENCE, HE'S DONE WONDERFUL WORK THAT SHOWS THIS TOO. I DON'T HAVE TIME TO TALK ABOUT MY RESEARCH BUT WANT TO EMPHASIZE WE'VE BEEN REALLY LOOKING INTO THIS IN A COMPREHENSIVE WAY WITH A RANGE OF SYSTEMS, AND SHOWING WE SIMPLY CANNOT MAKE SENSE OF THE DECISIONS ANIMALS MAKE, WITHOUT INCLUDING THIS RECURSIVE FEEDBACK, THIS REVEALS THE BRAIN IS BEHAVING AS DYNAMIC SYSTEM WITH BIFURCATION ALSO THAT BREAK UP THE COMPLEXITY OF THE WORLD INTO A SERIES OF BINARY, BINARY, BINARY DECISIONS. SO I THINK THIS COULD INTEGRATE VERY WELL WITH CLASSICAL NEUROSCIENCE IN TERMS OF DECISION MAKING BECAUSE WE FIND THE BRAIN CONTINUOUSLY CHANGES THE WORLD, NO MATTER HOW MANY OPTIONS, INTO A SERIES OF TWO OPTION DECISIONS. WE COMPARE THEORY WITH EXPERIMENTS. NO ANIMAL IS AN ISLAND. SOCIAL BEHAVIOR HAS BEEN POORLY DESCRIBED AND POORLY CONSIDERED, I GUESS, WITHIN THIS TYPE OF FRAMEWORK. ALMOST ALL DECISIONS ANY ANIMALS MAKE INVOLVES INDIRECT OR DIRECT SOCIAL INFORMATION. LOOKING AT CLASSIC MODELS, SORT OF MODELS THAT WE CAN USE IN OUR COMMUNITY, C. ELEGANS IS A HIGHLY SOCIAL ANIMAL. SO ARE DROSOPHILA, ODOR AVOIDANCE, WE ALSO USE SOCIAL INTEGRATION TO FORAGE, CHOOSE MATES, SO ON. BUT THOSE ORGANISMS, DROSOPHILA AND C. ELEGANS, ARE INCREDIBLY HARD TO STUDY IN NATURAL ENVIRONMENT. WE HAVE AT LEAST TWO MODEL SYSTEMS PERFECT FOR CROSSING SCALES BETWEEN NATURAL ENVIRONMENT AND SOCIAL COMPLEXITY AND CONTROLLED EXPERIMENT, ZEBRAFISH BEING ONE, THEY HAVE TERRITORIAL BEHAVIOR, THEY HAVE RICH FIGHTS THAT LEAD TO TERRITORIES, SCHOOLING OF COURSE. RATS, WE WERE ONE OF THE FIRST GROUPS TO EVER ASK THIS BASIC QUESTION, WHAT HAPPENS IF YOU PUT TWO RATS IN A MAZE? OR EIGHT RATS IN A MAZE? THEY SOLVE THIS MAZE COLLECTIVELY WITH SOME WONDERFUL EXAMPLES OF HOW INTERACTIONS CAN LEAD TO COGNITIVE ABILITIES BEYOND THE INDIVIDUAL. AND YET I'M GOING TO BE A LITTLE BIT DRAMATIC NOW, PUSH YOU A LITTLE BIT. I WOULD ARGUE THE BEHAVIOR OF THES ORGANISMS IS ALMOST ALWAYS STUDIED IN ABSENCE OF CONTEXT. BY DOING SO WE SEVERELY RESTRICT AND BIAS THE BEHAVIORS WE STUDY. ALSO LIMIT THE SCOPE OF OUR STUDIES BUT MEANS WE CAN MISUNDERSTAND DATA. IT BECOMES EXTREMELY HARD WITH EVOLUTIONARY PROCESS, ULTIMATELY WHAT MATTERS. EXAMPLE IS COLLECTIVE MIGRATION. PEOPLE PUT TAGS ON INDIVIDUAL ANIMALS, WHAT THE INDIVIDUAL IS DOING EVERYTHING THEMSELVES. BUT ALMOST ALL MIGRATIONS ARE KNOWN TO BE COLLECTIVE. SO WHY IS STUDYING COLLECTIVE BEHAVIOR CHALLENGING? WELL, IT'S VERY HARD TO DETERMINE CAUSAL RELATIONSHIPS. THIS IS ANOTHER AREA THAT WE NEED TO HAVE DEVELOPMENTS IN. EVEN IF WE HAVE TWO ANIMALS, THEN THINK OF THE THIRD ANIMAL, IS HE BEING INFLUENCED OR A, OR A AND C VIA INDIRECT EFFECT, AND CAN ADD MORE COMPLEX RELATIONSHIPS. WE'VE BEEN TRYING TO STUDY IN SCHOOLING FISH AND FLOCKING BIRDS, EXTREMELY CHALLENGING TO YOU CAN WORK OUT CAUSAL NATURE OF INFLUENCE. TOWARDS THE SOLUTION, SO I REALLY AM EXCITED BY DEVELOPING VIRTUAL REALITY ENVIRONMENTS, IMMERSEIVE HOLOGRAPHIC, WHERE INDIVIDUALS CAN INTERACT FREELY WIN VIRTUAL WORLDS. AND WE CAN NETWORK SYSTEMS TOGETHER. YOU CAN SEE AN INDIVIDUAL -- VIRTUAL INDIVIDUAL MOVING IN THE CIRCLE, THE REAL ZEBRAFISH FOLLOWING, BELIEVES IT TO BE IN THE TANK WITH IT EVEN THOUGH THIS IS AN ILLUSION. WE HAVE TO PROJECT THE WORLD PRECISELY FROM PERSPECTIVE OF THE ANIMAL, WE CAN CREATE MULTIPLE SYSTEMS AND NETWORK TOGETHER AND ALLOW REAL INDIVIDUALS TO INTERACT WITH EACH OTHER, NO T IN THE SAME PHYSICAL WORLD BUT SAME HOLOGRAPHIC WORLD, GIVES GREAT CONTROL TO UNDERSTAND WHAT'S GOING ON. WE HAVE NOW 11 OF THESE SYSTEMS HERE, YOU CAN SEE FOUR INDIVIDUALS. THEY ARE NOT INTERACTING IN THE REAL PHYSICAL WORLD, WITHIN A HOLOGRAPHIC WORLD. AND WORKING WITH HARVARD UNIVERSITY WE CAN CREATE A STRUCTURAL ENVIRONMENT WITH A MICROSCOPE, WHOLE BRAIN IMAGING AT CELLULAR RESOLUTION, WITH ANIMALS INTERACTING WITH THE OTHER HOLOGRAPHIC, ONE COULD BE IN THE U.S., WITH THE BANDWIDTH. WE'VE BEEN LOOKING AT ZEBRAFISH AS A MODEL FOR AUTISM SPECTRUM DISORDER. THERE'S A MASSIVE GAP IN OUR STUDIES OF BEHAVIOR. ALMOST ALL STUDIES FOCUS IN CONTROLLED SIMPLISTIC LABORATORY ENVIRONMENTS. MICROSCOPIC, IF YOU GO BACK TO THE BEGINNING OF MY TALK. ALL IN THE UNCONTROLLED CONDITIONS OF THE WORLD, MACROSCOPIC, I'M INVOLVED IN BOTH COMMUNITIES, VERY FEW PEOPLE ARE. SO WE HAVE THIS BEAUTIFUL WORK BY TIMOTHY AND OTHERS OF THE MICROSCOPIC SCALE, WORK WE'VE BEEN DOING, SO THIS IS A PAPER, STATE-OF-THE-ART TECHNOLOGY IN THE WILD TO TRACK ANIMALS AS THEY MIGRATE, STORKS IN SUB-SAHARAN AFRICA, BUT NOTHING IN THE MIDDLE. MESOSCOPEIC IS VOID, A BARREN LAND. WE DON'T MAKE THESE CONNECTIONS, AS IMPORTANT AS ANYTHING ELSE. WHEN I BUILT THIS, THE CONCEPT WITH THE LARGE INDOOR FACILITY, THAT ALLOW US TO BEGIN TO ADDRESS THIS. YOU HAVE RATS CHOOSING BETWEEN TWO OPTIONS, THOUSANDS OF TIMES, ALLOWING THEM TO MOVE WITHIN LARGE ENVIRONMENTS, WHILE DOING BRAIN IMAGING. ALSO THIS IS ANOTHER WHERE WE CAN ALSO DO THE WORK AND LOOK AT THE ACOUSTIC COMMUNICATION. >> I'M GOING TO BE TALKING ABOUT BEHAVIOR IN HUMANS. AND HOW BEST TO MEASURE BEHAVIOR. COMMONLY USED TO SELF-REPORT, AS A NUMBER OF ADVANTAGES, IT'S CONVENIENT, IT'S FAST, ALLOWS FOR QUANTIFICATION, IT'S EASILY OBTAINED. AND THERE ARE MANY RELIABLE QUESTIONNAIRES TO CHOOSE FROM. THE DISADVANTAGE OF COURSE IS THAT IT REQUIRES ABILITY TO SPEAK, WRITE, OR GESTURE. THAT CAN'T BE DONE IN YOUNG CHILDREN. IT CAN'T BE DONE IN PEOPLE WHO ARE ON BREATHING TUBES OR WHO ARE COMMUNICATIONALLY CHALLENGED. IT'S IDIOSYNCRATIC. THE MEANING OF THE MEASURES KIND OF VARY WITH THE PERSON. IT'S LIMITED TO WHAT IS CONSCIOUSLY AWARE. IT'S SUSCEPTIBLE TO SUGGESTION, IMPRESSION, DECEPTION. TEMPORARILY SPARSE, YOU CAN ONLY ASK SO MANY QUESTIONS IN ANY PERIOD OF TIME AND LACKS ANALOGS IN OTHER SPECIES. SO HOW ELSE COULD MEASURE BEHAVIOR? FACIAL EXPRESSION IS ONE WE'VE BEEN ESPECIALLY INTERESTED IN. FACE, HEAD, AND BODY MOTION PRESENTS ANOTHER. VOICE, VOICE QUALITY, TIMING. AND THEN PHYSIOLOGY WHICH CAN BE MEASURED EITHER WITH THERMAL CAMERA OR CONTACT SENSORS, ALL OF THESE MODALITIES HAVE BEEN WELL STUDIED IN PEOPLE AND IN DEVELOPMENTALLY AND CROSS-CULTURALLY. MY OWN APPROACH HAS EMPHASIZED VISUAL FACIAL EXPRESSION. AND THAT'S WHAT I'LL FOCUS ON TODAY. FACIAL EXPRESSION IS MADE POSSIBLE BY A DENSE MESH OF FACIAL MUSCLES, UNLIKE MUSCLES ANYWHERE ELSE IN THE BODY, THEY CONNECT EITHER TO EACH OTHER OR THEY CONNECT TO SOFT TISSUE. THIS GIVES A HIGH DEGREE OF FREEDOM, HIGH DEGREES OF FREEDOM FOR FACIAL EXPRESSION. THE APPROACH WE USE, ANNOTATION, IS BASED ON ECKMAN'S ENCODING SYSTEM, IN THAT SYSTEM EACH FACIAL -- EACH MUSCLE OR SMALL NUMBER OF MUSCLES IS REFERRED TO AS AN ACTION UNIT. AND THESE HAVE HIGH DEGREE OF CONTROL. WE ARE INTERESTED IN THE OCCURRENCE, THE INTENSITY, AND TIMING OF FACIAL ACTION. WE CAN TRACK CHANGE IN EXPRESSION AND DESCRIBE IT IN TERMS OF MOLECULAR MOVEMENTS. THE APPROACH THAT WE USE, DEVELOPED IN COLLABORATION, IS AUTOMATED FACIAL ACTION RECOGNITION WHICH WE REFER TO AFAR. THIS IS ONE VERSION OF THE SYSTEM, THERE ARE A NUMBER OF VERSIONS, YOU KNOW, THAT WE'VE BEEN USING, IS A COMPUTER VISION MACHINE-LEARNING BASED APPROVE, TO MEASURING FACIAL ACTIONS AND HEAD AND FACE DYNAMICS. IT'S BEEN TRAINED ON LARGE AND HETEROGENEOUS DATA, GENERALIZABLE TO PARTICIPANTS SUCH AS I'LL BE TALKING ABOUT WITH DEEP BRAIN STIMULATION. IN WORK WITH BAYLOR COLLEGE OF MEDICINE, WAYNE GOODMAN, NICOLE PROVENZA WHO WILL BE FOLLOWING ME, WE'VE BEEN INTERESTED IN RELATION BETWEEN STIMULATION OF THE VENTRAL STRIATUM AND (INDISCERNIBLE) PATIENTS, FIVE PATIENTS WHO ARE UNDERGOING TREATMENT WITH DEEP BRAIN STIMULATION, OR TREATMENT RESISTANT OBSESSIVE COMPULSIVE DISORDER. ONE CONTEXT IS PROGRAMMING SESSIONS IN WHICH THE PARAMETER SETTINGS OF THE DEVICE ARE ADJUSTED IN ORDER TO OPTIMIZE TREATMENT. AND THIS IS A SERIES OF PROGRAMMING SESSIONS, A PERIOD OF EIGHT MONTHS. THE SECOND CONTEXT IS INTERVIEWS. CLINICAL INTERVIEWS OVER THE SAME PERIOD OF TIME. SO FOR MORE KIND OF NATURALISTIC BEHAVIOR. THIS SLIDE SHOWS AN EXAMPLE FROM A PROGRAMMING SESSION. HERE YOU CAN SEE THE DBS AMPLITUDE, TRACKING POSITIVE AFFECT AND HEAD VELOCITY. NOTE THAT THERE IS KIND OF A CLOSE COORDINATION BETWEEN CHANGES, EXPRESSION OF FACIAL AFFECT, EXPRESSED IN TERMS OF (INDISCERNIBLE) CONTRACTION REPRESENTED BY UNITS AND HEAD VELOCITY. IN CLINICAL INTERVIEWS, OVER THE SAME PERIOD OF TIME, WE LOOK AT RELATION BETWEEN FACIAL AFFECT AND HEAD DYNAMICS IN RELATION TO THE DBS ENERGY, WHICH IS A FUNCTION OF AMPLITUDE, PULSE WIDTH AND FREQUENCY, AND FOUND CLOSE TO 30% OF THE VARIANTS IN DBS ENERGY DURING AND INTERVIEW WAS ACCOUNTED FOR -- CAN BE ACCOUNTED FOR BY HEAD DYNAMICS AND FACIAL ACTION UNITS. WE LOOKED AS WELL AT THE RELATION TO YBOCS II SEVERITY SCORES, HEAD DYNAMICS AND FACIAL ACTION UNITS ACCOUNTED FOR ABOUT 20% OF THE VARIANTS. WHICH IS KIND OF A STRONG -- I WOULD SAY STRONG EFFECT IN THIS CONTEXT. AS AN EXAMPLE OF DBS IN A CLINICAL INTERVIEW, WE HAVE TWO INTERVIEWS, THIS IS FROM A CASE STUDY, KIND OF EARLY ON. IN ONE, THIS INDIVIDUAL HAS THE DBS IS ON, THE SECOND VIDEO SHOWS DBS OFF, AFTER A PERIOD OF ABOUT THREE HOURS. NOTE THERE IS A DRAMATIC DIFFERENCE IN HIS FACIAL AFFECT DURING THESE TWO CONDITIONS. WHEN IT'S ON, THERE IS INTENSE POSITIVE AFFECT, CYCLING ON AND OFF, BROW RAISES, SMILES, SO ON, VERY LITTLE, VERY FEW IN THE NEGATIVE AFFECT, WITH OFF . YOU FACIAL EXPRESSION HAS REVEALED TIME-LOCKED CHANGES IN NEURAL ACTIVATION AND LONGITUDINAL VARIATION IN DBS AND SYMPTOMATOLOGY. FACIAL EXPRESSION IS ONE OF MANY MODALITIES THAT CAN BE APPLIED IN HUMAN AND COMPARATIVE STUDIES OF BRAIN AND BEHAVIOR. GOING FORWARD, WE'RE INTERESTED IN EXPANDING THE NUMBER OF NEURAL SITES, SO FAR WE'VE PRIMARILY BEEN LOOKING AT SITES ASSOCIATED WITH OCD, SEPARATE STUDY WE'RE LOOKING AT KIND OF DEPRESSION, AND USING FACILITIES OF EPILEPSY MONITORING UNIT AT BAYLOR TO LOOK AT A LARGER NUMBER OF SITES IN RELATION TO THEIR EFFECT ON -- OR OUTPUT IN TERMS OF FACIAL AFFECT AND OTHER MODALITIES. VOICE QUALITY AND TIMING, SPEECH AND SENTIMENT, PERIPHERAL PHYSIOLOGY AND SOCIAL INTERACTION. WE HAVE TWO BROAD GOALS. ONE IS BEHAVIORAL BIOMARKERS FOR ADAPTIVE DEEP BRAIN STIMULATION. NICOLE PROVENZA WILL TALK MORE ABOUT THAT IN THE VERY NEXT TALK. AND MORE OBJECTIVE MEASUREMENT OF BEHAVIOR IN CLINICAL TRIALS, CURRENTLY FOR CLINICAL TRIALS THE PRIMARY OUTCOME MEASURE IS SELF-REPORT. WE WANT TO USE MULTI-MODAL MEASURES TO MORE OBJECTIVELY MEASURE SYMPTOMATOLOGY AND RESPONSE TO TREATMENT AND WORK WITH DELIBERATE, WE'VE BEEN WORKING WITH PREDICTING HAMILTON DEPRESSION SCORES AND USING MULTI-MODAL MEASURES, AND SO FAR KIND OF ENCOURAGING RESULTS, ACCOUNTING FOR 60% OF THE VARIATION BETWEEN OBSERVED HAMILTON SCORES AND PREDICTIVE. LAST IN TERMS OF CHALLENGES, WE NEED MORE DATA. YOU KNOW, KIND OF DATA IS THE KEY FOR LEARNING. ONE OF THE CONSTRAINTS IS CONSTRAINTS OF CLINICAL SETTINGS, THAT WE'RE NOT SET UP AS KIND OF LABORATORIES. SO IN AN EPILEPSY UNIT, THE LIGHTING IS HOSPITAL LIGHTING, HOW DO YOU IMPROVE THE ENVIRONMENT IN A WAY THAT FACILITATES IMPROVED MEASUREMENT. A SECOND IS SYNCHRONIZING VIDEO AND NEURAL DATA. AND THIS IS -- THIS IS AN AREA THAT IS REALLY IN NEED OF IMPORTANT ADVANCES. AND SO WITH THAT I'D LIKE TO THANK YOU FOR THE OPPORTUNITY TO PRESENT TODAY. THANK YOU. >> THANK YOU VERY MUCH. HOPEFULLY YOU CAN HEAR ME BETTER NOW. WE'RE MOVING TO THE THIRD TALK, NICOLE PROVENZA. >> HI, EVERYONE. I'M A POSTDOC AT BAYLOR COLLEGE OF MEDICINE SPEAKING ON PROGRESS IN HUMAN NEUROSCIENCE. NO DISCLOSURES TO REPORT. THERE'S SEVERAL KEY CHALLENGES THAT I THINK NEED TO BE ADDRESSED TO LINK NEURAL BEHAVIORS TO PSYCHIATRIC DISORDERS. ONE IS SYMPTOM HETEROGENEITY. PSYCHIATRIC DIAGNOSIS ARE DEFINED BY CLUSTERS OF SYMPTOMS AND THERE'S HETEROGENEITY. SYMPTOMS ARE SUBJECTIVE. EVEN IF WE HAVE AN IDEA OF SYMPTOMS WE WANT TO MEASURE HOW ARE WE GOING TO MEASURE THEM. THIS IS REALLY CHALLENGING IN PSYCHIATRY AS THERE ARE NO ESTABLISHED PHYSICAL READOUTS OF SYMPTOM STATES LIKE IN MOVEMENT DISORDERS. CLINICIAN ADMINISTERED SKILLS, AS DR. COHM MENTIONED, WITH USED TO ASSESS SEVERITY AND DEPEND ON SUBJECTIVE INPUT. THREE IS SPARSITY OF BEHAVIORAL INFORMATION. SELF-REPORT SCALES ARE MEANT TO BE ADMINISTERED ABOUT EVERY TWO WEEKS, AND SO IF WE ADMINISTER THEM MORE OFTEN, SAY EVERY DAY, WE DON'T HAVE A METRIC FOR EVERY MOMENT OF EVERY DAY, STILL. SO THESE SELF-REPORT SKILLS ARE TEMPORALLY SPARSE COMPARED TO DENSE LEAD DATA. FOR MY WORK I'M MOTIVATED TO OVERCOME CHALLENGES TO IDENTIFY BIOMARKERS TO IMPROVE OUTCOMES OF DBS IN SEVERE OCD AND DEPRESSION, PART OF AN NIH EFFORT LED BY DR. WAYNE GOODMAN. WE RECORD NEURAL SIGNALS, USE THOSE TO CLASSIFY A RELEVANT BRAIN STATE SUCH AS OBSESSIVE COMPULSIVE SYMPTOM STATE, CONTROL STIMULATION PARAMETERS ACCORDINGLY TO RELIEVE SYMPTOMS. SO IDENTIFICATION OF A RELATIVE BEHAVIORAL STATE AND BRAIN STATE UNDERLYING THIS BEHAVIOR IS REALLY CRITICAL FOR ACHIEVING THIS GOAL. SO ONE OF THE MANY STRATEGIES TOWARD FINDING BIOMARKER TO BE TO RELATE TO DENSELY MEASURED BEHAVIORAL FEATURES, IN MY PREVIOUS WORK WE DID THIS WITH OCD PARTICIPANTS, IMPLANTED WITH SENSING CAPABLE DBS DEVICES. AND WE MEASURED BEHAVIOR DURING CONCURRENT NEURAL REPORTINGS IN THE CLINIC AND AT HOME. IN THE CLINIC WERE ABLE TO DO DENSE BEHAVIORAL MODELING, AT HOME WE HAVE PARTICIPANTS CONTROL INTRACRANIAL RECORDINGS THROUGH TABLET, REPORTING USING A SLIDER BAR ON A PHONE APPLICATION. ONE PARTICIPANT DID A THREE-DAY RECORDING AT HOME, NATURAL EXPRESSURES TO OCD TRIGGERS ON DAYS 1 AND 2, REFLECTED BY THE SYMPTOM INTENSITY ON THE Y-AXIS, TO ANALYZE THE LFP DATA I COMPUTED SPECTRAL POWER OVER THE TWO-MINUTE SEGMENT ALSO OF DATA AROUND EACH RATING. HERE I'M SHOWING RESULTS FROM THAT ANALYSIS, SPECTRAL POWER VERSUS SYMPTOM INTENSITY FOR LEFT VC/VS, AND SO I FOUND SPECTRAL POWER SHOWS CORRELATION OF OCD SYMPTOM INTENSITY WHERE CORRELATION BETWEEN DELTA BAND POWER SHOWED STRONG NEGATIVE CORRELATION WITH OCD SYMPTOMS. SO WE'VE IDENTIFIED A CANDIDATE NEURAL BIOMARKER DURING NATURAL SYMPTOM PROVOCATION AT HOME. WHERE DO WE GO FROM HERE AND CONTINUE CLOSING IN ON BIOMARKERS AND VALIDATE THIS FINDING IN A LARGER SAMPLE? I THINK OUR FOCUS SHOULD BE ON HOW TO COLLECT BEHAVIORAL DATA IN NATURAL ENVIRONMENTS BUT IS JUST AS RICH AS THE NEURAL DATA THAT WE'RE ALREADY COLLECTING. WE NEED TO IDENTIFY TEMPORALLY DENSE REAL WORLD BEHAVIORAL METRICS. AS JEFF MENTIONED, THERE ARE LIMITATIONS WITH SELF-REPORTS, SUBJECTIVE EXPERIENCE CAN BE DIFFERENT THAN OBSERVABLE BEHAVIOR, AGAIN TEMPORALLY SPARSE. SO IF WE WANT TO KNOW HOW BRAIN ACTIVITY IS VARYING WITH CHANGES IN SEVERITY WE NEED CONTINUOUS MEASURE,S REDUCING PATIENT BURDEN TO REPORT HOW THEY ARE FEELING ACCURATELY AT EVERY SINGLE MOMENT. PASSIVE OBJECTIVE MEASURES MIGHT ELUCIDATE BRAIN BEHAVIOR RELATIONSHIPS UNDERLYING WHY SOMEONE FEELS THE WAY THEY DO. SO TO GO BACK TO OUR EXAMPLE OF OCD, WE'VE BEEN WORKING ON IDENTIFYING MEANINGFUL CONTINUOUS METRICS BY USING AFAR AS DR. COHN DESCRIBED. THIS IS A STILL SHOT FROM THE VIDEO THAT HE SHOWED. WE SAW A PEAK IN POSITIVE AFFECT, A SMILE, DBS AMPLITUDE INCREASING. SO THIS DATA AND OTHER DATA LIKE THIS WITH CONTINUOUS METRICS SYNCHRONIZED TO NEURAL DATA COULD BE HARNESSED TO USE -- TO BUILD MACHINE LEARNING MODELS THAT PREDICT BEHAVIOR FROM NEURAL ACTIVITY OR VICE VERSA. THE NEXT ACTION ITEM ON MY LIST IS THAT I THINK WE NEED TO INCORPORATE MULTI-MODAL CONTEXT AWARE BEHAVIORAL MODELING, I MEAN SOME BEHAVIORS MIGHT BE MORE QUANTIFIABLE OR OBSERVABLE THAN OTHERS DEPENDING ON CONTEXT. OUR GOAL THOUGH IS TO DEVELOP METHODS FOR QUANTIFYING BEHAVIOR, VIA MULTIPLE MODALITIES THAT HAVE HIGHER RELIABILITY WITH MANUAL ANNOTATION. SO HERE I'VE LISTED SOME EXAMPLES OF BEHAVIORS THAT WE COULD BE MONITORING AND GROUPED THEM INTO CATEGORIES. SO IF WE WANT TO QUANTIFY BEHAVIOR DURING A SOCIAL INTERACTION, FOR EXAMPLE, ONLY THE SOCIAL CATEGORY WOULD BE MEANINGFUL TO MONITOR. MAYBE LIKE DURING A WORKOUT, THESE MORE SOCIALLY GEARED METRICS MIGHT NOT BE INFORMATIVE OR POSSIBLE TO ENCAPTURE. THERE'S SOME CHALLENGES WITH THIS. ONE INHERENT CHALLENGE FOR DEVELOPING THESE METHODS AND SEARCHING FOR OBJECTIVE BEHAVIORAL FEATURES, DOING THIS SORT OF SCIENCE, IS IT'S MORE EXPLORATORY AND MIGHT NOT FIT INTO THE FRAMEWORK OF TRADITIONAL HYPOTHESIS-DRIVEN STUDY. I THINK THIS WILL REQUIRE REALLY LARGE N, REQUIRES BROAD TECHNICAL EXPERTISE, IN NEUROSCIENCE AND APPLYING COMPUTER VISION TECHNIQUES AND ENGINEERING, AND RAISES QUESTIONS ABOUT HOW TO PROTECT THE PRIVACY OF THE PARTICIPANTS, AND THINK IT'S IMPORTANT TO CONSIDER THERE MAY NOT BE CONVERGENCE ACROSS DIFFERENT MODALITIES. SO THE THIRD ACTION ITEM IS THAT I THINK WE NEED TO EXPLORE THE RELATIONSHIP BETWEEN TASK BASE AND REAL WORLD BEHAVIORS, WE NEED TO BE PROPOSING NATURALISTIC EXPERIMENTS GROUNDED IN FINDINGS FROM CLASSIC PSYCHOPHYSIOLOGIC TASKS. SO, CAN WE BETTER ALIGN PSYCHOPHYSIOLOGIC TASK-BASED STUDIES WITH OBSERVATION TO UNDERSTAND THE RELATIONSHIP BETWEEN LAB-BASED TASKS AND NATURALISTIC BEHAVIORS? ANOTHER EXAMPLE FROM MY PREVIOUS WORK IN OCD IS SHOWN HERE, SO ON THE LEFT I'M SHOWING SCHEMATIC FOR A TASK MEANT TO PROVOKE OCD SYMPTOMS, ON THE RIGHT I'M SHOWING NATURALISTIC ANALOG OF THIS PROVOCATION TASKS, PARTICIPANT IS WORKING THROUGH A REAL-LIFE EXPOSURE OF IMAGINING A WART ON HIS HAND DURING A TELETHERAPY SESSION AT HOME. SO PAIRING TASKS-BASED AND NATURALISTIC EXPERIMENTS THIS WAY ALLOWS US TO ASK THE QUESTION ARE THERE MENTAL PROCESSES ELICITED BY BEHAVIORAL TASKS THAT ARE RELEVANT FOR REAL WORLD SYMPTOMS OR FUNCTIONAL DEFICITS? AND I THINK THIS QUESTION IS CRITICAL MOVING FORWARD FOR IDENTIFYING ECOLOGICALLY VALID NEURAL BIOMARKERS OF DISEASE-RELEVANCE BEHAVIORS. SO, TO WRAP UP, I'LL TALK A BIT ABOUT THE IMPACT THAT THIS WORK I THINK MIGHT HAVE ON THE FIELD. I THINK THAT MORE DENSE BEHAVIORAL INFORMATION COULD ENABLE PRECISION PSYCHIATRY, PERHAPS WITH BETTER BEHAVIORAL QUANTIFICATION COULD MATCH INDIVIDUALS TO TREATMENTS OR UNDERSTAND WHY CERTAIN PEOPLE RESPOND TO CERTAIN TREATMENTS. AND I THINK THAT MORE DENSE BEHAVIORAL INFO COULD ENABLE IDENTIFICATION OF NEURAL BIOMARKERS, RICH BEHAVIORAL QUANTIFICATION WOULD HELP US BETTER UNDERSTAND THE BRAIN BEHAVIOR RELATIONSHIP UNDERLYING PSYCHIATRIC DISORDERS, AND NEURAL BIOMARKERS COULD SERVE AS A MORE OBJECTIVE GOAL POST FOR RESPONSE TO TREATMENT. AND LASTLY, I THINK THAT THIS MORE DENSE BEHAVIORAL QUANTIFICATION COULD SERVE AS PART OF THE MUCH NEEDED BRIDGE BETWEEN CONTROLLED AND NATURALISTIC EXPERIMENTS, AND TASK EXPERIMENTS ALLOW US TO OBSERVE AND QUANTIFY BEHAVIOR IN A CONTROLLED WAY AND WE NEED TO APPROACH BEHAVIORAL QUANTIFICATION IN NATURAL ENVIRONMENTS WITH JUST AS MUCH IF NOT MORE RIGOR THAN TASK-BASED STUDIES. THANK YOU FOR YOUR ATTENTION. . >> THANKS, NICOLE. SO FOR THE LAST TALK, LENA TING. >> OKAY, GREAT. SO THANK YOU SO MUCH FOR INVITING ME HERE AS A PERSON WHO SPENT A LOT OF TIME REALIZING THAT THE MOVEMENT INVOLVED FROM THE -- EMERGED FROM INTERACTION OF THE BIOMECHANICS OF THE BODY, AND NEURAL CONTROL SYSTEMS, IT'S OVER THE YEARS IT'S BECOME CLEAR TO ME I HAVE TO MOVE BEYOND THINKING ABOUT MOVEMENT AS REALLY JUST THE OUTPUT THAT WE CARE ABOUT, TO THINK MORE ABOUT BEHAVIOR AS IT'S BEEN SAID BEFORE. SO, THIS MATCH WHAT IS WAS SAID BEFORE. BUT WE REALLY CAN'T DISASSOCIATE. IT'S BEEN CLEAR WE CAN'T DISASSOCIATE COGNITIVE FUNCTION, AFFECT AND MOVEMENT BECAUSE ALL OF THESE THINGS ARE INTERTWINED, AS THEY ARE ALL HELPING US WITH OUR BEHAVIORS, WITH SOLVING PROSECUTES IN THE WORLD THAT HAVE VALUE TO US, MAKING DECISIONS WITH INCOMPLETE INFORMATION, AND TAKING ACTIONS IN AN ENVIRONMENT THAT'S DYNAMIC. SO IN MY FIELD WE TALK ABOUT THE NERVOUS SYSTEM GENERATING INTERNAL MODELS OF THE EXTERNAL ENVIRONMENT, THAT LET US COMPUTE FASTER. THESE NECESSARILY INTRODUCE BIASES IN HOW WE MOVE THAT MIGHT SHOW UP AS SOMETHING WE MIGHT REFER TO AS OUR PERSONALITY OR OUR LIFE HISTORY. AND, YOU KNOW, THE WORDS ASSOCIATED WITH THESE SHOW UP IN DIFFERENT FIELDS IN DIFFERENT WAYS, BUT THE BASIC IDEA IS WHEN WE LOOK AT BEHAVIORS, CHOICES PEOPLE MAKE, WHETHER THEY ARE COGNITIVE OR MOTOR, THE WAY IN WHICH WE GENERATE THEM ARE NOT NECESSARILY -- THEY ARE NOT OBJECTIVE, WE'RE NOT GOING TO BE ABLE TO WRITE DOWN EQUATIONS FOR THEM BUT THEY DEPEND ON INTERACTIONS WITH THE ENVIRONMENT, ON MANY TIME SCALES, AND THE CONTEXT IN WHICH THOSE HAPPEN. AND SO THOSE ARE THE THINGS THAT CAME UP TO ME WHEN THEY ASKED ME TO PLEASE GIVE A TALK HOW WE'RE GOING TO UNDERSTAND BRAIN BEHAVIOR. I'M GOING TO GIVE EXAMPLES AND INSPIRATION, WHERE I DREW INSPIRATION FROM. AND TO EMPHASIZE WE REALLY WANT TO HANDLE ALL THIS BIG DATA, THIS DATA THAT'S GOING TO BE DIFFERENT ACROSS CONTEXT, WE REALLY HAVE TO UNDERSTAND RULES OF THE STRUCTURES THAT WE GET AND TO UNDERSTAND THE VARIABILITY THAT WE'RE GOING TO GET BECAUSE IT'S NOT GOING TO BE AS RIGID AND AS MY ENGINEER BRAIN WOULD LIKE. WE'D LIKE TO IN PARTICULAR LOOK AT SPATIAL TEMPORAL DATA THAT ARE ON TIME SCALES OF NEURAL ACTIVITY SO WE CAN ACTUALLY BRIDGE MULTIPLE SCALES FROM NEURAL ACTIVITY TO BEHAVIOR. SO, ONE EXAMPLE THAT, YOU KNOW, AS YOU'VE SEEN WHEN WE LOOK AT MOVEMENTS, I CAN MEASURE JOINT ANGLES AND THINGS LIKE THAT BUT IN REALITY THERE'S MUCH MORE INFORMATION THERE ABOUT ALL SORTS OF SOCIAL ASPECTS AND AFFECT, AS WE'VE HEARD, AND WE CAN LOOK AT THE DATA AND BREAK IT DOWN. WE ALSO NEED TO START USING SORT OF BIOPHYSICALLY BASED METHODS, HOW DOES THAT EVEN HAPPEN IN THE BODY, HOW DO WE GET BACK DOWN TO MECHANISMS MORE? HERE'S JUST AN EXAMPLE, WHERE THE BODY DOESN'T CHANGE, WE KNOW THIS IS A PHYSICS-BASED MODEL OF MOVEMENT WHERE THE CONTROL OF IT CHANGES, POTENTIALLY BIOLOGICALLY PLAUSIBLE WAY SO A PERSON CAN EXPRESS HAPPY OR SAT WALKING ACROSS VARIOUS BEHAVIORS. WE WANT TO FIND PRINCIPLES, IT DOESN'T MATTER WHAT KIND OF MOVEMENT I'M DOING, THERE'S SOMETHING COMMON THAT MAKES IT HAPPY. OR SAD. AND TO THINK ABOUT THAT, I WANT TO GO BACK TO A PRETTY OLD EXAMPLE THAT I LIKE TO THINK ABOUT, MY SLIDES ARE NOT ADVANCING. OKAY. WHICH IS THINKING ABOUT ACCENTS, WHEN WE TALK. SO WHEN WE'RE -- THERE'S AN OBJECTIVE SPACE HERE OF SOUNDS. THIS IS ONE FREQUENCY AND ANOTHER. EACH OF THESE BOXES REPRESENTS THE VOWEL SOUNDS WE CAN HEAR WHEN WE'RE BABIES. AND AS PEOPLE INTERACT WITH THEIR ENVIRONMENT, THESE SPACES KIND OF MERGE AND FUSE INTO NEW SHAPES THAT ARE RELEVANT TO THE WORLD. SO, THEY ARE SORT OF CHUNKING THEM INTO DIFFERENT TYPES OF WHAT WE MIGHT CALL LATENT VARIABLE, IN TODAY'S PARLANCE, BUT WHAT IT MEANS IS THAT THERE ARE OBJECTIVE -- THEY ARE BIASES THAT HAPPEN WHEN PEOPLE SPEAK THAT I CAN PREDICT IN OBJECTIVE AND QUANTIFIABLE WAY. ONE EXAMPLE IS IF YOU SPEAK JAPANESE, THEN ALL OF THE SOUNDS THAT FALL IN THIS REGION SOUND THE SAME. AND SO WHEN I'M LISTENING TO ENGLISH, I CAN'T DISTINGUISH THESE TWO SOUNDS, AND WHEN I'M SPEAKING THEM I MIGHT SAY RA AND LA, CONFUSE THOSE TWO SOUNDS. THOSE ARE THINGS WE CAN ACTUALLY MEASURE HAPPENING IN THE BRAIN, HOW THE BRAIN IS WARPING THE OBJECTIVE INFORMATION, AND THEN THE CONTEXT MATTERS TOO. WE CAN'T THINK OF THESE LOCATIONS AS BEING FIXED IN THIS OBJECTIVE SPACE, BUT WHEN A MOTHER SPEAKS TO A CHILD THE DISTANCE BETWEEN THE POINTS STRETCHES. WE NEED TO LEARN THE RULES BY WHICH THESE CLUSTERS OR UNITS OF ACTION MOVE AROUND WITH INDIVIDUAL, WITH CONTEXT, WITH MOOD. AND SO GETTING BACK EVEN FURTHER TO SOMETHING, MORE LINKED TO MOTOR OUTPUT, IN MY LAB WE LOOKED AT MUSCLE ACTIVITY, THE OUTPUT OF THE MOTOR NEURONS, AND DEMONSTRATED THAT IN MOVEMENT THERE ARE -- EACH PERSON HAS LIKE A LIBRARY OF MOTOR MODULES SIMILAR TO THESE FACIAL THINGS WE HEARD ABOUT AND WE CONSTRUCT MOVEMENTS BASED ON COORDINATING A SET OF MUSCLES ACROSS LIMBS IN ORDER TO PERFORM CERTAIN BIOMECHANICAL TASKS. I WAS LOOKING FOR SOMETHING OBJECTIVE HERE AND HAD TO CONCEDE DEPENDING ON THE ANIMAL I WAS LOOKING AT OR THE PERSON I'M LOOKING AT, TO PERFORM THE SAME TASKS WHICH MIGHT BE PRODUCING A FORCE EACH ANIMAL SORT OF COORDINATES THEIR SETS OF MUSCLES IN SLIGHTLY DIFFERENT WAYS. WHEN WE LOOKED ACROSS PEOPLE, THESE DIFFERED DEPENDING IF YOU'RE A BALLET DANCER OR HAVE GONE THROUGH REHABILITATION, OR IF YOU HAVE DIFFERENT NEUROLOGIC DISORDER. PERHAPS THAT'S ONE OF THE MECHANISMS BY WHICH WE CAN IDENTIFY DIFFERENT PEOPLE, JUST BY THE WAY THEY WALK. BUT THUS FAR WE DON'T HAVE A DIRECT LINKAGE BETWEEN THESE SORT OF VERY STATIC CARTOONS OF DIFFERENT WAYS IN WHICH PEOPLE ACTIVATE THEIR MUSCLE ACTIVITY OR DIFFERENT WAYS IN WHICH THEY MIGHT BE IMPAIRED AT MOVEMENT, AND ONE WAY THAT MANY PEOPLE HAVE BEEN TRYING TO IS TO NOW PUT THESE PATTERNS ON A BIOPHYSICAL MODEL OF WALKING. THIS APPROACH IS LACKING IN THAT WE DON'T KNOW ALL OF THE NEURAL CONSTRAINTS AND NEURAL DYNAMICS, EVEN IF WE HAD A CONNECTOME, IT WOULD BE VERY, VERY UNDERSPECIFIED. SO, WE'VE RECENTLY WORKED WITH GORDON BERMAN TO TAKE THIS MORE DATA-DRIVEN APPROACH TO SAY I CAN SEE THESE DIFFERENCES IN THE MOVEMENTS, IN THESE KINEMATIC PATTERNS. WHEN I COMPARE IN TRADITIONAL BIOMECHANICAL SENSE EVERYBODY LOOKS THE SAME WHEN THEY WALK. THERE'S MORE INFORMATION THAT OUR BRAINS ARE GETTING OUT THAT WE'RE NOT CAPTURING IN THE WAY THAT WE TREAT BIOMECHANICS DATA. AND SO WE PUSH IT THROUGH A NEURAL NETWORK, AND THEN TO CAPTURE THE SORT OF DYNAMICS, THAT IS THE SPATIAL AND TEMPORAL DEPENDENCIES WITHIN DATA, IN A WAY THAT WE CAN COMPARE DIFFERENT INDIVIDUALS, AND SHOW THESE PEOPLE WALK AT THE SAME SPEED ARE ACTUALLY WALKING AT DIFFERENT SPEEDS, THAT'S THE SHADING, MAINTAIN SOME CHARACTERISTICS OF THE MOVEMENT THAT IS THEIRS. AND PERHAPS BY NOW LOOKING AT THESE TWO TYPES OF MODELS, OF MOVEMENT, WE CAN HOPE TO LINK THIS EMERGENT BEHAVIOR WE SEE WITH SOME MORE CONNECTIVENESS TIMES OF TIMES OF PATTERNS THAT MIGHT INDICATE SOMETHING AT A LOWER LEVEL IN THE NERVOUS SYSTEM. THINK THIS KIND OF APPROACH IS GOING TO BE NECESSARY WHEN WE ARE TALKING ABOUT, OH, HOW DO WE DO MULTI-DISCIPLINARY RESEARCH ACROSS SPECIES AND SCALE, AND TO THINK ABOUT, OH, WELL, HOW NATURALISTIC BEHAVIORAL STUDIES TO SORT OF GATHER DATA AND MAKE HYPOTHESES AND EXPERIMENTAL STATE WHERE WE MANIPULATE BEHAVIORAL CONTEXT, WE HAVE TO BE ABLE TO UNDERSTAND THE STRUCTURE OF THE VARIATIONS THAT WE'RE GOING TO SEE AT ALL LEVELS. AND THEN ULTIMATELY AGAIN IF PEOPLE SAID TO USE TECHNOLOGIES TO ACTUALLY PERTURB THE CAUSAL MECHANISMS INVOLVED. I WANT TO REALLY TALK ABOUT REALLY WHY THIS IS SO IMPORTANT. RECENTLY INTERVIEWED ABOUT THE YIPS, YOU MAY HAVE SEEN AT THE OLYMPICS, ATHLETES UNABLE TO PERFORM THE MOTOR TASK IN A PARTICULAR CONTEXT, IT TURNS OUT THERE ARE HYPOTHESES ABOUT THESE BEING BOTH BOTTOM-UP, SOMETHING ABOUT SPINAL NETWORKS GOING WRONG, ALL THE WAY UP TO PREFRONTAL, SORT OF COGNITIVE REASONS. AS WELL AS LIFE HISTORY LIKE EARLY CHILDHOOD TRAUMA, GENETICS, CAN CONTRIBUTE TO THESE BEHAVIORS. AND SO THIS IS WHAT HAPPENS WHEN YOU HAVE TO MOVE IN A REAL SITUATION. IT'S JUST NOT ABOUT CONTROLLING YOUR MUSCLES. IT ALSO HAS RELEVANCE IN CLINICAL DISORDERS, FREEZING OF GAIT, PARKINSON'S DISEASE, SIMILAR THAT PEOPLE CAN WALK MOST OF THE TIME, DIFFERENT INDIVIDUALS HAVE COGNITIVE MOTOR AND EMOTIONAL TRIGGERS THAT MAKE THEM UNABLE TO ACTUALLY WALK, WALK AND THEY FREEZE. AND SO WHEN WE THINK HOW DO WE STUDY THAT IN AN ANIMAL AND MAKE INFERENCES, THERE'S BEEN SOME WORK BY STEVE CHASE'S GROUP, SET SET UP EXPERIMENTAL SITUATION WHERE MONKEYS WILL CHOKE, DEPENDING ON THE STAKES OF THE AS -- SCENARIO. THAT'S ONE WAY TO LOOK AT IT. IN MY LAB WE'VE JUST STUDIED ONE BEHAVIOR, WHICH IS STANDING POSTURAL CONTROL, AND STARTED TO MEASURE BRAIN ACTIVITY AND BASED ON SOME EPIDEMIOLOGICAL WORK SHOWING COGNITIVE STATE INFLUENCES FALLS WE'VE BEEN SHOWING THE ACTIVITY EVOKED DURING THIS VERY UNECOLOGYICAL IS MODULATED BY AFFECT. THE DIRTY SECRET, WE DON'T MEASURE THEM, DON'T SAVE THEIR DATA WHEN THEY ARE HAVING A BAD DAY BECAUSE WE DON'T FLOW HOW -- KNOW HOW TO TAKE THAT INTO ACCOUNT WHEN COMPARING DATA AND HAVE TO RECOGNIZE BOTTOM-UP MECHANISMS THAT WE'RE LOOKING AT LIKE EVEN THE SENSORY INFORMATION THAT'S COMING UP FROM THE PERIPHERY IS MODULATED BY MOTOR COGNITIVE STATE AND TRAINING. AND SO I WANTED TO SAY NEURODIVERSITY AND VARIABILITY IS REALLY IMPORTANT TO UNDERSTAND AND NOT TRY TO FIND SOME UNIQUE MAPPINGS BETWEEN BRAIN AND BEHAVIOR, AND THAT BY UNDERSTANDING THE STRUCTURE WE CAN HOPE TO FIND PRINCIPLES THAT CAN GO ACROSS LOTS OF BEHAVIORS, SCALES, AND SPECIES. SO, I'M ENCOURAGING PEOPLE TO COLLABORATE, JUST LIKE THIS WORKSHOP. THANK YOU. >> THANK YOU. SO WE'RE NOW MOVING FROM THE INDIVIDUAL TALKS TO THE DISCUSSANTS. FIRST DISCUSSANT IS MOLLY CUMMINGS. >> THANK YOU. >> GO AHEAD. >> OKAY. I WAS GOING TO THANK ALL THOSE AMAZING SPEAKERS. IT'S HAVEN A REALLY INTERESTING SET OF FOUR SPEAKERS. AND -- WELL, ACTUALLY FIVE SPEAKERS. THAT WAS AN AMAZING INITIAL KICKOFF INTRODUCTION. I WANT US TO NOTE A FEW THINGS THAT WERE THROUGHOUT THE FOUR SPEAKERS. THE FIRST -- TWO OF THE SPEAKERS FOCUSED ON MOVEMENT PROVIDED INCREDIBLE MULTI-MODAL DEEP METRICS IN THE HUMAN PERSPECTIVE, DR. TING JUST SHARED, A COUPLE ANIMAL -- OTHER ANIMAL MODEL SPECIES, ZEBRAFISH, RATS, INSECTS, DROSOPHILA, THAT DR. COUZIN SHARED, AND TWO SHARING DATA ON INTERNAL STATES, TRYING TO FIND BIOBEHAVIORAL MARKERS TO PREDICT INTERNAL STATES. AND IT SEEMED WHILE WE HAD GREAT INFORMATION ON INTERNAL STATES, WE HAD RELATIVELY FEW REALISTIC BIOBEHAVIORAL MARKERS TO PREDICT THAT WELL, AND DR. COHN'S EXAMPLE WAS FANTASTIC WHERE HE WENT FROM ONLY BEING ABLE TO PREDICT 8 TO 17% OF VARIATION IN OCD, UP TO 60% WHEN STARTING TO TAKE IN GREATER REPERTOIRE OF MULTI-MODAL INPUTS INTO THAT. AND I WANTED US TO TALK, THERE WILL BE MULTIPLE PANELISTS BRINGING UP DIFFERENT QUESTIONS, BUT I WANTED TO REALLY EMPHASIZE THE ENDING THEME FROM DR. TING'S TALK, HOW DO WE IDENTIFY RULES AND PRINCIPLES AND HOW WE HAVE TO GO ACROSS MULTI-SCALES IN DIFFERENT SPECIES TO DO THAT. AND AS A NEUROETHNOLOGYIST, I WANT TO THROW OUT THE QUESTION ARE WE AT A POINT WHERE WE NEED TO ABANDON STUDYING THE SPECIALIST, THAT WOULD BE INVOKING A DANISH PRINCIPLE FOR FROM A CENTURY AGO, THERE'S ONE WE SHOULD BEST STUDY TO FIND THE ANSWER, AND DO WE WANT TO CHANGE TACTIC AND LOOK MORE AT GENERALIST OR PERHAPS, I WOULD BE IN FAVOR OF THIS, GOING EXTREMELY COMPARATIVE, WE OFTEN PAT OURSELVES ON THE BACK IF WE COMPARE TWO SPECIES. BUT IT SEEMS JUST AS DR. TING SHOWED US THERE'S MULTIPLE WAYS TO SOLVE A MOVEMENT PATTERN, CAN WE LOOK ACROSS MULTIPLE SPECIES OF BRAINS, DIFFERENT SPECIES WITH DIFFERENT BRAINS THAT SOLVE THE SAME PROBLEM IN DIFFERENT WAYS, TO ENRICH OUR ABILITY TO SOLVE PROBLEMS AT THE HUMAN FRONT. I WANT TO THROW THAT OUT THERE TO START. AND IF ANYONE FEELS LIKE CHIMING IN ON THAT, I THINK THE IDEA IS YOU UNMUTE YOURSELF. THIS IS NOT MEANT FOR ME TO TALK FOR EIGHT MINUTES. >> SHOULD WE PUT OUR VIDEOS ON? >> YES PLEASE. >> OR SHOULD WE STICK WITH SPECIALIZING? ARE THERE THINGS THE RAT CAN TELL US MORE AND WE SHOULD BIG DEEPER INTO THE RAT OR DROSOPHILA TO FIND INSIGHT INTO HUMANS? >> LET ME START BY SAYING AFFECT DECISION-MAKING MOVEMENT AND LIKE AFFECT ARE NOT SPECIAL TO HUMANS. SO SOMETHING HAS TO MATTER TO YOU TO BE ABLE TO WANT TO MAKE A DECISION. SO IN THAT SENSE WE'RE GOING TO FIND IT IN MAY DIFFERENT SPECIES BUT TO DIFFERENT EXTENTS. >> IT'S INTERESTING THAT THE TWO TALKS THAT USED DBS, YOU KNOW, THEY ARE STIMULATING PARTS OF THE BRAIN EVERY VERTEBRATE HAS. THERE ARE WAYS TO FIGURE THIS OUT WITH DBS AT THE LEVEL OF CROSS-LINEAGE COMPARISON. ON THAT THEME, I WANTED TO, AGAIN DRAWING FROM DR. TING'S WORK, WHERE SHE FOUND THAT DIFFERENT ANIMALS, DIFFERENT HUMANS, USE DIFFERENT COORDINATED MODULES TO PRODUCE THE SAME WALKING SPEED. I THINK BY BECOMING MORE COMPARATIVE, AND LOOKING ACROSS BRAINS, THAT HAVE SOLVED WHATEVER PROBLEM WE ARE FACING, CHANGING ONE INTERNAL STATE TO ANOTHER INTERNAL STATE, WE CAN ACTUALLY IDENTIFY DIFFERENT BRAIN REGIONS AND PATHWAYS TO DO THAT. WHETHER OR NOT THE SOLUTION ENDS UP BEING DBS OR THE SOLUTION ENDS UP BEING PHARMACEUTICALS, THAT CAN STIMULATE DIFFERENT NEUROMODULATORY PATHWAYS TO AFFECT OUTCOME IS ANOTHER WAY TO LOOK AT THAT. >> I WANT TO BE CLEAR THOUGH THAT'S WHAT WE SEE IS SHAPED OVER LONG PERIODS OF TIME AND LEARNING AND REHABILITATION. SO THERE'S GOING TO BE -- I THINK THIS HAPPENS IN DBS OR DEPRESSION TOO, INSTANTANEOUS CHANGE THAT MIGHT HAPPEN WITH NEUROMODULATION THAT ENABLES A LONGER-TERM SORT OF SLOWER TIME CONSTANT OF CHANGE IN THE UNITS THAT SHAPE SOMEBODY'S BEHAVIOR. >> RIGHT. KAREN ROMMELFANGER. >> THANKS. I HAD A REALLY -- I ENJOYED THE OPENING PANEL. I HAVE CURIOSITY AROUND JEFF CON'S WORK WHERE HE NOTED AFAR WAS OPTIMIZED ACROSS CULTURES AND YET ALSO CLOSED WITH AN OPPORTUNITY FOR US TO EXPLORE MORE CONTEXT NEEDED FOR EXPLORING HOW THIS TECHNOLOGY WOULD WORK AND THAT MORE DATA WOULD BE NEEDED TO OPTIMIZE TECHNOLOGY. CAN YOU SHARE ABOUT BIAS, SOCIALLY CONSTRUCTED IDENTITIES, THE DEGREE OF CONFIDENCE IN INTERPRETATIONS ACROSS BEHAVIORS, SUCH AS WE FOCUSED ON MOOD BUT OPPORTUNITIES FOR LOOKING AT PAIN AND OTHER SUBJECTIVE STATES. AND WRAPPING THAT TOGETHER, WHAT ARE THE OPPORTUNITIES AND CHALLENGES FOR TECHNOLOGY LIKE AFAR IN THE BRAIN BEHAVIOR SYNCHRONIZATION MODEL IN LIGHT OF THOSE KINDS OF QUESTIONS AROUND BIAS OPPORTUNITIES AND LIMITATIONS. >> RIGHT. THANKS FOR THE QUESTION. THE-- AFAR IN DETECTING ACTION TUNES, ACTION OPPORTUNITIES ARE DESCRIPTIVE MOVEMENTS IN A WAY THAT WALKING IS A MOVEMENT. SO THERE'S -- IT'S PRIMARILY DESCRIPTIVE OTHER THAN INFERENCE CONSTRUCT. GENDER, CULTURAL IDENTITY, OR AFFECT, THOSE ARE HIGHLY INFERENTIAL. WE NEED SOME BRIDGING TO SUPPORT INFERENCES ABOUT THE MEANING OF ACTION UNITS. NOW, WITH RESPECT TO DESCRIPTIVE MEANING WE DO ACTION UNIT INPUTS, INFANT HAVE DIFFERENT-SHAPED FACES, DIFFERENT MOVEMENTS, YET THE SYSTEM WORKS, BECAUSE IT'S PRIMARILY ANATOMIC AND TRAINED ON INFANTS. FOR PEOPLE WITH VERY DARK SKIN, AFAR TO THE EXTENT WE'VE LOOKED, SO FAR IT APPEARS TO WORK WELL. IT DEPENDS ON THE AMOUNT OF LIGHTING. BUT, YOU KNOW, IT'S REALLY -- WHEN YOU HAVE STIMULATION IN THE VENTRAL STRIATUM, THAT SUPPORTS INFLUENCE YOU'RE ELICITING POSITIVE AFFECT, AND SO I MAKE THAT INFERENCE, BUT I'M MAKING IT FROM WHAT I KNOW ABOUT PARTICULAR COMBINATION OF ACTION UNITS THAT HAVE BEEN, YOU KNOW, STUDIED BROADLY IN AFFECTIVE CONTEXT. >> THANKS, JEFF. WE NEED TO MOVE TO THE NEXT DISCUSSANT BUT I THINK THERE WERE SEVERAL QUESTIONS THAT WE CAN GENERALIZE ABOUT WHEN WE'RE SETTING BEHAVIOR, IN WHAT SITUATIONS, AS HUMANS, DO WE NEED TO THINK MORE CAREFULLY HOW WE DO THAT BASED ON RACE OR ETHNICITY OR SEX OR GENDER. NOW WE'RE GOING TO MOVE TO TIMOTHY WRIGHT. >> THANK YOU. I WANT TO THANK MOLLY FOR STARTING US OFF. DAMON, THANK YOU FOR PUTTING THAT UP. I WONDER IF WE COULD SEE THE WHOLE PANEL RIGHT NOW AND THEN TOWARDS THE END OF MY TIME I'LL ASK A QUESTION SPECIFICALLY ABOUT THAT SLIDE. THANK YOU VERY MUCH. I WANT TO CONTINUE ON THIS THEME OF AFFECT, AND DIFFICULTIES WE MIGHT HAVE WITH SYSTEMS THAT ARE DEVELOPED FOR ONE SPECIES OR ONE SUBSET OF A SPECIES. AND THINK ABOUT HOW WE CAN EXPAND THAT APPROACH INTO A COMPARATIVE FASHION, THINKING ABOUT OTHER SPECIES. IT'S CLEAR PEOPLE WORKING WITH HUMANS HAVE THOUGHT QUITE A BIT ABOUT THE IMPORTANCE OF AFFECT. BUT ANIMAL BEHAVIORISTS ARE NOT BLIND TO THIS. THERE'S WELL KNOWN EFFECTS IF A CORRECT HAS LOST A FIGHT EARLY, IT'S MORE LIKELY TO LOSE A FIGHT LATER, EVEN IF IT'S A RELATIVELY EASY OPPONENT. BUT CLEARLY TRYING TO FIGURE OUT WHAT AFFECTIVE STATE IS IN ANIMALS IS A CHALLENGE. I I WANT TO START WITH THE HUMAN PANELIST, HUMAN ORIENTED PANELISTS. WE'RE ALL HUMAN I GUESS. ASK WHAT POTENTIAL THEY SEE FOR USING THESE SORTS OF APPROACHES WITH ANIMALS AND THEN I WANT TO MOVE ON TO IAN AND MOLLY AND BENCE, WHO WORKED WITH ANIMAL MODELS. I'LL THROW THAT OUT TO NICOLE AND JEFF AND LENA TO START WITH. LENA HAS A HAND RAISED. >> I'M NOT SURE ABOUT AFFECT, THERE HAVE BEEN INTERESTING STUDIES IN GUINEA FOWL BY MONICA DALY, SHOWING A PERSONALITY AFFECTS HOW THEY WALK AROUND THE ROOM OR SLIDE OR HOW THEY CARE ABOUT CRASHING INTO THINGS. AND SO, YOU KNOW, I THINK YOU'RE GOING TO SEE THAT IN MANY ANIMALS, CATS THAT I TRAINED ALL HAD DIFFERENT PERSONALITIES WHICH AFFECTED THE MUSCLE COORDINATION PATTERNS THEY USED AND HOW THEY USED THEM. >> OKAY. ANY OTHER COMMENTS FROM NICOLE OR JEFF? >> I THINK ONE INTERESTING CONCEPT -- WELL, SHIFTING FROM THINKING ABOUT HUMANS TO THINKING ABOUT ANIMALS YOU CAN'T ASK ANIMALS HOW THEY ARE FEELING OBVIOUSLY. YOU CAN ASK HUMANS HOW THEY ARE FEELING. WE HAVE TO INFER EVERYTHING ABOUT -- EVERYTHING WE ASSUME AGUESS ABOUT HOW ANIMALS ARE FEELING BASED ON THEIR BEHAVIOR. I THINK THE IDEA OF LIKE DOING THAT IN HUMANS IS REALLY COOL. BUT ONE QUESTION I'VE BEEN CHEWING ON FOR QUITE A WHILE, I'M SURE PEOPLE HAVE THOUGHTS ABOUT THIS, IS WHAT COMES FIRST, LIKE WHEN SOMEONE IS RECOVERING OR GETTING BETTER, IF THEY HAVE DEPRESSION, IS IT BEHAVIOR CHANGES AND THEN THEY FEEL BETTER, DOES IT HAPPEN AT THE SAME TIME OR DO THEY FEEL BETTER AND THEIR BEHAVIOR CHANGES? >> LENA, YOU KNOW, I WOULD SAY WE GIVE SPECIAL STATUS TO FEELING, SUBJECTIVE EXPERIENCE. IT'S THE SINE QUA NON OF EMOTION. BUT IT'S REALLY JUST ONE ASPECT. AND IT'S ONE THAT'S VERY HARD TO KNOW WHAT ANOTHER -- IT'S VERY HARD TO KNOW WHAT YOUR PET IS FEELING, VERY LARD -- HARD TO KNOW, BUT THERE ARE OTHER ASPECTS SUCH AS REACTION TENDENCIES, SO THE CRICKET THAT LOSES A FIGHT AND THEN GOES ON TO LOSE AN EASY FIGHT, YOU KNOW, THAT SEEMS LIKE AN AFFECTIVE RESPONSE. IT'S AN EMOTIONAL RESPONSE THAT THE EXPERIENCE OF LOSING THAT INITIAL FIGHT AFFECTS ITS ABILITY TO MOTIVATE COORDINATED AGGRESSIVE AFFECTIVE BEHAVIOR IN CONFLICT. THAT'S, YOU KNOW, THAT'S EMOTION. I DON'T KNOW WHAT THE CRICKET IS FEELING, BUT I WOULD SAY THAT IT'S NOT HIGH ON APPROACH, IT'S -- YOU KNOW, IT'S NOT EXHIBITING MASTERY OF WHICH ITS CAPABLE. THESE ARE AFFECTIVE RESPONSES. >> I SEE A HAND RAISED EARLIER FROM YOU, IAN? >> YEAH, IT'S CONTEXT, BUT I COULD IF IT'S INTEREST BUT OTHERWISE LET PEOPLE SPEAK. >> I'LL CHIME IN FROM THE NON-HUMAN ANIMAL CHARACTERIZATION OF BEHAVIOR. >> THAT WAS THE ANGLE WAY GOING TO TAKE, MOLLY. >> I WAS GOING TO CITE YOUR WORK AS WELL AS PITCH SIMILAR -- HOW WE ACTUALLY CAPTURE INTERNAL STATES AND AFFECTIVE INTERNAL STATE OF A NON-MODEL ORGANISM AT THIS POINT IS DOING GENOMIC PROFILING OF THE BRAIN. IAN GAVE A GREAT EXAMPLE, YOU CAN PAIR BEHAVIOR WITH A TRANSCRIPTOMIC PROFILE AND UNDERSTAND, YOU KNOW, WHAT'S GOING ON, DYNAMICALLY AT THE BRAIN, NOT AT NEURAL STIMULATION BUT GENE PATHWAYS THAT ARE BEING EVOKED IN THE LAST 30 MINUTES OF THE ANIMAL'S LIFE OR LESS THAN THAT. AND SO WE ACTUALLY CAN IDENTIFY GENE SUITES THAT DO DIVERGE AND WE DON'T KNOW IF TERMS OF AFFECTATION EXACTLY WHAT THEY REPRESENT BUT WE COMPARE THAT WITH MOVEMENTS WHICH WE CAN SURMISE HAVE SOMETHING TO DO WITH EITHER FEAR OR EXCITEMENT OR OTHER CONTINUUMS OF BEHAVIOR. SO THAT'S HOW WE DO IT INSTEAD OF SELF-REPORTING. WE LOOK AT WHAT'S GOING ON IN THE BRAIN. NOT AT THE LEVEL OF NEURONS BUT GENOMIC PROFILING OFTEN. >> I THINK THIS PROBLEM OF BRIDGING FROM HUMAN STUDIES TO ANIMAL STUDIES IS REALLY KEY AND CENTRAL, AT LEAST FOR WHAT I DO. IAIN SAID HE WAS STUDYING AUTISM IN IN FISH, WE STUDY IN RATS. HOW DO YOU KNOW WHAT YOU SEE IN A HUMAN PATIENT IS NEAR WHAT YOU SEE IN YOUR ANIMAL SO I DON'T KNOW WHAT MECHANISMS WE HAVE FOR FACILITATING COMPARISONS BETWEEN THE HUMAN CONDITION AND VARIOUS ANIMAL BEHAVIORS. BUT PERHAPS ONE APPROACH IS TO STUDY RATHER LOW LEVEL MANIFESTATIONS OF HIGH LEVEL PHENOMENA WHICH CAN BE COMPARED ACROSS SPECIES AND I'M THINKING HERE OF MOVEMENTS IN TERMS OF AUTONOMIC NERVOUS SYSTEM FUNCTION BECAUSE THAT GIVES US AT LEAST SOMETHING TO COMPARE, MORE OF AN APPLES-TO-APPLES MANNER, PERHAPS USE SIMILAR TECHNIQUES FOR MEASURING AND SIMILAR ANALYTICAL FRAMEWORKS, SO THAT COMPARISONS CAN BE MADE MORE EASILY. >> I DON'T KNOW IF THAT WAS THE BEGINNING OF YOUR ACTUAL FORMAL DISCUSSION COMMENTS BUT IT'S TIME FOR YOU THIS TRANSITIONS ORGANICALLY. >> IT'S A BIG QUESTION I'VE STRUGGLED WITH, LISTENING TO THE SPEAKERS, I REALIZE THERE'S A GULF IN APPROACH, AND MINDSET, AND TECHNIQUES, AND FRAME WORKS BETWEEN THOSE WHO STUDY DISEASE, HUMAN DISEASE IN A CLINICAL MANIFESTATION AND THOSE OF US WHO DO MECHANISTIC STUDIES IN ANIMALS WE TRY TO SIMPLIFY THINGS AS MUCH AS WE CAN AND MAKE INFERENCES ABOUT WHAT WE SEE TO COMPLEX HUMAN DISORDERS. I DON'T HAVE A PARTICULARLY GOOD REMEDY FOR THAT BUT I WOULD LOVE TO HEAR FROM THOSE WHO STUDY HUMAN DISEASE OR DISORDER, IF YOU HAVE ANY GUIDELINES OR CRITERIA FOR VALIDITY OF THE VARIOUS MODEL ORGANISMS THAT WE STUDY, YOU KNOW, IS IT AS MOLLY SAID, IS THERE A PARTICULAR ANIMAL MODEL THAT'S PARTICULARLY WELL SUITED TO STUDY OCD OR DEPRESSION? AND IF SO, WHAT ARE THOSE CRITERIA, HOW SHOULD WE THINK ABOUT THIS? >> IAIN, DO YOU WANT TO ANSWER THIS? >> I GUESS I PUT MY HAND DOWN AND UP AGAIN. I THINK YOU MAKE A REALLY GOOD POINT. SO TO MY READING, MOST OF THE WORK, PRE-CLINICAL WORK THAT'S BEEN DONE ON, SAY, PSYCHIATRIC ILLNESS, HAS BEEN DONE FOR HUMANS, A VERY GOOD REASON. I THINK WITH RESPECT TO ZEBRAFISH, YOU HAVE A HIGHER DEGREE OF CONTROL, CAN PUT THEM IN VIRTUAL REALITY, INFER CAUSALLY, WHICH ALGORITHMS ARE MISFUNCTIONING. YOU DETERMINE PRECISELY WHICH ALGORITHMS WORK AND COLLABORATING WITH PEOPLE LIKE FLORIAN AT YOUR UNIVERSITY WE CAN POTENTIALLY LOOK AT CIRCUITS THAT MIGHT BE AFFECTED. BUT I WILL SAY THIS, LIKE A FISHING EXTRADITION, IT'S NOT THE TYPE OF SCIENCE I LIKE TO DO. THE MORE WE LOOK INTO IT, IT'S LIKE A SCREENING FOR JUST TO SEE IF WE GET LUCKY. BUT THE MORE WE LOOK INTO IT THE MORE WE REALIZE THAT WE MIGHT GET LUCKY. CAN YOU HEAR ME NOW? >> NOW I CAN HEAR YOU. >> YEAH, SORRY. THERE ARE THESE BASAL SOCIAL BEHAVIORS, AT LEAST, THAT WE COULD STUDY. I THINK THE ANSWER TO YOUR QUESTION IS WE DON'T KNOW IF IT'S GOING TO BE SO, WE DON'T KNOW IF IT'S GOING TO BE CLINICALLY USEFUL. ZEBRAFISH ARE HIGHLY COMPLEX IN SOCIAL BEHAVIORS. PEOPLE HAVE MISUNDERSTOOD COMPLETELY HOW SOPHISTICATED THEY ARE BECAUSE THEY LOOK AT THEM IN AN ENVIRONMENT THAT DOESN'T ALLOW THEM TO BE SOPHISTICATED. THEY MIGHT BE A GOOD MODEL. AND I WOULD SAY ONE OF THE THINGS I WAS SAYING IN MY TALK, YOU KNOW, DROSOPHILA AND C. ELEGANS, YOU PROBABLY NEVER REALLY WILL BE ABLE TO UNDERSTAND THEM IN THE FIELD. RATS AND ZEBRAFISH, WE CAN. WE CAN DO THAT WITH FISH, WITH RATS, YET ALMOST NO ONE IS WORKING WITH RATS OR ZEBRAFISH IN THE FIELD. WHY NOT? MAYBE WE NEED TO PUSH THIS CONNECTION BETWEEN THESE SCALES. AND I DON'T SEE A PROBLEM WITH USING A VERY STRONG MODEL ORGANISM LIKE A RAT OR ZEBRAFISH TO MAKE THESE POWERFUL CONNECTIONS ACROSS SCALE BECAUSE WE CAN'T JUST RANDOMLY JUST -- I THINK THE OTHER OPTION, THE OTHER THING THAT WAS MENTIONED, CHOOSE AN ANIMAL FOR ITS SPECIALIST CAPABILITIES. MOLLY MENTIONED THAT. ABSOLUTELY RIGHT. FULLY AGREE. COMPLETELY AGREE. REALLY IMPORTANT. WE NEED TO HAVE NON-MODEL ORGANISMS, ABSOLUTELY. BUT WITH THE MODEL ORGANISMS WE DO HAVE, CAN WE MAYBE CONSIDER PROGRAMS THAT SPAN FROM THE WILD TO THE LAB BECAUSE I THINK THE TYPES OF TECHNOLOGY I'VE BEEN HEARING ABOUT TODAY COULD BE APPLIED IN THE NEAR FUTURE IF NOT NOW TO NATURALISTIC CONDITIONS OR EVEN NATURAL CONDITIONS, I THINK UNDERSTANDING HOW RATS BEHAVE IN THE WILD WOULD BE PHENOMENAL. THAT'S MY FEELINGS, I THINK WE COULD COMBINE THIS IN A VERY SYNERGISTIC WAY. >> MOLLY? >> WE'LL HAVE TO END SOON. >> I WAS JUST GOING TO POSIT MAYBE WE SHOULD GET AWAY FROM SPECIALISTS AND FIND THE LOWEST COMMON DENOMINATOR, IF WE STAY WITH DEPRESSION, SOME KIND OF SIMPLE BEHAVIORAL MARKER FOR WITHDRAWAL AND THEN IDENTIFY THE SAME COMPONENTS THAT ARE OPERATING OR DIFFERENT COMPONENTS THAT ARE OPERATING ACROSS BRAINS IN DIFFERENT LINEAGES. I THINK THAT MIGHT BE A SIMPLER WAY TO FIND A TREATMENT. IAIN, YOUR SYSTEM IS AMAZING BUT NOT EVERYBODY CAN CREATE HOLE HOLE GRAPH -- HOLOGRAPHICS AND I'M WONDERING IF WE COULD GO THE OPPOSITE DIRECTION. >> I TOOK THE SLIDE OUT BECAUSE OF TIME BUT I THINK DEMOCRATIZING TECHNOLOGIES, MAKING THEM CHEAP AND EASY FOR EVERYBODY IS A HUGE PRIORITY FOR US. I WOULD BE SURPRISED IF IN FIVE YEARS' TIME WE CAN'T PRODUCE A SYSTEM THAT ANYONE CAN USE FOR THIS TYPE OF TECHNOLOGY. MOLLY, I ALSO FULLY AGREE WITH WHAT YOU SAID. HAVING THAT EVOLUTIONARY PERSPECTIVE, BUT THEY ARE NOT MUTUALLY EXCLUSIVE. ONE CAN LOOK INTO DIVERSITY OF BIODIVERSITY TO REALLY UNDERSTAND THINGS. >> LAST COMMENT. >> HI. YES, I WOULD ALSO LIKE TO ECHO WHAT MOLLY JUST SAID. I THINK LOOKING AT WIDER RANGE OF SPECIES IS VERY IMPORTANT. IN PARTICULAR, I THINK WE COULD BE A BIT MORE STRATEGIC IN PICKING THE SPECIES. IN PARTICULAR, WE DON'T WANT TO -- I THINK WITHOUT DENYING THE VALUE OF CONVERGENT EVOLUTION, WE DON'T WANT TO STUDY SOCIAL BEHAVIOR, FOR EXAMPLE, IN BEES, MAY GIVE US A LOT OF INTERESTING INSIGHT, BUT IT'S DEFINITELY EVOLVED INDEPENDENTLY FROM SOCIAL BEHAVIOR AND HUMANS. ALSO PROBABLY IN THE CASE OF OTHER ANIMALS, DOGS, WOLVES, ET CETERA, SO I THINK WE CAN -- I THINK WE SHOULD PICK SPECIES ON THE BASIS OF THE WINDOW THEY GIVE US INTO SHARED MECHANISMS THAT CONCERN US. SO, IN THAT SENSE, SPECIES THAT DIVERGE FROM US AT A PARTICULAR MOMENT IN THE PAST BUT THEN DID NOT CHANGE TOO DRAMATICALLY. SO IN THAT SENSE ZEBRAFISH ARE WONDERFUL. BUT THEY REALLY DID CHANGE A LOT. THEY DUPLICATED THEIR GENOME, CHANGED THEIR FOREBRAIN DEVELOPMENT DEVELOPMENTALLY, CAME UP WITH A LOT OF THINGS THAT ARE SHARED COMMON ANCESTORS DID NOT HAVE. BIRDS LIKEWISE, DEVELOPED ALL KINDS OF AMAZING THINGS BUT TOTALLY INDEPENDENTLY. I THINK WE GET MORE OUT OF STUDYING SPECIES THAT SHALL HIGHLY NEGLECTED. BASAL VERTEBRATES LIKE SHARKS THAT DID NOT CHANGE SO MUCH, IF WE PICK THE RIGHT SPECIES. LIZARDS, OTHER REPTILES. I THINK -- I'M BLOCKING ON HIS NAME. SAID RECENTLY THAT IN NEUROSCIENCE WE USED TO STUDY MANY MORE SPECIES. NOW WE'RE BECOMING MONKEYS, RATS, ZEBRAFISH AND DROSOPHILA. AND I THINK WE NEED TO LOOK VERY BROADLY. I THINK WE CAN PICK SPECIFIC SPECIES, BASED ON PHYLOGENETIC RELATIONSHIPS WHILE STILL B RIG -- BEING BROAD. I WOULD LIKE TO ACKNOWLEDGE PART OF THE REASON WE SPECIALIZE IS SIMPLY BECAUSE THE MORE PEOPLE STUDY A CERTAIN SPECIES, THE MORE EASY IT IS. I STUDY RHESUS MONKEYS BECAUSE SO MANY OTHER PEOPLE DO, I CAN DRAW UPON THAT KNOWLEDGE. IT WOULD BE USEFUL TO DO SOMETHING MORE LIKE, FOR EXAMPLE, WHAT LEAH KRUBITZER DOES OR JOHN KOZ, MANY SPECIES OF PRIMATES, INSIGHT INTO THINGS RELEVANT TO HUMANS. >> I WANT TO SAY A COUPLE THINGS. WE HAVE TO WRAP UP. ONE, SOUNDS LIKE THERE MAY BE SOME MORE ROOM FOR DISCUSSION ABOUT WHICH SPECIES WE WANT TO BE STUDYING AND WHY BUT AGREEMENT STUDYING MULTIPLE SPECIES AND CONSIDERING CROSS-SPECIES IS IMPORTANT AND THE CONTEXT OF THIS WORKSHOP, IT'S IMPORTANT TO THINK HOW DO WE MEASURE BEHAVIORS OF DIFFERENT SPECIES. SO IF WE WANT TO STUDY SHARKS HOW ARE WE GOING TO MEASURE SHARK BEHAVIOR, ZEBRAFISH, AND HOW ARE WE GOING TO MEASURE HUMAN BEHAVIOR, SO ON. WE'RE KEEPING THE QUESTIONS IN THE CHAT AND Q&A TO SEED TOMORROW'S BREAKOUT SESSIONS SINCE WE HAVEN'T HAD TIME TO GET TO ALL OF THEM NOW. THERE'S SO MUCH TO TALK ABOUT. FOR RIGHT NOW WE'RE GOING TO MOVE TO PANEL 2. >> THAT WAS A RICH DISCUSSION. THANK YOU, JANINE. DIFFICULT ASPECT TO FOLLOW BUT IT'S MY PLEASURE TO INTRODUCE PANEL 2, ON SENSING BEHAVIOR IN ITS ENVIRONMENT. CENTRAL QUESTION FOR THIS PANEL IS HOW DO WE CAPTURE THE INFORMATION NEEDED FROM INDIVIDUAL ORGANISMS, SOCIAL GROUPS, PHYSICAL ENVIRONMENTS, SO WE CAPTURE COMPLEXITY OF BEHAVIOR. REALLY THE QUESTION, ONE WAY TO TRANSLATE THE BIG QUESTION, WHAT ARE THE DIFFERENT TYPES OF DATA NEEDED TO DO MODELS OF BEHAVIOR THAT REFLECT MULTI-DIMENSIONAL COMPLEXITY OF BEHAVIOR IN THE CONTEXT IN WHICH THE BEHAVIOR OCCURS NATURALLY, THE IDEA THAT CAME UP IN THE PREVIOUS PANEL. AND WHAT ARE SOME OF THE TECHNOLOGIES IN PARTICULAR SENSOR TECHNOLOGIES, WHAT ARE INFORMATIC TOOLS THAT ARE AVAILABLE AND WHICH ONES ARE NEEDED TO COLLECT AND INTEGRATE THESE TYPES OF DATA TO BUILD CONCEPTUAL FRAMEWORKS IN COMPUTATIONAL MODELS. WITH THAT THE SPEAKERS WE HAVE LINED UP, A GREAT SET, GALIT PELLED, BEN HAYDEN, MALCOLM MacIVER, AND RAJASHREE BASKARAN, AND WE SWITCH TO THE DISCUSSANTS, PETER HARTWELL, BOB DATTA, AND ANDRE GREEN. WITH THAT I HAND IT OVER TO GALIT. >> HI. I'M GALIT PELLED, MICHIGAN STATE UNIVERSITY. WE'RE INTERESTED IN THE PRACTICE OF IMPROVING MOTOR FUNCTION THAT WILL LEAD TO HIGHER LEVELS OF SPEED AND STRENGTH, COGNITIVE FUNCTION INCLUDING DECISION MAKING AND LEARNING, DEFINED AS BEHAVIOR, AND TO UNDERSTAND NORMAL BEHAVIOR, TO ASSESS CHANGES IN BEHAVIOR WE NEED TOOLS TO MEASURE BEHAVIOR. THERE WAS A GREAT DISCUSSION BEFORE, WHETHER WE SHOULD BE LOOKING AT DIVERSE ANIMAL MODELS TO UNDERSTAND BEHAVIOR. IN MY LAB I'LL SHOW YOU THAT WE ARGUE THAT THIS IS NECESSARY. SO HERE YOU SEE IN THE NEXT FEW SLIDES I'M GOING TO SHOW YOU TOOLS THAT WE'RE DEVELOPING AND CONCEPTS WE'RE STUDYING IN MORE CONTROLLED NEURAL SYSTEMS HOW THESE MAY BE ADOPTED TO MORE COMPLEX SYSTEMS. SO UNDERSTANDING DECISION MAKING AND GOAL-ORIENTED MOVEMENT IS REALLY A MAJOR GOAL OF NEUROSCIENTISTS AND BEHAVIORAL SCIENTISTS, THESE ARE OF COURSE INTEGRATION OF VERY COMPLEX, THE BRAIN AND PERIPHERAL NERVOUS SYSTEM. YOU SEE AN OCTOPUS, THERE ARE 300 SPECIES, THIS IS THE THE ONLY SPECIES OF OCTOPUS THE ENTIRE GENOME HAS BEEN SEQUENCED, YOU CAN IMAGINE ALL SORTS OF AMAZING THINGS WE WILL BE ABLE TO DO WITH THAT IN THE FUTURE. AND YOU CAN SEE IT HERE IN ITS TANK. INTERACTING, TRYING TO GRASP THE FINGER OF A STUDENT. AND IT'S QUITE AMAZING. SO WE'RE TRYING TO STUDY OCTOPUS GRASPING MOVEMENT HOPING TO IDENTIFY ALGORITHM THAT DESCRIBE HOW THE OCTOPUS CAN ADAPT MOVEMENT MIDWAY, CAN WE DEVELOP NEW TYPE OF DEVICES RAPID AND ADAPTIVE SUCH AS OCTOPUS ARM. IT HAS THE HEART, BLUE BLOOD, CAMOUFLAGES, REGENERATES. AND THEY ACTUALLY EXISTED, THERE'S THE FIRST FOSSILS OF OCTOPUS DATED TO 296 MILLION YEARS AGO. SO EVEN BEFORE DINOSAURS. SO IN THE COURSE OF EVOLUTION, THE OCTOPUS AMAZINGLY CAME UP WITH MANY SIMILAR SOLUTIONS FOR PHYSIOLOGY AND SENSING LIKE VERTEBRATES. FOR EXAMPLE, ITS EYES, OPTIC LOBES ARE SIMILAR TO VERTEBRATE SYSTEMS, BUT IN TERMS OF MOTION AND MOVEMENT IT DEVELOPED A COMPLETELY ALTERNATIVE STRATEGY. SO, IT HAS A HUGE NERVOUS SYSTEM WITH 500 MILLION NEURONS. SIMILAR TO WHAT A DOG HAS. BUT MOST OF THE NEURONS ARE NOT IN THE BRAIN. THEY ARE ACTUALLY IN THE ARM. AND EACH SINGLE ARM HAS AN AXIAL CORD THAT ACTS LIKE A SPINAL CORD. AND MOST OF THE DECISIONS THAT THE OCTOPUS MAKE IS NOT IN ITS BRAIN, IT'S MAKING IT IN THE ARMS. THE DISEMBODIED ARM IS KNOWN TO KEEP MOVING IN A WAY THAT AN INTACT OCTOPUS DOES. YOU CAN IMAGINE HOW THE OCTOPUS NOW GIVES US REALLY A WAY TO LOOK AT NEURAL PATTERNS IN NEURAL NETWORKS THAT ARE ASSOCIATED WITH MOVEMENT. SO, TO LOOK AT MOVEMENT, THE OCTOPUS -- HERE YOU CAN SEE WE'RE USING A.I. WITH DEEP LAB CUT, CAN PLACE 16 MARKERS ON A SINGLE ARM TO MEASURE MOVEMENT. SO THERE'S EIGHT ARMS. THERE'S CONSTANT MOVEMENT. THERE'S CONSTANT ELONGATION AND SHRINKING. IF WE WANT TO MEASURE OCTOPUS MOVEMENT IN NATURAL ENVIRONMENT, DEEPLABCUT IS A GOOD WAY TO START. TO GET HIGH RESOLUTION KINEMATICS WE HAVE TO GO AND USE DIFFERENT TECHNIQUES. AND WHAT YOU CAN SEE HERE WE CAN USE REFLECTIVE SENSORS, REFLECTIVE MARKERS ON AN OCTOPUS ARM, ON THE TOP IT'S AN ARM OUTSIDE OF THE WATER. AND NOW WE CAN ALSO USE IT IN AN OCTOPUS INSIDE THE WATER. I'M NOT SURE WE CAN SEE THE MOVIE. THANK YOU. OF THE PREVIOUS -- YEAH. SO SENSORED REFLECTIVE SENSORS OUTSIDE AND OF COURSE INSIDE THE WATER. WE ALSO WANT TO HAVE THE NEURAL CORRELATES THAT WE RECORD AT THE SAME TIME WITH THE MOVEMENT. AND FOR THAT TODAY WE CAN PLACE A RAY OF ELECTRODES INTO THE OCTOPUS ARM, AND YOU CAN ALREADY SEE HERE WE CAN RECORD MORE THAN 16 CHANNELS AT A TIME FOR OCTOPUS ARM AND START SEEING DIFFERENT PATTERNS IN TERMS OF THE AMPLITUDE, IN TERMS OF THE FREQUENCY, WHEN WE GIVE DIFFERENT TYPE OF STIMULATION. CENTRAL STIMULATION, PERIPHERAL AND SO ON. OF COURSE WE WANT TO RECORD ALL OF THAT IN AN INTACT OCTOPUS, IN ITS NATURAL ENVIRONMENT. WHEN WE CAN PROVIDE NATURAL CUES AND STIMULATION. SO NOW WITH WITH CINDY WE WORK ON THE ARM, AND HAVE IT IN ITS BIG TANK RECORDING ACTION POTENTIALS AND AT THE SAME TIME RECORDING ITS MOTION. WE CAN EVEN GO DEEPER AND LOOK AT THE NEURAL NETWORK, SINGLE ACTION POTENTIALS, YOU CAN SEE HERE WE CAN PLACE -- CAN WE PLAY THAT VIDEO, TOP LEFT? WE CAN PLACE A SLICE OF AN OCTOPUS ARM ON A MULTI-ELECTRODE ARRAY, THIS DISH HAS 26,000 SINGLE ELECTRODES, AND YOU CAN SEE HERE THAT WE CAN DELIVER EITHER A CENTRAL STIMULATION OR PERIPHERAL STIMULATION, AND WE CAN START SEEING DIFFERENT PATTERNS OF ACTIVITY. SO, ALL OF THAT TOGETHER HOPEFULLY WILL ENABLE US TO IDENTIFY -- YES, TO IDENTIFY THE CIRCUITS THAT ARE ASSOCIATED WITH GRASPING, HOW IS THE OCTOPUS HAVING -- ADAPT TO GRASPING AND CHANGING ITS MOTION MIDWAY, AND WE WILL BE -- WILL WE BE ABLE TO IDENTIFY THESE ALGORITHMS AND TEST THEM IN FLEXIBLE MATERIAL-BASED BIOROBOTIC ARMS. I SHOWED HOW WE CAN USE HIGH DIMENSIONAL TECHNOLOGIES TO MEASURE -- >> ONE MINUTE. >> OKAY. >> ONE MINUTE LEFT. >> YES, WHERE ENWE EN-- WHEN WE MEASURE IN OCTOPUS WITH SPECIAL CHALLENGES, IN MY LAB WE'RE INTERESTED IN OTHER ANIMAL MODELS, HOW WE CAN ASSESS BEHAVIOR IN LARGE ANIMALS. WE SPECIFICALLY ARE INTERESTED IN BRAIN INJURIES, OVER THE PAST FEW YEARS WE DEVELOPED A MODEL OF BRAIN INJURY, I WANT TO SHOW QUICKLY HOW WE USE EXACTLY THE SAME METHODS THAT WE USE IN THIS UNCONVENTIONAL SPECIES TO MORE PERHAPS CONVENTIONAL WAY. YOU CAN SEE WE'RE USING DEEPLABCUT TO LOOK AT MOTION, LOCOMOTION, THE SAME MOTION REFLECTIVE SENSORS, YOU CAN SEE THE PIG WITH MARSHMALLOWS, IT'S GOING TO BEHAVE WELL. AND ALSO THE WE CAN GET HIGH RESOLUTION KINEMATIC DATA, DATA ANALYSIS OF PIG MOTION BEFORE THE INJURY AND AFTER THE INJURY. OUR PIGS ALSO WEAR WEARABLE SENSORS. THEY EVER WEARING FIT BITS. SO WE CAN TELL HOW MANY STEPS, WHEN, AND IN THE NATURAL ENVIRONMENT WE CAN ALSO MEASURE HOW MUCH THEY SLEEP A DAY, WHEN DO THEY SLEEP, WE HAVE VIDEO CAMERAS TO LOOK WHO THEY LIKE TO SLEEP WITH. AND SO ON. SO I DIDN'T INCLUDE THAT BUT WE'RE INTERESTED IN PERSONALITY CHANGES IN PIGS, AND THEY HAVE REALLY VERY DEVELOPED PERSONALITY. WE CAN MEASURE AGGRESSION, DEPRESSION, ANXIETY, AND SHORT-TERM MEMORY AND LONG-TERM MEMORY, DEVELOPED A WHOLE ARRAY OF TECHNOLOGIES THAT WE CAN MEASURE IN THESE SPECIES. SO JUST TO CONCLUDE, I SHOWED YOU THAT WE DEVELOPED TOOLS TO ADDRESS -- TO RECORD BEHAVIOR, PERHAPS IN UNCONVENTIONAL SPECIES, AND IN LESS CONVENTIONAL ENVIRONMENTS, BUT HOPEFULLY THIS WILL BE ABLE TO BE ADOPTED IN MORE TRADITIONAL SPECIES. AND THE CONCEPTS THAT WE'RE GOING TO LEARN WILL HOPEFULLY TRANSLATE TO REALLY UNDERSTANDING OF HUMAN BEHAVIOR. THANK YOU. THANK YOU. BEN, ARE YOU READY? >> THANK YOU VERY MUCH. THANK YOU FOR INVITING ME. THE TITLE IS THE BEHAVIORAL IMAGING REVOLUTION, BEHAVIORAL IMAGING IS A TERM IN THE LAB TO REFER TO WHAT A LOT OF THINGS PEOPLE ARE TALKING ABOUT TODAY IS ABOUT, BASICALLY THE USE OF CAMERAS AND DOWNSTREAM SOFTWARE AND TECHNOLOGY TO BASICALLY SAY SOMETHING SOPHISTICATED ABOUT BEHAVIOR. I WANT TO START WITH ONE THOUGHT. WHICH IS THAT BEFORE THE TELESCOPE WAS INVENTED YOU COULD SEE THE STARS. THE DIFFERENCE BETWEEN THE TELESCOPE AND YOUR AS IS THEY LET YOU DO SOMETHING BETTER AND SEE BETTER, ORDERS OF MAGNITUDE BETTER, WHAT LED TO REVOLUTION 500 YEARS AGO, THE SAME THING IS HAPPENING NOW IN BEHAVIOR. SO, BASICALLY, LET'S SEE IF I CAN GET THIS VIDEO TO PLAY HERE. YEAH, THIS IS A VIDEO OF A MONKEY AT THE FIELD SITE IN SANTIAGO WHERE I'VE DONE FIELD WORK, WHAT REALLY STRIKES ME, ALWAYS HAS STRUCK ME ABOUT MONKEYS IN THE WILD, IS THAT THEY ARE JUST GENERATING ENORMOUS AMOUNTS OF DATA. DATA IS JUST LEAKING OUT CONSTANTLY. I JUST AM LIKE HOW DO WE GATHER ALL THIS DATA? I'VE BEEN TRYING TO FIGURE OUT WAYS TO TRY TO DO THAT. AND EVERYBODY HAS BEEN SAYING, I DON'T NEED TO REPEAT IT, BUT DATA IS NOT JUST INTERESTING TO PRIMATOLOGISTS, BUT OTHERS TOO. THAT'S WHAT THE SYMPOSIUM IS ABOUT. AND MY LAB, UNIVERSITY OF MINNESOTA, WE HAVE OUR OWN BESPOKE SYSTEM FOR STUDYING MONKEYS. THIS IS A PICTURE OF A MONKEY IN OUR CAGE SYSTEM MOVING AROUND INTERACTING WITH FEEDERS. WE HAVE 62 CAMERAS TRACKING HIM, A BUNCH OF SOFTWARE WE WROTE TO DO THAT. AND MY GOAL TODAY IN MY BRIEF TIME IS JUST TO TALK ABOUT CONSEQUENCE OF TRACKING THE MONKEYS, ABILITY THAT HAS TO DO BEHAVIORAL IMAGING TO SEEP WHAT THE BEHAVIOR IS AND THE WAY THAT CHANGED HOW TO BE BEHAVIOR NEUROSCIENTISTS. WE'RE FUNDAMENTALLY INTERESTED IN THE RELATIONSHIP BETWEEN BRAIN AND BEHAVIOR IN MY LAB. I KNOW THIS IS A BROAD MEETING WITH A BUNCH OF PEOPLE. NOT EVERYBODY IS THINKING IN TERMS OF THAT. BASICALLY WE WANT TO SEE HOW BRAIN ACTIVITIES PREDICTS AND DRIVES BEHAVIOR. THAT'S NOT EASY, EVEN WITH HIGH QUALITY BEHAVIORAL IMAGING, YOU HAVE TO WORRY, FOR EXAMPLE, ABOUT CONFOUNDERS, ABOUT THINGS THAT ARE CORRELATED WITH THE THING YOU'RE INTERESTED IN, AND THAT MAY ALSO BE DRIVING THE NEURAL ACTIVITY. SO, WE DO THINGS, STANDARD THINGS LIKE CAREFUL TASK DESIGN, BIG DATA, EVEN BIGGER DATA, MORE SOPHISTICATED ANALYSIS, IN ORDER TO DEAL WITH THE PROBLEM OF CONFOUNDERS, BUT THE REAL THING THAT KEEPS HITTING US IN HEAD AS WE DO THIS RESEARCH IN MY LAB IS THAT THE ABILITY TO MEASURE THE BEHAVIOR IS LIKE THE ABILITY TO LOOK AT STARS IN THE TELESCOPE, LEADS TO DIFFERENT QUESTIONS. IT DEFINES YOU TO FIND DIFFERENT THINGS TO BE INTERESTING. IN THE KALES OF A -- CASE OF A MONKEY, WE'VE HAD THE MONKEY SIT IN A CHAIR AND DO A LITTLE TASK WITH THEIR EYES. WITH OUR SYSTEM IN THE BIG CAGE, THE FIRST THING WE DID, WE TOOK HUGE AMOUNTS OF DATA, HUNDREDS OF HOURS OF A FORAGING TASK, WE DID A SIMPLE DIMENSIONALITY ANALYSIS, THE BEHAVIOR DIVIDES INTO 49 ISLANDS. THE NUMBER IS NOT PARTICULARLY IMPORTANT. IT DEPENDS ON THE WAY YOU DO THE PRE-PROCESSING. BUT YOU GET THESE DISCRETE ISLANDS OF BEHAVIOR, THAT CORRESPOND REALLY TO NATURAL THINGS THAT YOU SEE IN THE BEHAVIOR. AND THEN THESE HAVE A HIGH ORGANIZATION, YOU SHOULDN'T BE TOO SURPRISED. WE HAVE THE MONKEY DO A TASK OR DIFFERENT TASK, TASK OFF, WE SEE SUBTLE CHANGES IN THE STRUCTURE OF THE THIS HIERARCHY. AND I'M NOT GOING TO TALK ABOUT WHAT THAT MEANS TODAY EXCEPT TO SAY THAT THE KEY POINT OF THIS FOR THIS TALK, THIS ISN'T EVEN A QUESTION WE THOUGHT ABOUT ASKING IF WE DIDN'T HAVE BEHAVIORAL IMAGING, DIDN'T HAVE ABILITY TO DO HIGH QUALITY HIGH RESOLUTION TRACKING AND BEHAVIOR. WHEN THE MONKEY IS IN A CHAIR YOU ASSUME HE'S IN THE STATE OF DOING THE TASK, THE BEHAVIOR IS ON OR OFF TASK. THERE'S A VERY SOPHISTICATED COMPLEX STRUCTURE AND WE'RE LOOKING NEURAL CORRELATES. ONE MORE EXAMPLE TO ILLUSTRATE, THE WAY THAT BEHAVIORAL IMAGING HELPS YOU THINK ABOUT AND ASK NEW QUESTIONS, SO THE CORE TOPIC OF MY LAB'S RESEARCH IS ABOUT ECONOMIC DECISION MAKING, BASICALLY HOW HUMAN OR A MONKEY IN OUR CASE WILL CHOOSE BETWEEN THINGS, BASED ON MICROECONOMIC THEORY. YOU'RE SITTING AROUND DOING NOTHING, TWO OPTIONS APPEAR, YOU CHOOSE BETWEEN THEM AND IT'S OVER, YOUR CHOICE IS OVER. THAT KIND OF APPROACH TO THINKING ABOUT WHAT A CHOICE IS LEADS TO THESE DISCRETE MODEL OF HOW CHOICE WORKS AND IMPLICATION IS THESE WILL EACH CORRESPOND TO BRAIN AREA AND THAT DRIVES THE SHAPE OF THE NEUROSCIENCE OF ECONOMIC CHOICE. TO EMPHASIZE THE THEME NATURAL BEHAVIOR DRIVEN BY IMAGING LEADS TO DIFFERENT CONCEPTUALIZATIONS OF HOW THESE THINGS WORK, I THINK OF CHOICE AS CONTINUOUS, I FOUND THIS VIDEO OF DOGS CHASING A RABBIT. THIS IS WHAT I THINK MUCH AS CANONICAL ECONOMIC DECISION MAKING, THE DOG HAVING TO CHOOSE WHICH WAY TO GO AT EVERY SINGLE MOMENT. IT'S A CONTINUUM. A CONTINUUM OF CHOICE, CONTINUUM OF POSSIBLE ACTIONS. AND HE HAS TO DECIDE HIGHER LEVEL THINGS LIKE WHAT STRATEGY HE'S GOING TO EMPLOY TO FOLLOW THE RABBIT. THESE ARE THINGS THAT COULD IN A CLUMSY WAY DO WITH MICROECONOMICS BUT WOULDN'T BE A NATURAL FIT. SO THIS HAS LED OUR LAP TOWARDS THINKING OF CONTROL THEORY, ENGINEERING CONTROL THEORY, SORT OF AS A FOUNDATION FOR THINKING ABOUT WHAT'S DRIVING BEHAVIOR. IT'S BEEN PLEASING FOR ME TO SEE THE TALKS SO FAR TODAY, A LOT HAVE CONVERGED ON THE SAME IDEA THAT REALLY WE NEED TO USE CONTROL THEORY AND FEEDBACK AS A WAY OF UNDERSTANDING HOW DECISIONS OCCUR. I'M ALSO GOING TO TALK ABOUT ONE OTHER THING. JUST ONE MORE EXAMPLE, IN THE NATURAL WORLD, IN THE LABORATORY YOUR MONKEY OR PERSON IS SITTING THERE, YOU EXPLAIN THE TASK, THEY UNDERSTAND, THEY HAVE BEEN DOING IT FOR MONTHS, IF THEY ARE A MONKEY. BUT IN THE REAL WORLD WE'RE LIKE CONSTANTLY BASICALLY STARVING FOR INFORMATION. WE'RE HIGHLY UNDERSPECIFIED. I PUT THIS PICTURE OF THE MONKEY TRYING TO DRIVE A CAR, IT'S HOW I FEEL IN MY LIFE, BASICALLY TRYING TO DO COMPLICATED THINGS I'M WOEFULLY INADEQUATELY PREPARED FOR, CONSTANTLY NEEDING INFORMATION, SEEKING INFORMATION, SO MUCH SO TO DRIVE ALL MY DECISION MAKING, AND I THINK THIS IS SOMETHING FUNDAMENTAL. IT HELPS EXPLAIN THE WAY MONKEYS DEAL WITH UNCERTAINTY ISN'T PARTICULARLY WELL DESCRIBED BY ECONOMIC THEORIES, ECONOMIC THEORIES, WE'VE HAD TROUBLE WITH THAT, BEEN DOING THAT 15 YEARS. INCORPORATING CONCEPTS OF THE NEED TO RESOLVE UNCERTAINTY HAS HELPED US UNDERSTAND THAT KIND OF BEHAVIOR MORE. THESE ARE A FEW EXAMPLES OF THE WAY THAT TAKING THIS NATURALISTIC PERSPECTIVE WHICH IS REALLY ULTIMATELY EVOLUTIONALILY DRIVEN TO ASK DIFFERENT QUESTIONS, INEVITABLY GIVE US DIFFERENT ANSWER. ONE-MINUTE WARNING. I'LL SAY QUICKLY WE'RE RECORDING BRAIN ACTIVITIES IN THESE MORE NATURALISTIC TASKS. THIS OF THIS MODEL OF THE BRAIN WITH 150 AREAS, EACH WITH SOME PARTICULAR FUNCTION, DOESN'T REALLY WORK AS WELL. A LOT OF THESE BRAIN AREAS SEEM TO HAVE A FRANK MOTOR FUNCTION WHEN WE HAVE THE MONKEYS MOVING AROUND IN THE REAL WORLD, THAT SEEMS TO BE TRUE FOR RATS AND MICE AS WELL. THAT IS CAUSING US TO TRY TO THINK OF WAYS TO RETHINK WHAT BRAIN STRUCTURES DO AND HOW THEY RELATE TO EACH OTHER. AGAIN WE WOULDN'T HAVE DONE THAT IF WE DIDN'T HAVE ALL THIS DATA COMING FROM FREELY MOVING MONKEYS DRIVEN BY ABILITY TO MEASURE THEIR BEHAVIOR, TO IMAGE THEIR BEHAVIOR. LET'S SEE. IN THE FUTURE WE'RE GOING TO BE DOING EXPERIMENTS MORE AND MORE TOWARDS THIS WORLD, MONKEY BRAIN EMBODIED IN THE BODY OF MONKEY, EMBEDDED IN A COMPLEX NATURAL ENVIRONMENT HELPING US DO NEUROSCIENCE IN THE 21ST CENTURY. THANK YOU. >> THANKS VERY MUCH, BEN. MALCOLM, ARE YOU READY? >> YEAH, CAN YOU HEAR ME? >> YEP. >> GREAT. LET ME SHARE MY SCREEN. OKAY. DO YOU SEE MY FIRST SLIDE? >> YES. >> GREAT. ALL RIGHT. SO I'M MALCOLM MacIVER, I'M HAPPY TO TALK TO YOU TODAY. TOPIC OF MY QUICK TALK IS WIDENING THE APERTURE OF BEHAVIORAL RESEARCH. AND BY THIS I MEAN, YOU KNOW, 20 YEARS AGO WHEN I FINISHED MY PhD we in decision making neuroscience and other areas of behavioral research, we settled on idealized approaches to alternative choice, to make the task somewhat easier to manage and control. SIMILARLY AT THE NEURAL LEVEL, RECORDING SINGLE NEURONS AT A TIME, MODELING THEM WITH DEPARTMENTAL MODELS, WAS PRETTY COMMON, WHAT'S REALLY COOL IN THE PAST 20 YEARS IS SEEING THE APERTURE WIDEN TO ENCOMPASS SOME OF THESE THINGS NATURALISTIC BEHAVIOR IS COMING ON STRONG, WHOLE ANIMAL SIMULATION I'LL ADVOCATE FOR TODAY. WE SEE SOME IN THE FIELD, EVOLUTIONARY APPROACHES WE'VE HEARD ABOUT TODAY, SOME ROBOTICS. AND OF COURSE THERE'S LOTS OF OTHER THINGS, GENETICS APPROACHES, I DIDN'T WANT TO GET IT TOO MESSY, I DON'T HAVE THEM ON HERE. THE THE QUESTION IS HOW TO OPERATIONALIZE CONSENSUS THAT EMERGED INTO ACTIONABLE INTELLIGENCE FOR THOSE LABS WHICH NEED TO PURSUE IF THEY ARE NOT ALREADY ON THIS TRACK. AND TO MOTIVATE MY APPROACH, I THINK IT'S USEFUL TO POINT OUT THE BODY IS A COOPERATIVE ECOSYSTEM OF 40 TRILLION HUMAN CELLS, ANOTHER 40 TRILLION BACK TEAR VIRUSES AND FUNGI. BEHAVIOR IS THE OUTPUT OF THE ENTIRE ECOSYSTEM. STUDY IS A COOL OPPORTUNITY FOR PUTTING OUR ARMS AROUND THE ENTIRETY OF THE ORGANISM. TO MAKE THIS POINT, I LOVE THIS VIDEO OF PASSIVE DYNAMIC WALKER FROM 20 YEARS AGO AS WELL. THIS IS A ROBOT THAT HAS PRETTY MUCH THE SAME DYNAMICS AS A HUMAN WALKER. YET THERE'S NO MOTORS, NO CONTROL, THE ONLY INPUT OF ENERGY IS INCLINED PLANE. AND, YOU KNOW, IT GIVES A FORM OF DYNAMICS, DEFAULT DYNAMICS, ALMOST FOR FREE. AND SO WHAT THIS POINT AMOUNTS TO ESSENTIALLY THE BODY THROWS EVERYTHING IT CAN AT THE PROBLEM OF SURVIVAL, OF COURSE. AND WE HAVE TISSUES THAT HAVE VERY LOW ENERGETIC FOOTPRINTS, WATTS PER KILOGRAM LIKE BONE, BILLIONS OF YEARS, GRAVITY, DEALING WITH MOVEMENT, GRAVITATIONAL FIELD, AND NEURONS REALLY SPECIALIZE ON CONTINGENCIES AT THE MILLISECOND TIME SPAN, OF COURSE VERY IMPORTANT, BUT WE NEED TO OPEN OUR PURVIEW TO THE WHOLE THINGS. COMPUTATIONS LIKES SPREAD OVER THE ENTIRETY OF THE DOMAIN. HOW DO YOU DO THIS, BROADEN IT? I'M GOING TO ARGUE, WELL, USING TOOLS AS SYNTHESIS IS NOT NOVEL BUT HOW DO WE ACTUALLY USE THEM AND WHAT ARE THEY? THE POINT IS COUNTER INTUITIVELY WHAT SEEMS TO BE MAKING OUR JOB HARDER INCORPORATING A WHOLE OTHER BUNCH OF ASPECTS OF AN ANIMAL DOES MAKE IT EASIER IN THE END. SO SOME ENGINES OF SYNTHESIS ARE WHOLE ORGANISM, AT-ENVIRONMENT SIMULATION, ROBOTICS, EVOLUTIONARY APPROACHES, VIRTUAL REALITY. HERE IS OLD ELECTRIC FISH WORK FROM MY Ph.D. TO RECONSTRUCT SIGNAL INPUT, FLUCTUATION OF TRANSDERMAL VOLTAGE DOING A NATURALISTIC TASK. YOU USE MOTION CAPTURE DATA AND YOU SYNTHESIZE WITHIN A COMPUTER MODELS OF ELECTRIC IMAGE FORMATION, WHERE THE SENSORS THE BODY, ELECTRIC FIELD MODEL, A MODEL OF THE PREY THE ANIMAL HUNTS IN NATURAL ENVIRONMENT, AND THEN BEHAVIORAL MODEL CONSTRAINED BY EMPIRICAL DATA COLLECTED FROM PRECEDING STEP AND YOU GET VOLTAGE HISTORIES FROM A SPIKE MODEL, YOU CAN GET SPIKE TRAINS, ULTIMATELY WE GET THROUGH A LOT OF COMPUTATION. AFFERENT SPIKE TRAINS ACROSS 14,000 AFFERENTS ARE ELECTRO SENSORY -- I'M GETTING A LOT OF WEIRD SOUNDS FROM ANOTHER SPEAKER I THINK MAYBE? THERE WE GO. SO NOT ONLY ARE WE GETTING INPUT ACROSS 14,000 ELECTRO SENSORY AFFERENTS BUT WE CAN FOLLOW THE SAME PROCESS TO GET THE PASSIVE ELECTRO SENSE AFFERENTS AND MECHANOAS WELL, AND LOOK AT HOW THEY ARE ACTIVATED DURING NATURAL BEHAVIORS. WE AUGMENT THIS WITH ROBOTICS APPROACH TO LOOK AT HOW MOVEMENT IS BEING CONSTRAINED BY THE ANIMAL'S BIOMECHANICS, HOW THE ANIMAL HAS TO GENERATE THRUST THAT IT'S GENERATING, AND WE USE THOSE DATA TO CONSTRAIN NUMERICAL MODELS TO EVEN FINER RESOLUTION ANALYSIS. WE DO THE SAME THING IN THE SENSORY DOMAIN WITH SENSORY ROBOTICS. I DON'T HAVE TIME TO GO INTO IT BUT IT CAN NAVIGATE AUTONOMOUSLY THROUGH PYLONS USING ELECTRIC FIELDS. PAYOUTS HERE, WELL, BEHAVIORS THAT MADE NO SENSE AT ALL WHEN LOOKED FROM STRICTLY SENSORY PROCESSING PERSPECTIVE OR INFORMATION THEORY PERSPECTIVE, ONCE WE AUGMENTS THE WAYS THE ANIMAL GENERATED THRUST AND HOW MUCH COST THAT COST ENERGETICALLY COULD MAKE SENSE OF BEHAVIORS THAT DID NOT MAKE SENSE FROM THE SENSORY OR INFORMATION THEORY PERSPECTIVE. OBVIOUSLY THE ANIMAL IS CON JOINTLY OPTIMIZING ALL OF THE THESE FACTORS, ZERO INSIGHT INTO WHAT THEY WERE WITHOUT THAT MORE SYNTHETIC APPROACH. THE SECOND PAYOFF I MENTIONED, THERE WAS A BIG CHANGE AS A RESULT OF CHARACTERIZING THE SIGNALS THROUGH THE SYNTHESIS APPROACH, IN THE WAY SIGNALS ARE FED INTO THE NERVOUS SYSTEM OF ELECTRIC FISH, NEUROPHYSIOLOGY LABS, THAT LED US TO DISCOVER NEURAL ASSESSING STRATEGIES. I'M GOING TO TALK ABOUT EVOLUTIONARY WORK WHETHER WE DISCOVERED THAT DURING THE TRANSITION TO LAND ANIMALS TRIPLED EYE SIZE, THE IDEA PERHAPS AERIAL VISION LED TO TERRESTRIALITY, HOW THE SENSORY ECOLOGY OF ANIMALS VERSUS WATER ON LAND LEADS TO DIFFERENT DECISION-MAKING STRATEGIES. IN PARTICULAR IN WATER WE HAVE THESE SENSORY BUBBLES THAT ARE VERY TIGHTLY CHAINED TO THE ANIMAL, VERY SHORT. THEY DETECT PREDATOR, HAVE TO FIRE OFF MODELERS TO SURVIVE. WITH LONG SIGHT LINES LAND PROVIDES YOU CAN IMAGINE AN ANIMAL WHO NOW HAS A MUCH LARGER SENSORIUM LOCATING TRAJECTORIES AND PICKING ONE, PLANNING. ONE OF THE THINGS WE FOUND IN MEDIUM ENTROPY OR CLUTTER ENVIRONMENTS THAT MATCH SAVANNAH ENVIRONMENTS THAT HUMAN ALSO WENT INTO, MAXIMUM ADVANTAGE AFTER CLUTTER GEOMETRY. WE'RE TESTING USE AGRO BOUGHT WITH A CO2 -- USING A ROBOT, THE AIR PUFF TURNS RED GETTING THE MOUSE TO RUN AWAY FROM THAT. PLAN A ROUTE. WE'RE DOING SLOWDOWN IN THE HABITAT, WHETHER THEY ARE ENGAGED IN PLANNING BEHAVIORS. YOU CAN SEE A FEW INDICATIONS. THERE'S DTE PHENOMENA, A HEAD WAG, YOU CAN SEE THAT HAPPENING HERE AT THIS CHOICE POINT, AS IT SENSES A ROBOT NEARBY. AND SO -- LET'S SEE. NOW, WE WANT TO -- SINCE A LOT OF PEOPLE ARE INTERESTED IN HUMAN WORK HERE I WANTED TO MENTION WE'VE TRANSLATED THIS PARADIGM INTO A HUMAN VIRTUAL REALITY SYSTEM. THIS IS A VIEW WHEN YOU WALK IN, THE PREDATOR IS A SCARY LOOKING GHOST. WE HAVE A WIRELESS SYSTEM FOR VR. AND EVERY MOVEMENT, EVERY FOOT THAT YOU TAKE, EVERY FOOT THAT YOU MOVE IN REAL SPACE IS MATCHED IN VR. SO HERE'S A PERSON IN OUR VIRTUAL REALITY ENVIRONMENT MATCHING THE ONE THE RODENTS ARE GOING THROUGH RUNNING FROM THE PREDATOR MOVING AROUND OBSTACLES TO BE HIDDEN. WE JUST STARTED THIS WORK, COMPARING TO MICE, COMPARATIVE WORK ON YOU OF HOW THE TWO ANIMALS DO PLANNING AND COMPLEX ENVIRONMENT ALSO. I WANT TO END ON A NOTE THAT I THINK THAT IT'S VERY INTERESTING TO ME THAT COLLABORATIONS AND PARTICLE PHYSICS GO TO 5,000 CO-AUTHORS, PARTICLE PHYSICISTS I KNOW THINK WE'RE WORKING ON A MUCH HARDER PROBLEM. SO I WONDER WHETHER WE NEED A COUPLE ORDER OF MAGNITUDES MORE COOPERATION IN TERMS OF EFFORT TO MAKE FASTER PROGRESS. THIS IS WHERE THIS WORKSHOP CAN REALLY HELP OUT A LOT. AND A FEW POINTS ON HOW YOU MIGHT ADD ENGINES OF SYNTHESIS WHEN NOT NATIVE TO YOUR TOOLKIT, DOING A SABBATICAL, ATTENDING A CONFERENCE, FINDING A PERSON NEWER IN THE CAREER, THESE ARE ALL APPROACHES I'VE FOUND HELPFUL AS I'VE DONE A HUGE NUMBER OF COLLABORATIONS FOR THE WORK THAT YOU'VE SEEN. SO I'D BE HAPPY TO TAKE QUESTIONS LATER DURING THE DISCUSSION SECTION. THANK YOU VERY MUCH. >> THANKS, MALCOLM. LAST SPEAKER FOR THIS SESSION IS RAJI. WE'RE READY. TAKE OVER. GREAT. THANK YOU. >> THANK YOU. GOOD AFTERNOON. THANKS FOR THIS OPPORTUNITY. I'M GOING TO COMPLETELY SWITCH TRACKS. I'VE BEEN OVERSTIMULATED IN THE LAST COUPLE HOURS, I'M GOING TO KEEP A LINGO OF ENGINEERING DOMAIN WHERE I COME FROM AND BUSINESS AND REAL LIFE APPLICATION, AND KEEP IT PLAIN. TWO MAIN IDEAS, BOTH AROUND MULTI-SCALE SENSOR FUSION, I'M A MICRO SCALE SENSOR TECHNOLOGIST, MINIATURIZING, MAKING IT POSSIBLE TO COLLECT LARGER DATASETS YOU ALL REALLY LOVE USING, KIND OF WHERE I GREW UP. BUT I WANT TO SAY, MALCOLM, I DID MY MASTER'S AT CORNELL AT THE SAME TIME, I KNOW THAT ROBOT PERSONALLY, IN THE LAB NEXT DOOR. I'M A DYNAMICS PERSON SO I ENJOYED IN A AS -- THAT ASPECT. I WANT TO DO TWO THINGS. FOR THE LAST DECADE OR SO I HAVE THOUGHT ABOUT HUMAN-MACHINE INTERFACE IN THE CONTEXT OF BUILDING NEW TECHNOLOGIES, LIKE CONSUMER TECHNOLOGIES, BE IT LARGER COMPUTERS, MOBILE TECHNOLOGIES, ROUTABLE TECHNOLOGIES. I WANT TO BRING INSIGHT FROM THE THOUGHT PROCESS OF UNDERSTANDING THE HUMAN-MACHINE INTERFACE. HIGH LEVEL INTRODUCTION, I DON'T WANT TO SPEAK FOR DEEP LEARNING, THERE ARE PEOPLE HERE MORE EXPERTS, BUT I WANT MY PERSPECTIVE ON WHAT IS A PARADIGM THAT SHIFTED WITH RESPECT TO DATA PROCESSING AND ANALYTICS FROM DEEP LEARNING AND WHAT ITS IMPLICATIONS ARE FOR SENSOR FUSION, AND IN MY CURRENT ROLE, INDEPENDENT CONSULTANT, MAINLY BUILDING SYSTEMS FOR MANY APPLICATIONS BUT MANY ARE HEALTH AND WELLNESS. REALLY WANTED TO HIGHLIGHT ISSUES I HAVE TO REALLY DEAL WITH WHERE THE RUBBER MEETS THE ROAD, WE'RE TAKING SENSOR TECHNOLOGIES, TRYING TO DEVELOP SOLUTIONS, FOR EXAMPLE, IN ELDER CARE OR ADDICTION RECOVERY SOLUTIONS. AND WHAT REALLY ARE THE GAPS AND CHALLENGES IF YOU'RE TRYING TO ADAPT THESE TECHNOLOGIES TO REAL LIFE SOLUTIONS. AND IN THE END I WANT TO PROPOSE A FRAMEWORK, REALLY JUST THROW IT OUT THERE, VERY HIGH LEVEL THOUGHT PROCESS, IS THERE SOMETHING AROUND THE FACT WE HAVE LANGUAGE, HIERARCHY, A LANGUAGE WE'VE ALL DEVELOPED THAT ITSELF CAN BE USED TO BETTER BUILD SENSOR FUSION. WITH MULTI-MODAL DESCRIPTION IN THE PAST I WANT TO SAY MULTI-MODAL SHOULD BE THOUGHT OF AS A MUST-HAVE BECAUSE IT'S WAY WE PROS INFORMATION -- PROCESS INFORMATION. THIS IS MY GO-TO SLIDE, WAYS WE CAN THINK ABOUT HUMAN-MACHINE INTERFACE, SENSORS, ACTUATORS, FROM A GENETIC PERSPECTIVE. WE HAVE THE VISUAL WHICH WE TALKED ABOUT A LOT, WE HEARD ABOUT A LOT TODAY, FOR VERY GOOD REASON. OUR VISUAL SYSTEM IS, YOU KNOW, CONSUMES A LOT OF BANDWIDTH IN THE WAY THE INFORMATION IS PROCESSING. BUT THERE ARE HUMAN VISUAL AND NON-HUMAN VISUAL SENSING, SOME OF THE CHAT GOING ON TODAY MENTIONED. I.R. CAPABILITIES AS GOOD AS VISUAL CAMERA CAPABILITIES. BUT ARE NOT OFTEN CONSIDERED. BUT COULD BE REALLY USEFUL. SAME THING WITH OUR AUDIO PROCESSING, THERE ARE VERY, VERY GOOD MINIATURIZED MICROPHONES, AS MANY OF YOU WHO USE THE TECHNOLOGY KNOW. ALSO THAT TECHNOLOGY IS NOT LIMITED BY THE HUMAN BANDWIDTH. ULTRASONIC SENSORS JUST AS GOOD CAN BE LEVERAGES VERY WELL. WE HAVE A LOT OF TOUCH-BASED TECHNOLOGIES. THEN A LOT LESS SOPHISTICATED SMELL AND TASTE, USUALLY WHEN I PRESENT THIS TO A MEN'S AUDIENCE, THAT'S A GAP, BUT THERE'S A LOT OF REALLY LOT OF TECHNOLOGIES BUILT IN THE LAST I WOULD SAY FIVE YEARS THAT HAVE MADE BIG LEAPS IN TERMS OF ABILITY TO BE USED IN VARIOUS SETTINGS THAT INCLUDES VOCs, ALCOHOL, CARBON DIOXIDE, CARBON MONOXIDE GASES. THIS OF THE SAME SENSES CAN BE USED TO MEASURE NOT JUST HUMAN REACTION BUT ALSO ENVIRONMENT'S COMPLEXITIES, AS THE FIRST PANEL SET UP. IN THIS CONTEXT I WANT TO GIVE A FEW EXAMPLES TO SHOW WHY, THEY ARE ALL EXTRACTED FROM ONE PAPER, QUITE OLD. MORE THAN A DECADE. BY NO MEANS ANYTHING NEW. BUT I WANT TO SAY WE PROCESS AUDIO AND VISION ALMOST EXCLUSIVELY TOGETHER, BUT SURPRISINGLY SOMEWHAT PRIORITIZE -- THERE'S A DISSONANCE BETWEEN INFORMATION COMING VISUAL AND AUDIO CHANNEL, WHEN WE PRIORITIZE WHAT IS AN EVOLUTIONARY CONSTRUCT. IF YOU HAVE BLINKING LIGHTS, 3 HERTZ, AUDIO AT 6 HERTZ, WE CANNOT UNDO THAT NATURAL VERY LOW TIME CONSTRAINT FUSION THAT'S HAPPENING. IT'S THE SAME THING WITH A LOT TO DO HOW VISION AND MOTOR CONTROL WORKS, CLASSIC EXPERIMENTS. KEEP BOTH YOUR HANDS IN FRONT OF YOU, TRY TO MOVE THE LEFT LITTLE FINGER AND RIGHT THUMB. DO THIS AND TRY THE SAME THING. MOVE YOUR LEFT FINGER, IT TAKES LONGER, RIGHT? WE HAVE UNDO THE OFFICIAL CONTRADICTION. THERE'S THIS SENSORY FUSION, MULTIPLE TIME SCALES, ACROSS MULTIPLE MODES OF SENSOR THAT WE NEED TO BE COGNIZANT OF, ESPECIALLY IF YOU'RE TRYING TO THINK ABOUT USING A HUMAN REACTION ACROSS ONLY ONE MODE. AS REPRESENTATION OF SOMETHING THAT'S MORE COMPLEX LIKE BEHAVIOR. HERE I'M SWITCHING TRACKS TO SAY NOW THAT WE'VE MINIATURIZED SENSORS AND PUT SENSORS EVERYWHERE, THERE ARE A LOT OF IDEAS ABOUT MODELS BEING THROWN AROUND, UNDERSTANDING THIS LARGE DATASET, I WANT TO HIGHLIGHT A FEW IMPORTANT CONSIDERATIONS TO KEEP IN MIND. YES, IMAGE, ESPECIALLY VISUAL DEEP LEARNING HAS COME VERY LONG WAY IN A DECADE, BUT THE WHOLE BASIS OF LOT OF DEEP LEARNING IS THESE 14 MILLION IMAGES HAND ANNOTATED, LAND TAXONOMYIZED, HIERARCHICAL REPRESENTATION, NOT DONE WITHOUT RESTRAINING DATA, THERE'S THE HUMAN IN THE LOOP, THE ONLY THING THE HUMAN, YOU KNOW, HUMAN UNDERSTANDING OF THE CONTEXT IS REMOVED BY MAKING IT REALLY, REALLY SIMPLE, RIGHT? IT'S ONE OBJECT, ONE BOUNDING BOX. THE CONTEXT IS LOST IN MANY MODELS WHILE BUILDING THE MODEL. THERE ARE SOPHISTICATED IDEAS THAT ARE ADDED TO BUILD THE CONTEXT BACK IN. LARGE LANGUAGE MODELS THAT, YOU KNOW, ARE REALLY ALSO GATHERED A LOT OF MOMENTUM HAVE ALL THE CONTEXT BUILT IN BECAUSE THEY JUST ARE NOT SUPERVISED MODELS BUT TENT TO -- TEND TO HAVE OTHER ISSUES BECAUSE OF DATA GOING IN. THEY CAN ONLY REFLECT THE DATA OF THE CONTEXT OF DATA FED INTO THE MODEL. THERE'S NOT REALLY EASY WAY TO UNDERSTAND HOW TO VALIDATE THE MODELS INDEPENDENT OF THE DATA BECAUSE THEY ARE JUST FULLY BUILT IN TOGETHER. HAVING SAID THIS, I WANT TO SAY MAYBE THERE ARE SOME VERY CRITICAL LEARNINGS FROM THESE MODELS, AND SOME ARE AROUND, YOU KNOW, THE -- FOR EXAMPLE, LIKE IF YOU THINK ABOUT MEDICAL SENSORS, THERE'S A LOT OF QUESTIONS ABOUT A SIMPLE WEARABLE DEVICE, MEASURING HEART RATE ESPECIALLY AROUND VARIABLES NOT TAKEN INTO ACCOUNT, LIKE GENDER AND, YOU KNOW, SKIN TONE AND BMI. AND WE'VE SHOWN, FOR EXAMPLE, IN MY LAB THAT BY SIMPLY ADDING ONE CONTEXT VARIABLE, ACTIVITY CONTEXT, TO THE HEART RATE SENSORS, YOU KNOW, PREDICTION OF HEART RATE FROM THE RAW SIGNAL CAN PREDICT, THIS IS THE IDEA OF ADDING, YOU KNOW, CONTEXT HOW IT CAN, YOU KNOW, MAKE MODELS BE MORE RESPONSIVE, BUT AT THE SAME TIME ALSO I WANT TO POINT OUT THAT STILL THERE'S SYSTEMATIC VARIATION, VARIABLE ALSO NOT TAKEN INTO ACCOUNT IN DEEP MODELS, YOU NEED TO BE AWARE OF THIS IF YOU'RE BUILDING MODELS THAT WILL BE USED IN REAL APPLICATIONS. ON A LARGE DIVERSE POPULATION. SO I WANT TO END THIS WITH PROPOSAL, LIKE A QUESTION. IS THAT TO NATURALLY BUILD A FRAMEWORK TO MODEL MULTIPLE SENSOR FUSIONS, DOES IT MAKE SENSE TO ACTUALLY TAKE CLUES FROM LARGE LANGUAGE MODELS THAT HAVE BUILT-IN HIERARCHY, TEMPORAL HIERARCHY AND CONTEXT AROUND UNDERSTANDING, FOR EXAMPLE THESE ARE NOT REAL DATA, BUT IDEAS THAT CAN THINK OF GENERAL FRAMEWORK WHERE YOU START WITH LIKE VERY LOW LEVEL SENSOR, YOU KNOW, UNDERSTANDING THAT THEN GOES ACROSS MULTIPLE TIME SCALES SO YOU BUILD FROM LIKE AN ALPHABET TO A WORD TO A SENTENCE. IS THERE IDEAS HERE THAT WE CAN LEARN FROM THE LARGE LANGUAGE MODELS THAT WE CAN THEN BUILD DOMAIN-SPECIFIC MODELS TO UNDERSTAND SENSOR FUSION DOMAIN. WITH THAT QUESTION I WILL OPEN TO -- >> THANK YOU. THANKS VERY MUCH, RAJI. NOW WE'RE SWITCHING TO THE DISCUSSION PORTION. THREE DISCUSSANTS, PETER HARTWELL, BOB DATTA, ANDRE GREEN. YOU EACH HAVE NINE MINUTES IN PRINCIPLE, BUT AS YOU SAW WITH THE PREVIOUS DISCUSSION, PREVIOUS PANEL, IT WAS VERY LIVELY, PEOPLE CHIMING IN. I'M GOING TO GIVE IT TO YOU FIRST, PETER. I'LL LET YOU HAND OFF TO THE NEXT DISCUSSANT. >> I'LL KEEP IT GOING. THANK YOU, FLOH. I'M PETER HARTWELL, I'M CTO, THAT'S MY TITLE. I LOOK AT THIS FROM A VERY DIFFERENT WAY, IN FACT I REALLY ENJOYED TODAY, THE DISCUSSION. LET ME SEE, OUT OF MY ELEMENT, IF I CAN TRY TO GET MY QUESTIONS ANSWERED, MAYBE THAT'S GOING TO HELP ALL THE FOLKS ONLINE. IN PARTICULAR I'M GOING TO LEAN BACK, PANEL 1, TRY TO TIE IT TO PANEL 2 AS I GET THIS GOING. WE'VE SEEN A LOT OF OTHER STUDY OF DISEASE IN HUMANS, MECHANICS IN ANIMALS, I APPROACHED BEHAVIOR OF HUMANS FOR PROFIT, FRANKLY, THAT'S MY JOB. A LOT OF WHAT I DO IN THIS SPACE IS DRIVEN BY MY CUSTOMERS, CONSUMER ELECTRONIC FIELD, WHERE THEY HAVE GOTTEN PRETTY GOOD ABOUT IF YOU THINK ABOUT THE BIG GUYS, ALL THOSE APPS AND DEVICES YOU CARRY AROUND, UNDERSTANDING AND DRIVING YOUR BEHAVIOR TO KEEP YOU ON THAT APP OR TO GET YOU TO GO BUY SOMETHING OR LOOK AT SOMETHING. AND THEY ARE DOING THAT BY MONITORING YOUR SENSE AND YOUR BEHAVIOR. THE WAY YOU TYPE OR YOU SWIPE OR PAUSE ON A SCREEN ON YOUR DEVICE IS ACTUALLY MONITORING YOUR BEHAVIOR. THAT'S WHAT I DO. I PROVIDE THE SENSORS THAT GO INTO THAT, VERY MUCH WHAT RAJI GAVE JUST BEFORE ME, INTRO HOW SENSORS AND ELECTRONICS DRIVE THAT, THINGS YOU HOLD, THINGS YOU WEAR, THINGS YOU CARRY AROUND. AND I THINK MALCOLM, I WANT TO START WITH YOU, BECAUSE YOU BROUGHT UP A VERY INTERESTING POINT AT THE END, WHICH WAS HOW CAN WE, YOU KNOW, LOOK MORE ACROSS THE FIELD. SO I PUT ON THE SLIDE, AND I'LL ASK YOU TO GET RID OF IT, MY LANGUAGE, WE CALL IT SENSORY FUSION, RAJI CALLED IT MULTIPLE MODALITIES, HOW YOU PULL STUFF FROM DIFFERENT AREAS. AND TRY TO GET TO THE ROOT OF WHAT IS SOMEONE DOING, ABOUT TO DO, WANTED TO DO, THAT IS WHERE WE APPLY THE LEARNING INTO THAT. WE CALL IT ACTIVITY CLASSIFICATION. THE CONTEXT DETECTION WHERE IS THE ENVIRONMENT, WHERE YOU'RE AT, HOW IS THAT INFLUENCING YOU. OUR GOAL IS YOU'RE SITTING IN A RESTAURANT, YOUR PHONE DOESN'T BLING, IT VIBRATES, UNDERSTANDING THE CONTEXT OF WHERE YOU ARE, CHANGING THE BEHAVIOR OF THE DEVICE. I WANT TO SHOW THAT SLIDE BECAUSE WHEN I FIRST GOT ASKED TO JOIN THIS I WAS LIKE, I DON'T KNOW ABOUT ANYTHING ABOUT BEHAVIOR, BUT ALL I DO IS TRY TO UNDERSTAND WHAT HUMANS ARE DOING WITH ELECTRONICS. FLOH, GET RID OF THE SLIDE. MALCOLM, LET ME TOUCH ON HOW DO WE GO BROADER? I'M LOOKING AT THE PANELISTS HERE, A LOT OF YOU ARE AT UNIVERSITIES. THERE'S NOBODY HERE FROM MY CUSTOMER SPACE. >> ONE THING I THINK ABOUT FOR YOU, I GUESS YOU DO A LOT OF -- DO DO YOU ETHOLOGY ON HUMANS? OUR DEVICES HIJACK OUR BRAINS, IT WOULD BE NICE TO KNOW MORE ABOUT THAT. ACTUAL ETHOLOGY, COLLECTING CAREFUL BEHAVIOR, AND APPROACHING FROM THAT ANGLE, THAT'S ONE THOUGHT I HAVE GOING DIRECTLY OFF WHAT YOU SAID ABOUT GOING BROADER. BUT THE OTHER THING I THOUGHT OF IS IT'S REALLY INTERESTING THAT YOU'RE THINKING AS WAS WHAT RAJI MENTIONED, THINKING ABOUT FUSION. AND THE BRAIN USES FUSION ACROSS MODALITIES TO DO ESSENTIALLY NOISE REJECTION IN A WAY. >> YES. >> IF THERE'S A PART OF SPACE THAT IS JUST MAKING A SOUND, BUT NOT MOVING, VERSUS A PART OF SPACE THAT IS NOT ONLY MAKING A SOUND BUT ALSO MOVING, THAT CAN BE THE MARK OF THE LIVING, THE MARK OF SOMETHING THAT IS ACTUALLY A DANGER OR AN OPPORTUNITY. WE'VE KNOWN FOR DECADES IN NEUROSCIENCE THAT THERE'S CELLS IN THE COLLICULUS, PARTS OF THE MID-BRAIN EXCITED BY CONJUNCTION OF STIMULI. SEEMS LIKE YOU'RE COMING TO THE SAME CONCLUSION IN A SENSE. >> WE'RE DRIVEN BY A DIFFERENT MOTIVATION. I'LL SPIN BACK TO BEN. VIDEO IS TOO POWER INTENSIVE FOR ALMOST EVERYTHING WE DO. THE SOLE EXCEPTION RIGHT NOW IS IF YOU BUY THE MOST POPULAR VR HEADSETS, IT'S TRACKING THE ROOM WITH FOUR CAMERAS ON IT. AND BUT AGAIN TWO HOURS, IN THE META-VERSE BEFORE YOU UNPLUG AND RECHARGE. BUT, BEN, LOOKING AT -- YOU'RE SO HEAVILY INTO VIDEO, 62 CAMERAS TO TRACK A MONKEY, RIGHT? I HAVEN'T GOT THE POWER TO DO THAT. WE'RE FUSING OTHER THINGS, RAJI DID A NICE INTRO, DO YOU HAVE ANYTHING BEYOND CAMERAS. >> IN THE MONEY LAB WE HAVE A NICE MICROPHONE TO IMPROVE RECONSTRUCTIONS BUT SO FAR WE HAVEN'T HAD ANY SUCCESS WITH THAT. WE REALLY HAVE OUR HANDS FULL WITH THAT. I WANT TO SAY LIKE WE USE THESE VERY HIGH NUMBER OF CAMERAS, BECAUSE WE HAVE TO, TO SOLVE OUR PROBLEMS BECAUSE THE SPECIES MOVES AROUND IN A COMPLICATED WAY, THREE DIMENSIONS, JUMPS AROUND, CONTORTS, TWISTS ITS BODY. WE USE LESS IF WE COULD GET AWAY WITH IT. WITH ONE CAMERA, YOU CAN GET A LOT OF STUFF AND DO A LOT OF GOOD SCIENCE. I'M NOT A CHAUVINIST WHO THINKS -- I THINK IT'S ALL INTERESTING. >> GALIT, YOU HAD WEARABLES ON YOUR PIG, CLOSER TO MY WORLD, RIGHT? WE TRACK MY MAGIC RING AND WATCH, WEARABLES LIKE THAT. DO YOU FIND BEING ABLE TO FUSE OR SYNCHRONIZE BETWEEN CAMERAS AND SORT OF STRAP-ON SENSORS IS A HARD PROBLEM OR SOMETHING YOU GUYS HAVE TRIED TO -- >> SO WE'RE TRYING TO DO THAT. WE HAVE THE WEARABLES. AND WE HAVE THE VIDEOS. AND WE ARE ACTUALLY VIDEOING THE ANIMALS 24/7. SO WE GATHER JUST A HUGE AMOUNT OF DATA THAT IS UNREASONABLE FOR STUDENTS TO SIT AND ANALYZE. SO WE'RE ALWAYS ON THE LOOKOUT FOR A.I., NEW TYPE OF TECHNOLOGIES, HOW WE SYNCHRONIZE VIDEOS WITH WEARABLES. SO FAR WE'RE NOT IN A WAY THAT WE HAVE IT -- WHERE WE WANT IT TO BE. BUT THIS IS SOMETHING THAT WE'RE LOOKING TO DO. ACTUALLY HERE IN THIS PANEL THERE'S SO MUCH GOOD IDEAS. >> YEAH, NO, THOSE ARE THE -- THERE'S TWO OTHER -- WE'RE GOING TO RUN OUT OF TIME. I'LL PASS IT OFF IN A SECOND. I WOULD LOVE TO ASK THAT QUESTION, BEN, DO YOU HAVE TOO MUCH DATA. BUT I'M GOING TO SWITCH. THE ONE I WANT TO ASK, A PERSONAL EXPERIENCE HERE, FIVE YEARS AGO MY KIDS WERE SORT OF 4 AND 6, I PUT A GO PRO ON THEIR HEAD TO SO GETTING WHEN WE HAD SIX FRESH FEET OF SNOW. THEY FORGOT ABOUT IT. THE MOST ENDEARING VIDEO OF MY KIDS, TEN MINUTES OF BROTHER WITH BROTHER AGAINST THE WORLD. AS SOON AS I WALKED UP, AT THE END OF THE VIDEO OLDER PUSHED THE YOUNGER ONE IN THE SNOW. THEY WERE A TEAM WHEN I WASN'T OBSERVING. MAYBE IN THE ANIMAL WORLD IT'S NOT AS -- YOU'RE INFLUENCING THE BEHAVIOR BY OBSERVING BUT IN THE HUMAN WORLD, PARTICULARLY V R THING, AS YOU'RE RUNNING THROUGH THE FAKE RAT MAZE, ARE YOU REALLY AFRAID OF THE ZOMBIE OR NOT? ARE WE INFLUENCING BEHAVIOR BY THE ACT OF TRYING TO MEASURE IT? >> I CAN SPEAK TO THAT LAST BIT. YOU'RE TALKING ABOUT THE DIFFERENCE IN INCENTIVES STRUCTURES BETWEEN RODENT AND HUMAN WORK, WE'RE THINKING HARD ABOUT HOW TO MEASURE THE AVERSIVE STRENGTH OF AN AIR PUFF, WHICH IS UNPLEASANT FOR AN ANIMAL, BUT WE'RE ALSO BASICALLY TRYING TO MODEL PREDATOR-PREY SEQUENCES HERE, IT'S NOT AS AVERSIVE AS POSSIBLE DEATH. IN THE CONTEXT OF VR WITH THE GHOST IT'S KIND OF A STAND-IN FOR WHAT THE REAL PENALTY IS. THEY ARE GETTING PAID AS SUBJECTS, PAY IS ACCORDING TO HOW WELL THEY DODGE THE GHOST. >> PERFECT. >> SO THAT'S OUR WAY OF TRYING TO LINE UP INCENTIVE. WE CAN'T IN THE CONTEXT OF HUMAN WORK HAVE ELECTRIC SHOCK COLLARS. >> THINKING BACK TO THE GAMIFICATION OF BEHAVIOR FOR PROFIT, WHERE I STARTED, I THINK TO ANSWER YOUR QUESTION THERE'S A WHOLE OVERLAP BETWEEN THIS WORLD AND MAYBE IT'S WHAT'S GOING ON IN SPACE RIGHT NOW, WE NEED TO LOOK AT THE MODEL AND HOW THESE TWO WORLDS CAN COME TOGETHER AND REALLY ADVANCE WHAT YOU'RE TRYING TO DO. LET ME PASS TO ROB DATTA AND LET YOU TAKE ROUND TWO. >> THANKS. I'M BOB DATTA, HARVARD MEDICAL SCHOOL. AND MAYBE TO FOLLOW UP ON WHAT WAS BEING SAID, POSING MY FIRST QUESTION, I THINK IT'S INTERESTING, WE'VE HEARD ABOUT SENSOR FUSION IN THIS BLOCK OF TALKS. I THINK IT'S WORTH NOTING THERE'S PROBABLY A DEEP RELATIONSHIP BETWEEN PROBLEMS ENGINEERS ARE SOLVING WITH SENSOR FUSION AND CHALLENGES NEUROBIOLOGISTS FACE, HIGH DENSE DATA STREAMS, AND FOR CAUSAL RELATIONSHIPS, I SUSPECT THERE'S AN ACTIVE DIALOGUE, NEURAL BEHAVIOR RELATIONSHIPS, MANY ON THIS CALL, THOSE IN THE REAL WORLD TRYING TO SOLVE COMPLEX SENSOR FUSION PROBLEMS, A THEME I NOTICED. THE SESSION INCLUDED A LOT OF DIFFERENT PERSPECTIVES FROM THINKING ABOUT NEW MODEEL ORGANISMS TO EVOLUTION AND SENSORS. I WAS THINKING WHAT QUESTION MIGHT UNIFY ALL THESE PERSPECTIVES. I REALIZED SOMETHING IAIAN SAID MIGHT BE RELEVANT, POSING A QUESTION TO MALCOLM. RAJI AND PETER TALKED ABOUT SENSORS, IMPORTANCE OF SENSORS AND SENSOR FUSION, MALCOLM AND BEN TALKED ABOUT BEHAVIOR FROM THE PERSPECTIVE OF MOVEMENT. AS WAS BROUGHT UP IN HIS TALK, A LOT OF MOVEMENT IS ABOUT MOVING SENSORS AROUND, CLOSED LOOP BETWEEN BEHAVIOR AND SENSATION IT'S NOT SO MUCH BUT THAT BEHAVIOR GOVERNS INPUT. IF WE'RE GOING TO UNDERSTAND BEHAVIOR HOW IMPORTANT IS IT TO GAIN ACCESS TO INFORMATION ABOUT WHAT BIOLOGICAL SENSORS ARE SENSING DURING BEHAVIOR? IS PART OF THE GAME HERE NOT JUST TO MEASURE BEHAVIOR BUT ALSO TO UNDERSTAND WHAT THE EYES ARE SEEING, WHAT THE EARS ARE HEARING, AND WHAT THE NOSE IS SMELLING, IF MY PERSPECTIVE? >> IT'S A TOUGH JOB YOU'RE DOING, SYNTHESIZES ACROSS TALKS. A GREAT QUESTION. I WOULD SAY, YOU KNOW, THE PERSPECTIVE OF SENSORS, BEHAVIOR BEING THE BODY'S WAY OF GETTING SENSORS IN THE RIGHT POSITION, INDEED I FEEL LIKE ONE OF THE BIG ADVANTAGES OF THE SYNTHETIC APPROACH SIMULATING THE ORGANISM AND ENVIRONMENT, THE OPPORTUNITY ONCE YOU HAVE GOOD HIGH RES, 3D DATA WHERE AN ANIMAL IS IN SPACE, TO RECONSTRUCT WHAT SENSORS ARE GETTING AT THE RECEPTOR LEVEL IN A CAREFUL WAY AND USE THAT. WHAT'S NICE ABOUT THAT, YOU KNOW, YOU'RE NOT GOING TO GET IT EXACTLY RIGHT BECAUSE THERE'S A LOT OF NUANCE THERE, BUT WHAT'S BEEN SUPER HELPFUL IN TERMS OF THE EARLIER PART OF THE WORK THAT I TALKED ABOUT IN MY BIT IS THAT ONCE YOU'VE DONE THAT LEG WORK ON WHAT'S HAPPENING IN THIS CRAZY COMPLICATED CONDITION THAT ISN'T FEASIBLE FOR BUSTING NEURAL CIRCUITS IS THAT YOU CAN TAKE WHAT YOU GAIN FROM THAT SYNTHETIC APPROACH AND THEN PUT IT INTO THE ANIMAL IN A CONTEXT WHERE YOU CAN DO THE MORE REDUCTIVE FORMS OF NEUROSCIENCE THAT SOMETIMES NEED TO BE DONE. IT'S NOT ALWAYS CONVENIENT TO DO FREELY BEHAVING RECORDINGS IN ANIMALS. SOMETIMES YOU NEED TO PHOTON HEAD FIX, FOR EXAMPLE. BUT HAVING THE RECIPE FOR THE SIGNAL FROM THE SYNTHETIC APPROACH I THINK IS ALL IMPORTANT IN TERMS OF GETTING THE CIRCUITS TO BEHAVE IN THE WAY THAT'S MAXIMALLY INFORMATIVE FOR SUBSEQUENT INTERPRETATION. SO, I THINK IT'S A REALLY GOOD POINT AND, YEAH, THANKS FOR THE QUESTION. >> BEHAVIOR IS NOT JUST ABOUT GETTING INFORMATION, THE GOAL IS REWARDS AND SURVIVAL, RIGHT? >> YEP. >> I JUST WANT TO ADD THAT, YOU KNOW, WE CAN TALK ABOUT SENSORS STREAMS, BUT IN ANIMALS THOSE ARE REALLY DEPENDENT ON THE CONTEXT THEMSELVES. THERE'S AUTONOMIC RECEPTORS ON MUSCLE SPINDLING, MOTOR COMMANDS TO YOUR EYES AN EARS. IT'S HARD TO THINK ABOUT SENSORS BEING DECOUPLED FROM OTHER STAKES WE'VE TALKED ABOUT WE NEED TO RECOGNIZE. >> THAT'S A GOOD POINT. >> THINKING ABOUT GETTING ACCESSION TO SENSORS IS KIND OF PREMISED ON GETTING ACCESS TO INTERNAL SPEED. >> YEAH, CONVENIENCE OF ELECTRO SENSOR SYSTEM BUT DON'T HAVE IT IN MANY OTHER SYSTEMS, LACK OF EFFERENT CONTROL. BUT SUBSEQUENT TO THE SECOND, THIRD ORDER OF NEURONS INFORMATION COMES INTO PLAY, ABSOLUTELY TRUE. >> BEN, THE AREAS OF THE BRAIN YOU ARE RECORDING CONTAIN SENSORY INFORMATION, MOTOR INFORMATION, AND REWARD INFORMATION. HOW ARE YOU THINKING ABOUT RELATING ALL THOSE TYPES OF INFORMATION TO THE ONGOING BEHAVIOR OF THE MONKEYS YOU'RE RECORDING? >> CAN YOU SAY THAT AGAIN? >> YEAH, I WAS ASKING, YOU'RE RECORDING THESE FRONTAL AREAS IN THE MONKEYS WHICH ENCODE ADMIXTURES OF SENSORY, MOTOR, REWARD INFORMATION, AND HOW ARE YOU THINKING ABOUT UNDERSTANDING THOSE SIGNALS IN THE CONTEXT OF COMPLICATED NATURALISTIC BEHAVIORS WHERE MONKEYS ARE RUNNING AROUND. >> YEAH, I MEAN, I THINK KIND OF THE BIG PICTURE, THE HIGHEST LEVEL, THE ANSWER IS THE OLD-FASHIONED WAY OF LOOKING AT IT THAT I'VE BEEN DOING FOR 20 YEARS, LIKE WHAT DO NEURONS REPRESENT, THAT WHOLE APPROACH REALLY ISN'T WORKING VERY WELL, IT'S NOT REALLY DRIVING US TOWARDS ANSWERS, BUT I DON'T THINK IT'S A GOOD DESCRIPTION OF WHAT NEURONS ARE DOING. WE'RE DISMANTLING IT AND STARTING OVER SAYING WHAT CAN WE SAY ABOUT THESE NEURONS. WE COULD START WITH, SAY, WELL, EVEN NEURONS THROUGHOUT THE PREFRONTAL CORTEX HAVE A ROLE IN CONTROLLING BEHAVIOR, SOMETHING WE CAN SAY WITH CONFIDENCE. WE'RE GOING AS FAR AS WE CAN WITH THAT SAYING WHAT ELSE DO THEY DO BESIDES THAT, WHAT ELSE DO THEY DO BESIDES INDIRECTLY CONTROLLING BEHAVIOR, SETTING THE STAGE FOR BEHAVIOR, AND THAT'S GOING TO WIND UP EXPLAINING VARIANCE IN THE NEURONS, JUST A PREDICTION THOUGH. >> ONE LAST QUESTION, TOUCHED ON IN THE FIRST PANEL, A VERY BROAD AND GENERAL CHALLENGE, GIVEN DENSE MEASUREMENTS OF BEHAVIOR HOW DO WE ASSIGN MEANING TO MOVEMENT? MANY SPEAKERS IN THIS PANEL ARE MAKING HIGH DIMENSIONAL RECORDINGS. >> I'M SO SORRY, THIS IS DANA. I DON'T WANT TO INTERRUPT BUT I WANT TO MAKE SURE ANDRE HAS TIME TO GET HIS THOUGHTS AS A DISCUSSANT. >> PLEASE GO AHEAD. >> HE WILL GET HIS TIME, DANA. WE STARTED LATE SO WE'LL RUN A LITTLE BIT LATE. I DON'T KNOW, BOB, YOU'VE GOT IT. >> I'LL QUICKLY FINISH AND MOVE ON. LOTS OF US ARE PILING UP HIGH DIMENSIONAL DESCRIPTIONS OF BEHAVIOR, AND IT'S NOT ALWAYS CLEAR HOW TO UNDERSTAND WHAT THE DATA MEAN. IN THE SENSE THAT, YOU KNOW, MOVEMENT AND BEHAVIOR AREN'T SYNONYMOUS, RIGHT? AS WAS MENTIONED IN HER TALK, FOR THOSE ESPECIALLY GALIT LOOKING AT ANIMALS, OCTOPUS, PIGS, THERE'S A HUGE AMOUNT OF BEHAVIORAL DATA THAT WE CAN CENTER THINKING AROUND, HOW ARE YOU THINKING ABOUT ASSIGNING MEANING TO THE VARIOUS HIGH DIMENSIONAL MOVEMENTS YOU'RE OBSERVING? >> SO, I DIDN'T SHOW -- FOR THE OCTOPUS, FOR EXAMPLE, WE ARE INTERESTED IN A VERY SPECIFIC MOVEMENT WHICH IS THE GRASPING MOVEMENT. WE ACTUALLY TRAIN THE OCTOPUS TO GRASP, SO WE CAN -- IT IS INTENTIONALLY EXECUTING A VERY SPECIFIC MOTION. SO WE DON'T HAVE EVERYTHING THAT HAPPENS IN THE TANK, OF COURSE IN THE FUTURE WE WOULD LIKE TO HAVE -- TO SAMPLE THAT SIMULTANEOUSLY BUT WE DO FEEL WE HAVE THE INTENTION OF THE MOVEMENT. >> ANDRE, WOULD YOU MIND TURNING ON YOUR CAMERA AND JUMPING INTO THE CONVERSATION THIS? WE'LL GIVE YOU THE TIME AND PLATFORM TO SPEAK, WE'LL CANCEL THE BREAK. >> THANK YOU, DANA. ANDRE? >> I APPRECIATE IT. THANK YOU. SO, I MIGHT MAYBE BE RELATIVELY QUICK. ONE THING I HEARD ACROSS ALL TALKS, INCLUDING THE ONES IN THIS PANEL, THIS KIND OF TENSION BETWEEN WORKING IN CONTROLLED ENVIRONMENTS WHERE WE CAN HAVE A MUCH TIGHTER UNDERSTANDING OF A PARTICULAR PROCESS, VERSUS THE NATURAL ENVIRONMENT, HOW ARE THINGS BEHAVING IN THE NATURAL ENVIRONMENT. AND WHAT I'VE BEEN THINKING A LOT ABOUT IN TERMS OF WORKING WITH THE MONARCHS, BECAUSE I'M INTERESTED IN MONARCH MIGRATION, HOW ARE THEY USING MULTI-SENSORY CUES NAVIGATE, GOING TO SUCH PLACES WITH INCREDIBLE REGULARITY, WHEN I START MY OWN EXPERIMENTS WE USE THE USUAL ENVIRONMENTAL CUES, USUAL SUSPECTS, AND I'M NOT ABLE TO RECAPITULATE SOME OF THE PHENOTYPES AND BEHAVIORS I'M INTERESTED IN. SO IT MADE ME TAKE A STEP BACK AND ASK, WELL, HOW WOULD I IN AN UNBIASED WAY DETERMINE WHAT IS ENVIRONMENTALLY RELEVANT TO THE PHENOTYPE THAT I'M LOOKING AT. AND I STILL DON'T HAVE A GREAT ANSWER FOR THAT. BUT I'M THINKING ABOUT PETER AND RAJI, WHO ARE THINKING ABOUT MULTI-MODAL FUSION AND WE'VE ALREADY MADE THIS REALIZATION THAT OF COURSE IN THE NATURAL ENVIRONMENT THAT WE WANT TO UNDERSTAND BEHAVIOR IN THIS IS WHAT MORE RECAPITULATES WHAT'S ACTUALLY HAPPENING. SO FOR THE FOLKS IN THE PANEL, HOW DO YOU DETERMINE FOR YOUR OWN WORK WHAT IN THE ENVIRONMENT IS IMPORTANT, WHAT DO YOU TELL KIND OF THE PEOPLE THAT YOU ASK TO WORK WITH KIND OF WITH THE SENSORS WHAT SHOULD THEY BE MEASURING AND HOW DO YOU GUIDE THEM WITH THAT? >> I'LL TAKE A STAB AT ANSWERING THAT. I WOULD LOVE TOE BE -- TO BE AT THE STAGE YOU'RE THINKING ABOUT, YOUR SYSTEM, RIGHT AT THE PRESENT TIME ALL THE EXPERIMENTS WE DO WITH ANIMALS ARE FAIRLY CONSTRAINED. MINUS THE NEW WORK WHERE WE HAVE COMPLEX HABITATS WITH RODENTS RUNNING AROUND WITH ROBOTS CHASING THEM, THAT'S UNCONSTRAINED. BUT THE EARLIER SENSORY WORK WE DID A LOT OF THE ANALYSIS WE'VE DONE IS SITUATIONS WHERE THERE'S ONE OBJECT TO TRACK, SOMETHING LIKE THAT, WHERE WE SIMPLIFY SO THERE ISN'T A LOT OF AMBIGUITY AS TO WHAT IS GOING ON. ONE THING THAT'S EMERGED FROM ANALYSIS ACROSS A BUNCH OF SPECIES, WORKED WE PUBLISHED LAST YEAR, ANALYZED DATA FROM ELECTRIC FISH, COCKROACHES, AND A TYPE OF RODENT, OR A TYPE OF MOLE, THERE'S INFORMATION WAY TO GET AT WHAT YOU'RE SAYING, FROM AN INFORMATION THEORY PERSPECTIVE WHEN AN ANIMAL IS ENCOUNTERING SOMETHING THAT HAS A RICH AMOUNT OF INFORMATION IN THE SPACE, THERE'S CHARACTERISTICS BEHAVIORAL CHANCE WE DOCUMENT IN THE PAPER INCLUDING REDUCING VELOCITY IS SOMETHING THAT'S CHARACTERISTIC OF ENTERING A HIGH DENSITY SPACE OF INFORMATION FOR AN ANIMAL. ANOTHER THING IS THERE'S A FREQUENCY CHANGE IN THE WAY THEY MOVE SENSORS, THEY START INHABITING A DIFFERENT REGION OF THE SPACE WHEN YOU DO ANALYSIS AND FREQUENCY SPACE. ONCE SENSORS ARE ACTIVATED, AND WHEN THEY ARE NOT IN A HIGH INFORMATION ZONE IT CHANGES TO REQUIRE LESS INFORMATION. THERE MIGHT BE INFORMATION THEORETIC TOOLS THAT CAN START TO HELP WITH THE PROBLEM THAT YOU'RE TACKLING, WHICH IS NOT A PROBLEM I'M YET TACKLING BUT I THINK I WILL BE NEEDING TO SOON. >> WE DO IT CLINICALLY, THERE'S A BEHAVIOR, LIKE AS I WAS TALKING ABOUT FREEZING OF GAIT, WE NEVER CAPTURE IT IN THE LAB. IT'S ALWAYS LIKE AFTER THE TRIAL IS OVER, THEN THE PATIENT TURNS AND THEN IT HAPPENS. WHEN WE ACTUALLY CATCH IT, OH MY GOD, KEEP RECORDING. AND PEOPLE GO BACK INTO THIS LIKE REAL -- MORE REAL TYPE OF BEHAVIOR, REAR STRUGGLING WITH THINGS WE NEED TO CONTROL IN THE LAB TO ACTUALLY GET THE BEHAVIORS THAT MATTER AND I THINK DISTRACTION AND HIGH COMPLEXITY OF ENVIRONMENTS IS REALLY IMPORTANT. AND OF COURSE AGAIN I THINK THIS CAME UP, ANYTIME SOMEBODY WALKS INTO OUR LAB IT'S JUST THEY WANT TO MAKE SURE THEY ARE DOING OKAY. UNLESS THEY ARE A COLLEGE STUDENT, THEY JUST WANT THEIR MONEY. THE CONTEXT, WHEN A PATIENT COMES IN VERSUS A COLLEGE STUDENT, IT'S REALLY DIFFERENT TOO, THEIR MOTIVATION FOR EVEN BEING THERE. >> I'LL TAKE ONE QUICK STAB. WE THINK ABOUT TRYING TO WAKE UP YOUR PHONE IF IT'S ON THE DESK, KNOWING YOU'RE NOT HOLDING IT GOES INTO A DEEP SLEEP STATE. IF I KNOW YOU'RE HOLDING IT THAT'S ANOTHER ONE. IF I KNOW YOU'RE LOOKING AT IT, I LISTEN FOR AUDIO CUES TO SEE IF THERE'S ANYBODY IN THE ROOM. WE BUILD THE HIERARCHY INTO THE ELECTRICAL SYSTEM. YOU SORT OF TRIGGER ME, MALCOLM, FOR US THE CURRENCY IS POWER, IMPROVING THE USER EXPERIENCE MAKING YOUR BATTERY LIFE BETTER, MAKING YOUR DEVICE SLEEP MORE. MAYBE I'M SYNTHESIZE -- [ NO AUDIO ] OR EVOLVING, TO QUOTE SOMEBODY EARLIER TODAY. >> I THINK THERE'S ALSO A LITTLE BIT OF CONTEXT AROUND THE MISMATCH, BEHAVIOR CHANGE. THAT'S THE COST, AUXILIARY COST LIKE POWER FOR WHICH YOU CAN OPTIMIZE IF YOU'RE DOING SYSTEM LEVEL THINKING, ONE-TIME DESIGN. AS A CONTINUOUS EVOLUTIONARY USE, TEN YEARS AGO, FIFTEEN YEARS AGO HOW MANY WOULD HAVE DONE THIS, WAITING AT INTERSECTION, YOU SNAP A PHOTO OF THE SIGN BOARDS TO SEND YOUR FRIEND. THAT PICTURE IS NOT WORTH A THOUSAND WORDS, IT'S WORTH WHO WORDS, RIGHT? SOME OF THESE BEHAVIORS, WHICH SENSOR YOU CHOOSE WHEN, MIGHT BE NOT PURELY ABOUT SYSTEM DESIGN BUT ALSO CONTEXT AND THAT PARTICULAR -- MORE LOCAL, RIGHT? THERE ARE MULTIPLE TIME SCALES RATHER THAN PURELY ABOUT -- THAT'S WHAT'S INTERESTING, SOMEONE WHO HAD TO CHOOSE BETWEEN BIOLOGY AND COMPUTER SCIENCE, IN 11th GRADE, MY MOST DISASTERROUS LIVE, GIVEN THE CHANCE I MIGHT HAVE CHOSEN THE OCTOPUS. >> ANDRE, DO YOU HAVE OTHER QUESTIONS FOR THE GROUP? >> I THINK MANY OF MY QUESTIONS WERE DEFINITELY COVERED AND OVER BOTH OF THE SESSIONS. AGAIN, I APPRECIATED HEARING THESE, ALSO I'LL MENTION HERE THIS THOUGHT OF CONSTRUCTING VIRTUAL ENVIRONMENTS, ANALYZING BEHAVIOR PAIRED WITH WHAT IS IMPORTANT IN THE ENVIRONMENT TRIGGERED ME PAIRING THOSE TWO AND NOW I GUESS I'M GOING TO BE THINKING ABOUT THAT FOR THE REST OF THE DAY. I REALLY APPRECIATE ALL OF THE COMMENTS HERE. >> THANK YOU. TERRIFIC DISCUSSION. THEMES I NOTICED, DIFFERENT TEMPORAL AND SPATIAL SCALES, BOTH, WITH RESPECT TO THE ORGANISM, WHAT COULD BE MEASURED IN AND AROUND, IF YOU WISH, ON THE ORGANISM, ALSO WITH RESPECT TO THE CONTEXT, CONTEXT IN MANY WAYS OF INTERPRETING CONTEXT, AGAIN HAS DIFFERENT SPATIAL AND TEMPORAL SCALES, BUT THE PROBLEM, NOT ONLY WHAT DATA IS COLLECTED BUT HOW TO FUSE THE DATA STREAMS AND SYNCHRONIZATION ACROSS STREAMS. AND INTERESTING COMMENT ABOUT EFFECT OF OBSERVER AS WE CONSTRUCT ENVIRONMENTS, AS WE CONSTRUCT MEANS OF MEASURING ARE WE ALTERING WHAT WE WANT TO CAPTURE. AND WITH THAT, WE'RE GOING TO SKIP THE BREAK AND MOCH MOVE TO THE THIRD PANEL. >> LET'S GO TO PANEL 3. FOUR SPEAKERS, FOUR PRESENTATIONS, DR. DANI BASSETT FROM UNIVERSITY OF PENNSYLVANIA, DR. SCOTT LINDERMAN, STANFORD, DR. MARYAM SHANECHI, AND DR. GORDON BERMAN, EMORY UNIVERSITY. WE HAVE THREE DISCUSSANTS TO LEAD PANEL DISCUSSION AFTER THE PRESENTATIONS. DR. ALLISON WATERS FROM ICAHN SCHOOL OF MEDICINE, DOLU OBATUSIN, AND DR. TIM BROWN FROM UNIVERSITY OF WASHINGTON. EACH PRESENTATION WILL BE SHORT, ABOUT 8 MINUTES OF TIME FOR THE PRESENTERS, PLEASE BE MINDFUL OF YOUR TIME SO WE HAVE ENOUGH TIME AT THE END FOR DISCUSSION AND QUESTION. SO WE'RE GOING TO TAKE QUESTIONS DURING THE PANEL DISCUSSION PHASE AFTER ALL FOUR PRESENTATIONS ARE COMPLETE. LET'S MOVE TO THE FIRST PRESENTATION BY DR. BASSETT. PLEASE TAKE IT AWAY. >> THANK YOU SO MUCH FOR THE OPPORTUNITY TO BE PART OF THIS WORKSHOP. I'M REALLY EXCITED ABOUT THE TOPICS THAT HAVE BEEN RAISED THUS FAR. I'M EXCITED TO PROVIDE YOU WITH A LITTLE BIT OF OVERVIEW FOR THIS PARTICULAR PANEL. WHAT THE ORGANIZERS ASKED ME TO DO IN THIS PANEL IS TO TALK QUITE BROADLY ABOUT HOW WE CAN THINK ABOUT BEHAVIOR, AS A COMPLEX SYSTEM, IN THE NEXT THREE TALKS MY COLLEAGUES WILL BE DIGGING DEEPLY INTO SPECIFIC INSTANCES OF THIS GENERAL IDEA. SO YOU CAN SEE MINE IS A BROAD OVERVIEW, AND WE'LL MOVE INTO OUR SPECIFICS. SO FIRST I WOULD SAY THAT I'M GOING TO BE PROVIDING THIS OVERVIEW FROM THE PERSPECTIVE OF COGNITIVE NEUROSCIENCE, ONE OF THE AREAS IN WHICH I WORK, AND IN THAT FIELD THE TYPICAL VIEW OF BEHAVIOR IS THE BRAIN CAUSES BEHAVIOR WHICH PRODUCES A CHANGE IN OUR ENVIRONMENTS. AND WHAT'S INTERESTING IS THAT IT FOREGROUNDS A SIMPLE ONE-DIRECTIONAL CAUSAL STRUCTURE. THE BRAIN DRIVES BEHAVIOR IN A ONE ONE-DIMENSIONAL LINE, DRIVING THE ENVIRONMENT. THE PREFRONTAL CORTEX DRIVES A REACTION TIME, AND THAT MAKES AN IMPACT ON THE ENVIRONMENT, WITH A KEYBOARD PRESS. THIS IS A KIND OF TYPICAL SET OF UNITS OF INFORMATION, AND THEIR RELATIONSHIPS WE WOULD VIEW AS A GOOD STUDY OF BEHAVIOR IN COGNITIVE NEUROSCIENCE. I THINK THE CHALLENGE THAT MANY OF US ARE FACING IS THAT OF COURSE BEHAVIOR IS NOT SEDENTARY, NOR RELATED TO BRAIN AND ENVIRONMENT IN THIS SIMPLE UNIDIRECTIONAL CAUSAL STRUCTURE. SO FIRST I WANT YOU TO FOCUS ON THE LEFT-HAND SIDE OF THE SLIDE, THE POINT THAT BEHAVIOR IS NOT UNITARY. INSTEAD OF ONE LAW THAT IS BEHAVIOR, OFTEN REACTION TIME IN COGNITIVE NEUROSCIENCE, WE HAVE MANY BEHAVIORAL BITS, FOR EXAMPLE, MICROMOVEMENT OR HIGH TWITCH, NOT VERBAL SOUND, TIGHTENING OR RELAXATION OF MUSCLES, THOSE BITS DISPLAY STATISTICAL AND CAUSAL DEPENDENCIES UPON ONE ANOTHER. SECONDLY, SECOND CHALLENGE, BEHAVIOR IS NOT RELATED TO BRAIN AND ENVIRONMENT IN THIS SIMPLE CAUSAL CHAIN SO IN FACT WHAT'S HAPPENING IS MUCH MORE COMPLICATED, VARIOUS PIECES OF THIS BROADER SYSTEM INTERACT WITH ONE ANOTHER IN BOTH PAID FORWARD AND PAID BACK DIRECTIONS. FOR EXAMPLE, LONG-TERM BEHAVIORS CAN CHANGE THE BRAIN BY ALTERING INPUT, THE ENVIRONMENT CAN ALTER THE BRAIN THROUGH PERCEPTION WITHOUT ACTION, AND THE ENVIRONMENT CAN DRIVE BEHAVIOR THROUGH STRUCTURING CONSTRAINTS. THE PICTURE IS MORE COMPLICATED THAN THE SIMPLE LINEAR 3-POINT BOX DIAGRAM. I'LL PUT THE CHALLENGES TOGETHER, IT'S NOT UNITARY, BEHAVIOR IS NOT UNITARY, NOT A SIMPLE CAUSAL CHAIN. PUT THE PIECES TOGETHER WE HAVE SOMETHING MORE COMPLICATED, WE KNOW THE BRAIN IS COMPOSED OF UNITS, INDIVIDUAL BRAIN REGIONS INTERACTING IN A COMPLICATED WAY, INTERACTIONS ARE DRIVERS FROM EACH OF THE PIECES OF THE BRAIN TO A PARTICULAR BEHAVIORAL BIT, BEHAVIORAL BITS ARE DEPENDENT ON ONE ANOTHER. AND THEN THE THIRD INSTANCE IS THE ENVIRONMENT, AGAIN THE ENVIRONMENT IS NOT ONE THING, IT'S MANY THINGS INTERACTING IN A COMPLICATED WAY. IN THE BROADER PICTURE MULTIPLE UNITS EXIST AND INTERACT IN EACH OF THE SUBSYSTEMS. BUT MORE THAN THAT SINGLE UNITS OR SMALL GROUPS OF UNITS CAN INTERACT BETWEEN SUBSYSTEMS. NOW, WE COULD ASK, YOU KNOW, ARE WE DONE WHEN WE HAVE BRAIN BEHAVIOR AND ENVIRONMENT IN THIS BROADER STRUCTURE, AND I THINK THAT MANY OF US IN THE FIELD WOULD SAY, NO, OF COURSE WE'RE NOT DONE, THE BRAIN IS NOT NECESSARILY THE FIRST CAUSE ALWAYS, MAYBE WE NEED TO PRESS FURTHER BACK AND UNDERSTAND PROCESSES IN THE MIND LIKE BELIEFS, DESIRES, INTENTIONS, PURPOSES THAT MIGHT NOT BE IMMEDIATELY -- MIGHT NOT PRODUCE ACTION, EXCEPT BY GOING FIRST THROUGH THE BRAIN. UNDERSTANDING BEHAVIOR I THINK REQUIRES US TO RECKON WITH THE PSYCHOLOGICAL PROCESSES INVOLVED, UNDERSTAND THEY ARE MAPPED TO THE BRAIN NETWORK, AND THEN THE MAP TO THE BEHAVIORAL NETWORK AND ENVIRONMENT BY EXTENSION. SO I THINK HAVING THESE FOUR PIECES IN MIND IS IMPORTANT WHEN WE THINK ABOUT A SINGLE ORGANISM. OF COURSE, THE PIECE OF THE PICTURE THAT I HAVE NOT ADDED IN HERE IS WHEN WE HAVE MULTIPLE ORGANISMS, INTERACTING WITH ONE ANOTHER IN SOCIAL SYSTEMS OR OTHER COLLECTIVES. I WANT TO DIG DEEPER AND SAY ARE WE DONE? THE ANSWER IS NOT QUITE BECAUSE EACH OF THESE INTERACTIONS, BOTH WITHIN AND ACROSS SYSTEMS, CAN CHANGE IN TIME. SO WITHIN THE MIND, INDIVIDUAL MENTAL PROCESSES HAVE TIME VARYING ARCHITECTURES, THE BRAIN REGIONS HAVE TIME VARYING INTERACTIONS, RELATIONSHIPS BETWEEN EACH OF THESE SUBSYSTEMS HAVE TIME-VARYING INTERACTION, SO EVERY LINK IN THE PICTURE I PLACED ON T ON BECAUSE THEY ARE TIME VARYING, NOT STATIC, AND THE CHANGE IN ONE PIECE CAN ALTER THE CHANGE THROUGHOUT THE ENTIRE SYSTEM. AND MORE THAN THAT, ONE SORT OF LAYER EVEN FARTHER IS THAT UNITS, EACH OF THESE UNITS CAN HAVE ASSOCIATED TIME SCALES, TIME SCALES OF ACTIVITY IN THE BRAIN, TIME SCALES OF ACTION AND BEHAVIOR, TIME SCALES OF STRENGTHENING OR WEAKENING OF MENTAL PROCESSES IN THE MIND. SO I PLACED A TAU IN EVERY UNIT, IN THE SUBSYSTEM, FOR EACH SUBSYSTEM ITSELF. WHAT IS THIS? THIS PICTURE THAT I'M SHOWING YOU NOW? THIS PICTURE IS A PICTURE OF A COMPLEX SYSTEM. IT IS A SYSTEM THAT IS COMPOSED OF MANY INDIVIDUAL UNITS, THAT INTERACT WITH ONE ANOTHER IN COMPLICATED TIME-VARYING WAYS TO PRODUCE VERY SOPHISTICATED PHENOMENON THAT OCCUR ACROSS ALL OF THE SUBSYSTEMS. SO, UNDERSTANDING THIS COMPLEX SYSTEM REQUIRES US TO BRING COMPLEX SYSTEMS TOOLS TO THE TABLE, TOOLS MEANING ALGORITHMS, CONCEPTUAL FRAMEWORKS, THEORIES, COMPUTATIONAL MODELS. OF COURSE, YOU KNOW, YOU COULD BE SITTING IN THE AUDIENCE THINKING, HANG ON, EVERYTHING IN SCIENCE IS REALLY COMPLICATED, WE COULD SAY THIS ABOUT ANYTHING WE STUDY, BUT ISN'T THE POINT TO TRY TO SIMPLIFY, TO MAKE ASSUMPTION AND FOCUS IN ON THE VERY INDIVIDUAL PARTS AND HOW THEY MIGHT INTERACT WITH ONE ANOTHER? SO, COULD I TAKE THAT BROAD COMPLEX SYSTEM I SHOWED YOU AND MAP IT DOWN INTO FOUR UNITS, AND THIS SIMPLE UNIDIRECTIONAL CAUSAL STRUCTURE? THE ANSWER IS NOT QUITE. NOT IF THE TIME SCALES OF THE PARTS MATTER FOR HOW THE SYSTEM FUNCTIONS. THOSE ARE THE TAUS IN THE PREVIOUS SLIDE. NOT IF TEMPORAL VARIABILITY MATTERS HOW THE SYSTEM FUNCTIONS, THAT WAS OF THE INDIVIDUAL T ON THE PREVIOUS SLIDE. ALSO NOT IF THE PATTERN OF INTERACTIONS MATTERS FOR HOW A SYSTEM FUNCTIONS. IF TIME SCALE, TEMPORAL VARIABILITY AND PATTERNS MATTER WE CAN'T GET AWAY WITH THE SIMPLIFIED PICTURE. YOU COULD ALSO SAY, YOU KNOW, HANG ON AGAIN. IF WE DON'T HAVE THIS SIMPLE CHAIN I'M SHOWING ON THE LEFT-HAND SIDE HOW DO WE KNOW WHEN WE BUILT A RIGOROUS EXPLANATION, WHAT SORTS OF EXPLANATIONS ARE EVEN POSSIBLE? WHICH ARE CAUSAL? AND WHICH ARE NOT CAUSAL? ON THE LEFT-HAND SIDE WHAT I'M SHOWING IS THE RELEVANT CAUSAL CONCEPTS HERE CAN BE RELATIVELY STRAIGHTFORWARD, A SIMPLE CAUSAL DEPENDENCY BETWEEN TWO UNITS. >> YOU HAVE ONE MINUTE. SORRY. >> THANK YOU. WHEN WE HAVE COME PLEX COMPLEX SYSTEM ON THE RIGHT THERE CAN BE RELEVANT CAUSAL CONCEPTS, FIRST IS CAUSAL DEPENDENCY, SECOND IS STRUCTURING CAUSE, CAUSING DYNAMICAL PROCESSES, TRIGGERING PATHWAYS, MECHANISMS, CASCADES. THERE'S BEAUTIFUL WORK IN THE AREA OF VELOCITY, CONCEPTS THAT MAY BE HELPFUL IN UNDERSTANDING COMPLEX SYSTEMS. THIS IS MY LAST SLIDE, WHAT I THINK OF AS PATHS TO PROGRESS. FIRST IS TO PROBE INTERACTIONS WITHIN AND BETWEEN THESE SUBSYSTEMS. TO MAP TIME SCALES OF PARTS, TO EXAMINE TEMPORAL VARIABILITY OF INTERACTIONS. AND LASTLY TO EXPAND OUR UNDERSTANDING OF DIVERSE CAUSAL CONCEPTS AND USE THEM TO DEVELOP MORE PRECISE HYPOTHESES CLOSER TO THE NATURE OF THIS COMPLEX SYSTEM. WITH THAT I'LL END AND CEDE THE FLOOR TO MY COLLEAGUES. >> NOW WE MOVE TO THE NEXT PRESENTATION, DR. LINDERMAN. TAKE IT AWAY. >> ALL RIGHT. GREAT. I'M STILL WAITING FOR THE CONTROLS. THERE WE GO. GREAT TO BE HERE. I'M ASSISTANT PROFESSOR IN STATISTICS DEPARTMENT, A COMPUTATIONAL NEUROSCIENCE, TO TRY TO UNDERSTAND AND GLEAN INSIGHT INTO DATASETS SO TOPICS OF TODAY'S DISCUSSION ARE NEAR AND DEAR TO MY HEART. AS I WAS PREPARING THE TALK FOR TODAY, I WAS LOOKING AT THE TITLE OF THE PANEL REALIZING EQUALLY APPROPRIATE TITLE WOULD BE TO CALL THIS MACHINE LEARNING AND STATISTICS BECAUSE REALLY GOING FROM MODELS TO -- DATA TO MODELS AND BACK IS WHAT MACHINE LEARNING AND STATISTICS AIM TO DO, BUILD MODELS OF LARGE SCALE DATASETS TO MAKE PREDICTION, MAKE DECISIONS. HOW IDEAS IN THESE FIELDS CAN CONTRIBUTE, AS YOU'VE SEEN THE BUZZ AROUND MACHINE LEARNING HAS PERMEATED THE POPULAR PRESS. SOME EXCITEMENT IS HYPE, A LOT FOR GOOD REASON. OPEN A.I., GPT 3 LANGUAGE MODELS WRITE ESSAYS, NET FLEX'S ALPHAGO, A REAL IMPACT ON SCIENCE. A TEAM AT STANFORD USED DEEP NEURAL NETWORKS TO MAKE PREDICTION ABOUT RNA STRUCTURE, AND USED THOSE IN ALPHAGO TO CONTROL PLASMA, AN IMPORTANT STEP TO SUSTAINABLE ENERGY BASED ON NUCLEAR FUSION. WHAT ARE KEY IDEAS, KEY ADVANCES BEHIND THESE? ONE IS AVAILABILITY OF LARGE SCALE DATASETS. GPT 3 IS TRAINED ON A CORPUS OF TEXT. SECOND INGREDIENT IS ADVANCES IN ALGORITHMS. MANY VERY LARGE SCALE DATASETS ARE UNLABELED LIKE THOSE WE COLLECT, PART OF THIS FIELD. ONE OF THE TRENDS TOWARD TAKING ADVANTAGE OF LARGE SCALE UNLABELED SETS IS SELF-SUPERVISED LEARNING, WE COME UP WITH SYNTHETIC PROBLEMS, FILLING IN A MISSING WORD IN A SENTENCE, USING PREDICTION PROBLEMS TO TRAIN LARGE SCALE MODELS. THIRD IS HARDWARE. WE ARE FAMILIAR WITH THE GPUs, AND FOURTH IS ARCHITECTURE, DEEP NEURAL NETWORKS WHICH HAVE HUNDREDS OF BILLIONS OF PARAMETERS LENDING THEMSELVES TO COMPUTATION. THE STRUCTURE OF THESE ARCHITECTURES MAY NOT MATTER AS MUCH AS THEIR FLEXIBILITY. SO PIVOTING NOW HOW CAN WE TAKE ADVANTAGE OF SOME OF THESE ADVANCES IN MACHINE LEARNING, APPLY THEM TO PROBLEMS IN BRAIN AND BEHAVIORAL QUANTIFICATION? TO ANSWER THIS QUESTION I THINK IT MIGHT BE HELPFUL TO TRY TO CATEGORIZE THE TIMES OF -- TYPES OF PROBLEMS WE ENCOUNTER. THESE ARE COARSE BUT IT MIGHT BE A USEFUL FIRST PASS. FIRST, SIGNAL EXTRACTION, HOW DO WE GET SIGNALS FROM VIDEO, POST-EXTRACTION WE'LL TALK ABOUT SHORTLY. SECOND CATEGORY HAS TO DO WITH BUILDING ENCODING AND DECODING MODELS, PREDICTING NEURAL ACTIVITY GIVEN BEHAVIOR, ENCODING SIDE, AND HOW CAN WE PREDICT THE NEURAL SIDE, IMPORTANCE IN INTERFACES, ESTABLISHING THE UNDERLYING NEURAL BASES OF BEHAVIOR. TRANSLATIONAL END THESE ARE IMPORTANT FOR UNDERSTANDING BIOMARKERS OF DISEASE. THIRD, MODELS THAT CAN SIMULATE AND REPRODUCE PATTERNS. THESE ARE BUILT FROM LATENT VARIABLE MODELS, USEFUL FOR DIMENSIONALITY REDUCTION AND VISUALIZATION, BUT ALSO FOR SOME TOPICS THAT HAVE COME UP ABOUT INTERNAL STATE ESTIMATION, WHICH IS AN IMPORTANT ASPECT OF THIS TALK, AS IS INTERSUBJECT VARIABILITY. MANY ADVANCES IN MACHINE LEARNING HAVE TARGETED PROBLEMS OF DECISION MAKING AND CONTROL. SO HOW CAN THOSE START TO FEEDBACK INTO THE ANALYSIS OF NEURAL AND BEHAVIORAL DATA? ONE EXAMPLE IS TOWARD GUIDING OPTOGENETICKIC STIMULATION TO INDIVIDUALIZE TREATMENTS FOR EACH PATIENT AND COUNTERACT DISEASE AND DISORDER. I'D LIKE TO HIGHLIGHT WORK POINTING AND SHOWING OPPORTUNITIES OF ADVANCES IN THESE DIFFERENT DOMAINS. SIGNAL EXTRACTION IS THE CLEAREST OF THE FOUR, ONE WE'VE SEEN A FEW EXAMPLES OF. WE'VE HEARD ABOUT DEEPLAPCUT, POINTS OF INTEREST, THE ANIMAL FREELY NAVIGATES THE ENVIRONMENT. AND WHY IS THIS A GOOD SHOWCASE OF ADVANCES IN MACHINE LEARNING CONTRIBUTING TO BEHAVIORAL ANALYSIS? TAKE A FOUNDATION MODEL, TRAINED ON A MAT OF MASSIVE DATASET OF IMAGES, NOT NECESSARY FOR THIS TASK, USE TRANSFER LEARNING TO ADAPT TO THE PROBLEM OF POST-TRACKING. THIS IS SHOWING IMPROVEMENTS IN ABILITY. YOU CAN SEE FROM SLEEP IN THE MIDDLE, HIGHLY ACCURATE ABILITIES TO TRACK POINT OF INTEREST. THE SKELETONIZATION IS A HIGHLY COMPRESSED VERSION, ONE TREND THAT I'M SEEING THAT IS REALLY IMPORTANT AND EXCITING IS TOWARD A MORE COMPREHENSIVE QUANTIFICATION OF BEHAVIORAL, FINE GRAINED MEASUREMENT OF OROFACIAL FEATURES, AS YOU SEE ON BOTTOM RIGHT MAPPING OF THE WHOLE SURFACE, SHAPE OF ANIMAL'S BODY, THESE ARE THE TIP OF THE SPEAR THAT IS POINTING TO SOME OF THE POSSIBILITIES IN THIS DOMAIN. ENCODING AND DECODING, DECODING IS THE PROBLEM OF INFERRING BEHAVIORAL OUTPUTS FROM NEURAL MEASUREMENTS. ONE APPLICATION IS IN THE AREA OF DEVELOPING BRAIN-COMPUTER INTERFACE. I'LL HIGHLIGHT WORK FROM LABS AT STANFORD, THEY DEVELOPED A BRAIN-COMPUTER INTERFACE FOR DECODING HANDWRITTEN CHARACTERS. USING THAT ACTIVITY TO GUIDE A HANDWRITING -- >> SCOTT, YOU HAVE ONE MINUTE LEFT. >> ALL RIGHT. I'LL SPEED THIS UP CONSIDERABLY. WHERE DO WE GO FROM HERE? THEY HAVE SHOWN THIS FOR HANDWRITING, THE SAME STRATEGY APPLIES TO OTHER BEHAVIORS LIKE DECODING HIGHER DIMENSIONAL BEHAVIORAL OUTPUTS, MUSCLE ACTIVITY, SPEECH, ET CETERA. THE KEY ASPECT OF THIS IS BUILDING GOOD GENERATIVE MODELS FOR THOSE TYPES OF TIME SERIES, MORE THAN THE LANGUAGE MODELS THEY USED IN THIS EXAMPLE. THIRD EXAMPLE OF GENERATIVE MODELING, I TOLD YOU HOW THESE TOOLS ALLOW US TO GET LOW DIMENSIONAL REPRESENTATIONS OF NEURAL ACTIVITY. ONE EXAMPLE FROM EVA DYER'S LAB, DISENTANGLING CONTENT FROM STYLE OF BEHAVIOR. THEY ARE USING METHODS LIKE VARIATIONAL AUTOENCODERS, I THINK HERE AGAIN THERE'S TONS OF ROOM FOR IMPROVEMENT AS WE CONTINUE TO PUSH THIS FORWARD. THIS IS AN AREA WHERE MY LAB IN PARTICULAR IS EXCITED ABOUT THE POSSIBILITIES FOR DEVELOPING BETTER GENERATIVE MODELS FOR BEHAVIOR. I THINK WE HAVE THE INGREDIENTS NECESSARY TO DO SO. I'LL SKIP THIS LAST EXAMPLE. THE FOURTH AREA WHERE M.L. CAN CONTRIBUTE IS IN THIS ONLINE ENCLOSED-LOOP CONTROL OF NEURAL AND BEHAVIORAL SYSTEMS. ONE EXAMPLE FOR MAJOR DEPRESSIVE DISORDER HIGHLIGHTING WORK FROM KATHERINE SCANGOS, LEADING A BREAKOUT SESSION TOMORROW. I THINK THIS IS AN AREA WHERE M.L. STANDS TO GIVE MORE SUMMARIES THAN JUST SYMPTOMS, PATIENT-REPORTED SYMPTOM SCALES. COLLABORATORS AT HARVARD HAVE SHOWN HOW MOTION SEQUENCING TECHNOLOGY CAN GIVE RICH DESCRIPTION OF BEHAVIOR, CANDIDATES FOR DEVELOPING BETTER BIOMARKERS OF DISEASE TARGETED WITH REINFORCEMENT ALGORITHMS FOR CONTROL. THIS IS FOUR AREAS, MY ATTEMPT TO TRY TO ORGANIZE OUR THOUGHT WHERE WE CAN AND HOW WE CAN APPLY ADVANCES IN M.L. TO THE DEVELOPMENT OF BETTER TECHNOLOGIES FOR BRAIN AND BEHAVIORAL MODELING AND QUANTIFICATION. THEY ARE NOT COMPLETE. ONE THING THAT'S MISSING IS ANY THEORY BUILDING. THAT FALLS UNDER THE UMBRELLA OF GENERATEIVE MODELING BUT DESERVES ITS OWN PLACE ON THE PODIUM. AND I'LL LEAVE IT AT THAT, LOOKING FORWARD TO THE DISCUSSION WITH THE OTHER PANELISTS. >> THANK YOU, SCOTT. WONDERFUL TALK. WE MOVE TO THE NEXT REPRESENTATION, DR. SHANECHI. >> THANKS VERY MUCH. A PLEASURE TO BE HERE TODAY AND TALK ABOUT SOME OF OUR IDEAS ON JOINT MODELING AND BY HEIFER DEBEHAVIOR AND NEURAL ACTIVITY FOR DECODING AND MODULATION. THERE'S BEEN WORK ON DYNAMIC MODELING, HERE SHOWING FROM ONE TIME TO THE NEXT TIME STAMP. IN LINEAR FORM. THERE'S BEEN ALSO NON-LINEAR DEEP LEARNING APPROACHES TO BUILD THESE TYPES OF DYNAMIC MODELS. THE GOAL HAS BEEN TO DESCRIBE VARIANCE, BUT WE'RE INTERESTED IN THE NEURAL STATES THAT ARE BEHAVIORALLY RELEVANT, NEURAL ACTIVITY RELATING TO BEHAVIOR OF INTEREST. LET'S SAY ARM MOVEMENTS. IN THIS CASE WE WANT TO EXTRACT LATENT STATES. WE CAN DECOMPOSE INTO THOSE JUST IN NEURAL ACTIVITY, THOSE THAT ARE SHARED, GREEN ONES, THE BLUE ONES IN BEHAVIOR. IF YOU USE UNSUPERVISED METHODS WHAT HAPPENS? WHAT CAN HAPPEN IS THAT WE MAY EXTRACT RED STATES, THE NEW DIRECTLY RELEVANT TO THE BEHAVIOR OF INTEREST. THERE'S REPRESENTATION THAT DOES THE OPPOSITE. FIRST, DON'T LOOK AT NEURAL ACTIVITY, DYNAMIC MODEL OF BEHAVIOR. WHAT CAN HAPPEN IS WE MAY EXTRACT THOSE BLUE STATES THAT WHILE IN BEHAVIOR ARE NOT ENCODED IN A PARTICULAR RECORDING. WE EXTRACT AND DISASSOCIATE. I'M GOING TO SHOW WHAT BENEFITS THIS CAN HAVE. THE METHOD IS A LATENT STATE DYNAMIC METHOD, RED AND GREEN ELEMENTS, BASED ON IDEA OF PROJECTION. MORE IMPORTANTLY CAN PRIORITIZE THOSE SHARED BEHAVIORALLY RELEVANT STATES. TO SEE HOW THIS CAN BENEFIT OUR MODELING OF THESE BEHAVIORALLY RELEVANT DYNAMICS APPLY METHOD IN MONKEY DATASETS PERFORMING NATURALISTIC REACHING, GRASPING AND RETURNING MOVEMENTS. LOOK AT EXPLANATION OF BEHAVIOR FROM NEURAL ACTIVITY AND DECODING OF BEHAVIOR. WE FOUND THIS JOINT MODELING ALLOWED US TO ACTUALLY MUCH BETTER DECODE BEHAVIOR, COMPARED TO REPRESENTATIONAL MODELS AND UNSUPERVISED DYNAMICAL MODELS. MOREOVER, BY DISASSOCIATING RELEVANT DYNAMICS WE FOUND BEHAVIORALLY RELEVANT DYNAMICS HAD MUCH LOWER DIMENSIONALITY THAT WE WOULD OTHERWISE CONCLUDE. THE SAME HELD COMPARED WITH SEQUENTIAL. THE OTHER APPLICATION THAT THIS JOINT MODELING HAS IS FOR TARGETED DIMENSIONALITY REDUCTION, DYNAMIC MANNER. WE USED THIS JOINT MODEL AND UNSUPERVISED MODELS TO VISUALIZE LOW DIMENSIONAL REPRESENTATION OF NEURAL ACTIVITY, DURING THE REACH AND RETURN MOVEMENT ALSO. WHAT WE FOUND WAS THAT THE ALGORITHM FOUND ROTATIONAL PATTERNS THAT THEY ARE MORE CONGRUENT WITH THE FACT THAT MOVEMENT ITSELF WAS CHANGING DIRECTIONS AS YOU CAN SEE, ROTATIONS ARE CHANGING DIRECTION, THE OTHER ROTATIONS ARE NOT. IF YOU WANT TO MAINTAIN IT WAS, THE JOINT MODELING (INDISCERNIBLE). WHILE I'VE SHOWN LINEAR INSTANTIATIONS WE CAN EXTEND TO THE REALM OF NEURAL NETWORKS AS WELL, ALLOWING FOR COMPLEX NON-LINEAR MODELING. THINK ABOUT BUILDING A MODEL WHERE EACH PARAMETER CORRESPONDS TO ONE OF THESE INTERPRETABLE MAPPINGS, WE CAN DEVELOP LEARNING ALGORITHMS THAT CAN AGAIN DISASSOCIATE AND PRIORITIZE SHARED DYNAMICS BY COMING UP WITH COST FUNCTION THAT ENCOURAGE NOT ONLY NEURAL PREDICTION BUT ALSO BEHAVIOR PREDICTION. WE'VE SHOWED IMPROVING DESCRIPTION OF NEURAL AND BEHAVIORAL DATA. FINALLY THE MODELS CAN BE USED AS TOOLS FOR SCIENCE TO DECOMPOSE TRANSFORMATION FROM NEURAL ACTIVITY TO BEHAVIOR, MAPPINGS, WHERE IS NON-LINEARITY. WE'VE FOUND CONSISTENT RESULT ACROSS MULTIPLE DOMAINS, NON-LINEARITY CAN BE SUMMARIZED, SUGGESTING DOWNSTREAM CORTICAL PROCESSING THAT CAPTURES NON-LINEARITY. WHAT ABOUT APPLICATION TO NEUROTECHNOLOGY? THESE DYNAMICAL MODELS CAN BENEFIT BRAIN-MACHINE INTERFACE. A SECOND APPLICATION COULD BE BMIs THAT AIM TO REGULATE ABNORMAL STATES, FOR EXAMPLE, THE SAME MOOD IN DEPRESSION. WE WANT TO BUILD THESE CLOSED-LOOP STIMULATION SYSTEMS THAT CAN DECODE MENTAL STATE SUCH AS MOOD AND USE THAT AS FEEDBACK TO ADJUST DEEP BRAIN STIMULATION PARAMETERS, TAKE THE BRAIN TO HEALTHY TARGET STATE. BUILDING THESE CLOSED LOOP STIMULATION SYSTEMS, FIRST THEY CAN BE HELPFUL TO BUILD DECODER, BY BIDDING DYNAMIC ENCODING MODEL, HOW MOOD IS REPRESENTED IN NEURAL SIGNAL. IT CAN BE IMPORTANT IN BUILDING INPUT-OUTPUT MODELS, HOW STIMULATION INPUTS CHANGES NEURAL ACTIVITY AND THEREFORE UNDERLYING MENTAL STATE, FINALLY MODELS CAN BE INCORPORATED WITHIN FEEDBACK CONTROLLERS TO ADJUST THE STIMULATION. >> ONE MINUTE. >> SURE. SO, FOR EXAMPLE, WHEN WE TALK ABOUT MENTAL STATES SUCH AS MOOD OBSERVATIONS OF THESE MENTAL STATES, BEHAVIORAL OBSERVATIONS ARE IN THE FORM OF QUESTIONNAIRES, I'M SHOWING ECoG ACTIVITY, I SHOW THE QUESTIONNAIRE WE GIVE PATIENTS TO MEASURE THEIR MOOD, ESSENTIALLY IPAD-BASED QUESTIONNAIRE. YOU CAN SEE IN THE CASE OF MENTAL STATES, BEHAVIORAL MEASUREMENTS ARE UNLIKE MOVEMENT SPARSE. WE HAVE SPARSE LABELS TO TRAIN OUR MODELS. WE'VE SHOWN WITH UNSUPERVISED METHODS WE CAN DECODE MOOD, THERE'S A LOT OF POTENTIAL IN USING THESE JOINT APPROACHES TO SIGNIFICANTLY IMPROVE THE QUALITY OF DECODING. WHILE WE'VE SHOWN IN UNSUPERVISED MANNER THAT WE CAN DESCRIBE NEURAL VARIATION RESPONSE TO STIMULATION, THERE'S A LOT OF POTENTIAL IN EXTENDING THESE MODELS TO DESCRIBE THE RESPONSE ON JUST MOOD RELEVANT NEURODYNAMICS FOR EXAMPLE SO WE CAN TARGET JUST THOSE DYNAMICS WHILE MINIMIZING SIDE EFFECTS. THERE'S A LOT OF POTENTIAL FOR BUILDING LEARNING ALGORITHMS, THAT CAN CONSIDER JOINTLY BEHAVIOR AS WELL AS NEURAL ACTIVITY, BOTH TO STUDY THE NEURAL BASIS OF BEHAVIOR BUT ALSO TO DEVELOP NEXT GENERATION TECHNOLOGIES. WITH THAT I'LL PASS THE FLOOR TO THE NEXT SPEAKER. >> THANK YOU SO MUCH. WE MOVE TO THE NEXT AND LAST PRESENTATION FOR THE PANEL, DR. BERMAN. TAKE IT AWAY. >> THANK OF YOU. THANKS FOR THE INVITATION. AND YOU FOR STICKING AROUND TO THE LAST TALK. I'M GOING TO TALK TODAY ABOUT A MORE OVERVIEW TALK, THINKING ABOUT HOW DOES -- HOW CAN WE DEVELOP A THEORY OF BEHAVIOR THAT ALLOWS US TO MAKE SENSE OF UNDERLYING NEURAL ACTIVITY AND UNDERLYING STRUCTURES WE'RE SEEING. WHEN WE'RE TALKING ABOUT BEHAVIOR, BEHAVIOR OFTEN HAS THIS CONNOTATION OF THINKING ABOUT HOW -- NOT JUST THE PARTICULAR ACTIONS AND MOVEMENTS, REALLY ONE OF THE THINGS WE CARE ABOUT, ONE OF THE KEY ASPECTS OF THE WORKSHOP, THINKING ABOUT THE PARTICULAR MOVEMENTS THAT AN ANIMAL IS MAKING, ALSO WHAT ARE THE UNDERLYING FEATURE ALSO THAT ARE GENERATING THOSE MOVEMENTS. HOW CAN WE MAKE SENSE OUT OF THESE THINGS, FOR EXAMPLE BETWEEN THE MOVEMENTS AS FUNCTION OF ACTIVITY LEVEL DUE TO CIRCADIAN RHYTHM, HUNGER, AGE, SOCIAL INTERACTIONS, AND THIS IS VERY IMPORTANT FOR THINKING ABOUT CLINICALLY RELEVANT THINGS LIKE MOOD DISORDERS OR OTHER TYPES OF THINGS WE MIGHT BE CARRYING ABOUT AS WE THINK ABOUT THIS. OF COURSE, THERE'S BEEN -- WHEN I SAY THERE'S BEEN A PAUCITY OF THEORY TO THINK ABOUT HOW DO WE UNDERSTAND THESE INTERNAL STATES OF BEHAVIOR AND HOW THIS ACTUALLY WINDS UP RELATING TO THOSE OUTPUTS. BUT THERE'S BEEN A LOT OF LINGUISTIC THEORY THAT'S BEEN AROUND, WHAT I MEAN BY THAT IS BEAUTIFULLY WRITTEN THOUGHTFUL IDEAS THAT COME FROM, SAY, FOR EXAMPLE, THE CLASSICAL ETHOLOGY WORKS FROM THE 50s, '60s, 70s, THE STRUCTURED BEHAVIOR OFTEN IS A FULL REPERTOIRE, NOT JUST ONE THING AN ANIMAL IS DOING TO UNDERSTAND ONE THING AND UNDERSTAND HOW IT'S DOING IT BUT WHERE DOES THAT FIT IN THE CONTEXT AMONGST THE REPERTOIRE OF BEHAVIORS AND THINKING ABOUT HIERARCHY OF STATES, INTERNAL STATES, TRYING TO UNDERSTAND HOW THOSE THINGS ARE INTERACTING IN TIME. WE CAN LOOK TO SEE -- THIS IS SIMPLISTIC MODELS, IF WE LOOK AT BEAUTIFUL WORK OTHER PEOPLE HAVE DONE, THIS IS GREAT WORK INCLUDING BY SEVERAL PEOPLE THAT ARE HERE ON THE PANEL TODAY, WHAT WE SEE IS THESE TYPES OF MODELS FOR DESCRIBING BEHAVIOR THAT HAVE A TENDENCY TO HAVE A SHORT TIME SCALE AND BE DISCRETE. THERE'S A LOT OF JUSTIFICATION WHY THAT MIGHT BE A GOOD WAY TO TALK ABOUT BEHAVIOR. OR WE SAY, OKAY, I'M GROOMING, I AM COURTING. I AM DOING SOMETHING ELSE. OKAY, I'M GOING TO SWITCH BETWEEN THINGS. THAT DOESN'T NECESSARILY -- OOPS -- MAP ON BETWEEN -- THIS IS THE PICTURE. I'M SORRY, THE FONTS GOT TRANSLATED BETWEEN THE MAC AND P.C. WE SEE THIS PICTURE WHEREAS TIME MOVES ALONG, OUR BEHAVIORAL OUTPUTS ARE ESSENTIALLY A FILTER. >> ONE MINUTE REMAINING. >> ONE MINUTE REMAINING? OKAY. ESSENTIALLY A FILTER BETWEEN WHAT'S THE INTERNAL STATE AND EXTERNAL STATE. TO MOVE OUT, HOW CAN WE THINK ABOUT THIS FROM THE STANDPOINT OF HOW OTHER PEOPLE GENERATED MODELS? I'M GOING TO WHAT MALCOLM TALKED ABOUT, AND GO BACK TO THE HIGGS BOSUN. AND THEORETICAL PARTICLE PHYSICS, UNDERLYING THEORY INVOLVES WRITING COMPLICATED EQUATIONS, PHENOMENONOLOGY BRINGS THE THING INTO THE WORD OF THE MEASURABLE, IT'S A COMPLICATED EVENT, WE WIND UP GETTING THESE THINGS. SAY I'M LOOKING AT THIS WATER, CALCULATE THE SPEED OF THE WAVE, WHAT THE AMPLITUDE IS. IF I WERE A STATISTICIAN, I WOULD SAY, OKAY, OR MINIMALLY THIS IS WHAT MODERN MACHINE LEARNING IS DOING. I HAVE MY HEIGHT, EXTERNAL VARIABLES, I'LL TAKE DATA, TRAIN A MODEL TO PREDICT THAT, CORRECT LIP OUTPUT MY PREDICTION. IF I'M ABLE, ANOTHER WAY TO THINK ABOUT IT, CAN WE ISOLATE RELEVANT VARIABLES FROM OUR ENVIRONMENT THAT WIND UP BEING PREDICTIVE? AND FROM THAT THIS IS WHERE YOU WOULD START WITH A BROAD -- >> THANK YOU, GORDON. YOU STILL HAVE A FEW MINUTES REMAINING. SORRY ABOUT THAT. >> I THOUGHT THAT WAS A BIT QUICK. ALL RIGHT. WHEW. BASICALLY WHAT WE WIND UP HAVING IS YOU START OFF WITH SOMETHING WHICH LOOKS LIKE A NAVIER-STOKES EQUATION, CONSERVATION OF MASS, MOMENTUM, TRANSLATIONAL AND ROTATIONAL VARIANTS. YOU SAY, OH, THERE'S RELEVANT FEATURES I'M TRYING TO PREDICT, WHICH IS THE WAVELENGTH AND AMPLITUDE OF THAT WAVE. AND THEN FROM MAKING A SET APPROXIMATIONS, I HAVE A PREDICTIVE MODEL. THIS TAKES KNOWING A LOT ABOUT YOUR SYSTEM, SOMETIMES IT'S COMPLICATED, YOU CAN'T DO IT IF YOUR SYSTEM IS TOO COMPLICATED LIKE ANIMAL DEVELOP OF BEHAVIOR, GENE EXPRESSION, EVOLUTION, EVERYTHING. AND IN PHYSICS WE CAN DEAL WITH THIS A LITTLE BIT BY THINKING ABOUT WHAT'S CALLED RENORMALIZATION, YOU CAN SAY ALL THESE TINY SKINS POINTING UP AND DOWN, I CAN FIND COURSE GRADING VARIABLES, GROUP INTO ONE BIG SPIN, I EVENTUALLY FIND THESE ARE THE VARIABLES THAT ARE RELEVANT FOR ME UNDERSTANDING MY SYSTEM. IN OUR GROUP WE THINK ABOUT THESE THINGS. THIS SORT OF -- TAKE THIS METAPHORICALLY. THE IDEA OF IMAGINING A BALL ROLLING THROUGH THE LANDSCAPE, A DETERMINISTIC PART OF THIS, LIKE A BALL IN A WELL, SOME NOISE, PUSHING IT AROUND. EACH BOUNDARIES MIGHT BE A STEREOTYPE BEHAVIOR. WHAT HAPPENS IS THIS MOVES AROUND, THAT SHIFTING IS THE INTERNAL STATES, HOW TO UNDERSTAND IS WHAT MATTERS. I'LL SKIP THIS. THAT'S REALLY -- THERE'S BEEN A LOT OF WORK INCLUDING STUFF THAT WE'VE DONE, INCLUDING BEAUTIFUL WORK BY GREG STEVENS, LENA'S WORK, IN BIOLOCOMOTION. THIS IS THE THING THAT'S IMPORTANT TO UNDERSTAND HOW TO BUILD THEORETICAL MODELS OF BEHAVIOR. ONE OF THE BIG ISSUES, DISCRETENESS. WE KNOW THIS CAN'T BE TRUE IN REALLY IMPORTANT WAYS. I'M SHOWING TWO BEAUTIFUL EXAMPLES FROM FRUIT FLIES. ONE FROM DAVID ANDERSON'S GROUP WHERE YOU SEE BASICALLY BY CHANGING THE LEVEL OF ACTIVATION OF THE PARTICULAR NEURAL CIRCUIT, YOU CAN GET DIFFERENT TYPES OF ACTIONS, THIS QUALITATIVE WAY. QUANTITATIVE WAY. OR IN THESE FRUIT FLIES, DIFFERENT SPECIES HAVE SIMILAR NEURAL CIRCUITRY TOGETHER BUT AS YOU WIND UP GETTING INCREASING ACTIVATION DIFFERENT BEHAVIORS CAN RESULT IN DIFFERENT EVOLUTIONARY BEHAVIORS. THE PICTURE WE HAVE IN OUR HEAD, BASICALLY SETS OF INTERACTING COMPLICATED -- ONE MORE SLIDE AFTER THIS -- INTERACTING COMPLICATED STATES, BASICALLY LOOKING PUSHING AND PULLING INTERACTING WITH EACH OTHER, FILTER BACK AND THEN THE BEHAVIOR FILTERS BACK INTO EVERYTHING ELSE. THIS IS GETTING REALLY TOWARDS WHAT DANI WAS SAYING IN THE FIRST TALK OF THE SESSION, THINKING ABOUT THESE COMPLICATED DYNAMIC FEEDBACK LOOPS, REALLY WHERE THEORY IS GOING TO COME IN, UNDERSTANDING HOW TO INFER IN A DYNAMIC RATHER THAN STATISTICAL WAY. LASTLY, I'LL JUST ADD WHAT WE'RE TRYING TO DO, DATA MODELS AND BACK, WE NEED HIGH QUALITY BEHAVIORAL DATA OVER LONG TIME SPANS, WHERE A LOT OF PROGRESS HAS BEEN MADE AS YOU'VE SEEN. REALLY THE KEY HERE, THE POINT I WANT TO LEAVE WITH, WHAT WE NEED TO BE ABLE TO DO IS THINK ABOUT MODELING IN THEORY FROM THE STANDPOINT OF PHENOMENOLOGY. THIS IS WHERE IN SOME WAYS I WORRY ABOUT USING FULL THROATED MACHINE LEARNING APPROACHES, BECAUSE OF THE IDEA THAT WHAT WE'RE TRYING TO DO IS BRING THINGS TO THE LEVEL WHERE WE CAN UNDERSTAND THE INs AND OUTs IN THE SYSTEM AND MAYBE IT'S THE ARROGANCE THAT COMES WITH WORKING IN A LOT OF SYSTEMS BUT I THINK THAT SOME LEVEL, MUCH IS UNDERSTANDABLE, THE PROBLEM IS HOW CAN WE BUILD THEORIES AND MODELS TO BUILD THAT TYPE OF UNDERSTANDING. WITH THAT I'LL CLOSE. >> THANKS SO MUCH, GORDON. A LOT OF THINGS TO THINK ABOUT. WE HAVE COMPLETED ALL FOUR PRESENTATIONS, DOING VERY WELL ON TIME. NOW WE MOVE TO PANEL DISCUSSION. WE ARE GOING TO HAVE THREE DISCUSSIONS, EACH A SINGLE IDEA OR QUESTION, DISCUSSIONS ARE ALSO OPEN FOR QUESTIONS AND COMMENTS FROM AUDIENCE. EACH DISCUSSION WILL HAVE NINE TO TEN MINUTES, BE MINDFUL OF TIME. FIRST DR. ALLISON WATERS. DR. WATERS, THE FLOOR IS YOURS. >> OKAY. THANKS, DANI, SCOTT, MARYAM, GORDON, A WHIRLWIND TOUR. WE DID IT, FROM DATA TO MODELS AND BACK, AS WE WERE MANDATED. WE COULD TALK ABOUT SO MANY THINGS NOW. AND SO I SEE THAT WE'RE POISED TO DISCUSS A LITTLE BIT ABOUT APPLICATION. WE COULD GO IN LOTS OF DIRECTIONS BUT I THINK MAYBE I'LL REPRESENT MY GROUP'S MANDATE, HERE AT THE NATIONAL FAMILY CENTER FOR ADVANCEED CIRCUIT THERAPEUTICS, WE, LIKE YOU IN YOUR DOMAINS, WANT TO LEVERAGE THESE MODELING INNOVATIONS, TO SPECIFICALLY BETTER INTEGRATE PRECISION NEUROMODULATION WITH BEHAVIORAL INTERVENTION FOR NEUROPSYCHIATRIC PROBLEMS LIKE CBT, ERP, WHICH I WOULD ARGUE IS ACTUALLY THE CHANGE STRATEGY WHAT WE NEED TO BE OR ENHANCE WITH OUR DIRECT TO BRAIN TECHNIQUES. SO I'M KEEPING IN MIND THIS ANECDOTE THAT LENA ALREADY SHARED ABOUT THE FREEZING OF GAIT IN THE LAB, HOW MAYBE WE WANT TO STUDY TINKING PHENOMENON IN TOURETTEEST BUT BRING THE PATIENT IN, THEY DON'T TIC FOR US. WE'RE GOING TO WIRE THEM UP, SEND THEM INTO THE DAILY COMMUTE WITH THEIR VIDEOS AND THEIR SENSORS, MICROPHONES AND SO ON. I'M WONDERING HOW DOES CENTERING ON THAT PATIENT'S EXPERIENCE HIGHLIGHT GAPS OR OPPORTUNITIES IN MODEL BUILDING. IN BEHAVIORAL MEDICINE THE PATIENT IS THE CHANGE AGENT, NOT A PASSIVE OBSERVER OF EVENTS AND RESPONSES, LIKE WE ARE THROUGH THE LENS OF DATA COLLECTION AND MODEL BUILDING. HOW DO EMERGING TECHNOLOGIES KEEP THE PATIENTS IN THE DRIVER'S SEAT? IT'S NOT AN ETHICAL QUESTION. MORE LIKE A PRACTICAL QUESTION. HOW DO WE TAP THE FORCE OF THE INDIVIDUAL TRYING TO MAKE CHANGE IN THEIR LIVES? >> THE QUESTION CAME TO MIND WHEN DANI WAS SPEAKING, AND BOLDLY INCLUDED THE MIND IN HER SCHEMATIC. IT ALSO COMES TO MIND WHEN SCOTT'S DISCUSSING, SAY, MACHINE LEARNING TECHNOLOGIES TO DRIVE CLOSED-LOOP APPROACHES. OR MARYAM, WHAT YOU PROPOSED IN BEINGS ABLE TO SIMULTANEOUSLY MODEL BEHAVIOR AND OF IN BRAIN AND I'M WONDERING IF THAT PARADOXICALLY GIVES MORE FREEDOM TO INVOLVE THE HUMAN IN SELECTING THE MODEL INPUTS, INCLUDING PERHAPS CLOSED-LOOP SYSTEMS. >> BY INPUT, YOU MEAN THE TYPE OF THERAPY, FOR EXAMPLE, OR WHAT DO YOU EXACTLY MEAN BY INPUT? >> WELL, FOR YOU, WHAT IS THE ROLE OF THE INDIVIDUAL'S EXPERIENCE, THEIR AGENCY, IN HOW YOU THINK ABOUT AND INNOVATE YOUR COMPUTATIONAL MODELING APPROACHES? >> UH-HUH. SO IN CLOSED LOOP WE CAN DEFINITELY HAVE SOME SORT OF COOPERATIVE TYPE SYSTEM WE DESIGN WHERE WE DON'T JUST LET THE MACHINE DICTATE THE OPTIMAL THERAPY PARAMETERS OR DOSAGE, FOR EXAMPLE. WE ALLOW THE INDIVIDUAL TO ALSO HAVE A SAY IN THAT. FRICTION, THE SIMPLEST VERSION OF THIS IS AN OVERRIDE, YOU CAN OVERRIDE WHATEVER THE MACHINE DECIDES IS BEST FOR YOUR MENTAL STATE. WE CAN THINK ABOUT COOPERATIVE DESIGNS THAT ALLOW AGENCY TO THE PATIENT IN TERMS OF HOW THEY WANT TO BE TREATED. FOR EXAMPLE, MAYBE THE MACHINE CAN DISPLAY WHAT IT THINKS THEIR MENTAL STATE IS. AND ON THE BASIS OF MAKING A DECISION ABOUT THE NEXT LEVEL OF SIMULATION, AND THEN THE PATIENT, IF THEY DON'T, I DON'T KNOW, THAT'S SOMETHING THEY DISAGREE WITH MAYBE THEY CAN HAVE AN INPUT THERE. WE CAN PUT THAT WITHIN THE COMPUTATIONAL FRAMEWORK TO MODEL THIS KIND OF INTERACTION BETWEEN THE HUMAN AND THE MACHINE THAT IS ALSO A VERY INTERESTING PROBLEM TO THINK ABOUT. >> DANI WHY DID YOU CHOOSE TO MIND AND NOT MAKE THAT PART OF BRAIN BEHAVIOR? >> I THINK OF BRAIN AS SOMETHING THAT DOES CONNECT TO THE BODY WITH THE ENVIRONMENT. AND SOMETIMES THERE ARE MENTAL PROCESSES THAT DON'T NECESSARILY COME OUT AND AFFECT THE ENVIRONMENT. I WANTED TO HAVE A PLACE IN SCHEMATIC DIAGRAM WHERE THE MIND CAN BE DOING THINGS THAT DON'T COME OUT BUT THAT COULD DRIVE CHANGES THROUGH THE BRAIN, INTO THE ENVIRONMENT. I THINK THAT HAVING THAT SECTION IS IMPORTANT. THOSE INTENTIONS, BELIEFS, GOALS, OR HOPES OR MOTIVATIONS, WE STILL DON'T HAVE A FULL UNDERSTANDING OF HOW THOSE MENTAL PROCESSES ARE INSTANTIATED IN THE BRAIN AND THEN FROM THERE HOW BEHAVIOR IS ALTERED. FEELS LIKE THAT SECTION NEEDS TO BE THERE BECAUSE WE DON'T UNDERSTAND IT. I THINK YOUR IDEA, YOUR EXAMPLE OF THE PERSON WHO DOESN'T TIC IN THE LAB BUT DOES IN THEIR EVERYDAY LIFE IS FASCINATING, SUGGESTING IMPORTANCE, FOR ME WHAT THAT DOES IS SUGGESTS THERE'S A DYNAMIC REGIME THE PERSON IS WORKING IN, WHEN YOU CHANGE THE ENVIRONMENTAL CONTEXT THEY ARE IN A DIFFERENT WORKING REGIME, THAT OF THE LAB. THE GOAL IS TO UNDERSTAND HOW DO YOU MOVE FROM ONE DYNAMICAL REGIME TO ANOTHER. AND HOW DOES A GIVEN PATTERN OF INTERACTIONS IN EACH OF THESE SUBSYSTEMS OR BETWEEN THEM, FROM BEHAVIOR TO ENVIRONMENT, BRAIN TO ENVIRONMENT, WHATEVER IT IS, CHANGES THE LANDSCAPE ON WHICH YOU'RE MOVING AND A LOT OF WORK HAS SUGGESTED THAT THE PATTERN OF INTERACTIONS BETWEEN THESE SUBCOMPONENTS DEFINES THE NATURE OF THAT LANDSCAPE IN SOME WAY. IF WE CAN UNPACK AND UNDERSTAND THAT PATTERN OF INTERACTIONS MORE IT WOULD PROVIDE US WITH A LOT OF INTUITION ABOUT HOW EASY IT IS FOR THE PERSON TO MOVE FROM BEHAVIOR TO BEHAVIOR, COGNITIVE PROCESS TO COGNITIVE ENVIRONMENT, AND THAT BRINGS ME TO COGNITIVE BEHAVIORAL THERAPY. YOU'RE RIGHT, IT'S AN INTERESTING CHALLENGE TO COMBINE THAT WITH OTHER METHODS, AND UNDERSTAND HOW TO USE THEM TOGETHER OVER DIFFERENT TIME SCALES, OR DIFFERENT FREQUENCIES, TO HELP THE PERSON ENACT CHANGE. YOU'RE RAISING A SUPER CRITICAL QUESTION. I THINK THESE MORE COMPLEX APPROACHES ARE NEEDED TO ADDRESS IT, BUT THE PARTICULAR WAY TO DO THAT STILL FEELS FUZZY, IN MY MIND. >> GORDON, YOU DESCRIBED BEHAVE AS A FILTER BETWEEN INTERNAL AND EXTERNAL STATES. >> I WANT TO SAY THAT WAS MY DESCRIPTION, NOT WHAT I THINK IT IS, BUT WHAT MOST MODELS TREAT IT AS. THAT WAS MY CRITICISM OF MOST MODELS. >> HOW DO YOU FEEL LIKE THAT METAPHOR IS KEEPING THINGS FUZZY, NOT CLARIFYING? >> WELL -- >> ONE MINUTE REMAINING. >> I THINK IN MY MIND, I'LL BE QUICK, IT'S BASICALLY THERE'S -- IT SAYS THE CERTAINLY STATE IS ONE THING. INTERNAL STATE IS A SET OF THINGS PUSHING AND PULLING AGAINST EACH OTHER, FROM CONTROL THEORY WE NOW THE PUSH-PULL MECHANISM IS WAY MORE STABLE IN A LOT OF CASES THAN HAVING A SINGLE VARIABLE THAT WE'RE TRYING TO DO A FIXED POINT, A SET CONTROL ON. SO THIS NOTION OF THINKING ABOUT INTERACTIONS BETWEEN THE STATES AND HOW THOSE STATES ARE THEN BEING FED BACK ONTO BY THE BEHAVIOR, I THINK IS ALMOST A FUNDAMENTALLY DIFFERENT TYPE OF MODEL THAN WE'RE SWITCHING BETWEEN DIFFERENT SORT OF LIKE HIDDEN MARKUP MODEL TYPE STATES. I THINK THAT SORT OF IS A FUNDAMENTALLY DIFFERENT WAY OF VIEWING THE WORLD. >> OKAY. THANKS FOR THE BRIEF CONVERSATION. I LOOK FORWARD TO PICKING UP THE TOPICS WHEN I SEE YOU IN PERSON. >> THANKS, DR. WATERS, FOR LEADING THIS GREAT DISCUSSION. WE MOVE TO THE SECOND DISCUSSION LEADED BY DOLU OBATUSIN. PLEASE TAKE IT AWAY. >> YEAH, THANK YOU FOR THAT. THAT WAS AN AMAZING SET OF TALKS. AND DEFINITELY LEARNED A LOT. YOU KNOW, MY BACKGROUND IS BIOMEDICAL ENGINEERING, PRAGMATIC APPLICATION BUILDING DEVELOPMENT. ESSENTIALLY, LEVERAGING ENGINEERING PRINCIPLES TO DRESS HEALTH CARE CHALLENGES AND WHAT NOT. IN RECENT YEARS I'VE HAD THE OPPORTUNITY TO WORK AT THE CENTER FOR ADVANCED THERAPEUTICS, WORKING ON TREATMENT RESISTANT DEPRESSION. PREVIOUSLY AT EMORY, NOW AT SINAI. SO AT OUR CENTER, DR. WATERS MENTIONED THIS IN HER INTRODUCTION AS WELL, ESSENTIALLY WE ARE IN THE BUSINESS OF BIOMEDICAL IDENTIFICATION FOR TREATMENT RESISTANT DEPRESSION, PATIENTS TREATED WITH ASSISTIVE DEVICE, ALLOWS YOU TO COLLECT BRAIN RECORDINGS, WHERE ON A DAILY BASIS THEY DO RECORDINGS OF THE BRAIN, RECORDING FROM WEARABLES, ACTIVITY AND SLEEP, RATING SCALES, SO IT'S VERY MUCH IN ALIGNMENT WITH THE THEME OF THIS WORKSHOP ON BRAIN AND BEHAVIORAL QUANTIFICATION AND SYNCHRONIZATION. WE'RE COLLECTING A LOT OF DATA IN THE LAB. EVEN IN A SIMULATED ENVIRONMENT, PATIENTS CAN ESSENTIALLY INTERACT WITH SIMULATED ENVIRONMENT, AND ALLOWS US TO CAPTURE POSE. SCOTT MENTIONED BIORHYTHMS, OPEN POSE USED IN TRANSFORMATION MODELS, I WANTED TO RIF ON THAT. ONE THING THAT STRIKES ME WE KIND OF LIKE DEVELOP, YOU KNOW, REALLY DEEP MODELS FOR DIFFERENT SETS OF INPUTS. WHETHER IT'S VIDEO, ACTIVITY TRACKING, VOICE. BUT IT'S NOT QUITE CLEAR HOW MODELS CAN BE DEVELOPED FOR MULTI-MODAL DATA COLLECTION. THIS IS SOMETHING WE'RE GETTING INTO WHERE WE CAN ACQUIRE MULTIPLE DATA ALONG WITH INCLUDING NEURAL DATA, AND IT'S NOT QUITE CLEAR, RIGHT, HOW TO BUILD MODELS THAT CAN INTEGRATE DIFFERENT INPUTS, YOU KNOW, DIFFERENT MULTI-MODAL INPUTS. I WAS WONDERING IF MAYBE SCOTT CAN SPEAK TO THAT, AND THE REST OF THE PANELISTS AS WELL, ON HOW TO BETTER BILL THESE KIND OF MODELS THAT INTEGRATE DIFFERENT MULTI-MODAL INPUTS. >> GREAT QUESTION. YOU KNOW, IT'S ONE THAT IS -- MY CAMERA DOES THIS WEIRD ZOOM-OUT THING. IT'S ONE THAT'S NOT UNIQUE TO THE TOPICS OF THIS WORKSHOP. TAKING THAT BIGGER PICTURE OF WHERE ELSE IN THE BROADER M.L. WORLD ARE WE ENCOUNTERING SIMILAR ANALOGOUS PROBLEMS. PROBLEMS HAVE TO DO WITH MODEL JOINT IMAGES, CAPTIONS, SPEECH AND TEXT. MAYBE TRYING TO GENERATE OR EXPLAIN AS MUCH SHARED VARIANTS AS POSSIBLE WHILE ALLOWING FOR THINGS THAT ARE UNIQUE TO ONE DOMAIN VERSUS THE OTHER, MAYBE IF YOU'RE MEASURING NEURAL ACTIVITY, IN SOME PART OF THE BRAIN, IT MAY NOT HAVE THE NECESSARY SIGNALS TO PREDICT CERTAIN ASPECTS OF BEHAVIOR. YOU HAVE TO HAVE SOME OTHER WAY OF EXPLAINING THAT OTHERWISE UNEXPLAINABLE VARIANT. THEY ARE ANALOGOUS, APPEARED IN MARYAM'S TALK WITH SHARED LATENT VARIABLES PREDICTING NEURAL AND BEHAVIORAL MEASUREMENTS. AGAIN, NOT TO SAY THAT THE PROBLEM IS SOLVED, BUT I THINK THIS IS AN AREA OF ACTIVE RESEARCH, ONE THAT I'M OPTIMISTIC ABOUT, AN INTERESTING AREA FOR RESEARCH. WHILE I HAVE THE MIC I DO WANT TO RESPOND TO A QUESTION THAT CAME UP IN THE CHAT, FROM DAVID ANDERSON, ABOUT ENCODING AND DECODING. AND ASKING DO WE RUN THE RISK OF INFERRING SPURIOUS NON-CAUSAL RELATIONSHIPS BETWEEN NEURAL ACTIVITY AND BEHAVIOR ANALOGOUS TO A SPEEDOMETER WHERE CAN YOU PREDICT, BUT IF THE CAR BREAKS DOWN, YOU CAN'T FIX IT, HOW CAN WE GO BEYOND THE STRICTLY CORRELATIONAL ANALYSES TO SOMETHING THAT WOULD GIVE MORE CAUSAL UNDERSTANDING. I THINK WHAT THIS IS GETTING AT IS THAT'S FOUR BUCKETS OF ORGANIZATION, PERHAPS INSUFFICIENT TO EXPLAIN ALL OF THE DIFFERENT TYPES OF CHALLENGES THAT WE COME UP WITH. IF HAD YOU TO ADD A FIFTH ONE, I DON'T KNOW, I'M TORN HERE. I THINK GORDON'S EMPHASIS ON DEVELOPING MORE SIMPLE THEORETICALLY GROUNDED MODELS IS ONE THAT'S QUITE IMPORTANT BUT EQUALLY IMPORTANT IS ESTABLISHING CAUSAL RELATIONSHIPS BETWEEN NEURAL AND BEHAVIORAL DATA. WE CAN START TO DO THAT WITH OPTOGENETIC TOOLS, THIS IS AN AREA THAT'S AN EXCITING AREA OF ONGOING RESEARCH, WE NEED AS A FIELD TO HAVE OUR EYE ON. >> WE HAVE ONE MINUTE REMAINING. >> WHAT COMES TO MIND ALSO LIKE IN THE PRACTICAL ASPECT OF TRYING TO USE SOMETHING LIKE TRANSFER LEARNING IS SOMETIMES LIKE I THINK ABOUT OUR DATA SCIENTISTS, WHO TRIED TO USE SOMETHING LIKE THE FREE EXPRESSION ALGORITHM, TRAINED ON 6 MILLION VIDEOS AROUND THE WORLD, MEASURED IN EXPRESSION, NOT OF SOME ALGORITHMS ARE NOT EASILY OBTAINABLE, YOU HAVE TO MAKE A SPECIAL REQUEST. AND THE REASON WHY THIS MATTERS, A LOT OF PEOPLE ARE DOING FACIAL ANALYSIS, AND IF YOU THINK ABOUT THE CONSTRAINTS AND GUARD RAILS PUT IN PLACE FOR GOOD REASONS, AND SOME STUDY DESIGN, YOU WANT TO SHARE THE INDIVIDUAL VIDEOS, YOU SHOULD BE THINKING ABOUT IN TERMS OF HOW TO BETTER LEVERAGE SOME BIG DATA TOOLS OUT THERE MAYBE NOT SO EASILY ACCESSIBLE. >> ALL RIGHT. WE CAN CONCLUDE THIS DISCUSSION HERE. THANKS, EVERYONE. NOW THE THIRD DISCUSSION WITH DR. TIM BROWN TO LEAD THE DISCUSSION. >> HI, EVERYBODY. I WANT TO SAY THANKS TO DANI, SCOTT, MARYAM, AND FOR THE SERIES SERIES OF EXCELLENT TALKS. I LEARNED A LOT. NOW IT'S TIME FOR WHAT MIGHT SEEM LIKE A DEPARTURE. I'M A PHILOSOPHER BY TRAINING, BIOETHICIST BY PROFESSION, UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE, BIOETHICS AND HUMANITIES DEPARTMENT. MY BIGGEST WORRIES ARE ABOUT HOW WE CURATE DATA, THE CONCLUSIONS THROUGH ANALYSES AND IMPACT OF OUR CONCLUSIONS ON A VULNERABLE GROUP DOWNSTREAM, RIGHT? AND SO I WANT TO START US OFF BY WALKING THROUGH A LINE OF NOT SO MUCH ARGUMENT, BUT MORE COMMENTARY, THEN I'LL ASK A COUPLE QUESTIONS. ONE THING I'VE SEEN CONNECTING ALL OF THE TALKS TODAY IS THAT, YOU KNOW, THE IDEA THAT, YOU KNOW, FINDABLE ACCESSIBLE INTEROPERABLE AND REPRODUCIBLE DATA, SO FAIR DATE, WILL BE VITAL TO THE CREATION OF COMPUTATIONAL MODELS FROM MULTI-MODAL DATA. HAVING SO MANY DIFFERENT SOURCES OF DATA, AND SYNTHESIZING THOSE AND SYNCHRONIZING TO PRODUCE INSIGHTS THAT DRIVE THE MODELS THAT WE USE IN DEVICES, IN DISCOVERY, IN MEDICAL PRACTICE, IT WILL BE EXTREMELY IMPORTANT TO HAVE THESE FAIR DATA PRACTICES TO ENSURE EVERYBODY HAS ACCESS TO DATA. BUT WE ALSO NEED TO REFLECT ON FAIRNESS, ACCOUNTABILITY, TRANSPARENCY, AND ETHICS OF OUR DATA SCIENTIFIC PRACTICES. THIS IS FAMILIAR FROM THE A.I. M.L. SPACE WHERE THERE IS A LOT OF INSTITUTIONAL KNOWLEDGE HOW TO MAKE HEADWAY ON ENSURING FAIRNESS, ENSURING TRANSPARENCY, ACCOUNTABILITY. WE NEED TO MIND THESE FEATURES OF COMPUTATIONAL MODELS, OF SYSTEMS THAT DATA ARE REPORTED ON, DOWNSTREAM USES OF MODELS AND DATA, SO WHEN IT TRANSLATES AT ULTIMATELY LIVED EXPERIENCES OF PATIENTS, DANI IS RIGHT, BEHAVIOR IS COMPLEX BECAUSE OF THE DEEP CAUSAL RELATIONSHIPS BETWEEN BRAINS, BEHAVIORS, AND THE CONTEXT THAT BRAINS AND BEHAVIORS HAPPEN IN. THE STATISTICAL MODELS THAT WE APPLY TO THESE DATA COULD BE COMPLEX IN WAYS THAT COULD BE OPAQUE, THAT COULD STAND IN THE WAY OF ENSURING MODELS DRIVE THE SYSTEMS IN WAYS THAT LEAD TO FAIR OUTCOMES FOR USERS. WE HAVE TO REMEMBER METADATA MEASUREMENT AND ANALYSIS CAN IMPACT HOW GROUPS ARE REPRESENTED IN MODELS AND MEDICAL PRACTICES. SO, FOR EXAMPLE, WHEN MARGINALIZED GROUPS ARE UNDERREPRESENTED IN PUBLIC DATASETS OR IF THEIR DATA -- IF DATA IS SUBJECT TO CATEGORIZATION OR CHARACTERIZATION, COMING BACK INTO THE QUESTION OF HOW DO WE GO FROM DATA TO EXPLANATIONS OF BEHAVIOR, THAT ARE BLIND TO LIVED REALITIES, OUR PRACTICES COULD CREATE ENTIRE SYSTEMS OF PEOPLE'S BEHAVIORS THAT ARE HARMFUL TO THE GROUPS DOWNSTREAM TO VULNERABLE GROUPS DOWNSTREAM. ONE HYPOTHETICAL LATER ON, THIS IS MY SELFISH WORRY, THE POSSIBILITY OF ADAPTIVE DECODING OF BIOMARKER FOR DEPRESSIVE EPISODES, UNIT USING BEHAVIORAL DATA LIKE FACIAL GESTURES, SOME MOMENTS, BLIND TO ME IN OTHERS. THIS IS ONLY ONE POSSIBLE DOWNSTREAM OR CONSEQUENCE OF UPSTREAM DECISIONS IN THE LAB. HOW SHOULD DATA GOVERNANCE WORK? THINKING ABOUT INDIGENOUS POPULATIONS, FIRST NATIONS, PEOPLE WHO IDENTIFY AS LGBTQIA, WHAT ROLE SHOULD THEY PLAY IN THE DATA CURATION PROCESS? WHAT IS THE MOST PRESSING ETHICAL CHALLENGE EACH OF YOU SEE ARISING FROM THE CURATION OF THESE COMPLEX MULTI-MODAL DATA, AND HOW THESE DATA DRIVE THE CREATION OF COMPLICATED STATISTICAL MODELS, HOW DO YOU SEE US GETTING OVER THIS AND MITIGATE THE RISK OF CULTURAL HARM AND OF IN THESE PRACTICES, IN LIGHT OF THESE CHALLENGES? I THINK IN THE ORDER OF THE TALKS IS GREAT. I WANT TO HEAR FROM ALL OF YOU BUT MAYBE IT WOULD BE GOOD TO START WITH DANI AND MOVE TO SCOTT. >> THANK YOU SO MUCH FOR YOUR COMMENTS. I THINK THAT RAISING THESE POINTS IS EXTREMELY IMPORTANT AND I'M GLAD WE GET TO END ON THEM TODAY. SO, I THINK THAT THE PLACE WHERE I SEE POTENTIAL FOR HARM, A LOT OF DISPARITIES RIGHT NOW, CERTAINLY IS IN THE COLLECTION OF DATA. BUT I THINK IS MORE WHERE I FOCUSED, IS ON HOW IDEAS ARE SHARED AMONG SCIENTISTS, AND HOW IDEAS ARE VALUED ACROSS SCIENTISTS. IT'S MORE AT THE SORT OF META LEVEL OF WHOSE IDEAS ARE LISTENED TO, WHOSE QUESTIONS ARE BEING ASKED AND REVERBERATE AND HELP GUIDE THE WORK MOVING FORWARD. SO, SOME OF THE MORE QUANTITATIVE WORK IN MY LAB THAT'S FOCUSED ON THOSE QUESTIONS IS RELATED TO CITATION PATTERNS AND HOW THEY ARE VERY SKEWED ACCORDING TO BOTH GENDER AND RACE AND ETHNICITY. SUCH THAT PEOPLE WHO ARE FROM MARGINALIZED IDENTITIES ARE UNDERCITED IN COMPARISON TO WHAT YOU WOULD EXPECT GIVEN THE NUMBER OF PAPERS THEY PUBLISHED, THE JOURNALS IN WHICH THOSE ARE PUBLISHED AND ET CETERA, LOTS OF FACTORS THAT WOULD SOAK UP VARIANTS, EVEN AFTER YOU ACCOUNT, THERE'S STILL A LARGE GENDER AND RACE AND ETHNICITY EFFECT. I THINK THAT HARMS THE SCIENCE BECAUSE IT HARMS SCIENTISTS, THE PEOPLE DOING THE WORK, ALTERS WHO ACTUALLY MAKES IT TO PROMOTION AND TO RECEIVE A LOT OF THE AWARDS AND ET CETERA. I THINK THAT I'M WORRIED ABOUT THAT. IN TERMS OF WHO WE'RE COLLECTING DATA FROM, THAT'S A WHOLE OTHER QUESTION. OF WHICH IS ONE I CARE ABOUT BUT HAVE LESS IMMEDIATE EXPERTISE. I'LL PASS TO THE NEXT PERSON. >> I AGREE. THANK YOU FOR RAISING THESE REALLY IMPORTANT QUESTIONS AND POINTS. I'M NOT AN EXPERT IN A.I. FAIRLESS, BUT I COMPLETELY AGREE, THIS IS A CRITICAL FIELD FOR US TO BE PURSUING. I WOULD LOVE TO TALK OFFLINE TO HEAR ABOUT YOUR THOUGHTS ON THAT AREA, ONE THING THAT I WILL SAY, MY RESEARCH HAS BEEN ON THE BASIC SCIENCE SIDE UNTIL RECENTLY. >> WE HAVE ONE MINUTE REMAINING. >> OKAY. I'LL KEEP IT SHORT. MOSTLY IT'S BEEN ON THE BASIC SCIENCE SIDE, RECENTLY MOVED MORE IN THE TRANSLATIONAL AREAS, WHICH HAS FORCED THESE ISSUES TO THE FORE. YOU SEE EVIDENCE OF SOME THINGS YOU'RE CONCERNED ABOUT. APPLYING SOME MODELS TO MOUSE BEHAVIOR, WE FIND SOMETIMES THE REPRESENTATIONS THAT YOU GET FOR ONE VERSUS ANOTHER DIFFER NOT BECAUSE THEY ARE FUNDAMENTALLY DIFFERENT IN BEHAVIOR BUT ONE MOUSE IS BIGGER THAN THE OTHER. THE FUTURE EXTRACTION PROCESS, SLIGHTLY BIGGER MOUSE GETS CLUSTERED, MODELS DIFFERENTLY. WE CAN CORRECT THAT BUT IT'S DONE IN A REACTIVE WAY. AND I THINK ONE OF THE THINGS YOU'RE GETTING AT IS THAT AS WE DEPLOY THESE IN THE REAL WORLD WE HAVE TO BE MORE PRO-ACTIVE IN GETTING OUT AHEAD OF THOSE TYPES OF BIASES BEFORE THEY ADVERSELY AFFECT PEOPLE. >> GREAT NOTE TO END ON. I REALLY HATE CUTTING THE DISCUSSION SHORT. THERE WENT MY TIMER. BUT I WANT TO GIVE A SHOUT OUT TO EVERYBODY FOR SUCH HARD WORK YOU HAVE PUT INTO YOUR PRESENTATIONS, AND KEEPING US ON SCHEDULE. AND ALL THE ACTIVE GREAT DISCUSSIONS, WE'LL CLOSE WITH FINAL THOUGHTS FROM KAREN ROMMELFANGER AND JUSTIN BAKER. I'LL HAND THE FLOOR OVER TO THEM AND LET THEM PROCEED. >> I UNDERSTAND I HAVE TEN MINUTES, BUT I'LL USE LESS. IS THAT RIGHT, DANA? >> YOU HAVE 10 TO 15 MINUTES. >> WE'RE ALL TIRED. I'LL DO 10, MAYBE UNDER. THANK YOU FOR STICKING AROUND FOR THE RICH CONVERSATION. I WAS PART OF THE DISCUSSION NOT FROM THE INCEPTION BUT THREE YEARS AGO AFTER THE TEAM HAD ALREADY GIVEN A LOT OF THOUGHT. AND IT WAS BBQ, NOT BBQS, SO THE SYNCHRONIZATION PIECE IS EXCITING. I'M CHARGED WITH SYNTHESIZING IDEAS, CHALLENGES, OPPORTUNITIES TO ADDRESS THOSE CHALLENGES. I'LL BRING UP THREE MAIN POINTS, AND THREE BULLETS UNDER THOSE. FOR SOME WHO MAY NOT BE AWARE, I HAVEN'T HAD THE PLEASURE OF MEETING A LOT OF YOU YET. I'M TRAINED AS A NEUROSCIENTIST, MOVED INTO FORMAL TRAINING IN NEURAL ETHICS, LAUNCHED A NEUROETHICS PROGRAM AT EMORY. CO-CHAIR, BRAIN INITIATIVE, CONSORTIUM OF EXISTING BRAIN INITIATIVES, BRAIN IS PART OF ONE OF THOSE. RECENTLY MOVED TO LAUNCHING INDEPENDENT INTERNATIONAL NEUROETHICS THINK TANK, UNDERWRITING ETHOS IS INCLUSIVITY ACROSS GEOGRAPHY. WE BRING PRACTICAL LEVELS TO ETHICS WE TALK ABOUT AND WE NEED TO THINK ABOUT IN THE NEXT FEW DAYS. THE FIRST POINT IS THAT WE HAVE AN ETHICAL IMPERATIVE FOR BRAIN BEHAVIOR QUANTIFICATION AND SYNCHRONIZATION, IT'S THE TYPE OF ACTIVITY THAT I THINK EVERYONE HERE AND THERE WERE 400 PEOPLE ONLINE THIS MORNING, AGREE THAT BRAIN SHOULD BE TAKING UP IN SOME WAY. BUT IN ORDER TO DO THAT WE NEED TOOLS. EVERYBODY HERE UNDERSTANDS THAT THE LOCALLY GRAIL OF NEUROSCIENCE IS STILL CONNECTING THE BRAIN TO BEHAVIOR. WE STILL HAVE A LOT OF WORK TO DO IN THAT SPACE, MAYBE WE'LL HAVE OUR WHOLE CAREERS TO DO THAT. I WANT TO ORIENT YOU, THE NATIONAL INSTITUTES OF HEALTH, THERE'S A NEUROETHICS WORKING GROUP, AND A TEAM, AND NEUROETHICS IS A PRACTICAL TOOL IN YOUR TOOLKIT THAT CAN BE USED TO SCAN THE HORIZON, NOT MEANT TO SET UP ROAD BLOCKS OR PURELY A POLICING EFFORT, WHICH I THINK THAT RCR TRAINING HAS DONE TO A LOT OF YOU IN YOUR PERCEPTIONS, IT'S WHAT HAPPENED TO ME WHEN I WAS IN MY TRAINING. BUT USED AS A WAY TO ANTICIPATE ETHICAL ISSUES THAT TIM WAS BRINGING UP, AND SOME OTHERS THROUGH THE DAY. ULTIMATELY CAN ADVANCE AND ACCELERATE SCIENCE. AND WITH IN A IN MIND -- THAT IN MIND WE WANS WANT TO ENSURE THE WORK, ETHICAL ADVANCED, SYSTEMATIC IN EXPLORATION. AS SCOTT MENTIONED IN THE LAST PIECE, WE WANT TO GET AHEAD, BE GIVING THE ETHICS UP FRONT. SECOND I WANT TO HIGHLIGHT THE CHALLENGE OF DESCRIBING WHAT IS NORMAL WHEN YOU QUANTIFY EVERYTHING. CERTAINLY THERE ARE IMPORTANT REASONS TO WANT TO QUANTIFY. FAIRLY POWERFUL STORY I BELIEVE I HEARD FROM SOME OF THE NIH FOLKS IS THAT MANY OF THE INDIVIDUALS, PSYCHIATRISTS, MANY OF THE INDIVIDUALS, HER PATIENTS WHO SUBMITTED SUICIDE TOLD HER WHEN SHE ASKED THEY WOULD NOT COMMIT SUICIDE. OUR TOOLS ARE ASKING PATIENTS TO RESULT TO US, AND FALLING SHORT. HOW DO WE DESIGN STUDIES, COLLECT DATA WHERE WE CAN OFFER BETTER EXPLANATORY POWER? AND HOW DO WE USE NON-HUMAN ANIMALS EVEN TO RECAPITULATE COMPLEX HUMAN CONDITIONS LIKE AUTISM? LENA TING REMINDED US OF OPPORTUNITY DIVERSITY AND VARIABILITY, CONSIDER THOSE IN YOUR STUDIES, BEHAVIORS MIGHT BE MORE CONSISTENT THAN UNDERLYING ACTIVITY. JEFFREY NOTED TRYING TO DRAW FROM OUR QUANTIFIED DATA WOULD BE HIGHLY INFERENTIAL. SCIENCE NEVER HAS BEEN JUST ABOUT SCIENCE. IT'S TIED WITH POWER. IT'S IMPORTANT TO CONSIDER WHO IS DRIVING PARAMETERS OF NORMAL AND STANDARDIZATION. JONATHAN, A PROFESSOR OF SOCIOLOGY AND PSYCHIATRY AT VANDERBILT, REMINDS US THAT YET IN UNINTENDED AND OFTEN INVISIBLE WAYS PSYCHIATRIC DEFINITIONS CONTINUE TO POLICE RACIAL HIERARCHY, TENSION, UNSPOKEN CODES IN ADDITION TO SEPARATING NORMAL FROM ABNORMAL BEHAVIOR. PERSONS ARE STIGMATIZED BECAUSE THE ON GENERALIZATION ABOUT DEVIANCE, PERCEIVED VOLATILITY, BE A OF CHARACTERISTICS THAT REMAIN ACCEPTABLE MODES OF DISCRIMINATION. RHETORIC BECOMES WAYS OF POLICING THE BOUNDARIES OF KEEPING THESE PEOPLE OUTSIDE. THESE ARE THINGS WE NEED TO GET AHEAD OF, THINK ABOUT AS WE DESIGN OUR RESEARCH AND OUR QUESTIONS. DOROTHY ROBERTS AT U PENN SAYS HEALTH CANNOT BE DIVORCED FROM SOCIOPOLITICAL CONTEXT, IGNORES THE WAYS IN WHICH -- TO DO SO WOULD BE IGNORE WAYS BIOTECHNOLOGY SHAPE AND ARE SHAPED BY SOCIAL POWER. DATA PRIVACY WAS BROUGHT UP BY TIM AND WILL BE DISCUSSED HEAVILY TOMORROW. THE IMPLICATIONS FOR THE INFERENCES EVEN IN RESEARCH IN CONTEXT WILL LIKELY END UP STILL IMPACTING AREAS BEYOND YOUR LABORATORY, AND WILL LIKELY INTERFACE IN YOUR ANALYSES WITH PUBLICLY AVAILABLE DATA AT SOME POINT. KATE CRAWFORD AND TIM BROWN MENTIONED IN THE CHAT EARLIER, LEADING SCHOLAR OF IMPLICATIONS OF A.I. SAYS BIG DATA ANALYTICS WILL CHANGE HOW WE KNOW AND LIKEWISE SHOULD EXPECT MORE COMPLICATED ETHICAL IMPLICATIONS OF WHAT WE KNOW. DATA IS NOW CONNECTIBLE, ENDEFINITELY REPURPOSABLE, CONTINUOUSLY UPDATABLE, DISTANCED FROM THE CONTEXT OF COLLECTION. AND IN ONE OF THE TOOL KITS THE NIH BRAIN HAS, NEUROETHICS PRINCIPLES, ONE SAYS MOVE BEYOND THE BENCH WITH CAUTION, EVEN AS WE CONSIDER THE COMPARATIVE WORK AND IMPLICATIONS OF INTERPRETATIONS OF SCIENCE SHOULD NOTE THEY SHOULD MOVE OUT INTO A NUMBER OF DOMAINS, FROM THE LAB TO CLINIC, CONSUMER TO NATIONAL DEFENSE, REMINDED PAUL ECKMAN'S, THAT WE DISCUSSED TODAY, AFFECTED CUSTOMS ENFORCEMENT. THE FRAMEWORK IN THE BEGINNING BY HOLLY LISANBY, A SUGGESTION FOR FRAMEWORK THINKING ABOUT OUR SCIENTIFIC AND ENGINEERING CHALLENGES, EQUAL FOOTING WITH SOCIAL CHALLENGES AND SOCIAL CONTEXT AND IMPLICATIONS OF THE WORK WE DO IN THE LAB. DANI NOTED THE MIND IS THE DESIRE, INTENTION, PURPOSE, AND WE SHOULD NOTE THAT EVEN THAT DEFINITION OF BRAIN IS NOT CULTURALLY UNIVERSAL. THAT WILL HAVE IMPLICATIONS FOR HOW DATA IS INTERPRETED, AND HOW THAT RESEARCH MIGHT END UP FEEDING BACK INTO SYSTEMS OF HOW, HOW THE MIND AND BRAIN ARE CONNECTED CAN HAVE IMPLICATION FOR STIGMA AND APPLICATION IN MEDICAL SYSTEMS AND BEYOND. CULTURE INFLUENCES WHAT SCIENCE IS SUPPORTED, WHERE SCIENCE CAN BE CONDUCTED, AND THE TIMES OF ETHICAL FRAMEWORK THAT OF. GAPS IN UNDERSTANDING CAN LEAD TO MISMISSED OPPORTUNITIES FOR COLLABORATION, ADVANCEMENT TOWARDS FUTURE DISCOVERIES, LIMIT ABILITY TO BROADLY SHARE RESULTS, AND REAP THE BENEFITS OF FINDINGS, RESULTING IN FAILURE TO RECOGNIZE SHORT- AND LONG-TERM POTENTIAL AND RISKS OF NEURAL SCIENCE RESEARCH. IT'S IMPORTANT TO NOTE THESE DIFFERENCES DON'T JUST EXIST AMONG NATIONAL ENTITIES. THEY EXIST WITHIN DEFINED SOCIETIES, ALSO AMONG INDIVIDUAL RESEARCHERS AND PRACTITIONERS, THESE ARE THINGS THAT NONE OF US ARE DEVOID OF. SCIENCE IS CONTEXTUALIZED. ANOTHER FEATURE THAT DANI MENTIONED, BRAIN TO BEHAVIOR IS NOT NECESSARILY LINEAR. IT'S NOT A SIMPLE CAUSAL CHAIN. ANY INTERPRETATION OF THE DATA THAT WE FIND FROM ANY OF THE WORK WE MIGHT DO IN THE BBQS FRAMEWORK NEEDS TO BE SIMILARLY CONSIDERED. WHAT DOES IT MEAN TO PUT A BRAIN IN THE CONTEXT OF THE REAL WORLD? TIM REMINDED US OF CHALLENGES THAT WE HAVE TO EXPLORE. ETHICS CAN ADVANCE NEUROTECHNOLOGY AND NEURAL SCIENCE, ENSURE BY THINKING ABOUT ETHICS THAT SCIENTISTS AND THE FUNDERS, LIKE NIH AND NSF, SOCIAL CONTRACT TO SOCIETY IS MET AND SUSTAINED. THE SECOND PART HERE IS THAT WE DO NEED NEW TOOLS AND COLLABORATION, MALCOLM SAID TOOLS FOR SYNTHESIS, COLLABORATORS, THE CHAT MENTIONED MANY HALF A MILLION AUTHORS IS WHAT WE NEED TO TACKLE SOME OF THE MOST CHALLENGING NEUROSCIENCE QUESTIONS. HOW DO WE -- WHO SHOULD THESE COLLABORATORS BE AND WHO GETS TO DECIDE WHO THEY ARE? COMMUNITIES CAN BE AN IMPORTANT PART OF THINKING OF OUR COLLABORATIONS WITH BRAIN BEHAVIOR QUANTIFICATION. THERE ARE OPPORTUNITIES FOR US TO THINK ABOUT WHO MIGHT BE IMPACTED, CONSIDER COMMUNITIES AS PARTNERS WHO MIGHT BE ABLE TO GIVE US EVEN NEW INSIGHTS INTO SOME TOUGH QUESTIONS WE HAVE ABOUT HOW FAR WE CAN MODEL AUTISM, AND WHAT KINDS OF CONCLUSIONS WE CAN DRAW, WHAT KINDS OF UTILITY WE MIGHT FIND. TIM RAISED QUESTIONS HOW TO MITIGATE SOCIOCULTURAL HARM, ALLISON MENTIONED WONDERFUL QUESTION, HOW DOES CENTERING ON THE PATIENT'S EXPERIENCE IN MODELING HIGHLIGHT GAPS FOR US TO EXPLORE WITH BBQ, INTERESTING NOTE SHE SAID THIS IS NOT AN ETHICAL QUESTION BUT IT'S A PRACTICAL ONE. I WOULD ARGUE IT'S BOTH. THESE TYPE OF QUESTIONS, ETHICAL QUESTIONS, COULD BE PRACTICALLY INTEGRATED INTO SCIENCE TO HELP ENHANCE AND ELEVATE THAT SCIENCE. THE SECOND PIECE WITH RAJI AND PETER GAVE US GREAT INSIGHT FROM INDUSTRY, AND I THINK I'LL NEVER SEE A CHIHUAHUA AND A BLUEBERRY MUFFIN THE SAME WAY AGAIN. I DON'T KNOW IF ANYBODY ELSE FELT THAT WAY. PETER INVITED US TO CONSIDER HOW TO TRY NEW OPPORTUNITIES, THINK ON A PRACTICAL WAY TO SCALE AND TRANSLATE IN THE REAL WORLD BEYOND THINKING ABOUT THE TIMES OF CHALLENGES WE MIGHT HAVE AS WE STAY IN A WEDDEDNESS TO STATUS QUO HOW WE DO SCIENCE IN ACADEMIC LABORATORY. THE FINAL THING, THERE'S EMPHASIS, TEAM SCIENCE, HOW DO WE DO SCIENCE? HOW DO YOU CREATE A VALUE PROPOSITION? IT'S JUST AS MUCH A SOCIAL PROBLEM AS TECHNICAL ONE. DANI REMINDED US OF THE PROBLEM WHO WE CITE AS WE THINK ABOUT THE ENTERPRISE OF SCIENCE. WE MIGHT CONSIDER OTHER OUT OF THE BOX PARTERS IN THE CHAT, THERE'S AN EXISTING MOVEMENT ENTIRELY SEPARATE FROM THIS GROUP, DECENTRALIZED SCIENCE, ALSO EXPLORING ACTIVE TOOLS ON HOW TO DEMOCRATIZE SCIENTIFIC PROCESS AND OPEN SCIENCE PRACTICES. THAT'S MY SUMMARY FOR TODAY. THERE'S A LOT OF RICH DISCUSSION AND A LOT OF OPPORTUNITIES FOR -- I HOPE THIS PUT OUT A FEW SEEDS FOR CONVERSATION FOR TOMORROW'S BREAKOUT GROUPS. AND I'M HONORED TO BE HERE WITH ALL OF YOU, TO BE THINKING THROUGH THIS WITH YOU TODAY AND TOMORROW. THANK YOU. >> OKAY. I THINK I CAN GO LAST THEN AND I'M NOT ABLE TO TURN MY VIDEO ON. I'M GOING TO SHARE MY SCREEN FOR ONE SECOND. LET'S SEE. I'LL GO THROUGH IMPRESSIONS FROM THE DAY. WE'RE ALL REELING FROM THE EXTENT OF THE DEPTH OF WHAT WE DISCUSSED TODAY. I'M JUSTIN BAKER, ASSISTANT PROFESSOR OF PSYCHIATRY AT HARVARD MEDICAL SCHOOL, SCIENTIFIC DIRECTOR FOR INSTITUTE FOR PSYCHIATRY AT McLAIN HOSPITAL. SOME THEMES APPLIED TO OUR OWN WORK, THINKING ABOUT HOW DO WE ASSESS MENTAL HEALTH, GOING BEYOND THE TYPICAL CASE CONTROL MODELS THAT WE'VE HAD TO WRESTLE WITH IN THE PAST, GIVEN THE COMPLEXITY OF COMORBIDITIES, AND ALSO AS WE TALK ABOUT TODAY COMPLEXITY OF LOOKING AT VARIANTS, VARIATION, THAT WE SEE AT THE LEVEL OF PATIENTS. OR ORGANISMS. AND TRYING TO MAKE SENSE OF THE CAUSAL CHAIN OF EVENTS THAT GOES DOWN TO GENETICS AND OTHER INTERMEDIATE SCALES. ON TOP OF THIS, WHICH IS USUALLY SOMETHING WE INFER, AS A FIELD ACROSS MANY THOUSANDS OF INDIVIDUALS, WE USUALLY DON'T DO IT FOR INDIVIDUALS, AND YET WITH HUMANS AS WELL AS OTHER ORGANISMS WE HAVE TO THINK ABOUT MENTAL HEALTH EVOLVING OVER THE LIFESPAN, AS WELL AS OVER PERIODS OF DAYS TO MONTHS TO WEEKS, AS INDIVIDUALS CHANGE THEIR CLINICAL STATE. THESE ADD TO THE COMPLEXITY, WHAT IS IT WE'RE TRYING TO SOLVE FOR AND UNDERSTAND THE BRAIN'S BASIS FOR BEHAVIOR. AND ONE OF THE ISSUES WE'VE TRIED TO WRESTLE WITH WHEN YOU HEARD FROM JEFF COHN ABOUT ENCOUNTERS EVALUATE MENTAL HEALTH, NOT A ABOUT SYSTEM FOR AN INDIVIDUAL'S MENTAL STATE. LIKE JEFF'S GROUP WE'VE BEEN WORKING ON OTHER METHODS TO CAPTURE THIS INFORMATION FROM VIDEO DATA. ULTIMATELY, THE WAY WE ASSESS MENTAL HEALTH TODAY, OPEN LOOP. WE TALK ABOUT VARIOUS KINDS OF CLOSED LOOP WAYS TO BEGIN TO ASSESS MENTAL HEALTH. I WANT TO COME BACK TO THE NOTION OF THE CONTROL SYSTEM WHICH WE HEARD ABOUT TODAY, SEVERAL OF THE SESSIONS, THAT AS WE TRY TO UNDERSTAND ANY SYSTEM WHERE THERE'S A LATENT CONSTRUCT, WE'LL HAVE SOME SENSOR NETWORKS PROVIDING RAW INFORMATION, NOT JUST THE RAW SIGNALS, IT'S RELEVANT FEATURES THAT GET EXTRACTED FROM THOSE RAW SIGNALS, AND WE HEARD ALSO ABOUT THE IDEA THAT BEHAVIORS, IN MANY CASES, ARE THERE TO REORIENT AND GET ADDITIONAL INFORMATION, NOT THE ONLY REASON, BEN AND OTHERS MENTIONED BEHAVIORS ARE NECESSARY FOR SURVIVAL BUT IT'S VERY IMPORTANT WE THINK ABOUT OUR SENSORY SYSTEMS AND OUR MOTOR SYSTEMS AS ACTING WITHIN THESE CLOSED LOOP ENVIRONMENTS. AND WHAT I LIKE TO POINT OUT TO TECHNICAL AUDIENCE AS WELL AS TO ENGINEERING AUDIENCES, FUNDAMENTALLY THE BRAIN IS A BUNCH OF THESE CLOSED LOOP SYSTEMS TRYING TO COMPUTE LATENT INFORMATION ABOUT THE WORLD TO SOLVE THOSE SURVIVAL PROBLEMS. SO IT'S VERY MUCH RELEVANT TO THE WAY NEUROSCIENTISTS MIGHT THINK ABOUT THE WORLD, BUT ALSO QUITE RELEVANT TO HOW CLINICIANS THINK ABOUT THE WORLD, EVERY TIME THE CLINICIAN NEEDS TO INTERVIEW A PATIENT THEY HAVE TO USE THEIR OWN SENSORY INFORMATION, HONED OR EVOLVED THROUGH TRAINING, TO EXTRACT RELEVANT INFORMATION FROM THEIR PATIENTS. BUT IT'S REALLY NOT USED IN THE SYSTEMATIC WAY BUT THE SAME CONCEPT IS THERE THAT CLINICIANS ARE CONSTANTLY ITERATING TO SOLUTIONS. WE TRY TO USE THAT SAME APPROACH TO PUT SENSORS ON PATIENTS, AND TO COMBINE THOSE MUCH HIGHER TENSELY ASSESSMENTS LIKE VIDEO, WITH THE LOWER DENSITY LIKE WEARABLES, TO SEE WHETHER SIGNALS OF CLINICAL RELEVANCE ARE PRESENT IN EACH OF THOSE KINDS OF SOURCES OF DATA. JUST TO MAKE THIS MORE CONCRETE, MANY KINDS OF RAW DATA WE CAN PULL FROM IN TODAY'S ENVIRONMENTS FROM SELF-REPORTS AND SURVEYS THROUGH TO ELECTRONIC HEALTH RECORDS, AND INCREASINGLY ALL THOSE OTHER TYPES OF DATA WE HEARD FROM SEVERAL PRESENTERS ON TODAY. AND WE HEARD AGAIN WAYS TO PULL DATA FROM COMPUTER VISION, DIDN'T HEAR AS MUCH ABOUT MRI. WE CAN TRY TO COMBINE ALL OF THE RELEVANT FEATURES BUT IT'S ABOUT INFORMING THE LATENT CONSTRUCTS THAT ARE RELEVANT FOR PATIENTS IF WE'RE TRYING TO WORK ON CLINICAL APPLICATIONS. AND SO TO TAKE IT EVEN FURTHER OUT WE THINK OF THIS AS WHERE MENTAL HEALTH ASSESSMENT WILL GO, WHICH IS THAT WE'LL HAVE THESE SYSTEMS OF COMPREHENSIVE PHENOTYPING THAT TRY TO LINK TOGETHER, YOU KNOW, DIFFERENT DATA SOURCES BUT ULTIMATELY TRY TO DO THAT THROUGH RELEVANT LATENT CONSTRUCTS THAT WE KNOW AREN'T GOING TO HAVE THE EXACT SAME MAPPING IN EVERY SINGLE PATIENT THAT WE ENCOUNTER, BUT THAT BY UNDERSTANDING WHERE THOSE MAPPINGS ARE MORE RELIABLE WE CAN BEGIN TO HAVE BETTER SYSTEMS THAT ARE ABLE TO WORK WELL BOTH AT THE INDIVIDUAL LEVEL AND APPLIED TO GROUPS. I WON'T GO INTO ANY OF THIS TODAY BUT I THINK WE'VE HEARD A LITTLE BIT ABOUT WAYS WE CAN BEGIN DESIGNING CONSUMER EXPERIENCES, TO ACTUALLY LEVERAGE CHANGES IN THE SIGNALS, TO PROVIDE BETTER CARE SYSTEMS. AND ALSO BE ABLE TO ANTICIPATE CHANGES IN CARE CAPACITY RELEVANT. AND SO I WANTED TO BRIEFLY MENTION A STUDY THAT WE'VE BEEN DOING, SUPPORT FROM NIMH, PART OF WHAT'S CALLED INTENSIVE LONGITUDINAL NETWORK. DANA SCHLOSSER WAS INVOLVED, IT SPANNED MULTIPLE OF THE NIH INSTITUTES FROM NATIONAL CANCER INSTITUTE AS WELL AS SOME NIAAA TO TRY TO USE THESE MULTI-MODAL MULTI-LEVEL ASSESSMENTS TO FOLLOW INDIVIDUALS WITH A WHOLE RANGE OF DIFFERENT HEALTH CONDITIONS, OVER LONGER PERIODS OF TIME COULD BEGIN SEEING COMMON ELEMENTS FROM PATHOLOGY AS WELL AS THINGS THAT ARE SPECIFIC TO THE CONDITION BEING STUDIED. WHAT OUR GROUP HAS BEEN FOCUSED ON IS FOLLOWING 100 INDIVIDUALS FOR AT LEAST ONE YEAR, WHO ARE LIKELY TO EXPERIENCE DEPRESSION, BIPOLAR DISORDER, PSYCHOTIC CONDITION, WE PUT INTO REGISTER ALL THEIR SELF-REPORTS AS WELL AS CLINICAL EVALUATIONS AND OTHER OBJECTIVE MEASURES. AND I THINK JUST TO HIGHLIGHT THAT THESE ARE THE KINDS OF DATA THAT WE CAN BEGIN TO COLLECT AT SCALE, WHERE WE CAN THINK OF A SINGLE PIXEL CAMERA OR THREE PIXEL CAMERA ON THE WRIST, WRIST WEARABLE, WITH MORE PERIODIC ENCOUNTERS OF A VIDEO, AND TO REALLY TRY TO FIND INDIVIDUALS WHO ARE GOING THROUGH SIGNIFICANT CHANGE IN THEIR CLINICAL STATUS TO UNDERSTAND HOW THE SIGNALS ARE CHANGING IN MEANINGFUL WAYS. AGAIN, JUST TO HIGHLIGHT THAT THE CONCEPT OF THIS NETWORK WAS TO ENCOURAGE PROJECTS TO NOT JUST FOCUS THE CONSTRUCT THEY WERE MOST INTERESTED IN BUT ALSO TO FOCUS IN ON ALL SORTS OF BEHAVIORS THAT MIGHT BE COMMON ACROSS THE HUMANS IN THE STUDY AND FROM HEARING THE TALKS TODAY THE DIFFERENT SESSIONS WE CAN BEGIN TO THINK ABOUT WAYS TO MOTIVATE SIMILAR STUDIES THAT ARE ACROSS SPECIES NATURE, EVEN IF YOU'RE NOT NECESSARILY INTERESTED IN, YOU KNOW, A GROOMING BEHAVIOR, YOU COULD COLLECT ENOUGH HIGH DENSITY DATA OF OTHER ADJACENT BEHAVIORS AROUND OUR CONSTRUCT OF INTEREST TO INFORM A MORE DATA-DRIVEN ASSESSMENT IN LINE WITH OTHER STUDIES COLLECTING SIMILARLY GRANULAR DATA. ONE QUICK EXAMPLE OF THE KIND MUCH DATA WE CAN GET FROM A SINGLE PARTICIPANTS. THIS IS SOMEONE WE FOLLOWED FOR OVER TWO YEARS. AS YOU HEARD FROM OTHER TALKS TODAY WE CAN ASK QUESTIONS ON THE SMARTPHONE, THIS INDIVIDUAL EXPERIENCED PROFOUND LONG PERIODS OF POSITIVE AFFECT FOLLOWED BY PROFOUND PERIODS OF NEGATIVE AFFECT, AGAIN POSITIVE AFFECT, YOU CAN SEE THAT THERE'S A RHYTHMICITY, FAIRLY DENSE PERIODS, EVEN THOUGH THE PERIODS WERE -- SHE'S EXPERIENCE POSITIVE AND NEGATIVE AFFECT CHANGE, THEY ARE QUITE DENSE. AND I THINK AS WE WOULD START TO SEE THIS -- IT'S NOT GROUND TRUTH BUT IT'S A STRONG INDICATION OF WHAT STATE SHE'S IS, WE CAN THEN ADD OTHER CLINICAL MEASURES SUCH AS THE VERY LOW DENSITY ASSESSMENTS WE WOULD DO FROM CLINICAL ASSESSMENTS EVERY MONTH TO SHOW HOW THEY ARE MORE OR LESS CORRELATED WITH THE SELF-REPORTS BUT OBVIOUSLY DELAYED AND MUCH LESS TEMPORALLY DENSE AND COMBINE THOSE WITH OTHER VERY DENSE ASSESSMENTS FROM A WEARABLE, AND TODAY I DON'T WANT TO GET INTO TOO MUCH OF THE SCIENCE HERE BUT JUST TO POINT OUT THAT WE DO START TO SEE REALLY STRIKING CHANGES AS WAS BROUGHT UP IN ONE OF THE EARLY COMMENTS ABOUT WHEN SOMEONE DOES CHANGE FROM DEPRESSION TO NOT BEING DEPRESSED, WHAT'S THE SEQUENCE, DO THEY FIRST FEEL BETTER AND LOOK BETTER? HERE WE SEE STRIKING CHANGES IN THE SLEEP PATTERN, PATTERN OF HOW THEY ARE SLEEPING, ALSO THE LEVEL OF ACTIVITY DURING THE DAY, WHICH ROUGHLY SPEAKING DOES CHANGE IN ACCORDANCE WITH THOSE OTHER SUBJECTIVE STATES, SO WE ALREADY SEE THAT IN THESE EXAMPLES BY FOLLOWING PEOPLE OVER LONG PERIODS OF TIME WE BEGIN TO CAPTURE THAT NATURALLY OCCURRING PHENOMENONOLOGY IN THE LABORATORY, USUALLY ONLY IN A SINGLE TIME POINT. WE ONLY CAN FOLLOW AN INDIVIDUAL OR GROUP OF INDIVIDUALS TO RESOLVE SOME RELATIONSHIPS AND BEGIN TO HAVE MUCH STRONGER HIGH POTENTIAL APPROACHES THESE FOR WHAT CAUSAL SYSTEMS ARE UNDERLYING. WE HAVE DEVELOPED AN ENERGY SORT OF ALLOSTASIS-HOMEOSTASIS MODEEL, TESTABLE IN LARGER POPULATIONS. AND SO I'LL JUST END MY COMMENTS BY SAYING A LOT OF THIS WORK IS REALLY GEARED TOWARDS MOVING BEYOND CARICATURE VIEWS OF WHAT MENTAL ILLNESS IS OR UNDERLYING BEHAVIORS, TO BE ABLE TO CONNECT SUBJECTIVE EXPERIENCES WITH THINGS THAT ARE OBJECTIVELY MEASURABLE. AND SO I THINK WE'RE CERTAINLY DOING THIS AND TRYING TO HELP BUILD TOOLS TO DO THIS, TO BE ABLE TO COLLECT LARGER DATASETS, TO SCALE THEM INTO THE THOUSANDS OF INDIVIDUALS. I WILL JUST ALSO END BY NOTING, WE'RE SOMETIMES ASKED WHETHER EVENTUALLY PSYCHIATRY WILL BE REPLACED BY ARTIFICIAL INTELLIGENCE. PART OF A DEBATE A COUPLE YEARS AGO I WAS COMING DOWN AGAINST THE NOTION BUT WORTH THINKING ABOUT WHAT ARE THE TOOLS THAT WILL EVENTUALLY REPLACE WHAT A HUMAN IN THE LOOP IS DOING AND WHAT ARE THE ROLES THAT ARE GOING TO CONTINUES TO BE PERFORMED BY HUMANS. AND I THINK OF RELEVANCE TO SOME OF THE OTHER ETHICAL CONSIDERATIONS WE TALKED ABOUT TODAY WHICH IS HOW DO WE KNOW A PARTICULAR SET OF TOOLS IS ACTUALLY READY FOR A PARTICULAR USE, AND WHO GETS TO DECIDE? IS THIS SIMPLY A QUESTION OF THE MARKETPLACE WHERE PEOPLE ARE WILLING TO SPEND THEIR MONEY? IS IT THE GOVERNMENT REGULATIONS WILL DETERMINE THIS? OR PHYSICIANS WHO HAVE TO DEAL WITH COMPLEX CHALLENGING ISSUES WILL TURN TO WHATEVER SOLUTIONS ARE OUT THERE, EVEN WHETHER THE EVIDENCE IS STRONG OR NOT. AND SO ALSO JUST TO MENTION THAT TIMOTHY BROWN'S QUESTION IN THE LAST SESSION, WE TAKE THE ETHICS OF THE WORK SERIOUSLY BECAUSE I THINK THERE IS THIS QUESTION OF WHETHER WE CAN MOVE BEYOND THE BENCH WITH CAUTION BUT I THINK IT'S ALSO THE CASE THAT MANY OF THESE ANALYTICAL TECHNIQUES AND TECHNOLOGIES ARE NOW AVAILABLE AND SO I THINK THE PATIENTS WHO ULTIMATELY ARE EXPERIENCING THIS CONDITION DESERVE APPLICATION, AS SOON AS IT'S POSSIBLE, AS SOON AS IT'S NOT GOING TO BE UNSAFE FOR THEM TO DO SO. SO AS PART OF OUR WORK WE'RE TRYING TO MOVE AS AGGRESSIVELY TOWARDS PATIENT APPLICATIONS AS POSSIBLE SO WE RECENTLY PUBLISHED AND ETHICS CHECK LIST IN DIGITAL HEALTH GIVEN COMPLEXITIES ABOUT ACHIEVING INFORMED CONSENT IN MULTI-MODAL LONGITUDINAL STUDIES AND ENSURE EQUITY, DIVERSE, ACCESS, PRIVACY, REGULATIONS, AND IMPORTANTLY RETURN OF RESULTS. WITH THAT I WILL WRAP UP MY COMMENTS AND NOTE AGAIN JUST WANT TO THANK THE FOLKS FOR INVITING ME TO PARTICIPATE AND ALL OF THE AMAZING SPEAKERS THAT WE HAD TODAY. APPRECIATE IT. >> THANK YOU SO MUCH, JO TIN, -- JUSTIN AND KAREN. I APPRECIATE YOU ENDING US. OF ON THIS TALK. I KNOW THE DISCUSSION TIME SEEMED COMPACT, TALKS WERE SHORT, HIGH LEVEL CONCEPTUAL TALKS. AND I HOPE THIS WHETS YOUR APPETITE FOR THE DISCUSSIONS THAT ALL OF YOU PARTICIPANTS ARE GOING TO BE TAKING PART IN TOMORROW. SO TOMORROW WILL BE THE THREE MORE HIGH LEVEL TALKS FOLLOWED BY TWO CONCURRENT BREAKOUT SESSIONS WHETHER WE'LL GET TO DIG INTO MORE OF THESE THOUGHTS AND IDEAS WHAT WERE BROUGHT UP, AND THEN WE'LL FOLLOW UP WITH A SUMMARY AND CONCLUSION. WITH THAT, I WILL CLOSE THE MEETING. THANK YOU, EVERYONE, FOR STICKING IT OUT FOR THIS LONG.