>>I WANT TO SAY THANK YOU TO
EVERYBODY THAT HAS JOINED TODAY
FOR THIS VERY -- WHAT I THINK IS
GOING TO BE VERY EXCITING AND
FRUITFUL WORKSHOP THAT BRAIN
BEHAVIOR QUANTIFICATION AND
SYNCHRONIZATION WORKSHOP.
IT WILL BE DIFFERENT, VERY HEAVY
ON DISCUSSION, SO WE'RE REALLY
EXCITED ABOUT HAVING EVERYBODY
GIVE THEIR QUESTIONS AND HAVING
A VERY FRUITFUL DISCUSSION.
WITH THAT I'M GOING TO TURN IT
OVER TO OUR FIRST SPEAKER, DR.
HOLLY LISANBY.
>> WELCOME, EVERYONE, TO THE
BRAIN INITIATIVE SUPPORTED
WORKSHOP ON BRAIN BEHAVIOR
QUANTIFICATION AND
SYNCHRONIZATION.
THANK YOU FOR JOINING US FOR
WHAT WILL BE AN EXCITING AND FUN
NEXT TWO DAYS.
SO, MY ROLE IS TO SET THE STAGE
AND DESCRIBE THE GOALS OF THE
WORKSHOP.
WHY IS BRAIN INITIATIVE
SUPPORTING A WORKSHOP ON BRAIN
BEHAVIOR QUANTIFICATION AND
SYNCHRONIZATION?
THE RATIONALE IS OUTLINED HERE.
BASICALLY BEHAVIOR IS THE
PRIMARY OUTPUT OF THE BRAIN.
SO, TO REALLY UNDERSTAND THE
NEURAL ORIGINS OF BEHAVIOR IT'S
ESSENTIAL TO GOAL OF USING
SCIENCE TO SUPPORT BRAIN HEALTH.
BUT THIS IS A COMPLEX CHALLENGE
BECAUSE UNDERSTANDING BEHAVIOR
REQUIRES A DETAILED AND
MULTI-DIMENSIONAL ANALYSIS OF A
BROAD RANGE OF BEHAVIORS IN THE
CONTEXT OF ENVIRONMENT.
AND INTERACTION WITH THE
ENVIRONMENT.
NOW, OUR TOOLS FOR QUANTIFYING
THE BRAIN SIDE OF THE EQUATION
ARE QUITE ADVANCED AND ARE
GETTING MORE ADVANCED THROUGH
BRAIN INITIATIVE INVESTMENTS.
WE'VE GOT GREAT TOOLS NOW FOR
QUANTIFYING NEURAL ACTIVITY WITH
VERY HIGH TEMPORAL AND SPATIAL
RESOLUTION.
BUT ON THE BEHAVIOR SIDE,
TYPICALLY WE'RE MEASURING
BEHAVIOR AT LOWER RESOLUTION.
AND THIS CAN MAKE THE DISCOVERY
OF THE CAUSAL LINKAGES BETWEEN
BRAIN AND BEHAVIOR CHALLENGING.
INDEED, THE TOOLS FOR MEASURING
THE FULL RICHNESS OF
SPECIES-APPROPRIATE BEHAVIORS IN
ENVIRONMENTS AND SYNCHRONIZING
THESE WITH SIMULTANEOUSLY
RECORDED NEURAL ACTIVITY ARE
PRESENTLY LACKING.
THIS RESEARCH GAP WAS NOT LOST
ON THE BRAIN 2.0 RECORD WHICH
SPECIFICALLY CALLED FOR MORE
SOPHISTICATED METHODS.
THUS OUR TOPIC FOR TODAY'S
WORKSHOP, BRAIN BEHAVIOR
QUANTIFICATION AND SINK
SYNCHRONIZATION.
I'M GOING TO MAP OUT A
CONCEPTUAL TERRAIN, IN BLACK
FONT WHAT WE HAVE BUT COULD BE
IMPROVED, PURPLE, WHAT WE DON'T
HAVE, RESEARCH GAPS IN RED.
WE HAVE HIGH RESOLUTION TOOLS.
IN THE LABORATORY SETTING WE
HAVE TOOLS FOR STUDYING BEHAVIOR
BUT WE'RE STUDYING BEHAVIORS NOT
NATURALISTIC AND THE
MEASUREMENTS OFTEN FAIL TO
CAPTURE THE FULL RANGE OF
BEHAVIOR AND OFTEN FAIL TO
CAPTURE THE ENVIRONMENT THE
SUBJECT'S INTERACTING WITH.
WE HAVE TOOLS CURRENTLY FOR
LINKING NEURAL ACTIVITY RECORDED
SIMULTANEOUSLY WITH BEHAVIOR IN
THE LAB, BUT AS YOU'LL SEE
TYPICALLY THESE TOOLS REQUIRE
BULKY EQUIPMENT, LIMITS MOVEMENT
OF SUBJECT, AND WHEN WE'RE
TRYING TO MODEL THESE DATA THAT
ARE MISMATCHED IN TERMS OF
TEMPORAL RESOLUTION BETWEEN THE
NEURAL SIDE AND BEHAVIOR SIDE,
THIS POSES CHALLENGES FOR
MODELING AS WELL.
NOW, WE DO ALREADY HAVE TOOLS
THAT ALLOW US TO TAKE
EXPERIMENTS INTO NATURALISTIC
SETTINGS AND CAPTURE BEHAVIORS
THAT ARE ECOLOGICALLY VALID IN
THE REAL WORLD BUT CURRENT TOOLS
TYPICALLY HAVE ONLY A FEW
CHANNELS OF RECORDING AND LIMITS
IN TERMS OF RANGE OF MODALITIES
THAT CAN BE SIMULTANEOUSLY
SAMPLED.
NOW, IN RED HERE IS A BIG
RESEARCH GAP.
WE'RE PRESENTLY LACKING
PRACTICAL SYSTEMS THAT ALLOW US
TO BRING NEURAL CIRCUIT
RECORDING INTO THE REAL WORLD
SETTING SO WE CAN CAPTURE WHAT
THE BRAIN IS DOING WHEN IT'S
GENERATING THESE BEHAVIORS IN
THE REAL WORLD.
NOW, I'VE ILLUSTRATED HERE AS A
CLOUD THIS THING THAT'S
DIFFICULT TO GET A HANDLE ON,
INTERNAL SUBJECTIVE STATES.
WE CURRENTLY LACK DIRECT
OBJECTIVES MEASURES OF THESE AND
OFTEN ARE LIMITED TO SELF-REPORT
OF THE SUBJECT TELLING US WHAT
THEY ARE EXPERIENCING OR PROXY
BEHAVIORS WE THINK ARE RELATED
TO INTERNAL STATE.
AN IMPORTANT GAP BECAUSE MANY
INTERNAL STATES ARE TOPICS OF
SPECIFIC CLINICAL INTEREST, SUCH
AS INTENTION, CRAVING, MOOD,
THOUGHT, HUNGER, PERCEPTION,
PAIN, REWARD, SUICIDAL INTENT,
THINGS WE WOULD LIKE TO BE ABLE
TO OBJECTIVELY STUDY, AND IN
ABSENCE OF OBJECTIVES MEASURES
OF THESE WE'RE CURRENTLY LACKING
RELIABLE WAY OF TRAINING
DECODERS THAT COULD USE NEURAL
ACTIVITY TO INFORM US ABOUT
THESE INTERNAL STATES.
AND WITHOUT THESE, THIS HAS BEEN
AN IMPEDIMENT FOR DEVELOPMENT OF
INTERVENTIONS AS WELL.
HERE'S SOME EXAMPLES OF
STATE-OF-THE-ART TECHNOLOGIES
THAT ARE CURRENTLY BEING USED IN
RESEARCH.
THIS, FOR EXAMPLE, IS A WEARABLE
RECORDING OF IEEG IN SUBJECTS
NAVIGATING A SPACE, AND YOU CAN
SEE THAT AS THE SUBJECT IS
WALKING AROUND THIS ROOM, THE
CAMERAS ON THE CEILING AND
TRACKER ON HIS HEAD IS ABLE TO
TRACK WHERE IN SPACE HE'S
LOCATED, AND TO DO THIS
SIMULTANEOUSLY WITH BRAIN
RECORDINGS ON THE RIGHT PART OF
THE SLIDE.
THIS SETUP WAS ABLE TO
DEMONSTRATE WHEN THE SUBJECT
HIMSELF WAS NAVIGATING THE ROOM,
A SPECIFIC NEURAL SIGNATURE IS
SEEN WHICH IS DIFFERENT THAN
WHEN HE'S WATCHING SOMEONE ELSE
NAVIGATE, ILLUSTRATING
IMPORTANCE OF ENVIRONMENT,
WHAT'S GOING ON AROUND THE
SUBJECT.
AS POWERFUL AS THIS TOOL IS,
IT'S STILL LACKING IN SOME WAYS.
IT'S ONLY MEASURING ONE ASPECT
OF BEHAVIORAL RESPONSE, HAS
LIMITED CAPTURE OF ENVIRONMENT
AND SENSORY CUES, LIMITED TO THE
LABORATORY ENVIRONMENT, AND LOOK
AT THE BACKPACK HE'S WEARING.
IT'S BULKY EQUIPMENT, AND
HEAD-MOUNTED GEAR, IT ONLY
TRACKS WHERE HE IS IN SPACE, NOT
HIS ACTUAL BODY MOVEMENTS OR
POSTURES.
WE ALREADY HAVE SOFTWARE
PLATFORMS SUCH AS DANCE
ILLUSTRATED HERE WHICH ARE ABLE
TO USE DEEP LEARNING TO DERIVE
MEASURES OF NATURALISTIC
BEHAVIOR REPERTOIRE WITHOUT
HAVING TO PUT MARKERS ON THE
SUBJECT.
THIS IS A VERY EXCITING
OPPORTUNITY TO DISCOVER THE
NEURAL BASIS OF THESE
NATURALISTIC BEHAVIORS, BUT
STILL IT'S LACKING THE FULL
RANGE OF BEHAVIORAL RESPONSE AND
LACKS CAPTURE OF THE ENVIRONMENT
AND SENSORY CUES THAT MAY
INFLUENCE BEHAVIOR BEING
MEASURED.
NOW, LET'S GO BACK TO THE
CONCEPTUAL MAP, WHAT WOULD IT
MEAN FOR NEUROSCIENCE IF WE IS
COULD HAVE SENSORS TO CAPTURE
FULL RANGE OF BEHAVIOR IN THE
ENVIRONMENT AND IN THE LAB, AND
LINK THESE AT THE SAME TEMPORAL
RESOLUTION WITH SIMULTANEOUSLY
RECORDED NEURAL ACTIVITY AND
DEPLOY NEXT GENERATION SENSORS
IN REAL WORLD SETTINGS TO RECORD
NATURALISTIC BEHAVIORS AND
ENVIRONMENTS?
WHAT IF WE COULD DEVELOP
PORTABLE TOOLS FOR NEURAL
RECORDING, IN REAL TIME, IN
AMBULATORY SETTINGS TO LINK
THESE NATURALISTICALLY RECORDED
BEHAVIORS WITH WHAT THE BRAIN IS
DOING AT THE TIME THAT IT'S
GENERATING THOSE BEHAVIORS IN
THOSE NATURAL ENVIRONMENTS.
WE MIGHT HAVE THE POTENTIAL THEN
TO USE THESE HIGH DIMENSIONALITY
DATA STREAMS TO HAVE MORE
ACCURACY IN OUR ABILITY TO INFER
INTERNAL STATES FROM THESE
QUANTIFIED BEHAVIORS.
IF THIS COULD GIVE US A MORE
OBJECTIVE MEANS OF INFERRING
INTERNAL STATES OF INTEREST,
LIKE MOOD DISORDER OR SUICIDAL
INTENT, THIS THEN GIVES US THE
OPPORTUNITY TO TAG NEURONAL
ACTIVITY SIMULTANEOUSLY RECORDED
DURING INTERNAL STATES OF
INTEREST TO BEGIN TO BE ABLE TO
DECODE THESE INTERNAL STATES
FROM THE NEURAL ACTIVITY.
IF WE HAD THAT, THEN WE MIGHT BE
ABLE TO BUILD CLOSED LOOP
SYSTEMS THAT WOULD BE ABLE TO
MODULATE NEURAL ACTIVITY WITH
BEHAVIOR ACTUALLY IN THE LOOP.
NOW, WHY DO WE THINK THE TIME IS
RIGHT TO HAVE THIS DISCUSSION?
WE'RE CURRENTLY EXPERIENCING
EXPLOSION OF NEW AND ADVANCED
STATIONARY AND WEARABLE
TECHNOLOGIES, PLATFORMS TO
INTEGRATE MULTI-MODAL DATA, TO
USE ADVANCED TOOLS OF DATA
SCIENCE TO MAKE SENSE OF THESE
DATA, AND TO PACKAGE THIS IN A
WAY THAT ALLOWS DATA
VISUALIZATION AND ANALYSIS.
SO THERE'S A LOT OF NEW
DEVELOPMENTS THAT HAVE NOT YET
BEEN EXPLOITED FOR NEUROSCIENCE
APPLICATIONS.
THIS GIVES POTENTIAL TO USE NEXT
GENERATION SENSORS SUCH AS FOR
SELF-DRIVING CARS TO HELP US
MEET UNMET NEEDS IN BRAIN
HEALTH.
NOW, TALKING ABOUT DATA, WE KNOW
THAT INTERNAL STATES CAN
GENERATE A KALEIDOSCOPE OF
DIFFERENT TYPES OF BEHAVIORS
THAT CAN BE MEASURED USING A
VARIETY OF ARRAYS OF SENSORS
LIKE VIDEOGRAPHY, THERMAL
SENSORS, PIEZOSENSORS,
GENERATING HIGH DIMENSIONALITY
DATA, AN OPPORTUNITY FOR DATA
DRIVEN MODELING, THEORY BUILDING
AND TESTING.
THIS APPROACH COULD BE
SYNERGISTIC WITH THE BRAIN
INITIATIVE'S INVESTMENTS IN
INFORMATICS, IN THE DEVELOPMENT
OF NEW DATA ARCHIVES, DATA
STANDARDS, DATA INTEGRATION
APPROACHES, SOFTWARE
DEVELOPMENT, SECONDARY DATA
ANALYSIS.
SPEAKING OF INFORMATICS, WE HAVE
AN OPPORTUNITY TO LEVERAGE
EXISTING MULTIMODAL DATA
ARCHIVES SUCH AS NEURODATA
WITHOUT BORDERS, AND THE
DISTRIBUTOR ARCHIVES FOR NERVE
PHYSIOLOGY DATA INTEGRATION
WHICH ALREADY ACCEPTS BEHAVIORAL
DATA STREAMS.
WE HAVE THE OPPORTUNITY TO BUILD
ON ALREADY EXISTING DATA
STANDARDS THAT BRAIN INITIATIVE
HAS INVESTED IN FOR CATALOGING
BEHAVIORAL DATA AND
STANDARDIZING THE WAIT WE DEFINE
BEHAVIORAL TASKS, SUCH AS
BEHAVIORAL TASKS DESCRIPTION
LANGUAGE.
AND IT'S AN OPPORTUNITY TO
EXPAND ON BRAIN INFORMATICS
PORTFOLIO BY DEVELOPING DATA
ARCHIVES STANDARDS AND SOFTWARE
TOOLS THAT ARE CURRENTLY
LACKING, SPECIFICALLY FOR
CROSS-MODAL DATA FUSION, THAT
INCORPORATION INFORMATION ABOUT
THE ENVIRONMENT AS WELL.
AND WE HAVE AN OPPORTUNITY TO
GENERATE A COMMON DATA ECOSYSTEM
ACROSS THESE DATA STREAMS WITH
PATHWAYS FOR DISSEMINATION.
SO HERE'S AN EXAMPLE OF PROOF OF
CONCEPT OF WHAT A BRAIN BEHAVIOR
QUANTIFICATION SYNCHRONIZATION
PROJECT MIGHT LOOK LIKE.
WE'LL HEAR FROM DR. PROVENZA
LATER IN THE WORKSHOP.
THE GOAL WAS TO TRAIN AN
ADAPTIVE DEEP BRAIN STIMULATION
SYSTEM FOR TREATMENT OF
OBSESSIVE COMPULSIVE DISORDER,
STARTING WITH SIMULTANEOUS
NEURAL RECORDINGS WHILE
DIFFERENT BEHAVIORS ARE BEING
SENSED BOTH IN THE CLINIC AS
WELL AS AT HOME, AND TO BE ABLE
TO HAVE THIS HIGH TEMPORAL
RESOLUTION LINKAGE BETWEEN WHAT
THE BRAIN IS DOING WHEN IT'S
GENERATING THE BEHAVIORS IN
THESE DIFFERENT CONTEXTS.
AND TO REALLY PUT BEHAVIOR IN
THE LOOP TO TRAIN THIS ADAPTIVE
DBS SYSTEM, HERE'S THE EXAMPLE
OF DATA STREAMS THAT CAN BE
CAPTURED WITH THIS APPROACH.
WE'VE GOT COMPUTER VISION TO
CAPTURE MICROMOVEMENTS OF THE
FACE, FOR AFFECT, WE'VE GOT
AUTONOMIC RECORDING, EEG, YOU
CAN AMPLITUDE AND SEE HOW
BEHAVIORS RESPOND.
WITH THIS ENRICHED MULTIMODAL
DATA STREAM IT GIVES US THE
OPPORTUNITY TO BEGIN TO INFER
INTERNAL STATES FROM THIS
QUANTIFIED BEHAVIOR AND USE THAT
INFORMATION TO TRAIN ADAPTED DBS
SYSTEMS THAT WOULD BE ABLE TO
MODULATE STATE AND DISRUPT THE
MALADAPTIVE BEHAVIORS WHEN AND
WHERE THEY OCCUR.
THIS BRINGS US TO THE CONCEPT OF
BRAIN BEHAVIOR QUANTIFICATION
AND SYNCHRONIZATION.
WE PUBLISHED THIS CONCEPT, THE
GOALS ARE TO DEVELOP HIGH
RESOLUTION TOOLS AND PLATFORMS,
TO PRECISELY QUANTIFY BEHAVIOR,
TO SYNCHRONIZE WITH
SIMULTANEOUSLY RECORDED BRAIN
ACTIVITY.
ANOTHER GOAL IS TO USE THIS DATA
AND THESE TOOLS TO BUILD NEW
CONCEPTUAL AND COMPUTATIONAL
MODELS OF BEHAVIORAL SYSTEMS,
AND USE THOSE MODELS TO TEST
CAUSAL RELATIONSHIPS IN BRAIN
AND BEHAVIOR.
ANOTHER GOAL TO ESTABLISH
CROSS-DISCIPLINARY CONSORTIUM TO
DEVELOP AND DISSEMINATE TOOLS,
ONTOLOGIES, RESEARCH DESIGNS,
IMPORTANTLY ETHICAL FRAMEWORKS
THAT COULD HAVE THE POTENTIAL OF
TRANSFORMING HOW MECHANISTIC
BRAIN BEHAVIOR RESEARCH IS
CONDUCTED AND ESTABLISHMENT OF
CONSORTIA IS PART OF THE GOAL
FOR OUR WORKSHOP TODAY.
OUR ENVISION IMPACT OF THIS
CONCEPT COULD BE THE DIGS
DISSEMINATION, HOW WE UNDERSTAND
HOW THE BRAIN GENERATES
BEHAVIOR, DEVELOPMENT OF NEW
PARADIGMS TO ESTABLISH CAUSAL
RELATIONSHIPS BETWEEN NEURAL
ACTIVITY AND BEHAVIOR,
ULTIMATELY THIS COULD ENABLE THE
DEVELOPMENT OF THERAPEUTIC
INTERVENTIONS FOR COMPLEX NEURAL
BEHAVIORAL DISORDERS THAT
CURRENTLY LACK TREATMENTS, AND
WE WOULD LIKE TO SEE A ROBUST
ETHICAL FRAMEWORK THAT ADDRESSES
THE IMPLICATIONS OF USING THESE
TOOLS TO CAPTURE AND RECORD A
PERSON'S EXPERIENCE.
NOW, THERE ARE TWO CASE EXAMPLES
I'M GOING TO WALK THROUGH AND
INTRODUCE THE FORMAT OF THE
WORKSHOP.
FIRST USE CASE EXAMPLE WHEN THE
BEHAVIOR IS THE TARGET OF STUDY,
AND THIS IS THE MOST OBVIOUS USE
CASE SITUATION, BUT WE MIGHT
NEED NEW TOOLS TO DISCOVER THE
NEURAL BASIS OF THE BEHAVIOR
WHICH MIGHT BE CHALLENGE BE TO
STUDY FOR A NUMBER OF REASONS,
MIGHT BE THAT THE BEHAVIOR
OCCURS IN REAL WORLD SETTING
OUTSIDE THE LAB OR COULD BE
BEHAVIOR OCCURS TOO INFREQUENTLY
AND REQUIRES AMBULATORY
MONITORING TO CAPTURE OR THE
FREQUENCY AT WHICH THE BEHAVIOR
OCCURS OVER EXTENDED PERIODS OF
TIME IS ITSELF SALIENT SUCH AS
IN THE CASE OF SEIZURES OR BINGE
EATING.
EXAMPLE OF APPLYING COMPUTER
VISION TO CAPTURE AND QUANTIFY
NATURALISTIC BEHAVIOR, IN THIS
CASE SPONTANEOUS FLY BEHAVIOR,
BUT THE CONCEPT WOULD BE TO DO
THIS ACROSS DIFFERENT SPECIES IN
DIFFERENT SETTINGS.
THE POTENTIAL IMPACT COULD BE
ADVANCING OUR UNDERSTANDING OF
HOW THE BRAIN GENERATES THESE
COMPLEX BEHAVIORS IN THE CONTEXT
OF ENVIRONMENT, PARTICULAR
COMPLEX ENVIRONMENTS, AND ALSO
ENABLING THE DEVELOPMENT OF
INTERVENTIONS WHETHER CLOSED
LOOP OR OTHERWISE, WHEN THE
BEHAVIOR ITSELF IS THE TARGETED
THERAPY.
WHAT ABOUT A DIFFERENT USE CASE
SCENARIO, IF IT'S NOT THE
BEHAVIOR THAT'S THE TARGET BUT
THE INTERNAL STATE THAT'S THE
TARGET OF STUDY?
IN THIS CASE, BEHAVIORS ARE
BEING USED AS MARKERS OF
INTERNAL SUBJECTIVE EXPERIENCE
THAT CANNOT BE MEASURED DIRECTLY
LIKE A THOUGHT PROCESS,
AFFECTIVE STATE OR PERCEPTION.
HERE THE SUBJECTIVE EXPERIENCE
FOR WHICH THE OBJECTIVELY
MEASURED BEHAVIORS SERVE AS A
MARKER IS THE INTENDED TARGET OF
THE STUDY, NOT THE BEHAVIORS
THEMSELVES.
HERE'S EXAMPLE OF USING COMPUTER
VISION AGAIN, TO QUANTIFY
DYNAMIC MICROMOVEMENTS OF THE
FACE OVER TIME AND USE THESE TO
INFER AFFECTIVE -- INTERNAL
AFFECTIVE STATE AND THIS COULD
BE LINKED WITH SIMULTANEOUS
NEURAL RECORDING AND STIMULATION
AS YOU'LL SEE FROM SOME OF OUR
SPEAKERS IN THE WORKSHOP.
THE POTENTIAL IMPACT HERE COULD
BE TO ADVANCE OUR UNDERSTANDING
OF HOW THE BRAIN GENERATES
SUBJECTIVE EXPERIENCES AND COULD
ENABLE THE DEVELOPMENT OF
INTERVENTIONS WHEN THE
SUBJECTIVE EXPERIENCE IS THE
THERAPEUTIC TARGET.
NOW, THE GOALS OF THE WORKSHOP
ARE TO CONVENE A DIVERSE PANEL
OF EXPERTS FROM A VARIETY OF
FIELDS THAT MAYBE DON'T OFTEN
TALK TO EACH OTHER.
WE'VE GOT NEUROSCIENTIST, DATA
SCIENTISTS, SENSORY
DEVELOPMENTS, BEHAVIORAL
EXPERTS, ENGINEERS, WE'VE ASKED
THEM TO ENGAGE IN
CROSS-DISCIPLINARY DIALOGUE.
WE'VE ASKED THEM TO HELP US
HIGHLIGHT WHERE OPPORTUNITIES
ARE, WHERE RESEARCH COULD MAKE
THE MOST IMPACT.
WE'VE ASKED THEM TO IDENTIFY
WHERE THE NEEDS ARE, WHERE NEW
TECHNOLOGY NEEDS TO BE
DEVELOPED.
AND WE HOPE THIS WILL BE THE
START OF THE FORMATION OF
CONSORTIA TO STIMULATE PROGRESS
IN DEVELOPING NEW BEHAVIOR
ONTOLOGY, TECHNOLOGY,
INFORMATICS, COMPUTATIONAL TOOLS
AND ETHICAL FRAMEWORKS.
SO, I'M GOING TO GIVE YOU A
BIRDSEYE VIEW, THE PANEL IS
SPEAKERS AND DISCUSSANTS WITH A
MODERATOR, AND IN THIS FIRST
PANEL FORMATTED LIKE A
CONCEPTUAL DATA BLITZ,
CONCEPTUAL BLITZ, WE'VE ASKED
SPEAKERS AND DISCUSSANTS TO
THINK THROUGH HOW DO WE FRAME
OUR RESEARCH QUESTIONS TO STUDY
BEHAVIOR AS A COMPLEX DYNAMIC
SYSTEM, IN INTERACTION WITH A
COMPLEX ENVIRONMENT.
OUR SECOND PANEL WILL BE ON
SENSING BEHAVIOR IN ITS
ENVIRONMENTS, THE DIMENSIONS AND
DYNAMICS.
AGAIN WE'VE GOT FOUR SPEAKERS,
THREE DISCUSSANTS.
WE'VE ASKED THEM TO THINK ABOUT
HOW DO WE CAPTURE THE
INFORMATION NEEDED FROM
INDIVIDUAL ORGANISMS, SOCIAL
GROUPS, AND PHYSICAL
ENVIRONMENTS, THAT WE NEED TO
CAPTURE THE COMPLEXITY OF
BEHAVIOR, PARTICULARLY IN REAL
WORLD SETTINGS.
AND THEN THE THIRD PANEL WILL BE
ON WHAT ARE THE COMPUTATIONAL
OPPORTUNITIES AND CHALLENGES FOR
MODELING BRAIN BEHAVIOR
ENVIRONMENT, AS A COMPLEX
DYNAMIC SYSTEM.
MOVING, CLOSING OUT DAY 1, WE'VE
ASKED CAROL ROMMELFANGER AND
JUSTIN BAKER TO PROVIDE
SYNTHESIS AND CLOSING REMARKS
FROM THE THREE PANELS ACROSS DAY
ONE, TO HIGHLIGHT WHERE THE
OPPORTUNITIES AND CHALLENGES
MELEE.
DAY 2 IS GOING TO START WITH
THREE REALLY EXCITING TALKS,
WHICH WILL INTEGRATE THE
CONCEPTS FROM DAY 1 TO ADVANCE
NEUROSCIENCE, AND WE HAVE MALA
MURTHY, NANTHIA SUTHANA AND NITA
FARAHANY.
AFTER THE THREE TALKS WE'LL HAVE
CONCURRENT MODERATED BREAKOUT
SESSIONS, THE FIRST ONE IS ON
SHALL IT - I'LL BE MODERATING ON
HUMAN AND CLINICAL RESEARCH WITH
OUR BREAKOUT CO-LEADS, AND THEN
THE SIMULTANEOUS BREAKOUT WILL
BE MODERATED WITH DISCUSSANTS.
IF YOU CAN'T DECIDE, THEY ARE
BOTH RECORDED, BOTH WILL BE
STREAMED AFTER THE MEETING.
SO, THEN OF COURSE WE'RE REALLY
EXCITED TO BE WELCOMING DR. JOHN
NGAI, TO GIVE CLOSING REMARKS
AND SYNTHESIS AFTER WE HAVE THE
REPORT OUT FROM THE GROUPS AND
DISCUSSION.
IT WILL BE A REALLY EXCITING TWO
DAYS AND WE'LL BE LOOKING
FORWARD TO HEARING FROM OUR
BREAKOUT GROUPS, WHAT ARE THE
HIGH PRIORITY AREAS OF RESEARCH
TO PURSUE, WHAT ARE THE TOOLS
NEEDED TO PURSUE THEM, AND WHAT
ARE THE BARRIERS THAT WE NEED TO
ADDRESS.
THERE'S SO MANY PEOPLE TO THANK
HERE, AND WE'VE LISTED THE
PLANNING COMMITTEE AND MEMBERS
WHO HELPED TO ORGANIZE THIS
MEETING ON THIS SLIDE HERE.
THIS IS A COLLABORATION ACROSS
MULTIPLE NIH INSTITUTES AND
OFFICES AS WELL AS NSF.
TO SET US UP FOR PANEL 1, WHICH
IS GOING TO BE ON BEHAVIOR AS A
COMPLEX SYSTEM WITHIN A COMPLEX
SYSTEM, WE'LL HAVE FOUR
EIGHT-MINUTE TALKS FOLLOWED BY
DISCUSSION, EACH DISCUSSANT WILL
INTRODUCE AN IDEA OR QUESTION,
GENERATED BY THE TALKS, AND LEAD
DISCUSSION FOR NINE MINUTES.
THEN THEY WILL HAND OFF TO THE
NEXT DISCUSSANT.
WORKSHOP PARTICIPANTS ARE
INVITED TO USE THE CHAT TO POSE
QUESTIONS FOR THE DISCUSSION,
AND OTHER ATTENDEES PLEASE USE
THE Q&A BOX OR LIVE FEEDBACK
FUNCTIONS IN ZOOM OR VIDEOCAST.
WITHOUT FURTHER ADO I'D LIKE TO
INTRODUCE DR. JANINE SIMMONS,
OUR MODERATOR FOR PANEL 1.
>> THANKS, HOLLY.
I'M EXCITED TO BE LAUNCHING OUR
PANEL 1.
I'M SUPER EXCITED.
THE PANEL 1 AS HOLLY SAID IS
ENTITLED BEHAVIOR AS A COMPLEX
SYSTEM WITHIN A COMPLEX SYSTEM.
AND QUESTIONS ARE RESEARCH
QUESTIONS TO STUDY BEHAVIOR AS A
COMPLEX SYSTEM.
SPEAKERS ARE IAIN COUZIN,
JEFFREY KOEN AND LENA TING,
DISCUSSANTS MOLLY CUMMINGS,
TIMOTHY WRIGHT AND BENCE
ÖLVECZKY.
>> I HAVE TO APOLOGIZE.
MY FAMILY HAD COVID, I THOUGHT I
WAS GOING TO ESCAPE BUT HAD
REBOUND SYMPTOMS AND TESTED
POSITIVE.
EXCUSE THE VOICE, COUGHING,
FATIGUE, SO ON.
I COULDN'T MISS THIS AMAZING
EVENT.
YES, I WAS INTRIGUED BY BEHAVIOR
AS A COMPLEX SYSTEM WITHIN A
COMPLEX SYSTEM BECAUSE I THINK
THE NEUROSCIENCE OF COURSE IS
DEVELOPING INCREDIBLE TOOLS WE
JUST HEARD ABOUT THAT SPAN
ACROSS THESE VAST SPATIAL AND
ALSO TEMPORAL SCALES.
I LOVE THE PIECE BY BASSETT AND
MORROW, EXTENDS TO EVOLUTIONARY
SCALES.
BEHAVIOR DOESN'T MAKE SENSE IF
WE CONSIDER HOW THESE FEEDBACK
ON EACH OTHER.
MULTI-SCALE IMAGING IS SO
CRITICAL.
ANIMAL BEHAVIOR, ONE OF THE
WONDERFUL CHALLENGES IS THAT IT
INHERENTLY SPANS SCALES, ANIMALS
NEED TO RESPOND TO A RISK OR
THREAT, THEY HAVE TO RESPOND IN
TERMS OF FOOD RESOURCES, OVER
YEARS IN TERMS OF WITH WHOM THEY
CHOOSE TO MATE.
AND THERE ARE MANY FEEDBACK
PROCESSES ACROSS THE SCALES.
THEY CAN'T BE DECOUPLED IN A
SIMPLE WAY.
WHO YOU CHOOSE TO MATE WITH
IMPACTS MANY OTHER ASPECTS OF
YOUR LIFE.
AND WE'VE ALREADY SEEN THIS.
I DIDN'T SEE THE PREVIOUS TALKS,
JUST COINCIDENCE, THIS WONDERFUL
WORK FROM TIMOTHY DUNN,
CHARACTERIZING A BODY OF WORK TO
TRY TO DEVELOP QUANTITATIVE
TOOLS TO STUDY AND TO DESCRIBE
STEREOTYPICAL BEHAVIORS.
THIS IS TRANSFORMATIVE TO THE
WHOLE FIELD OF BEHAVIOR.
IN FACT, I THINK BEHAVIOR IS
UNDERGOING A RENAISSANCE.
WHEN I WAS A POSTDOC AT OXFORD
UNIVERSITY, IT WAS THOUGHT
BEHAVIOR WAS REDUNDANT, WE DID
IT IN THE 70s AND '80s, NOW
WE'RE REALIZING MECHANISM IS
FUNDAMENTAL TO UNDERSTANDING HOW
THE BRAIN PROCESSES INFORMATION.
BUT YET WE'RE STILL ONLY AT THE
VERY BEGINNING OF OUR STUDIES.
I THINK THAT'S THE EXCITING
THING.
SO, INSTEAD OF TALKING MY ABOUT
MY WORK, I'LL MENTION SOME BUT
WANTED TO REALLY TALK ABOUT WHAT
I'VE BEEN THINKING ABOUT LATE
LAST NIGHT, PREPARING MY SLIDES,
ABOUT WHAT ARE MAJOR CHALLENGES
AND GAPS IN THE STUDY OF
BEHAVIOR.
SOME OF THIS WILL REFLECT WHAT
YOU JUST HEARD BECAUSE I THINK
WE'RE INDEPENDENTLY THINKING THE
SAME TYPES OF THINGS.
SO, WE'RE GETTING PRETTY GOOD AT
DOING POSTURE ANALYSIS AND
DESCRIBING STEREOTYPICAL
PATTERNS.
WHAT WE'RE NOT GOOD AT DOING,
AND I WOULD LOVE TO HEAR MORE
FROM OTHER PARTICIPANTS, HOW
ANIMALS TAKE IN HIGH DIMENSIONAL
SENSORY INFORMATION,
MULTI-MODAL, THE BRAIN IS THIS
ENTITY THAT ALLOWS ANIMALS TO
MAKE SENSE OF THIS, TO MAKE
DECISIONS, BASED ON MEMORY,
PHYSIOLOGIC STATE, GENETICS, AS
WAS MENTIONED BEFORE BEHAVIOR
CAN ACTUALLY GIVE INSIGHT INTO
THESE INTERNAL STATES.
BUT WE STILL REALLY LACK THIS
FORMAL WAY, INFORMATIONAL
THEORETIC TOOLS, TO ALLOW US TO
CAUSALITY.
CORRELATION, YES, BUT CAUSALITY
IS STILL CHALLENGING.
ONE OF THE REASONS, I'LL COME
BACK TO THIS, WE HAVE THESE
RECURSIVE FEEDBACKS ACROSS
MULTIPLE TIME SCALES.
SO I THINK THAT'S A VERY
IMPORTANT ASPECT THAT WILL
IMPROVE ALL OF OUR RESEARCH.
ANOTHER ASPECT I'VE BECOME VERY
INTERESTED IN, I THINK, WEAK
LINK, OFTEN MOVEMENT IS
CONSIDERED THE OUTCOME OF THE
DECISION, ANIMAL DECIDES WHAT IT
WANTS TO DO, MOVES TO GET TO
LOCATION IT WANTS.
BUT THE MOVEMENT, THERE'S A
DYNAMIC FEEDBACK BETWEEN MOTION
SENSING AND DECISION MAKING AND
OTHERS BUT THIS IS YET A VERY
SMALL FIELD IN BODY DECISION
MAKING, IT'S VERY IMPORTANT TO
CONSIDER THE BRAIN AS EMBODIED
IN THE ORGANISM TO UNDERSTAND
SPATIAL TEMPORAL COMPUTATION.
TIMOTHY IS IN THE AUDIENCE, HE'S
DONE WONDERFUL WORK THAT SHOWS
THIS TOO.
I DON'T HAVE TIME TO TALK ABOUT
MY RESEARCH BUT WANT TO
EMPHASIZE WE'VE BEEN REALLY
LOOKING INTO THIS IN A
COMPREHENSIVE WAY WITH A RANGE
OF SYSTEMS, AND SHOWING WE
SIMPLY CANNOT MAKE SENSE OF THE
DECISIONS ANIMALS MAKE, WITHOUT
INCLUDING THIS RECURSIVE
FEEDBACK, THIS REVEALS THE BRAIN
IS BEHAVING AS DYNAMIC SYSTEM
WITH BIFURCATION ALSO THAT BREAK
UP THE COMPLEXITY OF THE WORLD
INTO A SERIES OF BINARY, BINARY,
BINARY DECISIONS.
SO I THINK THIS COULD INTEGRATE
VERY WELL WITH CLASSICAL
NEUROSCIENCE IN TERMS OF
DECISION MAKING BECAUSE WE FIND
THE BRAIN CONTINUOUSLY CHANGES
THE WORLD, NO MATTER HOW MANY
OPTIONS, INTO A SERIES OF TWO
OPTION DECISIONS.
WE COMPARE THEORY WITH
EXPERIMENTS.
NO ANIMAL IS AN ISLAND.
SOCIAL BEHAVIOR HAS BEEN POORLY
DESCRIBED AND POORLY CONSIDERED,
I GUESS, WITHIN THIS TYPE OF
FRAMEWORK.
ALMOST ALL DECISIONS ANY ANIMALS
MAKE INVOLVES INDIRECT OR DIRECT
SOCIAL INFORMATION.
LOOKING AT CLASSIC MODELS, SORT
OF MODELS THAT WE CAN USE IN OUR
COMMUNITY, C. ELEGANS IS A
HIGHLY SOCIAL ANIMAL.
SO ARE DROSOPHILA, ODOR
AVOIDANCE, WE ALSO USE SOCIAL
INTEGRATION TO FORAGE, CHOOSE
MATES, SO ON.
BUT THOSE ORGANISMS, DROSOPHILA
AND C. ELEGANS, ARE INCREDIBLY
HARD TO STUDY IN NATURAL
ENVIRONMENT.
WE HAVE AT LEAST TWO MODEL
SYSTEMS PERFECT FOR CROSSING
SCALES BETWEEN NATURAL
ENVIRONMENT AND SOCIAL
COMPLEXITY AND CONTROLLED
EXPERIMENT, ZEBRAFISH BEING ONE,
THEY HAVE TERRITORIAL BEHAVIOR,
THEY HAVE RICH FIGHTS THAT LEAD
TO TERRITORIES, SCHOOLING OF
COURSE.
RATS, WE WERE ONE OF THE FIRST
GROUPS TO EVER ASK THIS BASIC
QUESTION, WHAT HAPPENS IF YOU
PUT TWO RATS IN A MAZE?
OR EIGHT RATS IN A MAZE?
THEY SOLVE THIS MAZE
COLLECTIVELY WITH SOME WONDERFUL
EXAMPLES OF HOW INTERACTIONS CAN
LEAD TO COGNITIVE ABILITIES
BEYOND THE INDIVIDUAL.
AND YET I'M GOING TO BE A LITTLE
BIT DRAMATIC NOW, PUSH YOU A
LITTLE BIT.
I WOULD ARGUE THE BEHAVIOR OF
THES ORGANISMS IS ALMOST ALWAYS
STUDIED IN ABSENCE OF CONTEXT.
BY DOING SO WE SEVERELY RESTRICT
AND BIAS THE BEHAVIORS WE STUDY.
ALSO LIMIT THE SCOPE OF OUR
STUDIES BUT MEANS WE CAN
MISUNDERSTAND DATA.
IT BECOMES EXTREMELY HARD WITH
EVOLUTIONARY PROCESS, ULTIMATELY
WHAT MATTERS.
EXAMPLE IS COLLECTIVE MIGRATION.
PEOPLE PUT TAGS ON INDIVIDUAL
ANIMALS, WHAT THE INDIVIDUAL IS
DOING EVERYTHING THEMSELVES.
BUT ALMOST ALL MIGRATIONS ARE
KNOWN TO BE COLLECTIVE.
SO WHY IS STUDYING COLLECTIVE
BEHAVIOR CHALLENGING?
WELL, IT'S VERY HARD TO
DETERMINE CAUSAL RELATIONSHIPS.
THIS IS ANOTHER AREA THAT WE
NEED TO HAVE DEVELOPMENTS IN.
EVEN IF WE HAVE TWO ANIMALS,
THEN THINK OF THE THIRD ANIMAL,
IS HE BEING INFLUENCED OR A, OR
A AND C VIA INDIRECT EFFECT, AND
CAN ADD MORE COMPLEX
RELATIONSHIPS.
WE'VE BEEN TRYING TO STUDY IN
SCHOOLING FISH AND FLOCKING
BIRDS, EXTREMELY CHALLENGING TO
YOU CAN WORK OUT CAUSAL NATURE
OF INFLUENCE.
TOWARDS THE SOLUTION, SO I
REALLY AM EXCITED BY DEVELOPING
VIRTUAL REALITY ENVIRONMENTS,
IMMERSEIVE HOLOGRAPHIC, WHERE
INDIVIDUALS CAN INTERACT FREELY
WIN VIRTUAL WORLDS.
AND WE CAN NETWORK SYSTEMS
TOGETHER.
YOU CAN SEE AN INDIVIDUAL --
VIRTUAL INDIVIDUAL MOVING IN THE
CIRCLE, THE REAL ZEBRAFISH
FOLLOWING, BELIEVES IT TO BE IN
THE TANK WITH IT EVEN THOUGH
THIS IS AN ILLUSION.
WE HAVE TO PROJECT THE WORLD
PRECISELY FROM PERSPECTIVE OF
THE ANIMAL, WE CAN CREATE
MULTIPLE SYSTEMS AND NETWORK
TOGETHER AND ALLOW REAL
INDIVIDUALS TO INTERACT WITH
EACH OTHER, NO T IN THE SAME
PHYSICAL WORLD BUT SAME
HOLOGRAPHIC WORLD, GIVES GREAT
CONTROL TO UNDERSTAND WHAT'S
GOING ON.
WE HAVE NOW 11 OF THESE SYSTEMS
HERE, YOU CAN SEE FOUR
INDIVIDUALS.
THEY ARE NOT INTERACTING IN THE
REAL PHYSICAL WORLD, WITHIN A
HOLOGRAPHIC WORLD.
AND WORKING WITH HARVARD
UNIVERSITY WE CAN CREATE A
STRUCTURAL ENVIRONMENT WITH A
MICROSCOPE, WHOLE BRAIN IMAGING
AT CELLULAR RESOLUTION, WITH
ANIMALS INTERACTING WITH THE
OTHER HOLOGRAPHIC, ONE COULD BE
IN THE U.S., WITH THE BANDWIDTH.
WE'VE BEEN LOOKING AT ZEBRAFISH
AS A MODEL FOR AUTISM SPECTRUM
DISORDER.
THERE'S A MASSIVE GAP IN OUR
STUDIES OF BEHAVIOR.
ALMOST ALL STUDIES FOCUS IN
CONTROLLED SIMPLISTIC LABORATORY
ENVIRONMENTS.
MICROSCOPIC, IF YOU GO BACK TO
THE BEGINNING OF MY TALK.
ALL IN THE UNCONTROLLED
CONDITIONS OF THE WORLD,
MACROSCOPIC, I'M INVOLVED IN
BOTH COMMUNITIES, VERY FEW
PEOPLE ARE.
SO WE HAVE THIS BEAUTIFUL WORK
BY TIMOTHY AND OTHERS OF THE
MICROSCOPIC SCALE, WORK WE'VE
BEEN DOING, SO THIS IS A PAPER,
STATE-OF-THE-ART TECHNOLOGY IN
THE WILD TO TRACK ANIMALS AS
THEY MIGRATE, STORKS IN
SUB-SAHARAN AFRICA, BUT NOTHING
IN THE MIDDLE.
MESOSCOPEIC IS VOID, A BARREN
LAND.
WE DON'T MAKE THESE CONNECTIONS,
AS IMPORTANT AS ANYTHING ELSE.
WHEN I BUILT THIS, THE CONCEPT
WITH THE LARGE INDOOR FACILITY,
THAT ALLOW US TO BEGIN TO
ADDRESS THIS.
YOU HAVE RATS CHOOSING BETWEEN
TWO OPTIONS, THOUSANDS OF TIMES,
ALLOWING THEM TO MOVE WITHIN
LARGE ENVIRONMENTS, WHILE DOING
BRAIN IMAGING.
ALSO THIS IS ANOTHER WHERE WE
CAN ALSO DO THE WORK AND LOOK AT
THE ACOUSTIC COMMUNICATION.
>> I'M GOING TO BE TALKING ABOUT
BEHAVIOR IN HUMANS.
AND HOW BEST TO MEASURE
BEHAVIOR.
COMMONLY USED TO SELF-REPORT, AS
A NUMBER OF ADVANTAGES, IT'S
CONVENIENT, IT'S FAST, ALLOWS
FOR QUANTIFICATION, IT'S EASILY
OBTAINED.
AND THERE ARE MANY RELIABLE
QUESTIONNAIRES TO CHOOSE FROM.
THE DISADVANTAGE OF COURSE IS
THAT IT REQUIRES ABILITY TO
SPEAK, WRITE, OR GESTURE.
THAT CAN'T BE DONE IN YOUNG
CHILDREN.
IT CAN'T BE DONE IN PEOPLE WHO
ARE ON BREATHING TUBES OR WHO
ARE COMMUNICATIONALLY
CHALLENGED.
IT'S IDIOSYNCRATIC.
THE MEANING OF THE MEASURES KIND
OF VARY WITH THE PERSON.
IT'S LIMITED TO WHAT IS
CONSCIOUSLY AWARE.
IT'S SUSCEPTIBLE TO SUGGESTION,
IMPRESSION, DECEPTION.
TEMPORARILY SPARSE, YOU CAN ONLY
ASK SO MANY QUESTIONS IN ANY
PERIOD OF TIME AND LACKS ANALOGS
IN OTHER SPECIES.
SO HOW ELSE COULD MEASURE
BEHAVIOR?
FACIAL EXPRESSION IS ONE WE'VE
BEEN ESPECIALLY INTERESTED IN.
FACE, HEAD, AND BODY MOTION
PRESENTS ANOTHER.
VOICE, VOICE QUALITY, TIMING.
AND THEN PHYSIOLOGY WHICH CAN BE
MEASURED EITHER WITH THERMAL
CAMERA OR CONTACT SENSORS, ALL
OF THESE MODALITIES HAVE BEEN
WELL STUDIED IN PEOPLE AND IN
DEVELOPMENTALLY AND
CROSS-CULTURALLY.
MY OWN APPROACH HAS EMPHASIZED
VISUAL FACIAL EXPRESSION.
AND THAT'S WHAT I'LL FOCUS ON
TODAY.
FACIAL EXPRESSION IS MADE
POSSIBLE BY A DENSE MESH OF
FACIAL MUSCLES, UNLIKE MUSCLES
ANYWHERE ELSE IN THE BODY, THEY
CONNECT EITHER TO EACH OTHER OR
THEY CONNECT TO SOFT TISSUE.
THIS GIVES A HIGH DEGREE OF
FREEDOM, HIGH DEGREES OF FREEDOM
FOR FACIAL EXPRESSION.
THE APPROACH WE USE, ANNOTATION,
IS BASED ON ECKMAN'S ENCODING
SYSTEM, IN THAT SYSTEM EACH
FACIAL -- EACH MUSCLE OR SMALL
NUMBER OF MUSCLES IS REFERRED TO
AS AN ACTION UNIT.
AND THESE HAVE HIGH DEGREE OF
CONTROL.
WE ARE INTERESTED IN THE
OCCURRENCE, THE INTENSITY, AND
TIMING OF FACIAL ACTION.
WE CAN TRACK CHANGE IN
EXPRESSION AND DESCRIBE IT IN
TERMS OF MOLECULAR MOVEMENTS.
THE APPROACH THAT WE USE,
DEVELOPED IN COLLABORATION, IS
AUTOMATED FACIAL ACTION
RECOGNITION WHICH WE REFER TO
AFAR.
THIS IS ONE VERSION OF THE
SYSTEM, THERE ARE A NUMBER OF
VERSIONS, YOU KNOW, THAT WE'VE
BEEN USING, IS A COMPUTER VISION
MACHINE-LEARNING BASED APPROVE,
TO MEASURING FACIAL ACTIONS AND
HEAD AND FACE DYNAMICS.
IT'S BEEN TRAINED ON LARGE AND
HETEROGENEOUS DATA,
GENERALIZABLE TO PARTICIPANTS
SUCH AS I'LL BE TALKING ABOUT
WITH DEEP BRAIN STIMULATION.
IN WORK WITH BAYLOR COLLEGE OF
MEDICINE, WAYNE GOODMAN, NICOLE
PROVENZA WHO WILL BE FOLLOWING
ME, WE'VE BEEN INTERESTED IN
RELATION BETWEEN STIMULATION OF
THE VENTRAL STRIATUM AND
(INDISCERNIBLE) PATIENTS, FIVE
PATIENTS WHO ARE UNDERGOING
TREATMENT WITH DEEP BRAIN
STIMULATION, OR TREATMENT
RESISTANT OBSESSIVE COMPULSIVE
DISORDER.
ONE CONTEXT IS PROGRAMMING
SESSIONS IN WHICH THE PARAMETER
SETTINGS OF THE DEVICE ARE
ADJUSTED IN ORDER TO OPTIMIZE
TREATMENT.
AND THIS IS A SERIES OF
PROGRAMMING SESSIONS, A PERIOD
OF EIGHT MONTHS.
THE SECOND CONTEXT IS
INTERVIEWS.
CLINICAL INTERVIEWS OVER THE
SAME PERIOD OF TIME.
SO FOR MORE KIND OF NATURALISTIC
BEHAVIOR.
THIS SLIDE SHOWS AN EXAMPLE FROM
A PROGRAMMING SESSION.
HERE YOU CAN SEE THE DBS
AMPLITUDE, TRACKING POSITIVE
AFFECT AND HEAD VELOCITY.
NOTE THAT THERE IS KIND OF A
CLOSE COORDINATION BETWEEN
CHANGES, EXPRESSION OF FACIAL
AFFECT, EXPRESSED IN TERMS OF
(INDISCERNIBLE) CONTRACTION
REPRESENTED BY UNITS AND HEAD
VELOCITY.
IN CLINICAL INTERVIEWS, OVER THE
SAME PERIOD OF TIME, WE LOOK AT
RELATION BETWEEN FACIAL AFFECT
AND HEAD DYNAMICS IN RELATION TO
THE DBS ENERGY, WHICH IS A
FUNCTION OF AMPLITUDE, PULSE
WIDTH AND FREQUENCY, AND FOUND
CLOSE TO 30% OF THE VARIANTS IN
DBS ENERGY DURING AND INTERVIEW
WAS ACCOUNTED FOR -- CAN BE
ACCOUNTED FOR BY HEAD DYNAMICS
AND FACIAL ACTION UNITS.
WE LOOKED AS WELL AT THE
RELATION TO YBOCS II SEVERITY
SCORES, HEAD DYNAMICS AND FACIAL
ACTION UNITS ACCOUNTED FOR ABOUT
20% OF THE VARIANTS.
WHICH IS KIND OF A STRONG -- I
WOULD SAY STRONG EFFECT IN THIS
CONTEXT.
AS AN EXAMPLE OF DBS IN A
CLINICAL INTERVIEW, WE HAVE TWO
INTERVIEWS, THIS IS FROM A CASE
STUDY, KIND OF EARLY ON.
IN ONE, THIS INDIVIDUAL HAS THE
DBS IS ON, THE SECOND VIDEO
SHOWS DBS OFF, AFTER A PERIOD OF
ABOUT THREE HOURS.
NOTE THERE IS A DRAMATIC
DIFFERENCE IN HIS FACIAL AFFECT
DURING THESE TWO CONDITIONS.
WHEN IT'S ON, THERE IS INTENSE
POSITIVE AFFECT, CYCLING ON AND
OFF, BROW RAISES, SMILES, SO ON,
VERY LITTLE, VERY FEW IN THE
NEGATIVE AFFECT, WITH OFF .
YOU FACIAL EXPRESSION HAS
REVEALED TIME-LOCKED CHANGES IN
NEURAL ACTIVATION AND
LONGITUDINAL VARIATION IN DBS
AND SYMPTOMATOLOGY.
FACIAL EXPRESSION IS ONE OF MANY
MODALITIES THAT CAN BE APPLIED
IN HUMAN AND COMPARATIVE STUDIES
OF BRAIN AND BEHAVIOR.
GOING FORWARD, WE'RE INTERESTED
IN EXPANDING THE NUMBER OF
NEURAL SITES, SO FAR WE'VE
PRIMARILY BEEN LOOKING AT SITES
ASSOCIATED WITH OCD, SEPARATE
STUDY WE'RE LOOKING AT KIND OF
DEPRESSION, AND USING FACILITIES
OF EPILEPSY MONITORING UNIT AT
BAYLOR TO LOOK AT A LARGER
NUMBER OF SITES IN RELATION TO
THEIR EFFECT ON -- OR OUTPUT IN
TERMS OF FACIAL AFFECT AND OTHER
MODALITIES.
VOICE QUALITY AND TIMING, SPEECH
AND SENTIMENT, PERIPHERAL
PHYSIOLOGY AND SOCIAL
INTERACTION.
WE HAVE TWO BROAD GOALS.
ONE IS BEHAVIORAL BIOMARKERS FOR
ADAPTIVE DEEP BRAIN STIMULATION.
NICOLE PROVENZA WILL TALK MORE
ABOUT THAT IN THE VERY NEXT
TALK.
AND MORE OBJECTIVE MEASUREMENT
OF BEHAVIOR IN CLINICAL TRIALS,
CURRENTLY FOR CLINICAL TRIALS
THE PRIMARY OUTCOME MEASURE IS
SELF-REPORT.
WE WANT TO USE MULTI-MODAL
MEASURES TO MORE OBJECTIVELY
MEASURE SYMPTOMATOLOGY AND
RESPONSE TO TREATMENT AND WORK
WITH DELIBERATE, WE'VE BEEN
WORKING WITH PREDICTING HAMILTON
DEPRESSION SCORES AND USING
MULTI-MODAL MEASURES, AND SO FAR
KIND OF ENCOURAGING RESULTS,
ACCOUNTING FOR 60% OF THE
VARIATION BETWEEN OBSERVED
HAMILTON SCORES AND PREDICTIVE.
LAST IN TERMS OF CHALLENGES, WE
NEED MORE DATA.
YOU KNOW, KIND OF DATA IS THE
KEY FOR LEARNING.
ONE OF THE CONSTRAINTS IS
CONSTRAINTS OF CLINICAL
SETTINGS, THAT WE'RE NOT SET UP
AS KIND OF LABORATORIES.
SO IN AN EPILEPSY UNIT, THE
LIGHTING IS HOSPITAL LIGHTING,
HOW DO YOU IMPROVE THE
ENVIRONMENT IN A WAY THAT
FACILITATES IMPROVED
MEASUREMENT.
A SECOND IS SYNCHRONIZING VIDEO
AND NEURAL DATA.
AND THIS IS -- THIS IS AN AREA
THAT IS REALLY IN NEED OF
IMPORTANT ADVANCES.
AND SO WITH THAT I'D LIKE TO
THANK YOU FOR THE OPPORTUNITY TO
PRESENT TODAY.
THANK YOU.
>> THANK YOU VERY MUCH.
HOPEFULLY YOU CAN HEAR ME BETTER
NOW.
WE'RE MOVING TO THE THIRD TALK,
NICOLE PROVENZA.
>> HI, EVERYONE.
I'M A POSTDOC AT BAYLOR COLLEGE
OF MEDICINE SPEAKING ON PROGRESS
IN HUMAN NEUROSCIENCE.
NO DISCLOSURES TO REPORT.
THERE'S SEVERAL KEY CHALLENGES
THAT I THINK NEED TO BE
ADDRESSED TO LINK NEURAL
BEHAVIORS TO PSYCHIATRIC
DISORDERS.
ONE IS SYMPTOM HETEROGENEITY.
PSYCHIATRIC DIAGNOSIS ARE
DEFINED BY CLUSTERS OF SYMPTOMS
AND THERE'S HETEROGENEITY.
SYMPTOMS ARE SUBJECTIVE.
EVEN IF WE HAVE AN IDEA OF
SYMPTOMS WE WANT TO MEASURE HOW
ARE WE GOING TO MEASURE THEM.
THIS IS REALLY CHALLENGING IN
PSYCHIATRY AS THERE ARE NO
ESTABLISHED PHYSICAL READOUTS OF
SYMPTOM STATES LIKE IN MOVEMENT
DISORDERS.
CLINICIAN ADMINISTERED SKILLS,
AS DR. COHM MENTIONED, WITH USED
TO ASSESS SEVERITY AND DEPEND ON
SUBJECTIVE INPUT.
THREE IS SPARSITY OF BEHAVIORAL
INFORMATION.
SELF-REPORT SCALES ARE MEANT TO
BE ADMINISTERED ABOUT EVERY TWO
WEEKS, AND SO IF WE ADMINISTER
THEM MORE OFTEN, SAY EVERY DAY,
WE DON'T HAVE A METRIC FOR EVERY
MOMENT OF EVERY DAY, STILL.
SO THESE SELF-REPORT SKILLS ARE
TEMPORALLY SPARSE COMPARED TO
DENSE LEAD DATA.
FOR MY WORK I'M MOTIVATED TO
OVERCOME CHALLENGES TO IDENTIFY
BIOMARKERS TO IMPROVE OUTCOMES
OF DBS IN SEVERE OCD AND
DEPRESSION, PART OF AN NIH
EFFORT LED BY DR. WAYNE GOODMAN.
WE RECORD NEURAL SIGNALS, USE
THOSE TO CLASSIFY A RELEVANT
BRAIN STATE SUCH AS OBSESSIVE
COMPULSIVE SYMPTOM STATE,
CONTROL STIMULATION PARAMETERS
ACCORDINGLY TO RELIEVE SYMPTOMS.
SO IDENTIFICATION OF A RELATIVE
BEHAVIORAL STATE AND BRAIN STATE
UNDERLYING THIS BEHAVIOR IS
REALLY CRITICAL FOR ACHIEVING
THIS GOAL.
SO ONE OF THE MANY STRATEGIES
TOWARD FINDING BIOMARKER TO BE
TO RELATE TO DENSELY MEASURED
BEHAVIORAL FEATURES, IN MY
PREVIOUS WORK WE DID THIS WITH
OCD PARTICIPANTS, IMPLANTED WITH
SENSING CAPABLE DBS DEVICES.
AND WE MEASURED BEHAVIOR DURING
CONCURRENT NEURAL REPORTINGS IN
THE CLINIC AND AT HOME.
IN THE CLINIC WERE ABLE TO DO
DENSE BEHAVIORAL MODELING, AT
HOME WE HAVE PARTICIPANTS
CONTROL INTRACRANIAL RECORDINGS
THROUGH TABLET, REPORTING USING
A SLIDER BAR ON A PHONE
APPLICATION.
ONE PARTICIPANT DID A THREE-DAY
RECORDING AT HOME, NATURAL
EXPRESSURES TO OCD TRIGGERS ON
DAYS 1 AND 2, REFLECTED BY THE
SYMPTOM INTENSITY ON THE Y-AXIS,
TO ANALYZE THE LFP DATA I
COMPUTED SPECTRAL POWER OVER THE
TWO-MINUTE SEGMENT ALSO OF DATA
AROUND EACH RATING.
HERE I'M SHOWING RESULTS FROM
THAT ANALYSIS, SPECTRAL POWER
VERSUS SYMPTOM INTENSITY FOR
LEFT VC/VS, AND SO I FOUND
SPECTRAL POWER SHOWS CORRELATION
OF OCD SYMPTOM INTENSITY WHERE
CORRELATION BETWEEN DELTA BAND
POWER SHOWED STRONG NEGATIVE
CORRELATION WITH OCD SYMPTOMS.
SO WE'VE IDENTIFIED A CANDIDATE
NEURAL BIOMARKER DURING NATURAL
SYMPTOM PROVOCATION AT HOME.
WHERE DO WE GO FROM HERE AND
CONTINUE CLOSING IN ON
BIOMARKERS AND VALIDATE THIS
FINDING IN A LARGER SAMPLE?
I THINK OUR FOCUS SHOULD BE ON
HOW TO COLLECT BEHAVIORAL DATA
IN NATURAL ENVIRONMENTS BUT IS
JUST AS RICH AS THE NEURAL DATA
THAT WE'RE ALREADY COLLECTING.
WE NEED TO IDENTIFY TEMPORALLY
DENSE REAL WORLD BEHAVIORAL
METRICS.
AS JEFF MENTIONED, THERE ARE
LIMITATIONS WITH SELF-REPORTS,
SUBJECTIVE EXPERIENCE CAN BE
DIFFERENT THAN OBSERVABLE
BEHAVIOR, AGAIN TEMPORALLY
SPARSE.
SO IF WE WANT TO KNOW HOW BRAIN
ACTIVITY IS VARYING WITH CHANGES
IN SEVERITY WE NEED CONTINUOUS
MEASURE,S REDUCING PATIENT
BURDEN TO REPORT HOW THEY ARE
FEELING ACCURATELY AT EVERY
SINGLE MOMENT.
PASSIVE OBJECTIVE MEASURES MIGHT
ELUCIDATE BRAIN BEHAVIOR
RELATIONSHIPS UNDERLYING WHY
SOMEONE FEELS THE WAY THEY DO.
SO TO GO BACK TO OUR EXAMPLE OF
OCD, WE'VE BEEN WORKING ON
IDENTIFYING MEANINGFUL
CONTINUOUS METRICS BY USING AFAR
AS DR. COHN DESCRIBED.
THIS IS A STILL SHOT FROM THE
VIDEO THAT HE SHOWED.
WE SAW A PEAK IN POSITIVE
AFFECT, A SMILE, DBS AMPLITUDE
INCREASING.
SO THIS DATA AND OTHER DATA LIKE
THIS WITH CONTINUOUS METRICS
SYNCHRONIZED TO NEURAL DATA
COULD BE HARNESSED TO USE -- TO
BUILD MACHINE LEARNING MODELS
THAT PREDICT BEHAVIOR FROM
NEURAL ACTIVITY OR VICE VERSA.
THE NEXT ACTION ITEM ON MY LIST
IS THAT I THINK WE NEED TO
INCORPORATE MULTI-MODAL CONTEXT
AWARE BEHAVIORAL MODELING, I
MEAN SOME BEHAVIORS MIGHT BE
MORE QUANTIFIABLE OR OBSERVABLE
THAN OTHERS DEPENDING ON
CONTEXT.
OUR GOAL THOUGH IS TO DEVELOP
METHODS FOR QUANTIFYING
BEHAVIOR, VIA MULTIPLE
MODALITIES THAT HAVE HIGHER
RELIABILITY WITH MANUAL
ANNOTATION.
SO HERE I'VE LISTED SOME
EXAMPLES OF BEHAVIORS THAT WE
COULD BE MONITORING AND GROUPED
THEM INTO CATEGORIES.
SO IF WE WANT TO QUANTIFY
BEHAVIOR DURING A SOCIAL
INTERACTION, FOR EXAMPLE, ONLY
THE SOCIAL CATEGORY WOULD BE
MEANINGFUL TO MONITOR.
MAYBE LIKE DURING A WORKOUT,
THESE MORE SOCIALLY GEARED
METRICS MIGHT NOT BE INFORMATIVE
OR POSSIBLE TO ENCAPTURE.
THERE'S SOME CHALLENGES WITH
THIS.
ONE INHERENT CHALLENGE FOR
DEVELOPING THESE METHODS AND
SEARCHING FOR OBJECTIVE
BEHAVIORAL FEATURES, DOING THIS
SORT OF SCIENCE, IS IT'S MORE
EXPLORATORY AND MIGHT NOT FIT
INTO THE FRAMEWORK OF
TRADITIONAL HYPOTHESIS-DRIVEN
STUDY.
I THINK THIS WILL REQUIRE REALLY
LARGE N, REQUIRES BROAD
TECHNICAL EXPERTISE, IN
NEUROSCIENCE AND APPLYING
COMPUTER VISION TECHNIQUES AND
ENGINEERING, AND RAISES
QUESTIONS ABOUT HOW TO PROTECT
THE PRIVACY OF THE PARTICIPANTS,
AND THINK IT'S IMPORTANT TO
CONSIDER THERE MAY NOT BE
CONVERGENCE ACROSS DIFFERENT
MODALITIES.
SO THE THIRD ACTION ITEM IS THAT
I THINK WE NEED TO EXPLORE THE
RELATIONSHIP BETWEEN TASK BASE
AND REAL WORLD BEHAVIORS, WE
NEED TO BE PROPOSING
NATURALISTIC EXPERIMENTS
GROUNDED IN FINDINGS FROM
CLASSIC PSYCHOPHYSIOLOGIC TASKS.
SO, CAN WE BETTER ALIGN
PSYCHOPHYSIOLOGIC TASK-BASED
STUDIES WITH OBSERVATION TO
UNDERSTAND THE RELATIONSHIP
BETWEEN LAB-BASED TASKS AND
NATURALISTIC BEHAVIORS?
ANOTHER EXAMPLE FROM MY PREVIOUS
WORK IN OCD IS SHOWN HERE, SO ON
THE LEFT I'M SHOWING SCHEMATIC
FOR A TASK MEANT TO PROVOKE OCD
SYMPTOMS, ON THE RIGHT I'M
SHOWING NATURALISTIC ANALOG OF
THIS PROVOCATION TASKS,
PARTICIPANT IS WORKING THROUGH A
REAL-LIFE EXPOSURE OF IMAGINING
A WART ON HIS HAND DURING A
TELETHERAPY SESSION AT HOME.
SO PAIRING TASKS-BASED AND
NATURALISTIC EXPERIMENTS THIS
WAY ALLOWS US TO ASK THE
QUESTION ARE THERE MENTAL
PROCESSES ELICITED BY BEHAVIORAL
TASKS THAT ARE RELEVANT FOR REAL
WORLD SYMPTOMS OR FUNCTIONAL
DEFICITS?
AND I THINK THIS QUESTION IS
CRITICAL MOVING FORWARD FOR
IDENTIFYING ECOLOGICALLY VALID
NEURAL BIOMARKERS OF
DISEASE-RELEVANCE BEHAVIORS.
SO, TO WRAP UP, I'LL TALK A BIT
ABOUT THE IMPACT THAT THIS WORK
I THINK MIGHT HAVE ON THE FIELD.
I THINK THAT MORE DENSE
BEHAVIORAL INFORMATION COULD
ENABLE PRECISION PSYCHIATRY,
PERHAPS WITH BETTER BEHAVIORAL
QUANTIFICATION COULD MATCH
INDIVIDUALS TO TREATMENTS OR
UNDERSTAND WHY CERTAIN PEOPLE
RESPOND TO CERTAIN TREATMENTS.
AND I THINK THAT MORE DENSE
BEHAVIORAL INFO COULD ENABLE
IDENTIFICATION OF NEURAL
BIOMARKERS, RICH BEHAVIORAL
QUANTIFICATION WOULD HELP US
BETTER UNDERSTAND THE BRAIN
BEHAVIOR RELATIONSHIP UNDERLYING
PSYCHIATRIC DISORDERS, AND
NEURAL BIOMARKERS COULD SERVE AS
A MORE OBJECTIVE GOAL POST FOR
RESPONSE TO TREATMENT.
AND LASTLY, I THINK THAT THIS
MORE DENSE BEHAVIORAL
QUANTIFICATION COULD SERVE AS
PART OF THE MUCH NEEDED BRIDGE
BETWEEN CONTROLLED AND
NATURALISTIC EXPERIMENTS, AND
TASK EXPERIMENTS ALLOW US TO
OBSERVE AND QUANTIFY BEHAVIOR IN
A CONTROLLED WAY AND WE NEED TO
APPROACH BEHAVIORAL
QUANTIFICATION IN NATURAL
ENVIRONMENTS WITH JUST AS MUCH
IF NOT MORE RIGOR THAN
TASK-BASED STUDIES.
THANK YOU FOR YOUR ATTENTION. .
>> THANKS, NICOLE.
SO FOR THE LAST TALK, LENA TING.
>> OKAY, GREAT.
SO THANK YOU SO MUCH FOR
INVITING ME HERE AS A PERSON WHO
SPENT A LOT OF TIME REALIZING
THAT THE MOVEMENT INVOLVED FROM
THE -- EMERGED FROM INTERACTION
OF THE BIOMECHANICS OF THE BODY,
AND NEURAL CONTROL SYSTEMS, IT'S
OVER THE YEARS IT'S BECOME CLEAR
TO ME I HAVE TO MOVE BEYOND
THINKING ABOUT MOVEMENT AS
REALLY JUST THE OUTPUT THAT WE
CARE ABOUT, TO THINK MORE ABOUT
BEHAVIOR AS IT'S BEEN SAID
BEFORE.
SO, THIS MATCH WHAT IS WAS SAID
BEFORE.
BUT WE REALLY CAN'T
DISASSOCIATE.
IT'S BEEN CLEAR WE CAN'T
DISASSOCIATE COGNITIVE FUNCTION,
AFFECT AND MOVEMENT BECAUSE ALL
OF THESE THINGS ARE INTERTWINED,
AS THEY ARE ALL HELPING US WITH
OUR BEHAVIORS, WITH SOLVING
PROSECUTES IN THE WORLD THAT
HAVE VALUE TO US, MAKING
DECISIONS WITH INCOMPLETE
INFORMATION, AND TAKING ACTIONS
IN AN ENVIRONMENT THAT'S
DYNAMIC.
SO IN MY FIELD WE TALK ABOUT THE
NERVOUS SYSTEM GENERATING
INTERNAL MODELS OF THE EXTERNAL
ENVIRONMENT, THAT LET US COMPUTE
FASTER.
THESE NECESSARILY INTRODUCE
BIASES IN HOW WE MOVE THAT MIGHT
SHOW UP AS SOMETHING WE MIGHT
REFER TO AS OUR PERSONALITY OR
OUR LIFE HISTORY.
AND, YOU KNOW, THE WORDS
ASSOCIATED WITH THESE SHOW UP IN
DIFFERENT FIELDS IN DIFFERENT
WAYS, BUT THE BASIC IDEA IS WHEN
WE LOOK AT BEHAVIORS, CHOICES
PEOPLE MAKE, WHETHER THEY ARE
COGNITIVE OR MOTOR, THE WAY IN
WHICH WE GENERATE THEM ARE NOT
NECESSARILY -- THEY ARE NOT
OBJECTIVE, WE'RE NOT GOING TO BE
ABLE TO WRITE DOWN EQUATIONS FOR
THEM BUT THEY DEPEND ON
INTERACTIONS WITH THE
ENVIRONMENT, ON MANY TIME
SCALES, AND THE CONTEXT IN WHICH
THOSE HAPPEN.
AND SO THOSE ARE THE THINGS THAT
CAME UP TO ME WHEN THEY ASKED ME
TO PLEASE GIVE A TALK HOW WE'RE
GOING TO UNDERSTAND BRAIN
BEHAVIOR.
I'M GOING TO GIVE EXAMPLES AND
INSPIRATION, WHERE I DREW
INSPIRATION FROM.
AND TO EMPHASIZE WE REALLY WANT
TO HANDLE ALL THIS BIG DATA,
THIS DATA THAT'S GOING TO BE
DIFFERENT ACROSS CONTEXT, WE
REALLY HAVE TO UNDERSTAND
RULES OF THE STRUCTURES THAT WE
GET AND TO UNDERSTAND THE
VARIABILITY THAT WE'RE GOING TO
GET BECAUSE IT'S NOT GOING TO BE
AS RIGID AND AS MY ENGINEER
BRAIN WOULD LIKE.
WE'D LIKE TO IN PARTICULAR LOOK
AT SPATIAL TEMPORAL DATA THAT
ARE ON TIME SCALES OF NEURAL
ACTIVITY SO WE CAN ACTUALLY
BRIDGE MULTIPLE SCALES FROM
NEURAL ACTIVITY TO BEHAVIOR.
SO, ONE EXAMPLE THAT, YOU KNOW,
AS YOU'VE SEEN WHEN WE LOOK AT
MOVEMENTS, I CAN MEASURE JOINT
ANGLES AND THINGS LIKE THAT BUT
IN REALITY THERE'S MUCH MORE
INFORMATION THERE ABOUT ALL
SORTS OF SOCIAL ASPECTS AND
AFFECT, AS WE'VE HEARD, AND WE
CAN LOOK AT THE DATA AND BREAK
IT DOWN.
WE ALSO NEED TO START USING SORT
OF BIOPHYSICALLY BASED METHODS,
HOW DOES THAT EVEN HAPPEN IN THE
BODY, HOW DO WE GET BACK DOWN TO
MECHANISMS MORE?
HERE'S JUST AN EXAMPLE, WHERE
THE BODY DOESN'T CHANGE, WE KNOW
THIS IS A PHYSICS-BASED MODEL OF
MOVEMENT WHERE THE CONTROL OF IT
CHANGES, POTENTIALLY
BIOLOGICALLY PLAUSIBLE WAY SO A
PERSON CAN EXPRESS HAPPY OR SAT
WALKING ACROSS VARIOUS
BEHAVIORS.
WE WANT TO FIND PRINCIPLES, IT
DOESN'T MATTER WHAT KIND OF
MOVEMENT I'M DOING, THERE'S
SOMETHING COMMON THAT MAKES IT
HAPPY.
OR SAD.
AND TO THINK ABOUT THAT, I WANT
TO GO BACK TO A PRETTY OLD
EXAMPLE THAT I LIKE TO THINK
ABOUT, MY SLIDES ARE NOT
ADVANCING.
OKAY.
WHICH IS THINKING ABOUT ACCENTS,
WHEN WE TALK.
SO WHEN WE'RE -- THERE'S AN
OBJECTIVE SPACE HERE OF SOUNDS.
THIS IS ONE FREQUENCY AND
ANOTHER.
EACH OF THESE BOXES REPRESENTS
THE VOWEL SOUNDS WE CAN HEAR
WHEN WE'RE BABIES.
AND AS PEOPLE INTERACT WITH
THEIR ENVIRONMENT, THESE SPACES
KIND OF MERGE AND FUSE INTO NEW
SHAPES THAT ARE RELEVANT TO THE
WORLD.
SO, THEY ARE SORT OF CHUNKING
THEM INTO DIFFERENT TYPES OF
WHAT WE MIGHT CALL LATENT
VARIABLE, IN TODAY'S PARLANCE,
BUT WHAT IT MEANS IS THAT THERE
ARE OBJECTIVE -- THEY ARE BIASES
THAT HAPPEN WHEN PEOPLE SPEAK
THAT I CAN PREDICT IN OBJECTIVE
AND QUANTIFIABLE WAY.
ONE EXAMPLE IS IF YOU SPEAK
JAPANESE, THEN ALL OF THE SOUNDS
THAT FALL IN THIS REGION SOUND
THE SAME.
AND SO WHEN I'M LISTENING TO
ENGLISH, I CAN'T DISTINGUISH
THESE TWO SOUNDS, AND WHEN I'M
SPEAKING THEM I MIGHT SAY RA AND
LA, CONFUSE THOSE TWO SOUNDS.
THOSE ARE THINGS WE CAN ACTUALLY
MEASURE HAPPENING IN THE BRAIN,
HOW THE BRAIN IS WARPING THE
OBJECTIVE INFORMATION, AND THEN
THE CONTEXT MATTERS TOO.
WE CAN'T THINK OF THESE
LOCATIONS AS BEING FIXED IN THIS
OBJECTIVE SPACE, BUT WHEN A
MOTHER SPEAKS TO A CHILD THE
DISTANCE BETWEEN THE POINTS
STRETCHES.
WE NEED TO LEARN THE RULES BY
WHICH THESE CLUSTERS OR UNITS OF
ACTION MOVE AROUND WITH
INDIVIDUAL, WITH CONTEXT, WITH
MOOD.
AND SO GETTING BACK EVEN FURTHER
TO SOMETHING, MORE LINKED TO
MOTOR OUTPUT, IN MY LAB WE
LOOKED AT MUSCLE ACTIVITY, THE
OUTPUT OF THE MOTOR NEURONS, AND
DEMONSTRATED THAT IN MOVEMENT
THERE ARE -- EACH PERSON HAS
LIKE A LIBRARY OF MOTOR MODULES
SIMILAR TO THESE FACIAL THINGS
WE HEARD ABOUT AND WE CONSTRUCT
MOVEMENTS BASED ON COORDINATING
A SET OF MUSCLES ACROSS LIMBS IN
ORDER TO PERFORM CERTAIN
BIOMECHANICAL TASKS.
I WAS LOOKING FOR SOMETHING
OBJECTIVE HERE AND HAD TO
CONCEDE DEPENDING ON THE ANIMAL
I WAS LOOKING AT OR THE PERSON
I'M LOOKING AT, TO PERFORM THE
SAME TASKS WHICH MIGHT BE
PRODUCING A FORCE EACH ANIMAL
SORT OF COORDINATES THEIR SETS
OF MUSCLES IN SLIGHTLY DIFFERENT
WAYS.
WHEN WE LOOKED ACROSS PEOPLE,
THESE DIFFERED DEPENDING IF
YOU'RE A BALLET DANCER OR HAVE
GONE THROUGH REHABILITATION, OR
IF YOU HAVE DIFFERENT NEUROLOGIC
DISORDER.
PERHAPS THAT'S ONE OF THE
MECHANISMS BY WHICH WE CAN
IDENTIFY DIFFERENT PEOPLE, JUST
BY THE WAY THEY WALK.
BUT THUS FAR WE DON'T HAVE A
DIRECT LINKAGE BETWEEN THESE
SORT OF VERY STATIC CARTOONS OF
DIFFERENT WAYS IN WHICH PEOPLE
ACTIVATE THEIR MUSCLE ACTIVITY
OR DIFFERENT WAYS IN WHICH THEY
MIGHT BE IMPAIRED AT MOVEMENT,
AND ONE WAY THAT MANY PEOPLE
HAVE BEEN TRYING TO IS TO NOW
PUT THESE PATTERNS ON A
BIOPHYSICAL MODEL OF WALKING.
THIS APPROACH IS LACKING IN THAT
WE DON'T KNOW ALL OF THE NEURAL
CONSTRAINTS AND NEURAL DYNAMICS,
EVEN IF WE HAD A CONNECTOME, IT
WOULD BE VERY, VERY
UNDERSPECIFIED.
SO, WE'VE RECENTLY WORKED WITH
GORDON BERMAN TO TAKE THIS MORE
DATA-DRIVEN APPROACH TO SAY I
CAN SEE THESE DIFFERENCES IN THE
MOVEMENTS, IN THESE KINEMATIC
PATTERNS.
WHEN I COMPARE IN TRADITIONAL
BIOMECHANICAL SENSE EVERYBODY
LOOKS THE SAME WHEN THEY WALK.
THERE'S MORE INFORMATION THAT
OUR BRAINS ARE GETTING OUT THAT
WE'RE NOT CAPTURING IN THE WAY
THAT WE TREAT BIOMECHANICS DATA.
AND SO WE PUSH IT THROUGH A
NEURAL NETWORK, AND THEN TO
CAPTURE THE SORT OF DYNAMICS,
THAT IS THE SPATIAL AND TEMPORAL
DEPENDENCIES WITHIN DATA, IN A
WAY THAT WE CAN COMPARE
DIFFERENT INDIVIDUALS, AND SHOW
THESE PEOPLE WALK AT THE SAME
SPEED ARE ACTUALLY WALKING AT
DIFFERENT SPEEDS, THAT'S THE
SHADING, MAINTAIN SOME
CHARACTERISTICS OF THE MOVEMENT
THAT IS THEIRS.
AND PERHAPS BY NOW LOOKING AT
THESE TWO TYPES OF MODELS, OF
MOVEMENT, WE CAN HOPE TO LINK
THIS EMERGENT BEHAVIOR WE SEE
WITH SOME MORE CONNECTIVENESS
TIMES OF TIMES OF PATTERNS THAT
MIGHT INDICATE SOMETHING AT A
LOWER LEVEL IN THE NERVOUS
SYSTEM.
THINK THIS KIND OF APPROACH IS
GOING TO BE NECESSARY WHEN WE
ARE TALKING ABOUT, OH, HOW DO WE
DO MULTI-DISCIPLINARY RESEARCH
ACROSS SPECIES AND SCALE, AND TO
THINK ABOUT, OH, WELL, HOW
NATURALISTIC BEHAVIORAL STUDIES
TO SORT OF GATHER DATA AND MAKE
HYPOTHESES AND EXPERIMENTAL
STATE WHERE WE MANIPULATE
BEHAVIORAL CONTEXT, WE HAVE TO
BE ABLE TO UNDERSTAND THE
STRUCTURE OF THE VARIATIONS THAT
WE'RE GOING TO SEE AT ALL
LEVELS.
AND THEN ULTIMATELY AGAIN IF
PEOPLE SAID TO USE TECHNOLOGIES
TO ACTUALLY PERTURB THE CAUSAL
MECHANISMS INVOLVED.
I WANT TO REALLY TALK ABOUT
REALLY WHY THIS IS SO IMPORTANT.
RECENTLY INTERVIEWED ABOUT THE
YIPS, YOU MAY HAVE SEEN AT THE
OLYMPICS, ATHLETES UNABLE TO
PERFORM THE MOTOR TASK IN A
PARTICULAR CONTEXT, IT TURNS OUT
THERE ARE HYPOTHESES ABOUT THESE
BEING BOTH BOTTOM-UP, SOMETHING
ABOUT SPINAL NETWORKS GOING
WRONG, ALL THE WAY UP TO
PREFRONTAL, SORT OF COGNITIVE
REASONS.
AS WELL AS LIFE HISTORY LIKE
EARLY CHILDHOOD TRAUMA,
GENETICS, CAN CONTRIBUTE TO
THESE BEHAVIORS.
AND SO THIS IS WHAT HAPPENS WHEN
YOU HAVE TO MOVE IN A REAL
SITUATION.
IT'S JUST NOT ABOUT CONTROLLING
YOUR MUSCLES.
IT ALSO HAS RELEVANCE IN
CLINICAL DISORDERS, FREEZING OF
GAIT, PARKINSON'S DISEASE,
SIMILAR THAT PEOPLE CAN WALK
MOST OF THE TIME, DIFFERENT
INDIVIDUALS HAVE COGNITIVE MOTOR
AND EMOTIONAL TRIGGERS THAT MAKE
THEM UNABLE TO ACTUALLY WALK,
WALK AND THEY FREEZE.
AND SO WHEN WE THINK HOW DO WE
STUDY THAT IN AN ANIMAL AND MAKE
INFERENCES, THERE'S BEEN SOME
WORK BY STEVE CHASE'S GROUP, SET
SET UP EXPERIMENTAL SITUATION
WHERE MONKEYS WILL CHOKE,
DEPENDING ON THE STAKES OF THE
AS -- SCENARIO.
THAT'S ONE WAY TO LOOK AT IT.
IN MY LAB WE'VE JUST STUDIED ONE
BEHAVIOR, WHICH IS STANDING
POSTURAL CONTROL, AND STARTED TO
MEASURE BRAIN ACTIVITY AND BASED
ON SOME EPIDEMIOLOGICAL WORK
SHOWING COGNITIVE STATE
INFLUENCES FALLS WE'VE BEEN
SHOWING THE ACTIVITY EVOKED
DURING THIS VERY UNECOLOGYICAL
IS MODULATED BY AFFECT.
THE DIRTY SECRET, WE DON'T
MEASURE THEM, DON'T SAVE THEIR
DATA WHEN THEY ARE HAVING A BAD
DAY BECAUSE WE DON'T FLOW HOW --
KNOW HOW TO TAKE THAT INTO
ACCOUNT WHEN COMPARING DATA AND
HAVE TO RECOGNIZE BOTTOM-UP
MECHANISMS THAT WE'RE LOOKING AT
LIKE EVEN THE SENSORY
INFORMATION THAT'S COMING UP
FROM THE PERIPHERY IS MODULATED
BY MOTOR COGNITIVE STATE AND
TRAINING.
AND SO I WANTED TO SAY
NEURODIVERSITY AND VARIABILITY
IS REALLY IMPORTANT TO
UNDERSTAND AND NOT TRY TO FIND
SOME UNIQUE MAPPINGS BETWEEN
BRAIN AND BEHAVIOR, AND THAT BY
UNDERSTANDING THE STRUCTURE WE
CAN HOPE TO FIND PRINCIPLES THAT
CAN GO ACROSS LOTS OF BEHAVIORS,
SCALES, AND SPECIES.
SO, I'M ENCOURAGING PEOPLE TO
COLLABORATE, JUST LIKE THIS
WORKSHOP.
THANK YOU.
>> THANK YOU.
SO WE'RE NOW MOVING FROM THE
INDIVIDUAL TALKS TO THE
DISCUSSANTS.
FIRST DISCUSSANT IS MOLLY
CUMMINGS.
>> THANK YOU.
>> GO AHEAD.
>> OKAY.
I WAS GOING TO THANK ALL THOSE
AMAZING SPEAKERS.
IT'S HAVEN A REALLY INTERESTING
SET OF FOUR SPEAKERS.
AND -- WELL, ACTUALLY FIVE
SPEAKERS.
THAT WAS AN AMAZING INITIAL
KICKOFF INTRODUCTION.
I WANT US TO NOTE A FEW THINGS
THAT WERE THROUGHOUT THE FOUR
SPEAKERS.
THE FIRST -- TWO OF THE SPEAKERS
FOCUSED ON MOVEMENT PROVIDED
INCREDIBLE MULTI-MODAL DEEP
METRICS IN THE HUMAN
PERSPECTIVE, DR. TING JUST
SHARED, A COUPLE ANIMAL -- OTHER
ANIMAL MODEL SPECIES, ZEBRAFISH,
RATS, INSECTS, DROSOPHILA, THAT
DR. COUZIN SHARED, AND TWO
SHARING DATA ON INTERNAL STATES,
TRYING TO FIND BIOBEHAVIORAL
MARKERS TO PREDICT INTERNAL
STATES.
AND IT SEEMED WHILE WE HAD GREAT
INFORMATION ON INTERNAL STATES,
WE HAD RELATIVELY FEW REALISTIC
BIOBEHAVIORAL MARKERS TO PREDICT
THAT WELL, AND DR. COHN'S
EXAMPLE WAS FANTASTIC WHERE HE
WENT FROM ONLY BEING ABLE TO
PREDICT 8 TO 17% OF VARIATION IN
OCD, UP TO 60% WHEN STARTING TO
TAKE IN GREATER REPERTOIRE OF
MULTI-MODAL INPUTS INTO THAT.
AND I WANTED US TO TALK, THERE
WILL BE MULTIPLE PANELISTS
BRINGING UP DIFFERENT QUESTIONS,
BUT I WANTED TO REALLY EMPHASIZE
THE ENDING THEME FROM DR. TING'S
TALK, HOW DO WE IDENTIFY RULES
AND PRINCIPLES AND HOW WE HAVE
TO GO ACROSS MULTI-SCALES IN
DIFFERENT SPECIES TO DO THAT.
AND AS A NEUROETHNOLOGYIST, I
WANT TO THROW OUT THE QUESTION
ARE WE AT A POINT WHERE WE NEED
TO ABANDON STUDYING THE
SPECIALIST, THAT WOULD BE
INVOKING A DANISH PRINCIPLE FOR
FROM A CENTURY AGO, THERE'S ONE
WE SHOULD BEST STUDY TO FIND THE
ANSWER, AND DO WE WANT TO CHANGE
TACTIC AND LOOK MORE AT
GENERALIST OR PERHAPS, I WOULD
BE IN FAVOR OF THIS, GOING
EXTREMELY COMPARATIVE, WE OFTEN
PAT OURSELVES ON THE BACK IF WE
COMPARE TWO SPECIES.
BUT IT SEEMS JUST AS DR. TING
SHOWED US THERE'S MULTIPLE WAYS
TO SOLVE A MOVEMENT PATTERN, CAN
WE LOOK ACROSS MULTIPLE SPECIES
OF BRAINS, DIFFERENT SPECIES
WITH DIFFERENT BRAINS THAT SOLVE
THE SAME PROBLEM IN DIFFERENT
WAYS, TO ENRICH OUR ABILITY TO
SOLVE PROBLEMS AT THE HUMAN
FRONT.
I WANT TO THROW THAT OUT THERE
TO START.
AND IF ANYONE FEELS LIKE CHIMING
IN ON THAT, I THINK THE IDEA IS
YOU UNMUTE YOURSELF.
THIS IS NOT MEANT FOR ME TO TALK
FOR EIGHT MINUTES.
>> SHOULD WE PUT OUR VIDEOS ON?
>> YES PLEASE.
>> OR SHOULD WE STICK WITH
SPECIALIZING?
ARE THERE THINGS THE RAT CAN
TELL US MORE AND WE SHOULD BIG
DEEPER INTO THE RAT OR
DROSOPHILA TO FIND INSIGHT INTO
HUMANS?
>> LET ME START BY SAYING AFFECT
DECISION-MAKING MOVEMENT AND
LIKE AFFECT ARE NOT SPECIAL TO
HUMANS.
SO SOMETHING HAS TO MATTER TO
YOU TO BE ABLE TO WANT TO MAKE A
DECISION.
SO IN THAT SENSE WE'RE GOING TO
FIND IT IN MAY DIFFERENT SPECIES
BUT TO DIFFERENT EXTENTS.
>> IT'S INTERESTING THAT THE TWO
TALKS THAT USED DBS, YOU KNOW,
THEY ARE STIMULATING PARTS OF
THE BRAIN EVERY VERTEBRATE HAS.
THERE ARE WAYS TO FIGURE THIS
OUT WITH DBS AT THE LEVEL OF
CROSS-LINEAGE COMPARISON.
ON THAT THEME, I WANTED TO,
AGAIN DRAWING FROM DR. TING'S
WORK, WHERE SHE FOUND THAT
DIFFERENT ANIMALS, DIFFERENT
HUMANS, USE DIFFERENT
COORDINATED MODULES TO PRODUCE
THE SAME WALKING SPEED.
I THINK BY BECOMING MORE
COMPARATIVE, AND LOOKING ACROSS
BRAINS, THAT HAVE SOLVED
WHATEVER PROBLEM WE ARE FACING,
CHANGING ONE INTERNAL STATE TO
ANOTHER INTERNAL STATE, WE CAN
ACTUALLY IDENTIFY DIFFERENT
BRAIN REGIONS AND PATHWAYS TO DO
THAT.
WHETHER OR NOT THE SOLUTION ENDS
UP BEING DBS OR THE SOLUTION
ENDS UP BEING PHARMACEUTICALS,
THAT CAN STIMULATE DIFFERENT
NEUROMODULATORY PATHWAYS TO
AFFECT OUTCOME IS ANOTHER WAY TO
LOOK AT THAT.
>> I WANT TO BE CLEAR THOUGH
THAT'S WHAT WE SEE IS SHAPED
OVER LONG PERIODS OF TIME AND
LEARNING AND REHABILITATION.
SO THERE'S GOING TO BE -- I
THINK THIS HAPPENS IN DBS OR
DEPRESSION TOO, INSTANTANEOUS
CHANGE THAT MIGHT HAPPEN WITH
NEUROMODULATION THAT ENABLES A
LONGER-TERM SORT OF SLOWER TIME
CONSTANT OF CHANGE IN THE UNITS
THAT SHAPE SOMEBODY'S BEHAVIOR.
>> RIGHT.
KAREN ROMMELFANGER.
>> THANKS.
I HAD A REALLY -- I ENJOYED THE
OPENING PANEL.
I HAVE CURIOSITY AROUND JEFF
CON'S WORK WHERE HE NOTED AFAR
WAS OPTIMIZED ACROSS CULTURES
AND YET ALSO CLOSED WITH AN
OPPORTUNITY FOR US TO EXPLORE
MORE CONTEXT NEEDED FOR
EXPLORING HOW THIS TECHNOLOGY
WOULD WORK AND THAT MORE DATA
WOULD BE NEEDED TO OPTIMIZE
TECHNOLOGY.
CAN YOU SHARE ABOUT BIAS,
SOCIALLY CONSTRUCTED IDENTITIES,
THE DEGREE OF CONFIDENCE IN
INTERPRETATIONS ACROSS
BEHAVIORS, SUCH AS WE FOCUSED ON
MOOD BUT OPPORTUNITIES FOR
LOOKING AT PAIN AND OTHER
SUBJECTIVE STATES.
AND WRAPPING THAT TOGETHER, WHAT
ARE THE OPPORTUNITIES AND
CHALLENGES FOR TECHNOLOGY LIKE
AFAR IN THE BRAIN BEHAVIOR
SYNCHRONIZATION MODEL IN LIGHT
OF THOSE KINDS OF QUESTIONS
AROUND BIAS OPPORTUNITIES AND
LIMITATIONS.
>> RIGHT.
THANKS FOR THE QUESTION.
THE-- AFAR IN DETECTING ACTION
TUNES, ACTION OPPORTUNITIES ARE
DESCRIPTIVE MOVEMENTS IN A WAY
THAT WALKING IS A MOVEMENT.
SO THERE'S -- IT'S PRIMARILY
DESCRIPTIVE OTHER THAN INFERENCE
CONSTRUCT.
GENDER, CULTURAL IDENTITY, OR
AFFECT, THOSE ARE HIGHLY
INFERENTIAL.
WE NEED SOME BRIDGING TO SUPPORT
INFERENCES ABOUT THE MEANING OF
ACTION UNITS.
NOW, WITH RESPECT TO DESCRIPTIVE
MEANING WE DO ACTION UNIT
INPUTS, INFANT HAVE
DIFFERENT-SHAPED FACES,
DIFFERENT MOVEMENTS, YET THE
SYSTEM WORKS, BECAUSE IT'S
PRIMARILY ANATOMIC AND TRAINED
ON INFANTS.
FOR PEOPLE WITH VERY DARK SKIN,
AFAR TO THE EXTENT WE'VE LOOKED,
SO FAR IT APPEARS TO WORK WELL.
IT DEPENDS ON THE AMOUNT OF
LIGHTING.
BUT, YOU KNOW, IT'S REALLY --
WHEN YOU HAVE STIMULATION IN THE
VENTRAL STRIATUM, THAT SUPPORTS
INFLUENCE YOU'RE ELICITING
POSITIVE AFFECT, AND SO I MAKE
THAT INFERENCE, BUT I'M MAKING
IT FROM WHAT I KNOW ABOUT
PARTICULAR COMBINATION OF ACTION
UNITS THAT HAVE BEEN, YOU KNOW,
STUDIED BROADLY IN AFFECTIVE
CONTEXT.
>> THANKS, JEFF.
WE NEED TO MOVE TO THE NEXT
DISCUSSANT BUT I THINK THERE
WERE SEVERAL QUESTIONS THAT WE
CAN GENERALIZE ABOUT WHEN WE'RE
SETTING BEHAVIOR, IN WHAT
SITUATIONS, AS HUMANS, DO WE
NEED TO THINK MORE CAREFULLY HOW
WE DO THAT BASED ON RACE OR
ETHNICITY OR SEX OR GENDER.
NOW WE'RE GOING TO MOVE TO
TIMOTHY WRIGHT.
>> THANK YOU.
I WANT TO THANK MOLLY FOR
STARTING US OFF.
DAMON, THANK YOU FOR PUTTING
THAT UP.
I WONDER IF WE COULD SEE THE
WHOLE PANEL RIGHT NOW AND THEN
TOWARDS THE END OF MY TIME I'LL
ASK A QUESTION SPECIFICALLY
ABOUT THAT SLIDE.
THANK YOU VERY MUCH.
I WANT TO CONTINUE ON THIS THEME
OF AFFECT, AND DIFFICULTIES WE
MIGHT HAVE WITH SYSTEMS THAT ARE
DEVELOPED FOR ONE SPECIES OR ONE
SUBSET OF A SPECIES.
AND THINK ABOUT HOW WE CAN
EXPAND THAT APPROACH INTO A
COMPARATIVE FASHION, THINKING
ABOUT OTHER SPECIES.
IT'S CLEAR PEOPLE WORKING WITH
HUMANS HAVE THOUGHT QUITE A BIT
ABOUT THE IMPORTANCE OF AFFECT.
BUT ANIMAL BEHAVIORISTS ARE NOT
BLIND TO THIS.
THERE'S WELL KNOWN EFFECTS IF A
CORRECT HAS LOST A FIGHT EARLY,
IT'S MORE LIKELY TO LOSE A FIGHT
LATER, EVEN IF IT'S A RELATIVELY
EASY OPPONENT.
BUT CLEARLY TRYING TO FIGURE OUT
WHAT AFFECTIVE STATE IS IN
ANIMALS IS A CHALLENGE.
I I WANT TO START WITH THE HUMAN
PANELIST, HUMAN ORIENTED
PANELISTS.
WE'RE ALL HUMAN I GUESS.
ASK WHAT POTENTIAL THEY SEE FOR
USING THESE SORTS OF APPROACHES
WITH ANIMALS AND THEN I WANT TO
MOVE ON TO IAN AND MOLLY AND
BENCE, WHO WORKED WITH ANIMAL
MODELS.
I'LL THROW THAT OUT TO NICOLE
AND JEFF AND LENA TO START WITH.
LENA HAS A HAND RAISED.
>> I'M NOT SURE ABOUT AFFECT,
THERE HAVE BEEN INTERESTING
STUDIES IN GUINEA FOWL BY MONICA
DALY, SHOWING A PERSONALITY
AFFECTS HOW THEY WALK AROUND THE
ROOM OR SLIDE OR HOW THEY CARE
ABOUT CRASHING INTO THINGS.
AND SO, YOU KNOW, I THINK YOU'RE
GOING TO SEE THAT IN MANY
ANIMALS, CATS THAT I TRAINED ALL
HAD DIFFERENT PERSONALITIES
WHICH AFFECTED THE MUSCLE
COORDINATION PATTERNS THEY USED
AND HOW THEY USED THEM.
>> OKAY.
ANY OTHER COMMENTS FROM NICOLE
OR JEFF?
>> I THINK ONE INTERESTING
CONCEPT -- WELL, SHIFTING FROM
THINKING ABOUT HUMANS TO
THINKING ABOUT ANIMALS YOU CAN'T
ASK ANIMALS HOW THEY ARE FEELING
OBVIOUSLY.
YOU CAN ASK HUMANS HOW THEY ARE
FEELING.
WE HAVE TO INFER EVERYTHING
ABOUT -- EVERYTHING WE ASSUME
AGUESS ABOUT HOW ANIMALS ARE
FEELING BASED ON THEIR BEHAVIOR.
I THINK THE IDEA OF LIKE DOING
THAT IN HUMANS IS REALLY COOL.
BUT ONE QUESTION I'VE BEEN
CHEWING ON FOR QUITE A WHILE,
I'M SURE PEOPLE HAVE THOUGHTS
ABOUT THIS, IS WHAT COMES FIRST,
LIKE WHEN SOMEONE IS RECOVERING
OR GETTING BETTER, IF THEY HAVE
DEPRESSION, IS IT BEHAVIOR
CHANGES AND THEN THEY FEEL
BETTER, DOES IT HAPPEN AT THE
SAME TIME OR DO THEY FEEL BETTER
AND THEIR BEHAVIOR CHANGES?
>> LENA, YOU KNOW, I WOULD SAY
WE GIVE SPECIAL STATUS TO
FEELING, SUBJECTIVE EXPERIENCE.
IT'S THE SINE QUA NON OF
EMOTION.
BUT IT'S REALLY JUST ONE ASPECT.
AND IT'S ONE THAT'S VERY HARD TO
KNOW WHAT ANOTHER -- IT'S VERY
HARD TO KNOW WHAT YOUR PET IS
FEELING, VERY LARD -- HARD TO
KNOW, BUT THERE ARE OTHER
ASPECTS SUCH AS REACTION
TENDENCIES, SO THE CRICKET THAT
LOSES A FIGHT AND THEN GOES ON
TO LOSE AN EASY FIGHT, YOU KNOW,
THAT SEEMS LIKE AN AFFECTIVE
RESPONSE.
IT'S AN EMOTIONAL RESPONSE THAT
THE EXPERIENCE OF LOSING THAT
INITIAL FIGHT AFFECTS ITS
ABILITY TO MOTIVATE COORDINATED
AGGRESSIVE AFFECTIVE BEHAVIOR IN
CONFLICT.
THAT'S, YOU KNOW, THAT'S
EMOTION.
I DON'T KNOW WHAT THE CRICKET IS
FEELING, BUT I WOULD SAY THAT
IT'S NOT HIGH ON APPROACH, IT'S
-- YOU KNOW, IT'S NOT EXHIBITING
MASTERY OF WHICH ITS CAPABLE.
THESE ARE AFFECTIVE RESPONSES.
>> I SEE A HAND RAISED EARLIER
FROM YOU, IAN?
>> YEAH, IT'S CONTEXT, BUT I
COULD IF IT'S INTEREST BUT
OTHERWISE LET PEOPLE SPEAK.
>> I'LL CHIME IN FROM THE
NON-HUMAN ANIMAL
CHARACTERIZATION OF BEHAVIOR.
>> THAT WAS THE ANGLE WAY GOING
TO TAKE, MOLLY.
>> I WAS GOING TO CITE YOUR WORK
AS WELL AS PITCH SIMILAR -- HOW
WE ACTUALLY CAPTURE INTERNAL
STATES AND AFFECTIVE INTERNAL
STATE OF A NON-MODEL ORGANISM AT
THIS POINT IS DOING GENOMIC
PROFILING OF THE BRAIN.
IAN GAVE A GREAT EXAMPLE, YOU
CAN PAIR BEHAVIOR WITH A
TRANSCRIPTOMIC PROFILE AND
UNDERSTAND, YOU KNOW, WHAT'S
GOING ON, DYNAMICALLY AT THE
BRAIN, NOT AT NEURAL STIMULATION
BUT GENE PATHWAYS THAT ARE BEING
EVOKED IN THE LAST 30 MINUTES OF
THE ANIMAL'S LIFE OR LESS THAN
THAT.
AND SO WE ACTUALLY CAN IDENTIFY
GENE SUITES THAT DO DIVERGE AND
WE DON'T KNOW IF TERMS OF
AFFECTATION EXACTLY WHAT THEY
REPRESENT BUT WE COMPARE THAT
WITH MOVEMENTS WHICH WE CAN
SURMISE HAVE SOMETHING TO DO
WITH EITHER FEAR OR EXCITEMENT
OR OTHER CONTINUUMS OF BEHAVIOR.
SO THAT'S HOW WE DO IT INSTEAD
OF SELF-REPORTING.
WE LOOK AT WHAT'S GOING ON IN
THE BRAIN.
NOT AT THE LEVEL OF NEURONS BUT
GENOMIC PROFILING OFTEN.
>> I THINK THIS PROBLEM OF
BRIDGING FROM HUMAN STUDIES TO
ANIMAL STUDIES IS REALLY KEY AND
CENTRAL, AT LEAST FOR WHAT I DO.
IAIN SAID HE WAS STUDYING AUTISM
IN IN FISH, WE STUDY IN RATS.
HOW DO YOU KNOW WHAT YOU SEE IN
A HUMAN PATIENT IS NEAR WHAT YOU
SEE IN YOUR ANIMAL SO I DON'T
KNOW WHAT MECHANISMS WE HAVE FOR
FACILITATING COMPARISONS BETWEEN
THE HUMAN CONDITION AND VARIOUS
ANIMAL BEHAVIORS.
BUT PERHAPS ONE APPROACH IS TO
STUDY RATHER LOW LEVEL
MANIFESTATIONS OF HIGH LEVEL
PHENOMENA WHICH CAN BE COMPARED
ACROSS SPECIES AND I'M THINKING
HERE OF MOVEMENTS IN TERMS OF
AUTONOMIC NERVOUS SYSTEM
FUNCTION BECAUSE THAT GIVES US
AT LEAST SOMETHING TO COMPARE,
MORE OF AN APPLES-TO-APPLES
MANNER, PERHAPS USE SIMILAR
TECHNIQUES FOR MEASURING AND
SIMILAR ANALYTICAL FRAMEWORKS,
SO THAT COMPARISONS CAN BE MADE
MORE EASILY.
>> I DON'T KNOW IF THAT WAS THE
BEGINNING OF YOUR ACTUAL FORMAL
DISCUSSION COMMENTS BUT IT'S
TIME FOR YOU THIS TRANSITIONS
ORGANICALLY.
>> IT'S A BIG QUESTION I'VE
STRUGGLED WITH, LISTENING TO THE
SPEAKERS, I REALIZE THERE'S A
GULF IN APPROACH, AND MINDSET,
AND TECHNIQUES, AND FRAME WORKS
BETWEEN THOSE WHO STUDY DISEASE,
HUMAN DISEASE IN A CLINICAL
MANIFESTATION AND THOSE OF US
WHO DO MECHANISTIC STUDIES IN
ANIMALS WE TRY TO SIMPLIFY
THINGS AS MUCH AS WE CAN AND
MAKE INFERENCES ABOUT WHAT WE
SEE TO COMPLEX HUMAN DISORDERS.
I DON'T HAVE A PARTICULARLY GOOD
REMEDY FOR THAT BUT I WOULD LOVE
TO HEAR FROM THOSE WHO STUDY
HUMAN DISEASE OR DISORDER, IF
YOU HAVE ANY GUIDELINES OR
CRITERIA FOR VALIDITY OF THE
VARIOUS MODEL ORGANISMS THAT WE
STUDY, YOU KNOW, IS IT AS MOLLY
SAID, IS THERE A PARTICULAR
ANIMAL MODEL THAT'S PARTICULARLY
WELL SUITED TO STUDY OCD OR
DEPRESSION?
AND IF SO, WHAT ARE THOSE
CRITERIA, HOW SHOULD WE THINK
ABOUT THIS?
>> IAIN, DO YOU WANT TO ANSWER
THIS?
>> I GUESS I PUT MY HAND DOWN
AND UP AGAIN.
I THINK YOU MAKE A REALLY GOOD
POINT.
SO TO MY READING, MOST OF THE
WORK, PRE-CLINICAL WORK THAT'S
BEEN DONE ON, SAY, PSYCHIATRIC
ILLNESS, HAS BEEN DONE FOR
HUMANS, A VERY GOOD REASON.
I THINK WITH RESPECT TO
ZEBRAFISH, YOU HAVE A HIGHER
DEGREE OF CONTROL, CAN PUT THEM
IN VIRTUAL REALITY, INFER
CAUSALLY, WHICH ALGORITHMS ARE
MISFUNCTIONING.
YOU DETERMINE PRECISELY WHICH
ALGORITHMS WORK AND
COLLABORATING WITH PEOPLE LIKE
FLORIAN AT YOUR UNIVERSITY WE
CAN POTENTIALLY LOOK AT CIRCUITS
THAT MIGHT BE AFFECTED.
BUT I WILL SAY THIS, LIKE A
FISHING EXTRADITION, IT'S NOT
THE TYPE OF SCIENCE I LIKE TO
DO.
THE MORE WE LOOK INTO IT, IT'S
LIKE A SCREENING FOR JUST TO SEE
IF WE GET LUCKY.
BUT THE MORE WE LOOK INTO IT THE
MORE WE REALIZE THAT WE MIGHT
GET LUCKY.
CAN YOU HEAR ME NOW?
>> NOW I CAN HEAR YOU.
>> YEAH, SORRY.
THERE ARE THESE BASAL SOCIAL
BEHAVIORS, AT LEAST, THAT WE
COULD STUDY.
I THINK THE ANSWER TO YOUR
QUESTION IS WE DON'T KNOW IF
IT'S GOING TO BE SO, WE DON'T
KNOW IF IT'S GOING TO BE
CLINICALLY USEFUL.
ZEBRAFISH ARE HIGHLY COMPLEX IN
SOCIAL BEHAVIORS.
PEOPLE HAVE MISUNDERSTOOD
COMPLETELY HOW SOPHISTICATED
THEY ARE BECAUSE THEY LOOK AT
THEM IN AN ENVIRONMENT THAT
DOESN'T ALLOW THEM TO BE
SOPHISTICATED.
THEY MIGHT BE A GOOD MODEL.
AND I WOULD SAY ONE OF THE
THINGS I WAS SAYING IN MY TALK,
YOU KNOW, DROSOPHILA AND C.
ELEGANS, YOU PROBABLY NEVER
REALLY WILL BE ABLE TO
UNDERSTAND THEM IN THE FIELD.
RATS AND ZEBRAFISH, WE CAN.
WE CAN DO THAT WITH FISH, WITH
RATS, YET ALMOST NO ONE IS
WORKING WITH RATS OR ZEBRAFISH
IN THE FIELD.
WHY NOT?
MAYBE WE NEED TO PUSH THIS
CONNECTION BETWEEN THESE SCALES.
AND I DON'T SEE A PROBLEM WITH
USING A VERY STRONG MODEL
ORGANISM LIKE A RAT OR ZEBRAFISH
TO MAKE THESE POWERFUL
CONNECTIONS ACROSS SCALE BECAUSE
WE CAN'T JUST RANDOMLY JUST -- I
THINK THE OTHER OPTION, THE
OTHER THING THAT WAS MENTIONED,
CHOOSE AN ANIMAL FOR ITS
SPECIALIST CAPABILITIES.
MOLLY MENTIONED THAT.
ABSOLUTELY RIGHT.
FULLY AGREE.
COMPLETELY AGREE.
REALLY IMPORTANT.
WE NEED TO HAVE NON-MODEL
ORGANISMS, ABSOLUTELY.
BUT WITH THE MODEL ORGANISMS WE
DO HAVE, CAN WE MAYBE CONSIDER
PROGRAMS THAT SPAN FROM THE WILD
TO THE LAB BECAUSE I THINK THE
TYPES OF TECHNOLOGY I'VE BEEN
HEARING ABOUT TODAY COULD BE
APPLIED IN THE NEAR FUTURE IF
NOT NOW TO NATURALISTIC
CONDITIONS OR EVEN NATURAL
CONDITIONS, I THINK
UNDERSTANDING HOW RATS BEHAVE IN
THE WILD WOULD BE PHENOMENAL.
THAT'S MY FEELINGS, I THINK WE
COULD COMBINE THIS IN A VERY
SYNERGISTIC WAY.
>> MOLLY?
>> WE'LL HAVE TO END SOON.
>> I WAS JUST GOING TO POSIT
MAYBE WE SHOULD GET AWAY FROM
SPECIALISTS AND FIND THE LOWEST
COMMON DENOMINATOR, IF WE STAY
WITH DEPRESSION, SOME KIND OF
SIMPLE BEHAVIORAL MARKER FOR
WITHDRAWAL AND THEN IDENTIFY THE
SAME COMPONENTS THAT ARE
OPERATING OR DIFFERENT
COMPONENTS THAT ARE OPERATING
ACROSS BRAINS IN DIFFERENT
LINEAGES.
I THINK THAT MIGHT BE A SIMPLER
WAY TO FIND A TREATMENT.
IAIN, YOUR SYSTEM IS AMAZING BUT
NOT EVERYBODY CAN CREATE HOLE
HOLE GRAPH -- HOLOGRAPHICS AND
I'M WONDERING IF WE COULD GO
THE OPPOSITE DIRECTION.
>> I TOOK THE SLIDE OUT BECAUSE
OF TIME BUT I THINK
DEMOCRATIZING TECHNOLOGIES,
MAKING THEM CHEAP AND EASY FOR
EVERYBODY IS A HUGE PRIORITY FOR
US.
I WOULD BE SURPRISED IF IN FIVE
YEARS' TIME WE CAN'T PRODUCE A
SYSTEM THAT ANYONE CAN USE FOR
THIS TYPE OF TECHNOLOGY.
MOLLY, I ALSO FULLY AGREE WITH
WHAT YOU SAID.
HAVING THAT EVOLUTIONARY
PERSPECTIVE, BUT THEY ARE NOT
MUTUALLY EXCLUSIVE.
ONE CAN LOOK INTO DIVERSITY OF
BIODIVERSITY TO REALLY
UNDERSTAND THINGS.
>> LAST COMMENT.
>> HI.
YES, I WOULD ALSO LIKE TO ECHO
WHAT MOLLY JUST SAID.
I THINK LOOKING AT WIDER RANGE
OF SPECIES IS VERY IMPORTANT.
IN PARTICULAR, I THINK WE COULD
BE A BIT MORE STRATEGIC IN
PICKING THE SPECIES.
IN PARTICULAR, WE DON'T WANT
TO -- I THINK WITHOUT DENYING
THE VALUE OF CONVERGENT
EVOLUTION, WE DON'T WANT TO
STUDY SOCIAL BEHAVIOR, FOR
EXAMPLE, IN BEES, MAY GIVE US A
LOT OF INTERESTING INSIGHT, BUT
IT'S DEFINITELY EVOLVED
INDEPENDENTLY FROM SOCIAL
BEHAVIOR AND HUMANS.
ALSO PROBABLY IN THE CASE OF
OTHER ANIMALS, DOGS, WOLVES, ET
CETERA, SO I THINK WE CAN -- I
THINK WE SHOULD PICK SPECIES ON
THE BASIS OF THE WINDOW THEY
GIVE US INTO SHARED MECHANISMS
THAT CONCERN US.
SO, IN THAT SENSE, SPECIES THAT
DIVERGE FROM US AT A PARTICULAR
MOMENT IN THE PAST BUT THEN DID
NOT CHANGE TOO DRAMATICALLY.
SO IN THAT SENSE ZEBRAFISH ARE
WONDERFUL.
BUT THEY REALLY DID CHANGE A
LOT.
THEY DUPLICATED THEIR GENOME,
CHANGED THEIR FOREBRAIN
DEVELOPMENT DEVELOPMENTALLY,
CAME UP WITH A LOT OF THINGS
THAT ARE SHARED COMMON ANCESTORS
DID NOT HAVE.
BIRDS LIKEWISE, DEVELOPED ALL
KINDS OF AMAZING THINGS BUT
TOTALLY INDEPENDENTLY.
I THINK WE GET MORE OUT OF
STUDYING SPECIES THAT SHALL
HIGHLY NEGLECTED.
BASAL VERTEBRATES LIKE SHARKS
THAT DID NOT CHANGE SO MUCH, IF
WE PICK THE RIGHT SPECIES.
LIZARDS, OTHER REPTILES.
I THINK -- I'M BLOCKING ON HIS
NAME.
SAID RECENTLY THAT IN
NEUROSCIENCE WE USED TO STUDY
MANY MORE SPECIES.
NOW WE'RE BECOMING MONKEYS,
RATS, ZEBRAFISH AND DROSOPHILA.
AND I THINK WE NEED TO LOOK VERY
BROADLY.
I THINK WE CAN PICK SPECIFIC
SPECIES, BASED ON PHYLOGENETIC
RELATIONSHIPS WHILE STILL
B RIG -- BEING BROAD.
I WOULD LIKE TO ACKNOWLEDGE PART
OF THE REASON WE SPECIALIZE IS
SIMPLY BECAUSE THE MORE PEOPLE
STUDY A CERTAIN SPECIES, THE
MORE EASY IT IS.
I STUDY RHESUS MONKEYS BECAUSE
SO MANY OTHER PEOPLE DO, I CAN
DRAW UPON THAT KNOWLEDGE.
IT WOULD BE USEFUL TO DO
SOMETHING MORE LIKE, FOR
EXAMPLE, WHAT LEAH KRUBITZER
DOES OR JOHN KOZ, MANY SPECIES
OF PRIMATES, INSIGHT INTO THINGS
RELEVANT TO HUMANS.
>> I WANT TO SAY A COUPLE
THINGS.
WE HAVE TO WRAP UP.
ONE, SOUNDS LIKE THERE MAY BE
SOME MORE ROOM FOR DISCUSSION
ABOUT WHICH SPECIES WE WANT TO
BE STUDYING AND WHY BUT
AGREEMENT STUDYING MULTIPLE
SPECIES AND CONSIDERING
CROSS-SPECIES IS IMPORTANT AND
THE CONTEXT OF THIS WORKSHOP,
IT'S IMPORTANT TO THINK HOW DO
WE MEASURE BEHAVIORS OF
DIFFERENT SPECIES.
SO IF WE WANT TO STUDY SHARKS
HOW ARE WE GOING TO MEASURE
SHARK BEHAVIOR, ZEBRAFISH, AND
HOW ARE WE GOING TO MEASURE
HUMAN BEHAVIOR, SO ON.
WE'RE KEEPING THE QUESTIONS IN
THE CHAT AND Q&A TO SEED
TOMORROW'S BREAKOUT SESSIONS
SINCE WE HAVEN'T HAD TIME TO GET
TO ALL OF THEM NOW.
THERE'S SO MUCH TO TALK ABOUT.
FOR RIGHT NOW WE'RE GOING TO
MOVE TO PANEL 2.
>> THAT WAS A RICH DISCUSSION.
THANK YOU, JANINE.
DIFFICULT ASPECT TO FOLLOW BUT
IT'S MY PLEASURE TO INTRODUCE
PANEL 2, ON SENSING BEHAVIOR IN
ITS ENVIRONMENT.
CENTRAL QUESTION FOR THIS PANEL
IS HOW DO WE CAPTURE THE
INFORMATION NEEDED FROM
INDIVIDUAL ORGANISMS, SOCIAL
GROUPS, PHYSICAL ENVIRONMENTS,
SO WE CAPTURE COMPLEXITY OF
BEHAVIOR.
REALLY THE QUESTION, ONE WAY TO
TRANSLATE THE BIG QUESTION, WHAT
ARE THE DIFFERENT TYPES OF DATA
NEEDED TO DO MODELS OF BEHAVIOR
THAT REFLECT MULTI-DIMENSIONAL
COMPLEXITY OF BEHAVIOR IN THE
CONTEXT IN WHICH THE BEHAVIOR
OCCURS NATURALLY, THE IDEA THAT
CAME UP IN THE PREVIOUS PANEL.
AND WHAT ARE SOME OF THE
TECHNOLOGIES IN PARTICULAR
SENSOR TECHNOLOGIES, WHAT ARE
INFORMATIC TOOLS THAT ARE
AVAILABLE AND WHICH ONES ARE
NEEDED TO COLLECT AND INTEGRATE
THESE TYPES OF DATA TO BUILD
CONCEPTUAL FRAMEWORKS IN
COMPUTATIONAL MODELS.
WITH THAT THE SPEAKERS WE HAVE
LINED UP, A GREAT SET, GALIT
PELLED, BEN HAYDEN, MALCOLM
MacIVER, AND RAJASHREE
BASKARAN, AND WE SWITCH TO THE
DISCUSSANTS, PETER HARTWELL, BOB
DATTA, AND ANDRE GREEN.
WITH THAT I HAND IT OVER TO
GALIT.
>> HI.
I'M GALIT PELLED, MICHIGAN STATE
UNIVERSITY.
WE'RE INTERESTED IN THE PRACTICE
OF IMPROVING MOTOR FUNCTION THAT
WILL LEAD TO HIGHER LEVELS OF
SPEED AND STRENGTH, COGNITIVE
FUNCTION INCLUDING DECISION
MAKING AND LEARNING, DEFINED AS
BEHAVIOR, AND TO UNDERSTAND
NORMAL BEHAVIOR, TO ASSESS
CHANGES IN BEHAVIOR WE NEED
TOOLS TO MEASURE BEHAVIOR.
THERE WAS A GREAT DISCUSSION
BEFORE, WHETHER WE SHOULD BE
LOOKING AT DIVERSE ANIMAL MODELS
TO UNDERSTAND BEHAVIOR.
IN MY LAB I'LL SHOW YOU THAT WE
ARGUE THAT THIS IS NECESSARY.
SO HERE YOU SEE IN THE NEXT FEW
SLIDES I'M GOING TO SHOW YOU
TOOLS THAT WE'RE DEVELOPING AND
CONCEPTS WE'RE STUDYING IN MORE
CONTROLLED NEURAL SYSTEMS HOW
THESE MAY BE ADOPTED TO MORE
COMPLEX SYSTEMS.
SO UNDERSTANDING DECISION MAKING
AND GOAL-ORIENTED MOVEMENT IS
REALLY A MAJOR GOAL OF
NEUROSCIENTISTS AND BEHAVIORAL
SCIENTISTS, THESE ARE OF COURSE
INTEGRATION OF VERY COMPLEX, THE
BRAIN AND PERIPHERAL NERVOUS
SYSTEM.
YOU SEE AN OCTOPUS, THERE ARE
300 SPECIES, THIS IS THE THE
ONLY SPECIES OF OCTOPUS THE
ENTIRE GENOME HAS BEEN
SEQUENCED, YOU CAN IMAGINE ALL
SORTS OF AMAZING THINGS WE WILL
BE ABLE TO DO WITH THAT IN THE
FUTURE.
AND YOU CAN SEE IT HERE IN ITS
TANK.
INTERACTING, TRYING TO GRASP THE
FINGER OF A STUDENT.
AND IT'S QUITE AMAZING.
SO WE'RE TRYING TO STUDY OCTOPUS
GRASPING MOVEMENT HOPING TO
IDENTIFY ALGORITHM THAT DESCRIBE
HOW THE OCTOPUS CAN ADAPT
MOVEMENT MIDWAY, CAN WE DEVELOP
NEW TYPE OF DEVICES RAPID AND
ADAPTIVE SUCH AS OCTOPUS ARM.
IT HAS THE HEART, BLUE BLOOD,
CAMOUFLAGES, REGENERATES.
AND THEY ACTUALLY EXISTED,
THERE'S THE FIRST FOSSILS OF
OCTOPUS DATED TO 296 MILLION
YEARS AGO.
SO EVEN BEFORE DINOSAURS.
SO IN THE COURSE OF EVOLUTION,
THE OCTOPUS AMAZINGLY CAME UP
WITH MANY SIMILAR SOLUTIONS FOR
PHYSIOLOGY AND SENSING LIKE
VERTEBRATES.
FOR EXAMPLE, ITS EYES, OPTIC
LOBES ARE SIMILAR TO VERTEBRATE
SYSTEMS, BUT IN TERMS OF MOTION
AND MOVEMENT IT DEVELOPED A
COMPLETELY ALTERNATIVE STRATEGY.
SO, IT HAS A HUGE NERVOUS SYSTEM
WITH 500 MILLION NEURONS.
SIMILAR TO WHAT A DOG HAS.
BUT MOST OF THE NEURONS ARE NOT
IN THE BRAIN.
THEY ARE ACTUALLY IN THE ARM.
AND EACH SINGLE ARM HAS AN AXIAL
CORD THAT ACTS LIKE A SPINAL
CORD.
AND MOST OF THE DECISIONS THAT
THE OCTOPUS MAKE IS NOT IN ITS
BRAIN, IT'S MAKING IT IN THE
ARMS.
THE DISEMBODIED ARM IS KNOWN TO
KEEP MOVING IN A WAY THAT AN
INTACT OCTOPUS DOES.
YOU CAN IMAGINE HOW THE OCTOPUS
NOW GIVES US REALLY A WAY TO
LOOK AT NEURAL PATTERNS IN
NEURAL NETWORKS THAT ARE
ASSOCIATED WITH MOVEMENT.
SO, TO LOOK AT MOVEMENT, THE
OCTOPUS -- HERE YOU CAN SEE
WE'RE USING A.I. WITH DEEP LAB
CUT, CAN PLACE 16 MARKERS ON A
SINGLE ARM TO MEASURE MOVEMENT.
SO THERE'S EIGHT ARMS.
THERE'S CONSTANT MOVEMENT.
THERE'S CONSTANT ELONGATION AND
SHRINKING.
IF WE WANT TO MEASURE OCTOPUS
MOVEMENT IN NATURAL ENVIRONMENT,
DEEPLABCUT IS A GOOD WAY TO
START.
TO GET HIGH RESOLUTION
KINEMATICS WE HAVE TO GO AND USE
DIFFERENT TECHNIQUES.
AND WHAT YOU CAN SEE HERE WE CAN
USE REFLECTIVE SENSORS,
REFLECTIVE MARKERS ON AN OCTOPUS
ARM, ON THE TOP IT'S AN ARM
OUTSIDE OF THE WATER.
AND NOW WE CAN ALSO USE IT IN AN
OCTOPUS INSIDE THE WATER.
I'M NOT SURE WE CAN SEE THE
MOVIE.
THANK YOU.
OF THE PREVIOUS -- YEAH.
SO SENSORED REFLECTIVE SENSORS
OUTSIDE AND OF COURSE INSIDE THE
WATER.
WE ALSO WANT TO HAVE THE NEURAL
CORRELATES THAT WE RECORD AT THE
SAME TIME WITH THE MOVEMENT.
AND FOR THAT TODAY WE CAN PLACE
A RAY OF ELECTRODES INTO THE
OCTOPUS ARM, AND YOU CAN ALREADY
SEE HERE WE CAN RECORD MORE THAN
16 CHANNELS AT A TIME FOR
OCTOPUS ARM AND START SEEING
DIFFERENT PATTERNS IN TERMS OF
THE AMPLITUDE, IN TERMS OF THE
FREQUENCY, WHEN WE GIVE
DIFFERENT TYPE OF STIMULATION.
CENTRAL STIMULATION, PERIPHERAL
AND SO ON.
OF COURSE WE WANT TO RECORD ALL
OF THAT IN AN INTACT OCTOPUS, IN
ITS NATURAL ENVIRONMENT.
WHEN WE CAN PROVIDE NATURAL CUES
AND STIMULATION.
SO NOW WITH WITH CINDY WE WORK
ON THE ARM, AND HAVE IT IN ITS
BIG TANK RECORDING ACTION
POTENTIALS AND AT THE SAME TIME
RECORDING ITS MOTION.
WE CAN EVEN GO DEEPER AND LOOK
AT THE NEURAL NETWORK, SINGLE
ACTION POTENTIALS, YOU CAN SEE
HERE WE CAN PLACE -- CAN WE PLAY
THAT VIDEO, TOP LEFT?
WE CAN PLACE A SLICE OF AN
OCTOPUS ARM ON A MULTI-ELECTRODE
ARRAY, THIS DISH HAS 26,000
SINGLE ELECTRODES, AND YOU CAN
SEE HERE THAT WE CAN DELIVER
EITHER A CENTRAL STIMULATION OR
PERIPHERAL STIMULATION, AND WE
CAN START SEEING DIFFERENT
PATTERNS OF ACTIVITY.
SO, ALL OF THAT TOGETHER
HOPEFULLY WILL ENABLE US TO
IDENTIFY -- YES, TO IDENTIFY THE
CIRCUITS THAT ARE ASSOCIATED
WITH GRASPING, HOW IS THE
OCTOPUS HAVING -- ADAPT TO
GRASPING AND CHANGING ITS MOTION
MIDWAY, AND WE WILL BE -- WILL
WE BE ABLE TO IDENTIFY THESE
ALGORITHMS AND TEST THEM IN
FLEXIBLE MATERIAL-BASED
BIOROBOTIC ARMS.
I SHOWED HOW WE CAN USE HIGH
DIMENSIONAL TECHNOLOGIES TO
MEASURE --
>> ONE MINUTE.
>> OKAY.
>> ONE MINUTE LEFT.
>> YES, WHERE ENWE EN-- WHEN WE
MEASURE IN OCTOPUS WITH SPECIAL
CHALLENGES, IN MY LAB WE'RE
INTERESTED IN OTHER ANIMAL
MODELS, HOW WE CAN ASSESS
BEHAVIOR IN LARGE ANIMALS.
WE SPECIFICALLY ARE INTERESTED
IN BRAIN INJURIES, OVER THE PAST
FEW YEARS WE DEVELOPED A MODEL
OF BRAIN INJURY, I WANT TO SHOW
QUICKLY HOW WE USE EXACTLY THE
SAME METHODS THAT WE USE IN THIS
UNCONVENTIONAL SPECIES TO MORE
PERHAPS CONVENTIONAL WAY.
YOU CAN SEE WE'RE USING
DEEPLABCUT TO LOOK AT MOTION,
LOCOMOTION, THE SAME MOTION
REFLECTIVE SENSORS, YOU CAN SEE
THE PIG WITH MARSHMALLOWS, IT'S
GOING TO BEHAVE WELL.
AND ALSO THE WE CAN GET HIGH
RESOLUTION KINEMATIC DATA, DATA
ANALYSIS OF PIG MOTION BEFORE
THE INJURY AND AFTER THE INJURY.
OUR PIGS ALSO WEAR WEARABLE
SENSORS.
THEY EVER WEARING FIT BITS.
SO WE CAN TELL HOW MANY STEPS,
WHEN, AND IN THE NATURAL
ENVIRONMENT WE CAN ALSO MEASURE
HOW MUCH THEY SLEEP A DAY, WHEN
DO THEY SLEEP, WE HAVE VIDEO
CAMERAS TO LOOK WHO THEY LIKE TO
SLEEP WITH.
AND SO ON.
SO I DIDN'T INCLUDE THAT BUT
WE'RE INTERESTED IN PERSONALITY
CHANGES IN PIGS, AND THEY HAVE
REALLY VERY DEVELOPED
PERSONALITY.
WE CAN MEASURE AGGRESSION,
DEPRESSION, ANXIETY, AND
SHORT-TERM MEMORY AND LONG-TERM
MEMORY, DEVELOPED A WHOLE ARRAY
OF TECHNOLOGIES THAT WE CAN
MEASURE IN THESE SPECIES.
SO JUST TO CONCLUDE, I SHOWED
YOU THAT WE DEVELOPED TOOLS TO
ADDRESS -- TO RECORD BEHAVIOR,
PERHAPS IN UNCONVENTIONAL
SPECIES, AND IN LESS
CONVENTIONAL ENVIRONMENTS, BUT
HOPEFULLY THIS WILL BE ABLE TO
BE ADOPTED IN MORE TRADITIONAL
SPECIES.
AND THE CONCEPTS THAT WE'RE
GOING TO LEARN WILL HOPEFULLY
TRANSLATE TO REALLY
UNDERSTANDING OF HUMAN BEHAVIOR.
THANK YOU.
BEN, ARE YOU READY?
>> THANK YOU VERY MUCH.
THANK YOU FOR INVITING ME.
THE TITLE IS THE BEHAVIORAL
IMAGING REVOLUTION, BEHAVIORAL
IMAGING IS A TERM IN THE LAB TO
REFER TO WHAT A LOT OF THINGS
PEOPLE ARE TALKING ABOUT TODAY
IS ABOUT, BASICALLY THE USE OF
CAMERAS AND DOWNSTREAM SOFTWARE
AND TECHNOLOGY TO BASICALLY SAY
SOMETHING SOPHISTICATED ABOUT
BEHAVIOR.
I WANT TO START WITH ONE
THOUGHT.
WHICH IS THAT BEFORE THE
TELESCOPE WAS INVENTED YOU COULD
SEE THE STARS.
THE DIFFERENCE BETWEEN THE
TELESCOPE AND YOUR AS IS THEY
LET YOU DO SOMETHING BETTER AND
SEE BETTER, ORDERS OF MAGNITUDE
BETTER, WHAT LED TO REVOLUTION
500 YEARS AGO, THE SAME THING IS
HAPPENING NOW IN BEHAVIOR.
SO, BASICALLY, LET'S SEE IF I
CAN GET THIS VIDEO TO PLAY HERE.
YEAH, THIS IS A VIDEO OF A
MONKEY AT THE FIELD SITE IN
SANTIAGO WHERE I'VE DONE FIELD
WORK, WHAT REALLY STRIKES ME,
ALWAYS HAS STRUCK ME ABOUT
MONKEYS IN THE WILD, IS THAT
THEY ARE JUST GENERATING
ENORMOUS AMOUNTS OF DATA.
DATA IS JUST LEAKING OUT
CONSTANTLY.
I JUST AM LIKE HOW DO WE GATHER
ALL THIS DATA?
I'VE BEEN TRYING TO FIGURE OUT
WAYS TO TRY TO DO THAT.
AND EVERYBODY HAS BEEN SAYING, I
DON'T NEED TO REPEAT IT, BUT
DATA IS NOT JUST INTERESTING TO
PRIMATOLOGISTS, BUT OTHERS TOO.
THAT'S WHAT THE SYMPOSIUM IS
ABOUT.
AND MY LAB, UNIVERSITY OF
MINNESOTA, WE HAVE OUR OWN
BESPOKE SYSTEM FOR STUDYING
MONKEYS.
THIS IS A PICTURE OF A MONKEY IN
OUR CAGE SYSTEM MOVING AROUND
INTERACTING WITH FEEDERS.
WE HAVE 62 CAMERAS TRACKING HIM,
A BUNCH OF SOFTWARE WE WROTE TO
DO THAT.
AND MY GOAL TODAY IN MY BRIEF
TIME IS JUST TO TALK ABOUT
CONSEQUENCE OF TRACKING THE
MONKEYS, ABILITY THAT HAS TO DO
BEHAVIORAL IMAGING TO SEEP WHAT
THE BEHAVIOR IS AND THE WAY THAT
CHANGED HOW TO BE BEHAVIOR
NEUROSCIENTISTS.
WE'RE FUNDAMENTALLY INTERESTED
IN THE RELATIONSHIP BETWEEN
BRAIN AND BEHAVIOR IN MY LAB.
I KNOW THIS IS A BROAD MEETING
WITH A BUNCH OF PEOPLE.
NOT EVERYBODY IS THINKING IN
TERMS OF THAT.
BASICALLY WE WANT TO SEE HOW
BRAIN ACTIVITIES PREDICTS AND
DRIVES BEHAVIOR.
THAT'S NOT EASY, EVEN WITH HIGH
QUALITY BEHAVIORAL IMAGING, YOU
HAVE TO WORRY, FOR EXAMPLE,
ABOUT CONFOUNDERS, ABOUT THINGS
THAT ARE CORRELATED WITH THE
THING YOU'RE INTERESTED IN, AND
THAT MAY ALSO BE DRIVING THE
NEURAL ACTIVITY.
SO, WE DO THINGS, STANDARD
THINGS LIKE CAREFUL TASK DESIGN,
BIG DATA, EVEN BIGGER DATA, MORE
SOPHISTICATED ANALYSIS, IN ORDER
TO DEAL WITH THE PROBLEM OF
CONFOUNDERS, BUT THE REAL THING
THAT KEEPS HITTING US IN HEAD AS
WE DO THIS RESEARCH IN MY LAB IS
THAT THE ABILITY TO MEASURE THE
BEHAVIOR IS LIKE THE ABILITY TO
LOOK AT STARS IN THE TELESCOPE,
LEADS TO DIFFERENT QUESTIONS.
IT DEFINES YOU TO FIND DIFFERENT
THINGS TO BE INTERESTING.
IN THE KALES OF A -- CASE OF A
MONKEY, WE'VE HAD THE MONKEY SIT
IN A CHAIR AND DO A LITTLE TASK
WITH THEIR EYES.
WITH OUR SYSTEM IN THE BIG CAGE,
THE FIRST THING WE DID, WE TOOK
HUGE AMOUNTS OF DATA, HUNDREDS
OF HOURS OF A FORAGING TASK, WE
DID A SIMPLE DIMENSIONALITY
ANALYSIS, THE BEHAVIOR DIVIDES
INTO 49 ISLANDS.
THE NUMBER IS NOT PARTICULARLY
IMPORTANT.
IT DEPENDS ON THE WAY YOU DO THE
PRE-PROCESSING.
BUT YOU GET THESE DISCRETE
ISLANDS OF BEHAVIOR, THAT
CORRESPOND REALLY TO NATURAL
THINGS THAT YOU SEE IN THE
BEHAVIOR.
AND THEN THESE HAVE A HIGH
ORGANIZATION, YOU SHOULDN'T BE
TOO SURPRISED.
WE HAVE THE MONKEY DO A TASK OR
DIFFERENT TASK, TASK OFF, WE SEE
SUBTLE CHANGES IN THE STRUCTURE
OF THE THIS HIERARCHY.
AND I'M NOT GOING TO TALK ABOUT
WHAT THAT MEANS TODAY EXCEPT TO
SAY THAT THE KEY POINT OF THIS
FOR THIS TALK, THIS ISN'T EVEN A
QUESTION WE THOUGHT ABOUT ASKING
IF WE DIDN'T HAVE BEHAVIORAL
IMAGING, DIDN'T HAVE ABILITY TO
DO HIGH QUALITY HIGH RESOLUTION
TRACKING AND BEHAVIOR.
WHEN THE MONKEY IS IN A CHAIR
YOU ASSUME HE'S IN THE STATE OF
DOING THE TASK, THE BEHAVIOR IS
ON OR OFF TASK.
THERE'S A VERY SOPHISTICATED
COMPLEX STRUCTURE AND WE'RE
LOOKING NEURAL CORRELATES.
ONE MORE EXAMPLE TO ILLUSTRATE,
THE WAY THAT BEHAVIORAL IMAGING
HELPS YOU THINK ABOUT AND ASK
NEW QUESTIONS, SO THE CORE TOPIC
OF MY LAB'S RESEARCH IS ABOUT
ECONOMIC DECISION MAKING,
BASICALLY HOW HUMAN OR A MONKEY
IN OUR CASE WILL CHOOSE BETWEEN
THINGS, BASED ON MICROECONOMIC
THEORY.
YOU'RE SITTING AROUND DOING
NOTHING, TWO OPTIONS APPEAR, YOU
CHOOSE BETWEEN THEM AND IT'S
OVER, YOUR CHOICE IS OVER.
THAT KIND OF APPROACH TO
THINKING ABOUT WHAT A CHOICE IS
LEADS TO THESE DISCRETE MODEL OF
HOW CHOICE WORKS AND IMPLICATION
IS THESE WILL EACH CORRESPOND TO
BRAIN AREA AND THAT DRIVES THE
SHAPE OF THE NEUROSCIENCE OF
ECONOMIC CHOICE.
TO EMPHASIZE THE THEME NATURAL
BEHAVIOR DRIVEN BY IMAGING LEADS
TO DIFFERENT CONCEPTUALIZATIONS
OF HOW THESE THINGS WORK, I
THINK OF CHOICE AS CONTINUOUS, I
FOUND THIS VIDEO OF DOGS CHASING
A RABBIT.
THIS IS WHAT I THINK MUCH AS
CANONICAL ECONOMIC DECISION
MAKING, THE DOG HAVING TO CHOOSE
WHICH WAY TO GO AT EVERY SINGLE
MOMENT.
IT'S A CONTINUUM.
A CONTINUUM OF CHOICE, CONTINUUM
OF POSSIBLE ACTIONS.
AND HE HAS TO DECIDE HIGHER
LEVEL THINGS LIKE WHAT STRATEGY
HE'S GOING TO EMPLOY TO FOLLOW
THE RABBIT.
THESE ARE THINGS THAT COULD IN A
CLUMSY WAY DO WITH
MICROECONOMICS BUT WOULDN'T BE A
NATURAL FIT.
SO THIS HAS LED OUR LAP TOWARDS
THINKING OF CONTROL THEORY,
ENGINEERING CONTROL THEORY, SORT
OF AS A FOUNDATION FOR THINKING
ABOUT WHAT'S DRIVING BEHAVIOR.
IT'S BEEN PLEASING FOR ME TO SEE
THE TALKS SO FAR TODAY, A LOT
HAVE CONVERGED ON THE SAME IDEA
THAT REALLY WE NEED TO USE
CONTROL THEORY AND FEEDBACK AS A
WAY OF UNDERSTANDING HOW
DECISIONS OCCUR.
I'M ALSO GOING TO TALK ABOUT ONE
OTHER THING.
JUST ONE MORE EXAMPLE, IN THE
NATURAL WORLD, IN THE LABORATORY
YOUR MONKEY OR PERSON IS SITTING
THERE, YOU EXPLAIN THE TASK,
THEY UNDERSTAND, THEY HAVE BEEN
DOING IT FOR MONTHS, IF THEY ARE
A MONKEY.
BUT IN THE REAL WORLD WE'RE LIKE
CONSTANTLY BASICALLY STARVING
FOR INFORMATION.
WE'RE HIGHLY UNDERSPECIFIED.
I PUT THIS PICTURE OF THE MONKEY
TRYING TO DRIVE A CAR, IT'S HOW
I FEEL IN MY LIFE, BASICALLY
TRYING TO DO COMPLICATED THINGS
I'M WOEFULLY INADEQUATELY
PREPARED FOR, CONSTANTLY NEEDING
INFORMATION, SEEKING
INFORMATION, SO MUCH SO TO DRIVE
ALL MY DECISION MAKING, AND I
THINK THIS IS SOMETHING
FUNDAMENTAL.
IT HELPS EXPLAIN THE WAY MONKEYS
DEAL WITH UNCERTAINTY ISN'T
PARTICULARLY WELL DESCRIBED BY
ECONOMIC THEORIES, ECONOMIC
THEORIES, WE'VE HAD TROUBLE WITH
THAT, BEEN DOING THAT 15 YEARS.
INCORPORATING CONCEPTS OF THE
NEED TO RESOLVE UNCERTAINTY HAS
HELPED US UNDERSTAND THAT KIND
OF BEHAVIOR MORE.
THESE ARE A FEW EXAMPLES OF THE
WAY THAT TAKING THIS
NATURALISTIC PERSPECTIVE WHICH
IS REALLY ULTIMATELY
EVOLUTIONALILY DRIVEN TO ASK
DIFFERENT QUESTIONS, INEVITABLY
GIVE US DIFFERENT ANSWER.
ONE-MINUTE WARNING.
I'LL SAY QUICKLY WE'RE RECORDING
BRAIN ACTIVITIES IN THESE MORE
NATURALISTIC TASKS.
THIS OF THIS MODEL OF THE BRAIN
WITH 150 AREAS, EACH WITH SOME
PARTICULAR FUNCTION, DOESN'T
REALLY WORK AS WELL.
A LOT OF THESE BRAIN AREAS SEEM
TO HAVE A FRANK MOTOR FUNCTION
WHEN WE HAVE THE MONKEYS MOVING
AROUND IN THE REAL WORLD, THAT
SEEMS TO BE TRUE FOR RATS AND
MICE AS WELL.
THAT IS CAUSING US TO TRY TO
THINK OF WAYS TO RETHINK WHAT
BRAIN STRUCTURES DO AND HOW THEY
RELATE TO EACH OTHER.
AGAIN WE WOULDN'T HAVE DONE THAT
IF WE DIDN'T HAVE ALL THIS DATA
COMING FROM FREELY MOVING
MONKEYS DRIVEN BY ABILITY TO
MEASURE THEIR BEHAVIOR, TO IMAGE
THEIR BEHAVIOR.
LET'S SEE.
IN THE FUTURE WE'RE GOING TO BE
DOING EXPERIMENTS MORE AND MORE
TOWARDS THIS WORLD, MONKEY BRAIN
EMBODIED IN THE BODY OF MONKEY,
EMBEDDED IN A COMPLEX NATURAL
ENVIRONMENT HELPING US DO
NEUROSCIENCE IN THE 21ST
CENTURY.
THANK YOU.
>> THANKS VERY MUCH, BEN.
MALCOLM, ARE YOU READY?
>> YEAH, CAN YOU HEAR ME?
>> YEP.
>> GREAT.
LET ME SHARE MY SCREEN.
OKAY.
DO YOU SEE MY FIRST SLIDE?
>> YES.
>> GREAT.
ALL RIGHT.
SO I'M MALCOLM MacIVER, I'M
HAPPY TO TALK TO YOU TODAY.
TOPIC OF MY QUICK TALK IS
WIDENING THE APERTURE OF
BEHAVIORAL RESEARCH.
AND BY THIS I MEAN, YOU KNOW, 20
YEARS AGO WHEN I FINISHED MY
PhD we in decision making
neuroscience and other areas of
behavioral research, we settled
on idealized approaches to
alternative choice, to make the
task somewhat easier to manage
and control.
SIMILARLY AT THE NEURAL LEVEL,
RECORDING SINGLE NEURONS AT A
TIME, MODELING THEM WITH
DEPARTMENTAL MODELS, WAS PRETTY
COMMON, WHAT'S REALLY COOL IN
THE PAST 20 YEARS IS SEEING THE
APERTURE WIDEN TO ENCOMPASS SOME
OF THESE THINGS NATURALISTIC
BEHAVIOR IS COMING ON STRONG,
WHOLE ANIMAL SIMULATION I'LL
ADVOCATE FOR TODAY.
WE SEE SOME IN THE FIELD,
EVOLUTIONARY APPROACHES WE'VE
HEARD ABOUT TODAY, SOME
ROBOTICS.
AND OF COURSE THERE'S LOTS OF
OTHER THINGS, GENETICS
APPROACHES, I DIDN'T WANT TO GET
IT TOO MESSY, I DON'T HAVE THEM
ON HERE.
THE THE QUESTION IS HOW TO
OPERATIONALIZE CONSENSUS THAT
EMERGED INTO ACTIONABLE
INTELLIGENCE FOR THOSE LABS
WHICH NEED TO PURSUE IF THEY ARE
NOT ALREADY ON THIS TRACK.
AND TO MOTIVATE MY APPROACH, I
THINK IT'S USEFUL TO POINT OUT
THE BODY IS A COOPERATIVE
ECOSYSTEM OF 40 TRILLION HUMAN
CELLS, ANOTHER 40 TRILLION BACK
TEAR VIRUSES AND FUNGI.
BEHAVIOR IS THE OUTPUT OF THE
ENTIRE ECOSYSTEM.
STUDY IS A COOL OPPORTUNITY FOR
PUTTING OUR ARMS AROUND THE
ENTIRETY OF THE ORGANISM.
TO MAKE THIS POINT, I LOVE THIS
VIDEO OF PASSIVE DYNAMIC WALKER
FROM 20 YEARS AGO AS WELL.
THIS IS A ROBOT THAT HAS PRETTY
MUCH THE SAME DYNAMICS AS A
HUMAN WALKER.
YET THERE'S NO MOTORS, NO
CONTROL, THE ONLY INPUT OF
ENERGY IS INCLINED PLANE.
AND, YOU KNOW, IT GIVES A FORM
OF DYNAMICS, DEFAULT DYNAMICS,
ALMOST FOR FREE.
AND SO WHAT THIS POINT AMOUNTS
TO ESSENTIALLY THE BODY THROWS
EVERYTHING IT CAN AT THE PROBLEM
OF SURVIVAL, OF COURSE.
AND WE HAVE TISSUES THAT HAVE
VERY LOW ENERGETIC FOOTPRINTS,
WATTS PER KILOGRAM LIKE BONE,
BILLIONS OF YEARS, GRAVITY,
DEALING WITH MOVEMENT,
GRAVITATIONAL FIELD, AND NEURONS
REALLY SPECIALIZE ON
CONTINGENCIES AT THE MILLISECOND
TIME SPAN, OF COURSE VERY
IMPORTANT, BUT WE NEED TO OPEN
OUR PURVIEW TO THE WHOLE THINGS.
COMPUTATIONS LIKES SPREAD OVER
THE ENTIRETY OF THE DOMAIN.
HOW DO YOU DO THIS, BROADEN IT?
I'M GOING TO ARGUE, WELL, USING
TOOLS AS SYNTHESIS IS NOT NOVEL
BUT HOW DO WE ACTUALLY USE THEM
AND WHAT ARE THEY?
THE POINT IS COUNTER INTUITIVELY
WHAT SEEMS TO BE MAKING OUR JOB
HARDER INCORPORATING A WHOLE
OTHER BUNCH OF ASPECTS OF AN
ANIMAL DOES MAKE IT EASIER IN
THE END.
SO SOME ENGINES OF SYNTHESIS ARE
WHOLE ORGANISM, AT-ENVIRONMENT
SIMULATION, ROBOTICS,
EVOLUTIONARY APPROACHES, VIRTUAL
REALITY.
HERE IS OLD ELECTRIC FISH WORK
FROM MY Ph.D. TO RECONSTRUCT
SIGNAL INPUT, FLUCTUATION OF
TRANSDERMAL VOLTAGE DOING A
NATURALISTIC TASK.
YOU USE MOTION CAPTURE DATA AND
YOU SYNTHESIZE WITHIN A COMPUTER
MODELS OF ELECTRIC IMAGE
FORMATION, WHERE THE SENSORS THE
BODY, ELECTRIC FIELD MODEL, A
MODEL OF THE PREY THE ANIMAL
HUNTS IN NATURAL ENVIRONMENT,
AND THEN BEHAVIORAL MODEL
CONSTRAINED BY EMPIRICAL DATA
COLLECTED FROM PRECEDING STEP
AND YOU GET VOLTAGE HISTORIES
FROM A SPIKE MODEL, YOU CAN GET
SPIKE TRAINS, ULTIMATELY WE GET
THROUGH A LOT OF COMPUTATION.
AFFERENT SPIKE TRAINS ACROSS
14,000 AFFERENTS ARE ELECTRO
SENSORY -- I'M GETTING A LOT OF
WEIRD SOUNDS FROM ANOTHER
SPEAKER I THINK MAYBE?
THERE WE GO.
SO NOT ONLY ARE WE GETTING INPUT
ACROSS 14,000 ELECTRO SENSORY
AFFERENTS BUT WE CAN FOLLOW THE
SAME PROCESS TO GET THE PASSIVE
ELECTRO SENSE AFFERENTS AND
MECHANOAS WELL, AND LOOK AT HOW
THEY ARE ACTIVATED DURING
NATURAL BEHAVIORS.
WE AUGMENT THIS WITH ROBOTICS
APPROACH TO LOOK AT HOW MOVEMENT
IS BEING CONSTRAINED BY THE
ANIMAL'S BIOMECHANICS, HOW THE
ANIMAL HAS TO GENERATE THRUST
THAT IT'S GENERATING, AND WE USE
THOSE DATA TO CONSTRAIN
NUMERICAL MODELS TO EVEN FINER
RESOLUTION ANALYSIS.
WE DO THE SAME THING IN THE
SENSORY DOMAIN WITH SENSORY
ROBOTICS.
I DON'T HAVE TIME TO GO INTO IT
BUT IT CAN NAVIGATE AUTONOMOUSLY
THROUGH PYLONS USING ELECTRIC
FIELDS.
PAYOUTS HERE, WELL, BEHAVIORS
THAT MADE NO SENSE AT ALL WHEN
LOOKED FROM STRICTLY SENSORY
PROCESSING PERSPECTIVE OR
INFORMATION THEORY PERSPECTIVE,
ONCE WE AUGMENTS THE WAYS THE
ANIMAL GENERATED THRUST AND HOW
MUCH COST THAT COST
ENERGETICALLY COULD MAKE SENSE
OF BEHAVIORS THAT DID NOT MAKE
SENSE FROM THE SENSORY OR
INFORMATION THEORY PERSPECTIVE.
OBVIOUSLY THE ANIMAL IS CON
JOINTLY OPTIMIZING ALL OF THE
THESE FACTORS, ZERO INSIGHT INTO
WHAT THEY WERE WITHOUT THAT MORE
SYNTHETIC APPROACH.
THE SECOND PAYOFF I MENTIONED,
THERE WAS A BIG CHANGE AS A
RESULT OF CHARACTERIZING THE
SIGNALS THROUGH THE SYNTHESIS
APPROACH, IN THE WAY SIGNALS ARE
FED INTO THE NERVOUS SYSTEM OF
ELECTRIC FISH, NEUROPHYSIOLOGY
LABS, THAT LED US TO DISCOVER
NEURAL ASSESSING STRATEGIES.
I'M GOING TO TALK ABOUT
EVOLUTIONARY WORK WHETHER WE
DISCOVERED THAT DURING THE
TRANSITION TO LAND ANIMALS
TRIPLED EYE SIZE, THE IDEA
PERHAPS AERIAL VISION LED TO
TERRESTRIALITY, HOW THE SENSORY
ECOLOGY OF ANIMALS VERSUS WATER
ON LAND LEADS TO DIFFERENT
DECISION-MAKING STRATEGIES.
IN PARTICULAR IN WATER WE HAVE
THESE SENSORY BUBBLES THAT ARE
VERY TIGHTLY CHAINED TO THE
ANIMAL, VERY SHORT.
THEY DETECT PREDATOR, HAVE TO
FIRE OFF MODELERS TO SURVIVE.
WITH LONG SIGHT LINES LAND
PROVIDES YOU CAN IMAGINE AN
ANIMAL WHO NOW HAS A MUCH LARGER
SENSORIUM LOCATING TRAJECTORIES
AND PICKING ONE, PLANNING.
ONE OF THE THINGS WE FOUND IN
MEDIUM ENTROPY OR CLUTTER
ENVIRONMENTS THAT MATCH SAVANNAH
ENVIRONMENTS THAT HUMAN ALSO
WENT INTO, MAXIMUM ADVANTAGE
AFTER CLUTTER GEOMETRY.
WE'RE TESTING USE AGRO BOUGHT
WITH A CO2 -- USING A ROBOT,
THE AIR PUFF TURNS RED GETTING
THE MOUSE TO RUN AWAY FROM THAT.
PLAN A ROUTE.
WE'RE DOING SLOWDOWN IN THE
HABITAT, WHETHER THEY ARE
ENGAGED IN PLANNING BEHAVIORS.
YOU CAN SEE A FEW INDICATIONS.
THERE'S DTE PHENOMENA, A HEAD
WAG, YOU CAN SEE THAT HAPPENING
HERE AT THIS CHOICE POINT, AS IT
SENSES A ROBOT NEARBY.
AND SO -- LET'S SEE.
NOW, WE WANT TO -- SINCE A LOT
OF PEOPLE ARE INTERESTED IN
HUMAN WORK HERE I WANTED TO
MENTION WE'VE TRANSLATED THIS
PARADIGM INTO A HUMAN VIRTUAL
REALITY SYSTEM.
THIS IS A VIEW WHEN YOU WALK IN,
THE PREDATOR IS A SCARY LOOKING
GHOST.
WE HAVE A WIRELESS SYSTEM FOR
VR.
AND EVERY MOVEMENT, EVERY FOOT
THAT YOU TAKE, EVERY FOOT THAT
YOU MOVE IN REAL SPACE IS
MATCHED IN VR.
SO HERE'S A PERSON IN OUR
VIRTUAL REALITY ENVIRONMENT
MATCHING THE ONE THE RODENTS ARE
GOING THROUGH RUNNING FROM THE
PREDATOR MOVING AROUND
OBSTACLES TO BE HIDDEN.
WE JUST STARTED THIS WORK,
COMPARING TO MICE, COMPARATIVE
WORK ON YOU OF HOW THE TWO
ANIMALS DO PLANNING AND COMPLEX
ENVIRONMENT ALSO.
I WANT TO END ON A NOTE THAT I
THINK THAT IT'S VERY INTERESTING
TO ME THAT COLLABORATIONS AND
PARTICLE PHYSICS GO TO 5,000
CO-AUTHORS, PARTICLE PHYSICISTS
I KNOW THINK WE'RE WORKING ON A
MUCH HARDER PROBLEM.
SO I WONDER WHETHER WE NEED A
COUPLE ORDER OF MAGNITUDES MORE
COOPERATION IN TERMS OF EFFORT
TO MAKE FASTER PROGRESS.
THIS IS WHERE THIS WORKSHOP CAN
REALLY HELP OUT A LOT.
AND A FEW POINTS ON HOW YOU
MIGHT ADD ENGINES OF SYNTHESIS
WHEN NOT NATIVE TO YOUR TOOLKIT,
DOING A SABBATICAL, ATTENDING A
CONFERENCE, FINDING A PERSON
NEWER IN THE CAREER, THESE ARE
ALL APPROACHES I'VE FOUND
HELPFUL AS I'VE DONE A HUGE
NUMBER OF COLLABORATIONS FOR THE
WORK THAT YOU'VE SEEN.
SO I'D BE HAPPY TO TAKE
QUESTIONS LATER DURING THE
DISCUSSION SECTION.
THANK YOU VERY MUCH.
>> THANKS, MALCOLM.
LAST SPEAKER FOR THIS SESSION IS
RAJI.
WE'RE READY.
TAKE OVER.
GREAT.
THANK YOU.
>> THANK YOU.
GOOD AFTERNOON.
THANKS FOR THIS OPPORTUNITY.
I'M GOING TO COMPLETELY SWITCH
TRACKS.
I'VE BEEN OVERSTIMULATED IN THE
LAST COUPLE HOURS, I'M GOING TO
KEEP A LINGO OF ENGINEERING
DOMAIN WHERE I COME FROM AND
BUSINESS AND REAL LIFE
APPLICATION, AND KEEP IT PLAIN.
TWO MAIN IDEAS, BOTH AROUND
MULTI-SCALE SENSOR FUSION, I'M A
MICRO SCALE SENSOR TECHNOLOGIST,
MINIATURIZING, MAKING IT
POSSIBLE TO COLLECT LARGER
DATASETS YOU ALL REALLY LOVE
USING, KIND OF WHERE I GREW UP.
BUT I WANT TO SAY, MALCOLM, I
DID MY MASTER'S AT CORNELL AT
THE SAME TIME, I KNOW THAT ROBOT
PERSONALLY, IN THE LAB NEXT
DOOR.
I'M A DYNAMICS PERSON SO I
ENJOYED IN A AS -- THAT ASPECT.
I WANT TO DO TWO THINGS.
FOR THE LAST DECADE OR SO I HAVE
THOUGHT ABOUT HUMAN-MACHINE
INTERFACE IN THE CONTEXT OF
BUILDING NEW TECHNOLOGIES, LIKE
CONSUMER TECHNOLOGIES, BE IT
LARGER COMPUTERS, MOBILE
TECHNOLOGIES, ROUTABLE
TECHNOLOGIES.
I WANT TO BRING INSIGHT FROM THE
THOUGHT PROCESS OF UNDERSTANDING
THE HUMAN-MACHINE INTERFACE.
HIGH LEVEL INTRODUCTION, I DON'T
WANT TO SPEAK FOR DEEP LEARNING,
THERE ARE PEOPLE HERE MORE
EXPERTS, BUT I WANT MY
PERSPECTIVE ON WHAT IS A
PARADIGM THAT SHIFTED WITH
RESPECT TO DATA PROCESSING AND
ANALYTICS FROM DEEP LEARNING AND
WHAT ITS IMPLICATIONS ARE FOR
SENSOR FUSION, AND IN MY CURRENT
ROLE, INDEPENDENT CONSULTANT,
MAINLY BUILDING SYSTEMS FOR MANY
APPLICATIONS BUT MANY ARE HEALTH
AND WELLNESS.
REALLY WANTED TO HIGHLIGHT
ISSUES I HAVE TO REALLY DEAL
WITH WHERE THE RUBBER MEETS THE
ROAD, WE'RE TAKING SENSOR
TECHNOLOGIES, TRYING TO DEVELOP
SOLUTIONS, FOR EXAMPLE, IN ELDER
CARE OR ADDICTION RECOVERY
SOLUTIONS.
AND WHAT REALLY ARE THE GAPS AND
CHALLENGES IF YOU'RE TRYING TO
ADAPT THESE TECHNOLOGIES TO REAL
LIFE SOLUTIONS.
AND IN THE END I WANT TO PROPOSE
A FRAMEWORK, REALLY JUST THROW
IT OUT THERE, VERY HIGH LEVEL
THOUGHT PROCESS, IS THERE
SOMETHING AROUND THE FACT WE
HAVE LANGUAGE, HIERARCHY, A
LANGUAGE WE'VE ALL DEVELOPED
THAT ITSELF CAN BE USED TO
BETTER BUILD SENSOR FUSION.
WITH MULTI-MODAL DESCRIPTION IN
THE PAST I WANT TO SAY
MULTI-MODAL SHOULD BE THOUGHT OF
AS A MUST-HAVE BECAUSE IT'S WAY
WE PROS INFORMATION -- PROCESS
INFORMATION.
THIS IS MY GO-TO SLIDE, WAYS WE
CAN THINK ABOUT HUMAN-MACHINE
INTERFACE, SENSORS, ACTUATORS,
FROM A GENETIC PERSPECTIVE.
WE HAVE THE VISUAL WHICH WE
TALKED ABOUT A LOT, WE HEARD
ABOUT A LOT TODAY, FOR VERY GOOD
REASON.
OUR VISUAL SYSTEM IS, YOU KNOW,
CONSUMES A LOT OF BANDWIDTH IN
THE WAY THE INFORMATION IS
PROCESSING.
BUT THERE ARE HUMAN VISUAL AND
NON-HUMAN VISUAL SENSING, SOME
OF THE CHAT GOING ON TODAY
MENTIONED.
I.R. CAPABILITIES AS GOOD AS
VISUAL CAMERA CAPABILITIES.
BUT ARE NOT OFTEN CONSIDERED.
BUT COULD BE REALLY USEFUL.
SAME THING WITH OUR AUDIO
PROCESSING, THERE ARE VERY, VERY
GOOD MINIATURIZED MICROPHONES,
AS MANY OF YOU WHO USE THE
TECHNOLOGY KNOW.
ALSO THAT TECHNOLOGY IS NOT
LIMITED BY THE HUMAN BANDWIDTH.
ULTRASONIC SENSORS JUST AS GOOD
CAN BE LEVERAGES VERY WELL.
WE HAVE A LOT OF TOUCH-BASED
TECHNOLOGIES.
THEN A LOT LESS SOPHISTICATED
SMELL AND TASTE, USUALLY WHEN I
PRESENT THIS TO A MEN'S
AUDIENCE, THAT'S A GAP, BUT
THERE'S A LOT OF REALLY LOT OF
TECHNOLOGIES BUILT IN THE LAST I
WOULD SAY FIVE YEARS THAT HAVE
MADE BIG LEAPS IN TERMS OF
ABILITY TO BE USED IN VARIOUS
SETTINGS THAT INCLUDES VOCs,
ALCOHOL, CARBON DIOXIDE, CARBON
MONOXIDE GASES.
THIS OF THE SAME SENSES CAN BE
USED TO MEASURE NOT JUST HUMAN
REACTION BUT ALSO ENVIRONMENT'S
COMPLEXITIES, AS THE FIRST PANEL
SET UP.
IN THIS CONTEXT I WANT TO GIVE A
FEW EXAMPLES TO SHOW WHY, THEY
ARE ALL EXTRACTED FROM ONE
PAPER, QUITE OLD.
MORE THAN A DECADE.
BY NO MEANS ANYTHING NEW.
BUT I WANT TO SAY WE PROCESS
AUDIO AND VISION ALMOST
EXCLUSIVELY TOGETHER, BUT
SURPRISINGLY SOMEWHAT
PRIORITIZE -- THERE'S A
DISSONANCE BETWEEN INFORMATION
COMING VISUAL AND AUDIO CHANNEL,
WHEN WE PRIORITIZE WHAT IS AN
EVOLUTIONARY CONSTRUCT.
IF YOU HAVE BLINKING LIGHTS, 3
HERTZ, AUDIO AT 6 HERTZ, WE
CANNOT UNDO THAT NATURAL VERY
LOW TIME CONSTRAINT FUSION
THAT'S HAPPENING.
IT'S THE SAME THING WITH A LOT
TO DO HOW VISION AND MOTOR
CONTROL WORKS, CLASSIC
EXPERIMENTS.
KEEP BOTH YOUR HANDS IN FRONT OF
YOU, TRY TO MOVE THE LEFT LITTLE
FINGER AND RIGHT THUMB.
DO THIS AND TRY THE SAME THING.
MOVE YOUR LEFT FINGER, IT TAKES
LONGER, RIGHT?
WE HAVE UNDO THE OFFICIAL
CONTRADICTION.
THERE'S THIS SENSORY FUSION,
MULTIPLE TIME SCALES, ACROSS
MULTIPLE MODES OF SENSOR THAT WE
NEED TO BE COGNIZANT OF,
ESPECIALLY IF YOU'RE TRYING TO
THINK ABOUT USING A HUMAN
REACTION ACROSS ONLY ONE MODE.
AS REPRESENTATION OF SOMETHING
THAT'S MORE COMPLEX LIKE
BEHAVIOR.
HERE I'M SWITCHING TRACKS TO SAY
NOW THAT WE'VE MINIATURIZED
SENSORS AND PUT SENSORS
EVERYWHERE, THERE ARE A LOT OF
IDEAS ABOUT MODELS BEING THROWN
AROUND, UNDERSTANDING THIS LARGE
DATASET, I WANT TO HIGHLIGHT A
FEW IMPORTANT CONSIDERATIONS TO
KEEP IN MIND.
YES, IMAGE, ESPECIALLY VISUAL
DEEP LEARNING HAS COME VERY LONG
WAY IN A DECADE, BUT THE WHOLE
BASIS OF LOT OF DEEP LEARNING IS
THESE 14 MILLION IMAGES HAND
ANNOTATED, LAND TAXONOMYIZED,
HIERARCHICAL REPRESENTATION, NOT
DONE WITHOUT RESTRAINING DATA,
THERE'S THE HUMAN IN THE LOOP,
THE ONLY THING THE HUMAN, YOU
KNOW, HUMAN UNDERSTANDING OF THE
CONTEXT IS REMOVED BY MAKING IT
REALLY, REALLY SIMPLE, RIGHT?
IT'S ONE OBJECT, ONE BOUNDING
BOX.
THE CONTEXT IS LOST IN MANY
MODELS WHILE BUILDING THE MODEL.
THERE ARE SOPHISTICATED IDEAS
THAT ARE ADDED TO BUILD THE
CONTEXT BACK IN.
LARGE LANGUAGE MODELS THAT, YOU
KNOW, ARE REALLY ALSO GATHERED A
LOT OF MOMENTUM HAVE ALL THE
CONTEXT BUILT IN BECAUSE THEY
JUST ARE NOT SUPERVISED MODELS
BUT TENT TO -- TEND TO HAVE
OTHER ISSUES BECAUSE OF DATA
GOING IN.
THEY CAN ONLY REFLECT THE DATA
OF THE CONTEXT OF DATA FED INTO
THE MODEL.
THERE'S NOT REALLY EASY WAY TO
UNDERSTAND HOW TO VALIDATE THE
MODELS INDEPENDENT OF THE DATA
BECAUSE THEY ARE JUST FULLY
BUILT IN TOGETHER.
HAVING SAID THIS, I WANT TO SAY
MAYBE THERE ARE SOME VERY
CRITICAL LEARNINGS FROM THESE
MODELS, AND SOME ARE AROUND, YOU
KNOW, THE -- FOR EXAMPLE, LIKE
IF YOU THINK ABOUT MEDICAL
SENSORS, THERE'S A LOT OF
QUESTIONS ABOUT A SIMPLE
WEARABLE DEVICE, MEASURING HEART
RATE ESPECIALLY AROUND VARIABLES
NOT TAKEN INTO ACCOUNT, LIKE
GENDER AND, YOU KNOW, SKIN TONE
AND BMI.
AND WE'VE SHOWN, FOR EXAMPLE, IN
MY LAB THAT BY SIMPLY ADDING ONE
CONTEXT VARIABLE, ACTIVITY
CONTEXT, TO THE HEART RATE
SENSORS, YOU KNOW, PREDICTION OF
HEART RATE FROM THE RAW SIGNAL
CAN PREDICT, THIS IS THE IDEA OF
ADDING, YOU KNOW, CONTEXT HOW IT
CAN, YOU KNOW, MAKE MODELS BE
MORE RESPONSIVE, BUT AT THE SAME
TIME ALSO I WANT TO POINT OUT
THAT STILL THERE'S SYSTEMATIC
VARIATION, VARIABLE ALSO NOT
TAKEN INTO ACCOUNT IN DEEP
MODELS, YOU NEED TO BE AWARE OF
THIS IF YOU'RE BUILDING MODELS
THAT WILL BE USED IN REAL
APPLICATIONS.
ON A LARGE DIVERSE POPULATION.
SO I WANT TO END THIS WITH
PROPOSAL, LIKE A QUESTION.
IS THAT TO NATURALLY BUILD A
FRAMEWORK TO MODEL MULTIPLE
SENSOR FUSIONS, DOES IT MAKE
SENSE TO ACTUALLY TAKE CLUES
FROM LARGE LANGUAGE MODELS THAT
HAVE BUILT-IN HIERARCHY,
TEMPORAL HIERARCHY AND CONTEXT
AROUND UNDERSTANDING, FOR
EXAMPLE THESE ARE NOT REAL DATA,
BUT IDEAS THAT CAN THINK OF
GENERAL FRAMEWORK WHERE YOU
START WITH LIKE VERY LOW LEVEL
SENSOR, YOU KNOW, UNDERSTANDING
THAT THEN GOES ACROSS MULTIPLE
TIME SCALES SO YOU BUILD FROM
LIKE AN ALPHABET TO A WORD TO A
SENTENCE.
IS THERE IDEAS HERE THAT WE CAN
LEARN FROM THE LARGE LANGUAGE
MODELS THAT WE CAN THEN BUILD
DOMAIN-SPECIFIC MODELS TO
UNDERSTAND SENSOR FUSION DOMAIN.
WITH THAT QUESTION I WILL OPEN
TO --
>> THANK YOU.
THANKS VERY MUCH, RAJI.
NOW WE'RE SWITCHING TO THE
DISCUSSION PORTION.
THREE DISCUSSANTS, PETER
HARTWELL, BOB DATTA, ANDRE
GREEN.
YOU EACH HAVE NINE MINUTES IN
PRINCIPLE, BUT AS YOU SAW WITH
THE PREVIOUS DISCUSSION,
PREVIOUS PANEL, IT WAS VERY
LIVELY, PEOPLE CHIMING IN.
I'M GOING TO GIVE IT TO YOU
FIRST, PETER.
I'LL LET YOU HAND OFF TO THE
NEXT DISCUSSANT.
>> I'LL KEEP IT GOING.
THANK YOU, FLOH.
I'M PETER HARTWELL, I'M CTO,
THAT'S MY TITLE.
I LOOK AT THIS FROM A VERY
DIFFERENT WAY, IN FACT I REALLY
ENJOYED TODAY, THE DISCUSSION.
LET ME SEE, OUT OF MY ELEMENT,
IF I CAN TRY TO GET MY QUESTIONS
ANSWERED, MAYBE THAT'S GOING TO
HELP ALL THE FOLKS ONLINE.
IN PARTICULAR I'M GOING TO LEAN
BACK, PANEL 1, TRY TO TIE IT TO
PANEL 2 AS I GET THIS GOING.
WE'VE SEEN A LOT OF OTHER STUDY
OF DISEASE IN HUMANS, MECHANICS
IN ANIMALS, I APPROACHED
BEHAVIOR OF HUMANS FOR PROFIT,
FRANKLY, THAT'S MY JOB.
A LOT OF WHAT I DO IN THIS SPACE
IS DRIVEN BY MY CUSTOMERS,
CONSUMER ELECTRONIC FIELD, WHERE
THEY HAVE GOTTEN PRETTY GOOD
ABOUT IF YOU THINK ABOUT THE BIG
GUYS, ALL THOSE APPS AND DEVICES
YOU CARRY AROUND, UNDERSTANDING
AND DRIVING YOUR BEHAVIOR TO
KEEP YOU ON THAT APP OR TO GET
YOU TO GO BUY SOMETHING OR LOOK
AT SOMETHING.
AND THEY ARE DOING THAT BY
MONITORING YOUR SENSE AND YOUR
BEHAVIOR.
THE WAY YOU TYPE OR YOU SWIPE OR
PAUSE ON A SCREEN ON YOUR DEVICE
IS ACTUALLY MONITORING YOUR
BEHAVIOR.
THAT'S WHAT I DO.
I PROVIDE THE SENSORS THAT GO
INTO THAT, VERY MUCH WHAT RAJI
GAVE JUST BEFORE ME, INTRO HOW
SENSORS AND ELECTRONICS DRIVE
THAT, THINGS YOU HOLD, THINGS
YOU WEAR, THINGS YOU CARRY
AROUND.
AND I THINK MALCOLM, I WANT TO
START WITH YOU, BECAUSE YOU
BROUGHT UP A VERY INTERESTING
POINT AT THE END, WHICH WAS HOW
CAN WE, YOU KNOW, LOOK MORE
ACROSS THE FIELD.
SO I PUT ON THE SLIDE, AND I'LL
ASK YOU TO GET RID OF IT, MY
LANGUAGE, WE CALL IT SENSORY
FUSION, RAJI CALLED IT MULTIPLE
MODALITIES, HOW YOU PULL STUFF
FROM DIFFERENT AREAS.
AND TRY TO GET TO THE ROOT OF
WHAT IS SOMEONE DOING, ABOUT TO
DO, WANTED TO DO, THAT IS WHERE
WE APPLY THE LEARNING INTO THAT.
WE CALL IT ACTIVITY
CLASSIFICATION.
THE CONTEXT DETECTION WHERE IS
THE ENVIRONMENT, WHERE YOU'RE
AT, HOW IS THAT INFLUENCING YOU.
OUR GOAL IS YOU'RE SITTING IN A
RESTAURANT, YOUR PHONE DOESN'T
BLING, IT VIBRATES,
UNDERSTANDING THE CONTEXT OF
WHERE YOU ARE, CHANGING THE
BEHAVIOR OF THE DEVICE.
I WANT TO SHOW THAT SLIDE
BECAUSE WHEN I FIRST GOT ASKED
TO JOIN THIS I WAS LIKE, I DON'T
KNOW ABOUT ANYTHING ABOUT
BEHAVIOR, BUT ALL I DO IS TRY TO
UNDERSTAND WHAT HUMANS ARE DOING
WITH ELECTRONICS.
FLOH, GET RID OF THE SLIDE.
MALCOLM, LET ME TOUCH ON HOW DO
WE GO BROADER?
I'M LOOKING AT THE PANELISTS
HERE, A LOT OF YOU ARE AT
UNIVERSITIES.
THERE'S NOBODY HERE FROM MY
CUSTOMER SPACE.
>> ONE THING I THINK ABOUT FOR
YOU, I GUESS YOU DO A LOT OF --
DO DO YOU ETHOLOGY ON HUMANS?
OUR DEVICES HIJACK OUR BRAINS,
IT WOULD BE NICE TO KNOW MORE
ABOUT THAT.
ACTUAL ETHOLOGY, COLLECTING
CAREFUL BEHAVIOR, AND
APPROACHING FROM THAT ANGLE,
THAT'S ONE THOUGHT I HAVE GOING
DIRECTLY OFF WHAT YOU SAID ABOUT
GOING BROADER.
BUT THE OTHER THING I THOUGHT OF
IS IT'S REALLY INTERESTING THAT
YOU'RE THINKING AS WAS WHAT RAJI
MENTIONED, THINKING ABOUT
FUSION.
AND THE BRAIN USES FUSION ACROSS
MODALITIES TO DO ESSENTIALLY
NOISE REJECTION IN A WAY.
>> YES.
>> IF THERE'S A PART OF SPACE
THAT IS JUST MAKING A SOUND, BUT
NOT MOVING, VERSUS A PART OF
SPACE THAT IS NOT ONLY MAKING A
SOUND BUT ALSO MOVING, THAT CAN
BE THE MARK OF THE LIVING, THE
MARK OF SOMETHING THAT IS
ACTUALLY A DANGER OR AN
OPPORTUNITY.
WE'VE KNOWN FOR DECADES IN
NEUROSCIENCE THAT THERE'S CELLS
IN THE COLLICULUS, PARTS OF THE
MID-BRAIN EXCITED BY CONJUNCTION
OF STIMULI.
SEEMS LIKE YOU'RE COMING TO THE
SAME CONCLUSION IN A SENSE.
>> WE'RE DRIVEN BY A DIFFERENT
MOTIVATION.
I'LL SPIN BACK TO BEN.
VIDEO IS TOO POWER INTENSIVE FOR
ALMOST EVERYTHING WE DO.
THE SOLE EXCEPTION RIGHT NOW IS
IF YOU BUY THE MOST POPULAR VR
HEADSETS, IT'S TRACKING THE ROOM
WITH FOUR CAMERAS ON IT.
AND BUT AGAIN TWO HOURS, IN THE
META-VERSE BEFORE YOU UNPLUG AND
RECHARGE.
BUT, BEN, LOOKING AT -- YOU'RE
SO HEAVILY INTO VIDEO, 62
CAMERAS TO TRACK A MONKEY,
RIGHT?
I HAVEN'T GOT THE POWER TO DO
THAT.
WE'RE FUSING OTHER THINGS, RAJI
DID A NICE INTRO, DO YOU HAVE
ANYTHING BEYOND CAMERAS.
>> IN THE MONEY LAB WE HAVE A
NICE MICROPHONE TO IMPROVE
RECONSTRUCTIONS BUT SO FAR WE
HAVEN'T HAD ANY SUCCESS WITH
THAT.
WE REALLY HAVE OUR HANDS FULL
WITH THAT.
I WANT TO SAY LIKE WE USE THESE
VERY HIGH NUMBER OF CAMERAS,
BECAUSE WE HAVE TO, TO SOLVE OUR
PROBLEMS BECAUSE THE SPECIES
MOVES AROUND IN A COMPLICATED
WAY, THREE DIMENSIONS, JUMPS
AROUND, CONTORTS, TWISTS ITS
BODY.
WE USE LESS IF WE COULD GET AWAY
WITH IT.
WITH ONE CAMERA, YOU CAN GET A
LOT OF STUFF AND DO A LOT OF
GOOD SCIENCE.
I'M NOT A CHAUVINIST WHO
THINKS -- I THINK IT'S ALL
INTERESTING.
>> GALIT, YOU HAD WEARABLES ON
YOUR PIG, CLOSER TO MY WORLD,
RIGHT?
WE TRACK MY MAGIC RING AND
WATCH, WEARABLES LIKE THAT.
DO YOU FIND BEING ABLE TO FUSE
OR SYNCHRONIZE BETWEEN CAMERAS
AND SORT OF STRAP-ON SENSORS IS
A HARD PROBLEM OR SOMETHING YOU
GUYS HAVE TRIED TO --
>> SO WE'RE TRYING TO DO THAT.
WE HAVE THE WEARABLES.
AND WE HAVE THE VIDEOS.
AND WE ARE ACTUALLY VIDEOING THE
ANIMALS 24/7.
SO WE GATHER JUST A HUGE AMOUNT
OF DATA THAT IS UNREASONABLE FOR
STUDENTS TO SIT AND ANALYZE.
SO WE'RE ALWAYS ON THE LOOKOUT
FOR A.I., NEW TYPE OF
TECHNOLOGIES, HOW WE
SYNCHRONIZE VIDEOS WITH
WEARABLES.
SO FAR WE'RE NOT IN A WAY THAT
WE HAVE IT -- WHERE WE WANT IT
TO BE.
BUT THIS IS SOMETHING THAT WE'RE
LOOKING TO DO.
ACTUALLY HERE IN THIS PANEL
THERE'S SO MUCH GOOD IDEAS.
>> YEAH, NO, THOSE ARE THE --
THERE'S TWO OTHER -- WE'RE GOING
TO RUN OUT OF TIME.
I'LL PASS IT OFF IN A SECOND.
I WOULD LOVE TO ASK THAT
QUESTION, BEN, DO YOU HAVE TOO
MUCH DATA.
BUT I'M GOING TO SWITCH.
THE ONE I WANT TO ASK, A
PERSONAL EXPERIENCE HERE, FIVE
YEARS AGO MY KIDS WERE SORT OF 4
AND 6, I PUT A GO PRO ON THEIR
HEAD TO SO GETTING WHEN WE HAD
SIX FRESH FEET OF SNOW.
THEY FORGOT ABOUT IT.
THE MOST ENDEARING VIDEO OF MY
KIDS, TEN MINUTES OF BROTHER
WITH BROTHER AGAINST THE WORLD.
AS SOON AS I WALKED UP, AT THE
END OF THE VIDEO OLDER PUSHED
THE YOUNGER ONE IN THE SNOW.
THEY WERE A TEAM WHEN I WASN'T
OBSERVING.
MAYBE IN THE ANIMAL WORLD IT'S
NOT AS -- YOU'RE INFLUENCING THE
BEHAVIOR BY OBSERVING BUT IN THE
HUMAN WORLD, PARTICULARLY V R
THING, AS YOU'RE RUNNING THROUGH
THE FAKE RAT MAZE, ARE YOU
REALLY AFRAID OF THE ZOMBIE OR
NOT?
ARE WE INFLUENCING BEHAVIOR BY
THE ACT OF TRYING TO MEASURE IT?
>> I CAN SPEAK TO THAT LAST BIT.
YOU'RE TALKING ABOUT THE
DIFFERENCE IN INCENTIVES
STRUCTURES BETWEEN RODENT AND
HUMAN WORK, WE'RE THINKING HARD
ABOUT HOW TO MEASURE THE
AVERSIVE STRENGTH OF AN AIR
PUFF, WHICH IS UNPLEASANT FOR AN
ANIMAL, BUT WE'RE ALSO BASICALLY
TRYING TO MODEL PREDATOR-PREY
SEQUENCES HERE, IT'S NOT AS
AVERSIVE AS POSSIBLE DEATH.
IN THE CONTEXT OF VR WITH THE
GHOST IT'S KIND OF A STAND-IN
FOR WHAT THE REAL PENALTY IS.
THEY ARE GETTING PAID AS
SUBJECTS, PAY IS ACCORDING TO
HOW WELL THEY DODGE THE GHOST.
>> PERFECT.
>> SO THAT'S OUR WAY OF TRYING
TO LINE UP INCENTIVE.
WE CAN'T IN THE CONTEXT OF HUMAN
WORK HAVE ELECTRIC SHOCK
COLLARS.
>> THINKING BACK TO THE
GAMIFICATION OF BEHAVIOR FOR
PROFIT, WHERE I STARTED, I THINK
TO ANSWER YOUR QUESTION THERE'S
A WHOLE OVERLAP BETWEEN THIS
WORLD AND MAYBE IT'S WHAT'S
GOING ON IN SPACE RIGHT NOW, WE
NEED TO LOOK AT THE MODEL AND
HOW THESE TWO WORLDS CAN COME
TOGETHER AND REALLY ADVANCE WHAT
YOU'RE TRYING TO DO.
LET ME PASS TO ROB DATTA AND LET
YOU TAKE ROUND TWO.
>> THANKS.
I'M BOB DATTA, HARVARD MEDICAL
SCHOOL.
AND MAYBE TO FOLLOW UP ON WHAT
WAS BEING SAID, POSING MY FIRST
QUESTION, I THINK IT'S
INTERESTING, WE'VE HEARD ABOUT
SENSOR FUSION IN THIS BLOCK OF
TALKS.
I THINK IT'S WORTH NOTING
THERE'S PROBABLY A DEEP
RELATIONSHIP BETWEEN PROBLEMS
ENGINEERS ARE SOLVING WITH
SENSOR FUSION AND CHALLENGES
NEUROBIOLOGISTS FACE, HIGH DENSE
DATA STREAMS, AND FOR CAUSAL
RELATIONSHIPS, I SUSPECT THERE'S
AN ACTIVE DIALOGUE, NEURAL
BEHAVIOR RELATIONSHIPS, MANY ON
THIS CALL, THOSE IN THE REAL
WORLD TRYING TO SOLVE COMPLEX
SENSOR FUSION PROBLEMS, A THEME
I NOTICED.
THE SESSION INCLUDED A LOT OF
DIFFERENT PERSPECTIVES FROM
THINKING ABOUT NEW MODEEL
ORGANISMS TO EVOLUTION AND
SENSORS.
I WAS THINKING WHAT QUESTION
MIGHT UNIFY ALL THESE
PERSPECTIVES.
I REALIZED SOMETHING IAIAN SAID
MIGHT BE RELEVANT, POSING A
QUESTION TO MALCOLM.
RAJI AND PETER TALKED ABOUT
SENSORS, IMPORTANCE OF SENSORS
AND SENSOR FUSION, MALCOLM AND
BEN TALKED ABOUT BEHAVIOR FROM
THE PERSPECTIVE OF MOVEMENT.
AS WAS BROUGHT UP IN HIS TALK, A
LOT OF MOVEMENT IS ABOUT MOVING
SENSORS AROUND, CLOSED LOOP
BETWEEN BEHAVIOR AND SENSATION
IT'S NOT SO MUCH BUT THAT
BEHAVIOR GOVERNS INPUT.
IF WE'RE GOING TO UNDERSTAND
BEHAVIOR HOW IMPORTANT IS IT TO
GAIN ACCESS TO INFORMATION ABOUT
WHAT BIOLOGICAL SENSORS ARE
SENSING DURING BEHAVIOR?
IS PART OF THE GAME HERE NOT
JUST TO MEASURE BEHAVIOR BUT
ALSO TO UNDERSTAND WHAT THE EYES
ARE SEEING, WHAT THE EARS ARE
HEARING, AND WHAT THE NOSE IS
SMELLING, IF MY PERSPECTIVE?
>> IT'S A TOUGH JOB YOU'RE
DOING, SYNTHESIZES ACROSS TALKS.
A GREAT QUESTION.
I WOULD SAY, YOU KNOW, THE
PERSPECTIVE OF SENSORS, BEHAVIOR
BEING THE BODY'S WAY OF GETTING
SENSORS IN THE RIGHT POSITION,
INDEED I FEEL LIKE ONE OF THE
BIG ADVANTAGES OF THE SYNTHETIC
APPROACH SIMULATING THE ORGANISM
AND ENVIRONMENT, THE OPPORTUNITY
ONCE YOU HAVE GOOD HIGH RES, 3D
DATA WHERE AN ANIMAL IS IN
SPACE, TO RECONSTRUCT WHAT
SENSORS ARE GETTING AT THE
RECEPTOR LEVEL IN A CAREFUL WAY
AND USE THAT.
WHAT'S NICE ABOUT THAT, YOU
KNOW, YOU'RE NOT GOING TO GET IT
EXACTLY RIGHT BECAUSE THERE'S A
LOT OF NUANCE THERE, BUT WHAT'S
BEEN SUPER HELPFUL IN TERMS OF
THE EARLIER PART OF THE WORK
THAT I TALKED ABOUT IN MY BIT IS
THAT ONCE YOU'VE DONE THAT LEG
WORK ON WHAT'S HAPPENING IN THIS
CRAZY COMPLICATED CONDITION THAT
ISN'T FEASIBLE FOR BUSTING
NEURAL CIRCUITS IS THAT YOU CAN
TAKE WHAT YOU GAIN FROM THAT
SYNTHETIC APPROACH AND THEN PUT
IT INTO THE ANIMAL IN A CONTEXT
WHERE YOU CAN DO THE MORE
REDUCTIVE FORMS OF NEUROSCIENCE
THAT SOMETIMES NEED TO BE DONE.
IT'S NOT ALWAYS CONVENIENT TO DO
FREELY BEHAVING RECORDINGS IN
ANIMALS.
SOMETIMES YOU NEED TO PHOTON
HEAD FIX, FOR EXAMPLE.
BUT HAVING THE RECIPE FOR THE
SIGNAL FROM THE SYNTHETIC
APPROACH I THINK IS ALL
IMPORTANT IN TERMS OF GETTING
THE CIRCUITS TO BEHAVE IN THE
WAY THAT'S MAXIMALLY INFORMATIVE
FOR SUBSEQUENT INTERPRETATION.
SO, I THINK IT'S A REALLY GOOD
POINT AND, YEAH, THANKS FOR THE
QUESTION.
>> BEHAVIOR IS NOT JUST ABOUT
GETTING INFORMATION, THE GOAL IS
REWARDS AND SURVIVAL, RIGHT?
>> YEP.
>> I JUST WANT TO ADD THAT, YOU
KNOW, WE CAN TALK ABOUT SENSORS
STREAMS, BUT IN ANIMALS THOSE
ARE REALLY DEPENDENT ON THE
CONTEXT THEMSELVES.
THERE'S AUTONOMIC RECEPTORS ON
MUSCLE SPINDLING, MOTOR COMMANDS
TO YOUR EYES AN EARS.
IT'S HARD TO THINK ABOUT SENSORS
BEING DECOUPLED FROM OTHER
STAKES WE'VE TALKED ABOUT WE
NEED TO RECOGNIZE.
>> THAT'S A GOOD POINT.
>> THINKING ABOUT GETTING
ACCESSION TO SENSORS IS KIND OF
PREMISED ON GETTING ACCESS TO
INTERNAL SPEED.
>> YEAH, CONVENIENCE OF ELECTRO
SENSOR SYSTEM BUT DON'T HAVE IT
IN MANY OTHER SYSTEMS, LACK OF
EFFERENT CONTROL.
BUT SUBSEQUENT TO THE SECOND,
THIRD ORDER OF NEURONS
INFORMATION COMES INTO PLAY,
ABSOLUTELY TRUE.
>> BEN, THE AREAS OF THE BRAIN
YOU ARE RECORDING CONTAIN
SENSORY INFORMATION, MOTOR
INFORMATION, AND REWARD
INFORMATION.
HOW ARE YOU THINKING ABOUT
RELATING ALL THOSE TYPES OF
INFORMATION TO THE ONGOING
BEHAVIOR OF THE MONKEYS YOU'RE
RECORDING?
>> CAN YOU SAY THAT AGAIN?
>> YEAH, I WAS ASKING, YOU'RE
RECORDING THESE FRONTAL AREAS IN
THE MONKEYS WHICH ENCODE
ADMIXTURES OF SENSORY, MOTOR,
REWARD INFORMATION, AND HOW ARE
YOU THINKING ABOUT UNDERSTANDING
THOSE SIGNALS IN THE CONTEXT OF
COMPLICATED NATURALISTIC
BEHAVIORS WHERE MONKEYS ARE
RUNNING AROUND.
>> YEAH, I MEAN, I THINK KIND OF
THE BIG PICTURE, THE HIGHEST
LEVEL, THE ANSWER IS THE
OLD-FASHIONED WAY OF LOOKING AT
IT THAT I'VE BEEN DOING FOR 20
YEARS, LIKE WHAT DO NEURONS
REPRESENT, THAT WHOLE APPROACH
REALLY ISN'T WORKING VERY WELL,
IT'S NOT REALLY DRIVING US
TOWARDS ANSWERS, BUT I DON'T
THINK IT'S A GOOD DESCRIPTION OF
WHAT NEURONS ARE DOING.
WE'RE DISMANTLING IT AND
STARTING OVER SAYING WHAT CAN WE
SAY ABOUT THESE NEURONS.
WE COULD START WITH, SAY, WELL,
EVEN NEURONS THROUGHOUT THE
PREFRONTAL CORTEX HAVE A ROLE IN
CONTROLLING BEHAVIOR, SOMETHING
WE CAN SAY WITH CONFIDENCE.
WE'RE GOING AS FAR AS WE CAN
WITH THAT SAYING WHAT ELSE DO
THEY DO BESIDES THAT, WHAT ELSE
DO THEY DO BESIDES INDIRECTLY
CONTROLLING BEHAVIOR, SETTING
THE STAGE FOR BEHAVIOR, AND
THAT'S GOING TO WIND UP
EXPLAINING VARIANCE IN THE
NEURONS, JUST A PREDICTION
THOUGH.
>> ONE LAST QUESTION, TOUCHED ON
IN THE FIRST PANEL, A VERY BROAD
AND GENERAL CHALLENGE, GIVEN
DENSE MEASUREMENTS OF BEHAVIOR
HOW DO WE ASSIGN MEANING TO
MOVEMENT?
MANY SPEAKERS IN THIS PANEL ARE
MAKING HIGH DIMENSIONAL
RECORDINGS.
>> I'M SO SORRY, THIS IS DANA.
I DON'T WANT TO INTERRUPT BUT I
WANT TO MAKE SURE ANDRE HAS TIME
TO GET HIS THOUGHTS AS A
DISCUSSANT.
>> PLEASE GO AHEAD.
>> HE WILL GET HIS TIME, DANA.
WE STARTED LATE SO WE'LL RUN A
LITTLE BIT LATE.
I DON'T KNOW, BOB, YOU'VE GOT
IT.
>> I'LL QUICKLY FINISH AND MOVE
ON.
LOTS OF US ARE PILING UP HIGH
DIMENSIONAL DESCRIPTIONS OF
BEHAVIOR, AND IT'S NOT ALWAYS
CLEAR HOW TO UNDERSTAND WHAT THE
DATA MEAN.
IN THE SENSE THAT, YOU KNOW,
MOVEMENT AND BEHAVIOR AREN'T
SYNONYMOUS, RIGHT?
AS WAS MENTIONED IN HER TALK,
FOR THOSE ESPECIALLY GALIT
LOOKING AT ANIMALS, OCTOPUS,
PIGS, THERE'S A HUGE AMOUNT OF
BEHAVIORAL DATA THAT WE CAN
CENTER THINKING AROUND, HOW ARE
YOU THINKING ABOUT ASSIGNING
MEANING TO THE VARIOUS HIGH
DIMENSIONAL MOVEMENTS YOU'RE
OBSERVING?
>> SO, I DIDN'T SHOW -- FOR THE
OCTOPUS, FOR EXAMPLE, WE ARE
INTERESTED IN A VERY SPECIFIC
MOVEMENT WHICH IS THE GRASPING
MOVEMENT.
WE ACTUALLY TRAIN THE OCTOPUS TO
GRASP, SO WE CAN -- IT IS
INTENTIONALLY EXECUTING A VERY
SPECIFIC MOTION.
SO WE DON'T HAVE EVERYTHING THAT
HAPPENS IN THE TANK, OF COURSE
IN THE FUTURE WE WOULD LIKE TO
HAVE -- TO SAMPLE THAT
SIMULTANEOUSLY BUT WE DO FEEL WE
HAVE THE INTENTION OF THE
MOVEMENT.
>> ANDRE, WOULD YOU MIND TURNING
ON YOUR CAMERA AND JUMPING INTO
THE CONVERSATION THIS?
WE'LL GIVE YOU THE TIME AND
PLATFORM TO SPEAK, WE'LL CANCEL
THE BREAK.
>> THANK YOU, DANA.
ANDRE?
>> I APPRECIATE IT.
THANK YOU.
SO, I MIGHT MAYBE BE RELATIVELY
QUICK.
ONE THING I HEARD ACROSS ALL
TALKS, INCLUDING THE ONES IN
THIS PANEL, THIS KIND OF TENSION
BETWEEN WORKING IN CONTROLLED
ENVIRONMENTS WHERE WE CAN HAVE A
MUCH TIGHTER UNDERSTANDING OF A
PARTICULAR PROCESS, VERSUS THE
NATURAL ENVIRONMENT, HOW ARE
THINGS BEHAVING IN THE NATURAL
ENVIRONMENT.
AND WHAT I'VE BEEN THINKING A
LOT ABOUT IN TERMS OF WORKING
WITH THE MONARCHS, BECAUSE I'M
INTERESTED IN MONARCH MIGRATION,
HOW ARE THEY USING MULTI-SENSORY
CUES NAVIGATE, GOING TO SUCH
PLACES WITH INCREDIBLE
REGULARITY, WHEN I START MY OWN
EXPERIMENTS WE USE THE USUAL
ENVIRONMENTAL CUES, USUAL
SUSPECTS, AND I'M NOT ABLE TO
RECAPITULATE SOME OF THE
PHENOTYPES AND BEHAVIORS I'M
INTERESTED IN.
SO IT MADE ME TAKE A STEP BACK
AND ASK, WELL, HOW WOULD I IN AN
UNBIASED WAY DETERMINE WHAT IS
ENVIRONMENTALLY RELEVANT TO THE
PHENOTYPE THAT I'M LOOKING AT.
AND I STILL DON'T HAVE A GREAT
ANSWER FOR THAT.
BUT I'M THINKING ABOUT PETER AND
RAJI, WHO ARE THINKING ABOUT
MULTI-MODAL FUSION AND WE'VE
ALREADY MADE THIS REALIZATION
THAT OF COURSE IN THE NATURAL
ENVIRONMENT THAT WE WANT TO
UNDERSTAND BEHAVIOR IN THIS IS
WHAT MORE RECAPITULATES WHAT'S
ACTUALLY HAPPENING.
SO FOR THE FOLKS IN THE PANEL,
HOW DO YOU DETERMINE FOR YOUR
OWN WORK WHAT IN THE ENVIRONMENT
IS IMPORTANT, WHAT DO YOU TELL
KIND OF THE PEOPLE THAT YOU ASK
TO WORK WITH KIND OF WITH THE
SENSORS WHAT SHOULD THEY BE
MEASURING AND HOW DO YOU GUIDE
THEM WITH THAT?
>> I'LL TAKE A STAB AT ANSWERING
THAT.
I WOULD LOVE TOE BE -- TO BE AT
THE STAGE YOU'RE THINKING ABOUT,
YOUR SYSTEM, RIGHT AT THE
PRESENT TIME ALL THE EXPERIMENTS
WE DO WITH ANIMALS ARE FAIRLY
CONSTRAINED.
MINUS THE NEW WORK WHERE WE HAVE
COMPLEX HABITATS WITH RODENTS
RUNNING AROUND WITH ROBOTS
CHASING THEM, THAT'S
UNCONSTRAINED.
BUT THE EARLIER SENSORY WORK WE
DID A LOT OF THE ANALYSIS WE'VE
DONE IS SITUATIONS WHERE THERE'S
ONE OBJECT TO TRACK, SOMETHING
LIKE THAT, WHERE WE SIMPLIFY SO
THERE ISN'T A LOT OF AMBIGUITY
AS TO WHAT IS GOING ON.
ONE THING THAT'S EMERGED FROM
ANALYSIS ACROSS A BUNCH OF
SPECIES, WORKED WE PUBLISHED
LAST YEAR, ANALYZED DATA FROM
ELECTRIC FISH, COCKROACHES, AND
A TYPE OF RODENT, OR A TYPE OF
MOLE, THERE'S INFORMATION WAY TO
GET AT WHAT YOU'RE SAYING, FROM
AN INFORMATION THEORY
PERSPECTIVE WHEN AN ANIMAL IS
ENCOUNTERING SOMETHING THAT HAS
A RICH AMOUNT OF INFORMATION IN
THE SPACE, THERE'S
CHARACTERISTICS BEHAVIORAL
CHANCE WE DOCUMENT IN THE PAPER
INCLUDING REDUCING VELOCITY IS
SOMETHING THAT'S CHARACTERISTIC
OF ENTERING A HIGH DENSITY SPACE
OF INFORMATION FOR AN ANIMAL.
ANOTHER THING IS THERE'S A
FREQUENCY CHANGE IN THE WAY
THEY MOVE SENSORS, THEY START
INHABITING A DIFFERENT REGION OF
THE SPACE WHEN YOU DO ANALYSIS
AND FREQUENCY SPACE.
ONCE SENSORS ARE ACTIVATED, AND
WHEN THEY ARE NOT IN A HIGH
INFORMATION ZONE IT CHANGES TO
REQUIRE LESS INFORMATION.
THERE MIGHT BE INFORMATION
THEORETIC TOOLS THAT CAN START
TO HELP WITH THE PROBLEM THAT
YOU'RE TACKLING, WHICH IS NOT A
PROBLEM I'M YET TACKLING BUT I
THINK I WILL BE NEEDING TO SOON.
>> WE DO IT CLINICALLY, THERE'S
A BEHAVIOR, LIKE AS I WAS
TALKING ABOUT FREEZING OF GAIT,
WE NEVER CAPTURE IT IN THE LAB.
IT'S ALWAYS LIKE AFTER THE TRIAL
IS OVER, THEN THE PATIENT TURNS
AND THEN IT HAPPENS.
WHEN WE ACTUALLY CATCH IT, OH MY
GOD, KEEP RECORDING.
AND PEOPLE GO BACK INTO THIS
LIKE REAL -- MORE REAL TYPE OF
BEHAVIOR, REAR STRUGGLING WITH
THINGS WE NEED TO CONTROL IN THE
LAB TO ACTUALLY GET THE
BEHAVIORS THAT MATTER AND I
THINK DISTRACTION AND HIGH
COMPLEXITY OF ENVIRONMENTS IS
REALLY IMPORTANT.
AND OF COURSE AGAIN I THINK THIS
CAME UP, ANYTIME SOMEBODY WALKS
INTO OUR LAB IT'S JUST THEY WANT
TO MAKE SURE THEY ARE DOING
OKAY.
UNLESS THEY ARE A COLLEGE
STUDENT, THEY JUST WANT THEIR
MONEY.
THE CONTEXT, WHEN A PATIENT
COMES IN VERSUS A COLLEGE
STUDENT, IT'S REALLY DIFFERENT
TOO, THEIR MOTIVATION FOR EVEN
BEING THERE.
>> I'LL TAKE ONE QUICK STAB.
WE THINK ABOUT TRYING TO WAKE UP
YOUR PHONE IF IT'S ON THE DESK,
KNOWING YOU'RE NOT HOLDING IT
GOES INTO A DEEP SLEEP STATE.
IF I KNOW YOU'RE HOLDING IT
THAT'S ANOTHER ONE.
IF I KNOW YOU'RE LOOKING AT IT,
I LISTEN FOR AUDIO CUES TO SEE
IF THERE'S ANYBODY IN THE ROOM.
WE BUILD THE HIERARCHY INTO THE
ELECTRICAL SYSTEM.
YOU SORT OF TRIGGER ME, MALCOLM,
FOR US THE CURRENCY IS POWER,
IMPROVING THE USER EXPERIENCE
MAKING YOUR BATTERY LIFE BETTER,
MAKING YOUR DEVICE SLEEP MORE.
MAYBE I'M SYNTHESIZE --
[ NO AUDIO ]
OR EVOLVING, TO QUOTE SOMEBODY
EARLIER TODAY.
>> I THINK THERE'S ALSO A LITTLE
BIT OF CONTEXT AROUND THE
MISMATCH, BEHAVIOR CHANGE.
THAT'S THE COST, AUXILIARY COST
LIKE POWER FOR WHICH YOU CAN
OPTIMIZE IF YOU'RE DOING SYSTEM
LEVEL THINKING, ONE-TIME DESIGN.
AS A CONTINUOUS EVOLUTIONARY
USE, TEN YEARS AGO, FIFTEEN
YEARS AGO HOW MANY WOULD HAVE
DONE THIS, WAITING AT
INTERSECTION, YOU SNAP A PHOTO
OF THE SIGN BOARDS TO SEND YOUR
FRIEND.
THAT PICTURE IS NOT WORTH A
THOUSAND WORDS, IT'S WORTH WHO
WORDS, RIGHT?
SOME OF THESE BEHAVIORS, WHICH
SENSOR YOU CHOOSE WHEN, MIGHT BE
NOT PURELY ABOUT SYSTEM DESIGN
BUT ALSO CONTEXT AND THAT
PARTICULAR -- MORE LOCAL, RIGHT?
THERE ARE MULTIPLE TIME SCALES
RATHER THAN PURELY ABOUT --
THAT'S WHAT'S INTERESTING,
SOMEONE WHO HAD TO CHOOSE
BETWEEN BIOLOGY AND COMPUTER
SCIENCE, IN 11th GRADE, MY
MOST DISASTERROUS LIVE, GIVEN
THE CHANCE I MIGHT HAVE CHOSEN
THE OCTOPUS.
>> ANDRE, DO YOU HAVE OTHER
QUESTIONS FOR THE GROUP?
>> I THINK MANY OF MY QUESTIONS
WERE DEFINITELY COVERED AND OVER
BOTH OF THE SESSIONS.
AGAIN, I APPRECIATED HEARING
THESE, ALSO I'LL MENTION HERE
THIS THOUGHT OF CONSTRUCTING
VIRTUAL ENVIRONMENTS, ANALYZING
BEHAVIOR PAIRED WITH WHAT IS
IMPORTANT IN THE ENVIRONMENT
TRIGGERED ME PAIRING THOSE TWO
AND NOW I GUESS I'M GOING TO BE
THINKING ABOUT THAT FOR THE REST
OF THE DAY.
I REALLY APPRECIATE ALL OF THE
COMMENTS HERE.
>> THANK YOU.
TERRIFIC DISCUSSION.
THEMES I NOTICED, DIFFERENT
TEMPORAL AND SPATIAL SCALES,
BOTH, WITH RESPECT TO THE
ORGANISM, WHAT COULD BE MEASURED
IN AND AROUND, IF YOU WISH, ON
THE ORGANISM, ALSO WITH RESPECT
TO THE CONTEXT, CONTEXT IN MANY
WAYS OF INTERPRETING CONTEXT,
AGAIN HAS DIFFERENT SPATIAL AND
TEMPORAL SCALES, BUT THE
PROBLEM, NOT ONLY WHAT DATA IS
COLLECTED BUT HOW TO FUSE THE
DATA STREAMS AND SYNCHRONIZATION
ACROSS STREAMS.
AND INTERESTING COMMENT ABOUT
EFFECT OF OBSERVER AS WE
CONSTRUCT ENVIRONMENTS, AS WE
CONSTRUCT MEANS OF MEASURING ARE
WE ALTERING WHAT WE WANT TO
CAPTURE.
AND WITH THAT, WE'RE GOING TO
SKIP THE BREAK AND MOCH MOVE TO
THE THIRD PANEL.
>> LET'S GO TO PANEL 3.
FOUR SPEAKERS, FOUR
PRESENTATIONS, DR. DANI BASSETT
FROM UNIVERSITY OF PENNSYLVANIA,
DR. SCOTT LINDERMAN, STANFORD,
DR. MARYAM SHANECHI, AND DR.
GORDON BERMAN, EMORY UNIVERSITY.
WE HAVE THREE DISCUSSANTS TO
LEAD PANEL DISCUSSION AFTER THE
PRESENTATIONS.
DR. ALLISON WATERS FROM ICAHN
SCHOOL OF MEDICINE, DOLU
OBATUSIN, AND DR. TIM BROWN FROM
UNIVERSITY OF WASHINGTON.
EACH PRESENTATION WILL BE SHORT,
ABOUT 8 MINUTES OF TIME FOR THE
PRESENTERS, PLEASE BE MINDFUL OF
YOUR TIME SO WE HAVE ENOUGH TIME
AT THE END FOR DISCUSSION AND
QUESTION.
SO WE'RE GOING TO TAKE QUESTIONS
DURING THE PANEL DISCUSSION
PHASE AFTER ALL FOUR
PRESENTATIONS ARE COMPLETE.
LET'S MOVE TO THE FIRST
PRESENTATION BY DR. BASSETT.
PLEASE TAKE IT AWAY.
>> THANK YOU SO MUCH FOR THE
OPPORTUNITY TO BE PART OF THIS
WORKSHOP.
I'M REALLY EXCITED ABOUT THE
TOPICS THAT HAVE BEEN RAISED
THUS FAR.
I'M EXCITED TO PROVIDE YOU WITH
A LITTLE BIT OF OVERVIEW FOR
THIS PARTICULAR PANEL.
WHAT THE ORGANIZERS ASKED ME TO
DO IN THIS PANEL IS TO TALK
QUITE BROADLY ABOUT HOW WE CAN
THINK ABOUT BEHAVIOR, AS A
COMPLEX SYSTEM, IN THE NEXT
THREE TALKS MY COLLEAGUES WILL
BE DIGGING DEEPLY INTO SPECIFIC
INSTANCES OF THIS GENERAL IDEA.
SO YOU CAN SEE MINE IS A BROAD
OVERVIEW, AND WE'LL MOVE INTO
OUR SPECIFICS.
SO FIRST I WOULD SAY THAT I'M
GOING TO BE PROVIDING THIS
OVERVIEW FROM THE PERSPECTIVE OF
COGNITIVE NEUROSCIENCE, ONE OF
THE AREAS IN WHICH I WORK, AND
IN THAT FIELD THE TYPICAL VIEW
OF BEHAVIOR IS THE BRAIN CAUSES
BEHAVIOR WHICH PRODUCES A CHANGE
IN OUR ENVIRONMENTS.
AND WHAT'S INTERESTING IS THAT
IT FOREGROUNDS A SIMPLE
ONE-DIRECTIONAL CAUSAL
STRUCTURE.
THE BRAIN DRIVES BEHAVIOR IN A
ONE ONE-DIMENSIONAL LINE,
DRIVING THE ENVIRONMENT.
THE PREFRONTAL CORTEX DRIVES A
REACTION TIME, AND THAT MAKES AN
IMPACT ON THE ENVIRONMENT, WITH
A KEYBOARD PRESS.
THIS IS A KIND OF TYPICAL SET OF
UNITS OF INFORMATION, AND THEIR
RELATIONSHIPS WE WOULD VIEW AS A
GOOD STUDY OF BEHAVIOR IN
COGNITIVE NEUROSCIENCE.
I THINK THE CHALLENGE THAT MANY
OF US ARE FACING IS THAT OF
COURSE BEHAVIOR IS NOT
SEDENTARY, NOR RELATED TO BRAIN
AND ENVIRONMENT IN THIS SIMPLE
UNIDIRECTIONAL CAUSAL STRUCTURE.
SO FIRST I WANT YOU TO FOCUS ON
THE LEFT-HAND SIDE OF THE SLIDE,
THE POINT THAT BEHAVIOR IS NOT
UNITARY.
INSTEAD OF ONE LAW THAT IS
BEHAVIOR, OFTEN REACTION TIME IN
COGNITIVE NEUROSCIENCE, WE HAVE
MANY BEHAVIORAL BITS, FOR
EXAMPLE, MICROMOVEMENT OR HIGH
TWITCH, NOT VERBAL SOUND,
TIGHTENING OR RELAXATION OF
MUSCLES, THOSE BITS DISPLAY
STATISTICAL AND CAUSAL
DEPENDENCIES UPON ONE ANOTHER.
SECONDLY, SECOND CHALLENGE,
BEHAVIOR IS NOT RELATED TO BRAIN
AND ENVIRONMENT IN THIS SIMPLE
CAUSAL CHAIN SO IN FACT WHAT'S
HAPPENING IS MUCH MORE
COMPLICATED, VARIOUS PIECES OF
THIS BROADER SYSTEM INTERACT
WITH ONE ANOTHER IN BOTH PAID
FORWARD AND PAID BACK
DIRECTIONS.
FOR EXAMPLE, LONG-TERM BEHAVIORS
CAN CHANGE THE BRAIN BY ALTERING
INPUT, THE ENVIRONMENT CAN ALTER
THE BRAIN THROUGH PERCEPTION
WITHOUT ACTION, AND THE
ENVIRONMENT CAN DRIVE BEHAVIOR
THROUGH STRUCTURING CONSTRAINTS.
THE PICTURE IS MORE COMPLICATED
THAN THE SIMPLE LINEAR 3-POINT
BOX DIAGRAM.
I'LL PUT THE CHALLENGES
TOGETHER, IT'S NOT UNITARY,
BEHAVIOR IS NOT UNITARY, NOT A
SIMPLE CAUSAL CHAIN.
PUT THE PIECES TOGETHER WE HAVE
SOMETHING MORE COMPLICATED, WE
KNOW THE BRAIN IS COMPOSED OF
UNITS, INDIVIDUAL BRAIN REGIONS
INTERACTING IN A COMPLICATED
WAY, INTERACTIONS ARE DRIVERS
FROM EACH OF THE PIECES OF THE
BRAIN TO A PARTICULAR BEHAVIORAL
BIT, BEHAVIORAL BITS ARE
DEPENDENT ON ONE ANOTHER.
AND THEN THE THIRD INSTANCE IS
THE ENVIRONMENT, AGAIN THE
ENVIRONMENT IS NOT ONE THING,
IT'S MANY THINGS INTERACTING IN
A COMPLICATED WAY.
IN THE BROADER PICTURE MULTIPLE
UNITS EXIST AND INTERACT IN EACH
OF THE SUBSYSTEMS.
BUT MORE THAN THAT SINGLE UNITS
OR SMALL GROUPS OF UNITS CAN
INTERACT BETWEEN SUBSYSTEMS.
NOW, WE COULD ASK, YOU KNOW, ARE
WE DONE WHEN WE HAVE BRAIN
BEHAVIOR AND ENVIRONMENT IN THIS
BROADER STRUCTURE, AND I THINK
THAT MANY OF US IN THE FIELD
WOULD SAY, NO, OF COURSE WE'RE
NOT DONE, THE BRAIN IS NOT
NECESSARILY THE FIRST CAUSE
ALWAYS, MAYBE WE NEED TO PRESS
FURTHER BACK AND UNDERSTAND
PROCESSES IN THE MIND LIKE
BELIEFS, DESIRES, INTENTIONS,
PURPOSES THAT MIGHT NOT BE
IMMEDIATELY -- MIGHT NOT PRODUCE
ACTION, EXCEPT BY GOING FIRST
THROUGH THE BRAIN.
UNDERSTANDING BEHAVIOR I THINK
REQUIRES US TO RECKON WITH THE
PSYCHOLOGICAL PROCESSES
INVOLVED, UNDERSTAND THEY ARE
MAPPED TO THE BRAIN NETWORK, AND
THEN THE MAP TO THE BEHAVIORAL
NETWORK AND ENVIRONMENT BY
EXTENSION.
SO I THINK HAVING THESE FOUR
PIECES IN MIND IS IMPORTANT WHEN
WE THINK ABOUT A SINGLE
ORGANISM.
OF COURSE, THE PIECE OF THE
PICTURE THAT I HAVE NOT ADDED IN
HERE IS WHEN WE HAVE MULTIPLE
ORGANISMS, INTERACTING WITH ONE
ANOTHER IN SOCIAL SYSTEMS OR
OTHER COLLECTIVES.
I WANT TO DIG DEEPER AND SAY ARE
WE DONE?
THE ANSWER IS NOT QUITE BECAUSE
EACH OF THESE INTERACTIONS, BOTH
WITHIN AND ACROSS SYSTEMS, CAN
CHANGE IN TIME.
SO WITHIN THE MIND, INDIVIDUAL
MENTAL PROCESSES HAVE TIME
VARYING ARCHITECTURES, THE BRAIN
REGIONS HAVE TIME VARYING
INTERACTIONS, RELATIONSHIPS
BETWEEN EACH OF THESE SUBSYSTEMS
HAVE TIME-VARYING INTERACTION,
SO EVERY LINK IN THE PICTURE I
PLACED ON T ON BECAUSE THEY ARE
TIME VARYING, NOT STATIC, AND
THE CHANGE IN ONE PIECE CAN
ALTER THE CHANGE THROUGHOUT THE
ENTIRE SYSTEM.
AND MORE THAN THAT, ONE SORT OF
LAYER EVEN FARTHER IS THAT
UNITS, EACH OF THESE UNITS CAN
HAVE ASSOCIATED TIME SCALES,
TIME SCALES OF ACTIVITY IN THE
BRAIN, TIME SCALES OF ACTION AND
BEHAVIOR, TIME SCALES OF
STRENGTHENING OR WEAKENING OF
MENTAL PROCESSES IN THE MIND.
SO I PLACED A TAU IN EVERY UNIT,
IN THE SUBSYSTEM, FOR EACH
SUBSYSTEM ITSELF.
WHAT IS THIS?
THIS PICTURE THAT I'M SHOWING
YOU NOW?
THIS PICTURE IS A PICTURE OF A
COMPLEX SYSTEM.
IT IS A SYSTEM THAT IS COMPOSED
OF MANY INDIVIDUAL UNITS, THAT
INTERACT WITH ONE ANOTHER IN
COMPLICATED TIME-VARYING WAYS TO
PRODUCE VERY SOPHISTICATED
PHENOMENON THAT OCCUR ACROSS ALL
OF THE SUBSYSTEMS.
SO, UNDERSTANDING THIS COMPLEX
SYSTEM REQUIRES US TO BRING
COMPLEX SYSTEMS TOOLS TO THE
TABLE, TOOLS MEANING ALGORITHMS,
CONCEPTUAL FRAMEWORKS, THEORIES,
COMPUTATIONAL MODELS.
OF COURSE, YOU KNOW, YOU COULD
BE SITTING IN THE AUDIENCE
THINKING, HANG ON, EVERYTHING IN
SCIENCE IS REALLY COMPLICATED,
WE COULD SAY THIS ABOUT ANYTHING
WE STUDY, BUT ISN'T THE POINT TO
TRY TO SIMPLIFY, TO MAKE
ASSUMPTION AND FOCUS IN ON THE
VERY INDIVIDUAL PARTS AND HOW
THEY MIGHT INTERACT WITH ONE
ANOTHER?
SO, COULD I TAKE THAT BROAD
COMPLEX SYSTEM I SHOWED YOU AND
MAP IT DOWN INTO FOUR UNITS, AND
THIS SIMPLE UNIDIRECTIONAL
CAUSAL STRUCTURE?
THE ANSWER IS NOT QUITE.
NOT IF THE TIME SCALES OF THE
PARTS MATTER FOR HOW THE SYSTEM
FUNCTIONS.
THOSE ARE THE TAUS IN THE
PREVIOUS SLIDE.
NOT IF TEMPORAL VARIABILITY
MATTERS HOW THE SYSTEM
FUNCTIONS, THAT WAS OF THE
INDIVIDUAL T ON THE PREVIOUS
SLIDE.
ALSO NOT IF THE PATTERN OF
INTERACTIONS MATTERS FOR HOW A
SYSTEM FUNCTIONS.
IF TIME SCALE, TEMPORAL
VARIABILITY AND PATTERNS MATTER
WE CAN'T GET AWAY WITH THE
SIMPLIFIED PICTURE.
YOU COULD ALSO SAY, YOU KNOW,
HANG ON AGAIN.
IF WE DON'T HAVE THIS SIMPLE
CHAIN I'M SHOWING ON THE
LEFT-HAND SIDE HOW DO WE KNOW
WHEN WE BUILT A RIGOROUS
EXPLANATION, WHAT SORTS OF
EXPLANATIONS ARE EVEN POSSIBLE?
WHICH ARE CAUSAL?
AND WHICH ARE NOT CAUSAL?
ON THE LEFT-HAND SIDE WHAT I'M
SHOWING IS THE RELEVANT CAUSAL
CONCEPTS HERE CAN BE RELATIVELY
STRAIGHTFORWARD, A SIMPLE CAUSAL
DEPENDENCY BETWEEN TWO UNITS.
>> YOU HAVE ONE MINUTE.
SORRY.
>> THANK YOU.
WHEN WE HAVE COME PLEX COMPLEX
SYSTEM ON THE RIGHT THERE CAN BE
RELEVANT CAUSAL CONCEPTS, FIRST
IS CAUSAL DEPENDENCY, SECOND IS
STRUCTURING CAUSE, CAUSING
DYNAMICAL PROCESSES, TRIGGERING
PATHWAYS, MECHANISMS, CASCADES.
THERE'S BEAUTIFUL WORK IN THE
AREA OF VELOCITY, CONCEPTS THAT
MAY BE HELPFUL IN UNDERSTANDING
COMPLEX SYSTEMS.
THIS IS MY LAST SLIDE, WHAT I
THINK OF AS PATHS TO PROGRESS.
FIRST IS TO PROBE INTERACTIONS
WITHIN AND BETWEEN THESE
SUBSYSTEMS.
TO MAP TIME SCALES OF PARTS, TO
EXAMINE TEMPORAL VARIABILITY OF
INTERACTIONS.
AND LASTLY TO EXPAND OUR
UNDERSTANDING OF DIVERSE CAUSAL
CONCEPTS AND USE THEM TO DEVELOP
MORE PRECISE HYPOTHESES CLOSER
TO THE NATURE OF THIS COMPLEX
SYSTEM.
WITH THAT I'LL END AND CEDE THE
FLOOR TO MY COLLEAGUES.
>> NOW WE MOVE TO THE NEXT
PRESENTATION, DR. LINDERMAN.
TAKE IT AWAY.
>> ALL RIGHT.
GREAT.
I'M STILL WAITING FOR THE
CONTROLS.
THERE WE GO.
GREAT TO BE HERE.
I'M ASSISTANT PROFESSOR IN
STATISTICS DEPARTMENT, A
COMPUTATIONAL NEUROSCIENCE, TO
TRY TO UNDERSTAND AND GLEAN
INSIGHT INTO DATASETS SO TOPICS
OF TODAY'S DISCUSSION ARE NEAR
AND DEAR TO MY HEART.
AS I WAS PREPARING THE TALK FOR
TODAY, I WAS LOOKING AT THE
TITLE OF THE PANEL REALIZING
EQUALLY APPROPRIATE TITLE WOULD
BE TO CALL THIS MACHINE LEARNING
AND STATISTICS BECAUSE REALLY
GOING FROM MODELS TO -- DATA TO
MODELS AND BACK IS WHAT MACHINE
LEARNING AND STATISTICS AIM TO
DO, BUILD MODELS OF LARGE SCALE
DATASETS TO MAKE PREDICTION,
MAKE DECISIONS.
HOW IDEAS IN THESE FIELDS CAN
CONTRIBUTE, AS YOU'VE SEEN THE
BUZZ AROUND MACHINE LEARNING HAS
PERMEATED THE POPULAR PRESS.
SOME EXCITEMENT IS HYPE, A LOT
FOR GOOD REASON.
OPEN A.I., GPT 3 LANGUAGE MODELS
WRITE ESSAYS, NET FLEX'S
ALPHAGO, A REAL IMPACT ON
SCIENCE.
A TEAM AT STANFORD USED DEEP
NEURAL NETWORKS TO MAKE
PREDICTION ABOUT RNA STRUCTURE,
AND USED THOSE IN ALPHAGO TO
CONTROL PLASMA, AN IMPORTANT
STEP TO SUSTAINABLE ENERGY BASED
ON NUCLEAR FUSION.
WHAT ARE KEY IDEAS, KEY ADVANCES
BEHIND THESE?
ONE IS AVAILABILITY OF LARGE
SCALE DATASETS.
GPT 3 IS TRAINED ON A CORPUS OF
TEXT.
SECOND INGREDIENT IS ADVANCES IN
ALGORITHMS.
MANY VERY LARGE SCALE DATASETS
ARE UNLABELED LIKE THOSE WE
COLLECT, PART OF THIS FIELD.
ONE OF THE TRENDS TOWARD TAKING
ADVANTAGE OF LARGE SCALE
UNLABELED SETS IS
SELF-SUPERVISED LEARNING, WE
COME UP WITH SYNTHETIC PROBLEMS,
FILLING IN A MISSING WORD IN A
SENTENCE, USING PREDICTION
PROBLEMS TO TRAIN LARGE SCALE
MODELS.
THIRD IS HARDWARE.
WE ARE FAMILIAR WITH THE GPUs,
AND FOURTH IS ARCHITECTURE, DEEP
NEURAL NETWORKS WHICH HAVE
HUNDREDS OF BILLIONS OF
PARAMETERS LENDING THEMSELVES TO
COMPUTATION.
THE STRUCTURE OF THESE
ARCHITECTURES MAY NOT MATTER AS
MUCH AS THEIR FLEXIBILITY.
SO PIVOTING NOW HOW CAN WE TAKE
ADVANTAGE OF SOME OF THESE
ADVANCES IN MACHINE LEARNING,
APPLY THEM TO PROBLEMS IN BRAIN
AND BEHAVIORAL QUANTIFICATION?
TO ANSWER THIS QUESTION I THINK
IT MIGHT BE HELPFUL TO TRY TO
CATEGORIZE THE TIMES OF -- TYPES
OF PROBLEMS WE ENCOUNTER.
THESE ARE COARSE BUT IT MIGHT BE
A USEFUL FIRST PASS.
FIRST, SIGNAL EXTRACTION, HOW DO
WE GET SIGNALS FROM VIDEO,
POST-EXTRACTION WE'LL TALK ABOUT
SHORTLY.
SECOND CATEGORY HAS TO DO WITH
BUILDING ENCODING AND DECODING
MODELS, PREDICTING NEURAL
ACTIVITY GIVEN BEHAVIOR,
ENCODING SIDE, AND HOW CAN WE
PREDICT THE NEURAL SIDE,
IMPORTANCE IN INTERFACES,
ESTABLISHING THE UNDERLYING
NEURAL BASES OF BEHAVIOR.
TRANSLATIONAL END THESE ARE
IMPORTANT FOR UNDERSTANDING
BIOMARKERS OF DISEASE.
THIRD, MODELS THAT CAN SIMULATE
AND REPRODUCE PATTERNS.
THESE ARE BUILT FROM LATENT
VARIABLE MODELS, USEFUL FOR
DIMENSIONALITY REDUCTION AND
VISUALIZATION, BUT ALSO FOR SOME
TOPICS THAT HAVE COME UP ABOUT
INTERNAL STATE ESTIMATION, WHICH
IS AN IMPORTANT ASPECT OF THIS
TALK, AS IS INTERSUBJECT
VARIABILITY.
MANY ADVANCES IN MACHINE
LEARNING HAVE TARGETED PROBLEMS
OF DECISION MAKING AND CONTROL.
SO HOW CAN THOSE START TO
FEEDBACK INTO THE ANALYSIS OF
NEURAL AND BEHAVIORAL DATA?
ONE EXAMPLE IS TOWARD GUIDING
OPTOGENETICKIC STIMULATION TO
INDIVIDUALIZE TREATMENTS FOR
EACH PATIENT AND COUNTERACT
DISEASE AND DISORDER.
I'D LIKE TO HIGHLIGHT WORK
POINTING AND SHOWING
OPPORTUNITIES OF ADVANCES IN
THESE DIFFERENT DOMAINS.
SIGNAL EXTRACTION IS THE
CLEAREST OF THE FOUR, ONE WE'VE
SEEN A FEW EXAMPLES OF.
WE'VE HEARD ABOUT DEEPLAPCUT,
POINTS OF INTEREST, THE ANIMAL
FREELY NAVIGATES THE
ENVIRONMENT.
AND WHY IS THIS A GOOD SHOWCASE
OF ADVANCES IN MACHINE LEARNING
CONTRIBUTING TO BEHAVIORAL
ANALYSIS?
TAKE A FOUNDATION MODEL, TRAINED
ON A MAT OF MASSIVE DATASET OF
IMAGES, NOT NECESSARY FOR THIS
TASK, USE TRANSFER LEARNING TO
ADAPT TO THE PROBLEM OF
POST-TRACKING.
THIS IS SHOWING IMPROVEMENTS IN
ABILITY.
YOU CAN SEE FROM SLEEP IN THE
MIDDLE, HIGHLY ACCURATE
ABILITIES TO TRACK POINT OF
INTEREST.
THE SKELETONIZATION IS A HIGHLY
COMPRESSED VERSION, ONE TREND
THAT I'M SEEING THAT IS REALLY
IMPORTANT AND EXCITING IS TOWARD
A MORE COMPREHENSIVE
QUANTIFICATION OF BEHAVIORAL,
FINE GRAINED MEASUREMENT OF
OROFACIAL FEATURES, AS YOU SEE
ON BOTTOM RIGHT MAPPING OF THE
WHOLE SURFACE, SHAPE OF ANIMAL'S
BODY, THESE ARE THE TIP OF THE
SPEAR THAT IS POINTING TO SOME
OF THE POSSIBILITIES IN THIS
DOMAIN.
ENCODING AND DECODING, DECODING
IS THE PROBLEM OF INFERRING
BEHAVIORAL OUTPUTS FROM NEURAL
MEASUREMENTS.
ONE APPLICATION IS IN THE AREA
OF DEVELOPING BRAIN-COMPUTER
INTERFACE.
I'LL HIGHLIGHT WORK FROM LABS AT
STANFORD, THEY DEVELOPED A
BRAIN-COMPUTER INTERFACE FOR
DECODING HANDWRITTEN CHARACTERS.
USING THAT ACTIVITY TO GUIDE A
HANDWRITING --
>> SCOTT, YOU HAVE ONE MINUTE
LEFT.
>> ALL RIGHT.
I'LL SPEED THIS UP CONSIDERABLY.
WHERE DO WE GO FROM HERE?
THEY HAVE SHOWN THIS FOR
HANDWRITING, THE SAME STRATEGY
APPLIES TO OTHER BEHAVIORS LIKE
DECODING HIGHER DIMENSIONAL
BEHAVIORAL OUTPUTS, MUSCLE
ACTIVITY, SPEECH, ET CETERA.
THE KEY ASPECT OF THIS IS
BUILDING GOOD GENERATIVE MODELS
FOR THOSE TYPES OF TIME SERIES,
MORE THAN THE LANGUAGE MODELS
THEY USED IN THIS EXAMPLE.
THIRD EXAMPLE OF GENERATIVE
MODELING, I TOLD YOU HOW THESE
TOOLS ALLOW US TO GET LOW
DIMENSIONAL REPRESENTATIONS OF
NEURAL ACTIVITY.
ONE EXAMPLE FROM EVA DYER'S LAB,
DISENTANGLING CONTENT FROM STYLE
OF BEHAVIOR.
THEY ARE USING METHODS LIKE
VARIATIONAL AUTOENCODERS, I
THINK HERE AGAIN THERE'S TONS OF
ROOM FOR IMPROVEMENT AS WE
CONTINUE TO PUSH THIS FORWARD.
THIS IS AN AREA WHERE MY LAB IN
PARTICULAR IS EXCITED ABOUT THE
POSSIBILITIES FOR DEVELOPING
BETTER GENERATIVE MODELS FOR
BEHAVIOR.
I THINK WE HAVE THE INGREDIENTS
NECESSARY TO DO SO.
I'LL SKIP THIS LAST EXAMPLE.
THE FOURTH AREA WHERE M.L. CAN
CONTRIBUTE IS IN THIS ONLINE
ENCLOSED-LOOP CONTROL OF NEURAL
AND BEHAVIORAL SYSTEMS.
ONE EXAMPLE FOR MAJOR DEPRESSIVE
DISORDER HIGHLIGHTING WORK FROM
KATHERINE SCANGOS, LEADING A
BREAKOUT SESSION TOMORROW.
I THINK THIS IS AN AREA WHERE
M.L. STANDS TO GIVE MORE
SUMMARIES THAN JUST SYMPTOMS,
PATIENT-REPORTED SYMPTOM SCALES.
COLLABORATORS AT HARVARD HAVE
SHOWN HOW MOTION SEQUENCING
TECHNOLOGY CAN GIVE RICH
DESCRIPTION OF BEHAVIOR,
CANDIDATES FOR DEVELOPING BETTER
BIOMARKERS OF DISEASE TARGETED
WITH REINFORCEMENT ALGORITHMS
FOR CONTROL.
THIS IS FOUR AREAS, MY ATTEMPT
TO TRY TO ORGANIZE OUR THOUGHT
WHERE WE CAN AND HOW WE CAN
APPLY ADVANCES IN M.L. TO THE
DEVELOPMENT OF BETTER
TECHNOLOGIES FOR BRAIN AND
BEHAVIORAL MODELING AND
QUANTIFICATION.
THEY ARE NOT COMPLETE.
ONE THING THAT'S MISSING IS ANY
THEORY BUILDING.
THAT FALLS UNDER THE UMBRELLA OF
GENERATEIVE MODELING BUT
DESERVES ITS OWN PLACE ON THE
PODIUM.
AND I'LL LEAVE IT AT THAT,
LOOKING FORWARD TO THE
DISCUSSION WITH THE OTHER
PANELISTS.
>> THANK YOU, SCOTT.
WONDERFUL TALK.
WE MOVE TO THE NEXT
REPRESENTATION, DR. SHANECHI.
>> THANKS VERY MUCH.
A PLEASURE TO BE HERE TODAY AND
TALK ABOUT SOME OF OUR IDEAS ON
JOINT MODELING AND BY HEIFER
DEBEHAVIOR AND NEURAL ACTIVITY
FOR DECODING AND MODULATION.
THERE'S BEEN WORK ON DYNAMIC
MODELING, HERE SHOWING FROM ONE
TIME TO THE NEXT TIME STAMP.
IN LINEAR FORM.
THERE'S BEEN ALSO NON-LINEAR
DEEP LEARNING APPROACHES TO
BUILD THESE TYPES OF DYNAMIC
MODELS.
THE GOAL HAS BEEN TO DESCRIBE
VARIANCE, BUT WE'RE INTERESTED
IN THE NEURAL STATES THAT ARE
BEHAVIORALLY RELEVANT, NEURAL
ACTIVITY RELATING TO BEHAVIOR OF
INTEREST.
LET'S SAY ARM MOVEMENTS.
IN THIS CASE WE WANT TO EXTRACT
LATENT STATES.
WE CAN DECOMPOSE INTO THOSE JUST
IN NEURAL ACTIVITY, THOSE THAT
ARE SHARED, GREEN ONES, THE BLUE
ONES IN BEHAVIOR.
IF YOU USE UNSUPERVISED METHODS
WHAT HAPPENS?
WHAT CAN HAPPEN IS THAT WE MAY
EXTRACT RED STATES, THE NEW
DIRECTLY RELEVANT TO THE
BEHAVIOR OF INTEREST.
THERE'S REPRESENTATION THAT DOES
THE OPPOSITE.
FIRST, DON'T LOOK AT NEURAL
ACTIVITY, DYNAMIC MODEL OF
BEHAVIOR.
WHAT CAN HAPPEN IS WE MAY
EXTRACT THOSE BLUE STATES THAT
WHILE IN BEHAVIOR ARE NOT
ENCODED IN A PARTICULAR
RECORDING.
WE EXTRACT AND DISASSOCIATE.
I'M GOING TO SHOW WHAT BENEFITS
THIS CAN HAVE.
THE METHOD IS A LATENT STATE
DYNAMIC METHOD, RED AND GREEN
ELEMENTS, BASED ON IDEA OF
PROJECTION.
MORE IMPORTANTLY CAN PRIORITIZE
THOSE SHARED BEHAVIORALLY
RELEVANT STATES.
TO SEE HOW THIS CAN BENEFIT OUR
MODELING OF THESE BEHAVIORALLY
RELEVANT DYNAMICS APPLY METHOD
IN MONKEY DATASETS PERFORMING
NATURALISTIC REACHING, GRASPING
AND RETURNING MOVEMENTS.
LOOK AT EXPLANATION OF BEHAVIOR
FROM NEURAL ACTIVITY AND
DECODING OF BEHAVIOR.
WE FOUND THIS JOINT MODELING
ALLOWED US TO ACTUALLY MUCH
BETTER DECODE BEHAVIOR, COMPARED
TO REPRESENTATIONAL MODELS AND
UNSUPERVISED DYNAMICAL MODELS.
MOREOVER, BY DISASSOCIATING
RELEVANT DYNAMICS WE FOUND
BEHAVIORALLY RELEVANT DYNAMICS
HAD MUCH LOWER DIMENSIONALITY
THAT WE WOULD OTHERWISE
CONCLUDE.
THE SAME HELD COMPARED WITH
SEQUENTIAL.
THE OTHER APPLICATION THAT THIS
JOINT MODELING HAS IS FOR
TARGETED DIMENSIONALITY
REDUCTION, DYNAMIC MANNER.
WE USED THIS JOINT MODEL AND
UNSUPERVISED MODELS TO VISUALIZE
LOW DIMENSIONAL REPRESENTATION
OF NEURAL ACTIVITY, DURING THE
REACH AND RETURN MOVEMENT ALSO.
WHAT WE FOUND WAS THAT THE
ALGORITHM FOUND ROTATIONAL
PATTERNS THAT THEY ARE MORE
CONGRUENT WITH THE FACT THAT
MOVEMENT ITSELF WAS CHANGING
DIRECTIONS AS YOU CAN SEE,
ROTATIONS ARE CHANGING
DIRECTION, THE OTHER ROTATIONS
ARE NOT.
IF YOU WANT TO MAINTAIN IT WAS,
THE JOINT MODELING
(INDISCERNIBLE).
WHILE I'VE SHOWN LINEAR
INSTANTIATIONS WE CAN EXTEND TO
THE REALM OF NEURAL NETWORKS AS
WELL, ALLOWING FOR COMPLEX
NON-LINEAR MODELING.
THINK ABOUT BUILDING A MODEL
WHERE EACH PARAMETER CORRESPONDS
TO ONE OF THESE INTERPRETABLE
MAPPINGS, WE CAN DEVELOP
LEARNING ALGORITHMS THAT CAN
AGAIN DISASSOCIATE AND
PRIORITIZE SHARED DYNAMICS BY
COMING UP WITH COST FUNCTION
THAT ENCOURAGE NOT ONLY NEURAL
PREDICTION BUT ALSO BEHAVIOR
PREDICTION.
WE'VE SHOWED IMPROVING
DESCRIPTION OF NEURAL AND
BEHAVIORAL DATA.
FINALLY THE MODELS CAN BE USED
AS TOOLS FOR SCIENCE TO
DECOMPOSE TRANSFORMATION FROM
NEURAL ACTIVITY TO BEHAVIOR,
MAPPINGS, WHERE IS
NON-LINEARITY.
WE'VE FOUND CONSISTENT RESULT
ACROSS MULTIPLE DOMAINS,
NON-LINEARITY CAN BE SUMMARIZED,
SUGGESTING DOWNSTREAM CORTICAL
PROCESSING THAT CAPTURES
NON-LINEARITY.
WHAT ABOUT APPLICATION TO
NEUROTECHNOLOGY?
THESE DYNAMICAL MODELS CAN
BENEFIT BRAIN-MACHINE INTERFACE.
A SECOND APPLICATION COULD BE
BMIs THAT AIM TO REGULATE
ABNORMAL STATES, FOR EXAMPLE,
THE SAME MOOD IN DEPRESSION.
WE WANT TO BUILD THESE
CLOSED-LOOP STIMULATION SYSTEMS
THAT CAN DECODE MENTAL STATE
SUCH AS MOOD AND USE THAT AS
FEEDBACK TO ADJUST DEEP BRAIN
STIMULATION PARAMETERS, TAKE THE
BRAIN TO HEALTHY TARGET STATE.
BUILDING THESE CLOSED LOOP
STIMULATION SYSTEMS, FIRST THEY
CAN BE HELPFUL TO BUILD DECODER,
BY BIDDING DYNAMIC ENCODING
MODEL, HOW MOOD IS REPRESENTED
IN NEURAL SIGNAL.
IT CAN BE IMPORTANT IN BUILDING
INPUT-OUTPUT MODELS, HOW
STIMULATION INPUTS CHANGES
NEURAL ACTIVITY AND THEREFORE
UNDERLYING MENTAL STATE, FINALLY
MODELS CAN BE INCORPORATED
WITHIN FEEDBACK CONTROLLERS TO
ADJUST THE STIMULATION.
>> ONE MINUTE.
>> SURE.
SO, FOR EXAMPLE, WHEN WE TALK
ABOUT MENTAL STATES SUCH AS MOOD
OBSERVATIONS OF THESE MENTAL
STATES, BEHAVIORAL OBSERVATIONS
ARE IN THE FORM OF
QUESTIONNAIRES, I'M SHOWING
ECoG ACTIVITY, I SHOW THE
QUESTIONNAIRE WE GIVE PATIENTS
TO MEASURE THEIR MOOD,
ESSENTIALLY IPAD-BASED
QUESTIONNAIRE.
YOU CAN SEE IN THE CASE OF
MENTAL STATES, BEHAVIORAL
MEASUREMENTS ARE UNLIKE MOVEMENT
SPARSE.
WE HAVE SPARSE LABELS TO TRAIN
OUR MODELS.
WE'VE SHOWN WITH UNSUPERVISED
METHODS WE CAN DECODE MOOD,
THERE'S A LOT OF POTENTIAL IN
USING THESE JOINT APPROACHES TO
SIGNIFICANTLY IMPROVE THE
QUALITY OF DECODING.
WHILE WE'VE SHOWN IN
UNSUPERVISED MANNER THAT WE CAN
DESCRIBE NEURAL VARIATION
RESPONSE TO STIMULATION, THERE'S
A LOT OF POTENTIAL IN EXTENDING
THESE MODELS TO DESCRIBE THE
RESPONSE ON JUST MOOD RELEVANT
NEURODYNAMICS FOR EXAMPLE SO WE
CAN TARGET JUST THOSE DYNAMICS
WHILE MINIMIZING SIDE EFFECTS.
THERE'S A LOT OF POTENTIAL FOR
BUILDING LEARNING ALGORITHMS,
THAT CAN CONSIDER JOINTLY
BEHAVIOR AS WELL AS NEURAL
ACTIVITY, BOTH TO STUDY THE
NEURAL BASIS OF BEHAVIOR BUT
ALSO TO DEVELOP NEXT GENERATION
TECHNOLOGIES.
WITH THAT I'LL PASS THE FLOOR TO
THE NEXT SPEAKER.
>> THANK YOU SO MUCH.
WE MOVE TO THE NEXT AND LAST
PRESENTATION FOR THE PANEL, DR.
BERMAN.
TAKE IT AWAY.
>> THANK OF YOU.
THANKS FOR THE INVITATION.
AND YOU FOR STICKING AROUND TO
THE LAST TALK.
I'M GOING TO TALK TODAY ABOUT A
MORE OVERVIEW TALK, THINKING
ABOUT HOW DOES -- HOW CAN WE
DEVELOP A THEORY OF BEHAVIOR
THAT ALLOWS US TO MAKE SENSE OF
UNDERLYING NEURAL ACTIVITY AND
UNDERLYING STRUCTURES WE'RE
SEEING.
WHEN WE'RE TALKING ABOUT
BEHAVIOR, BEHAVIOR OFTEN HAS
THIS CONNOTATION OF THINKING
ABOUT HOW -- NOT JUST THE
PARTICULAR ACTIONS AND
MOVEMENTS, REALLY ONE OF THE
THINGS WE CARE ABOUT, ONE OF THE
KEY ASPECTS OF THE WORKSHOP,
THINKING ABOUT THE PARTICULAR
MOVEMENTS THAT AN ANIMAL IS
MAKING, ALSO WHAT ARE THE
UNDERLYING FEATURE ALSO THAT ARE
GENERATING THOSE MOVEMENTS.
HOW CAN WE MAKE SENSE OUT OF
THESE THINGS, FOR EXAMPLE
BETWEEN THE MOVEMENTS AS
FUNCTION OF ACTIVITY LEVEL DUE
TO CIRCADIAN RHYTHM, HUNGER,
AGE, SOCIAL INTERACTIONS, AND
THIS IS VERY IMPORTANT FOR
THINKING ABOUT CLINICALLY
RELEVANT THINGS LIKE MOOD
DISORDERS OR OTHER TYPES OF
THINGS WE MIGHT BE CARRYING
ABOUT AS WE THINK ABOUT THIS.
OF COURSE, THERE'S BEEN -- WHEN
I SAY THERE'S BEEN A PAUCITY OF
THEORY TO THINK ABOUT HOW DO WE
UNDERSTAND THESE INTERNAL STATES
OF BEHAVIOR AND HOW THIS
ACTUALLY WINDS UP RELATING TO
THOSE OUTPUTS.
BUT THERE'S BEEN A LOT OF
LINGUISTIC THEORY THAT'S BEEN
AROUND, WHAT I MEAN BY THAT IS
BEAUTIFULLY WRITTEN THOUGHTFUL
IDEAS THAT COME FROM, SAY, FOR
EXAMPLE, THE CLASSICAL ETHOLOGY
WORKS FROM THE 50s, '60s,
70s, THE STRUCTURED BEHAVIOR
OFTEN IS A FULL REPERTOIRE, NOT
JUST ONE THING AN ANIMAL IS
DOING TO UNDERSTAND ONE THING
AND UNDERSTAND HOW IT'S DOING IT
BUT WHERE DOES THAT FIT IN THE
CONTEXT AMONGST THE REPERTOIRE
OF BEHAVIORS AND THINKING ABOUT
HIERARCHY OF STATES, INTERNAL
STATES, TRYING TO UNDERSTAND HOW
THOSE THINGS ARE INTERACTING IN
TIME.
WE CAN LOOK TO SEE -- THIS IS
SIMPLISTIC MODELS, IF WE LOOK AT
BEAUTIFUL WORK OTHER PEOPLE HAVE
DONE, THIS IS GREAT WORK
INCLUDING BY SEVERAL PEOPLE THAT
ARE HERE ON THE PANEL TODAY,
WHAT WE SEE IS THESE TYPES OF
MODELS FOR DESCRIBING BEHAVIOR
THAT HAVE A TENDENCY TO HAVE A
SHORT TIME SCALE AND BE
DISCRETE.
THERE'S A LOT OF JUSTIFICATION
WHY THAT MIGHT BE A GOOD WAY TO
TALK ABOUT BEHAVIOR.
OR WE SAY, OKAY, I'M GROOMING, I
AM COURTING.
I AM DOING SOMETHING ELSE.
OKAY, I'M GOING TO SWITCH
BETWEEN THINGS.
THAT DOESN'T NECESSARILY --
OOPS -- MAP ON BETWEEN -- THIS
IS THE PICTURE.
I'M SORRY, THE FONTS GOT
TRANSLATED BETWEEN THE MAC AND
P.C.
WE SEE THIS PICTURE WHEREAS TIME
MOVES ALONG, OUR BEHAVIORAL
OUTPUTS ARE ESSENTIALLY A
FILTER.
>> ONE MINUTE REMAINING.
>> ONE MINUTE REMAINING?
OKAY.
ESSENTIALLY A FILTER BETWEEN
WHAT'S THE INTERNAL STATE AND
EXTERNAL STATE.
TO MOVE OUT, HOW CAN WE THINK
ABOUT THIS FROM THE STANDPOINT
OF HOW OTHER PEOPLE GENERATED
MODELS?
I'M GOING TO WHAT MALCOLM TALKED
ABOUT, AND GO BACK TO THE HIGGS
BOSUN.
AND THEORETICAL PARTICLE
PHYSICS, UNDERLYING THEORY
INVOLVES WRITING COMPLICATED
EQUATIONS, PHENOMENONOLOGY
BRINGS THE THING INTO THE WORD
OF THE MEASURABLE, IT'S A
COMPLICATED EVENT, WE WIND UP
GETTING THESE THINGS.
SAY I'M LOOKING AT THIS WATER,
CALCULATE THE SPEED OF THE WAVE,
WHAT THE AMPLITUDE IS.
IF I WERE A STATISTICIAN, I
WOULD SAY, OKAY, OR MINIMALLY
THIS IS WHAT MODERN MACHINE
LEARNING IS DOING.
I HAVE MY HEIGHT, EXTERNAL
VARIABLES, I'LL TAKE DATA, TRAIN
A MODEL TO PREDICT THAT, CORRECT
LIP OUTPUT MY PREDICTION.
IF I'M ABLE, ANOTHER WAY TO
THINK ABOUT IT, CAN WE ISOLATE
RELEVANT VARIABLES FROM OUR
ENVIRONMENT THAT WIND UP BEING
PREDICTIVE?
AND FROM THAT THIS IS WHERE YOU
WOULD START WITH A BROAD --
>> THANK YOU, GORDON.
YOU STILL HAVE A FEW MINUTES
REMAINING.
SORRY ABOUT THAT.
>> I THOUGHT THAT WAS A BIT
QUICK.
ALL RIGHT.
WHEW.
BASICALLY WHAT WE WIND UP HAVING
IS YOU START OFF WITH SOMETHING
WHICH LOOKS LIKE A NAVIER-STOKES
EQUATION, CONSERVATION OF MASS,
MOMENTUM, TRANSLATIONAL AND
ROTATIONAL VARIANTS.
YOU SAY, OH, THERE'S RELEVANT
FEATURES I'M TRYING TO PREDICT,
WHICH IS THE WAVELENGTH AND
AMPLITUDE OF THAT WAVE.
AND THEN FROM MAKING A SET
APPROXIMATIONS, I HAVE A
PREDICTIVE MODEL.
THIS TAKES KNOWING A LOT ABOUT
YOUR SYSTEM, SOMETIMES IT'S
COMPLICATED, YOU CAN'T DO IT IF
YOUR SYSTEM IS TOO COMPLICATED
LIKE ANIMAL DEVELOP OF BEHAVIOR,
GENE EXPRESSION, EVOLUTION,
EVERYTHING.
AND IN PHYSICS WE CAN DEAL WITH
THIS A LITTLE BIT BY THINKING
ABOUT WHAT'S CALLED
RENORMALIZATION, YOU CAN SAY ALL
THESE TINY SKINS POINTING UP AND
DOWN, I CAN FIND COURSE GRADING
VARIABLES, GROUP INTO ONE BIG
SPIN, I EVENTUALLY FIND THESE
ARE THE VARIABLES THAT ARE
RELEVANT FOR ME UNDERSTANDING MY
SYSTEM.
IN OUR GROUP WE THINK ABOUT
THESE THINGS.
THIS SORT OF -- TAKE THIS
METAPHORICALLY.
THE IDEA OF IMAGINING A BALL
ROLLING THROUGH THE LANDSCAPE, A
DETERMINISTIC PART OF THIS, LIKE
A BALL IN A WELL, SOME NOISE,
PUSHING IT AROUND.
EACH BOUNDARIES MIGHT BE A
STEREOTYPE BEHAVIOR.
WHAT HAPPENS IS THIS MOVES
AROUND, THAT SHIFTING IS THE
INTERNAL STATES, HOW TO
UNDERSTAND IS WHAT MATTERS.
I'LL SKIP THIS.
THAT'S REALLY -- THERE'S BEEN A
LOT OF WORK INCLUDING STUFF THAT
WE'VE DONE, INCLUDING BEAUTIFUL
WORK BY GREG STEVENS, LENA'S
WORK, IN BIOLOCOMOTION.
THIS IS THE THING THAT'S
IMPORTANT TO UNDERSTAND HOW TO
BUILD THEORETICAL MODELS OF
BEHAVIOR.
ONE OF THE BIG ISSUES,
DISCRETENESS.
WE KNOW THIS CAN'T BE TRUE IN
REALLY IMPORTANT WAYS.
I'M SHOWING TWO BEAUTIFUL
EXAMPLES FROM FRUIT FLIES.
ONE FROM DAVID ANDERSON'S GROUP
WHERE YOU SEE BASICALLY BY
CHANGING THE LEVEL OF ACTIVATION
OF THE PARTICULAR NEURAL
CIRCUIT, YOU CAN GET DIFFERENT
TYPES OF ACTIONS, THIS
QUALITATIVE WAY.
QUANTITATIVE WAY.
OR IN THESE FRUIT FLIES,
DIFFERENT SPECIES HAVE SIMILAR
NEURAL CIRCUITRY TOGETHER BUT AS
YOU WIND UP GETTING INCREASING
ACTIVATION DIFFERENT BEHAVIORS
CAN RESULT IN DIFFERENT
EVOLUTIONARY BEHAVIORS.
THE PICTURE WE HAVE IN OUR HEAD,
BASICALLY SETS OF INTERACTING
COMPLICATED -- ONE MORE SLIDE
AFTER THIS -- INTERACTING
COMPLICATED STATES, BASICALLY
LOOKING PUSHING AND PULLING
INTERACTING WITH EACH OTHER,
FILTER BACK AND THEN THE
BEHAVIOR FILTERS BACK INTO
EVERYTHING ELSE.
THIS IS GETTING REALLY TOWARDS
WHAT DANI WAS SAYING IN THE
FIRST TALK OF THE SESSION,
THINKING ABOUT THESE COMPLICATED
DYNAMIC FEEDBACK LOOPS, REALLY
WHERE THEORY IS GOING TO COME
IN, UNDERSTANDING HOW TO INFER
IN A DYNAMIC RATHER THAN
STATISTICAL WAY.
LASTLY, I'LL JUST ADD WHAT WE'RE
TRYING TO DO, DATA MODELS AND
BACK, WE NEED HIGH QUALITY
BEHAVIORAL DATA OVER LONG TIME
SPANS, WHERE A LOT OF PROGRESS
HAS BEEN MADE AS YOU'VE SEEN.
REALLY THE KEY HERE, THE POINT I
WANT TO LEAVE WITH, WHAT WE NEED
TO BE ABLE TO DO IS THINK ABOUT
MODELING IN THEORY FROM THE
STANDPOINT OF PHENOMENOLOGY.
THIS IS WHERE IN SOME WAYS I
WORRY ABOUT USING FULL THROATED
MACHINE LEARNING APPROACHES,
BECAUSE OF THE IDEA THAT WHAT
WE'RE TRYING TO DO IS BRING
THINGS TO THE LEVEL WHERE WE CAN
UNDERSTAND THE INs AND OUTs
IN THE SYSTEM AND MAYBE IT'S THE
ARROGANCE THAT COMES WITH
WORKING IN A LOT OF SYSTEMS BUT
I THINK THAT SOME LEVEL, MUCH IS
UNDERSTANDABLE, THE PROBLEM IS
HOW CAN WE BUILD THEORIES AND
MODELS TO BUILD THAT TYPE OF
UNDERSTANDING.
WITH THAT I'LL CLOSE.
>> THANKS SO MUCH, GORDON.
A LOT OF THINGS TO THINK ABOUT.
WE HAVE COMPLETED ALL FOUR
PRESENTATIONS, DOING VERY WELL
ON TIME.
NOW WE MOVE TO PANEL DISCUSSION.
WE ARE GOING TO HAVE THREE
DISCUSSIONS, EACH A SINGLE IDEA
OR QUESTION, DISCUSSIONS ARE
ALSO OPEN FOR QUESTIONS AND
COMMENTS FROM AUDIENCE.
EACH DISCUSSION WILL HAVE NINE
TO TEN MINUTES, BE MINDFUL OF
TIME.
FIRST DR. ALLISON WATERS.
DR. WATERS, THE FLOOR IS YOURS.
>> OKAY.
THANKS, DANI, SCOTT, MARYAM,
GORDON, A WHIRLWIND TOUR.
WE DID IT, FROM DATA TO MODELS
AND BACK, AS WE WERE MANDATED.
WE COULD TALK ABOUT SO MANY
THINGS NOW.
AND SO I SEE THAT WE'RE POISED
TO DISCUSS A LITTLE BIT ABOUT
APPLICATION.
WE COULD GO IN LOTS OF
DIRECTIONS BUT I THINK MAYBE
I'LL REPRESENT MY GROUP'S
MANDATE, HERE AT THE NATIONAL
FAMILY CENTER FOR ADVANCEED
CIRCUIT THERAPEUTICS, WE, LIKE
YOU IN YOUR DOMAINS, WANT TO
LEVERAGE THESE MODELING
INNOVATIONS, TO SPECIFICALLY
BETTER INTEGRATE PRECISION
NEUROMODULATION WITH BEHAVIORAL
INTERVENTION FOR
NEUROPSYCHIATRIC PROBLEMS LIKE
CBT, ERP, WHICH I WOULD ARGUE IS
ACTUALLY THE CHANGE STRATEGY
WHAT WE NEED TO BE OR ENHANCE
WITH OUR DIRECT TO BRAIN
TECHNIQUES.
SO I'M KEEPING IN MIND THIS
ANECDOTE THAT LENA ALREADY
SHARED ABOUT THE FREEZING OF
GAIT IN THE LAB, HOW MAYBE WE
WANT TO STUDY TINKING PHENOMENON
IN TOURETTEEST BUT BRING THE
PATIENT IN, THEY DON'T TIC FOR
US.
WE'RE GOING TO WIRE THEM UP,
SEND THEM INTO THE DAILY COMMUTE
WITH THEIR VIDEOS AND THEIR
SENSORS, MICROPHONES AND SO ON.
I'M WONDERING HOW DOES CENTERING
ON THAT PATIENT'S EXPERIENCE
HIGHLIGHT GAPS OR OPPORTUNITIES
IN MODEL BUILDING.
IN BEHAVIORAL MEDICINE THE
PATIENT IS THE CHANGE AGENT, NOT
A PASSIVE OBSERVER OF EVENTS AND
RESPONSES, LIKE WE ARE THROUGH
THE LENS OF DATA COLLECTION AND
MODEL BUILDING.
HOW DO EMERGING TECHNOLOGIES
KEEP THE PATIENTS IN THE
DRIVER'S SEAT?
IT'S NOT AN ETHICAL QUESTION.
MORE LIKE A PRACTICAL QUESTION.
HOW DO WE TAP THE FORCE OF THE
INDIVIDUAL TRYING TO MAKE CHANGE
IN THEIR LIVES?
>> THE QUESTION CAME TO MIND
WHEN DANI WAS SPEAKING, AND
BOLDLY INCLUDED THE MIND IN HER
SCHEMATIC.
IT ALSO COMES TO MIND WHEN
SCOTT'S DISCUSSING, SAY, MACHINE
LEARNING TECHNOLOGIES TO DRIVE
CLOSED-LOOP APPROACHES.
OR MARYAM, WHAT YOU PROPOSED IN
BEINGS ABLE TO SIMULTANEOUSLY
MODEL BEHAVIOR AND OF IN BRAIN
AND I'M WONDERING IF THAT
PARADOXICALLY GIVES MORE FREEDOM
TO INVOLVE THE HUMAN IN
SELECTING THE MODEL INPUTS,
INCLUDING PERHAPS CLOSED-LOOP
SYSTEMS.
>> BY INPUT, YOU MEAN THE TYPE
OF THERAPY, FOR EXAMPLE, OR WHAT
DO YOU EXACTLY MEAN BY INPUT?
>> WELL, FOR YOU, WHAT IS THE
ROLE OF THE INDIVIDUAL'S
EXPERIENCE, THEIR AGENCY, IN HOW
YOU THINK ABOUT AND INNOVATE
YOUR COMPUTATIONAL MODELING
APPROACHES?
>> UH-HUH.
SO IN CLOSED LOOP WE CAN
DEFINITELY HAVE SOME SORT OF
COOPERATIVE TYPE SYSTEM WE
DESIGN WHERE WE DON'T JUST LET
THE MACHINE DICTATE THE OPTIMAL
THERAPY PARAMETERS OR DOSAGE,
FOR EXAMPLE.
WE ALLOW THE INDIVIDUAL TO ALSO
HAVE A SAY IN THAT.
FRICTION, THE SIMPLEST VERSION
OF THIS IS AN OVERRIDE, YOU CAN
OVERRIDE WHATEVER THE MACHINE
DECIDES IS BEST FOR YOUR MENTAL
STATE.
WE CAN THINK ABOUT COOPERATIVE
DESIGNS THAT ALLOW AGENCY TO THE
PATIENT IN TERMS OF HOW THEY
WANT TO BE TREATED.
FOR EXAMPLE, MAYBE THE MACHINE
CAN DISPLAY WHAT IT THINKS THEIR
MENTAL STATE IS.
AND ON THE BASIS OF MAKING A
DECISION ABOUT THE NEXT LEVEL OF
SIMULATION, AND THEN THE
PATIENT, IF THEY DON'T, I DON'T
KNOW, THAT'S SOMETHING THEY
DISAGREE WITH MAYBE THEY CAN
HAVE AN INPUT THERE.
WE CAN PUT THAT WITHIN THE
COMPUTATIONAL FRAMEWORK TO MODEL
THIS KIND OF INTERACTION BETWEEN
THE HUMAN AND THE MACHINE THAT
IS ALSO A VERY INTERESTING
PROBLEM TO THINK ABOUT.
>> DANI WHY DID YOU CHOOSE TO
MIND AND NOT MAKE THAT PART OF
BRAIN BEHAVIOR?
>> I THINK OF BRAIN AS SOMETHING
THAT DOES CONNECT TO THE BODY
WITH THE ENVIRONMENT.
AND SOMETIMES THERE ARE MENTAL
PROCESSES THAT DON'T NECESSARILY
COME OUT AND AFFECT THE
ENVIRONMENT.
I WANTED TO HAVE A PLACE IN
SCHEMATIC DIAGRAM WHERE THE MIND
CAN BE DOING THINGS THAT DON'T
COME OUT BUT THAT COULD DRIVE
CHANGES THROUGH THE BRAIN, INTO
THE ENVIRONMENT.
I THINK THAT HAVING THAT SECTION
IS IMPORTANT.
THOSE INTENTIONS, BELIEFS,
GOALS, OR HOPES OR MOTIVATIONS,
WE STILL DON'T HAVE A FULL
UNDERSTANDING OF HOW THOSE
MENTAL PROCESSES ARE
INSTANTIATED IN THE BRAIN AND
THEN FROM THERE HOW BEHAVIOR IS
ALTERED.
FEELS LIKE THAT SECTION NEEDS
TO BE THERE BECAUSE WE DON'T
UNDERSTAND IT.
I THINK YOUR IDEA, YOUR EXAMPLE
OF THE PERSON WHO DOESN'T TIC IN
THE LAB BUT DOES IN THEIR
EVERYDAY LIFE IS FASCINATING,
SUGGESTING IMPORTANCE, FOR ME
WHAT THAT DOES IS SUGGESTS
THERE'S A DYNAMIC REGIME THE
PERSON IS WORKING IN, WHEN YOU
CHANGE THE ENVIRONMENTAL CONTEXT
THEY ARE IN A DIFFERENT WORKING
REGIME, THAT OF THE LAB.
THE GOAL IS TO UNDERSTAND HOW DO
YOU MOVE FROM ONE DYNAMICAL
REGIME TO ANOTHER.
AND HOW DOES A GIVEN PATTERN OF
INTERACTIONS IN EACH OF THESE
SUBSYSTEMS OR BETWEEN THEM, FROM
BEHAVIOR TO ENVIRONMENT, BRAIN
TO ENVIRONMENT, WHATEVER IT IS,
CHANGES THE LANDSCAPE ON WHICH
YOU'RE MOVING AND A LOT OF WORK
HAS SUGGESTED THAT THE PATTERN
OF INTERACTIONS BETWEEN THESE
SUBCOMPONENTS DEFINES THE NATURE
OF THAT LANDSCAPE IN SOME WAY.
IF WE CAN UNPACK AND UNDERSTAND
THAT PATTERN OF INTERACTIONS
MORE IT WOULD PROVIDE US WITH A
LOT OF INTUITION ABOUT HOW EASY
IT IS FOR THE PERSON TO MOVE
FROM BEHAVIOR TO BEHAVIOR,
COGNITIVE PROCESS TO COGNITIVE
ENVIRONMENT, AND THAT BRINGS ME
TO COGNITIVE BEHAVIORAL THERAPY.
YOU'RE RIGHT, IT'S AN
INTERESTING CHALLENGE TO COMBINE
THAT WITH OTHER METHODS, AND
UNDERSTAND HOW TO USE THEM
TOGETHER OVER DIFFERENT TIME
SCALES, OR DIFFERENT
FREQUENCIES, TO HELP THE PERSON
ENACT CHANGE.
YOU'RE RAISING A SUPER CRITICAL
QUESTION.
I THINK THESE MORE COMPLEX
APPROACHES ARE NEEDED TO ADDRESS
IT, BUT THE PARTICULAR WAY TO DO
THAT STILL FEELS FUZZY, IN MY
MIND.
>> GORDON, YOU DESCRIBED BEHAVE
AS A FILTER BETWEEN INTERNAL AND
EXTERNAL STATES.
>> I WANT TO SAY THAT WAS MY
DESCRIPTION, NOT WHAT I THINK IT
IS, BUT WHAT MOST MODELS TREAT
IT AS.
THAT WAS MY CRITICISM OF MOST
MODELS.
>> HOW DO YOU FEEL LIKE THAT
METAPHOR IS KEEPING THINGS
FUZZY, NOT CLARIFYING?
>> WELL --
>> ONE MINUTE REMAINING.
>> I THINK IN MY MIND, I'LL BE
QUICK, IT'S BASICALLY THERE'S --
IT SAYS THE CERTAINLY STATE IS
ONE THING.
INTERNAL STATE IS A SET OF
THINGS PUSHING AND PULLING
AGAINST EACH OTHER, FROM CONTROL
THEORY WE NOW THE PUSH-PULL
MECHANISM IS WAY MORE STABLE IN
A LOT OF CASES THAN HAVING A
SINGLE VARIABLE THAT WE'RE
TRYING TO DO A FIXED POINT, A
SET CONTROL ON.
SO THIS NOTION OF THINKING ABOUT
INTERACTIONS BETWEEN THE STATES
AND HOW THOSE STATES ARE THEN
BEING FED BACK ONTO BY THE
BEHAVIOR, I THINK IS ALMOST A
FUNDAMENTALLY DIFFERENT TYPE OF
MODEL THAN WE'RE SWITCHING
BETWEEN DIFFERENT SORT OF LIKE
HIDDEN MARKUP MODEL TYPE STATES.
I THINK THAT SORT OF IS A
FUNDAMENTALLY DIFFERENT WAY OF
VIEWING THE WORLD.
>> OKAY.
THANKS FOR THE BRIEF
CONVERSATION.
I LOOK FORWARD TO PICKING UP THE
TOPICS WHEN I SEE YOU IN PERSON.
>> THANKS, DR. WATERS, FOR
LEADING THIS GREAT DISCUSSION.
WE MOVE TO THE SECOND DISCUSSION
LEADED BY DOLU OBATUSIN.
PLEASE TAKE IT AWAY.
>> YEAH, THANK YOU FOR THAT.
THAT WAS AN AMAZING SET OF
TALKS.
AND DEFINITELY LEARNED A LOT.
YOU KNOW, MY BACKGROUND IS
BIOMEDICAL ENGINEERING,
PRAGMATIC APPLICATION BUILDING
DEVELOPMENT.
ESSENTIALLY, LEVERAGING
ENGINEERING PRINCIPLES TO DRESS
HEALTH CARE CHALLENGES AND WHAT
NOT.
IN RECENT YEARS I'VE HAD THE
OPPORTUNITY TO WORK AT THE
CENTER FOR ADVANCED
THERAPEUTICS, WORKING ON
TREATMENT RESISTANT DEPRESSION.
PREVIOUSLY AT EMORY, NOW AT
SINAI.
SO AT OUR CENTER, DR. WATERS
MENTIONED THIS IN HER
INTRODUCTION AS WELL,
ESSENTIALLY WE ARE IN THE
BUSINESS OF BIOMEDICAL
IDENTIFICATION FOR TREATMENT
RESISTANT DEPRESSION, PATIENTS
TREATED WITH ASSISTIVE DEVICE,
ALLOWS YOU TO COLLECT BRAIN
RECORDINGS, WHERE ON A DAILY
BASIS THEY DO RECORDINGS OF THE
BRAIN, RECORDING FROM WEARABLES,
ACTIVITY AND SLEEP, RATING
SCALES, SO IT'S VERY MUCH IN
ALIGNMENT WITH THE THEME OF THIS
WORKSHOP ON BRAIN AND BEHAVIORAL
QUANTIFICATION AND
SYNCHRONIZATION.
WE'RE COLLECTING A LOT OF DATA
IN THE LAB.
EVEN IN A SIMULATED ENVIRONMENT,
PATIENTS CAN ESSENTIALLY
INTERACT WITH SIMULATED
ENVIRONMENT, AND ALLOWS US TO
CAPTURE POSE.
SCOTT MENTIONED BIORHYTHMS, OPEN
POSE USED IN TRANSFORMATION
MODELS, I WANTED TO RIF ON THAT.
ONE THING THAT STRIKES ME WE
KIND OF LIKE DEVELOP, YOU KNOW,
REALLY DEEP MODELS FOR DIFFERENT
SETS OF INPUTS.
WHETHER IT'S VIDEO, ACTIVITY
TRACKING, VOICE.
BUT IT'S NOT QUITE CLEAR HOW
MODELS CAN BE DEVELOPED FOR
MULTI-MODAL DATA COLLECTION.
THIS IS SOMETHING WE'RE GETTING
INTO WHERE WE CAN ACQUIRE
MULTIPLE DATA ALONG WITH
INCLUDING NEURAL DATA, AND IT'S
NOT QUITE CLEAR, RIGHT, HOW TO
BUILD MODELS THAT CAN INTEGRATE
DIFFERENT INPUTS, YOU KNOW,
DIFFERENT MULTI-MODAL INPUTS.
I WAS WONDERING IF MAYBE SCOTT
CAN SPEAK TO THAT, AND THE REST
OF THE PANELISTS AS WELL, ON HOW
TO BETTER BILL THESE KIND OF
MODELS THAT INTEGRATE DIFFERENT
MULTI-MODAL INPUTS.
>> GREAT QUESTION.
YOU KNOW, IT'S ONE THAT IS -- MY
CAMERA DOES THIS WEIRD ZOOM-OUT
THING.
IT'S ONE THAT'S NOT UNIQUE TO
THE TOPICS OF THIS WORKSHOP.
TAKING THAT BIGGER PICTURE OF
WHERE ELSE IN THE BROADER M.L.
WORLD ARE WE ENCOUNTERING
SIMILAR ANALOGOUS PROBLEMS.
PROBLEMS HAVE TO DO WITH MODEL
JOINT IMAGES, CAPTIONS, SPEECH
AND TEXT.
MAYBE TRYING TO GENERATE OR
EXPLAIN AS MUCH SHARED VARIANTS
AS POSSIBLE WHILE ALLOWING FOR
THINGS THAT ARE UNIQUE TO ONE
DOMAIN VERSUS THE OTHER, MAYBE
IF YOU'RE MEASURING NEURAL
ACTIVITY, IN SOME PART OF THE
BRAIN, IT MAY NOT HAVE THE
NECESSARY SIGNALS TO PREDICT
CERTAIN ASPECTS OF BEHAVIOR.
YOU HAVE TO HAVE SOME OTHER WAY
OF EXPLAINING THAT OTHERWISE
UNEXPLAINABLE VARIANT.
THEY ARE ANALOGOUS, APPEARED IN
MARYAM'S TALK WITH SHARED LATENT
VARIABLES PREDICTING NEURAL AND
BEHAVIORAL MEASUREMENTS.
AGAIN, NOT TO SAY THAT THE
PROBLEM IS SOLVED, BUT I THINK
THIS IS AN AREA OF ACTIVE
RESEARCH, ONE THAT I'M
OPTIMISTIC ABOUT, AN INTERESTING
AREA FOR RESEARCH.
WHILE I HAVE THE MIC I DO WANT
TO RESPOND TO A QUESTION THAT
CAME UP IN THE CHAT, FROM DAVID
ANDERSON, ABOUT ENCODING AND
DECODING.
AND ASKING DO WE RUN THE RISK OF
INFERRING SPURIOUS NON-CAUSAL
RELATIONSHIPS BETWEEN NEURAL
ACTIVITY AND BEHAVIOR ANALOGOUS
TO A SPEEDOMETER WHERE CAN YOU
PREDICT, BUT IF THE CAR BREAKS
DOWN, YOU CAN'T FIX IT, HOW CAN
WE GO BEYOND THE STRICTLY
CORRELATIONAL ANALYSES TO
SOMETHING THAT WOULD GIVE MORE
CAUSAL UNDERSTANDING.
I THINK WHAT THIS IS GETTING AT
IS THAT'S FOUR BUCKETS OF
ORGANIZATION, PERHAPS
INSUFFICIENT TO EXPLAIN ALL OF
THE DIFFERENT TYPES OF
CHALLENGES THAT WE COME UP WITH.
IF HAD YOU TO ADD A FIFTH ONE, I
DON'T KNOW, I'M TORN HERE.
I THINK GORDON'S EMPHASIS ON
DEVELOPING MORE SIMPLE
THEORETICALLY GROUNDED MODELS IS
ONE THAT'S QUITE IMPORTANT BUT
EQUALLY IMPORTANT IS
ESTABLISHING CAUSAL
RELATIONSHIPS BETWEEN NEURAL AND
BEHAVIORAL DATA.
WE CAN START TO DO THAT WITH
OPTOGENETIC TOOLS, THIS IS AN
AREA THAT'S AN EXCITING AREA OF
ONGOING RESEARCH, WE NEED AS A
FIELD TO HAVE OUR EYE ON.
>> WE HAVE ONE MINUTE REMAINING.
>> WHAT COMES TO MIND ALSO LIKE
IN THE PRACTICAL ASPECT OF
TRYING TO USE SOMETHING LIKE
TRANSFER LEARNING IS SOMETIMES
LIKE I THINK ABOUT OUR DATA
SCIENTISTS, WHO TRIED TO USE
SOMETHING LIKE THE FREE
EXPRESSION ALGORITHM, TRAINED ON
6 MILLION VIDEOS AROUND THE
WORLD, MEASURED IN EXPRESSION,
NOT OF SOME ALGORITHMS ARE NOT
EASILY OBTAINABLE, YOU HAVE TO
MAKE A SPECIAL REQUEST.
AND THE REASON WHY THIS MATTERS,
A LOT OF PEOPLE ARE DOING FACIAL
ANALYSIS, AND IF YOU THINK ABOUT
THE CONSTRAINTS AND GUARD RAILS
PUT IN PLACE FOR GOOD REASONS,
AND SOME STUDY DESIGN, YOU WANT
TO SHARE THE INDIVIDUAL VIDEOS,
YOU SHOULD BE THINKING ABOUT IN
TERMS OF HOW TO BETTER LEVERAGE
SOME BIG DATA TOOLS OUT THERE
MAYBE NOT SO EASILY ACCESSIBLE.
>> ALL RIGHT.
WE CAN CONCLUDE THIS DISCUSSION
HERE.
THANKS, EVERYONE.
NOW THE THIRD DISCUSSION WITH
DR. TIM BROWN TO LEAD THE
DISCUSSION.
>> HI, EVERYBODY.
I WANT TO SAY THANKS TO DANI,
SCOTT, MARYAM, AND FOR THE
SERIES SERIES OF EXCELLENT
TALKS.
I LEARNED A LOT.
NOW IT'S TIME FOR WHAT MIGHT
SEEM LIKE A DEPARTURE.
I'M A PHILOSOPHER BY TRAINING,
BIOETHICIST BY PROFESSION,
UNIVERSITY OF WASHINGTON SCHOOL
OF MEDICINE, BIOETHICS AND
HUMANITIES DEPARTMENT.
MY BIGGEST WORRIES ARE ABOUT HOW
WE CURATE DATA, THE CONCLUSIONS
THROUGH ANALYSES AND IMPACT OF
OUR CONCLUSIONS ON A VULNERABLE
GROUP DOWNSTREAM, RIGHT?
AND SO I WANT TO START US OFF BY
WALKING THROUGH A LINE OF NOT SO
MUCH ARGUMENT, BUT MORE
COMMENTARY, THEN I'LL ASK A
COUPLE QUESTIONS.
ONE THING I'VE SEEN CONNECTING
ALL OF THE TALKS TODAY IS THAT,
YOU KNOW, THE IDEA THAT, YOU
KNOW, FINDABLE ACCESSIBLE
INTEROPERABLE AND REPRODUCIBLE
DATA, SO FAIR DATE, WILL BE
VITAL TO THE CREATION OF
COMPUTATIONAL MODELS FROM
MULTI-MODAL DATA.
HAVING SO MANY DIFFERENT SOURCES
OF DATA, AND SYNTHESIZING THOSE
AND SYNCHRONIZING TO PRODUCE
INSIGHTS THAT DRIVE THE MODELS
THAT WE USE IN DEVICES, IN
DISCOVERY, IN MEDICAL PRACTICE,
IT WILL BE EXTREMELY IMPORTANT
TO HAVE THESE FAIR DATA
PRACTICES TO ENSURE EVERYBODY
HAS ACCESS TO DATA.
BUT WE ALSO NEED TO REFLECT ON
FAIRNESS, ACCOUNTABILITY,
TRANSPARENCY, AND ETHICS OF OUR
DATA SCIENTIFIC PRACTICES.
THIS IS FAMILIAR FROM THE A.I.
M.L. SPACE WHERE THERE IS A LOT
OF INSTITUTIONAL KNOWLEDGE HOW
TO MAKE HEADWAY ON ENSURING
FAIRNESS, ENSURING TRANSPARENCY,
ACCOUNTABILITY.
WE NEED TO MIND THESE FEATURES
OF COMPUTATIONAL MODELS, OF
SYSTEMS THAT DATA ARE REPORTED
ON, DOWNSTREAM USES OF MODELS
AND DATA, SO WHEN IT TRANSLATES
AT ULTIMATELY LIVED EXPERIENCES
OF PATIENTS, DANI IS RIGHT,
BEHAVIOR IS COMPLEX BECAUSE OF
THE DEEP CAUSAL RELATIONSHIPS
BETWEEN BRAINS, BEHAVIORS, AND
THE CONTEXT THAT BRAINS AND
BEHAVIORS HAPPEN IN.
THE STATISTICAL MODELS THAT WE
APPLY TO THESE DATA COULD BE
COMPLEX IN WAYS THAT COULD BE
OPAQUE, THAT COULD STAND IN THE
WAY OF ENSURING MODELS DRIVE THE
SYSTEMS IN WAYS THAT LEAD TO
FAIR OUTCOMES FOR USERS.
WE HAVE TO REMEMBER METADATA
MEASUREMENT AND ANALYSIS CAN
IMPACT HOW GROUPS ARE
REPRESENTED IN MODELS AND
MEDICAL PRACTICES.
SO, FOR EXAMPLE, WHEN
MARGINALIZED GROUPS ARE
UNDERREPRESENTED IN PUBLIC
DATASETS OR IF THEIR DATA -- IF
DATA IS SUBJECT TO
CATEGORIZATION OR
CHARACTERIZATION, COMING BACK
INTO THE QUESTION OF HOW DO WE
GO FROM DATA TO EXPLANATIONS OF
BEHAVIOR, THAT ARE BLIND TO
LIVED REALITIES, OUR PRACTICES
COULD CREATE ENTIRE SYSTEMS OF
PEOPLE'S BEHAVIORS THAT ARE
HARMFUL TO THE GROUPS DOWNSTREAM
TO VULNERABLE GROUPS DOWNSTREAM.
ONE HYPOTHETICAL LATER ON, THIS
IS MY SELFISH WORRY, THE
POSSIBILITY OF ADAPTIVE DECODING
OF BIOMARKER FOR DEPRESSIVE
EPISODES, UNIT USING BEHAVIORAL
DATA LIKE FACIAL GESTURES, SOME
MOMENTS, BLIND TO ME IN OTHERS.
THIS IS ONLY ONE POSSIBLE
DOWNSTREAM OR CONSEQUENCE OF
UPSTREAM DECISIONS IN THE LAB.
HOW SHOULD DATA GOVERNANCE
WORK?
THINKING ABOUT INDIGENOUS
POPULATIONS, FIRST NATIONS,
PEOPLE WHO IDENTIFY AS LGBTQIA,
WHAT ROLE SHOULD THEY PLAY IN
THE DATA CURATION PROCESS?
WHAT IS THE MOST PRESSING
ETHICAL CHALLENGE EACH OF YOU
SEE ARISING FROM THE CURATION OF
THESE COMPLEX MULTI-MODAL DATA,
AND HOW THESE DATA DRIVE THE
CREATION OF COMPLICATED
STATISTICAL MODELS, HOW DO YOU
SEE US GETTING OVER THIS AND
MITIGATE THE RISK OF CULTURAL
HARM AND OF IN THESE PRACTICES,
IN LIGHT OF THESE CHALLENGES?
I THINK IN THE ORDER OF THE
TALKS IS GREAT.
I WANT TO HEAR FROM ALL OF YOU
BUT MAYBE IT WOULD BE GOOD TO
START WITH DANI AND MOVE TO
SCOTT.
>> THANK YOU SO MUCH FOR YOUR
COMMENTS.
I THINK THAT RAISING THESE
POINTS IS EXTREMELY IMPORTANT
AND I'M GLAD WE GET TO END ON
THEM TODAY.
SO, I THINK THAT THE PLACE WHERE
I SEE POTENTIAL FOR HARM, A LOT
OF DISPARITIES RIGHT NOW,
CERTAINLY IS IN THE COLLECTION
OF DATA.
BUT I THINK IS MORE WHERE I
FOCUSED, IS ON HOW IDEAS ARE
SHARED AMONG SCIENTISTS, AND HOW
IDEAS ARE VALUED ACROSS
SCIENTISTS.
IT'S MORE AT THE SORT OF META
LEVEL OF WHOSE IDEAS ARE
LISTENED TO, WHOSE QUESTIONS ARE
BEING ASKED AND REVERBERATE AND
HELP GUIDE THE WORK MOVING
FORWARD.
SO, SOME OF THE MORE
QUANTITATIVE WORK IN MY LAB
THAT'S FOCUSED ON THOSE
QUESTIONS IS RELATED TO CITATION
PATTERNS AND HOW THEY ARE VERY
SKEWED ACCORDING TO BOTH GENDER
AND RACE AND ETHNICITY.
SUCH THAT PEOPLE WHO ARE FROM
MARGINALIZED IDENTITIES ARE
UNDERCITED IN COMPARISON TO WHAT
YOU WOULD EXPECT GIVEN THE
NUMBER OF PAPERS THEY PUBLISHED,
THE JOURNALS IN WHICH THOSE ARE
PUBLISHED AND ET CETERA, LOTS OF
FACTORS THAT WOULD SOAK UP
VARIANTS, EVEN AFTER YOU
ACCOUNT, THERE'S STILL A LARGE
GENDER AND RACE AND ETHNICITY
EFFECT.
I THINK THAT HARMS THE SCIENCE
BECAUSE IT HARMS SCIENTISTS, THE
PEOPLE DOING THE WORK, ALTERS
WHO ACTUALLY MAKES IT TO
PROMOTION AND TO RECEIVE A LOT
OF THE AWARDS AND ET CETERA.
I THINK THAT I'M WORRIED ABOUT
THAT.
IN TERMS OF WHO WE'RE COLLECTING
DATA FROM, THAT'S A WHOLE OTHER
QUESTION.
OF WHICH IS ONE I CARE ABOUT BUT
HAVE LESS IMMEDIATE EXPERTISE.
I'LL PASS TO THE NEXT PERSON.
>> I AGREE.
THANK YOU FOR RAISING THESE
REALLY IMPORTANT QUESTIONS AND
POINTS.
I'M NOT AN EXPERT IN A.I.
FAIRLESS, BUT I COMPLETELY
AGREE, THIS IS A CRITICAL FIELD
FOR US TO BE PURSUING.
I WOULD LOVE TO TALK OFFLINE TO
HEAR ABOUT YOUR THOUGHTS ON THAT
AREA, ONE THING THAT I WILL SAY,
MY RESEARCH HAS BEEN ON THE
BASIC SCIENCE SIDE UNTIL
RECENTLY.
>> WE HAVE ONE MINUTE REMAINING.
>> OKAY.
I'LL KEEP IT SHORT.
MOSTLY IT'S BEEN ON THE BASIC
SCIENCE SIDE, RECENTLY MOVED
MORE IN THE TRANSLATIONAL AREAS,
WHICH HAS FORCED THESE ISSUES TO
THE FORE.
YOU SEE EVIDENCE OF SOME THINGS
YOU'RE CONCERNED ABOUT.
APPLYING SOME MODELS TO MOUSE
BEHAVIOR, WE FIND SOMETIMES THE
REPRESENTATIONS THAT YOU GET FOR
ONE VERSUS ANOTHER DIFFER NOT
BECAUSE THEY ARE FUNDAMENTALLY
DIFFERENT IN BEHAVIOR BUT ONE
MOUSE IS BIGGER THAN THE OTHER.
THE FUTURE EXTRACTION PROCESS,
SLIGHTLY BIGGER MOUSE GETS
CLUSTERED, MODELS DIFFERENTLY.
WE CAN CORRECT THAT BUT IT'S
DONE IN A REACTIVE WAY.
AND I THINK ONE OF THE THINGS
YOU'RE GETTING AT IS THAT AS WE
DEPLOY THESE IN THE REAL WORLD
WE HAVE TO BE MORE PRO-ACTIVE IN
GETTING OUT AHEAD OF THOSE TYPES
OF BIASES BEFORE THEY ADVERSELY
AFFECT PEOPLE.
>> GREAT NOTE TO END ON.
I REALLY HATE CUTTING THE
DISCUSSION SHORT.
THERE WENT MY TIMER.
BUT I WANT TO GIVE A SHOUT OUT
TO EVERYBODY FOR SUCH HARD WORK
YOU HAVE PUT INTO YOUR
PRESENTATIONS, AND KEEPING US ON
SCHEDULE.
AND ALL THE ACTIVE GREAT
DISCUSSIONS, WE'LL CLOSE WITH
FINAL THOUGHTS FROM KAREN
ROMMELFANGER AND JUSTIN BAKER.
I'LL HAND THE FLOOR OVER TO THEM
AND LET THEM PROCEED.
>> I UNDERSTAND I HAVE TEN
MINUTES, BUT I'LL USE LESS.
IS THAT RIGHT, DANA?
>> YOU HAVE 10 TO 15 MINUTES.
>> WE'RE ALL TIRED.
I'LL DO 10, MAYBE UNDER.
THANK YOU FOR STICKING AROUND
FOR THE RICH CONVERSATION.
I WAS PART OF THE DISCUSSION NOT
FROM THE INCEPTION BUT THREE
YEARS AGO AFTER THE TEAM HAD
ALREADY GIVEN A LOT OF THOUGHT.
AND IT WAS BBQ, NOT BBQS, SO THE
SYNCHRONIZATION PIECE IS
EXCITING.
I'M CHARGED WITH SYNTHESIZING
IDEAS, CHALLENGES, OPPORTUNITIES
TO ADDRESS THOSE CHALLENGES.
I'LL BRING UP THREE MAIN POINTS,
AND THREE BULLETS UNDER THOSE.
FOR SOME WHO MAY NOT BE AWARE, I
HAVEN'T HAD THE PLEASURE OF
MEETING A LOT OF YOU YET.
I'M TRAINED AS A NEUROSCIENTIST,
MOVED INTO FORMAL TRAINING IN
NEURAL ETHICS, LAUNCHED A
NEUROETHICS PROGRAM AT EMORY.
CO-CHAIR, BRAIN INITIATIVE,
CONSORTIUM OF EXISTING BRAIN
INITIATIVES, BRAIN IS PART OF
ONE OF THOSE.
RECENTLY MOVED TO LAUNCHING
INDEPENDENT INTERNATIONAL
NEUROETHICS THINK TANK,
UNDERWRITING ETHOS IS
INCLUSIVITY ACROSS GEOGRAPHY.
WE BRING PRACTICAL LEVELS TO
ETHICS WE TALK ABOUT AND WE NEED
TO THINK ABOUT IN THE NEXT FEW
DAYS.
THE FIRST POINT IS THAT WE HAVE
AN ETHICAL IMPERATIVE FOR BRAIN
BEHAVIOR QUANTIFICATION AND
SYNCHRONIZATION, IT'S THE TYPE
OF ACTIVITY THAT I THINK
EVERYONE HERE AND THERE WERE 400
PEOPLE ONLINE THIS MORNING,
AGREE THAT BRAIN SHOULD BE
TAKING UP IN SOME WAY.
BUT IN ORDER TO DO THAT WE NEED
TOOLS.
EVERYBODY HERE UNDERSTANDS THAT
THE LOCALLY GRAIL OF
NEUROSCIENCE IS STILL CONNECTING
THE BRAIN TO BEHAVIOR.
WE STILL HAVE A LOT OF WORK TO
DO IN THAT SPACE, MAYBE WE'LL
HAVE OUR WHOLE CAREERS TO DO
THAT.
I WANT TO ORIENT YOU, THE
NATIONAL INSTITUTES OF HEALTH,
THERE'S A NEUROETHICS WORKING
GROUP, AND A TEAM, AND
NEUROETHICS IS A PRACTICAL TOOL
IN YOUR TOOLKIT THAT CAN BE USED
TO SCAN THE HORIZON, NOT MEANT
TO SET UP ROAD BLOCKS OR PURELY
A POLICING EFFORT, WHICH I THINK
THAT RCR TRAINING HAS DONE TO A
LOT OF YOU IN YOUR PERCEPTIONS,
IT'S WHAT HAPPENED TO ME WHEN I
WAS IN MY TRAINING.
BUT USED AS A WAY TO ANTICIPATE
ETHICAL ISSUES THAT TIM WAS
BRINGING UP, AND SOME OTHERS
THROUGH THE DAY.
ULTIMATELY CAN ADVANCE AND
ACCELERATE SCIENCE.
AND WITH IN A IN MIND -- THAT IN
MIND WE WANS WANT TO ENSURE THE
WORK, ETHICAL ADVANCED,
SYSTEMATIC IN EXPLORATION.
AS SCOTT MENTIONED IN THE LAST
PIECE, WE WANT TO GET AHEAD, BE
GIVING THE ETHICS UP FRONT.
SECOND I WANT TO HIGHLIGHT THE
CHALLENGE OF DESCRIBING WHAT IS
NORMAL WHEN YOU QUANTIFY
EVERYTHING.
CERTAINLY THERE ARE IMPORTANT
REASONS TO WANT TO QUANTIFY.
FAIRLY POWERFUL STORY I BELIEVE
I HEARD FROM SOME OF THE NIH
FOLKS IS THAT MANY OF THE
INDIVIDUALS, PSYCHIATRISTS, MANY
OF THE INDIVIDUALS, HER PATIENTS
WHO SUBMITTED SUICIDE TOLD HER
WHEN SHE ASKED THEY WOULD NOT
COMMIT SUICIDE.
OUR TOOLS ARE ASKING PATIENTS TO
RESULT TO US, AND FALLING SHORT.
HOW DO WE DESIGN STUDIES,
COLLECT DATA WHERE WE CAN OFFER
BETTER EXPLANATORY POWER?
AND HOW DO WE USE NON-HUMAN
ANIMALS EVEN TO RECAPITULATE
COMPLEX HUMAN CONDITIONS LIKE
AUTISM?
LENA TING REMINDED US OF
OPPORTUNITY DIVERSITY AND
VARIABILITY, CONSIDER THOSE IN
YOUR STUDIES, BEHAVIORS MIGHT BE
MORE CONSISTENT THAN UNDERLYING
ACTIVITY.
JEFFREY NOTED TRYING TO DRAW
FROM OUR QUANTIFIED DATA WOULD
BE HIGHLY INFERENTIAL.
SCIENCE NEVER HAS BEEN JUST
ABOUT SCIENCE.
IT'S TIED WITH POWER.
IT'S IMPORTANT TO CONSIDER WHO
IS DRIVING PARAMETERS OF NORMAL
AND STANDARDIZATION.
JONATHAN, A PROFESSOR OF
SOCIOLOGY AND PSYCHIATRY AT
VANDERBILT, REMINDS US THAT YET
IN UNINTENDED AND OFTEN
INVISIBLE WAYS PSYCHIATRIC
DEFINITIONS CONTINUE TO POLICE
RACIAL HIERARCHY, TENSION,
UNSPOKEN CODES IN ADDITION TO
SEPARATING NORMAL FROM ABNORMAL
BEHAVIOR.
PERSONS ARE STIGMATIZED BECAUSE
THE ON GENERALIZATION ABOUT
DEVIANCE, PERCEIVED VOLATILITY,
BE A OF CHARACTERISTICS THAT
REMAIN ACCEPTABLE MODES OF
DISCRIMINATION.
RHETORIC BECOMES WAYS OF
POLICING THE BOUNDARIES OF
KEEPING THESE PEOPLE OUTSIDE.
THESE ARE THINGS WE NEED TO GET
AHEAD OF, THINK ABOUT AS WE
DESIGN OUR RESEARCH AND OUR
QUESTIONS.
DOROTHY ROBERTS AT U PENN SAYS
HEALTH CANNOT BE DIVORCED FROM
SOCIOPOLITICAL CONTEXT, IGNORES
THE WAYS IN WHICH -- TO DO SO
WOULD BE IGNORE WAYS
BIOTECHNOLOGY SHAPE AND ARE
SHAPED BY SOCIAL POWER.
DATA PRIVACY WAS BROUGHT UP BY
TIM AND WILL BE DISCUSSED
HEAVILY TOMORROW.
THE IMPLICATIONS FOR THE
INFERENCES EVEN IN RESEARCH IN
CONTEXT WILL LIKELY END UP STILL
IMPACTING AREAS BEYOND YOUR
LABORATORY, AND WILL LIKELY
INTERFACE IN YOUR ANALYSES WITH
PUBLICLY AVAILABLE DATA AT SOME
POINT.
KATE CRAWFORD AND TIM BROWN
MENTIONED IN THE CHAT EARLIER,
LEADING SCHOLAR OF IMPLICATIONS
OF A.I. SAYS BIG DATA ANALYTICS
WILL CHANGE HOW WE KNOW AND
LIKEWISE SHOULD EXPECT MORE
COMPLICATED ETHICAL IMPLICATIONS
OF WHAT WE KNOW.
DATA IS NOW CONNECTIBLE,
ENDEFINITELY REPURPOSABLE,
CONTINUOUSLY UPDATABLE,
DISTANCED FROM THE CONTEXT OF
COLLECTION.
AND IN ONE OF THE TOOL KITS THE
NIH BRAIN HAS, NEUROETHICS
PRINCIPLES, ONE SAYS MOVE BEYOND
THE BENCH WITH CAUTION, EVEN AS
WE CONSIDER THE COMPARATIVE WORK
AND IMPLICATIONS OF
INTERPRETATIONS OF SCIENCE
SHOULD NOTE THEY SHOULD MOVE OUT
INTO A NUMBER OF DOMAINS, FROM
THE LAB TO CLINIC, CONSUMER TO
NATIONAL DEFENSE, REMINDED PAUL
ECKMAN'S, THAT WE DISCUSSED
TODAY, AFFECTED CUSTOMS
ENFORCEMENT.
THE FRAMEWORK IN THE BEGINNING
BY HOLLY LISANBY, A SUGGESTION
FOR FRAMEWORK THINKING ABOUT OUR
SCIENTIFIC AND ENGINEERING
CHALLENGES, EQUAL FOOTING WITH
SOCIAL CHALLENGES AND SOCIAL
CONTEXT AND IMPLICATIONS OF THE
WORK WE DO IN THE LAB.
DANI NOTED THE MIND IS THE
DESIRE, INTENTION, PURPOSE, AND
WE SHOULD NOTE THAT EVEN THAT
DEFINITION OF BRAIN IS NOT
CULTURALLY UNIVERSAL.
THAT WILL HAVE IMPLICATIONS FOR
HOW DATA IS INTERPRETED, AND HOW
THAT RESEARCH MIGHT END UP
FEEDING BACK INTO SYSTEMS OF
HOW, HOW THE MIND AND BRAIN ARE
CONNECTED CAN HAVE IMPLICATION
FOR STIGMA AND APPLICATION IN
MEDICAL SYSTEMS AND BEYOND.
CULTURE INFLUENCES WHAT SCIENCE
IS SUPPORTED, WHERE SCIENCE CAN
BE CONDUCTED, AND THE TIMES OF
ETHICAL FRAMEWORK THAT OF.
GAPS IN UNDERSTANDING CAN LEAD
TO MISMISSED OPPORTUNITIES FOR
COLLABORATION, ADVANCEMENT
TOWARDS FUTURE DISCOVERIES,
LIMIT ABILITY TO BROADLY SHARE
RESULTS, AND REAP THE BENEFITS
OF FINDINGS, RESULTING IN
FAILURE TO RECOGNIZE SHORT- AND
LONG-TERM POTENTIAL AND RISKS OF
NEURAL SCIENCE RESEARCH.
IT'S IMPORTANT TO NOTE THESE
DIFFERENCES DON'T JUST EXIST
AMONG NATIONAL ENTITIES.
THEY EXIST WITHIN DEFINED
SOCIETIES, ALSO AMONG INDIVIDUAL
RESEARCHERS AND PRACTITIONERS,
THESE ARE THINGS THAT NONE OF US
ARE DEVOID OF.
SCIENCE IS CONTEXTUALIZED.
ANOTHER FEATURE THAT DANI
MENTIONED, BRAIN TO BEHAVIOR IS
NOT NECESSARILY LINEAR.
IT'S NOT A SIMPLE CAUSAL CHAIN.
ANY INTERPRETATION OF THE DATA
THAT WE FIND FROM ANY OF THE
WORK WE MIGHT DO IN THE BBQS
FRAMEWORK NEEDS TO BE SIMILARLY
CONSIDERED.
WHAT DOES IT MEAN TO PUT A BRAIN
IN THE CONTEXT OF THE REAL
WORLD?
TIM REMINDED US OF CHALLENGES
THAT WE HAVE TO EXPLORE.
ETHICS CAN ADVANCE
NEUROTECHNOLOGY AND NEURAL
SCIENCE, ENSURE BY THINKING
ABOUT ETHICS THAT SCIENTISTS AND
THE FUNDERS, LIKE NIH AND NSF,
SOCIAL CONTRACT TO SOCIETY IS
MET AND SUSTAINED.
THE SECOND PART HERE IS THAT WE
DO NEED NEW TOOLS AND
COLLABORATION, MALCOLM SAID
TOOLS FOR SYNTHESIS,
COLLABORATORS, THE CHAT
MENTIONED MANY HALF A MILLION
AUTHORS IS WHAT WE NEED TO
TACKLE SOME OF THE MOST
CHALLENGING NEUROSCIENCE
QUESTIONS.
HOW DO WE -- WHO SHOULD THESE
COLLABORATORS BE AND WHO GETS TO
DECIDE WHO THEY ARE?
COMMUNITIES CAN BE AN IMPORTANT
PART OF THINKING OF OUR
COLLABORATIONS WITH BRAIN
BEHAVIOR QUANTIFICATION.
THERE ARE OPPORTUNITIES FOR US
TO THINK ABOUT WHO MIGHT BE
IMPACTED, CONSIDER COMMUNITIES
AS PARTNERS WHO MIGHT BE ABLE TO
GIVE US EVEN NEW INSIGHTS INTO
SOME TOUGH QUESTIONS WE HAVE
ABOUT HOW FAR WE CAN MODEL
AUTISM, AND WHAT KINDS OF
CONCLUSIONS WE CAN DRAW, WHAT
KINDS OF UTILITY WE MIGHT FIND.
TIM RAISED QUESTIONS HOW TO
MITIGATE SOCIOCULTURAL HARM,
ALLISON MENTIONED WONDERFUL
QUESTION, HOW DOES CENTERING ON
THE PATIENT'S EXPERIENCE IN
MODELING HIGHLIGHT GAPS FOR US
TO EXPLORE WITH BBQ, INTERESTING
NOTE SHE SAID THIS IS NOT AN
ETHICAL QUESTION BUT IT'S A
PRACTICAL ONE.
I WOULD ARGUE IT'S BOTH.
THESE TYPE OF QUESTIONS, ETHICAL
QUESTIONS, COULD BE PRACTICALLY
INTEGRATED INTO SCIENCE TO HELP
ENHANCE AND ELEVATE THAT
SCIENCE.
THE SECOND PIECE WITH RAJI AND
PETER GAVE US GREAT INSIGHT FROM
INDUSTRY, AND I THINK I'LL NEVER
SEE A CHIHUAHUA AND A BLUEBERRY
MUFFIN THE SAME WAY AGAIN.
I DON'T KNOW IF ANYBODY ELSE
FELT THAT WAY.
PETER INVITED US TO CONSIDER HOW
TO TRY NEW OPPORTUNITIES, THINK
ON A PRACTICAL WAY TO SCALE AND
TRANSLATE IN THE REAL WORLD
BEYOND THINKING ABOUT THE TIMES
OF CHALLENGES WE MIGHT HAVE AS
WE STAY IN A WEDDEDNESS TO
STATUS QUO HOW WE DO SCIENCE IN
ACADEMIC LABORATORY.
THE FINAL THING, THERE'S
EMPHASIS, TEAM SCIENCE, HOW DO
WE DO SCIENCE?
HOW DO YOU CREATE A VALUE
PROPOSITION?
IT'S JUST AS MUCH A SOCIAL
PROBLEM AS TECHNICAL ONE.
DANI REMINDED US OF THE PROBLEM
WHO WE CITE AS WE THINK ABOUT
THE ENTERPRISE OF SCIENCE.
WE MIGHT CONSIDER OTHER OUT OF
THE BOX PARTERS IN THE CHAT,
THERE'S AN EXISTING MOVEMENT
ENTIRELY SEPARATE FROM THIS
GROUP, DECENTRALIZED SCIENCE,
ALSO EXPLORING ACTIVE TOOLS ON
HOW TO DEMOCRATIZE SCIENTIFIC
PROCESS AND OPEN SCIENCE
PRACTICES.
THAT'S MY SUMMARY FOR TODAY.
THERE'S A LOT OF RICH DISCUSSION
AND A LOT OF OPPORTUNITIES
FOR -- I HOPE THIS PUT OUT A FEW
SEEDS FOR CONVERSATION FOR
TOMORROW'S BREAKOUT GROUPS.
AND I'M HONORED TO BE HERE WITH
ALL OF YOU, TO BE THINKING
THROUGH THIS WITH YOU TODAY AND
TOMORROW.
THANK YOU.
>> OKAY.
I THINK I CAN GO LAST THEN AND
I'M NOT ABLE TO TURN MY VIDEO
ON.
I'M GOING TO SHARE MY SCREEN FOR
ONE SECOND.
LET'S SEE.
I'LL GO THROUGH IMPRESSIONS FROM
THE DAY.
WE'RE ALL REELING FROM THE
EXTENT OF THE DEPTH OF WHAT WE
DISCUSSED TODAY.
I'M JUSTIN BAKER, ASSISTANT
PROFESSOR OF PSYCHIATRY AT
HARVARD MEDICAL SCHOOL,
SCIENTIFIC DIRECTOR FOR
INSTITUTE FOR PSYCHIATRY AT
McLAIN HOSPITAL.
SOME THEMES APPLIED TO OUR OWN
WORK, THINKING ABOUT HOW DO WE
ASSESS MENTAL HEALTH, GOING
BEYOND THE TYPICAL CASE CONTROL
MODELS THAT WE'VE HAD TO WRESTLE
WITH IN THE PAST, GIVEN THE
COMPLEXITY OF COMORBIDITIES, AND
ALSO AS WE TALK ABOUT TODAY
COMPLEXITY OF LOOKING AT
VARIANTS, VARIATION, THAT WE SEE
AT THE LEVEL OF PATIENTS.
OR ORGANISMS.
AND TRYING TO MAKE SENSE OF THE
CAUSAL CHAIN OF EVENTS THAT GOES
DOWN TO GENETICS AND OTHER
INTERMEDIATE SCALES.
ON TOP OF THIS, WHICH IS USUALLY
SOMETHING WE INFER, AS A FIELD
ACROSS MANY THOUSANDS OF
INDIVIDUALS, WE USUALLY DON'T DO
IT FOR INDIVIDUALS, AND YET WITH
HUMANS AS WELL AS OTHER
ORGANISMS WE HAVE TO THINK ABOUT
MENTAL HEALTH EVOLVING OVER THE
LIFESPAN, AS WELL AS OVER
PERIODS OF DAYS TO MONTHS TO
WEEKS, AS INDIVIDUALS CHANGE
THEIR CLINICAL STATE.
THESE ADD TO THE COMPLEXITY,
WHAT IS IT WE'RE TRYING TO SOLVE
FOR AND UNDERSTAND THE BRAIN'S
BASIS FOR BEHAVIOR.
AND ONE OF THE ISSUES WE'VE
TRIED TO WRESTLE WITH WHEN YOU
HEARD FROM JEFF COHN ABOUT
ENCOUNTERS EVALUATE MENTAL
HEALTH, NOT A ABOUT SYSTEM FOR
AN INDIVIDUAL'S MENTAL STATE.
LIKE JEFF'S GROUP WE'VE BEEN
WORKING ON OTHER METHODS TO
CAPTURE THIS INFORMATION FROM
VIDEO DATA.
ULTIMATELY, THE WAY WE ASSESS
MENTAL HEALTH TODAY, OPEN LOOP.
WE TALK ABOUT VARIOUS KINDS OF
CLOSED LOOP WAYS TO BEGIN TO
ASSESS MENTAL HEALTH.
I WANT TO COME BACK TO THE
NOTION OF THE CONTROL SYSTEM
WHICH WE HEARD ABOUT TODAY,
SEVERAL OF THE SESSIONS, THAT AS
WE TRY TO UNDERSTAND ANY SYSTEM
WHERE THERE'S A LATENT
CONSTRUCT, WE'LL HAVE SOME
SENSOR NETWORKS PROVIDING RAW
INFORMATION, NOT JUST THE RAW
SIGNALS, IT'S RELEVANT FEATURES
THAT GET EXTRACTED FROM THOSE
RAW SIGNALS, AND WE HEARD ALSO
ABOUT THE IDEA THAT BEHAVIORS,
IN MANY CASES, ARE THERE TO
REORIENT AND GET ADDITIONAL
INFORMATION, NOT THE ONLY
REASON, BEN AND OTHERS MENTIONED
BEHAVIORS ARE NECESSARY FOR
SURVIVAL BUT IT'S VERY IMPORTANT
WE THINK ABOUT OUR SENSORY
SYSTEMS AND OUR MOTOR SYSTEMS AS
ACTING WITHIN THESE CLOSED LOOP
ENVIRONMENTS.
AND WHAT I LIKE TO POINT OUT TO
TECHNICAL AUDIENCE AS WELL AS TO
ENGINEERING AUDIENCES,
FUNDAMENTALLY THE BRAIN IS A
BUNCH OF THESE CLOSED LOOP
SYSTEMS TRYING TO COMPUTE LATENT
INFORMATION ABOUT THE WORLD TO
SOLVE THOSE SURVIVAL PROBLEMS.
SO IT'S VERY MUCH RELEVANT TO
THE WAY NEUROSCIENTISTS MIGHT
THINK ABOUT THE WORLD, BUT ALSO
QUITE RELEVANT TO HOW CLINICIANS
THINK ABOUT THE WORLD, EVERY
TIME THE CLINICIAN NEEDS TO
INTERVIEW A PATIENT THEY HAVE TO
USE THEIR OWN SENSORY
INFORMATION, HONED OR EVOLVED
THROUGH TRAINING, TO EXTRACT
RELEVANT INFORMATION FROM THEIR
PATIENTS.
BUT IT'S REALLY NOT USED IN THE
SYSTEMATIC WAY BUT THE SAME
CONCEPT IS THERE THAT CLINICIANS
ARE CONSTANTLY ITERATING TO
SOLUTIONS.
WE TRY TO USE THAT SAME APPROACH
TO PUT SENSORS ON PATIENTS, AND
TO COMBINE THOSE MUCH HIGHER
TENSELY ASSESSMENTS LIKE VIDEO,
WITH THE LOWER DENSITY LIKE
WEARABLES, TO SEE WHETHER
SIGNALS OF CLINICAL RELEVANCE
ARE PRESENT IN EACH OF THOSE
KINDS OF SOURCES OF DATA.
JUST TO MAKE THIS MORE CONCRETE,
MANY KINDS OF RAW DATA WE CAN
PULL FROM IN TODAY'S
ENVIRONMENTS FROM SELF-REPORTS
AND SURVEYS THROUGH TO
ELECTRONIC HEALTH RECORDS, AND
INCREASINGLY ALL THOSE OTHER
TYPES OF DATA WE HEARD FROM
SEVERAL PRESENTERS ON TODAY.
AND WE HEARD AGAIN WAYS TO PULL
DATA FROM COMPUTER VISION,
DIDN'T HEAR AS MUCH ABOUT MRI.
WE CAN TRY TO COMBINE ALL OF THE
RELEVANT FEATURES BUT IT'S ABOUT
INFORMING THE LATENT CONSTRUCTS
THAT ARE RELEVANT FOR PATIENTS
IF WE'RE TRYING TO WORK ON
CLINICAL APPLICATIONS.
AND SO TO TAKE IT EVEN FURTHER
OUT WE THINK OF THIS AS WHERE
MENTAL HEALTH ASSESSMENT WILL
GO, WHICH IS THAT WE'LL HAVE
THESE SYSTEMS OF COMPREHENSIVE
PHENOTYPING THAT TRY TO LINK
TOGETHER, YOU KNOW, DIFFERENT
DATA SOURCES BUT ULTIMATELY TRY
TO DO THAT THROUGH RELEVANT
LATENT CONSTRUCTS THAT WE KNOW
AREN'T GOING TO HAVE THE EXACT
SAME MAPPING IN EVERY SINGLE
PATIENT THAT WE ENCOUNTER, BUT
THAT BY UNDERSTANDING WHERE
THOSE MAPPINGS ARE MORE RELIABLE
WE CAN BEGIN TO HAVE BETTER
SYSTEMS THAT ARE ABLE TO WORK
WELL BOTH AT THE INDIVIDUAL
LEVEL AND APPLIED TO GROUPS.
I WON'T GO INTO ANY OF THIS
TODAY BUT I THINK WE'VE HEARD A
LITTLE BIT ABOUT WAYS WE CAN
BEGIN DESIGNING CONSUMER
EXPERIENCES, TO ACTUALLY
LEVERAGE CHANGES IN THE SIGNALS,
TO PROVIDE BETTER CARE SYSTEMS.
AND ALSO BE ABLE TO ANTICIPATE
CHANGES IN CARE CAPACITY
RELEVANT.
AND SO I WANTED TO BRIEFLY
MENTION A STUDY THAT WE'VE BEEN
DOING, SUPPORT FROM NIMH, PART
OF WHAT'S CALLED INTENSIVE
LONGITUDINAL NETWORK.
DANA SCHLOSSER WAS INVOLVED, IT
SPANNED MULTIPLE OF THE NIH
INSTITUTES FROM NATIONAL CANCER
INSTITUTE AS WELL AS SOME NIAAA
TO TRY TO USE THESE MULTI-MODAL
MULTI-LEVEL ASSESSMENTS TO
FOLLOW INDIVIDUALS WITH A WHOLE
RANGE OF DIFFERENT HEALTH
CONDITIONS, OVER LONGER PERIODS
OF TIME COULD BEGIN SEEING
COMMON ELEMENTS FROM PATHOLOGY
AS WELL AS THINGS THAT ARE
SPECIFIC TO THE CONDITION BEING
STUDIED.
WHAT OUR GROUP HAS BEEN FOCUSED
ON IS FOLLOWING 100 INDIVIDUALS
FOR AT LEAST ONE YEAR, WHO ARE
LIKELY TO EXPERIENCE DEPRESSION,
BIPOLAR DISORDER, PSYCHOTIC
CONDITION, WE PUT INTO REGISTER
ALL THEIR SELF-REPORTS AS WELL
AS CLINICAL EVALUATIONS AND
OTHER OBJECTIVE MEASURES.
AND I THINK JUST TO HIGHLIGHT
THAT THESE ARE THE KINDS OF DATA
THAT WE CAN BEGIN TO COLLECT AT
SCALE, WHERE WE CAN THINK OF A
SINGLE PIXEL CAMERA OR THREE
PIXEL CAMERA ON THE WRIST, WRIST
WEARABLE, WITH MORE PERIODIC
ENCOUNTERS OF A VIDEO, AND TO
REALLY TRY TO FIND INDIVIDUALS
WHO ARE GOING THROUGH
SIGNIFICANT CHANGE IN THEIR
CLINICAL STATUS TO UNDERSTAND
HOW THE SIGNALS ARE CHANGING IN
MEANINGFUL WAYS.
AGAIN, JUST TO HIGHLIGHT THAT
THE CONCEPT OF THIS NETWORK WAS
TO ENCOURAGE PROJECTS TO NOT
JUST FOCUS THE CONSTRUCT THEY
WERE MOST INTERESTED IN BUT ALSO
TO FOCUS IN ON ALL SORTS OF
BEHAVIORS THAT MIGHT BE COMMON
ACROSS THE HUMANS IN THE STUDY
AND FROM HEARING THE TALKS TODAY
THE DIFFERENT SESSIONS WE CAN
BEGIN TO THINK ABOUT WAYS TO
MOTIVATE SIMILAR STUDIES THAT
ARE ACROSS SPECIES NATURE, EVEN
IF YOU'RE NOT NECESSARILY
INTERESTED IN, YOU KNOW, A
GROOMING BEHAVIOR, YOU COULD
COLLECT ENOUGH HIGH DENSITY DATA
OF OTHER ADJACENT BEHAVIORS
AROUND OUR CONSTRUCT OF INTEREST
TO INFORM A MORE DATA-DRIVEN
ASSESSMENT IN LINE WITH OTHER
STUDIES COLLECTING SIMILARLY
GRANULAR DATA.
ONE QUICK EXAMPLE OF THE KIND
MUCH DATA WE CAN GET FROM A
SINGLE PARTICIPANTS.
THIS IS SOMEONE WE FOLLOWED FOR
OVER TWO YEARS.
AS YOU HEARD FROM OTHER TALKS
TODAY WE CAN ASK QUESTIONS ON
THE SMARTPHONE, THIS INDIVIDUAL
EXPERIENCED PROFOUND LONG
PERIODS OF POSITIVE AFFECT
FOLLOWED BY PROFOUND PERIODS OF
NEGATIVE AFFECT, AGAIN POSITIVE
AFFECT, YOU CAN SEE THAT THERE'S
A RHYTHMICITY, FAIRLY DENSE
PERIODS, EVEN THOUGH THE PERIODS
WERE -- SHE'S EXPERIENCE
POSITIVE AND NEGATIVE AFFECT
CHANGE, THEY ARE QUITE DENSE.
AND I THINK AS WE WOULD START TO
SEE THIS -- IT'S NOT GROUND
TRUTH BUT IT'S A STRONG
INDICATION OF WHAT STATE SHE'S
IS, WE CAN THEN ADD OTHER
CLINICAL MEASURES SUCH AS THE
VERY LOW DENSITY ASSESSMENTS WE
WOULD DO FROM CLINICAL
ASSESSMENTS EVERY MONTH TO SHOW
HOW THEY ARE MORE OR LESS
CORRELATED WITH THE SELF-REPORTS
BUT OBVIOUSLY DELAYED AND MUCH
LESS TEMPORALLY DENSE AND
COMBINE THOSE WITH OTHER VERY
DENSE ASSESSMENTS FROM A
WEARABLE, AND TODAY I DON'T WANT
TO GET INTO TOO MUCH OF THE
SCIENCE HERE BUT JUST TO POINT
OUT THAT WE DO START TO SEE
REALLY STRIKING CHANGES AS WAS
BROUGHT UP IN ONE OF THE EARLY
COMMENTS ABOUT WHEN SOMEONE DOES
CHANGE FROM DEPRESSION TO NOT
BEING DEPRESSED, WHAT'S THE
SEQUENCE, DO THEY FIRST FEEL
BETTER AND LOOK BETTER?
HERE WE SEE STRIKING CHANGES IN
THE SLEEP PATTERN, PATTERN OF
HOW THEY ARE SLEEPING, ALSO THE
LEVEL OF ACTIVITY DURING THE
DAY, WHICH ROUGHLY SPEAKING DOES
CHANGE IN ACCORDANCE WITH THOSE
OTHER SUBJECTIVE STATES, SO WE
ALREADY SEE THAT IN THESE
EXAMPLES BY FOLLOWING PEOPLE
OVER LONG PERIODS OF TIME WE
BEGIN TO CAPTURE THAT NATURALLY
OCCURRING PHENOMENONOLOGY IN THE
LABORATORY, USUALLY ONLY IN A
SINGLE TIME POINT.
WE ONLY CAN FOLLOW AN
INDIVIDUAL OR GROUP OF
INDIVIDUALS TO RESOLVE SOME
RELATIONSHIPS AND BEGIN TO HAVE
MUCH STRONGER HIGH POTENTIAL
APPROACHES THESE FOR WHAT CAUSAL
SYSTEMS ARE UNDERLYING.
WE HAVE DEVELOPED AN ENERGY SORT
OF ALLOSTASIS-HOMEOSTASIS
MODEEL, TESTABLE IN LARGER
POPULATIONS.
AND SO I'LL JUST END MY COMMENTS
BY SAYING A LOT OF THIS WORK IS
REALLY GEARED TOWARDS MOVING
BEYOND CARICATURE VIEWS OF WHAT
MENTAL ILLNESS IS OR UNDERLYING
BEHAVIORS, TO BE ABLE TO CONNECT
SUBJECTIVE EXPERIENCES WITH
THINGS THAT ARE OBJECTIVELY
MEASURABLE.
AND SO I THINK WE'RE CERTAINLY
DOING THIS AND TRYING TO HELP
BUILD TOOLS TO DO THIS, TO BE
ABLE TO COLLECT LARGER DATASETS,
TO SCALE THEM INTO THE THOUSANDS
OF INDIVIDUALS.
I WILL JUST ALSO END BY NOTING,
WE'RE SOMETIMES ASKED WHETHER
EVENTUALLY PSYCHIATRY WILL BE
REPLACED BY ARTIFICIAL
INTELLIGENCE.
PART OF A DEBATE A COUPLE YEARS
AGO I WAS COMING DOWN AGAINST
THE NOTION BUT WORTH THINKING
ABOUT WHAT ARE THE TOOLS THAT
WILL EVENTUALLY REPLACE WHAT A
HUMAN IN THE LOOP IS DOING AND
WHAT ARE THE ROLES THAT ARE
GOING TO CONTINUES TO BE
PERFORMED BY HUMANS.
AND I THINK OF RELEVANCE TO SOME
OF THE OTHER ETHICAL
CONSIDERATIONS WE TALKED ABOUT
TODAY WHICH IS HOW DO WE KNOW A
PARTICULAR SET OF TOOLS IS
ACTUALLY READY FOR A PARTICULAR
USE, AND WHO GETS TO DECIDE?
IS THIS SIMPLY A QUESTION OF THE
MARKETPLACE WHERE PEOPLE ARE
WILLING TO SPEND THEIR MONEY?
IS IT THE GOVERNMENT REGULATIONS
WILL DETERMINE THIS?
OR PHYSICIANS WHO HAVE TO DEAL
WITH COMPLEX CHALLENGING ISSUES
WILL TURN TO WHATEVER SOLUTIONS
ARE OUT THERE, EVEN WHETHER THE
EVIDENCE IS STRONG OR NOT.
AND SO ALSO JUST TO MENTION THAT
TIMOTHY BROWN'S QUESTION IN THE
LAST SESSION, WE TAKE THE ETHICS
OF THE WORK SERIOUSLY BECAUSE I
THINK THERE IS THIS QUESTION OF
WHETHER WE CAN MOVE BEYOND THE
BENCH WITH CAUTION BUT I THINK
IT'S ALSO THE CASE THAT MANY OF
THESE ANALYTICAL TECHNIQUES AND
TECHNOLOGIES ARE NOW AVAILABLE
AND SO I THINK THE PATIENTS WHO
ULTIMATELY ARE EXPERIENCING THIS
CONDITION DESERVE APPLICATION,
AS SOON AS IT'S POSSIBLE, AS
SOON AS IT'S NOT GOING TO BE
UNSAFE FOR THEM TO DO SO.
SO AS PART OF OUR WORK WE'RE
TRYING TO MOVE AS AGGRESSIVELY
TOWARDS PATIENT APPLICATIONS AS
POSSIBLE SO WE RECENTLY
PUBLISHED AND ETHICS CHECK LIST
IN DIGITAL HEALTH GIVEN
COMPLEXITIES ABOUT ACHIEVING
INFORMED CONSENT IN MULTI-MODAL
LONGITUDINAL STUDIES AND ENSURE
EQUITY, DIVERSE, ACCESS,
PRIVACY, REGULATIONS, AND
IMPORTANTLY RETURN OF RESULTS.
WITH THAT I WILL WRAP UP MY
COMMENTS AND NOTE AGAIN JUST
WANT TO THANK THE FOLKS FOR
INVITING ME TO PARTICIPATE AND
ALL OF THE AMAZING SPEAKERS THAT
WE HAD TODAY.
APPRECIATE IT.
>> THANK YOU SO MUCH, JO TIN,
-- JUSTIN AND KAREN.
I APPRECIATE YOU ENDING US.
OF ON THIS TALK.
I KNOW THE DISCUSSION TIME
SEEMED COMPACT, TALKS WERE
SHORT, HIGH LEVEL CONCEPTUAL
TALKS.
AND I HOPE THIS WHETS YOUR
APPETITE FOR THE DISCUSSIONS
THAT ALL OF YOU PARTICIPANTS ARE
GOING TO BE TAKING PART IN
TOMORROW.
SO TOMORROW WILL BE THE THREE
MORE HIGH LEVEL TALKS FOLLOWED
BY TWO CONCURRENT BREAKOUT
SESSIONS WHETHER WE'LL GET TO
DIG INTO MORE OF THESE THOUGHTS
AND IDEAS WHAT WERE BROUGHT UP,
AND THEN WE'LL FOLLOW UP WITH A
SUMMARY AND CONCLUSION.
WITH THAT, I WILL CLOSE THE
MEETING.
THANK YOU, EVERYONE, FOR
STICKING IT OUT FOR THIS LONG.