WELCOME TO DAY TWO
THANK YOU FOR THE RICH
DISCUSSIONS YESTERDAY.
I START WITH AN UPDATE ON
FOLLOW-UP FROM THE BREAKOUT
ROOMS YESTERDAY.
THERE WAS NO REPORT BACK.
THAT WAS THE PRINCIPLE.
WE CAPTURED ALL THE BOARDS THAT
WERE IN THE ROOMS AND THERE IS
WAY TOO MUCH TO SYNTHESIZE IN
ANY MEANINGFUL WAY THIS MORNING
BUT WILL WORK WITH THE
MODERATORS FROM EACH ROOM TO
COME BACK WITH A SYNTHESIS AFTER
THE MEETING.
DON'T WORRY.
THE WORK THAT WAS DONE YESTERDAY
IS ABSOLUTELY CAPTURED AND WILL
BE SYNTHESIZED AND DISCUSSED
FURTHER.
SIMILARLY WE'LL GO INTO OTHER
EXERCISES THIS MORNING AND THE
GOOD THING ABOUT THE SETUP THIS
MORNING IS LUNCH IS RIGHT AFTER
THE BREAKOUT ROOMS SO YOU'LL
HAVE THE ABILITY TO SORT OF WALK
AROUND THE BREAKOUT ROOMS IF YOU
WANT TO SEE HOW THE SITUATION
HAS EVOLVED SINCE YOU'VE BEEN IN
THE FIRST ONE.
SO YESTERDAY WE REALLY FOCUSSED
ON INDIVIDUALS.
WE TALKED ABOUT MOTIVATIONS OF
INDIVIDUALS.
WE TALKED ABOUT HOW THEY
INTERACT IN NETWORK.
AND TODAY WE'RE GOING TO MOVE UP
A LEVEL AND TRY TO STE THE
CONTEXT IN WHICH THEY OPERATE.
THIS MORNING WE'LL FOCUS ON
INSTITUTIONS AND THIS AFTERNOON
WE'LL GO ONE MORE LEVEL UP AND
LOOK AT THE SYSTEMS.
THIS IS HOW WE'RE GOING TO SPEND
THE DAY AND HAVE THESE DIFFERENT
LENSES AND LEVELS.
THE CONTEXT IS EXTREMELY
IMPORTANT.
IT'S DETERMINANT FOR THE
IMPROVEMENTS THAT WE WANT TO SEE
HAPPENING.
AND I WAS REMINDED -- REFLECTING
ON THE KEY NOTE YESTERDAY.
THERE WORE TWO THINGS THAT STUCK
WITH ME.
ONE WAS REALLY THERE ARE NO
NEUTRAL INTERACTIONS.
NON-ADOPTERS ARE CONTRAVEILING
INFLUENCES AND THAT IS IMPORTANT
AND THEN THE OTHER CONCEPT WAS
THE FACT THAT IT'S NOT ABOUT
SPECIAL PEOPLE IT'S ABOUT
SPECIAL PLACES IN THE NETWORK
WHERE THESE PEOPLE ARE AND WHAT
IS IT ABOUT THE SPECIAL PLACES
WHERE CLUSTER OF INDIVIDUALS CAN
BE SHELTERS FROM CONTRAVALING
INFLUENCES AND OTHER CLUSTERS
WHERE IT CAN SPREAD.
THESE CONCEPTS I THINK ARE
USEFUL TO KEEP IN MIND AS WE ARE
TRYING TO DERIVE ACTIONABLE
INSIGHT FROM THE MEETING TODAY
AND AS WE THINK ABOUT WHAT THE
CLUSTERS MIGHT MEAN FOR THE
COMMUNITIES OF RIGOR CHAMPION WE
WANT TO SUPPORT.
I WAS REMINDED LAST NIGHT THERE
ARE DIFFERENT KINDS OF
INSTITUTIONS.
WE'RE FOCUSSED ON ONE TYPE OF
INSTITUTIONS BUT THERE ARE THE
HEAVILY NIH-RELIANT RESEARCH
INSTITUTIONS AND THEN THERE ARE
INSTITUTION ARE UNDERGRAD
FOCUSSED, FOR EXAMPLE.
WE HAVE REPRESENTATIVES FROM THE
DIFFERENT GROUPS AND DIFFERENT
TYPES OF INSTITUTIONS IN THE
ROOM.
I WANT TO ENCOURAGE YOU TO SPEAK
UP FROM YOUR PERSPECTIVE AND
DRAW THE DIFFERENCES AND THE
IMPORTANT CHARACTERISTICS OF
THESE TYPES OF INSTITUTIONS.
THERE WILL BE THREE OR FOUR
THEMES THIS MORNING WE WILL
EXPLORE.
LIKE YESTERDAY WE'LL START BY A
PANEL DISCUSSION AND THEN GO TO
THE BREAKOUT ROOMS.
THE FIRST THEME SO THE THEMES
ARE TOOLS, NORMS AND REWARDS.
SO AROUND TOOLS IT'S HOW DO WE
ENABLE THIS IN RESEARCH AND WHAT
ARE THE TOOLS THAT EXIST AND HOW
CAN THEY BE IMPLEMENTED TO
FACILITATE THE IMPROVED
BEHAVIORS WE WANT TO SEE AND HOW
CAN WE CREATE AN EASY PATH FOR
PEOPLE WHO ARE DECIDING TO ADOPT
AN IMPROVEMENT.
IT MAY NOT BE THE EASIEST PATH
BUT LET'S NOT MAKE IT THE MOST
BURDENSOME AND CURRENTLY THAT'S
OFTEN THE CASE TO DO THE RIGHT
THING IS THE HARDER THING TO DO.
WE'VE HEARD ABOUT SOME OF THE
TOOLS YESTERDAY AND WE WILL HEAR
MORE TODAY AND ALSO DISCUSS HOW
THE TOOLS CAN BE IMPLEMENTED.
SECOND THEME IS AROUND NORMS.
NORMS THAT EXIST IN THE CONTEXT
IN WHICH RESEARCHERS ARE WORKING
AND HOW THE NORMS CAN BE
ADJUSTED OR INFLUENCED TO WHICH
MECHANISMS AND ENCOURAGE YOU TO
THINK OF A DIFFERENT UNIT AND
LAB UNIT AND HARD NORMS AND
MAINTAINED IN LABS THROUGH PEER
NETWORKS, THROUGH MENTORS.
HOW ARE THESE IMPORTANT AND HOW
CAN WE HELP INFLUENCE THEM.
AND THE THIRD IS AROUND REWARDS
AND THAT'S AROUND THE FORMAL
EVALUATION SYSTEM AND CARRIER
PROGRESSION THAT EXISTS AT THE
INSTITUTIONAL LEVEL.
DIFFERENT UNITS OF SCALE THERE
BUT REALLY LOOKING AT THESE
THREE MECHANISMS AND WE'RE
LOOKING AT TOOLS, NORMS AND
REWARDS BOTH IN TERMS OF
CREATING POSITIVE ENVIRONMENT
AND POSITIVE REINFORCEMENT BUT
ALSO REMOVING OBSTACLES WHERE
THEY EXIST CURRENTLY.
THESE WILL BE THE THEMES THIS
MORNING.
THIS AFTERNOON WE'LL MOVE TO
MORE SYSTEMS LEVEL AND HOW WE
CAN WORK TOGETHER AND WHAT ARE
OTHER STAKEHOLDERS THAT ARE
IMPORTANT AND ANOTHER THING I
WAS REMINDED OF LAST NIGHT WE
DON'T KNOW WHICH INTERVENTIONS
IMPROVE RESEARCH.
WE HAVE A GOOD HUNCH.
WE THINK SOME ARE DEMONSTRATED
IN SOME CONTEXT BUT IF WE ARE
ALL ABOUT RIGOR WE ARE TO APPLY
RIGOR TO THE TYPES OF
INTERVENTIONS WE ARE TALKING
ABOUT TO IMPROVE RESEARCH AS
WELL.
SO REALLY THINKING ABOUT WHAT
ARE GOING TO BE THE SYSTEMS THAT
WILL ALLOW US TO TEST RIGOROUSLY
INTERVENTIONS AND HELP
DEMONSTRATE THEIR EFFECTIVENESS
AND SPREAD THAT WAY AS WELL.
THINGS TO KEEP IN MIND AS WE GO
THROUGH THE DAY.
I'D LIKE TO INVITE ON THE STAGE
THE PANELIST FOR THE SESSION
THIS MORNING.
THE SESSION WILL BE MODERATED BY
JUAN PABLO ALPERIN FROM FRASER
UNIVERSITY.
HE IS ASSOCIATE PROFESSOR IN THE
PUBLISHING PROGRAM AND
CO-SCIENTIFIC DIRECTOR OF THE
PUBLIC KNOWLEDGE PROJECT AND
CO-DIRECTOR OF THE SCHOLARLY
COMMUNICATIONS LAB AND I WOULD
REALLY ENCOURAGE YOU, IF YOU
HAVEN'T ALREADY, TO REALLY CHECK
THE AMAZING WORK THAT THEY'VE
DONE ACTUALLY AT THE LABS AROUND
THE RPT, REVIEW, PROMOTION AND
TENURE COMMITTEE IN THE U.S. AND
CANADA.
THERE'S A REALLY SOBERING
ANALYSIS AND I HOPE WE'LL HEAR
SOME OF THAT TODAY FROM JUAN
PABLO.
SO THANK YOU.
>> I'VE BEEN ASKED TO REMIND
EVERYONE TO MAINTAIN SIX FEET
SOCIAL DISTANCING IN THE
AUDITORIUM WHERE POSSIBLE.
THANK YOU.
>> I WEAR THREE HATS.
ONE AS AN ASSOCIATE PROFESSOR
AND DIRECTOR OF THE PROJECT I'VE
BEEN INVOLVED WITH SINCE BEFORE
I HAD MY MASTER'S GOING ON 17
YEARS NOW AND NOW CO-SCIENTIFIC
DIRECTOR AND THE THIRD ROLE I
HAVE IS CO-DIRECTOR OF A
SCHOLARLY COMMUNICATIONS LAB A
RESEARCH GROUP I CO-DIRECTOR AT
THE UNIVERSITY OF OTTAWA.
LOOKING BROADLY AT ISSUES OF
SCHOLARLY COMMUNICATION.
I ALWAYS BRING ALL OF MYSELF
WHEN I GO ANYWHERE.
I THINK I AM HERE WITH TWO OF
THOSE ROLES IN PARTICULAR.
ONE IS I THINK IN THIS PANEL
TALKING ABOUT TOOLS IN MY
CAPACITY AS DIRECTOR OF A
PROJECT THAT HAS BEEN DOING OPEN
SOURCE SOFTWARE FOR PUBLISHING.
THE PRODUCT'S BEEN AROUND 20
YEARS AND IT'S AND THE SOFTWARE
IS USED AROUND THE WORLD AND I
THINK IT'S A GREAT EXAMPLE OF
HOW WE CAN SEE AND SHAPE WHAT
THE LANDSCAPE LOOKS LIKE.
WHILE IT'S NOT SPECIFIC TO RIGOR
AND SCIENCE IT'S BUT IS IT ABOUT
THE MODES OF PUBLISHING AND
WHOSE VOICE HEARD AND ABLE TO
TAKE PART IN THE SCHOLARLY
CONVERSATION WHICH IS AN
IMPORTANT PART OF QUALITY IN
RESEARCH.
I'M GOING TO STAY BRIEF TO KEEP
TIME FOR PANELISTS I WANTED TO
PRESENT A LITTLE BIT OF MYSELF
AND WHAT I'M BRINGING SO AS I
MODERATE AND TAKE AND
PARTICIPATE IN THE CONVERSATION
I HOPE YOU'LL HAVE A LITTLE BIT
OF THAT CONTEXT.
THE WORK WE DO IS AROUND WHAT
FEATURES DO WE BUILD IN THE
SOFTWARE NOW THAT IT'S GOT THIS
BROAD ADOPTION?
WE THINK OF WHAT FEATURES DO WE
BRING TO CREATE THE CHANGES IN
THE SCHOLARLY PRACTICES
UNDERNEATH.
SOME THINGS ARE PRAGMATIC
INCLUDING CERTAIN METADATA
FIELD.
WE WANT TO MAKE SURE PEOPLE ARE
ADOPTING THE USE TO HAVE GREATER
TRANSPARENCY AND BUILD THAT IN
THE SOFTWARE AND PUT IT IN FRONT
OF PEOPLE AND SOME IS AROUND HOW
WE PHRASE THE E-MAIL TEMPLATES
IN THE SOFTWARES TO SO YOU
NUDGE THE EDITORS, MY ROLE IS TO
SYNTHESIZE WHAT'S GOING ON WITH
THE REVIEWERS HAVE DONE.
AGAIN, WE USE THE TECHNOLOGIES
IN THIS WAY AND THE PROJECT AND
PEOPLE OFTEN THINK OF IT AS
PUBLISHING PLATFORM THE
SCHOLARLY LEADING OF THAT
PROJECT MEANS WE'RE THINKING
ABOUT WHERE THE QUALITIES OF HOW
THAT SCHOLARSHIP SHOULD TAKE
PLACE AND HOW CAN WE THROUGH THE
CREATION OF THE SOFTWARE NUDGE
OR ENCOURAGE CERTAIN KINDS OF
BEHAVIORS AND ACTIONS IN HOW
THEY DO THEIR WORK.
THAT'S SOMETHING TO KEEP IN MIND
AS WE HAVE THE CONVERSATION.
THE SECOND MAIN PART AND THE
PIECE THAT CAUGHT THE ATTENTION
OF THE ORGANIZERS AND WE TEST
CONCLUDED AND I INVITE YOU TO
CHECK OT THE WEBSITE FOR CANADA
FOR WE DID A STUDY ACROSS
REPRESENTATIVE SET OF
UNIVERSITIES IN U.S. AND CANADA
LOOKING AT OUR TYPE OF
INSTITUTIONS AND BACCALAUREATE
INSTITUTIONS AND I SAW YESTERDAY
IN ONE OF THE ROOMS THERE WAS A
LOT OF DOTS OF INCLUDING RIGOR
IN THE RPT PROCESS AND THERE WAS
A CONVERSATION WHERE THEY WANTED
TO HAVE A PROJECT ABOUT CHANGING
THE FORM IT WAS LIKE LET'S
INCLUDE OPENNESS AND THE
RESEARCHER WAS LIKE WHAT
EVIDENCE DO WE HAVE AND WHAT IS
OUR BENCHMARK FOR HOW MUCH THOSE
ELEMENTS ARE ACTUALLY IN THE RPT
PROCESS RIGHT NOW.
SO WE TOOK IT UPON OURSELVES TO
CLETH THE -- COLLECT THOSE
THINGS AND WE MANAGED TO GET
FUNDING TO DO A NICE SET OF INFO
GRAPHICS AND WE FORMALLY WRAPPED
UP THAT PROJECT.
ONE OF THE MAIN TAKEAWAYS WAS
ADDING TO THE RPT PROCESS OUR
CONCLUSION IS IT WAS NOT ENOUGH
TO CHANGE THE CULTURE.
YOU CAN ADD MORE THINGS AND
THERE'S A LOT OF A LOT OF LIP
SERVICE TO THOSE PROCESSES AND
UNIVERSITY VERY GOOD ABOUT
TALKING THOSE THINGS AND THERE'S
TRADITIONAL METRICS SO THE
INCENTIVES CONTINUE TO BE IN
PLACE.
I WANTED TO CHALLENGE YESTERDAY
BUT THERE WERE TOO MANY PEOPLE
LINED UP AT THE MIC AND I'LL
TAKE THE OPPORTUNITY TO SAY IT'S
NOT A REVOLUTION.
IT'S AN EVOLUTION NOT A
REVOLUTION.
FOR ME I QUESTION WHY WE MOVE
THINGS IN AN EVOLUTION WAY.
THE EVOLUTION ADDS SOMETHING TO
THEM AND THE REVOLUTION IS LET'S
BUILD IT FROM THE GROUND UP.
WE SHOULDN'T ALWAYS GET CAUGHT
UP ON THE PRAGMATIC NEXT STEPS.
IT WOULD BE EASIER TO MAKE A
LITTLE CHANGE AND PUT ON THE
TABLE SOMETIMES THE RIGHT THING
TO DO IS WHAT WE BUILT SO FAR IS
GARB
GARBAGE.
WE NEED TO TEAR IT DOWN AND PUT
SOMETHING PROPER IN PLACE AND IT
REQUIRES A VERY STRONG PARADIGM
SHIFT IF WE WANT TO SEE CHANGES
IN HOW WE DO THINGS.
I DON'T WANT TO TAKE UP TOO MUCH
TIME.
WE HAVE FOUR GREAT PANELISTS AND
I WON'T GIVE THEIR BIOS BUT GIVE
A SENSE OF WHAT'S TO COME FOR
ROLLING OUT AND SIGN POSTING.
WE'LL HAVE TIM ERRINGTON TALKING
ABOUT INFRASTRUCTURE AND CHANGE
AND THE LIMITS OF WHAT THAT
MIGHT LOOK LIKE.
FOLLOWED BY BECKY LUNDWALL
SPEAKING OF CHANGES AT THE
DEPARTMENT LEVEL AND REVIEW AND
RANK DOCUMENTS.
THAT'S BUILDING ON WHAT I WAS
TALKING ABOUT BUT HER SPECIFIC S
OF HAVING DONE THOSE CHANGES.
THEN MELISSA RETHLEFSEN WELL
BRING US UP TO THE INSTITUTION
LEVEL BY SHARING EXPERIENCE OF
TWO DIFFERENT INSTITUTIONS AND
HAVING THE IMPORTANCE OF
SUCCESSFUL CHANGES TALKS ABOUT
CHANGE IN DIFFERENT INSTITUTIONS
AND TRYING TO MAKE CHANGES AND
FINALLY UL -- ULRICH DIRNAGL
WILL TALK ABOUT CHANGE MODEL AND
WHAT'S BEHIND IT.
THAT'S A LITTLE BIT OF WHAT'S TO
COME AND I'LL STEP ASIDE AND
TURN IT TO TIM TO GET IT
STARTED.
>> FIRST, THANKS FOR THE
INVITATION AND CHANCE TO TALK.
I'M GOING TO TEE UP A LITTLE BIT
ON HOW INFRASTRUCTURE PLAYS A
ROLE BUT FIRST SET UP THE STAGE
AND GO A LITTLE BIT BIG PICTURE
IF THAT'S OKAY.
FIRST I WANT TO START A LITTLE
BIT BY PUTTING BACK WHAT WE WERE
HEARING YESTERDAY AND THERE'S
LOTS OF AND GOING ON THE '50s
AND '60s OF THINGS WE FACE AND
THINGS WE WERE ALL TALKING ABOUT
YESTERDAY AND I'VE TALKED ABOUT
THIS AND THERE'S A LOT THERE BUT
NOT NECESSARY UNIQUE AND THE
SOLUTIONS.
THESE ARE NOT UNIQUE EITHER.
THEY'VE BEEN TALKED ABOUT IN THE
PAST.
NOT SAYING WE SHOULDN'T KEEP
TALKING ABOUT THEM, I'M A BIG
FAN OF IT TOO BUT THINKING WHY
WE'VE BEEN ABLE TO SOLVE AND
DISCUSS THE CHALLENGES, TALK
ABOUT SOLUTIONS BUT NOT GET TO
IMPLEMENTATION.
WHEN WE THINK ABOUT IT, IT'S A
BIT OF A STRATEGY TAKEN TO DATE
WHICH IS LET'S STUDY IT.
I'M A FIRM BELIEVER IN BECAUSE I
DO IT MYSELF AND EDUCATE AND
TRAIN AND WE KNOW THAT'S
IMPORTANT AND GET THE VALUES
EMBEDDED WITHIN THE COMMUNITY.
WE KNOW THESE ARE THE NECESSARY
STEPS.
BUT IT'S NOT SUFFICIENT TO GET
CHANGE.
I THINK THAT'S THE KEY.
ONE OF THE ISSUES IN ACADEMIC
MODEL IS WHERE THE INCENTIVES
LIE.
IF THEY'RE NOT IN THE RIGHT
PLACE IT'S GOING TO BE ONE OF
THE MANY BARRIERS THERE.
WHAT IT MEANS IS WE NEED TO
START ADDING THINGS TO THE
SYSTEM AND THINK ABOUT THE
REWARD SYSTEM AND THINK HOW
INFRASTRUCTURE PLAYS A ROLE IN
THIS.
SO AGAIN GETTING UP AT A HIGHER
LEVEL TO PUT BACK INTO CONTEXT
WHAT I THINK AND HOW
INFRASTRUCTURE CAN PLAY A ROLE.
SO WE THINK ABOUT THIS A LOT AT
AND TRYING TO ARTICULATE IS WHAT
DOES IT MEAN TO CHANGE A
RESEARCH CULTURE AND WHAT ARE
THOSE PIECES AND WHY THEY'RE IN
PLACE.
INFRASTRUCTURE IS THE MEANS TO
MAKE IT POSSIBLE.
WE HAVE TO THINK ABOUT WHAT IS
THE EXPERIENCE FOR THE USER AND
MAKE SURE IT WILL BE ANNESY AN
EASY PROCESS AND IT'S THE
NORMATIVE BEHAVIOR THAT'S
IMPORTANT AND WE'RE NOT -- WE
ARE ALL SCIENTISTS AND WE ALL
COME FROM DIFFERENT BACKGROUNDS.
PREVIOUSLY A PRE-CLINICAL CANCER
BIOLOGIST AND WOULD LIKE TO SEE
BEHAVIORS IN THAT COMMUNITY AND
WE WANT TO HAVE INCENTIVES TO
MAKE IT REWARDING AND POLICIES
TO MATCH.
THE KEY I THINK IN THIS AT THE
WAY OF URBAN SCIENCE IS
ALIGNMENT.
YOU WANT TO MAKE SURE YOU DON'T
HAVE POLICIES MISALIGNED WITH
THE INFRASTRUCTURE TO ENABLE IT
TO OCCUR.
AND REMEMBER WE'RE DOING THAT IN
AN ECO SYSTEM.
THE RESEARCHER IS NOT ALONE.
THEY'RE SUPPORTING THEM.
I THINK THAT WAS MENTIONED
BEFORE.
WE'RE ALL HUMANISTIC.
THAT'S A COMMON ELEMENT AND IN
THE SYSTEM WORKING TOGETHER.
I'LL GIVE YOU EXAMPLES AND SPEND
MORE TIME ON THE INFRASTRUCTURE
PIECE BUT I WANT TO PUT IT BACK
IN CONTEXT.
HERE'S THINGS WE'RE DOING AT THE
CENTER FOR URBAN SCIENCE AND
COME DOWN AS DIFFERENT ELEMENTS
OF THE PAIR -- PYRAMID AND
TRYING TO SHIFT AT JOURNALS AND
FUNDING AGENCIES AND
INSTITUTIONS TO SAY DO YOU HAVE
A FRAMEWORK TO ALIGN WHAT YOU
ARE HOPING TO GET FROM
RESEARCHERS WITH WHAT YOUR
POLICIES ACTUALLY ARE.
THINGS AROUND DATA CITATION AND
REGISTRATION, ANALYSIS.
YOU CAN ALSO HAVE INCENTIVES
BAKED IN AND REGISTERED REPORTS
WHICH IS A DIFFERENT
COMPLEMENTARY WAY OF STILL
HAVING PUBLICATIONS OCCUR BUT
EMBEDDING RIGOR IN THE PROCESS
AND NOT LETTING IT BE A BARRIER
TO THE WAY THE PAPER GETS
PUBLISHED.
IT'S NOT WHAT YOU FIND BUT HOW
YOU DICTATE THE PUBLICATION
PROCESS.
THIS CAN EMBED GOOD RIGOR AND
PRACTICE WITHIN THE SYSTEM BY
INNOCENTSIV
IN -- INCENTIVIZING AND IT'S NOT
THAT I'M GIVING YOU A BADGE BUT
I AM SHOWING IT TO THE
INDIVIDUALS IN THE COMMUNITY.
I'M SHOWING THAT'S PART OF OUR
NORMATIVE BEHAVIOR.
YOU CAN SEE I'M DOING IT AS
WELL.
IT'S NOT THE ENTIRE PART.
WE NEED TO MAKE SURE WE'RE ABLE
TO DO THAT AND THIS IS WHERE THE
INFRASTRUCTURE COMES IN TRYING
TO MAKE IT EASIER.
I'LL GIVE YOU A LITTLE BIT OF A
CENTER FOR URBAN SCIENCE CENTRIC
VIEW ON THIS.
THE WAY WE THINK ABOUT THIS AND
IT'S A BIT OF A BUSY SLIDE.
WE CAN BUILD OPEN SCIENCE
FRAMEWORK AND IT'S A TOOL THAT
ENABLES COLLABORATION AND
PRACTICES TO OCCUR.
IT'S NOT ALONE.
IT'S PART OF A LARGER SYSTEM
WHERE THERE'S OTHER DATA
REPOSITORIES AND OTHER SERVICES
PART OF THE SYSTEM.
AND OSF IN TERMS OF MAKING IT A
TOOL TO HAVE BETTER PRACTICES
AND DO THE BEHAVIORS THEY WANT
TO.
THE TRICK IS IF YOU JUST GIVE
THEM THAT TOOL WITH ALL THE WAYS
TO DO IT IT'S NOT VERY EASY AND
NOT EMBEDDED WITHIN THEIR
COMMUNITIES.
THE APPROACH WE TAKE IS TO THINK
OF THE INTERFACES SOMEBODY WILL
INTERACT WITH AND FOCUSSING ON
PREPRINTS OR PRE-REGISTRATION OR
HAVE THE COLLECTION OF THE
ARTIFACTS FROM THE RESEARCH OR
TRYING TO SHOW THE INSTITUTION
EVERYTHING I'M DOING WITH IT?
THAT CREATES INTERFACES BUT IT'S
STILL NOT IF YOU HAVE BECAUSE
IT'S NOT ALL PRE PRINTS
EVERYWHERE AND ALL INSTITUTION
IN ONE PLACE AND HAVE YOU
COMMUNITY INTERFACES.
AS THEY IS WHAT WE'RE STRUCTURED
AROUND AND SOCIAL SYSTEM WHERE
THERE'S SOCIETIES AND
INSTITUTIONS.
SO WE BASICALLY USE THAT AS AN
INTERFACE INTO THE DIFFERENT
INTERFACES TO TECHNOLOGY.
LET ME GIVE AN EXAMPLE.
THAT WAS A LOT.
I'M NOT SURE I WAS ABLE TO CLEAR
ARTICULATE THAT.
OSF YOU CAN PRE-REGISTER YOUR
STUDIES AND THIS ALLOWS YOU TO
GO AND UNDER AND DISCOVER
REGISTRATIONS AS WELL TO DO ONE
YOURSELF AND HAS A WORK FLOW TO
TRY TO MAKE IT EASIER.
I TRY TO GUIDE THE USER THROUGH
WHAT YOU'RE TRYING TO DOONED HOW
DO I CREATE THE METADATA TO MAKE
SURE YOU'RE REGISTERING IT AND
MAKE SURE YOU HAVE THE PROPER
DATA RESPONSES AND WHETHER
YOU'LL EMBARGO IT.
IT'S IT MAKES IT EASIER.
IT'S NOT JUST DO THIS BEHAVIOR
BUT A GUIDED FLOW TO DO THE
BEHAVIOR.
YOU CAN SAY OKAY, THE WAY I DO
IT IN ECONOMICS WON'T BE THE
SAME AS ANIMAL RESEARCH OR
CLINICAL RESEARCH.
SO WHAT WE DO IS WE DO BRANDED
REGISTRIES.
IT'S A WAY TO MAKE THE FORM, THE
TEMPLATE THE INFORMATION YOU PUT
INTO A REGISTRATION CUSTOMIZED
TO THE COMMUNITY.
THEY CREATE THOSE COMMUNITIES TO
DETERMINE WHAT ARE THE BEST
PRACTICES LIKE WE TALKED ABOUT
WHAT ARE CONSENSUS OF WHAT IS TO
THE METADATA AND IT INCLUDES
COLOR AND LOGO AND SIMPLE
INDICATORS THE INFRASTRUCTURE IS
PART OF YOUR COMMUNITY.
HAVE YOU A LANDING PAGE AND IT'S
DISCOVERABLE BY OTHERS.
THAT WAY YOU CAN HAVE A BASE
INFRASTRUCTURE ABLE TO SUPPORT
THIS VARIATION IN THE WAY OUR
COMMUNITIES ARE ABLE TO DO
THINGS.
NOW YOU CAN GO AND SEE LIKE YOU
CAN ON PUB MED SEE ALL THE
REGISTRATIONS BUT IF YOU LOOK
HERE YOU CAN LOOK AT SPECIFIC
REGISTRATIONS ON OSF AND NOT.
CLINICALTRIALS.gov.
YOU SHOULD BE ABLE TO TO ONE
PLACE AND FIND ALL THE
INFORMATION YOU NEED BUT IT'S
NOT NECESSARILY MY COMMUNITY, IT
MAY BE EGAP AND THAT'S HOW
TECHNOLOGY CAN HELP SUPPORT
THAT.
I'LL STOP THERE AND THANK YOU
AND WE'LL HAVE A GREAT
DISCUSSION HOPEFULLY.
>> I HAVE HANDOUTS SO YOU CAN
LOOK AT THEM BECAUSE I WON'T BE
ABLE TO COVER THEM.
I'M BECKY LUNDWALL AT BRIGHAM
YOUNG UNIVERSITY AND I'LL TALK
HOW WE CHANGED OUR RANK AND
STATUS DOCUMENT.
SO THE TABLES ARE HERE BUT THE
FIRST TABLE IS PROCESS BUT ALL
I'LL MENTION LIKE WHAT A REAL
ESTATE AGENT TELLS YOU, IT'S
ABOUT LOCATION, LOCATION,
LOCATION.
TALK TO PEOPLE, TALK TO PEOPLE,
TALK TO PEOPLE.
THE COMMITTEE WORKING ON
CHANGING RANK AND STATUS READ A
LOT OF LITERATURE.
WE CAN HAVE LOTS OF STATUS
PEOPLE INTERESTED IN DOCUMENTS
TO SUPPORT MORE RIGOROUS
SCIENCE.
WE HAD TO TALK TO THE CHAIR AND
DEAN AND UNIVERSITY TO MAKE SURE
THEY WOULD BACK US UP IF WE
CHANGED OUR RANK AND STATUS
DOCUMENT AND PERHAPS MOST
IMPORTANT TO ME IS WE HAD TO
TALK TO FACULTY OVER AND OVER
AGAIN.
WHAT ARE YOUR CONCERNS?
HOW WOULD YOU LIKE TO SEE THE
RANK AND STATUS DOCUMENTS
CHANGE.
THAT'S A KEY POINT ABOUT THE
PROCESS.
WE PROVIDE IN OUR RANK AND
STATUS DOCUMENT A LIST OF WHAT
WE CONSIDER QUALITY.
ESSENTIALLY WE DIVIDED AND
YOU'RE EVALUATING ON TEACHING
AND CITIZENSHIP AND RESEARCH AND
WE DIVIDED RESEARCHER
SCHOLARSHIP.
THIS IS A PSYCHOLOGIY DEPARTMEN
INTO QUALITY AND QUANTITY.
WE PROVIDED A LIST OF WHAT WE
CONSIDERED QUALITY.
I DO THINK WE NEED TO REVISE THE
LIST.
IT'S MOSTLY OPEN SCIENCE BUT
MADE A CASE FOR WHAT THEY
THOUGHT WAS MORE QUALITY
RESEARCH.
KEY POINTS IS BEING IBLE.
WE HAVE FUND IN OUR DEPARTMENT I
DON'T THINK IT'S SURVEILLANCE IS
PEOPLE THAT ALREADY HAD TENURE
WERE WILLING TO CHANGE.
THEY WERE LIKE, YEAH, YEAH, I'LL
TRY THAT, I'VE GOT TENURE I'M
PROI PROTECTED AND INCOMING
PEOPLE WERE EXCITED AND THE
PEOPLE MOST RESISTANT WERE THE
PEOPLE THREE TO FOUR YEARS FROM
SUBMITTING THEIR TENURE SO WE
HAVE COMPASSION FROM THEM.
YOU CAN'T CHANGE THE RULES IN
THE MIDDLE OF THEIR PIPELINE,
RIGHT?
SO WE TRIED TO BE FLEXIBLE
THERE'S TWO PATHS TO SUCCEED YOU
CAN GO THROUGH THESE QUALITY
STEPS THAT WE'RE SUGGESTING OR
OTHERS YOU SUGGEST.
ALSO SUPER CRITICAL TO BE
TRANSPARENT.
THIS REALLY HELPS CALM PEOPLE
DOWN.
WE HAVE PRACTICE DOCUMENTS EVERY
YEAR FOR ANNUAL REVIEW AND FOR
RANK AND STATUS THAT EVERYBODY
WHO'S EVALUATING ANYBODY ELSE
DOES THE PRACTICE DOCUMENTS AND
KEEP DOING THEM UNTIL THEY'RE
CALIBRATED.
UNTIL WE GET PRETTY CONSISTENT
RESULTS AND THEN EVERY APPLICANT
HAS TWO RATERS AND IF A THIRD
NEEDS TO BE BROUGHT IN WE'LL
BRING THEM IN AND HAVE
DISCUSSION UNTIL HOW TO SCORE
THE PACKET.
THAT'S THE SECOND TABLE IN THE
HANDOUT.
WHAT HOST FACULTY DO IS TRY TO
GET AS MANY PUBLICATIONS AS THEY
CAN BUT WILL ALSO USE QUALITY
STEPS THAT WE'RE RECOMMENDING OR
COME UP WITH ON THEIR OWN TO
BOOST THEIR SCORE.
WE MAKE IT CLEAR IN JOB
INTERVIEWS AND MAKE IT CLEAR AND
ASSIGN MENTORS TO NEW FACULTY
COMING INTO OUR DEPARTMENT.
AND IT'S CLEAR IN THE RANK AND
STATUS PACKET GUIDANCE.
AGAIN, FLEXIBLE.
SO HOW ARE WE DOING?
2016 ON THE X AXIS AND THE Y IS
ARBITRARY SCALE WE USE AND 2016
IS PRE USING THE SYSTEM SO WE
HAVE THE LOW LINE IS QUALITY AND
GREEN LINE IS QUANTITY.
YOU CAN SEE THE FIRST YEAR AFTER
WE IMPLEMENTED THIS WE GOT A
BOOST.
THEN IT GRADUALLY GOES DOWN.
IT'S POSSIBLE PEOPLE LOST
INTERESTED.
IT'S TOO HARD AND COMPLICATED
AND THE SOFTWARE AND
APPLICATIONS SO EVERYBODY GETS
TWO SCORES.
ONE FOR QUALITY AND ONE FOR
QUANTITY AND THEY'RE AVERAGED.
WE THINK THE DECLINE IN SCORES
MIGHT BE BECAUSE WE WERE PRETTY
GENEROUS IN 2017 AND WE
GRADUALLY GOT A LITTLE BIT
PICKIER AND PICKIER.
I'M OPEN TO IDEAS.
I WANT TO HEAR YOUR IDEAS.
I THINK WE WANT TO UPDATE WHAT
WE CONSIDER TO BE RIGOR AND
INCLUDE OPEN SCIENCE BECAUSE IT
SEEMS TO BE FOUNDATIONAL AND
MORE ABOUT RIGOR AND I THINK WE
NEED TO DO A LITTLE MORE OF THE
SOCIAL REWARDS AND YOU DID A
GREAT JOB.
AND THE FACULTY DEFINITELY TALK
ABOUT THIS WITH THEIR GRADUATE
STUDENTS.
STUDENTS GET MULTIPLE COURSES.
STATUS, RESEARCH METHODS FROM
THEIR MENTORS IN A PROFESSIONAL
SEMINAR AND ETHICS CLASS MOST
FACULTY ARE MODELLING HOW TO DO
MORE RIGOROUS SCIENCE.
THE FACULTY TYPICALLY PROVIDE A
MIX OF EVIDENCE, NUMBER OF STEPS
OF QUALITY.
NOBODY IS DOING ONE OR THE
OTHER.
A FEW FACULTY, MAYBE 20% ARE
STILL RESISTANT.
THE IDEA IS IT'S A PHASE, IT
WILL PASS.
NEW HIRES SEEM TO THRIVE ON IT
AND BE EXCITED ABOUT THE NEW
STANDARDS.
I'D LOVE TO HEAR YOUR IDEAS.
THANK YOU.
>> THANK YOU FOR HAVING ME HERE
TODAY.
IT'S BEEN A PLEASURE TO SPEAK
WITH YOU ALL THROUGHOUT THE
CONFERENCE AND LOOKING FORWARD
TO ANOTHER GREAT DAY.
I'M A MEDICAL LIBRARIAN AND RUN
THE INFORMATICS CENTER THE
UNIVERSITY OF NEW MEXICO.
I WAS REALLY HAPPY TO HEAR
LIBRARIES MENTIONED YESTERDAY
PARTICULARLY AS A PARTNER FOR
THE CARPENTRIES BECAUSE LIBRARY
ESSENTIAL BUT UNDER TAPPED
RESOURCE THAT IS REALLY ABLE TO
HELP OUR ACADEMIC INSTITUTIONS
MOVE FORWARD TO PROMOTE RIGOR,
TRANSPARENCY AND
REPRODUCIBILITY.
IN MY BRIEF TIME WITH YOU TODAY,
I'M GOING TO TALK ABOUT SOME
WAYS I WORKED ON THE ISSUE FROM
THE LIBRARY PERSPECTIVE AND A
FEW OF THE LESSONS I LEARNED IN
THE PROCESS.
IN 2016 I BEGAN COLLABORATING
WITH THE VICE PRESIDENT FOR
RESEARCH AT THE UNIVERSITY OF
UTAH WHERE I WAS AT THE TIME.
WE WERE BOTH REALLY INTERESTED
IN RESEARCH REPRODUCIBILITY IN
ALL DIMENSIONS OF THAT WORD AND
IN HIS ROLE HE WAS ESPECIALLY
INTERESTED IN HOW TO CHANGE THE
INSTITUTION TO VALUE RIGOR AND
REPRODUCIBILITY MORE.
LIBRARIES TOUCH ALL THE
DISCIPLINES IN OUR UNIVERSITY
AND AS SUCH I WAS INTERESTED IN
HOW TO DO THIS ON A LARGE
MULTI-DISCIPLINARY SCALE.
WE CAME UP WITH A TWO-DAY
CONFERENCE OPEN TO TENDE ATTEN
NATIONALLY TO PRODUCE
REPRODUCIBILITY.
THIS CONFERENCE WAS HELD IN 2016
AND FEATURES PANELS, PLENARY
SPEAKERS, A SUBMITTED POSTER
SESSION AND WORK SHOPS TO TEACH
PEOPLE HOW TO USE THE OPEN
SCIENCE FRAMEWORK.
THE CONFERENCE WAS A HUGE
SUCCESS.
PEOPLE PARTICIPATED AND THE
REVIEWS WERE GLOWING BUT PART OF
WHAT WE WANTED TO LEARN FROM
THIS CONFERENCE WAS HOW TO KEEP
THE MOMENTUM GOING.
WE ASKED PARTICIPANTS FOR FE
FEEDBACK AND GOT A FOUR-PRONGED
APPROACH.
WE CAME UP WITH A CONFERENCE IN
2018 AND THIS WE WERE GOING TO
THEME MORE AROUND INDIVIDUAL
CHANGES THAT COULD BE MADE WHICH
WAS A REQUEST OF A PARTICIPANT.
WE ALSO FORMED A RESEARCH
REPRODUCIBILITY COALITION AND
MET QUARTERLY TO CONTINUE
DISCUSSIONS HOW TO TAKE THE
LESSONS FROM THIS CONFERENCE AND
IMPLEMENT THEM IN PRACTICE TO
CHANGE THE CULTURE OF THE
INSTITUTION.
PROBABLY MY FAVORITE THING THAT
WE HOSTED A WEEKLY GRAND ROUNDS
RESEARCH REPRODUCIBILITY SEMINAR
WHERE WE INVITED SPEAKERS FROM
ALL OVER OUR CAMPUS TO TALK
ABOUT THE REPRODUCIBILITY ISSUES
IN OUR OWN DISCIPLINES WHETHER
IT WAS BUSINESS OR MANAGEMENT,
PRE-CLINICAL RESEARCH, SOFTWARE
ENGINEERING.
ALL THE DISCIPLINES WE COULD
FIND.
THE IT WAS DESIGNED TO BUILD A
COMMUNITY AND BUILD MOMENTUM FOR
THE 2018 CONFERENCE.
WE ALSO PARTNERED WITH OUR
DEPARTMENT OF PHILOSOPHY TO
CREATE A CREDIT COURSE
ENCOMPASSING THE CONFERENCE AND
PREPARING LEARNERS TO ATTEND THE
CONFERENCE AND DEVELOP THEIR OWN
SKILL SETS AT THE SAME TIME.
MOST IMPORTANTLY THOUGH THE
CONFERENCE SET UP THE HEALTH
SCIENCES LIBRARY AS THE GO-TO
EXPERTS ON CAMPUS FOR
REPRODUCIBILITY AND LED TO
OPPORTUNITIES TO PRESENT AND
DIRECTLY TO RESEARCHERS AND TO
RESEARCH LEADERS ACROSS CAMPUS.
I LEFT AFTER THE 2018 CONFERENCE
THE DAY AFTER AND LITERALLY AND
MOVED TO THE UNIVERSITY OF
FLORIDA WHERE I BEGAN MY EFFORTS
AGAIN.
THERE, ONE OF MY POST IMPORTANT
EFFORTS WAS TO HIRE A
REPRODUCIBILITY LIBRARIAN YOU
MAY HAVE MET YESTERDAY.
AND DEVOTE TIME AND ENERGY TO
TEACHING AND EDUCATING ABOUT
REPRODUCIBILITY TOPICS ACROSS
DISCIPLINES AND TO LEAD CHANGE.
I WAS ABLE TO TEACH
RE MODELS AND
DEVELOP A REPRODUCIBILITY FOR A
T32 AND HOST THE THIRD ITERATION
OF THE REPRODUCIBILITY
CONFERENCE IN 2020.
THEN I MOVED TO MY CURRENT
INSTITUTION AND MOVED TO NEW
MEXICO WHERE WE'RE IN THE
BEGINNING STAGES AND WE PROVIDER
CARPENTRIES WORKSHOPS AS WELL AS
TO PROVIDE DATA MANAGEMENT WORK
SHOPS, WORK SHOPS ON OTHER TYPES
OF COURSES.
WE ALSO DO A LOT OF GUEST
LECTURES FOR VARIOUS COURSES ON
TOPICS RELATED TO RIGOR AND
REPRODUCIBILITY AND CREATE
CREDIT-BASED COURSE.
WE WERE INVITED TO CREATE
TRAINING FOR POST-DOCS AND GRAD
STUDENTS ON BETTER LITERATURE
SEARCHING SKILLS WHICH IS ONE OF
OUR CORE SERVICES.
THIS WAS TO HELP THEM TO BE ABLE
TO BETTER NOT ONLY FIND BUT
ASSESS THE RESEARCH TO ADDRESS
RIGOR RESEARCH IN THEIR FUTURE
GRANTS.
WITHIN THE HEALTH SCIENCES
LIBRARY AND INFORMATICS CENTER
WE HAVE NEW TENURE AND PROMOTION
GUIDELINES FOR FACULTY THAT
EMPHASIZE RIGOR, TRANSPARENCY,
OPEN SCIENCE AND
REPRODUCIBILITY.
FROM ALL THESE THINGS A FEW
LESSONS I'VE LEARNED.
THIS CAME UP A LITTLE BIT
YESTERDAY BUT IF INSTITUTIONS DO
VALUE THIS KIND OF WORK, WE NEED
TO FUND IT.
WE NEED TO GET RELEASE TIME AND
REWARDS AND RECOGNITION FOR THE
INDIVIDUALS LEADING THESE
EFFORTS EVEN AND PERHAPS
ESPECIALLY THE GRASSROOTS
EFFORTS HAPPENING ACROSS OUR
CAMPUSES.
AT UTAH, I ACTUALLY HAD FIVE
COLLEAGUES IN THE LIBRARY WHOSE
WORK WAS ESSENTIAL TO
MAINTAINING THE MOMENTUM OF
WHERE WE HAD GONE AFTER OUR
FIRST CONFERENCE.
AFTER I LEFT, THREE OF THEM ALSO
LEFT AND THE REPRODUCIBILITY AND
RIGOR WORK AT UTAH IS NOW GONE.
COLLABORATION IS ESSENTIAL.
I WOULD LIKE TO THINK OF
LIBRARIES AS ONE OF THOSE NODES
THAT HAS WIDE BRIDGES DAMON
MENTIONED IN HIS TALK YESTERDAY.
WE ARE DISCIPLINE NEUTRAL AND WE
ARE OF COURSE THE CENTRAL UNIT
BUT POWER DYNAMICS BEING WHAT
THEY ARE TO MOVE FORWARD WE
NEEDED TO USE THOSE BRIDGES.
I ALSO THINK THAT WE NEED A
COMBINATION OF TOP-DOWN AND
BOTTOM-UP APPROACH AND WE WOULD
NOT HAVE GOTTEN THE WIDE SUPPORT
WE DID.
HAVING LEADERS THAT ACTIVELY
PROMOTED, ENCOURAGED AND
SUPPORTED WHAT WE DID WAS REALLY
ESSENTIAL TO THE SUCCESS.
ALSO WE NEED TO BE
MULTI-DISCIPLINARY IN SOME OF
THESE EFFORTS.
I THINK THAT'S A CHALLENGE
BECAUSE EVERYONE TENDS TO BE
SILOED BUT IT'S AN AMAZING WHEN
YOU BRING TOGETHER PEOPLE FROM
DIFFERENT FIELD AND DISCIPLINES
TO TALK WITH ONE ANOTHER BECAUSE
THEY START TO SEE THE
COMMONALITIES AND CAN SHARE AND
LEARN AND GROW.
AND THE WORK WE DO ALIGNS WITH
RESEARCH RIGOR TRANSPARENCY AND
REPRODUCIBILITY TOPICS ON
MOTIVATIONS ARE SIMILAR TO THIS
WORK AND THIS IS AN AREA MORE
LIBRARIES ARE ACTIVELY WORKING
IN SO THANK YOU VERY MUCH.
>> SO LET ME FIRST THANK
VERONIQUE AND STEVEN AND CHI AND
I THINK CHI HAS THE ORIGINAL
CHAMPION OF RIGOR FOR MAYBE 10
YEARS.
I DON'T EVEN KNOW.
I STILL WONDER WHETHER HIS
IMPACT ON OUR COMMUNITY IS PROOF
OR WHAT WE HEARD OF THE NETWORK
THEORY AND IT DEPENDS ON HOW YOU
DEFINE PERIPHERY IN THE END.
WHAT I WOULD LIKE TO PRESENT IS
A SYSTEMIC INTERVENTION AND
CHANGE INK -- AND WE HAD $1.2
BILLION BUDGET INCLUDING THE
HOSPITAL, I HAVE TO SAY.
WE STARTED OUR ACTIVITIES WITH A
FRAMEWORK.
YOU CAN ALWAYS CALL THE ACTION
WHICH STARTS WITH WHAT WE WANT
TO ACHIEVE, TRUSTWORTHINESS AND
ETHICAL ARE THE PRINCIPLES.
WE HAVE DERIVED PROCESSES WHICH
UNDER PIN IN OUR CONTEXT THE
PRINCIPLES AND FROM THOSE WE
HAVE DERIVED SERVICES AND
ACTIVITIES WHICH ARE WHAT WE ARE
DOING ON A DAILY BASIS AND THIS
WE ACCOMPANY WITH META RESEARCH
FOR IMPLEMENTATION AND THE RIGHT
THING.
YOU CAN PUT OUR ACTIVITIES IN
VARIOUS BUCKETS.
THIS IS A BRIEF OVERVIEW AND
I'LL GIVE EXAMPLES IN A SECOND.
WE'RE TRYING TO HELP WITH
QUALITY ASSURANCE, OPEN SCIENCE
AND INCENTIVES AND DO
STAKEHOLDER ENGAGEMENT AND THEN
WE HAVE METARESEARCH AND ACT AS
A THINK TANK AND THERE'S A CORE
GROUP OF PEOPLE WHO DO
ADMINISTRATION AND WE ALSO HAVE
GROUPS AND TRACEY PRESENTED
YESTERDAY AND MOST ARE TOPICAL.
THEY HAVE TOPICS RELATED TO OUR
ACTIVITIES AND FUNDED
EXTERNALLY.
TO GIVE YOU AN IDEA WHAT WE ARE
DOING AND THIS IS A MIXTURE
BECAUSE WE HAVE AN ECLECTIC
ACTIVITY AND ARE CHOSEN ALMOST
AT RANDOM.
WE ARE HELPING OUR RESEARCHERS
IN THE BASIC REALM TO WORK WITH
ELECTRONIC LABORATORIES AND NOTE
BOOKS MEANING WE SHOW THEM HOW
TO USE THEM AND PAY FOR THE
LICENSES AND FIX PROBLEMS.
OBVIOUSLY E DO A
EDUCATION.
TRACY HAS TALKED ABOUT IT
YESTERDAY.
THERE'S OTHER ACTIVITIES.
THE USUAL STUFF YOU COULD ARGUE
LIKE THE COURSES IN ADVANCED
PRODUCIBLE RESEARCH WITH
STATISTICS AND WE HAVE
PRE-CLINICALAL ANALYSIS AND
SYSTEMATIC REVIEWS.
THESE ARE AN IMPORTANT
COMPONENTS.
WE DEVELOPED TOOLS AND ALL OF
THEM BY THE WAY ARE OPEN AND WE
PUBLISHED THEM AND HELP OTHERS
TO USE THEM OUTSIDE OUR
ENVIRONMENT.
THIS IS AN EXAMPLE OF A TOOLBOX.
THIS HELPS OUR RESEARCHERS TO
BUILD THE PROBLEM OF HAVING THE
RESULTS AND ASKING THE QUESTION
WHERE CAN I PUBLISH THEM, WHAT
ARE THE POSSIBILITIES.
ANOTHER POTENTIALLY INTERESTING
ACTIVITY FOR THE GROUP IS A
DASHBOARD WHICH WE HAVE CREATED
FOR OUR INSTITUTION.
AND IT'S ONLINE SO YOU CAN CHECK
IT OUT AND YOU CAN FIND OUT HOW
THE ORGANIZATION IS DOING WITH
RESPECT TO OPEN AND RESPONSIBLE
RESEARCH PRACTICES.
AND THIS IS UPDATED SOME OF IT
ON A WEEKLY BASIS AND IT'S ABOUT
FOR EXAMPLE OPEN SCIENCE.
HOW MANY OF OUR PUBLICATIONS ARE
OPEN AND THIS IS CUT OFF -- I
DON'T KNOW.
HOW WE'RE DOING THE PREPRINTS
AND THEN VERY APPEAR FOR US TO
DO CLINICAL TRIALS HOW MUCH OF
THEM HAVE PUBLISHES SUMMARY
RESULTS AND SO FORTH AND
INTERESTING HOW WE'RE FARING
WITH RESPECT TO YEARS AGO.
I FORGOT TO MENTION IT'S BEEN
AROUND FOUR OR FIVE YEARS NOW.
WE'RE ALSO DOING A BENCHMARKING
NOT ONLY IN OUR INSTITUTION WE
ALSO BENCHMARK OTHERS.
SO SET IN RELATION TO THEM AND
IT'S USUALLY NOT AT THE TOP.
WE'RE VERY MUCH IN THE MIDDLE
VERY OFTEN SO THESE ARE ALL THE
GERMAN UNIVERSITY MEDICAL
CENTERS AND WHAT THIS SHOWS IS
PUBLICATION AND I GUESS THIS IS
PUBLICATION OF ANY RESULT WITHIN
I THINK ANY FORMAT COULD BE A
WEBSITE OR PUBLICATION IN A
REGISTRY WITHIN I THINK WHAT IS
IT 24 MONTHS.
THAT'S AN OLD W.H.O. STANDARD
AND WE WANT TO HELP RESEARCHERS
DO BEST.
IN THIS CONTEXT WE HAVE COURSES
AND ALERT THEM IF THEY'RE
DELAYED.
WE HAVE AN INTERVENTION.
SOME GET A REPORT CARD AND SOME
DON'T GET ANYTHING AND SOME GET
SOME HELP AND THE QUESTION IS
WHAT HAPPENS MONTHS AND YEARS
LATER, WHICH INTERVENTION IS
BETTER.
IN THE MEDICAL CONTEXT IF WE
COMPARE OURSELVES TO BRITAIN
WE'RE VERY BAD AT PATIENT AND
STAKEHOLDER ENGAGEMENT.
SO THIS IS SOMETHING WE'RE KNOW
PICKING UP WITH RESEARCH AND
FUNDING RESEARCH GROUPS TO DO
BEFORE THEY START A STUDY TO
ENGAGE STAKEHOLDERS.
WE DO A LOT OF IN TERMS OF
INCENTIVES.
FOR EXAMPLE, WE GIVE OUT WE HAVE
ALREADY HELPED TO CHANGE OUR
UNIVERSITY OF THE PREVIOUS
METRICS BASED ON THE IMPACT
FACTOR AND PUBLICATION WE HAVE
NOW ADDED THE NIR/NIH
CITATION-BASED METRIC AND MORE
IMPORTANTLY WE GIVE AN OPEN DATA
ORIENTED FUNDING FOR EVERY PAPER
THAT HAS IT'S DATA FAIR AND OPEN
AUTHORS GET A CERTAIN MONETARY
REWARD FOR THEIR RESEARCH.
WE HAVE A LOT OF AWARDS WHICH
OUR RESEARCHERS AND CLINICIANS
CAN APPLY FOR, FOR EXAMPLE, IF
THEY HAVE INVOLVED PATIENTS AND
STAKEHOLDERS.
IF THEY HAVE RE-USED DATA OR
HAVE IT IN THE PRE CLINICAL
REALM OR HAVE A PUBLISHED RESULT
OR EDUCATION AND THIS IS NOT
ABOUT THE MONEY BUT HIGHLIGHTING
THE EFFORTS.
MOST IMPORTANTLY VEEP WE HAVE
CHANGED OUR PROFESSORSHIP
APPLICATION SERVICE AND THAT IS
SOMETHING THAT IS VERY MUCH
HELPED OUR LEADERSHIP ESPECIALLY
IT'S VERY MUCH IN FAVOR OF WHAT
WE'RE DOING AND WE HAVE CHANGED
THIS TO NOW INCLUDE NARRATIVES.
THIS IS ONE OF THE NARRATIVES
THAT IS ABOUT OPEN SCIENCE.
WE'RE ASKING WHAT HAVE YOU DONE
AND WHAT WILL YOU BE DOING WHEN
YOU ARE POTENTIALLY WORKING WITH
US.
WE ARE HELPING OUR COMMISSIONS
TO WORK WITH THIS NEW
INFORMATION.
THIS IS VERY TRICKY BECAUSE IT'S
CONFUSING TO THEM.
IT'S SORTING SPREAD SHEETS WITH
JOURNAL NAMES.
WE HAVE DEVELOPED SOFTWARE WHICH
HELPS THE COMMITTEE WE WORK WITH
AND THIS IS OPEN AND AVAILABLE
TO OTHER INSTITUTIONS.
THIS WAS REALLY JUST A
SELECTION.
WE'RE VERY GRATEFUL BECAUSE WE
GET A LOT OF SUPPORT FROM
EXTERNAL AGENCIES FOR THIS WORK
OTHERWISE THIS WOULD NOT BE
POSSIBLE AND OF COURSE WE HAVE A
LOT OF INTERACTIONS WITH OTHER
GROUPS THAT WORK IN THIS JOINT
PROJECTS.
SOME OF THEM FUNDERS AND SOME OF
THEM WE'RE JUST LEARNING FROM
THEM.
SO TO FINISH UP, I WOULD JUST
LIKE TO MENTION MY TWO BARRIERS
I SEE MOST PROBLEMATIC AND ONE
IS THE BRAINS OF SCIENTISTS AND
LEADERSHIP AND ADMINISTRATORS IN
ACADEMIA ARE HARD WIRED WITH THE
REPUTATION AND FUNDING.
IF I CAN PICK ONE THING I COULD
CHANGE AND NOW THAT WE'RE
SUCCESSFUL I THINK AS A SECOND
PROBLEM THAT'S COMING UP AND I
THINK IT'S MORE LICENSING AND
THAT'S THE KIND OF NARRATIVE
SINCE WE HAVE QUESTIONS WE'RE
DOING THE RIGHT THING AND CAN DO
DO OTHER THINGS.
THANK YOU VERY MUCH.
>> THANK YOU FOR MAKING MY
MODERATE BEING EASY.
WE HAVE A FEW MINUTES FOR
QUESTIONS AND THEN WE'LL TAKE A
BREAK AND COME BACK FOR THE
PLENARY.
I THINK WE CAN TAKE A COUPLE
QUESTIONS NOW.
WE CAN GET THE MICS IN THE
AISLES ON PLEASE.
MIKE DOHERTY, UNIVERSITY OF
MARYLAND, DEPARTMENT OF
PSYCHOLOGY.
SO YOUR COMMENT EARLIER ON ABOUT
REVOLUTION VERSUS EVOLUTION AND
I JUST WANT TO EMPHASIZE I THINK
REVOLUTION IS MAYBE THE ONLY
APPROACH THAT WILL WORK AND I'LL
GIVE YOU THIS FROM A POINT OF
EXPERIENCE WHERE WHEN I STARTED
CHANGING THE CRITERIA FOR
PROMOTION AND TENURE AND MERIT
REVIEW IN OUR DEPARTMENT I
STARTED WITH EVOLUTION AND WHAT
I QUICKLY FOUND WAS PEOPLE WERE
FALLING BACK INTO HISTORICAL
METRICS OF SUCCESS, HISTORICAL
METRICS OF IMPACT AND QUICKLY
FOUND OUT AND REALIZED THAT WAS
NOT GOING TO GO ANYMORE AND WE
THREW OUT THE OLD APPROACHES AND
STARTED FRESH AND REQUIRED AN
OVERALL NOT A REVISION.
THEN TO REBECCA'S POINTS
COMMUNICATION WAS KEY.
IT WAS COMMUNICATING WITH
FACULTY OVER AND OVER AGAIN.
IT WAS A THREE-YEAR PROCESS TO
GET THEM ON BOARD COMMUNICATING
WITH THE DEAN'S OFFICE OF
FACULTY AFFAIRS TO GET THEIR
BUY-IN OR SO THEY CAN SAY YES
YOU CAN DO THIS AND GOING BACK
WITH THE FACULTY AND SAYING YES,
WE CAN DO THIS OVER AND NEVER
UNTIL THEY THOUGHT, WE CAN DO
THIS AND THEN THEY BOUGHT IN.
SO EXACTLY COMMUNICATION,
COMMUNICATION IS THE MOST
IMPORTANT THING AND OVERHAULING
I THINK IS PROBABLY THE ONLY
THING THAT WILL WORK.
>> WE LOOKED AT HOW PEOPLE
DEFINE QUALITY, PRESTIGE AND
IMPACT AND THEY DO DEFINE ALL
THESE THINGS IN CIRCULAR WAYS
AND USE WORDS TO DESCRIBE THE
OTHERS AND THEN THIS IDEA SOME
PEOPLE WERE STILL CHOOSING TO GO
TO THOSE AND TIMING TWO TRACTS
AND WONDER IF YOU CAN TALK ABOUT
HOW IT'S PLAYED OUT HOW PEOPLE
ARE CHOOSING ONE OVER THE OTHER.
IS THERE STILL A SENSE IN THE
DEPARTMENT THAT THE REAL THING
IS TO STILL GO FOR THE NUMBER OF
PUBLICATIONS AND NOW WE'RE ALSO
LEARNING SOMETHING THAT CAN HELP
US.
THAT'S WHAT YOU DESCRIBED BUT IS
THERE A REAL SENSE OF WHEN YOU
TALK ABOUT OKAY WE BETTER STILL
FOCUS ON A NUMBER OF POSITIVES
AND NOT GIVE THAT UP?
>> SO TECHNICALLY YOUR QUALITY
SCORES ONLY APPLY TO YOUR
CURRENT PUBLICATIONS.
SO FOR EACH PUBLICATION, YOU
HAVE TO HAVE YOUR QUALITY
OTHERWISE YOU GET NO POINTS.
THEY STILL HAVE TO HAVE A NUMBER
OF PUBS.
BECAUSE WE MAY HAVE LOST SOME
INTEREST OVER TIME I'M LEARNING
TOWARDS MAYBE WE NEED A
REVOLUTION AND NOT AN EVOLUTION.
BUT I KIND OF LIKE TO THINK WE
DIDN'T WASTE TIME TAKING THIS
FIRST STEP.
IT WAS A GOOD FIRST STEP AND
THEN WE NEED TO GO BACK TO THE
FACULTY AND SAY WE'RE SLIDING.
WE NEED TO REALLY PUSH THIS A
LOT HARDER.
I THINK WE'LL GET SUPPORT FROM
THE COLLEGE AND UNIVERSITY
THOUGH.
THAT'S THE GOOD THING.
>> JESSICA POLE BEINGA AND HOW
DOES THE SYSTEM PLAY INTO
RECRUITMENT?
SO FOR EXAMPLE YOU JUST
MENTIONED HAVING A REVOLUTION.
IS THERE A POINT AT WHICH YOU
WOULD SAY WE'RE NOW ONLY GO TO
RECRUIT A COHORT FROM THIS POINT
ON?
PEOPLE RECRUITED WILL BE
EVALUATED IN THIS NEW FASHION.
DO YOU ENVISION THAT AS AN
APPROACH AND SECOND HAS IT
PLAYED INTO VALUES OF PEOPLE
COMING INTO THE DEPARTMENT AS
NEW HIRES.
HEY, DO YOU SEE THIS AS A
SELLING POINT OF PEOPLE.
>> AND IF THEY'RE NOT GETTING
REINFORCED AT OTHER UNIVERSITIES
FOR DOING IT THE OLD WAY THEN IT
SEEMS UNFAIR RIGHT NOW TO SWITCH
IT.
>> WE'LL BE EVALUATING FOR
TENURE IN THIS FASHION.
>> YES, WE DO SAY THAT.
>> DO YOU THINK THERE'S A
MEASURABLE DIFFERENCE IN THE
APPLICANTS COMING IN NOW WHEN
YOU'RE PUTTING THEM ON THE
PROGRAM AND THE QUESTIONS?
>> YES, WE GET APPLICANTS THAT
SEEM TO BE MORE FOCUSSED ON IT.
OF COURSE, YOU KNOW, WHEN YOU'RE
APPLYING FOR A JOB, IF YOU
REALLY WANT IT YOU'LL TAILOR
WHAT YOU SAID TOWARDS THAT.
BUT WE DO SEEM TO BE ATTRACTING
PEOPLE THAT ARE EXCITED TO DO
THIS.
THEY SEEM GENERALLY EXCITED AND
COME IN AND START DOING IT.
THE NERVOUS PEOPLE ARE THOSE
THREE OR FOUR YEARS OUT FROM
TENURE AND PEOPLE WHO THINK IT'S
JUST A PHASE.
AND I THINK WE NEED TO DOUBLE
DOWN AND CONVINCE THEM THIS IS
NOT A PHASE.
THIS IS WHAT WE'RE DOING.
>> SO WE DOCUMENT ALL THE
DISCUSSIONS THAT ARE GOING ON
AND IT'S VERY MIXED HOW MUCH
UPTAKE THIS GETS BECAUSE YOU CAN
IGNORE IT AND OUR GEP OFFICER
WILL SORT OF START AND HAVE YOU
LOOKED AT THIS AND THAT.
SO FAR IT'S A MIXED EXPERIENCE.
I THINK THE MOST IMPORTANT
EFFECT WE ARE HAVING I THINK IS
THAT CANDIDATES ARE STARTING TO
THINK ABOUT THOSE THINGS AND
IT'S ALSO INTERESTING WE'RE
ELEVATING WHAT THEY'RE WRITING
IN THOSE THINGS AND OFTEN SAY
DOES NOT APPLY.
SO WE ASKED THEM ABOUT THEIR
PRACTICES AND THEY SAY, DOES NOT
APPLY.
I THINK WE'RE IN A STAGE OF
EVALUATING WHETHER HOW WE'RE
GOING TO DO THIS.
I DON'T THINK WE CAN PROVE WE
HAVE ANY EFFECT ON THE SELECTION
WHETHER WE HAVE A DIFFERENT
SELECTION AND A DOUBT IT.
>> IS THERE AN EXAMPLE OF A
UNIVERSITY THAT'S DONE THIS
SUCCESSFULLY FROM TOP TO BOTTOM,
ALL DEPARTMENTS AND HAVE THEY
SHOWN ANY CONNECTION WITH
OUTPUT?
DO YOU HAVE ANY EXAMPLES?
HAVE YOU TRIED TO DO IT?
>> THE NETHERLANDS HAVE CHANGED
NOT ONLY THE SYSTEM IN ONE
UNIVERSITY, THEY HAVE CHANGED IT
IN THE WHOLE COUNTRY AND HAVE A
LOT OF BACKLASH FROM THEIR OWN
SCIENTISTS.
IT'S AN INTERESTING SPACE TO
WATCH.
FOR EXAMPLE THE UNIVERSITY OF
ULTRICHT HAVE DONE IT A FEW
YEARS NOW AND I DON'T KNOW THEY
CAN THEY CHANGED SELECTION BUT
THIS IS WHAT TO LOOK FOR.
>> THIS GOES TO 10:40?
>> YEP.
>> GREAT, THANKS.
>> UNIVERSITY OF BRITISH
COLUMBIA IS ONE TO LOOK AT.
THEY'VE BEEN WORKING ON IT FOR
YEARS AND TO KNOW THEY'RE NOT
OUT TO ANY OUTPUT YET AND STILL
WORKING ON THE TRANSFORMATION
AND WORKING HARD ON SHIFTING
THEIR TENURE AND HIRING POLICIES
AND I THINK THEY'RE GETTING
RESISTANCE BUT THAT'S ONE TO
KEEP AN EYE ON.
ONE THEY'RE LOOKING AT IN TERMS
OF BEING AN EXEMPLAR.
>> DAVID CENTOLA TALKED ABOUT
GOING AROUND THE PERIPHERY BUT
IN THIS AREA IT SEEMED YOU NEED
INSTITUTIONAL LEADERS.
YOU CANNOT CHANGE HIRING
CRITERIA WITHOUT SUPPORT FROM
THE TOP.
AND MY QUESTION IS IS I DON'T
KNOW IF YOU KNOW WHAT HAPPENED
OR HOW THEY DID IT AT BRITISH
COLUMBIA BUT WHAT ARE THE
STRATE
STRATEGIES AND THERE WAS A BIG
LEADER THERE.
WHAT WAS THE STRATEGY TO GET
LEADERSHIP FROM THE TOP AND IF
YOU HAVE BACKLASH FROM THEIR OWN
FACULTY THEY'RE LIMITED IN THEIR
POWER.
>> AN INSIDE JOB.
>> YOU'RE RIGHT IN THAT IT'S
TOP-DOWN.
HELIOS IS AN INTERESTING THING
WITH OPEN RESEARCH FUNDERS GROUP
LEADING THE WAY.
THAT ONE IS CLEVER BECAUSE IT'S
STILL AT THE TOP BUT STILL USING
PEER PRESSURE WITHIN THE SYSTEM.
IT'S ANOTHER APPROACH DIFFERENT
THE ISOLATED CASES.
IT WILL BE INTERESTING TO SEE
WHERE IT GOES BUT ANOTHER ONE
I'M AWARE OF.
>> AT THE UNIVERSITY OF FLORIDA
THEY DIDN'T CHANGE THE PROMOTION
AND TENURE STANDARDS TO INCLUDE
RIGOR SPECIFICALLY BUT DID
CHANGE THEM TO INCLUDE CREDIT
FOR TEAM SCIENCE WHICH IS A STEP
IN THE RIGHT DIRECTION.
THEY DID THAT WITH SUPPORT FROM
THE HEALTH SCIENCES CENTER, THE
CTSA HUB AND ESSENTIALLY BY
CONTINUING TO WORK WITH THE
FACULTY SENATE WHO THEN ALLOWED
IT.
WHETHER OR NOT THE CHANGE TO THE
INSTITUTIONAL TENURE AND
PROMOTION GUIDELINES WAS
ACTUALLY EFFECTIVE I DON'T THINK
THEY'VE STUDIED THAT YET AND IT
WILL BE INTERESTING AND WOULD BE
REALLY INTERESTED TO STUDY
WHETHER OR NOT FOR EXAMPLE THE
STATISTICIANS WHO DO WORK ON
CLINICAL TRIALS ARE ACTUALLY IN
FACT GETTING EQUAL CREDIT.
FROM WHAT I HEARD THEY'RE NOT
BUT THESE THINGS I THINK DO TAKE
TIME TO CHANGE INCREMENTALLY.
>> THERE'S A DIFFERENCE BETWEEN
WHAT YOU PUT IN THE GUIDELINES
AND WHAT HAPPENS IN PRACTICE.
THERE'S LOTS OF INSTITUTIONS
THAT MADE CHANGES TO THEIR
GUIDELINES.
WE FOUND EVEN IN THE THINGS WE
STUDIED FOR OPEN ACCESS INTO
THEIR GUIDELINES AND YOU HEAR
ABOUT ALTERNATIVES AND MEASURING
THINGS IN DIFFERENT WAYS AND
USING NARRATIVES AND HOW MUCH
DOES IT CHANGE THE CULTURE AND
THE FACULTY ARE NOT NECESSARY
FEELING THEY'LL BE EVALUATED BY
THEIR BROADER COMMUNITY THAT
SPANS OTHER INSTITUTIONS.
AND THE RESISTANCE I'VE HEARD
FROM UNIVERSITY ADMINISTRATION
IS OFTEN STILL VERY HOOKED AND
I'M WILLING TO ADD THEY'RE
INTERESTED IN LISTENING TO
ALTERNATIVE APPROACHES AND
MEASURES BUT HOW WILL THIS
AFFECT ME IN THE RANKING AND
RECRUITMENT OF STUDENTS.
SPECIAL INSTITUTIONS PRESSURED
FOR THE ENROLLMENT TO MAINTAIN
BUDGET.
WHEN YOU COME SPEAK CLOSELY INTO
THE MIC AND LET'S GET A COUPLE
DIFFERENT COMMENTS AND KEEP THEM
BRIEF SO WE GET A CHANCE TO HEAR
DIFFERENT PERSPECTIVES.
>> I'M VERY GRAD TO HEAR I'M NOT
THE ONLY ONE CALLING FOR THE
REVOLUTION.
IN SUPPORT OF YOU, WE CANNOT
MAKE THE SYSTEM ANY WORSE BUT
TRYING SIMPLE FORMS.
QUESTION TO MELISSA YOU SAID
LIBRARY CUTS ACROSS ALL
DISCIPLINES AND DISCIPLINE
AGNOSTIC.
I'D LIKE EXAMPLES ON HOW YOU
HELP VERY DIFFERENT DEPARTMENTS
TAKE UP SOME CITATIONS AND ALSO
HOW LIBRARY CAN PROMULGATE IT
AND BEFORE YOU ANSWER I WAS
THINKING OF ONE OF THE ADDITION
TO INCLUDE REPRODUCIBILITY AND
MOST STUDENTS DEVELOP THEIR
THESIS BEFORE THEY GRADUATE
ESPECIALLY Ph.D. AND MASTER'S.
IS THERE DIFFERENT WAYS IN WHICH
THEY'RE REQUIRING VARIOUS LEVELS
OF DOCUMENTATION AND RIGOR.
YOU HAVE MATERIALS TO DEPOSIT
BEFORE THEY GRADUATE.
THAT'S ANOTHER WAY AND A WAY TO
ACHIEVE OPEN ACCESS IN THE
THESIS MATERIAL AND SOFTWARE AND
DATA ASSOCIATED WITH THE THESIS.
>> THANKS FOR THAT QUESTION.
LIBRARIES DO CUT ACROSS
DISCIPLINES AND MOST ACADEMIC
LIBRARIES YOU'D BE USUALLY
THAT'S ACROSS INSTITUTIONS AND
ACROSS LIBRARIES IS THE WORK WE
DO AND EVERY DISCIPLINE BEING
DIFFERENT THAT'S WHY WE HAVE
PEOPLE WHO WORK WITH DIFFERENT
DISCIPLINES.
THAT SAID THERE ARE A LOT OF
THINGS LIBRARIES ARE DOING THAT
ARE REALLY LOOKING AT MULTIPLE
DIFFERENT KINDS OF DISCIPLINES
AND ONE EXAMPLE I CAN THINK I OF
AT THE UNIVERSITY OF MINNESOTA
[TECHNICAL DIFFICULTIES]
AND TEACHING SYSTEMATIC REVIEW
TECHNOLOGIES AND TEACHING PEOPLE
HOW TO ASSESS A PAPER, HOW TO DO
LITERATURE SEARCHING, HOW TO
CREATE A DATA MANAGEMENT PLAN, A
LOT OF THOSE KINDS OF THINGS
HAVE SOME FIELD-SPECIFIC AREAS
BUT REALLY THE CONCEPTS BEHIND
THEM ARE GENERAL.
SO WE'RE ABLE TO AFFECT AND TO
REACH A LOT OF DIFFERENT
DISCIPLINES ACROSS CAMPUS BY
HAVING THOSE MORE INTRODUCTORY
BASIC COURSES AND ONCE PEOPLE
LEARN SOME OF THOSE BASIC SKILLS
THEN THEY CAN GO ON AND TO DO
MORE EDUCATION AND LEARNING
WITHIN THEIR OWN FIELD.
>> I'LL TALK ABOUT REVOLUTION
AND EVOLUTION AND WHAT WE SHOULD
BE DOING IS THOUGHTFUL
REENGINEERING.
LET'S FIX WHAT'S BROKEN WITHOUT
BREAKING WHAT WORKS.
INTERVENTIONS CAN HAVE
UNANTICIPATED CONSEQUENCES AND
THAT CAN BE PART OF EITHER THE
FIELD OR REALITY PEOPLE PUSHBACK
AND WITHOUT LOOKING AT LEVELS
AND INSTITUTIONS, I WANTED TO
SUGGEST A COUPLE IDEAS AT ANY
GIVEN LEVEL PAYING ATTENTION TO
THE THINGS THAT ARE GOING WELL
AND ARE BEING DONE RIGHT AND
HIGHLIGHT THOSE AND MAKE SURE
AND ACKNOWLEDGE THE SUCCESSES
HAPPENING AND LOOK AT A
UNIVERSITY AND SAY EVERY
DEPARTMENT HERE HAS A TENURE
PROCESS.
WHICH DEPARTMENTS REALLY SHINE?
AND THAT PROVIDES AN OPPORTUNITY
FOR A PEER MODELLING AND FINDING
WHAT'S WORKING IS GOING TO BE AN
IMPORTANT PART OF LOOKING FOR
CHANGE.
>> I'M GLAD THAT COMMENT ABOUT
PUSHING BACK ON THE REVOLUTION
GENERATED COMMENTS.
I DON'T KNOW -- I FORGET THE
NAME OF THE PERSON WHO PROPOSED
THE EVOLUTION IS RIGHT THERE
NEXT TO YOU.
>> THIS IS A CONVERSATION AND
PUSHBACK ON THE IDEA AND SEEMS
TO BE RESONATING AND I WANTED
YOU TO COMMENT ON IT.
>> I DON'T MIND REVOLUTION.
I LIKE THE IDEA BUT PEOPLE GET
KILLED IN THEM AND YOU DON'T
ALWAYS WIN.
AND THE STARTING POINT THERE ARE
BITS OF PROGRESS THAT WE CAN
MAKE BY PERSUADING OTHERS.
WE BELIEVE AS RESEARCHERS THAT
WE CREATE DATA WHICH WE USING TO
PERSUADE PEOPLE TO CHANGE THE
WAY THE WORLD IS.
IF WE CANNOT SHIFT THINGS BY
PERSUASION MAYBE THERE'S
SOMETHING WRONG WITH THE
DIRECTION IN WHICH WE WISH TO
GO.
I'VE BEEN ON THE WRONG END OF
SELF-APPOINTED EXPERTS WHO KNOW
THE WORLD SHOULD BE WHO SEEK TO
IMPLEMENT IT AND DON'T THINK WE
SHOULD FALL IN THE SAME TRAP.
A GRADUALIST APPROACH IS MAYBE
BETTER BUT I MAY BE SURPRISING
PEOPLE WHO KNOW ME WITH MY
CONSERVATISM.
MAYBE I'M JUST GETTING OLD.
>> THANK YOU.
I WANT TO MAKE A QUICK COMMENT
ON HELIOS.
IT'S A COALITION OF WILLING
UNIVERSITIES TO EMBRACE OPEN
SCIENCE THAT'S BEEN SET UP BY
THE ROUNDTABLE FOR ALIGNING
INCENTIVES FOR OPEN SCIENCE WITH
THE NATIONAL ACADEMY.
I THINK INSTITUTIONS ARE GIVEN A
LIST OF THINGS THEY COULD DO BUT
IT ISN'T FORMULATED IN THE WAY
THAT TOPS FRAMEWORK IS SET UP
WHERE PEOPLE CAN GET ON A BUS
AND DECIDE WHERE THEY WANT TO GO
AND BENCHMARK AGAINST EACH
OTHER.
THAT ONE THING THAT WOULD
BENEFIT HELIOS IS HAVING A ROAD
MAP FOR ADOPTING OPEN.
>> YOU ARE ALL SEEDS AT
DIFFERENT PARTS OF THE ECO
SYSTEM REFLECT ON HOW YOU
ENVISION YOUR CLUSTER AND/OR HOW
SPREAD MIGHT OCCUR WITHIN THE
CLUSTER LET ALONE THE BRIDGES.
I REFLECT BACK ON THAT AND HOW
YOU SEE YOURSELVES IN THE ECO
SYSTEM.
>> I SEE IT AS PART OF THE
PUZZLE.
I REALLY APPRECIATE AND
RECOMMEND "THE TALK" YESTERDAY
AND THE BOOK IS IMPORTANT FOR
HOW TO INSTILL CHANGE.
I SEE IT IN TWO WAYS.
HOW DO YOU HELP FACILITATE THAT
TYPE OF CHANGE TO OCCUR.
THAT'S ONE STEP.
THE OTHER I TRIED TO ILLUSTRATE
AND IT'S DEFINITELY A HARD
CHALLENGE IS ONE PIECE OF A
PUZZLE.
AND I GIVE NIH A TON OF CREDIT
FOR THIS WITH THE GENERALIST
REPOSITORY APPROACH IT'S NOT
NECESSARY A COLLABORATION OR
COMPETITION.
THERE'S A MIDDLE GROUND WHERE
YOU'RE SEEING THERE'S DIFFERENT
THINGS BUT YOU'LL WORK TOGETHER
BECAUSE THAT'S HOW WE'LL ALL
RISE.
WE'RE NOT SAYING MERGE IT INTO
ONE TOOL.
YOU HAVE TO TRULY MEET THE FAIR
PRINCIPLES AND BE INTEROPERABLE
AND HOW I VIEW IT IN TERMS OF
THAT APPROACH BUT WE CAN CHAT
MORE.
IT'S A GOOD TOPIC.
>> THANK YOU.
YOU DON'T NEED TO ASK A QUESTION
AND WE CAN KEEP THE CONVERSATION
GOING DIRECT THEM TO EACH OTHER
OR HAVE OPEN QUESTIONS FOR
ANYONE.
CLOSER TO THE MIC, PLEASE.
>> TWO BRIEF COMMENTS TO
STIMULATE THOUGHT AND
DISCUSSION.
IN RESPONSE TO QUESTIONS ABOUT
EXAMPLES OF GOOD PRACTICE AND
ONE I FIND ESPECIALLY
INTERESTING IS GENT UNIVERSITY
IN BELGIUM.
CHANGE THERE CAME YESTERDAY IN
MY TALK FROM THIS FRUSTRATION
WITH PUBLISH OR PERISH.
IT WAS STIMULATED AT THE TALK
BUT IT WAS THE OFFICE OF
RESEARCH IN COLLABORATION WITH
THE HUMAN RESOURCES DEPARTMENT.
YOU DON'T ALWAYS HEAR HUMAN
RESOURCES BEING BROUGHT INTO THE
CONVERSATIONS VERY INTERESTING
OR OFTEN.
AND THEN THE OTHER INTERESTING
THINGS ABOUT CHANGE AT GENT WAS
THEY RECOGNIZED THAT BESIDES U
TRACT THEY'D BE ONE OF THE FIRST
INSTITUTIONS TO DO IT AND THEY
EMBRACED IT AND SAID ALL RIGHT,
WE WANT TO BE A LEADER.
THIS IS GOING TO BE PART OF OUR
IDENTITY.
AND THE PART OF THE CHANGE
PROCESS I THINK IS
TRANSFORMATIVE IS THEY SWITCHED
IT FROM AN EVALUATION.
IT'S STILL AN EVALUATION BUT
IT'S MORE OF A COACHING MODEL.
SO THEY LENGTHENED THE TENURE
PROCESS AND SAID YOUR EVALUATION
COMMITTEE IS NOW GOING TO BE
LIKE A PROFESSIONAL DEVELOPMENT
COMMITTEE.
SO NOW NEW FACULTY MEMBERS ARE
ABLE TO MEET WITH THIS COMMITTEE
EVERY YEAR, TALK ABOUT THE
PROGRESS THEY'VE MADE, WHAT THEY
WANT TO DO IN THE FUTURE.
SO THEY'RE WORKING THROUGH OR
WORKING WITH THAT COMMITTEE UP
UNTIL THE POINT OF PROMOTION OR
TENURE.
AND ALL THIS IS DOCUMENTED IN
THE CASE STUDY ON THE WEBSITE.
YOU CAN LEARN ABOUT IT THERE.
THEN THE OTHER EXAMPLE THAT I
WANTED TO GO BACK TO IS TOUCHING
ON TOP-DOWN, BOTTOM-UP AND THE
CREATION OF SIPS IN ABOUT 2013
AND A COUPLE YEARS AGO AND THE
GRADUATE STUDENTS STARTED HAVING
CONVERSATIONS ABOUT WELL,
THERE'S ALL THIS CHANGE FOR
FACULTY EVALUATION BUT WE'RE
STILL EVALUATED ON THE NUMBER OF
PAPERS WE PUBLISH SO WHAT'S
GOING ON.
ORGANIZED WITH A NEW FRAME WORE
FRAMEWORK AND PITCHED IT AND HAD
A TAKER AND NOW THAT DEPARTMENT
IMPLEMENT THE NEW GRADUATE
EVALUATION WHICH IS SPANNED OUT
TO OTHER GRADUATE DEPARTMENTS
AND THEY'RE IN THE PROGRESS OF
EVALUATING SUCCESSES.
THROWING THAT OUT AS EXAMPLES.
>> I HAVE A COUPLE OF THOUGHTS
HERE.
FIRST ON PROMOTION AND TENURE,
MANY INSTITUTIONS ARE INTENSELY
FOCUSSED IN DIVERSITY, EQUITY
AND INCLUSION.
THAT'S A MAJOR EFFORT AND THE
EFFORTS DISCUSSED HERE NEED TO
TAKE THAT INTO ACCOUNT AND NEED
TO CONSIDER WHETHER FOCUS ON THE
METRICS OF RIGOR BEING DISCUSSED
AND BEAR RESOURCES TO MAY OR MAY
NOT BE AVAILABLE TO CANDIDATES
THAT MAY OR MAY NOT BE
DISADVANTAGED AS A RESULT OF
WHERE THEY'RE COMING FROM AND
SIT IN DISHA OF RESOURCES AND DO
WE LOOK AT DIVERSITY AND
INCLUSION?
AND SHOULD WE BE DOING THIS IN A
VACUUM AND IT WOULD BE AN
OPPORTUNITY TO GET AND ENGAGE IN
A DISCUSSION ABOUT HOW RIGOR AND
DIVERSITY GO TOGETHER SO THAT
WHEN YOU'RE LOOKING FOR DIVERSE
CANDIDATES ONE METRIC YOU MIGHT
LOOK FOR IF THEY'RE CLINICAL
RESEARCHERS IS HOW DIVERSE ARE
THE SUBJECTS THEY'RE RECRUITING
INTO THEIR TRIALS.
WHAT STEPS ARE THEY TAKING
BECAUSE IT'S PART OF RIGOR.
THOSE ARE MY COMMENTS.
>> IT MAKES ME OF WHAT WHAT IS
CRITERIA AND THE SAME LIP
SERVICE IN HIRING COMMITTEES AND
DOCUMENTING WHERE WE SEE
SOMETIMES ON PAPER THEY PUT THE
REQUIREMENTS AND LANGUAGE AROUND
DIVERSITY AND INCLUSION BUT
THERE ISN'T NECESSARILY THE
PRACTICES TO MAKE SURE THAT IS
TAKING PLACE AND THOSE
CONSIDERATIONS ARE PUT IN.
TO BE MADE.OOK AT
>> IT'S AN IMPORTANT POINT IN
OUR ACTIVITY.
I WAS HIGHLIGHTING THOSE OPEN
PRACTICES.
THE WHOLE THING WE'RE DOING IS
CALLED GOOD EVALUATION PRACTICE.
THIS IS ABOUT EVALUATION IN
GENERAL.
WE'RE TALKING ABOUT ACADEMIC AGE
AND ANONYMIZING THE AGE OF
CANDIDATES.
THE ISSUES OF UNCONSCIOUS BIAS.
IT'S PART OF THE PRACTICE AND SO
I FULLY AGREE.
AND ARE WE ALREADY SUCCESSFUL IN
DIFFERENT SELECTIONS.
MAYBE THIS IS EVEN THE WRONG
QUESTION.
THE DISCUSSION ABOUT EVALUATION
IN PRACTICE IS A WAY TO START
THE CONVERSATION WITH THE
FACULTY.
THIS IS WHAT THEY'RE INTERESTED
IN.
THIS IS ONE OF THE MOST
IMPORTANT DISCUSSIONS WITH
FACULTY AND TO ME THE MOST
IMPORTANT IMPACT THIS IS HAVING
IS THAT WE ARE IN THE MIDST OF A
VERY INTERESTING CONVERSATION.
>> WHEN I STARTED AT THE
UNIVERSITY OF NEW MEXICO I WAS
IN A FORTUNATE POSITION WHERE WE
WENT TO OUR OWN ACADEMIC UNIT
AND WE WROTE OUR OWN GUIDELINES
AND DIVERSITY, EQUITY AND
INCLUSION WERE THINGS WE ADDED
AS WELL AS THE REPRODUCIBILITY
AND RIGOR AND THERE'S THE
DIVERSITY AND IN OUR PARTICULAR
FIELD THERE'S ALL SORTS OF
DIFFERENT THINGS WHETHER IT'S
WEBSITES, EXHIBITS, MATERIAL
THAT HADN'T BEEN HIGHLY VALUED.
AND CEMENT THESE ARE IMPORTANT
THINGS AND LED TO RIGOROUS
SCHOLARSHIP.
AND PEOPLE ARE THINKING ABOUT IT
BUT THE QUESTION IS IS IT THEN
GETTING INSTITUTIONAL SUPPORT
FOR PEOPLE TO ACT ON THE
CONVERSATION?
WHETHER WE AGREE WE WANT TO HAVE
THESE THINGS AND MAYBE AS A UNIT
WE HAVE THE CONVERSATION OR I
HADN'T THOUGHT ABOUT WHETHER I
SHOULD HAVE THAT THING BUT THEN
DO YOU GET THE SUPPORTS TO BE
ABLE TO PUT IT INTO PRACTICE AND
WHETHER IT BE IN THE HIRING OR
THE EVALUATION.
>> MY NAME IS WILLIAM YOUNG FROM
THE UNIVERSITY OF CHICAGO AND
WOULD LIKE TO ADVOCATE FOR THE
EARLIER CAREER GRADUATES AND
THERE'S NO FORMALIZED PROFESTHE
PROCESS AND THOSE WHO TRY TO
LEARN THE RESEARCH SKILLS AND BE
RIGOROUS IN THEIR RESEARCH.
I'M WONDERING IF YOU HAVE
THOUGHTS ON WHETHER GRADUATE
STUDENTS OR POST-DOCS SHOULD BE
PROMOTE AND PROVIDED STABLE
CONDITIONS AND CAREERS AS DOING
THE JOB AS FORMALIZED RESEARCH
TRAINERS AND RIGOROUS
RESEARCHERS OR DO YOU SEE A MORE
TOP-DOWN EFFECT FROM TENURE,
PROMOTION AND METRICS BEING
CHANGED?
>> MAYBE YOUNG PEOPLE WILL COME
IN AND HAVE THE GOOD PRACTICES.
WE FOUND WHEN WE SURVEYED
FACULTY AS WELL SOMETIMES THE
MORE ESTABLISHED PEOPLE FEEL
MORE COMFORTABLE THEY CAN TAKE
THE RISKS AROUND MAKING THOSE
CHANGES.
NOT SPECIFIC TO THE ECRs BUT TO
THE STUDENTS BUT THERE ISN'T
ALWAYS A CLEAR WHO ARE THE RIGHT
LEADERS OF CHANGE.
SOMETIMES IT IS THE OLDER AND
SOMETIMES THE YOUNGER.
>> I THINK IT WOULD BE A GREAT
IDEA TO ACKNOWLEDGE GRAD
STUDENTS.
IN OUR DEPARTMENT THE GRAD
STUDENTS SEEM TO BE EXCITED
ABOUT DOING THIS AND SOMETIMES
THEIR P.I. IS NO, NO, NO.
IT'S MUCH FASTER IF WE DO IT
THIS WAY.
I HEAR THEM TALK ABOUT IT ALL
THE TIME SO IT WOULD BE GREAT TO
REWARD THEM FOR DOING THESE
THINGS.
>> IF I UNDERSTOOD WILL'S
QUESTION CORRECTLY, HE'S
THINKING ABOUT PERMANENT
POSITIONS TO HELP BRING ABOUT
CHANGE AS COMING FROM THE BOTTOM
UP NOT SOMEONE LANDING AND
SAYING I'M GOING TO MAKE A
CHANGE NOW.
IS THAT CORRECT, WILL?
>> IT FEELS LIKE ECRs ARE NOT
RECOGNIZED AS A STAKEHOLDER IN
THE CHANGE AND COMMITMENT TO
CHANGING AND IMPROVING RESEARCH
RIGOR.
ONE EXAMPLE IN TENURE EVALUATION
MIGHT BE IDENTIFYING THE FACULTY
WHO ARE IMPROVING THE TRAINEE
EXPERIENCE AND ALSO PROVIDING
THOSE OPPORTUNITIES TO IMPROVE
ONE'S RESEARCH PRACTICES AND
PERHAPS THE ROLE OF IMPROVING
RESEARCH MAY NOT REQUIRE A
PERMANENT POSITION OR
INDEPENDENT POSITION THAT WILL
ENFORCE OR ALLOW THIS KIND OF
CHANGE IF THAT MAKES SENSE.
>> I'D LIKE TO PIGGYBACK OFF
THAT.
FROM THE UNIVERSITY OF TEXAS AT
AUSTIN.
STARTING A FACULTY JOB IS YOU'RE
REALLY IN ISOLATION IT'S ABOUT
YOU BUILDING YOUR INDIVIDUAL
CAREER IN GRADUATE SCHOOL AND
DOING THE SAME THING AND
THINKING OF CATALYZING
COMMUNITIES AND MANY ARE DOING
CLUSTER HIRES.
THE IDEA THE PEOPLE IN THAT
CLUSTER FORM A COMMUNITY AND
PERHAPS IF THOSE PEOPLE WERE
TRAINED AS PART OF THEIR CLUSTER
HIRE BOOT CAMP TO HAVE RIGOR
TRAINING AND THEN ONGOING
TRAINING THAT WOULD BE SORT OF
THE LINK BETWEEN THE TOP-DOWN
AND BOTTOM-UP BECAUSE THEY WOULD
BE THE WIDE BRIDGE.
>> THERE'S DATA LIBRARIANS.
HOW DOES THE BUDGET WORK?
DO THE LABS PAY YOU FOR YOUR
SERVICES IF YOU GIVE A CLASS OR
UNIVERSITY OF MINNESOTA HAS
PEOPLE DO DATA CURATION.
DO THE LABS PAY OR HAS THE
UNIVERSAL ALLOCATED IT'S OWN
AND IF THERE'S COLLABORATIONS
WHAT'S SPECIAL ABOUT THE
UNIVERSITY?
>> THAT'S A GOOD AND COMPLICATED
QUESTION.
I WOULD SUGGEST TALKING TO BOTH
MYSELF AND HOW ABOUT IT WHEN
WE'RE OFF THE RECORD.
I WILL SAY THAT LIBRARIANS WE
ARE A PROFESSION THAT TENDS TO
BE ALTRUISTIC.
WE WANT TO DO EVERYTHING FOR
FREE AND THAT'S TO OUR DETRIMENT
A LOT OF THE TIME.
SO NO LABS RARELY IF EVER PAY
US.
SOMETIMES WE'LL STICK SOMEONE ON
A GRANT.
BUT FOR THE MOST PART MOST
LIBRARIES AREN'T PUSHING FOR
THAT.
I'M PROBABLY ONE OF THE FEW THAT
ACTIVELY LOOKED AT CREATING
SERVICES LIKE NIH RECHARGE
CENTERS WITHIN THE LIBRARY TO
TRY TO RECOUP COSTS AND
ENCOURAGE PARTNERSHIPS ON GRANTS
TO NOT ALLOW US TO DO THINGS
BEYOND OUR SCOPE IF WE'RE DOING
IT FOR FREE.
AND I'VE RECEIVED A LOT OF
PUSHBACK ON THIS AS A LIBRARY
LEADER.
BUT CREATING REPRODUCIBILITY
DOES TEND TO BE AT THE WEALTHIER
INSTITUTIONS.
THERE'S NO TWO WAYS AROUND THAT.
THAT BEING SAID A LOT OF THE
THINGS LIBRARIANS ARE ALREADY
DOING ARE ALIGNED WITH RIGOR AND
REPRODUCIBILITY AND CAN BE A
COMPONENT OF THEIR OR BRING IN
EXTERNAL RESOURCES AS WELL.
A LOT OF WHAT WE DO IS
COORDINATE AND ALSO MAKE
REFERRALS.
SO FOR EXAMPLE WORKING WITH THE
CARPENTRIES, WORKING WITH
REPRODUCIBILITY FOR EVERYONE AND
SOME OF THESE OTHER LARGE-SCALE
EFFORTS ARE THINGS WE CAN DO
EVEN IF WE DON'T HAVE FULL-TIME
FINANCIAL SUPPORT FOR THAT.
HAVING INSTITUTIONS FUND THEIR
LIBRARY AS WELL BECAUSE WE'RE A
CENTRAL RESOURCE AND CAN PROVIDE
DIFFERENT RESOURCES THAT REACH
EVERYONE INSTEAD OF SMALLER
UNITS.
VOTE FOR YOUR LIBRARY BUT
THEY'RE USUALLY FUNDED THROUGH
CENTRALIZED FUND SO A LOT OF
LIBRARY BUDGETS ARE SHRINKING.
>> UNIVERSITY OF FLORIDA.
I WOULD LIKE TO COMMENT ABOUT
THE TOPIC OF RESEARCHER
EVALUATION BECAUSE I THINK WE
SHOULD GET RID OF JOURNAL IMPACT
FACTORS ENTIRELY AND WOULD LIKE
TO POSE THE QUESTION TO THE
FUNDERS IN THE ROOM WHETHER IT'S
EVEN REASONABLE OR POSSIBLE TO
RANK GRANT PROPOSALS IN A TOP
1%, TOP 2% SORT OF WAY.
AND WHETHER YOU CONSIDER THE
LOTTERY SYSTEMS IN OTHER PLACES
WHERE YOU HAVE A CUT OFF FOR
THESE PROJECTS AREN'T TO FIND
AND LOOK AMONG THE PROPOSALS
INSTEAD OF TRYING TO DO THIS
VERY FINE-GRAIN RANKING WE KNOW
IS NOT POSSIBLE, RIGHT.
THERE'S A LOT OF STUDIES SHOWING
THERE'S A VERY LOW RELIABILITY
IN REVIEWER RATINGS FOR GRANT
PROPOSALS.
RETHINKING EVALUATION OR OTHER
MODELS THAT LOOK AT THAT AND THE
COMPETITION CULTURE INSTEAD OF
COLLABORATION CULTURE ENDS UP
HAVING NEGATIVE EFFECTS BECAUSE
WE ARE CREATING ALL MODELS OF
COMPETITION AND RANKING AT EVERY
LEVEL FROM INDIVIDUALS IN THE
CLASSROOM AND STUDENTS ALL THE
WAY TO THE FACULTY AND HOW WE DO
THE HIRING AND HOW WE DO
GRANTING AND EVALUATE
INSTITUTIONS.
PRESSURE ON PEOPLE.
TWO MORE QUESTIONS AND WE'RE
NEAR THE END OF OUR TIME BUT
LET'S TAKE THE LAST TWO
>> THANK YOU FOR AN INTERESTING
QUESTION.
I'D LIKE TO FOLLOW-UP ON WILL'S
COMMENT ABOUT ECRs AS IT RELATES
TO PEOPLE WORKING FOR SCIENCE
IMPROVEMENT BECAUSE RE-THINKING
OUR EVALUATION POLICIES --
>> CLOSER TO THE MIC.
>> RE-THINKING OUR EVALUATION
POLICIES HERE IS IMPORTANT TO
WHERE THE FUTURE PEOPLE WHO WILL
BE IN THIS ROOM ARE GOING TO
COME FROM.
WE HELD A VIRTUAL BRAIN STORMING
EVENT ABOUT A YEAR AGO WITH 54
ECRs INVOLVED IN INITIATIVES TO
IMPROVE CULTURE AND PRACTICE.
THEY WERE NOT WORK ON IMPROVING
THEIR OWN SCIENCE BUT THE
SCIENTIFIC SYSTEM.
THE NUMBER ONE BARRIER ALL THOSE
PEOPLE IDENTIFIED WAS THAT THEIR
WORK TO IMPROVE SCIENCE WAS NOT
REWARDED OR INCENTIVIZED FOR
CAREER CAREER PROGRESSION.
IF WE'RE NOT REWARDING OR
INCENTIVIZING THOSE ACTIVITIES
THEN THOSE PEOPLE WILL BE PUSHED
OUT OF SCIENCE AND WE'RE MISSING
THE STRONGEST PEOPLE WE SHOULD
HAVE COMING IN FOR THE NEXT
GENERATION.
THEY'RE ACTIVITIES MAY BE
DETRIMENTAL FOR CAREER
PROGRESSION BECAUSE THEY'RE
TAKING AWAY TIME FROM THE THINGS
THAT ARE COUNTING TOWARDS CAREER
PROGRESSION.
I THINK THAT'S REALLY IMPORTANT
TO KEEP IN MIND.
THE OTHER LESSONS LEARN FROM
THAT COMMENT WE HAVE OR FROM
THAT EVENT WE HAVE COMPLIED INTO
A DOCUMENT WITH 45 OF THE
PARTICIPANTS LISTED AS AUTHORS.
THAT WILL BE A CONSENSUS
STATEMENT AND EMPOWER
RESEARCHERS FOR CAREER AND
PRACTICE INCLUDES
RECOMMENDATIONS FOR
STAKEHOLDERS, INSTITUTIONS,
JOURNALS, FUNDING AGENCIES, PEER
COMMUNITIES AND SO ON AND SO
FORTH AND INCLUDES LESSONS
LEARNED FOR ECRs WORKING ON
THEIR OWN INITIATIVES OR TOWARDS
SCIENCE IMPROVEMENT.
WHEN WE'RE TALKING ABOUT HOW TO
CHANGE INCENTIVES, IT'S NOT JUST
FOR NORMAL RESEARCHERS, IT'S
ALSO FOR OURSELVES.
WE NEED TO INCENTIVIZE AND
REWARD OUR OWN WORK.
>> THANK YOU.
>> A BIT CHEEKY SINCE WE'RE
GOING TO THE NEXT SESSION AND
HOLD A POINT OF FIRE AND I DID
WANT TO PICK ON NAOMI'S COMMENT
AND REFLECT IT BACK WITH THE
PANEL AND AT THE EARLIER PANEL
THERE ARE ASPECTS I THINK WITH
WHAT MALCOLM WAS DESCRIBING
YESTERDAY OF RESEARCH INGRIT.
I WANTED TO PUT BACK ON THE
TABLE CONSIDERATIONS OF WHAT WE
REFER TO AS INCLUSION BY DESIGN.
COMMENTS ON DIVERSITY AND
INCLUSION OF AN APPLICANT OR
THOSE WITHIN THE SYSTEM AND
INCLUSION BY DESIGN IS THINKING
ABOUT THE DESIGN OF THE ACTUAL
RESEARCH ITSELF.
IN THE U.K. WE'RE SEEING
CRITICAL QUESTIONS BEING ASKED
ABOUT CLINICAL TRIALS AND TRYING
TO REVOLUTIONIZE WHAT THAT LOOKS
LIKE.
I THINK WE'RE NOT TALK THAT IN
THIS WORKSHOP BECAUSE THERE'S
FUM
FUNDAMENTAL ASPECTS OF STRUCTURE
AND HOW THINGS LOOK THAT MAY PUT
PRESSURE ON RIGOR.
NOT NECESSARILY ON BEING
RIGOROUS BUT CHANGING THE
CULTURE OF HOW RIGOR MIGHT LOOK
AND IT'S A QUESTION OF WHAT THE
FEEDBACK MIGHT LOOK LIKE WHETHER
IT'S FOR AN EARLY CAREER
RESEARCHER OR SOMEONE ELSE IF
PEOPLE ARE COMING UP WIN OR
CHALLENGING HOW THINGS -- WITH
OR CHALLENGING HOW THINGS ARE
DONE AND ASKING DIFFERENT TYPES
OF QUESTIONS ESPECIALLY IF
THEY'RE TALKING WITH DIFFERENT
POPULATIONS OR WANT DIFFERENT
ENGAGEMENT WITH THE WORK THEY'RE
DOING.
THERE'S SOMETHING THERE THAT
WOULD BE INTERESTING TO CONTINUE
TO TEASE APART.
>> WE HAVE TIME TO CONTINUE
OR -- I THINK WE BETTER --
THAT'S A NICE NOTE TO END ON AND
OPENS UP MORE CONVERSATION ON
THE DIVERSITY AND THE WORK I'VE
DONE IS LOOKING AT DIFFERENT
ELEMENTS OF SCHOLARSHIP AND
DIFFERENT OUTPUTS AND WHAT KIND
OF WORK AND HOW ACADEMICS SPEND
THEIR TIME AND HOW THE REWARD
SYSTEMS PUT IN PLACE OFTEN
CHANNEL US TO A PARTICULAR KIND
OF ACTIVITY THAT THE PATH OF
LEAST RESISTANCE AND THERE'S TWO
COUNTS ONES VERY MOTIVATED
BECAUSE THEY WANT TO DO
SOMETHING DIFFERENT AND THE ONES
THAT ARE WANTING TO SAY WHAT I
NEED TO KNOW THE RULES AND THE
RULES OF THE GAME ARE CLEAR WHEN
IT'S PUBLICATIONS AND THESE
JOURNALS THAT FOLLOW THIS MODEL.
THERE'S TWO DIFFERENT WAYS OF
SEEING OR TYPES OF ECRs.
THOSE ARE I DON'T CARE ABOUT THE
REWARDS, I'M JUST GOING TO DO IT
AND THEN I'M GOING TO FIRST GET
ESTABLISHED BUT I THINK WE NEED
TO THINK OF THE CONVERSATION AND
RIGOR AND REPRODUCIBILITY OR
QUALITY.
ALL THOSE IS LIKE HOW DO OUR
SYSTEMS AND WE HAVE SEEN THE
NEEDS FOR THE SUPPORTS FOR THE
TECHNOLOGIES IN PLACE AND POEM
TO PROVIDE THE SUPPORTS AND WE
NEED TO BE VALUING THOSE THINGS
AND STARTING THE CONVERSATIONS.
THANK YOU ALL FOR YOUR
PRESENTATIONS AND FOR ENGAGING
WITH US.
WE'RE GOING TO START THE SESSION
ON WORKING TOGETHER SO REALLY
SORT OF TAKING A LOOK AT THE
SYSTEM IN WHICH IN WHICH WE
OPERATE AND THE PANEL DISCUSSION
WILL BE MODERATED BY KAREN SALT.
SHE'S THE DEPUTY DIRECTOR FOR
R&D, CULTURE AND ENVIRONMENT AT
UKRI.
AND I'LL LET HER INTRODUCE THE
REST OF THE PANEL.
SIMILAR PRESENTATIONS FOLLOWED
BY A PLENARY DISCUSSION AND THEN
THE LAST 30 MINUTES OF TODAY AND
I'M SAYING THIS NOW SO YOU CAN
GET YOUR THINKING JUICES GOING,
WE WILL DISCUSS HOW WE ARE
BEFORE WE LEAVE THIS ROOM, HOW
DO WE WANT TO TAKE THE IDEAS
THAT WERE DISCUSSED OVER THE
LAST TWO DAYS AND TURN THEM INTO
ACTIONABLE THINGS.
HOW DO WE ACT ON ALL THE
STRATEGIES AND IDEAS DISCUSSED
IN THE TWO DAYS.
YOU'LL BE SUGGESTING THESE WAYS
SO I JUST WANTED TO LET YOU
PREP
PREPARE FOR THAT.
AND WE'LL BE SKIPPING THE BREAK.
WE GOT MOST OF OUR FOLKS BACK.
IT'S ALWAYS SLOW THE AFTER-LUNCH
CROWD WHERE YOU'RE A BIT SLEEPY
AND TIRED AND TOWARDS THE END OF
THE DAY A BIT FED UP.
HOPEFULLY WE CAN STIMULATE YOU
AND GIVE YOU THINGS TO THINK
ABOUT AND START TO DRAW TOGETHER
WHAT WE'VE BEEN TALKING ABOUT.
I'M EXCITED BY THIS PANEL AND
THE BRILLIANT WORK I THINK
THEY'LL BE ABLE TO SHARE WITH
YOU.
WE'LL BE GOING IN ORDER OF THE
AGENDA.
WE'LL START WITH SOPHIA CRUWELL
AND THEN MARYROSE FRANCO AND
MARCUS MUNAFÒ AND NAY OMIC
SCHRAG FROM COLOMBIA UNIVERSITY.
YOU SHOULD HAVE ALL THEIR BIOS
CONTAINED WITHIN THE AGENDA
INFORMATION AND CAN I ASK THE
PANELISTS IF THERE'S ANYTHING
CRUCIAL YOU HAVEN'T PUT IN YOUR
BIO YOU WOULD LIKE TO ADD TO
SHARE PLEASE DO AND COULD BE A
NICE WAY TOO .
THAT WOULD BE GREAT.
THIS PARTICULAR PANEL I THINK IS
A REALLY GOOD ONE THE ORGANIZERS
HAVE DONE A FANTASTIC JOB IN
ANCHORING THE CONVERSATION AND
WHAT MIGHT WE DO AS A GROUP AND
USE WHAT OTHERS ARE DOING.
WHAT ARE THE CHARACTERISTICS OF
AND MOVEMENTS AND WHAT HAS NOT
WORKED AND WE'RE NOT USUALLY
GOOD AT STOPPING AND
ACKNOWLEDGING WHAT DOESN'T WORK
AND IT'S A BIG BOOK OF MINE IN
THE POLICY WORLD AND KEEPS
ADDING MORE THINGS AND HOW TO
BUILD STRONGER.
WHAT ARE WAYS TO ULTIMATELY DO
THAT.
I WOULD ADD PROBABLY ANOTHER
QUESTION AND OUR ORGANIZERS PUT
TOGETHER, HOW DO WE BE MORE
STRATEGIC?
WHAT DOES THAT LOOK LIKE?
I'VE OFFERED YESTERDAY AND
THROUGHOUT MOST THE DAY
QUESTIONS ABOUT SYSTEMS.
SOME IS SYSTEMS THINKING.
IT'S NOT NECESSARILY TRYING TO
CREATE SOLUTIONS AIMED AT ALL
THE ENTIRE SOLUTION BUT THINK
SYSTEMS AND ORIENTATION AND
THINGS I'D LIKE TO PUT ON THE
SESSION MOVING FORWARD.
WHAT'S THE WORK LOAD LOOK LIKE
FOR THIS WORK?
WE THINK OF INFRASTRUCTURE BUT
WHO WILL DO IT AND WHAT'S THE
WORKLOAD AND WHAT DOES IT FEEL
LIKE TO DO THAT KIND OF WORK
WHAT'S WITH NEGATIVE FEEDBACK
AND SEEN AS A POLICE OFFICER
THAN A COLLABORATOR.
AND WHAT DO WE DO WITH THE
COMPETING PRIORITIES MANY ARE
GRAPPLING WITH THAT ARE ALL
ESSENTIAL AND THINGS YOU WANT TO
DO RIGHT NOW.
HOW DO WE SITUATE RIGOR SO IT
DOESN'T LOOK LIKE A TRADE-OFF.
WE HAVE TO GIVE UP ONE THING TO
DO ANOTHER?
AND TO PICK UP ON MALCOLM'S
POINT FROM BEFORE ABOUT
INCREMENTAL REVOLUTION AND
CONTINUED CONVERSATIONS AND
PROVOCATIONS OTHERS RAISED I'D
LIKE TO TAKE THAT TO A SLIGHTLY
DIFFERENT SPACE.
I'M THINKING MACRO AND MICRO.
WE WERE TALKING ABOUT
DEPARTMENTAL AND COMMUNITY AND
LOCATED AROUND THE STRUCTURE AND
MACRO AND MICRO ECONOMICS ARE TO
UNDERSTAND DIFFERENT LEVELS AND
SCALES.
SOMETIMES AT THE MICROLEVEL BUT
SOMETIMES THE MACRO LEVEL AND
INFLUENCES ENTAIL.
WE KIND OF NEED BOTH TO GET
UNDERNEATH WHAT WE'RE TALKING
ABOUT WITH RIGOR.
I ROUTINELY STRUGGLE ABOUT WHAT
WORKS.
WHAT HAVE A PROBLEM OF
INTERVENTION CONVERGENCE.
PEOPLE DO A LOT OF THINGS AND
GET ANY RETURN OF ANY KIND
DOESN'T MATTER HOW SPELL AND GET
POSITIVE REINFORCEMENT
EVERYTHING WORKS.
WE CONTINUE WITH THIS BARRAGE
AND BIG LIST OF STUFF OPPOSED TO
UNDERSTANDING THE PARTICULAR
INTERVENTION AND THAT'S WHAT I
MEAN BY INTERVENTION CONVERGENCE
AND WHEN WE WANT TO PICK STUFF
UP BECAUSE OFTEN PICK UP THE
WHOLE THING AND TRY TO MOVE IT
TO ANOTHER LOCATION OPPOSED TO
THINKING OF THE MICRO LEVEL
THINGS PEOPLE WERE INVOLVED WITH
AND WE'VE GOT A SET OF PANELIST
S -- AND THINGS AT MICRO LEVEL
AND OTHERS AT THE MACRO LEVEL
AND PULLING TOGETHER LARGE
AMBITIONS AND MAKE THE AIMS ARE
SMALL AND MAYBE THE GOALS AND
TRANSFORMATIONS HAVE A CLEAR
OUTPUT AND OUTCOMES.
BUT THEY ARE REALLY TRYING TO
TARGET AND MOVE THROUGH
HOPEFULLY YOU CAN SEE THE WAYS
THEY HAVE INTERVENTIONS.
WE'LL TAKE 10 MINUTES FOR EACH
OF OUR PRESENTERS TODAY AND THEN
WHAT WE WILL DO IS AS VERONIQUE
JUST MENTIONED WE'LL FLOW INTO
AN OPEN Q&A.
I MIGHT HAVE SOME DIRECT
QUESTIONS FOR OUR PANELISTS
FIRST TO GET US GOING.
SO WHAT I WOULD REALLY
APPRECIATE YOU DOING AS YOU
LISTEN TO THEM IS START JOTTING
QUESTIONS COMING TO YOU AND USE
THIS TO LINK THROUGH THE LAST
DAY AND THINGS DISCUSSED AT THE
BREAKOUTS, CONVERSATIONS YOU'VE
HAD AT LUNCHES OR ON THE SIDE
WITH OTHERS AND THEN OTHER
THINGS THAT OTHERS HAVE
MENTIONED AND YOU DON'T JUST
HAVE TO TALK TO PEOPLE WHO HAVE
BEEN THE SPEAKERS.
WE WANT TO TRY TO TALK TO THE
WHOLE AUDIENCE AND ENGAGE WITH
EACH OTHER DYNAMICALLY MOVING
FORWARD.
I THINK THAT'S ALL FOR RIGHT
NOW.
SOME FR PROVOCATIONS AND THINKI
FORWARD AND I'VE BEEN ASSURED
WE'RE READY TO GO ON ALL THE
TECH.
WE HAVE OUR TEAM AT THE READY.
WITHOUT FURTHER ADO OVER TO YOU.
>> SO HERE'S WHY I'M HERE.
I'LL BE TALKING ABOUT WHAT I
HOPE IS AN EXAMPLE OF A
SUCCESSFUL COMMUNITY OF RIGOR
CHAMPIONS, REPRODUCIBILITY.
I'LL BRIEFLY TELL YOU WHAT
REPRODUCIBILITY IS AND WHY IT
SEEMS TO BE WORKING AND HOW WE
GOT HERE.
AS YOU SAID, WE'VE GOT LOTS OF
Rs IN THIS WORKSHOP AND I'M
ADDING ANOTHER TO CONFUSE THAT
AND ASIDE FROM THAT THERE'S A
GRASSROOTS INITIATIVE WE STARTED
FOUR YEARS AGO.
IT'S LED AND ORGANIZED ON THE
LOCAL AND GLOBAL LEVEL BY
VOLUNTEERS.
THERE'S A STEERING COMMITTEE AND
TRY TO TAKE THAT AS BEST WE CAN
AND IN THIS CONTEXT IT'S
IMPORTANT TO MENTION OUR FOCUS
IS LARGELY ON PSYCHOLOGY.
WE DO HAVE A FOCUS ON
NEUROSCIENCE AND IN THE
PSYCHOLOGY AREA.
AND I JUST WANTED TO DO THIS IN
FRONT OF LOTS OF PEOPLE AND THIS
IS ME FOUR YEARS AGO.
WE WERE ALL AND WE WERE TALKING
ABOUT CAREER AT THAT POINT AND
SAM JUST STARTED HIS FIRST
POST-DOC AND A STARTED MY
MASTER'S AND WE WERE A BIT
DISSATISFIED BY THE LACK OF
DISCUSSION AROUND OPEN AND
RIGOROUS RESEARCH OR DISCUSSIONS
ACCESSIBLE TO US.
THERE WASN'T MUCH GOING ON AND
LOOKING FOR UCRs BEYOND THE
THREE OF US RANTING OR TALKING
EXCITEDLY ABOUT THE THINGS
CHANGING.
WE STARTED THIS JOURNEY AT
OXFORD AND WENT QUITE WELL AND
TALKED TO PEOPLE ABOUT IT WE
HAVE LOCAL ORGANIZERS AND
SOMEONE ELSE AND WE BEEN
DISCUSSING THIS ALMOST THE
ENTIRE TIME.
WHAT IS IT WE'RE ACTUALLY DOING
IN ALL THESE CLUBS?
THAT'S REALLY IMPORTANT BECAUSE
OFTEN THERE'S NO FORMAL TRAINING
IN THESE PRACTICES.
AND WE'RE PLANNING THE GAP TO
SOME EXTENT BY DISCUSSING
RELEVANT PAPERS AND EXPLORING
IDEAS SURROUNDING OPEN AND
RIGOROUS RESEARCH.
BY DOING THAT WE ARE ALSO MAKING
THESE ISSUES VISIBLE AT
INSTITUTION WHERE'S POTENTIALLY
THAT WASN'T THAT VISIBLE BEFORE
AND THAT'S AN IMPORTANT AIM.
I THINK IT'S A TROJAN HORSE
BECAUSE IT'S WHAT YOU CAN SELL
TO YOUR SUPERVISOR AND I'M
LEARNING ABOUT THEM.
I MAY TAKE THEM UP.
I THINK THE KEY IS COMMUNITY AND
I ENJOY THIS THE COMMUNITY WORKS
ON TWO DIFFERENT LEVELS AND MADE
UP BY SUPPORTIVE PEERS AND
SUPPORTING EACH OTHER THROUGH
DIFFERENT CHALLENGES THEY MAY BE
EXPERIENCING AND TRYING TO
IMPLEMENT NEW PRACTICES THAT MAY
BE AREN'T NORMAL IN THE CONTEXT
AND SHARING EXPERIENCES AND
ISSUES.
LOWERING BARRIERS, MAKING THINGS
A BIT MORE ACCESSIBLE AND THEN
ON THE OTHER HAND WE HAVE THE
GLOBAL COMMUNITY WE'VE HAD SOME
DIFFERENT TIME ZONES.
IT'S BEEN TWICE FUN AND WE HAVE
THAT ONE HAND LOCAL AUTONOMY AND
WE DON'T KNOW WHAT'S GOING ON IN
THE INDIVIDUAL JOURNAL CLUBS
UNTIL WE COLLABORATE ON BIGGER
THINGS AND WE HAVE THE SUPPORT
SYSTEM AND ENTITY TO USE.
SO WHAT HAVE WE LEARNED FROM THE
LAST FOUR YEARS, WHY DO I THINK
THIS IS WORKING?
LOCAL AUTONOMY AND SUPPORT IS
IMPORTANT AND WE TRIED TO MAKE
THE THRESHOLD BY STARTING THIS
FUNDER GAPS AND WE SENT OUT
STICKERS AND ALL THESE THINGS --
IT SEEMS SILLY BUT IT BRINGS ME
TO THE NEXT POINT AND IT SOUNDS
SILLY BUT WE KIND OF CREATED A
BRAND WITH THE LOGO AND THE
MATERIALS WE CREATED AND THE BAD
PUNS.
IT PROVIDES A LEGITIMACY AND
LOOKS LIKE AN EVENT NOT JUST A
JOURNAL CLUB.
WE HAD A PODCAST BRINGING THE
EXPERIENCE TO PEOPLE WHO MIGHT
NOT HAVE THAT COMMUNITY WHERE
THEY ARE.
AND WE GOT THAT FUNDING WHICH
WAS SUPER HELPFUL.
AND WE WERE ABLE TO SEE WHAT WAS
WORKING AND WHAT ISN'T.
THIS ONE IS SAYING IT'S NOT ONLY
GRAS ROOTS IT'S COMMUNED
BUILDING AND GIVING A HOME TO
THOSE FOR SOLUTIONS AND JOINING
AND WANTING TO BE PART OF A BIG
CHANGE.
IT'S ONE OF THE FEW THINGS THAT
GIVES ME HOPE BECAUSE WE ARE THE
CHANGE THAT'S NEEDED IN PRACTICE
AND CULTURE.
>> THE MOST EXCITING THINGS IS
WATCHING YOU SMILE WHEN YOU WERE
TALKING ABOUT THE WORK.
.
I KNOW WE'VE BEEN TALKING ABOUT
RIGOR AND BEING SERIOUS OVER THE
LAST DAY AND A BIT BUT IT'S
EXCITING TO GET THAT ENERGY OF
FUN AND FRIVOLITY AND EXCITEMENT
THAT'S DRAWN US TOGETHER.
I HOPE WE CAN CONTINUE TO FIND
FUN IN WHAT WE WANT TO DO HERE
OPPOSED TO VERY SERIOUS.
I HAVE LOADS OF QUESTIONS AND
WILL HOLD MINE TO THE END AND
SEND IT OVER TO MARYROSE.
>> I'M MARYROSE FRANCO AND IT'S
I'M PART AVA LICENSE OF
NON-PROFIT, NON GOVERNMENTAL
FUNDERS AND MAXIMIZING RESEARCH
TO IMPROVE HUMAN HEALTH AND I
THINK EVERYONE AGREES INCREASING
RIGOROUS SCIENCE WOULD IMPACT
RESEARCH.
THAT'S OUR LOGO SLIDE.
YOU CAN SEE DIVERSE WE ARE IN
TERMS OF MISSION AND SIZE AND
THEY'RE VERY DIVERSE OF COURSE
IN THE RESOURCES THEY'RE WILLING
TO DEVOTE TOWARDS RIGOR AND OPEN
SCIENCE.
SOME ARE ON THE CUTTING EDGE AND
OF COURSE YOU HAVE RESISTERS AS
WELL.
WE TRY TO BRING RESOURCES TO
BEAR AND PRESTIGE FROM FUNDING
ORGANIZATIONS AND HOPEFULLY
BROADEN THE COALITION OF THE
WILLING.
I'LL TALK ABOUT STRATEGIES THAT
CAN BE USED TO FACILITATE MORE
RIGOROUS RESEARCH AND ERIC
CHARACTERISTICS THAT CAN WORK IN
THIS EFFORT AND WHAT DO WE DO
AFTER THE MEETING.
MY FIRST EXAMPLE IS FROM THE
NATIONAL ACADEMIES ROUNDTABLE ON
INCENTIVES FOR OPEN SCIENCE.
I REALIZE OPEN IS MAYBE
NECESSARY BUT CERTAINLY NOT
SUFFICIENT FOR RIGOROUS BUT
THERE'S OVERLAP.
I THINK THIS EFFORT CAN LEARN A
LOT FROM WHERE WE'VE COME IN THE
OPEN SCIENCE EFFORT AND TRY TO
PUSH THAT.
AND FOSTER R.
IN THE ROUNDTABLE IT WAS
IMPORTANT WE HAD UNIVERSITIES
AND FUNDERS OF DIVERSE STRIPES.
WE ALSO HAD GOVERNMENT THERE.
I REMEMBER GARY SHEENAN SAID YOU
HAD TO DO THIS AND IT WAS ALWAYS
THE NIH TO DO SOMETHING AND IT
WAS GOOD FOR UNIVERSITIES AND
FUNDERS TO KNOW WE NEED TO BE
PART OF THE SOLUTION AND CAN'T
ALWAYS TURN TO THE NIH.
FIRST THING WE DID WAS BROKE
INTO WORK STREAMS.
WE HAD A GROUP ASSAYS AND
GATHERED WHY THIS IS IMPORTANT.
YOU NEED THIS ONE PAGER.
YOU NEED GRAPHICS.
AN EASY WAY TO CONVINCE PEOPLE
WHY IT'S IMPORTANT.
AND ALSO HOW THESE PRIMERS AND
HOW QUICKLY HOW TO DO IT AND
WHAT ARE WE TALKING ABOUT
POSTING A PREPRINT AND THINGS
LIKE THAT.
RE-IMAGINING OUTPUTS.
WE'RE NOT JUST TALKING IN OPEN
SCIENCE ABOUT OPEN DATA BUT WHAT
ARE ALL THE OUTPUTS AND THE
DATABASE.
THIS IS AWESOME.
PEOPLE SAY THERE'S NO PROOF OPEN
SCIENCE WORKS.
WELL, YES THERE IS AND THEY
CREATED A DATABASE.
THIS IS MORE ABOUT EVOLUTION NOT
REVOLUTION MAKING THE BAR LOW.
OUR INITIAL TITLE WAS CALLED THE
NUDGE GROUP.
WE WERE NUDGING OUR RESEARCHERS.
AND THE HYPOTHESIS WAS IF YOU
SIGNAL TO YOUR RESEARCHERS YOU
VALUE THIS BEHAVIOR IN VARIOUS
WAYS THEY'LL SAY I'M GOING TO DO
THAT.
IN APPLICATIONS AND RESOLUTION
REPORTS AND FUNDERS ON OUR TEAM
AS WELL AND UNIVERSITY P & T
PACKAGES AND JOB APPLICATIONS
YOU CAN ADD THIS LANGUAGE AND
IT'S JUST A LOW BAR.
JUST ASK, FOUNDATION, X, Y, Z,
OPEN SHARING AND OUTPUTS.
DESCRIBE INSTANCES WHERE YOU'VE
ENGAGED IN OPEN ACTIVITIES.
THIS IS VERY CUSTOMIZABLE.
YOU CAN SAY WHAT YOUR LOOKING
FOR MAKING ARTICLES.
ACCESS.
IF YOU DON'T CARE ABOUT SHARING
DATA, DON'T PUT THAT IN.
IF YOU WANT PEOPLE TO POST PRE
PRINTS, PUT THAT IN.
BUT THE NEXT TWO WERE REALLY FUN
NOT ONLY HAS YOUR OPEN RESEARCH
OUTPUTS BEEN USED BY OTHERS BUT
HOW HAVE YOU USED OTHERS OPEN
RESEARCH OUTPUTS?
AND THOSE DATA PARASITES WE
HEARD ABOUT BEFORE AND IN YOUR
DISCIPLINE, AGAIN, YOU CAN TWEAK
THIS HOW YOU WANT.
WHAT WAS FUN ABOUT THIS WAS WE
DIDN'T EVEN KNOW THAT AWARDEES
DIDN'T KNOW WE VALUE THIS AS
FUNDERS.
THEY OBVIOUSLY THOUGHT WE WANTED
JUST THEM TO BE CREATING THE
DATA AND NOT USING OTHER DATA.
THIS WAS AN EYE-OPENER FOR THE
FUNDERS AND HOW DO YOU PLAN TO
ENGAGE IN OPEN ACTIVITIES IN THE
FUTURE?
MANY FUNDERS PUT THAT IN.
THE BAR WAS LOWEST FOR PUTTING
IT IN PROGRESS REPORTS OF COURSE
AND WHAT THEY FOUND IN PROGRESS
REPORTS WERE THE RESEARCHERS
WEREN'T THE PROBLEM.
THEY DIDN'T GET ANY PUSHBACK.
IN FACT ALMOST EVERY PROGRESS
REPORT LISTED ACTIVITIES AND
DOIs FOR DATA THEY USED OR THEY
SHARED, CODE, MATERIALS, TALKED
ABOUT SHARING RE-AGENTS.
THAT WAS KIND OF A SUCCESS BUT
WE'RE STILL IN THE ANECDOTAL
STAGES AND EARLY GATHER, DATA
GATHERING AND HOW MANY FUNDERS
AND UNIVERSITIES ARE USING THIS.
IT'S STILL EARLY DAYS.
ONE THING THAT CAME I THINK
NAOMI SAID THIS TO ME THE OTHER
DAY, WE HAVE THE TOOL KIT.
THE NATIONAL ACADEMY CREATED
THIS TOOL KIT AND IT'S FABULOUS.
IF TOOL KITS COULD SOLVE THE
PROBLEM IT WOULD BE SOLVED.
WE WERE TALKING IF YOU BUILD IT
THEY WILL COME AND WE BUILT IT
AND THEY'RE NOT NECESSARILY
COMING.
THIS AGAIN IS OPEN SCIENCE.
I THINK WE HAVE THE TOOLS AND
KNOW HOW TO TAKE WHAT WAS DONE
FOR OPEN SCIENCE AND CONVERT
THAT TO SIGNALLING THE RIGOR AND
RIGOROUS AND VALUE AND THE
PHENOTYPE OF RIGOR.
FUNDERS ARE START BEING TO ASK
IF YOU REGISTERED THAT AND SOME
ARE REQUIRING REGISTRATION OR
PRE-REGISTERED REPORTS.
MANY ASK AS PART OF THE
EXPERIMENTAL DESIGN, HAVE YOU
HAD A STATISTICIAN SIGN OFF AND
LOOKING AT DESIGN REQUIRES US AS
FUNDERS TO BE ABLE TO REVIEW
THAT ADEQUATELY.
SO IT'S INCUMBENT ON US TO GET
THAT TECHNOLOGY OR EXPERTISE ON
OUR OWN REVIEW PANELS.
I MENTIONED THE NINDS AND NIH
POLICY AND THAT'S GREAT.
WE AS FUNDERS AND THE COMMUNITY
CAN ASK ABOUT THE ACTUAL
BEHAVIORS AND IF WE ASK, THEY'RE
GOING TO SAY, WELL, OKAY, I'M
NOT GOING TO LEAVE THAT BLANK
AND I'LL SAY I HAVE PLAN FOR
TRAINING AND RIGOROUS DESIGN AND
THESE ARE THE PLANS.
AND SOME REQUIRE THE PUBLICATION
BE MADE AVAILABLE WITHIN 12
MONTHS OF THE FUNDING.
OPEN SCHOLARSHIP IS VERY MUCH
SYNERGISTIC AND THE ROUNDTABLE
IS MORE TRYING TO CREATE
SYSTEM-LEVELS CHANGE AND CREATE
THE HELIOS HIGHER INITIATIVE.
OVER 70 UNIVERSITIES HAVE SIGNED
ON.
IT'S VERY MUCH A TOP-DOWN
APPROACH.
THE PRESIDENT HAS TO COMMIT OR
THE PRESIDENT'S DESIGNEE BUT
IT'S ALSO A LOT OF PEER
PRESSURE.
VERY MUCH CAMPUS ENGAGEMENT AND
COMMUNITY OF PRACTICE.
IF ANYONE WANTS TO JOIN HELIOS,
IT'S OPEN AND FIGURING IT'S THE
TOP DOWN.
IT DOESN'T PAY MUCH ATTENTION TO
THE WHY BRIDGES BUT WE'LL SEE
HOW IT WORKS.
IT'S JUST STARTING AND
RECOGNIZING AND THERE'S COHORTS
BEING STARTED AND USING THE
STATEMENT ACTION METHOD.
AS A FUNDER WE'RE COMMITTED TO
ADVANCING DISCOVERY IN
CARDIOVASCULAR DISEASE AND
AFFIRM THE RIGOROUS AND
COMMITTED TO DOING THESE THINGS.
WE'RE WORKING WITH THE FUNDERS
TO COPY THE HELIOS MODEL.
WE TALKED ABOUT IT TODAY.
IT LOOKS LIKE IT MAY BE
SUCCESSFUL BUT I HOPE WHATEVER
WE DO WE PUT RIGOROUS EVALUATION
IN AND HOW CAN WE PLUG IN INTO
THE EFFORT.
WHAT ARE THESE MUTUALLY
REINFORCING VECTORS AND IT MUST
BE A WIN-WIN.
WHATEVER YOU DO YOU HAVE TO
REWARD BUT IT HAS TO BE A WIN.
IF YOU'RE DOING TO TAKE
RESOURCES AWAY YOU MUST REWARD
FOR THAT REBAFR.
HAVE BEHAVIOR. P
I BELIEVE IN THER OF
COHORTS AND AND IT'S A SPECIAL
PLACE AND A DIDN'T REALIZE HOW
MUCH IT MAKES SENSE AND WHY WE
HADN'T GOTTEN THE TRACTION WEED.
YOU CAN MOF ROOS ONE WAY AND YOU
CAN BE GREAT AT SOMETHING AND
NOT BE SO GREAT AT SOMETHING BUT
IT'S NOT ACCOUNTABLE.
YOU'RE MOVING FROM LEFT TO
RIGHT.
WE LOVE RUBRICS AND GUIDELINES.
EVALUATION WITHOUT DATA YOU
CAN'T SAY ANYTHING WORKS SO
SEEMS LIKE A WEST.
POWER OF CONVENING AND
CATALYZING.
SUCCESS STORIES ARE VERY
IMPORTANT.
YOU CAN'T REALLY CONVINCE PEOPLE
TO CHANGE THEIR BEHAVIOR UNLESS
YOU CAN SHOW IT'S BEEN
SUCCESSFUL.
AS A FUNDER IT'S INCUMBENT ON US
TO LOOK AT MONEY, TIME AND
EMPOWERING OUR STAKEHOLDERS AND
EARLY CAREER SCIENTISTS AND HOW
DO WE WORK TOGETHER?
TOP GUIDELINES, TARGET WHAT I
THINK MIGHT BE INTERESTING IS TO
CREATE A FLOW OF CAREER STAGE
AND PLUG IN STRATEGIES INTO THE
PARTS MUTUALLY ENFORCING AND
CREATE THE WIDE BRIDGES.
>> EXCELLENT AND BRILLIANT.
>> THERE'S ANOTHER POLICY AREA
I'M LEADING ON WITHIN RESEARCH
INNOVATION.
I'M INTERESTED IN THE RESIDENCES
BETWEEN RIGOR AND SECURITY.
THEY TEND TO BE BROUGHT UP
QUITE A LITTLE AND THERE MAY BE
LOTS OF SIMILARITIES IF WE THINK
OF BEHAVIOR AND CULTURE CHANGE
MORE BROADLY.
WE STORE YOUR QUESTIONS AND
WE'LL NOW TURN OUT OVER TO
MARCUS.
>> THANK YOU, KAREN.
>> I'M GOING TO BE ABOUT THE
U.K. NETWORK IN EXISTENCE OVER
THREE YEARS AND DESCRIBE WHERE
THAT CAME FROM AND HOW IT'S
STRUCTURED AND WHO OUR FOCUS IS
GOING FORWARD.
IN THE U.K. THERE WAS A MEET
SUPPORTED BY A MAJOR
ENVIRONMENTAL CHARITY AND SOME
RESEARCH WAS THE ORGANIZATION
PARN REPRESENTS AND WELCOME HAS
BEEN DOING A GREAT DEAL OF WORK
AROUND RESEARCH CULTURE WITHIN
WHICH THE ISSUES WE'RE TALKING
ABOUT CAN BE SITUATE AND THEY
COMMISSIONED THIS REPORT ON
RESEARCH INTEGRITY THE LANDSCAPE
STUDY WITH THE RESEARCH
INTEGRATE OFFICE AND LAST YEAR
THE HOUSE OF COMMONS LAUNCHED AN
INQUIRY AND THE REPORT WILL BE
FORTHCOMING.
TO SOME EXTENT WE'RE IN A
POSITION WHERE WE RUN THE RISK
OF SOLUTIONS BEING IMPOSED ON US
FROM THE TOP-DOWN AND I THINK
THERE'S AN IMPETUS FOR
RESEARCHERS TO ENGAGE IN THESE
ISSUES AND WE DID THIS AFTER
LINING UP OUR DUCKS AND LOBBYING
FUNDERS AND POSITIONING
OURSELVES AS A GROUP OF
ACADEMICS AND ACADEMIC
COLLABORATION EFFECTIVELY THAT
WOULD TAKE ON THAT
RESPONSIBILITY OF JOINING UP THE
ISSUES MANY OF THESE DIFFERENT
FUNDERS HAD RECOGNIZED AS AN
ISSUE BUT WHICH THEY DIDN'T FEEL
WAS ENTIRELY THEIR
RESPONSIBILITY TO ADDRESS, IF
YOU LIKE.
IT'S A PEER-LED COLLABORATION
THAT BRINGS TOGETHER COMMUNITIES
WITHIN THE WIDER RESEARCH ECO
SYSTEM WITH THE AIM OF
CONNECTING, HARMONIZING,
COORDINATING ACTIVITY.
IT'S NOT SO MUCH ABOUT INVENTING
NEW ACTIVITY AS THE ACTIVITY
ALREADY IN PLACE.
WE'VE TALKED ABOUT RESEARCH
RIGOR AND INTEGRATE ITY AND WE D
WORDS LIKE THIS AND THEY HAVE
DIFFERENT MEANINGS IN DIFFERENT
CONTEXT.
OUR FOCUS IS ON THE RESEARCH
PROCESS AND WE DESCRIBED SOME OF
THE RATIONALE FOR WHY THAT FOCUS
CAN BE VALUABLE IN THIS PAPER
PUBLISHED IN TECHNOLOGY A FEW
YEARS AGO AND THAT DERIVES FROM
APPROACHES USED IN OTHER CONTEXT
SUCH AS MANUFACTURING BY
FOCUSSING ON THE PROCESS AND
INTRODUCING CHECKS ON THE
PROCESS RATHER THAN ON THE
OUTPUTS OF THE PROCESS AND WHAT
YOU SEE ON THE BOTTOM LEVEL HAVE
THE COYOTE -- KIOTO PROCESS AND
THE FOCUS ON PROCESS I THINK IS
A THEME THAT'S COME FROM THE
DISCUSSIONS I'VE BEEN PART OF
OVER THE LAST COUPLE DAYS.
WE HAVE A STEERING GROUP AND
ALEXANDER COLLINS AND CHRIS
CHAMBERS AND LOCAL NETWORKS AND
ETIENNE ROSH.
THE MAJOR PARTS OF THE U.K.
STRUCTURE ARE OUR LOCAL NETWORK
AND INSTITUTIONAL MEMBERS AND
EXTERNAL STAKEHOLDERS.
LOCAL NETWORKS ARE IN FORMAL
SELF-ORGANIZING GROUPS OF
RESEARCHERS THAT COME TOGETHER
TO RUN THINGS LIKE
REPRODUCIBILITY AND JOURNAL
CLUBS AND A RESEARCHER CREATING
COMMUNITIES OF PRACTICE.
THOSE ARE OFTEN LED BY EARLY
CAREER RESEARCHERS AND WE HAVE
INSTITUTIONAL MEMBERS THAT
APPOINT A SENIOR LEAD FOR THE
PERSON TO FOCUS ON THE ROLE.
AND THE MODEL IS IDEALLY EVERY
INSTITUTION WILL HAVE BOTH ROLES
THE LOCAL NETWORK LEAD IS FROM
THE GRASSROOTS COMMUNITY AND
THERE'S THINGS LIKE TRAINING AND
HIRING AND PROMOTION PRACTICES
AND THOSE TWO WORK IN
PARTNERSHIP ADVOCATING FOR EACH
OTHER IN BOTH DIRECTIONS AND
CHALLENGING EACH OTHER AND
ALLOWS MANAGEMENT TO HAVE A
DIRECTOR ROUTE TO THE GRASS
ROOTS COMMUNITY SO THEY CAN HOLD
SENIOR MANAGEMENT'S FEET TO THE
FIRE INSTITUTIONALLY AND THE
STAKEHOLDERS ARE OTHER
ORGANIZATIONS.
WE ENCOURAGE THE LOCAL NETWORKS
TO RUN REPRODUCIBILITY JOURNAL
CLUBS.
WE'RE LOOKING AT PROMOTION
PRACTICES TO INCENTIVIZE
PRACTICES SO THEY CAN MOVE FROM
ONE INSTITUTION TO ANOTHER
WITHOUT A DISADVANTAGE BY
WORKING IN A WAY ONLY TAILORED
IT TO THAT INSTITUTION AND THAT
COORDINATION IS ALSO IMPORTANT
AND WITHIN THE STAKEHOLDER GROUP
WE HAVE PARTNERSHIPS BETWEEN
FUNDERS AND WE'RE RUNNING A
REPORT WITH A MAJOR CHARITABLE
FUNDER AND A SUITE TO LINK THE
GRANT REVIEW PROCESS AS PART OF
THE RELY RACE AND WHEN WE LOOKED
AT THE INTEGRATE WE PUT MANY
LOCAL LEADS DIRECTLY IN TOUCH
KURCHLT RISE IN TERMS OF WHAT
THEY FELT WAS NEEDED FROM THAT
COMMUNITY.
NOW WE HAVE LOCAL NETWORKS AT
ABOUT 60 INSTITUTIONS ACROSS THE
U.K. AND HAVE INSTITUTIONAL
MEMBERS AT 25 INSTITUTIONS AND
THAT NUMBER IS GROWING ALL THE
TIME AND THE KEY FEATURES IS
IT'S A PEER-LED CONSORTIUM AND
WE TRIED TO BE RELEVANT ACROSS A
BROAD RANGE AND THEY'RE LARGELY
WITHIN THE LIFE SCIENCES.
IN TERMS OF THE STAKEHOLDER
ENGAGEMENT GROUP THESE ARE PART
OF THE ORGANIZATIONS PART OF THE
STAKEHOLDER GROUP.
A RANGE OF DIFFERENT
ORGANIZATIONS THAT INCLUDE
ORGANIZATIONS WITH PROFESSIONAL
SERVICES STAFF AND WE RUN
ACTIVITIES WITH DIFFERENT
ORGANIZATIONS AND PRESENT OUR
COMMUNITY AND OUR WORK THEIR
COMMUNITY AND VICE VERSA AND TO
DELIVER HARMONIZED TRAINING.
WE DELIVER OPEN RESEARCH
TRAINING ACROSS OUR
INSTITUTIONAL MEMBERS AND 25
JOINED U.K. RN FORMALLY AND THAT
IS A TRAIN THE TRAINER MODEL SO
THE TRAINING WILL ALLOW FOR
INTEROPERABILITY BECAUSE THE
TRAINING RECEIVED AT YOUR
INSTITUTION EVEN IF IT'S
TAILORED TO YOUR INSTITUTIONAL
CONTEXT WILL RETAIN COMMON DNA
WITH THE TRAINING ACROSS THE
OTHER FIVE INSTITUTIONS.
AGAIN, PROMOTE A COMMON APPROACH
TO SOME OF THESE FOUNDATIONAL
SKILLS WE WANT TO ENCOURAGE ALL
RESEARCHERS TO ENGAGE IN.
AND THE FINAL PIECE OF THE
JIGSAW IS THE ISSUES ARE NOT
SPECIFIC TO OR LOCATED WITHIN
ANY ONE COUNTRY AND WANT TO WORK
COLLABORATIVELY ACROSS
COUNTRIES.
WE'VE SEEN OTHER COUNTRIES ADOPT
THE STRUCTURE AND HAVE
REPRODUCIBILITY ACROSS AUSTRALIA
AND BRAZIL AND REGULARLY CONVENE
MEETING OF THE NETWORKS SO IS
THE COLLABORATION AND CREATION
OF COMMUNITIES OF PRACTICE CAN
HAPPEN AT AN INTERNATIONAL SCALE
AS WELL AS NATIONAL SCALE.
THANK YOU.
>> FABULOUS, MARCUS.
I HOPE WHAT YOU'VE GIVEN THE
AUDIENCE A CHANCE TO RECOGNIZE
IS THE INFRASTRUCTURE THAT
YOU'VE CREATED OVER UKRN BUT
ALSO THE EMPOWERMENT THAT IS
SPREAD THROUGH ALL THAT AND
QUITE A LOT OF OF THE LINK
THROUGH THROUGH THE SECTOR AND
LED INTO THE LOCAL.
THAT'S PROBABLY CHALLENGING.
I THINK THE DISCUSSIONS IN THE
LAST DAY OR SO TALKING ABOUT THE
NIH DOING SOMETHING OR PUTTING
SOMETHING IN PLACE AND THEN HOW
PEOPLE ARE THEN SUPPOSED TO
RESPOND TO THAT WHERE YOU GET
THE SENSE THERE ARE OTHER
DYNAMIC MODELS AND I WOULD
IMAGINE THE NIH WOULD BE QUITE
INTERESTED IN PLAYING AN
INTERESTING ROLE.
I WOULD TAKE A MOMENT BECAUSE
MARCUS THANKFULLY MENTIONED THE
U.K. COMMITTEE ON RESEARCH
INTEGR
INTEGRITY.
I'M MORE THAN HAPPY TO TALK
ABOUT THAT COMMITTEE.
IT'S A NATIONAL COMMITTEE FROM
FROM THE COMMUNITY THAT
REPRESENTS ACROSS PEOPLE ACROSS
MULTIPLE CAREER STAGES AND
PERSPECTIVES INCLUDING TECHNICAL
STAFF AND LOCATIONS AND WHERE
THEY'RE POSITIONED.
THAT IS ALSO SOMETHING THAT A
FUNDER IS INTERESTING IN TRYING
TO SUPPORT WHICH ISN'T
NECESSARILY ABOUT MONEY OR POST
OR GUIDELINES AND GUIDELINES AND
EXPECTATIONS.
THANK YOU FOR ALLOWING ME TO
TAKE A MOMENT, NAOMI.
I'LL TURN IT OVER TO YOU FOR OUR
FINAL PRESENTATION BEFORE WE
HAVE A NICE DISCUSSION BETWEEN
US AND THEN WE'LL TURN IT OVER
TO THE AUDIENCE.
>> THANK YOU SO MUCH.
IT'S A TOUGH ACT TO FOLLOW THESE
THREE AND NOW I'M GOING TO TAKE
US MORE IN THE PAST AND I'M A
GET LATE -- LITIGATOR AND LAWYER
BY TRAINING MAYBE ONE OF THE FEW
HERE TODAY AND MY CASE LOAD WAS
FALSE ADVERTISING CASES AND MY
JOB WAS TO IDENTIFY THE FLAWS
AND THE DATA COMPANIES WERE
USING TO SUPPORT THEIR
ADVERTISING CLAIMS EITHER OUR
OWN CLIENTS OR OUR CLIENTS'
COMPETITORS.
THAT'S HOW I GOT INTERESTED IN
THE QUESTION OF RESEARCH QUALITY
AND CAME TO MY WORK AT COLUMBIA
WITH A REAL COMMITMENT IN A LESS
ADVERSARIAL THINKING.
SO A LITTLE BIT ABOUT MY
INSTITUTION.
WE'RE BIG AND INCREDIBLY DIVERSE
AND DECENTRALIZED.
MANY OF US THAT COME FROM BIG
INSTITUTIONS SHARE AN IMPORTANT
CHAMPION WITHIN OUR INSTITUTIONS
IS THE SENIOR RESEARCH OFFICER
AND MOST INSTITUTIONS NOW AT
LEAST MOST HAVE SOMEONE IN THIS
ROLE AT COLUMBIA.
IT'S CALLED THE EXECUTIVE VICE
PRESIDENT FOR RESEARCH AND SEE
THAT PERSON'S ROLE.
MISS IS MY BOSS, THIS IS WHO I
REPORT TO AND IMPORTANT TO HAVE
THAT SUPPORT AND YOU CAN SEE IN
2017 THE THEN EVP FOR RESEARCH
MICHAEL PERDY AND I WROTE IN
EDITORIAL HOW INSTITUTIONS
NEEDED TO DO MORE.
IT'S HARD.
IT'S HARD FOR BIG INSTITUTIONS
LIKE MINE TO DO MORE.
WE CALLED FOR LOTS OF THINGS IN
THIS.
I WOULDN'T SAY WE'RE DOING THEM
ALL.
WHERE DID MY PARTICULAR ROLE
HERE COME FROM?
ONE OF MY OTHER HATS IN COLUMBIA
IS I'M THE RESEARCH INTERYY --
INTEGRITY OFFICER AND IF THE
UNIVERSITY IS GOING TO MAKE A
FINDING OF RESEARCH MISCONDUCT
THE EVP FOR RESEARCH IS THE
FINAL ADJUDICATOR.
THAT PERSON WILL READ EVERY
REPORT OF RESEARCH AND THAT'S
MORE THAN RESEARCH MISCONDUCT.
THERE'S SO MANY OTHER KINDS OF
PROBLEMS THAT EMERGE IN THOSE
INVESTIGATIONS YOU SEE ONE HERE.
WE HAD ONE CASE AT COLUMBIA A
PARTICULARCATION AND WE HAD AN
AD HOC COMMITTEE THAT CARRIED
THROUGH THE MASSIVE
INVESTIGATION.
AND THEY BECAME CHAMPIONS OF THE
IDEA THE INSTITUTION NEEDED TO
DO SOMETHING TO PREVENT THIS.
TOGETHER WE MADE THE CASE TO
LEADERSHIP AND EVENTUALLY
LEADERSHIP AGREED AND THEY
FUNDED A POSITION IN MY OFFICE.
ONE POSITION WHICH HAS HAD TO
TAKE ON OTHER THINGS AS OUR
MISSION CREEPS AND CREATED A
PROGRAM AND WAS NOT AS
SUCCESSFUL AS REPRODUCIBILITY
AND LAUNCHED THE PROGRAM WITH
THREE PRIMARY FOCUSES TO CREATE
RESOURCES THAT WOULD ASSIST IN
RESEARCH QUALITY.
TO DO OUTREACH WORK SHOPS,
TRAININGS, FACULTY ORIENTATIONS,
GRADUATE STUDENTS, ETCETERA AND
TO PROVIDE CONSULTS FOR
INDIVIDUAL P.I.s.
WE WERE ASKED AT THE OUTSET TO
INCLUDE A DATA AUDIT COMPONENT.
WE DEVELOPED A WHOLE PROGRAM AND
IN FACT THERE'S SOME LITERATURE
THAT SUPPORTS DATA AUDITS WOULD
BE A WAY TO CONTAIN RESEARCH
MISCONDUCT.
WE ASK DID YOU IMPLEMENT THE
DATA AUDITS AND SAID NO BECAUSE
THE CULTURE WAS NOT ARE THE TO
EMBRACE IT.
WE DEVELOPED THIS PROGRAM AT
COLUMBIA TO DO DATA AUDITS AND
ASKED LEADERSHIP AND INDIVIDUAL
CHAIRS AND P.I.s TO LOOK AT YOUR
DATA AND NOBODY TOOK US UP ON
IT.
IS THAT A SURPRISE?
PERHAPS NOT.
BUT I DO THINK THAT IT'S IN
TERMS OF CULTURE AND NORMS I
THINK IT'S SOMETHING THAT TAUGHT
ME SOMETHING.
WE GRADUALLY EVOLVED AND OUR
FUNDING AGENCIES HELP US EVOLVE.
WHEN ANNOUNCED THE RIGOR IT GAVE
US THE ABILITY TO GO AROUND AND
WE HAVE $6 MILLION OF NIH
FUNDING AND 1,000 P.I.s AND
WANTED TO KNOW WHAT THEY CAN DO
TO MAKE THEIR GRANTS MORE
FUNDABLE AND WE CAN GIVE THEM
INFORMATION ON THAT AND IN THAT
MOMENT DEUCE THEM TO DATA
RESOURCES AND MATERIALS WE
PROVIDE.
I WANT TO HIGHLIGHT TWO
INCREDIBLE PEOPLE THAT DID A LOT
OF THE WORK.
WE BUILT A WEBSITE.
ALL THE RESOURCES ARE AVAILABLE.
WE TRIED TO ORGANIZE IT BASED ON
THE RESEARCH LIFE CYCLE STARTING
WITH WHEN SOMEONE IS
ESTABLISHING A RESEARCH GROUP
AND ADDRESSING LOTS OF DIFFERENT
ISSUES INCLUDING REPRODUCIBILITY
RESOURCES FROM TOPIC WE COLLECT
FROM LITERATURE.
COLUMBIA SPECIFIC RESOURCES AND
YOU CAN FIND A STATISTICIAN TO
HELP YOU AND VARIOUS TUTORIALS
AND TEMPLATES AND A LOT OF THE
RESOURCES WERE DRIVEN BY THINGS
WE HAD OBSERVED.
WE OBSERVED IN OUR RESEARCH
MISCONDUCT CASES THERE WAS A
PROBLEM WITH LAB NOTE BOOKS.
THIS IS NOT NEWS TO ANY OF US
BUT WE CREATED A TRAINING
SPECIFICALLY ON GOOD LAB
NOTEBOOK PRACTICES AN ONLINE
TRAINING.
IT'S NOT REQUIRED.
IF YOU BUILD IT WILL THEY COME
YOU SAW 500 COMPLETIONS.
ANOTHER ON COLLECTING DATA WHEN
USING INSTRUMENTATION AND
SLOPPINESS WHERE WE DEVELOPED
TRAINING THERE ORGANIZATION OF
SAMPLES IN A LABORATORY AND THE
SOIL SAMPLES AT ISSUE WERE IN
PLASTIC BAGS BUT NOT
PARTICULARLY LABELLED.
THE ONE THAT WE'RE WORKING ON
NOW IMAGE MANIPULATION AND
DETECTION WHICH IS INSPIRED AND
A NUMBER OF PROBLEMS WITH
WESTERN BLOTS.
THE CHECKLIST WE SAW ISSUES WE
TRIED TO CREATE CHECKLISTS THAT
WOULD HELP.
WE CREATED A CATALOG OF THE DATA
STORAGE OPTIONS AT COLUMBIA.
WE CREATED A WEBSITE ON DATA AT
COLUMBIA SO RESEARCHERS COULD
HAVE A ONE-STOP SHOP TO FIND THE
INFORMATION THEY NEEDED AS TO
DATA AND WE COLLABORATE A LOT
WITH OUR LIBRARIES AS WE HEARD
EARLIER WITH OUR I.T. DEPARTMENT
AND PART OF THAT COLLABORATION
INVOLVED MAKING REAL TOOLS
AVAILABLE FOR RESEARCHERS AND
SOME ARE FREE AND SOME ARE NOT.
OUR LAB BOOK IS FREE AND WE'VE
HAD PRETTY GOOD UPTAKE ON THAT
AND THEN OTHER SOLUTIONS HERE AS
WELL.
THE OUTREACH AND CONSULTATIONS
RUN THE GAMUT.
WE'RE TRYING TO TARGET EVERY
RELEVANT AUDIENCE WE CAN REACH
OR INVITE OURSELVES TO.
WE ARE INVOLVED WITH CARPENTRIES
AND DOING RIGOR S AND THE
CONSULTATIONS ARE INDIVIDUAL
P.I. LEVEL AND SOMETIMES WE GET
REFERRALS.
SO A VICE DEAN FOR RESEARCH WILL
SAY GO TALK TO SO AND SO AND
OTHER TIMES PEOPLE COME TO US
WITH QUESTIONS OR CONCERNS WE'RE
GOING TO PILOT HOPEFULLY A NEW
PROGRAM A BADGE PROGRAM FOR
P.I.'S AND LABS THAT MEET
CERTAIN DATA MANAGEMENT CRITERIA
AND WILL GET MONETARY
UNRESTRICTED FUNDS AS PART OF
THE INCENTIVE.
WE ALSO HELD THE SYMPOSIA AND
THEY'VE BEEN GREAT WAYS TO CON
SEEN OUR COMMUNITY TOGETHER AND
RESEARCH TECH TO SENIOR P.I.s TO
COME TOGETHER FOR A DAY TO THINK
ABOUT THESE ISSUES.
THAT'S KIND OF WHAT WE'VE BEEN
DOING BUT WHERE DO WE GO FROM
HERE.
HOW DO WE INCREASE UPTAKE?
WIDE BRIDGES IS PART OF THAT AND
THE COLLABORATIONS ARE PART OF
THAT.
I THINK NORMS IS A HUGE PART AND
WE'RE ONLY AT THE BEGINNING OF
THE WORK AND THERE'S A LONG WAY
TO GO.
I'LL STOP THERE.
>> IT'S AS IF IMAGE -- I
MAGICALLY DID THAT.
BRILLIANT TALKED AND HOPE YOU
FEEL ENERGETIC.
YOU DON'T LOOK IT.
I JUST WANT YOU TO KNOW, YOU ALL
LOOK REALLY TIRED.
HERE'S THE DEAL.
WE WERE GOING TO HAVE YOU NOT
BREAK AND MOVE DIRECTLY INTO
QUESTION AND ANSWERS BUT I THINK
I MIGHT NEED TO LET YOU GO DO
SOME JUMPING JACKS FOR A FEW
MOMENTS BEFORE WE COME BACK.
I HAVE A QUESTION FOR EACH ONE
OF THEM THAT'S GOING TO
STIMULATE THINKING HOPEFULLY AND
I'M LIMITING MYSELF TO ONE AND
LIMIT THEM TO ANSWER ONE AND
WE'LL GIVE YOU A BREAK BUT A
STRETCH YOUR LEGGY TYPE OF THING
AND STANDING UP OR CHATTING TO
THE PERSON ACROSS THE WAY.
FIVE MINUTES VERONIQUE?
WE'LL KEEP IT SHORT.
THIS IS A LITTLE BIT OF AN
ENERGY BREAK.
IF YOU NEED TO DO SOME YOGA IN
THE BACK, WHATEVER YOU NEED TO
DO TO GET IT TOGETHER AND THEN
WE'LL HAVE OUR PLENARY AND GO IN
THE ORDER OF HOW EVERYONE HAS
SPOKEN TODAY WHICH HAS BEEN
GREAT.
SOFIA, OVER TO YOU FIRST.
I WAS INTERESTED IN THE LINE
ABOUT LOOKING FOR COMMUNITY AND
WHAT MOTIVATED YOU BECAUSE I'M
FASCINATED BY WHAT THAT
COMMUNITY HAS BECOME AND HOW IT
SPREAD.
ALSO THE PRACTICE THAT YOU'VE
CREATED.
THIS PRACTICE OF DOING NOT JUST
THE RIGOR WORK BUT THE WORK OF
CREATING COMMUNITY.
WOULD YOU SPEAK A LITTLE BIT
ABOUT THAT.
>> MOSTHE THIS
LOCAL LEVEL.
MY EXPERIENCES COME FROM THE
THREE LOCAL LEVELS I'VE
EXPERIENCED AND AT SOME POINT WE
LIKE THE WHOLE READING PAPERS
THINGS BECAME A GUISE FOR
EVERYONE MEETING AND EXCITING
SHARING OF THINGS WE'VE SEEN.
>> IT WAS FANTASTIC.
WE NEVER KNEW IT WAS GOING TO
HAPPEN BUT WE WERE HAVING TEA
AND EATING BISCUITS AND EVERYONE
TOOK THE TIME OUT IN THEIR WEEK
TO DEDICATE TO THAT.
>> THAT'S LOVELY.
AN AGENDA, PROBABLY SOMETHING
FOR US TO THINK ABOUT AS WE'RE
THINKING OF THIS COMMUNITY,
RIGHT?
WHAT THE NEXT STEPS MIGHT
ULTIMATELY BE.
I'M GIVING YOU SOME HINTS, STEVE
AND VERONIQUE.
WE WANT FUN AND FRIVOLITY.
FRIV-OL-O-TEA.
WE'RE ON FIRE.
YOU'RE GOING THROUGH THE WAY THE
HEALTH ALLIANCE HAS BEEN
THINKING AND GROWING.
I HAVE A HARD QUESTION FOR YOU.
I'M CURIOUS BASED ON ALL THE
KNOWLEDGE YOU'VE HAD NOW.
WHAT DO YOU WISH YOU WOULD HAVE
KNOWN FIVE YEARS AGO YOU KNOW
NOW ABOUT WORKING STRATEGICALLY
ACROSS THE VARIOUS DIFFERENT
GROUPS?
>> THE RIGHT THING IS NOT
NECESSARILY OBVIOUS.
WHAT I THINK MISSION, PASSIONS,
EMOTION PLAYS SO MUCH INTO THE
DECISIONS OF THE LEADERSHIP
ESPECIALLY PATIENT-BASED
ORGANIZATIONS.
IT'S HARD TO CONVINCE PEOPLE TO
CHANGE BEHAVIORS WHEN THINGS ARE
TO THE HEART.
SUCCESS IS HUGE AND IN THAT WAY.
>> BRILLIANT.
THANK YOU.
MARCUS I'M GOING TO IMPOSE
BECAUSE I KNOW THINGS.
IT WOULD BE HELPFUL TO
UNDERSTAND THE SCALE OF UKRN AND
WHAT DOES THAT FEEL LIKE AND
LOOK LIKE BEING AT THE FOREFRONT
OF SCALE.
>> A BIG CHALLENGE WAS MANAGING
GROWTH AND WE WAITED UNTIL THE
TIME WAS RIGHT.
WE WERE DOING A LOT OF LOBBYING
WITH FUNDERS AND OTHER
ORGANIZATIONS AND MANAGED TO GET
THEM IN THE END OF THE ROOM
AROUND SEPTEMBER 2018 AND GOT
THE BUY-IN TO LAUNCH AND OUR
STRATEGY WAS TO ASK FOR A LARGE
AMOUNT OF MONEY FROM A LARGE
AMOUNT OF FUNDERS AND KNEW THERE
WAS GRASSROOTS ACTIVITY AND MANY
EARLIER CAREER RESEARCHERS WHO
WERE PUSHING FOR CHANGE, IF YOU
LIKE AND WE WERE ABLE TO HARNESS
THAT.
THE GROWTH INITIALLY WAS IN
LOCAL NETWORKS BRINGING TOGETHER
COMMUNITIES AND USED THAT
FUNDING TO SUPPORT ANOTHER
INITIATIVES THAT WERE ALIGNED
LIKE REPRODUCIBILITY AND SEND
OUT TEA AND BISCUITS AND
ARTICLES AND IT CREATED BUZZ ON
TWITTER AND FED INTO A POSITIVE
FEEDBACK LOOP.
WE LINED UP A SMALL NUMBERS BUT
WAITED UNTIL WE HAD CRITICAL
PASS.
I THINK WE HAD A COMMENT IN
NATURE TO ANNOUNCE WE HAD THAT
TOGETHER.
THERE WAS A WAITING UNTIL THE
TIME WAS RIGHT.
WHEN THERE WERE 10 UNIVERSITIES
OTHERS WANT TO BE PART OF THE
CLUB AND GOT OTHERS TO JOIN.
THAT'S WHERE WE STARTED TO
MANAGE THE GROWTH AND THE
STRATEGY WAS ALWAYS TO HAVE THE
FUNDING FOR THREE YEARS AND TO
GET MORE SUBSTANTIAL
PROGRAMMATIC FUNDING THAT WOULD
THEN EXTEND BEYOND THAT AND GOT
THAT FIVE YEARS FROM RESEARCH
WHICH IS PART OF UKRI WHICH IS
THIS OPEN RESEARCH PROGRAM.
WE NOW HAVE MORE SUSTAINABLE
PUTTING AND THE NEXT CHALLENGE
WILL MOVE TO A MORE SUSTAINABLE
MODEL OF FUNDING SO WE DON'T
HAVE TO GO BACK FOR THAT
SUPPORT.
>> THANK YOU FOR THAT MARCUS.
I KNEW I WASN'T PUTTING YOU ON
THE SPOT TO DISCUSS THAT BECAUSE
THAT GROWTH AND MODEL OF HOW TO
SCALE AND THEN BEING STRATEGIC
ABOUT IT WILL BE A POWERFUL
CONSIDERATION MOVING FORWARD FOR
THIS GROUP.
NAME -- NAOMI, FROM WHAT YOU
DESCRIBED YOU WERE VERY TACTICAL
ABOUT HOW THE OFFICE WAS GOING
TO WORK AND HOW YOU MIGHT
INTERACT ACROSS SOMETHING AS
LARGE AS THE SYSTEM AT COLUMBIA.
TALK ABOUT IMPLEMENTATION.
HOW MUCH HAS YOUR OFFICE TRIED
TO DEVOLVE IN THE LOCAL LEVEL.
WE TALKED ABOUT IMPLEMENTATION
AMONG THIS GROUP AND TALKED
ABOUT IT IN THE NICE ABSTRACT
WAY LIKE IT MAGICALLY HAPPENED
BY SOMETHING UP THERE WHO WILL
DO SOMETHING.
YOU'RE CLOSER TO THAT AT THAT
MICROLEVEL.
>> SO IMPLEMENTATION.
WE DID A FAIR AMOUNT OF
PLANNING.
WE PLANNED AND DOCUMENTED AND WE
HAD A PROJECT PLAN THAT LAID OUT
WHAT OUR OBJECTIVES WERE AND GOT
FEEDBACK ON IT FROM CHAMPIONS
AND LOTS OF PLACES INCLUDING THE
FACULTY COMMITTEES.
WE KIND OF FORMED A LITTLE
ADVISORY COMMITTEE THAT INCLUDED
MEMBERS OF A FEW COMMITTEES THAT
HAVE BEEN INVOLVED.
SO THEY HELPED US AND
RELATIONSHIPS PLAY A HUGE ROLES.
ONCE WE DEVELOPED A PLAN WE
NEEDED TO PARTNER WITH LOTS OF
PEOPLE IN OUR UNIVERSITY.
I NEEDED THE ASSOCIATE DEAN TO
GIVE US A TIME SLOT AND SAME WAS
TRUE FOR NEW FACULTY
ORGANIZATIONS AND THEY NEEDED TO
KNOW THEM TO REACH OUT TO THEM
TO SAY HEY, I KNOW WE'VE GOT TEN
NEW ENGINEERING PROFESSORS
COMING END AND I'D LOVE 30
MINUTES OF YOUR ORIENTATION FOR
THEM.
IT'S A LOT OF RELATIONSHIPS AND
A LOT OF NETWORKING AND THEN
HAVING THE OBJECTIVES IN MIND OF
WHERE WE'RE HOPING TO GET TO AT
THE SAME TIME AND ALWAYS HAVING
A VALUE TO ADD.
IT'S NOT ABOUT -- THIS PROCESS
HAS NEVER BEEN ABOUT COMPLIANCE.
THIS PARTICULAR PIECE OF MY ROLE
HAS NEVER BEEN ABOUT COMPLIANCE.
IT'S HOW CAN YOU MAKE THE BEST
USE OF YOUR TIME AND WHAT
RESEARCH AND WHAT CAN YOU OFFER?
>> YOU HELP US GET BACK TO THE
QUESTIONS OF VALUES WE WERE
TALKING ABOUT OVER THE LAST FEW
DAYS.
THE POWER OF COLLABORATIONS AND
NETWORKS AND STAYING QUITE
FOCUSES NOT ONLY THE COMPLIANCE
BUT THE BRILLIANT WORK WE WANT
TO ENCOURAGE AND ENABLE OUR
COMMUNITIES TO PREPARE AND
PRODUCE.
ALL RIGHT, YOU GET FIVE MINUTES.
MAKE THE MOST OF THIS AND WE'LL
SEE YOU BACK AT JUST BEFORE A
QUARTER 'TILL.
>>THANKS TO THE PANEL NOR GREAT
PRESENTATIONS AND KAREN THANKS
FOR THE WONDERFUL JOB LEADING
THE GROUP.
MY QUESTION IS TARGETED AT
MARYROSE FRANCO.
SO I WAS INTERESTING TO SEE THE
WORK OF HELIOS AND WANTED TO SEE
IT AS A COLLECTIVE PROBLEM AND
MY HOME INSTITUTION THE
UNIVERSITY OF WASHINGTON IS NOT
LISTED SO WANTED ADVICE AND THE
LEADERS AT THE INSTITUTION.
DO I E-MAIL, THE PRESIDENT, THE
PROVOST OF THE UNIVERSITY?
WHAT DO I DO?
>> THAT'S A LOCAL QUESTION.
I DON'T KNOW WHO THE CHAMPIONS
ARE AT YOUR UNIVERSITY AND THOSE
PEOPLE SHOULD RUN IT UP THE FLAG
POLE.
ALSO IT'S A LOT OF PEER
PRESSURE.
YOU CAN CONTACT THE HELIOS STAFF
AND I CAN E-MAIL YOU.
WHAT'S HELPED IS THE FUNDERS
E-MAIL DIRECTLY SAYING WHY
AREN'T YOU PART OF OUR GROUP?
DANIELS FROM HOPKINS OR WHOEVER
HOLDS SWAY SAY WE NEED YOU TO BE
PART OF THE HELIOS.
YOU CAN E-MAIL ME AND A CAN SET
YOU UP.
THOUGH THE PRESIDENTS MAY BE THE
INFLUENCES MARRY ROSE IS THE
WIDE BRIDGE.
>> ONE THING THAT'S WORKED WELL
FOR US AT THE UNIVERSITY OF
FLORIDA IS WE HAVE A LIST OF OUR
PEER COMPETITOR UNIVERSITIES AND
ONCE WE TELL SOMEONE IN
ADMINISTRATION, OH, THE U.C.
SYSTEM IS DOING X, Y, Z, THEN
SUDDENLY THE WHEELS START
TURNING TO GET THAT IMPLEMENTED
FOR US.
>> A QUESTION COMMENT BEFORE THE
QUESTION.
I LOVE THE WORK NAOMI'S DONE AT
COLUMBIA.
IT'S A FABULOUS SOLUTION FOR A
VERY IMPORTANT PROBLEM AND COULD
BE IMPLEMENTED AND SCALED UP.
A QUESTION TO MARY ON HOW TO
CREATE MUTUALLY ENFORCING
VECTORS.
I'M NOT SO SURE.
IT FEELS LIKE COMPETING VECTORS
SOMETIMES AND YOU SUGGESTED WE
IDENTIFY COMMON INTERESTS ACROSS
STAKEHOLDERS.
YOU SAY IT MUST BE A WIN-WIN BUT
IT'S OFTEN THE CASE IT'S NOT A
WIN-WIN.
THAT IS SOME POWERFUL FORCES
INVOLVED.
WHEN BIG POLICY CHANGES ARE
COMING FROM INSTITUTES LIKE NIH.
I HEARD FROM SHAW ABOUT
UNIVERSITIES LOBBY AGAINST THE
AGENCIES BECAUSE THEY HAVE THEIR
OWN INTERESTS TO PROTECT.
I WAS WONDERING IF YOU CAN
COMMENT ON HOW YOU GO ABOUT
ACHIEVING FUNDAMENTAL REFORMS.
NOT SMALL SIMPLE THINGS BUT
SOMETHING FUNDAMENTAL LIKE RIGOR
INTO MOST STAGES OF RESEARCH.
THANK YOU.
>> I STILL BELIEVE THE ONLY WAY
YOU'RE GOING TO -- SOMEONE IS
WILLING TO MAKE A HUGE BEHAVIOR
CHANGE IS IF IN SOME WAY IT
DOESN'T HARM THEM.
YOU NEED IT MAKE SURE THAT
CHANGE IS IN THAT STAKEHOLDERS
OR INDIVIDUAL'S INTEREST.
IT'S HARD TO GET THEM TO DO
SOMETHING THAT WILL CAUSE HARM.
>> I THINK THE NEGATIVE
RETURN -- AND YOU KNOW THERE
WILL BE PUSHBACK FROM THE HIGHER
UPS AND UNIVERSITIES --
>> TO DO WHAT?
>> ADVOCATING AGONISTS
PUBLISHING IN GLAMOUR JOURNALS
AS A WAY TO SHOW IMPACT AND
PROGRESS, SO TO SAY.
>> I'M GOING TO HELP US FOCUS ON
YOUR QUESTION BECAUSE I THINK
WHAT WE'VE BEEN TRYING TO GIVE
IS NOT CONCRETE TO ONE SOLUTION
BUT EXAMPLES OF HOW FAR YOU
MIGHT BREAK STUFF DOWN AND HOW
YOU MIGHT GROW AND ACTUALLY LINK
UP AND IT MAY BE THAT WE'VE GOT
PUBLISHES IN THE ROOM HAVE YOU
CHATTED TO A FEW WHAT CAN YOU DO
TO BRING THE NETWORKS TOGETHER?
I'D GO BACK TO THE POINT MARCUS
RAISED, THE REPRODUCIBILITY
NETWORK AND REPRODUCIBILITY HAVE
BEEN FUNDED BY A RANGE OF
ENTITIES INCLUDING PEOPLE GIVING
THEIR TIME AND BY UKRI WITHOUT
IT BEING A MANDATE AND POLICY
CHANGE.
THAT'S NOT AN EXCELLENT TO BE
ABLE TO CONTRIBUTE.
IT'S PROBABLY SOMETHING TO
CONTINUE TO THINK STRATEGICALLY
HOW WE MIGHT PACKAGE SOME OF
THIS UP MOVING FORWARD IN TERMS
OF THE TYPE OF CHANGE I THINK
TAKING ON MALCOLM'S POINT.
>> IT'S NOT -- WHAT YOUR
INTELLECTUAL IMPACT TO THAT
DISCOVERY.
WE'RE MOVING TO THAT DIRECTION
AND I THINK RESEARCHERS CAN TAKE
ADVANTAGE.
>> WE HAVE EVERYTHING THE RESUME
FOR RESEARCHERS AND INNOVATORS
BUT WORKING COLLABORATIVELY WITH
OTHERS ON THE NARRATIVE CVs
MOVING FORWARD BUT HOLD ON THAT
TO THAT BECAUSE THERE'S A PLACE
FOR THAT IN OUR NEXT
CONVERSATION.
WE'LL MOVE TO THE LEFT AND THEN
RIGHT.
>> KAREN, YOU MENTIONED
SCALABILITY AND WE SCALE THE WAY
AND YOU SHOWED IT NICELY PUSHING
OUT FURTHER TO MORE AND MORE
NETWORKS.
WE KNOW THERE'S CONNECTIONS
BETWEEN THE STAKEHOLDERS THERE
AND WE ALL KNOW WE CAN WRITE ON
OUR BOARDS AND USE WORDS LIKE
RIGOR AND REPRODUCIBILITY AND
OPEN SCIENCE THAT HAVE MANY
ELEMENTS THAT CUT ACROFT.
I'M WONDERING IF YOU'VE HAD BOTH
RESISTANCE TO TRY TO FIND LIKE
THE PLANTING SEED WITHIN A GIVEN
COMMUNITY TO ALLOW IT TO SCALE
ACROSS COMMUNITIES SAY DATA.
DATA IS ONE YOU CAN LATCH ON TO
AND DOES THAT START TO OPEN
DOORS TO OTHER CONVERSATIONS?
IF I GET COMMUNITY ENCOURAGEMENT
ON OPEN DATA AND ALL OF A SUDDEN
YOU KNOCK DOWN A BARRIER.
WONDERING IF YOU CAN REFLECT ON
ANY OF THAT.
I THINK THAT'S ANOTHER ELEMENT
OF SCALABILITY.
YOU DON'T HAVE TO DO IT.
YOU CAN START TO CONNECT THE
BRIDGES TO ORGANIZATIONS.
>> MARCUS, DO YOU WANT TO DO?
>> YOU NEED TO CREATE STRUCTURED
APPROACHES FLEXIBLE ENOUGH TO BE
RELEVANT TO A BROADER RANGE AND
WE WERE TALKING ABOUT HOW IN THE
U.S. AND THE CONVERSATION
BETWEEN SESSIONS MAYBE THEY NEED
TO USE BEER RATHER THAN TEA AND
OTHERS DRINK COFFEE NOT TEA.
WITHIN UKRN WE HAVE THE
UNIVERSITY OF OTHERS WHICH IS
MORE TEACHING INTENSIVE SO A
DIVERSE RANGE OF INSTITUTIONS
WITH DIFFERENT NEEDS AND LOCAL
STRUCTURES AND THAT'S TRUE AT
THE NATIONAL LEVEL IN THE
CONTEXT WHAT YOU'RE TALKING
ABOUT AND THERE ARE A SUITE OF
OPEN RESEARCH PRACTICES IF IT'S
GOING TO BE SOMETHING WE DECIDE
TO FOCUS ON BUT OUR ATTITUDE HAS
BEEN PICK A PLACE TO START.
DATA AND WHAT IS CLOSEST TO HOW
YOU ALREADY WORK IT'S ABOUT
RETAINING THE FLEXIBILITY TO
MAKE WHAT WE DO RELEVANT AND
TAILOR IT TO THEIR LOCAL CONTEXT
AND REDUCING THE BARRIERS TO
ENTRY AS MUCH AS POSSIBLE.
>> NAOMI OR SOFIA WOULD YOU LIKE
TO ADD?
>> I'D LIKE TO MAKE A PLUG FOR
DATA MANAGEMENT AS A PLACE TO
START BECAUSE IT'S APPLICABLE TO
EVERY FIELD AND SOMETHING EVERY
RESEARCHER CARES ABOUT.
RIGOR CAN IMPLY JUDGMENT.
YOUR RESEARCH ISN'T RIGOROUS
ENOUGH.
DATA MANAGEMENT, NOT SO MUCH
JUDGMENT OF THE SAME KIND LESS
VALUE JUDGMENT MORE FOCUS ON
PROCESS I THINK THAT CAN BE
USEFUL IN THE KIND OF
ENVIRONMENT WE'RE IN.
>> FIRST THANK YOU FOR WONDERFUL
PRESENTATIONS YOU DID THIS
AFTERNOON.
I WANT TO ACKNOWLEDGE YOUR
COMMITMENT TO WHAT WE ARE ALL
HEAR TO TALK ABOUT TODAY.
IN PARTICULAR I WANT TO
CONGRATULATION SOPHIA FOR THE
INITIATIVE.
I THINK STUDENTS AND THE NEW
GENERATION KNOW WHAT WE SHOULD
BE AIMING AT AND WE'RE BOTH
LEARNING FROM EACH OTHER AND I
WANT YOU TO ENCOURAGE YOU TO
CONTINUE IN THAT PATH.
WE'RE TALKING ABOUT TRYING TO
WHAT I HEAR IS HOW CAN WE COME
TOGETHER?
THERE'S THE WORLD HEALTH
ORGANIZATION?
I'M NOT MAKING A JOKE BUT WE'RE
GATHERED TOGETHER FROM ALL OVER
THE WORLD.
PERHAPS WE HAD PEOPLE FROM ASIA
OR AFRICA BECAUSE WE COULD HAVE
DONE THAT AS WELL BUT AMONG
OURSELVES BUT IN THE TIME OF
DIGITALIZATION MAYBE -- I KNOW
IT'S HARD AND MAYBE WE NEED
LEVERAGE THAT COULD MAKE IT LIKE
A HUB AT LEAST A DIGITAL HUB OF
ALL OF US.
A PLACE WHERE WE CAN CONNECT
BECAUSE WE DON'T WANT A LIST OF
THINGS WE DON'T KNOW WHAT TO DO
WITH BUT SOMETHING THAT MAKES
SENSE TO THE COMMUNITY OF
LEARNERS BECAUSE WE'RE ALL
LEARNERS AT DIFFERENT LEVELS.
>> YOU PUT IT ON THE TABLE WHERE
TO GO NEXT.
>> IF WE SAY WE NEED TO LOOK AT
OUR INDIVIDUAL COMMUNITIES FROM
NEW YORK CITY TO SAN JUAN,
PUERTO RICO BUT THEN AGAIN
THERE'S A LOT OF THINGS THAT
MAKE US COMMON AND WHAT WE
SHOULD BE LOOKING AT.
I KNOW THE EFFORTS OF ORGANIZERS
TODAY ARE AIMED AT THAT BUT I
THINK WE CAN CREATE SOMETHING
THAT WILL BE WORTH OUR WHILE SO
THAT'S WHAT I WOULD LIKE TO
LEAVE IT.
>> WE HAVE THE WRAP-UP, CARMEN
AND IT SEEMS LIKE A GOOD PLACE
TO COME BACK TO SIMILAR TO OUR
EARLIER CONVERSATION WHERE WE
CAN GO NEXT BUT MARRY ROSE AND
SOPHIA DO YOU WANT TO COME IN?
>> A QUICK COMMENT I THINK IT'S
SPOT ON.
WE ALSO HAVE TO RECOGNIZE WE'RE
ADDING RESOURCE NEEDS.
THEY'RE NOT FREEBIE -- FREE
BEHAVIORS AND DON'T WANT TO
DISADVANTAGE THE COMMUNITIESES
WE DON'T WANT TO DISADVANTAGE
AND THAT HAS TO BE AN IMPORTANT
PART OF OPEN SCIENCE AND RIGOR
OF SCIENCE.
>> MARCUS.
>> IT'S A GREAT POINT.
ONE THING WE'RE TRYING TO DO
WITH THE STRUCTURE WE HAVE IS TO
CREATE THESE COMMUNITIES IN
DIFFERENT PARTS OF THE SYSTEM.
RESEARCHERS THEMSELVES AND
SEEING MANAGEMENT OF
INSTITUTIONS SO THERE'S
CHAMPIONS WIN THE DIFFERENT
ELEMENTS OF THE SYSTEM AND
FUNDERS AND PUBLISHERS AND HAVE
A RESIDENTIAL RETREAT FOR
NETWORKS AND RETREAT FOR OUR
INSTITUTIONAL LEAD AND COME
TOGETHER AND SHARE EXPERIENCES
AND DECIDE ON NEXT STEPS.
THE U.K. HAS A NATURAL ADVANTAGE
AND IT'S THAT KIND OF CREATION
OF COMMUNITIES INCLUDING
INTERNATIONAL THAT THINK WILL BE
KEY.
THIS QUESTION IS TO OUR EUROPEAN
PARTNERS HERE WHICH IS MANY
EUROPEANS ARE HERE AND IF YOU
LOOK AT SYSTEMS OF CHANGE AND MY
IMPRESSION IS THE EUROPEAN
MODELS HAVE MOVED FASTER AND
MORE EFFECTIVELY.
WHAT IS IT ABOUT THE U.K. AND
REST OF EUROPE THAT HAS ENABLED
CHANGE TO MOVE MORE QUICKLY
COMPARED TO THE U.S. WHERE WE
SEEM TO BE SORT OF STUCK.
>> WHO WANTS TO JUMP ON THAT
ONE?
>> WE'RE MESSED UP IN MANY WAYS.
EVERYONES MESSED UP IN THEIR OWN
WAY.
THE U.K. HAS NATURAL ADVANTAGES
AND MANY PRINCIPLES BROADLY
APPLY.
THE U.K. AS A RESEARCH ENTITY IF
YOU LIKE PUNCHES ABOVE ITS
WEIGHT AS A RELATIVELY SMALL
COMPANY WITH A GLOBAL FOOTPRINT
IN TERMS OF IT'S RESEARCH AND
BIG ENOUGH TO MATTER AND
INTERCONNECTED ENOUGH BECAUSE WE
HAVE SOMETHING LIKE UKRI THAT
CUTS ACROSS AND HUMANITY AND
SOCIAL SCIENCE.
WE HAVE A RELATIVELY COMPACT
SYSTEM BOTH PHYSICALLY AND IN
TERMS OF STRUCTURE AND AT THE
EUROPEAN LEVEL YOU HAVE EUROPEAN
FUNDING AGENCIES THROUGH THE
EUROPEAN UNION FOR EXAMPLE THE
HORIZON 2020 PROGRAM MANDATES
OPEN RESEARCH.
SOME OF THAT CONNECTIVITY WAS
ALREADY IN PLACE AND WE WERE
ABLE TO HARNESS AND CAPTURE
THAT.
THERE'S ALSO A CULTURAL
DIFFERENCE POSSIBLY AND THIS IS
ME FREE STYLING MORE.
THE U.S. IS A MORE
INDIVIDUALISTIC CULTURE.
I THINK THAT'S TRUE AT AN
INSTITUTIONAL LEVEL.
AND THESE ARE QUITE
COLLABORATIVE APPROACHES WHERE
WE'RE SHARING KNOWLEDGE AND
LEARNING FROM EACH OTHER AND
PUTTING COMPETITION TO ONE SIDE,
IF YOU LIKE IN CONTEXT WHERE THE
COMPETITION DOESN'T HELP
ANYBODY.
ACTUALLY IT'S BENEFICIAL TO
EVERYBODY TO WORK IN THAT
COLLABORATIVE WAY IN AREAS THAT
ARE NOT ZERO SUM, IF YOU LIKE.
MAYBE THERE'S A CULTURAL
DIFFERENCE THERE AS WELL.
>> I THINK YOUR GRAD STUDENTS
ARE FINE.
MAYBE YOU JUST NEED TO EMPOWER
THE GRAT STUDENTS AND UCR.
>> THE AND THERE'S A DIFFERENT
SET OF FUNDERS HAVING DIFFERENT
GOALS AND NEEDS AND THEN HAVING
TO TRY TO ORGANIZE AND
COORDINATE.
THAT MAKES A DIFFERENCE BUT WE
ARE ONLY ONE FUNDER.
THAT MEANS COLLABORATING ACROSS
ENTITIES INCLUDING MANY MARYROSE
LISTED AND WORK WITH NATIONAL
ACADEMIES AND LEARNED SOCIETIES
WHO HAVE BEEN AT THE FOREFRONT
OF A LOT OF THIS WORK AS WELL
WITHIN THE U.K.
THERE'S THINGS TO LEARN AND
THINGS TO SHARE ON BOTH SIDES
ACTUALLY.
ON THAT NOTE WE'RE GOING OVER
HERE.
>> I'M JENNY HIMEBERG AND THE
WORK AT THE NATIONAL ACADEMY OF
SCIENCE, ENGINEERING AND
MEDICINE.
I WANTED TO ACKNOWLEDGE THE
COMMENT ABOUT RIGOR AS SOMETHING
THAT COULD BE OPINION-BASED I
WANTED US TO THINK MORE OF
QUANTITATIVE RIGOR AND DATA IS
AVAILABLE AND THERE'S LOTS OF
PEOPLE THAT TALKED ABOUT OPEN
DATA BEING AN INDICATOR OF
RIGOR.
ANOTHER MIGHT BE STATISTICAL
CHECKS FROM THE OUTPUT OR
VERIFICATION OR VALIDATION OF
THE CODE.
ALL THESE THINGS COULD BE
INCLUDED AND I'M NOT THE FIRST
PERSON TO SUGGEST THIS AS
ACKNOWLEDGEMENT AND AUTHORS OF
PAPER.
YOU HAVE A PAPER AND YOU KNOW
WHO VALIDATED THE CODE.
YOU KNOW WHO MANAGED THE DATA
WHO HAD REPRODUCIBILITY OF THE
WHOLE SET.
BUT IT ALSO GETS TO WORK LOAD.
WHAT'S THE ADDITIONAL WORK LOAD
OF HAVING ALL OF THOSE PROCESSES
DONE FOR THE PAPERS?
TWO POINTS.
HOW CAN WE START MOVING FROM
TALKING ABOUT RIGOR AS AN
ESOTERIC GOAL AS INDICATORS OF
OUR PRODUCTS AND DOES ANYONE
KNOW -- HAS ANYONE DONE AN
EXPERIMENT OR ANALYSIS OF THE
EXTRA WORK LOAD FOR THE
ADDITIONAL THINGS THAT HAVE
RIGOR?
>> CAN WE GO DO NAOMI FIRST AND
IF YOU ANYTHING TO ADD.
>> I THINK IT'S A GREAT COMMENT.
>> I THINK IT'S REALLY IMPORTANT
TO COME UP WITH SOME CRITERIA
THAT WE CAN MEASURE AND SEE THAT
ARE IN SOME WAYS SOMEWHAT
OBJECTIVE.
I THINK GOING BACK TO EARLIER
MEANT AND ONE THING TO BEAR IN
MIND IN THIS COMMUNITY IS A LOT
OF PEOPLE FEEL MY WAY HAS GOTTEN
ME WHERE I GOT AND MY WAY IS
WORKING FOR ME.
WHY SHOULD I CHANGE?
I THINK THAT IMPLIES SOME
ANXIETY.
MAYBE IT WON'T WORK FOR ME AS
WELL AS ANY MORE.
>> WE HAVE SOPHIA AND MARCUS AND
THEN MARYROSE.
>> AND THERE WAS WORKLOAD AND
REPRODUCIBILITY.
I THINK IT'S CHRIS -- CRAZY TO
ME IT SHOULD BE DONE BY SOMEONE
EMPLOYED AT THE JOURNAL FOR
CHECKING REPRODUCIBILITY.
THERE'S EXAMPLES IN ECONOMICS
AND I DON'T UNDERSTAND HOW
THAT'S NOT THE CASE ALL JOURNALS
DO THIS BECAUSE IT'S JUST
QUALITY CONTROL.
I THINK A LOT OF THIS TECHNICAL
WORK LOAD SHOULD BE DONE BY
JOURNALS.
WHY ARE WE PAYING THEM?
>> GOOD QUESTIONS.
THERE'S ALSO SOME TRANSPARENCY
ALLOWING FOR AND WE CAN ALLOW
FOR QUALITY ASSURANCE OVERLAYS
AND THEN THINK OF SPOT-CHECKS ON
PUBLICATIONS AND OUTPUTS AS A
CHECK OF THE SYSTEM RATHER THAN
CHECKING THE THING IN ITSELF.
IT'S THE PROCESS IN THE SYSTEM I
THINK WE NEED TO GET RIGHT.
THE MEANS WE HAVE TO SHIFT TO
WHAT WOULD BE QUANTIFIED ABOUT
THE SYSTEM AND PROCESS.
IT'S IMPORTANT TO QUANTIFY THE
RESOURCES BECAUSE IF WE'RE GOING
TO RESOURCE APPROPRIATELY THE
NEW BEHAVIORS WE NEED TO KNOW
HOW MUCH THEY'RE GOING TO COST.
IT'S INCUMBENT ON FUNDERS SO
IT'S IMPORTANT WE CAPTURE THE
DATA AND WHAT IT'S GOING TO
COST.
>> MARCUS?
>> THAT REMINDS ME OF ONE THING
WORTH MENTIONING.
THE STATISTICIAN TOOK THIS TO
THE INDUSTRY AND TRANSFORMED THE
AUTO INDUSTRY AND QUALITY
CONTROL DURING THE PROCESS IS
INTUITIVE AND YOU ALSO IMPROVE
PRODUCTIVITY.
THAT LED TO THE JAPANESE
AUTOMOBILE INDUSTRY LEAPING
AHEAD AND NOW EVERY INDUSTRY
USING THE SAME APPROACH.
WE CAN MAKE AN ARGUMENT HOW THAT
INVEST WILL PAY BACK AND THE
EFFICIENCY WITH WHICH WE WORK.
>> AND CHALLENGE US TO THINK
ABOUT EFFICIENCY AND EFFICACY
AND ALSO DISPLACING.
MAKING SURE WE'RE NOT DISPLACING
MORE WORK SOME PLACE ELSE WITHIN
THE SYSTEM MOVING FORWARD.
STEVE.
>> MARCUS JUST MADE THE POINT I
WAS GOING TO MAKE.
THERE'S A COST TO THE CURRENT
SYSTEM AND WE'RE PAYING IT.
WE'RE JUST NOT ALWAYS AWARE OF
IT.
I AGREE WITH THAT.
I'LL MAKE TWO QUICK COMMENTS BUT
THEN I HAVE A QUESTION.
ONE IS THE ISSUE ON RIGOR AND
QUANTIFYING IT.
WITHOUT GIVING A LECTURE I'LL
JUST SAY ASSESSING RIGOR IS HARD
AND THERE'S NO WAY AROUND IT.
THERE'S NO QUANTITATIVE MEASURE.
I'LL SAY THAT.
DOES THE QUESTION FIT THE DESIGN
OR THE OUTCOME MEASURE ANSWER
THE QUESTION.
IF YOU DOUBLE THE SAMPLE SIZE
AND ALL STUDIES HAVE 80% POWER.
THE PROBLEM WITH RIGOR IS THAT
IT IS HARD AND IT IS SUBJECTIVE.
THAT IS TRUE AND IT'S AN EASIER
TO MEASURE PROCESS MEASURES AND
ARGUABLY THE ONLY THING WE CAN
DO.
WE STILL HAVE RANDOMIZED CONTROL
TRIALS AND WE HAVE 25 SCORECARDS
AND DIFFERENT ASSESSMENTS.
WE KNOW WHAT MAKES A GOOD RCT
BUT THERE'S NO SYNTHETIC
MEASURE.
HERE'S THE QUESTION.
THIS IS SORT OF TO NAOMI AND
MARYROSE.
WE HAVE A VERY WELL ESTABLISHED
SYSTEM FOR RESEARCH INTEGRATE OF
MONITORING IN THE U.S. AND THERE
ARE PEOPLE LIKE YOU RESEARCH
INTEGRITY OFFICERS AND WE DON'T
ARE THE BROADER CONCEPTION OF
RESEARCH INTEGRAITY AND THERE'S
REPORTS THIS IDEA RESEARCH
INTEGRITY IS MEASURED IN THESE
MORE COMPLEX WAYS AND NEEDS
SUPPORT STRUCTURE.
THE QUESTION I HAVE FOR YOU AND
FOR THE PANEL, HOW DID YOUR
OFFICE OF RESEARCHER INTEGRITY
THE TRADITIONAL ROLE GET
EXPANDED INTO THIS BROADER
RESOURCE PROVIDING ENTITY AND
WHAT'S THE VALUE PROPOSITION TO
THE INSTITUTION AND HOW MANY
INSTITUTIONS IN THE U.S. ARE
TAKING ADVANTAGE OF THAT
STRUCTURE?
WE'RE CURRENTLY DOING A SURVEY
OF ALL CTSA HOLDING INSTITUTIONS
TO FIND OUT WHAT THEY'RE DOING
BUT THE PROBLEM IS THE CTSA'S
THAT FUND THE INFRASTRUCTURE FOR
CLINICAL RESEARCH.
THE P.I.s DON'T EVEN KNOW THIS
IS GOING ON IN THEIR OWN
INSTITUTIONS.
THEY THEMSELVES DON'T KNOW WHO'S
DOING IT BUT DON'T KNOW WHOSE
HANDLING THIS AND THE QUESTIONS
ARE COUPLED SO I'D LOVE TO HEAR.
>> HOW DID WE MAKE THIS
TRANSITION ON MY TEAM?
IT'S WHERE I WANTED TO TAKE IT.
I DON'T HAVE A BETTER ANSWER
THAN THAT.
THE CHARGE TO US WAS FIGURE OUT
A WAY TO TRY TO PREVENT RESEARCH
MISCONDUCT AND THE ONLY ANSWER I
COULD COME UP WITH IS WE GOT TO
HAVE BETTER DATA MANAGEMENT AND
WE'VE GOT TO HAVE BETTER
PRACTICES IN THE LAB AND THE
OTHER BIG TRIGGER WAS NIH AND
NIH'S ADDICTION TO EVERY GRANT
PROPOSAL OF THE RIGOR
REQUIREMENTS.
AND SO THOSE RIGOR REQUIREMENTS
AGAIN THAT GAVE US THE
OPPORTUNITY TO REALLY ENGAGE
WITH RESEARCHERS ON HOW DO YOU
ADDRESS THIS IN YOUR PARTICULAR
PRACTICE.
THERE ARE OTHER INSTITUTIONS
DOING THINGS SIMILAR TO WHAT
WE'RE DOING.
THERE AREN'T A LOT BUT SOME AND
DUKE IS ONE AND HARVARD IS ONE.
>> IN THE WAKE OF SCANDAL.
>> YES.
AND SOMETIMES THESE KINDS OF
THINGS HAPPEN IN RESPONSE TO A
HUGE PROBLEM.
WE DON'T THINK IT'S THE CASE
EVERY P.I. KNOWS WHO THEIR
RESEARCH INTEGRATE OFFICE IS BUT
I THINK SHY, AREN'T YOU THE
RESEARCH QUALITY OFFICER OR
SOMETHING LIKE THAT?
>> HE SAID TROUBLE MAKER.
>> I THINK IT'S DIRECTOR OF
RESEARCH QUALITY.
>> IT'S PART OF THE SCOPE OF THE
RESEARCH INTEGRITY.
>> I GOT A POSITION FUNDED ON MY
TEAM.
OUR -- PEOPLE WOULD DO THE WORK
THAT WE DO WE'RE ALL UNDER
FUNDED.
THERE IS A COST NOT JUST A COST
TO THE INDIVIDUAL RESEARCHER BUT
THE INSTITUTION.
>> THE TITLE WE SULTH IS
ACADEMIC LEAD FOR RESEARCH
IMPROVEMENT.
IT'S SIMILAR TO WHAT YOU'RE
DESCRIBING AND IT'S IMPORTANT TO
HAVE THAT ACADEMIC VOICE AS PART
OF THIS AS WELL TO BRING IN THE
LIBRARTEAMS AND MAKE IT CLEAR
THIS IS FUNDAMENTALLY FOR
RESEARCHERS AND WITH THE VOICE
OF RESEARCHERS AS PART.
>> FANTASTIC POINT.
THANK YOU.
>> WE'RE SEEING SIMILAR THINGS
IN THE U.K. WITH THE VARIOUS
ROLES.
SOME INSTITUTIONS SET UP
DIRECTOR OF RESEARCH CULTURE OR
A PVV FOR CULTURE.
OTHERS ARE EXTENDING THE WORK
AROUND SECURITY TO BRING IN
ASPECTS OF INTEGRITY OR BULLYING
AND HARASSMENT.
THERE'S INTEGRAL TIES WITH
THINKING WHAT IT LOOKS LIKE FROM
A STRUCTURE PERSPECTIVE AND OVER
TO YOU.
WE HAVE OUR LAST TWO AFTER THAT.
>> IF SOMEONE'S THINKING OF
CHANGE IN BEHAVIOR FOR
SCIENTISTS, I THINK ABOUT A
QUOTE WHO WAS AN ATMOSPHERE
CHEMIST WHO WORKED ON THE OZONE
LAYER AND ONE OF THE FEW
SUCCESSFUL PEOPLE TO CREATE A
SITUATION WHERE PEOPLE RESPONDED
TO AN ECOLOGICAL DISASTER WITH
SOME MEASURE OF SUCCESS.
SHE SAID THE REASON WHY THE
EFFORTS TO MAKE THAT CHANGE
WORKED WERE THAT IT WAS
SOMETHING THAT WAS PERCEPTABLE
AND SOMETHING TO REACH FOR AND
DO.
PIGGYBACKING OFF THE COMMENTS
HOW HARD RESEARCH RIGOR IS AND
REFLECTING ON THE LAST COUPLE
DAYS DO YOU THINK WE'RE CLOSER
TO MAKING THIS IDEA OF RESEARCH
RIGOR SOMETHING THAT SCIENTISTS
COULD DEFINITELY PERCEIVE?
SOMETHING THEY CAN FEEL PERSONAL
ABOUT AND CURIOUS WHAT YOU THINK
ABOUT THE THREE WORDS AND HOW
THEY'VE STIMULATED YOUR OWN
EFFORT AND LOOK AT THE MINIMUM
SET OF THINGS WE CAN DO TOGETHER
TO GET US START AND SAY THAT'S
GOOD.
CURIOUS FOR REFLECTIONS ON THAT
LISTENING THE PAST COUPLE DAYS.
>> IT SOUNDS LIKE ANOTHER ONE
FOR THE WRAP-UP BUT MARYROSE HAS
A SOLUTION.
>> NO, LOW-HANGING FRUIT.
I'M A HUGE FAN OF RUBRICS.
THE TOP GUIDELINES AND LOOKING
AT IMPROVEMENT AND
ACCOUNTABILITY.
THE BEHAVIORS ARE LISTED IN
THEIR VARIOUS FORMS GETTING
BETTER.
VAIRS BEHAVIORS GETTING BETTER.
YOU CAN -- IT'S MOTIVATIONAL,
IT'S PRACTICAL AND YOU CAN MOVE
FORWARD.
I LOVE THAT RUBRIC TOP-GUIDELINE
MODEL.
>> IF YOU ASK SOMEBODY IF YOU'RE
A BETTER THAN AVERAGE DRIVER,
THEY ALL THINK THEY ARE.
NO ONE THINKS THEY'RE DOING
SLOPPY RESEARCH AND NEED TO
FRAME IT AS IMPROVEMENT.
HOW CAN WE BE EVEN BETTER OTHER
THAN YOU'VE ALL DONE A TERRIBLE
JOB.
MAKE IT POSITIVE.
>> I HAVE TWO QUESTIONS, IS THAT
OKAY?
ONE IS BUILDING ON THE ROLE OF
EXISTING INFRASTRUCTURE AND THE
I.O.s AND BUILDING THE RESEARCH
IMPROVE
IMPROVEMENT OFFICERS AND THAT'S
THE BALANCE BETWEEN TOP-DOWN AND
BOTTOM UP.
YOU HAVE A GRASS-ROOT LOCAL
NETWORK AND SOMEBODY IN THE
LEADERSHIP SEEMS TO BE AN
IMPORTANT ASPECT OF THAT.
IS THERE SOMETHING ABOUT U.K.
INSTITUTIONS THAT ALLOWED YOU TO
HAPPEN OR HAS IT BEEN REP KATED
IN THE OTHER -- REPLICATED AND
IS THERE SOMETHING LIKE THAT
PART OF HELIOS?
PEOPLE HAVE COMMITTED TO DO
SOMETHING?
IS THERE A PERSON WHO PLAYS THAT
KIND OF ROLE?
>> IN TERMS OF WHETHER IT'S
REPLICATED ACROSS NETWORKS,
THERE'S DIFFERENT STAGES OF
EVOLUTION BUT THE VAST MAJORITY
START WITH THE GRASSROOTS ENERGY
WHO WILL BE THE ADVOCATES OF
THAT LEVEL.
THERE'S PLENTY OF EXAMPLES OF
THE GRASSROOTS COMMUNITIES BUT
IF THEY'RE NOT SUPPORTED AND
AMPLIFIED BY PEOPLE FORTUNATE
ENOUGH TO BE IN POSITIONS WHERE
THEIR JOB DOESN'T DEPEND ON IT
ANYMORE IT'S NOT GOING TO GET AS
FAR AS AS IF WE HAVE THE
CONNECTIVITY.
I DON'T THINK THERE'S ANYTHING
UNIQUE ABOUT THE U.K.
THERE'S NO SECRETE SAUCE THERE.
ONE OF THE THINGS IT'S ABOUT
THINKING HAVING A CLEAR THEORY
OF CHANGE AND HOW TO GET WHERE
WE ARE TO WHERE YOU WANT TO BE.
WE'VE BEEN TALKING ABOUT
EVOLUTION VERSUS REVOLUTION.
I'M NOT A BIG FAN OF REVOLUTIONS
BECAUSE THEY END UP BEING
DESTRUCTIVE AND FACTIONAL AND A
PROBLEM WITH THE REFORM MOVEMENT
NARRATIVE IS PEOPLE BECOME WED
TO THE WAY OF DOING THINGS AND
SOME GOOD IDEAS MIGHT TURN OUT
NOT TO WORK AND HAVE UNINTENDED
CONSEQUENCES AND WE NEED TO KEEP
EACH OTHER HONEST IN TERMS OF
THAT.
THERE'S A POLITICAL SOCIETY IN
THE U.K. CALLED FABIANS AND THE
MODEL OF REFORM IN THE CONTEXT
OF LIBERAL DEMOCRACY IS YOU NEED
TO DRIP ADVICE INTO THE EARS OF
THOSE IN POSITION TO ENACT THAT
BUT IN THE OTHER EAR THEY NEED
TO HEAR THE CLAMORING AT THE
GATES AND YOU NEED BOTH AND
LINKING THE TOGETHER THE VOICE
OF THE GRASSROOTS COMMUNITY CAN
BE POWERFUL I THINK.
>> GOING BACK TO THE POINTS OF
INTERVENTIONS AND WHAT THOSE
MIGHT BE.
I CALL IT SHINY OBJECT SYNDROME.
ESPECIALLY AROUND CERTAIN POLICY
AREAS WHERE IT'S A NEW GUIDE BUT
NOT AN UNDERSTANDING OF WILL IT
WORK OR NOT AND THEN CALLING IT
REFORM OR REVOLUTION.
IT COULD BE TRAINING BUT IT'S
PROBABLY JUST A TRAINING.
THERE'S PROBABLY SOMETHING HOW
DO THOSE THINGS SIMULTANEOUSLY.
WE'RE PICKING UP MALCOLM'S
POINTS FROM BEFORE AND I THINK
UNLESS THERE'S ANOTHER POINT
FROM THE PANEL.
>> I CAN TAKE THE HELIOS ONE BUT
QUICKLY DO YOU THINK YOU GUYS
ARE LESS AGAINST MANDATES SOME
SEEMS LIKE IN THE U.S. WE HATE
ANYTHING TO BE MANDATE AND
YOU'VE USED THE WORD MANDATE A
LOT.
>> I'M NOT SURE.
WE DO HAVE FUNDER REQUIREMENTS
BUT I DON'T THINK MORE SO THAN
THE U.S.
WE'VE GONE TO THE FUNDERS AND
SAY WE'LL OFFER A PARTIAL
SOLUTION BY TAKING SOME OF THAT
OFF YOU BECAUSE IT SHOULDN'T ALL
BE TOP DOWN AND NEED TO WORK IN
PARTNERSHIP.
I THINK MANDATES HAVE A PLACE IN
SOME SPECIFIC CONTEXT BUT
ACTUALLY I WOULDN'T EXPECT THEM
TO BE THE EXCEPTION.
>> AND ACROSS DISCIPLINES.
THAT'S BEEN ANOTHER REAL BIG
KEY.
IT'S NOT A SPECIFIC SOCIETY OR
DISCIPLINE SOLUTION TRYING TO
THINK MORE BROADLY SO THAT
CULTURAL ORGANIZATIONS WOULD BE
IN THE ROOM TO HAVE THIS
CONVERSATION.
PEOPLE INVOLVED IN SOCIAL
RESEARCH IN THE COMMUNITY WOULD
BE IN THE SAME RIGOR
CONVERSATION AS OPPOSED TO THIS
IS ONE SPECIFICALLY FOR THOSE
INVOLVED IN BIO AND BIO MEDICINE
OR SOMETHING ELSE.
>> GOING BACK TO THE EVOLUTION
VERSUS REVOLUTION, I WOULDN'T
WANT TO APPEAR TOO CUDDLY
BECAUSE THERE'S FORCES OF
DARKNESS OUT THERE SUCCEEDING
VERY WELL AND DO EVERYTHING THEY
CAN TO GET IN OUR WAY.
THE NAIVE WAY IS BY JUST
OBSTRUCTING WHATEVER.
THE MORE BRITISH WAY IS TO
ACCOMMODATE, TO COERCE AND TO
APPEAR TO GIVE MORE AND SUCK THE
ENERGY OUT OF THESE REFORM
MOVEMENTS WHILE MOVING BEHAVIORS
JUST A TINY AMOUNT.
THERE'S THE VIEW OF THE OTHER
PANELISTS HOW THEY SEE WHAT
THEY'RE BEING GIVEN, IF YOU
LIKE, PERMISSION TO DO AS AN
ACCOMMODATION TO PREVENT A MORE
EXTREME REFORM AND HOW MUCH THE
WHOLE IS GENERALLY ON THE MARCH
TOGETHER AND MAY BE MOVING AT
DIFFERENT SPEEDS BUT ALL
COMMITTED TO A DIRECTION OF
MOVEMENT.
>> OKAY.
WHO'S JUMPING?
SOFIA IS READY FOR YOU.
>> IN TERMS OF REPRODUCE ACT.
WE MOSTLY KEEP OUT THE MORE
SENIOR PEOPLE SO YOU DON'T WANT
TO KNOW WHAT WE'RE TALKING
ABOUT.
I DON'T THINK IT'S ABOUT BEING
ALLOWED TO DO SOMETHING BUT MORE
ON THE REVOLUTIONARY SIDE AND
MEANS A SUPPORT GROUP AT THE END
OF THE DAY.
MAYBE THE QUESTION WASN'T FOR
ME.
>> ALL QUESTIONS ARE FOR YOU.
ALL QUESTIONS.
MARCUS.
>> I DON'T KNOW WHERE TO START.
>> ONE OF THE THINGS THAT GIVES
THE UPRN ITS LEGIT MASSIMACY WE
TO REMEMBER THAT ACADEMIA CAN BE
HIGH RANKING BUT IT'S FOR ALL OF
US AND WE'RE ALL PART OF THE
ACADEMY.
WE NEED TO SUPPORT THE
INITIATIVES THAT CREATE THAT
CLAMORING AT THE GATES I WAS
TALKING ABOUT SO PEOPLE IN
SENIOR POSITIONS REALIZE THERE'S
A NEED TO CHANGE OTHERWISE AND
WE NEED TO THINK ABOUT
ARTICULATING WHY THOSE IN SENIOR
POSITIONS SHOULD BE CHANGING
THINGS BECAUSE ULTIMATELY IT'S
IN OUR INTEREST AND WE'LL
ADVANCE KNOWLEDGE MORE RAPIDLY
AND WE'LL RETAIN THE BEST TALENT
AND THEREFORE DO BETTER RESEARCH
AS A RESULT OF THAT.
ALL THAT FITS TOGETHER.
IT'S COMPLEX BUT ONE OF THE
GUIDING PRINCIPLES FOR ME IS WE
NEED TO LISTEN AS MUCH ADVOCATE.
WHICH IS WHY I STRUGGLE WITH THE
REVOLUTIONARY BIT BECAUSE IT'S
TELLING PEOPLE HOW TO DO THINGS.
WE NEED TO BE IN LISTENING MODE
AS WELL AND LISTEN TO THE EARLY
CAREER RESEARCHERS AND THEY ALSO
THERE NEEDS TO BE BACK AND FORTH
DIALOGUE AND THERE'S INTERESTING
CONVERSATIONS WHEN YOU START TO
TAKE PRINCIPLES OF OPEN RESEARCH
TO THE SOCIAL SCIENCES FOR
EXAMPLE.
AND WITHOUT BECOMING TOO FLUFFY
OR ALLOWING IT TO DRIFT IS THE
CHALLENGE.
>> THERE'S A POWER DYNAMIC IN
ACADEMIA BUT I DOWN THINK IT'S
SET IN STONE AND AS OUR
UNIVERSITY WENT THROUGH A MAJOR
UNIONIZATION BY OUR GRADUATE
STUDENTS ILLUSTRATES THERE CAN
BE MAJOR SHIFTS AND WE HAVE TO
BE OPEN TO THOSE AND BE READY TO
JUMP WHEN THERE'S AN
OPPORTUNITY.
I THINK WE HAVE TO LOOK FOR THE
OPPORTUNITIES TO JUMP IN AND ADD
THIS COMPONENT TO A BROADER
DISCUSSION OF ISSUES.
I THINK THIS PIECE FITS AND
SYNERGIZE WITH OTHER ISSUES.
>> THE MAGIC OF A GROUP LIKE
THIS IS THE WIDE ARRAY OF PEOPLE
FROM OFFICERS TO PEOPLE LEADING
LABS TO INTERYY -- INTEGRITY
OFFICERS AND THERE'S A WIDE
ARRAY OF FOLKS WE CAN FIND AS
COCONSPIRATORS AND NOT WAIT FOR
PERMISSION FROM ANYBODY, MEL
COME.
-- MALCOLM.
>> DO YOU WANT TO COME BACK?
VERONIQUE.
>> I WANT TO COME BACK TO THE
VOICES WE DON'T HEAR BECAUSE
THEY LEAVE ACADEMIA BECAUSE THEY
ARE FED UP WITH THE SYSTEM AND
THEY ENTERED ACADEMIA TO DO
SCIENCE AND DO GOOD IN THE WORLD
AND REALIZED IT'S A LOSING
BATTLE AND HAVE NO POWER AND
LEAVE AND WE DON'T LISTEN TO
THOSE VOICES.
I'M IMPRESSED ON THE WORK ON
RESEARCH CULTURE IN THE U.K.
I'M WONDERING KAREN OR ANYBODY
ELSE IN THE ROOM, IF WE HAVE DO
WE HAVE DATA ON EXIT INTERVIEWS
OR SURVEYS OF PEOPLE AT EXIT
POINTS AND WHY ARE THEY LEAVING
AND WHAT'S THE SCALE OF THE
PROBLEM AND WOULD THAT BE -- DO
WE HAVE THAT?
I WOULD LOVE TO SEE THAT.
IF WE DON'T WOULD IT BE HELPFUL
HERE IN THE U.S.
>> WE KNOW A BIT BUT NOT A LOT
OF THOSE WHO EXITED.
WE HAVE A SOCIETY BRINGING
RETURNERS BACK IN AND PROVIDE
FELLOWSHIPS INCLUDING UKRI FOR
RETURNERS AND WHAT'S INTERESTING
IS THEIR MONEY GOES DIRECTLY TO
THEM AND NOT AN INSTITUTION WHO
WILL THEN FUNNEL IT OUT AND TAKE
IT TO INTERESTING SETS OF ROLES
AND THERE'S SOMETHING ABOUT WHAT
BRINGS PEOPLE BACK IN AND IN
WHAT WAY.
ONE PERSON WAS AT THE FOREFRONT
OF HER FIELD AND MADE A DECISION
TO FIND SOMEBODY AND MAKE SOME
BABIES BUT WHEN SHE CAME BACK
THE TOOLS AND TECHNIQUES
ADVANCED SO SIGNIFICANTLY SHE
HAD TO START OVER AND FOUND
SOMEONE ALLOWING HER TO PRACTICE
MESSING WITH THE KIT AND THE
STUFF AND THAT'S WHAT SHE USED
HER FELLOWSHIP FOR WAS TO RETOOL
AND MOVE FORWARD BUT DIDN'T
THINK SHE'D BE ABLE FIND A
REGULAR FELLOWSHIP SO ALLOW HER
TO DO THAT THE WAY THE
CRITERIA'S SET UP.
IT'S NOT JUST WHO MAY EXIT OR
ENTER BACK IN BUT WHO'S ALSO
NEVER THERE AND NOT A PART OF
THESE GROUPS NOT IN THE ROOMS OR
INSTITUTIONS.
THEY'RE NOT PART OF THE
CONVERSATIONS.
AND WE HAVE GOING TO NEED TO
DYNAMICALLY KEEP THAT ALIVE
SIMULTANEOUSLY.
I'LL TURN TO MY PANEL.
>> THE MPA HAS COLLECTED THAT
DATA.
I DON'T KNOW HOW RECENT IT WAS.
LAST TIME I READ A BIG CAUSE WAS
BALANCING CAREER AND FAMILY.
MANY FUNDERS THEIR AWARDEES GO
ON TO STEP TWO.
THEY'RE DATA IS THERE BUT I
DON'T KNOW HOW RECENT IT IS
BECAUSE THE CAREER FAMILY THING
MAY OR MAY NOT BE AS PREVALENT
AS IT WAS THE LAST TIME I READ
THE SURVEY BUT I KNOW PEOPLE ARE
DOING SURVEY AND THINK IT'S A
FABULOUS POINT.
WHOEVER HAS DONE THAT SURVEY
BEFORE, LET'S GET THEM TO ASK
THE QUESTIONS WE NEED TO.
[OFF MIC]
>> AND PEOPLE CARING FOR OTHER
PARTNERS AND CARING FOR ADULTS.
THERE'S ACTUALLY QUITE A LITTLE
WILL TO START TO UNPACK
INCLUDING PEOPLE WHO MAY GO INTO
FIELD WHERE A MASTER'S MAY BE
THE PENULTIMATE DEGREE.
THERE'S QUITE A LOT TO
UNDERSTAND ABOUT PIPELINE GROUPS
IN THE U.K. LOOKING AT THAT.
>> THERE'S PEOPLE IN ACADEMIA
AND THERE'S BITS IN THERE ABOUT
REASONS THEY'RE THINKING OF
LEAVING.
ONE THING WE NEED TO CHALLENGE
IS THIS PIPELINE NOTION.
THE FACT WE HAVE A PIPELINE I
THINK IS A NARROW CONCEPTION OF
WHAT A CAREER CAN LOOK LIKE AND
WE NEED TO CREATE OPPORTUNITIES
FOR PEOPLE TO RETURN BACK TO
ACADEMIA TO KAREN'S POINT.
THAT WOULD ALLOW OTHER SECTORS
TO COME BACK AND ONE OF THE
REASONS WE'RE SO UNIMAGINATIVE
IS MOST OF US HAVE LITTLE
EXPERIENCE OUTSIDE ACADEMIA.
>> NOT AN ALT TRACT BUT WHAT
DOES POROSITY LOOK LIKE ACROSS
SECTORS INCLUDING A NARROW TRACK
IN VARIOUS AREAS.
I'M NOT SURE WHERE WE'RE AT.
I THINK WE'RE HERE.
>> WE THINK NOT ENOUGH ARE
LEAVING ACADEMIA AND SHOULD BE
LEAVING SOONER.
>> YOU'RE PUSHING THEM OUT THE
DOOR?
I'M KIDDING.
IT'S SARCASM.
>> FUNDAMENTALLY WE PRODUCE TOO
MANY Ph.D.s FOR NO GOOD REASON
AND SHOULD STOP THE PRODUCTION
WHICH WOULD SOLVE A NUMBER OF
PROBLEMS INCLUDING LARGE NUMBERS
OF PAPER AND NOT ENOUGH QUALITY
AND WHATNOT AND WHY DO WE WANT
PEOPLE BACK?
IT'S NOT A GOOD LIFE FOR MANY
PEOPLE IN ACADEMIA.
>> WE TALKED ABOUT WHO IS IN
ACADEMIA AND DO CONSIDER THE
GREENHOUSE GROWER AS PART OF
ACADEMIA.
WHAT WOULD CHANGE IF WE MOVED
FROM WHAT THE ROLES MIGHT END UP
BEING?
THIS IS MARCUS' POINT.
NOT DISAGREEING CONCEPTUALLY
THOUGH MANY ARE NOT HIRING AFTER
SOME RETIRE AND THE LINES ARE
DISAPPEARING IN THE U.S. AND
U.K. BUT I THINK THESE ARE REAL
BIG FUNDAMENTAL QUESTIONS AROUND
ROLES.
AND RESPONSIBILITIES.
MAYBE WE CAN PICK THIS UP AS
WELL WHEN WE GET INTO OUR NEXT
BIT OF THE CONVERSATION ABOUT
NEXT STEPS.
I'M WARNING YOU WE'RE ON CLOCK
NOW.
YOU'VE MANAGED TO PERK UP, FIND
YOUR ENERGY AND COME UP WITH
BRILLIANT IDEAS BUT WE'LL
PROBABLY TAKE THE LAST THREE
BEFORE WE TURN THE STAGE TO
VERONIQUE.
>> ONE COMMENT IS WE CAN'T
UNDERESTIMATE THE IMPORTANCE OF
SAVING THIS GENERATION OF
SCIENCE.
THERE'S THE DISCUSSION OF
REVOLUTION VERSUS EVOLUTION.
IT'S NOT REALLY A DICHOTOMY.
IT'S HOW FAST CAN WE MAKE THE
EVOLUTION GO.
AT SOME POINT IT LOOKS LIKE A
REVOLUTION BUT IT'S JUST RAPID
EVOLUTION AND I THINK WE NEED TO
THINK OF IT IN TERMS OF SYSTEMS
OF CHANGE TO GET US THERE MORE
QUICKLY TO PRESERVE AND SAVE THE
CURRENT GENERATION OF
SCIENTISTS.
I WANT TO USE A COUPLE
ANECDOTES.
ONE COMES FROM THE RESEARCH
PROJECT I DID SEVERAL YEARS AGO
AT THIS POINT WHERE WE LOOKED AT
HEURISTIC DECISION MAKING IN
SCIENCE AND HOW DO THEY DECIDE
WHEN THEY'RE STOP COLLECTING
DATA AND COMMENTS THAT CAME FROM
THE STUDY WERE THINGS LIKE I
WORK IN THE LAB WHERE WHERE HE
USE OPTIONAL STOPPING.
I KNOW IT'S WRONG AND WE DO IT
BUT WHEN I GET MY OWN LAB I
WON'T DO IT.
IR HAD STUDENTS THAT WOULD COME
AND SAY I WANT TO SWITCH ADVISER
AND WANT TO DO SO BECAUSE I WANT
TO BE PART OF THE SOLUTION NOT
PART OF THE PROBLEM.
THESE ARE REAL PROBLEMS
AFFECTING DECISIONS OF OUR
GRADUATE STUDENTS AND SHOULDN'T
HAVE TO BE IN THIS POSITION AND
THAT'S THE KIND OF SAVING WE
NEED TO DO WE CAN'T
UNDERESTIMATE THE NO CHANGE ON
WHAT IT'S DOING TO OUR
PROFESSIONAL AND THE STUDENTS
WE'RE PRODUCING.
>> OKAY.
>> WHEN YOU SAY THIS GENERATION
WHAT GENERATION?
>> THE NEW FACULTY.
I TELL THEM WHAT OUR NEW
POLICIES ARE.
>> EARLIER CAREER PEOPLE IN
PARTICULAR GRADUATE STUDENTS,
POST-DOCS, EARLY CAREER FACULTY.
>> WE TALKED ABOUT REVOLUTION.
IT'S NOT SAYING HOW IT SHOULD BE
DONE BUT IT'S NOT A DICTATOR
SAYING HOW THINGS SHOULD BE DONE
OR THESE ARE THE RULES.
IT'S BUILT BY GATHERING AND
FIGURING OUT CREATING
COMMUNITIES OF ALLIES TO
PUSHBACK AND THEY TAKE PRINCIPLE
STANCES SOMETIMES AND BASED ON
AN IDEOLOGY AND MAKES IT AN
EVOLUTION.
A REVOLUTION IS THE PREVAILING
IDEOLOGY AND I'LL COME AT IT
FROM A COMPLETE CHANGE OF
PARADIGM WHAT ARE THE PRACTICAL
NEXT STEPS AND CARING ABOUT A
PRINCIPLE STANCE AND THE RISK
I'M WILLING TO TAKE AND SAYING
THE SYSTEM WILL NOT HELP ME BUT
I'LL STILL DO IT AND WE NEED TO
BE ON THE PATH OF A SLOW
MODERATE CHANGE AND AND THAT'S
WHAT I WAS REFERRING TO AND IT'S
ABOUT SAYING THESE ARE THE
ENCOURAGING THE ARGUMENTATION
AND USING PARADIGMS SAYING IT'S
WORTHWHILE TO TAKE THE RISKS AND
IF THE SYSTEM IS NOT ENCOURAGING
YOU, THAT'S SOMETHING WE NEED TO
ACCEPT AS A COMMUNITY TRYING TO
ADVOCATE CHANGE THERE'LL BE
CASUALTIES ALONG THE WAY IN
TRYING TO DO IT.
THANK YOU.
>> HOW QUICKLY DO WE WANT TO
MAKE CHANGE?
I THINK THE DIMENSION OF TIME IS
INCREDIBLY IMPORTANT IN THE
DECISION.
WHAT ARE THE NEXT STEPS.
I WANT TO BRING THIS UP AS
PERHAPS AS A PLATFORM ON
THINKING ABOUT WHERE WE MOVE
FORWARD IF YOU WANT TO GET
SOMETHING DONE BY NEXT YEAR YOU
SAY OKAY, THIS IS WHAT WE HAVE
TO DO BUT IF YOUR PROJECTION IS
10, 20 YEARS IT'S VERY
DIFFERENT.
>> ALL RIGHT.
LET'S GO IN OERDER
>> I'LL TALK ABOUT THE
GENERATION THING.
IT DEPENDS ON WHAT YOU MEAN.
IF IT'S SAVING THIS GENERATION
ARE THEY PAYING THE GAME THAT
WILL CURRENTLY GIVE THEM AND
MAKE THEM WIN OR PLAYING THE
GAME THAT THE SECOND THING YOU
MENTIONED WHERE THEY'RE TRYING
TO DO THE RIGHT THING AND THE
TANDEM SOLUTIONS WILL END UP
BENEFITING MAINLY THE PEOPLE WHO
ARE PLAYING THE GAME CORRECTLY
GIVEN THE RULES OF THE SYSTEM
AND HAVE THE PROBLEM WHERE
PEOPLE WHO MAY BE HAVE REALIZED
A BIT EARLIER WHAT'S GOING ON
HAVING DONE SOMETHING THAT
REALLY DID GO AGAINST THEIR BEST
INTEREST BECAUSE YOU ALREADY
SAID PEOPLE ARE NOT GOING TO DO
ANYTHING THAT GOES AGAINST THEIR
BEST INTEREST.
I DON'T THINK IT'S RIGHT FOR
GRAD STUDENTS AND FOR MORE
EASTERN PEOPLE AND THOSE WHO
HAVE TO LOSE IN AGAIN.
AND HOW TO BEST SUPPORT IT I
DON'T KNOW IN I REALIZED SO SOON
I REALIZED I HAVE TO GO TO A
META FIELD AND WE ENCOUNTERED
THIS A LOT IN REPRODUCIBILITY
MEETINGS.
THEY SAID I'M ON BOARD BUT WHAT
I DO DO NOW AND PEOPLE CHANGED
THEIR DISSERTATIONS IN THEIR
FIELD AND YOU CAN GO THAT ROOT.
THERE ARE WAYS OF SOLVING THAT
BUT THEN YOU'RE AT A DEAD END
AND LOTS OF PEOPLE DO LEAVE AND
I DON'T KNOW WHAT TO DO ABOUT
THAT AND I'M NOT THE PERSON TO
SOLVE THAT BUT IT'S A HUGE
PROBLEM.
I'M NOT SURE HOW TO SOLVE IT.
>> YOU DON'T NEED SOLUTIONS AND
WE'LL DO DOWN HERE AND THEN
VERONIQUE.
>> WHAT DO WE DO NOW COMMENT
GOES TO SHY'S QUESTION.
THERE'S A LOT THE ANSWER IS ALL
OF THE ABOVE.
IT WOULD BE GREAT TO HAVE EARLY
WINS AND THEY'LL BUILD MOMENTUM
BUT THERE ARE THINGS THAT ARE
LONG-TERM AND WRITING DOWN FLOW
CHARTS AND USING THE HIGH VALUE,
LOW RESOURCES, LET'S DO THOSE
QUICKLY AND PUT THEM ALL
TOGETHER AND DECIDE WHAT'S THE
ORDER AND WHAT WE CAN USE AND
WHAT RESOURCES WE'LL TAKE AND
PROJECT MANAGEMENT EXERTION.
>> MARCUS.
>> IT'S IMPORTANT TO HAVE A
VALUES-LED APPROACH BUT THEY
NEED TO BE VALUES THAT WE ARE ON
BOARD WITH AS A COMMUNITY RATHER
THAN AN INDIVIDUAL VALUES AND
THOSE CAN BE LINKED DIRECTLY TO
VALUES AND IN TERMS OF TIME
SCALE, I THINK WE NEED TO DO A
SERIES OF THINGS.
WE NEED A GOOD UNDERSTANDING OF
WHERE WE ARE AT THE MOMENT.
A LOT OF META RESEARCH AND WHERE
THE FAULTS LIE AND WOULD LIKE A
VISION OF WHAT IS THE FUTURE OF
JOURNALS AND THE FUTURE OF
CAREER PATH HOW TO CREATE THAT
AND MAP TO WHERE WE ARE FROM
WHERE WE WANT TO BE AND IT'S NOT
TRIVIAL AND PROBABLY WILL TAKE
TIME.
THERE'S A GOOD QUOTE THAT
REVOLUTIONS HAPPEN VERY SLOWLY
AND THEN ALL AT ONCE.
I THINK THERE'LL BE A TIPPING
POINT WHERE THINGS WILL SUDDENLY
CHANGE QUICKLY AND IT FEELS LIKE
WE'RE APPROACHING THAT.
I'VE BEEN TALKING ABOUT THIS
STUFF 15, 20 YEARS.
FOR MOST THE TIME I WASN'T
LISTENED TO AND THE CLAMORING AT
THE GATE IS LOUDER NOW THAN FIVE
YEARS AGO.
>> NAOMI.
>> I THINK WE'RE IN SUCH A
COMPLEX ECO SYSTEM ALL THE
STRATEGIES WE HEARD ABOUT OVER
THE LAST TWO DAYS CAN PLAY A
ROLE.
IN TERMS OF TIME SCALE, WHEN A
MAJOR FUNDING AGENCY TELLS YOU
TO DO SOMETHING, INSTITUTIONS
FIGURE OUT HOW TO DO IT.
IF THEY SAY YOU HAVE TO DO IT BY
JANUARY 25, WE'LL FIGURE OUT HOW
TO DO IT BY JANUARY 25.
IF THERE'S THINGS WE THINK
SHOULD BE PRIORITIZED, THAT'S A
LEVER THAT'S EFFECTIVE.
HOW EFFECTIVE CAN IT BE?
WELL, I THINK THAT'S A QUESTION
THAT REALLY WE DON'T KNOW THE
ANSWER TO.
WHEN NIH MADE THE BIG CHANGES TO
REQUIRE RIGOR AND
REPRODUCIBILITY OR THE FOUR
CATEGORIES OF RIGOR IN EVERY
GRANT APPLICATION, IF P.I.s WERE
GETTING FEEDBACK BACK FROM
REVIEWERS THEN P.I.s WILL MAKE
CHANGES.
THAT'S PART OF THE FEEDBACK
PROCESS THAT WILL CHANGE THE WAY
THAT PEOPLE -- HOW PEOPLE
PRIORITIZE AS THEIR THINKING
ABOUT THEIR SCIENCE.
THAT MEANS THE REVIEWERS HAVE TO
BE TRAINED TO BE ABLE TO GIVE
MEANINGFUL FEEDBACK THAT
ACTUALLY IS HELPFUL RATHER THAN
NICK PICKY THINGS PEOPLE GET
ANNOYED BY AND THE OTHER THING I
WOULD SAY IS AND THEY SAID
SOMETIMES TO GET SOMETHING DONE
AT UNIVERSITY YOU HAVE TO WAIT
FOR SOMEONE TO DIE.
I HOPE THAT IS NOT THE CASE FOR
US.
THERE ARE SOME THINGS THAT TAKE
LONGER THAN OTHERS.
>> I HAVE TO ADD A POINT BECAUSE
THAT WAS FRIGHTENING THINKING
ABOUT DEATH FOR A FEW MOMENTS.
HAVING STUDIED REVOLUTIONS FOR A
LONG TIME, SOMETIMES YOU DON'T
KNOW WHEN THEY ARE HAPPENING.
THEY CAN BE SUBTLE AND PROFOUND
AND THERE'S SOMETHING ABOUT
RECOGNIZING THE MOMENT WE'RE IN
AND THE OPPORTUNITIES WE HAVE
MOVING FORWARD AND WAYS TO
MOBILIZE AND THINK OF A SYSTEM.
I WANT TO THINK THE PANELISTS
AND ALL YOU FOR A FANTASTIC SET
OF QUESTIONS.
YOU MANAGED TO GET IT TOGETHER
WHICH IS GREAT.
THANK YOU VERY MUCH AND OVER TO
YOU.
>> SO YOU'RE NOT DONE YET BUT
GETTING CLOSE.
IN TERMS OF WRAP-UP WE TALKED
ABOUT THINGS DONE AT THE LEVEL
OF THE SYSTEM AND ALL THAT.
I WANT TO BRING IT BACK TO THIS
ROOM.
WHEN WE WERE DISCUSSING AND
PUTTING THE THING TOGETHER WE
HAD THE DISCUSSION IF WE ALL HAD
A GREAT MEETING AND GO BACK AND
DO NOTHING ABOUT IT, WE WILL
HAVE FAILED BECAUSE WE REALLY
WANT THINGS TO COME OUT OF THIS
AND TO BE ACTIONABLE.
SO WE HAVE ABOUT 20 MINUTES
REMAINING AND A WANT YOUR
SUGGESTIONS ON WHAT ARE THE
CONCRETE THINGS WE CAN DO WHEN
WE GET OUT OF THIS ROOM TO KEEP
THE MOMENTUM AND CONVERSATION
GOING AND GET ACTIONABLE
INSIGHTS OUT OF THIS.
I'LL LIST A COUPLE THINGS WE'VE
ALREADY GRASPED DURING THE
MEETING AND WHETHER IT'S
FEEDABLE OR -- FEASIBLE OR NOT
AND IT'S JUST FOR BRAIN STORMING
AND GATHERING INFORMATION.
WE WILL DO A SYNTHESIS OF THE
IDEAS DISCUSSED HERE AND IN
PARTICULAR WHAT WAS DISCUSSED IN
THE BREAKOUT SESSIONS.
WE'LL WORK WITH THE MODERATORS
MAKE SINCE OF THE STICKY NOTES
AND RETURNING THIS INTO A USEFUL
SYNTHESIS WOULD BE WELCOME.
SOME ARE ASKED ABOUT
COMMUNICATION CHANNEL FOR PEOPLE
IN THIS ROOM AND IT WILL SET UP
A SLACK CHANNEL.
THERE'S A LACK OF RESOURCES FOR
TRAINING.
IF YOU HAVE INTERESTING WAYS TO
SHARE WOULD YOU BE WILLING TO
SHARE THAT ON AN OPEN RESOURCE?
WE HAVE A GOOD SET OF
VOLUNTEERS.
I THINK WE CAN WORK WITH TIM TO
HAVE A REPOSITORY FOR THE
MATERIAL ANY VOLUNTEERS THAT
WOULD LIKE TO HELP CURATE THE
MATERIAL, PLEASE LET ME KNOW.
COME UP TO THE MICROPHONE IF YOU
HAVE IDEAS.
>> I LEAD THE PROGRAM CALLED
RIGOROUS FOUNDATION RESEARCH.
I THINK SOCIETIES CAN PLAY A
PUBLIC HEALTH ROLE IN WHAT
YOU'RE DISCUSSING HERE AS WELL.
IT'S TOUGH NOW TO BE A
NEUROSCIENTIST AND NOT KNOW
ABOUT THE TOPIC.
MARCUS, I FEEL I'VE BEEN IN THIS
SPACE A LONG TIME.
LOOK AT WHAT SFN HAS DONE IN THE
LAST SIX TO EIGHT YEARS IS
PROVIDE A LOT OF TRAINING
MATERIALS AND DISCUSSIONS.
THEY'RE SOLUTION ORIENTED.
NOT JUST ABOUT WE'RE HAVING A
PROBLEM.
WHAT CAN WE DO TO GET TO
SOLUTIONS?
THE TRAINING MATERIALS ARE ALL
AVAILABLE AND IN NEUROSCIENCE
THINGS ARE BEING USED WIDELY
ACROSS THE WORLD.
I'LL HELP FOR SURE.
>> ANY COMMENTS IN THE
MICROPHONE BECAUSE STEVE WE
COULDN'T HEAR YOU AT THE BACK?
>> WHAT I MAY KNOW IN
NEUROSCIENCE IS NOT WHAT ALL
SOCIETIES COMPLETELY AGREE AND
WE NEED TO SYNTHESIZE THAT AND
HAVE THE SOCIETIES WORK TOGETHER
AND TO BRING THAT TOGETHER.
THE NIH HAS DONE A GREAT JOB
LIVING RESOURCES AND IT'S A GRA
START AND THAT WOULD BE ANOTHER
WAY.
SOCIETIES CAN PLAY AN IMPORTANT
ROLE BECAUSE THEY'RE
INTERNATIONAL.
>> THERE'S RESEARCH AND FUNDING
RESOURCES.
A LOT OF CONVERSATIONS I WAS IN
I SAID IF ONLY SUCH AND SUCH
EXISTED AND SOMEONE WOULD SAY WE
HAVE THAT.
I THINK CONSOLIDATING A LIST OF
RESOURCES SO PEOPLE WHO HAVE
DONE A LOT OF HARD WORK TO
CREATE A RUBE RICK OR FUNDING
STREAM OR SOMETHING LIKE THAT TO
ALL FIND OUT ABOUT THOSE.
>> IN CERTAIN FIELD AND AREAS
WHAT WE'RE TRYING TO DO IS NOT
CREATE A ONE-STOP SHOP AND NOT
JUST A LIBRARY, LIBRARIES ARE
LOVELY WHERE WE STORE STUFF AND
THINK OF CURATING AND CREATING
NEW SPACE FOR PEOPLE WHO JUST
STARTED TO THOSE WHO ARE
ADVANCED MOVING FORWARD.
I WOULD ENCOURAGE IF THERE'S THE
CREATION OF SOMETHING IT MAY
HAVE CASE STUDIES OR HAVE
STRATEGIC EXAMPLES OF THINGS
PEOPLE HAVE DONE SOME OF THE
STRATEGIC THINKING WILL BE
BENEFICIAL TO OTHERS.
GETTING THE CONCRETE SPECIFICS
WILL BE CRUCIAL AND HOW TO EMBED
IT IN MY INSTITUTION OR WHAT ARE
WAYS OTHERS HAVE DONE THIS AND
ENGAGED WITH OTHER GROUPS AND
PICKING UP QUESTIONS PEOPLE WERE
POSING TO THE LAST PANEL.
ALSO BE READY TO SHARE MATERIAL
NOT POLISHED AND PERFECT BUT
WORKED AT YOUR LAB OR DEPARTMENT
LEVEL OR THINGS LIKE THAT.
>> I'M FIND -- MINDFUL WE'RE
HERE WITH THE CONVENERS, THE
NINDS AND THOSE OF US FROM MANY
PARTS OF THE WORLD AND MANY
PLACES THAT HAVE MANY INTERESTS.
I THINK THAT FRAMES FOR ME THE
THREE POINTS.
ONE IS NARRATIVE, THE SECOND IS
PROGRESSIVENESS OF CONVERSATION
AND THE THIRD IS TANGIBLE
ACTION.
THE NARRATIVE I THINK IS
IMPORTANT AND THE ETHOS OF
SCIENCE IS ONE OF CONSTANT
SELF-IMPROVEMENT.
ADAM SETH SAID SCIENCE IS THE
ANECDOTE TO WHAT SUPERSTITION
IS.
IT'S NOT A NEW CONVERSATION BUT
THE FUNDAMENTAL ESSENCE OF
SCIENCE.
CHARLES BAB ITCH 1830 PUTS OUT A
BOOK ON COOKING DATA.
THEY DIDN'T HAPPEN WHEN BEAR PUT
OUT A THING 15 YEARS AGO.
IT GOES BACK TO THE BEGINNING OF
SCIENCE.
WE CAN GRAB THAT NARRATIVE AND
MAY NOT HAVE SOLVED EVERYTHING
BUT IT'S A TANGIBLE THING DONE
AND LOOK AT HOW MANY MORE TRIALS
ARE NOW PREREGISTERED AND HOW
MANY FEWER END POINTS ARE
CHANGED.
THAT'S THE NARRATIVE.
HAVING GRABBED THAT WE MAY NEED
TO BRANCH IN TWO WAYS.
THERE'S A CONVERSATION WHERE
WE'VE GONE FROM MENTORING TO
DIVERSITY TO INCENTIVES TO AND
TO THINK NINDS IS HERE, WHAT
COULD NINDS DO THEY COULD OWN
THAT THEY COULD PUT THEIR MARK
IN AND SAY WE DID THAT WE DID
THAT AND WE MADE A DIFFERENCE
AND IT'S ONE THING MA MADE
SCIENCE BETTER, IT'S CONCRETE
AND BY THE WAY, OTHERS COULD
FOLLOW US AND DO BETTER AND
NINDS CAN DO NINDS-FUNDED
STUDIES.
A CRON CREASE TANGIBLE THING
THAT HAS THE REST OF US ENGAGE
IN THE BIG DIALOGUE TO PULL
FORWARD THE NEW NUG ELTS OF
IDEAS WE HADN'T THOUGHT OF YET
THAT NINDS CAN SAY ONE THING, DO
ONE THING AND FOLLOW UP ON
NINDS-FUNDED DATA AND GET RFPs
TOUT HIRE PROFESSIONALS TO
ENGAGE IN SPOT CHECKING AND
QUANTIFYING THE RIGOR OF THOSE
STUDIES AND PUTTING OUT METRIC
ON HOW WELL NINDS-FUNDED STUDIES
ARE DOING.
THAT GIVES THE TANGIBLE CONCRETE
THING TO POINT TO AND WE DID
SOMETHING AND HOPEFULLY MAKE IT
BETTER WHILE WE CONTINUE THE
BROAD CONVERSATION THAT'S
EQUALLY ESSENTIAL.
>> DEFINITELY.
>> THE FUNDING AGENCIES ARE
ASKING LIKE INSTITUTIONS,
PUBLISHERS, EVERYBODY REACTS TO
THAT.
IT'S IMPORTANT TO KEEP THE LEVER
IN MIND.
I HAD A RELATED QUESTION.
I THINK NINDS HAS DONE A LOT OR
A FEW PEOPLE WITHIN THAT HAVE
DONE A LOT AND WHAT OTHER
INSTITUTES ARE LOOKING AT THE
PROBLEM AND DO WE HAVE THE WIDE
BRIDGES BETWEEN INSTITUTES AT
NIH WHERE SOME COLLECTIVE ACTION
IS POSSIBLE?
>> THERE'S DIFFERENT ENTITIES
AND DIFFERENT WAYS OF DOING
THINGS AND WE DON'T NECESSARILY
KNOW WHAT OTHERS ARE DOING.
BUT IF THEY ARE DOING IT,
THEY'RE KIND OF DOING IT
QUIETLY.
THE QUESTION IS WHAT ARE THEY
DOING?
NINDS CAN ONLY DO SO MUCH
BECAUSE THE MISSION OF NINDS IS
TO ADVANCE NEUROLOGICAL
DISORDERS AND STROKE AND NOT
CANCER RESEARCH AND NOT OTHER
THINGS BUT THE QUALITY OF
RESEARCH IS CENTRAL TO
EVERYTHING.
SO THAT'S WHY WE HAVE BEEN
FORTUNATE TO HAVE GOTTEN SUPPORT
TO WORKSHOPS SUCH AS THIS.
THERE'S A LIMIT TO WHAT NINDS
CAN DO AND WE NEED TO THINK MORE
BROADLY AND MAYBE IT HAS TO GO
TO CENTRAL NIH OR SOMETHING.
>> EVERY TIME I ATTEND THESE
WORK SHOPS I GET REMINDED WHAT
ENORMOUS A PROBLEM IS.
THERE WAS A CONFERENCE ON THE
SAME TOPIC AND A LOT OF ACADEMIC
LEADERS WERE AT THAT WORKSHOP
AND AT THE END OF THE SESSION IT
WAS THE SAME QUESTION, WHAT ARE
WE EACH GOING TO DO AND I'LL
REITERATE WHAT I VOLUNTEERED TO
DO AND VOLUNTEERED TO WORK
WITHIN MY COMMUNITY AND WE HAVE
A COUNCIL OF GRADUATE STUDIES
AND PSYCHOLOGY AND HAVE A LIST
SERVE TO RUN A WORKSHOP ON
POLICY CHANGE AND I'LL REITERATE
THAT NOW AND VOLUNTEER MYSELF TO
WORK WITHIN MY COMMUNITY TO
INSTITUTE CHANGE IN AS MANY
DEPARTMENTS AS WANT TO SIGN UP.
SO FAR THREE OR FOUR BUT I
HAVEN'T ADVERTISED WELL.
>> THANK YOU.
THANK YOU FOR OFFERING THAT.
>> WE HEARD PROMISING THINGS ON
WHAT'S HAPPENING WITH THE U.K.
REPRODUCIBILITY NETWORK AND
ITALIAN AND AUSTRALIAN.
WHAT ABOUT A
U.S. REPRODUCIBILITY NETWORK?
IS THERE A REGIONAL
REPRODUCIBILITY NETWORK?
IS THIS SOMETHING WE NEED TO
LOOK FURTHER INTO?
WE'LL SEND A SURVEY AFTER THE
MEETING TO GET SPEEDBACK ON THE
ORGANIZATION OF THE MEETING AND
LEARN ABOUT BECAUSE WE'VE
EXPERIMENTED WITH A NUMBER OF
THINGS AS WELL AND PERHAPS WE
CAN ADD THAT QUESTION IN THERE
ABOUT WHAT WOULD BE THE VALUE OF
SOMETHING LIKE THAT.
>> THIS IS HOW IT STARTED IN THE
U.K.
MANY ELEMENTS WERE IN PLACE AND
THEY'VE SPOKE TO THE OTHER
REPRESENTS AT OTHER NETWORKS BUT
THERE'S VALUE IN HAVING A COMMON
IDENTITY SO WE'RE SEEN AS A
COMMUNITY AND THAT THEN GIVES A
BIGGER VOICE WHEN WE SPEAK TO
FUNDERS AND SUPER NATIONAL
AGENCIES LIKE THE EUROPEAN
COMMISSION, FOR EXAMPLE.
THAT PART OF IT IS STILL
SOMETHING WE CAN IMPROVE ON.
>> AND THE INTERPRETABILITY
PART.
I THINK A LOT OF ANYTHING WE
PROPOSE WILL COST TIME OR MONEY
OR LABOR WHICH MONEY CAN PAY FOR
SO I'D LIKE TO THROW OUT AN IDEA
TO GENERATE MONEY WHICH IS WE
ALL REFUSE TO PAY APCs FOR ANY
FOR-PROFIT PUBLISHERS AND
RECOVER ALL THAT MONEY AND PUT
IT INTO RESEARCH RIGOR INSTEAD.
>> I THINK GOING OFF DAVID'S
COMMENT I APPRECIATE TAKING THE
NARRATIVE AND MAKE SURE THERE'S
TANGIBLE THINGS AND HAVE SHORT
TANGIBLE AND LONG-TERM VISIONS
BECAUSE IT'S SO COMPLEX.
WHAT I GET REMINDED OF IS YOU
DON'T NEED TO WORRY SO MUCH
ABOUT THE BIG THING JUST START
SMALL THINGS AND SEE IF IT HAS
LEGS.
THERE'S A WANT ADD, WHO IS
WILLING TO TRY THINGS IS I THINK
JUST AS IMPORTANT.
THERE'S A LOT OF PEOPLE HERE
SAYING I HAVE IDEAS BUT I'M NOT
IN THE POSITION BECAUSE I NEED
OTHERS TO HELP ME.
THERE'S -- I'M WONDERING HOW
BEST TO CATALYZE THAT BECAUSE
YOU DO MATCH MAKING TO SAY I'M
AT AN INSTITUTION AND I KNOW
WHAT YOU'RE DOING AND WILLING TO
ENTERTAIN THAT SO LET'S TALK.
AND OTHERS ARE AN EARLY CAREER
RESEARCHER AND WILLING TO TALK
AND THAT MAY GO A LONG WAY TO
PILOT AND SPARK IDEAS.
THERE'S IDEAS IT'S JUST MATCH
MAKING WE NEED TO DO.
LET'S PROTOTYPE IT AND FIT
FAILS, LET'S LEARN AND TRY
AGAIN.
>> I WOULD LIKE TO WORK OUT WHAT
THE TOP GUIDELINES WOULD BE FOR
RIGOR.
IT'S COME UP A NUMBER OF TIMES.
IF WE WERE ABLE TO GIVE A
PATHWAY FOR INSTITUTIONS, FOR
EXAMPLE, TO FOCUS ON RIGOR, WHAT
WOULD THAT LOOK LIKE OR A
RESEARCH GROUP AND SO ON SO IT'S
NOT YOU HAVE TO DIVE INTO A
MILLION RESOURCES, YOU HAVE AN
ARCHITECTURE TO WORK FROM AND
FIND RESOURCES THAT APPLY TO
YOUR DISCIPLINE.
I THINK WE'RE ALSO ALL SEEDS AND
WE MIGHT EACH THINK OF HOW TO DO
THAT AND MIKE JUST GAVE AN
EXAMPLE AND FOR ME THAT MIGHT BE
GOING BACK AND TALKING TO MY
RESEARCH OFFICE AND TRYING TO
ORGANIZE A CONFERENCE ON THIS TO
GET OTHER PEOPLE INTERESTED IN
MY INSTITUTION.
>> THE TOPIC DOESN'T TRANSLATE
NICELY BUT THEY'RE ON TO
SOMETHING AND THERE'S A VERY
USEFUL ROAD MAP AND DEVELOPMENT
THERE.
>> WE'VE BEEN TALKING ABOUT
SYMPTOMS AND SYMPTOMS OF A
DISEASE.
THERE'S MERIT IN TREATING
SYMPTOMS THAT MAKES THE PATIENT
FEEL BETTER.
I WOULD ARGUE THERE'S A ROOT
CAUSE AND DISEASE THAT NEEDS TO
BE TREATED AND THE ROOT CAUSE OF
THE SYMPTOMS WE'VE BEEN TALKING
ABOUT HAS TO DO WITH RESEARCH
AND HOW THEIR PRODUCTS ARE
EVALUATED AND INCENTIVIZED.
AND THIS WILL HAVE THE LARGEST
EFFECT.
IN FACT WHAT'S INTERESTING IS
AND I JUST RECENTLY STUDIED
APPROXIMATELY 20 MANIFESTOS,
RECOMMENDATIONS, GUIDELINES,
WHATEVER FROM THE EUROPEAN UNION
YOU NAME IT.
IT SEEMS TO ME THE WHOLE OPEN
SCIENCE DISCUSSION, HOW DO WE
GET TO MAKING SCIENCE MORE HAS
SHIFTED TO THIS PROBLEM BECAUSE
THEY'RE ALL FOCUSSING ON IT NOW.
IT'S GOOD NEWS.
I THINK THIS IS WHAT WE SHOULD
DO.
I'M A BIT DISAPPOINTED WE
HAVEN'T SOLVED THIS BECAUSE THIS
WOULD TAKE CARE OF MUCH OF THE
REST.
>> THANK YOU.
>> WE NEED TO MEET AGAIN AND
WOULD LIKE TO CLOSE ON AN
OPTIMISTIC NOTE THIS WAS BY FAR
AT AN EARLIER STAGE DISCUSSION.
I LIKE HOW THE DISCUSSION
EVOLVED AND LOOKING AT THE
NETWORKS INDIVIDUALS HAVE.
IT'S AT A CRITICAL STEP.
WE HAVE DIFFERENT IDEAS HOW TO
EXPAND THE NETWORKS AND EACH HAS
A NETWORK.
I HAVE THREE NETWORKS I CAN
CONTRIBUTE.
LET'S JUST CONNECT AND THE
SURVEY OF GETTING THAT TOGETHER
AND MARCUS I'M GOING TO HARASS
YOU BECAUSE WE TALKED ABOUT
JOINING YOUR NETWORK AND I THINK
WE HAVE THE POWER TO CATALYZE
AND THE TIME IS NOW AND WE COULD
HAVE STEPS TO GO AND I LOVE THE
COMMUNITY THAT DEVELOPED
TOGETHER.
I WANTED TO THANK EVERYBODY FOR
CONTRIBUTING TO ALL OF THIS.
>> THANK YOU.
>> THANK YOU, EVERYONE.
>> I WOULD LOVE FOR US TO PUT
OUT SOMETHING VERY TANGIBLE AT
THE END OF THIS MEETING.
I'D LOVE FOR US TO MEET AGAIN
AND KEEP CONNECTED.
WE'RE NOW IN THE ERA THAT WE'RE
NOT AFRAID ANY MORE OF VIRTUAL
MEETINGS SO CONTINUE BUT I FOUND
IT POWERFUL TO PUT OUT A PAPER
LIKE AN OPINION PIECE, HOW CAN
WE GET THINGS TOGETHER AND SHARE
RESOURCES MAY BE A GOOD START
POINT TO WORK TOGETHER AND MAY
BE SEPARATE WORKING GROUPS BUT
BRING AN ACTUAL PAPER, SOMETHING
THAT THE SCIENTIFIC COMMUNITY
CAN SEE THE INVESTMENT OF ALL OF
US IN THE CONCEPT.
JUST A THOUGHT.
>> THANK YOU.
>> I DO WANT TO MENTION THAT WE
HAVE AGENCY HERE.
AND IT WAS MENTIONED WE'RE
FORCING GRADUATE STUDENTS OR THE
SYSTEM IS FORCING GRADUATE
STUDENTS TO DO THINGS THEY KNOW
ARE WRONG.
YOU DON'T HAVE TO FINISH A PH.D.
YOU DON'T HAVE TO NECESSARILY
WORK IN RESEARCH.
THAT'S EASILY SAID FROM MY POINT
OF VIEW.
NO ONE'S FORCING US TO DO THINGS
THAT ARE WRONG AND WE DON'T
ACCEPT THAT WHEN IT COMES TO
PLAGIARISM.
WE WOULDN'T SAY WE WERE FORCED
TO PLAGIARIZE AND WE DON'T
ACCEPT THAT AND THEREFORE WE
DON'T DO THAT.
WE'RE CAREFUL NOT TO DO THAT.
THE COST OF PLAGIARISM SEEM
DIFFERENT TO THE BAD PRACTICES
WE'VE SORT OF ACCEPTED AS
QUESTIONABLE THOUGH THEY'RE
QUITE BAD SO AGENCY.
>> MARCUS AND ULLI AND THEN
WE'LL HAVE TO CLOSE.
THERE'S A LOT WE CAN DO
COLLECTIVELY.
OUR CULTURE IS US.
WE'RE THE PEOPLE WHO REVIEW
PAPERS AND GRANTS AND MAKE
DECISIONS.
IT'S ALL US.
IF WE CHANGE COLLECTIVELY AND
UNIONIZE BY NOT PAYING
PROCESSING CHARGES AND
REDISCOVERING OUR SENSE OF
AGENCY AND PUSHING BACK AGAINST
THE LEARNED HELPLESSNESS THAT
EMPOWERED HIERARCHY AND ACADEMIA
CREATE WE CAN ACHIEVE A GREAT
DEAL AND WE ALSO NEED TO BE MORE
IMAGINATIVE AND THIS IS A
CHALLENGE TO YOU TO SOME EXTENT
WHAT IS THE ROLE OF JOURNALS IN
THE FUTURE VISION OF WHAT
SCIENCE AND ACADEMIA SHOULD LOOK
LIKE?
THE IDEA WE DISSEMINATE
KNOWLEDGE USING A 400-YEAR-OLD
MODEL STILL PREDICATED ON THE
ASSUMPTION WE DON'T PUBLISH
ONLINE AND CREATE PDFs AND POST
ONLINE AND THE WHOLE CONCEPT IS
DATE AND NEED TO LOOK AT THE
FUNCTION OF JOURNALS ARE IN THE
NEW FUTURE WE WANT TO MOVE
TOWARDS.
>> ABSOLUTELY.
>> I WOULD LIKE TO CHALLENGE THE
IDEA OF ANOTHER PAPER.
WE HAD SO MANY PAPERS.
WHO'S GOING READ THEM?
>> I'VE HEARD INTERESTING
ACTIVITIES AND STUFF I'VE NOT
KNOWN BEFORE TO MENTION HE'LL
-- HELIOS AND I RESEARCH SOME
THINGS THAT ARE EXTREMELY
HELPFUL.
MY VOTE WOULD BE TO PUT AN
EFFORT INTO COLLECTING THE
THINGS THE RIGOR CHAMPION
ACTIVITIES BECAUSE IT'S USEFUL
IN OUR DAILY WORK.
IF I CAN POINT TO OTHERS DOING
SIMILAR OR BETTER THINGS THIS IS
EXTREMELY HELPFUL.
BUT NOT ANOTHER PAPER.
>> CAN I SECOND THAT IN THIS
COVID ERA STILL SEEM TO STILL BE
IN AND ARE THERE OUTPUTS THAT
ARE COMMUNITY BUILDING AND WHAT
CAN EACH OF US TO CONTRIBUTE TO
BUILDING COMMUNITIES WITHIN OUR
ORGANIZATIONS AND WITHIN WITH
OTHER ORGANIZATIONS AND CAN THAT
BE PART OF THE METRIC?
>> I WANTED TO PUSH BACK AGAINST
THE IDEA OF THE PUSH BACK IN THE
PAPER.
FOR THE JUNIORS IN THE ROOM THEY
NEED THAT CACHE TO SHOW THEY'RE
PART OF THE COLLECTIVE BUT I
THINK OF THE PEOPLE WHO WOULD
NOT BENEFIT FROM THAT STRATEGY.
>> THE TIME CHECK FROM THE
ORGANIZERS, ARE WE GOING TO BE
THROWN OUT OR CAN WE GO ON?
WE ARE OFFICIALLY AFTER THE
CLOSING TIME BUT I DON'T WANT TO
CLOSE THE CONVERSATION BECAUSE
THERE'S TONS OF GREAT IDEAS, IF
YOU HAVE TO LEAVE WE UNDERSTAND.
WE'VE BEEN TALKING ABOUT THE
INCREDIBLE AMOUNT OF CREATIVITY
PEOPLE HAVE IN THE WAYS THEY
HAVE BEEN SUPPORTING EACH OTHER
BUILDING, CREATING.
IF A PAPER WOULD SATISFY SOME
FOL
FOLKS, GREAT.
BUT THERE'S AN OPPORTUNITY TO
POTENTIALLY CROWD SOURCE AN
INCREDIBLE AMOUNT OF RESOURCES
AND REACH OUT TO VARIOUS
COMMUNITIES.
PEOPLE ISOLATED WHO DON'T EVEN
KNOW THINGS LIKE THIS EXIST AND
MAY NOT HAVE THAT COMMUNITY AND
CONNECTION TO THEN START TO
BUILD THAT BECAUSE THERE'S
SOMETHING ABOUT GETTING UNDER
THE COMMUNITY ASPECT WE'VE BEEN
TALKING ABOUT AND NOT JUST THE
STUFF AND MOVING TOWARDS THE
STUFF.
AND PUSHED TO THE OUTCOME
ORIENTED IDEA AND WHAT CAN BE
CREATED TO BRING PEOPLE TOGETHER
IN USEFUL WAYS.
>> WE'LL TRY TO CLOSE AT 4:30.
>> I WANT DO SAY I 100% AGREE
WITH AND WHY WE'RE IN THE
SITUATION WE'RE IN.
WHY DID I SAY THAT?
BECAUSE I DISAGREE ABOUT THE
OTHER THING.
AND THE REASON I DISAGREE IS
THAT NIH WOULD NOT HAVE COME OUT
WITH THE RIGOR GUIDELINES IF IT
WASN'T FOR THE LANCET ET AL.
PAPER AND PROBABLY THE
EDUCATIONAL PLATFORM THE NINDS
IS GOING TO FUND IS WOULDN'T
HAVE HAPPENED IF WE DIDN'T
PUBLISH THE PAPER AFTER THE LAST
WORKSHOP BECAUSE THAT'S JUST HOW
THINGS ARE DONE.
IF WE WANT TO GET US -- NOT THE
COMMUNITY, BUT GET THE FUNDERS
OR THE OTHER ENTITIES THAT COULD
ACTUALLY BRING ABOUT CHANGE TO
ACT WE NEED TO GIVE THEM
SOMETHING TO DO THAT.
>> REGARDING THE TANGIBLE
ACTIONS WE COULD TAKE, I BELIEVE
ONE OF THE EFFECTIVE WAYS TO
CATALYZE COMMUNITIES IS TO LOOK
AT THE RESEARCH THAT EXISTS IN
THE UNIVERSITIES IN THE U.S.
WE DON'T HAVE ENOUGH ENERGY TO
PUSH THIS FORWARD GIVEN THE
MAGNITUDE AND COMPLEXITY OF THE
ISSUE.
WE NEED TO TRY AND BRING THEM IN
AND THEN USE THEIR POWER AS WELL
AS INSTITUTIONAL INTEGRATION TO
PUSH OUR IDEAS FORWARD.
AND WE CAN GO BACK TO THE
UNIVERSITY AND SAY WE DID THIS,
WHAT DO YOU THINK, HOW CAN WE
MAKE CHANGE.
DO YOU AGREE, DO YOU NOT
DISAGREE?
KEEP THE CONVERSATION GOING WITH
THEM.
I THINK ONCE WE BRING THEM IN, I
THINK IT WOULD CATALYZE CHANGE
AND THEN WE CAN TRY TO PROP UP
OUR OTHER NETWORK.
THE REASON I SAY THAT IS I THINK
MANY OF US HAVE BEEN INVOLVED IN
MANY OF THE OPEN SCIENCE
EFFORTS.
A LOT OF TIMES AND I THINK
LEVERAGING THE EXISTING ORIs IN
A WAY IS BASIC INTEGRITY IS THE
SAME AS RIGOR.
>> I THINK THERE'S SOMETHING AND
WHAT NAOMI HAS DONE IS AMAZING
BUT THERE'S A HIGH RISK NOT
DOING IT AS WELL AS SHE'S DONE
IT HAVING THE COMPLIANCE THERE'S
WORK TO DO HOW TO IT WELL.
>> I THINK RESEARCH INTEGRATE
SHOULD BE PART OF THIS.
>> I AGREE PAPERS HAVE VALUE AS
ACADEMIC OCCURRENCE AND PEOPLE
DO USE THAT TO GET JOBS BUT I
THINK WE CAN SHORTCUT IT AND
HIGHER THE ECRs AS THE
INSTITUTION LEADS FOR ACADEMIC
INTEGRITY.
>> TRACEY, LAST WORDS.
>> IT'S VALUE IN THINKING HOW TO
CREATE SUSTAINABLE COMMUNITIES
WITHIN OUR COMMUNITY FOR FUTURE
PROBLEM SOLVING AND DISCUSSIONS
AND SETTING UP SMALLER WORKING
GROUPS WHERE EVERYONE HAS A
CHANCE TO TALK ABOUT PROJECTS
THAT THEY'RE WORKING ON THAT ARE
SIMILAR AND SHARED INTERESTING
AND TALK HOW ARE YOU HANDLING IT
AND HOW CAN I LEARN FROM THAT
AND HOW CAN WE AMPLIFY AND
STRENGTHEN EFFORTS THROUGH THE
PROJECTS AND MAKING SURE THESE
ARE SMALL TARGETED GROUPS AND
WORTH THINKING ABOUT THE
DIFFERENT ROLES PEOPLE PLAY
WITHIN DIFFERENT STAKEHOLDER
GROUPS AND ORGANIZATIONS AND
OFFERING STRATEGIES PEOPLE CAN
USE WITHIN EACH OF THOSE ROLES
TO STRENGTHEN THE GOALS THAT
WE'RE TRYING TO ACHIEVE.
>> THANK YOU.
I THINK THIS THESE ARE VERY WISE
WORDS TO CLOSE.
IT'S 4:30.
THANK YOU SO MUCH TO EVERYONE OF
YOU FOR ALL THE ENERGY AND THE
CREATIVITY AND EVERYTHING YOU
BROUGHT TO THESE TWO DAYS.
IT'S BEEN INTENSE AND AMAZING.
THANK YOU VERY MUCH THANKS AGAIN
TO THE HOST AND SHY AND DEVAN
AND EVERYTHING THEY'VE DONE TO
MAKE THIS POSSIBLE.
I THINK THEY DESERVE A ROUND OF
APPLAUSE.
>> ON BEHALF OF NINDS I WANT TO
THANK YOU ALL FOR BEING HERE AND
TAKING THE TIME OUT AND COMING
TO AN IN-PERSON MEETING IT'S THE
FIRST WE HELD IN TWO YEARS AND
WE THINK IT WAS VALUABLE AND
WANT TO THANK YOU ALL FOR BEING
HERE.
THANK YOU.