NIH VideoCast - Battling the Matthew Effect: Can Search Results be Fair?

Skip Navigation

Battling the Matthew Effect: Can Search Results be Fair?

Loading... Loading video...

67 Views

Air date:	Thursday, September 28, 2023, 2:00:00 PM Time displayed is Eastern Time, Washington DC Local
Views:	Total views: 67 (35 Live, 32 On-demand)
Category:	Special
Runtime:	00:59:39
Closed captioning:	Check box to display CC outside of the video Note: You can drag the captioning window around and resize it
Description:	Search engines are mediators between producers and consumers of information, and the order in which an engine presents its results affects both stakeholder groups in significant ways (e.g., economic livelihood). Top-ranked items provide their producers with much more exposure to the consumer than do lower-ranked items. If a search engine repeatedly returns comparably relevant items in the same order, the rich (more exposed) items will keep getting richer while lower-ranked items will get less and less relative exposure. The Text REtrieval Conference (TREC, trec.nist.gov) project at NIST contained a Fair Ranking track from 2019--2022 to study the problem of producing fair rankings. The track defined fairness as providing appropriate exposure to items over a set of searches, where `appropriate exposure' encompasses both relevance and rank within the search results. Organizers of the track faced multiple challenges in operationalizing the evaluation task that system implementers will also face. Chief among these challenges are defining the attributes the engine should be fair with respect to (there isn't a universally fair ranking) and having enough information regarding those attributes to implement some desirable policy. Bio: Ellen Voorhees is a scientist emeritus in the Information Technology Laboratory at the U.S. National Institute of Standards and Technology (NIST). Her primary responsibility at NIST was managing the Text REtrieval Conference (TREC), an international workshop series that defines the methodology and builds the infrastructure required for large-scale evaluation of search and related technologies. Her research focuses on developing and validating appropriate evaluation schemes to measure system effectiveness in these areas. Voorhees is a fellow of the ACM, a member of the ACM SIGIR Academy, and has been elected as a fellow of the Washington Academy of Sciences. She has published numerous articles on information retrieval techniques and evaluation methodologies and serves on the review boards of several journals and conferences.

Author:	Ellen Voorhees, Scientist Emeritus, NIST

CIT Live ID:	52214
Permanent link:	https://videocast.nih.gov/watch=52214

Related Videos...