arxiv Preprint – FACET: Fairness in Computer Vision Evaluation Benchmark

In this episode we discuss FACET: Fairness in Computer Vision Evaluation Benchmark
by Laura Gustafson, Chloe Rolland, Nikhila Ravi, Quentin Duval, Aaron Adcock, Cheng-Yang Fu, Melissa Hall, Candace Ross. The paper introduces a new benchmark called FACET, which measures performance disparities in computer vision models across different attributes such as gender and skin tone. It consists of a large evaluation set of 32k images and expert reviewers manually annotated person-related attributes. The benchmark revealed performance disparities across demographic attributes and aims to contribute to the development of fairer and more robust vision models.