Скачать или смотреть Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API

Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API
Zining Ye, Xinran Yuan, Shaurya Gaur, Aaron Halfaker, Jodi Forlizzi, Haiyi Zhu

DIS '21: ACM Designing Interactive Systems Conference 2021
Session: AI

Abstract
With the growing industry applications of Artificial Intelligence (AI) systems, pre-trained models and APIs have emerged and greatly lowered the barrier of building AI-powered products. However, novice AI application designers often struggle to recognize the inherent algorithmic trade-offs and evaluate model fairness before making informed design decisions. In this study, we examined the Objective Revision Evaluation System (ORES), a machine learning (ML) API in Wikipedia used by the community to build anti-vandalism tools. We designed an interactive visualization system to communicate model threshold trade-offs and fairness in ORES. We evaluated our system by conducting 10 in-depth interviews with potential ORES application designers. We found that our system helped application designers who have limited ML backgrounds learn about in-context ML knowledge, recognize inherent value trade-offs, and make design decisions that aligned with their goals. By demonstrating our system in a real-world domain, this paper presents a novel visualization approach to facilitate greater accessibility and human agency in AI application design.

DOI:: https://doi.org/10.1145/3461778.3462099
WEB:: https://dis.acm.org/2021

Presentation Videos for ACM DIS 2021 Conference Papers & Pictorials