Abstract: Even if algorithms make better forecasts than human decision-makers on average, humans may have “private” information which the forecasting algorithm does not have access to that can improve algorithmic forecasts. How can we help humans effectively use and adjust recommendations made by prediction algorithms in such situations? We propose a mathematical model that captures how humans combine information they directly observe (some of which is private) with an algorithmic prediction to make a final demand forecast. In the model, humans take a weighted average between their own forecast and the algorithm’s, where the weights depend only on the aggregate relative historical performances. This leads to humans over-adjusting the algorithm’s predictions when it performs well and under-adjusting the algorithm’s predictions when it performs poorly. We validate our model using a lab experiment where participants are tasked with making demand forecasts for 20 products while having access to an algorithm’s recommendations. In a follow-up experiment, we leverage these insights to inform and design algorithmic transparency to help humans use the private information more effectively. We find that feature transparency – even when the underlying algorithm is a black box – helps users incorporate algorithmic recommendations in their decisions better than other types of transparency that increase algorithmic adherence.

Bio: Kris Ferreira is an Assistant Professor at Harvard Business School in the Technology and Operations Management (TOM) Unit. She teaches the Supply Chain Management course in the MBA elective curriculum and analytics in various Executive Education programs. In her research, Kris focuses on helping e-commerce companies make more profitable revenue-management decisions, including pricing, assortment planning, ranking, etc. For much of this work, she partners with online retailers to design new machine learning and optimization algorithms that use the massive amount of data at their fingertips to improve their decision making in this domain. Most recently, Kris has started studying how users make decisions with such algorithmic input, and correspondingly how to improve algorithmic development and deployment. Kris’s work has been awarded the M&SOM Best OM Paper in OR Award, two-time finalist for the M&SOM Best Paper Award, first place for the Revenue Management and Pricing Section Practice Award, finalist for the M&SOM Practice-Based Research Competition, and first place and finalist for the Innovative Applications in Analytics Award. Kris earned her PhD in Operations Research at the Massachusetts Institute of Technology and her BS in Industrial and Systems Engineering at the Georgia Institute of Technology. Before entering graduate school, she was a supply chain consultant for Alvarez & Marsal.

Meeting ID: 971 7550 2219
Passcode: 210402

Zoom link for the talk