Project Similarity Bias and Variance Neglect in Forecast Metric Evaluation

Abstract

Business executives often have to allocate resources across very dissimilar projects. They use financial measures, such as Net Present Value (NPV) that simplify this difficult comparison because they aim to be equally applicable to any kind of project, but these measures vary in their reliability. Psychological research suggests that comparing alignable objects will be easier than comparing non-alignable objects (Markman & Gentner, 1993; Markman & Medin, 1995). However, it is unclear how alignment might moderate people’s use of financial measure such as NPV. We found that laypeople accommodate their use of a financial measure (NPV) based on its reliability (as explicitly described in the introduction to the task) when allocating resources to a set of alignable projects, but use it regardless of reliability when allocating to a set of non-alignable projects. However, when NPV reliability information was presented numerically using ranges, participants’ allocation did not depend on the ranges—participants used NPV even when they had an opportunity to use the intrinsic features of the project. Overall, however, participants relied on NPV more when projects were low in alignment than when they were high in alignment. The result with numerical reliability was replicated with Masters of Management students. Our results demonstrate that considering dissimilar choices may hinder people’s ability to evaluate their importance, and that people might not be using useful variance information in their decisions.