How to connect values in decision theory, instruments in psychometrics, and norms in classical sociology and anthropology?
Many difficulties have emerged from decades of striving to model human behavior and experience on the basis of utility (e.g. wellbeing, preferences, and other notions from behavioral economics and decision theory). That framework assumes that goals and values are a stable or slowly-changing property of individuals, and that the object of theory is to explain shorter time scales: how humans will (or should) therefore use resources and strategies to achieve these goals.
When dissecting that framework, we should avoid naive objections: humans do occasionally behave in such a way, and it may have been fair to hope that these occasional behaviors would help investigate the complexity of human values. Much like a physical instrument may convert temperature or forces into a more easily measurable electric current, revealed preferences were hoped to convert subjective properties into objectively-measurable actions in a market setting. Yet it seems to us that the physical instrument succeeds because all forms of physical energy and work are really interchangeable, whereas in the second case, what is measured is very specific to the market setting - and so we end up making a theory of the instrument (the electric current) rather than of the target (temperature or forces).
Rather than modeling humans as optimizing (or even satisficing) agents, and slowly complicating their objective function to explain anomalous behaviors (along the lines of prospect theory), we believe that a more elegant formulation can be constructed by drawing inspiration from:
- Generative and multiagent models of the mind: the free energy principle, predictive processing, etc.
- Psychological results on conformity, influence and learning
- Social norms and cultural anthropology
Together, these sources draw a picture of humans as being in a constant process of discovering and reevaluating their own objectives. Classical decision theory is not well-equipped to deal with situations where objectives change at least as much as actions -- for instance, where an individual can deduce their own objective from what they are currently doing (resulting in commitment and consistency "biases").