Heuristic mean-variance optimization in Markov decision processes using state-dependent risk aversion
- In dynamic decision problems, it is challenging to find the right balance between maximizing expected rewards and minimizing risks. In this paper, we consider NP-hard mean-variance (MV) optimization problems in Markov decision processes with a finite time horizon. We present a heuristic approach to solve MV problems, which is based on state-dependent risk aversion and efficient dynamic programming techniques. Our approach can also be applied to mean-semivariance (MSV) problems, which particularly focus on the downside risk. We demonstrate the applicability and the effectiveness of our heuristic for dynamic pricing applications. Using reproducible examples, we show that our approach outperforms existing state-of-the-art benchmark models for MV and MSV problems while also providing competitive runtimes. Further, compared to models based on constant risk levels, we find that state-dependent risk aversion allows to more effectively intervene in case sales processes deviate from their planned paths. Our concepts are domain independent,In dynamic decision problems, it is challenging to find the right balance between maximizing expected rewards and minimizing risks. In this paper, we consider NP-hard mean-variance (MV) optimization problems in Markov decision processes with a finite time horizon. We present a heuristic approach to solve MV problems, which is based on state-dependent risk aversion and efficient dynamic programming techniques. Our approach can also be applied to mean-semivariance (MSV) problems, which particularly focus on the downside risk. We demonstrate the applicability and the effectiveness of our heuristic for dynamic pricing applications. Using reproducible examples, we show that our approach outperforms existing state-of-the-art benchmark models for MV and MSV problems while also providing competitive runtimes. Further, compared to models based on constant risk levels, we find that state-dependent risk aversion allows to more effectively intervene in case sales processes deviate from their planned paths. Our concepts are domain independent, easy to implement and of low computational complexity.…
Author details: | Rainer SchlosserORCiDGND |
---|---|
DOI: | https://doi.org/10.1093/imaman/dpab009 |
ISSN: | 1471-678X |
ISSN: | 1471-6798 |
Title of parent work (English): | IMA journal of management mathematics / Institute of Mathematics and Its Applications |
Publisher: | Oxford Univ. Press |
Place of publishing: | Oxford |
Publication type: | Article |
Language: | English |
Date of first publication: | 2021/05/17 |
Publication year: | 2022 |
Release date: | 2023/01/02 |
Tag: | Markov decision process;; dynamic pricing; dynamic programming; heuristics; mean-variance optimization; risk aversion |
Volume: | 33 |
Issue: | 2 |
Number of pages: | 19 |
First page: | 181 |
Last Page: | 199 |
Organizational units: | An-Institute / Hasso-Plattner-Institut für Digital Engineering gGmbH |
DDC classification: | 5 Naturwissenschaften und Mathematik / 51 Mathematik / 510 Mathematik |
Peer review: | Referiert |