Refine
Document Type
- Article (3)
- Monograph/Edited Volume (1)
- Doctoral Thesis (1)
Is part of the Bibliography
- yes (5)
Keywords
- heuristics (5) (remove)
Learning from failure
(2022)
Regression testing is a widespread practice in today's software industry to ensure software product quality. Developers derive a set of test cases, and execute them frequently to ensure that their change did not adversely affect existing functionality. As the software product and its test suite grow, the time to feedback during regression test sessions increases, and impedes programmer productivity: developers wait longer for tests to complete, and delays in fault detection render fault removal increasingly difficult.
Test case prioritization addresses the problem of long feedback loops by reordering test cases, such that test cases of high failure probability run first, and test case failures become actionable early in the testing process. We ask, given test execution schedules reconstructed from publicly available data, to which extent can their fault detection efficiency improved, and which technique yields the most efficient test schedules with respect to APFD?
To this end, we recover regression 6200 test sessions from the build log files of Travis CI, a popular continuous integration service, and gather 62000 accompanying changelists. We evaluate the efficiency of current test schedules, and examine the prioritization results of state-of-the-art lightweight, history-based heuristics. We propose and evaluate a novel set of prioritization algorithms, which connect software changes and test failures in a matrix-like data structure.
Our studies indicate that the optimization potential is substantial, because the existing test plans score only 30% APFD. The predictive power of past test failures proves to be outstanding: simple heuristics, such as repeating tests with failures in recent sessions, result in efficiency scores of 95% APFD. The best-performing matrix-based heuristic achieves a similar score of 92.5% APFD. In contrast to prior approaches, we argue that matrix-based techniques are useful beyond the scope of effective prioritization, and enable a number of use cases involving software maintenance.
We validate our findings from continuous integration processes by extending a continuous testing tool within development environments with means of test prioritization, and pose further research questions. We think that our findings are suited to propel adoption of (continuous) testing practices, and that programmers' toolboxes should contain test prioritization as an existential productivity tool.
In many revenue management applications risk-averse decision-making is crucial. In dynamic settings, however, it is challenging to find the right balance between maximizing expected rewards and minimizing various kinds of risk. In existing approaches utility functions, chance constraints, or (conditional) value at risk considerations are used to influence the distribution of rewards in a preferred way. Nevertheless, common techniques are not flexible enough and typically numerically complex. In our model, we exploit the fact that a distribution is characterized by its mean and higher moments. We present a multi-valued dynamic programming heuristic to compute risk-sensitive feedback policies that are able to directly control the moments of future rewards. Our approach is based on recursive formulations of higher moments and does not require an extension of the state space. Finally, we propose a self-tuning algorithm, which allows to identify feedback policies that approximate predetermined (risk-sensitive) target distributions. We illustrate the effectiveness and the flexibility of our approach for different dynamic pricing scenarios. (C) 2020 Elsevier Ltd. All rights reserved.
In dynamic decision problems, it is challenging to find the right balance between maximizing expected rewards and minimizing risks. In this paper, we consider NP-hard mean-variance (MV) optimization problems in Markov decision processes with a finite time horizon. We present a heuristic approach to solve MV problems, which is based on state-dependent risk aversion and efficient dynamic programming techniques. Our approach can also be applied to mean-semivariance (MSV) problems, which particularly focus on the downside risk. We demonstrate the applicability and the effectiveness of our heuristic for dynamic pricing applications. Using reproducible examples, we show that our approach outperforms existing state-of-the-art benchmark models for MV and MSV problems while also providing competitive runtimes. Further, compared to models based on constant risk levels, we find that state-dependent risk aversion allows to more effectively intervene in case sales processes deviate from their planned paths. Our concepts are domain independent, easy to implement and of low computational complexity.
In many businesses, firms are selling different types of products, which share mutual substitution effects in demand. To compute effective pricing strategies is challenging as the sales probabilities of each of a firm's products can also be affected by the prices of potential substitutes. In this paper, we analyze stochastic dynamic multi-product pricing models for the sale of perishable goods. To circumvent the limitations of time-consuming optimal solutions for highly complex models, we propose different relaxation techniques, which allow to reduce the size of critical model components, such as the state space, the action space, or the set of potential sales events. Our heuristics are able to decrease the size of those components by forming corresponding clusters and using subsets of representative elements. Using numerical examples, we verify that our heuristics make it possible to dramatically reduce the computation time while still obtaining close-to-optimal expected profits. Further, we show that our heuristics are (i) flexible, (ii) scalable, and (iii) can be arbitrarily combined in a mutually supportive way.
Heuristiken der Urteilsbildung umfassen bottom-up bzw. schemagesteuerte Strategien innerhalb der sozialen Informationsverarbeitung, mit deren Hilfe trotz unsicherer Datenlage hinreichend genaue Urteile gefällt werden können. Die Anker- und Anpassungsheuristik als eine Form solcher Faustregeln beschreibt im Wesentlichen die Wirkung von vorgegebenen Zahlen (den so genannten Ankerwerten) auf numerische Schätzungen. Urteile unter Unsicherheit sind zum Beispiel im Bereich der Rechtsprechung zu beobachten, wobei die Entscheidungsprozesse hier eher normativ auf der Basis der vorliegenden Informationen, d.h. einer datengesteuerten Verarbeitung, erfolgen sollten. In einer Serie von drei Experimenten wurde die Ankerheuristik auf den Bereich der Rechtsprechung übertragen. Mit Hilfe der Vignettentechnik wurden N = 229 Rechtsreferendare sowie N = 600 Studierende der Rechtswissenschaften zu ihrem Strafverhalten befragt. Im Mittelpunkt standen drei Zielsetzungen: (1) die Replikation und Erweiterung der Ankereffekts in Bezug auf eine größere Gruppe von Deliktarten; (2) die Analyse individueller Unterschiede in der Ankernutzung unter Berücksichtigung verschiedener Persönlichkeitsvariablen (Need for Cognition und Need for Cognitive Closure) sowie (3) die Anregung zu verstärkter systematischer Informationsverarbeitung durch die Indizierung einer Genauigkeitsmotivation. Der Ankereffekt in der juristischen Urteilsfindung konnte für die verschiedenen Deliktgruppen repliziert werden. Die Ergebnisse zeigen, dass die wahrgenommene Schwere der geschilderten Taten mit dem Strafmaß korrelierte. Dieser Zusammenhang wurde durch die Einführung von Ankerwerten deutlich reduziert. Entgegen den bisherigen Untersuchungen war zwar auch bei den Rechtsreferendaren ein Ankereffekt zu beobachten, der jedoch geringer ausfiel als bei den Studierenden der Rechtswissenschaften. Im Hinblick auf die Persönlichkeitsmerkmale konnte die Erwartung bestätigt werden, dass ein geringes Kognitionsbedürfnis sowie ein hohes Geschlossenheitsbedürfnis mit höherer Anfälligkeit für die Ankerheuristik einhergehen. Die Erzeugung eines Rechtfertigungsdrucks dagegen veranlasste die Probanden, sich intensiver mit den Materialien zu beschäftigen und eher datengeleitet vorzugehen. Implikationen für die juristische Praxis werden diskutiert.