Evaluating Approaches to Resource Demand Estimation

Motivation

Managing the performance, availability and resource efficiency of running software systems proactively requires techniques to predict the system’s performance and resource consumption. Typically, performance predictions are based on performance models that capture the performance-relevant aspects of the considered software system. Building performance models involves the estimation of resource demands, i.e., estimating the time a unit of work spends obtaining service from a resource.
There exist already many different resource demand estimation methods, e.g., based on regression analysis or stochastic filtering, differing in their accuracy, their robustness and their applicability. For instance, there are notable differences in the amount and type of measurement data that is required as method input. However, to the best of our knowledge, a comprehensive evaluation and comparison of these methods in a realistic context does not exist.

Goals

Starting with a literature research on existing resource demand estimation methods, you should develop a classification scheme for the evaluation of the different approaches. The focus will be on estimation techniques that are applicable during system operation which implies that required measurements must not impose a significant overhead. This indicates that the evaluation has to take the characteristics (e.g., overhead, accuracy) of existing system monitoring approaches into account.
A selected set of methods should be implemented and evaluated in a realistic experiment environment we will provide. We will work with the novel SPECjEnterprise2010 Benchmark deployed on a cluster of Oracle WebLogic Java application servers.

Contact person: Fabian Brosig