Adaptive Framework for Configuration Tuning

Modern software systems are often equipped with hundreds to thousands of configurationoptions, many of which greatly affect performance. However, determining the best configuration is difficult, both because it requires some knowledge of software internals, and often the best configuration changes due to unpredictable changes in workload or operating environment. Among all types of configurations, performance-sensitive configurations (PerfConfs) are challenging to set because they represent tradeoffs; e.g. , between memory usage and response time. Aiming for automatic configuration tuning and improve the modern software performance and reliability, this dissertation works on these three parts and makes the following contributions: First, this dissertation conducts an empirical study to understand performance-sensitive configurations and the challenges of setting them in the real-world. We look at 80 developerpatches and 54 user-posts concerning PerfConfs in 4 widely used large-scale systems. The study reveals several main findings: (1) about half of PerfConfs threaten hard performance constraints like out-of-memory or out-of-disk problems; (2) about half of PerfConfs affect performance indirectly through setting thresholds for other system variables; (3) more than half of PerfConfs are associated with specific system events and hence only take effect conditionally; and (4) often different configurations affect the same performance goal simultaneously, requiring coordination. Second, guided by our study, we design a systematic and general control-theoretic framework, SmartConf , to automatically set and dynamically adjust performance-sensitive configurations to meet required operating constraints while optimizing other performance metrics. Evaluation shows that SmartConf is effective in solving real-world configuration problems, often providing better performance than even the best static configuration developers can choose under existing configuration systems. Third, observed that existing self-adaptive approaches are required to configure a new set of internal configurations (AdapConfs), we take a different approach: building self-adaptive software with as few externally configurable components as possible. Specifically, we create a framework—called AgileCtrl—by extensively modifying an existing control-theoretic framework for self-adaptive software. AgileCtrl monitors the quality of its adaptations and reconfigures its own internals to provide even greater robustness in the face of user error or unexpectedly volatile environments. We evaluate AgileCtrl by comparing against other self-adaptive frameworks that require careful human tuning. Across a number of case studies, we find AgileCtrl can withstand user errors of up to 106× while achieving similar performance.