Overview

Contrary to compile-time solutions, system level optimization can maximize application performance in the face of growing hardware diversity. Because system-level automated (or, self-) optimization is rare in practice, we approach it conservatively by attaching to the Streamline data- and transport planes a control system that is both demonstrably utilitarian and fully comprehensible. In application tailoring, we distinguish between optimization itself, automation of this process and execution of the chosen solution in a real system.

Optimization (6.2Optimization)

is the basis for high throughput on ranges of hard- and software. Streamline applications are pipelines with competing candidate streams and filters. Streamline enables selection through filter name reification, buffer specialization and - stacking and graph substitution.

Automation (6.3Automation)

supplants tedious manual system administration. Streamline computes a network flow program that incorporates all application choice and relevant system features (e.g., cache layout) and that evaluates all possible configurations in tens of milliseconds to demonstrate practicality of online quantitative optimization.

Execution (6.4Control System)

builds an application that implements the selected configuration. Streamline extends basic resource discovery and pipeline construction with transactions for fault tolerance, graph sharing for increased performance and just-in-time compilation and loading of filters for embedded operation and further increased performance.

Fine-grained Access Control (6.5Fine-grained Access Control)

safely grants access to kernel and devices to untrusted users. Streamline reuses Unix file permission where possible and supplements these with safe languages and mandatory stream sanitization. Besides Unix-type isolation, fine grained permissions enable trading security for throughput on a case-by-case basis.

false and analyzes which will, given a set of design decisions, consistently select the best fit at the smallest possible cost. These two sections form the core exposition on optimization automation. They are - by design - ignorant, however, of the practical issues concerning application to a specific problem. Section 6.3Automation constructs a model of I/O application tailoring in Streamline, transforms the intuitive model in one suitable for automated logic and explains how this logic interacts with its ecosystem through sensors and actuators.