Firmament: data-flow computation over heterogeneous resources - Talk by Malte Schwarzkopf

Title: Firmament: data-flow computation over heterogeneous resources
Speaker: Malte Schwarzkopf, University of Cambridge Computer Laboratory
Host: Andrew Warfield

Abstract:  Heterogeneity at many levels abounds even in the poster child of parallel programming: today's seemingly homogeneous data centres. In this talk, I will motivate the development of a new task-parallel data-flow framework, called Firmament, which makes resource heterogeneity a first-class citizen in task scheduling. Unlike previous approaches, Firmament aims to pro-actively anticipate the impact of scheduling decisions, rather than retrospectively working around bad decisions. Furthermore, Firmament adds support for contained non-determinism to the task-parallel programming model, allowing heterogeneity to be exposed to tasks, which can make introspective decisions based on it. Finally, Firmament can operate in a master-less mode, making it suitable for environments even more heterogeneous and less tightly interconnected than a data centre. This talk covers work under active development, with some parts still open to speculation; interaction and feedback is thus welcome and encouraged.

Speaker bio  Malte is a PhD student at the University of Cambridge Computer Laboratory, working on the boundary between operating systems and distributed systems. He is supervised by Steve Hand, and one of the authors of the NSDI 2011 paper on the CIEL universal execution engine for distributed data-flow computing.