Global-Scale Applications Rely on Datacenters, Datacenters Rely on Scalable Computer Systems

Technology Masterclass at Delft International Festival of Technology

Download PDF Slides


Google, Facebook, Amazon are all major tech companies that rely on scalable computer systems to survive. To cope with increasing computation demands and with the data deluge, we have already started to build complex hardware and software systems of systems (ecosystems), which a global user community accesses as cloud services. These users demand high performance or high throughput, and may switch at any time among the hundreds of service providers and technologies. This masterclass focuses on interesting new challenges in the operation of the datacentres that form the infrastructure of cloud services, in particular supporting the dynamic workloads of demanding users, ensuring various forms of scalability, and also looking at efficiency and fairness. You will learn here vital skills for the industry: if we succeed in addressing these challenges, we may not only enable the advent of big science and engineering, and the almost complete automation of many large-scale processes, but also reduce the ecological footprint of datacentres and the entire ICT industry. We will combine theory and practice, and talk about how we won the prestigious IEEE Scale Challenge 2014.

Elastic Big Data and Computing
[1] B. Ghit, N. Yigitbasi (Intel Research Labs, Portland), A. Iosup, and D. Epema. Balanced Resource Allocations Across Multiple Dynamic MapReduce Clusters. SIGMETRICS 2014 (elastic analytics)
[2] L. Fei, B. Ghit, A. Iosup, D. H. J. Epema: KOALA-C: A task allocator for integrated multicluster and multicloud environments. CLUSTER 2014: 57-65
[3] K. Deng, J. Song, K. Ren, A. Iosup: Exploring portfolio scheduling for long-term execution of scientific workloads in IaaS clouds. SC 2013: 55

Time-Based Analytics
[4] B. Ghit, M. Capota, T. Hegeman, J. Hidders, D. Epema, and A. Iosup. V for Vicissitude: The Challenge of Scaling Complex Big Data Workflows. Winners IEEE Scale Challenge 2014

Graph Processing
[5] A. L. Varbanescu, M. Verstraaten, C. de Laat, A. Penders, A. Iosup, H. J. Sips: Can Portability Improve Performance?: An Empirical Study of Parallel Graph Analytics. ICPE 2015: 277-287