Massivizing Computer Systems: VU on Sustainable Distributed Computer Ecosystems

Abstract

Wherever we look, our society is turning digital. Science and engineering, business-critical and government operations, and online education, shopping, and gaming rely increasingly on digitalization---on core processes powered by data, AI, and computers. For digitalization to succeed, we must integrate computer systems into larger _ecosystems_, effectively and efficiently managed. However successful until now, we cannot take computer ecosystems for granted: they energy use and other climate impact raise an unprecedented sustainability challenge that we cannot solve merely through higher spending, and which will not go away as we focus on the more traditional goals related to performance, scalability, and dependability.

In this talk, inspired by this challenge and our experience with distributed computer ecosystems for over 15 years, we focus on understanding the sustainability cost of deploying, scaling, and evolving computer ecosystems, and on what we can do to alleviate this cost.

We present scientific and technical results achieved together with our VU team and partners, motivated by an ambitious research, development, and innovation program, where the key is understanding the combined computer ecosystem (the full stack) rather than only its individual, small-scale computer systems (the stack's components). We present a pyramid of metrics for different kinds of stakeholders and a framework of short- and long-term operational techniques that, combined, can address these metrics. On the data side, we introduce the computing continuum Memex to record long-term operational signals, and make them available efficiently and meaningfully. We showcase how digital twinning, which combines advanced telemetry ingestion from the Memex with sophisticated simulation, can help understanding sustainability issues in the context of other metrics, and steering the ecosystem toward improved operations.

Although, in a scientific sense, we can only show early results - only a handful of teams working only for the past 5 years -, they are obtained through controlled real-world experiments, multi-year observation of real-world systems, and what-if analysis of short- and long-term scenarios, and they are accompanied by open-science reproducibility packages and educational materials. We encourage more research teams to leverage these software tools and data, collaborate with us, and join this vision - aligned with the IPN SIG on Future Computer Systems and Networking [1] (but any mistakes remain the speaker's), and with the work of the Cloud group of SPEC RG [2]. Building a science of sustainability for distributed computer ecosystems is paramount for the 21st century!


[1] Future Computer Systems and Networking Research in the Netherlands: A Manifesto, 2022. [Online] https://arxiv.org/pdf/2206.03259
[2] SPEC RG Cloud https://research.spec.org/working-groups/rg-cloud/