Towards Logging and Preserving the Entire History of Distributed Systems

Abstract

Towards Logging and Preserving the Entire History of Distributed Systems: a Sistributed Systems Memex

The implications of archiving large amounts of daily information for science and society are clear since at least the 1940s, when Vannevar Bush defined the concept of the personal memex as an individual’s device for storing and accessing all information and communication involving that individual. Among these benefits are learning about and eradicating humankind diseases, enabling human beings more creative and thought-related time by eliminating tasks that can be automated, etc. Similarly, we posit that archiving large amounts of operational traces collected from the many distributed systems that currently underpin societies across the world would be beneficial for tuning today’s systems and designing better systems in the future. What is the Distributed Systems Memex? How can such a Memex be designed and implemented? To address these and related questions, we have taken a bottom-up approach, in which we focus on the archival needs of specific application areas in which distributed systems are prominent and hope to gain sufficient understanding for the future.

In the mid 1990s, the grid computing community promised the “compute power grid,” a utility computing infrastructure for scientists and engineers. Since then, a variety of grids have been built worldwide, for academic purposes, specific application domains, and general production work. The Grid Workloads Archive (GWA) [4] is a workload data exchange and a meeting point for the grid community. We have defined the requirements for building a workload archive and have described the approach taken to meet these requirements with the GWA. We have introduced a format for sharing grid workload information and the tools associated with this format. Using these tools, we have collected, ingested in the archive, and processed data from over fifteen well-known grid environments, covering thousands of scientists submitting millions of jobs over a period of over twenty operational years, and with working environments spanning hundreds of operational sites comprised of tens of thousands of processing machines.

Resource failures are currently common in distributed systems, likely as a side-effect of the increasing complexity and scale of these systems in real- life deployments. To facilitate the design, validation, and comparison of fault-tolerant models and algorithms, we have created the Failure Trace Archive (FTA) [2] as an online public repository of availability traces taken from diverse parallel and distributed systems. We have designed a new data format and a toolbox that facilitates automated analysis of trace data sets. We have collected, ingested in the archive, and processed data from over twenty-five well-known distributed systems, covering millions of users over a period of over twenty operational years, and with working environments spanning the entire world and millions of computers. Peer-to-Peer (P2P) systems have gained a phenomenal popularity in the past few years; among them, BitTorrent alone serves daily tens of millions of people and generates an important fraction of the Internet traffic. Measurement data collected from real P2P systems are fundamental for gaining solid knowledge of the usage patterns and the characteristics of these systems and can improve the modeling, the design, and the evaluation of P2P systems. We have created the P2P Trace Archive (P2PTA) [3], an archive that facilitates the collection and exchange of P2P traces. Currently, the P2PTA hosts over twenty traces of various P2P applications (from file-sharing to VoIP to video-streaming), with over 60 million sessions, tens of millions of content items, and multiple years of system operation.

Spurred by the rapid development of the gaming industry and the expansion of Online Meta-Gaming Networks (OMGNs), we have designed the Game Trace Archive (GTA) [1] to be a virtual meeting space for the game community. We have proposed a unified format for game traces and introduced a number of tools associated with the format. With these tools, we have collected, processed, and analyzed 9 traces of both games and OMGNs. We have already collected in the GTA traces corresponding to more than 8 million real players and more than 200 million information items, spanning over 14 operational years. We also show that the GTA can be extended to include a variety of real-game trace types. Our work in designing and building a Distributed Systems Memex has only just begun.

References
[1] Yong Guo, Alexandru Iosup: The Game Trace Archive. NetGames 2012: 1-6
[2] Derrick Kondo, Bahman Javadi, Alexandru Iosup, Dick H. J. Epema: The Failure Trace Archive: Enabling Comparative Analysis of Failures in Diverse Distributed Systems. CCGRID 2010: 398-407. Best paper award.
[3] Boxun Zhang, Alexandru Iosup, Johan Pouwelse, and Dick Epema. 2010. The peer-to-peer trace archive: design and comparative trace analysis. In Proceedings of the ACM CoNEXT Student Workshop (CoNEXT ’10 Student Workshop). ACM, New York, NY, USA, Article 21.
[4] Alexandru Iosup, Hui Li, Mathieu Jan, Shanny Anoep, Catalin Dumitrescu, Lex Wolters, Dick H. J. Epema: The Grid Workloads Archive. Future Generation Comp. Syst. 24(7): 672-686 (2008)
URL: http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=12472