Storage Systems, MSc, VU (2020-ongoing)

The goal of the course is to teach students about the rise of Non-Volatile Memory (NVM) storage technologies in commodity computing, their impact on system design (architecture, operating system), distributed systems, storage services, and application designs.

Syllabus

Storage System is a unique course due to its sole focus on NVM storage and its impact on research and education. We take inspiration from the 2018 Data Storage Research Vision 2025 report (https://dl.acm.org/doi/book/10.5555/3316807) which identifies (see section 6.1):

"Many students may only associate storage systems with hard disk drives or a specific file system, which is obviously less attractive compared to, say, self-driving cars. This situation is partly due to the fact that there is no clearly defined course on storage systems in the majority of universities."

Group Photo

The new Storage-Memory Continuum that this course covers.

Lecture slides

Storage Systems (VU catalogue number XM_0092) is a MSc-level course that is first established and offered in 2020. The course covers the rise of Non-Volatile Memory (NVM) storage technologies in commodity computing, their impact on system design (architecture, operating system), distributed systems, storage services, application designs, and emerging trends (CXL and io_uring). We cover the following topics in 2023:

Practical Work

For the practical work, students develop an NVM flash translation layer FTL (the essential part of any modern NVM storage device) for NVMe ZNS devices, and integrate a file system in RocksDB. There are five milestone in the practical work:

  1. A new device is in town - setup the development environment with ZNS devices in QEMU and read the NVMe 1.4 and ZNS specifications, and test the nvme command to interact with nvme devices. Download PDF
  2. I can’t read, is there a translator here? - implement a host-side hybrid log-data FTL. The log segment is page-mapped, while the data-segment is zone-mapped. No GC at this stage. Download PDF
  3. It’s 2023, we recycle - implement a choice of garbage collection algorithm for your FTL. Download PDF
  4. We love Rock(sDB) ‘n’ Roll! - design and implement a file system on top of your FTL and integrate it with the RocksDB FileSystem API. Download PDF
  5. Wake up, Neo - the last milestone requires you to persist and restart your FTL and filesystem states and pass the RocksDB persistency tests. Download PDF

The project handbook is publicly available here :download:2023-2024-stosys-handbook-4.0.pdf </teaching/slides/stosys/2023-2024-stosys-handbook-4.0.pdf>. Drop me an email if you want access to more project related resources.

License

This course content are distributed under the Creative Commons Attribution 4.0 International (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/.

Feel free to modify and use the slides in your course as you see fit with attribution.

Acknowledgement

The project work is generously supported by Western Digital with their donation of ZNS devices and software support.

Course Code

XM_0092

Course Coordinators

Past Contributors