DBOS A Database oriented Operating System

DBOS: A Database-oriented Operating System
Michael Stonebraker

Abstract: For the last three years, a team of us at MIT, Stanford, CMU, Google and VMware have built a new operating system stack, based on a high-performance distributed DBMS. In other words, all OS state (files, messages, scheduling information, etc.) is stored in the DBMS and all OS services are written in SQL plus stored procedures. At the present time, we have a secured venture capital funding and are about to release a commercial open source version. In this talk, I report on aspects of our system, including: Performance: DBOS is competitive with the state of the art concerning file system performance, message performance and scheduling performance. All of these are implemented in SQL. Provenance: Because all OS state is in DBMS tables, DBOS change capture moves all state to a warehouse DBMS (currently Vertica or Redshift) with a runtime overhead of about 5%. In this case, security queries (e.g. looking for outliers) supporting the “right to be forgotten” in GDPR-style systems and other provenance operations can be coded in SQL. Experiments with a current security product show that DBOS can both capture provenance data and query it with higher performance. Serverless environment: We have written a Java serverless environment on top of DBOS. It is an order of magnitude faster than current systems (AWS Lambda, Open Whisk) because it co-locates computation and data whenever possible. Also, provenance facilitates a novel time-travel debugger. Early enterprise usage: I will report on early DBOS usage in enterprise environments at three large enterprises. Commercialization changes to DBOS: These include moving to TypeScript and open source DBMSs. Bio: Dr. Stonebraker has been a pioneer of data base research and technology for more than forty years. He was the main architect of the INGRES relational DBMS, and the object-relational DBMS, POSTGRES. These prototypes were developed at the University of California at Berkeley where Stonebraker was a Professor of Computer Science for twenty-five years. More recently at M.I.T. he was the co-architect of the C-Store column- oriented DBMS, the H-Store transaction processing engine, the Data Tamer data integration system, the SciDB array processing engine, the Kyrix visualization system and the operating system DBOS. He is the founder of ten venture-capital backed startups which have commercialized his prototypes. Professor Stonebraker is the author of scores of research papers on data base technology, operating systems and the architecture of system software services. He was awarded the ACM System Software Award in 1992, for his work on INGRES. Additionally, he was awarded the first annual Innovation award by the ACM SIGMOD special interest group in 1994 and was elected to the National Academy of Engineering in 1997. He was awarded the IEEE John Von Neumann award in 2005, and the ACM Turing Award in 2014. Presently he is an Adjunct Professor of Computer Science at M.I.T., where he is working on a variety of future-generation data-oriented projects. This joint meeting of the Boston Chapter of the IEEE Computer Society and GBC/ACM will be hybrid (in person and online), part of getting back to normal after the COVID-19 lockdown.