BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations WASI: a New Kind of System Interface

WASI: a New Kind of System Interface

Bookmarks
37:11

Summary

Lin Clark walks through what WASI means and shares examples of opportunities that could be unlocked.

Bio

Lin Clark is a senior principal engineer at Fastly, focusing on WebAssembly. She is a co-founder of the Bytecode Alliance, which is building a vision of a future WebAssembly ecosystem that extends beyond the browser. She has also worked on web standardization and devtools at Mozilla, helped people understand the JS ecosystem at npm, and was a Drupal core module maintainer.

About the conference

QCon Plus is a virtual conference for senior software engineers and architects that covers the trends, best practices, and solutions leveraged by the world's most innovative software organizations.

Transcript

Clark: I'm Lin Clark. I make Code Cartoons. I also work at Fastly, which is doing a lot of really cool things with WebAssembly, to make better edge compute possible. I'm also a co-founder of the Bytecode Alliance, which is building a vision of a future WebAssembly ecosystem that extends beyond the browser. If you haven't been keeping up with WebAssembly, you might be thinking, why would you want to run WebAssembly outside of the browser? First, I'm going to explain how we got here. I'm going to start at the beginning for that.

Why Invent WebAssembly?

The first question is, why was WebAssembly created in the first place? The browsers wanted developers to be able to compile code bases written in languages like C++ and Rust to a single file, and then have that file run at near native speeds in the browser. They wanted it to run in a very secure way, in a well isolated sandbox, because you really need that when you're running untrusted code that you've downloaded from somewhere on the internet. To get that near native speed, the bytecode for these WebAssembly binaries need to be as close as possible to the native instruction set architectures or ISAs, like x86 or ARM, but without specializing to any particular ISA. This meant creating a really low level abstraction over various ISAs. This made it easy to run the same binary across a bunch of different machines with different machine architectures. This got developers really excited, even those developers who were working completely outside of the browser.

As these developers started bringing WebAssembly to the server and other places, they left some of the key properties of WebAssembly behind. They were giving these WebAssembly binaries full access to the operating system's system call library. That compromised security and also compromised portability, since now the binary was tied to a particular operating system. Given this, we realized that we didn't just need an abstract ISA, we also needed an abstract operating system. One that made it possible to run the same binary across a bunch of different operating systems, while preserving the effectiveness of the WebAssembly sandbox.

The WebAssembly System Interface (WASI)

We started work on WASI, the WebAssembly System Interface. The goal of WASI is to create a very modular set of system interfaces. These include all of the low level kinds of interfaces that you'd expect from a system interface layer. It also includes some of the higher level ones too, like neural networks in crypto, and we expect many more of these higher level APIs to be added. These interfaces need to follow capability based security principles to ensure that we maintain the integrity of the sandbox. For the most part, these interfaces also need to be portable across the major operating systems. Although we are ok with system specific interfaces for some narrowly scoped use cases. It was when we started trying to make this portability work that we started getting into some problems. These problems started coming to light when we were thinking about a pretty core concept in many operating systems, the filesystem. A lot of code today depends on the filesystem. That code uses the filesystem for lots of different tasks. It's where you persist data. It's where you share data between two different programs running in different processes. It's where you put the code for executables. It's where configuration lives. It's where assets get stored.

Files are like the Swiss Army knives that are used for all of these different tasks. As we were thinking about it, all the places where we want WASI to run, we started thinking whether this was really the right abstraction to use. The file achieved a central position in system interfaces during a very different time in software development. There were a few operating systems that really entrenched the file in this privileged position. These operating systems were first being developed in the 1970s and '80s. This was when you had the rise of mini computers, and after that, the personal computer, mostly to help with office work, which of course was organized in paper files. For those kinds of systems, having a filesystem and having direct access to that filesystem made a whole lot of sense.

If you look at the systems that we're building today, the ones that we're building applications for, things look a bit different. We're building applications for the personal computer still, that's true. With things like browsers, we started running applications inside of other applications, places where you probably don't want that inner application to have direct access to the filesystem. Then, as we started moving applications to the cloud and edge networks, and as IoT devices started proliferating, we suddenly had an entirely different landscape where direct access to a real filesystem was the exception, not the norm. On top of all of that, as we've moved towards having modular ecosystems of open source code that you just plug together, like npm or PyPI, these filesystems are presenting maintainability and security problems. Because the way that these filesystems are used, it's basically like having one big pile of global shared mutable state.

What Is a File?

Given all of this, files don't really feel like the right universal abstraction anymore. If we're going to try and break out of this file-centric paradigm, we need to think about what the file actually is and what it does. What exactly is a file? A file consists of two things, some bytes that encode content. You can think of this as an array or a stream. This is the data. Then there are other bytes that contain metadata about that data. This includes things like the name of the file, timestamps, permissions, and what underlying device the file is stored on. It's the second part here, where we start to have problems. When you're working with this metadata, that's when you need to know about the conventions of the host system that you're running on. When you think about what most programs are actually doing, what they actually care about, most of them only care about the data in those files. They just want to get that array or stream of bytes and start working on it. They don't care about where this data lives. Of course, there are some applications that do need to know the details about the metadata as well. For example, if you're building backup software, then you want to know the file name and which directory each file is in. Most of the time, that metadata is unnecessary for what the program is trying to do.

Compute vs. Metacompute

My colleague and the architect of a lot of WASI, Dan Gohman, has called this distinction, the difference between compute and metacompute. He had the thought that what if we were to push as much of this metacompute to the edges of the system as possible, either up to an orchestrating module, or, even better, out to the host itself? To see exactly what this means, let's walk through an example. Let's say that you're writing a utility that shrinks an image down to a particular size, and you want to run this utility from the command line. How would this work in the filesystem centric paradigm? We have the host system that's around the outside here, the gray box. Then the Wasm module is running inside of the host as a guest in the white box. The Wasm module would be parsed in an array of arguments, which are all strings, and it would take the string that's at a particular index, and use that as a file name. Then, that Wasm module would call the open syscall with that string. The operating system would give the Wasm module a handle to the file. Then the Wasm module would read the bytes from the file. With this, we're requiring the module to think about the filesystem. We're requiring it to think about the context that it's running in. This module wouldn't really need to know about any of these details. All it really needs is a stream of bytes to come in so that it can operate on that stream.

Let's try moving this metacompute out of the module and over to the host. By convention, a program's main function takes a very generic set of parameters. For example, in C, it takes the ARG Count, and a pointer to the array of strings that are the ARGs. Let's say that we introduced a convention in tooling support for more application specific parameters. For example, let's say that the main function for this application accepts a stream and returns a result that contains either a stream or an error. When you run this on the command line, the host will be able to look at that string, and see that the type that was being asked for is actually a stream. The host would know that it can convert a file to a stream. Instead of just parsing in a string, the host would instead open the file itself and get a handle, which the host can then use to stream bytes into the Wasm module. With this, we've moved all of the metacompute over to the host. This module no longer has any concept baked into it, of whether or not there's a filesystem, and this makes it more portable. This architecture also makes things more secure, because this way, we don't need to give the program access to that open syscall. That way, even if the code in this utility gets exploited, or is subject to a supply chain attack, it doesn't have access to the open syscall, so it can't be opening files willy-nilly when you don't expect it to.

Will Developers Use It?

Of course, none of this matters if developers don't use it. We need to have a gradual adoption path. We need a way for everyone in the community to transition to this new paradigm at their own pace, so that the whole community doesn't have to move in lockstep. We're planning three different options for how to compile a module to use WASI in this way. These three options represent that gradual adoption path. Let's say that you already have some legacy code that you want to compile, and this code makes extensive use of some of the not-so-good parts of traditional filesystem APIs. The parts that bake in expectations about the host environment. In that case, you would signal to the compiler that you want to use the legacy filesystem interface. This might be through a flag or through a target triple. This would link your code against the version of libc or whatever your language's standard library is that's implemented in terms of the WASI filesystem interface. This is in many ways the same API as the filesystem API that's exposed by POSIX. Your code can act like it has direct access to a filesystem, which it might have in some cases, or the host might provide a virtualized filesystem. Either way, this looks pretty much like the run of the mill filesystem APIs that most operating systems expose to your code. This code would not work on hosts that didn't either provide direct access to the filesystem or provide a virtualized filesystem. It won't provide full portability, but it would be an easy on-ramp to moving code to using WebAssembly.

What if you do want that portability, and you want the isolation between different modules that WebAssembly can give you, where you aren't sharing the filesystem between the different modules? For that case, we're providing a compatibility layer, that the developer would still write their code using their language's normal file APIs. In this case, what we're currently thinking is that the host wouldn't actually be the one providing the filesystem. Instead, the module itself would be virtualizing its own filesystem. These "files" would be in the linear memory of the Wasm module. This means that we don't have that global shared mutable state problem that the filesystem introduces. Even though these look like files in the source code, under the hood, they would use WASI I/O types, things like streams and arrays that would give them that full portability.

However, this virtualization would introduce some inefficiencies, including larger file sizes for the Wasm module. In the case where you want full portability and efficiency all at the same time, you would have a different API in your source code, the WASI I/O API. That means that you would change the code so that instead of parsing files around, you'd be parsing around those I/O types, like streams and arrays of bytes around. With this, the developer no longer even thinks in terms of files. It's all just these pure I/O types. The developer doesn't think that I have a file with this name in this directory, I'll open the file to get a stream of bytes from it. They just think, I have a stream of bytes. This means that the code really can run anywhere, it doesn't matter what host system. All systems can represent these basic primitive types. We've completely gotten rid of the potential for global shared mutable state, while also eliminating the overhead of the per module virtualized filesystems. This path also potentially opens up opportunities for further optimizations, because the engine now has more detailed type information.

In talking through these three options, there's something I want to be clear about, you don't need to make the same choice for all of the different modules that you have in your application. Part of this gradual adoption path is having the ability to convert certain modules before others. With both the second and the third option that I just talked about, you're using WASI I/O types, either explicitly or implicitly. In both cases, you're not expecting to share the filesystem between these two modules. This means that you can just use these two together, and they can simply pass values back and forth between each other. It's not quite as trivial to plug these modules up to ones that use WASI filesystem, but it's still pretty easy. If you want a module that is using WASI filesystem to call something from a module that uses WASI I/O, then you just need to have some code in between to extract a stream or array of bytes from the file's content, and pass that in to the WASI I/O module. There are some kinds of modules that will always require the full WASI filesystem that can't use only the portable parts. We expect this to represent a very small fraction of the modules that developers are creating, and we're hoping to see the rest of the ecosystem gradually migrate to only using WASI I/O. This is the thinking that we're applying as we're building out this ecosystem.

Example Opportunity for Cloud Native

How can we move these details out to the edges, so that orchestrating code or the host can take charge of them and potentially optimize them? It's one of these potential host optimizations. This is one opportunity that we see that is specific to the cloud native space. We're sure that there are lots of other ways that this paradigm can help for different kinds of use cases in different kinds of communities. We're excited to explore all of those more. This opportunity has to do with requests between containers, and how to make those faster. Let's walk through what happens when you make a request. I want to be clear here, this is just based on conversations that I've had. I haven't actually set anything like this up myself and walked through it, stepping through it in a debugger, or anything like that. There's a chance that I've gotten some of the details wrong here. I think that this is at least directionally correct. Don't worry if you aren't familiar with the container world, you should still be able to follow exactly how we're making things more efficient here. I'll just give you a quick rundown of the terms that I'll be using, so that you can understand a little bit better.

While containers often are on different machines, you can also have multiple containers on the same machine in something called a pod. Sometimes a container in one of these pods needs to have some additional functionality bolted on to it. For that, you use another container, which is called a sidecar container. Let's say that you have your pod, and in that pod, you have a main container, and a sidecar container that does some work before any request gets sent out to the network. A good example of this that's used commonly is something called a service mesh. Now you're sending a request to another service across the network in another pod. What does that look like? The data that you're sending over gets serialized using the format, something like protobufs. Then this serialization is saved into the memory in user space. Then the system makes a system call, and the memory is copied over to kernel space memory. That's already two copies of this data.

Let's say that you are using this sidecar. The sidecar is another container in that pod. The data gets sent over to the sidecar container as an incoming packet. Then the data gets copied over again into kernel space memory by the network drivers. Then it's copied into user space in the sidecar proxy. Then the system deserializes the data into objects that it can use. Only then does the service mesh actually run on this data. We haven't even gotten the data out of the pod yet, we have to go through steps one through four again to get the data out to the network. Then the other side, there's a very good chance that this whole process has to happen again.

Two-thirds of the steps here were actually to make requests that's on the same machine to pipe data through the sidecar. You'll see that documentation about the sidecar pattern calls this out as a tradeoff. These docs suggest that you ask yourself whether the isolation is really worth it, whether it's worth that additional overhead for your use case. This overhead isn't inherent to the problem, we can actually eliminate this as a tradeoff. Since we can do fine-grained sandboxing with WebAssembly, we can actually make this relationship between the container and the sidecar much more efficient, even running them in the same process. We still get all of the isolation between the two. In fact even more if we're not sharing the filesystem. Because of this, we don't need the socket to be our interface between the isolated units of code, instead, our interface between these two is just typed function calls. To communicate between these two, we simply do a synchronous function call on a single threaded stack. We use direct copies for registers, and potentially direct memory copies if needed. There are no intermediate serialization and deserialization steps here, and no heavyweight calls to the kernel or inter-process communication. This puts us into the nanosecond range for calls between the two. This would be much faster than the call we just looked at from container to sidecar.

However, sometimes you actually do need things to be on different machines that are across the network. It would be inconvenient to have different APIs for representing that, and to have to change which API you're using based on whether or not the other container is on the same machine or not. We actually don't have to. In this paradigm, we've moved all of that decision making related to where the code is running, out to the edges. The module you'd write imports the callee, specifying a function signature that's appropriate for a cross-network call. For example, allowing for various network failure modes and supporting non-blocking calls. In the case where the callee module is on a different machine, the host could take care of serializing the data stream and streaming it over the socket. If a service mesh is being used, the host could instead supply just a proxy module that's on the same machine that's using the much cheaper calling convention that I described just now.

The important thing is that the host that handles is the host that handles this distinction, not your code. In this way, you can get the optimal performance when you're talking to a container on the same machine, while not sacrificing the ability to communicate with a container that's over the network. We don't have all of these pieces in place right now, but once these foundational primitives are in place, we think that somebody could build this efficiency into the existing cloud native ecosystem. We're excited to explore this further. We'll be writing about all of this more over the coming months as we push to move these standards forward. We'd be interested in hearing from people coming from all different software communities, about what they see this architecture opening up for their communities and use cases.

Questions and Answers

Eberhardt: Within your talk, you concentrated quite a bit on the next iteration of the filesystem APIs. I really like the historical context revisiting the idea of, do we really need to make the filesystem central? That was really interesting. It'd be great if you could give a broad overview of, where do you think WASI is at, at the moment? Because when it first came out, I think it had filesystem console, and maybe timer, but what does it look like from 30,000 feet at the moment?

Clark: I think one thing is, people didn't realize when we announced WASI, that we were announcing this beginning of the standardization effort, and not necessarily that there was something that people should be using in production already. In the early days, the filesystem was there. We had timer, random, a lot of the stuff that you would have in basically POSIX was there. There were only a handful of things like sockets that we did not include in the first iteration. This is still the first iteration of WASI. We're still in the early days. We're still figuring out what that basic platform should be. I think that we are just now getting to the point where we do have that picture pretty clear, and this push that we're going to be doing around WASI I/O and around WASI filesystem is really to bring this first iteration of WASI to the production ready stage.

Eberhardt: That explains why your presentation was focusing on quite a significant rethink of filesystem APIs, and questioning the need for the filesystem API, and what it means to WASI. From a versioning perspective, do you consider WASI be a 0. product at the moment?

Clark: Very much so. That's actually pretty explicit in the standardization process. Nothing has actually reached phase three yet. Phase three is when it's ready for widespread implementation, for people to start finding the flaws in it. We're pushing WASI I/O and WASI filesystem to phase three soon. WASI I/O will probably be first, because we are now starting to feel like it's actually ready for people to really start playing with it, really start seeing whether or not it meets their use cases. Then after that, we'll go to phase four, which is where we're really putting finishing touches on it. Then after that it's phase five, which is where the W3C basically rubberstamps it. We really haven't reached WASI 1.0 until a lot of these things have reached phase four.

Eberhardt: You mentioned that WASI is at a stage where it's ready for people to start using it. I think looking at it from the flip side, people are ready for there to be a WASI. WebAssembly is taking off in a big way outside of the browser, and we need the standardization so that we don't keep reinventing the wheel. On that particular note, what are you most excited about? If you can't choose one, by all means, choose multiple things. What excites you most about WebAssembly at the moment?

Clark: There's some really excellent foundational work going on right now in the community group around the component model. WASI is part of this, so are other proposals like interface types, and module linking. If you look at the WebAssembly ecosystem today, you have ways of taking modules and putting them together into larger applications. In order to do that, you have to do a lot of gluing yourself. You have to do a lot of binding and all sorts of other things. With the component model, a lot of that additional work goes away. Basically, you can take these Lego blocks that you didn't write yourself and assemble them together really easily, in the same way that you do in JavaScript with the npm ecosystem, or Rust with crates ecosystem. I think that that's going to unlock so much potential. One of the neat things about WebAssembly that goes above and beyond those existing ecosystems is the fact that these components, they can be written in different languages, but still interoperate really easily and efficiently. They are isolated from each other, so you have the sandboxing around them, which does protect you from a lot of supply chain attacks. This is going to be a completely new ecosystem that solves a lot of the problems that other ecosystems have faced historically.

Eberhardt: We did have a question which relates to the current state of WASI, and about making network requests from Wasm. A lot of the early adopters of WebAssembly outside the browser, things like blockchain or smart contract engines, where you work yourself at Fastly, edge networks, almost all of them are relying on some network I/O, rather than filesystem I/O. What's the current state of I/O beyond filesystem access within WASI?

Clark: There was a team pushing a WASI-sockets proposal. That's still open. We have been thinking that sockets might actually be a little bit too low level for WASI. Somebody will probably push that across, in the same way that WASI filesystem is being pushed across so that we can support these legacy applications. That's another case where thinking about higher level, like what can we do higher level that moves a lot of this metacompute out to the host, so that the application itself isn't having to think about the socket layer. That last bit where I was talking about containers talking to each other, for that use case, we would have a higher level API that allows for that network connectivity.

Eberhardt: It's a bit like filesystem. Once again, you're trying to work out exactly what level you need to pitch the WASI interface at. Getting that right is the difference between success and failure almost, for WASI.

Clark: Exactly. We have some really good partners who we're working with on that. The Envoy team from Google is currently driving work through the WASI process. We're also working with the Krustlet team from Microsoft. There are some other folks working in this space that are also collaborating with us on that.

Eberhardt: The expectation is that it's a common problem. You have people partnering and collaborating to try to solve it, but at the moment, it sounds like you're still trying to work out, again, the level to set it at.

Clark: Exactly, yes. We expect that to progress pretty quickly. After this push to get WASI I/O to phase three, we expect that to be basically the next chunk of work that we're pushing to phase three.

Eberhardt: Yes, because everyone is going to be using their own custom implementation of some networking layer on top of WebAssembly. The other day, I saw Wagi, which is a CGI style interface, which I quite like. I love simple things. I really enjoyed the simplicity of that solution.

Clark: That's actually the Krustlet team at Microsoft that put that out. They're one of the teams that's collaborating on figuring out the ideal path.

Eberhardt: Getting back to nano processes. Because you've alluded to it a few times, some of the thing you mentioned about what excites you about WebAssembly is the way that it makes it easier to plug things together. It makes it easier to plug things together written in different languages, with interface types coming along, that will become easier. You've got the inherent security model, you've mentioned things like supply chain attacks. I'm predominantly a JavaScript developer, but I can see this wonderful future where I can pull down modules written in C++ and Rust and not really care what language they're written in. I'll have the confidence that I'm not giving them unfettered access to my filesystem and network. It sounds wonderful. The thing that I struggle with is, how do we actually get there? Take for example, someone like myself, who spends most of their time messing around with JavaScript, and Node, and npm, what are the stepping stones? How do you think that WebAssembly could initially be integrated into that tool chain?

Clark: I think that mostly the tool chain can stay the same. One of the things that we really want to see is deep integration with those tool chains. We've actually started prototyping some deep integrations with different tool chains. Nothing that is ready to show off yet. The idea is that developers would be able to just work in their own language and be pulling in these modules from the WebAssembly ecosystem as needed. For the JavaScript ecosystem, if you were at the Wasm Summit, one of my colleagues actually showed a little bit of this prototype we've been working on for how we can actually package up JavaScript modules with a very quick startup time inside of WebAssembly modules.

Eberhardt: I'm assuming they package as a JavaScript virtual machine within WebAssembly to achieve that.

Clark: Exactly. We use a snapshotting tool called Wizer to get very fast startup. It's actually, if I remember correctly, faster than a JavaScript engine by itself starting up. You already get to, the application is completely initialized.

Eberhardt: I say yes, because you know what code you're going to be running within that virtual machine already, ahead of time.

Clark: Exactly. You can basically bake in all of the bytecode that's already been parsed and everything, into the linear memory, and then set up the instance with that linear memory ready to go when it's running.

Eberhardt: Yes, because it was the migration challenge of the existing JavaScript modules, which is the thing that I got hung up on. It's good to know you're exploring a migration path there, because I think that's necessary. You need an easy path for people to follow to get there. You need to be able to provide a path where some of the popular tooling, whether it's Babel, or Webpack, or whatever else can relatively easily move across.

Java had a similar migration from applets in the browser to servlets in the data center. I'm not entirely sure that's correct. Java existed outside of the browser before it tried to penetrate the browser ecosystem. What have you learned from that experience in adapting WebAssembly to the pod sidecar model? Personally, I think Java took a very different route. I don't know whether you've got any particular comments on that one.

Clark: Java has a lot of the same goals. It's like this is another iteration of the same goals, but yes, taking a lot of lessons from Java and from other things that were happening around the same time. One of the things is that like interface types is pretty close to the component models that you would have seen around that time. I think with Java, there's DICOM, all of this stuff. We are taking a lot of lessons, for example, from the component model, the idea is that you don't bake in the distribution, the idea that it's distributed into the component model, that's a layer above. We're taking a lot of these lessons and applying them to this next iteration on the same goals.

Eberhardt: I think when WebAssembly first came out, there were obvious immediate comparisons to Java and the applet model. I think there were quite a lot of differences. To me some of the things that have been really important to the success of WebAssembly has been the simplicity of it in the first instance. The complete lack of I/O initially sounds like it's an inhibitor. I think, actually, that was quite an important decision to make. Also, making it immediately multi-language from the outset. I know you can compile multiple languages to the Java Virtual Machine, but it's still, cut it in half, and it has Java all the way through. I think there were many early design decisions that each one of them probably felt small, but in composite when you look at them all together, I do think it sets WebAssembly far apart from Java and the JVM.

 

See more presentations with transcripts

 

Recorded at:

Feb 04, 2022

BT