BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations Rust, WebAssembly, and Javascript Make Three: An FFI Story

Rust, WebAssembly, and Javascript Make Three: An FFI Story

Bookmarks
53:13

Summary

Ashley Williams talks about the Rust and WebAssembly toolchain and the technical work involved in creating a developer-friendly experience designed to grow adoption of WebAssembly- particularly amongst JS developers. She takes a deep dive into the foundational tool, "wasm-bindgen”, and how (and why) they‘ve taught Rust compiled WebAssembly to interact seamlessly with JavaScript APIs & toolchains.

Bio

Ashley Williams works as a Systems Engineer at Cloudflare. She is on the Rust core team and leads the Rust community team, where she focuses on the RustBridge educational initiative. Previously, she worked on the Rust Programming Language and WebAssembly for Mozilla, and before that, she wrote and maintained Rust and Node.js backend services at NPM, Inc.

About the conference

Software is changing the world. QCon empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams.

Transcript

Williams: My name is Ashley [Williams] and as Justin said, I'm here to talk about WebAssembly. As you can tell from my incredibly long talk title, I'm also here to talk about several other things including Rust and JavaScript.

One of the biggest problems about this topic is that I'm here to talk to you about Rust and WebAssembly which are both incredibly new edgy things that most people haven't used, and this is supposed to be a practical conversation about technology you can use today. I'm going to take these two things that were born pretty recently and try and explain to you that you can absolutely use them. That's a bit of a challenge, but we'll see.

Additionally, I'm really excited about this talk because I'm going to go into some depths of things. I was going to do that because I was told that I could do some workshops and they would be before the talk and so that would help everyone get up to speed, but the workshops are actually after the talk, and so it's just going to be a little in-between. I'm going to try and go a little fast because I feel like WebAssembly is one of those things where people want to ask questions a lot and so hopefully I can make some time.

I named this Rust WebAssembly and JavaScript Make Three, an FFI Story. The FFI I'm going to be talking about is slightly different than the one that you might expect. FFI stands for Foreign Function Interface and, classically, you've got to do the really scientific Wikipedia definition. A mechanism by which a program written in one programming language can call routines or make use of services written in another.

You may note that I actually had three languages written in my title. Some people may think this is a talk about how to teach Rust how to JavaScript. Believe it or not, this is something that exists but it's not what this talk is about. If you want your Rust to talk directly to JavaScript, check out Neon. Arguably, I would say it's a little bit slow; you're going to want to use the WebAssembly version but Neon is also very cool. This talk is going to be about how we teach WebAssembly to talk to JavaScript using Rust function signatures. If that sounds super intense, I'm hoping by the end of this that you will actually realize that not only is it really not all that technical, potentially a little bit tedious, but actually exciting and something that you can actually look into and understand on your own. Let's see how well I do with that.

My name is Ashley Williams, this is a picture of me dressed as a crab. If you're unfamiliar with Rust, Rust has an unofficial mascot called Ferris, who is a crab because Rust people are called Rustaceans. The puns get worse, so I'll stop there but I am on the Rust core team, I'm also on the Rust wasm core team. I wrote this tool called wasm-pack. If you've written Rust generated WebAssembly, you've almost certainly used that. I used to work at NPM and node, so balancing this Rust JavaScript game is my jam. I currently work at a company called CloudFlare on building out more WebAssembly tools. If you want to learn how to run a WebAssembly on the Edge because there’s more Edge technologies there, come talk to me because that's what I'm working on.

What is Rust?

Today we are going to talk about how we can connect all of these three things and make them live very happily together on the web being very small and very fast. Before I do this, I think it's probably important to explain what some of these technologies are. What is Rust? On its website, Rust declares itself as a programming language designed to empower everyone to build reliable and efficient software. Fundamentally, you could understand it as a C++ replacement, but instead of being C++ where maybe you could shoot yourself in the foot and the tooling is terrible and the docs are terrible, we actually make the tooling really nice, the docs really nice, because we want really everyone to be able to program at a systems programming level. In order to do that you have to make your environment inclusive and a great place for people to learn.

That's what Rust is really trying to do and it's what really inspires me about Rust, but some people like the technical details, so here are a couple. We are statically typed but we also have type inference. You can say let, name, equal, quotes, Ashley and you can get variable. The biggest thing that people really pay attention to about Rust is we have a concept called ownership and lifetimes, which gives us this guaranteed memory safety without a garbage collector. If you want to be really pedantic about it, we actually have what you could call a static garbage collector, which is really a blood packed with the compiler that says “You'll write your code this way and I'll take care of it for you.” If you are a newbie, don't worry about this, people often say that this is scary. This is definitely the location of the largest learning curve in Rust but great responsibility, great power. We're constantly trying to work on those docs every day.

Some people like the hacker news version of pitching Rust and because I work at CloudFlare I'm allowed to say this, but one of the reasons you might consider Rust for your company is Cloudbleed was an incredibly huge security problem that happened for cloud alerts similar to the Heartbleed security vulnerability. Fundamentally, what Rust says is the safest program is one that doesn't compile. If you're doing some iffy stuff with your memory, unless you explicitly tell Rust “No, trust me,” Rust is going to go, “Let's not do this, I'm not compiling this program for you until you fix it.” That's the orange website pitch for Rust.

What is WebAssembly?

With Rust out of the way, let's talk a little bit about WebAssembly. I asked people how many people here had heard about WebAssembly, and there were not that many hands. I'm going to go quite quick but I'm excited to talk about this in the hallway if anyone wants to. WebAssembly, if you go to webassembly.org, will say it is a binary instruction format for a stack-based virtual machine. Which is a tagline similar to the old Rust tagline, which has the problem that if you look at that, you may not be able to parse any of the words in it. A more friendly definition would be, it is a compilation target for running bytecode on the web.

WebAssembly, of course, is on the web, and so people are always worried about reading source. It does have human-readable versions, you can see the top version is what you could call an s expression of WebAssembly. Who here writes Lisp? I finally won, see? You got it. Additionally you can take a look down there at the bottom, the bytecode stuff also looks like that.

Fundamentally, you're not going to be wanting to write WebAssembly from hand. You're going to be wanting to write it in a language that then is going to target WebAssembly as this compilation target. Don't worry, I'm sure recruiters are going to be looking for people who have handwritten WebAssembly for the last 10 years tomorrow, but that's not a thing, so don't worry about that.

Another thing, just before I move on from this, I'm really excited about WebAssembly being on the web, but if we think about what's happened in the last six months, if anyone's followed the hype cycle with WebAssembly, the most fun things that are happening with WebAssembly right now are not happening on the web. A great tagline for WebAssembly is, “WebAssembly: neither web, nor assembly”. Just remember that because again, naming things is very difficult.

Why?

You might be asking yourself, “Cool, maybe I have this idea of what WebAssembly is, but why? Why would you make this?” You could, but should you? There's a couple of reasons. This is one of the original white papers that announced WebAssembly. You can see that it's from a conglomeration of different browsers. You have folks from Google, folks from Mozilla, Microsoft, and Apple all coming together to talk about this. The primary motivation for WebAssembly is something like this. If you read this, this is an announcement from Google I/O last year, where AutoCAD announced that you can now AutoCAD in your browser, that's a big deal. Perhaps this is very specific to me, my father is an architect, I remembered going to his job and having to run AutoCAD on a SPARC station. The idea that you could that software that needed such a huge computer not a decade ago and now you can put it in your browser is an incredible thing.

When I talk about WebAssembly, I'd like to try and motivate it when I define it. I say it's a technology that invites new types of applications, written in multiple different languages, to be discovered and distributed on the web. Perhaps you can call me a millennial, but I personally think that it has been relatively proven that the web is by far the best discovery and distribution technology that has existed to date. If you want someone to use your thing, read your information, do whatever, it really needs to be on the web. Right now, because of the web's capabilities, there's some things that are just literally excluded from it because the browser and the web are literally not capable of being able to include those very performant intensive applications. Now, WebAssembly is like, “No, come on in. The web is also for you”.

You might actually realize that this is not a new idea at all. Does anyone recognize these? Has everyone updated their Adobe Flash? This is not a new idea, in fact, it's an incredibly old idea. Adobe Flash is that big red F there. This did not really have a logo and so it's an interesting one, but Google used to have something called NaCl, and you could add a portable Native Client called pnacl, and this was an attempt to do that as well. Then, of course, we had Java applets and all of those types of things. Why WebAssembly? Are we just repeating the same things that we did before? Is this a new Flash?

I would just remind you, if we take a look at this, this is all the browsers coming together. WebAssembly has been formed through the standards process and continues to be. We'll talk a little bit more about that later, but this is, what if Adobe Flash was something that everyone agreed on, was an open standard and was open source and was inherently built into your browser, so as long as you keep your browser up-to-date, you should never get a little pop-up that asks you to update your WebAssembly? Because that would be incredibly embarrassing.

Speed and Size

WebAssembly is exciting because it's inviting these new applications, but why is this important? Why is WebAssembly able to invite these new applications into the web? It's largely due to this idea of speed and size. WebAssembly can go fast and it can be smaller than JavaScript. That's the main idea. Now, I want to caution, with all new technologies, there's this question, “How fast is it?” We're in a situation today where there are certain applications because browser engines are really awesome. V8 optimizes the heck out of things, where JavaScript will actually be a little bit faster. The browsers now have to write WebAssembly optimization engines, and they haven't done that yet, or they're working on it.

In general, WebAssembly is going to be relatively faster because it's less code, it does not need to be interpreted because it's compiled, and it's going to be smaller, because remember I said, it is a bytecode, so it has a lot less instructions. Wouldn't want to be writing it from hand because it's very unpleasant. One way to be faster is just to be less code, and so WebAssembly is less code.

Instead of me telling you all of that in an abstract sense, let's talk about a specific edge case. Mozilla is one of the front runners in developing WebAssembly and getting it out there in production. This is a blog post written January 2018. This is OG WebAssembly days, by Nick Fitzgerald who I work on with the Rust WebAssembly working group with, and he wrote an article called "Oxidizing Source Maps with Rust and WebAssembly." This is a super awesome blog post, and I always show this graph. These are a little hard to see, but the orange dots are going to be measurements of JavaScript and the blue ones are measurements of WebAssembly, these are all in milliseconds. Then across the bottom, the first two are Chrome, the second two are Firefox, the second two are Safari. Beyond going, “Hey, JavaScript and Firefox, what's going on there?”, what you might immediately recognize is, “Oh, cool, the blue ones take less time than the orange ones”. The WebAssembly is faster, but that is not what is interesting to me in this graph.

Is anyone here on call, on Peter duty? I used to babysit a whole bunch of node processes when I worked at NPM for the NPM registry. This graph makes me get really excited about WebAssembly, because check out that standard deviation, oh my gosh, the performance of the WebAssembly is so much more predictable. Can anyone guess why? Garbage collection. WebAssembly does not currently have a garbage collection. Memory management in JavaScript is going to need garbage collection and it's going to be a lot harder to predict. Because WebAssembly does not have that, it's going to be significantly faster and significantly easier to predict its behavior. I think that over time, we'll see that WebAssembly is more consistently faster than JavaScript but the killer feature really is its predictability, and I think that that is really great.

Immediately after posting this blog post, because it's the internet, another engineer who I really respect but I'm very bad at pronouncing his name, so I'm just not going to. His Twitter handle is “mrlf” that's easier. He immediately wrote a blog post using the classic meme phrase “maybe you don't need,” and so maybe you don't need Rust and WASM just beat up your JS. If you are a JavaScript developer and you want to learn about V8 optimizations, read this blog post because it is amazing. However, all of the things that he tells you to do to make your JavaScript faster, makes your JavaScript inscrutable. It is impossible to read, it is very difficult to maintain, but you can learn a whole ton of cool stuff about how things get optimized.

Nick followed this up by saying, “Hey, speed without wizardry”. When you can write an application in WebAssembly, you can get this kind of speed in performance, but you can do it without having to use all of these fantastic optimization hacks. You can just write your code the way you would write your code, and by using WebAssembly as this compilation target, you can get that great stuff that you get from the hacks sort of for free. What you might be realizing here is that there's a sort of sense that JavaScript and WebAssembly are at odds.

How many people here have heard that JavaScript is going to die because of WebAssembly? I'm going to say it eight times in this talk but I just want to say it now, that's not true. It's not what WebAssembly wants to do, WebAssembly is not a replacement for JavaScript. In fact, it is going to be its complement, and the entire rest of this talk is going to be about an intense amount of effort from a relatively small number of people, to work desperately hard to make sure that WebAssembly can play super nicely with JavaScript, because we believe fundamentally that is how WebAssembly is going to be successful and we'll talk more about that.

We believe that WebAssembly needs to inter-operate with JavaScript. How many people here, regardless of what I just said, think that WebAssembly should kill JavaScript? This talk is for you, I'm so glad you're here. Before I talk about some more motivations, let's talk about technically what it means to make WebAssembly play nicely with JavaScript. The first thing is a little bit basic, which is WebAssembly needs to be able to store and work with JavaScript objects. Maybe that makes sense, we'll talk about the challenges there. Then additionally, JavaScript is very much an ecosystem-based language. We need to be able to interact with the JavaScript ecosystem via modules and workflows, particularly things like NPM and bundlers.

When I talked about a small group of people, this is the Rust WebAssembly working group. We meet weekly and we post all our videos on YouTube, so if you want to hear these conversations, please go check them out. We'd love for more people to get involved. Let's talk about the work that came to this. The work I'm going to be talking about is primarily work that happened all last year. It's like a big year-long sprint to get this over-the-head.

Demo

What I'd like to show you, is I'm going to demo writing some Rust. I hear the Internet is not going great and so I am very glad that I pre-baked something. One thing I will it's my little crib notes, we've set up a whole bunch of tooling because we want people to get involved in this. One thing that you should note is that with these three things, we can just get a hello world in Rust wasm working. Assuming you have Cargo and NPM, you can also do this, and I'll be posting the tutorials along with my slides.

Let's hop into demo two. If we take a look inside demo two, if you've ever done any Rust work, what you'll see is primarily a Rust crate. Package.json and Rust is cargo tamil. Picking a markup language is hard, we can fight about it in the hallway. When you write Rust you're going to write your code in the source directory, and so we're just going to open that up, go into our source directory in here, I just have a lib, some utils that we baked for you. Let's hop into this. Before anyone comments on it, yes, I use vim with no syntax highlighting, it's fine.

This is our lib.rs, it's got 19 lines of code. The first nine shouldn't matter to you, we're doing some cool stuff to make sure that you're allocating a little bit less memory but those are all just nice to haves. The big thing that we have in here is our Rust code that is going to be using a very important web interface called alert. Who can guess what this would do if I ran it in a browser? It will poof or it will explode your computer? It's going to pop up an alert and it's going to say “hello demo two”. Which is not all that interesting, but if we take a look at this code, there are some interesting things happening here.

We're using this - as the kids call it, hashtag bracket - wasm-bindgen, which is called an attribute in Rust to mark the touch points of where we want our Rust to talk to our JavaScript. If we look at 16 through 19, we are creating a function called greet, and we want that function to be available in the browser. From JavaScript I want to be able to call the greet function and that greet function is going to call alert.

Guess what Rust doesn't have? An alert function, at all. The browser does, and so we need to tell our Rust which will then be compiled to WebAssembly, “You don't have this here, but where you're going, expect it. It will be there, I promise”. From lines 12 through 14, we've created an extern block and inside there, we can just list our function signatures for the things that we expect to have. Here I expected to have an alert function that should take a string. You might see that that type is ampersand str. If you've done Rust, strings are a little weird. I'm sorry but that fundamentally just means it's a string.

By doing so, we're able to compile this to WebAssembly and it's going to make that greet function work. It's also going to mean that when I call the greet function, it's going to go look in my browser for the alert function and call it with the string that I constructed in my WebAssembly. If you don't know how that's working, one, that makes a lot of sense, two that's what this talk is about, but I wanted to show it to you working before I explain how it works.

Here we have our Rust and let's compile this to WebAssembly. In here, so I have pre-baked this because I hear the internet is very bad. What you would do is you would run a tool called wasm-pack, which is the tool I wrote which says, “Hey, I want you to make me a WebAssembly but I don't want to think about it that hard” so wasm-pack does it almost all the thinking for you. You would run wasm-pack build and wasm-pack build is going to create this pkg directory here. Let's poke in there and see what's in there. There's a whole bunch of stuff, we got some JavaScript, we got some WebAssembly, we have a package.json and we get some time, we got a types file in there. That's pretty cool.

Just for people aside, if you love typescript web, wasm-pack will generate type files for you automatically, but it's also going to build your wasm and that's the wasm file right there demo2 underscore bg. You'll also notice that we created some JavaScript and that JavaScript is very heavy lifting and it's going to allow you to interact with that WebAssembly. We'll talk more about that in a second, but let's see this working in real life. Let's leave our pkg file or our directory and then we'll go into the dub-dub-dub one.

In here, we've got a whole bunch more, we've got a webpack config. If you don't like that, please know you can use WebAssembly without web pack. It's just nicer with web pack, people have opinions. We've got some other stuff in here, but fundamentally, we have an index.html, an index.js. I'll quickly show you the index.html. Nothing really fancy to see here, you see that we're loading something called bootstrap.js. Currently, WebAssembly needs to be loaded asynchronously and we thought, “Hey, we always know how we need to load it. Why don't we just create a template so we load it for you?” You can also load it yourself but all bootstrap.js is doing is loading that WebAssembly for you.

Then in index.js, we can see that we are importing as wasm from demo2. You might notice that we're importing from demo2 which is actually a JavaScript file, so we're using our web something via JavaScript and we're calling wasm.greet. That's a small tour of the code, let's pop this open. Should be able to run serve just here, pop it up. Alas, that was supposed to be a beautiful, amazing alert message that we all really wanted to see. For folks who are wondering what the issue was, I have another demo where I just serve it as a static site, but this one I needed to make sure that the webpack magic happened and so I should have run webpack serve instead of just serve.

Here we go, we've got some stuff working. Hopefully this was a deeply compelling and not at all naive example that will make you desperately want to know what is going on under the hood here, because it turns out it's a lot. That is the thing that we all worked so hard to make happen, and while that was of course a very naive example, there are a ton of things that people are now building in Rust and WebAssembly that are not naive and are actually quite big and are production level applications. It is super cool but again, this is where we started because the stuff that's happening under the hood is pretty big.

Design Principles

wasm-pack is the orchestration build tool but under the hood we're using a tool called wasm-bindgen. How many people here think that bindgen sounds like a city in Germany and not actually a technical tool at all? bindgen stands for bindings generator and what I'm going to do is I'm going to talk about how we generate both Rust code and JavaScript code while compiling to WebAssembly so that you can talk to your WebAssembly as if it is JavaScript. Then I'm going to tell you why.

One of our primary design principles is the idea that WebAssembly should be an implementation detail. WebAssembly is super cool but almost all the things that are super cool and technical probably are things that most people when they're using it should never even have to think about. Our first real belief was that WebAssembly should just act like a JavaScript module. We should just be able to interact with it like a module, you shouldn't really have to know whether or not there's JS or WebAssembly in there.

When you're originally thinking about compiling Rust or WebAssembly, you may imagine that the process looks a little bit like this. You've got some Rust, you use Cargo, you compile it to WebAssembly and then just throw it into the lands where you heard that WebAssembly is supported, which by the way, WebAssembly is supported in all the major browsers in addition to standalone run times and node.js. It turns out that this doesn't work super well. How many people have any type of JavaScript or web application static site on the web? Keep your hands up if it doesn't have JavaScript in it? Well now, people put their hands down already, it's got JavaScript I know it does. This is really what the web looks like.

There is a universe where people believe that you're going to take an application, compile it to WebAssembly, just throw it up on the web and you don't need to interact with JavaScript at all, but very few people actually have the privilege of being able to write greenfield project, and the vast majority people are dealing with legacy code and legacy code on the web is written in JavaScript. When you have a universe like this, this is actually quite complicated because the WebAssembly cannot talk to the JavaScript very well.

What we have created, which I'm going to explain today, is how we take your WebAssembly, we jam it into a JavaScript module, then we generate a whole bunch of JavaScript so that when you send it over to the browser, everybody around it, all of the other JavaScript is just like, “Hey, cool, this is a new kid and he hangs out just like we do.”

To reiterate, we really need to interoperate with JavaScript and this is why. This is module accounts. I used to work at NPM. I don't know what is going on recently, I took this picture today and something happened with the stats, I don't know what's going on. It doesn't matter what that last little huge jump is, NPM is clearly one of the biggest packaged ecosystems that exists, and regardless of whether or not you think it's full of garbage or not, I don't really actually care.

It turns out that a ton of people are using this. There are around 9 million JavaScript developers in the world. Asterisk, I did a search for how many JavaScript developers are in the world and then I just averaged the first five numbers I found. This drives what I believe, though, the web is a big thing, we've already talked about what a delightful discovery and distribution mechanism it is. It does not surprise me that there's a ton of developers who are behind making that work.

What I thought was most interesting is that at least on the couple that had the next languages in them, it seems like it was around 2.5 million more than whatever they decided was the next most popular programming language. There are a lot of JavaScript developers in here. How many people are hiring managers in here? How many people who are hiring managers have to hire JavaScript people? It's also because there's a lot more of them so they're a lot easier to hire. It's a lot easier to find a JavaScript developer then a Rust developer. Finding a WebAssembly developer, as I said, 10 years’ experience, right here - no, I'm just kidding.

There are a lot of JavaScript developers in the world, which means that if you really want WebAssembly to be successful, you need to make that a tool in their toolbox. Not a tool just for people who want to write systems programming languages and think that the lack of type safety in JavaScript is a huge security problem. I know that you're out there and I'm coming for you. JavaScript is important and WebAssembly will only be successful if we can make sure that it plays well with them.

As we move on from the idea that we really to interoperate with JavaScript, we can talk more about our implementation details. This is where I put a lot of code on the screen, and I try and do color highlights to make it easier, but I also half expected to have my slides here so that I don't have a pointer. You're just going to see wild hand gestures now. With this idea that Rust could compile to WebAssembly, we've wanted wasm support to be unobtrusive. You should be able to write a Rust library that someone could compile to WebAssembly but also could just use in a normal Rust program. It shouldn't be that implementing WebAssembly support would be so huge and obtrusive that it would make it very unpleasant to have a multi-purpose crate. Crate is what we call packages in Rust. I told you the puns were bad.

We believe it should be unobtrusive and, as you saw from my demo, what we've been able to bring it down to is the idea that you're simply going to be annotating any type of public function that you wish to export with this wasm-bindgen attribute. That is the vast majority of what you're going to have to do.

Here's the big crux, I've been talking about WebAssembly needs to play well with JavaScript. I haven't really told you why maybe it wouldn't. We need to work with and pass JavaScript objects into WebAssembly, but WebAssembly doesn't support that at all.

Currently, WebAssembly only supports integers and floats. It only does numbers, is how I like to say it. Maybe you're going to put your nerd glasses on and push them up a little and go, “Well, Ashley, any type of programming could be expressed in only numbers”. Of course, it's computer science, but I'm just like “Nope, that is not a game that I'm here for.” Particularly with WebAssembly being something for the web, I like to describe web development as fancy string concatenation. I know I'm not wrong, I've been a web developer for a while. If I had to take every single string I ever needed and convert it into a whole bunch of numbers to send it over a boundary and then bring it back--take those numbers, turn them into strings--that sounds like a whole bunch of work and I'm lazy.

Who else here is a lazy programmer? Good, this is a value, being lazy is good. About computers, not about people though, note that. What our desire was, was we need to enhance the ABI of wasm modules. I never thought I'd be saying JavaScript has a rich type system, but if you take a look at WebAssembly, “It's relative here, but JavaScript definitely has a significantly richer type system than WebAssembly, so we need to enhance that.” This comes down to the question of how do we shoehorn JavaScript objects into an u32 for wasm to use? I am going to do a short tour of how we do that today.

WebAssembly is also very interesting because it has something called linear memory, which is to say the way it thinks about memory is that it's just one big array. This is not really a simplification in any way, that's just really what it is. One of the things that I like to point out when I start doing more low-level programming is that it's not actually harder. I think it actually just gets simpler and just way more tedious, very tedious. The model here is just a big old giant array, and we've got to put stuff into it and take it out and the only thing we can put in is numbers.

By the way, I have a side operating system project and we mimic the VGA driver the same way. You can get close to the metal just playing around with arrays, it's fun.

What we created with wasm-bindgen was what we call a polyfill for JavaScript objects in WebAssembly. You may be looking at this slide and go, “Wait, Ashley, why did you cut and paste two little code snippets that are exactly the same?” They are off by one very weird but load-bearing punctuation, which is the ampersand. I'm going to quickly just run through because of this linear memory, the way we handle passing JavaScript objects in WebAssembly is going to be different depending on whether or not that object should live for a long time, which is the one over on the left, the one without the ampersand, or for a short time, which is the one with the ampersand.

We need to remember that we're dealing with one big array and so, what we end up doing is actually just making one big array in the JavaScript that we generate. I'm going to be showing you both generated Rust and generated JavaScript, and you'll see this consistently but, I am not lying to you, we're just talking about big old arrays.

Short-Lived JS Objects on the Stack

Let's talk about short-lived JavaScript objects. Here is a bunch of code, the Rust that we're going to write is where I have this foo.rs. We have our foo function, I don't like using foo for examples but this is actually supposed to be as generic as possible, so that actually works here. You have that annotation, and we're writing this because fundamentally, the typescript or JavaScript interface we want, is we want to be able to just export this function and it can take any sort of JavaScript value whatsoever.

In order to make this happen, you're going to write that Rust and then this is what wasm-bindgen is going to create. The first thing it's going to create is this generated Rust. What you'll note here is that your original function still exists and it's completely unmodified. It's a pub fn foo, still right there. However, right below is a very funky looking function, we can see we have a pub extern "C". This is we're defining this boundary and the function actually has a generated name, so if you go and use the toolchain that I just showed you, the generated files, you're going to see a checksum in the name. You're going to be like, “I hate this function signature, it's a gross number.” That's because it's generated and you shouldn't touch generated files.

Here, we're going to be exporting this as foo, that'll be the name still. What we can see in here is that we are actually going to be grabbing some data from our giant array. We're passing it in index and that index is going to be where all of our file for our string, our data for our string, is going to be or our data for our JS value, so you're like, “Ashley, what do you mean that data for your JS is value? What is going on there?” What we can see up in foo.js, which is JavaScript that we also generate, we're first constructing this new array which is going to be our heap. For short-lived JS objects on the stack, we're actually going to treat the first set of that array as a stack.

When we add a borrowed object, which is going to be this object that we're trying to call our function with, we are going to pop that right on there and it's going to operate the same way as a stack. You push them on, you push them off. This means that that value is not going to be available once that function gets called. It can't be long-lived; it's only going to live for the duration of that function.

In our JavaScript - and this is the JavaScript function you end up calling as an end user, export function foo - the first thing that we're going to do is we are going to create this index, and this index actually calls this ad borrowed object. We have this argh 0, and what that's doing is it's taking the object, and it's going to pop it onto the stack that's in our heap array. Then we're going to use that index, we're going to call foo with that index, it's going to pop that right off, it's going to do the thing it needs to do, and then we're going to immediately erase it from our array. That's what's going on in the final a section.

What you can understand is we just have a big old array, we have this JavaScript, we're going to add the borrowed object, and then we're going to pass that index of where we've stuck it in memory to our wasm function and then our wasm function is going to use that index to pull it right back out and be able to call it. This memory is being managed as an array and we're really passing around indices, and what you'll see when we're talking about the long one is that you're going to pass around indices and links. To check in on exactly what's going on here, we wanted export function foo, the foo function that we call comes from generated JS. That generated JS is going to be calling wasm.foo and that wasm.foo is actually a generated function from Rust which then ends up calling the Rust that you wrote.

Man, that's a whole bunch of stuff, that probably didn't make a whole bunch of sense. You have to read these things over and over a couple of times to get it.

Long-Lived JS Objects on the Stack

For long-lived JavaScript objects, I won't settle on this too long but what I do want to point out is that for these, instead of popping them in that little stack we've created at the front of our array, we're actually just going to be pushing them on to that array and they're going to be staying there for a lot longer.

To access them, we're going to be saving an index and a length so that when we call it from WebAssembly, WebAssembly is going to go look into its memory, figure out that indices to see that JavaScript passed it, grab all of the data, and then pull it back out. Everything is stored as just these numbers in this array and you're passing around indices and links to do so. This is a bunch of code and then additionally, because it lasts for a long time, we actually have to manually manage the memory by removing it from that. For long-lived JavaScript objects, this gets a little bit trickier and so as you're writing WebAssembly, as a practical takeaway, you want things that live short and you don't want them to persist as we go back and forth between Rust and JavaScript, because we call that the “trampoline” and like crossing any type of border, it's complicated and it takes a long time.

Exporting to JavaScript

Let's try and use something that maybe is a little bit more concrete, we don't want to talk super generally about how we're crossing this boundary. Let's talk about just exporting to JavaScript. If we export to JavaScript, let's say we have this greet function, I want a wasm function that works like this. I want this greet function, I'm going to stay alert, it's going to have a name. I'm going to write Rust like this, this is the extern that we already saw and we want to be able to use it in JavaScript like this. What happens? We write this and then what Rust does, and what you need to realize about what wasm-bindgen is doing, is it is writing a wrapper around your Rust and then it's also writing a wrapper around your WebAssembly in JavaScript to make these things play together.

The first thing, we see our greet function but then we see that generated greet and, it's taking that pointer and it's taking the length. That's how we're noting what that data is, and then if you take a look at alert, we're actually creating a placeholder and we're being able to pass that data into the placeholder with WebAssembly knowing that it's going to have to call that from your browser. Then when we look on the JavaScript side, we have this JavaScript that we'd really like to write, a nice, simple interface. We see this manipulation of the pointer, which is really just the index of where is in the array and then the length of where that data is, and so this is what it is called.

I would encourage you as you use this toolchain, if you ever want to peer under the hood and take a look at this code, you should try and understand at least how it's stringing everything together. As we move into WebAssembly being something that everyone is using, the more you know about how this works, it's just array manipulation and there's definitely ways to do it more efficiently than we currently are. This is a place where I think speeding up WebAssembly could be made so much better if we just had more eyes on it.

Exporting a Function to JavaScript

You'll see when you create something like this, these functions called passStringToWasm, getStringFromWasm and we are using text decoder right now to take those from strings, put them in numbers, take them from numbers, and put them into strings. What you need to realize is this was a whole bunch of work and this is just strings. Right now, wasm-bindgen supports literally every js API, Web API, and the ECMO strips standard for being able to convert a Rust function signature into a JavaScript one. There are some really beautiful moments where you go, “Whoa, how do we teach Rust to pretend it's doing JavaScript inheritance?” We do that, but there are a lot of very interesting lessons in there and that would take unfortunately an incredibly a long time to explain.

The real thing here is to say the best part about all this random generated code, manipulating arrays, you saw some unsafe blocks in there and probably didn't know what they were, but assumed they were bad and you're not wrong. The best part of this is that it is designed so that you literally never have to think about it if you don't want to. If you just want to build cool web apps and you want to be able to write it in Rust, then you want to talk to it like JavaScript, you just can. You can peer into the box, like this bird here and maybe you see something you don't like. Maybe you did not like that last 20 minutes, it bummed you out, just close the box. Just close the box, it's totally ok, sometimes it's fun to open the box. I love this gif, this is how I feel about looking at my own code, it's like, “No, close it.”

Why would we do all of this weird work? This was so much effort to make WebAssembly talk to JavaScript. Why would we do that now? That was so much, why would we do it? It's because WebAssembly isn't done. My favorite thing about the web is that we are still building it and the cool thing about the web is that you can help people build it. WebAssembly is just getting started. The thing that's been released is not final WebAssembly, it's the MVP, really. I don't know if you know Lin Clark but also Till Schneidereit and Luke Wagner, I worked with them all at Mozilla and they are brilliant people. Lin loves to draw code cartoons about WebAssembly and they are awesome.

WebAssembly has a lot of growing to do. In fact, this is a map of the growing that the WebAssembly wants to do. The interesting thing about languages and their growth is that in order to get people to be willing to participate, particularly a larger group of people, you have to give them a way to use it.

This is very close to me, but the keynote here is called we are not waiting and it's about the open pancreas system. It's about why type 1 diabetics are not waiting for this cool technology to become approved by the FDA, they're just starting to hack on it and use it now. I feel that way about WebAssembly right now, which is I'm not waiting. Eventually, they're going to build something but it's really cool that I'm alive right now and that maybe I can help them build the thing that I think they should build or be able to participate. We need to get people starting to use it now.

The best thing about wasm-bindgen is probably, in five years, it will be completely unnecessary and that's awesome. That's so cool to build this type of technology where you can already tell it won't be necessary but it will have helped so many people get involved so they could have helped make WebAssembly a tool that everyone's going to love. A lot of people in here really hated on JavaScript, it was kind of hard to get involved in JavaScript when they first released it. It'd be harder to be mad at it if you had an opportunity to participate that you didn't take. This is that moment for WebAssembly, and so this is your opportunity to get involved and a really great way to get involved is just to try it out.

You can help by joining the WebAssembly working group, we spend a lot of time working on this stuff, tooling what is the thing that we should build. Right now, WebAssembly, there's a lot of really awesome people in it but people are really focused on the technical details and they're not at all focused on the part about how people use it. Adobe Flash succeeded because it was an amazing developer experience, it exceeded for so long because it was such a good developer experience even despite pretty much everything else being a pretty bad design decision on their part. I want WebAssembly to succeed, which means people need to be able to use it. There's also webassembly.org, you can submit feedback and issues and you can join the community group as well. These are all open to anyone.

Why?

I'll leave you this, I have a lot of passion about this and you probably are, “Why does this lady care so much about WebAssembly? She loves it, what's the deal?” I care a lot about building tools that people use. This is a tweet of mine that went viral and you know it went viral because there's a typo in it. The cartoon says “I like this painting because it has a bench,” and I feel that very heavily. The cool thing about jokes is it's hard to know if they mean it positively or negatively and when people like the joke, you still can't tell how they feel about it. To make it clear, I build benches and that's what I care about and I think that that's the way to make technology that's successful.

The HTML5 design principle spec says in cases of conflict, consider users over authors over implementers over specifiers over theoretical purity. I feel like we should all follow this design principle all the time, which is to say I want people to use this tooling. We're going to do that huge song and dance with that array because I care more about you being able to use it than the pain of actually implementing it.

I want to do a small tour of these tools that I've talked about before I leave. First off, we have wasm-bindgen. This is a very fun code base. If you want to read the tests, they're organized just by the types of JavaScript APIs and literally their string concatenation with how we write their JavaScript. It's just fun to see that. This is wasm-pack, this is the tool that makes it so you don't have to worry about this tool at all but it will do it for you, which is super cool and you can publish your stuff up to MPN if you feel you'd like to do that. Additionally, if you were wondering if we can turn Rust futures into JavaScript promises, we can. You can use this to do it. It's getting there, we just got some good news for Rust async. It will be better soon but you can do it.

Additionally, when I showed that example, I wrote an extern block and wrote the alert, and if you maybe in your head went, “Oh shoot, do I have to literally define every function signature of every API I expect to use in my Rust?” No, but at one point you did and it sucked and we were like, “We've got to get rid of that.” This will do all that for you, so if it's anything in the Ekman script standard, for browser JS this will just make those for you. You can just import the function signatures. You can just be like, “I want to use this and it'll be fine.” We do the same thing for web-sys but just because a lot of people don't know the full first - how many people know what at a web IDL is? Fair, pretty much no one does. What if the spec actually compiled to code, is the pitch for web IDL. It's a way to programmatically define the web API's and so we actually generated web-sys with some difficulty but we pulled it off.

How many people here have heard that WebAssembly can't access the DOM? If you get into WebAssembly people will say that. It's true it doesn't have native access, but with web-sys you can and that's pretty cool. Then lastly, what if React was written in WebAssembly or just little pieces of it? We're building this, it's called Glue, check it out.

Then lastly, I've got to pitch my company, if you want to write Rust and WebAssembly, you can also do it on CloudFlare workers. I made another CLI with another crab logo because I'm insufferable.

To wrap this up, WebAssembly should be an implementation detail. Most people should never know they are even using WebAssembly and I think that that's its power and not necessarily a curse, though certainly growing the technology makes it difficult. We're going to see a lot of really cool things come onto the web and it's going to be unlocked by WebAssembly, but realize that with this power, I want to make sure it's inclusive to all types of developers and not just the types that want to, “It's fine, I can convert all my strings to numbers, I don't care” type of person. I want it to be accessible to everyone and so that's why this developer tooling really matters.

Let's go forth and make amazing things that expand the web platform or, if you came here because you didn't really care that much about WebAssembly, try to teach some languages how to talk to each other, you will learn so much. Translation projects are some of the best possible ways to learn languages. I know I learned about both JavaScript and Rust working on this project. If you're looking to get to know languages better, translation's an awesome activity.

 

See more presentations with transcripts

 

Recorded at:

Jul 22, 2019

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT