Transcript
Swan: I'm Chris Swan. Welcome to my talk on Backends in Dart. This was the final picture taken by the vehicle before it crashed into the asteroid. I wish this was a talk about the double asteroid redirection test. That would be super cool. It'd be even extra cool if that project had happened to use the Dart programming language. Dart is a client-optimized language for fast apps on any platform. At least that's what it says at dart.dev. Today is going to be a story about the three bears. We start off with the big bear, and he's wearing the t-shirt for JIT virtual machine. The big bear has got a lot of capabilities, can do a lot of stuff, but tends to use a bit more resource as a result of that. Then we have the little bear. The little bear is ahead-of-time compilation. Little bear is small and nimble and light, but misses some of the capabilities that we saw with big bear. Then, of course, lastly, we have the medium-sized bear. Medium-sized bear is representing jit-snapshot. This is something that the Dart compiler can produce, which should give us those best of both world qualities. Quick startup time, but without some of the overheads of running a full virtual machine. The question is, does that bear actually give us the perfect recipe for running our Dart applications?
I'm going to talk about what I do. This is the job description I wrote for myself when I joined Atsign, about a year and a half ago. Part one, improve the quality of documentation, samples and examples, so that atSigns are the easiest way for developers to build apps that have privacy preserving identity. Part two is to build continuous delivery pipelines as a first step towards progressive delivery and chaos engineering, so that we can confidently deploy high quality, well-tested software. Then part three is to lay the foundations for an engineering organization that's optimized for high productivity by minimizing cognitive load on the teams and the people working in those teams.
Outline
I'm going to start out talking about why Dart, both from an industry perspective and from a company perspective. Then take a look at some of the language features, so that if you're not familiar with Dart, you've at least got an overview of what it's generally capable of. I'm going to spend a bunch of time looking at what's involved in just-in-time runtime versus the ahead-of-time compilation, and the various tradeoffs between those. Zoom in to some of the considerations that are put in place running Dart in containers. Take a look at the profiling and performance management capabilities that are offered by the Dart virtual machine. Then try and answer that question about the middle bear. Is the middle way possible with the jit-snapshot?
Why Dart (Industry Big Picture)?
Why Dart, and look at the industry big picture. This is a snapshot from RedMonk's top 20 programming languages. They take a bunch of data from GitHub and Stack Overflow in order to figure out which are the languages that people are using and talking about and struggling with. From that, the top 20 is assembled, and there's a bit of discussion in each successive report every 6 months or so, about why particular languages are moving up and down in this environment. Dart, when I started using it was just outside of the top 20 and seemed to be rising fast. Now it's positioned equal 19th along with Rust. That's largely come about due to the adoption of Flutter. Flutter is a cross-platform frontend framework that's come out of Google. Dart is the underlying language that's used for writing Flutter apps. It's become an immensely popular way of building applications that can be build once, run anywhere. That's been pulling Dart along with it. Dart's also interesting for the things you can do across the full stack.
If I look at why we adopted Dart for Atsign, then I commonly say that we adopted Dart for many of the same reasons that RabbitMQ chose Erlang for their implementation of the AMQP protocol. Atsign is a personal datastore. What we're doing is providing a mechanism for end-to-end encryption. As usual, we talk about Alice being able to have a secured conversation with Bob. In this illustration, we're focusing on internet of things. Alice is a thing, wanting to send data from the thing to Bob using an application to get a view onto that data. We use personal datastores as a way of mediating the data flow and the network connectivity, and the sharing of keys. Alice and Bob each have their private keys, but they have their public keys in their personal datastores. The personal datastore acts as an Oracle for those. Then the network connectivity is through the personal datastores. Alice only needs to talk to their personal datastore, Bob only needs to talk to their personal datastore. The personal datastores can take care of talking to each other, in order to complete the end-to-end message path, which is encrypted all the way.
When we create a Dart ahead-of-time binary, that makes for a very small container that we can use to implement these personal datastores. Looking at the sizing here, on ARMv7, that container is as small as a 5-meg image. It's only a little bit larger, going up to about 6 megs for an AMD64 image. That is the full implementation of the personal datastore, including everything it needs for end-to-end encryption, and the JSON key-value store, which is used to share data between Alice and Bob, or whatever else is trying to share data in an end-to-end encrypted manner. This is the infrastructure that we use. This sizing of infrastructure will support about 6500 atSigns. We're using Docker Swarm to avoid the overhead of Kubernetes, for pod management there, and 3 manager nodes along with 18 worker nodes will comprise a cluster. One of those clusters will be able to support those thousands of atSigns. As we build out more atSigns for our customers, then we build out more clusters like this, in what I refer to as a coal truck model. As we've filled up one coal truck, we just pull the next one into the hopper and start filling that up with atSigns.
Language Features
Having a look at the language features then. Dart is a C-like language, and so Hello World in Dart looks pretty much like Hello World in C, or Java, or any of the other C derived languages. It's got an Async/Await feature that's very similar to JavaScript. In this example code, we're making use of Await in order to eventually get back the createOrderMessage. In the example, there's just a fixed delay being put in there in order to show that things are taking a little bit of time. In reality, that would normally be network delays, processing delays, database query delays, and whatever else. Its concurrency model is based upon the notion of isolates. Where an isolate is an independent worker that's similar to threads but not sharing memory, so communicating only through messages. Since March last year, Dart has been implementing sound null safety. What this means is, by default, the types are not nullable. If I say that something is an integer, then it must actually be an integer and it can't be null. However, if I want to make sure that I can still have a null case for an integer like type, then I can declare it with the question mark at the end. Then I'll have a type which can be either null or an integer. Dart 3 is going to be coming out next year. A big part of the shift from the Dart 2 versions at the moment, to Dart 3 is going to be greater enforcement of sound null safety across everything that Dart's doing.
Licensing for Dart itself is using the BSD-3 license, which is one of the OSI approved licenses. Generally speaking, the packages that go along with Dart tend to be licensed with BSD-3 as well, just about everything that we open sourced from atSign is using BSD-3. As we look at our dependencies elsewhere, they're typically BSD-3 as well. The package manager for Dart and Flutter is called pub.dev. Going into that package manager, you can see a huge variety of packages supporting many typical things that you would want to do with code. As package managers go, I found it to be one of the best that I've worked with. It's very recently got support in GitHub, so that Dependabot will identify updates to packages and automatically raise pull requests for those.
JIT vs. AOT
Moving on to just-in-time versus ahead-of-time. I'm going to illustrate this using this showplatform mini application, which is really just a slightly more useful Hello World. It's importing from the dart:io package, two things, the platform module, and standard out. That then lets me have the ability to print out the platform version. This gives you a similar output to invoking Dart with --version, which will give you the platform version that it's been built on and for. At the time that I was putting these slides together, the latest release of Dart was 2.18.2, at least the latest stable release. You can see here, I was running that on a Linux ARM64 virtual machine. With just-in-time, I can use dart run and just run it in the virtual machine. Here I'm preceding it with time so I can see how long these things take. If I just dart run showplatform.dart, I get the same output as before. Time took a little over 6.5 seconds. That 6.5 seconds is the time it's taking to spin up the virtual machine for the virtual machine to bring in the code, turn it into bytecode, run the bytecode, produce its output and shut itself down. That's quite a big cold start time. What I can do instead with Dart is use it to compile an ahead-of-time binary. This will result in the output of a native binary that I can run on that platform. Then I can time running the native binary and see how long that takes. We can see here that the native binary is taking a fraction under three-tenths of a second. It's really quick in comparison to that 6.5 seconds of spinning up the VM. For things like Cloud Functions, AOT binaries are a good choice, because it's getting past that cold start time.
Of course, there's a tradeoff. In this case, the tradeoff is that compilation is slower than just running the application in the first place. If I time how long it takes me to compile that executable, then I can see that that was taking of the order of 20 seconds to create the executable. Generally speaking, that's a tradeoff that's going to be well worth taking. Because that time is going to be taken in a continuous delivery pipeline, where it's not affecting the user experience in any way. The user experience is going to be the one of getting a quick response from the native binary.
Dart in Containers
Taking a brief look at Dart in containers. Here's a typical Dockerfile for an ahead-of-time compiled Dart binary. I start off with saying FROM dart AS build. This is going to be a two-stage Dockerfile. In stage one, I'm basically doing all of the Dart stuff and compiling my binary. Then in stage two, I'm creating the much smaller environment that's going to run the binary. Dart has for a while now been an official Docker image, which is why I can simply say FROM dart. Then, AS build means that later on, I can use that label as something to reference when I'm copying parts out of the build image into the runtime image. I then set up my working directory, I copy my source file, and then dart pub get will ensure that all of the dependencies are put in place. Then as before, dart compile exe is going to create a native binary. The second stage here of the build is beginning with, FROM scratch. That's an empty container, so absolutely minimal. Dart is a dynamically linked language. I think that's one of the more fundamental philosophical decisions that has been made about Dart. Hence, there's a need to copy in some runtime dependencies. Dart dynamically links to libc components, and the runtime directory in the Dart Docker image contains everything that's needed from a dependency perspective to run those ahead-of-time compiled binaries. With runtime in place, I can then copy the binary itself into the container, and I can then define the ENTRYPOINT into that for the container image. The image size for that dartshowplatform trivial application is pretty small. It comes out at just over 3 megs for ARMv7, and about 4.5 megs for AMD64. As I mentioned earlier on, even for something non-trivial like an atSign, then the images can still be incredibly small. The atSign image for AMD64 which is the architecture we most commonly use to run it is just under 6 megs there.
Profiling and Performance Management
One of the nice things about Dart is it's got a whole bunch of profiling and performance management capabilities built into the tool chain. Where with other languages, we might need to buy sometimes expensive third-party tools in order to get that profiling and performance management data, that's all there built in with Dart. I think that's often very important from a developer accessibility point of view. Because if there's a hurdle of having to pay for tools, then very often developers will then be deprived of it, or have to do a bunch of extra work to get the tools that they need. This chart explores the range of Dart DevTools that are on offer. You can see here that that is a most complete for Flutter applications running on mobile or desktop environments. Flutter can also cross-compile to JavaScript for the web. In doing so, it's then divorcing itself from the Dart runtime environment, and so there's much less available there in the way of profiling. The same goes for other web approaches. Then Dart command line applications of the type that I've been using examples in which we might use to implement APIs to be the backend services that we're using for full stack Dart applications, have everything except for Flutter inspector. Flutter inspector is a Dart DevTool that's specific to building Flutter applications. Dart command line applications get everything else. That's quite a good range of tools that are on offer.
Here's a look at the memory view. Very often, we're concerned about how memory is being used, and what's happening in terms of garbage collection cycles, how frequently they're running and how long they're taking. As this illustration shows, we can connect to a running Dart application in its virtual machine and get a really comprehensive profile of the memory utilization and garbage collection cycles using the memory view. Similarly, Flame Charts give us a performance understanding of where an application is spending its time. That allows us to focus effort on some of the more timely aspects of the stack, and maybe some of the underlying dependencies and how those are being used. There's a little bit of a catch here, DevTools needs to connect to a virtual machine. DevTools are for just-in-time compiled applications only. I reached out to one of the Dart engineers to ask if there was any way of tracking down memory leaks for ahead-of-time Dart apps running in containers. His response was, essentially, "Yes, we got that for Flutter." All of the pieces are there, but right now, that's not part of the SDK offering. You'd have to compile your own SDK in order to make use of that.
A Middle Way with JIT-Snapshot?
Is there a middle way then with jit-snapshot? Let's have a look at what's involved with that. Just as I can dart compile exe, I can just parse in a different parameter here and say, dart compile jit-snapshot with my application. Then once that compilation process has taken place, I can run the jit-snapshot in the Dart virtual machine. What we see here is, having made a jit-snapshot, the Dart startup time is much quicker than before. Before it was 6.5 seconds, we're now under 1.8 seconds in order to run that Dart application. We haven't completely gotten past the cold start problem but we're a good way there. I'd note that for a non-trivial application, really what's needed here is some training mode. That would be so that the application goes through the things that you would want to be taking a snapshot of, but then cleanly exits, so that the compiler knows that that's the time to complete the jit-snapshot, and close that out.
There's a bit more involved in putting a jit-snapshot into a container than there was with an AOT binary. The first stage of the Docker build here goes pretty much as before, except we're compiling a jit-snapshot instead of an executable. In the second stage, as well as those runtime dependencies, we also then need the Dart virtual machine. Not just the single executable for the Dart virtual machine, but the complete environment of the machine and its libraries and snapshots that comes with it. That results in a pretty substantially larger than before container image. With that in hand, what we can then do is define an ENTRYPOINT where we're parsing in that observed parameter, and that allows us to have the Dart DevTools connecting into that container and able to make use of the observability that's on offer there, rather than before having to fly blind with the AOT binaries.
In terms of, are we in the Goldilocks Zone here? Let's compare the container image size. A secondary is the name that we gave to the implementation of an atSign. The previous observable secondary that we use with a complete Dart SDK inside of it was weighing in at 1.25 gigs. Now compare that to the uncompressed image for a production AOT binary at only 14.5 megs. It's a huge difference. My hope had been that the jit-snapshot would give us images, that would be of the order of a 10th of the size of the full SDK. They're not that small. I think I could have maybe squeezed them a little bit more, but they're coming in at about a third of the size. In the end, that's not the main consideration for these. Running a larger state of atSigns, we care about resource usage. To get 6500 atSigns on to one of those Swarms I illustrated earlier, that's primarily dependent upon the memory footprint of the running atSigns. If I look at resource utilization here at runtime, then the AOT binary in a quiescent state is using about 8.5 megs of RAM, whereas the observable secondary using jit-snapshot is using more than 10 times that. The practical consequence of that is my cluster that's presently able to support 6500 atSigns would only be able to support about 650, if I move them all over to being observable, by making use of the jit-snapshot.
In this particular case, the jit-snapshot wasn't the Goldilocks Zone I was looking for. It's a vast improvement over the approach that we've been taking before, but still not quite there in terms of giving us the ability to make all of the environment observable, which would have been a desirable feature. I think this might be possible if I was using a memory bubbling virtualized environment, because a lot of these containers are going to have essentially the same contents. You need a really smart memory manager in order to be able to determine the duplicate blocks, and work around those in terms of how it's allocating the memory. This was the picture of the Dart vehicle crashing into the asteroid. My exploration of jit-snapshot has crashed into reality in terms of the memory overhead is still far much more than I would want it to be to make widespread use of it.
Review
It brings us around to a review of what I've been going through here. I started out talking about why Dart, and looked at how Dart is becoming much more popular in the industry, as especially people use it to write Flutter applications. I think that pushes towards the use of full stack Dart so that people aren't having to use multiple languages in order to implement all of their application. The language features of Dart are pretty much those that people would expect from a modern language, especially in terms of the way it deals with asynchronicity and its capabilities for concurrency. I spent some time examining the tradeoff between just-in-time and ahead-of-time. Just-in-time is able to offer really good profiling and performance management tools through Dart DevTools. We'll also be able to optimize code better over a long duration, but it comes with that cold start problem that we're all familiar with from languages like Java. Ahead-of-time is what we have been using in order to have quickly starting small applications that are quick to deploy.
Getting Dart into containers is pretty straightforward, given that it's now an official image. That official image also contains that minimal runtime environment that we need to put underneath any AOT binaries that we make. A pretty straightforward two-stage build allows us to construct really small containers with those binaries. When we use AOT, we miss out on some of the great profiling and performance management tools that Dart has on offer. This led me to exploring a middle way with jit-snapshot. Jit-snapshot is where we ask the compiler to start an application, to spend some of its time doing JIT optimizations. Then, essentially, take a snapshot of what it's got so that we can launch back into that point later on, and get past some of the cold start overhead. This did minimize resource utilization somewhat, but not enough to be a direct replacement for the ahead-of-time binaries that we've previously been using.
Call to Action: Try Dart
I'd like to finish with a call to action, and suggest, try Dart yourself. There's a really great set of resources called Codelabs. One of those is an intro to Dart for Java developers. If you're already knowledgeable about Java, and I expect many people will be, then that's a great way of essentially cross training. Then the Dart cheat sheet is a really good run-through of all of the things that make Dart different from some of the other languages that you may have come across. When I was going through this myself a few months back, it was a really good illustration of how code can be powerful but concise and easy to understand, rather than just screen after screen of boilerplate. Try those out at dart.dev/codelabs.
Questions and Answers
Montgomery: Is there anything that you'd like to update us on, like any changes to Dart, or to things that you've noticed since you put the material together originally?
Swan: As I was watching that playback through, I noticed I was using Dart 2.18.2 at the time. Nothing too major has happened since. Dart 2.18.5 came out, but that's just been a set of patches. I might have mentioned RISC-V support coming in Dart. That had been appearing in the beta channel on the Dart SDK download page. In fact, one of my colleagues has been doing a whole bunch of work with RISC-V single-board computers based on that. I went to do some stuff myself. I was actually having a view on trying to get the RISC-V Docker support knocked into shape. It turns out that it shouldn't have been in the beta channel, it's still in dev. There's still not a test infrastructure regularly running around that. It's going to be a little longer for RISC-V support than maybe people were thinking. Then, also, as I was embarking on that adventure, things were a little bit dicey with upstream dependencies as well. The Dart Docker image is based on Debian. Debian hasn't yet actually released their RISC-V full support. You can't get RISC-V images from Debian itself, and there's no Docker RISC-V images yet. Ubuntu are a little ahead of them there. I can't see folk changing track at the moment from Debian to Ubuntu in order to support that.
The big news coming is Dart 3. That's going to be more null safety. Null safety was released last March with the 2.12 version. That ended up being much more of a breaking change than I think was anticipated. Although there was a mechanism there that you could continue compiling things without sound null safety, and you could continue casting variables without null safety. I think as people were upgrading their Pub packages and stuff, it was a much more disruptive change. Maybe that should have been Dart 3, and then we'd be talking about Dart 4, because what's coming next is essentially enforced null safety. We can think of Dart 2.12 to now, and that will take us through 2.19, and maybe 2.20, as being easing into null safety. Then with Dart 3, it's null safety all the way. Yes, that should result in faster code, it should result in safer code. Obviously, it's something of an upheaval to get there. This transitionary period should have helped. It also means that they're acknowledging there's a whole bunch of stuff, won't carry on working with that, and we'll be leaving behind the world of unsafe nulls.
Montgomery: It will also probably shake out a lot of, not only usage, but I do know that a little bit of when you're having an environment and you're changing, and you're enforcing that, you're also adopting best practices to support that. That will be very interesting to see how that shakes out. It's just like heavy use of Optional within lots of languages, has always introduced a fun ride for the users and maintainers. I know C# went through a very interesting time. Java with Optional is going through, has in various places been very interesting to adopt. I anticipate with Dart, it will come out the other side and be very interesting.
For which type of application do you recommend using Dart?
Swan: The primary use case that people have for Dart at the moment is building Flutter applications. Flutter being, as I said, the cross-platform development framework that's come out of Google. Especially mobile apps that have iOS and Android from one code base, Flutter is just great for that. Dart's the natural language to do it. I think the place where full stack Dart then becomes most attractive is you've written your frontend in Flutter, you've had to learn Dart to do that, why then use another language for the backend? For a team that's then using Dart as a matter of habit for their frontend, it really makes sense to be able to do that on the backend. There's a bunch of accommodations that have been put in place. There's a functions framework that can be used in order to build functions that are going to run in Cloud Run. That's offering a bit of help. I think beyond that, we've found Dart to be a great language for implementing a protocol. There's aspects of Dart that I feel are quite Erlang-ish. Erlang being the language out of Ericsson, to run on routers, and again, very much a language for implementing protocols. If we look at how Erlang became popularized over the last decade or so, the main thing with that was RabbitMQ. I think Rabbit on its own led to massive proliferation of Erlang VMs, and got a lot of people to be Erlang curious. The same things that made Erlang good for AMQP make Dart good for implementing our protocol. I would extend that to other protocols.
Then, really, we get back to this lightweight AOT thing. I think Dart is probably neck and neck with languages like Go for being able to create these very compact Docker images that have got the functionality inside of them. I was having a conversation with a customer, and the question was asked, how do you secure your Docker images? There's no OS in there. It's giving you the opportunity to get rid of a whole bunch of security surface area, by not having to have the operating system in there. Of course, there's other languages that can get you either a statically compiled binary, or a dynamically compiled binary and a minimum set of libraries that you need to slide in underneath that. Then into the tradeoffs around expressiveness and package manager support and things like that.
We've also found it pretty good on IoT. We've ourselves for demos and stuff been building IoT applications for devices, so system-on-chip devices, not MCU devices, because it does need Linux underneath it. In that case, we can do interfacing to things like I2C, and then go straight up the stack into our protocol implementation and onto the wire with that. That's been an easier journey than we might have anticipated. Because things like the I2C driver implementation is just there as a pub.dev package. We didn't have to go and write that ourselves.
Montgomery: I think one of the things that has stood out to me is some of the things that Dart does do that is quite different for most frontend languages, that actually does lend itself pretty well.
What are the advantages of Dart compared to TypeScript? Considering the fame of TypeScript being one of the go-to frontend languages now that's used.
Swan: I think there's frontend and front end. I didn't talk about Flutter on the web. Maybe this just gives me an opportunity to briefly touch on that. Dart, when it was originally conceived, Google thought that they were going to put a Dart VM into Chrome, and people would run Dart inside the browser. That didn't end up happening. History went off a different leg of the fork. I think one way we can think of frontend these days is anything where we're presenting the user interface of an application. The natural language of the web is JavaScript. Then TypeScript has clearly improved upon that by giving us types and type safety in JavaScript. The natural language of devices is Swift or Kotlin. Then we've got all sorts of cross-platform frameworks, Flutter being one of them, that allow us to write once and run on those. There's also Flutter web. What happens with that is Dart actually transpiles into JavaScript. We can take a single code base and not just have Android, iOS, and macOS, Linux, and Windows, but we can also have web applications from that as well. I think the advantages versus TypeScript, taking you into considerations around a title defined set of packages, and some of the idioms around those. Also, it's not just JavaScript with types bolted on, but a language where types and now null safety have been baked in from the very beginning. It's much more stronger leaning in that direction. The result is you can still get stuff that runs on the web out of it. It's not a big thing for my own work, because key management inside of browsers is a nightmare. There's lots of other applications for it.
Montgomery: Can Dart be used for serverless functions? I don't know if there's any AWS support, or any other cloud provider supporting it.
Swan: If we're talking about Lambda and Google Cloud Functions and Azure Functions, none of those have native support for Dart. Interestingly, even Google doesn't have native support for Dart in their own functions offering. However, this is where the functions framework comes into play, because the functions framework is all about helping you build functions that you're going to deploy as containers. Of course, all of those clouds now have mechanisms for you to deploy your function in a container, and route traffic through to that and have the on-demand calling and stuff like that. You're going to be getting some of the advantage I was talking about there, because, generally speaking, you'd use an AOT compiled binary in that container, which is going to have super swift spin-up. You're not then worrying about VM startup as you might be with a language like Java, which people also use for serverless.
In terms of support from AWS, AWS doesn't have native support for lambda. AWS in another part of AWS with Amplify, has now got a really strong Dart and Flutter team. Some of those guys, GDE, so Google Developer Experts, I keep an eye on what they're up to, and of course, they were all at re:Invent. Dart and Flutter has been embraced to that extent, by AWS. I've also seen both AWS and Azure people showing up in contexts where it's a conversation about what we're here for, Dart on the backend. Because if people are going to be writing Dart on the backend, they don't want them to be just defaulting to Google, because it's a Google language. They want to be saying, our serverless implementations or Kubernetes implementations are going to be just as good at running your Dart code, when it's packaged up in that manner.
Montgomery: Maybe take a look and see what gets announced out of re:Invent. You never know what may happen.
Swan: Some pre:Invent stuff came out from the AWS Amplify team about new stuff that they're doing with Dart. They've put together a strong team there, they're really active, and showing up to not just AWS events, but community events. There was a Flutter community event called FlutterVikings, and there was a whole stand of AWS people there, in the mix alongside a lot of the Googlers.
See more presentations with transcripts
