BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations npm and the Future of JavaScript

npm and the Future of JavaScript

Bookmarks
52:10

Summary

Laurie Voss talks about what npm knows about JavaScript users and how JavaScript usage patterns are changing. He also talks about JavaScript security, what tools people use, what the community believes best practices really are, what frameworks are on the rise and which are on the wane, where the major pain points are for devs right now, and future direction.

Bio

Laurie Voss has been a web developer for 22 years and he is currently the co-founder and COO of npm, Inc.

About the conference

Software is changing the world. QCon empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams.

Transcript

Voss: It's important before we start talking, then I take a crowd selfie. Otherwise, it didn't really happen. So look happy. Like you just learned some shit. There we go. I can't take it afterwards because I never remember. So I have to do it before. I'm just, like, I'm borrowing some goodwill from you and hoping that I can pay it back. Anyway, so with that important task done? Hello. As she said, I'm Laurie Voss. I am the co-founder of npm, Inc. I have been a web developer for 22 years. The title is COO, but when you're a co-founder they have to give you some kind of C title. Don't take it very seriously. It will probably change.

But what I'm trying to do today is give you some sense of what's going on in the world of JavaScript, because I have found that this is a thing that JavaScript developers don't have. A lot of people who work in the web space operate in a sort of vacuum of whether the stuff they're doing is a best practice or a fad or some weird trick that they picked up from a coworker. And everyone's sort of like, "Well, this seems to work and I don't really know if I'm just making it up or if this is what everyone does." But we know because we can see you doing it every single day. So I'm here to give you some reassurance on that front. But also, I can give you some data about what is and isn't popular, what is going up, what is going down. And the idea there, is that I hope that you can use that information to help you make some choices about what to do in 2019, technical choices about what you should adopt, what you should learn, what you should move away from. And I'm also going to be adding to this talk a little bit about the longer-term future, beyond the 2019 stuff.

So there are three parts to this talk. First, there's just going to be a quick tour of some of the newer features of npm, especially the stuff that we've added in the last year. And the second is going to be the part where I'm talking about you, the demographics we have about you, and what you're up to and where you are in the space. And then finally, I'm going to make some predictions about 2019 and what tools I think you should look into. So stick around to watch me be extremely wrong about what is going to happen next year.

What You Should Know about npm

The first thing that you should know about npm is that these days, npm is extremely popular. This comes as either news or not at all news, depending where you sit in this space. npm has more than 10 million users these days. And they download more than 8.1 billion packages every seven days. We think that more than 90% of developers who use JavaScript's are currently using npm to put it together. And that's heading towards 100%. There is no final holdouts who are like, "This will never be useful." No, that's just been people haven't got around to it yet.

And that's even more amazing because by a lot of measures, JavaScript is the most popular programming language in the world right now. It has been the biggest language on GitHub for the last five years running. 70% of Stack Overflow developers report that they use JavaScript, at least some of the time, which means that if you do the math, it is possible that 50% of all programmers use JavaScript at least some of the time. Whether or not they consider themselves Javascript developers, 50% of working programmers are using JavaScript to do something at least some of the time.

We actually went and manually checked. And all 50 of the Fortune 50 are using npm in production, all 50 of the 50 biggest financial companies, all 50 of the 50 biggest tech companies. We haven't checked every single one of the Fortune 500 yet. But every single one that we've checked is using npm in production. Fun fact. Berkshire Hathaway's website is ridiculously terrible. It's, like, html with a pile of font tags and a link saying, "Please don't email us about our website." But luckily one of their subsidiaries uses npm in production. So we were okay.

What this all boils down to is the JavaScript has become, for better or worse, the most important programming language in the world. And that is a strange thing to say. It's some sort of a self-serving thing to say, obviously, since I work at npm. But this was not the plan, right? When we were doing form validation in 1997, this is not what we were thinking would happen. We were like, “We're going to conquer the world and everyone's going to write JavaScript for everything." No, that was not what we thought was going to happen. But here we are. So we have to start taking Javascript seriously, especially in the enterprise.

The next thing to know about npm is that npm is being used everywhere that JavaScript is being used. And JavaScript is being used everywhere. People are building websites in JavaScript, obviously. But people are also writing server site apps in Node-JS. A bunch of people are doing robotics in IoT in JavaScript. And a bunch of people writing mobile apps in JavaScript, either mobile web apps or actual mobile apps that get transpiled from JavaScripts down into a mobile code. And npm has become the package manager for all of them. Npm has become the package manager for all of JavaScript, which again wasn't the plan. And npm invented itself to be the package manager for Node-JS. And we've had to reinvent ourselves and rethink about our defaults and who we cater to because of this massive shift in our user base.

Because npm’s primary users are now web developers, that's who we have to care about first. Ninety-three percent of npm users are writing code that will eventually run in a browser. Of course, 70% of our users are still writing server side code. And those people will always be well-served. But npm has become the way that you build websites in 2018. One way that you can tell how central npm has become to web development, is that 97% of the code in a modern web application is downloaded from npm. You, the application developer, write only the last 3% of the code. It's the most important 3%. It's the bit that makes you unique and useful. But 97% of the code is coming from npm. That's a triumph of code reuse, right? Everybody is saving an enormous amount of time. Everyone's getting much higher quality software because we're doing an enormous amount of reuse of code and debugging of code that already exists and it's great. But it changes some things. And it creates some new risks. And I'm going to be talking about those, as well.

But first, what version of npm are you on? If you are using anything less than version six, which something like 40% of you are, then you are woefully behind the times, it's time to upgrade to npm 6 because it is 20 times faster than npm 4 was. If you don't need any other reasons, it is 20 times faster and does all of the same stuff. And whenever I say it's faster, somebody in the audience is like, "But is it faster than YARN?" And the answer is yes, it is faster than YARN or rather, it's basically the same speed as YARN and all of the package managers the same speed now because that's how open source works. YARN came out and npm and PMPM and all of the other package managers for JavaScript that I bet you didn't even know existed. They all formed a big community. And they started talking to each other and sharing techniques and exchanging code snippets. And as a result, all of the package managers got a lot faster in the last two years. But now most of the low-hanging fruit is gone. And so they're all much faster, but roughly the same speed.

So another big thing in npm 6 is npm ci. It is a huge use case of npm 6. One of the reasons that people are downloading 8.1 billion packages a month, sorry, a week, is not because people are like, "Yes, typing npm install 8 billion times." No, people are running CI servers a lot. And CI servers just trigger every time anybody does a commit to anything. And it downloads 10,000 packages every time that happens, which is how you would get very rapidly to 8.1 billion packages. But we discovered that there are some properties of CI environments, which allow npm to run a lot faster because, we don't have to do a bunch of error checking and stuff. We know what a CI environment looks like. So if you use npm ci, instead of npm install, in your test environment, your installs will magically be twice as faster. Your builds will be twice as fast. Nothing else will change. And it's just free performance for you.

As npm has got bigger, security has become an even bigger concern for us. Earlier this year, npm acquired a company, which was new. We have never done that before. We acquired Lift Security. And we immediately incorporated the node security platform into npm itself. This resulted in a bunch of new features. The first of those was two-factor authentication. Two-factor authentication, it's basically table stakes now. It's sort of embarrassing that it took us this long. It's important for everyone. But it's especially important for package authors. Package authors need to have two-factor auth enabled so that nobody can publish a package while pretending to be you, because that is a thing that happens now. And I'm going to talk about that in a bit.

In May, we also launched npm Quick Audits. If you use any recent version of npm, you've probably seen them because they run every single time you install. npm will tell you every time you install if any of the packages that you have just downloaded contain any known security vulnerabilities. This is an extremely useful ambient signal to the community about whether or not the code that they are installing is secure. It works in nearly every version of npm. But in npm 6, the warnings are better and they're more detailed.

We're now doing around 4 million of these quick audits every week. And the stats that we're getting from those scans are pretty alarming honestly. Eleven percent of the scans that we run contain a critical vulnerability. That should be, like, a smash the glass, hit the red button. You should not be shipping this code vulnerability. And it's in 11% of installs, despite the fact that we're telling you every time you do it. Critical vulnerability, it's still happening. But fortunately, improving that is easy because running npm audit will give you these detailed reports that I talked about that will tell you not just what is wrong, but where that is coming from. And most importantly, it will tell you how to fix it.

And the way to fix it nearly all of the time is to just upgrade from the insecure version of the package that you're using, to the fixed version that the author has already released. And that created an interesting situation where we were like, "Well, npm audit is mostly just telling you to run npm again. And if npm is telling you to run npm, why doesn’t npm just run npm, like why doesn't he just do that?" So that's what we did. So npm audit fix will automatically follow its own advice. npm audit fix will run everything that it can safely run within the bounds of Semver, to repair your software and bring it up to currently secure versions. By default, it will only bring in Semver compatible fixes. But if you -- force, it will bring in breaking changes as well. We have not solved the halting problem. We have not written a piece of software that can write your software for you. But what we have done is pulled a lot of the grunt work out of keeping a very large JavaScript application secure, so that you can just pull in everything and then make sure that everything is still working.

We've also put a ton of effort into security in general because of these changes I've mentioned. When 97% of the code that you're using is coming from us, we, as a company, have a responsibility to help you keep it secure. When the average tree of JavaScript modules and the average JavaScript application is using over 1,000 sub-dependencies, we have an obligation to automate that. We can't possibly pretend it's your responsibility to manually verify that 1,000 modules are all okay. That's not a reasonable thing for us to ask you to do. So us automating it is part of our responsibility to the community.

And this brings me briefly back to YARN. We used to say that we didn't care what client you used. Every client is great, open source, open source. But we can no longer say that with a straight face. With all the security features that we've added, with the addition of TFA and npm audit, nmp is a lot safer to use than YARN. And so we, in a groundbreaking never before seen event, our company recommends its own product. npm recommends that you use npm to access npm. That link on the slide is to a blog post from a company that switched to YARN and then back to npm, and a link to the tool that they wrote that will migrate your YARN projects to npm projects.

The other big recent change about npm that I should mention is that we turned off our GitHub issue tracker. We are big fans of GitHub. But the issue tracker just was not working for us. So we now have a discourse at npm.community, which is a domain because this is 2018. It's a combination of an issue tracker and also a forum so you can get help, as well as track new features and report bugs.

And the last thing to know about npm, which I keep including because people keep asking, is how does npm make money? Are you some sort of charity? No, no, fuck no. We are a company. The registry costs millions of dollars a year to run, just the servers and bandwidth, like 8 billion packages a week. They're not cheap to ship. So we make money. And the way that we make money is by selling services that npm users love. The first way is called npm Organizations. You just use the same registry you've always been using. But now you can publish private packages. And you can invite all of your team members in and give them granular access controls on publishing and installing. And you can see what's up. It's very popular and it's very simple. Tens of thousands of people are using it.

The next step up from that is npm Enterprise. This gives you your own private registry with its own domain. People can sign in using single sign on, like Google or OCTA or whatever it is that your corporation users. And you get your own private npm website where you can search and browse documentation of your own company's packages. You can do discovery of other people's work within your big corporation. It's super fun. And in both products you get enhanced npm security. The same npm audit command that works on the public registry will work on your private registry. But in addition to public-known vulnerabilities, you will also get embargoed security vulnerabilities that our security team has found and are in the process of fixing, so you can have advance warning that something is wrong. And you can fix it before the public disclosure goes out. I apologize for this being an obvious and transparent sales pitch. But this stuff really is genuinely useful and genuinely popular.

What npm Knows about You

But now let's move on to what npm knows about you. First off, how do we know all of this stuff? We had said that we interviewed 600,000 people, which is a mild overstatement. It was 16,000. We did a survey of 16,000 npm users earlier this year. And we got people to tell us directly what is it that you are up to. And also when people are downloading 8 billion packages a week, that gives us an enormous amount of data that we can mine about what's going on. We can see what people are downloading, what people are downloading more, what people are downloading less. And it gives us a bunch of insight into what's going on in the world of JavaScript. And the results are fascinating.

But first I want to try a party trick. And I never know with any given audience whether or not it's going to work. So I'd like everybody in the room to stand up. I know how much this audience loves audience participation. And I'm going to make statements about you. And if the statement is true, then you stay standing. So if it's not true, then you can sit down. So stay standing if you use npm. Good, and you have understood how this works. If you write JavaScript for browsers, if you write JavaScript at work, if you're concerned that maybe the open source code you use isn't always secure; these are all really easy. You mostly taught yourself JavaScript. Wow. In addition to JavaScript, you also write PHP or Java sometimes. These are where they begin to get tricky, and I start eliminating people. You work at a company that isn't really considered a tech company. You started using npm less than two years ago. We're going to get really tricky now. You're using Webpack and Babel. You're writing a React app using TypeScript. Wow. I got all of you. I know so much about you. I basically know your last names. That's it. That's the whole party trick.

So we know enormous amounts about what's going on in the world of JavaScript. We know that this is the typical JavaScript user these days, right? It's somebody using Webpack and Babel and React and TypeScript and putting together a web app. It's mostly on the front end, blah, blah, blah, blah. But let's go through and explore those a little bit. The first thing to dig into is that npm users aren't only writing JavaScript. And, in fact, most npm users don't primarily consider themselves JavaScript developers. They consider themselves web developers working in other languages and using npm and JavaScript sort of incidentally to that. So 30% of you are either writing Java, PHP, or Python. C-sharp is also a popular back-end language. Go and C++ and Ruby are all in there, as well.

And that was surprising to us, that so many npm users don't consider themselves JavaScript uses. But it possibly shouldn't have been. There is a study that I love quoting by a guy called Leo Meyerovich at UC Berkeley, who did a study, in fact, a whole PHD, about why people choose programming languages. What makes them choose programming language? Is it the performance of that language? Is it the features of that language? Is it that their team knows that language? Is that that they don't have any choice? You know what? Why do you pick a programming language? And he discovered that the dominant reason was the availability of open source libraries that help you get the job done. So whether or not you know the language, if there is a library that will help you do the thing you're trying to get done, you will tend to use that language.

And that was interesting because that's exactly what you told us when we asked. You said that the reason that you picked JavaScript is because of the libraries available. So it's no surprise that npm's popularity is growing because JavaScript is growing in popularity. But what is surprising is that it's quite possible that JavaScript is growing in popularity because of npm. There are 828,000 modules in the JavaScript registered in the npm registry these days. And that is contributing to the success and popularity of JavaScript as a language. There was also a sad 15% in there who said that they didn't get any choice in the language they use. And interestingly, they were mostly Ruby developers.

Another important finding was how big a concern to most users the security of open source was. This was at the beginning of the year before we added all of these security features. Seventy seven percent of people said that they were concerned about security. And even more worryingly, 52% of people said that the tools that existed, at least back then, were not good enough, that they didn't have anything to do about that stuff, which meant that there's a percentage who are like, "I'm concerned about security and I don't care." So that's a little about who npm users are.

The Tools We Use

But what I promised you was information to help you make technical choices in 2019. And for that, we need to look at the tools that we're already using and what we're doing with them right now. And this is where I begin to run into trouble because developers are very passionate about their tools. When there's a tool that you like, you tend to get defensive about it, especially if I tell you that it's not popular anymore. People send me angry emails and tweets and yell at me about my methodology. And all I can say is that I have no dog in this fight. I am not a maintainer or contributor to any major framework. All I'm doing is showing you the numbers and telling you what's going up and what's going down. I'm not saying your framework is bad. I'm just talking about its relative popularity. If you want me to tell you that your framework is bad, that is what the Q&A is for.

So before I show you these angry-making graphs, let's put this in context. This is how the registry grows. This enormous, enormous growth. The registry has grown 19,500% since npm, Inc. started in 2014. It is very, very hard to wrap a human brain around that kind of growth. The fact that the registry is growing means that npm users are showing up all the time. There's constantly new users. That's why half of people who are using npm have only been using it for two years. It means that everything in the registry grows. Every single package in the registry, no matter how shitty it is, is constantly acquiring new users because there's just new people showing up all the time going,"I don't know what anything is. Let me use eval package one."

So that line at the top of the graph there that is Express. Express is bedrock to npm. Everybody's using Express for something. It sneaks its way into everything. Let's look at Express as a percentage of all download, instead of as a percentage of absolute downloads. And you get this very different line. You get this flat line. We call this metric the share of registry. This is expressed downloads as a percentage of all of the downloads in the registry. So this flat line means 19,500% growth. Express has kept pace with the registry exactly. Express used to be 1%, 1.5% of all registry downloads. And these days it's only 0.1% percent. But 0.1% of downloads is 4.8 million downloads a week. Express is staggeringly popular. So this metric is hard to get your mind around. But it does give a very interesting picture of relative growth of packages. And all of these graphs that I'm about to show you are using this metric.

So first let's take a look at some front end frameworks, starting with the very oldest framework, which is Backbone. As you can see back in 2013, Backbone was the shit. Everybody was using it for everything. It was almost as popular as Express. But now, basically nobody uses Backbone. And by basically nobody, I mean 250,000 downloads a week. The thing that you can see here better than anywhere else, is how a framework dies. Very few people switch frameworks once a project has started. Nobody takes a project that's already existing and rewrites it in a new framework. They take the old projects, they maintain the old project until it gets retired, and they start new projects in the new language. So this means that frameworks tend to have a decay curve. They tend to fall off as they're getting unpopular. So nobody's writing new software in Backbone. But lots of people are in maintaining existing legacy applications written in Backbone.

Now let's look at React. React is goddamn running away with the web. I have never seen a framework act like this. Sixty percent of respondents to our survey said that they are using React for something. That is absurd. That's 6 million people. It is huge usage. And it's impressive growth. But it's no longer runaway growth. If you see that growth curve, it seems to be sort of leveling off. Is that temporary? Is it going to spike up again next year? I'm going to look into that a little bit more later.

But let's talk about Angular. And I get into a lot of trouble with Java developers, especially when I talk about Angular. Angular is an extremely popular framework. And you never heard me say different. In our survey, about 40% of people said that they used Angular, so about two-thirds as many as use React. Our downloads data says something slightly different. It says about half as many. And there's a bunch of reasons for why that discrepancy could exist in Angular. Users were very, very generous in all of their suggestions about why that might be the case. So I'm not going to say that Angular is getting less popular. I definitely don't have enough data for that. I will when I run the survey again next year. But what I will say is that Angular is seeing fewer downloads. Its downloads peaked in 2017 and have been in slow decline since then.

Ember is an unusual story. Ember was pretty popular in 2015. And then it went through a rough patch in 2016. But in 2017 and 2018, it's making a comeback, which is strange. I've never seen a framework that was in decline, reverse the decline before. But that's exactly what Ember is doing. In January this year, about 4% of npm users reported that they were using Ember. But now it's something like 10%. Ember has done a good job of clawing back market share.

Roughly as popular as Ember right now is Vue. But Vue is a very different growth story. Vue is just in rocket ship mode. Vue is taking off. If I had to guess why React's slowdown is happening, Vue is my number one guess. New users tend to report that Vue is very easy to adapt, very easy to get started with, requires less early decisions than React does. And that seems to be pulling some of the momentum out of React's growth. I don't have a perfect picture. There are a bunch of other reasons that it could be happening. Ember’s research is just also probably, to some degree, responsible for React's slowdown.

But I want to dig a little bit deeper into the React ecosystem, partly because so many of you are using it, and partly because it's unique. React isn't really a framework, as anybody who's using knows. It is a solution to one part of a framework. It is a solution to the Vue problem. It is a way of making components that share state. And there are lots of types of applications that need that. Rich web apps obviously need to be able to do that. But a desktop applications and mobile applications also need to be able to do that. And that's why React has made big inroads into those areas. But rich web apps also have another requirement, which is that they need to be able to map that state to a URL. And to do that, you can't use React, or React doesn't have a solution. So you use React Router instead. Much like React itself, React Router grew quickly and has leveled off. But as you see. it leveled off at roughly half the popularity of React itself. And that's fascinating because it points to the triumph of React.

One of the triumphs of React is that they successfully managed to decouple the functions of this component model from everything else about a framework. Prior to this, all of the frameworks that I was familiar with were soup-to-nuts frameworks. They were like, "We've got a router. We've got a component model. We've got a view model. We've got a data model. We've got everything that you need. You just use our framework and it's done." React went, "We don't actually know what you want to do. But we do have a really great component model and you can use that." And because of that smaller and that more flexible design, React has found its way into a whole bunch of other applications. So React is more than twice as popular as React Router, because there are lots of React apps that don't use React Router. They use other routers. And there are lots of React apps that don't need a router at all because they're a desktop app or a mobile app.

Speaking of React decoupled model, let's look at Flux. Flux was released by Facebook roughly at the same time that React was. It was coupled with GraphQL. Flux was how Facebook expected that you would manage state in large React applications because it was how Facebook did it. But as you can see, that hasn't happened. Imagine if Facebook had released all of this as a single big framework. If you had to use React and Flux at the same time, the unpopularity of Flux would have dragged React down with it. But instead, what happened is that took off. And people use Redux instead to manage state. As you can see, Flux’s down-turned in popularity started the very second that an alternative was available. As soon as Redux was available, it started taking off. And Flux began its decline. Redux and React Router attract each other. They're are very popular combination, so people are nearly always using them at the same time.

Of course, a few weeks ago, Facebook announced an alpha of React that has a new way of managing state called Hooks. Hooks are really interesting because it's possible that they might make Redux redundant. So I'm going to be tracking very closely what happens to Redux in 2019. And the other thing that's interesting about Hooks, if you are not familiar with them yet, is that you can package them up. They're a lot easier to encapsulate into a single package of data. So it's possible to get a bunch of state management functionality from Hooks that you can npm-install, which means that even more of your application can be downloaded from npm. And you can write even less of your application, pushing the percentage up to 98% or 99%.

I mentioned GraphQL earlier. GraphQL started in React land and immediately expanded beyond it. And it is red hot right now. There are two big libraries for using GraphQL, but Apollo is by far the more popular. In fact, there's a whole conference about Apollo GraphQL going on this week. It starts Wednesday. If you know, you're like, "Hey, I haven't done enough conferencing this week. Let me get another ticket." One line in this graph that is there, but I haven't mentioned at all, is RxJS, the bright green line. As you can see, its use is absolutely skyrocketing and has expanded far beyond its react use case. And luckily, Hannah Howard is here to give you a whole talk about RxJS, which is right after this one. So it's in this room right after lunch. And you can get Hannah to explain why RxJS is taking off like that.

So far I've spent a lot of time focused on front end frameworks, because that is what npm users are mostly doing. But 70% of you are also doing back end stuff. So should talk about that, at least a little bit. It's kind of tricky because if you look at what frameworks people are using on the back end, Express just totally dominates. Relative to Express, nothing else even shows up. Everybody's using Express to do something. So let's take Express out of the graph and see what we get instead. In blue is Koa. Koa is a sort of spiritual successor to Express. Koa's relative popularity is pretty flat. But remember, this is share of registry so that flat line means 19,000% growth. So Koa is going really, really well. And it's an example of why share of registries are useful, but also a confusing metric.

Sales, as the name suggests, is a straight up port of Ruby on Rails to JavaScript. And as the graph suggests, that seemed like a good idea at the time. Hapi as a framework that npm used to use on our own website. We switched earlier this year to React. Hapi's relative growth is also pretty much flat. So again, it's growing nicely in absolute terms. And then there's next year's from Next.js from Zeit. If you are interested in React, but you don't want to have to make all of these upfront decisions about how to use React in the router and all of that stuff, Next.js is a soup-to-nuts framework based on React. It's still relatively small. And it's just getting started. But it's growing in registry share. And I personally think it's very interesting.

So how are we all doing so far? I've been showing you a bunch of graphs. And everyone's slowly losing consciousness. So I'm going to do a thing where I split the room up into groups. And it's a little bit tricky because the room is kind of lopsided. But I'm going to say that everybody on this … No, everybody on this side of the line is team A. Everybody on this side of the line is team B. Let me hear it from team A. Team B. Team A again. Team B again. Team A again. Team B again. Excellent. I'm not using it for anything. It's just to make you wake up.

So I've talked a ton about frameworks. And you've all sort of woken up again now. A big part of what people use npm for is their tool chain. Like everything else, we have a bunch of fascinating data about tool chains and how people use them. But the biggest takeaway about our research into how people use npm for tool chains is that people hate their tool chains. They wished there was a lot less tooling. They're really tired of reading config files. Everybody wishes it would just go away.

But if we're going to make it go away, we need to know what it is that we're using in the first place. So 85% of us are using web frameworks. That is interesting because it's compared to the 93% of us who are doing web apps. So there's 7% of us who are just rolling our own, which sounds like a bad idea. Seventy four percent of us are using Transpilers. Sixty nine percent of us are using Linters. Sixty seven percent of us are using Bundlers. And 80% of those are using Webpack. Fifty eight percent of us are using CSS preprocessing of various kinds. And 58% of us are using testing frameworks.

Let's dig a little bit more into some of those. Transpilers or tools that translate JavaScript into other flavors of JavaScript's mostly. The most popular one that a bunch of you already said that you're using, is Babel, which mostly is just translating newer JavaScript into older JavaScript, with the important exception of JSX. JSX is a big part of the React ecosystem. And it's not really JavaScript at all. It's not part of any JavaScript framework. It's not on a standards track. It's just this thing that React decided would be useful. And I'm going to talk a little bit more about that. As you can see, CoffeeScript is clinging on and Elm and ClojureScripts are in there.

But the really big surprise is TypeScript. A bunch of you were still standing when I talked about TypeScript. Forty six percent of npm users report that they are using TypeScript for something. That was very surprising to us when we found that in stat. TypeScript is mostly JavaScript. It just adds types, which are built in testing that helps big teams work together better. Microsoft launched TypeScript with its own package manager. But the community sort of rebelled and said, "We don't want to use two package managers, just use npm." And so they did, which was great and very new Microsoft of them. And they made that change in TypeScript 2 without asking us if that would be okay, which was very old Microsoft of them. Took down the registry.

Linters. As 70% of you know, our tools that tell you, if your code is nice. They will check for obvious errors and flag or even correct coding styles for you. They are super popular. And by far the most popular of them is ESLint. And you may have heard about ESLint earlier this year because it had a major security problem. ESLint, like most open source, is maintained by a group of experienced developers who really know what they're doing when it comes to security. Unfortunately, one of them was being a little bit lazy. He had used a very simple password that he had reused on other websites. And that other website that he used it on got hacked. The password got stolen. And long story short, somebody got that and used it on npm and it worked. And they used that password to publish a new version of ESLint that did bad things. Specifically what it did was it stole your npm credentials.

This could have been a huge problem because it could have been used by the attacker to steal other author's credentials and then publish their packages with the same malicious code, creating this manually-propagated worm that would expand and steal tons and tons of people's credentials. Luckily, that didn't happen because it could have been awful. What actually happened is that people noticed really quickly, because there's 10 million of you. It turns out when there's 10 million developers, the slightest weird thing immediately gets noticed by, like, 100,000 people all of whom email our security group at the same time. So within about 30 minutes of anything happening, people noticed that something was up. And we'd taken down the offending package. And we'd put it into our security database. So anybody who was using npm 6 was getting a big red warning saying, "This is a huge problem. Take it out." So we did that. And then we reset the login tokens of every single npm user, so that even if somebody's login tokens had been stolen, they didn't work anymore.

As far as we can tell, that worked. It's been a couple of months now. As far as we can tell, nobody else had their accounts stolen as a result. But this is why npm is taking security so much more seriously than we used to. We used to hope that it would work and that everyone was nice. But it turns out when you have 10 million people, the tiny percentage of people who are evil is a very large number, a very large absolute number, of people. So we could no longer rely on that. So please use two-factor auth. It is now possible to enforce two factor auth at the package level, so that not only do you have to use two factor auth, but everybody who has published access to that package also has to use two-factor auth.

And let's get back to tooling frameworks. Looking at testing frameworks, a great deal of you are using Mocha. Jasmine is also very popular. Jest is, understandably, popular, given its ties to React and how popular React is. But sitting up there at number three is none. That's not a good idea. Surely, we can do better than that. And that's interesting because that's what the data says will happen. We first noticed this effect when we were building our security tools. And we split people up by how long they had been a developer. We split them up by years of experience. And there was this really clear linear progression, that the longer you had been a developer, they more concerned about security you got.

So we dug a little further. And we discovered that this is true of nearly everything that we would consider a best practice. People just the longer they've been doing JavaScript, the better they get at doing JavaScript. They pick up more and more of the best practices. So bundling and linting and various security steps like code reviews and audits and code scanning, they all get more popular the more experience you have. So that's great because not only can you do better, you will do better. The least experienced group, only about 58% of them, aren't doing anything about security at all. But in the most experienced group, it's 85%. And you can get from one group to the other just by upgrading to npm 6. So you should probably do that.

The Future of JavaScript

So now, we come to the final part of my talk, which is the future of JavaScript, where I make bad predictions., and you take pictures of me standing in front of bad predictions, so you can make fun of me later. The easiest prediction I can make, and the one that is most likely to be true, is that nothing lasts forever. Backbone was once dominant. And it is now an afterthought. jQuery ,which gave web development so much power and flexibility back in the battle days, has been superseded. Any framework or tool that exists today has a lifetime that's going to be measured in a handful of years. So don't cling too tightly to your tools. Don't get stuck in one framework. Don't decide that one thing is the best and fight off the new things.

So if you're looking for a new thing, in 2019 I think the thing that people should learn and get their minds around is GraphQL. GraphQL is about to be huge. There are whole startups built around it. And it has some real advantages. If you are building APIs, it is worth investigating, if GraphQL can help you build them better. It is also definitely time to give TypeScript a try. At 46% adoption, it is not just a fad. It is not just a thing that people at Microsoft use. This has definitely got some real juice behind it. I'm still surprised about TypeScript's adoption. And I've been staring at these numbers all year.

So the biggest prediction of this talk maybe that you people will stop writing JavaScript and start writing TypeScript instead, which immediately raises the question, what happens to npm, if people don't write JavaScript anymore? If people are writing TypeScript instead, what goes on? And the answer is nothing. And the way that I know, is that it already happened, and you didn't notice. First off, 46% of you are writing TypeScript. And also, there's a bunch of other stuff in the registry that isn't JavaScript. There are other languages being transpiled to JavaScript and published to npm. There are a bunch of native modules that get compiled every time you install them, and you didn't notice that either. npm has become the way that you build websites. And it has become decoupled from JavaScript itself. It's just what you use to put a website together, whether or not that code is JavaScript or not.

One of the best examples of that is React. React isn't really written in JavaScript. React is written in a thing that looks very similar to JavaScript, but has some import semantics and other features that aren't really part of JavaScript. And nobody noticed that React is written in an imaginary version of JavaScript. And that's cool because there's an even more exciting development in this npm that isn't really JavaScript world, which is a web assembly or WASM. Web Assembly lets you write in any language and compile it down to byte code that runs in the browser as JavaScript, but at the same speed as native code. And this is very, very exciting because it's a way of taking the entire universe of existing code and bringing it into the web without having to rewrite it in a different language. You can just take your existing libraries and have them available as code inside of your web app.

And one of the most exciting parts of that is that you can already do this. There's a tool called WASM Pack from Mozilla that lets you write code in Rust and compile it down to WASM, which you can then publish to npm and run just like ordinary JavaScripts. This is already happening. In fact, there are some popular modules that have incorporated WASM modules underneath. So it's possible that your stack already includes some Rust code that you're running in a browser and you didn't even notice.

Another very safe prediction for 2019 is that you're going to be doing a whole bunch of bundling and transpiling. Webpack will stay the bundler of choice and Babel is … Sorry, Webpack will stay the bundled choice. Babel is probably going to be the transplant of choice, mostly because of its connection to React.

But this is not great news for JavaScript as a language. This the more future part of my talk. Everybody says that they want less tooling. But now to get almost anything done in JavaScript, you have to set up this big pain in the ass tool chain of a bundler and a Linter in a compiler and a whole bunch of other stuff. And if you have to do that to get anything done anyway, why would you still write JavaScript? Why couldn't you just pick your favorite language and write that instead and compile it down to JavaScript, if you have to compile it down anyway?

As I said, even if everybody stops writing JavaScript, I'm pretty confident that they'll still use npm to do it just because there's this huge volume of already-written code inside of npm that will make people write new solutions that are backwards-compatible. But it's not good news for JavaScript itself. JavaScript and the node project need to get themselves together and make sure that they are reducing the amount of tooling that is necessary to write code for browsers. Otherwise, people are going to start writing code for browsers that isn't JavaScript.

So now I have to get to my biggest prediction, which is what framework should I use in 2019? How do you pick a web framework? I always say that the best web framework is the one that has the most users. Frameworks seldom have overwhelming technical advantages. They often have small ones. But what they do have is a human advantage. If everyone is using a framework, then people will write the tutorials for you. People will find the bugs for you. People will answer your questions on Stack Overflow. And so by that standard, obviously you should pick React. It's huge, you can tell by the amount of time I spent talking in this talk about what you should do, that React is probably where you should go. It is the overwhelming favorite right now. And just its popularity alone is often a good enough reason to pick it. But there are a couple other reasons that are interesting, as well.

I mentioned just now that libraries die after awhile. And I mentioned jQuery. But jQuery is unusual in that it didn't really die. The primary innovation of jQuery was CSS selectors for [inaudible 00:43:25] elements and that just became part of the browser. We never stopped using those. We just stopped having to download jQuery to do it. jQuery sort of transcended and became part of browsers. And that is a very, very interesting pattern.

I've been in this industry a long time, 22 years, like I said. And one of the things that I've noticed is that standards bodies are very, very bad at inventing new APIs. They're very bad at inventing new technology. What they're good at is finding what everybody is doing and then standardizing that long after the fact. That's great. It's a useful function. But they're not good at inventing new things to use. And React is the single most popular framework since jQuery. So it is very worth investigating whether or not some of the concepts in React land are worth transcending into browsers themselves.

A big part of React is JSX syntax. Like I said, it's not really JavaScript, it’s not part of JavaScript, no one intends to make it part of JavaScript. But at 60% adoption, it should probably become part of Javascript now. We should probably stop having to transpile to make JSX work. And whenever I say this, people are like, "Oh, it's got fundamental syntactical incompatibilities with JavaScript." And I'm like, "Fuck you. I remember E4X. Not only can you put html inside of your JavaScript, we did it already." We built this before it was called E4X. It combined XML and JavaScript. And anybody who's familiar with JSX would find it extremely familiar. We basically built it already. It was part of Mozilla. And it got deprecated. But we know that we can do this because we already did this. It just wasn't quite good enough.

And React itself is also a candidate for transcendence. React has invented a very popular way of creating components that are truly reusable. You can npm install React to color. And you get a color picker with a bunch of configuration options that just works out of the box. It doesn't pretend to work like a jQuery plugin. It actually works. And you can just use it out of the box and put it into your application. And that's awesome. That's been the dream of web development for a long time, is to not have to keep reinventing UI wheels all of the time.

So could React style components become part of browsers themselves? I'm not sure that this will happen. And I'm not predicting that it will because we've already seen React slow down, which might've been Vue. It might've been anything else. Web devs are very demanding users. So I can't be sure that React will transcend. But one important factor is that libraries drive usage. Like I said, that's part of what's driving JavaScript. It's all the libraries in npm. If people love libraries enough and people write enough reusable components in React, people will keep using React forever because there's all of this stuff already written in React that they don't want to write again. If people, you know, design a truly reusable, truly popular, a design system in React and make it available open source, it could be that, that would create the critical mass necessary to keep React rolling forever.

Another thing that could make a really big difference is if one of the other major frameworks decided to make React a first-class component. If Vue or Angular said, "Hey, in addition to our own component model, it's fine. You can just use your existing React components inside of our framework," that would be enough impetus to make the browser manufacturers have to sit up and pay attention and go, "Well, maybe this should really be a standard. This isn't just Facebook forcing us to do stuff. This is genuine user adoption."

The problem here is that browser makers already have a component standard, literally called web components. However, even web components' biggest fans say that they are either not ready yet, or that using them is a little bit inconvenient and they tend to use a framework, especially one called Stencil, to get them done. And if you have to use a framework to get your web components done, why would you use any other framework than React? If the web components require framework, you should use the one that's popular, not the one that the browser manufacturers decided that was good because like I said, browser makers and standards bodies, they're not good at coming up with these things off the top of their heads. The thing that was actually invented by web developers is React. And that's the one that we like.

So this raises a really exciting possibility. With amass adoption of a component model that really works and really allows reuse, we could raise the web to an entirely new level of abstraction. Instead of putting together CSS and html, there could be web developers whose only job is to select the React components they need and glue them together into a functioning application. That would be amazing. Some of you might think that it's scary or horrible. Imagine web developers who don't know CSS or html. That might cause a visceral reaction for you. But I'm here to tell you not to be afraid because I've seen this happen before, over and over and over.

People always think that the framework that they use, the technologies that they learned first, those are the fundamentals and that everything else that came after that is not efficient enough, or it's not close to the metal or it's too easy. And I'm here to tell you that that is bullshit. There's no such thing as too easy. If we can make web development significantly easier, we should, because imagine how many more web developers there would be. Imagine how much bigger the web would be, if we could massively simplify how complicated it is to make a web app. Imagine how fast you could work, how many more useful apps would get built that currently nobody has the time or the money to build. The web is the world's most popular and most powerful application platform. And if we made it radically simpler to build on, that would really genuinely change the world.

And no matter what happens, npm is here to stay. And my last and final prediction is that whether JavaScript stays big or we all end up writing web apps in Rust, whether we stay with html, CSS or we go to React components for everything, the huge pile of npm libraries is going to keep us coming back to npm. I've been doing web development for 22 years now. And it is always thrilled me and no less so than right now. It is amazing and wonderful and scary the stuff that we can do with the web. And the tools that we use to make it are ad hoc and a mess and partially broken.

And my final prediction is that’s how it's always going to be. The way that you can tell that the web is still growing, the thing that makes the web exciting, is that it's never done. We're always still adding bits to it. So these changes and these challenges are surmountable. They're scary, but they're fun. I hope you find this as fantastically exciting a time to be a web developer as I do. And thank you all very much for your patience. Thank you.

Woman: Okay. Thank you so much. We have four minutes, right? So we have four minutes for questions that … Laurie will actually be in the AMA room after lunch at 1:40. So you can also ask questions there. If you'd like to ask a question now, just skip the thank you's in the intros and let's get straight to question.

Voss: Everybody wants to go to lunch.

Woman: Or go to lunch. By all means, please do.

Man: Yeah. Thanks. I'd really just maybe 30 seconds, 60 seconds, on Internet of Things. Any trends you're observing. You didn't have time in that talk to address it so far.

Voss: It's not really my specialty, but I do know that a JavaScript STKs have become sort of the de facto for Internet of Things. Everybody who releases a device picks the language that everybody else wrote their STK in. And that's just become sort of the JavaScript place. So people who are big into IoT say that npm is indispensable to use IoT stuff. But it's interesting because they're mostly distributing like Lula or like C, or something that actually runs on the device, like not really distributing JavaScript at all. But they're very much using npm to do it. Other than that, I don't really have a whole bunch of experience with IoT stuff. Anybody else not hungry enough to ask a question.

Woman: Next question.

Man 2: So switching from YARN back to using npm as a package manager, I saw the link that you shared. It mentions that transitive dependencies change quite a bit. So is there anything that we could do, if we want to switch back over to make that process slightly easier?

Voss: There is, but not in a way that I could tell you in a 15-second answer. And also, the people who are doing that are not me. So I would connect you to those people. But you should definitely drop me a DM or an email and I can connect you to those people.

See more presentations with transcripts

 

Recorded at:

Dec 12, 2018

BT