BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Tensorflow with Javascript Brings Deep Learning to the Browser

Tensorflow with Javascript Brings Deep Learning to the Browser

This item in japanese

GDPR for Software Engineers

This eMag examines what software engineers, data engineers, and operations teams need to know about GDPR, along with the implications it has on data collection, storage and use for any organization dealing with customer data in the EU. Download Now.

At the recent TensorFlow Dev Summit 2018, Google announced the release of Tensorflow.js, a Javascript implementation of Tensorflow, its open-source deep-learning framework. Tensorflow.js allows training models directly in the browser by leveraging the WebGL JavaScript API for faster computations.

Tenforflow.js is an evolution of deeplearn.js, a Javascript library released by Google in August 2017. Deeplearn.js was born out of the success of the Tensorflow Playground, an interactive visualization of neural networks written in TypeScript.

Tensorflow.js has four layers: The WebGL API for GPU-supported numerical operations, the web browser for user interactions, and two APIs: Core and Layers. The low-level Core API corresponds to the former deeplearn.js library. It provides hardware-accelerated linear algebra operations and an eager API for automatic differentiation. The higher-level Layers API is used to build machine-learning models on top of Core. The Layers API is modeled after Keras and implements similar functionality. It also allows to import models previously trained in python with Keras or TensorFlow SavedModels and use it for inference or transfer learning in the browser.

With Tensorflow.js, machine-learning models can be utilized in the browser in three ways: by importing already pre-trained models and using them for inference only, by training models from scratch directly in the browser, or by using transfer learning to first adapt imported models to the user's context and then use these improved models for inference.

As Nikhil Thorat and Daniel Smilkov, members of the Tensorflow team, point out in their announcement video, running Tensorflow in the browser has several advantages: the infrastructure and set of requirements are simplified as the need for background API requests is removed; the available data is richer in nature thanks to newly accessible sensors, such as webcam and microphone on computers or GPS and gyroscope on mobile devices; and the data also remains on the client side which addresses privacy concerns. 

Javascript based deep-learning also lowers the barrier of entry into the field of data science for many web developers. It enables them to integrate machine learning functionality into their applications while bringing an innovative approach to the domain. Online demos show the improvements in terms of speed and quality of interactions which result from running machine learning directly in the browser: from sentiment analysis to hand gesture detection or style transfer.

Similar Javascript implementations of deep learning frameworks have existed for a while with the precursor brain.js project and the ConvNet.js library from Standford for convoluted neural networks. More recent libraries include KerasJS and TensorFire. However, these libraries either lack the ability to leverage the browser WebGL components for GPU based computations, or do not allow training the models directly in the browser. Propel for scientific computing and ml5 a for machine learning are more recent projects based on Tensorflow.js, which do not suffer from these shortcomings.

The Tensorflow team is currently working on making TensorFlow.js support Node.js, but no timeframe has been given yet. Several other significant Tensorflow developments were announced during the TensorFlow Dev Summit including TensorFlow for Swift, due for release in April 2018 and TensorFlow Hub which "provides a repository for sharing different pre-built modules developers can reuse across multiple models".

Rate this Article

Adoption
Style

BT