Open Sourcing Artificial Intelligence Research

| by Jonathan Allen Follow 612 Followers on Apr 27, 2016. Estimated reading time: 2 minutes |

As with many companies over the last couple of years, InfoSys is seeing a major shift in away from “big data” to more of an emphasis on machine learning an AI research. But unlike their competitors, which are heavily investing in proprietary solutions such as Microsoft’s Azure Machine Learning Studio, InfoSys decided a cooperative approach would be more efficient.

The result of this decision is OpenAI, a non-profit artificial intelligence research company. Officially launched in December, this research group has a billion dollars in funding from InfoSys, Amazon Web Services, and several private donors.

The reason we’re talking about OpenAI today is they just released the public beta of OpenAI Gym. This toolkit is used to develop and compare reinforcement learning (RL) algorithms, a cornerstone of modern machine learning research. The announcement cites two main reasons they are focusing on reinforcement learning algorithms,

RL is very general, encompassing all problems that involve making a sequence of decisions: for example, controlling a robot's motors so that it's able to run and jump, making business decisions like pricing and inventory management, or playing video games and board games. RL can even be applied to supervised learning problems with sequential or structured outputs.

RL algorithms have started to achieve good results in many difficult environments. RL has a long history, but until recent advances in deep learning, it required lots of problem-specific engineering. DeepMind's Atari results, BRETT from Pieter Abbeel's group, and AlphaGo all used deep RL algorithms which did not make too many assumptions about their environment, and thus can be applied in other settings.

Currently RL research is hampered the need for better benchmarks and the “lack of standardization of environments used in publications”. As you can imagine, it is hard to reproduce another scientist’s results when their research paper presumes that you have access to a proprietary set of tools. Or worse, an internally built toolkit that isn’t available for any price.

An important aspect of machine learning is the having an experimental environment to work in. Not only is there a significant development cost in creating an experimental environment, one can’t meaningfully compare two algorithms unless they share a common environment. So out of the box, OpenAI Gym offers these environments: Classic control, Toy text, Algorithmic, Atari (based on the Arcade Learning Environment), Board games, and 2D/3D robots. (The last one requires a MuJoCo physics engine license.)

OpenAI Gym currently supports Python 2.7 on Linux and OSX. If there is sufficient interest, Python 3 and Windows will also be considered. The code is offered under the MIT license.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you