Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage News Gremlin Adds Automated Service Discovery for Targeting Chaos Experiments

Gremlin Adds Automated Service Discovery for Targeting Chaos Experiments

This item in japanese

Gremlin, a chaos engineering platform, recently announced automated service discovery. This new feature will auto discover services running within dynamic environments. These services are then available to target for chaos experiments. Gremlin has also added role based access control for their API keys.

The new service discovery feature simplifies locating services to target for chaos experiments. The Gremlin agent can auto-detect processes running within Kubernetes clusters, containers, or on hosts. When a process is identified, operational metadata is also processed by the agent. This includes active UDP and TCP sockets, process names and IDs, the path to the process executables, and the command line arguments.

Gremlin service details interface

Gremlin service details interface (credit: Gremlin)


The service discovery page allows for selecting services to target for attacks. Gremlin will auto select all systems that the desired service is running on. The experiment can then be tailored by isolating and targeting a subset of the systems.

The interface also shows a history of experiments performed by service including month-over-month activity. This view can help identify gaps in experimental coverage and which services may need testing. It is also possible to re-run past experiments from this view.

Gremlin service discovery page showing history of chaos experiments

Gremlin service discovery page showing history of chaos experiments (credit: Gremlin)


In addition to the auto-collected metadata, it is possible to input additional information into the service details page. This includes adding a description for the service, identifying the owner, and linking out to service dashboards and incident response runbooks.

The Gremlin agent will discover Kubernetes based services automatically with no additional configuration. For host and container based services, there is additional configuration required. Once configured, new services and changes in existing service data will be discovered hourly. New pods, containers, and hosts within already identified services will be registered within 30 seconds.

Also recently released is role based access control (RBAC) for API keys. With this change, all new API keys will be created at the user level, sharing the same permissions as the user who created it. The API is also enhanced with new routes for adding and removing users, managing client certificates and keys, and generating reports.

This addition sees the deprecation of team-level API keys. Team-level keys will continue to work until June 15, 2021, at which point only the new user API keys will be supported. Existing keys can be migrated to user-level API keys through the Gremlin application.

Service discovery is available free for all Gremlin users. More information can be found on the Gremlin blog.

Rate this Article