BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News GitLab Protocol Fuzzer CE Now Open-Source

GitLab Protocol Fuzzer CE Now Open-Source

This item in japanese

Bookmarks

GitLab has open-sourced the core protocol fuzz testing engine it has been using since its 13.4 release. Fuzz testing aims to more effectively find security issues and flaws in business logic by passing randomly generated inputs to an app.

GitLab introduced Fuzz Testing at the end of last year, on the heels of its acquisition of Peach Tech and Fuzz it.

GitLab’s API fuzz testing lets you provide an OpenAPI v2 specification or a HAR file of your application, and then automatically generates random inputs designed to exercise edge cases and find bugs. Results are then immediately shown as part of the pipeline.

Protocol fuzz testing extends fuzzing capabilities already built-in into GitLab by making it possible to fuzz test network protocols or file formats.

InfoQ has spoken with GitLab principal product manager Sam Kerr to learn more.

InfoQ: Could you briefly explain what fuzz testing is and why it is getting interest from many software companies?

Sam Kerr: Fuzz testing is about finding bugs and vulnerabilities that other techniques can miss. It does this by generating random inputs to cause crashes and find parts of the app that may not have been tested previously. Since fuzz testing is using randomly generated inputs, rather than predetermined signatures or patterns to look for, it will find issues that are unique to the business logic of the app that could never have a signature written to check for.

Many companies are getting more and more interested in fuzz testing as a great way to complement the existing testing and security scanning they already do. Other techniques can be used to find certain classes of issues and fuzz testing can find others. It is complementary to, rather than a replacement of, other testing and security methods.

There are several types of fuzz testing which can be used depending on the app type and use case. GitLab has two different types of fuzz testing available inside our product today, coverage-guided fuzz testing and API fuzz testing. These are for “inside-out” and “outside-in” testing, respectively, of the application and its code. This new open source repository brings protocol fuzz testing to the community and allows for testing low-level network protocols, such as TCP, or file formats, such as JPG.

InfoQ: What challenges does fuzz testing pose to organizations that wish to integrate it into their CI pipelines?

Kerr: One of the main challenges with fuzz testing is perceived difficulty of use and setup. At GitLab, one of our key focus areas is on usability, to ensure users can be successful and find bugs and vulnerabilities easily without becoming a security expert. We plan to take the same approach with protocol fuzz testing going forward.

Another step organizations will need to take with fuzz testing is determining how long to run fuzz testing, to balance between being responsive to developers and giving the fuzzers enough time to find issues. We typically see users run short fuzz jobs on every commit of their code and then also have a long-running fuzz job against a given release of the app. This gives a good balance between not slowing down developers while also finding as many issues as possible.

InfoQ: Who is GitLab Protocol Fuzzer Community Edition aimed at and how can it help them adopt fuzz testing?

Kerr: GitLab Protocol Fuzzer Community Edition is open source and everyone can contribute. This release provides a lot of the core engine pieces that GitLab acquired from PeachTech to allow for users to build support for their own protocols.

We anticipate security researchers and users already familiar with fuzz testing to use the repository to build protocols as well as contribute any features they would like to see in the core engine.

If users have used the older Peach Fuzzer Community Edition, we expect them to be able to quickly get started with the new GitLab Protocol Fuzzer Community Edition.

Over time, we plan to offer support for some pre-built protocols to allow users less familiar with fuzz testing to get started more quickly.

InfoQ: How does GitLab Protocol Fuzzer Community Edition differ from your other Fuzz Testing offering?

Kerr: GitLab offers coverage-guided fuzz testing and API fuzz testing today, which are for slightly different use cases than protocol fuzz testing.

Coverage-guided fuzz testing is about putting a small “test harness” around either a whole app or a part of an app to test. The fuzz harness can monitor the app while it runs and intelligently make changes to the test to find results. This can be used for any web, desktop, or server app written in one of our many supported languages.

API fuzz testing is about using a definition of an API or a recording of interacting with an API to tell the fuzzer what types of info the app is expecting. This could be either an OpenAPI specification, HAR recording, or Postman collection. The API fuzzer is smart enough to take those inputs and generate random values to find bugs in the API. This is especially useful for using with an app that exposes an API over HTTP.

Protocol fuzz testing is different because it is about testing low-level network protocols or file formats that an app might use. The protocol fuzzer is given a definition of what the protocol is and then intelligently makes changes to valid protocol messages to try and find bugs in the app’s implementation of that protocol.

For example, if a protocol message was supposed to include a length value, the fuzzer may provide an incorrect length value to see if the app can handle it appropriately. This is especially important for embedded devices or low-level software that may be implementing their own versions of protocols, rather than using a version provided by an operating system.

Another difference is that GitLab Protocol Fuzzer Community Edition is an open source project hosted and maintained by GitLab, but is not yet integrated into GitLab the product. This means users can look at the source code, make changes, run it locally, and add it to their own pipelines. Coverage-guided fuzz testing and API fuzz testing are both part of GitLab the product today and are deeply integrated. That means their results will appear in the Security Dashboard and developer merge requests.

In the future, we have plans to extend GitLab Protocol Fuzzer Community Edition and integrate it into GitLab to provide a similar experience for those users.

The latest version of GitLab, GitLab 13.10, introduced a number of improvements to fuzz testing, including support for OpenAPI v3 and a new UI to make it easier to modify fuzz testing settings.

Rate this Article

Adoption
Style

BT