Go 1.21 toolchain is the first Go toolchain to be perfectly reproducible. This makes it possible to reduce the risk that a malicious actor can tamper with the output binaries, explains Google engineer Russ Cox, to carry through a supply chain attack.
A reproducible build generates the same outputs every time it starts from the same sources and inputs. This makes it possible for anyone to verify that any posted binary is free of malicious changes by re-creating the build and comparing the output with the posted binary.
That approach proves the binaries have no backdoors or other changes not present in the source code, without having to disassemble or look inside them at all. Since anyone can verify the binaries, independent groups can easily detect and report supply chain attacks.
Lack of reproducibility means that a malicious actor could modify the generated binaries in a downstream computer, which is usually easier than modifying the upstream source code, without those changes being detectable by its original authors or users alike. The momentum that reproducibility is gaining in the open source community is also attested by the new Reproducible Builds initiative.
This is particularly a concern for developers collaborating on privacy or security software: attacking these typically result in compromising particularly politically-sensitive targets such as dissidents, journalists and whistleblowers, as well as anyone wishing to communicate securely under a repressive regime.
As Cox explains, older Go toolchains enabled reproducible builds, but only with a significant effort. Go 1.21 has introduced a number of changes that make this process seamless by eliminating any relevant inputs, that is, inputs to the build process that affect its output, except for the source code itself.
Common relevant inputs that end up being detrimental to reproducibility include the specific versions of the source code or any dependencies, the OS or CPU architecture, the compiler version, the user/group running the build, and so on. According to Cox, while documenting any relevant inputs does indeed provide a way to make builds reproducible, the only way to make them easily reproducible, aka perfectly reproducible, is by getting rid of any relevant input besides the source code proper.
The road to have reproducible Go toolchain started in Go 1.10, where a number of key decisions where taken, including reducing randomness when iterating over maps or executing goroutines and copying libraries used by the compiler into the source tree, such as sort and compress/zlib to avoid that updating any of those libraries on the build system could produce different binaries.
Go 1.20 and 1.21 added the remaining building blocks towards achieving perfect reproducibility. Those included rewriting the net package in Go to avoid having to depend on the build host C toolchain, improving the portability of binaries generated by Go linker, using the module path instead of the source directory path for all internal paths in the runtime and debugging metadata, making floating point code generation uniform across platforms, and more.
With all these changes in place, says Cox, the Go toolchain is perfectly reproducible. As a matter of fact, Google builds all Go distributions on two very different systems, a trusted Linux/x86-64 host and a Windows/x86-64 host, and verifies they produce bit-for-bit identical binaries or else scrap the build.
For other developers to carry through the same verification, Google has published a verifier, which will rebuild the current Go version, checking it matches the official archives.
Cox's article includes far more technical information than can be covered here, so do not miss it if you are interested in the full details.