BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Go 1.13 Brings Centralized Module Mirror Proxy and Checksum Database

Go 1.13 Brings Centralized Module Mirror Proxy and Checksum Database

This item in japanese

Bookmarks

While leaving the language mostly unchanged, Go 1.13 brings a number of improvements to the toolchain, runtime, and libraries, including a relatively controversial default Go module proxy and Checksum database that requires developers to accept a specific Google privacy policy.

At the language level, Go 1.13 supports a more uniform set of prefixes for numeric literals, such as 0b for binary numbers, 0x for hexadecimal numbers, 0o for octals, and so on. The new version also enables the use of signed shift counts with operators << and >>, thus removing the need for uint type casting when using them. Both changes do not break Go's promise of source compatibility with previous versions of the language.

More extensive are the improvements to the toolchain, including: some polish for go get so it works more consistently in both module-aware and GOPATH mode; support for version suffix @patch when getting a package; additional validation on the requested version string; improved analysis of which variables and expressions could be allocated on the stack instead of heap; and more.

In particular, the Go module system now supports out-of-the-box the module mirror and checksum database Google recently launched. The module mirror is aimed to improve go get performance by implementing a more efficient fetching mechanism in comparison with that of what version management systems such as Git provide, and effectively acts as a proxy to the package sources. For example, the module mirror does not require downloading the full commit history of a repository. In addition, it provides an API more suited to package management, which includes operations to get a list of versions of a package, information about a package version, a mode and zip file for a given version, and so on.

The checksum database also aims to ensure your packages are trusted. Prior to its introduction, the Go package manager used to ensure authenticity of a package using its go.sum file, which basically contains a list of SHA-256 hashes. With this approach, the authenticity of a package is established on its first download, then checked against for each subsequent download. The checksum database is a single source of truth which can be used to ensure a package is legit, even the first time it is downloaded by comparing its go.sum with the information stored in the database.

This checksum database allows the go command to safely use an otherwise untrusted proxy. Because there is an auditable security layer sitting on top of it, a proxy or origin server can’t intentionally, arbitrarily, or accidentally start giving you the wrong code without getting caught.

The announcement of the new module mirror and checksum database sparked a reaction from developers worried about Google being able to track what packages developers use if they do not modify go get default configuration:

This is the first language in my 23 years of programming that reports my usage back to the language authors. It's unprecedented.

Another complaint concerns the requirement to accept a specific Google privacy policy which could change at any time in future. It should be noted that developers can disable the go get default behaviour by setting the following environment variables:

go env -w GOPROXY=direct
go env -w GOSUMDB=off

This is considered not robust enough, though, due to the possibility of those variables not being set by mistake, e.g., on a new machine, or being inadvertently cleared, e.g. when using su -.

While the new module mirror and checksum database undoubtely improve Go module system performance and security, there is a chance that their acceptance may not be very high among privacy-conscious developers until the Google Go team reviews its approach by making the usage `off` by default, or by providing a more robust way to disable them, especially in enterprise environments, where accepting the Google privacy policy could entail some complication. InfoQ will keep reporting on this as new information becomes available.

Rate this Article

Adoption
Style

BT