BT

Mercurial 2.0 Released with LargeFiles extension

by Alex Blewitt on Nov 03, 2011 |

Selenic have released Mercurial 2.0, their trianual upgrade to their namesake DVCS tool.

This release brings the Largefiles Extension, a way of bringing centralised data storage to an otherwise distributed version control system. As the Mercurial Revlog storage mechanism is not well suited to the storage of large binary files, the Largefiles Extension provides a centralised server location where such files can be downloaded on an as-needed basis.

Storing large binary files in any DVCS can sometimes be a problem if those files are updated frequently; even if the binary file can be stored using delta compression (i.e. just storing the changed parts) large files can quickly inflate the size of the repository. In addition, if these resources form part of the repository's history then they can't be pruned from the repository without affecting the content hashes, and therefore the repository's version.

Depending on the version system used, whether you store large assets in different branches (i.e. ones that you don't fetch regularly) can affect the size of the cloned repository (but not the source). Mercurial largefiles takes a different route, and provides (in effect) a symlink to the large file in the cloned repository.

Providing that the file is not needed in the currently checked out version, it is not downloaded as part of the clone or in fetch/push operations. However, if a revision is checked out which contains one or more of these largefile links, it will perform a separate connection to download the content from the server. Clearly this implies that there needs to be connectivity to check out some versions of the repository, but the local Mercurial clone will keep a cache of previously downloaded largefiles and can supply them if they've been downloaded already.

A new option to the hg add command has been added, --large, which allows a file to be specified as large (and therefore taking this alternative route to centralised-but-cached-on-demand storage). Alternatively, a fixed size can be set (those over 10Mb by default) or a naming pattern (e.g. *.zip) can be used to automatically flag large files. Note that the repository format must be upgraded with the hg lfconvert command to take advantage of this new format. Note that the dirstate command, which tracks the state of largefiles, is currently limited to 2Gb in Mercurial 2.0, although there are hopes for a fix post the 2.0 release.

Mercurial 2.0 also brings in the graft command, which implements cherry picking for Mercurial. It uses the merge logic to determine what changes should be present, and then applies them on a piecemeal basis but without creating a merge node. As with other implementations of cherry-picking, if the change has already been copied in the past then it will not be duplicated.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT