Thursday, October 27, 2011

The move to Mercurial

oyatsukai recently migrated all code from the old Subversion repository to Mercurial, and there has been a little bit of interest about what we did and why / how we did it.

Why move?
In 2011 this probably doesn't even need an explanation.  Subversion still doesn't handle branching in a sensible way and distributed version control is just so much more flexible.

With the core engine technology evolving quickly to handle more game titles, we were aware we needed to start using branches in some form, if only to maintain releases and keep them shielded from changes in the shared libraries.  I was using git-svn with our old svn repo, and keeping local release branches of Final Freeway for android.  It was time to start doing this across other games and platforms, keeping everything on the server where it should be.

Why Mercurial (hg)?
As a pure revision control system for code, my first choice would have been git.  Partly because I already know it, but mainly because it's just better.  It's true that, barring some subtle differences like how branches are handled, under he hood the two systems seem to be very similar and what you can do with one generally seems possible with the other.  But what stands out for me is that with git the default commands do exactly what you want in the common case.  git pull does what I generally want to do when I sync from the server.  On the other hand, hg pull will usually leave you in a state you don't want to be in unless you add the correct combination of flags.  It can't be good design that the most common operations tend to require extra parameters.  However, it was clear that either git or hg would be a better solution than svn.

Of course, it wasn't just a simple case of choosing the system we personally prefer.  Since oyatukai is a game developer there are a lot of binary assets to deal with.  This complicates things because neither git nor hg are particularly good with large binaries.  We also didn't want artists to have to deal with anything more complex than SVN (which can already be a burden for them sometimes) and the hg and git GUI interfaces are not that great.

The feature of hg that made us choose it was the ability to reference external subversion repositories.  Git has 'submodules' for pulling in external repositories, but they have to be git repositories.  But the hg subrepos feature is just excellent, allowing you to include hg, svn or git repositories.

Final Solution
So we went with hg on the basis that, like git, it gives us proper support for branching, merging, rebasing, etc and also allows us to keep art in an svn repository.

Since svn can sync sub-sections of a repository, we have always given artists a svn url that only includes the stuff they need - the art data and the tools to edit and preview their changes.  We use hg subrepos to pull this same url into the source tree for developers, and as far as the artists are concerned nothing changed.  Not having to migrate them over to another system really saved some headaches.

A side-effect of using this system is that there is a bit of separation between art data and code.  A given hg revision includes the version number of the assets in svn, and when there are art changes someone has to manually tell hg to reference a new revision.  However, this means that we can adopt new art into the development repo at whatever frequency we like.  Until we are happy with the state of the art repo, the development line keeps referencing a known good version of the asset data.  Similarly, when we make changes to the art tools we can 'release' them to the artists when they are ready by just submitting them to the svn subrepo, and the artists don't have to be concerned about any intermediate code changes.

I think overall it's working well.  Moving from svn to hg required learning a new interface, but with several titles in development and more recently projects that involve sharing sections of code with other developers, we are making good use of the flexibility of the system.  It was certainly worth the effort.