Tag Archives: darcs

hg.complete.org is no more

Programming, , , ,

As of today, hg.complete.org is no more. I have removed mercurial and hgwebdir from my server, removed hg from my DNS zone, and converted everything that was in Mercurial over to Git. (Except for hg-buildpackage, which I have orphaned) So there is now stuff at git.complete.org.

I still have a ton of Darcs repos to convert, which will take more time.

Also I have heard a lot of people say that the GitPlugin for Trac is not very good. I have two Trac instances running it: one for commithooks and another for ListLike. Both seem OK so far, but I haven’t pushed them very much yet.

bzr, again

Programming, , ,

I’ve talked a lot lately about different VCSs.

I got some interesting comments in reply to my most recent post. One person took issue with my complaint that nobody really understood how to specify a revision to git format-patch, and proceeded to issue an incorrect suggestion. And a couple of people complained about my comments about bzr, which generally came down to the released version of bzr didn’t have anything compelling and also didn’t support tags.

So I went into , asked them what bzr has that git, Mercurial, and darcs don’t. And gave bzr the benefit of the doubt that 0.15 will be out soon and will be stable. What I got back were these general items:

  1. Renaming of directories (not in hg, git)
  2. 2-way sync with Subversion (not in hg, darcs)
  3. Checkouts (not in any others by default)
  4. No server-side push requirement

Let’s look at these in more detail.

1. Renaming of directories

All of them can rename files and (excepting git) completely accurately track back the file’s history. But consider this: if person A commits a change to branch A that adds a file, and person B then renames the directory that the file is in on his branch, will a merge cause person A’s file to appear in the new directory name? In darcs and bzr, yes. In Mercurial and git, no.

So yes, this is a nice thing. But I have never actually had this situation crop up in practice, and even if it did, could be trivially remedied. I would say that for me, I don’t really care.

[Update: Current stable releases of Mercurial can do this too. I’m not quite sure how, but it does work. So git is the only one that can’t do that.]

2. 2-way sync with Subversion

This is a really nice feature and is present in both git and bzr. I haven’t tested it either place, but if it works as advertised — and properly supports tracking multiple related svn branches and merges — would be slick. That was enough to make me consider using git, but in retrospect, I so rarely interact with people using svn that it is not that big a deal to me.

Still, for those that have to work with svn users, this feature in bzr and git could be a big one.

Better yet would be to get all those svn holdouts over to DVCS.

3. Checkouts

A bzr checkout is basically a way to make local commits be pushed to the remote repo immediately, as with svn. This is of no utility to me, though I can see some may have a use for it. But it can be done with hg hooks and probably approximated with scripting in others.

4. No server-side process necessary for pushing repos

bzr has built-in support to push to a server that has sftp only, and doesn’t require a copy of itself on the server. While I believe that none of the other three have that, it is possible to rsync (and probably ftp) darcs and Mercurial repos to a server in a safe fashion by moving repo files in a defined order. Probably also possible with git. All four can pull repos using nothing but regular HTTP.

What bzr still doesn’t have

Integrated patch emailing. The big thing is that it has no built-in emailing of patches support. darcs is extremely strong in this area, followed by hg, and git is probably third. “darcs send” is all it takes to have darcs look at the remote repo, figure out what you have that they don’t, and e-mail a bundle of changesets to them. I posted an extension and later a patchset that does all this for Mercurial except for automatically figuring out what default email address to do (that’ll come in a few days, I think). One feature Mercurial has had for awhile that Darcs hasn’t is sending multiple textual diffs as a thread, with one message per changeset. bzr doesn’t have any support for emailing patches yet, which is disappointing. Because of the strong support for this in darcs and Mercurial, people running those systems feel less of a need to publish their repos.

[Update: There is a plugin for bzr that seems to address some of this. I haven’t tested it, and it’s not in bzr core (so doesn’t quite meet my very friendly for a newbie requirement), but this does exist, though apparently not as advanced as Mercurial]

Performance. Supposedly 0.15 is supposed to be better on this, but even if bzr achieves the claimed doubling of performance, most benchmarks I have seen would rate it as still being significantly behind git and Mercurial, though it may overtake darcs in some tests.

Extensive documentation. I would say that bzr’s docs are better in some ways than git’s (its tutorials especially), but lack depth. If you want to know some detail about how the repository works on-disk, it’s not really documented. Darcs still has David’s excellent manual, and Mercurial has the hg book which is still great as well.

Merging not as advanced. darcs is pretty obviously way on top here, but of the others, Mercurial does a pretty good job with its automatic handling of renames and automatic resolving of different branches that commit the same change (even if that same change is a rename, or an add of the same content). bzr can’t resolve as much automatically.

Summary

Well, I’ll say that bzr still doesn’t look compelling enough for my use cases to use, and the lack of an easy-for-a-newbie-to-use automated email submission feature is a pretty big disappointment. Though I did appreciate the time those on spent with me, and if I needed to sync with svn users frequently, I’d probably choose bzr over git.

For now, I’m happy with sticking with darcs for my code and hg for my Debian work.

But all four communities are aggressively working on their weaknesses, and this landscape may look very different in a year.

More on Git, Mercurial, and Bzr

Programming, , ,

I’ve been writing a lot about this lately, I know, but it’s an interesting landscape.

I had previously discarded git, but in light of git-cvsserver (which provides a plausible way for Windows people to participate), I gave it a try.

The first thing I noticed is that git documentation, in general, is really poor. Some tutorials that claim to cover git actually cover cogito. Still others use commands that are much more complex than those in the current git — and these just the ones linked to from the git homepage.

git’s manpages aren’t much better. There are quite a few git commands (such as log) that take arguments that other git commands accept. Sometimes this fact is documented with a pointer to these other commands, but often not; a person is left guessing what the full range of accepted arguments are.

My complaint that git is overly complex still exists. They’ve made progress, but still have a serious issue here. Part is because of the docuemtnation, and part is because of the interface. I wanted to export to diffs all patches on the current branch in a repo. I asked on , and someone suggested using the revision specifier ..HEAD. Nope, didn’t work. A few other git experts chimed in, and none could come up with the correct recipe. I finally used -500, which worked but is hackish.

git’s lack of even offering support for a human to indicate renames also bothers me, though trustworthy people have assured me that it doesn’t generally cause a problem in practice.

git does have nicer intra-repo branching than Mercurial does, for the moment. But the Mercurial folks are working on that anyway, and branching to new directories still works fine for me.

But in general, git’s philosophy is to make things easy for the upstream maintainer, and doesn’t spend much effort making things easy for contributors (except to make it mildly easier to contribute to a large project like Linux). Most of my software doesn’t have a large developer community, and I want to make it as easy as possible for new developers to join in and participate. git still utterly fails on that.

I tried bzr again. It seems that every time I try it, after just a few minutes, I am repulsed. This time, I stopped when I realized that bzr doesn’t support tags and has no support for emailing changesets whatsoever. As someone that has really liked darcs send (and even used tags way back with CVS!), this is alarming. The tutorial on the bzr website referenced a command “bzr help topics”, which does not work.

So I’ll stick with my mercurial and darcs combination for now.

I announced the first version of a hg send extension yesterday as well. I think Mercurial is very close to having a working equivalent to darcs send.

Mercurial & Git

General, ,

About two weeks ago, I wrote about my thoughts on Mercurial and how I was switching to it from Darcs.

At the time, I had skipped Git because of its lack of Windows support. I have some contributors to pieces of Free Software that I write that use Windows, and that seemed a pretty big flaw.

But I recently discovered git-svn and git-svnimport, both of which look like great tools for working with our friends using svn that haven’t yet gotten ahold of the DVCS light. Then I noticed that Git has a CVS server emulation tool, which means that Windows users can use TortoiseCVS to interact with it. Nice.

I spent some time today learning Git. This was a lot easier having already learned Mercurial. Git and Mercurial have very similar philosophies to a number of things, but the Mercurial documentation explains all this far better than the Git documentation does.

I’m going to have to try both of them out more and see what I think. But git-svn (which is bi-directional) certainly looks like a very nice thing.

Neither of them have something as nice as darcs send, though.

Re-Examining Darcs & Mercurial

Programming,

I recently wrote an article or two about distributed version control systems.

I’ve been using Darcs since 2005. I switched to Darcs, in fact, 10 days after the simultaneous founding announcements of git and Mercurial.

Overall, I have been happy. I continue to believe that it is the most distributed of the distributed VCSs, which is a Good Thing.

However, I have lately started having trouble with Darcs hanging while working on my Debian packages. My post to the Darcs user list drew out a few other people whith this problem, which is a design flaw of Darcs.

So I revisited the VCS landscape. I re-examined git, Mercurial, and bzr. I eventually decided to give Mercurial a try. I avoided git because I write some code that is portable to Windows, and git isn’t (or isn’t very well). Also, git is complex to pick up for me, and I certainly don’t want to force something complex onto my contributors. bzr seemed to still have some strange behaviors that it’s had for awhile, and I couldn’t find even one advantage of it over Mercurial. So off I went with Mercurial.

I quickly learned a bit of a philosophical difference from Darcs to Mercurial.

Darcs avoids conflicts at all costs. Mercurial makes handling conflict easy and, in many cases, automatic.

It is exactly this Darcs behavior that permits both is excellent “darcs send” feature (still unmatched in any other VCS), but also causes its hang problems.

I found Mercurial quite pleasant to work with, and *fast*. It seems to be edging out git in speed tests sometimes these days.

It is easy to get started with Mercurial. The mq system — similar to quilt or other patch-management programs — is really quite an amazing hybrid between patch management and version control. I frankly don’t see any need for other patch-management tools anymore.

Mercurial has a “patchbomb” feature where you can select a range of changesets to send off, and it will generate nice emails with one changeset per email, and send them to your selected destination, optionally with an introductory message. The normal way of interacting with other Mercurial users is via the hg export/import commands, which send around simple unified diffs plus some additional header information, optionally in the git extended diff format.

I am happy with Mercurial and am in the process of converting my Debian repositories from Darcs to Mercurial. I’m going to keep my personal code in Darcs for the moment because “darcs send” is still easier than “hg email”, but that may change before long, depending on how my experience goes.

I’d encourage others to give Mercurial a try. The community is also very nice and helpful.

I have contributed patches to Tailor to make it make exact copies of Darcs repos into Mercurial, which are now in its Darcs repo. There is also a thread on the Mercurial list with some of my initial questions/concerns coming from a Darcs perspective.

Whose Distributed VCS Is The Most Distributed?

Software,

Lately I have been trying out a number of distributed version control systems (VCS or SCM).

One of my tests was a real problem: I wanted to track the Linux 2.6.16.x kernel tree, apply the Xen patches to it, and pull only specific patches (for the qla2xxx driver) from 2.6.17.x into this local branch. I wanted also to be able to upgrade to 2.6.17.x later (once Xen supports it) and have the version control system properly track which patches I already have.

But before going on, let’s establish what it means to be an ideal distributed VCS:

  • 1. The fundamental method of collaboration must be a branch. A checkout should mean creating a local branch on which a person can commit and work without having to involve the server. An update from some central server should take the form of a merge from that branch to the local branch.
  • 2. Branching should be cheap. It should be easy to create a local branch, the operation to do so should be fast, and it shouldn’t take an inordinate amount of space. It should also be as easy as possible to branch from a remote repository.
  • 3. Merging between branches is intelligent. It should be easy to merge another branch with your own. The VCS should know which changesets from the other branch are already on yours, and should not attempt to merge changesets that you have already merged previously.
  • 4. Inividial changesets should be mergeable without bringing across the whole history. You should be able to bring across the minimum number of changesets necessary to effect a specific change. This corresponds to my test case above. Future merges from the whole branch should, of course, recognize that these changesets are present already.
  • 5. Branching preserves full history. A branch should be a first-class copy of a repository, even if the repository is remote. It should contain the full history of the branch it was made from, including diffs for each individual changeset and full commit logs, unless otherwise requested by the user.
  • 6. Merging preserves full history. A merge from one branch to another should also preserve full history. Changesets merged to the local branch should retain the individual, distinct diffs and commit logs for each changeset.

There are also some things that we would generally want:

  • 7. It is possible to commit, branch, merge, and work with history offline.
  • 8. The program is fast enough for general-purpose use.

Evaluation

Let’s look at some common VCSs against these criteria. I’ll talk about Arch (tla, baz, etc), bzr (bazaar-ng), Darcs, Git, Mercurial (hg), and Subversion (svn) for reference.

1. The fundamental method of collaboration must be a branch

All of the tools pass this test except for svn.

2. Branching should be cheap

Everyone except svn generally does this reasonably well.

The tla interface for Arch had a pretty terrible interface for this, so it took awhile simply due to all the typing involved. That’s better these days.

Darcs supports hardlinking of history to other local repositories and will do this automatically by default. Git also supports that, but defaults to not doing it, or you can store a path to look in for changesets that aren’t in the current repo. I believe Mercurial also can use hardlinks, though I didn’t personally verify that. bzr appears to have some features in this area, but not hardlinks, and the features were too complex (or poorly documented) to learn about quickly.

svn does not support branching across repositories, so doesn’t really pass this test. Branches within a repository are not directly supported either, but are conventionally simulated by doing a low-cost copy into a specially-named area in the repository.

3. Merging between branches is intelligent

Arch was one of the early ones to work on this problem. It works reasonably well in most situations, but breaks in spectacular and unintelligble ways in some other situations.

When asked to merge one branch to another, Darcs will simply merge in any patches from the source branch onto the destination which the destination doesn’t already have. This goes farther than any of the other systems, which generally store a “head” pointer for each branch that shows how far you’ve gone. (Arch is closer to darcs here, though ironically bzr is more like the other systems)

Merging between branches in svn is really poor, and has no support for recognizing changesets that have been applied both places, resulting in conflicts in many development models.

4. Inividial changesets should be mergeable without bringing across the whole history

Darcs is really the only one that can do this right. I was really surprised that nobody else could, since it is such a useful and vital feature for me.

Both bzr and git have a cherry-pick mode that simulates this, but really these commands just get a diff from the specific changeset requested, then apply the diff as with patch. So you really get a different changeset committed, which can really complicate your history later — AND lead to potential conflicts in future merges. bzr works around some of the conflict problems because on a merge, it will silently ignore patches that attempt to perform an operation that has already occured. But that leads to even more confusing results, as the merge of the patch is recorded for a commit that didn’t actually merge it. (That could even be a commit that doesn’t modify the source.) Sounds like a nightmare for later.

Arch has some support for it, but in my experience, actually using this support tends to get it really confused when you do merges later.

Neither Mercurial nor svn have any support for this at all.

5. Branching preserves full history

git, darcs, and Mercurial get this right. Making a branch from one of these repos will give you full history, including individual diffs and commit logs for each changeset.

Arch and bzr preserve commit logs but not the individual changesets on a new branch. I was particularly surprised at this shortcoming with bzr, but sure enough, a standard bzr merge from a remote branch commited three original changesets into one and did not preserve the individual history on the one commit.

svn doesn’t support cross-repo branching at all.

6. Merging preserves full history

Again, darcs, git, and Mercurial get this right (I haven’t tested this in Mercurial, so I’m not 100% sure).

Arch and bzr have the same problem of preserving commit logs, but not individual changesets. A merge from one branch to another in Arch or bzr simply commits one big changeset on the target that represents all the changesets pulled in from the source. So you lose the distinctness of each individual changeset. This can result in the uncomfortable situation of being unable to re-create full history without access to dozens of repositories on the ‘net.

Subversion has no support for merging across repositories, and its support for merging across simulated local branches isn’t all that great, either.

7. It is possible to commit, branch, merge, and work with history offline

Everyone except Subversion does a good job of this.

8. The program is fast enough for general-purpose use

All tools here are probably fast enough for most people’s projects. Subversion can be annoying at times because many more svn commands hit the network than those from others.

In my experience, Arch was the slowest. Though it was still fine for most work, it really bogged down with the Linux kernel. bzr was next, somewhere between arch and darcs. bzr commands “felt” sluggish, but I haven’t used it enough to really see how it scales.

Darcs is the next. It used to be pretty slow, but has been improving rapidly since 1.0.0 was released. It now scales up to a kernel-sized project very well, and is quite usable and reasonably responsive for such a thing. The two main things that slow it down are very large files (10MB or above) and conflicts during a merge.

Mercurial and git appear to be fastest and pretty similar in performance.

All of these tools perform best with periodic manual (or scheduled cron jobs) intervention — once a month to once a year, depending on your project’s size. Arch users have typically created a new repo each year. Darcs users periodically tag things (if things are tagged as part of normal work, no extra work is needed here) and can create checkpoints to speed checkouts over the net. git and Mercurial also use a form of checkpoints. (not sure about bzr)

Subversion works so differently from the others that it’s hard to compare. (For one, a checkout doesn’t bring down any history.)

Conclusions

I was surprised by a few things.

First, that only one system actually got #4 (merging individual changesets) right. Second, that if you had to pick losers among VCSs, it seems to be Arch and bzr — the lack of history in branching and merging is a really big issue, and they don’t seem to have any compelling features that git, darcs, or Mercurial lack. #4 was a unique feature to Darcs a few years ago, but I figured it surely would have been cloned by all the other new VCS projects that have popped up since. It seems that people have realized it is important, and have added token workaround support for it, but not real working support.

On the other hand, it was interesting to see how VCS projects have copied from each other. Everyone (except tla) seems to use a command-line syntax similar to CVS. The influence of tla Arch is, of course, plainly visible in baz and bzr, but you can also see pieces of it in all the other projects. I was also interested to see the Darcs notion of patch dependencies was visible (albeit in a more limited fashion) in bzr, git, and Mercurial.

So, I will be staying with Darcs. It seems to really take the idea of distributed VCS and run with it. Nobody else seems to have quite gotten the merging thing right yet — and if you are going to make it difficult to do anything but merge everything up to point x from someone’s branch, I just don’t see how your tool is as useful as Darcs. But I am glad to see ideas from different projects percolating across and getting reused — this is certainly good for the community.

Updates / Corrections

I got an e-mail explaining how to get the individual patch diffs out of bzr. This will work only for “regular”, non-cherry-picked merges, and requires some manual effort.

You’ll need to run bzr log, and find the patch IDs (these are the long hex numbers on the “merged:” line) of the changeset you’re interested in, plus the changeset immediately before it on the same branch (which may not be on the same patch and may not be obvious at all on busy projects.) Then, run bzr diff -r revid:old-revid-string..new-revid-string.

I think this procedure really stinks, though, since it requires people to manually find previous commits from the same branch in the log.

First steps with git and I’m not all that pleased

Software,

I have been tracking 2.6.16.x here because Xen doesn’t have patches for 2.6.17 yet (why on earth that is, I don’t know.) But I need a few 2.6.17.x patches to the qla2xx driver. So I figure this is an opportunity to learn git.

I learned git, but then quickly learned that I can’t just pull random commits from one branch to another. I have to use git cherry-pick, which doesn’t actually pull the commit unmodified — it takes a diff, and commits a new patch based on that diff. So I expect problems later when I bump the local branch to 2.6.17.

This seems depressingly like arch/baz/bzr, and is a good reason for me to stay with darcs for now. This same operation with darcs would have been trivial, AND darcs would have automatically pulled in prerequisite patches rather than giving a merge failure and making me find them manually.