Category Archives: Programming

Thoughts on Redmine

Programming,

A few days ago, I discussed Trac and Redmine. Redmine is a project management tool, similar to Trac, with built-in download tools, bug tracking, etc.

Redmine has a lot of nice features. Chief among them is better integration between multiple projects, so I don’t have to go to 17 separate pages to see the open bugs on my projects.

But I’m worried about the Redmine community. It appears to live in an insular Ruby world, without much participation outside. I wrote about some of those concerns in their forums. I’ve also submitted bugs to Redmine, some with patches.

Also, it’s concerning that, although Redmine includes a very nice forum module, the Redmine forums are still on RubyForge. Also, there are many bugs in the Redmine BTS that have patches but little, if any, comments from the Redmine people that have commit access.

It could just be that Redmine is a fairly new project and just needs some time to get on its feet more. It’s been around since July, 2006, which isn’t all that long on the one hand… or quite awhile, depending on how you look at it.

The git support patch for Redmine looks very nice. However, after a month, it still hasn’t been replied and there’s no indication why. Which also is troubling.

So I think I’ll sit with Trac for a little while until I get a better feel of how Redmine is progressing.

hg.complete.org is no more

Programming, , , ,

As of today, hg.complete.org is no more. I have removed mercurial and hgwebdir from my server, removed hg from my DNS zone, and converted everything that was in Mercurial over to Git. (Except for hg-buildpackage, which I have orphaned) So there is now stuff at git.complete.org.

I still have a ton of Darcs repos to convert, which will take more time.

Also I have heard a lot of people say that the GitPlugin for Trac is not very good. I have two Trac instances running it: one for commithooks and another for ListLike. Both seem OK so far, but I haven’t pushed them very much yet.

Trac & Git

Programming, ,

For quite some time now, I’ve been running Trac over at software.complete.org. Most of my free software projects — well, the ones where I actually go to the effort to make formal releases — have a Trac instance. This Trac instance provides a wiki, bug tracker, downloads area, timeline (with RSS feeds), and VCS integration.

Trac is a nice program, but one thing has bugged me about it all this time:

Every trac instance is its own island.

I have 17 trac instances out there for my projects. To see what bugs are out there on my own server, I have to check 17 websites (or 17 RSS feeds or whatnot). Publishing a new program is not a lightweight process.

So today I started poking around looking for something better. I really like Trac’s way of integrating the wiki with the BTS and the commits; wiki markup can refer to a bug or a changeset, and bugs can use wiki markup too.

I looked at Redmine, Mantis, and Roundup, and I also have experience with RT.

Of these, Redmine looks the most interesting. Multiple projects support, per project wiki and forums, gantt charting even, and support for SVN, CVS, Mercurial, Bazaar, and Darcs — with Git support out there as patches to their development tree already too. Oh, and I saw references to a Trac importer as well. One thing that makes me nervous, though, is that they have no links to sites that use Redmine (except one in the news section), and Google isn’t turning up users either. Does nobody use this thing?

What else should I be looking at?

Over on the Git side, I’m still liking Git. I have now migrated several Mercurial projects over to git (see git.complete.org). I am also playing with Darcs to git migration using darcs2git, which also is going well. Sometimes gitk shows a nicer representation of a Git repo converted from Darcs than I was able to get from Darcs.

Experimenting with Git

Programming,

I’ve been writing about Git a bit lately.

I’ve decided to switch some of my Debian work over to it to start with, as well as some of my other projects.

Although I was thoroughly frustrated with Git a year ago, now I am quite pleased with it. What’s different? The documentation is a LOT better. So far I have only found one manpage (git-show) that omits lots of its options. The system is friendlier, keystroke-happier, and powerful.

Compared to Mercurial, I’ve found some nice things:

In-directory branching. I didn’t expect to care about this, since both git and hg permit lightweight clones. But it turns out to be so easy to use that it is great. Especially since I don’t have to setup multiple branch repos on the server. I really like this. Note that “hg branch” is not the same as a git branch, and see the discussion on the hg lists about renaming that before 1.0.0 for why.

Flexibility in getting things around. Plain HTTP works fine (no static-http:// hack). ssh. git daemon. rsync. Very slick.

Performance. Surprisingly, git actually feels faster than Mercurial, especially when pushing or pulling. I didn’t expect that.

Tags. They seem smarter in git. No more merging of .hgtags all the time. Also I like that I can attach a message to a tag and sign it.

All that power. There is a *lot* that Git can do. I should have been taking notes about it all.

My main complaint is still that Git doesn’t have something as nice as “darcs send”. Mercurial doesn’t either, but it’s a bit closer. Git has moved closer, but still has room to improve on that.

So I have set up git.complete.org and am starting to publish my Debian stuff on Debian’s alioth server as well.

Also, hg-fast-export in the fast-export project is *awesome*. Branch-aware and everything. It made a perfect Git version of my Mercurial work.

Git looks really nice, until….

Programming

So I have been learning about Git this weekend. It has some really nice-looking features for sure — some things Mercurial doesn’t have.

I was getting interested in switching, until I found what I consider a big problem.

Many projects that use git require you to submit things using git-format-patch instead of pushing/pulling from you. They don’t want your merge history.

git-format-patch, though, doesn’t preserve SHA1s, nor does it preserve merges.

Now, say we started from a common base where line 10 of file X said “hi”, I locally changed it to “foo”, upstream changed it to “bar”, and at merge time I decide that we were both wrong and change it to “baz”. I don’t want to lose the fact that I once had it at “foo”, in case it turns out later that really was the right decision.

When we track upstream changes, and submit with git format-patch, the canonical way to merge upstream appears to be:

git fetch; get rebase origin/master

Now, problem with that is it loses your original pre-conflict code on a case like this.

There appears to be no clean way around that whatsoever. I tried a separate “submission” branch, that rebases a local development-with-merge branch, but it requires a ton of git rebase –skip during the rebase process.

Thoughts?

Two new bashisms

Programming

I learned about two bash features I hadn’t known about today.

From a colleague, GLOBIGNORE. A colon-separated glob of files to ignore when expanding globs. Helpful behavior when set to “*~” and used with grep.

From the Git FAQ, in a section explaining that it breaks the Git build process, CDPATH. A colon-separated search path to use when you type cd. Possibly useful to refer to subdirs of ~ or other common areas. Seems like it’s prone to break a ton of scripts if exported though.

Haskell Fun

Programming

Bryan O’Sullivan noted over at the Real World Haskell blog that Haskell made quite the impact at OSCon. And I can attest to Simon Peyton-Jones having trouble leaving the building because of all the people that wanted to talk to him about Haskell. It was interesting to think about “why now” for Haskell’s popularity. Bryan’s post has links to the video of Simon’s talks, which are great. (Sample quote: “Oh look, a whiteboard has appeared as if by magic! What joy!”)

I followed Bryan’s link to the Haskell/OSCon-related blog posts at Technorati. Here’s an interesting one by chromatic, who gave some Perl talks at OSCon. Favorite quote:

I sat next to Nat Torkington at the tutorial. He kept rubbing his temples. At one point I leaned over and said, “The interesting thing about Haskell is that its functions only take one argument.” He turned green.

In all seriousness, well-factored Haskell code resembles well-factored Smalltalk code: if you have functions (or methods) longer than a handful of lines, you’re probably doing too much. Lower level languages such as C rarely give the opportunity for composition and abstraction that you can get out of functional languages. The presence of pure functions–functions which never change global state and which return the same output for the same input–is also immensely important.

It’s actually the combination of the two features which give these languages such power. When Haskell forces you to mark impure functions explicitly, it gives you tools to isolate behavior which can change global state in the smallest possible scopes, and prevents you from composing impure and pure code together accidentally. When Haskell lets you compose functions into larger functions, not only does it help you write code more concisely, but it provides well-defined units of behavior which work along well-defined and isolated boundaries.

So what is this business about Haskell functions taking only one argument? Let’s look at a quick example. Say I wanted to write a function to multiply two numbers. I’d write:

mul a b = a * b

I could give the type of this function like this:

mul :: Int -> Int -> Int

You could read that as “mul takes two Ints and returns an Int”. And you can think of it this way. But you could also write the type this way:

mul :: Int -> (Int -> Int)

It means the exact same thing and is valid to Haskell. To read it, you’d say “mul takes an Int and returns a function that takes an Int and returns an Int.” And truly this is what Haskell functions that take multiple parameters are doing. Now, I can say:

fifteen = mul 3 5
mulByThree = mul 3
fifteen' = mulByThree 5
fifteen'' = (mul 3) 5

mulByThree has type Int -> Int; it’s a function that we got by simply not applying “mul” to all its arguments. fifteen” illustrates what is going on internally when you write “mul 3 5”. It turns out that being able to call a function with multiple arguments is just some syntactic sugar to Haskell.

This is a tremendously useful feature. For instance:

filter (>= 10) [1..20]

I’ve applied >= to only one argument here. That’s fine; it returns a function that’s exactly what filter wants.

I’ve been watching the video of Simon’s Taste of Haskell talk. I could *hear* when the audience grasped the utility of this because there was a collective “Oooooo!” from them.

By the way, Haskell has type inference, so I didn’t have to give types at all. Oh, and you can also refer to the function itself by not giving any arguments.

mul = (*)

This is one small reason I like to say “Haskell manipulates functions with the same ease that Perl manipulates strings.” Mind-bending, isn’t it?

Some more git, mercurial, and darcs

Programming

Ted Ts’o had an interesting post about git recently. He has a lot of good thoughts on the subject. He comments that he wound up using git because it’s so Unixy (with its small commands to do things), that he sees the git community developing innovations faster than Mercurial, and that they are working to improve the documentation and user interface problems.

The being so Unixy is a double-edged sword. On the one hand, it can make it easy to write shell scripts to extend Git. That itself can be a double-edged sword (think filename quoting and the like). But one doesn’t have to use the shell. The other downside is that being Unixy makes it hard to run on platforms that aren’t, such as Windows. So if one is working on Unix-only software (X, the kernel, e2fsprogs, etc.), there’s no need to care about it. But if you’re a person like me, who has Windows users using my software, or a large organization like Mozilla, it’s maybe a showstopper. Of course, workarounds exist (cygwin, git-cvsserver), but none of them are particularly nice.

I think that both Git and Mercurial are working to address their shortcomings. I’ve chosen hg for now because it does what I need now. And because there are very nice tools to convert hg to git, and vice-versa. So if Ted’s right, and a year from now git is easier to use, better documented, more featureful, and runs well on Windows, it won’t be that hard to switch over and preserve history. Ted’s the sort of person that usually is right, so maybe I should starting looking at hg2git right now

So following up on my bzr post, here are the things that Mercurial is great at right now:

  1. Performance. Approximately even with git, occasionally faster. Nobody else can compete with these two right now.
  2. Simplicity. It’s almost as easy to get started as with darcs, and with recent patches, will be even closer in the future.
  3. Lots of ways to interact. You can send hg bundles, which preserve all metadata (parents, hash, authors, etc), or you can send git-format email patches, or you can push and pull between repos. The email tools will shortly be able to automatically detect what patches to send. Your choice. git doesn’t seem to support lossless emailing of bundles like this, and bzr doesn’t make emailing of anything easy by default.
  4. Merging. hg seems to be able to automatically resolve more merge conflicts than anything else, and when it can’t automatically resolve them, has a nicely configurable system to let you use your choice of tool to manually resolve them.
  5. Community. The Mercurial community is open and inviting, and open to new/different ideas. It seems similar to Darcs in that respect, and somewhat dissimilar to git.
  6. Rebase does not trash history like it does (barring undocumented manual intervention) in git.

I’ve written before about Darcs, so I won’t duplicate that here.

bzr, again

Programming, , ,

I’ve talked a lot lately about different VCSs.

I got some interesting comments in reply to my most recent post. One person took issue with my complaint that nobody really understood how to specify a revision to git format-patch, and proceeded to issue an incorrect suggestion. And a couple of people complained about my comments about bzr, which generally came down to the released version of bzr didn’t have anything compelling and also didn’t support tags.

So I went into , asked them what bzr has that git, Mercurial, and darcs don’t. And gave bzr the benefit of the doubt that 0.15 will be out soon and will be stable. What I got back were these general items:

  1. Renaming of directories (not in hg, git)
  2. 2-way sync with Subversion (not in hg, darcs)
  3. Checkouts (not in any others by default)
  4. No server-side push requirement

Let’s look at these in more detail.

1. Renaming of directories

All of them can rename files and (excepting git) completely accurately track back the file’s history. But consider this: if person A commits a change to branch A that adds a file, and person B then renames the directory that the file is in on his branch, will a merge cause person A’s file to appear in the new directory name? In darcs and bzr, yes. In Mercurial and git, no.

So yes, this is a nice thing. But I have never actually had this situation crop up in practice, and even if it did, could be trivially remedied. I would say that for me, I don’t really care.

[Update: Current stable releases of Mercurial can do this too. I’m not quite sure how, but it does work. So git is the only one that can’t do that.]

2. 2-way sync with Subversion

This is a really nice feature and is present in both git and bzr. I haven’t tested it either place, but if it works as advertised — and properly supports tracking multiple related svn branches and merges — would be slick. That was enough to make me consider using git, but in retrospect, I so rarely interact with people using svn that it is not that big a deal to me.

Still, for those that have to work with svn users, this feature in bzr and git could be a big one.

Better yet would be to get all those svn holdouts over to DVCS.

3. Checkouts

A bzr checkout is basically a way to make local commits be pushed to the remote repo immediately, as with svn. This is of no utility to me, though I can see some may have a use for it. But it can be done with hg hooks and probably approximated with scripting in others.

4. No server-side process necessary for pushing repos

bzr has built-in support to push to a server that has sftp only, and doesn’t require a copy of itself on the server. While I believe that none of the other three have that, it is possible to rsync (and probably ftp) darcs and Mercurial repos to a server in a safe fashion by moving repo files in a defined order. Probably also possible with git. All four can pull repos using nothing but regular HTTP.

What bzr still doesn’t have

Integrated patch emailing. The big thing is that it has no built-in emailing of patches support. darcs is extremely strong in this area, followed by hg, and git is probably third. “darcs send” is all it takes to have darcs look at the remote repo, figure out what you have that they don’t, and e-mail a bundle of changesets to them. I posted an extension and later a patchset that does all this for Mercurial except for automatically figuring out what default email address to do (that’ll come in a few days, I think). One feature Mercurial has had for awhile that Darcs hasn’t is sending multiple textual diffs as a thread, with one message per changeset. bzr doesn’t have any support for emailing patches yet, which is disappointing. Because of the strong support for this in darcs and Mercurial, people running those systems feel less of a need to publish their repos.

[Update: There is a plugin for bzr that seems to address some of this. I haven’t tested it, and it’s not in bzr core (so doesn’t quite meet my very friendly for a newbie requirement), but this does exist, though apparently not as advanced as Mercurial]

Performance. Supposedly 0.15 is supposed to be better on this, but even if bzr achieves the claimed doubling of performance, most benchmarks I have seen would rate it as still being significantly behind git and Mercurial, though it may overtake darcs in some tests.

Extensive documentation. I would say that bzr’s docs are better in some ways than git’s (its tutorials especially), but lack depth. If you want to know some detail about how the repository works on-disk, it’s not really documented. Darcs still has David’s excellent manual, and Mercurial has the hg book which is still great as well.

Merging not as advanced. darcs is pretty obviously way on top here, but of the others, Mercurial does a pretty good job with its automatic handling of renames and automatic resolving of different branches that commit the same change (even if that same change is a rename, or an add of the same content). bzr can’t resolve as much automatically.

Summary

Well, I’ll say that bzr still doesn’t look compelling enough for my use cases to use, and the lack of an easy-for-a-newbie-to-use automated email submission feature is a pretty big disappointment. Though I did appreciate the time those on spent with me, and if I needed to sync with svn users frequently, I’d probably choose bzr over git.

For now, I’m happy with sticking with darcs for my code and hg for my Debian work.

But all four communities are aggressively working on their weaknesses, and this landscape may look very different in a year.