Monthly Archives: February 2008

hg.complete.org is no more

Programming, , , ,

As of today, hg.complete.org is no more. I have removed mercurial and hgwebdir from my server, removed hg from my DNS zone, and converted everything that was in Mercurial over to Git. (Except for hg-buildpackage, which I have orphaned) So there is now stuff at git.complete.org.

I still have a ton of Darcs repos to convert, which will take more time.

Also I have heard a lot of people say that the GitPlugin for Trac is not very good. I have two Trac instances running it: one for commithooks and another for ListLike. Both seem OK so far, but I haven’t pushed them very much yet.

Trac & Git

Programming, ,

For quite some time now, I’ve been running Trac over at software.complete.org. Most of my free software projects — well, the ones where I actually go to the effort to make formal releases — have a Trac instance. This Trac instance provides a wiki, bug tracker, downloads area, timeline (with RSS feeds), and VCS integration.

Trac is a nice program, but one thing has bugged me about it all this time:

Every trac instance is its own island.

I have 17 trac instances out there for my projects. To see what bugs are out there on my own server, I have to check 17 websites (or 17 RSS feeds or whatnot). Publishing a new program is not a lightweight process.

So today I started poking around looking for something better. I really like Trac’s way of integrating the wiki with the BTS and the commits; wiki markup can refer to a bug or a changeset, and bugs can use wiki markup too.

I looked at Redmine, Mantis, and Roundup, and I also have experience with RT.

Of these, Redmine looks the most interesting. Multiple projects support, per project wiki and forums, gantt charting even, and support for SVN, CVS, Mercurial, Bazaar, and Darcs — with Git support out there as patches to their development tree already too. Oh, and I saw references to a Trac importer as well. One thing that makes me nervous, though, is that they have no links to sites that use Redmine (except one in the news section), and Google isn’t turning up users either. Does nobody use this thing?

What else should I be looking at?

Over on the Git side, I’m still liking Git. I have now migrated several Mercurial projects over to git (see git.complete.org). I am also playing with Darcs to git migration using darcs2git, which also is going well. Sometimes gitk shows a nicer representation of a Git repo converted from Darcs than I was able to get from Darcs.

Church Choir

General,

Every year on Good Friday, our church choir joins with the other church choir in the area for an evening choir program.

This year we are performing The Seven Last Words of Christ by Theodore Dubois. It’s a challenging and beautiful piece. We’ll have our combined choir, plus pipe organ and various other instruments.

Easter starts with a sunrise service, then breakfast at church. During the service, we’ll sing a number of additional songs. I think the list includes The heavens are telling by Haydn, Ave Verum (KV 618) by Mozart, Love Is Come Again arr. by Parker & Shaw, Praise the Lord by Handel, Alleluia (from Veni Sancte Spiritus, K 47) by Mozart, and Thanks Be to Thee by Handel.

It’s a lot of fun to be in the choir, and also quite a joyful time. I’m looking forward to Easter.

Experimenting with Git

Programming,

I’ve been writing about Git a bit lately.

I’ve decided to switch some of my Debian work over to it to start with, as well as some of my other projects.

Although I was thoroughly frustrated with Git a year ago, now I am quite pleased with it. What’s different? The documentation is a LOT better. So far I have only found one manpage (git-show) that omits lots of its options. The system is friendlier, keystroke-happier, and powerful.

Compared to Mercurial, I’ve found some nice things:

In-directory branching. I didn’t expect to care about this, since both git and hg permit lightweight clones. But it turns out to be so easy to use that it is great. Especially since I don’t have to setup multiple branch repos on the server. I really like this. Note that “hg branch” is not the same as a git branch, and see the discussion on the hg lists about renaming that before 1.0.0 for why.

Flexibility in getting things around. Plain HTTP works fine (no static-http:// hack). ssh. git daemon. rsync. Very slick.

Performance. Surprisingly, git actually feels faster than Mercurial, especially when pushing or pulling. I didn’t expect that.

Tags. They seem smarter in git. No more merging of .hgtags all the time. Also I like that I can attach a message to a tag and sign it.

All that power. There is a *lot* that Git can do. I should have been taking notes about it all.

My main complaint is still that Git doesn’t have something as nice as “darcs send”. Mercurial doesn’t either, but it’s a bit closer. Git has moved closer, but still has room to improve on that.

So I have set up git.complete.org and am starting to publish my Debian stuff on Debian’s alioth server as well.

Also, hg-fast-export in the fast-export project is *awesome*. Branch-aware and everything. It made a perfect Git version of my Mercurial work.

Git looks really nice, until….

Programming

So I have been learning about Git this weekend. It has some really nice-looking features for sure — some things Mercurial doesn’t have.

I was getting interested in switching, until I found what I consider a big problem.

Many projects that use git require you to submit things using git-format-patch instead of pushing/pulling from you. They don’t want your merge history.

git-format-patch, though, doesn’t preserve SHA1s, nor does it preserve merges.

Now, say we started from a common base where line 10 of file X said “hi”, I locally changed it to “foo”, upstream changed it to “bar”, and at merge time I decide that we were both wrong and change it to “baz”. I don’t want to lose the fact that I once had it at “foo”, in case it turns out later that really was the right decision.

When we track upstream changes, and submit with git format-patch, the canonical way to merge upstream appears to be:

git fetch; get rebase origin/master

Now, problem with that is it loses your original pre-conflict code on a case like this.

There appears to be no clean way around that whatsoever. I tried a separate “submission” branch, that rebases a local development-with-merge branch, but it requires a ton of git rebase –skip during the rebase process.

Thoughts?

Revisiting Git and Mercurial

Software, , ,

Exactly one year ago today, I wrote about Git, Mercurial, and Bzr. I have long been interested in VCS, and looked at the three main DVCS systems back then.

A Quick Review

Mercurial was, and for the moment, remains, my main VCS. Bzr remains really uninteresting; I don’t see it offering anything compelling that Mercurial or Git can’t do. My Git gripes mainly revolved around its interface and documentation. Also, I do have Windows people using my software, and need a plausible solution for them, even though I personally do no development on that platform.

Ted Tso wrote his own article in reply to mine, noting that the Git community had identified many of the same things I had ans was working on them.

I followed up to Ted with:

… So if Ted’s right, and a year from now git is easier to use, better documented, more featureful, and runs well on Windows, it won’t be that hard to switch over and preserve history. Ted’s the sort of person that usually is right, so maybe I should starting looking at hg2git right now.

So I guess that means it’s time to start looking at Git again.

This is rather rambly, I know. It’s late and I want to get these thoughts down before going to sleep…

Looking at Git

I started at the Git wikipedia page for an overview of the software. It linked to two Google Tech Talks about Git: one by Linus Torvalds and another by Randal Schwartz. Of the two, I found Linus’ more entertaining and Randal’s more informative. Linus’ point that CVS is fundamentally broken, and that SVN trying to be “a better CVS” (an early goal of svn, at least) means it too is fundamentally broken, strikes me as quite sound.

One other interesting tidbit I picked up is that git can show you where functions have moved from one file to another, thanks to its rename-detection heuristic. That sounds really sweet, and is the best reason I’ve yet heard for Git’s stubborn refusal to track renames.

The Landscape

I’ve been following Mercurial and Darcs somewhat, and not paying much attention to Git. Mercurial has been adding small features, and is nearing version 1.0. Darcs has completed a major overhaul both of its repository format and internal algorithms and is nearing version 2.0, and appears to have finally killed the doppleganger (aka conflict spinlock) bug for good.

Git, meanwhile, seems to have made strides in usability and documentation in its 1.5.x versions.

One thing particularly interesting to me is: what projects are using the different VCSs. High-profile projects now using Mercurial include OpenSolaris, OpenJDK (Java 7), and Mozilla’s projects. Git has, of course, the Linux kernel. It also has just about everything associated with freedesktop.org, including X. Also a ton of Unixy stuff.

Both Mercurial and Git communities are working on TortoiseHg/TortoiseGit types of GUIs for Windows users. Git appears to have a sane Windows port now as well, putting it on pretty much even footing with Mercurial and Darcs there. However, I didn’t spot anything with obvious Windows ties in the Git “what projects use git” pages.

The greater speed of Mercurial and Git — even for pushing and pulling small patches — likely will keep me away from Darcs for the moment.

Onwards…

As time allows (I do have other things keeping me busy), I plan to install git and work through some tutorials and try to use it in practice as much as possible, to get a good feel for it.

Future

It is beneficial to be using a VCS that is popular, though that is certainly not a major criterion for me. I refuse to use SVN because its lack of distributed functionality makes it too unproductive to be useful. But it looks like Git is gaining a lot of traction these days, especially in Debian circles, which also makes it more interesting.

I notice that Ted did convert e2fsprogs over to git as he said he might, incidentally.

Two new bashisms

Programming

I learned about two bash features I hadn’t known about today.

From a colleague, GLOBIGNORE. A colon-separated glob of files to ignore when expanding globs. Helpful behavior when set to “*~” and used with grep.

From the Git FAQ, in a section explaining that it breaks the Git build process, CDPATH. A colon-separated search path to use when you type cd. Possibly useful to refer to subdirs of ~ or other common areas. Seems like it’s prone to break a ton of scripts if exported though.

Death sure is cost-effective, isn’t it?

Society,

I just read Death Be Not Proud (But It Is Cost-Effective) by Chez Pazienza. In his story, Chez talks about his stay in the hospital to have a marble-sized brain tumor removed. Across the room during his stay in neuro ICU, he saw a person far worse off than himself: staples all around his head, barely able to stay conscious, unable to speak. After a few days of this, Chez asked the nurse what had happened to the other person.

It was the same thing.

The difference? Chez had good insurance, and the other person didn’t. So Chez got the modern surgery with the latest technology, and the other guy got the Neolithic version. The other patient’s family came to visit, clearly heartbroken at his condition, not knowing whether he’d ever be the same. And knowing that even if he’d survive, he’d have years of physical therapy ahead of him.

Then there was the story of the girl whose insurance company denied a liver transplant, calling it “experimental”, sending her to her death. He says:

Regardless of what Fox business-creature Neil Cavuto may have to say on the subject, healthcare and profit are two thoroughly antithetical concepts. Giving CEOs the authority to stand on the edge of the arena and issue a final thumbs-up or down while we lay incapacitated or dying is like charging a lion with protecting the Christians.

I entirely agree.

Registrar Dynadot Conspires to Help Take Down Wikileaks?

Technology

Yesterday, Wired ran a story on the Cayman Islands bank that got Wikileaks.org blocked. This story said, in part:

When the bank’s lawyers indicated they would be filing a suit, she asked them to tell her where so that Wikileaks could find an attorney in the appropriate jurisdiction to represent it. She says the lawyers refused to tell her. Two and a half weeks later, the bank filed a restraining order against Dynadot and Wikileaks in San Francisco. Wikileaks received notice only a few hours before the case went to a judge who accepted the agreement between Dynadot and the bank.

(emphasis mine)

Now, Dynadot and this Cayman Islands bank apparently had an agreement to block wikileaks.org already. Not only did Dynadot effectively take wikileaks.org down, but also they “lock(ed) the wikileaks.org domain name to prevent transfer of the domain name to a different domain registrar.” The U.S. Disctrict Court for Northern California issued this injunction without ever giving Wikileaks a chance to respond. The bank only filed the request against Dynadot. Apparently Wikileaks received notification that this was going to happen only 6 hours before the hearing (an incredibly short time in legal terms), nowhere near enough time to prepare a case.

Now, the reason I post this is because I have looked at Dynadot as a registrar before. They have good prices and a whois privacy service that makes sense: where you remain the owner of record, making it easier if you need to transfer the domain or prove your ownership of it.

But before signing up, I read their AUP carefully. Among many alarming things, I noticed this paragraph:

You further agree that Dynadot, in its sole discretion and without liability to You for any resulting loss or damages, may take immediate corrective action, including, but not limited to, removal of all or a portion of Your domain services and/or deletion, suspension, cancellation, termination, or other interruption of domain services or Your customer account with Dynadot, at any time during the term of this Agreement, in the event of notice of any possible violation of this Agreement by You or Your end users, or if such service or account is used in association with morally objectionable activities, or for any reason whatsoever. In such cases, any and all fees paid to Dynadot will be non-refundable and ineligible for account credit.

So, I thought I would write to them about it. Here is an excerpt from their response:

We always conduct an investigation before taking action against a domain. We will give you a chance to respond to the complaints.

From Wired’s story, it doesn’t look like that really happened. The US Government has already issued advisories about Cayman Islands banks, and it is unclear (to me at least) what law Wikileaks broke, or how Dynadot could find their actions of exposing fraud “morally objectionable”. What’s more, collaborating with the bank to get a takedown order written, but not talking to Wikileaks, seems to go against their statements to me (assuming again that the Wired story is accurate).


Here is my entire mail. You may also find archive.org’s copy of the AUP from last July to be helpful. (I wrote the email in November, and they’ve added some sections since then, so the section numbers don’t necessarily match up)

Subject: Re: SITE: Questions about your AUP
From: Dynadot Info <info@dynadot.com>
Date: Sat, 3 Nov 2007 13:42 -0800
To: jgoerzen@complete.org 

Hello,

Thank you for your email. Responses are below.

Best Regards,
Dynadot Staff

--------------------------------------------------
DYNADOT... $8.99 domain names... $1/mo. web hosting
http://www.dynadot.com




Hi,

I currently have several domains being hosted with Gandi.  I have long been looking for someone that can provide a level of privacy for my whois data in a sane way.  I think Dynadot is the first I've seen that looks like it can do that and still be affordable.

In preparation to transfer over a first test domain, I read your AUP and frankly am quite troubled by what I saw.  I hope that you can allay my fears.

In section 4 of part 2, the last paragraph states that Dynadot "for any reason whatsoever" may delete, cancel, or terminate my domain services or customer account.  It also lists "notice of possible violation" as a justification for that.  That makes me even more nervous -- any random person could send you an email claiming something nefarious is happing with my domain name, and I'm agreeing to just let you delete it because of that?



We always conduct an investigation before taking action against a domain. We will give you a chance to respond to the complaints. 


A very similar clause appears in section 7 of part 1 ("cancellation of services").  It goes on to cast a very broad net around objectionable material and says that Dynadot decides what's objectionable.  Now, if you go to my website at www.complete.org or blog at changelog.complete.org, you'll see I'm an upstanding netizen.  But the AUP says that things that "are designed to or effectively... embarrass" third parties could get my domain cancelled.  So if I post a review of Vista that says I think Microsoft did a poor job of engineering, would my account be yanked if one of their engineers complained?  What if I (legally!) linked to a Comedy Central sketch using real TV footage to mock George W. Bush or Hillary Clinton?



The cases you described above would never happen with us. Complaining about Microsoft or making fun of George Bush are protected free speech. The service agreement is designed to give us some flexibility in dealing with customers that break the law. 


I'm particularly concerned about this because apparently DynaDot felt it worthwhile to try to take down the website of someone that found a security hole: http://www.jhuskisson.com/friends/dynadot-fights-back-bans-nick-from-everything

While I wouldn't condone step-by-step cracking instructions in most cases, this is concerning to me.



He was posting a step by step guide to hacking our website, which is illegal. The hack did not work, but we noticed an upsurge in strange activity in our logs, so we requested the hacking guide be taken down. 


Finally, item (i) under the Domain Privacy Service section says that you could immediately reveal all my information upon the receipt of merely a *claim* (not even a court order), even if it's invalid.  But I thought that NOT doing this was what you were saying made your service better, over at http://www.dynadot.com/resource/article/qa.html?aid=0



Once again we need to build some flexibility into our service agreement to deal with people who use their domains for criminal activity. Otherwise we could be liable for the damages that they cause to others. So far, we have never dropped anyones privacy except in the few cases we were forced to by a FBI subpoena.


I know this is a long message, and I appreciate your time.  I really do want to use your service, but -- no offense intended -- I want to make sure I'm not dealing with scammers first, and from reading the AUP, I'm not so sure!



No offence taken. I will email our counsel to see if we can tighten up the agreement a bit. 


-- John Goerzen