Saturday, 13 March 2010

Fork'n Evolution!

I just saw a post suggesting it is time for Ubuntu to fork evolution and thought it deserved a comment ... Although now i've just about finished this post I realise just how silly the whole notion is to start with, so it probably doesn't.

It is a pretty bizarre suggestion.

Even just on a technical level, the author may not realise just how much work is involved in maintaining a piece of code the size of Evolution. Or the amount of organisation and effort required to make the sort of major structural changes it probably needs before any real progress can be made.

And politically it is a naive and arrogant to suggest one GNU/Linux company can 'go it alone' like that on any major piece of software, particularly one developed by a competitor. It is also pretty rude - I haven't even touched Evolution for 5 years as a user or developer, but as a free software developer and advocate I think forking should really be the point of last resort. And even then it would only be after a demonstrable and intractable conflict had arisen - e.g. refusal to accept many reasonable patches of significant size. A few 10 line patches hardly turns one into a maintainer. And I wonder if anyone who might be involved has even done that.

Evolution was always a strange free software project, which i've written about plenty of times before. Whilst I was working on it it had only a few external contributions, and even less so from any of the GNU/Linux vendors (apart from Sun). Most had patches just for (re)branding, but the occasional bug fix they did have rarely made it to the code-base through normal processes i.e. they rarely if ever submitted them to us. Not that we made it terribly easy, anally retarded patch submission and review processes, the copyright assignment crap, and simply being busy most of the time.

But even despite that - it is a free software project, and open to contributions from anyone. There is simply no need to fork it, if anyone desires to improve it they can, and indeed are encouraged to. It is part of the `social contract' if you will, of any GPL project. You get the code for free, and if you want to make changes you can, and add them to the public pool for the good of all. And unlike many 'open source' projects - Evolution was always a real free software product and not just an alpha-grade product looking for free beta-testers and code-monkeys like many seem to be.

"They didn’t understand that there is just one killer feature (just as with integrated desktop social networking) that needs to be in there which is Exchange support."

Back on topic ... this one statement probably undermines the post itself more than any other. As Jeff said in comments on the post this was Ximian's business model pretty much from the start, and one of the reasons Novell bought them. It was pretty much the reason Evolution existed anyway, although more to replace the need for Microsoft Exchange than to work with it - at least in my mind. As a free software developer I never saw the need to inter-operate with proprietary crap like that - which is probably why I was never working on that part of the code.

Also the whole bit about `social networking' mentioned in the post - this was a major part of the internal thinking at Ximian right from the start - before the term even existed. Although like then, I still don't really see the need for it, particularly in a corporate environment. We all just used IRC and it worked just fine for us.

I was working on the mail code even longer than Jeff - I can't believe I started over 10 years ago. I haven't looked at the code since leaving Novell and the project, and I never worked on the `microsoft exchange component' anyway, but i'd suggest the code was simply never up to snuff given the way it originally designed, and even just because who wrote it.

Nice guy and all, but not great at coming up with a maintainable design let alone writing reliable C. Jeff and I's experience with the original IMAP code was a nightmare that started when he simply abandoned it in a fit of terrible management. One day he simply began refusing emails or dealing with patches saying he'd moved to another project (might have even been the microsoft exchange plugin). So we were left maintaining a really nasty bit of code with no deep knowledge of how it worked. It was all too lisp-like with little or no structure which made it very difficult to grok particularly given it's requirement for multi-threading. Lisp might be a great language, but it simply isn't how you write C.

And apart from the code itself, I believe the microsoft exchange component had a messy architecture layered on top of the already overly complex ones (yes, multiple) within evolution. Partly because all the different data-types were routed through the same connection, and also because it started as a proprietary extension and had silly things like a symbol obfuscater. To be honest once I found out how it worked I was surprised it worked at all ... it was not surprising in the least that it was slow and buggy and probably always would be.

Disbanding the original team and moving the project wholesale to India can't have helped either, at least at the time. Apart from the brain drain that involved, India has a tendency to add layers of middle-management too, so big decisions we could have gotten away with by the end (probably without asking ;-) became completely impossible due to risk averse middle-managers only worried about their next promotion-due-to-tenure. High staff turn-over due to a liquid job market was also an issue.

The entire component architecture of the system should have been replaced years ago (I wouldn't know - perhaps it has been) - almost all of the early design decisions were wrong, and particularly outside of the mail code none of it was ever reviewed because of the high turn-over of developers. Inside the mail component there was a huge scope for doing things in a much more re-usable way. CORBA got a bad wrap because we had an absolutely terrible design and often simply used it incorrectly. The original bonobo design was just a COM clone, so no surprises there - pluggable multi-process widget systems was a stupid use of CORBA. But the evolution-data-server wasn't, although the interfaces were not as good as they could have been.

I don't think there is any conspiracy - but Novell are a proprietary software company first and foremost and a project like evolution never really fit it's culture - probably the major reason I left in the end. Once HR starts impacting on engineering I think you have worries as a technology company, and even more as an `open sauce' one where the most valuable `eye-pee' you have is in your employee's heads.

Canonical make some odd decisions about Ubuntu too - more odd since they are purporting to be such a community focused organisation. But I don't think even they would be arrogant or silly enough to seriously consider such a strange notion as forking evolution. I'm sure they could surprise me though.


Sankar said...

Look at the reMAP link that I gave in the end. Though, It is still in-theory-only afaics.

Frej said...

I'm not an expert on evo code, but 2.30 actually does replace bonobo/corba with d-bus for communication with evo-data-server.
The rest is replaced with gtkuimanager or other non IPC solutions, even if bonobo was used in-process. I'm kinda vague because i really don't know that much :).

All driven by matthew barnes from redhat. Great stuff!

The fun point is that ubuntu hasn't even updated to the 2.29 branch for lucid..... I think the forking guy is a somewhat clueless.

Christopher Parker said...

Who was the original author who thought up the design of evolution?

NotZed said...

Sankar: Hmm, dunno about that remap thing, imap has issues but the proposed solution doesn't really address them. Just simplifying imap and particularly mime would do more (MIME is good, just the i18n stuff, multipart/signed, and 7-bit wire stuff needs to go). IMAP is actually a very simple protocol considering it is self-contained (e.g. for restful you've already combined several protocols, let alone the remap stuff on-top of that).

Christopher: Well my memory isn't too hot ... but the about page has a few names: ... and using that as a guide ...

Betrand designed Camel (email library), and it is more-or-less based on the JavaMail api. I then took it over and although the guts changed quite a bit the class hierarchy didn't change much. If I were to do it again though, I would focus on simplification and streaming - probably just about halve the API.

Miguel & Ettore did the early `shell' stuff (the main component architecture), and when they moved on it languished a bit because everyone else was busy and nobody wanted to touch it. Particularly after years of small bug fixes by everyone - since none of the rest of us really knew the way it was intended to work, nor wanted to learn lest we get stuck with it!

I actually can't remember who did the main EDS design - I was too busy to take note (or more importantly, over 10000 miles away). I know Federico, JPR, Rodgrigo, and Chris Toshok all worked on it at various times, and probably Seth, Miguel and Nat in the early days.

I'm pretty sure they'd all agree that we all made lots of mistakes at the start because we were still learning so much. Some of those mistakes got fixed and others didn't. Most of them were in the details but a few were pretty serious and hampered the project for a long time (the shell was nasty).

Frej: good to hear redhat contributing lately - like I said haven't used it in 5 years, but those were my experiences. I did actually think CORBA was a really good fit for EDS by the end ... the interfaces were on the right track but needed an overhaul. Replacing CORBA would not have solved such things on its own.

And yes the `forking guy' probably didn't even deserve any comment, but I had nothing better to do than get suckedered into a response ...