/read/online

1: The Debian Universe

Debian is generally regarded as a good Linux distribution with great package management but a terrible installer. However, it's actually a lot more than that. Technically it's not even really a Linux distribution in the traditional sense, and it can be a hard thing to define for those who have dealt primarily with commercial distributions like Red Hat, Mandrake and SuSE, because Debian even embraces alternative kernels such as the BSD kernel and The Hurd.

Linux itself is grounded in community involvement and accessibility, concepts it inherited from the GNU project. But when we think of Linux distributions today they are almost all commercial ventures. What goes into each commercial distribution is a decision made by paid employees with the company bottom line in mind. That may not in itself be a bad thing, but it does leave them open to the possibility of commercial failure as we saw recently with Mandrake. If an organisation needs to make money to survive, that danger always remains.

But Debian is different. It's a totally open, cooperative project involving a great diversity of people, each doing what they do either because they want to or because they feel it's worthwhile. In fact Debian doesn't really exist in the legal sense. There is no Debian Inc, there are no shareholders, no board, not even a non-profit organisation. There is an umbrella organisation called Software in the Public Interest (SPI), but Debian itself is really just a big cooperative project. It's probably one of the best large scale examples of a true 'bazaar' style project as described by Eric S Raymond in "The Cathedral And The Bazaar" that exists today. It doesn't have to make any sales, it doesn't have to meet investor expectations, its members just get on with doing what they do best: create one of the best ever collections of open source software.

That can be both a good and a bad thing. One of the recent problems, for example, has been obtaining AMD x86-64 prototype hardware for porting and testing. AMD have limited supplies of hardware, and while it's still at the prototype phase they will only release machines to organisations that can both demonstrate a need and enter into a non-disclosure agreement. Because Debian doesn't really exist legally, it can't enter into an agreement binding on all it's developers and so AMD have been unable to provide hardware for Debian developers to test on.

However, problems like that are few and far between, and for the most part Debian's lack of structure is its strength. It's diversity and inclusiveness have resulted in its ability to package a huge range of software on more hardware architectures than any other distribution, or indeed any other operating system.

Something that many people don't know is that Debian officially supports 11 different hardware architectures: x86/IA-32(i386), Motorola 68k, Sparc, Alpha, PowerPC, ARM, MIPS, MIPSel, HP PA-RISC, IA-64 and S/390. And that doesn't mean that everything is developed for i386 first, with other architectures lagging behind and treated as poor cousins, with distribution releases delayed by weeks or months. When a release such as Woody (Debian 3.0) happens, it happens simultaneously on all 11 architectures.

That's a pretty mind-blowing concept when you consider that even the big boys such as Red Hat only officially try to support one or two. Managing development on 11 architectures has required Debian to put in place a very sophisticated auto-builder system that allows a developer to create a software package on whatever their local architecture happens to be, then upload the package to a build queue. Once in the queue the package is sanity checked, then distributed to machines in the build farm: a group of machines loaned or donated to Debian that represent all 11 architectures. The package is then compiled from source on each architecture, with any resulting errors being sent back to the developer. Assuming everything went smoothly, the source package and compiled binary packages are then moved across to the master servers that feed mirrors located all around the world. A short time later, the package is available to download and install on any supported architecture.

The end result is that a package can be created by a developer on their local machine such as an Intel or AMD system, uploaded to the build queue, then installed a few hours later anywhere in the world as a pre-compiled binary on anything from an iBook to an S/390 mainframe.

Pretty impressive.

Quite apart from its technical achievements, Debian has laid the groundwork for much of the community environment that Open Source developers enjoy today. You may be familiar with the Open Source Initiative's Open Source Definition, for example, but you may not know that it is directly derived from the Debian Free Software Guidelines. And Debian has a Social Contract, a document that defines how Debian will relate to users and developers. All developers joining the project are required to agree to uphold the Open Source Definition and the Social Contract. In some ways the Social Contract is like the conscience of Debian, making sure that developers keep their perspective correct and always try to do what is best for the community as a whole rather than compromise their standards for short term gain. It also ensures that Debian itself will remain free and open, accessible to the greatest number of people possible without prejudice. If you want to understand their ideals and the standards to which Debian developers are held, I highly recommend you read both the Open Source Definition and the Social Contract.

For the technical and social reasons mentioned Debian enjoys a great deal of respect within the developer community at large. While it has a small marketshare among users with less expertise, a very large proportion of core Open Source developers use it as their primary system. Legends in the Open Source world such as Andrew Tridgell (of Samba and Rsync fame) and Paul "Rusty" Russell (author of Ipchains, Iptables, and all-round kernel guru) are among those who use it every day. At the Australian Linux Technical Conference held in Perth in January 2003 and attended by Linus Torvalds, Rasmus Lerdorf, Alan Cox, Rusty Russell, Andrew Tridgell and many other Open Source luminaries, a survey showed Debian was by far the most commonly used distribution among the delegates with Red Hat a distant second.

Debian also leads the way in internationalisation, with a very active team dedicated to translating documentation and web pages into many languages, and developers working to abstract text within menus and dialogs from the software itself so that users can run software in their native language. There is still a lot of work to be done in this regard, but Debian is already one of the most flexible distributions in terms of accessibility by users for whom English is not their primary language. The various translation and internationalisation teams are always happy to receive offers of help, so if that's an area in which you can contribute they'd love to hear from you. Making software accessible to all, regardless of ethnic background or economic circumstances, is one of the strongest ideals within Debian.

Debian developers come from all corners of the globe, and you can even see the geographic diversity for yourself because many developer locations have been plotted on a world map.

So how does such a diverse, geographically separated collection of individuals manage to communicate or get anything done?

The short answer is mailing lists. Debian has over 100 internal mailing lists handling an enormous volume of daily traffic, everything from newbie questions to focused discussions within internal subgroups. Probably the busiest and rowdiest is debian-devel, where Debian developers discuss everything from specifics of software packaging and policy to overall project objectives. In the spirit of a truly open project, all mailing lists (with one notable exception) are public lists where anyone can join in the fun. The only closed list is a special list where very sensitive issues such as disclosure of upcoming security patches are discussed.

In terms of membership, Debian has a fairly flat and democratic structure. The first layer are Debian users everywhere, who may be system administrators running many machines, or casual users with a Debian box at home. It also includes many people who contribute directly to the project, such as by submitting patches and bug reports or even creating Debian packages. The next layer is people known as Debian Developers, or DDs. At present there are approximately 1100 DDs. DDs have additional privileges: because they are expected to create and maintain Debian software packages, they can directly upload software packages to the build queue. Non-DDs who want to have a package included in Debian must have it sponsored by a DD, otherwise it can't be uploaded to the queue. DDs can also vote on issues such as changes to policy or elections for official positions, and they can stand for official positions themselves. The final layer is those few DDs who hold official positions, such as the annually elected Debian Project Leader (DPL), Technical Committee members, the Release Manager for each release, and managers of various infrastructure items such as the build farm and mirrors.

The Debian community prides itself on being very result-oriented. Debates on debian-devel often wander off into left field, but the people within the community who command the greatest respect are the ones who see that something needs to be done and just get in and do it rather than debate issues without taking any action. One perennial issue that's guaranteed to start a flamewar if mentioned on debian-devel is the long and tedious process of moving from being a general Debian user to being recognised as an official Debian Developer. After wandering around for a while, that argument always ends up coming back to the same conclusion: the way to gain respect and be considered a member of the community is to just get involved, whether you are an official DD or not. In a future article I hope to walk you through the New Maintainer process step by step so you can see what's involved, but for now if you're wanting to contribute and you're not an official DD my advice is to subscribe to debian-devel and just get involved.

OK, so why is this article titled "The Debian Universe"?

In the early days of Debian, there was a general concept that it could become a solid software base on which other projects could be built. By using Debian as the underlying framework, various groups could create their own custom software environments to meet their specific needs without re-inventing the common areas.

However, Debian Project Leader Bdale Garbee made an observation at the recent Australian Linux Technical Conference that rather than being a foundation on which other projects are built, Debian has become a universe from which sub-projects select a specific subset of items. He joked that based on the current growth figures for Debian, it will eventually encompass every software package that has ever been useful to any person for all of 5 minutes.

His observation was only half joking: with the vast quantity of software now in Debian (the core distribution is 7 Cds, and as I write this the master mirrors are about to pass 80GB and carry around 11,000 software packages!) there is almost no need to add anything external to Debian in order to create a derived project. It's more a matter of defining your end result and selecting from Debian just the elements you need to create it.

This approach has already been happening for some time with a number of sub-projects, such as Debian Jnr, Debian-Med, Debian-Edu, and Debian Desktop. Each of these sub-projects is working to make Debian accessible and relevant to specific groups of users by working within the existing Debian structure rather than building layers on top of it. That way the whole community benefits from advances that are made by sub-projects, because each one never really leaves the mother ship: they just extend and arrange it to make things better for their particular subset of users.

I hope this article has shown you that Debian is not just another Linux distribution. It's a whole universe of software made available for everyone to benefit from, but which also happens to provide arguably the best Linux distribution available.

So welcome to The Debian Universe!