Drupal distributions

Drupal 5.0, the upcoming Drupal release, will have better support for Drupal distributions. Each distrubtion takes some set of Drupal themes and modules and packages them together with the Drupal core, along with custom installation steps, documentation, and so on. For example, one could create a distribution called "Drupal for Education"; it could have pre-configured roles and permissions for both teachers and students, and ship with additional modules that allow one to offer online courses and testing.

New and different markets

Distributions allow people to create ready-made downloadable packages with their own focus and vision. This will enable Drupal to reach out to both new and different markets. If we had a "Drupal for Education" distribution it might, at one point, be able to compete head on with the other course management systems like Moodle and Blackboard.

Not only will distributions allow Drupal to compete in existing markets, it will also enable people to create new markets. For example, someone could create a "Drupal for Student Organizations" distribution and instantly be the number one tool in that market. The number of verticals is nearly unlimited and the opportunities are numerous. Divide and conquer.

While the possibilities are nearly unlimited, it does impose a number of challenges ... (Click read more to learn more about these.)

Collaboration, not competition

It is important that Drupal distributions collaborate, and not compete. To do so, we have to provide Drupal distributions an environment that encourages collaboration, and that allows for specialization (such as custom documentation and support) without introducing incompatibilities that drive competition.

The good news is that we know how to do this. We've been through this already with CivicSpace (previously called "DeanSpace"), a Drupal distribution for online campaign management and grassroots activism. They were quick to realize that the success of the CivicSpace distribution depends on the success Drupal core, and vice versa. The decided they shouldn't fork core development. Instead, CivicSpace decided to do all its development on the drupal.org infrastructure, to synchronize releases, to submit all patches upstream, to centralize bug reports, and to share documentation where possible. Collaboration, not competition.

The bad news is that can be hard work. People will find that creating a distribution is fun and easy, but that being a responsible maintainer might be a lot less fun. Who wants to track changes, write documentation, maintain modules, provide upgrade paths, manage releases and provide support for years to come?

So how can we can encourage distributions to collaborate, and how can we avoid users getting stuck with an old Drupal version because a distribution is not maintained properly? Remember the "Drupal for Bloggers" distribution? Hundreds of users had no upgrade path and were stuck with an insecure Drupal fork. The author decided to compete rather than to collaborate, and introduced various incompatibilities that, in the end, negatively affected the users. It is a good example of what we want to avoid.

So here is a simple rule: don't create a distribution because you can. Create a distribution because you want to provide users a service. If you don't want to collaborate or if you can't commit to providing a service around your distribution, you are likely to do more harm than good.

We also have to keep in mind that there will never be a perfect solution. Even if a distribution is maintained properly, consider this simple fact: no distribution will be able to provide timely bug fixes, and no matter what you do, users will always complain that it takes too long to roll a new security or bugfix release.

What exactly the solution is going to look like, I don't know, but I think the short answer is: community responsibility. We can't implement a technical solution for what might be a social issue. This is a community challenge that needs a community solution.

As a community we should disapprove Drupal distributions that do not intend to collaborate, that have no signs of long term commitment, or that risk locking people in. It might involve new and bigger growing pains but so far, we have done amazingly well figuring these things out -- and that is reflected by our healthy growth.

Fragmentation and natural selection

Things might be more subtle than that. Even if we collaborate and all Drupal distributions are built on the exact same Drupal core, there will still be incompatibilities in terms of documentation, support and vision. After all, most Drupal distributions will want to provide specialized documentation and support. Being able to create this kind of value is the very reason why distributions exist.

These incompatibilities can create an incentive to compete. Just look at what is happening with Debian and Ubuntu. Ubuntu began as a fork of Debian with the aim of drawing from Debian's code regularly in order to allow for frequent releases. So Debian diverged and Ubuntu was created. Now, they are driven more and more apart by these incompatibilities, and as a result, Debian risks being superseded. The same is happening with Mambo and Joomla, with NetBSD and FreeBSD, already happened with Unices and Linux, and might happen with RedHat and Oracle.

If we are not careful, Drupal risks suffering a similar fate: splintering into multiple competing distributions. It is a slow process and often takes many years (and is therefore often disregarded by the people who are too closely involved). The good news is that this isn't necessarily a bad thing because ultimately more people win. It is how nature works and it is called "mutation" and "natural selection".

That said, we should disapprove distributions that are created out of frustration (eg. a "Drupal Pro" distribution or a "Drupal Lite" distribution whose main reason of existence is driven by frustration). The difference between a distribution and a fork can be a subtle. Listening to our users and changing the way things work should continue to be the preferred answer to user frustration. As Darwin said, it's not the strongest organisms that win, it's the most adaptable to change.

Distributions

Conclusions

The fact that Drupal 5.0 will support distributions is big, and most people have yet to see its full potential. I don't think that any other Open Source project has done something like this before -- or at least, not on the scale that we might end up doing this. Time will tell. It is going to be an exciting change, and I'm confident that we'll be able to untap the incredible potential.

But as a community, we have to take responsibility, and make sure that distributions collaborate rather than compete. We also have to make sure that we disapprove distributions that are not properly maintained. Community responsibility will become ever important as we move forward with this awesome adventure. I hope it all works out.

Comments

Jan (not verified):

These is both great news and a challenge. I would very much love to collaborate on an e-learning distribution, and if I'm weak on code I can throw on the table my master degree in e-learning!

October 30, 2006 - 18:48
Jeff Eaton (not verified):

I think you've correctly identified a lot of the risks, Dries. Though I think some of the changes that went in late in the game with 5.0 give us a good chance of avoiding them.

A good Drupal distribution will not, IMO, remove anything from core or change any core files. It will only be *additional* modules, themes, and setup data. It's possible for all of that information to be kept in the /profiles directory, separate from both core code and custom site-specific stuff in the /sites directory...

The danger of forking with this scenario is a lot less than it could be...

October 30, 2006 - 18:51
DaveNotik (not verified):

Very well put, Dries. Your keen insight and soft guidance make you a great father figure for Drupal. :)

Looking forward to a new era in the life of Drupal!

October 30, 2006 - 19:06
Anonymous (not verified):

Might an easier way to do Drupal distributions start with simplifying core? Example: Detach the forum into a separate module. Detach the statistics/logs module. Detach everything that isn't "inherent" to doing things the Drupal way.

This way, core moves faster and the core focus is more powerful at implementing new pieces of functionality that benefit all mods and distributions.

What Drupal does best (and is of most benefit to all) is to provide functionality that is extendable and configurable. Why include a forum? Why try to be called a CMS? Of course it will be all that, cause the community will provide, as it has thus far.

October 30, 2006 - 20:24
Bèr Kessels (not verified):

Well said! Avoiding fragmentation is indeed a big challenge. And you state it very well that frustration and "because you can" are two bad reasons for maintaining a distro.

There are some more bad reasons:

  • "Just to try". It is not bad a experiment, but don't present your experiments as solutions. :) We'll end up with loads of duplicate, half-witted distros that do Drupal more harm than good.
  • "Because Foo distro has no included module Bar". Please collaborate. Don't create a new distro, just because you don't like a tiny part in another distro.

I also think that having the infrastructure ready is very important: how to distribute distributions? On your own SVN? On Drupal.org? As a tarball on your own website?

Last but not least, is to make sure that the "natural" selection is visible. Either in very visible download stats, or as some rating system. Modules on Drupal.org don't have the natural selection visible. The only way to get a feeling about that selection is to follow the 'buzz' on IRC, the forums and so on. I really hope that either drupal.org or some external site (drupaldistrowatch.com anyone?) offers a good visible way of browsing trough a natural selection.

October 30, 2006 - 20:53
Farsheed (not verified):

I think the first step is to create working groups around distribution services or ideas. A great place to do this is http://groups.drupal.org/. There people can introduce several ideas or variations for Drupal distributions for their market. Most likely there will be slight variations on the same theme. But it's important to target a focused audience of people with a distribution and concretely identify why and how this group can benefit from it. Trying to provide a technological solution for a community can actually introduce MORE problems if the author doesn't research and listen to what the community needs are. We should try and identify what problems need to be solved first before trying to apply a technological solution, otherwise, the community will not see the value in a solution without a problem. As you say, collaboration on all fronts is key.

October 30, 2006 - 21:56
Erik B-W (not verified):

Checkout the Drupal Distribution Profiles group.

January 16, 2007 - 08:43
xamox (not verified):

That is awesome. I was talking to someone exactly about this today in #drupal-support. The one I discussed was a Darknet version though. You can think of it as a VPN with Drupal. Basically for sharing files with encryption. It would need Drupal with SSL support and a way to encrypt data on the host. I haven't really looked into how it would get accomplished but it was an idea I had.

I agree that Drupal needs an "out-of-the-box type of solution". I'm sure it would persuade many more people to move to Drupal. With a basic Drupal core install (4.7) the system seems very bland and I don't think people get to understand the power of Drupal as opposed to a base install of Joomla or Mambo. I'm sure if you compared the 2 side by side with a default install it would appear that Joomla blows the doors off Drupal, but like I said people don't get to see the inner workings and know the power of it.

October 30, 2006 - 23:37
Bill Fitzgerald (not verified):

Hello, Dries,

As others have noted, you do a nice job laying out the advantages and pitfalls.

Collaboration is essential, as is the commitment not to fork core code or contrib modules.

As we discussed in Portland at the Rose and Raindrop, I'd be glad to play a role in the Drupal for Education distro --

I look forward to helping with these developments.

Cheers,

Bill

October 31, 2006 - 00:11
Robin (not verified):

You might also want to look at how Debian maintains custom versions while preventing forks (whatbuntu?). They've been thinking about this particular problem for a few years now.

October 31, 2006 - 04:21
Max Bell (not verified):

Indeed. Maybe a group like this would make sense?

This is, as usual, a very excellent set of observations, Dries -- and hopefully my little group won't prove to be too redundant, but I see a lot of opportunity in this development and have been looking forward to it for some time. You're very right that cooperation between developers needs to be emphasized from the very beginning and besides just creating a group, I'm going to try to keep tabs on what kind of distribution profiles are being considered and developed, and post about them myself if no one else does.

October 31, 2006 - 04:43
Dominik Lukes (not verified):

Very interesting analysis of community collaboration (quite apart from the Drupal things). On things Drupal I have a few comments. I think that one thing that will make this whole thing easier is a module versioning system (in the works I hear). It would take much of the sting of maintaining a distribution.

I'd also like to resurrect my old idea of having three rather then two tiers of modules. 1. core (very few) 2. core supported (a few more whose continued functionality would receive support and attention from core developers) 3. contributed (all remaining). That way, a maintainer of a distribution would not have to worry about obsolescence so much. I love Drupal 5 but can't see upgrading my sites to it any time soon because the current installations are so dependent on specific module functionality.

I'm planning on creating a several private distributions (I'm running too many sites that are almost identical and took a day instead of an hour to set up). I will probably never have the skill or frankly the time to be a long-term maintainer but I also like to share. Therefore I'd like to suggest another possible analogy. That of VMWare Virtual Appliance, which is basically a distribution of an operating system with different things preinstalled. There is no suggestion of forking because it depends on the VMWare player/Drupal engine but many can be easily created that differ only slightly. If there was some way to get a repository of these that is separate maybe more people would contribute and the pressure would be less. For instance, I'm in the process of setting up an elaborate site for a University department - if I could turn it into a Drupal appliance (distro) with a few clicks (or even a day's work), I would happily share that with the world. If I'm asked to create a distribution and maintain it, I will probably never get around to maintaining it. Imagine links on Drupal sites: Set up a similar site, download a template.

October 31, 2006 - 09:56
Roel De Meester (not verified):

Good points. An example of what this "because you can" attitude results in ... have a look at RBuilder online (RBuilder allows you to 'easily' create a custom Linux distribution including applications).

You'll see 4 unfinished Linux distributions that want to provide a Drupal experience out-of-the-box. None of them have ever released a distribution. Even worse, only 2 out of 4 wrote some code. No collaboration, no commitment, no product!

That is definitely NOT the direction that Drupal 5.0 should go in, and I agree that technical tools can only help to some extend to prevent such things from happening. Convincing people to collaborate and contribute to core is far more effective to reach well-maintained stable distributions.

October 31, 2006 - 10:28
Dominik Lukes (not verified):

You are probably right. Actually, I'm responsible for creating one of those abortive Drupal appliance projects. I wanted to make one for myself but it was more difficult than I thought plus I discovered Drupal on a Stick. But as I say, I'm mostly interested in being able to share as much of what I do as is possible for a non-coder like me.

November 1, 2006 - 02:32
Peter Dedecker (not verified):

Wow, great! A "Drupal for Student Organizations" would be very, very nice!

October 31, 2006 - 13:31
Caleb G (not verified):

My own long-winded reply to Dries article here to the Drupal forums. (some of it is along the same lines as Dominik, whose comment I didn't read until after I posted there..)

November 3, 2006 - 21:33
Kieran (not verified):

I am reposting my comments to the Drupal infrastructure mailing list here, as I feel it is relevant to this discussion.

Drupal has been behind a curve for a while. While winning in elegance of programing and architecture it's failed to attract an increasing important part of the talent market place, designers.

With Drupal 5.0 we can catch up in attracting designers due to usability improvements and ease of use. See this quote:

"I am happy to say that I found the Drupal two click installation as painless and simple as a Wordpress installation." (source)

However, projects like Wordpress still overwhelmingly attract designers in an order of magnitude greater than Drupal. With profile and distribution support we have a chance to give designers the place to express themselves. Welcoming profiles and distributions is a solid step forward in attracting these designers. Attracting these
designer will give Drupal a competitive advantage in ensuring that we have the talent on our team to win in the competition of making it easy to publish content and applications to the web.

Wordpress is hard at work making their plugins more powerful. Joomla is hard at work building a new installer and an application node system. Google is hard at work making their home page plug-ins and applications better integrated, and more powerful.

But Drupal can pull ahead, if we attract hundreds of designers to build these profiles and ultimately distributions. Over the last couple of years, dozens and even hundreds of Drupal module developers have been able to quit their day jobs and work on Drupal full time delivering solutions to their clients, because of their showcased
work in Drupal module contributions. We should be very proud of this. But we can go further. Hundreds of designers can come to Drupal.org and start showcasing their talent, if we give them a place to do so.

Will their be bad distributions? Yes. Will their be great distributions? Yes. But more importantly, Drupal distributions will attract hundreds of designers who will help us create a great user experience.

Kieran
CivicSpace

November 4, 2006 - 19:44
Jose A. Reyero (not verified):

Drupal has been behind a curve for a while. While winning in elegance of programing and architecture it's failed to attract an increasing important part of the talent market place, designers.

I think this is a serious problem and the main reason why this happens is that we are failing to provide some PHP-free and more or less standard templating system.

I really don't share the general enthusiasm about phpTemplate, as it is nothing else but more PHP code. And as long as we don't strongly support any real templating system -- by 'real' I mean no PHP code in it and manageable with graphical design tools -- it will be more and more difficult to have pure graphical designers creating Drupal themes.

November 13, 2006 - 12:04
Jose A. Reyero (not verified):

About distributions I really don't see that bad if people want to fork Drupal core for specific distributions, as long as they provide the patch to be considered for core inclusion which for sure will be in their best interest. Then we'd have some interesting already tested features just waiting for inclusion into core.

The thing is that, while I think this could benefit Drupal, it will add a good deal of additional work for the distribution maintainer. And I would just ask these people to add some kind of warning about it in the same line in which they mention Drupal.

In the meanwhile we could somehow promote Drupal distributions with unpatched core having an official list somewhere in drupal.org.

So the official policy may be something like "All distributions are welcomed. Only distributions with no core patches will be promoted from drupal.org. If you want to add core patches, that's ok, but you do it at your own risk and please warn the users that you are running a Drupal fork."

Some competition will be a good driver for innovation and some core patches may prove being worth for inclusion after they've been field tested in some distribution. So it's really the distribution maintainer who's taking the risk and somehow committing for extra work, while Drupal itself could benefit from that.

The same may apply for forked contributed modules.

November 13, 2006 - 12:26
zoon_unit (not verified):

I think the key to maintaining a Drupal distribution will come from an automated update system, similar to Ubuntu. If a user could run an auto update to keep his or her Drupal distribution current, then the distro maintainers would have much less work to do. If the distro updates itself, then the original distribution will not need to be updated nearly as often.

An auto-update feature would be invaluable to all Drupal developers. (the system should allow users to mark modules to be updated automatically, in case a developer has customized one or more modules)

I also believe that a strong focus on these three modules could simplify and strengthen Drupal's expandability:

1) CCK, 2) Views, & 3) Panels. Expanding the power of these development packages would reduce the need for custom modules. CCK and Views should have powerful import and export capabilities. That way, instead of installing a new module to enable some custom page or view, the user simply imports a predefined layout. This would also ensure that all data travels properly through Drupal's core security and caching system, instead of leaving this up to the skills of the module developer.

Combining automated module updates with more powerful development modules like CCK and Views would simplify and strengthen Drupal's capabilities at the same time.

December 14, 2006 - 18:11
Lopo Lencastre ... (not verified):

This approach that 5.0 will have is exactly what we were planning but were expecting a huge pain to get there just starting from a plain CivicSpace-based system striped out from what didn't interest to us.

Thanks for going in that direction. It will ease our task a lot and let us being concentrate in what is not CMS related.

January 9, 2007 - 21:22
Erik B-W (not verified):

Drupal Groups is nice, but what about Drupal Distributions?

I'm enjoying the way Drupal Groups is helping me connect with people interested in similar target markets, like school alumni associations and churches, but find that the organic groups construct is good for only basic discussion and organizing, and not for the real work of organizing a specific Drupal distribution. I began to create a structure for thinking about an Alumni distribution in the Alumni Group, but there's just too little flexibility to do it there. That's no fault of organic groups, it's just not what they were intended to do.

What do people think about creating something like a Drupal Distributions site, specifically designed to facilitate the development of all the different possibly distribution interest groups? This common "distribution assembly infrastructure" would, IMHO, improve the odds of collaboration over competition and also increase the consistency of how distributions are developed and released.

January 16, 2007 - 09:02
Anonymous (not verified):

Dunno, Drupal is more powerful but I still prefer Wordpress - it's simpler and more widespread - that means more plugins and themes.

May 6, 2008 - 13:30
root internet d... (not verified):

When I was selecting a CMS system for multiple sites I have checked out Drupal, but it seemed a lot more cumbersome to install and manage than Joomla, but I think that has all changed now.

May 15, 2008 - 01:13
Cosmetic dentis... (not verified):

Hey Thats excellent.... a collaboration of different peoples strength would create such a synchronization which would create great results across the board.

Excellent work there....i would also look forward to collaborate going further.

Thanks

March 6, 2009 - 17:59
raj41pur (not verified):

We are working on a community-powered news, rating and review service dedicated to helping you find Drupal distributions!

http://drupaldistrowatch.com/

July 31, 2009 - 13:37

Add new comment

© 1999-2014 Dries Buytaert Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Drupal is a Registered Trademark of Dries Buytaert.