State of Drupal presentation (March 2009)

Last week at DrupalCon DC I gave my traditional state of Drupal presentation in front of 1400 Drupalistas. The video of the presentation is provided below, and you can download a copy of my slides (PDF, 20 MB) as well. The video is available in alternative encoding formats from archive.org. Topics I talked about: the history of Drupal, the Drupal 7 release, the future of Drupal, etc. Have a look!

Source: archive.org.

My predictions for 2009

It is that time of year again. Time to reflect on 2008, and to put on my Drupal Nostradamus hat and look forward to 2009. But first of all, thanks for 2008! It's been a pretty crazy ride.

Drupal

My personal Drupal highlights for 2008 include the Drupal 6 release (the best Drupal release ever!), both DrupalCon Boston and DrupalCon Szeged, the Drupal.org redesign that is in progress, and, of course, beating Joomla and Wordpress at the Packt awards. ;-) As I predicted last year, more than ten books were written about Drupal, compared to a single book in 2007. The increase in Drupal books is another highlight as I actively helped connect authors to publishers. I truly enjoyed being part of the Drupal community in 2008.

My personal low for 2008 is regret that some key modules lagged behind the Drupal 6 release. The majority of these modules have now been released, and Drupal 6 is finally getting on the fast lane now. The message is clear: we'll continue to see tremendous growth and adoption in 2009.

Why?

  1. Drupal 6 is easier to use, runs faster, and comes with many great new features. The work we did on Drupal 6 throughout 2007 and 2008 will pay off in 2009.
  2. Economic pressure will help accelerate Drupal's growth, and that of Open Source in general. More site owners will discover that with Drupal, you can build a better website cheaper than with many of its proprietary counterparts.
  3. Social publishing (blogs, forums, wikis, social networks, etc.) will become more pervasive and continue to make inroads in organizations seeking to facilitate collaboration between teams and departments. These applications, while nothing new, make many aspects of business better, are here to stay, and will mature over time. Drupal continues to be in that sweet spot.

I'll continue to have a software love affair with Drupal in 2009. At the moment, I'm very excited about the community's growing interest in the semantic web -- and all the related interoperability and decentralization technologies. The seed of what I hope will become a strong marriage between Drupal and semantic web technologies was planted in my DrupalCon Boston 2008 keynote in February (with the help, hard work and preparation of many others), and will continue to grow in 2009. Drupal continues to be a technology pioneer in 2009.

I predict that Drupal 7 will be released in the fourth quarter of 2009. The two most exciting features in Drupal 7 core will be custom content types and radical improvements in usability. To reduce the risk of important modules falling behind in support or update path, a significant portion of the Content Construction Kit (CCK) related modules will move to core and we'll pave the way for the Views modules. The same holds true for other important contributed modules, including token module, path auto module, and image handling functionality. In 2009, core becomes bigger, not smaller. The Drupal 7 code freeze will be longer than expected regardless our new continuous test framework, and the upgrade path to Drupal 7 will be more painful than hoped for. But like always, we'll come out stronger than before ...

Despite Drupal being loved by many, we'll have to work hard in 2009. The thing that holds Drupal back is failure to execute many of the ideas and plans that we have. As a community, we need to grow more mentors in 2009, and we must all make sure that they are set up for success rather than failure. The community's responsibility to itself should be to remove barriers to participation and single points of failure. Alarm bells should go off when there is a desire to introduce red tape, unnecessary hurdles or dependencies, or when we fail to collaborate and make progress in key areas of the project. At the same time, we need to help more Drupal companies figure out how to contribute back to Drupal in substantial ways. Contributions are gold, talk is silver. Helping people contribute must become platinum.

Last year, I predicted that we would see the first signs of consolidation in the Open Source CMS market. I believe that prediction was correct. The "big three" (i.e. Wordpress, Joomla! and Drupal) continued to grow in 2008, while many of the other systems faded into the background a bit. I think that trend will continue in 2009. In the long run, the winners will be platform providers that enable people to connect, create and share value in different ways -- and that do so with the lowest barrier to entry. Expect other systems to (continue to) attack Drupal from both below and above. We're the best platform today, and others will have to move in to stay viable.

Oh, and IBM starts to embrace Drupal in 2009!

Acquia

I'm proud of Acquia. Acquia is the Drupal company that I started with Jay Batson. We announced the start of Acquia at the end of November 2007, and we announced our funding just before the end of 2007. People had a lot of questions about Acquia early in 2008, but throughout the year we demonstrated over and over again that we're committed to Drupal's success and that we want to do what is right for the community. We built a great team and grew from 2 employees early in the year to 30 people today. In September 2008, we launched our first products and started to offer commercial support for a defined software distribution called Acquia Drupal. Today, 3 months after we opened to doors for business, we are serving customers. We worked hard and made our milestones. It has been fun to see a new business take off. I also racked up way more frequent flyer points (i.e. air miles) than what is generally considered healthy.

The first thing you learn when selling in tough economic times is that you must figure out how to give customers exactly what they want and you must do it fast. It didn't take long for us to realize that people wanted more than Acquia Drupal: they wanted support for everything Drupal 6.x -- all modules, themes and custom code. The good news is that Acquia is a nimble company so the last weeks we worked on changing our support model to address customer demands. Starting tomorrow, we will support everything Drupal 6.x -- not just Acquia Drupal but all modules and themes available on drupal.org as well as custom code. I'm still a firm believer in Drupal distributions so Acquia Drupal still has a role as a packaged on-ramp for people getting started with Drupal. However, anyone will be able to connect any Drupal 6.x site to the Acquia Network -- helping us achieve our goal of helping people build and operate great websites with Drupal. Keep an eye on acquia.com if you want to learn more about these changes.

We're passionate about getting our value proposition right, so expect us to continue to tweak and extend our current offering in 2009. We'll also launch a number of new products. Some, like our hosted search service, we've already talked about, and I think we'll finally be ready to talk about a few others in the first quarter of 2009.

Regardless of the down-turn in the economy, I think that Acquia's business will continue to take off nicely in 2009. My heart and gut are convinced that Acquia has a tremendous opportunity to do well, and to do good. I believe (and hope) that Acquia will have the success it takes to continue to invest in Drupal.

Mollom

Together with Benjamin Schrauwen, I also launched Mollom, a web service whose purpose is to dramatically reduce the effort of keeping websites free of spam and the quality of user-generated content high. Mollom is a self-funded company and nowhere near the size or scope of Acquia (Acquia is my full-time commitment) but nevertheless, a lot of progress has been made. We announced Mollom in March, and opened the doors for business at the end of September 2008. Today, we're actively protecting 4,500 websites of which 75-100 have paid subscriptions. Mollom has caught almost 21 million spam messages since it started.

In 2009, I predict that Mollom will continue to experience steady growth and that we'll introduce a premium subscription (i.e. "Mollom Premium" in addition to "Mollom Plus" and "Mollom Free") with enterprise level features. I also predict that our efficiency in blocking spam will raise from our current 99.88% (i.e. 12 in 10,000 spam messages were not caught) to 99.95% or more (i.e. 5 in 10,000 spam messages or less were not caught). While this might sound like a marginal improvement, it actually means we make 2.4 times fewer mistakes.

Mollom has a ton of potential and is great fun, so I have all reasons to believe that 2009 will be a good year for Mollom. If fact, I predict that 'good' will be an understatement.

Conclusion

2008 was a great year, and continues Drupal's great run. The economic realities of 2009 will present challenges, but also opportunities. I believe Drupal's success will continue -- and accelerate -- in 2009, though we'll have to work hard. I predict we'll do exactly that.

Open Source and Free Puppies

Seth Gottlieb reported that Annie Weinberger of Interwoven, a proprietary CMS vendor, launched some good old Open Source FUD comparing Open Source to a free puppy:

"We look differently at the cardboard box full of free puppies outside the super market once we become adults. As children what could be more fun than to get a puppy who is going to be your friend for life? Why not mom…it’s FREE!! But as adults we have learned the truth. We know that taking home that puppy is going to cost us in the end. The free price tag hides all the costs we are going to spend on food, training, shots, and a new couch once the puppy discovers you are not coming home at 5:00 every night to walk him. Open source WCM solutions are very similar. The free price tag is attractive at first, but for online strategies that have multiple initiatives (intranet, extranet, portal, landing pages, micro-sites, etc.), the hidden fees lie in the heavy customization, maintenance and engineering work."

Puppy analogies -- especially those with free puppies -- are powerful stuff.

Is Open Source more expensive than proprietary systems? It depends. You can't generalize. Open Source implementations can be more expensive if you try to bend the software too much. However, you don't have to be a genius to understand that because there are no licensing costs, Open Source has the potential to be much cheaper than proprietary solutions, and that Open Source solutions come with freedom and flexibility not found in proprietary products. Implementation cost is an important factor, but it is in providing freedom and flexibility that Open Source wins and commercial vendors lose. Open Source puppies are "free" as in "free speechbark".

One thing is for sure: puppies attract attention; these days, Open Source does as well and proprietary vendors tend to be of the jealous type.

The great thing about FUD, though, is that it validates our work in the Open Source community. Blog posts like Annie's trigger the competitive gene in hundreds of Open Source developers around the world, and in the end, makes Drupal stronger.

A free puppy, anyone?

The freemium business model: giving away pays

I've recently been thinking a lot about the freemium business model. For those unfamiliar with the freemium business model, it was first articulated by venture capitalist Fred Wilson in 2006:

"Give your service away for free, possibly ad supported but maybe not, acquire a lot of customers very efficiently through word of mouth, referral networks, organic search marketing, etc., then offer premium priced value added services or an enhanced version of your service to your customer base."

I've been thinking about the freemium business model because, inspired by Drupal and Open Source, both my companies, Acquia and Mollom, use a freemium business model. (Technically, Acquia uses an Open Source business model which is different from the freemium business model, but there is plenty of overlap and similarities -- pointing out the differences could be a blog post and discussion on its own.)

At Acquia, we currently provide community subscriptions for free -- people that want help with Drupal installation and configuration can get free support from Acquia's Drupal experts. While our free support is limited to certain channels (i.e., forum only), certain support questions (i.e., no module development help and no security best practices) and comes without response time guarantees, we have people on staff whose full-time job is to help you (example customer story). Further, we invest heavily in Drupal and give those contributions away for free.

Similarly, at Mollom, our basic spam filtering service is available for free to sites with limited post volumes. Our free website protection service provides all the features of our commercial Mollom Plus product, but is limited in the number of posts it will protect each day and in its access to our high-availability back-end infrastructure. The great majority of our Mollom clients are using our free filtering service with great success.

There are a number of things that attract me to the freemium business model. The first, and certainly foremost, is the opportunity to do “good” and “well” at the same time. It’s a great thing to help people build quality websites with Drupal, and it’s a great thing to provide Mollom to help deal with spam. Second, I believe a company is better off with a large install base than a small install base, even if the majority of clients ride free. A large install base translates to direct and indirect network effects, including efficient marketing, greater brand awareness, the collective intelligence of your users, and faster product adoption. And, last, I strongly believe that a successful company built on the freemium business model is simply a stronger and more defensible business in the long run.

The freemium business model is relatively new because it didn't become a serious option until the internet gave us a low-cost distribution channel. Ultimately, I can't help but think the freemium business model is the business model of the future for the sole reason that it puts the customer first. With the freemium business model customers only have to pay when they get significant value from the software (i.e. they have reached the limits of the free version). Compare this to the current model where people have to pay to get access to the bits, or where people have to pay before they got enough value from the software (e.g. most shareware software).

That all sounds great but you have to make the freemium business model work first. Getting free users to convert to paying customers is hard. Conversion rates of less than 1% are not uncommon. Free is often “good enough” and only a few people choose to pay for additional features and services. You have to put enough value in the free version to drive adoption (so that you get the scale and the network effects that derive from it), while providing enough incentive for people to pay for premium features or services. The marketing and sales funnel is really wide at the top, and very narrow at the bottom. Plus, you have to make sure that the paying users subsidize all the free users.

Achieving the right balance between free and paid customers is difficult and requires close attention to a number of variables. As a result, I've been trying to answer questions like: how much should we invest to acquire additional free users? How do you estimate the value of a free user? What is the cost of a free user? How long does it take for a free user to convert to a paying customer, and how many will do so? What are the triggers that convince free users to convert?

For example, in Mollom's case, one could argue that we get thousands of dollars worth of value from free users already. We currently have more than 3,000 active users that use Mollom for free. Say each user spends on average 15 minutes a week moderating his site's content and reporting classification errors to Mollom. Mollom learns from this feedback and automatically adjusts its spam filters so that all other Mollom users benefit from it. At a rate of $10 USD/hour, we get $390,000 USD worth of value from free users a year -- 3,000 users x 15 minutes/week x 52 weeks/year x 10 USD/hour = $390,000 USD/year. If these numbers hold up, the value of a free Mollom user could be estimated at $130 USD/year. And that doesn't include the marketing value they add. That said, the value of a free user probably declines as you get more of them and the business becomes stronger.

Both Acquia and Mollom have just opened for business so we have a ton to learn. It will be interesting to look at the different variables and questions a year from now, and to see what we have learned. I hope we can make it work so we can do good and well at the same time ...

Drupal in the cloud

It is not always easy to scale Drupal -- not because Drupal sucks, but simply because scaling the LAMP stack (including Drupal) takes no small amount of skill. You need to buy the right hardware, install load balancers, setup MySQL servers in master-slave mode, setup static file servers, setup web servers, get PHP working with an opcode cacher, tie in a distributed memory object caching system like memcached, integrate with a content delivery network, watch security advisories for every component in your system and configure and tune the hell out of everything.

Either you can do all of the above yourself, or you outsource it to a company that knows how to do this for you. Both are non-trivial and I can count the number of truly qualified companies on one hand. Tag1 Consulting is one of the few Drupal companies that excel at this, in case you're wondering.

My experience is that MySQL takes the most skill and effort to scale. While proxy-based solutions like MySQL Proxy look promising, I don't see strong signals about it becoming fundamentally easier for mere mortals to scale MySQL.

It is not unlikely that in the future, scaling a Drupal site is done using a radically different model. Amazon EC2, Google App Engine and even Sun Caroline are examples of the hosting revolution that is ahead of us. What is interesting is how these systems already seem to evolve: Amazon EC2 allows you to launch any number of servers but you are pretty much on your own to take advantage of them. Like, you still have to pick the operating system, install and configure MySQL, Apache, PHP and Drupal. Not to mention the fact that you don't have access to a good persistent storage mechanism. No, Amazon S3 doesn't qualify, and yes, they are working to fix this by adding Elastic IP addresses and Availability Zones. Either way, Amazon doesn't make it easier to scale Drupal. Frankly, all it does is making capacity planning a bit easier ...

Then comes along Amazon SimpleDB, Google App Engine and Sun Caroline. Just like Amazon EC2/S3 they provide instant scalability, only they moved things up the stack a level. They provide a managed application environment on top of a managed hosting environment. Google App Engine provides APIs that allow you to do user management, e-mail communication, persistent storage, etc. You no longer have to worry about server management or all of the scale-out configuration. Sun Caroline seems to be positioned somewhere in the middle -- they provide APIs to provision lower level concepts such as processes, disk, network, etc.

Unfortunately for Drupal, Google App Engine is Python-only, but more importantly, a lot of the concepts and APIs don't map onto Drupal. Also, the more I dabble with tools like Hadoop (MapReduce) and CouchDB, the more excited I get, but the more it feels like everything that we do to scale the LAMP stack is suddenly wrong. I'm trying hard to think beyond the relational database model, but I can't figure out how to map Drupal onto this completely different paradigm.

So while the center of gravity may be shifting, I've decided to keep an eye on Amazon's EC2/S3 and Sun's Caroline as they are "relational database friendly". Tools like Elastra are showing a lot of promise. Elastra claims to be the world's first infinitely scalable solution for running standard relational databases in an on-demand computing cloud. If they deliver what they promise, we can instantly scale Drupal without having to embrace a different computing model and without having to do all of the heavy lifting. Specifically exciting is the fact that Elastra teamed up with EnterpriseDB to make their version of PostgreSQL virtually expand across multiple Amazon EC2 nodes. I've already reached out to Elastra, EnterpriseDB and Sun to keep tabs on what is happening.

Hopefully, companies like Elastra, EnterpriseDB, Amazon and Sun will move fast because I can't wait to see relational databases live in the cloud ...

State of Drupal presentation (March 2008)

Last week at DrupalCon Boston I gave my traditional state of Drupal presentation in front of 850 Drupalistas. The video of the presentation is provided below, and you can download a copy of my slides (PDF, 15 MB) as well. The video is available in alternative encoding formats from archive.org.

Source: archive.org.

Topics I talked about: the Drupal 6 release, the state of our union, the need for a drupal.org redesign, the Drupal 7 killer release, the Drupal 7 development cycle, usability, test-driven development, the future of Drupal and the semantic web, etc. There is a lot of material in this presentation and during the course of the next few weeks, I plan to decompose this presentation in a number of extended blog posts. Stay tuned!

My Drupal predictions for 2008

My personal Drupal highlights for 2007 include the Drupal 5 release, bootstrapping the Drupal Association, the two Drupal conferences we organized, the Pro Drupal development book, and co-founding Acquia.

But more than anything else, I enjoyed being part of the Drupal community and helping it navigate through some of its growing pains. Thanks to you, 2007 was a blast.

In good tradition, here are my Drupal predictions for the next year.

Growth predictions for 2008

First, let's predict Drupal's growth in 2008. The short answer is that Drupal will continue to grow more, not less.

Much of Drupal's growth in 2008 will depend on the work we did in 2007. I'm extremely happy with the upcoming Drupal 6 release as it will be easier to use, it will run faster, and it will come with some great new features. Our work on Drupal 6 will pay off in 2008.

Many metrics can be used to predict Drupal's growth in 2008, so let's use a non-conventional one: to date, four books have been published on Drupal, but only one of these was published in 2007. In 2008, ten Drupal books will be published ...

What I care most about is not Drupal's growth, but that we will continue to democratize web publishing and web development in 2008. By growing Drupal and giving it away for free we accomplish two things: (i) we empower more individuals to publish online and (ii) we help grow a successful ecosystem that allows more people to make a living with web development. So not only will Drupal continue to grow in 2008, it will continue to make a positive change.

The more you give away, the more you get back, and because of this, working on Drupal will continue to be a labor of love in 2008.

Market predictions for 2008

We are still in a young market: there are hundreds of Open Source CMSes, there is no real competition amongst them ("We're all colleagues and friends!"), not to mention we all benefit from what seems unwieldy growth. (Note that I'm talking about the Open Source CMS market, not the proprietary CMS market.)

However, near the end of 2008, we'll see the first signs of consolidation in the Open Source CMS market. The Open Source CMS space will become less fragmented; the "big three" (i.e. Wordpress, Joomla! and Drupal) will continue to grow but the growth of many other systems (i.e. Plone, Typo3, Xoops, e107, ezPublish, dotNetNuke, etc) will slow down significantly.

The good news is that the Open Source CMS market becomes easier to shop in. The bad news is that there will be a competitive edge.

Unless we manage to put more effort into (i) marketing, (ii) support, (iii) documentation and (iv) drupal.org this might turn out to be a tough battle for Drupal. Drupal.org will be our biggest challenge in 2008, and much of that will determine whether we'll be one of the "big three" Open Source CMSes at the end of 2008.

End-user predictions for 2008

From an end-user's point of view, 2008 will be characterized by the fact that we'll continue to give our users what they want. There will be a big and concentrated effort to further improve Drupal's ease of use. As a result, Drupal 7 will ship with one or two install profiles, many UI improvements, more AJAX, a basic WYSIWYG editor (or better WYSIWYG support), some wizards, and improved image and file handling. Yes, that is a lot.

Developer predictions for 2008

While we listen to our users in 2008, most of the excitement will be developer-centric.

A significant portion (but not all) of the Content Construction Kit (CCK) will move to core and we'll pave the path for Views by extending Drupal 6's query builder. There are three important motivations for this: (i) the desire to write less and less code to improve developer productivity, (ii) the desire to reduce the risk of these modules falling behind in terms of support and updates and (iii) drupal.org becoming dependent on them.

Integration of the CCK and Views will trigger a strong focus on improving our internal data models and APIs. While unheard in Drupal circles right now, object-relational mapping (ORM) will be the buzzword du jour by the summer of 2008. This, in turn, will lead to better web service integration, RIA integration (specifically Flex), and improved import/export functionality in Drupal 7.

The desire to reduce risks, combined with drastic API changes and a growing code base, will lead us to adopt a test driven development methodology. Drupal 7 will ship with some first regression tests.

All in all, the net result is that Drupal 7 will be an even better web application development platform. Comments and users as nodes continues to be a pipe dream though.

My final prediction is that I will get all of this year's predictions right, but that you still want to get a second opinion.

State of Drupal presentation (September 2007)

Robert Garrigos from the Drupal Catalan user group just uploaded a video of the State of Drupal presentation that I gave at DrupalCon Barcelona 3 months ago. The presentation shares the results of a survey that I conducted in August; the survey ran for 30+ days, collected more than 1000 responses, and should do a pretty good job capturing the Drupal Zeitgeist.

Have a look at the video below (my presentation starts 12 minutes into the video) and/or download a copy of my slides (PDF, 14MB).

Gartner on pricing pressure

In a research note, Gartner predicted what most people already knew: proprietary software is going to face serious pricing pressure from a range of different software trends. Specifically, Gartner identified seven trends putting pricing pressure on software business models:

These include business process outsourcing; software as a service (SaaS); low-cost development environments, such as China and India, combined with modular architectures and service-oriented architectures; the emergence of third-party software maintenance and support; growing interest in open source; the rise of Chinese software companies; and the expansion of the Brazilian, Chinese and Indian markets.

I wonder how many executives took a poop after reading Gartner's research note ... Not too many, you would hope.

Anyway, it looks like Drupal is in good shape: it is open source software with a modular architecture that lends itself to delivering web services. Also, Drupal 6 is all about reaching out to more people; I'm confident that Drupal 6's localization (l10n) and internationalization (i18n) improvements will help us get more traction in Brazil, China and India. Plus, with Drupal companies like Raincity Studios opening offices in China, there have been talks about organizing a Chinese Drupal conference.

Reaching out with Drupal 6

I continue to be amazed about how many people believe that Drupal 6's localization (l10n) and internationalization (i18n) improvements aren't a killer feature.

Time to beat some sense into the nay-sayers: according to the CIA's factbook, only 4.84% of all the people in the world have English as one of their native languages. While it is believed that one out of every five people on earth knows some English (but not necessarily much), the fact remains that for more than 95% of the people in the world, English is not the main tool for communication, nor is English part of their cultural identity.

Admittedly, I knew that l10n and i18n improvements weren't going to be compelling for a large portion of our current install base, but I placed my bets, and wanted to see this happen nonetheless. That is exactly why I made Gábor a Drupal 6 branch maintainer.

I learned that every time we release a new version of Drupal, Drupal attracts more users. In other words, making a better product translates to a bigger install base. Every major Drupal release asserts this observation. It took me a couple years to realize that it was actually that simple a formula. Thus, I'm confident that the l10n and i18n improvements in Drupal 6 will convince many more people to use Drupal.

Also, Drupal isn't a panacea, but I'd like to believe that the l10n and i18n improvements in Drupal 6 could mark a small but important step to make a positive change in the lifes of folks in remote parts of the world. I'd expect that NGOs (like Greenpeace or Amnesty International) -- or even the NATO -- take interest in seeing Drupal's translation community flourish.

So if you want to learn more about the l10n and i18n improvements in Drupal 6, or how they compare to those of other Open Source CMSes, I'd encourage you to read Gábor's master thesis (PDF, 650KB, mirror) or to check out DevelopmentSeed's comparison chart.

Updates from Dries straight to your mailbox