Two recent blog posts explained what I think the Drupal development cycle is like; see the Gartner hype cycle and Drupal and the Drupal mood cycle. These thoughts came from living through many major Drupal releases and noticing patterns of developer and user mood as release dates approached and receded. Make sure to read these posts first, before reading this one.
Developers like to release code. "Release early, release often" wrote Eric S. Raymond in his famed essay on open source, The Cathedral and the Bazaar, and that philosophy has facilitated the rise of many open-source projects -- including Drupal. At the same time, many end users dislike change: they would prefer that software versions stay stable as long as possible, because change means work or cost.
Personally, based on listening to a lot of people, I believe that for Drupal 8, the release date should be 18 months after Drupal 7 reaches its Plateau of Productivity.
I think 18 months gives users plenty of time to enjoy Drupal 7's productivity at its peak before facing a possible update. At the same time, it's short enough that developers won't get bored. That said, I'm happy to brainstorm about this more but at least it puts a first stake in the ground.
I can't say how I'll determine that Drupal 7 has reached its Plateau of Productivity. It will be evident in several ways -- the decrease of new Drupal 6 sites, the stability of contributed modules, the number of websites on Drupal 7, the feedback I get when attending Drupal events and so forth. I'll make an announcement at that time to start the clock running.
Right now, I'd predict that the Plateau of Productivity is 6-9 months away, meaning Drupal 8 won't be released for at least another 2 years.
Yesterday, I talked about the Gartner hype cycle and how the Drupal 7 release will evolve through it. Now I'd like to generalize it to show how the hype cycle affects the Drupal community -- particularly core developers -- during the development lifecycle of a major release.
Here is the curve of one lifecycle, as seen by Drupal's developers:
The development cycle is like a song, and each phase has its own stanza. Slow motion's lyrics are, "We need more core committers to review patches!" and "Drupal development is so broken!". Patch frenzy wails, "Crap, my patch isn't going to make it in!" and "Wait, we can't freeze the code like this!". User scream is a call-and-response with the market, which sings "You shouldn't release core until contrib is updated" and "I can't believe you're already working on the next version when I haven't even upgraded yet".
Complicating matters is the fact that we're always working on both the current version of Drupal and the next one. (We also maintain the previous version, but its cycle has already finished.) When you overlay Drupal 7 and Drupal 8's cycles, they look something like this:
Adding the two graphs together gives us a new curve that reflects the total picture.
I've put a line where I think we are now. Drupal 7 is on its way into the "Slope of enlightenment", while Drupal 8 is near the beginning of its path.
Unfortunately, a low mood characterizes both of these stages, so the total mood can be discouraging these days. But if you look into the future, it's clear that both Drupal 7 and Drupal 8 are entering a good time. The excitement will only grow until the release of Drupal 8. When will that be? I'll talk about that in my next blog post.
I'd like to discuss some observations the technology analysis company Gartner Group has made about product lifecycles -- and how their model affects Drupal release management.
Here is how I think the hype cycle's stages played out for Drupal 7's development cycle.
- Technology Trigger. As Gartner puts it, this is where “A potential technology breakthrough kicks things off". For Drupal 7 that occurred when we started development on 4 February, 2008.
- Peak of Inflated Expectations. This is a period when excitement about the forthcoming release makes people dream big. We entered this phase shortly before Drupal 7's release, as news of all the wonderful things we were doing entered the consciousness of early adopters.
- Trough of Disillusionment. This stage came after Drupal 7's release, as people examined it closely to decide whether it would work for their new projects. Some discovered that a necessary module wasn't available yet, or that they had to learn some new APIs. Many chose “the devil they know" in Drupal 6 while cursing Drupal 7's differences. That's understandable, and it will pass.
- Slope of Enlightenment. I believe this is where Drupal 7 is now. It is where modules are being upgraded, bugs are being fixed, and the whole ecosystem is becoming more stable overall. Some people adopt it, while others still wait.
- Plateau of Productivity. This is where we can breathe easy. It occurs when developing in Drupal 6 seems pointless, because the Drupal 7 ecosystem exhibits more promise.
That last stage is where a product really shines. I think we reached the Plateau of Productivity for Drupal 6 sometime in late 2009, about 18 months after its release. At that point there was no advantage to developing in Drupal 5, and Drupal 7 was still a long way off.
I have my own opinions about Drupal 7's path through the hype cycle, but would like to hear your thoughts. When do you think Drupal 7 will reach the Plateau of Productivity?
This is my first blog post in 2011, so first things first: happy New Year to all of you!
As I do each year, I want to stop for a moment and reflect on the past, before jumping head-first into the new year.
Last year was another year of growth for Drupal: the number of active committers grew; the number of visitors to drupal.org grew; the number of Drupal sites grew; and the number of Drupal events and meet-ups grew. Drupal grew by almost every discernible metric.
Overall, we became a better and more well-rounded team in 2010. We've always had many kick-ass engineers in the Drupal community. However, 2010 was the year in which we achieved noticeably better usability, design, accessibility and marketing. These are important developments, as these new skills will help prepare us for even more success in the long run.
On a personal level, 2010 was a very busy Drupal year for me. I committed 1,683 patches -- that's up from 1,567 patches in 2009 and 1,031 patches in 2008. I accepted many more speaking engagements to evangelize our work. While in 2009 I flew about 100,000 km (62,000 miles), in 2010 I flew over 300,000 km (190,000 miles). That's a lot of evangelizing. I was more 'outward facing' in 2010 than ever before, which seems to be a trend. Although I enjoy meeting Drupal users, I'm glad to stay closely involved with the day to day development of Drupal core. Being able to combine these two essential elements is both important and healthy. I hope and predict that 2011 will bring more of the same.
My personal low for Drupal in 2010 is the fact that we didn't release Drupal 7. It is a consolation, however, it has been delayed for good reasons and we didn't compromise on quality. One thing is a given though: 2011 will be the year of Drupal 7. It will be a nicer-looking, more powerful, and more scalable Drupal that will be easier to use. If you've overlooked Drupal previously in favor of some other system, it's time to revisit it again.
My biggest wish for 2011 is that our community remains strong. We are our biggest asset. We make Drupal both vibrant and innovative. It has been an honor to be a part of the Drupal project, and it remains so today. I have no reason to believe that that will change in 2011.
After working on Drupal 7 for three years, I'm looking forward to start working on Drupal 8. One of the things that I like most is figuring out the future, and leading the community in the right direction.
I'm utterly convinced that user experience remains the single most important thing that is holding back Drupal adoption. I want to spend time listening and talking to each of Drupal's users and learn more about their experiences. That includes content creators, site builders, developers, designers, system administrators, owners of small Drupal sites, owners of large Drupal sites and more. I also want to learn more about other systems and learn from them. There is only so much we can do in one release, especially if we want shorter release schedules. So I want to have a very clear picture of how we can improve the user experience of each of Drupal's target audiences.
Based on the 300,000 km of listening I did last year, I predict that the following three items will make it in the top five new features of Drupal 8: general usability improvements for content creators and site builders; performance improvements for single machine installations; life cycle and release management (e.g., development-staging-production, configuration management, testability).
It's clear that the mobile web will become mainstream in a big way in 2011. On top of that, the iPad is changing our world. We should think long and hard about how we can make Drupal the go-to-platform for building web applications in a world of tablets and mobile handheld devices. One of my objectives is to write a small iPhone or Android application that connects with Drupal. I want to learn more about it so I'm going to try and make it one of my weekend projects.
I also predict that in 2011, we'll see the first Drupal website that serves a billion page views per month, and that Drupal.org will be upgraded to Drupal 7 before the end of 2011.
I've spent a lot of time time thinking about Drupal distributions in 2010. While we made a lot of progress in making distributions feasible from a technical point of view, we have yet to figure out the business model around Drupal distributions. Unless we make it commercially interesting or at least commercially viable for organizations to build and maintain high quality distributions, distributions might not be as popular as we'd like. I think 2011 might be the year where many of us test how important Drupal distributions are really going to be for us.
Looking back at 2010, I'm happy with the way the Drupal Association evolved. We did some unusual things this year. We hired two full-time employees; Megan Sanicki (sponsor wrangler) and Neil Kent (events manager), and paid for the drupal.org redesign to be completed. We're also paying for the migration from CVS to Git. It isn't always easy to mix paid staff into a volunteer-driven open source project, but without that, two of my 2010 highlights would not have happened.
Relative to the bigger market, 2010 was also the year where I felt core development of Joomla! slowed down a bit. As part of that, I noticed that the Joomla community started to expand to Drupal. WordPress, on the other hand, seems to be accelerating. Slowly but certainly, it's growing from a blogging platform into a content management system. Kudos to Matt Mullenweg and Automattic for their progress. While WordPress has a long way to go to compete with Drupal on the high-end, I expect that we will see it compete more and more in the low-end of the market. I won't let them take the low-end of the market -- it has been too important for Drupal's growth and adoption. I'm very passionate about making small sites successful, and I wish they were better represented in the Drupal.org issue queues. It is going to be interesting to see how these things play out. Either way, the real competition is not other Open Source projects, but proprietary software vendors and the many static HTML websites. There is so much room for growth that we shouldn't worry too much about Open Source competition.
2011 is also the year that Drupal turns 10 years old. We're starting off the year with our biggest release ever, followed by our biggest party ever. We'll have lots of things to talk about, lots of planning to do, and lots of initiatives to kick off. Expect 2011 to get very busy.
No blog post of this type would be complete if I didn't end with a sincere, heart-felt "Thank you!" to the many members of our community. Without your contributions, Drupal wouldn't exist, and my past years and future years to come would be devoid of something I love dearly. So, from me to you, for making Drupal what it is today, and for working with me to make it better day by day, let me say, simply, thank you.
Ten years ago, the average organization had one website. Since then, the world has become a more complex place with a diverse set of needs. If you're like most organizations the number of sites you have continues to grow at a rapid clip. You've dipped your toe into the social media waters by setting up one or more blogs, you use microsites for the foundation of your marketing efforts to promote products and events, and community sites to engage with the people who use your products. And of course, you have your corporate website, as well as your intranet and a number of internal collaboration websites for different projects. This is today's reality.
During the past year, I've met with many organizations that have hundreds of sites; some even have thousands of sites. Most of these sites are vastly different in terms of scale, functionality, complexity and longevity. As a result, the level of investment and the time to market requirements are usually very different. Some of the websites are owned by the company's IT department while other websites may be owned by their marketing department.
For most organizations, one tool cannot get the job done, so they keep multiple tools in their toolbox - whether they intend to or not. Unfortunately this is a common scenario in many enterprises. We see many organizations that run on Vignette, but they add WordPress to power their blog, and use SharePoint for their intranet. Managing integration and multiple (proprietary) solutions is not only costly but it acts as a roadblock to innovation and slows time to market when changes are needed.
It can be a complete mess.
CIOs are now waking up; they're realizing that the cost of running twenty different content management systems on twenty different stack configurations is an expensive, unnecessary burden for the organization. They can see that there are cost savings to be made if they standardize. What they want is something more than a content management system: they need a single platform that meets their needs. They need Drupal.
The New York Stock Exchange is launching 30 sites on Drupal. Turner Broadcasting is migrating 80 sites to Drupal. A large consumer pharmaceuticals company might end up with over 3000 Drupal sites. Why? Because Drupal gives their IT team an agile platform to deliver new sites quickly with the breadth of capabilities to meet their needs, but broad enough in scope to serve as a web platform too.
More and more, I see large organizations standardize on Drupal. Drupal's low-cost entry point and open, modular architecture provides the perfect foundation. Drupal is one of the few solutions that can scale from very small to extremely large, and has the depth and breadth of functionality to support thousands of different use cases. Drupal sites can span a wide range of functionality; from blogs, to marketing microsites, social business community sites, corporate intranets, e-commerce sites and more. There are incredible success stories for each of these use cases already. Plus, we've now reached a point where many open source content management systems outperform proprietary solutions on technical superiority and pace of innovation. Drupal is in a very unique position.
This is an immensely powerful trend. If we navigate this well, Drupal could become truly ubiquitous and go from powering 1% of the web, to quickly powering 5% of the web and even further. It requires us to keep making Drupal a better platform, to crack the code on how to make Drupal distributions truly compete with commercial point solutions, and to build out an even larger commercial ecosystem and community. Needless to say, I'm super excited about Drupal's future and the opportunity ahead of us.