In my travels to talk about Drupal, everyone asks me about Drupal 8's performance and scalability. Modern websites are much more dynamic and interactive than 10 years ago, making it more difficult to build modern sites while also being fast. It made me realize that maybe I should write up a summary of some of the most exciting performance and scalability improvements in Drupal 8. After all, Drupal 8 will leapfrog many of its competitors in terms of how to architect and scale modern web applications. Many of these improvements benefit both small and large websites, but also allow us to build even bigger websites with Drupal.
One of the strategies we employ in making Drupal fast is "caching". This means we try to generate pages or page elements one time and then store them so future requests for those pages or page elements can be served faster. If an item is already cached, we can simply grab it without going through the building process again (known as a "cache hit"). Drupal stores each cache item in a "cache bin" (a database table, Memcache object, or whatever else is appropriate for the cache backend in use).
In Drupal 7 and before, when one of these cache items changes and it needs to be re-generated and re-stored (the cache gets "invalidated"), you can only delete a specific cache item, clear an entire cache bin, or use prefix-based invalidation. None of these three methods allow you to invalidate all cache items that contain data of, say, user 200. The only method that is going to suffice is clearing the entire cache bin, and this means that usually we invalidate way too much, resulting in poor cache hit ratios and wasted effort rebuilding cache items that haven't actually changed.
This problem is solved in Drupal 8 thanks to the concept of "cache tags": each cache item can have any number of cache tags. A cache tag is a compact string that describes the object being cached. Thanks to this extra metadata, we can now delete all cache items that use the
user:200 cache tag, for example. This means we've deleted all the cache items we must delete, but not a single one more: optimal cache invalidation!
And don't worry, we also made sure to expose the cache tags to reverse proxies, so that efficient and accurate invalidation can happen throughout a site's entire delivery architecture.
While accurate cache invalidation makes caching more efficient, there is more we did to improve Drupal's caching. We also make sure that cached items are optimally varied. If you vary too much, duplicate cache entries will exist with the exact same content, resulting in inefficient usage of caches (low cache hit ratios). For example, we don't want a piece of content to be cached per user if it is the same for many users. If you vary too little, users might see incorrect content as two different cache entries might collide. In other words, you don't want to vary too much nor too little.
In Drupal 7 and before, it's easy to program any cached item to vary by user, by user role, and/or by page, and could even be configured through the UI for blocks. However, more targeted variations (such as by language, by country, or by content access permissions) were more difficult to program and not typically exposed in a configuration UI.
In Drupal 8, we introduced a Cache Context API to allow developers and site builders to express these variations and to make them automatically available in the configuration UI.
Usually a page can be cached almost entirely except for a few dynamic elements. Often a page served to two different authenticated users looks identical except for a small "Welcome $name!" and perhaps their profile picture. In Drupal 7, this small personalization breaks the cacheability of the entire page (or rather, requires a cache context that's way too granular). Most parts of the page, like the header, the footer and certain blocks in the sidebars don't change often nor vary for each user, so why should you regenerate all those parts at every request?
In Drupal 8, thanks to the addition of #post_render_cache, that is no longer the case. Drupal 8 can render the entire page with some placeholder HTML for the name and profile picture. That page can then be cached. When Drupal has to serve that page to an authenticated user, it will retrieve it from the cache, and just before sending the HTML response to the client, it will substitute the placeholders with the dynamically rendered bits. This means we can avoid having to render the page over and over again, which is the expensive part, and only render those bits that need to be generated dynamically!
Some things that Drupal has been rendering for the better part of a decade, such as the "new" and "updated" markers on comments, have always been rendered on the server. That is not ideal because these markers are different for every visitor and as a result, it makes caching pages with comments difficult.
The just-in-time substitution of placeholders with dynamic elements that
#post_render_cache provides us can help address this. In some cases, as is the case with the comment markers, we can even do better and offload more work from the server to the client. In the case for comment markers, a certain comment is posted at a certain time — that doesn't vary per user. By embedding the comment timestamps as metadata in the DOM with a
With Drupal 8, we're very close to taking the client-side dynamic content substitution a step further, just like some of the world's largest dynamic websites do. Facebook has 1.35 billion monthly active users all requesting dynamic content, so why not learn from them?
The traditional page serving model has not kept up with the increase of highly personalized websites where different content is served to different users. In the traditional model, such as Drupal 7, the entire page is generated before it is sent to the browser: while Drupal is generating a page, the browser is idle and wasting its cycles doing nothing. When Drupal finishes generating the page and sends it to the browser, the browser kicks into action, and the web server is idle. In the case of Facebook, they use BigPipe. BigPipe delivers pages asynchronously instead; it parallelizes browser rendering and server processing. Instead of waiting for the entire page to be generated, BigPipe immediately sends a page skeleton to the the client so it can start rendering that. Then the remaining content elements are requested and injected into their correct place. From the user's perspective the page is rendered progressively. The initial page content becomes visible much earlier, which improves the perceived speed of the site.
We've made significant improvements to the way Drupal 8 renders pages (presentation). By default, Drupal 8 core still implements the traditional model of assembling these pieces into a complete page in a single server-side request, but the independence of each piece and the architecture of the new rendering pipeline enable different “render strategies" to be experimented with — different methods for dynamic content assembly, such as BigPipe, Edge Side Includes, or other ideas for making the most optimal use of client, server, content delivery networks and reverse proxies. In all those examples, the idea is that we can send the primary content first so the client can start rendering that. Then we send the remaining Drupal blocks, such as the navigation menu or a 'Related articles' block, and have the browser, content delivery network or reverse proxy assemble or combine these blocks into a page.
Some early experiments by Wim Leers in Acquia's OCTO show that we can improve performance by a factor of about 2 compared to a recent Drupal 8 development snapshot. These breakthroughs are enabled by leveraging the various improvements we made to Drupal 8.
All in all, there is a lot to look forward to in Drupal 8!
When I visited Brazil in 2011, I was so impressed by the Latin American Drupal community and how active and passionate the people are. The region is fun and beautiful, with some of the most amazing sites I have seen anywhere in the world. It also happens to be a strategic region for the project.
Latin American community members are doing their part to grow the project and the Drupal community. In 2014, the region hosted 19 Global Training Day events to recruit newcomers, and community leaders coordinated many Drupal camps to help convert those new Drupal users into skilled talent. Members of the Latin American community help promote Drupal at local technology and Open Source events, visiting events like FISL (7,000+ participants), Consegi (5,000+ participants) and Latinoware (4,500+ participants).
You can see the results of all the hard work in the growth of the Latin American Drupal business ecosystem. The region has a huge number of talented developers working at agencies large and small. When they aren't creating great Drupal websites like the one for the Rio 2016 Olympics, they are contributing code back to the project. For example, during our recent Global Sprint Weekend, communities in Bolivia, Colombia, Costa Rica, and Nicaragua participated and made valuable contributions.
The community has also been instrumental in translation efforts. On localize.drupal.org, the top translation is Spanish with 500 contributors, and a significant portion of those contributors come from the Latin America region. Community members are also investing time and energy translating Drupal educational videos, conducting camps in Spanish, and even publishing a Drupal magazine in Spanish. All of these efforts lower the barrier to entry for Spanish speakers, which is incredibly important because Spanish is one of the top spoken languages in the world. While the official language of the Drupal project is English, there can be a language divide for newcomers who primarily speak other languages.
Last but not least, I am excited that we are bringing DrupalCon to Latin America next week. This is the fruit of many hours spent by passionate volunteers in the Latin American local communities, working together with the Drupal Association to figure out how to make a DrupalCon happen in this part of the world. At every DrupalCon we have had so far, we have seen an increase in energy for the project and a bump in engagement. Come for the software, stay for the community! Hasta pronto!
It's that time again. Time to look back at 2014, and to look forward to 2015. For Drupal in 2014, it was all about Drupal 8. As Drupal 8's development enters its fourth (and hopefully, final) year of development, it's a good time to reflect on all the work achieved by the Drupal 8 team so far, and to talk about Drupal 8's momentum heading into the final stretch to the release.
Drupal 8 will have 200 new features. Among the larger features that I'm excited about are the responsive design, HTML5 support, the native web service support, the much improved multilingual support, the configuration management system, a built-in WYSIWYG editor, in-place editing, streamlined content editing, the improved entity system, and more. The list of improvements is long!
My favorite part of Drupal 8 is that it will make building all types of Drupal sites — both big and small — much easier than with Drupal 7.
Key accomplishments in 2014 include:
October 1, 2014, amidst the fanfare at DrupalCon Amsterdam, we released Drupal 8 beta 1. This was an important milestone in the project, marking the finalization of major APIs, which enables contributed modules to begin porting in earnest.
Drupal 8's new object-oriented API represents a significant paradigm shift for developers (there are many benefits to this). To help Drupal 7 pros make the jump to Drupal 8, Acquia funded the Drupal Module Upgrader project. This project will not only scan a Drupal 7 module and generate a report pointing off to the appropriate documentation on how to port it, there is even a mode that automatically re-writes much of your module's code to Drupal 8 to eliminate a huge chunk of the work.
We organized dozens of sprints all around the world, and together hundreds of people came together in "real life" to help get Drupal 8 released. Sprints are a key part of momentum-building in Drupal, by laser-focusing on a specific goal, or by pairing both new and experienced contributors together for mentorship. Not only do sprints make solving tough issues easier, they also provide opportunities for building relationships and "leveling up" your skills.
Though it was launched just a month ago, the Drupal Association's Drupal 8 Accelerate Fund is already helping to add velocity to Drupal 8, by paying key contributors to help fix particularly onerous critical issues.
Our current focus is resolving the Drupal 8 upgrade path issues, which will allow early adopters of Drupal 8 to upgrade their site data between beta releases, and should result in a further uptick to Drupal 8 development velocity.
Once we reach zero critical issues, we begin the release candidate phase. Among the areas left to polish up after the Drupal 8 upgrade path issues are bringing external libraries up to date, finalizing documentation, and performance.
Unlike prior versions of Drupal, Drupal 8 has adopted a new release cycle that will provide backwards-compatible "feature" releases every 6 months. I'm extremely excited about this change, as it means we can innovate on the core platform for years to come after release, versus holding all of the new goodies until Drupal 9.
We're now one of the largest Open Source projects in terms of active contributors, if not the largest. That growth requires us to evolve how we work. Over the years, we've grown from a 100% volunteer-driven model to a model where there is a mix of volunteers, contributors who are partially funded by their customers or employers, and contributors who are paid full-time to work on Drupal.
While this shift has big benefits in making Drupal more sustainable, it also means there is increasingly more corporate participation and influence. One of our biggest challenges for 2015 is to figure out how we can get more commercial organizations to step up to take on more of the shared maintenance of Drupal, while at the same time respecting the needs and desires of our entire community.
There has also been a lot of talk about optimizing the way in which we work, to make it more explicit who is responsible for what, how decisions are made, and so on. This year I plan to work with others in the community to revamp Drupal core's governance model to bring more transparency and appoint additional leadership.
Overall, I'm thrilled with the progress that the Drupal core contributors have made in 2014, and want to extend an enormous thanks to each and every one of our 2,500 contributors who have brought us this far. I'm feeling very positive about our momentum going into 2015.
Drupal 8 will set a new standard for ease of use, power and flexibility, and will have something for everyone to love. Without a doubt, Drupal 8 will take our community to new heights. Let's do this!
As is now a tradition for me, here is my annual Acquia retrospective, where I look back at 2014 and share what is on my mind as we start the new year. I take the time to write these retrospectives not only for you dear reader, but also for myself, because I want to keep a record of the changes we've gone through as a company and how my personal thinking is evolving from year to year. But I also write them for you, because you might be able to learn from my experiences or from analyzing the information provided. If you would like to, you can read my previous retrospectives: 2009, 2010, 2011, 2012 and 2013.
For Acquia, 2014 was another incredible year, one where we beat our wildest expectations. We crossed the major milestone of $100 USD million in annual revenue, the majority of which is recurring subscription revenue. It is hard to believe that 2014 was only our sixth full year as a revenue-generating business.
We've seen the most growth from our enterprise customers, but our number of small and medium size customers has grown too. We helped launch and host some incredible sites last year: from weather.com (a top 20 site) to the Emmys. Our efforts in Europe and Asia-Pacific are paying off; our EMEA business grew substantially, and the Australian government decided to switch the entire government to Drupal and the Acquia Platform.
We hired 233 people in 2014 and ended the year with 575 employees. About 25% of our employees work from home. The other 75% work from offices around the world; Burlington MA (US), Portland OR (US), Washington DC (US), Paris (France), Reading (United Kingdom), Ghent (Belgium), Singapore, Delhi (India), Brisbane (Australia) and Sydney (Australia). About 75% of our employees are based in the United States. Despite our fast growth rate in staff, recruiting remains a key challenge; it's hard to hire as fast as we do and maintain the high bar we've set for ourselves in terms of talent and commitment.
We raised venture funding twice in 2014: a $50MM series F round led by New Enterprise Associates (NEA) followed by Amazon investing an undisclosed amount of money in our business. It's not like Tom Erickson and I enjoy raising money, but building and expanding a sales and marketing team is notoriously difficult and requires big investments. At the same time, we're building and supporting the development of multiple products in parallel. Most companies only build one product. We're going after a big dream to become the preferred platform for what has been called the "pivot point of many enterprise tech stacks" -- the technologies that permit organizations to deliver on the promises of exceptional digital customer experiences from an agile, open, resilient platform. We are also competing against behemoths. We can't show up to a gunfight with a knife, so to speak.
Digital has changed everything, and more and more organizations need or want to transform into digital-first businesses to stay in step with the preferences of their customers. Furthermore, technology innovations keep occurring at an ever faster and more disruptive pace. No organization is immune to the forces of digital disruption. At Acquia, we help our customers by providing a complete technology platform and the support necessary to support their digital initiatives. The Acquia Platform consists of tools and support for building and managing dynamic digital experiences. It includes Acquia Cloud, which helps developers deliver complex applications at scale, and Acquia Lift, our digital engagement services for bringing greater context to highly personalized experiences. Let me give you an update on each of the major components.
Drupal gives organizations the ability to deliver a unified digital experience that includes mobile delivery, social and commerce. Great inefficiencies exist in most organizations that use a variety of different, disconnected systems to achieve those three essentials. They are tired of having to tie things together; content is important, social is important, commerce is important but connecting all these systems seamlessly and integrating them with preferred applications and legacy systems leads to massive inefficiencies. Companies want to do things well, and more often than not, Drupal allows them to do it better, more nimbly and in a far more integrated framework.
In 2010, we laid out our product vision and predicted more and more organizations would start to standardize on Drupal. Running 20 different content management systems on 20 different technology stacks is both an expensive and unnecessary burden. We've seen more and more large organizations re-platform most of their sites to Drupal and the Acquia Platform. They realize they don't need multiple content management systems for different sites. Great examples are Warner Music and Interscope Records, who have hundreds of sites on Drupal across the organization, resulting in significant cost savings and efficiency improvements. The success of our Acquia Cloud Site Factory solution has been gratifying to witness. According to a research study by Forrester Consulting, which we released late last year, ACSF is delivering a 944% return on investment to its adopters.
After many years of discussion and debate in the Drupal community, we launched the Acquia Certification Program in March 2014. So far, 546 Drupal developers from more than 45 countries have earned certification. The exams focus on real world experience, and the predominant comments we've heard this past year are that the exams are tough but fair. Acquia delivered six times the amount of training in 2014 compared to the previous year, and demand shows no sign of slowing.
Last, but definitely not least, is Drupal 8. We contributed significantly to Drupal 8 and helped it to achieve beta status; of the 513 critical Drupal 8 bugs fixed in 2014, Acquia's Office of the CTO helped fix 282 of them. We also funded work on the Drupal Module Upgrader to automate much of the work required to port modules from Drupal 7 to Drupal 8.
Drupal alone isn't enough for organizations to succeed in this digital-first world. In addition to adopting Drupal, the cloud continues to enable organizations to save time and money on infrastructure management so they can focus on managing websites more efficiently and bringing them to market faster. Acquia customers such as GRAMMY.com have come to depend on the Acquia Cloud to provide them with the kind of rugged, secure scale that ensures when the world's attention is focused on their sites, they will thrive. On a monthly basis, we're now serving more than 33 billion Apache hits, almost 5 billion Drupal bootstraps (requests handled by Drupal instead of one of different caching layers or the web server directly), 9 petabytes of data transferred (excluding instance to instance traffic), and logging 13 billion Drupal watchdog log lines. We added many new features to Acquia Cloud in 2014, including log streaming, self-service diagnosis tools, support for teams and permissions, two-factor authentication, new dashboards, improved security with support for Virtual Private Networks (VPNs), an API for Acquia Cloud, and more.
As powerful as the Drupal/Acquia Cloud combination may be, our customers demand far more from their digital properties, focusing more and more on optimizing them to fully deliver the best possible experience to each individual user. Great digital experiences have always been personal; today they have to become contextual, intuitively knowing each user and dynamically responding to each user's personal preference from device to location to history with the organization. After two years of development and the acquisition of TruCentric, we launched Acquia Lift in 2014.
It's surprising how many organizations aren't implementing any form of personalization today. Even the most basic level of user segmentation and targeting allows organizations to better serve their visitors and can translate into significant growth and competitive differentiation. Advanced organizations have a single, well-integrated view of the customer to optimize both the experience and the lifetime value of that customer, in a consistent fashion across all of their digital touchpoints. Personalization not only leads to better business results, customers have come to expect it and if they don't find it, they'll go elsewhere to get it. Acquia Lift enables organizations to leverage data from multiple sources in order to serve people with relevant content and commerce based on intent, locations and interests. I believe that Acquia Lift has tremendous opportunity and that it will grow to be a significant business in and of itself.
While our key areas of investment in 2014 were Acquia Cloud and Acquia Lift, we did a lot more. Our Mollom service blocked more than 7.8 billion spam messages with an error rate of only 0.01%. We continue to invest in commerce; we helped launch the new Puma website leveraging our Demandware connector and continue to invest and focus on the integration of content and commerce. Overall, the design and user experience of our products has improved a lot, but it is still an area for us to work on. Expect us to focus more heavily on user experience in 2015.
The results of all our efforts around the launch of the Acquia Platform have not gone unnoticed. In October, Acquia was identified as a Leader in the 2014 Gartner Magic Quadrant for Web Content Management.
I'm very optimistic about Acquia's future in 2015. I believe we've steered the company to be positioned at the right place at the right time. As more organizations are shifting to becoming digital-first businesses they want to build digital experiences that are more pervasive, more contextual, more targeted, more integrated, and last but not least, more secure.
The consolidation from many individual point solutions to one platform is gaining momentum, although re-platforming is usually a long process. Organizations want the unified or integrated experience that Drupal has to offer, as well as the flexibility of Open Source. It is still time consuming and challenging to create quality content, and I believe there is plenty of opportunity for us and our partners to help with that going forward.
Without a doubt, organizations want to better understand their customers and use data-driven decisions to drive growth. Data is becoming the new product. The opportunity this creates in commerce is massive.
Cloud computing and Software-as-a-Service (SaaS) continues to be on the rise. Cloud is top of mind and the transition away from on-premise solutions is accelerating even as the arguments around security and privacy issues in the cloud continue to be raised. While there is a certain amount of emotion, and sometimes politics, people are beginning to realize that the cloud is usually more secure and more robust against cyber-attacks than traditional on-premise systems.
The promise of Drupal 8, arguably the most significant advance in the evolution of the Drupal software, has me very excited. It is shaping up to be a great release, and I'm confident it will further secure Drupal's reputation among developers, designers, agencies and site managers as the most flexible, powerful content management solution available.
All of this is not to say 2015 will be easy. This is an incredibly exciting and fast-changing space in the world of technology. Acquia is growing in an incredibly fast-paced, dynamic sector and we realize our mission is to help our customers understand how to think ahead to ever more innovation and change. Simplifying our overall messaging and defining ourselves around the Acquia Platform is a significant first step.
Of course, none of this success would be possible without the support of our customers, partners, the Drupal community, the Acquia team, and our many friends. Thank you for your support in 2014, and I look forward to working with you to find out what 2015 will bring!
The older I get, the quicker the years seem to fly by. As I begin to reflect on a great 2014, one thing is crystal clear again. People are the most important thing to any organization. Having a great team is more important than having a great idea. A good team will figure out how to make something great happen; they'll pivot, evolve and claw their way to success. I see it every day at Acquia, the Drupal Association or the Drupal community. I'm fortunate to be surrounded by so many great people.
By extension, recruiting is serious business. How do you figure out if someone is a great fit for your organization? Books have been written about finding and attracting the right people, but for me the following quote from Dee Hock, the founder of Visa, sums it up perfectly.
"Hire and promote first on the basis of integrity; second, motivation; third, capacity; fourth, understanding; fifth, knowledge; and last and least, experience. Without integrity, motivation is dangerous; without motivation, capacity is impotent; without capacity, understanding is limited; without understanding, knowledge is meaningless; without knowledge, experience is blind." — Dee Hock, founder of Visa.
Most hiring managers get it wrong and focus primarily on experience. While experience can be important, attitude is much more important. Attitude, not experience, is what creates a strong positive culture and what turns users and customers into raving fans.
Today the Drupal Association announced a new program: the Drupal 8 Accelerate Fund. Drupal 8 Accelerate Fund is a $125,000 USD fund to help solve critical issues and accelerate the release of Drupal 8.
The Drupal Association is guaranteeing the funds and will try to raise more from individual members and organizations within the Drupal community. It is the Drupal 8 branch maintainers — Nathaniel Catchpole, Alex Pott, Angie Byron, and myself — who will decide on how the money is spent. The fund provides for both "top-down" (directed by the Drupal 8 branch maintainers) and "bottom-up" (requested by other community members) style grants. The money will be used on things that positively impact the Drupal 8 release date, such as hiring contributors to fix critical bugs, sponsoring code sprints to fix specific issues, and other community proposals.
Since the restructuring of the Drupal Association, I have encouraged the Drupal Association staff and Board of Directors to grow into our ambitious mission; to unite a global open source community to build and promote Drupal. I've also written and talked about the fact that scaling Open Source communities is really hard. The Drupal 8 Accelerate Fund is an experiment with crowdsourcing as a means to help scale our community which is unique compared to other efforts because it is backed by the official non-profit organization that fosters and supports Drupal.
I feel that the establishment of this fund is an important step towards more sustainable core development. My hope is that if this round of funding is successful that this can grow over time to levels that could make an even more meaningful impact on core, particularly if we complement this with other approaches and steps, such as organization credit on Drupal.org.
This is also an opportunity for Drupal companies to give back to Drupal 8 development. The Drupal Association board is challenging itself to raise $62,500 USD (half of the total amount) to support this program. If you are an organization who can help support this challenge, please let us know. If you're a community member with a great idea on how we might be able to spend this money to help accelerate Drupal 8, you can apply for a grant today.
Business model innovation is usually more powerful than technical innovation; it is more disruptive and harder to copy than technical innovation. And yet, so many companies are focused on technical innovation to compete.
Consider Airbnb. What makes them so successful is not a technical advantage, but a business model advantage that provides them near-zero marginal cost. For a traditional hotel chain to increase its capacity, it needs to build more physical space at significant cost. Instead of shouldering that setup cost, Airbnb can add another room to its inventory at almost no cost by enabling people to share their existing houses. That is a business model innovation. Furthermore, it is extremely difficult for the traditional hotel chain to switch its business model to match Airbnb's.
The same is true in Open Source software. While it is true that Open Source often produces technically superior software, its real power may be its business model innovation: co-creation. Open Source software like Drupal or Linux is a co-created product; thousands of contributors build and enhance Drupal and everyone benefits from that. A large Open Source community produces vastly more software than a proprietary competitor, and shares in the production and go-to-market costs. It disrupts proprietary software companies where the roles of production and consumption are discrete and the production and go-to-market costs are high. While established companies can copy key technical innovations, it is extremely difficult to switch a proprietary business model to an Open Source business model. It affects how they build their software, how they monetize the software, how they sell and market their software, their cost structure, and more. Proprietary software companies will lose against thriving Open Source communities. I don't see how companies like HP, Oracle and SAP could change their business model while living quarter to quarter in the public markets; changing their business model would take many years and could disrupt their revenues.
Take Amazon Web Services (AWS), one of the most disruptive developments in the IT world the past decade. While AWS' offerings are rich and often ahead of the competition, the biggest reason for the company's success is its business model. Amazon not only offers consumption-based pricing ('pay as you consume' vs 'pay as you configure'), it's also comfortable operating a low-margin business. Almost 10 years after AWS launched, at a time that vast amounts of computing are moving into the cloud, HP, Oracle and SAP still don't have competitive cloud businesses. While each of these companies could easily close technical gaps, they have been unable to disrupt their existing business models.
If you're in a startup, innovating on a business model is easier than if you're in a large company. In fact, an innovative business model is the best weapon you have against large incumbents. Technical innovation may give you a 6 to 18 month competitive advantage, but the advantage from business model innovation can be many years. Too many startups focus on building or acquiring innovative or proprietary technology in order to win in the market. While there is usually some technical innovation around the edges, it is business model innovation that makes a successful, long-standing organization -- it tends to be a lot harder to copy than technical innovation.
The concept of official initiatives came out of lessons learned from the Drupal 7 development. We learned a lot from that and in a recent blog post about Drupal initiative leads, I recognized that we need to evolve our tools, our processes, and our organizational design. Others like Nathaniel Catchpole, Larry Garfield and Gábor Hojtsy have shared some of their thoughts already. One of the things I'm most proud of is that the Drupal community is always looking to improve and reinvent itself. Evolving is an important part of our culture. Each time it will get better, but still won't be perfect.
For me, one of the biggest take-aways (but not the only one) is that for an initiative to succeed, it needs to be supported by a team. An initiative needs to carry out a technical vision, plan the work, communicate with all stakeholders, mobilize volunteers, raise funding, organize sprints, and more. It can easily be more than one person can handle -- especially if it isn't your full-time job or if your initiative is complex.
More specifically, we have learned that the most successful initiatives appear to be run by teams that are self-managed; the team members collaborate in the development of the initiative, but also share both managerial and operational responsibilities like planning, coordinating, communicating, sprint organizing and more.
Because self-managed teams are both responsible for their outcomes and in control of their decision-making process, members of a self-managing team are usually more motivated than traditional hierarchical teams. This independence and greater responsibility are important in volunteer communities. Self-managed teams also build and maintain institutional knowledge. The outcome of their work is also more easily accepted by other stakeholders (like core committers) because they have already built a lot of consensus.
If I were to be an initiative lead, I'd feel strongly about building my own team rather than being handed a team. My initial assumption was that each initiative lead would build his/her own team. In hindsight, that was a mistake. Team building is not easy. It requires a time investment that can seem to compete with technical priorities. This is an important lesson and something we can do better going forward. Before making an initiative official, we have to make sure that each initiative has a good team and the support to be successful -- either we can help create a team, provide more coaching or formal training around team building, or we shouldn't designate the initiative official until such a team has coalesced.
One of the world's most trafficked websites, with more than 100 million unique visitors every month and more than 20 million different pages of content, is now using Drupal. Weather.com is a top 20 U.S. site according to comScore. As far as I know, this is currently the biggest Drupal site in the world.
Weather.com has been an active Drupal user for the past 18 months; it started with a content creation workflow on Drupal to help its editorial team publish content to its existing website faster. With Drupal, Weather.com was able to dramatically reduce the number of steps that was required to publish content from 14 to just a few. Speed is essential in reporting the weather, and Drupal's content workflow provided much-needed velocity. The success of that initial project is what led to this week's migration of Weather.com from Percussion to Drupal.
The company has moved the entire website to Acquia Cloud, giving the site a resilient platform that can withstand sudden onslaughts of demand as unpredictable as the weather itself. As we learned from our work with New York City's MTA during Superstorm Sandy in 2012, “weather-proofing” the delivery of critical information to insure the public stays informed during catastrophic events is really important and can help save lives.
Drupal 8 is the first time we introduced the concept of formal initiatives and initiative leads. Over the course of these Drupal 8 initiatives we learned a lot and people are floating several ideas to increase the initiatives' success and provide Drupal initiative leads with more support. As we grow, it is crucial that we evolve our tools, our processes, and our organizational design based on these learnings. We've done so in the past and we'll continue to do so going forward.
But let's be honest, no matter how much support we provide, leading a Drupal initiative will unquestionably remain difficult and overwhelming. As a Drupal initiative lead, you are asked to push forward some of the most difficult and important parts of Drupal.
You will only succeed if you are able to build a strong team of volunteers that is willing to be led by you. You have to learn how to inspire and motivate by articulating a vision. You establish credibility by setting clear objectives and roadmaps in partnership with others. You have to motivate, guide and empower people to participate. You have to plan and over-communicate.
Not only do you have to worry about building and leading a team, you also have to make sure the rest of the community has shared goals and that everyone impacted has a shared understanding of why those decisions are being made. You use data, ideas and feedback from different sources to inform and convince people of your direction. Your "soft skills" are more important than your "hard skills". Regardless, you will lose many battles. You only "win" when you remain open to feedback and value change and collaboration. To lead a community, you need both a thick skin and a big heart.
Success is never a coincidence. You put in long hours to try and keep your initiative on track. You need relentless focus on doing whatever is necessary to succeed; to be the person who fills all the gaps and helps others to be successful. Instead of just doing the things you love doing most, you find yourself doing mundane tasks like updating spreadsheets or planning a code sprint to help others be successful. In fact, you might need to raise money for your code sprint. And if you succeed, you still don't have enough money to achieve what is possible and you feel the need to raise even more. You'll be brushing aside or knocking down obstacles in your path, and taking on jobs and responsibilities you have never experienced before.
Your objectives will constantly shift as Drupal itself iterates and evolves. You will want to go faster and you will struggle with the community processes. Imagine working on something for a month and then having to throw it out completely because you realize it doesn't pass. Frustration levels will be off the charts. Your overall goal of achieving the perfect implementation might never be achieved and that feeling haunts you for weeks or months. You will feel the need to vent publicly, and you probably will. At the worst moments, you'll think about stepping down. In better times, you realize that if most of your initiative succeeds it could take years of follow-up work. You will learn a lot about yourself; you learn that you are bad at many things and really good at other things.
Leading is incredibly hard and yet, it will be one of the best thing you ever did. You work with some of the finest, brightest, and most passionate people in the world. You will see tangible results of your hard work and you will impact and help hundreds of thousands of people for the next decade. There is no better feeling than when you inspire or when you help others succeed. Leading is hard, but many of you will look back at your time and say this was the most gratifying thing you ever did. You will be incredibly proud of yourself, and the community will be incredibly proud of you. You will become a better leader, and that will serve you for the rest of your life.
Updates from Dries straight to your mailbox