There has been a lot of discussion around the future of the Drupal front end both on Drupal.org (#2645250, #2645666, #2651660, #2655556) and on my blog posts about the future of decoupled Drupal, why a standard framework in core is a good idea, and the process of evaluating frameworks. These all relate to my concept of "progressive decoupling", in which some portions of the page are handed over to client-side logic after Drupal renders the initial page (not to be confused with "full decoupling").
My blog posts have drawn a variety of reactions. Members of the Drupal community, including Lewis Nyman, Théodore Biadala and Campbell Vertesi, have written blog posts with their opinions, as well as Ed Faulkner of the Ember community. Last but not least, in response to my last blog post, Google changed Angular 2's license from Apache to MIT for better compatibility with Drupal. I read all the posts and comments with great interest and wanted to thank everyone for all the feedback; the open discussion around this is nothing short of amazing. This is exactly what I hoped for: community members from around the world brainstorming about the proposal based on their experience, because only with the combined constructive criticism will we arrive at the best solution possible.
Improving Drupal's user experience is a topic near and dear to my heart. Drupal's user experience challenges led to my invitation to Mark Boulton to redesign Drupal 7, the creation of the Spark initiative to improve the authoring experience for Drupal 8, and continued support for usability-related initiatives. In fact, the impetus behind progressive decoupling and adopting a client-side framework is the need to improve Drupal's user experience.
To iterate or to disrupt?
To date, much of our UX improvements have been based on an iterative process, meaning it converges on a more refined end state by removing problems in the current state. However, we also require disruptive thinking, which is about introducing entirely new ideas, for true innovation to happen. It's essentially removing all constraints and imagining what an ideal result would look like.
I think we need to recognize that while some of the documented usability problems coming out of the Drupal 8 usability study can be addressed by making incremental changes to Drupal's user experience (e.g. our terminology), other well-known usability problems most likely require a more disruptive approach (e.g. our complex mental model). I also believe that we must acknowledge that disruptive improvements are possibly more impactful in keeping Drupal relevant and widening Drupal's adoption.
At this point, to get ahead and lead, I believe we have to do both. We have to iterate and disrupt.
From inside-out to outside-in
Let's forget about Drupal for a second and observe the world around us. Think of all the web applications you use on a regular basis, and consider the interaction patterns you find in them. In popular applications like Slack, the user can perform any number of operations to edit preferences (such as color scheme) and modify content (such as in-place editing) without incurring a single full page refresh. Many elements of the page can be changed without the user's flow being interrupted. Another example is Trello, in which users can create new lists on the fly and then add cards to them without ever having to wait for a server response.
Contrast this with Drupal's approach, where any complex operation requires the user to have detailed prior knowledge about the system. In our current mental model, everything begins in the administration layer at the most granular level and requires an unmapped process of bottom-up assembly. A user has to make a content type, add fields, create some content, configure a view mode, build a view, and possibly make the view the front page. If each individual step is already this involved, consider how much more difficult it becomes to traverse them in the right order to finally see an end result. While very powerful, the problem is that Drupal's current model is "inside-out". This is why it would be disruptive to move Drupal towards an "outside-in" mental model. In this model, I should be able to start entering content, click anything on the page, seamlessly edit any aspect of its configuration in-place, and see the change take effect immediately.
Drupal 8's in-place editing feature is actually a good start at this; it enables the user to edit what they see without an interrupted workflow, with faster previews and without needing to find what thing it is before they can start editing.
Making it real with content modeling
Eight years ago in 2007, I wrote about a database product called DabbleDB. I shared my belief that it was important to move CCK and Views into Drupal's core and learn from DabbleDB's integrated approach. DabbleDB was acquired by Twitter in 2010 but you can still find an eight-year-old demo video on YouTube. While the focus of DabbleDB is different, and the UX is obsolete, there is still a lot we can learn from it today: (1) it shows a more integrated experience between content creation, content modeling, and creating views of content, (2) it takes more of an outside-in approach, (3) it uses a lot less intimidating terminology while offering very powerful capabilities, and (4) it uses a lot of in-place editing. At a minimum, DabbleDB could give us some inspiration for what a better, integrated content modeling experience could look like, with the caveat that the UX should be as effortless as possible to match modern standards.
This sort of vision was not possible in 2007 when CCK was a contributed module for Drupal 6. It still wasn't possible in Drupal 7 when Views existed as a separate contributed module. But now that both CCK and Views are in Drupal 8 core, we can finally start to think about how we can more deeply integrate the two. This kind of integration would be nontrivial but could dramatically simplify Drupal's UX. This should be really exciting because so many people are attracted to Drupal exactly because of features like CCK and Views. Taking an integrated approach like DabbleDB, paired with a seamless and easy-to-use experience like Slack, Trello and Backand, is exactly the kind of disruptive thinking we should do.
We shouldn't limit ourselves to this one example, as there are a multitude of Drupal interfaces that could all benefit from both big and small changes. We all want to improve Drupal's user experience — and we have to. To do so, we have to constantly iterate and disrupt. I hope we can all collaborate on figuring out what that looks like.
On December 29, 2000, I made a code commit that would change my life; it is in this commit that I called my project "Drupal" and added the GPL license to it.
A couple weeks later, on January 15, 2001, exactly 15 years ago from today, I released Drupal 1.0.0 into the world. The early decisions to open-source Drupal and use the GPL license set the cornerstone principles for how our community shares with one another and builds upon each other's achievements to this day.
Drupal is now 15 years old. In internet terms, that is an eternity. In 2001, only 7 percent of the world's population had internet access. The mobile internet had not entered the picture, less than 50% of the people in the United States had a mobile phone, and AT&T had just introduced text messaging. People searched the web with Lycos, Infoseek, AltaVista and Hot Bot. Google -- launched in 1998 as a Stanford University research project -- was still a small, private company just beginning its rise to prominence. Google AdWords, now a $65 billion business, had less than 500 customers when Drupal launched. Chrome, Firefox, and Safari didn't exist yet; most people used Netscape, Opera or Internet Explorer. New ideas for sharing and exchanging content such as "public diaries" and RSS had yet to gain widespread acceptance and Drupal was among the first to support those. Wikipedia was launched on the same day as Drupal and sparked the rise of user-generated content. Facebook and Twitter didn't exist until 4-5 years later. Proprietary software vendors started to feel threatened by open source; most didn't understand how a world-class operating system could coalesce out of part-time hacking by several thousand developers around the world.
Looking back, Drupal has not only survived massive changes in our industry; it has also helped drive them. Over the past decade and a half, I've seen many content management systems emerge and become obsolete: Vignette, Interwoven, PHP-Nuke, and Scoop were all popular at some point in the past but Drupal has outlived them all. A big reason is from the very beginning we have been about constant evolution and reinvention, painful as it is.
Keeping up with the pace of the web is a funny thing. Sometimes you'll look back on choices made years ago and think, "Well, I'm glad that was the right decision!". For example, Drupal introduced "hooks" and "modules" early on, concepts that are commonplace in today's platforms. At some point, you could even find some of my code in WordPress, which Matt Mullenweg started in 2003 with some inspiration from Drupal. Another fortuitous early decision was to focus Drupal on the concept of "nodes" rather than "pages". It wasn't until 10 years later with the rise of mobile that we started to see the web revolve less and less around pages. A node-based approach makes it possible to reuse content in different ways for different devices. In a way, much of the industry is still catching up to that vision. Even though the web is a living, breathing thing, there is a lot of things that we got right.
In the end, I feel fortunate that our community is willing to experiment and break things to stay relevant. Most recently, with the release of Drupal 8, we've made many big changes that will fuel Drupal's continued adoption. I believe we got a lot of things right in Drupal 8 and that we are on the brink of another new and bright era for Drupal.
I've undergone a lot of personal reinvention over the past 15 years too. In the early days, I spent all my time writing code and building Drupal.org. I quickly learned that a successful open source project requires much more than writing code. As Drupal started to grow, I found myself an "accidental leader" and worried about our culture, scaling the project, attracting a strong team of contributors, focusing more and more on Drupal's end-users, growing the commercial ecosystem around Drupal, starting the Drupal Association, and providing vision. Today, I wear a lot of different hats: manager of people and projects, evangelist, fundraiser, sponsor, public speaker, and BDFL. At times, it is difficult and overwhelming, but I would not want it any other way. I want to continue to push Drupal to reach new heights and new goals.
Today we risk losing much of the privacy, serendipity and freedom of the web we know. As the web evolves from a luxury to a basic human right, it's important that we treat it that way. To increase our impact, we have to continue to make Drupal easier to use. I'd love to help build a world where people's privacy is safe and Drupal is more approachable. And as the pace of innovation continues to accelerate, we have to think even more about how to scale the project, remain agile and encourage experimentation. I think about these issues a lot, and am fortunate enough to work with some of the smartest people I know to build the best possible version of the web.
So, here is to another 15 years of evolution, reinvention, and continued growth. No one knows what the web will look like 15 years in the future, but we'll keep doing our best to guide Drupal responsibly.
I am confident that adopting a client-side framework through progressive decoupling will give us the best of both worlds. Of course, this does not mean I oppose fully decoupling through any other framework; in fact, I believe we should redouble our efforts toward a first-class API-first Drupal. But progressive decoupling means that we will be able to work toward a next-generation user experience without abandoning much of the work we've done so far.
Special thanks to all of the following experts who provided review and input: Miško Hevery (creator of Angular; Google) and Igor Minar (technical lead for Angular; Google); Ed Faulkner (core maintainer for Ember); Amitai Burstein (Drupal and Elm contributor; Gizra); Sebastian Siemssen (Drupal contributor, Elm and React developer; Zensations); and John Albin Wilkins (Drupal 8 mobile initiative lead), Alex Bronstein (Drupal core maintainer; Acquia), Wim Leers (Drupal core contributor; Acquia), and Preston So (Drupal contributor, Acquia).
First, have we decided on the right criteria regardless of the frameworks themselves? This is probably the most important at this stage. While many organizations choose to adopt client-side frameworks for fully decoupled implementations, Drupal is the first to consider layering a framework on top to allow both richly dynamic and more traditional modules to coexist gracefully through progressive decoupling. What issues around templates, rendering, and client-side caching should we incorporate into these criteria? Is there anything missing or overemphasized?
Finally, have we drawn the right conclusions against these criteria? In other words, did we fill out the cells correctly? While they have been reviewed by some of the frameworks' experts, there might be unexpected gotchas or caveats.
At the moment, the most promising candidates in the comparison matrix appear to be Angular 2, Ember, and React, given their technical robustness, relative suitability for progressively decoupled Drupal, and their strong levels of community support and broader adoption. Given that Backbone is already in core and several modules already rely on it, we have included it too.
What we've learned from talking to the different projects is that they are often converging on similar techniques and best practices; they are by and large adding support for Virtual DOM implementations or rehydration (seamless state transfer), and they are all obsessing over small payload size and performance, better testability, etc. Therefore it is important to focus on the fundamental, often philosophical, differences between the projects that will likely be unchanged in time; key architectural differences, their release cadence and stance on backward compatibility, their license, their governance model, their flexibility and learning curve, etc.
From a quick glance at the criteria and our needs, it seems that Ember is currently our best candidate, as it appears to have a slight technical edge overall. Ember 2.0 has an all-new rendering engine named Glimmer, and it has server-side rendering through FastBoot. On the other hand, however, Ember is quite bulky and opinionated (enforcing patterns for code structure) compared to other candidate frameworks. A more fundamental difference is that unlike Angular and React, which have corporate governance and funding, Ember is a community-driven project like Drupal.
While React is lightweight, it needs integration with a variety of other libraries in the React ecosystem to work as a full-fledged implementation, which gives it a steep learning curve from an implementation standpoint. Because React is a relatively young project, best practices are shifting quickly and making it less attractive. The Virtual DOM, among React's most compelling features, has also seen its core ideas filter into other framework projects. But more importantly, React is licensed with what I believe to be a potentially unacceptable patent clause, which states that an organization can no longer use React once it sues Facebook for any (unrelated) patent infringement. This has already generated some concerted pushback from both WordPress's Calypso and React contributors.
Whatever the result of the debate around which client-side framework to adopt, I believe that Drupal needs to move quickly to embrace a framework that will aid development of a progressively decoupled architecture and corresponding user experience improvements. By providing some baseline criteria and including our accomplished community, I have no doubt we can reach this decision quickly but also intelligently.
Special thanks to Preston So for contributions to this blog post.
As many of my loyal blog readers know, I sit down to write a retrospective at the end of each year. It's helpful from the perspective of seeing how far Acquia has come in a single year, but it should also give you a glimpse of what is in store for Acquia in 2016 and beyond. If you find it interesting to take a look back at previous retrospectives, here they are: 2014, 2013, 2012, 2011, 2010 and 2009. These posts provide a pretty detailed overview of Acquia's trajectory as a company.
Since Acquia started eight years ago, we've believed that open source offers significant advantages over proprietary software because of its faster innovation, higher quality, freedom from vendor lock-in, greater security, and lower total cost of ownership. Early in our life as a company, we made a big bet that open source combined with the cloud, offered through a subscription model rather than perpetual license, would be a very compelling solution for the market. Few people believed us at the time, but now it is clear that our early vision is starting to pay off; perpetual software licenses are on the decline and Deutsche Bank analyst Karl Keirstead recently called cloud and open source the two leading themes in Silicon Valley.
The market demand for Acquia's digital business platform continues to grow; three of the top analyst firms, IDC, Gartner and Forrester have all named digital business transformation a top strategic imperative for the C-suite in 2016 and beyond. Open source, cloud computing, big data, the Internet of Things (IoT), and artificial intelligence are all catalysts for the expansion of digital transformation into all corners of the organization.
Organizations are rapidly expanding the range of digital interactions with their customers and partners, moving Drupal and Acquia to the core of their business. There is a growing focus on personalization and data-driven automation, which bodes well for products like Acquia Lift. In general, I believe that the growing reliance on digital provides Drupal and Acquia with a multi-decade opportunity.
Within the last 12 months, some of the largest technology companies including Apple, Google, Microsoft and Facebook have open-sourced key components of their business. There are many motivations for this shift. According to Apple, the company open-sourced its Swift programming language to extend it to new platforms. Google open-sourced TensorFlow, its artificial intelligence platform to make an even bigger impact outside Google, even though the company employs 2,000 engineers working on artificial intelligence alone. Microsoft open-sourced .NET to increase its relevance with developers and play nicely with other operating systems. In Deutsche Bank's 2016 predictions, Keirstead says "open source keeps eating the world", causing major price deflation for the traditional enterprise software industry. Whether the motive is faster innovation or increased adoption, companies are relying less and less on proprietary software and embracing open source.
Amazon SVP of Web Services, Andy Jassy, explained in his 2015 AWS re:Invent presentation that websites have been a "critical gateway" to AWS' wider cloud adoption in the enterprise. His rationale: nearly every organization has one or more websites, and many aren't considered "mission-critical applications". Therefore, most organizations feel comfortable moving some or all of their websites to the cloud. Websites are an important stepping stone for organizations to build up the knowledge and confidence to migrate their entire businesses to the cloud.
As cloud adoption grows, we're seeing our customers use a mixture of SaaS, PaaS and IaaS. In particular the Platform-as-a-Service (PaaS) model continues to grow fast, which is great news for Acquia Cloud. A growing number of enterprises are choosing PaaS ahead of Infrastructure-as-a-Service (IaaS) to save time and money on building, scaling and maintaining infrastructure so they can focus on building websites. Gartner sizes the PaaS market at $4.1B in 2015, attaining a compound annual growth rate (CAGR) of 65%+ through 2018. We are a few years ahead of our competitors (Adobe, Sitecore, IBM, Oracle) when it comes to PaaS, and I don't see that changing in 2016.
The migration to the cloud is only getting started. When I met with Eric Schmidt this year, he told me that he believes Google's cloud business could outpace its advertising business in five years. To put that in context, Google made more than $65 billion in advertising in 2015, roughly 90% of its total revenue. I don't think Google Cloud can possibly grow that fast, but directionally it's an eye-opening goal. Amazon's cloud business, bigger than its four closest competitors combined (including Google's cloud business), generated roughly $7 billion in revenues in 2015 and is expected to grow 80% year-over-year. Needless to say, it is exciting for Acquia be a "critical gateway" in such a massive movement to the cloud.
As the web evolves, the idea of a "digital business" takes on an entirely new meaning. I've said this before but digital is not just about building websites anymore; many of our customers are using digital to change the way their business operates, automate manual processes, and save millions of dollars in the process. Digital strategies are no longer confined to the marketing department; they're quickly becoming a boardroom priority.
Digital experiences are also getting more sophisticated. What used to be as simple as building a website now involves getting the right information, to the right user, at the right time, in the right context -- an idea I call B2One and talked about as part of my Big Reverse of the Web thesis. It's all about understanding the user's context and preferences to deliver the best next experience. We started investing in this area in late 2013, acquired a personalization company in 2014, and expect this trend to grow really big, especially as big data, the Internet of Things (IoT) and artificial intelligence mature.
Beyond personalization and contextualization, companies have a greater need for flexibility and freedom to integrate a variety of external services, ranging from commerce to marketing automation solutions. While there are plenty of point solutions on the market that achieve pieces of this puzzle, Acquia and Drupal uniquely provide a platform to do it all.
Acquia continues its growth
Acquia's growth is an indicator that businesses are already betting big on open source, cloud, personalization, and digital transformation. Looking at our numbers for 2015, it is hard to believe that last year was only our seventh full year as a revenue-generating business. Our new logo subscriptions -- the business that we get from companies new to Acquia -- continued its fast growth, while our renewal rates are among the best around.
To support our growth, we added $55 million in new venture capital funding in 2015, bringing Acquia's total raised to $188.6 million.
We dramatically increased headcount last year. In May, we moved into a beautiful new corporate headquarters in Boston, where we hosted a launch party with mayor Martin J. Walsh that established us as an anchor growth company for the city. Globally, we added 150 more employees in 2015, bringing us to 720 people. The executive team changed substantially in 2015, with the addition of Al Nugent as CISO, Preston Bradford as COO, Heather Hartford as Chief People Officer, and most recently Loren Jarrett as CMO. We also announced the appointment of Christine Komola, CFO of Staples, to Acquia's board of directors. We expanded European operations with the opening of a new office in Munich. Acquia now operates out of 10 global offices on three continents!
We made key investments in talent for our partner team, grew the number of Drupal certifications achieved globally, and strengthened our relationship with agency partners. We were proud to announce our WPP Global Alliance partnership this year, which brings Acquia closer to the organizations building the world's most amazing digital experiences.
Top industry analysts continued to recognize Acquia as a leading provider of digital experience software and services. In October 2015, we were named a "Strong Performer" in The Forrester Wave: Digital Experience Platforms Q4 2015. In August, Gartner named us a "Leader" in the 2015 Magic Quadrant for Web Content Management. This type of external validation supports Acquia as a viable alternative to proprietary solutions provided by companies like Adobe, IBM, Sitecore or Oracle.
Acquia Cloud continues to enable organizations to run their websites securely and reliably, while providing them with the tools to accelerate web development and shorten time to market. We manage over 13,000 AWS instances (virtual servers) powering approximately 197,000 websites. In December 2015, Acquia Cloud served 10.8 billion Apache hits and 6.2 billion Drupal bootstraps (requests handled by Drupal instead of one of different caching layers or the web server directly), with billions more served from our caching layers and Acquia Cloud Edge.
Last year, we launched Acquia Cloud support for the São Paulo, Brazil and Frankfurt, Germany regions. With our German expansion we became the first Drupal PaaS to offer pan-European high availability to accommodate EU data sovereignty requirements.
With the launch of Drupal 8 in late 2015, the Drupal community achieved our most significant release in the history of the platform. We implemented a more modern development framework, reimagined the usability and authoring experience, and made improvements that will help everyone build the multilingual, mobile and highly personalized digital experiences of the future. From how we model content and get content in and out the system, to how we assemble experiences on various devices, to how we scale that to millions of pageviews — it all got much better with Drupal 8.
Now that Drupal 8 is released, I'm convinced that it will attract new developers and site-builders to the platform. Nonetheless, the wait for Drupal 8 has been long and painful, temporarily slowing down much of the commercial Drupal ecosystem. Despite some turbulence, I'm proud that the Acquia team was a force in helping to push Drupal 8 over the finish line.
Acquia employs more than 150 Drupal experts, and has fixed upwards of 1,200 issues in Drupal 8. I reassigned our Drupal team from feature development (i.e. Spark) to working on criticals. This team was a major force in bringing the total number of criticals down from a high of 90 at the beginning of the year to zero in early October, through development, performance work, patch reviews, sprint coordination, and in helping to manage the Drupal 8 Accelerate program. To help jumpstart faster Drupal 8 adoption, Acquia is investing significantly in porting the top 50+ contributed modules. We have always believed in giving back more as a core part of our company's DNA. Our entire team is ready to enable and support companies working with Drupal 8.
Last year was an exciting one when it came to new products. We announced Content Hub, a cloud-based content distribution and discovery service. As more of our customers scale with Acquia across hundreds of sites, Content Hub lets authors and site owners reuse content from internal and external sites. And we added a critical commerce integration through a partnership with Hybris, which provides even more options for enterprises to drive commerce experiences.
We announced a variety of important security and compliance milestones that will be crucial to protecting our customers. First, we introduced Acquia Cloud Edge, a new DDoS security product developed in partnership with CloudFlare to keep our customers safer from external threats. Soon after, Acquia achieved HIPAA compliance upon passing an independent audit of Acquia Cloud Site Factory and Acquia Cloud. HIPAA compliance is significant because of Acquia's roster of healthcare customers, who require certain safeguards for data security and look to scale Acquia Cloud across their portfolio of sites.
In addition to this compliance milestone, our spam-blocking software, Mollom, has blocked over 10 billion spam comments.
Acquia Lift customers challenged some of our original assumptions about personalization. We worked to improve our Acquia Lift personalization product with the help of our customers, creating a new workflow and UX that supported more flexibility and freedom depending on the individual organization. In 2015, we learned a lot about the challenges organizations face when starting out with personalization and doubled down to help our customers become successful. Personalization will continue to be a huge focus of ours in 2016.
This year, we will be rolling out many new products and enhancing existing ones. Acquia Cloud will get a brand-new, responsive, card-based UX in early 2016, and we'll give development teams the ability to create on-demand environments. We will continue to focus on security enhancements from audits like SOC and ISO, as well as key control planes including FedRAMP, Single Sign-On, and much more. Our team was challenged to get the first cloud service running on our new grid architecture by the end of 2015. With a few hours to spare on the 31st, the "Uptime" service is now running on our grid architecture, a major milestone. A continued focus on developer tools, more microservices, personalization and a "jumpstart" Drupal 8 distribution are just some of the technology we will be rolling out in 2016. Overall, you'll see us getting faster, more secure and more efficient, and providing even more options for our customers to create highly personalized digital experiences.
Longer term, I'm very excited about Acquia's opportunity. I believe we've steered the company to be positioned at the right place at the right time. Time will tell, but 2016 promises to be another big year for Acquia.
File this under "better late than never". Before the year closes out, I wanted to post my 2015 DrupalCon Barcelona keynote video and slides. I archive all my DrupalCon keynotes on my site so anyone who is interested in taking a trip to memory lane or studying the evolution of Drupal, can check out all my previous DrupalCon keynotes.
My DrupalCon Barcelona keynote is focused on having a realistic, open and honest conversation about the state of Drupal. In it, I broke down my thoughts on Drupal's market position, development process, and "decoupled Drupal". You can watch the recording of my keynote or download a copy of my slides (PDF, 27 MB).