Selecting a client-side framework for Drupal

Last month, my blog post about whether a client-side framework is right for Drupal stimulated some excellent insights into how the future of the Drupal front end might look. There was broad agreement that incorporating more JavaScript into Drupal's administrative interface is important for a future-ready user experience.

I am confident that adopting a client-side framework through progressive decoupling will give us the best of both worlds. Of course, this does not mean I oppose fully decoupling through any other framework; in fact, I believe we should redouble our efforts toward a first-class API-first Drupal. But progressive decoupling means that we will be able to work toward a next-generation user experience without abandoning much of the work we've done so far.

With that in mind, I tasked a small team made up of various experts from the Drupal and various JavaScript communities with putting together a list of criteria and a comparison matrix to help us decide on the most appropriate client-side framework for progressive decoupling in Drupal.

JavaScript framework comparison matrix

After a hundred hours of work, we came up with a criteria document and comparison matrix (PDF version):

Special thanks to all of the following experts who provided review and input: Miško Hevery (creator of Angular; Google) and Igor Minar (technical lead for Angular; Google); Ed Faulkner (core maintainer for Ember); Amitai Burstein (Drupal and Elm contributor; Gizra); Sebastian Siemssen (Drupal contributor, Elm and React developer; Zensations); and John Albin Wilkins (Drupal 8 mobile initiative lead), Alex Bronstein (Drupal core maintainer; Acquia), Wim Leers (Drupal core contributor; Acquia), and Preston So (Drupal contributor, Acquia).

Though this is a good starting point that lays the groundwork for a formal adoption process in Drupal core, it is time to open our comparison for review by members of both the Drupal and JavaScript communities. We should also more rigorously evaluate these frameworks' appropriateness through actual development.

First, have we decided on the right criteria regardless of the frameworks themselves? This is probably the most important at this stage. While many organizations choose to adopt client-side frameworks for fully decoupled implementations, Drupal is the first to consider layering a framework on top to allow both richly dynamic and more traditional modules to coexist gracefully through progressive decoupling. What issues around templates, rendering, and client-side caching should we incorporate into these criteria? Is there anything missing or overemphasized?

Second, have we selected the right frameworks to focus on? Are there other frameworks, libraries, or even compile-to-JavaScript projects (such as Elm) that should be included in the matrix? In my view, it is best for Drupal to adopt a framework with an already large community and ecosystem that would allow us to more quickly bridge the gap and resolve any friction during adoption. To provide a good frame of reference, we've also included Backbone, Angular, and Knockout, all three slightly older among client-side frameworks. Others on our shortlist that we didn't consider due to low levels of adoption and small communities are Mithril, Riot, and Vue. Still other ambitious projects such as the Elm and ClojureScript languages are also candidates, but their adoption will mean more up-front work to support typical features of client-side frameworks, as well as buy-in into an approach where JavaScript is compiled from a different language.

Finally, have we drawn the right conclusions against these criteria? In other words, did we fill out the cells correctly? While they have been reviewed by some of the frameworks' experts, there might be unexpected gotchas or caveats.

I'm sharing the initial comparison matrix on my blog for maximum reach; I want both the Drupal community and the JavaScript framework communities, as well as the broader front-end community, to take note. After three installments on my blog ("The future of decoupled Drupal", "Should we decouple Drupal with a client-side framework?"), it's time to move the technical conversation to drupal.org. There is now an issue in the core issue queue on drupal.org to iterate on the matrix.

Preliminary conclusions

At the moment, the most promising candidates in the comparison matrix appear to be Angular 2, Ember, and React, given their technical robustness, relative suitability for progressively decoupled Drupal, and their strong levels of community support and broader adoption. Given that Backbone is already in core and several modules already rely on it, we have included it too.

What we've learned from talking to the different projects is that they are often converging on similar techniques and best practices; they are by and large adding support for Virtual DOM implementations or rehydration (seamless state transfer), and they are all obsessing over small payload size and performance, better testability, etc. Therefore it is important to focus on the fundamental, often philosophical, differences between the projects that will likely be unchanged in time; key architectural differences, their release cadence and stance on backward compatibility, their license, their governance model, their flexibility and learning curve, etc.

From a quick glance at the criteria and our needs, it seems that Ember is currently our best candidate, as it appears to have a slight technical edge overall. Ember 2.0 has an all-new rendering engine named Glimmer, and it has server-side rendering through FastBoot. On the other hand, however, Ember is quite bulky and opinionated (enforcing patterns for code structure) compared to other candidate frameworks. A more fundamental difference is that unlike Angular and React, which have corporate governance and funding, Ember is a community-driven project like Drupal.

While React is lightweight, it needs integration with a variety of other libraries in the React ecosystem to work as a full-fledged implementation, which gives it a steep learning curve from an implementation standpoint. Because React is a relatively young project, best practices are shifting quickly and making it less attractive. The Virtual DOM, among React's most compelling features, has also seen its core ideas filter into other framework projects. But more importantly, React is licensed with what I believe to be a potentially unacceptable patent clause, which states that an organization can no longer use React once it sues Facebook for any (unrelated) patent infringement. This has already generated some concerted pushback from both WordPress's Calypso and React contributors.

Angular 2 is a complete rewrite of Angular 1 to address issues with that version of the framework, including performance problems stemming from a large file size. The new version also introduces a new template engine that incorporates both directives familiar to Angular 1 developers and nestable component-based templates. But Angular 2's Apache 2.0 licensing is incompatible with Drupal's own GPLv2 license. While Drupal's PHP code and JavaScript code run in isolated processes, it appears that an Apache 2.0-licensed project can't be jointly distributed within an umbrella project that uses a GPLv2 license.

In addition to being at a slight technical disadvantage to Ember, the legal concerns with React and Angular make me believe Ember might be our best bet. But I'm not a lawyer nor a JavaScript expert, so I can't make this decision alone; I'm looking for lots of feedback, particularly from JavaScript experts, to determine the best option for Drupal.

Conclusion

Whatever the result of the debate around which client-side framework to adopt, I believe that Drupal needs to move quickly to embrace a framework that will aid development of a progressively decoupled architecture and corresponding user experience improvements. By providing some baseline criteria and including our accomplished community, I have no doubt we can reach this decision quickly but also intelligently.

Special thanks to Preston So for contributions to this blog post.

Acquia retrospective 2015

As many of my loyal blog readers know, I sit down to write a retrospective at the end of each year. It's helpful from the perspective of seeing how far Acquia has come in a single year, but it should also give you a glimpse of what is in store for Acquia in 2016 and beyond. If you find it interesting to take a look back at previous retrospectives, here they are: 2014, 2013, 2012, 2011, 2010 and 2009. These posts provide a pretty detailed overview of Acquia's trajectory as a company.

Market trends

Since Acquia started eight years ago, we've believed that open source offers significant advantages over proprietary software because of its faster innovation, higher quality, freedom from vendor lock-in, greater security, and lower total cost of ownership. Early in our life as a company, we made a big bet that open source combined with the cloud, offered through a subscription model rather than perpetual license, would be a very compelling solution for the market. Few people believed us at the time, but now it is clear that our early vision is starting to pay off; perpetual software licenses are on the decline and Deutsche Bank analyst Karl Keirstead recently called cloud and open source the two leading themes in Silicon Valley.

The market demand for Acquia's digital business platform continues to grow; three of the top analyst firms, IDC, Gartner and Forrester have all named digital business transformation a top strategic imperative for the C-suite in 2016 and beyond. Open source, cloud computing, big data, the Internet of Things (IoT), and artificial intelligence are all catalysts for the expansion of digital transformation into all corners of the organization.

Organizations are rapidly expanding the range of digital interactions with their customers and partners, moving Drupal and Acquia to the core of their business. There is a growing focus on personalization and data-driven automation, which bodes well for products like Acquia Lift. In general, I believe that the growing reliance on digital provides Drupal and Acquia with a multi-decade opportunity.

Open source

Within the last 12 months, some of the largest technology companies including Apple, Google, Microsoft and Facebook have open-sourced key components of their business. There are many motivations for this shift. According to Apple, the company open-sourced its Swift programming language to extend it to new platforms. Google open-sourced TensorFlow, its artificial intelligence platform to make an even bigger impact outside Google, even though the company employs 2,000 engineers working on artificial intelligence alone. Microsoft open-sourced .NET to increase its relevance with developers and play nicely with other operating systems. In Deutsche Bank's 2016 predictions, Keirstead says "open source keeps eating the world", causing major price deflation for the traditional enterprise software industry. Whether the motive is faster innovation or increased adoption, companies are relying less and less on proprietary software and embracing open source.

Cloud adoption

Amazon SVP of Web Services, Andy Jassy, explained in his 2015 AWS re:Invent presentation that websites have been a "critical gateway" to AWS' wider cloud adoption in the enterprise. His rationale: nearly every organization has one or more websites, and many aren't considered "mission-critical applications". Therefore, most organizations feel comfortable moving some or all of their websites to the cloud. Websites are an important stepping stone for organizations to build up the knowledge and confidence to migrate their entire businesses to the cloud.

As cloud adoption grows, we're seeing our customers use a mixture of SaaS, PaaS and IaaS. In particular the Platform-as-a-Service (PaaS) model continues to grow fast, which is great news for Acquia Cloud. A growing number of enterprises are choosing PaaS ahead of Infrastructure-as-a-Service (IaaS) to save time and money on building, scaling and maintaining infrastructure so they can focus on building websites. Gartner sizes the PaaS market at $4.1B in 2015, attaining a compound annual growth rate (CAGR) of 65%+ through 2018. We are a few years ahead of our competitors (Adobe, Sitecore, IBM, Oracle) when it comes to PaaS, and I don't see that changing in 2016.

The migration to the cloud is only getting started. When I met with Eric Schmidt this year, he told me that he believes Google's cloud business could outpace its advertising business in five years. To put that in context, Google made more than $65 billion in advertising in 2015, roughly 90% of its total revenue. I don't think Google Cloud can possibly grow that fast, but directionally it's an eye-opening goal. Amazon's cloud business, bigger than its four closest competitors combined (including Google's cloud business), generated roughly $7 billion in revenues in 2015 and is expected to grow 80% year-over-year. Needless to say, it is exciting for Acquia be a "critical gateway" in such a massive movement to the cloud.

Andy jassy reinvent
A photo of Andy Jassy's keynote at AWS re:Invent 2015.

Digital experiences

As the web evolves, the idea of a "digital business" takes on an entirely new meaning. I've said this before but digital is not just about building websites anymore; many of our customers are using digital to change the way their business operates, automate manual processes, and save millions of dollars in the process. Digital strategies are no longer confined to the marketing department; they're quickly becoming a boardroom priority.

Digital experiences are also getting more sophisticated. What used to be as simple as building a website now involves getting the right information, to the right user, at the right time, in the right context -- an idea I call B2One and talked about as part of my Big Reverse of the Web thesis. It's all about understanding the user's context and preferences to deliver the best next experience. We started investing in this area in late 2013, acquired a personalization company in 2014, and expect this trend to grow really big, especially as big data, the Internet of Things (IoT) and artificial intelligence mature.

Beyond personalization and contextualization, companies have a greater need for flexibility and freedom to integrate a variety of external services, ranging from commerce to marketing automation solutions. While there are plenty of point solutions on the market that achieve pieces of this puzzle, Acquia and Drupal uniquely provide a platform to do it all.

Acquia continues its growth

Acquia's growth is an indicator that businesses are already betting big on open source, cloud, personalization, and digital transformation. Looking at our numbers for 2015, it is hard to believe that last year was only our seventh full year as a revenue-generating business. Our new logo subscriptions -- the business that we get from companies new to Acquia -- continued its fast growth, while our renewal rates are among the best around.

To support our growth, we added $55 million in new venture capital funding in 2015, bringing Acquia's total raised to $188.6 million.

We dramatically increased headcount last year. In May, we moved into a beautiful new corporate headquarters in Boston, where we hosted a launch party with mayor Martin J. Walsh that established us as an anchor growth company for the city. Globally, we added 150 more employees in 2015, bringing us to 720 people. The executive team changed substantially in 2015, with the addition of Al Nugent as CISO, Preston Bradford as COO, Heather Hartford as Chief People Officer, and most recently Loren Jarrett as CMO. We also announced the appointment of Christine Komola, CFO of Staples, to Acquia's board of directors. We expanded European operations with the opening of a new office in Munich. Acquia now operates out of 10 global offices on three continents!

Acquia state street
A photo of Acquia's headquarters in downtown Boston.

We made key investments in talent for our partner team, grew the number of Drupal certifications achieved globally, and strengthened our relationship with agency partners. We were proud to announce our WPP Global Alliance partnership this year, which brings Acquia closer to the organizations building the world's most amazing digital experiences.

Top industry analysts continued to recognize Acquia as a leading provider of digital experience software and services. In October 2015, we were named a "Strong Performer" in The Forrester Wave: Digital Experience Platforms Q4 2015. In August, Gartner named us a "Leader" in the 2015 Magic Quadrant for Web Content Management. This type of external validation supports Acquia as a viable alternative to proprietary solutions provided by companies like Adobe, IBM, Sitecore or Oracle.

Acquia Cloud continues to enable organizations to run their websites securely and reliably, while providing them with the tools to accelerate web development and shorten time to market. We manage over 13,000 AWS instances (virtual servers) powering approximately 197,000 websites. In December 2015, Acquia Cloud served 10.8 billion Apache hits and 6.2 billion Drupal bootstraps (requests handled by Drupal instead of one of different caching layers or the web server directly), with billions more served from our caching layers and Acquia Cloud Edge.

Last year, we launched Acquia Cloud support for the São Paulo, Brazil and Frankfurt, Germany regions. With our German expansion we became the first Drupal PaaS to offer pan-European high availability to accommodate EU data sovereignty requirements.

Drupal 8

With the launch of Drupal 8 in late 2015, the Drupal community achieved our most significant release in the history of the platform. We implemented a more modern development framework, reimagined the usability and authoring experience, and made improvements that will help everyone build the multilingual, mobile and highly personalized digital experiences of the future. From how we model content and get content in and out the system, to how we assemble experiences on various devices, to how we scale that to millions of pageviews — it all got much better with Drupal 8.

Now that Drupal 8 is released, I'm convinced that it will attract new developers and site-builders to the platform. Nonetheless, the wait for Drupal 8 has been long and painful, temporarily slowing down much of the commercial Drupal ecosystem. Despite some turbulence, I'm proud that the Acquia team was a force in helping to push Drupal 8 over the finish line.

Acquia employs more than 150 Drupal experts, and has fixed upwards of 1,200 issues in Drupal 8. I reassigned our Drupal team from feature development (i.e. Spark) to working on criticals. This team was a major force in bringing the total number of criticals down from a high of 90 at the beginning of the year to zero in early October, through development, performance work, patch reviews, sprint coordination, and in helping to manage the Drupal 8 Accelerate program. To help jumpstart faster Drupal 8 adoption, Acquia is investing significantly in porting the top 50+ contributed modules. We have always believed in giving back more as a core part of our company's DNA. Our entire team is ready to enable and support companies working with Drupal 8.

Acquia products

Last year was an exciting one when it came to new products. We announced Content Hub, a cloud-based content distribution and discovery service. As more of our customers scale with Acquia across hundreds of sites, Content Hub lets authors and site owners reuse content from internal and external sites. And we added a critical commerce integration through a partnership with Hybris, which provides even more options for enterprises to drive commerce experiences.

We announced a variety of important security and compliance milestones that will be crucial to protecting our customers. First, we introduced Acquia Cloud Edge, a new DDoS security product developed in partnership with CloudFlare to keep our customers safer from external threats. Soon after, Acquia achieved HIPAA compliance upon passing an independent audit of Acquia Cloud Site Factory and Acquia Cloud. HIPAA compliance is significant because of Acquia's roster of healthcare customers, who require certain safeguards for data security and look to scale Acquia Cloud across their portfolio of sites.

In addition to this compliance milestone, our spam-blocking software, Mollom, has blocked over 10 billion spam comments.

Acquia Lift customers challenged some of our original assumptions about personalization. We worked to improve our Acquia Lift personalization product with the help of our customers, creating a new workflow and UX that supported more flexibility and freedom depending on the individual organization. In 2015, we learned a lot about the challenges organizations face when starting out with personalization and doubled down to help our customers become successful. Personalization will continue to be a huge focus of ours in 2016.

Looking ahead

This year, we will be rolling out many new products and enhancing existing ones. Acquia Cloud will get a brand-new, responsive, card-based UX in early 2016, and we'll give development teams the ability to create on-demand environments. We will continue to focus on security enhancements from audits like SOC and ISO, as well as key control planes including FedRAMP, Single Sign-On, and much more. Our team was challenged to get the first cloud service running on our new grid architecture by the end of 2015. With a few hours to spare on the 31st, the "Uptime" service is now running on our grid architecture, a major milestone. A continued focus on developer tools, more microservices, personalization and a "jumpstart" Drupal 8 distribution are just some of the technology we will be rolling out in 2016. Overall, you'll see us getting faster, more secure and more efficient, and providing even more options for our customers to create highly personalized digital experiences.

Longer term, I'm very excited about Acquia's opportunity. I believe we've steered the company to be positioned at the right place at the right time. Time will tell, but 2016 promises to be another big year for Acquia.

The oil bottom

The rapid decline of oil prices have dominated the financial news the second half of 2015. Each time you turn on CNBC chances are a Wall Street analyst is discussing the latest movement in oil prices. Just the other day one proclaimed "I believe we hit the bottom!".

The price of oil has fallen from $110 per barrel in 2014 to $35 at the end of 2015. The reason for this massive drop boils down to the economics of supply and demand. On the supply side, the world produces more and more oil. The United States and Canada keep raising production year after year. The Organization of the Petroleum Exporting Countries (OPEC), a cartel of oil producers including Saudi Arabia, Iraq, the United Arab Emirates and other gulf allies, recently agreed to increase the production of oil because they don't want to lose marketshare and because they know it drives the U.S. shale oil producers out of the market. Last, but not least, Iran is about to re-enter the world of oil markets in 2016 after years of sanctions and isolation. On the demand side, we see that demand for fuel is lagging a bit, resulting in an overproduction of oil.

When there is more supply than demand and no one is cutting production, the price of oil will keep dropping, and eventually the world might run out of places to store all the excess oil. Many oil producer are struggling to maintain their profitability and incurring losses.

I'm not an expert on the oil industry, and I certainly understand that it is impossible to predict the future, but to me it seems the worst is yet to come. As I see it, there are four scenarios that could play out: (1) oil producers agree to balance supply and demand by cutting production, (2) the demand for oil picks up as the world's economies strengthen, (3) the supply of oil starts to slow down as oil producers can't afford the investments to sustain production, or (4) enough oil producers go out business and supply and demand is rebalanced that way.

I don't have a crystal ball, but I consider a combination of (3) and (4) to be the most likely outcome. Oil production companies have been slashing their capital budgets throughout 2015 to stay in business. Multibillion-dollar explorations and production projects have been delayed or cancelled. This should cause oil production to drop naturally in the future. To weather the storm, cutting capital budgets might not be enough; oil companies might have to increase their debt levels, and those that can't raise more debt might have to dig deeper and cut their dividend. I wouldn't be surprised if we saw various oil producers cut their dividends in the second half of 2016, and witnessed a series of bankruptcies or acquisitions in 2017.

Because of the drop in oil price, many of the oil companies have become unusually cheap, often trading at 10 year lows. For people with a 5-10 year time horizon, it is probably a great time to buy stock in one or more integrated oil and gas super majors, especially if you stick with the few that are still able to support their dividend. At the same time, I believe the worst is yet to come. As someone investing part of his paycheck for retirement, I'll sit on the sideline a bit longer until the above scenario starts to play out. In the mean time, I'll keep wondering and learning why Wall Street analysts keep calling the oil bottom ...

Disclaimer: I'm long XOM with a cost basis of $68 per share, and would probably add to my position if XOM hits $70 per share again. Before making an investments, you should do your own proper due diligence. Any material in this article should be considered general information, and not a formal investment recommendation.

State of Drupal presentation (September 2015)

File this under "better late than never". Before the year closes out, I wanted to post my 2015 DrupalCon Barcelona keynote video and slides. I archive all my DrupalCon keynotes on my site so anyone who is interested in taking a trip to memory lane or studying the evolution of Drupal, can check out all my previous DrupalCon keynotes.

My DrupalCon Barcelona keynote is focused on having a realistic, open and honest conversation about the state of Drupal. In it, I broke down my thoughts on Drupal's market position, development process, and "decoupled Drupal". You can watch the recording of my keynote or download a copy of my slides (PDF, 27 MB).

In addition, here are three related blog posts I wrote about the future of decoupled Drupal, having to pick a JavaScript framework for Drupal, and a proposal for how to evolve our development process, which will allow us to ship new features every 6 months rather than every 4-5 years.

Should we decouple Drupal with a client-side framework?

As user experiences evolve from static pages to application-like experiences, end users' expectations of websites have become increasingly demanding. The Facebook newsfeed, the Gmail inbox, and the Twitter live stream are all compelling examples that form a baseline for the application-like experiences users now take for granted.

Many of Drupal's administrative interfaces and Drupal sites could benefit from a similarly seamless, instantaneous user experience. Drupal's user interfaces for content modeling (Field UI), layout management (Panels), and block management would benefit from no page refreshes, instant previews, and interface previews. These traits could also enrich the experience of site visitors; Drupal's commenting functionality could similarly gain from these characteristics and resemble an experience more like the commenting experience found in Facebook.

As Drupal's project lead, I ask myself: how can our community feel enabled and encouraged to start building rich user experiences?

In recent years, the emergence of decoupled architectures with client-side frameworks and libraries have helped our industry build these experiences. Does that mean we have to decouple Drupal's front-facing site experience (for site visitors or end users) and/or the administration experience (for content editors and site builders)? If so, should Drupal decouple the administration layer differently from the front-facing site experience? By extension, should a client-side framework guide this change?

Here is my current thinking: in the short term, Drupal should work toward a next-generation user experience under progressive decoupling for both the administration layer and the end user experience. At the same time, we should enable fully decoupled end-user and administrative experiences to be built on Drupal too. In my view, the best way to achieve this is to formally standardize on a full-featured MV* framework (e.g. Angular, Ember, Backbone, and Knockout) or a component-based view library (e.g. React, Vue, and Riot). In this blog post, I want to kick off the discussion and walk you through my current position in some more detail.

Should we decouple Drupal itself?

Should we decouple drupal framework responsibility
A framework can assume responsibility over very little, such as with Backbone, or a great deal of the stack, including the rendering process.

The question we have to answer is: do we stand to benefit from decoupling Drupal? In this context, decoupled Drupal refers to a separation between the Drupal back end and one or more front ends. In a decoupled architecture, the back end is built in a server-side language like PHP, the front end is built using a client-side framework written in JavaScript, and data is transferred between the two. Decoupling your content management system gives front-end developers greater control over an application or site's rendered markup and user experience, and using a client-side framework gives them better tools to build application-like experiences.

There is no doubt that Drupal's administration layer is very application-like and would benefit from an experience that is more interactive. For the end-user experience, a decoupled implementation is not always the best option. Some sites may not need or want the application-like interactivity that a client-side framework would provide, since not every site needs interaction. For instance, news sites or blogs do not need much interactivity, custom applications need a great deal, while e-commerce sites lie somewhere in the middle. It's not clear-cut, so let's look at our options in more detail.

The first option is to preserve traditional Drupal, the status quo where there is no decoupling. This would leave the Drupal theme layer intact apart from additional JavaScript embedded freely by the developer, but as a result, the onus is on the developer to introduce client-side user experience improvements. The likely outcome is that different modules will start using different client-side frameworks. Such a scenario would steepen Drupal's learning curve, impede collaboration, make Drupal more brittle, and damage performance on pages whose display is influenced by front-end components of multiple modules.

The second option is to work toward full decoupling, in which Drupal is solely a themeless content repository for client-side applications. But fully decoupling both the Drupal administrative interface and the front end would require us to rebuild much of our page building functionality (e.g. site preview, layouts, and block placement) and to abandon many performance benefits of Drupal 8. Moreover, more JavaScript, especially on the server side, means complicating infrastructure requirements and a large-scale overhaul of server-side code.

The third option is what I call progressive decoupling, where Drupal renders templates on the server to provide an initial application state, and JavaScript then manipulates that state. This means we can have the best of both worlds: unobtrusive JavaScript and all the out-of-the-box performance advantages of Drupal 8 — a short time to first paint (when the user sees the page materialize) via BigPipe as well as interface previews and static fallbacks during the time to first interaction (when the user can perform an action). Plus, while some additional JavaScript would enter the picture, progressive decoupling mitigates the amount of change needed across the stack by using a baseline of existing theme functionality.


Approach User experience Developer experience Ease of implementation
Traditional coupling Page refreshes
No immediate feedback
Interrupted workflow
Theme layer preserved
Modules in mostly PHP
Ad-hoc client-side code
PHP and minimal JavaScript
No server-side change
No client-side change
Progressive decoupling No page refreshes
Immediate feedback
Uninterrupted workflow
Theme layer preserved
Modules in PHP and JavaScript
Unified client-side code
Both PHP and JavaScript
Some server-side change
Some client-side change
Full decoupling No page refreshes
Immediate feedback
Uninterrupted workflow
Theme layer replaced
Module rewrites in JavaScript
Unified client-side code
Primarily JavaScript
Much server-side change
Much client-side change

How should Drupal decouple its user experience?

Should we decouple Drupal front end experiences?
This diagram shows some of the possible front-end experiences that could rely on a single Drupal back-end. In full decoupling, a custom application built using a client-side framework encompasses the entire front-end. In progressive decoupling, JavaScript manipulates an initial state already provided by the theme layer.

Drupal's administration layer (content editor and site builder experience) is effectively an application. Fully decoupling may be the appropriate call to achieve the best possible user experience for creating, managing and presenting content. However, rewriting the administration layer from the ground up is a monumental task, especially since its modules provide powerful interfaces that allow site builders to build robust, complex sites without a line of code.

The same expectations for application-like interactivity often hold for the end-user experience: users expect shopping carts, comments, notifications, and searches to be updated instantaneously, without having to wait for a page to load.

For both the administration layer and the end-user experience, I believe the Drupal front end should not be fully decoupled out of the box. We should advance from our traditional paradigm and default to progressive decoupling. It allows us to achieve the user experience we want without significant downsides, since not every use case would benefit from fully decoupling. Through progressive decoupling, Drupal could potentially reach the ideals of the assembled web more quickly by preserving a tight link between Drupal modules and their front-end components.

Nonetheless, we should empower people building fully decoupled sites and applications. Depending on the use case, Drupal 8 is a good match for decoupled applications but we should improve and extend Drupal's REST API, enhance contributed modules such as Services, and shore up new features such as GraphQL (demo video) so more functionality can be decoupled. Front-end developers can then use any framework of their choice — whether it is Angular, Ember, React, or something else — to build a fully decoupled administrative application.

Should Drupal standardize on a single framework?

All things considered, I do believe Drupal should standardize on a single client-side framework, but it should only make such an explicit recommendation for progressively decoupled Drupal, not fully decoupled architectures. It would result in a more seamless user experience, better compatibility across interactive components in modules, maximum code reuse, a more consistent front-end developer experience, more maintainable code, and better performance as we don't have to load multiple frameworks.

Despite the potential benefits, there are also good reasons not to embrace a single client-side framework. New front-end frameworks are being created at what feels like an unsustainable pace; every nine months there is a new kid on the block. It's hard for a project as large as Drupal to embrace a single technology when there is no guarantee of its longevity.

For instance, Backbone, with its underlying library Underscore, currently powers interactions in the toolbar and in-place editing in Drupal 8. Though Drupal could expand the scope of Backbone in core and encourage front-end developers to build with it, it means buying even further into a framework that is quite old among its peers.

To deal with the fast-evolving nature of the front-end landscape, we need to be thoughtful about which framework we choose, to reassess our choice from time to time, and to make sure we can migrate fairly easily if we so decide.

What client-side framework should Drupal standardize on?

Assuming we agree that embracing a single client-side framework makes sense for Drupal, there are actually three additional questions: what framework to standardize on, how to do it, and when to decide.

I'm the first to admit I'm not the best person to answer the first question. As I'm not a front-end developer, I'm looking at you, the Drupal community, to help answer this question. I'd hope that the chosen framework aligns well with both our architecture (modular, flexible) and community (collaborative, community-driven).

The second question — how to standardize on a framework — I can help answer. On the one extreme, Drupal could be opinionated and ship a client-side framework with Drupal core, meaning that every installation of Drupal ships with the chosen framework. This would be much like the adoptions of jQuery and Backbone.js. On the other end of the spectrum, Drupal could recommend a specific framework but not ship with it. Finally, somewhere in between, Drupal could provide a default standard framework in core but make it easy to replace with it a framework of a developer's choice, though the likelihood of this is quite small. This is akin to Drupal core shipping with a template engine (i.e. PHPTemplate) that could be (but was rarely) replaced with another. Ultimately, I think we get the best result if Drupal ships with a specific framework—much like the adoption of jQuery in Drupal 5.

The last question, when to standardize on a framework, is important too. I would recommend we experiment with possible directions as soon as possible in order to decide on a final vision sooner rather than later.

Conclusion

I believe that, for now, it makes more sense to progressively decouple Drupal sites and their administration layer by first building our pages with Drupal. Once the page is loaded, we can let a unified client-side framework take over as much of the page as needed to foster a next-generation user experience without reinventing the wheel or alienating developers.

Whatever the result of this debate, Drupal needs your help. We will need initiatives to build a foundation for better decoupled experiences. For all decoupled implementations - whether full or progressive - we will want to redouble our efforts on existing web services to create the best APIs for client-side applications. As for progressively decoupled experiences, we will need a process for selecting a client-side framework and defining how the existing Drupal JavaScript behaviors system would need to evolve. Finally, we will need to coordinate all of these efforts to focus on offering a better administrative experience.

That is not all! We will need module developers to bring rich interactions to their user interfaces with the help of the framework we choose. We will need designers to guide module developers in building a graceful user interface. We will need front-end developers to demonstrate how they want to develop the user experiences that will define Drupal for years to come. We will need users to test and validate all of our work. It will be tough going, but together, we can ensure Drupal stays ahead of the curve well into the future.

Special thanks to Preston So and Wim Leers for contributions to this blog post and to Moshe Weitzman and Gábor Hojtsy for feedback during its writing.

Pages

Dries Buytaert is the original creator and project lead of Drupal and the co-founder and CTO of Acquia. He writes about Drupal, startups, business, photography and building the world we want to exist in.

Updates from Dries straight to your mailbox