File this under "better late than never". Before the year closes out, I wanted to post my 2015 DrupalCon Barcelona keynote video and slides. I archive all my DrupalCon keynotes on my site so anyone who is interested in taking a trip to memory lane or studying the evolution of Drupal, can check out all my previous DrupalCon keynotes.
My DrupalCon Barcelona keynote is focused on having a realistic, open and honest conversation about the state of Drupal. In it, I broke down my thoughts on Drupal's market position, development process, and "decoupled Drupal". You can watch the recording of my keynote or download a copy of my slides (PDF, 27 MB).
As user experiences evolve from static pages to application-like experiences, end users' expectations of websites have become increasingly demanding. The Facebook newsfeed, the Gmail inbox, and the Twitter live stream are all compelling examples that form a baseline for the application-like experiences users now take for granted.
Many of Drupal's administrative interfaces and Drupal sites could benefit from a similarly seamless, instantaneous user experience. Drupal's user interfaces for content modeling (Field UI), layout management (Panels), and block management would benefit from no page refreshes, instant previews, and interface previews. These traits could also enrich the experience of site visitors; Drupal's commenting functionality could similarly gain from these characteristics and resemble an experience more like the commenting experience found in Facebook.
As Drupal's project lead, I ask myself: how can our community feel enabled and encouraged to start building rich user experiences?
In recent years, the emergence of decoupled architectures with client-side frameworks and libraries have helped our industry build these experiences. Does that mean we have to decouple Drupal's front-facing site experience (for site visitors or end users) and/or the administration experience (for content editors and site builders)? If so, should Drupal decouple the administration layer differently from the front-facing site experience? By extension, should a client-side framework guide this change?
Here is my current thinking: in the short term, Drupal should work toward a next-generation user experience under progressive decoupling for both the administration layer and the end user experience. At the same time, we should enable fully decoupled end-user and administrative experiences to be built on Drupal too. In my view, the best way to achieve this is to formally standardize on a full-featured MV* framework (e.g. Angular, Ember, Backbone, and Knockout) or a component-based view library (e.g. React, Vue, and Riot). In this blog post, I want to kick off the discussion and walk you through my current position in some more detail.
Should we decouple Drupal itself?
There is no doubt that Drupal's administration layer is very application-like and would benefit from an experience that is more interactive. For the end-user experience, a decoupled implementation is not always the best option. Some sites may not need or want the application-like interactivity that a client-side framework would provide, since not every site needs interaction. For instance, news sites or blogs do not need much interactivity, custom applications need a great deal, while e-commerce sites lie somewhere in the middle. It's not clear-cut, so let's look at our options in more detail.
|Approach||User experience||Developer experience||Ease of implementation|
No immediate feedback
Theme layer preserved
Modules in mostly PHP
Ad-hoc client-side code
No server-side change
No client-side change
No page refreshes
Theme layer preserved
Unified client-side code
Some server-side change
Some client-side change
No page refreshes
Theme layer replaced
Unified client-side code
Much server-side change
Much client-side change
How should Drupal decouple its user experience?
Drupal's administration layer (content editor and site builder experience) is effectively an application. Fully decoupling may be the appropriate call to achieve the best possible user experience for creating, managing and presenting content. However, rewriting the administration layer from the ground up is a monumental task, especially since its modules provide powerful interfaces that allow site builders to build robust, complex sites without a line of code.
The same expectations for application-like interactivity often hold for the end-user experience: users expect shopping carts, comments, notifications, and searches to be updated instantaneously, without having to wait for a page to load.
For both the administration layer and the end-user experience, I believe the Drupal front end should not be fully decoupled out of the box. We should advance from our traditional paradigm and default to progressive decoupling. It allows us to achieve the user experience we want without significant downsides, since not every use case would benefit from fully decoupling. Through progressive decoupling, Drupal could potentially reach the ideals of the assembled web more quickly by preserving a tight link between Drupal modules and their front-end components.
Nonetheless, we should empower people building fully decoupled sites and applications. Depending on the use case, Drupal 8 is a good match for decoupled applications but we should improve and extend Drupal's REST API, enhance contributed modules such as Services, and shore up new features such as GraphQL (demo video) so more functionality can be decoupled. Front-end developers can then use any framework of their choice — whether it is Angular, Ember, React, or something else — to build a fully decoupled administrative application.
Should Drupal standardize on a single framework?
All things considered, I do believe Drupal should standardize on a single client-side framework, but it should only make such an explicit recommendation for progressively decoupled Drupal, not fully decoupled architectures. It would result in a more seamless user experience, better compatibility across interactive components in modules, maximum code reuse, a more consistent front-end developer experience, more maintainable code, and better performance as we don't have to load multiple frameworks.
Despite the potential benefits, there are also good reasons not to embrace a single client-side framework. New front-end frameworks are being created at what feels like an unsustainable pace; every nine months there is a new kid on the block. It's hard for a project as large as Drupal to embrace a single technology when there is no guarantee of its longevity.
For instance, Backbone, with its underlying library Underscore, currently powers interactions in the toolbar and in-place editing in Drupal 8. Though Drupal could expand the scope of Backbone in core and encourage front-end developers to build with it, it means buying even further into a framework that is quite old among its peers.
To deal with the fast-evolving nature of the front-end landscape, we need to be thoughtful about which framework we choose, to reassess our choice from time to time, and to make sure we can migrate fairly easily if we so decide.
What client-side framework should Drupal standardize on?
Assuming we agree that embracing a single client-side framework makes sense for Drupal, there are actually three additional questions: what framework to standardize on, how to do it, and when to decide.
I'm the first to admit I'm not the best person to answer the first question. As I'm not a front-end developer, I'm looking at you, the Drupal community, to help answer this question. I'd hope that the chosen framework aligns well with both our architecture (modular, flexible) and community (collaborative, community-driven).
The second question — how to standardize on a framework — I can help answer. On the one extreme, Drupal could be opinionated and ship a client-side framework with Drupal core, meaning that every installation of Drupal ships with the chosen framework. This would be much like the adoptions of jQuery and Backbone.js. On the other end of the spectrum, Drupal could recommend a specific framework but not ship with it. Finally, somewhere in between, Drupal could provide a default standard framework in core but make it easy to replace with it a framework of a developer's choice, though the likelihood of this is quite small. This is akin to Drupal core shipping with a template engine (i.e. PHPTemplate) that could be (but was rarely) replaced with another. Ultimately, I think we get the best result if Drupal ships with a specific framework—much like the adoption of jQuery in Drupal 5.
The last question, when to standardize on a framework, is important too. I would recommend we experiment with possible directions as soon as possible in order to decide on a final vision sooner rather than later.
I believe that, for now, it makes more sense to progressively decouple Drupal sites and their administration layer by first building our pages with Drupal. Once the page is loaded, we can let a unified client-side framework take over as much of the page as needed to foster a next-generation user experience without reinventing the wheel or alienating developers.
That is not all! We will need module developers to bring rich interactions to their user interfaces with the help of the framework we choose. We will need designers to guide module developers in building a graceful user interface. We will need front-end developers to demonstrate how they want to develop the user experiences that will define Drupal for years to come. We will need users to test and validate all of our work. It will be tough going, but together, we can ensure Drupal stays ahead of the curve well into the future.
Volkswagen's recent emissions scandal highlighted the power that algorithms wield over our everyday lives. As technology advances and more everyday objects are driven almost entirely by software, it's become clear that we need a better way to catch cheating software and keep people safe.
A solution could be to model regulation of the software industry after the US Food and Drug Administration's oversight of the food and drug industry. The parallels are closer than you might think.
The case for tighter regulation
When Volkswagen was exposed for programming its emissions-control software to fool environmental regulators, many people called for more transparency and oversight over the technology.
One option discussed by the software community was to open-source the code behind these testing algorithms. This would be a welcome step forward, as it would let people audit the source code and see how the code is changed over time. But this step alone would not solve the problem of cheating software. After all, there is no guarantee that Volkswagen would actually use the unmodified open-sourced code.
Open-sourcing code would also fail to address other potential dangers. Politico reported earlier this year that Google's algorithms could influence the outcomes of presidential elections, since some candidates could be featured more prominently in its search results.
Research by the American Institute for Behavioral Research and Technology has also shown that Google search results could shift voting preferences by 20% or more (up to 80% in certain demographic groups). This could potentially flip the margins of voting elections worldwide. But since Google's private algorithm is a core part of its competitive advantage, open-sourcing it is not likely to be an option.
The same problem applies to the algorithms used in DNA testing, breathalyzer tests and facial recognition software. Many defense attorneys have requested access to the source code for these tools to verify the algorithms' accuracy. But in many cases, these requests are denied, since the companies that produce the proprietary criminal justice algorithms fear a threat to their businesses' bottom line. Yet clearly we need some way to ensure the accuracy of software that could put people behind bars.
What we can learn from the FDA
So how exactly could software take a regulatory page from the FDA in the United States? Before the 20th century, the government made several attempts to regulate food and medicine, but abuse within the system was still rampant. Food contamination caused widespread illness and death, particularly within the meatpacking industry.
Meanwhile, the rise of new medicines and vaccines promised to eradicate diseases, including smallpox. But for every innovation, there seemed to be an equal amount of extortion by companies making false medical claims or failing to disclose ingredients. The reporting of journalists like Upton Sinclair made it abundantly clear by the early 1900s that the government needed to intervene to protect people and establish quality standards.
In 1906, President Theodore Roosevelt signed the Food and Drug Act into law, which prevented false advertising claims, set sanitation standards, and served as a watchdog for companies that could cause harm to consumers' welfare. These first rules and regulations served as a foundation for our modern-day FDA, which is critical to ensuring that products are safe for consumers.
The FDA could be a good baseline model for software regulation in the US and countries around the world, which have parallel FDA organizations including the European Medicines Agency, Health Canada, and the China Food and Drug Administration.
Just as the FDA ensures that major pharmaceutical companies aren't lying about the claims they make for drugs, there should be a similar regulator for software to ensure that car companies are not cheating customers and destroying the environment in the process. And just as companies need to disclose food ingredients to prevent people from ingesting poison, companies like Google should be required to provide some level of guarantee that they won't intentionally manipulate search results that could shape public opinion.
It's still relatively early days when it comes to discovering the true impact of algorithms in consumers' lives. But we should establish standards to prevent abuse sooner rather than later. With technology already affecting society on a large scale, we need to address emerging ethical issues head-on.
(I originally wrote this blog post as a guest article for Quartz.)
One thing that is exciting to me, is how much we appear to have gotten right in Drupal 8. The other day, for example, I stumbled upon a recent article from the LinkedIn engineering team describing how they completely changed how their homepage is built. Their primary engineering objective was to deliver the fastest page load time possible, and one of the crucial ingredients to achieve that was Facebook's BigPipe.
When a very high-profile, very high-traffic, highly personalized site like LinkedIn uses the same technique as Drupal 8, that solidifies my belief in Drupal 8.
LinkedIn supports both server-side and client-side rendering. While Drupal 8 does server-side rendering, we're still missing explicit support for client-side rendering. The advantage of client-side rendering versus server-side rendering is debatable. I've touched upon it in my blog post on progressive decoupling, but I'll address the topic of client-side rendering in a future blog post.
However, there is also something LinkedIn could learn from Drupal! Every component of a LinkedIn page that should be delivered via BigPipe needs to write BigPipe-specific code which is prone to errors and requires all engineers to be familiar with BigPipe. Drupal 8 on the other hand has a level of abstraction that allows BigPipe to work without the need for BigPipe-specific code. Thanks to Drupal's higher-level API, Drupal module developers don't have to understand BigPipe: Drupal 8 knows what page components are poorly cacheable or not cacheable at all, and what page components are renderable in isolation, and uses that information to automatically optimize the delivery of page components using BigPipe.
It is exciting to see Drupal support the advanced techniques that were previously only within reach of the top 50 most visited sites of the world! Drupal's BigPipe support will benefit websites small and large.
Building Drupal 8 with all of you has been a wild ride. I thought it would be fun to take a little end-of-week look back at some of our community's biggest milestones through Twitter. If you can think of others important Tweets, please share them in the comments, and I'll update the post.
Feeling nostalgic? See every single version of Drupal running!
— Cheppers (@cheppers) November 19, 2015
Here is how we opened the development branch for Drupal 8: live at Drupalcon!
The secretsauce of #drupal isn't code or features or market share, important thought they are. The secret sauce is community.
— Sean Yo (@seanyo) March 10, 2011
— Jeff Geerling (@geerlingguy) March 10, 2011
Drupal 8's first beta showed the power of community
Drupal 8.0.0 beta 1 released! https://t.co/FwdmRYaZUx Ahh the power of COMMUNITY driven software! :-)
— Doug Vann (@dougvann) October 1, 2014
— Gábor Hojtsy (@gaborhojtsy) October 1, 2014
We had issues ... but the queue steadily declined
— xjm (@xjmdrupal) September 19, 2014
Drupal 8.0.x-rc1 release window is today. Good sign of real stability is major issue count going down for 6+ weeks. pic.twitter.com/5VnHGmL9zb
— catch (@catch56) October 7, 2015
We held sprints around the world: here are just a few
— xjm (@xjmdrupal) July 5, 2015
Working on D8 Criticals at the Ghent DA critical sprint, this is how the "My issues" page looks for me right now! pic.twitter.com/y5SnavVtND
— Sascha Grossenbacher (@berdir) December 13, 2014
— Cameron Eagans (@cweagans) March 23, 2012
And we created many game-changing features
— Wim Leers (@wimleers) April 8, 2015
And.... there we go! http://t.co/ed6XtMIs MOTHER BLEEPING VIEWS IN MOTHER BLEEPING CORE!
— webchick (@webchick) October 22, 2012
— Alex Pott (@alexpott) February 15, 2014
With Content + Config Translation in core D8 core is more translatable than D7 with all of contrib. #drupal
— Tobias Stöckler (@tstoeckler) November 18, 2013
Amazing to see Drupal 8's multilingual capabilities explained on the multilingual release page (for example Farsi): pic.twitter.com/9owVE3xABo
— Gábor Hojtsy (@gaborhojtsy) November 19, 2015
The founder of PHP said: Drupal 8 + PHP7 = a lot of happy people
— Rasmus Lerdorf (@rasmus) April 21, 2015
We reached the first release candidate and celebrated ... a little
— Whitney Hess (@whitneyhess) October 7, 2015
— Manuel Garcia (@drupalero) October 7, 2015
Kudos to the 3000+ contributors and to the entire Drupal community that helped make this happen. https://t.co/FtATRtSmCU
— Leslie Glynn (@leslieglynn) October 7, 2015
And, just yesterday, we painted the world blue and celebrated Drupal 8 ... a lot!
— Drupal (@drupal) November 10, 2015
— Drupal (@drupal) November 19, 2015
— Taco Potze˙ (@tacopotze) November 19, 2015
— Duo (@DuoConsulting) November 19, 2015
— Shakeel Tariq (@shakeeltariq) November 19, 2015
— Agustin Rojas Silva (@Aguztinrs) November 19, 2015
— HornCologne (@HornCologne) November 19, 2015
— webchick (@webchick) November 19, 2015
— Paul Johnson (@pdjohnson) November 19, 2015
— Dries Buytaert (@Dries) November 18, 2015
— Peter Decuyper (@sgrame) November 23, 2015