In March, I did a presentation at SxSW that asked the audience a question I've been thinking about a lot lately: "Can we save the open web?".
The web is centralizing around a handful of large companies that control what we see, limit creative freedom, and capture a lot of information about us. I worry that we risk losing the serendipity, creativity and decentralization that made the open web great.
While there are no easy answers to this question, the presentation started a good discussion about the future of the open web, the role of algorithms in society, and how we might be able to take back control of our personal information.
I'm going to use my blog to continue the conversation about the open web, since it impacts the future of Drupal. I'm including the video and slides (PDF, 76 MB) of my SxSW presentation below, as well as an overview of what I discussed.
Here are the key ideas I discussed in my presentation, along with a few questions to discuss in the comments.
Idea 1: An FDA-like organization to provide oversight for algorithms. While an "FDA" in and of itself may not be the most ideal solution, algorithms are nearly everywhere in society and are beginning to impact life-or-death decisions. I gave the example of an algorithm for a self-driving car having to decide whether to save the driver or hit a pedestrian crossing the street. There are many other life-or-death examples of how unregulated technology could impact people in the future, and I believe this is an issue we need to begin thinking about now. What do you suggest we do to make the use of algorithms fair and trustworthy?
Idea 2: Open standards that will allow for information-sharing across sites and applications. Closed platforms like Facebook and Google are winning because they're able to deliver a superior user experience driven by massive amounts of data and compute power. For the vast majority of people, ease-of-use will trump most concerns around privacy and control. I believe we need to create a set of open standards that enable drastically better information-sharing and integration between websites and applications so independent websites can offer user experiences that meet or exceeds that of the large platforms. How can the Drupal community help solve this problem?
Idea 3: A personal information broker that allows people more control over their data. In the past, I've written about the idea for a personal information broker that will give people control over how, where and for how long their data is used, across every single interaction on the web. This is no small feat. An audience member asked an interesting question about who will build this personal information broker -- whether it will be a private company, a government, an NGO, or a non-profit organization? I'm not really sure I have the answer, but I am optimistic that we can figure that out. I wish I had the resources to build this myself as I believe this will be a critical building block for the web. What do you think is the best way forward?
Ultimately, we should be building the web that we want to use, and that we want our children to be using for decades to come. It's time to start to rethink the foundations, before it's too late. If we can move any of these ideas forward in a meaningful way, they will impact billions of people, and billions more in the future.
Today is another big day for Drupal as we just released Drupal 8.1.0. Drupal 8.1.0 is an important milestone as it is a departure from the Drupal 7 release schedule where we couldn't add significant new features until Drupal 8. Drupal 8.1.0 balances maintenance with innovation.
On my blog and in presentations, I often talk about the future of Drupal and where we need to innovate. I highlight important developments in the Drupal community, and push my own ideas to disrupt the status quo. People, myself included, like to talk about the shiny innovations, but it is crucial to understand that innovation is only a piece of how we grow Drupal's success. What can't be forgotten is the maintenance, the bug fixing, the work on Drupal.org and our test infrastructure, the documentation writing, the ongoing coordination and the processes that allow us to crank out stable releases.
We often recognize those who help Drupal innovate or introduce novel things, but today, I'd like us to praise those who maintain and improve what already exists and that was innovated years ago. So much of what makes Drupal successful is the "daily upkeep". The seemingly mundane and unglamorous effort that goes into maintaining Drupal has a tremendous impact on the daily life of hundreds of thousands of Drupal developers, millions of Drupal content managers, and billions of people that visit Drupal sites. Without that maintenance, there would be no stability, and without stability, no room for innovation.
Earlier this month, the international media group Hubert Burda Media (about 2.5 billion annual revenue, more than 10,000 employees, and more than 300 titles) released its Drupal 8 distribution, Thunder. Thunder includes custom modules specifically tailored to the needs of professional publishers.
This is great news for three reasons: (1) I've long been a believer in Drupal distributions, (2) I believe that publishers shouldn't compete through CMS technology, but through strong content and brands, and (3) Thunder is based on Drupal 8.
Distributions enable Drupal to compete against a wide range of turnkey solutions, as well as break into new markets. The number of vertical distributions that can be created is nearly limitless, and the possibilities are endless. Thunder is a great example of that.
Professional publishing is one of the industries that has faced the most extensive business model disruption because of the web. Many companies are feeling pressure on their revenue streams, yet you'll find that some companies still focus their efforts on building proprietary, custom CMS platforms as a way to differentiate. This doesn't have to be the case – I've long believed that Drupal (and open source, more generally) can give publishers endless ways to differentiate themselves at much lower costs.
The following video gives an overview of the Thunder approach:
Custom features for publishers
Thunder adds a range of publisher-centric Drupal modules to Drupal 8 core. Specifically, Burda added integrations with audience "circulation" counting tools and ad servers, as well as single sign-on (SSO) support across multiple sites. They've also developed a theme which implements infinite scrolling.
Thunder users also benefit from a range of channel- and feature-specific enhancements through collaboration with industry partners. The following extensions are already available or in the final stages of development:
- Riddle.com provides an easy-to-use editor for interactive content. The data from the resulting polls and quizzes is available to the publisher.
- nexx.tv offers a video CMS and their video player. And Microsoft will support the video solution with 100,000 free video streamings per month through their Azure cloud.
- Facebook will provide a module for integrated publishing to their Instant Articles, exclusively for Thunder users.
I admire the approach Burda is taking to bring publishers, partners and developers together from throughout the industry to develop the best open-source CMS platform for publishers.
At the core is a team of publishing experts and developers led by Ingo Rübe, CTO for Burda's German publishing operations, and initiator of Thunder. This team will also be responsible for coordinating the continuous development and enhancement of Thunder.
Under Thunder's policy, all features provided by industry partners must be offered for free or with a freemium model; in other words, a significant part of the functionality has to be provided at no cost at all. Smaller publishers will likely benefit from this approach, as they will be able to use a full-fledged publishing solution that is continuously enhanced and maintained by larger partners.
Big brands are already using Thunder
Although Thunder is still in public beta, Burda has migrated three brands to Thunder. The German edition of Playboy (about 2M monthly visits) was the first to move at the end of 2015. The fashion brand InStyle (about 1.8M monthly visits) and gardening website "Mein schöner Garten" (about 1.5M monthly visits) are also running on Thunder. Most of the other German Burda brands are planning to adopt Thunder in the next 12 months. This includes at least 20 brands such as Elle.de and Bunte.de, which have more than 20 million monthly visits each.
You can download Thunder from https://www.drupal.org/project/thunder.
I've been writing a lot about what I believe is important for the future of Drupal, but now it is your turn. After every major release of Drupal I do a "product management survey" to get your ideas and data on what to focus on for future releases of Drupal (8.x/9).
The last time we had such a survey was after the release of Drupal 7, six months into the development of Drupal 8. I presented the results at DrupalCon London in 2011. The results informed the Drupal community at large, but were also the basis for defining initiatives for Drupal 8. This time around, I'm hoping for similar impact, but also some higher-level strategic thinking about how Drupal should respond to various market trends.
It shouldn't take more than 10-15 minutes to fill out the survey. We'd like to hear from everyone who cares about Drupal: content managers, site owners, site builders, module developers, front-end developers, people selling and marketing Drupal, etc. Whether you are a Drupal expert or just getting started with Drupal, every voice counts! Best of all, with Drupal 8's new 6-month release cycle, we can act on the results of this survey much sooner than in the past.
I will be presenting the results during my DrupalCon New Orleans keynote (the video recording of the keynote, the presentation slides and the survey results will be downloadable on my blog after). Please tell us what you think about Drupal; your feedback will shape future versions of Drupal.
At DrupalCon Mumbai I sat down for several hours with the Drupal team at Pfizer to understand the work they have been doing on improving Drupal content management features. They built a set of foundational modules that help advance Drupal's content workflow capabilities; from content staging, to multi-site content staging, to better auditability, offline support, and several key user experience improvements like full-site preview, and more. In this post, I want to point a spotlight on some of Pfizer's modules, and kick-off an initial discussion around the usefulness of including some of these features in core.
Before jumping to the technical details, let's talk a bit more about the problems these modules are solving.
- Cross-site content staging — In this case you want to synchronize content from one site to another. The first site may be a staging site where content editors make changes. The second site may be the live production site. Changes are previewed on the stage site and then pushed to the production site. More complex workflows could involve multiple staging environments like multiple sites publishing into a single master site.
- Content branching — For a new product launch you might want to prepare a version of your site with a new section on the site featuring the new product. The new section would introduce several new pages, updates to existing pages, and new menu items. You want to be able to build out the updated version in a self-contained 'branch' and merge all the changes as a whole when the product is ready to launch. In an election case scenario, you might want to prepare multiple sections; one for each candidate that could win.
- Preview your site — When you're building out a new section on your site for launch, you want to preview your entire site, as it will look on the day it goes live. This is effectively content staging on a single site.
- Offline browse and publish — Here is a use-case that Pfizer is trying to solve. A sales rep goes to a hospital and needs access to information when there is no wi-fi or a slow connection. The site should be fully functional in offline mode and any changes or forms submitted, should automatically sync and resolve conflicts when the connection is restored.
- Content recovery — Even with confirmation dialogs, people delete things they didn’t want to delete. This case is about giving users the ability to “undelete” or recover content that has been deleted from their database.
- Audit logs — For compliance reasons, some organizations need all content revisions to be logged, with the ability to review content that has been deleted and connect each action to a specific user so that employees are accountable for their actions in the CMS.
All these use cases share a few key traits:
- Content needs to be synchronized from one place to another, e.g. from workspace to workspace, from site to site or from frontend to backend
- Full revision history needs to be kept
- Content revision conflicts needs to be tracked
Much of this started as a single module: Deploy. The Deploy module was first created by Greg Dunlap for Drupal 6 in 2008 for a customer of Palantir. In 2012, Greg handed over maintainership to Dick Olsson. Dick continued to improve Deploy module for Al Jazeera while working at Node One. Later, Dave Hall created a second Drupal 7 version which more significant improvements based on feedback from different users. Today, both Dick and Dave work for Pfizer and have continued to include lessons learned in the Drupal 8 version of the module. After years of experience working on Deploy module and various redesigns, the team has extracted the functionality in a set of modules:
This module does three things: (1) it adds revision support for all content entities in Drupal, not just nodes and block content as provided by core, and (2) it introduces the concept of parent revisions so you can create different branches of your content or site, and (3) it keeps track of conflicts in the revision tree (e.g. when two revisions share the same parent). Many of these features complement the ongoing improvements to Drupal's Entity API.
Built on top of Multiversion module, this lightweight module reads revision information stored by Multiversion, and uses that to determine what revisions are missing from a given location and lets you replicate content between a source and a target location. The next two modules, Workspace and RELAXed Web Services depend on replication module.
This module enables single site content staging and full-site previews. The UI lets you create workspaces and switch between them. With Replication module different workspaces on the same site can behave like different editorial environments.
RELAXed Web Services
This module facilitates cross-site content staging. It provides a more extensive REST API for Drupal with support for UUIDs, translations, file attachments and parent revisions — all important to solve unique challenges with content staging (e.g. UUID and revision information is needed to resolve merge conflicts). The RELAXed Web Services module extends the Replication module and makes it possible to replicate content from local workspaces to workspaces on remote sites using this API.
In short, Multiversion provides the "storage plumbing", whereas Replication, Workspace, and RELAXed Web Services, provide the "transport plumbing".
Historically Deploy module has taken care of everything from bottom to top related to content staging. But for Drupal 8 Deploy has been rewritten to just provide a UI on-top of the Workspace and Replication modules. This UI lets you manage content deployments between workspaces on a single site, or between workspaces across sites (if used together with RELAXed Web Services module). The maintainers of the Deploy module have put together a marketing site with more details on what it does: http://www.drupaldeploy.org.
To handle use case #5 (content recovery) the Trash module was implemented to restore entities marked as deleted. Much like a desktop trash or recycle bin, the module displays all entities from all supported content types where the default revision is flagged as deleted. Restoring creates a newer revision, which is not flagged as deleted.
Synchronizing sites with a battle-tested API
Drupal 8.0 core packed many great improvements, but we didn't focus much on advancing Drupal's content workflow capabilities. As we think about Drupal 8.x and beyond, it might be good to move some of our focus to features like content staging, better audit-ability, off-line support, full-site preview, and more. If you are a content manager, I'd love to hear what you think about better supporting some or all of these use cases. And if you are a developer, I encourage you to take a look at these modules, try them out and let us know what you think.