Thunder, a Drupal distribution for publishers

Earlier this month, the international media group Hubert Burda Media (about 2.5 billion annual revenue, more than 10,000 employees, and more than 300 titles) released its Drupal 8 distribution, Thunder. Thunder includes custom modules specifically tailored to the needs of professional publishers.

This is great news for three reasons: (1) I've long been a believer in Drupal distributions, (2) I believe that publishers shouldn't compete through CMS technology, but through strong content and brands, and (3) Thunder is based on Drupal 8.

Distributions enable Drupal to compete against a wide range of turnkey solutions, as well as break into new markets. The number of vertical distributions that can be created is nearly limitless, and the possibilities are endless. Thunder is a great example of that.

Professional publishing is one of the industries that has faced the most extensive business model disruption because of the web. Many companies are feeling pressure on their revenue streams, yet you'll find that some companies still focus their efforts on building proprietary, custom CMS platforms as a way to differentiate. This doesn't have to be the case – I've long believed that Drupal (and open source, more generally) can give publishers endless ways to differentiate themselves at much lower costs.

The following video gives an overview of the Thunder approach:

Custom features for publishers

Thunder adds a range of publisher-centric Drupal modules to Drupal 8 core. Specifically, Burda added integrations with audience "circulation" counting tools and ad servers, as well as single sign-on (SSO) support across multiple sites. They've also developed a theme which implements infinite scrolling.

Thunder users also benefit from a range of channel- and feature-specific enhancements through collaboration with industry partners. The following extensions are already available or in the final stages of development:

  • Riddle.com provides an easy-to-use editor for interactive content. The data from the resulting polls and quizzes is available to the publisher.
  • nexx.tv offers a video CMS and their video player. And Microsoft will support the video solution with 100,000 free video streamings per month through their Azure cloud.
  • Facebook will provide a module for integrated publishing to their Instant Articles, exclusively for Thunder users.

Smart collaboration

I admire the approach Burda is taking to bring publishers, partners and developers together from throughout the industry to develop the best open-source CMS platform for publishers.

At the core is a team of publishing experts and developers led by Ingo Rübe, CTO for Burda's German publishing operations, and initiator of Thunder. This team will also be responsible for coordinating the continuous development and enhancement of Thunder.

Under Thunder's policy, all features provided by industry partners must be offered for free or with a freemium model; in other words, a significant part of the functionality has to be provided at no cost at all. Smaller publishers will likely benefit from this approach, as they will be able to use a full-fledged publishing solution that is continuously enhanced and maintained by larger partners.

Big brands are already using Thunder

Although Thunder is still in public beta, Burda has migrated three brands to Thunder. The German edition of Playboy (about 2M monthly visits) was the first to move at the end of 2015. The fashion brand InStyle (about 1.8M monthly visits) and gardening website "Mein schöner Garten" (about 1.5M monthly visits) are also running on Thunder. Most of the other German Burda brands are planning to adopt Thunder in the next 12 months. This includes at least 20 brands such as Elle.de and Bunte.de, which have more than 20 million monthly visits each.

You can download Thunder from https://www.drupal.org/project/thunder.

State of Drupal 2016 survey

I've been writing a lot about what I believe is important for the future of Drupal, but now it is your turn. After every major release of Drupal I do a "product management survey" to get your ideas and data on what to focus on for future releases of Drupal (8.x/9).

The last time we had such a survey was after the release of Drupal 7, six months into the development of Drupal 8. I presented the results at DrupalCon London in 2011. The results informed the Drupal community at large, but were also the basis for defining initiatives for Drupal 8. This time around, I'm hoping for similar impact, but also some higher-level strategic thinking about how Drupal should respond to various market trends.

Take the State of Drupal 2016 survey now

It shouldn't take more than 10-15 minutes to fill out the survey. We'd like to hear from everyone who cares about Drupal: content managers, site owners, site builders, module developers, front-end developers, people selling and marketing Drupal, etc. Whether you are a Drupal expert or just getting started with Drupal, every voice counts! Best of all, with Drupal 8's new 6-month release cycle, we can act on the results of this survey much sooner than in the past.

I will be presenting the results during my DrupalCon New Orleans keynote (the video recording of the keynote, the presentation slides and the survey results will be downloadable on my blog after). Please tell us what you think about Drupal; your feedback will shape future versions of Drupal.

Improving Drupal's content workflow

At DrupalCon Mumbai I sat down for several hours with the Drupal team at Pfizer to understand the work they have been doing on improving Drupal content management features. They built a set of foundational modules that help advance Drupal's content workflow capabilities; from content staging, to multi-site content staging, to better auditability, offline support, and several key user experience improvements like full-site preview, and more. In this post, I want to point a spotlight on some of Pfizer's modules, and kick-off an initial discussion around the usefulness of including some of these features in core.

Use cases

Before jumping to the technical details, let's talk a bit more about the problems these modules are solving.

  1. Cross-site content staging — In this case you want to synchronize content from one site to another. The first site may be a staging site where content editors make changes. The second site may be the live production site. Changes are previewed on the stage site and then pushed to the production site. More complex workflows could involve multiple staging environments like multiple sites publishing into a single master site.
  2. Content branching — For a new product launch you might want to prepare a version of your site with a new section on the site featuring the new product. The new section would introduce several new pages, updates to existing pages, and new menu items. You want to be able to build out the updated version in a self-contained 'branch' and merge all the changes as a whole when the product is ready to launch. In an election case scenario, you might want to prepare multiple sections; one for each candidate that could win.
  3. Preview your site — When you're building out a new section on your site for launch, you want to preview your entire site, as it will look on the day it goes live. This is effectively content staging on a single site.
  4. Offline browse and publish — Here is a use-case that Pfizer is trying to solve. A sales rep goes to a hospital and needs access to information when there is no wi-fi or a slow connection. The site should be fully functional in offline mode and any changes or forms submitted, should automatically sync and resolve conflicts when the connection is restored.
  5. Content recovery — Even with confirmation dialogs, people delete things they didn’t want to delete. This case is about giving users the ability to “undelete” or recover content that has been deleted from their database.
  6. Audit logs — For compliance reasons, some organizations need all content revisions to be logged, with the ability to review content that has been deleted and connect each action to a specific user so that employees are accountable for their actions in the CMS.

Technical details

All these use cases share a few key traits:

  1. Content needs to be synchronized from one place to another, e.g. from workspace to workspace, from site to site or from frontend to backend
  2. Full revision history needs to be kept
  3. Content revision conflicts needs to be tracked

Much of this started as a single module: Deploy. The Deploy module was first created by Greg Dunlap for Drupal 6 in 2008 for a customer of Palantir. In 2012, Greg handed over maintainership to Dick Olsson. Dick continued to improve Deploy module for Al Jazeera while working at Node One. Later, Dave Hall created a second Drupal 7 version which more significant improvements based on feedback from different users. Today, both Dick and Dave work for Pfizer and have continued to include lessons learned in the Drupal 8 version of the module. After years of experience working on Deploy module and various redesigns, the team has extracted the functionality in a set of modules:

Pfizer content workflow improvements modules

Multiversion

This module does three things: (1) it adds revision support for all content entities in Drupal, not just nodes and block content as provided by core, and (2) it introduces the concept of parent revisions so you can create different branches of your content or site, and (3) it keeps track of conflicts in the revision tree (e.g. when two revisions share the same parent). Many of these features complement the ongoing improvements to Drupal's Entity API.

Replication

Built on top of Multiversion module, this lightweight module reads revision information stored by Multiversion, and uses that to determine what revisions are missing from a given location and lets you replicate content between a source and a target location. The next two modules, Workspace and RELAXed Web Services depend on replication module.

Workspace

This module enables single site content staging and full-site previews. The UI lets you create workspaces and switch between them. With Replication module different workspaces on the same site can behave like different editorial environments.

RELAXed Web Services

This module facilitates cross-site content staging. It provides a more extensive REST API for Drupal with support for UUIDs, translations, file attachments and parent revisions — all important to solve unique challenges with content staging (e.g. UUID and revision information is needed to resolve merge conflicts). The RELAXed Web Services module extends the Replication module and makes it possible to replicate content from local workspaces to workspaces on remote sites using this API.

In short, Multiversion provides the "storage plumbing", whereas Replication, Workspace, and RELAXed Web Services, provide the "transport plumbing".

Deploy

Historically Deploy module has taken care of everything from bottom to top related to content staging. But for Drupal 8 Deploy has been rewritten to just provide a UI on-top of the Workspace and Replication modules. This UI lets you manage content deployments between workspaces on a single site, or between workspaces across sites (if used together with RELAXed Web Services module). The maintainers of the Deploy module have put together a marketing site with more details on what it does: http://www.drupaldeploy.org.

Trash

To handle use case #5 (content recovery) the Trash module was implemented to restore entities marked as deleted. Much like a desktop trash or recycle bin, the module displays all entities from all supported content types where the default revision is flagged as deleted. Restoring creates a newer revision, which is not flagged as deleted.

Synchronizing sites with a battle-tested API

When a Drupal site installs and enables RELAXed Web Services it will look and behave like the REST API from CouchDB. This is a pretty clever trick because it enables us to leverage the CouchDB ecosystem of tools. For example, you can use PouchDB, a JavaScript implementation of CouchDB, to provide a fully-decoupled offline database in the web browser or on a mobile device. Using the same API design as CouchDB also gives you "streaming replication" with instant updates between the backend and frontend. This is how Pfizer implements off-line browse and publish.

This animated gif shows how a content creator can switch between multiple workspaces and get a full-site preview on a single site. In this example the Live workspace is empty, while there has been a lot of content addition on the Stage workspace in.
This animated gif shows how a workspace is deployed from 'Stage' to 'Live' on a single site. In this example the Live workspace is initially empty.

Conclusion

Drupal 8.0 core packed many great improvements, but we didn't focus much on advancing Drupal's content workflow capabilities. As we think about Drupal 8.x and beyond, it might be good to move some of our focus to features like content staging, better audit-ability, off-line support, full-site preview, and more. If you are a content manager, I'd love to hear what you think about better supporting some or all of these use cases. And if you are a developer, I encourage you to take a look at these modules, try them out and let us know what you think.

Thanks to Tim Millwood, Dick Olsson and Nathaniel Catchpole for their feedback on the blog post. Special thanks to Pfizer for contributing to Drupal.

Examples of how to make Drupal "outside-in"

The authoring experience improvements we made in Drupal 8 appear to be well-received, but that doesn't mean we are done. With semantic versioning in Drupal 8, we can now make more UX changes in follow-up releases of Drupal 8. So now is a good time to start moving Drupal's user experience forward.

The goal of this post is to advance the conversation we started over a month ago in a blog post talking about the concept of turning Drupal outside-in. In today's blog post, we'll show concrete examples of how we could make site building in Drupal easier by embracing the concept of outside-in. We hope to provide inspiration to designers, core contributors and module maintainers for how we can improve the user experience of Drupal 8.2 and beyond.

What is outside-in?

In Drupal you often have to build things from the ground up. If you want to make a list of events you need to wade through 20 different menu options to a user interface with configuration options like "boolean" and "float", build a content type, content, and then a view. Essentially you need to understand everything before you can build anything.

In an "outside-in" experience – or what Kevin OLeary (Director of Design on my team at Acquia) calls Literal UI – you can click anything on the page, edit its configuration in-place, and watch it take effect immediately.

Over the past few years Drupal has adopted a more outside-in approach, the most obvious example being Drupal 8's in-place editing. It enables you to edit what you see with an uninterrupted workflow, faster preview, and removes the need to visit the administration backend before you can start editing.

To evaluate how outside-in can be applied more broadly in order to make Drupal easier to use, Kevin created a few animated gifs.

Example #1: editing menu items

The current inside-out experience

Editing menu options in Drupal has already become more outside-in with contextual links. The contextual links take away some of the guesswork of finding the proper administration UI, but once you arrive at the configuration page there is still a lot of stuff to take in, only some of which is relevant to this task.

The current inside-out experience for editing a menu item: adding a menu link and changing its position involves 2 separate administration pages, 4 page refreshes and more than 400 words of text on the UI.

Anyone familiar with Drupal will recognize the pattern above; you go to an administration UI, make some changes, than you go back to the page to see if it worked. This context switching creates what UX people call "cognitive load". On an administration page you need to remember what was on the site page and vice-versa.

To complete this task you need to:

  1. Find the contextual link for the menu (not simple since it's on the far side of the page)
  2. Choose the correct option "edit menu"
  3. Find and click the "add link" button
  4. Type the menu link name
  5. Type the menu link path beginning with a forward slash
  6. Understand that this change needs to be saved
  7. Scroll to the bottom of the page
  8. Save the link
  9. Find the link in the list of links
  10. Understand that dragging up/down in this abstraction is equivalent to moving right/left in that actual page
  11. Scroll to the bottom of the page
  12. Save the menu

The problem is not just that there are too many pages, clicks, or words, it's that each step away from the actual page introduces new opportunities for confusion, error and repetition. In user testing, we have seen users who were unable to find the contextual link, or to understand which option to choose, or to find the "add link" button, or to add a path, or drag-drop links, or to save before leaving the UI. When these things happen in context, feedback about whether you are "doing it right" is immediate.

The proposed outside-in experience

The proposed outside-in experience for editing a menu item. Rather than moving out-of-context to an administration page to get the job done, configuration happens right on the page where you see the effect of each action. You start adding a menu item simply by selecting the menu on the page. Both the menu item and the item's path are in focus to reinforce the connection. As you type a path is proposed from the link text.

Now all you need to do is:

  1. Click the menu on the page
  2. Find and click the "add link" button
  3. Type the name of the menu item
  4. Revise the menu item's path if needed
  5. Drag the link to its new location
  6. Close the drawer

One important aspect of this approach is that all actions that produce a visible change have bi-directional control and bi-directional feedback. In other words, if you can drag something in the configuration drawer you should also be able to drag it on the page, and the changes should happen simultaneously.

Example #2: adding a block to a page

The current inside-out experience

The process of placing a block on a page can be difficult. Once you discover where to go to add a block, which is in itself a challenge, the experience requires a lot of reading and learning, as well as trial and error.

The current inside-out experience for adding a block. Not all steps are shown.

To complete this task you need to:

  1. Figure out where to go to place a block
  2. Go to /block-layout
  3. Go to /display-regions to find out where the region is on the page
  4. Go back to /block-layout
  5. Find the region and click "add block"
  6. Find the block you want to place and click "place block"
  7. Configure the block
  8. Read about how blocks can appear on multiple pages and for different content types and roles
  9. Read what content types are
  10. Read what roles are
  11. Read what a "path" is
  12. Read how to find the path for a page
  13. Go back to the page and get its path
  14. Go back to /block-layout and add the path to the visibility settings
  15. Drag your block to the position where you want it
  16. If your blocks are arranged horizontally, learn that "up and down" in the block layout UI will mean "left and right" on the actual page
  17. Find the"back to site" link
  18. Go back to the page to see if you did it right

Eventually you'll use what you just learned, but Drupal makes you learn it first instead of just showing what is immediately necessary. Both the task and the learning can be simplified by bringing the configuration closer to the object you are configuring.

The proposed outside-in experience

The proposed outside-in experience for adding a block. Everything happens on the actual page rather than on an administration page. Places where things can be added appear on hover. On click they show thumbnails that you can filter with autocomplete.

Now all you need to do is:

  1. Choose where to place the block
  2. Find the block you want to place
  3. Place the block
  4. Close the drawer

The "plus" button, the drop target (blue dotted rectangle) and the autocomplete are all commonly understood patterns used by other software. The task requires no or little explanation as the interactions reveal the process. By starting with selecting the location of where to place the block, we avoid the need for drag-and-drop and the complexity of dragging a block on a page that requires scrolling.

Principles and patterns

These examples show the principle that rather than taking the site builder to a separate administration backend, the experience should begin with the actual page and show how the changes will be seen by the end-user. The patterns shown here are less important. For example, the animated gifs above show two different approaches to the options panel and there are certainly others. The important thing is not yet where the panel comes from or how it looks, but that the following criteria are met:

  • An option panel is triggered by direct interaction.
  • When triggered it only shows options for what you selected.
  • It primarily shows configuration options that produce a visible change to the page. More complex options could be accessed through an 'Advanced options' link.

The ideas in this post are meant to provide some food for thought and become the seed of some patches for Drupal 8.2. Rather than dive right into development, we're looking to the Drupal community to take these designs as a starting point to prototype and user-test more outside-in experiences in Drupal.

The next post in this series will cover outside-in experiences with more complex contexts and the problem of "leaky abstractions". If you don't want to miss the next blog post, make sure to subscribe. In the mean time, I'd love to hear what you think!

Special thanks to Kevin OLeary for advocating outside-in thinking. Thanks to Preston So, Gábor Hojtsy, Angie Byron, Bojhan Somers and Roy Scholten for their feedback.

State of Drupal presentation (February 2016)

I was excited to travel to India a few months ago for DrupalCon, an area where we have a really big opportunity for the growth of Drupal. In keeping with tradition, here are the slides and video from my keynote presentation. You can watch the recording of my keynote (starting at 20:15) or download a copy of my slides (PDF, 158 MB).

The main areas of focus for the talk included Drupal's rapid growth and progress in India, key technology trends driving the future of the web, and how Drupal is responding to these trends. As a call-to-action, I encouraged Drupalists in India to form grassroots communities locally, to become a part of the larger Drupal community conversation, and to port modules from Drupal 7 to Drupal 8 to accelerate its adoption.

Have a look and as always, feel free to leave your thoughts in the comments!

How should you decouple Drupal?

With RESTful web services in Drupal 8 core, Drupal can function as an API-first back end serving browser applications, native applications on mobile devices, in-store displays, even in-flight entertainment systems (Lufthansa is doing so in Drupal 8!), and much more. When building a new website or web application in 2016, you may ask yourself: how should I decouple Drupal? Do I build my website with Drupal's built-in templating layer or do I use a JavaScript framework? Do I need Node.js?

There is a lot of hype around decoupled architectures, so before embarking on a project, it is important to make a balanced analysis. Your choice of architecture has implications on your budget, your team, time to launch, the flexibility for content creators, the ongoing maintenance of your website, and more. In this blog post, I'd like to share a flowchart that can help you decide when to use what technology.

Decoupled decision flowchart

This flowchart shows three things:

First, using coupled Drupal is a perfectly valid option for those who don't need extensive client-side rendering and state management. In this case, you would use Drupal's built-in Twig templating system rather than heavily relying on a JavaScript framework. You would use jQuery to take advantage of limited JavaScript where necessary. Also, with BigPipe in Drupal 8.1, certain use cases that typically needed asynchronous JavaScript can now be done in PHP without slowing down the page (i.e. communication with an external web service delaying the display of user-specific real-time data). The advantage of this approach is that content marketers are not blocked by front-end developers as they assemble their user experiences, thus shortening time to market and reducing investment in ongoing developer support.

Second, if you want all of the benefits of a JavaScript framework without completely bypassing Drupal's HTML generation and all that you get with it, I recommend using progressively decoupled Drupal. With progressive decoupling, you start with Drupal's HTML output, and then use a JavaScript framework to add client-side interactivity on the client side. One of the most visited sites in the world, The Weather Channel (100 million unique visitors per month), does precisely this with Angular 1 layered on top of Drupal 7. In this case, you can enjoy the benefits of having a “decoupled" team made up of both Drupal and JavaScript developers progressing at their own velocities. JavaScript developers can build richly interactive experiences while leaving content marketers free to assemble those experiences without needing a developer's involvement.

Third, whereas fully decoupled Drupal makes a lot of sense when building native applications, for most websites, the leap to fully decoupling is not strictly necessary, though a growing number of people prefer using JavaScript these days. Advantages include some level of independence on the underlying CMS, the ability to tap into a rich toolset around JavaScript (e.g. Babel, Webpack, etc.) and a community of JavaScript front-end professionals. But if you are using a universal JavaScript approach with Drupal, it's also important to consider the drawbacks: you need to ask yourself if you're ready to add more complexity to your technology stack and possibly forgo functionality provided by a more integrated content delivery system, such as layout and display management, user interface localization, and more. Losing that functionality can be costly, increase your dependence on a developer team, and hinder the end-to-end content assembly experience your marketing team expects, among other things.

It's worth noting that over time we are likely to see better integrations between Drupal and the different JavaScript frameworks (e.g. Drupal modules that export their configuration, and SDKs for different JavaScript frameworks that use that configuration on the client-side). When those integrations mature, I expect more people will move towards fully decoupled Drupal.

To be performant, fully decoupled websites using JavaScript employ Node.js on the server to improve initial performance, but in the case of Drupal this is not necessary, as Drupal can do the server-side pre-rendering for you. Many JavaScript developers opt to use Node.js for the convenience of shared rendering across server and client rather than for the specific things that Node.js excels in, like real-time push, concurrent connections, and bidirectional client-server communication. In other words, most Drupal websites don't need Node.js.

Decoupled delivery architectures

In practice, I believe many organizations want to use all of these content delivery options. In certain cases, you want to let your content management system render the experience so you can take full advantage of its features with minimal or no development effort (coupled architecture). But when you need to build a website that needs a much more interactive experience or that integrates with unique devices (i.e. on in-store touch screens), you should be able to use that same content management system's content API (decoupled architecture). Fortunately, Drupal allows you to use either. The beauty of choosing from the spectrum of fully decoupled Drupal, progressively decoupled Drupal, and coupled Drupal is that you can do what makes the most sense in each situation.

Special thanks to Preston So, Alex Bronstein and Wim Leers for their contributions to this blog post. We created at least 10 versions of this flowchart before settling on this one.

A "MAP" for accelerating Drupal 8 adoption

Contributed modules in Drupal deliver the functionality and innovation proprietary content management solutions simply can't match. With every new version of Drupal comes the need to quickly move modules forward from the previous version. For users of Drupal, it's crucial to know they can depend on the availability of modules when considering a new Drupal 8 project or migrating from a previous version.

I'm pleased that many agencies and customers who use Drupal are donating time and attention to maintaining Drupal's module repository and ensuring their contributed modules are upgraded. I believe it's the responsibility of Drupal companies to give back to the community.

I'm proud that Acquia leads by example. It was with great pride that Acquia created a Drupal 8 Module Acceleration Program, or MAP. Led by Acquia's John Kennedy, MAP brings financial, technical and project management assistance to Drupal module maintainers. Acquia kicked off MAP in mid-October and to date we have helped complete production-ready versions of 34 modules. And it is not just any modules; we've been focused on those modules that provide critical pieces of functionality used by most Drupal sites.

When MAP was formed Acquia allocated $500,000 to fund non-Acquia maintainers in the community. In addition, we have so far invested more than 2,500 hours of our own developers' time to support the effort (the equivalent of three full-time developers).

What is impressive to me about MAP is both the focus on mission-critical modules that benefit a huge number of users, as well as the number of community members and agencies involved. John's team is leading a coalition of the best and brightest minds in the Drupal community to address the single biggest obstacle holding Drupal 8 adoption back.

Drupal 8 has already made a significant impact; in the 90 days following the release of Drupal 8.0.0, adoption has outpaced Drupal 7 by more than 200 percent. And as more modules get ported, I expect Drupal 8 adoption to accelerate even more.

White House deepens its commitment to Open Source

Yesterday, the White House announced a plan to deepen its commitment to open source. Under this plan, new, custom-developed government software must be made available for use across other federal agencies, and a portion of all projects must be made open source and shared with the public. This plan will make it much easier to share best practices, collaborate, and save money across different government departments.

However, there are still some questions to address. In good open source style, the White House is inviting developers to comment on this policy. As the Drupal community we should take advantage and comment on GitHub within the 30-day feedback window.

The White House has a long open source history with Drupal. In October 2009, WhiteHouse.gov relaunched on Drupal and shortly thereafter started to actively contribute back to Drupal -- both were a first in the history of the White House. White House's contributions to Drupal include the "We the People" petitions platform, which was adopted by other governments and organizations around the world.

This week's policy is big news because it will push open source deeper into the roots of the U.S. government, requiring more government agencies to become active open source contributors. We'll be able to solve problems faster and, together, build better software for citizens across the U.S.

I'm excited to see how this plays out in the coming months!

The rise of Drupal in India

Earlier this week I returned from DrupalCon Asia, which took place at IIT Bombay, one of India's premier engineering universities. I wish I could have bottled up all the energy and excitement to take home with me. From dancing on stage, to posing for what felt like a million selfies, to a motorcycle giveaway, this DrupalCon was unlike any I've seen before.

Drupalcon group photo
A little over 1,000 people attended the first DrupalCon in India. For 82% of the attendees, it was their first DrupalCon. There was also much better gender diversity than at other DrupalCons.

The excitement and interest around Drupal has been growing fast since I last visited in 2011. DrupalCamp attendance in both Delhi and Mumbai has exceeded 500 participants. There have also been DrupalCamps held in Hyderabad, Bangalore, Pune, Ahmedabad Jaipur, Srinagar, Kerala and other areas.

Indian Drupal companies like QED42, Axelerant, Srijan and ValueBound have made meaningful contributions to Drupal 8. The reason? Visibility on Drupal.org through the credit system helps them win deals and hire the best talent. ValueBound said it best when I spoke to them: "With our visibility on drupal.org, we no longer have to explain why we are a great place to work and that we are experts in Drupal.".

Also present were the large System Integrators (Wipro, TATA Consultancy Services, CapGemini, Accenture, MindTree, etc). TATA Consultancy Services has 400+ Drupalists in India, well ahead of the others who have between 100 and 200 Drupalists each. Large digital agencies such as Mirum and AKQA also sent people to DrupalCon. They are all expanding their Drupal teams in India to service the needs of growing sales in other offices around the world. The biggest challenge across the board? Finding Drupal talent. I was told that TATA Consultancy Services allows many of its developers to contribute back to Drupal, which is why they have been able to hire faster. More evidence that the credit system is working in India.

The government is quickly adopting Drupal. MyGov.in is one of many great examples; this portal was established by India's central government to promote citizen participation in government affairs. The site reached nearly two million registered users in less than a year. The government's shifting attitude toward open source is a big deal because historically, the Indian government has pushed back against open source because large organizations like Microsoft were funding many of the educational programs in India. The tide changed in 2015 when the Indian government announced that open source software should be preferred over proprietary software for all e-government projects. Needless to say, this is great news for Drupal.

Another initiative that stood out was the Drupal Campus Ambassador Program. The aim of this program is to appoint Drupal ambassadors in every university in India to introduce more students to Drupal and help them with their job search. It is early days for the program, but I recommend we pay attention to it, and consider scaling it out globally if successful.

Last but not least there was FOSSEE (Free and Open Source Software for Education), a government-funded program that promotes open source software in academic institutions, along with its sister project, Spoken Tutorial. To date, 2,500 colleges participate in the program and more than 1 million students have been trained on open source software. With the spoken part of their videos translated into 22 local languages, students gain the ability to self-study and foster their education outside of the classroom. I was excited to hear that FOSSEE plans to add a Spoken Tutorial series on Drupal course to its offerings. There is a strong demand for affordable Drupal training and certifications throughout India's technical colleges, so the idea of encouraging millions of Indian students to take a free Drupal course is very exciting -- even if only 1% of them decides to contribute back this could be a total game changer.

Open source makes a lot of sense for India's thriving tech community. It is difficult to grasp the size of the opportunity for Drupal in India and how fast its adoption has been growing. I have a feeling I will be back in India more than once to help support this growing commitment to Drupal and open source.

A history of JavaScript across the stack

Did you know that JavaScript was created in 10 days? In May 1995, Brendan Eich wrote the first version of JavaScript in 10 days while working at Netscape.

For the first 10 years of JavaScript's life, professional programmers denigrated JavaScript because its target audience consisted of "amateurs". That changed in 2004 with the launch of Gmail. Gmail was the first popular web application that really showed off what was possible with client-side JavaScript. Competing e-mail services such as Yahoo! Mail and Hotmail featured extremely slow interfaces that used server-side rendering almost exclusively, with almost every action by the user requiring the server to reload the entire web page. Gmail began to work around these limitations by using XMLHttpRequest for asynchronous data retrieval from the server. Gmail's use of JavaScript caught the attention of developers around the world. Today, Gmail is the classic example of a single-page JavaScript app; it can respond immediately to user interactions and no longer needs to make roundtrips to the server just to render a new page.

A year later in 2005, Google launched Google Maps, which used the same technology as Gmail to transform online maps into an interactive experience. With Google Maps, Google was also the first large company to offer a JavaScript API for one of their services allowing developers to integrate Google Maps into their websites.

Google's XMLHttpRequest approach in Gmail and Google Maps ultimately came to be called Ajax (originally "Asynchronous JavaScript and XML"). Ajax described a set of technologies, of which JavaScript was the backbone, used to create web applications where data can be loaded in the background, avoiding the need for full page refreshes. This resulted in a renaissance period of JavaScript usage spearheaded by open source libraries and the communities that formed around them, with libraries such as Prototype, jQuery, Dojo and Mootools. (We added jQuery to Drupal core as early as 2006.)

In 2008, Google launched Chrome with a faster JavaScript engine called V8. The release announcement read: "We also built a more powerful JavaScript engine, V8, to power the next generation of web applications that aren't even possible in today's browsers.". At the launch, V8 improved JavaScript performance by 10x over Internet Explorer by compiling JavaScript code to native machine code before executing it. This caught my attention because I had recently finished my PhD thesis on the topic of JIT compilation. More importantly, this marked the beginning of different browsers competing on JavaScript performance, which helped drive JavaScript's adoption.

In 2010, Twitter made a move unprecedented in JavaScript's history. For the Twitter.com redesign in 2010, they began implementing a new architecture where substantial amounts of server-side code and client-side code were built almost entirely in JavaScript. On the server side, they built an API server that offered a single set of endpoints for their desktop website, their mobile website, their native apps for iPhone, iPad, and Android, and every third-party application. As a result, they moved much of the UI rendering and corresponding logic to the user's browser. A JavaScript-based client fetches the data from the API server and renders the Twitter.com experience.

Unfortunately, the redesign caused severe performance problems, particularly on mobile devices. Lots of JavaScript had to be downloaded, parsed and executed by the user's browser before anything of substance was visible. The "time to first interaction" was poor. Twitter's new architecture broke new ground by offering a number of advantages over a more traditional approach, but it lacked support for various optimizations available only on the server.

Twitter suffered from these performance problems for almost two years. Finally in 2012, Twitter reversed course by passing more of the rendering back to the server. The revised architecture renders the initial pages on the server, but asynchronously bootstraps a new modular JavaScript application to provide the fully-featured interactive experience their users expect. The user's browser runs no JavaScript at all until after the initial content, rendered on the server, is visible. By using server-side rendering, the client-side JavaScript could be minimized; fewer lines of code meant a smaller payload to send over the wire and less code to parse and execute. This new hybrid architecture reduced Twitter's page load time by 80%!

In 2013, Airbnb was the first to use Node.js to provide isomorphic (also called universal or simply shared) JavaScript. In the Node.js approach, the same framework is identically executed on the server side and client side. On the server side, it provides an initial render of the page, and data could be provided through Node.js or through REST API calls. On the client side, the framework binds to DOM elements, "rehydrates" (updates the initial server-side render provided by Node.js) the HTML, and makes asynchronous REST API calls whenever updated data is needed.

The biggest advantage Airbnb's JavaScript isomorphism had over Twitter's approach is the notion of a completely reusable rendering system. Because the client-side framework is executed the same way on both server and client, rendering becomes much more manageable and debuggable in that the primary distinction between the server-side and client-side renders is not the language or templating system used, but rather what data is provisioned by the server and how.

Universal Javascript
In a universal JavaScript approach utilizing shared rendering, Node.js executes a framework (in this case Angular), which then renders an initial application state in HTML. This initial state is passed to the client side, which also loads the framework to provide further client-side rendering that is necessary, particularly to “rehydrate” or update the server-side render.

From a prototype written in 10 days to being used across the stack by some of the largest websites in the world, long gone are the days of clunky browser implementations whose APIs changed depending on whether you were using Netscape or Internet Explorer. It took JavaScript 20 years, but it is finally considered an equal partner to traditional, well-established server-side languages.

Updates from Dries straight to your mailbox