The Big Reverse of the Web

I believe that for the web to reach its full potential, it will go through a massive re-architecture and re-platforming in the next decade. The current web is "pull-based", meaning we visit websites or download mobile applications. The future of the web is "push-based", meaning the web will be coming to us. In the next 10 years, we will witness a transformation from a pull-based web to a push-based web. When this "Big Reverse" is complete, the web will disappear into the background much like our electricity or water supply. We'll forget what 'www' stood for (which was kind of dumb to begin with). These are bold statements, I understand, but let me explain you why.

In the future, content, products and services will find you, rather than you having to find them. Puma will let us know to replace our shoes and Marriott will automatically present you room options if you missed your connecting flight. Instead of visiting a website, we will proactively be notified of what is relevant and asked to take action. The dominant function of the web is to let us know what is happening or what is relevant, rather than us having to find out.

Facebook and Flipboard are early examples of what such push-based experience looks like. Facebook "pushes" a stream of personalized information designed to tell you what is happening with your friends and family; you no longer have "pull" them and ask how they are doing. Flipboard changes how we consume content by aggregating the best of the web and filtering it based on our interests; it "pushes" the relevant and interesting content to you rather than you having to "pull" the news from multiple sources. Also consider the rise of notification-centric experiences; your smartphone's notification center provides you with a stream of relevant information that is pushed to you. More recently, these notifications have become interactive; you can check in for a flight without having to open your travel app. You can buy a product without having to visit their website.

What people really want is to tune into information rather than having to work to get information. It saves them time and effort and in the long run, an improved user experience always wins. In most cases, "Show me what I want" is more useful than "Let me search around and see what I can find".

With some imagination, it's not too hard to picture how these kind of experiences could expand to other areas of the web. The way the e-commerce works today is really no different than having to visit a lot of separate physical stores or wading through hundreds of products in a department store. We shouldn't have to work so hard to find what we want. In a push-based world, we would sit back as if we were watching a fashion show -- the clothing presented could come for hundreds of different online brands but the stream is "personalized" to our needs, budget, sizes and style preferences. When the Big Reverse is complete, it will be the end of department stores and malls. Keep an eye on personalized clothing services like Trunk Club or Stitch Fix.

Ten years from now we're going to look back and recognize that search-based content discovery was broken. Today the burden is put on the user to find relevant content either via directly typing in a URL or by crafting complex search queries. While pull-based experiences might not go away; push-based experiences will dominate as they will prove to be much more efficient.

Many of you won't like it (at first), but push will win over pull. Healthcare is going through a similar transformation from pull to push; instead of going to a doctor, we'll have web-enabled hardware and software that is able to self-diagnose. Wearables like activity trackers are just the start of decades of innovation and opportunity in healthcare. Helped by the web, education is also moving from pull to push. Why go to a classroom when personalized training can come to you?

We are at the beginning of a transition bridging two distinctly different types of economies. First, a "push economy" that tries to anticipate consumer demand, creates standardized or generic products in large amounts, and "pushes" them into the market via global distribution channels and marketing. Now, a "pull economy" that—rather than creating standardized products—will create highly customized products and services produced on-demand and delivered to consumers through one-on-one relationships and truly personal experiences.

This new paradigm could be a very dramatic shift that disrupts many existing business models; advertising, search engines, app stores, online and offline retailers, and much more. For middlemen like online retailers or search engines, the push-based means they risk being disintermediated as the distribution chain becomes less useful. It marks a powerful transformation that dematerializes and de-monetizes much of the current web. While this might complicate the lives of many organizations, it will undoubtedly simplify and better the lives of consumers everywhere.

How much money to raise for your startup? [Flowchart]

From time to time, people ask me how much money to raise for their startup. I've heard other people answer that question from "never raise money" to "as little as you need" to "as much as you can".

The reason the answers vary so much is because what is best for the entrepreneur is seemingly at odds with what is best for the business. For the entrepreneur, the answer can be as little as necessary to avoid dilution or giving up control. For the business, more money can increase its chances of success. I feel the right answer is somewhere in the middle -- focus on raising enough money, so the company can succeed, but make sure you still feel good about how much control or ownership you have.

But even "somewhere in the middle" is a big spectrum. What makes this so difficult is that it is all relative to your personal risk profile, the quality of the investors you're attracting, the market conditions, the size of the opportunity, and more. There are a lot of parameters to balance.

I created the flowchart below (full-size image) to help you answer the question. This flowchart is only a framework -- it can't take into account all decision-making parameters. The larger the opportunity and the better the investors, they more I'd be willing to give up. It's better to have a small part of something big, than to have a big part of something small.

How much money to raise for your startup

Some extra details about the flowchart:

  • In general, it is good to have 18 months of runway. It gives you enough time to figure out how to get your company to the next level, but still keeps the pressure on.
  • Add 6 months of buffer to handle unexpected bumps or budgeting oversights.
  • If more money is available, I'd take it as long you don't give away too much of your company. As a starting point for how much control to give up, I use the following formula: 30% - (5% x number of the round). So if you are raising your series A (round 1), don't give away more than 25% (30 - (5 x 1)). If you are raising your series B (round 2), don't give away more than 20% (30 - (5 x 2)). If you start with 50% of the shares, using this formula, you'll still have roughly 20% of the company after 5 rounds (depending on other dilutive events such as option pool increases).

My view is that of an entrepreneur having raised over $120 million for one startup. If you're interested in an investor's view that has funded many startups, check out Michael Skok's post. Michael Skok is Acquia's lead investor and one of Acquia's Board of Directors. We both tried to answer the question from our own unique viewpoint.

5 things a government can do to grow its startup ecosystem

Building a successful company is really hard. It is hard no matter where you are in the world, but the difficulty is magnified in Europe, where people are divided by geography, regulation, language and cultural prejudice. If governments can provide European startups a competitive advantage, that could come a long way in helping to offset some of the disadvantages. In this post, I'm sharing some rough ideas for what governments could do to encourage a thriving startups ecosystem. It's my contribution to the Belgian startup manifesto (#bestartupmanifesto).

  1. Governments shouldn't obsess too much about making it easier to incorporate a company; while it is certainly nice when governments cut red tape, great entrepreneurs aren't going to be held back by some extra paperwork. Getting a company off the ground is by no means the most difficult part of the journey.
  2. Governments shouldn't decide what companies deserve funding or don't deserve funding. They will never be the best investors. Governments should play towards their strength, which is creating leverage for all instead for just a few.
  3. Governments can do quite a bit to extend a startup's runway (to compensate for the lack of funding available in Belgium). Relatively simple tax benefits result in less need for venture capital:
    • No corporate income taxes on your company for the first 3 years or until 1 million EUR in annual revenue.
    • No employee income tax or social security contributions for the first 3 years or until you hit 10 employees. Make hiring talent as cheap as possible; two employees for the price of one. (The cost of hiring an employee would effectively be the net income for the employee. The employee would still get a regular salary and social benefits.)
    • Loosen regulations on hiring and firing employees. Three months notice periods shackle the growth of startups. Governments can provide more flexibility for startups to hire and fire fast; two week notice periods for both incoming and outgoing employees. Employees who join a startup are comfortable with this level of job insecurity.
  4. Create "innovation hubs" that make neighborhoods more attractive to early-stage technology companies. Concentrate as many technology startups as possible in fun neighborhoods. Provide rent subsidies, free wifi and make sure there are great coffee shops.
  5. Build a culture of entrepreneurship. The biggest thing holding back a thriving startup community is not regulation, language, or geography, but a cultural prejudice against both failure and success. Governments can play a critical role in shaping the country's culture and creating an entrepreneurial environment where both failures and successes are celebrated, and where people are encouraged to better oneself economically through hard work and risk taking. In the end, entrepreneurship is a state of mind.

Making Drupal 8 fly

In my travels to talk about Drupal, everyone asks me about Drupal 8's performance and scalability. Modern websites are much more dynamic and interactive than 10 years ago, making it more difficult to build modern sites while also being fast. It made me realize that maybe I should write up a summary of some of the most exciting performance and scalability improvements in Drupal 8. After all, Drupal 8 will leapfrog many of its competitors in terms of how to architect and scale modern web applications. Many of these improvements benefit both small and large websites, but also allow us to build even bigger websites with Drupal.

More precise cache invalidation

One of the strategies we employ in making Drupal fast is "caching". This means we try to generate pages or page elements one time and then store them so future requests for those pages or page elements can be served faster. If an item is already cached, we can simply grab it without going through the building process again (known as a "cache hit"). Drupal stores each cache item in a "cache bin" (a database table, Memcache object, or whatever else is appropriate for the cache backend in use).

In Drupal 7 and before, when one of these cache items changes and it needs to be re-generated and re-stored (the cache gets "invalidated"), you can only delete a specific cache item, clear an entire cache bin, or use prefix-based invalidation. None of these three methods allow you to invalidate all cache items that contain data of, say, user 200. The only method that is going to suffice is clearing the entire cache bin, and this means that usually we invalidate way too much, resulting in poor cache hit ratios and wasted effort rebuilding cache items that haven't actually changed.

This problem is solved in Drupal 8 thanks to the concept of "cache tags": each cache item can have any number of cache tags. A cache tag is a compact string that describes the object being cached. Thanks to this extra metadata, we can now delete all cache items that use the user:200 cache tag, for example. This means we've deleted all the cache items we must delete, but not a single one more: optimal cache invalidation!

Drupal cache tags

Example cache tags for different cache IDs.

And don't worry, we also made sure to expose the cache tags to reverse proxies, so that efficient and accurate invalidation can happen throughout a site's entire delivery architecture.

More precise cache variation

While accurate cache invalidation makes caching more efficient, there is more we did to improve Drupal's caching. We also make sure that cached items are optimally varied. If you vary too much, duplicate cache entries will exist with the exact same content, resulting in inefficient usage of caches (low cache hit ratios). For example, we don't want a piece of content to be cached per user if it is the same for many users. If you vary too little, users might see incorrect content as two different cache entries might collide. In other words, you don't want to vary too much nor too little.

In Drupal 7 and before, it's easy to program any cached item to vary by user, by user role, and/or by page, and could even be configured through the UI for blocks. However, more targeted variations (such as by language, by country, or by content access permissions) were more difficult to program and not typically exposed in a configuration UI.

In Drupal 8, we introduced a Cache Context API to allow developers and site builders to express these variations and to make them automatically available in the configuration UI.

Drupal cache contexts

Server-side dynamic content substitution

Usually a page can be cached almost entirely except for a few dynamic elements. Often a page served to two different authenticated users looks identical except for a small "Welcome $name!" and perhaps their profile picture. In Drupal 7, this small personalization breaks the cacheability of the entire page (or rather, requires a cache context that's way too granular). Most parts of the page, like the header, the footer and certain blocks in the sidebars don't change often nor vary for each user, so why should you regenerate all those parts at every request?

In Drupal 8, thanks to the addition of #post_render_cache, that is no longer the case. Drupal 8 can render the entire page with some placeholder HTML for the name and profile picture. That page can then be cached. When Drupal has to serve that page to an authenticated user, it will retrieve it from the cache, and just before sending the HTML response to the client, it will substitute the placeholders with the dynamically rendered bits. This means we can avoid having to render the page over and over again, which is the expensive part, and only render those bits that need to be generated dynamically!

Client-side dynamic content substitution

Some things that Drupal has been rendering for the better part of a decade, such as the "new" and "updated" markers on comments, have always been rendered on the server. That is not ideal because these markers are different for every visitor and as a result, it makes caching pages with comments difficult.

The just-in-time substitution of placeholders with dynamic elements that #post_render_cache provides us can help address this. In some cases, as is the case with the comment markers, we can even do better and offload more work from the server to the client. In the case for comment markers, a certain comment is posted at a certain time — that doesn't vary per user. By embedding the comment timestamps as metadata in the DOM with a data-comment-timestamp="1424286665" attribute, we enable client-side JavaScript to render the comment markers, by fetching (and caching on the client side) the “last read" timestamp for the current user and simply comparing these numbers. Drupal 8 provides some framework code and API to make this easy.

A "Facebook BigPipe" render pipeline

With Drupal 8, we're very close to taking the client-side dynamic content substitution a step further, just like some of the world's largest dynamic websites do. Facebook has 1.35 billion monthly active users all requesting dynamic content, so why not learn from them?

The traditional page serving model has not kept up with the increase of highly personalized websites where different content is served to different users. In the traditional model, such as Drupal 7, the entire page is generated before it is sent to the browser: while Drupal is generating a page, the browser is idle and wasting its cycles doing nothing. When Drupal finishes generating the page and sends it to the browser, the browser kicks into action, and the web server is idle. In the case of Facebook, they use BigPipe. BigPipe delivers pages asynchronously instead; it parallelizes browser rendering and server processing. Instead of waiting for the entire page to be generated, BigPipe immediately sends a page skeleton to the the client so it can start rendering that. Then the remaining content elements are requested and injected into their correct place. From the user's perspective the page is rendered progressively. The initial page content becomes visible much earlier, which improves the perceived speed of the site.

We've made significant improvements to the way Drupal 8 renders pages (presentation). By default, Drupal 8 core still implements the traditional model of assembling these pieces into a complete page in a single server-side request, but the independence of each piece and the architecture of the new rendering pipeline enable different “render strategies" to be experimented with — different methods for dynamic content assembly, such as BigPipe, Edge Side Includes, or other ideas for making the most optimal use of client, server, content delivery networks and reverse proxies. In all those examples, the idea is that we can send the primary content first so the client can start rendering that. Then we send the remaining Drupal blocks, such as the navigation menu or a 'Related articles' block, and have the browser, content delivery network or reverse proxy assemble or combine these blocks into a page.

Drupal render pipeline

A snapshot of the Drupal 8 render pipeline diagram that highlights where alternative render strategies can be implemented.

Some early experiments by Wim Leers in Acquia's OCTO show that we can improve performance by a factor of about 2 compared to a recent Drupal 8 development snapshot. These breakthroughs are enabled by leveraging the various improvements we made to Drupal 8.

And much more

But that is not all. The Drupal community has actually done much more, including: complete asset dependency information (which allowed us to ensure zero JavaScript is loaded by default for anonymous users and send less data on AJAX requests), pluggable CSS/JS aggregation and minification (to support more optimal optimization algorithms), and more. We've also made sure Drupal 8 is fast by default, by having better defaults: CSS/JS aggregation enabled, JS assets being loaded from the bottom, block caching enabled, and so on.

All in all, there is a lot to look forward to in Drupal 8!

Special thanks to Acquia's Wim Leers, Alex Bronstein and Angie Byron for their contributions to this blog post.

Growing Drupal in Latin America

When I visited Brazil in 2011, I was so impressed by the Latin American Drupal community and how active and passionate the people are. The region is fun and beautiful, with some of the most amazing sites I have seen anywhere in the world. It also happens to be a strategic region for the project.

Latin American community members are doing their part to grow the project and the Drupal community. In 2014, the region hosted 19 Global Training Day events to recruit newcomers, and community leaders coordinated many Drupal camps to help convert those new Drupal users into skilled talent. Members of the Latin American community help promote Drupal at local technology and Open Source events, visiting events like FISL (7,000+ participants), Consegi (5,000+ participants) and Latinoware (4,500+ participants).

You can see the results of all the hard work in the growth of the Latin American Drupal business ecosystem. The region has a huge number of talented developers working at agencies large and small. When they aren't creating great Drupal websites like the one for the Rio 2016 Olympics, they are contributing code back to the project. For example, during our recent Global Sprint Weekend, communities in Bolivia, Colombia, Costa Rica, and Nicaragua participated and made valuable contributions.

The community has also been instrumental in translation efforts. On, the top translation is Spanish with 500 contributors, and a significant portion of those contributors come from the Latin America region. Community members are also investing time and energy translating Drupal educational videos, conducting camps in Spanish, and even publishing a Drupal magazine in Spanish. All of these efforts lower the barrier to entry for Spanish speakers, which is incredibly important because Spanish is one of the top spoken languages in the world. While the official language of the Drupal project is English, there can be a language divide for newcomers who primarily speak other languages.

Last but not least, I am excited that we are bringing DrupalCon to Latin America next week. This is the fruit of many hours spent by passionate volunteers in the Latin American local communities, working together with the Drupal Association to figure out how to make a DrupalCon happen in this part of the world. At every DrupalCon we have had so far, we have seen an increase in energy for the project and a bump in engagement. Come for the software, stay for the community! Hasta pronto!


Dries Buytaert is the original creator and project lead of Drupal and the co-founder and CTO of Acquia. He writes about Drupal, startups, business, photography and building the world we want to exist in.

Updates from Dries straight to your mailbox