The future of software is data-driven

Marc Andreessen famously said that software is eating the world. While I certainly agree with Marc that software companies are redefining our economies, I believe that much of that technological shift is being driven by data. So is the value of a business in the data or is it in the software? I believe data is eating the world because the value is increasingly more in the data and not the software. Let's investigate why.

Data-driven experiences

Netflix provides a great example of a data-driven customer-centric company. By introducing streaming video, their software "ate" the traditional DVD business. But Netflix soon realized their future wasn't in the medium of delivery -- it was in the wealth of data generated simply by people using the service. The day-to-day data generated by Netflix viewers provides a crucial ingredient to competing in the marketplace and defining the company's mission: improving the quality of the service.

To that end, Netflix uses passive data -- the information gathered quietly in the background without disrupting users' natural behaviors -- to provide TV and movie recommendations, as well as to optimize the quality of services, such as streaming speed, playback quality, subtitles, or closed captioning. Of course, Netflix subscribers can contribute active feedback to the company, such as movie reviews or feedback on the accuracy of a translation, but the true value of Netflix's user data is in the quiet, zero-effort observation that allows the company to optimize experiences with no friction or disruption to regular user behavior. In fact, the company even hosted several competitions to invent better algorithms for user ratings, with a winning prize of $1M USD.

Within very saturated marketplaces, data is also becoming a key differentiator for some companies. For example, when Google first started, its value was almost entirely centered around the quality of its Pagerank algorithm, or its "software". But Google did not rest on the laurels of having good software, and prioritized data-driven insights as the future of the company. Consider Google Waze, the world's largest community-based traffic and navigation app. Google Waze relies heavily on both active consumer input and passive location-based data, combined with a sophisticated routing algorithm. The routing algorithm alone would not be enough to differentiate Waze from the other navigation systems of the world. Consumers are demanding more accurate maps and real-time traffic information, which could not happen without the use of data.

The future of software

There is another element in the rising importance of data: not only is the sheer amount of consumer data growing, but software is simultaneously becoming much easier to build. Developers can leverage new software programming tools, open source, and internet-based services to build more complex software in less time. As a result, the underlying intrinsic value of software companies is diminishing.

Netflix and Google are still disruptive companies, but no longer primarily because of their software -- it's their ability to use the data their customers produce to extend their engagement with customers. Their actual software is increasingly being commoditized; recommendation engines and navigation software both exist in open source and are no longer trade secrets.

Tomorrow's applications will consume multiple sources of data to create a fine-grained context; they will leverage calendar data, location data, historic clickstream data, social contacts, information from wearables, and much more. All that rich data will be used as the input for predictive analytics and personalization services. Eventually, data-driven experiences will be the norm.

And this basic idea doesn't even begin to cover the advances in machine learning, artificial intelligence, deep learning and beyond -- collectively called "machine intelligence". Looking forward even more, computers will learn to do things themselves from data rather than being programmed by hand. They can learn faster themselves than we'd be able to program them. In a world where software builds itself, computers will only be limited by the data they can or cannot access, not by their algorithms. In such a future, is the value in the software or in the data?

Rethinking business

As value shifts from software to the ability to leverage data, companies will have to rethink their businesses, just as Netflix and Google did. In the next decade, data-driven, personalized experiences will continue to accelerate, and development efforts will shift towards using contextual data collected through passive user behaviors.

Companies of the future have a lot on their plates. More than ever, they'll need to adapt to all types and formats of data (closed, open, structured and unstructured); leverage that data to make their product or service better for users; navigate the gray area around privacy concerns; and even reconsider the value of their intellectual property derived from software. They'll have to do all this while providing more contextualized, personalized, and automated experiences. "Data-driven" will spell a win-win situation for both users and businesses alike.

Attitude beats experience

The older I get, the quicker the years seem to fly by. As I begin to reflect on a great 2014, one thing is crystal clear again. People are the most important thing to any organization. Having a great team is more important than having a great idea. A good team will figure out how to make something great happen; they'll pivot, evolve and claw their way to success. I see it every day at Acquia, the Drupal Association or the Drupal community. I'm fortunate to be surrounded by so many great people.

By extension, recruiting is serious business. How do you figure out if someone is a great fit for your organization? Books have been written about finding and attracting the right people, but for me the following quote from Dee Hock, the founder of Visa, sums it up perfectly.

"Hire and promote first on the basis of integrity; second, motivation; third, capacity; fourth, understanding; fifth, knowledge; and last and least, experience. Without integrity, motivation is dangerous; without motivation, capacity is impotent; without capacity, understanding is limited; without understanding, knowledge is meaningless; without knowledge, experience is blind." — Dee Hock, founder of Visa.

Most hiring managers get it wrong and focus primarily on experience. While experience can be important, attitude is much more important. Attitude, not experience, is what creates a strong positive culture and what turns users and customers into raving fans.

Announcing the Drupal 8 Accelerate Fund

Today the Drupal Association announced a new program: the Drupal 8 Accelerate Fund. Drupal 8 Accelerate Fund is a $125,000 USD fund to help solve critical issues and accelerate the release of Drupal 8.

The Drupal Association is guaranteeing the funds and will try to raise more from individual members and organizations within the Drupal community. It is the Drupal 8 branch maintainers — Nathaniel Catchpole, Alex Pott, Angie Byron, and myself — who will decide on how the money is spent. The fund provides for both "top-down" (directed by the Drupal 8 branch maintainers) and "bottom-up" (requested by other community members) style grants. The money will be used on things that positively impact the Drupal 8 release date, such as hiring contributors to fix critical bugs, sponsoring code sprints to fix specific issues, and other community proposals.

Since the restructuring of the Drupal Association, I have encouraged the Drupal Association staff and Board of Directors to grow into our ambitious mission; to unite a global open source community to build and promote Drupal. I've also written and talked about the fact that scaling Open Source communities is really hard. The Drupal 8 Accelerate Fund is an experiment with crowdsourcing as a means to help scale our community which is unique compared to other efforts because it is backed by the official non-profit organization that fosters and supports Drupal.

I feel that the establishment of this fund is an important step towards more sustainable core development. My hope is that if this round of funding is successful that this can grow over time to levels that could make an even more meaningful impact on core, particularly if we complement this with other approaches and steps, such as organization credit on

This is also an opportunity for Drupal companies to give back to Drupal 8 development. The Drupal Association board is challenging itself to raise $62,500 USD (half of the total amount) to support this program. If you are an organization who can help support this challenge, please let us know. If you're a community member with a great idea on how we might be able to spend this money to help accelerate Drupal 8, you can apply for a grant today.

Business model innovation beats technical innovation

Business model innovation is usually more powerful than technical innovation; it is more disruptive and harder to copy than technical innovation. And yet, so many companies are focused on technical innovation to compete.

Consider Airbnb. What makes them so successful is not a technical advantage, but a business model advantage that provides them near-zero marginal cost. For a traditional hotel chain to increase its capacity, it needs to build more physical space at significant cost. Instead of shouldering that setup cost, Airbnb can add another room to its inventory at almost no cost by enabling people to share their existing houses. That is a business model innovation. Furthermore, it is extremely difficult for the traditional hotel chain to switch its business model to match Airbnb's.

The same is true in Open Source software. While it is true that Open Source often produces technically superior software, its real power may be its business model innovation: co-creation. Open Source software like Drupal or Linux is a co-created product; thousands of contributors build and enhance Drupal and everyone benefits from that. A large Open Source community produces vastly more software than a proprietary competitor, and shares in the production and go-to-market costs. It disrupts proprietary software companies where the roles of production and consumption are discrete and the production and go-to-market costs are high. While established companies can copy key technical innovations, it is extremely difficult to switch a proprietary business model to an Open Source business model. It affects how they build their software, how they monetize the software, how they sell and market their software, their cost structure, and more. Proprietary software companies will lose against thriving Open Source communities. I don't see how companies like HP, Oracle and SAP could change their business model while living quarter to quarter in the public markets; changing their business model would take many years and could disrupt their revenues.

Take Amazon Web Services (AWS), one of the most disruptive developments in the IT world the past decade. While AWS' offerings are rich and often ahead of the competition, the biggest reason for the company's success is its business model. Amazon not only offers consumption-based pricing ('pay as you consume' vs 'pay as you configure'), it's also comfortable operating a low-margin business. Almost 10 years after AWS launched, at a time that vast amounts of computing are moving into the cloud, HP, Oracle and SAP still don't have competitive cloud businesses. While each of these companies could easily close technical gaps, they have been unable to disrupt their existing business models.

If you're in a startup, innovating on a business model is easier than if you're in a large company. In fact, an innovative business model is the best weapon you have against large incumbents. Technical innovation may give you a 6 to 18 month competitive advantage, but the advantage from business model innovation can be many years. Too many startups focus on building or acquiring innovative or proprietary technology in order to win in the market. While there is usually some technical innovation around the edges, it is business model innovation that makes a successful, long-standing organization -- it tends to be a lot harder to copy than technical innovation.

The power of self-managed teams in Drupal

The concept of official initiatives came out of lessons learned from the Drupal 7 development. We learned a lot from that and in a recent blog post about Drupal initiative leads, I recognized that we need to evolve our tools, our processes, and our organizational design. Others like Nathaniel Catchpole, Larry Garfield and Gábor Hojtsy have shared some of their thoughts already. One of the things I'm most proud of is that the Drupal community is always looking to improve and reinvent itself. Evolving is an important part of our culture. Each time it will get better, but still won't be perfect.

For me, one of the biggest take-aways (but not the only one) is that for an initiative to succeed, it needs to be supported by a team. An initiative needs to carry out a technical vision, plan the work, communicate with all stakeholders, mobilize volunteers, raise funding, organize sprints, and more. It can easily be more than one person can handle -- especially if it isn't your full-time job or if your initiative is complex.

More specifically, we have learned that the most successful initiatives appear to be run by teams that are self-managed; the team members collaborate in the development of the initiative, but also share both managerial and operational responsibilities like planning, coordinating, communicating, sprint organizing and more.

Because self-managed teams are both responsible for their outcomes and in control of their decision-making process, members of a self-managing team are usually more motivated than traditional hierarchical teams. This independence and greater responsibility are important in volunteer communities. Self-managed teams also build and maintain institutional knowledge. The outcome of their work is also more easily accepted by other stakeholders (like core committers) because they have already built a lot of consensus.

If I were to be an initiative lead, I'd feel strongly about building my own team rather than being handed a team. My initial assumption was that each initiative lead would build his/her own team. In hindsight, that was a mistake. Team building is not easy. It requires a time investment that can seem to compete with technical priorities. This is an important lesson and something we can do better going forward. Before making an initiative official, we have to make sure that each initiative has a good team and the support to be successful -- either we can help create a team, provide more coaching or formal training around team building, or we shouldn't designate the initiative official until such a team has coalesced.


Dries Buytaert is the original creator and project lead of Drupal and the co-founder and CTO of Acquia. He writes about Drupal, startups, business, photography and building the world we want to exist in.

Updates from Dries straight to your mailbox