Last year around this time, I wrote that The Big Reverse of Web would force a major re-architecture of the web to bring the right information, to the right person, at the right time, in the right context. I believe that conversational interfaces like Amazon Echo are further proof that the big reverse is happening.
New user experience and distribution platforms only come along every 5-10 years, and when they do, they cause massive shifts in the web's underlying technology. The last big one was mobile, and the web industry adapted. Conversational interfaces could be the next user experience and distribution platform – just look at Amazon Echo (aka Alexa), Facebook's messenger or Microsoft's Conversation-as-a-Platform.
Today, hardly anyone questions whether to build a mobile-optimized website. A decade from now, we might be saying the same thing about optimizing digital experiences for voice or chat commands. The convenience of a customer experience will be a critical key differentiator. As a result, no one will think twice about optimizing their websites for multiple interaction patterns, including conversational interfaces like voice and chat. Anyone will be able to deliver a continuous user experience across multiple channels, devices and interaction patterns. In some of these cross-channel experiences, users will never even look at a website. Conversational interfaces let users disintermediate the website by asking anything and getting instant, often personalized, results.
To prototype this future, my team at Acquia built a fully functional demo based on Drupal 8 and recorded a video of it. In the demo video below, we show a sample supermarket chain called Gourmet Market. Gourmet Market wants their customers to not only shop online using their website, but also use Echo or push notifications to do business with them.
We built an Alexa integration module to connect Alexa to the Gourmet Market site and to answer questions about sale items. For example, you can speak the command: "Alexa, ask Gourmet Market what fruits are on sale today". From there, Alexa would make a call to the Gourmet Market website, finding what is on sale in the specified category and pull only the needed information related to your ask.
On the website's side, a store manager can tag certain items as "on sale", and Alexa's voice responses will automatically and instantly reflect those changes. The marketing manager needs no expertise in programming -- Alexa composes its response by talking to Drupal 8 using web service APIs.
The demo video also shows how a site could deliver smart notifications. If you ask for an item that is not on sale, the Gourmet Market site can automatically notify you via text once the store manager tags it as "On Sale".
From a technical point of view, we've had to teach Drupal how to respond to a voice command, otherwise known as a "Skill", coming into Alexa. Alexa Skills are fairly straightforward to create. First, you specify a list of "Intents", which are basically the commands you want users to run in a way very similar to Drupal's routes. From there, you specify a list of "Utterances", or sentences you want Echo to react to that map to the Intents. In the example of Gourmet Market above, the Intents would have a command called
GetSaleItems. Once the command is executed, your Drupal site will receive a webhook callback on
/alexa/callback with a payload of the command and any arguments. The Alexa module for Drupal 8 will validate that the request really came from Alexa, and fire a Drupal Event that allows any Drupal module to respond.
It's exciting to think about how new user experiences and distribution platforms will change the way we build the web in the future. As I referenced in Drupalcon New Orleans keynote, the Drupal community needs to put some thought into how to design and build multichannel customer experiences. Voice assistance, chatbots or notifications are just one part of the greater equation. If you have any further thoughts on this topic, please share them in the comments.
In March, I did a presentation at SxSW that asked the audience a question I've been thinking about a lot lately: "Can we save the open web?".
The web is centralizing around a handful of large companies that control what we see, limit creative freedom, and capture a lot of information about us. I worry that we risk losing the serendipity, creativity and decentralization that made the open web great.
While there are no easy answers to this question, the presentation started a good discussion about the future of the open web, the role of algorithms in society, and how we might be able to take back control of our personal information.
I'm going to use my blog to continue the conversation about the open web, since it impacts the future of Drupal. I'm including the video and slides (PDF, 76 MB) of my SxSW presentation below, as well as an overview of what I discussed.
Here are the key ideas I discussed in my presentation, along with a few questions to discuss in the comments.
Idea 1: An FDA-like organization to provide oversight for algorithms. While an "FDA" in and of itself may not be the most ideal solution, algorithms are nearly everywhere in society and are beginning to impact life-or-death decisions. I gave the example of an algorithm for a self-driving car having to decide whether to save the driver or hit a pedestrian crossing the street. There are many other life-or-death examples of how unregulated technology could impact people in the future, and I believe this is an issue we need to begin thinking about now. What do you suggest we do to make the use of algorithms fair and trustworthy?
Idea 2: Open standards that will allow for information-sharing across sites and applications. Closed platforms like Facebook and Google are winning because they're able to deliver a superior user experience driven by massive amounts of data and compute power. For the vast majority of people, ease-of-use will trump most concerns around privacy and control. I believe we need to create a set of open standards that enable drastically better information-sharing and integration between websites and applications so independent websites can offer user experiences that meet or exceeds that of the large platforms. How can the Drupal community help solve this problem?
Idea 3: A personal information broker that allows people more control over their data. In the past, I've written about the idea for a personal information broker that will give people control over how, where and for how long their data is used, across every single interaction on the web. This is no small feat. An audience member asked an interesting question about who will build this personal information broker -- whether it will be a private company, a government, an NGO, or a non-profit organization? I'm not really sure I have the answer, but I am optimistic that we can figure that out. I wish I had the resources to build this myself as I believe this will be a critical building block for the web. What do you think is the best way forward?
Ultimately, we should be building the web that we want to use, and that we want our children to be using for decades to come. It's time to start to rethink the foundations, before it's too late. If we can move any of these ideas forward in a meaningful way, they will impact billions of people, and billions more in the future.
Today is another big day for Drupal as we just released Drupal 8.1.0. Drupal 8.1.0 is an important milestone as it is a departure from the Drupal 7 release schedule where we couldn't add significant new features until Drupal 8. Drupal 8.1.0 balances maintenance with innovation.
On my blog and in presentations, I often talk about the future of Drupal and where we need to innovate. I highlight important developments in the Drupal community, and push my own ideas to disrupt the status quo. People, myself included, like to talk about the shiny innovations, but it is crucial to understand that innovation is only a piece of how we grow Drupal's success. What can't be forgotten is the maintenance, the bug fixing, the work on Drupal.org and our test infrastructure, the documentation writing, the ongoing coordination and the processes that allow us to crank out stable releases.
We often recognize those who help Drupal innovate or introduce novel things, but today, I'd like us to praise those who maintain and improve what already exists and that was innovated years ago. So much of what makes Drupal successful is the "daily upkeep". The seemingly mundane and unglamorous effort that goes into maintaining Drupal has a tremendous impact on the daily life of hundreds of thousands of Drupal developers, millions of Drupal content managers, and billions of people that visit Drupal sites. Without that maintenance, there would be no stability, and without stability, no room for innovation.
The web felt very different fifteen years ago, when I founded Drupal. Just 7 percent of the population had internet access, there were only around 20 million websites, and Google was a small, private company. Facebook, Twitter, and other household tech names were years away from being founded. In these early days, the web felt like a free space that belonged to everyone. No one company dominated as an access point or controlled what users saw. This is what I call the "open web".
But the internet has changed drastically over the last decade. It's become a more closed web. Rather than a decentralized and open landscape, many people today primarily interact with a handful of large platform companies online, such as Google or Facebook. To many users, Facebook and Google aren't part of the internet -- they are the internet.
I worry that some of these platforms will make us lose the original integrity and freedom of the open web. While the closed web has succeeded in ease-of-use and reach, it raises a lot of questions about how much control individuals have over their own experiences. And, as people generate data from more and more devices and interactions, this lack of control could get very personal, very quickly, without anyone's consent. So I've thought through a few potential ideas to bring back the good things about the open web. These ideas are by no means comprehensive; I believe we need to try a variety of approaches before we find one that really works.
It's undeniable that companies like Google and Facebook have made the web much easier to use and helped bring billions online. They've provided a forum for people to connect and share information, and they've had a huge impact on human rights and civil liberties. These are many things for which we should applaud them.
But their scale is also concerning. For example, Chinese messaging service Wechat (which is somewhat like Twitter) recently used its popularity to limit market choice. The company banned access to Uber to drive more business to their own ride-hailing service. Meanwhile, Facebook engineered limited web access in developing economies with its Free Basics service. Touted in India and other emerging markets as a solution to help underserved citizens come online, Free Basics allows viewers access to only a handful of pre-approved websites (including, of course, Facebook). India recently banned Free Basics and similar services, claiming that these restricted web offerings violated the essential rules of net neutrality.
Beyond market control, the algorithms powering these platforms can wade into murky waters. According to a recent study from the American Institute for Behavioral Research and Technology, information displayed in Google could shift voting preferences for undecided voters by 20 percent or more -- all without their knowledge. Considering how narrow the results of many elections can become, this margin is significant. In many ways, Google controls what information people see, and any bias, intentional or not, has a potential impact on society.
In the future, data and algorithms will power even more grave decisions. For example, code will decide whether a self-driving car stops for an oncoming bus or runs into pedestrians.
It's possible that we're reaching the point where we need oversight for consumer-facing algorithms. Perhaps it's time to consider creating an oversight committee. Similar to how the FDA monitors the quality and safety of food and drugs, this regulatory body could audit algorithms. Recently, I spoke at Harvard's Berkman Center for the Internet and Society, where attendees also suggested a global "Consumer Reports" style organization that would "review" the results of different company's algorithms, giving consumers more choice and transparency.
But algorithmic oversight is not enough. In numbers by the billions, people are using free and convenient services, often without a clear understanding of how and where their data is being used. Many times, this data is shared and exchanged between services, to the point where people don't know what's safe anymore. It's an unfair trade-off.
I believe that consumers should have some level of control over how their data is shared with external sites and services; in fact, they should be able to opt into nearly everything they share if they want to. If a consumer wants to share her shoe size and color preferences with every shopping website, her experience with the web could become more personal, with her consent. Imagine a way to manage how our information is used across the entire web, not just within a single platform. That sort of power in the hands of the people could help the open web gain an edge on the hyper-personalized, easy-to-use "closed" web.
In order for a consumer-based, opt-in data sharing system described above to work, the entire web needs to unite around a series of common standards. This idea in and of itself is daunting, but some information-sharing standards like OAuth have shown us that it can be done. People want the web to be convenient and easy-to-use. Website creators want to be discovered. We need to find a way to match user preferences and desires with information throughout the open web. I believe that collaboration and open standards could be a great way to decentralize power and control on the web.
The web will only expand into more aspects of our lives. It will continue to change every industry, every company, and every life on the planet. The web we build today will be the foundation for generations to come. It's crucial we get this right. Do we want the experiences of the next billion web users to be defined by open values of transparency and choice, or the siloed and opaque convenience of the walled garden giants dominating today?
I believe we can achieve a balance between companies' ability to grow, profit and innovate, while still championing consumer privacy, freedom and choice. Thinking critically and acting now will ensure the web's open future for everyone.
(I originally wrote this blog post as a guest article for The Daily Dot. I also gave a talk yesterday at SXSW on a similar topic, and will share the slides along with a recording of my talk when it becomes available in a couple of weeks.)
This blog post is co-authored with Devesh Raj, Senior Vice President and Head of Strategy and Planning at Comcast-NBCU. Devesh and I are friends and both Young Global Leader at the World Economic Forum. In this blog post we share some of our observations and thoughts after attending the World Economic Forum's annual meeting in Davos.
This year's World Economic Forum Annual Meeting in Davos focused on the Fourth Industrial Revolution, a term coined by Klaus Schwab to describe the new generation of technological advances – sensors, robotics, artificial intelligence, 3D printing, precision medicine – coming together to define the next wave of progress.
These new technologies have the potential to transform our lives. Beyond sci-fi like scenarios – such as each of us having our own personal R2-D2, summoning our Batmobile, or colonizing Mars – these advances also have the potential to solve many real-world problems. With more intelligent, automated technology, we could generate renewable energy, address climate change, connect billions of people to the internet, develop affordable housing solutions and cure chronic diseases.
These advances are not far into the future. A recent report on Technology Tipping Points and Societal Impact anticipates many such moments of inflection within our lifetimes – in fact, we may see major advances in transportation, artificial intelligence, and new payment technology as soon as the next decade. Yet, somewhat surprisingly, much of the discussion in Davos last month focused on the negative impacts of these technologies, rather than their positive potential.
|- Storage for all||- Robots and services||- The Internet of Things
- Wearable internet
- 3D printing and manufacturing
|- Implantable technologies
- Big Data for decisions
- Vision as the new interface
- Our digital presence
- Governments and the block chain
- A supercomputer in your pocket
|- Ubiquitous computing
- 3D printing and human health
- The connected home
|- 3D printing and consumer products
- AI and white-collar jobs
- The sharing economy
|- Driverless cars
- AI and decision-making
- Smart cities
One consistent, fearful theme was the potential for job losses. As automation continues to replace manufacturing or blue collar jobs, artificial intelligence will subsequently do the same for skilled, white collar jobs in banking, law or medicine. Estimates as to the impact this will have on jobs vary, but many prognostications in Davos suggested a depressive impact on the global economy. While it's true that technological leaps have often eliminated older, human-powered methods of doing things, many in Davos also recognized that advances in technology create new jobs, most of which we can't even dream of today. For example, the invention of the airplane created hundreds of thousands of jobs, from pilots, to stewards, to airport personnel, to international agents and more prognostications not to mention the transformative economic impact of billions of people traveling vast distances in a short span of time.
A second concern at Davos was growing inequality in the world between "digital haves" and "have-nots". This was reflected both as a challenge among nations – developed vs. developing – but also an issue for specific socio-economic groups within individual nations, some of which arguably are still not past the second or third industrial revolution. What does 3D printing or precision medicine do, for example, for rural parts of India and Africa that still don't have reliable electricity, while urban centers in those same countries race towards an era of smart, automated living?
A third common concern (particularly driven by robotics and artificial intelligence) was the "dehumanization" of our lives. There was a case for a renewed emphasis on qualities that make us uniquely human – empathy, sensitivity, creativity and inspiration.
Another issue centered on the ethical and moral challenges of many advances. Some conversations at Davos discussed the dangerous potential of eugenics-like scenarios in medicine, enabled by advances such as CRISPR/Cas9. On the flip side, could machines make positive decisions regarding human lives, such as a self-driving car making a choice between hitting a pedestrian or sacrificing its passenger?
One could argue some of these concerns are overblown Luddism. But in some ways, it doesn't matter – the march of technological progress is inevitable, as it has always been. Certainly, no one at Davos suggested slowing down the pace of technological advancement. The gist of the discussions was that we should figure out how to avoid, or address, the negative, unintended consequences of these changes.
We believe there is a major challenge with the Fourth Industrial Revolution that didn't get adequate attention in Davos – the issue of prioritization.
To date, the technological innovation that has driven the Fourth Industrial Revolution is shaped by the commercial prospects of small or large firms in the market. After all, one definition of "innovation" is the commercial application of invention. As an example, investment in alternative energy R&D fluctuates depending on oil prices, just as demand for hybrid or electric vehicles become more or less attractive depending on gasoline prices.
What if, instead of being driven solely by commercial returns, we could focus the Fourth Industrial Revolution more directly on the big problems our world faces? What if we could prioritize technological advances that have the most beneficial impact to society?
The world has recently defined its problems very clearly in a set of 17 Sustainable Development Goals (Wikipedia), also known as the Global Goals, that were adopted by all countries last year to "end poverty, protect the planet and ensure prosperity for all". The goals cover poverty, hunger and food security, health, education, energy, and water and sanitation – to name a few. A successor list to the earlier Millennium Development Goals, the Sustainable Development Goals get quite specific.
Take Goal 3 as an example: "Ensure healthy lives and promote well-being for all at all ages". This goal is linked to 12 targets, including these top three:
Of course, technological advancement is not the only solution to all Sustainable Development Goals – there is much more to do – but it is likely one of the major contributors.
As the world thinks through how to harness the Fourth Industrial Revolution, we think it is worth questioning which technologies we should be prioritizing to meet these Sustainable Development Goals. How do we draft policies and create economic incentives to encourage the right types of technology advances? What should governments and the private sector do differently to focus technology on addressing these goals? How do we direct the energy and creativity of millions of entrepreneurs towards improving the state of the world?
The world's innovation system is powerful and has generally worked well. However, it could use a guiding hand to nudge it in a direction that will benefit the planet beyond the incentives of commercial returns. Expanding our criteria for importance to solving areas of global need is not an inherently anti-capitalist idea. But it is one that would channel capitalism in the best direction for humanity as a whole. That, we hope, is the real agenda initiated by the focus in Davos on the Fourth Industrial Revolution, which the world will seek to address in the coming year.
Last week, a member of my family died in a car accident. Jasper was on his way home and was hit by a taxi. He fought for his life, but died the next day. Jasper was only 16 years old. I was at Davos and at one point I had to step out of the conference to cry. Five years ago, another family member died after she was hit by a truck when crossing the road.
It's hard to see a tragedy like this juxtaposed against a conference filled with people talking about improving the state of the world. These personal losses make me want to fast-forward to a time in the future where self-driving cars are normal, and life-saving innovations don't have as much regulatory red tape to cut through before they can have an impact. It's frustrating that we may have the right technology in sight today, but aren't making it available, especially when people's lives are at stake.
Imagine two busses full of people crashing, killing everyone on board, every single day. That is how many people die on America's roads every day. In fact, more people are killed by cars than guns, but I don't see anyone calling for a ban on automobiles. Car accidents (and traffic jams) are almost always the result of human error. It is estimated that self-driving cars could reduce deaths on the road by 90%. That is almost 30,000 lives saved each year in the US alone. The life-saving estimates for driverless cars are on par with the efficacy of modern vaccines. I hope my children, now ages 6 and 8, will never need a driver's license and can grow up in a world with driverless cars.
The self-driving car isn't as far off as you might think but is still being held back by government regulators. Delayed technology isn't limited to self-driving cars. Life-saving innovations in healthcare are often held back by regulatory requirements. The challenge of climate change could be addressed faster if the regulatory uncertainty around solar and wind power permits and policies were reduced. The self-serving interest of lobbying groups focused on maintaining the status quo for industries like Big Oil make it harder for alternative energies to gain momentum.
Regulators need to frame their jobs differently; they need to ask how they can facilitate and enable emerging disruptive innovations, rather than maintain existing systems. Their job should focus more on removing any barriers that prevent disruptions from having a faster impact. If they do this job well, some established institutions will fail. In some cases, economic sacrifices by the incumbents should be of lesser concern than advancing social health and safety for the benefit of society. I'm less concerned about technology destroying jobs, and more concerned about our children not being able to benefit from available technical advances that improve their lives. We should realize that opportunities for long-term economic growth come with short-term disruption or temporary pain.
Losing family members in fatal accidents makes one think about what could have been done. I'm often asked how one can create a "Silicon Valley" model elsewhere in the world. I may have an answer. If you want to out-"Silicon Valley" Silicon Valley, create a region with a regulatory environment that supports prompt, responsible innovation to drive the adoption and iteration of new technologies. A region where people can responsibly launch self-driving cars, fast-track healthcare and address climate change. A region where long-term advantages are valued more than short-term disadvantages. Such a region would attract capital and entrepreneurs, and would be much better for our children.
Volkswagen's recent emissions scandal highlighted the power that algorithms wield over our everyday lives. As technology advances and more everyday objects are driven almost entirely by software, it's become clear that we need a better way to catch cheating software and keep people safe.
A solution could be to model regulation of the software industry after the US Food and Drug Administration's oversight of the food and drug industry. The parallels are closer than you might think.
When Volkswagen was exposed for programming its emissions-control software to fool environmental regulators, many people called for more transparency and oversight over the technology.
One option discussed by the software community was to open-source the code behind these testing algorithms. This would be a welcome step forward, as it would let people audit the source code and see how the code is changed over time. But this step alone would not solve the problem of cheating software. After all, there is no guarantee that Volkswagen would actually use the unmodified open-sourced code.
Open-sourcing code would also fail to address other potential dangers. Politico reported earlier this year that Google's algorithms could influence the outcomes of presidential elections, since some candidates could be featured more prominently in its search results.
Research by the American Institute for Behavioral Research and Technology has also shown that Google search results could shift voting preferences by 20% or more (up to 80% in certain demographic groups). This could potentially flip the margins of voting elections worldwide. But since Google's private algorithm is a core part of its competitive advantage, open-sourcing it is not likely to be an option.
The same problem applies to the algorithms used in DNA testing, breathalyzer tests and facial recognition software. Many defense attorneys have requested access to the source code for these tools to verify the algorithms' accuracy. But in many cases, these requests are denied, since the companies that produce the proprietary criminal justice algorithms fear a threat to their businesses' bottom line. Yet clearly we need some way to ensure the accuracy of software that could put people behind bars.
So how exactly could software take a regulatory page from the FDA in the United States? Before the 20th century, the government made several attempts to regulate food and medicine, but abuse within the system was still rampant. Food contamination caused widespread illness and death, particularly within the meatpacking industry.
Meanwhile, the rise of new medicines and vaccines promised to eradicate diseases, including smallpox. But for every innovation, there seemed to be an equal amount of extortion by companies making false medical claims or failing to disclose ingredients. The reporting of journalists like Upton Sinclair made it abundantly clear by the early 1900s that the government needed to intervene to protect people and establish quality standards.
In 1906, President Theodore Roosevelt signed the Food and Drug Act into law, which prevented false advertising claims, set sanitation standards, and served as a watchdog for companies that could cause harm to consumers' welfare. These first rules and regulations served as a foundation for our modern-day FDA, which is critical to ensuring that products are safe for consumers.
The FDA could be a good baseline model for software regulation in the US and countries around the world, which have parallel FDA organizations including the European Medicines Agency, Health Canada, and the China Food and Drug Administration.
Just as the FDA ensures that major pharmaceutical companies aren't lying about the claims they make for drugs, there should be a similar regulator for software to ensure that car companies are not cheating customers and destroying the environment in the process. And just as companies need to disclose food ingredients to prevent people from ingesting poison, companies like Google should be required to provide some level of guarantee that they won't intentionally manipulate search results that could shape public opinion.
It's still relatively early days when it comes to discovering the true impact of algorithms in consumers' lives. But we should establish standards to prevent abuse sooner rather than later. With technology already affecting society on a large scale, we need to address emerging ethical issues head-on.
(I originally wrote this blog post as a guest article for Quartz.)
The Industrial Revolution, started in the middle of the 18th century, transformed the world. It marks the start of a major turning point in history that would influence almost every aspect of daily life. The Industrial Revolution meant the shift from handmade to machine-made products and increased productivity and capacity. Technological change also enabled the growth of capitalism. Factory owners and others who controlled the means of production rapidly became very rich and working conditions in the factories were often less than satisfactory. It wasn't until the 20th century, 150 years after its beginning, that the Industrial Revolution ended creating a much higher standard of living than had ever been known in the pre-industrial world. Consumers benefited from falling prices for clothing and household goods. The impact on natural resources, public health, energy, medicine, housing and sanitation meant that chronic hunger, famines and malnutrition started to disappear and the life expectancy started to increase dramatically.
An undesired side-effect of the Industrial Revolution is that instead of utilizing artisans to produce hand-made items, machines started to take the place of the artisans. Before the industrial revolution, custom-made goods and services were the norm. The one-on-one relationships that guilds had with their customers sadly got lost in an era of mass-production. But what is exciting me about the world today is that we're on the verge of being able to bring back one-on-one relationships with our customers, while maintaining increased productivity and capacity.
As the Big Reverse of the Web plays out and information and services are starting to come to us, we'll see the rise of a new trend I call "B2One". We're starting to hear a lot of buzz around personalization, as evidenced by companies like The New York Times making delivery of personalized content a core part of their business strategy. Another recent example is Facebook testing shopping concepts, letting users browse a personal feed of clothing and other items based on their "likes". I'd imagine these types of feeds could get smarter and smarter, refining themselves over time as a user browses or buys. Or just yesterday, Facebook launched Notify, an iOS app that pushes you personalized notifications from up to 70 sites.
These recent examples are early signs of how we're evolving from B2C to B2One (or from B2B2C to B2B2One), a world where all companies have a one-on-one relationship with their customers and personalized experiences will become the norm. Advances in technology allow us to get back what we lost hundreds of years ago in the Industrial Revolution, which in turn enables the world to innovate on business models. The B2One paradigm will be a very dramatic shift that disrupts existing business models (advertising, search engines, online and offline retailers) and every single industry.
For example, an athletic apparel company such as Nike could work sensor technology into its shoes, telling you once you've run a certain number of miles and worn them out. Nike would have enough of a one-on-one relationship with you to push an alert to your smartphone or smartwatch with a "buy" button for new shoes, before you even knew you needed them. This interaction is a win-win for both you and Nike; you don't need to re-enter your sizing and information into a website, and Nike gets a sale directly from you disrupting both the traditional and online retail supply chain (basically, this is bad news for intermediaries like Amazon, Zappos, clothing malls, Google, etc).
I believe strongly in the need for data-driven personalization to create smarter, pro-active digital experiences that bring back one-on-one relationships between producers and consumers. We have to dramatically improve delivering these personal one-on-one interactions. It means we have to get better at understanding the user's journey, the user's context, matching the right information/service to the user and making technology disappear in the background.
Algorithms are shaping what we see and think -- even what our futures hold. The order of Google's search results, the people Twitter recommends us to follow, or the way Facebook filters our newsfeed can impact our perception of the world and drive our actions. But think about it: we have very little insight into how these algorithms work or what data is used. Given that algorithms guide much of our lives, how do we know that they don't have a bias, withhold information, or have bugs with negative consequences on individuals or society? This is a problem that we aren't talking about enough, and that we have to address in the next decade.
In the past several weeks, Volkswagen's emissions crisis has raised new concerns around "cheating algorithms" and the overall need to validate the trustworthiness of companies. One of the many suggestions to solve this problem was to open-source the software around emissions and automobile safety testing (Dave Bollier's post about the dangers of proprietary software is particularly good). While open-sourcing alone will not fix software's accountability problems, it's certainly a good start.
As self-driving cars emerge, checks and balances on software quality will become even more important. Companies like Google and Tesla are the benchmarks of this next wave of automotive innovation, but all it will take is one safety incident to intensify the pressure on software versus human-driven cars. The idea of "autonomous things" has ignited a huge discussion around regulating artificially intelligent algorithms. Elon Musk went as far as stating that artificial intelligence is our biggest existential threat and donated millions to make artificial intelligence safer.
While making important algorithms available as Open Source does not guarantee security, it can only make the software more secure, not less. As Eric S. Raymond famously stated "given enough eyeballs, all bugs are shallow". When more people look at code, mistakes are corrected faster, and software gets stronger and more secure.
Automobiles aside, there is possibly a larger scale, hidden controversy brewing on the web. Proprietary algorithms and data are big revenue generators for companies like Facebook and Google, whose services are used by billions of internet users around the world. With that type of reach, there is big potential for manipulation -- whether intentional or not.
There are many examples as to why. Recently Politico reported on Google's ability to influence presidential elections. Google can build bias into the results returned by its search engine, simply by tweaking its algorithm. As a result, certain candidates can display more prominently than others in search results. Research has shown that Google can shift voting preferences by 20 percent or more (up to 80 percent in certain groups), and potentially flip the margins of voting elections worldwide. The scary part is that none of these voters know what is happening.
And, when Facebook's 2014 "emotional contagion" mood manipulation study was exposed, people were outraged at the thought of being tested at the mercy of a secret algorithm. Researchers manipulated the news feeds of 689,003 users to see if more negative-appearing news led to an increase in negative posts (it did). Although the experiment was found to comply with the terms of service of Facebook's user agreement, there was a tremendous outcry around the ethics of manipulating people's moods with an algorithm.
In theory, providing greater transparency into algorithms using an Open Source approach could avoid a crisis. However, in practice, it's not very likely this shift will happen, since these companies profit from the use of these algorithms. A middle ground might be allowing regulatory organizations to periodically check the effects of these algorithms to determine whether they're causing society harm. It's not crazy to imagine that governments will require organizations to give others access to key parts of their data and algorithms.
The explosion of software and data can either have horribly negative effects, or transformative positive effects. The key to the ethical use of algorithms is providing consumers, academics, governments and other organizations access to data and source code so they can study how and why their data is used, and why it matters. This could mean that despite the huge success and impact of Open Source and Open Data, we're still in the early days. There are few things about which I'm more convinced.
It's hard to ignore the strong force of digital distributors on the open web. In a previous post, I focused on three different scenarios for the future of the open web. In two of the three scenarios, the digital distributors are the primary way for people to discover news, giving them an extraordinary amount of control.
When a small number of intermediary distributors get between producers and consumers, history shows us we have to worry -- and brace ourselves for big industry-wide changes. A similar supply chain disruption has happened in the food industry, where distributors (supermarkets) changed the balance of power and control for producers (farmers). There are a few key lessons we can take from the food industry's history and apply them to the world of the web.
When food producers and farmers learned that selling directly to consumers had limited reach, trading posts emerged and began selling farmers' products more than their own. As early as the 14th century, these trading posts turned into general stores, and in the 18th century, supermarkets and large-scale grocery stores continued that trend in retailing. The adoption of supermarkets was customer-driven; customers benefit from being able to buy all their food and household products in a single store. Today, it is certainly hard to imagine going to a dozen different speciality stores to buy everything for your day-to-day needs.
But as a result, very few farmers sell straight to consumers, and a relatively small amount of supermarkets stand between thousands of suppliers and millions of consumers. The supermarkets have most of the power; they control how products are displayed, and which ones gain prominent placement on shelves. They have been accused of squeezing prices to farmers, forcing many out of business. Supermarkets can also sell products at lower prices than traditional corner shops, leading to smaller grocery stores closing.
In the web's case, digital distributors are the grocery stores and farmers are content producers. Just like supermarket consumers, web users are flocking to the convenience and relevance of digital distributors.
Web developers and designers devote a tremendous amount of time and attention to creating beautiful experiences on their websites. One of my favorites was the Sochi Olympics interactive stories that The New York Times created.
That type of experience is currently lost on digital distributors where everything looks the same. Much like a food distributor's products are branded on the shelves of a supermarket to stand out, publishers need clear ways to distinguish their brands on the “shelves” of various digital distribution platforms.
While standards like RSS and Atom have been extended to include more functionality (e.g. Flipboard feeds), there is still a long way to go to support rich, interactive experiences within digital distributors. As an industry, we have to develop and deploy richer standards to "transport" our brand identity and user experience from the producer to digital distributor. I suspect that when Apple unveils its Apple News Format, it will come with more advanced features, forcing others like Flipboard to follow suit.
In China, WeChat is a digital distributor of different services with more than 0.5 billion active users. Recently, WeChat blocked the use of Uber. The blocking comes shortly after Tencent, WeChat's parent company, invested in rival domestic car services provider Didi Kuaidi. The shutdown of Uber is an example of how digital distributors can use their market power to favor their own businesses and undermine competitors. In the food industry, supermarkets have realized that it is more profitable to exclude independent brands in favor of launching their own launching grocery brands.
Digital distributors have an enormous amount of power through their ability to manipulate curation algorithms. This type of control is not only bad for the content producers, but could be bad for society as a whole. A recent study found that the effects of search engine manipulation could pose a threat to democracy. In fact, Google rankings may have been a deciding factor in the 2014 elections in India, one of the largest elections in the world. To quote the study's author: "search rankings could boost the proportion of people favoring any candidate by more than 20 percent -- more than 60 percent in some demographic groups". By manipulating its search results, Google could decide the U.S. presidential election. Given that Google keeps its curation algorithms secret, we don't know if we are being manipulated or not.
Finally, a last major fear is control of monetization. Like supermarkets, digital distributors have a lot of control over pricing. Individual web publishers must maintain their high quality standards to keep consumers demanding their work. Then, there is some degree of leverage to work into the business model negotiation with digital distributors. For example, perhaps The New York Times could offer a limited-run exclusive within a distribution platform like Facebook's Instant Articles or Flipboard.
I'm also interested to see what shapes up with Apple's content blocking in iOS9. I believe that as an unexpected consequence, content blocking will place even more power and control over monetization into the hands of digital distributors, as publishers become less capable of monetizing their own sites.
Updates from Dries straight to your mailbox