Turning Drupal outside-in

There has been a lot of discussion around the future of the Drupal front end both on Drupal.org (#2645250, #2645666, #2651660, #2655556) and on my blog posts about the future of decoupled Drupal, why a standard framework in core is a good idea, and the process of evaluating frameworks. These all relate to my concept of "progressive decoupling", in which some portions of the page are handed over to client-side logic after Drupal renders the initial page (not to be confused with "full decoupling").

My blog posts have drawn a variety of reactions. Members of the Drupal community, including Lewis Nyman, Théodore Biadala and Campbell Vertesi, have written blog posts with their opinions, as well as Ed Faulkner of the Ember community. Last but not least, in response to my last blog post, Google changed Angular 2's license from Apache to MIT for better compatibility with Drupal. I read all the posts and comments with great interest and wanted to thank everyone for all the feedback; the open discussion around this is nothing short of amazing. This is exactly what I hoped for: community members from around the world brainstorming about the proposal based on their experience, because only with the combined constructive criticism will we arrive at the best solution possible.

Based on the discussion, rather than selecting a client-side JavaScript framework for progressive decoupling right now, I believe the overarching question the community wants to answer first is: How do we keep Drupal relevant and widen Drupal's adoption by improving the user experience (UX)?

Improving Drupal's user experience is a topic near and dear to my heart. Drupal's user experience challenges led to my invitation to Mark Boulton to redesign Drupal 7, the creation of the Spark initiative to improve the authoring experience for Drupal 8, and continued support for usability-related initiatives. In fact, the impetus behind progressive decoupling and adopting a client-side framework is the need to improve Drupal's user experience.

It took me a bit longer than planned, but I wanted to take the time to address some of the concerns and share more of my thoughts about improving Drupal's UX (and JavaScript frameworks).

To iterate or to disrupt?

In his post, Lewis writes that the issues facing Drupal's UX "go far deeper than code" and that many of the biggest problems found during the Drupal 8 usability study last year are not resolved with a JavaScript framework. This is true; the results of the Drupal 8 usability study show that Drupal can confuse users with its complex mental models and terminology, but it also shows how modern features like real-time previews and in-page block insertion are increasingly assumed to be available.

To date, much of our UX improvements have been based on an iterative process, meaning it converges on a more refined end state by removing problems in the current state. However, we also require disruptive thinking, which is about introducing entirely new ideas, for true innovation to happen. It's essentially removing all constraints and imagining what an ideal result would look like.

I think we need to recognize that while some of the documented usability problems coming out of the Drupal 8 usability study can be addressed by making incremental changes to Drupal's user experience (e.g. our terminology), other well-known usability problems most likely require a more disruptive approach (e.g. our complex mental model). I also believe that we must acknowledge that disruptive improvements are possibly more impactful in keeping Drupal relevant and widening Drupal's adoption.

At this point, to get ahead and lead, I believe we have to do both. We have to iterate and disrupt.

From inside-out to outside-in

Let's forget about Drupal for a second and observe the world around us. Think of all the web applications you use on a regular basis, and consider the interaction patterns you find in them. In popular applications like Slack, the user can perform any number of operations to edit preferences (such as color scheme) and modify content (such as in-place editing) without incurring a single full page refresh. Many elements of the page can be changed without the user's flow being interrupted. Another example is Trello, in which users can create new lists on the fly and then add cards to them without ever having to wait for a server response.

Contrast this with Drupal's approach, where any complex operation requires the user to have detailed prior knowledge about the system. In our current mental model, everything begins in the administration layer at the most granular level and requires an unmapped process of bottom-up assembly. A user has to make a content type, add fields, create some content, configure a view mode, build a view, and possibly make the view the front page. If each individual step is already this involved, consider how much more difficult it becomes to traverse them in the right order to finally see an end result. While very powerful, the problem is that Drupal's current model is "inside-out". This is why it would be disruptive to move Drupal towards an "outside-in" mental model. In this model, I should be able to start entering content, click anything on the page, seamlessly edit any aspect of its configuration in-place, and see the change take effect immediately.

Drupal 8's in-place editing feature is actually a good start at this; it enables the user to edit what they see without an interrupted workflow, with faster previews and without needing to find what thing it is before they can start editing.

Making it real with content modeling

Eight years ago in 2007, I wrote about a database product called DabbleDB. I shared my belief that it was important to move CCK and Views into Drupal's core and learn from DabbleDB's integrated approach. DabbleDB was acquired by Twitter in 2010 but you can still find an eight-year-old demo video on YouTube. While the focus of DabbleDB is different, and the UX is obsolete, there is still a lot we can learn from it today: (1) it shows a more integrated experience between content creation, content modeling, and creating views of content, (2) it takes more of an outside-in approach, (3) it uses a lot less intimidating terminology while offering very powerful capabilities, and (4) it uses a lot of in-place editing. At a minimum, DabbleDB could give us some inspiration for what a better, integrated content modeling experience could look like, with the caveat that the UX should be as effortless as possible to match modern standards.

Other new data modeling approaches with compelling user experiences have recently entered the landscape. These include back end-as-a-service (BEaaS) solutions such as Backand, which provides a visually clean drag-and-drop interface for data modeling and helpful features for JavaScript application developers. Our use cases are not quite the same, but Drupal would benefit immensely from a holistic experience for content modeling and content views that incorporates both the rich feature set of DabbleDB and the intuitive UX of Backand.

This sort of vision was not possible in 2007 when CCK was a contributed module for Drupal 6. It still wasn't possible in Drupal 7 when Views existed as a separate contributed module. But now that both CCK and Views are in Drupal 8 core, we can finally start to think about how we can more deeply integrate the two. This kind of integration would be nontrivial but could dramatically simplify Drupal's UX. This should be really exciting because so many people are attracted to Drupal exactly because of features like CCK and Views. Taking an integrated approach like DabbleDB, paired with a seamless and easy-to-use experience like Slack, Trello and Backand, is exactly the kind of disruptive thinking we should do.

What most of the examples above have in common are in-place editing, immediate previews, no page refreshes, and non-blocking workflows. The implications on our form and render systems of providing configuration changes directly on the rendered page are significant. To achieve this requires us to have robust state management and rendering on the client side as well as the server side. In my vision, Twig will provide structure for the overall page and non-interactive portions, but more JavaScript will more than likely be necessary for certain parts of the page in order to achieve the UX that all users of the web have come to expect.

We shouldn't limit ourselves to this one example, as there are a multitude of Drupal interfaces that could all benefit from both big and small changes. We all want to improve Drupal's user experience — and we have to. To do so, we have to constantly iterate and disrupt. I hope we can all collaborate on figuring out what that looks like.

Special thanks to Preston So and Kevin O'Leary for contributions to this blog post and to Wim Leers for feedback.

Comments

chx (not verified):

First, let's double down on "Drupal's User Experience problems go far deeper than code" -- improving our processes. We can't "disrupt" with the current glacial pace.

February 10, 2016
Dries:

Improving our velocity is another topic near and dear to my heart. While the topic for another blog post, I agree it should be an area of focus.

February 10, 2016
Shannon (not verified):

Velocity is dependent on so much. People, prioritization, coordination/communication, integration process... all of these things are blockers to increase our release speed & efficiency. What I saw being most painful for D8 was a lack of man/woman-power due to the headache-inducing-difficulty involved in learning how core works.

I wonder if this has been improved with D8?

February 15, 2016
nod_ (not verified):

Thanks, it makes things clearer knowing what you have in mind.

February 10, 2016
davidhernandez (not verified):

What I get from this is more user approachable than the other blog posts. The experience, from the content editor's perspective, should feel more fluid. People want to be able to add a new page and start adding content directly with in-place editing. But, it would be great to also manage the form, and possibly function, of that content as well.

I'm imagining clicking the menu item to add a new page, and while adding content you have the ability to move the fields around and add new ones. Drupal will, incognito, add a new content type while doing this, but the user doesn't care. They'll just "save this as a new page type," which they can reuse later when they make more pages. Move a field into what is technically another region? The user doesn't care, just find a way to do it.

Some of these actions have been really difficult to conceptualize, because of the separation we have between frontend and admin themes. So in order to do it we may need to do away with the interaction with the admin theme where content is concerned, or maybe remodel how some of those forms/settings work.

February 10, 2016
Dries:

Exactly! I agree that some of these concepts can be hard to conceptualize, especially in writing. I think we should try to conceptualize them in wireframes or design mockups as that makes it more concrete. Content creation and content modeling is a difficult place to start; we might want to focus on easier parts of Drupal to introduce more of the concepts and patterns (like we did with in-place editing).

February 10, 2016
Bill Winett (not verified):

If a new content type is created each time a new page is created this way, you will soon have thousands of content types.

I'd rather see something like I see in our Confluence wiki. When I click on the button to create a new page, it displays a popup with a list of content types I can use as the basis for the new page. Here's a screenshot of what I'm talking about:

https://danahertm.box.com/s/0dvp0eaz2vxzwk1nxam3k3ji8gvbidlf

By doing this, the site developers can provide control over the set of content types available, and provide flexibility to authors.

WRT people being able to move fields around, I like the capability that the Arrange Fields module provides (https://www.drupal.org/project/arrange_fields). Of course it can be greatly improved, but hopefully you get my drift.

February 18, 2016
vijaycs85 (not verified):

Thanks for the great post Dries. We can integrate config schema + views to generate almost all current Drupal 8 admin config entity CURD pages and configuration forms. I guess, it's kinda related to what you have mentioned above.

February 10, 2016
Dries:

It is partially what I'm talking about, but I also want to make it easy to use for both site builders and developers.

February 10, 2016
Michael Keara (not verified):

Dries, I think you hit the nail on the head here. I’d like to help drive the point home by following through from a UX Dev perspective.

First of all ‘outside-in’ is an extremely important concept. In my own UX and developer activity I’ve been pushing that concept since 2000 in desktop and web contexts.

Also JavaScript is important as it prevents web user experiences from being a cure for insomnia. Personally I switched from desktop development to web ONLY after I encountered AJAX – simply because the web suddenly became usable.

Having said that, I believe if we are serious about engaging in disruptive thinking, we also need to recognize the tremendous impact of ‘The Screen’ as both a focus and a limit.

What does this mean? I believe that most non-technical people (users and website owners) tend to equate a web site with The Screen. That is the part that is most real to them and understandably so. Client side JavaScript can help make the screen experiences delicious and satisfying.

But Javascript is only the ‘oil’ of the UX and not the engine. The ‘engine’ must come from a solid UX strategy which is much deeper and takes into account many, many more things than just the behaviour of screenware.

One UX strategy concept I have been sharing with my peers in the Drupal community is that of the Inductive Pull Model. This is the UX equivalent of dependency injection. The client (web site owner) states their needs in their naturally screen-oriented manner and the *system* provides a means of delivering it to them in a manner that matches their own evolving thought processes.

Technically the Inductive Pull Model is based on the concept of articulate but empty placeholders that form the basis of requests to ... not the web server but the *sitebuilder*. We must recognize the important difference in meaning of the word 'request' in the context of *site production* versus *site consumption*. This corresponds to 2 entirely different usage contexts and dialogs – the site owner communicating with the site builder (e.g. in planning and review sessions) versus the end user communicating with the site owner (i.e. interaction with the web site itself).

For more details on why this is important and how it could work I invite you to view this video of a presentation I made at DrupalCamp Toronto back in 2014:
https://www.youtube.com/watch?v=9WM7k8AXgYc

The aim of that presentation (called ‘Intentional Sitebuilding and the Evolution of Drupal’) was to expand on a concept that I shared with the community in 2012 regarding the new meaning of web ‘pages’:

‘A page is a unit of business intent’.

That phrase has been quoted many times by some of the leaders in the Drupal community and I’m happy if it helped clarify certain strategies. But it is only scratching the surface. There is so much more constructive disruption we can do.

Michael Keara
@useradvocate

February 10, 2016
Dries:

Thanks for the thoughtful comment, Michael. I'll make sure to check out your DrupalCamp Toronto presentation.

If you have design ideas/prototypes for how to move certain parts of Drupal towards an outside-in model, I'd love to see them.

February 10, 2016
Michael Keara (not verified):

Unfortunately I haven’t had time to build a working prototype but I did include a simulated UI at the end of the Intentional Sitebuilding presentation (around the 20:00 mark in the video).

I do have another video from 2012 that actually starts with a reference to Spark and then explores the concept of ‘outside-in’ through a simulated sitebuilder UX. This was an ‘envisioning exercise’ that explores an outside-in conceptualization of site building through a sort of narrative approach - so let your mind go and enjoy :)
https://www.youtube.com/watch?v=T2U_AyM5ePM

And finally Dries on the topic of disruptive innovation, this talk (also from 2012) begins with a reference you made to ongoing innovation as a requirement for Drupal’s growth and continuity. It explores what I think that means in terms of problem solving techniques and UX.
https://www.youtube.com/watch?v=CgTpN9fUqhM

I hope you enjoy these videos and I look forward to more discussions of this topic. I think the time is right for it.

Michael

February 10, 2016
philsward (not verified):

My only request for this pursuit is to make sure from the beginning, that it is clear to the user that "something happened". A full page load clearly shows the user that something happened. With the addition of JS libraries that load incremental pieces of the page, it isn't always clear that something changed or updated. Point is, if this approach is taken, I believe Drupal will need to define and outline a strict API that can be utilized for developers to show that a block dynamically updated, or an edit was successfully saved. I've encountered this with JS calculators on websites where hitting submit, successfully runs the calculation, but there's nothing there to tell you it ran or did something. The user is left sitting there clicking calculate 3-4 times before their eyes train to the right ouput field and supidly realize "oh, there it is". Solving one UX problem may create another. Food for thought.

Another thought on this topic is that while it will be great to have this, I personally believe Drupal should solve the problem of mobile first, then work on integrating a JS framework into it. Will it be easier to deal with mobile before or after the JS framework is implemented? Drupal currently hands off layout and display to the theme to handle various viewports. It's fine for themes that are built around this mindset, but there is a lack of flexibility for people to come in and build custom sites, without reverse engineering things. Maybe a JS framework will make mobile first easier, I dunno, but I believe they should be built together with each respective outcome in mind. If anything, once this topic is put to bed, I think the next project to tackle should be "handling viewports". Prime example, I'm writing this comment and while the page is formatted beautifully for mobile, I can't make the text area box larger...

Finally, what are the implications of SEO? I realize this topic is more for admin related UX design, but if the API is built for the backend, I can only assume it will traverse to the frontend as well. Google does deal with JS generated content, but from what I've read, it isn't fully there yet. Something to think about.

Overall, I'm excited to see where this discussion goes. I think that a JS framework within Drupal will open the doors for some pretty amazing things from the community.

February 10, 2016
Josh Mitchell (not verified):

Dries, thank you for taking the conversation this direction. It is much needed.

I've been having some pretty frank conversations with the part of the Drupal.org team that is spending a bit of sponsor-funded time on Panels, Panelizer, and IPE for Drupal 8. Even the abstraction of concepts like "panels" (or "contexts" if you prefer) on top of "content types" and "displays" is problematic. These all have meaning to the well-versed Drupal site builder, but even our most accomplished developers are challenged by the ability to choose between when to use a node reference field versus a taxonomy reference field versus a custom page to determine layout. For nearly everyone reading this post, those words trigger meaning and a mental model of "how Drupal works" with very strong biases.

The mental models we have around content modeling and "page" layout can be quite a blocker to good design and usability. We've created abstractions on top of abstractions to the point that a simple query to the database requires a ton of inside knowledge of "how Drupal works".

As we all figure more of this out, let's try and get prototypes and demonstrations on Drupal.org issues and linked from personal blogs so we can "show" rather than "tell" in order to get faster buy in. (If a picture is worth 1,000 words, a prototype is worth 10,000 words.)

I'm excited to see what people come up with that helps to simplify the visual clutter of designing layouts with complex relationships.

Oh... and speaking of demonstrations, I loved that Dabble demo. Such a cool concept for dealing with data models. I had forgotten all about that one.

February 10, 2016
Jeff Eaton (not verified):

There's a lot of meat to this blog post, and it's great to see the topic being tackled. Drupal's modular, contrib-driven approach has resulted in a pile of amazing tools hampered by inconsistent (and often conflicting) terminology and mental models. While we could realize some easy wins by shipping Drupal with "prefab" install profiles tailored to certain types of sites, that doesn't address the underlying challenge you're talking about.

One of the issues touched on during the UX Research that deserves special attention is the distinction between site architecture, content management, and visual layout/design. The interplay between those types of tasks (and the people who are performing them) varies dramatically from project to project. Some sites (all of the sites I work on, in fact) would benefit greatly from clearer separation between site-building and day to day creation of content. On others, particularly smaller sites run by one or two people, highly variable content makes engaging with themes and blocks a regular part of the editing and management process. On still other sites, distinctions between content *managers* and content *creators* expose big gaps in the tools Drupal provides out-of-box.

Often, there are easy solutions to one scenario's UX problems that would unfortunately complicate the others. The tug-of-war has meant that most core UX improvements have focused on the cosmetic, where common ground is easier to find. At its heart, these are not simply UX problems — they're product design problems. The question of Who Drupal Is For has been debated for years, with no consensus emerging.

Any UX undertaking worth the name needs that question — whose problems are we solving? — answered clearly and unambiguously. Answering it honestly may alienate those who fall outside the target audience, but avoiding the question ensures our community's limited UX resources will always be stretched too thin.

February 10, 2016
John Kennedy (not verified):

I had a very interesting discussion about this issue with Mike Potter today. Lightning will be more opinionated about the separation of authoring, site building activities and design layout. Would be great to hear your extended thoughts on this at some point.

February 16, 2016
Bojhan Somers (not verified):

Thanks Dries, its good to see the discussion cycle back to this again. We have talked on and off about this for over 7 years. I could write an essay about this, this is me trying to keep it short :)

This is from my experience in the past 7 years as UX Maintainer, the key challenges are:

1. Process optimised for iteration
2. Lack of competitive disruption
3. Lack of economical support or incentive

1. Process optimised for iteration

We are largely unable to achieve disruptive UX change. Many of our users, and this extends to core developers - are heavily invested in our current UX approaches. This leads to much aversion, as they feel its conceptually the right outing (this makes disruption hard, iteration easy).

Most of the organisations I deal with are only able to make disruptive change by programs that are highly funded, top-down, and politically controversial. D7UX had this ambition, funding and did make significant change.

On the surface, it might seem like a lack of pretty mock-ups/ideas or UX process. But chx is right in stating, that we need to double-down on the process problem and thats not perse pace - but its the almost impossible translation of design ideas into core code.

It is unlikely we can radically change our quality process. However we can invent new ways of getting design ideas in core, that is not subject to the same rigour as it serves a different purpose.

Eaton is right, that we never really chose an audience - and that missing fundaments introduces many inefficiencies in our discussions (this makes on-boarding UX people really hard). However even with a choice, fundamentally we have a inherent disconnect with the needs and behaviours our intermediate users (the largest part of your audience bell curve, typically). Organisations have this all the time, basing product decisions on their most vocal customers (e.g. people who reach the issue queue, in Drupal's case) rather than their largest audience

IDEA 1: Lets use the "Experimental" package, as a way to introduce very disruptive but not yet mature UX concepts. Following a Lean UX approach; iterating on an idea, and possibly bringing into core or kicking it out determined by its quality. We just need a concept to kickstart it, Field UI being the ideal candidate.

IDEA 2: Lets make our product strategy discussions more visible, and invite competencies in product strategy and UX to take a key role in helping shape the direction. We currently don't have a very open visible community process, on these discussions other than these blog posts and it feels (likely not is) very closely tied to solely the core committers.

2. Lack of competitive UX awareness

We are blessed to be in a highly competitive market, and continue to have a significant market share without a competitive UX. Vendors try to resolve this with distributions, but they lack the cohesion thats needed to disrupt the market.

With the rise of front-end only UX, irrespective of the back-end - we can expect more and more competition on the UX level. With users expecting more consumer level UX, and the market reaching a plateau where UX is a main differentiator - the effect on sales will be immense.

Companies that are ahead of the curve, in size like Acquia and Wunderkraut are likely noticing this most. It is my hope, that the expectations of what good UX is like and what it should be like to remain competitive will grow across the board.

IDEA 3: Usability tests only go so far. We need to do more open competitive benchmarking that creates the awareness what our measurable values and weaknesses are. This is difficult for commercial entities to publish on, but it is part of a mature product strategy.

3. Lack of economical support or incentive

To impact UX, you need UX people. Just the 6 months of funded work by Leisa Reichelt and Mark Boulton in 2009, was immense.

With just market dynamics, we seem to be unable to repeat this type of funding for UX. The designers I have on-boarded often stepped out, simply because its economically unviable - there is no career path.

You need a very special breed of UX people who thrive in highly dynamic, very complex and combative environment like Drupal core. Working myself at a 25 person company, with only UX oriented designers, researchers and strategists - these people are very hard to come by.

At the time of writing this only 1 UX person (Kevin O'Leary) is dedicated to core (and from time to time even not). As opposed to 10+ paid developers, and the ratio of UX designers to developers in the core community is close to 1:800.

IDEA 4: We've found ways to better fund our cat herders, core committers, core developers. Let's ensure that we create awareness, that in key competitive competences like UX we have none of this. You need companies to be bold, and step in this gap - even without their contribution in that tag-cloud :)

February 10, 2016
Dries:

Bojhan, lots of great points and ideas in your comment. Thank you!

I'd love to see us talk more about our competition, process improvements, the concept of experimental modules, target audience, financial support, or incentive programs to get more UX talent involved, etc. Believe it or not, but I'm passionate about all these topics and have written or talked about most of those in the past in the past.

Rather than me trying to reply to all your ideas in the comments of this blog post, let me follow up with individual blog posts. Now that Drupal 8 is released, the time is right to amp up these conversations.

If you, or others, have other important questions that you want me to address, let me know and I'll do my best to do so. If you don't want to miss the follow-up blog posts, you can opt to receive them by e-mail.

February 11, 2016
Roy (not verified):

I made a list of topics to address for increasing velocity and quality of the process: http://yoroy.com/pieces/drupal-ux-process-improvements-shopping-list. Would be good to hear your and others thoughts on. Thanks!

February 20, 2016
Roy (not verified):

This touches on a lot, fodder for multiple blog posts indeed, but some thoughts:

For the process part, there is a first bit of discussion about that here: https://www.drupal.org/node/2573119 It seems to me that we have to get better at:

  • Exploring and prototyping *multiple* solutions to a given problem. 10 ideas -> 3 prototypes -> 1 solution.
  • Timeboxing different phases in the design process. Whether the steps are called Build Measure Learn or Discover, Define, Design, Develop is less important then getting very explicit about knowing which type of phase we’re in and when it ends and the next one begins.
  • Choosing a direction. At this point, for some of the long standing critical UX issues any change should be preferred over stalling on getting things exactly right.
  • “If you want to go fast, go alone, if you want to go far, go together” is often dropped to stress the benefits of large scale collaboration. We need to find out how we can benefit from small teams working through problems faster in a way that does not smother that work in a consensus driven implementation phase.

If devops is about continuous deployment etc, we have to find a similar mode for design work. Designops. Experimental modules is definitely a part of that. Not having to rely on big usability lab testing alone is another. It will be interesting to balance “always be shippable” with the need to make disruptive changes. One thing I’m wondering about:

Where in the stack does the translation from Drupal technological model to the users mental model happen? We have a lot of underlying complexity to absorb to turn it into flows and interfaces that map to user expectations. We’ll have to get a lot more opinionated in how we connect, sequence and prioritise different parts of the UI. This *will* be at odds with the Drupal as a general framework concept. Of course we don’t have to choose between user optimised and clean generic framework, but then we have to get specific about where this translation between the two happens: Experimental modules? A very opinionated core install profile? The frontend data model?

Process and culture changes will help but not fix the resource problem. If UX is the main differentiator, then significant investments are needed to enable designers to work on this. Basically, what Bojhan said. :)

I agree that Fields and Views are a great place to start exploring these new UX paradigms. They are both core Drupal strengths and will stay relevant for the next couple of years at least. They both need to be made less explicit, more blended into the way people work to achieve things, much like davidhernandez describes in his comment above.

February 11, 2016
tic2000 (not verified):

To add to what @chx said, look at this issue https://www.drupal.org/node/77245. It was created on August 5, 2006. jQuery first stable release was on August 26th of same year. "Only" 6 months and that issue asking for a way to "indicate the status of an (ajax) action properly" will be 10 years old. And the solution in the last patch is not even addressing the issue properly in real world scenarios.

UX people will be driven away if they try to help and will be told that the CSS fix they provided is an "API change" and they are forced to find a way to fix a BUG by not making an "API change".

What is even worse is that API straight additions are also in the same boat of API changes and not considered until the next release. I will point to 1 issue I opened because I know of it, but examples like that are a lot. https://www.drupal.org/node/2660272. This would fall in the same category https://www.drupal.org/node/2660292

By the time you get some improvements into Drupal, standards change and you have to do it again. This way of doing things doesn't really work for UX.

Inspired by this comment by @effulgentsia (https://www.drupal.org/node/1515688#comment-10770008) I decided to write the https://www.drupal.org/project/field_ui_ajax module (also a way to familiarize myself with D8). Working on that I found that 10 years old issue because my goal was to write the module using the least custom code as possible and take full advantage of what Drupal provides.

But in the process I found myself duplicating the whole Drupal.behaviors.Ajax.attach just because I wanted to add a namespace to a click event and implementing my own "notification" because using Drupal.dialog was to disruptive and a message added somewhere on the page was not UX friendly enough since that may not even be visible from where the action took place.

This are my issues and I'm not even a UX guy and fortunately I can find my way around Drupal code.

What @Bojhan said is also very true, and I will add that it takes a special kind of people to do what Drupal needs even if payed. Either the "this is just a job, I don't care if it takes 2 years to have something committed that should take few weeks at most" or the very dedicated people. I don't know if you want the 1st and at some point you will lose the 2nd if they can find a place where they feel their work provides more personal satisfaction.

February 11, 2016
catch (not verified):

@tic2000

Note that what you're referring to as 'the next release' will be in beta at the end of this month and released on April 20th, and we've been committing patches to it since the beginning of December. While the criteria for patches that can get into monthly patch releases is quite strict, and can be annoying in many cases, it does mean more license to change things in the 6-monthly minor releases than we have/had with 7.x.

February 12, 2016
tic2000 (not verified):

Yes, I'm aware of this. At least for Drupal 8 the time is reduced from years to months.

February 13, 2016
Bojan (not verified):

A key factor in enlarging the install base are newbies' comfort levels upon first install. A first time WordPress user, after following instructions on a website, can see a useful dashboard with things to do. Drupal doesn't have this (though I haven't tried 8). It would be great if new Drupal installs came with some "demo projects" that the first-time user could simply pick from a menu (as in 1. Blog, 2. Shop, 3. Static Site)...etc.

February 11, 2016
Shannon (not verified):

YES YES YES! I've been thinking this for YEARS!!! Though I'd go even farther and not call them demos, but project types. Why can't Drupal be usable for dummies like me out of the box?? ;) Why must we assume that everyone who wants to use Drupal wants to build it from scratch?? There is a market for both, and we should be catering to both.

WP has shown effectively through its mass adoption that having a base use case serves many purposes and increases its popularity - and thus, lifespan/advancement. I suggested this wrt Drupal Commerce, (eg: selection of payment solutions pre-configured, ability to apply different themes, multiple useful modules pre-configured like comments & reviews), but saw no traction because the "people want to build it for their own needs, we shouldn't assume what they want to do" opinion prevailed. I disagree. I think developers can still edit or turn off things that are set up in a few easy clicks, and if we are smart, we would preconfigure things in such a way that reduces their dev-to-launch sprint time.

I see no reason why we cannot focus groups of users around usecases, especially the ones you are citing here Bojan, which make perfect sense to me. It would solve many of the ux and usability issues we face by allowing us to make assumptions about our audience & what they want to do, and getting them to those goals faster & less painfully.*

Conversely, nothing is stopping us from having the good 'ole drupal "vanilla" install with nothing pre-built for them! This would make everybody happy imho.

Another comment on this, there's no reason why there couldn't be more than one version of a "blog". Distributions allow us this flexibility, and it is very useful. It could open an entire marketplace of valuable blog, shop, and other distribution variants and associated themes for Drupal site builders to choose from.

*I'm no fool disclaimer: I do realize that this could cause complications and increased complexity for core, which is increasing dev experience "pain". That needs to be managed. If we have a core that cannot easily serve distributions with different purposes, then our core is not disruption-ready and we should, in the words of Cap'n Picard, make it so.

February 15, 2016
Randall Knutson (not verified):

This is the exact same question that me and a few other long time Drupal devs have been contemplating for the last year.

What Drupal does really well is make the process of creating a form and the data model behind content really easy to develop. It took away the complexities of form states, database schemas and validations. Site developers could simply build a content type and start entering data. That was all handled for them.

In the new world of front end javascript frameworks we lose a lot of those niceties and are suddenly back in the world of manually building front end forms that map to Rest APIs. This is very much like the CMS days before Drupal made this easy.

We've been working the last year on a solution to exactly this problem. What we've come up with is a Drupal like approach to front end forms.

Here's how it works:

  1. Build a content type (form in our lingo). This content type is represented by a json schema.
  2. The schema is rendered with angular/react/ember/etc on the front end to represent the content type.
  3. The schema is also used on the backend to define the Rest API, including validation.
  4. The front end form is connected to the back end API automatically.

By doing it this way, we have the power of a configurable CMS without requiring the rendering to be done on the server. Any changes to the json schema will automatically change both the Rest API and the front end Form to match. This is very similar to how Content Types currently work.

You can check out our solution at https://form.io. We have open source our backend server, form builder and angular and react renderers at https://github.com/formio.

It wouldn't be to difficult to implement a similar solution in Drupal. I believe there is already a Rest API for each content type so all that would be needed would be to get the form definition for each content type, render it on the front end and connect it to the Rest API. As the content type changes, the Rest API changes and the form definition that is sent to the front end it changed as well.

February 11, 2016
philsward (not verified):

How do we keep Drupal relevant and widen Drupal's adoption by improving the user experience (UX)?

The question of the day... I think Dries, you are taking the right approach by asking "What do we need to do different", instead of "What can we use to make things different?"

I think the first step, is to dive down to the lowest micro levels of "User Experience" and make a list of all of the things that could / need to be improved on. A lot of folks have brought up the comment about "Revamp the Field Manager!". While yes, this would be nice, we're focusing on a single part of the Drupal admin interface and haven't even targeted the entire backend admin interface, let alone the front end user experience. We shouldn't be focusing on the Macro level, when the little things need just as much or more attention.

I think the approach we need to take on this is to first determine WHAT needs attention first at a micro level and break each item out into their own categories underneath a global UX initiative. Next, do the same thing from a macro level. Once we determine what needs attention, we can then begin determine how to combat those issues. The entire system really, needs overhauled in this regard as opposed to fixing this or that because they are the squeakiest wheels, for example in-place editing...

The initial micro categories I think we need to focus on are:

- Something Needs To Happen
- Something Is Going To Happen
- Something Is Happening
- Something Did Happen
- (Insert additional category here)

So for example, here are a few micro level things I would love to see addressed:

- Messages: System, Warning, Error, Descriptions, etc. Drupal does a really poor job with presenting information that matters at the lowest level. As an example, take a look at the "Friendly Register" module. If a username exists, the user is presented with a JS generated error message. Think about required form fields on the site for a minute. If it's required, and the requirement isn't met, you don't know it until the page is saved and you're presented with a generic message that some sort of field needs attention. Then you have to go hunt for it. Why not gently let the user know ahead of time, that the field is required while they are filling everything else out? It could be as simple as a red border around the field item that goes away when something is selected or the user types into the text box. This would probably fall under "Something needs to happen" and "Something Did Happen".

- Loading Indicators: Currently, the only indicator (at least for 7) we have to show that "Something is happening", is the small circular loading icon. (haven't played with 8 enough yet) Build out an entire api ecosystem for easily adding or changing the part of Drupal that lets people know "Something is happening". The more Drupal relies on JS, the more important this will become.

- Overlays: I know the overlay of D7 went over about as well as a fart in an astronaut suit, but I have seen other sites that implement overlays beautifully. I know it can be done in Drupal with the right JS framework, but it couldn't be done well in Drupal 7 because the overlay mechanism required entirely too much information to perform well. As much as I hate Intuit's products as a whole, they have done a really good job with some overlays on their web system. I'm not saying we need overlays, but they can be used as really good signals that "Something is happening" and "Something did happen" if they are executed well.

- Better signals that "Something is going to happen". Drupal does a pretty decent job of this already, but I think it could be improved on. No idea how exactly.

Once we figure out WHAT needs changed at the micro level, we can then proceed to look at it from the macro level. Fixing things like content creation, field management, layout, etc.

I think my biggest problem so far, is the issue has been presented: "We need to fix the UX of Drupal" but I haven't seen a clear set of guidelines presented for the community to focus on for feedback. We have an end goal, but we don't have a strategy yet.

"Goals without a strategy are like ideas without action. They're both fun to think about, but never get anything done."

I realize right now the strategy stage is forming which is why the questions that have cropped up, are the ones that have cropped up. But, what I would like to see is a singular place (preferably not on d.o) where an outline on this issue can be created and each topic has discussion, direction and resolution to move to the next topic in the outline. Without this, we're throwing mud at the wall, hoping something sticks. Take it back to the basics and start by asking "What needs changed?". Once that list is collected, prioritize it and focus on the #1 item. I'd agree that UX is probably #1 at this point. If so, move to the next question and ask: "Why does the UX need an overhaul? Speed issues? Usability issues? Keeping up with the Joneses?" Assuming we need the overhaul due to speed and usability, the next question could be: "What needs a UX overhaul on a micro level? What needs a UX overhaul on a macro level?" And the questions continue...

I think once a lot of the basic questions are answered, it will begin to pave the way for the HOW to fall in place which is determining which JS framework (if any) is used and how deeply integrated it becomes. I would love to see this topic and future topics advertised with authority. Maybe create a new domain and use it to house: "The future of Drupal". drupalup.org is available... ;-) A place where people know the topics are transparent of where Drupal is going, open for serious discussion, constructed with clarity and authored by the leader(s) of Drupal. Get out on social media and post when things update or decisions are made that need the community to discuss it. If it weren't for Chris Weber on the Drupal Community on Google+, I wouldn't even know about these discussions.

I'll close by saying: "We all know the UX needs love." Is this the single most important thing that needs addressed at this time in order to move forward? If so, let's tackle it from the beginning.

February 12, 2016
Mark Tinsley (not verified):

I haven't been following these developments very closely but am very interested in seeing how they turn out.

Has Pjax come up at all? Possibly something Drupal could find value in.

https://github.com/defunkt/jquery-pjax

February 13, 2016
Wim Leers (not verified):

Yes, PJAX is on the radar. I'm working on it. Follow this issue to see progress: https://www.drupal.org/node/2603794.

February 15, 2016
philsward (not verified):

In addition, UX needs to be grouped and categorized by audience.

A few possibilities:
Global
Dev
Admin
Content Manager
Content Creator
Authenticated User
Anonymous User
(Others?)

Forms for example, could fall under Global.

Token Browsing, could fall under a variety of audiences.

Site messages could be designed one way for backend and another for frontend.

So, tackle UX from an audience standpoint to figure out who needs what UX improvements, then break each audience out into a micro and macro level to address both the little things and big things for each respective audience.

February 13, 2016
Oren (not verified):

Another inspiration similar to DabbleDB, but under active development:

https://www.silk.co/

February 15, 2016
Christopher Estes (not verified):

I'm both a learner and "Past Customer" of Drupal. I've been in an organization's IT department when we selected Drupal for both internal and external sites. Our communications department had people with lots of experience in CMS-based websites, but they struggled with Drupal (7). Looking at technologies to learn I've encountered fewer problems with Wordpress (and of course fewer options), but I see why it's so popular. Drupal is far better than something like DotNetNuke, but still has a high learning curve for users. Drupal enjoyed a surge when the US government began adopting it, but a lot of agencies are clamoring for something different. Without a quick improvement in how Drupal can be used, it will begin to decline.

February 18, 2016
GreggShort (not verified):

I think that the save/post pages interface needs some work as more javascript processing is added. The Google docs, save all the time model would seem to be best in class, but you need a revision/undo feature, as well as a clear make live (post) option. How we display a wysiwyg page during editing is also a concern...could a multiple browser window approach be used here? Where the page being edited is displayed in another window, near realtime, as a particular user?

February 18, 2016
Charles Belov (not verified):

The biggest concern I have about pushing things to the Client side, as webmaster for a government website is that many of our customers/constituents are not computer savvy and some are low-income. They can't necessarily upgrade their old, slow computer to meet the latest demands websites are putting on their CPUs. Progressive enhancement is not necessarily the answer, as a browser might say it's capable while the computer says it's choking on the browser's demands.

February 20, 2016
John Graffio (not verified):

I'm a programmer and web developer. I built a .NET web based CMS in 2007 that never took off, tried some other CMS's and finally started working with WordPress two years ago. Getting a WP site up and functional takes about 10 minutes. The caveat is extending WP to be useful for more than two content types, and without in depth knowledge of WP internals or picking the right plugin to extend WP, you are stuck with a blogging platform.

I tried to work with Drupal 7 last year, and I felt like I needed a Ph.D in physics to make it work, so I gave up. I just started using Drupal 8 this week. I needed to watch the videos on www.ostraining.com to get a handle on content creation, and even having done so, I got stuck for two days trying to do it on my own as I retraced the steps in the videos. That learning process was extremely frustrating.

Now that I'm slightly familiar with Drupal 8, I can see that it is light years ahead of WordPress. The process of building content is upside down and backwards compared to WordPress, but once past that learning curve there is no comparison on functionality and speed. And that causes another frustration.

If Drupal's workflow was like WordPress, Drupal's market share would quickly jump ahead of WordPress. There are a lot of frustrated WordPress developers out there, mucking around, trying to make WordPress to do what Drupal does out of the box. In fact, I have a radical suggestion: create an alternate admin them that mimics WordPress. Make it default out of the box. If you can mimic WordPress, 99% of the barriers to Drupal will disappear overnight; web designers and web developers will see how really powerful it is when they realize they can create any kind of content type, blocks and views with only a few mouse clicks.

March 3, 2016
Casey Wight (not verified):

We've been working internally on a project (with the intention of sharing with the community) to help accomplish just that. It's still in its infancy, but we're excited to provide an improved authoring experience: https://www.chapterthree.com/blog/better-way-administrate-drupal-part-1

March 9, 2016
Dries:

For those who enjoyed this blog post, have a look at the follow-up post where I share more concreted examples of how to make Drupal outside-in: http://buytaert.net/examples-of-how-to-make-drupal-outside-in.

April 4, 2016

Pages

Add new comment

Updates from Dries straight to your mailbox