Handling context in "outside-in"

In a recent post we talked about how introducing outside-in experiences could improve the Drupal site-building experience by letting you immediately edit simple configuration without leaving the page. In a follow-up blog post, we provided concrete examples of how we can apply outside-in to Drupal.

The feedback was overwhelmingly positive. However, there were also some really important questions raised. The most common concern was the idea that the mockups ignored "context".

When we showed how to place a block "outside-in", we placed it on a single page. However, in Drupal a block can also be made visible for specific pages, types, roles, languages, or any number of other contexts. The flexibility this provides is one place where Drupal shines.

Why context matters

For the sake of simplicity and focus we intentionally did not address how to handle context in outside-in in the last post. However, incorporating context into "outside-in" thinking is fundamentally important for at least two reasons:

  1. Managing context is essential to site building. Site builders commonly want to place a block or menu item that will be visible on not just one but several pages or to not all but some users. A key principle of outside-in is previewing as you edit. The challenge is that you want to preview what site visitors will see, not what you see as a site builder or site administrator.
  2. Managing context is a big usability problem on its own. Even without outside-in patterns, making context simple and usable is an unsolved problem. Modules like Context and Panels have added lots of useful functionality, but all of it happens away from the rendered page.

The ingredients: user groups and page groups

To begin to incorporate context into outside-in, Kevin Oleary, with input from yoroy, Bojhan, Angie Byron, Gábor Hojtsy and others, has iterated on the block placement examples that we presented in the last post, to incorporate some ideas for how we can make context outside-in. We're excited to share our ideas and we'd love your feedback so we can keep iterating.

To solve the problem, we recommend introducing 3 new concepts:

  1. Page groups: re-usable collections of URLs, wildcards, content types, etc.
  2. User groups: reusable collections of roles, user languages, or other user attributes.
  3. Impersonation: the ability to view the page as a user group.

Page groups

Most sites have some concept of a "section" or "type" of page that may or may not equate to a content type. A commerce store for example may have a "kids" section with several product types that share navigation or other blocks. Page groups adapts to this by creating reusable "bundles" of content consisting either of a certain type (e.g. all research reports), or of manually curated lists of pages (e.g. a group that includes /home, /contact us, and /about us), or a combination of the two (similar to Context module but context never provided an in-place UI).

User groups

User groups would combine multiple user contexts like role, language, location, etc. Example user groups could be "Authenticated users logged in from the United States", or "Anonymous users that signed up to our newsletter". The goal is to combine the massive number of potential contexts into understandable "bundles" that can be used for context and impersonation.

Impersonation

As mentioned earlier, a challenge is that you want to preview what site visitors will see, not what you see as a site builder or site administrator. Impersonation allows site builders to switch between different user groups. Switching between different user groups allow a page to be previewed as that type of user.

Using page groups, user groups and impersonation

Let's take a look at how we use these 3 ingredients in an example. For the purpose of this blog post, we want to focus on two use cases:

  1. I'm a site builder working on a life sciences journal with a paywall and I want to place a block called "Download report" next to all entities of type "Research summary" (content type), but only to users with the role "Subscriber" (user role).
  2. I want to place a block called "Access reports" on the main page, the "About us" page, and the "Contact us" page (URL based), and all research summary pages, but only to users who are anonymous users.

Things can get more complex but these two use cases are a good starting point and realistic examples of what people do with Drupal.

Step #1: place a block for anonymous users

Let's assume the user is a content editor, and the user groups "Anonymous" and "Subscriber" as well as the page groups "Subscriber pages" and "Public pages" have already been created for her by a site builder. Her first task is to place the "Access reports" block and make it visible only for anonymous users.

Outside in block placement anonymous
First the editor changes the impersonation to "Anonymous" then she places the block. She is informed about the impact of the change.

Step #2: place a block for subscribers

Our editor's next task is to place the "Download reports" block and make it visible only for subscribers. To do that she is going to want to view the page as a subscriber. Here it's important that this interactions happens smoothly, and with animation, so that changes that occur on the page are not missed.

Outside in block placement subscriber
The editor changes the impersonation to "Subscribers". When she does the "Access reports" block is hidden as it is not visible for subscribers. When she places the "Download report" block and chooses the "Subscriber pages" page group, she is notified about the impact of the change.

Step #3: see if you did it right

Once our editor has finished step one and two she will want to go back and make sure that step two did not undo or complicate what was done in step one, for example by making the "Download report" block visible for Anonymous users or vice versa. This is where impersonation comes in.

Outside in impersonate
The anonymous users need to see the "Access reports" block and subscribers need to see the "Download report" block. Impersonation lets you see what that looks like for each user group.

Summary

The idea of combining a number of contexts into a single object is not new, both context and panels do this. What is new here is that when you bring this to the front-end with impersonation, you can make a change that has broad impact while seeing it exactly as your user will.

Comments

Marcus Morba (not verified):

I love the UX/UI - which distribution offers this?

May 02, 2016
Dries:

No distribution offers this today. The goal of this blog post is to brainstorm and discuss options for how to improve Drupal's user experience. Feel free to contribute with more thoughts and/or code! :)

May 02, 2016
Larry Garfield (not verified):

What you're describing here is not "context" as the Panels/ctools world uses the term. What you're describing is block visibility. That's well and good, and being able to pre-define visibility clusters sounds like a good step forward.

It falls down, though, when the underlying layout may vary between pages, too. If a given region doesn't even exist on certain pages, then it becomes difficult to map a block visibility rule between them. You still need multiple block instances in that case. Probably solvable, but will involve more steps for users. Really, though, at this point I don't see any functionality that Panelizer with IPE doesn't already provide today. Maybe the Impersonate functionality, but there's already a module for that. :-)

The bigger problem is "context" in the sense of "context-sensitive input arguments." Consider this use case:

"As a site builder, I want to place a block in the left sidebar on all news nodes that shows the title, date, and author of all news items in the same organic group as the news node I'm on."

In Panels/Views today, that's really quite simple, IF you are OK with defining back-to-front. You build a View Mode for news nodes that shows the title, date, and author. Then you build a View (easy) with a single contextual filter and give it a "Content Pane" (D7) or "Block" (D8) display, showing that view mode. Then you go to Panels/Panelizer/Panels Everywhere (the distinction is very fuzzy, I agree, that needs to be fixed), and configure News node pages. Then I create a new derived context for the Organic Group of the node being displayed. I now have 2 node contexts. Then I can place the View block I just created, and tell it to use the OG context. Poof, done.

If you know how to do that, it takes about 3 minutes the first time, and to place the same block again takes about 1 minute. It's not obvious that's what you need to do, of course, which is where people run into trouble with the Panels suite.

Now, try to achieve that by placing the block first, without the context defined. And without giving users a list of 50 possible blocks to place, 2/3 of which can't even work on that page and would be non-functional (and thus not display). *THAT'S* the challenge. That's what the Panels folks mean by "context is hard". Block visibility isn't a problem. Contextual input values are.

Really, I'd love to see that UX problem solved. It's a hard one, and an important one. If we can come up with a better approach for that (which results in a view mode, view, etc. still being created under the hood because that's how Drupal works today), I'll be ecstatic. But at this point I'd settle for the use case being acknowledged. :-/

May 02, 2016
Dries:

Really great comment! Thanks Larry. We should brainstorm some more on how we could solve this use case outside-in, and that of contextual input values in general.

It's worth remembering that an outside-in UI doesn't have to solve every use case (and I think you acknowledged that in your last paragraph). The idea is to keep the current inside-out UI for complex use cases, and to provide an outside-in UI for 80% of the use cases.

Furthermore, we might also be able to simplify the experience you described by using a hybrid approach; certain steps in that experience could probably be made outside-in.

May 02, 2016
catch (not verified):

"Now, try to achieve that by placing the block first, without the context defined. And without giving users a list of 50 possible blocks to place, 2/3 of which can't even work on that page and would be non-functional (and thus not display)."

There are two sets of information that need to be matched:

1. The context available on the current page (node type, organic group the node is posted to, node author).

2. The contextual inputs of the available page elements on the site (list of nodes based on current node's organic group membership, list of nodes by a user, rendered user entity etc.)

So an outside-in interface for those is really about presenting the 'available contexts on the current page' - possibly at least initially limited to those contexts which are actually configured elsewhere. Then presenting blocks filtered/limited by those currently available contexts.

That seems very achievable.

Where it does get tricky is where rather than re-using an existing block, you need to create a new view from scratch. There's not going to be a way to make the entire Views UI available as a contextual menu item within a page, but it might be possible to do something approaching the current view wizards with some additional pointers from the current page context. So "Show me a list of nodes, related to the current node, by the current node's organic group". Then load the full Views UI, but with a preview of that listing which in turn could lead to configuring a new view mode if one is required. You end up with a lot of nested steps that way, but at least one naturally leads to the other compared to having to jump around menus like now.

May 02, 2016
Kevin Oleary (not verified):

"I don't see any functionality that Panelizer with IPE doesn't already provide today. Maybe the Impersonate functionality, but there's already a module for that."

Yes, Panels and Panelizer do achieve this result, but I think you are dramatically underplaying the effort:

"You build a View Mode for news nodes that shows the title, date, and author. Then you build a View (easy) with a single contextual filter and give it a "Content Pane" (D7) or "Block" (D8) display, showing that view mode. Then you go to Panels/Panelizer/Panels Everywhere (the distinction is very fuzzy, I agree, that needs to be fixed), and configure News node pages. Then I create a new derived context for the Organic Group of the node being displayed. I now have 2 node contexts. Then I can place the View block I just created, and tell it to use the OG context. Poof, done."

Poof done? For you maybe. I've been using Drupal for seven years and I'd have to seriously tax my brain to make that happen without your cheat sheet. The fact that all of that functionality exists is fantastic because we can build UIs like this on top of it, but let's not kid ourselves that it's in any way easy.

Let's sit down in New Orleans and hash this out because I think that outside-in workflows can be designed that cover the kinds of examples you are talking about—potentially using panels as their underlying architecture—without necessarily exposing the complexity of the panels UI to most users.

May 03, 2016
Larry Garfield (not verified):

"... but let's not kid ourselves that it's in any way easy."

I didn't say it's easy, at least not in the sense of "few obvious steps". On the contrary, I don't think anyone in the Panels/ctools/Views world thinks it's "easy" in that sense. You have to know how the moving parts work, and the UI currently does a horrid job of teaching you what those moving parts are and how they work. The lack of consistency in Drupal itself (we offer both view modes and fielded views, then there's Panels, and display suite, and context module (which shouldn't be used anymore in D8, IMO), etc.), and then the Panels UI has never been regarded by anyone as approachable.

However, I and others have been asking for help for a long time making understanding these modern components (view modes, design components, views, context, etc.) more approachable within the "bottom up" world that Drupal has been for years. There's absolutely room for improvement there, within the "100%" system. And that improvement, frankly, desperately needs to happen *regardless* of whether an "80%, outside-in" UI is built, because that other 20% is still there and is, frankly, one of the key value-adds of Drupal and design components.

By analogy, it's like asking for help with the node edit page for years and only getting quick-edit as a counter-proposal. OK, yeah, useful for certain cases but the edit page still needs help, please stop ignoring it! (Note: I know the node edit page did get a lot of help in D8, which is wonderful, I'm just using it as an analogy here.)

Unrelated side note: How did you get the italic/cite tags in there when Dries' blog doesn't allow HTML tags???

May 03, 2016
catch (not verified):

"Authenticated users logged in from the United States"

This is where context starts to get tricky in terms of scaling.

Let's say you have a site serving the US, Canada and Mexico, and you mainly care about auth vs. anonymous users.

You can either have a user group that's like:

"Authenticated users logged in from the United States".

Or you can have two groups, "Authenticated" and "United States", then when configuring the block or masquerading:

"Authenticated users" AND "logged in from the United States".

The ability to combine groups for both visibility and preview complicates the UI a bit especially for impersonation, but it also means that on a site serving 30 countries, you end up with a user group for each country + auth and anonymous (i.e. 32 groups), instead of every combination which would be 62 groups. Once you add more roles the difference becomes even more stark 33 vs. 93, 34 vs. 124 etc.)

When evaluating whether a user can see something restricted to a user group, we just need to evaluate if the user is a member of the groups specified on the page element - this is something that should be straightforward to either pre-calculate or to cache - you either have a role or not, you either have a country set in your user profile or geo-located or not. Although again the less groups there are the less total evaluations will need to be made.

May 02, 2016
Kevin Oleary (not verified):

I think what you are describing is a problem of complexity itself, not necessarily this approach. In Drupal 7 many sites end up with at least the level of complexity you describe. In real-world situations I believe that routing the creation of these user groups through impersonation (ie. clicking "+new user group" in the dropdown) will lead to groups that more accurately match the business goals of the people using them.

Let's take your example. I have 32 countries I serve and I have authenticated and anonymous users, but what am I serving them?

If I'm a publisher and all my countries are in the Americas, I could use a small number of groups like: Anon-EN, Auth-EN, Anon-ES, Auth-ES, Anon-PS, Auth-PS, to cover anonymous, and authenticated in English, Spanish and Portuguese.

If my countries are in the US and EU and my site is a commerce site I might only need Auth-US, Anon-US, Auth-EU, Anon-EU, because my main concerns are currency and compliance.

The business needs of the customer—plus their desire to keep things simple and manageable—are going to drive them to focus only on the contexts that *significantly alter the page*, and ignore contexts that do not. What this in turn does is nudge the user towards thinking more economically about how they contextualize things and away from creating overly complex and crufty sites.

I do think it will be valuable to have the ability to use multiple user groups as part of a page group (as an advanced setting), but not to allow multiple groups for impersonation, which doesn't make sense because impersonation is subtractive by nature (show me only...) not additive (show me both...).

What would be valuable, and I have designs for, is a more complex faceted impersonation *builder* UI that would let you preview while changing multiple contexts and then save a user group if you want to re-use it. I would not envision that being used in at the same time as page building, however, because understanding the delta between the facets changing the preview and the visibility settings of the content would produce cognitive overload.

May 05, 2016
catch (not verified):

"I do think it will be valuable to have the ability to use multiple user groups as part of a page group (as an advanced setting), but not to allow multiple groups for impersonation, which doesn't make sense because impersonation is subtractive by nature (show me only...) not additive (show me both...)"

How do you handle user roles then? Those are not only additive, but also orthogonal to user groups.

"If I'm a publisher and all my countries are in the Americas, I could use a small number of groups like: Anon-EN, Auth-EN, Anon-ES, Auth-ES, Anon-PS, Auth-PS, to cover anonymous, and authenticated in English, Spanish and Portuguese.

If my countries are in the US and EU and my site is a commerce site I might only need Auth-US, Anon-US, Auth-EU, Anon-EU, because my main concerns are currency and compliance."

I've worked on a site with over ten and over twenty countries. They had some content that they had to tailor per-country, some that applied to all countries, some that applied to different groups of countries (not consistently though), then some of that content also had to be translated. On top of that there were user groups, orthogonal to countries, different to Drupal roles, and content was targeted to those too.

Is that ridiculously complex? Yes it is. However there's no way that a single 'user group' could have reflected those needs. Now it might be valid to say to people that tough and they're on their own, but that's more valid for the UI that core provides, less for the underlying system (which does not exist either).

Language is another tricky one. It's common to have blocks that only show in one language or a set of languages - especially on site that are multilingual but where a lot of content isn't translated between the different languages. Additionally language is sometimes a user group (i.e. based on the user's own configuration) sometimes a page context (taken from the URL). Either language isn't a user group - in which case it needs to be its own thing in this system, or it is and there's an extra dimension of cross sections to worry about.

May 06, 2016
James Williams (not verified):

I'm pretty excited to see these discussions, with regards to getting Drupal core working towards this way. It's cool to hear that the outside-in approach may intentionally just be to cover the majority of cases (80%), rather than trying to be completely comprehensive. I think that is a good way to make the outside-in UX goal (which is noble in itself) actually happen, as 100% coverage could indeed be very tricky. Larry's point about context-sensitive input arguments is a strong challenge, but that kind of complexity is going to be less common on sites where users struggle with Drupal's current inside-out approach anyway, I suspect (I have no evidence for this!).

May 03, 2016
Alan Dixon (not verified):

Great to see this, I have two separate comments that I'll squeeze together ...

1. My initial reaction to the post is: great, this sounds like the concepts I'm used to (I'm a context guy). But then I see Larry, who's clearly a panels guy, who brings into this discussion that set of labels and abstractions. So we might as well just lay that out: there are two competing models of how to do this kind of thing, and there may be a political decision to be made. At least, we should be acknowledging some competing language.

2. I like that you're giving names to the abstractions and being rigorous in defining them and how they will be used. I love that about Drupal. It also reminds me of the awesome beauty in the Organic Groups implementation for Drupal 7. And it also reminds me that every time I work with Organic Groups, it takes me longer than I care to admit to re-wrap my head around how it works. So my point is: let's not forget that people have to understand this stuff, people who don't spend all their time with their heads in Drupal. I recognize that 'make it as simple as possible, but not simpler' may not actually have a workable implementation for what you're trying to achieve, so that brings me back to reinforcing the recurring theme you don't want to go too far chasing beautiful abstractions that solve every problem.

May 05, 2016
Matthias Lünemann (not verified):

"User groups" sounds a lot like "Personas".
Personally I would not use the name "groups" in these concepts. This could be mixed up with concepts of the organic groups or group modules, which bundle entities of different types (nodes/users) into groups.
Currently I have no other name for "Page groups".

July 07, 2016