Principles and techniques

Productive interaction requires a different approach to design, and a different view of the audience. To help frame these differences, we can look at the development of productive interaction systems through four major vectors:

• Content: Information, narrative elements, meanings and sensations as communicated in text, image, video, sound, tactile and other modes.
• Context: The integrated presentation of content in form, decoration, attitude, organization, selection, values, and experiences.
• Affordance: The handles that enable the audience to work with and manipulate the content and context.
• Audience: The users as integral elements of the total system, who operate it through the affordances, and who create the final expressions.

In this framework, design is a radically non-linear process of engagement and making, spinning a complex and recursive relationship between the domains of content, context, affordance and audience. Here, each element influences the design of the other as the designer builds up a full, yet flexible and open work that relies on the user as an active collaborator.

With the vectors of content, context, affordance, and audience, what approaches are available to the designer? The following is a selected taxonomy of design principles and techniques for productive interaction. Note that the categorizations are somewhat arbitrary, because each of the four is tightly interwoven and interdependent.

Content Context Affordance Audience

Information, narrative elements, meanings and sensations as communicated in text, image, video, sound, tactile and other modes.

• not – linear – Interactive content elements are designed from the beginning to work by themselves, in combination and juxtaposition, or in dynamically constructed linearities. This is a break from the linear design of content, where things can be predetermined to work well together. In not-liner design, content will often be recombined in ways the designer did not anticipate. As a consequence, each content element needs to have a greater integrity on its own, yet also have an openness that allows it to be connected up with, or juxtaposed against other content elements. For example, in the Physical Music project, musical elements like the bass and drum parts are composed so they will combine with a variety of other elements. This is a very different composition approach than writing for a traditional song, because the composer has to think in a multi-layered way to successfully anticipate the many different combinations. At the same time, there are certain combinations in Physical Music that technically work, but are “bad” sounding. This is okay, and even beneficial to the overall effectiveness of the work because the user needs to have the flexibility of failure to fully explore the music. In addition, what may sound “bad” to the composer may well fit into a “good” composition by the user. This productive misuse of a system often indicates a truly successful interactive design, and is an important design goal.
• smart – Interactive content can be enhanced with attached metadata to give it machine-actionable semantics. Software can then manipulate the content based on that metadata, providing multiple, rich views and handles on it. For example, in Topic Explorer the body text has selected phrases tagged as interesting pull-quotes, these quotes can be separately displayed as appropriate to provide interesting gateways into the body text. Similarly, if text facets are tagged with different keywords, the software can organize and present the content and summaries based on the explicit or implicit actions of the user. XML is an obvious system for implementing this approach, and is how Topic Explorer tags author identified pull quotes and excerptable headings. A more sophisticated way to look at content is to think of each element as a smart object that behaves through its own set of designer imposed rules and behaviors. A kind of artificial-life version of text, image, and data, where meaning and behavior emerge from the collective actions of the media elements as choreographed by the user.
• associative – Links and associations between content facets are both authored and generated algorithmically, providing a means for the user (and system) to build on-the-fly relationships between content facets. These associations can be as simple as hyperlinks, or they can define complex interrelationships that are neither hierarchical nor one-toone. Associations don’t need to change the entire context as typical web links do when they load a new page. Instead, they can help build up and vary a rich, multi-faceted collection of content that’s accessible all at once.
• dynamically updatable – Content can be revised, added to, and subtracted from after the initial work is released. The changes may come from the original author, be applied algorithmically, come from external sources, or be contributed by the audience. They Rule changes over time because users contribute new maps, which then change in their prominence because users rate the maps.

Content Context Affordance Audience

The integrated presentation of content in form, decoration, attitude, organization, selection, values, and experiences.

• system as facilitator/provocateur – Productive interactive systems are designed to actively help the user produce the meaning and knowledge they want. The designer may author the application to facilitate this in an assistive and collaborative way, or she may use a more provocative style, pushing the user in directions they may not be considering. 
• real-time juxtaposition – The system can dynamically present combinations of elements that evoke new contexts, perceptions, and connections and enable the user to experiment and engage with these combinations. There’s a qualitative difference between static, fixed juxtaposition and the dynamic, real-time juxtaposition of productive interaction, especially when the user participates in the creation of the juxtaposition. The act of causing juxtaposition makes the user complicit and creates a more visceral connection with it. In the Topic Explorer, the user creates real-time juxtapositions by selecting the search keywords. New combinations of the two different texts are simultaneously displayed each time a keyword is selected, allowing the user to see the politicians’ views side by side in ways they might never have seen or considered before. 
• adaptation – The system and its elements should be aware of current and past contexts, and behave in interesting and productive ways based on those contexts. Actionable contexts include user preferences, the state of other content elements, geographic location, time, events, news, etc. The adaptive system reconfigures itself based on these contexts. For example, mobile phones can adapt to a new location by presenting appropriate shopping information for the current neighborhood. Further, the phone could know the user’s current interests and preferences and provide more appropriate information according to that context. 
• serendipity – Natural systems such as libraries and design studios provide ample opportunity for productive browsing, happenstance, coincidence, diversion and digression. These valuable opportunities for random and intentional unexpectedness can trigger new directions and new ways of seeing things, and should be built into interactive systems. For example, in the Topic Explorer experiment, the serendipity stream provides an alternative mode to reading the material in a linear way. The user can turn to this approach as a way to see the material in a new light and be surprised by perspectives they may not have seen otherwise. 
• constructed simultaneity – Material can be presented in a live, multi-channel format, where content facets are played back at the same time, but in different combinations as driven by the interaction. This real-time, constructed simultaneity provides a rich, intensive interaction environment for the user to chart a path through. In the Physical Music project, the user is playing with simultaneous combinations of musical elements, orchestrating arrangements, adding and subtracting, filtering and modifying. In Topic Explorer, the user views two texts at once, driving them to related issues by using the keyword search. See also remixing below. 
• center/periphery, explicit/implicit , attending/attuning – John Sealy Brown identifies the importance of enabling the user to have simultaneous access to a range of material, some of which is in the center of attention, much of which is attuned to on the periphery. Designers must use subtle, wide-bandwidth techniques for enriching the flow of use, offering means of focus and unfocus that provide a fuller simultaneous content landscape without overwhelming the user. 
• passive interaction – While active interaction by the user is central, sometimes the system interacts with itself to advance facilitation and provocation. It does this by taking the place of the user, getting a handle on the affordances and changing things, while the user remains passive or focused on other elements. The video game “attract mode” is a classic example of this, where if an arcade game is left unused for a period, it drops into a self-playing mode to attract new customers. In the Topic Explorer, the serendipity stream runs on its own, displaying a continuous set of options that can be can be attended to or not.
• presets – The designer can provide different presets that configure a system with many capabilities. These presets establish a starting point with immediate usefulness, and can assist the user in successfully climbing the learning curve and/or returning to a more known state. They also provide the designer with an opportunity to frame a highly crafted presentation in the context of dynamic and sometimes entropic systems. Physical Music uses presets to provide the user with “good” sounding arrangements from which they can diverge, experiment, and create their own presets.

Content Context Affordance Audience

The handles that enable the audience to work with and manipulate the content and context.

• faceting – Content and meaning can be viewed in different perspectives, individually or in combinations as a way to provide focus/periphery, juxtaposition and simultaneity. Faceting is a system of dynamically working content, like a table full of different books opened to pages of related information. In the Text Facets exploration, the user is able to see four different facets at once rather than having to scroll through a long, linear set of text or opening up individual pages for each set of text. 
• excerpting – The user needs different ways of seeing and accessing material at different levels – full versions, clips, indexes, tables of contents and hyperlinks are “traditional” ways. But interaction allows for dynamically generated, user-controlled excerpts derived from the actual material – a kind of real-time clipping and summarizing service. This provides the user with means to pull out, highlight, and juxtapose content elements in new and useful ways. These excerpts can be created from metatagged sections of the content (e.g. predefined pull-quotes), or generated through algorithms (e.g. random pullquotes). In the Topic Explorer, the serendipity stream does exactly this kind of excerpting as an alternative to manually browsing through the full text or using a more traditional table of contents or index. 
• filtering – With large amounts of content available, there’s a need to look at different subsets of material. Filters provide a way to view material by commonalities (filtering in), or eliminate material to clarify and reveal what’s left (filtering out). The filtering can be “hard”, eliminating material completely, or “soft”, pushing the extra material to the periphery. 
• remixing – As a way to provide a deep engagement with the material, the system can enable the re-ordering, re-combination, and re-constitution of the material. This makes it possible for the user to make their own sequences, change the layering and mixing of simultaneous materials, mute and solo parts, and alter elements. The user becomes a DJ, with an array of media EQ, mixers, turntables, and effects. In the case of They Rule, the user turns organizations and board members on and off, expands and contracts displays, and moves elements around on the screen in a process of building a “mix,” or map that builds a new understanding of the relationships. 
• wide bandwidth interaction – To rival other media, interactive systems must move beyond the impoverished mouse-crouch of current computing, to systems that more fully Philip van Allen Productive Interaction v.11/14/04 Page 14 engage human physical-spatial powers. This will involve designing for large, highresolution screens, tangible interfaces, multiple interaction devices, haptic feedback, gestural control, and new sensors that increase the resolution and quality of the interaction and outcomes. It requires moving beyond the zoetrope character of today’s interactive media. As a simple example, Topic Explorer is designed for a large video display to allow the user to see a full range of information at the same time. Similarly, the Minority Report interface provides a rich information landscape and a variety of mechanisms to manipulate and interact with that information. 
• authoring, configuration, performance – Interactive systems commonly have use patterns that involve authoring, configuration and performance. The designer must provide different affordances for each of the modes. For example, a person fluidly takes photos (hopefully) without thinking about interface – a kind of performance. In a different mode, the photographer needs to configure their camera to take a self-timer picture. And in another mode, they would author a slideshow for their friends. Interactive designers must consider these different uses (and the transitions between them); providing the most appropriate affordances. These may be nicely integrated in a single “non-modal” interface, or they may be separated across independent devices (e.g. a camera for performance and configuration, and iPhoto on a computer for the slide show authoring). 
• separation and clarity of function – Integrating all capabilities into a single system is sometimes useful, but it’s often better to create different affordance systems for different uses. Separating the capabilities may involve a reconfiguration of one device with different sets of affordance, or separation of capability into different devices. “Convergence” is not always the best approach. 
• custom affordance – Systems can have a variety of capabilities, and rather than imposing a single set of affordances, the designer may enable the user to pick or create their preferred affordance set. Macro keys that perform a series of OS tasks are a simple example of this. An advanced approach for custom affordance would have the system publish its capabilities and allow various independent affordances/devices to “latch” onto selected capabilities. With this, the user could pick a personal system of affordance. E.g. someone could buy a “core” of an iPod with a “published” set of capabilities, and then wrap it with a separately purchased custom skin/interface that taps into a particular subset of those capabilities. Some might pick a skin that that is super small with limited access to capability. Others might want something bigger with handles on more features. Yet others might go primarily for cool style and just the features they like. The fanatics will make and modify their own hotrodded skins.

Content Context Affordance Audience

The users as integral elements of the total system, who operate it through the affordances,and who create the final expressions.

• contribution – The audience can move beyond interacting, and contribute their own material, providing content and content structures for later by themselves or a community of users. In They Rule, users can build “maps” and make them available to other users and sharing their insights. 
• social design – Interaction design not only involves consideration of the current user(s), but of the social context in which the system is used. The “interaction” extends beyond the “interface” and into the social system that surrounds the system – people discuss and advise each other about the system, use it collaboratively, contribute to the system and share with others, fit the system into existing working methodologies, and use it to create new ones. 
• user as content – If we extend the interactive system out to include the user, we begin to create very rich relationships and unique outcomes. For example, the system could automatically (with permission) sense the user’s information (age, preferences, past interactions, opinions, medical info, etc.), and adjust to it. In other words, the idea of metadata is extended to the user, who is tagged with a variety of information that can be incorporated into the interactive presentation.

Abstract 

There have been many research efforts devoted to tangible user interfaces (TUIs), but it has proven difficult to create a definition or taxonomy that allows us to compare and contrast disparate research efforts, integrate TUIs with conventional interfaces, or suggest design principles for future efforts. To address this problem, we present a taxonomy, which uses metaphor and embodiment as its two axes. This 2D space treats tangibility as a spectrum rather than a binary quantity. The further from the origin, the more ‘‘tangible’’ a system is. We show that this spectrum-based taxonomy offers multiple advantages. It unifies previous categorizations and definitions, integrates the notion of ‘‘calm computing,’’ reveals a previously un-noticed trend in the field, and suggests design principles appropriate for different areas of the spectrum.

a-taxonomy-for-tangible-interfaces

To state that I once hated graphic design is neither, provocative nor original. Many graphic designers before me, have felt the physical and creative malaise I experienced after twelve-hour days of mouse-clicking week in, week out. This chapter describes the creative equivalent of a recovery program for disenchanted designers. This story narrates how my colleagues and I conceived of an ongoing research project, that positioned our studio practice as the research subject, our professional dissatisfaction as the primary issue and calling for creative speculation within the workplace as our key argument.

Speculation, Serendipity, and Studio Anybody

ABSTRACT

This work presents the results of a comparative study in which we investigate the ways manipulation of physical versus digital media are fundamentally different from one another. Participants carried out both a puzzle task and a photo sorting task in two different modes: in a physical 3-dimensional space and on a multi-touch, interactive tabletop in which the digital items resembled their physical counterparts in terms of appearance and behavior. By observing the interaction behaviors of 12 participants, we explore the main differences and discuss what this means for designing interactive surfaces which use aspects of the physical world as a design resource.

Affordances for Manipulation of Physical versus Digital Media on Interactive Surfaces

Large websites and intranets can benefit from improved methods of search and navigation. These include site maps, A-Z indexes, sophisticated search engines, and generally improved navigational design—and playing a potential role in all of these methods is well-planned taxonomy.

better-living-through-taxonomies

INTRODUCTION
Traditionally, input devices have been discussed in terms of their mechanical and electrical properties (Foley & Van Dam, 1982; Sherr, 1988). Discussions centre on “joysticks,” “trackballs,” and “mice,” for example. Several studies have attempted to evaluate the technologies from the perspective of human performance. Many of these are summarized in Greenstein and Arnaut (1988) and Milner (1988). A common problem with such studies, however, is that they are often overly device-specific. While they may say something about a particular device in a particular task, many do not contribute significantly to the development of a general model of human performance. (There are exceptions, of course, such as Card, English and Burr, 1978.) With the objective of isolating more fundamental issues, some researchers have attempted to categorize input technologies and/or techniques along dimensions more meaningful than simply “joystick” or “trackball.” The underlying assumption in such efforts is that better abstractions can lead us from phenomenological descriptions to more general models, and hence better analogies.

Taxonomies of Input