• Tag Folders: http://web.me.com/jonstovell/Tag_Folders/Tag_Folders_Home.html
• Punakea: http://www.nudgenudge.eu/punakea
• TagBot: http://bigrobotsoftware.com/
• Tags: http://gravityapps.com/tags/overview/
• Leap: http://www.ironicsoftware.com/leap/

Info about tagging

• CommonTag – a new standard for tagging that includes metadata and semantic organization for tags

Principles and techniques

Productive interaction requires a different approach to design, and a different view of the audience. To help frame these differences, we can look at the development of productive interaction systems through four major vectors:

• Content: Information, narrative elements, meanings and sensations as communicated in text, image, video, sound, tactile and other modes.
• Context: The integrated presentation of content in form, decoration, attitude, organization, selection, values, and experiences.
• Affordance: The handles that enable the audience to work with and manipulate the content and context.
• Audience: The users as integral elements of the total system, who operate it through the affordances, and who create the final expressions.

In this framework, design is a radically non-linear process of engagement and making, spinning a complex and recursive relationship between the domains of content, context, affordance and audience. Here, each element influences the design of the other as the designer builds up a full, yet flexible and open work that relies on the user as an active collaborator.

With the vectors of content, context, affordance, and audience, what approaches are available to the designer? The following is a selected taxonomy of design principles and techniques for productive interaction. Note that the categorizations are somewhat arbitrary, because each of the four is tightly interwoven and interdependent.

Content Context Affordance Audience

Information, narrative elements, meanings and sensations as communicated in text, image, video, sound, tactile and other modes.

• not – linear – Interactive content elements are designed from the beginning to work by themselves, in combination and juxtaposition, or in dynamically constructed linearities. This is a break from the linear design of content, where things can be predetermined to work well together. In not-liner design, content will often be recombined in ways the designer did not anticipate. As a consequence, each content element needs to have a greater integrity on its own, yet also have an openness that allows it to be connected up with, or juxtaposed against other content elements. For example, in the Physical Music project, musical elements like the bass and drum parts are composed so they will combine with a variety of other elements. This is a very different composition approach than writing for a traditional song, because the composer has to think in a multi-layered way to successfully anticipate the many different combinations. At the same time, there are certain combinations in Physical Music that technically work, but are “bad” sounding. This is okay, and even beneficial to the overall effectiveness of the work because the user needs to have the flexibility of failure to fully explore the music. In addition, what may sound “bad” to the composer may well fit into a “good” composition by the user. This productive misuse of a system often indicates a truly successful interactive design, and is an important design goal.
• smart – Interactive content can be enhanced with attached metadata to give it machine-actionable semantics. Software can then manipulate the content based on that metadata, providing multiple, rich views and handles on it. For example, in Topic Explorer the body text has selected phrases tagged as interesting pull-quotes, these quotes can be separately displayed as appropriate to provide interesting gateways into the body text. Similarly, if text facets are tagged with different keywords, the software can organize and present the content and summaries based on the explicit or implicit actions of the user. XML is an obvious system for implementing this approach, and is how Topic Explorer tags author identified pull quotes and excerptable headings. A more sophisticated way to look at content is to think of each element as a smart object that behaves through its own set of designer imposed rules and behaviors. A kind of artificial-life version of text, image, and data, where meaning and behavior emerge from the collective actions of the media elements as choreographed by the user.
• associative – Links and associations between content facets are both authored and generated algorithmically, providing a means for the user (and system) to build on-the-fly relationships between content facets. These associations can be as simple as hyperlinks, or they can define complex interrelationships that are neither hierarchical nor one-toone. Associations don’t need to change the entire context as typical web links do when they load a new page. Instead, they can help build up and vary a rich, multi-faceted collection of content that’s accessible all at once.
• dynamically updatable – Content can be revised, added to, and subtracted from after the initial work is released. The changes may come from the original author, be applied algorithmically, come from external sources, or be contributed by the audience. They Rule changes over time because users contribute new maps, which then change in their prominence because users rate the maps.

Content Context Affordance Audience

The integrated presentation of content in form, decoration, attitude, organization, selection, values, and experiences.

• system as facilitator/provocateur – Productive interactive systems are designed to actively help the user produce the meaning and knowledge they want. The designer may author the application to facilitate this in an assistive and collaborative way, or she may use a more provocative style, pushing the user in directions they may not be considering. 
• real-time juxtaposition – The system can dynamically present combinations of elements that evoke new contexts, perceptions, and connections and enable the user to experiment and engage with these combinations. There’s a qualitative difference between static, fixed juxtaposition and the dynamic, real-time juxtaposition of productive interaction, especially when the user participates in the creation of the juxtaposition. The act of causing juxtaposition makes the user complicit and creates a more visceral connection with it. In the Topic Explorer, the user creates real-time juxtapositions by selecting the search keywords. New combinations of the two different texts are simultaneously displayed each time a keyword is selected, allowing the user to see the politicians’ views side by side in ways they might never have seen or considered before. 
• adaptation – The system and its elements should be aware of current and past contexts, and behave in interesting and productive ways based on those contexts. Actionable contexts include user preferences, the state of other content elements, geographic location, time, events, news, etc. The adaptive system reconfigures itself based on these contexts. For example, mobile phones can adapt to a new location by presenting appropriate shopping information for the current neighborhood. Further, the phone could know the user’s current interests and preferences and provide more appropriate information according to that context. 
• serendipity – Natural systems such as libraries and design studios provide ample opportunity for productive browsing, happenstance, coincidence, diversion and digression. These valuable opportunities for random and intentional unexpectedness can trigger new directions and new ways of seeing things, and should be built into interactive systems. For example, in the Topic Explorer experiment, the serendipity stream provides an alternative mode to reading the material in a linear way. The user can turn to this approach as a way to see the material in a new light and be surprised by perspectives they may not have seen otherwise. 
• constructed simultaneity – Material can be presented in a live, multi-channel format, where content facets are played back at the same time, but in different combinations as driven by the interaction. This real-time, constructed simultaneity provides a rich, intensive interaction environment for the user to chart a path through. In the Physical Music project, the user is playing with simultaneous combinations of musical elements, orchestrating arrangements, adding and subtracting, filtering and modifying. In Topic Explorer, the user views two texts at once, driving them to related issues by using the keyword search. See also remixing below. 
• center/periphery, explicit/implicit , attending/attuning – John Sealy Brown identifies the importance of enabling the user to have simultaneous access to a range of material, some of which is in the center of attention, much of which is attuned to on the periphery. Designers must use subtle, wide-bandwidth techniques for enriching the flow of use, offering means of focus and unfocus that provide a fuller simultaneous content landscape without overwhelming the user. 
• passive interaction – While active interaction by the user is central, sometimes the system interacts with itself to advance facilitation and provocation. It does this by taking the place of the user, getting a handle on the affordances and changing things, while the user remains passive or focused on other elements. The video game “attract mode” is a classic example of this, where if an arcade game is left unused for a period, it drops into a self-playing mode to attract new customers. In the Topic Explorer, the serendipity stream runs on its own, displaying a continuous set of options that can be can be attended to or not.
• presets – The designer can provide different presets that configure a system with many capabilities. These presets establish a starting point with immediate usefulness, and can assist the user in successfully climbing the learning curve and/or returning to a more known state. They also provide the designer with an opportunity to frame a highly crafted presentation in the context of dynamic and sometimes entropic systems. Physical Music uses presets to provide the user with “good” sounding arrangements from which they can diverge, experiment, and create their own presets.

Content Context Affordance Audience

The handles that enable the audience to work with and manipulate the content and context.

• faceting – Content and meaning can be viewed in different perspectives, individually or in combinations as a way to provide focus/periphery, juxtaposition and simultaneity. Faceting is a system of dynamically working content, like a table full of different books opened to pages of related information. In the Text Facets exploration, the user is able to see four different facets at once rather than having to scroll through a long, linear set of text or opening up individual pages for each set of text. 
• excerpting – The user needs different ways of seeing and accessing material at different levels – full versions, clips, indexes, tables of contents and hyperlinks are “traditional” ways. But interaction allows for dynamically generated, user-controlled excerpts derived from the actual material – a kind of real-time clipping and summarizing service. This provides the user with means to pull out, highlight, and juxtapose content elements in new and useful ways. These excerpts can be created from metatagged sections of the content (e.g. predefined pull-quotes), or generated through algorithms (e.g. random pullquotes). In the Topic Explorer, the serendipity stream does exactly this kind of excerpting as an alternative to manually browsing through the full text or using a more traditional table of contents or index. 
• filtering – With large amounts of content available, there’s a need to look at different subsets of material. Filters provide a way to view material by commonalities (filtering in), or eliminate material to clarify and reveal what’s left (filtering out). The filtering can be “hard”, eliminating material completely, or “soft”, pushing the extra material to the periphery. 
• remixing – As a way to provide a deep engagement with the material, the system can enable the re-ordering, re-combination, and re-constitution of the material. This makes it possible for the user to make their own sequences, change the layering and mixing of simultaneous materials, mute and solo parts, and alter elements. The user becomes a DJ, with an array of media EQ, mixers, turntables, and effects. In the case of They Rule, the user turns organizations and board members on and off, expands and contracts displays, and moves elements around on the screen in a process of building a “mix,” or map that builds a new understanding of the relationships. 
• wide bandwidth interaction – To rival other media, interactive systems must move beyond the impoverished mouse-crouch of current computing, to systems that more fully Philip van Allen Productive Interaction v.11/14/04 Page 14 engage human physical-spatial powers. This will involve designing for large, highresolution screens, tangible interfaces, multiple interaction devices, haptic feedback, gestural control, and new sensors that increase the resolution and quality of the interaction and outcomes. It requires moving beyond the zoetrope character of today’s interactive media. As a simple example, Topic Explorer is designed for a large video display to allow the user to see a full range of information at the same time. Similarly, the Minority Report interface provides a rich information landscape and a variety of mechanisms to manipulate and interact with that information. 
• authoring, configuration, performance – Interactive systems commonly have use patterns that involve authoring, configuration and performance. The designer must provide different affordances for each of the modes. For example, a person fluidly takes photos (hopefully) without thinking about interface – a kind of performance. In a different mode, the photographer needs to configure their camera to take a self-timer picture. And in another mode, they would author a slideshow for their friends. Interactive designers must consider these different uses (and the transitions between them); providing the most appropriate affordances. These may be nicely integrated in a single “non-modal” interface, or they may be separated across independent devices (e.g. a camera for performance and configuration, and iPhoto on a computer for the slide show authoring). 
• separation and clarity of function – Integrating all capabilities into a single system is sometimes useful, but it’s often better to create different affordance systems for different uses. Separating the capabilities may involve a reconfiguration of one device with different sets of affordance, or separation of capability into different devices. “Convergence” is not always the best approach. 
• custom affordance – Systems can have a variety of capabilities, and rather than imposing a single set of affordances, the designer may enable the user to pick or create their preferred affordance set. Macro keys that perform a series of OS tasks are a simple example of this. An advanced approach for custom affordance would have the system publish its capabilities and allow various independent affordances/devices to “latch” onto selected capabilities. With this, the user could pick a personal system of affordance. E.g. someone could buy a “core” of an iPod with a “published” set of capabilities, and then wrap it with a separately purchased custom skin/interface that taps into a particular subset of those capabilities. Some might pick a skin that that is super small with limited access to capability. Others might want something bigger with handles on more features. Yet others might go primarily for cool style and just the features they like. The fanatics will make and modify their own hotrodded skins.

Content Context Affordance Audience

The users as integral elements of the total system, who operate it through the affordances,and who create the final expressions.

• contribution – The audience can move beyond interacting, and contribute their own material, providing content and content structures for later by themselves or a community of users. In They Rule, users can build “maps” and make them available to other users and sharing their insights. 
• social design – Interaction design not only involves consideration of the current user(s), but of the social context in which the system is used. The “interaction” extends beyond the “interface” and into the social system that surrounds the system – people discuss and advise each other about the system, use it collaboratively, contribute to the system and share with others, fit the system into existing working methodologies, and use it to create new ones. 
• user as content – If we extend the interactive system out to include the user, we begin to create very rich relationships and unique outcomes. For example, the system could automatically (with permission) sense the user’s information (age, preferences, past interactions, opinions, medical info, etc.), and adjust to it. In other words, the idea of metadata is extended to the user, who is tagged with a variety of information that can be incorporated into the interactive presentation.

Abstract 

There have been many research efforts devoted to tangible user interfaces (TUIs), but it has proven difficult to create a definition or taxonomy that allows us to compare and contrast disparate research efforts, integrate TUIs with conventional interfaces, or suggest design principles for future efforts. To address this problem, we present a taxonomy, which uses metaphor and embodiment as its two axes. This 2D space treats tangibility as a spectrum rather than a binary quantity. The further from the origin, the more ‘‘tangible’’ a system is. We show that this spectrum-based taxonomy offers multiple advantages. It unifies previous categorizations and definitions, integrates the notion of ‘‘calm computing,’’ reveals a previously un-noticed trend in the field, and suggests design principles appropriate for different areas of the spectrum.

a-taxonomy-for-tangible-interfaces

Contents

[hide]

1. 1 Location/scale
2. 2 Sketching
3. 3 Refined, precise, detailed
4. 4 Broad, large, expressive
5. 5 Tools
6. 6 Reflection
7. 7 Collaboration
8. 8 Idea generation
9. 9 Learning/design process
10. 10 Manipulating Psychological State

Here is a first rough attempt at a list of affordances. These don’t always get to the afforded activity – needs fleshing out and re-working. This is kind of brain-dump and so the list is not consistent in tone or approach.

Location/scale

• handheld and can be easily moved
  • portability
  • think with hands
  • vary location
  • enable social interaction, collaboration
  • enables multiples for crit, comparison, juxtaposition in small field of view
• fixed in space, not easily moved
  • it’s a place to go to – exists in space (and in the head)
  • enables people to come together without planning
  • site specificity – relation to culture, objects, context
• various scales
  • hand scale, 4″ – intimate, fine use of hands
  • book scale, 8″ – good for writing and reading, narrow field of view enables single focus
  • arm’s length scale, 3′-4′ – enables simultaneity, broad gestures, fills field of view
  • human scale, 5′-8′ – evokes emotion, enables standing, full body sensation
  • car scale, 12′ – requires movement to see entire, 
  • building scale, 20′ – evokes shelter, social, modulates light
• defines and visualize a physical or imaginary location in space – enables spatial mental representation & manipulation
• sit/stand
• can spread-out work

Sketching

• little prep
• rough work easy
• lots of ideas quickly
• low cost materials means little censoring
• tangible output to assist in metaphor, reflection, ??? 
• helps brainstorm, idea generation

Refined, precise, detailed

• engages fingers, hands in gestures
• learning curve enables fine, controlled motor skills
• wide range of expression possible

Broad, large, expressive

• big gestures
• enables use of arms, other parts of body
• wide field of vision

Tools

• evocative, tool has character
  • specific, narrow-range, idiosyncratic, rich, evocative
  • general, adjustable, wide-range
• learning curve
  • learning curve minimal, fast to use
  • learning curve maximal, lots of nuance to expression
  • both are possible in some, but not always
• tools for making tools – custom for the designer’s practice, or specific project
• tools for making processes - custom for the designer’s practice, or specific project

Reflection

• juxtaposition – see ideas in comparison to each other
• isolation – of person, or of content
  • focus, eliminates context, losing yourself in it, flow
• viewable outside the mind – tangible, in the world, sensible
• evokes ideas in the mind
• editable

Collaboration

• social interaction
  • group can see the same thing, shared experience
  • each can communicate with others
  • each can manipulate thing
  • “eye-contact”
• pin-up
• synchronous and asynchronous discussion
• divide into parts for individual work
• scale of collaboration requires some different approaches

Idea generation

• visualization of ideas
• externalize & organize
• think by writing
• think by sketching
• think by making
• serendipity
• inspiration
• define principles/constraints
• enables complete absorption/flow
• disrupt flow/absorption

Learning/design process

• test ideas
• experiment with multiple approaches
• learn medium in context of project
• learn materials in context of project
• organize
• make a mess
• reorganize/refactor
• sequence
• breakdown into tasks

Manipulating Psychological State

• ability to change modes through the day
• concentration
• distraction
• entry into “flow”/absorption/focus
• contemplation
• daydreaming
• discharge nervous energy
• create nervous energy
• enable thinking/analysis by moving around (physically/mentally)

Contents

[hide]

1. 1 Overview
2. 2 How Designers Work – Areas of Interest
3. 2.1 Focus on Meaning Making
4. 2.2 Designers operate in two different modes: Making Ideas and Making Bits
5. 2.3 Design Process: Hand -> Tool -> Content -> Format
6. 2.4 Gestures of Expression
7. 3 Our Approach to a Taxonomy of Affordance
8. 3.1 Goals for a taxonomy
9. 3.2 Focus on Meaning Making Affordances
10. 3.3 What kinds of affordances should we look at?
11. 3.4 Format of the Taxonomy
12. 4 Prototype Directions
13. 4.1 Tools for Making Ideas

Overview

We covered a lot of ground in the first couple days, and I think it sets the stage well for our research. It seems to me that we’re covering three main areas.

1. defining how designers work and what parts of that we’re interested in.
2. defining our approach to building a taxonomy
3. identifying potential directions for our prototypes

How Designers Work – Areas of Interest

In order to focus our work and chart out a unique area of research we have to narrow and refine what we’re looking at. I think we agreed that we are less concerned about the specifics of mark-making and gestural aspects of the designer’s tools. Others have covered much of that, and we think we can contribute more in the modes of design as they relate to making/crafting/forming/generating meaning. A big breakthrough in our thinking about design process was that designers not only make tangible outcomes, but as important to their process, designers make ideas from which the tangible outcomes result. In addition, we identified a pattern of making (at least for the tangible output) of, roughly, Hand ->Tool ->  Content -> Format.

Focus on Meaning Making

For both the taxonomy and working prototypes, we’re focusing on how designers make meaningful results rather than the lower level aspects of gesture, form-making and mark-making. In particular:

• making bits and ideas: the design process that we consider must include affordances around idea making as well as making bits (see below for more)
• meaningful modes of expression: we’re interested in how tools afford expressive and interesting making of meaning. The metaphor of musical instruments was raised in terms of how simple they are, yet how wide a range of output is possible (with skill). Further, we’re interested in the specific, idiosyncratic output tools afford. And lastly, we like tools that afford creative abuse.

Designers operate in two different modes: Making Ideas and Making Bits

We realized that a designers process for design includes two parts. Often the designers spends a significant amount of time generating ideas, whether through thinking, note making, sketching or other process. Ultimately, this process is about helping the designer shape what’s inside their head (or the collective heads of a group) rather than making tangible output. This part of design requires significantly different kinds of affordances than producing tangible output (bits). The designer often slides easily between the realm of ideas and made outcomes, and there is not always a clear dividing line.

Nevertheless, we believe that the affordances needed for making ideas vs. making bits are substantially different. And we believe that the realm of tools with affordances for making ideas for designers is fertile ground for digital enhancement.

Design Process: Hand -> Tool -> Content -> Format

In trying to understand the methodology of design in relation to affordance, we came to the following understanding:

• The designer makes a gesture with part of their body, typically the HAND
• That gesture intersects with a TOOL of some sort
• The tool operates on some form of CONTENT, either turning raw materials into communication, or reworking in some way existing “samples” into new forms.
• The working of the content intersects with its context which is the FORMAT of the communication, such as an 8 1/2″ x 11″ paper, or multimedia installation in a 20′x40′ room.

Gestures of Expression

We found 3 types of high level gestures for working with content as a way of creating a meaningful expression. Any of these three modes can be done in isolation or in combination.

• Additive/Generative – This is create new content out of thin air. Whether typing letters into a word-processor, applying graphite to paper, or making new vector shapes in Illustrator, the designer builds up new content.
• Subtractive – This approach involves a process of removing material from existing content in a process of editing and eliminating.
• Manipulative – Here the designer works with existing material to rearrange, distort, combine, and juxtapose.

Our Approach to a Taxonomy of Affordance

In trying to situate our research and develop a taxonomy of affordance, we discussed the goals of the taxonomy, the areas of affordance we’re interested in, and the format of the taxonomy.

Goals for a taxonomy

We identified the following goals for our taxonomy of affordance:

• Provide designers with inspiration for invention
• Analyze and understand design practice
• Inform the discourse of design
• Provide tools for design pedagogy

To achieve these goals, we’ll want to eventually publish our results in a range of forms including a website, conference papers, and articles.

Focus on Meaning Making Affordances

Rather than concentrate on detailed gestures of mark-making or the detailed affordances of tools and how they operate, we decided to focus our efforts on the quality of meaning making that affordances offer. As discussed earlier, we’re interested in:

• range of output – that affordances offer useful ranges of output in terms of making meaningful output
• idiosyncratic output – we think that the character of the affordance is extremely important, and that often the best tools have a personality, and a somewhat idiosyncratic and rich way of interpreting the designer’s gesture. This means that the design can select a particular tool knowing its character and how that may fit their desired meaning.
• creative abuse – often the best tools do their particular job well, but also afford creative abuse. i.e. permit unintended uses that the designer discovers in through process of design.

More generally, we’re interested in the designer’s process at the level of meaning making, the creative moment, and also as discussed the making of ideas.

What kinds of affordances should we look at?

We’re still in discussion on what affordances to look at. Should it be for making bits, making ideas, or? And should we also look at the affordances of humans? I.e. what role do they play in the mix? Hunter discusses this more in this post: Human Affordance and Ideation

Format of the Taxonomy

We discussed different forms of taxonomy, from hierarchical to grid based. But we are leaning towards some form of database driven taxonomy that affords the user to get at the collection in a range of ways, producing their own conclusions through an interactive and computational driven approach. In particular, we think that some form of tagging and possibly rating characteristics may provide the best and most flexible taxonomy.

We may need to build a custom database to achieve this.

Prototype Directions

While more discussion is needed, we identified several areas of potential work in prototypes. First, each RA is interested in the following (these need expansion):

• Hyun – How tools and affordances can be customizable, and how tools can learn the designer
• Chris – how can affordances help develop a culture for designers. also what affordances can help give media elements “thing qualities” through their behavior, agency, etc.
• Hunter – How does the workspace itself become a tool for design. How does it shape context, help with collaboration (within design and across other disciplines), and how can mobile tools afford the designer more power as they operate in different environments including working in other people’s workspaces.

Tools for Making Ideas

In addition to these areas of interest, a new idea was developed. Out of our discussion on making bits and making ideas, we came to the realization that there is an interesting opportunity in creating something that will help designers in their generation and refinement of ideas. This system might have the following characteristics:

• Helps manage project ideas and documents
• Permits sketching in many forms, and maintains a history of sketches
• Helps with brainstorming
• Is a separate device from the “workstation”
• Automatically tracks documents as they are created, and assists in the collection of those documents into projects
• Assists in the management of projects in terms of organizing ideas, documents, and outcomes.
• Also permits the leveraging of previous projects in the starting of new projects, e.g. copying an old project to establish a starting point for something new.