Research for Responsive Media

The theory of conceptual metaphor is explained as a kind of metaphor-making. Metaphor-making in this instance refers to representations of the human body and of the manner in which human beings negotiate their physical and cultural environments (Lakoff & Johnson, 1999). The concept of ‘understanding’ is commonly represented in the form of a mental image, for example as the ‘grasping’ of an object. Humans construct their understanding of the world and their place in it through transactions with its physical nature and build them into abstract concepts. (Lakoff & Johnson, 1980, 1999). Making intentional connections with concrete experience involves cross-domain relational mapping. Cross-domain mappings originate as concrete episodes in human experience shaping the relational structure of concepts. The process of metaphor-making which underpins this mental mapping, occurs spontaneously (Cantril, 1960) . During the 1980’s computer scientists designed and developed the graphical user interface currently exemplified in the Apple Macintosh desktop. Human-computer interaction expanded from this point to include virtual knobs (widgets), sliders and other forms of physical interactions more widely used to engage with the concrete world. However, commercially designed interfaces lack a formally explained linkage between the sensory input of interactive encounters and the narrative reasoning they impose. Through the experimental application of sensor/physical interfaces, designed to reduce the cognitive load or split attention of users (Saul 1999), this study tests a metaphorical scaffolding for the navigation of abstract cinematic events that promises a more seamless transition between narrative content and its physical encounter. D.A. Norman argues that interface designers assume that users will appreciate intuitively their conceptual intentions. He claims that most interface designers provide insufficient interactive diversity to render content conceptually transparent. Many examples are sighted in Norman’s The Psychology of Every Day Things. In response to Norman’s observations we seek to revise the staring point for our conceptual model of user interface by borrowing Primary Conceptual Metaphors (PCM) from conceptual metaphor theory and using cross-domain relational mapping of these metaphoric responses to embodied experience. Computers allow complete control of the interactive cinematic environment, the computer can only project the images and connections designed into the content. The CI’s can control the cinematic content and the design of the source-target connections within the sensor/physical user interface that reinforce association between relevant movements or positions and/or spatial relations and the actual link from the source (the interface) to the target (content). Specifically this research aims to: Employ Lakoff’s metaphor-based approach to concept formation in the generation of sensor/physical user interfaces within interactive cinematic environments. This research will by means of place, orientation and distance allow beholders to navigate a world of projected moving images encountered in the expanded cinematic form via the concrete physical domain of PCM-based sensor/physical user interfaces. Test the hypothesis that PCM as a model for sensor/physical computing user interface design, by developing a series of prototypical sensor/physical user interfaces. Two different scenarios are going to be tested. A single spectator model in a single and/or multi-projection screen environment. The spectator is tracked by the vision-system and there absolute position and relative position to the screen(s) is the trigger for content change. The PCM utilised for this scenario will be derived in consultation with the dramaturge and video director. Potential candidates are: “more is up”, “time is motion”, “states are locations”. The second scenario is tracking multiple people (up to three) and make their relative position to each other and their orientation the trigger for screen events. The PCM utilised for this scenario will be derived in consultation with the dramaturge and video director but for example the PCM could be “Intimacy is closeness”, “relationships are enclosures”. Interpret and refine the PCM sensor user interfaces we developed in particular evaluate the split attention problem (Saul 1999) between interface negotiations and the potentialities of beholder generated multi-temporal narrative experiences. By watching the spectators interacting with the environment we should be able to get a better understanding if the approach is successful and the user understood intuitively the underlying metaphorical concept. This may be on an unconscious level or the user is aware of the structure. Utilising the vision based tracking system we will be able to gather empirical data about user movement in the environment. This data will be used for analysing user movement, their dynamic and length of stay. We will ask the users to fill out a questionnaire after exploring the environment. Questions asked are about their understanding of the underlying structure, their emotional engagement, problems, and their level of enjoyment. Through the combination of the information we collected (objective computer data, subjective from the CI’s perspective and subjective from user’s point-of-view) we will be able to analyse and evaluate the success level of our PCM based user interface.