AudioMostly 2008, Piteå norther Sweden. First day.

Here are some notes from Piteå, day 1. There were no parallel sessions, which was a really good thing, so this is really an overview of the whole conference.

Keynote by Nick Laviers
Nick Laviers is a video game Audio Director, based at EA’s Los Angeles Studio. Nick’s talk was about the music in Red Alert 3 and some of his earlier work such as the Harry Potter games. He took a very broad approach, and in his speech covered the design process both in terms of design decisions as well as the practical aspects of the process, from finding the right composers to organizing orchestral recordings. The talk was really engaging and peppered with useful insights about the realities you face when working on a huge title such as the C&C.
Some strategies I really want to look into:

• Using the memory of play (at restart of level etc.) to influence sounds. Nick’s example was with the background music remembering earlier encounters, I think you could extend that to the acoustic space as well. A room with a ‘worn’ feel.
• "Up yours" music. Basically, playing the other teams music when they really hit you down. That’s almost like the game is taking sides, temporarily abandoning your music to play the other players theme…
• Resting moments, reflection moments. Giving time for the players to absorb the consequences of their actions.

Talks

Mark Grimshaw, Craig A. Lindley & Lennart Nacke: Sound and Immersion in the First-Person Shooter: Objectively Measuring the Player´s Sonic Experience

Mark and colleagues are trying to measure immersion. Their experiment had people playing a Half Life 2 mod with different sound conditions and then measuring their subjective and psychophysiological responses to the sound. Not surprisingly, the subjective reports show people do perceive more ‘immersion’ when the sounds are on. The unexpected result was that they found no sign of difference in the psychophysiological (tonic) data depending on audio condition. Personally, I do think this has something to do with the measurement strategy: using a tonic measurement would probably even out all effects. The thing then is to decide suitable chunks of data for phasic analysis – now that’s a real challenge. It will surely be interesting to see how they proceed.

Daniel Kromand: Sound and the diegesis in survival-horror games

Daniel gave us a fascinating theoretic/analytic investigation into sound in two survival horror game titles: FEAR and SilentHill(2). He’s extending on Joergensen´s notion of the transdiegetic and looks at the tension created through functional use of sound, essentially the very same thing I’ve been working on the last few months. What Daniel is saying is emotional effect exists in the collapse of functional reliability. So, tying it down with the notion of functional fit, if positive emotions come from facilitating gameplay, certain negative emotions (fear/anxiety) are created by selectively making the cues in sound unreliable, this would make it more difficult to use game sounds for building up a strategy.

I had my paper as well on the first day:

Inger Ekman: Psychologically Motivated Techniques for Emotional Sound in Computer Games.

I looked at the complex tangle that is emotion design and what tools a game designer will use to create emotional impact in a game. The crux of my talk were the two competing frameworks for emotion design in games; you could think of them as story-emotions and gameplay-emotions. The two approaches employ partly conflicting strategies in sound, so you should try to find a way for either alternating or going for one of them, but not both at once. I also looked into all the other emotional evaluation that is going on alongside the cognitive evaluation processes and suggested how that maps into the general emotional experience. It seems to me the talk was well received and I was fortunate to have many interesting exchanges about it throughout the rest of the conference. And, how lovely it was to know in the evening of the first day I don’t have to be completely sober presenting the next morning

I’ll be putting up my slides soon and add a link so you can find them.

Louise Valgerður Nickerson & Thomas Hermann: Interactive sonification of grid-based games

Louise presented work on a sonified sudoku game and a four-in-a-row game, basically both really challenging grid-sonifications. The Sudoku had this touch-to-play approach, which would essentially let you use sound as a sort of tactile information and feel your way through the lines. The four-in-a-row game plays constantly, looping all seven piles in two bars of 4/4 beats and pitch representing the columns. I thought it sounded really funky in a cool way. I got to try it out in the evening, and I would say it really is playable and fun. Could be cool to have something like that on a mobile working over a Bluetooth connection!

David Moffat & David Carr: Using audio aids to augment games to be playable for blind people

David presented a sonified Asteroids! game, that is the normal Asteroids! but with added sound. They evaluated the game with sighted players playing either visual-only, audio-only or audio-visual conditions. Interesting results, as the best audio-only player was actually equal with a novice playing in visual-only condition! Adding audio did mostly not affect playing, however there were signs of cognitive owerload for the novice player when playing the audio-visual game condition. Had a very interesting discussion with David afterwards, where we talked about the real challenges for audio gaming and which sonification tasks will be the most challenging to solve as we try to create blind-accessible gaming. He suspected the most difficult tasks will be sonifying abstract things, and that just adding more realistic ‘naturalistic’ representations will get us quite far. I think we have quite a challenge left there, though, because the goal is also really to use ‘naturalistic’ in a suitably non-naturalistic way for other effects, like emotion. So finding a good balance there will be important.

Mats Liljedahl & Nigel Papworth: !BeoWulf - Sound based game

Nigel presented fresh research done with their BeoWulf game, specifically user studies of how people perceive the sounds of BeoWulf. I really enjoy the position they argue – on how sound opens up the creative processes of people and that lack of visual information actually enhances the sense of ‘being there’ instead of the opposite. This research was looking into how people describe the audio-mostly game world (visually they only see a simple map). The results demonstrate the richness of interpretations when using mainly sounds as a stimulus. I wouldn’t myself go so far as to say that the visual information closes down imagination, but certainly sound and visuals work in slightly different ways to open different ‘doors of imagination’.

Scott Beveridge & Don Knox: Control of Sound Environment using Genetic Algorithms

Scott started the presentation by showing an earlier project, a sonified cow (yeah, real one!). We all laughed. Then he went on to describe their current system for creating meaningful sonic representation of crowds. They collect data by webcam, map the crowd actions to sound. The main goal was to strive for musical output, which creates a question of mapping. Too literal mapping breaks up the structure, which will lead to atonality. Too abstract mapping will make the system feel detached and unresponsive.
The system uses direct and indirect translation, directly mapping certain features of an emotion classification engine and mapping them onto an arousal/valence map. The indirect mapping will produce high-level features such as chord progression and melodic contour, using genetic algorithms evaluated for consonance. The system seems interesting and complex enough, it will be interesting to follow their future work, especially as Scott said they are planning for inclusion of suprasegmental features such as melodic contour. That’s a tough one.

Daniel Hug: Genie in a Bottle: Object-Sound Reconfigurations for Interactive Commodities

Daniel is looking at interactive commodities and considering the semantics relating to the sound of objects. Especially interesting are electroacoustically augmented gadgets, intelligent, enhanced commodities that can play with the sound-object relationship. I’ve heard Daniel talk about these things before and I’m always fascinated by the way he ties together information from all these different fields: computer games, film sound, design tradition, sonification… Good overview of different fields with implications for game sound design as well.