Citation info: Ekman, I. (2009) "Modelling the Emotional Listener: Making Psychological Processes Audible", Proc. AudioMostly 2009, Sepbember, Glasgow, Great Britain.
 
Abstract. There is an increasing tendency to use of procedural strategies for the creation and manipulation of sound in computer games. This development is motivating a shift in the design process; meaning is no longer tied to a specific asset or asset type, but is instead linked to the procedural manipulation of the sound material. Often the design phase for this type of game sound includes modelling sound within the virtual environment in terms of a source, a medium and a listener. This way of thinking about sound is not new; however, how it relates to emotional expression has not been investigated before. Particularly what has not been modelled is the listener as a perceptual entity, whose perception changes to reflect psychological states and processes. This study addresses the lack of research in this area. We identify four psychological processes that influence what and how sounds are heard: attention, emotion, multimodal perception and internal sound. We also provide a detailed investigation of a special case of psychological process: the perceptual distortions people suffer during extreme stress. Drawing from this empirical data, we form a listener model capable of expressing the avatar’s psychological qualities through sound manipulation. The listener model is described, along with examples on how to apply it in practise.
 
Read more…

Horror Video Games: Essays on the Fusion of Fear and Play, is a collection of essays concerning horror video games. The book is edited by Bernard Perron and has a foreword by Clive Barker.

This publication is a first of its kind, as nowhere has the themes and variations of horror games previously been so thoroughly analyzed. The book provides 14 essays on various topics, by the following authors: Clive Barker, Inger Ekman, Ewan Kirkland, Tanya Krzywinska, Petri Lankoski, Christian McCrea, Simon Niedenthal, Michael Nitsche, Martin Picard, Bernard Perron, Dan Pinchbeck, Richard Rouse III, Guillaume Roux-Girard, Laurie N. Taylor, Carl Therrien, and Matthew Weise.

Together with Petri Lankoski, we contributed with an essay on horror sound called Hair-Raising Entertainment: Emotions, Sound, and Structure in Silent Hill 2 and Fatal Frame

The book will be available in October, but you can already pre-order it straight from McFarland, or alternatively from Amazon.

Autumn is here, and AudioMostly is again getting near. This year, AM goes Scotland, and the event will be hosted by the Glasgow Caledonian University. I really look forward to the event, not least because of the conference theme this year: "Sound and Emotion". The program can be found here.

Looks like a nice selection of presentations, but also intimate enough to foster great conversations. And I’m really looking forward to hear Patrik Juslin’s keynote… I will also give a presentation of my own, entitled "Modelling the emotional listener: making psychological processes audible". I’ll put it up online after the event, so stay tuned!

Earlier this year, I was fortunate to be able to attend the 35th conference on Audio for Games held by the Acoustical Engineering Society. This was a very special event since it was the first ever AES conference (in a long history of events) to deal with game sound. I enjoyed the conference very much. Particularly I was delighted with the relatively even representation of both academia and industry, and the fact that the two camps really did share thoughts and views with each other. Academia and industry mixing, you don’t get that so often.

The conference was a huge success, the downside of which was that the organizers were unable to squeezen in all of those who wanted to attend. (!) Well, in addition to organizing great conferences, the AES has a fantastic habit of providing conference reports on their webpage, and now the report for the 35th is also out. So if you weren’t able to attend, you can at least browse through the topics and the whos and whens.

The report is accessible directly from here [pdf] or alternatively you can browse your way through the long list of conference reports on this page. Maybe stumble on some other interesting topics there, too…

Unfortunately, the conference papers are not provided online by the AES. However, for anyone interested in the full story on my paper (with Raine Kajastila) "Localisation Cues Affect Emotional Judgements – Results from a User Study on Scary Sound", I provide it in full length online here.

The Finnish Composers’ Copyright Society Teosto is a copyright organization for composers, lyric writers, arrangers and music publishers in Finland. Teosto provides licensing products for performing and reproducing music that in their words "meets the needs of music using customers, and distributes the royalties to the rightholders - the music authors."

Teosto is commonly known to have a stiff structure, and consequently Teosto agreements may sometimes present problems for composers who e.g. want to cross between paid and unpaid work or, for example want to showcase their works online. Also, Teosto’s apparent lack of procedures for handling the lifespan of game music (the game itself, marketing material, derivatory products like soundtracks, their marketing material, etc.) and a narrow understanding for the particular practical conventions of the industry (for example, some game companies only hire musicians who will provide the game music as commissioned work) has made life hard for Finnish game sound people.

Teosto appears already to have unofficcially accepted certain common practises, overlooking instances such as when artists publishing one’s own music on private webpages. (Even so, commercial-but-private spaces like MySpace have been a problematic thing and may be so even in the future.) Teosto is also known to have made some individual agreements with musicians, but according to my understanding these private agreements have been a pain to negotiate and even still, the solutions have been at most inventive workaraounds to tackle the dated structures of Teosto agreements.

Now, the situation may be changing. According to a recent article in taloussanomat (in Finnish) Teosto is apparently reconsidering the limitations their deals put on musicians. The article tells us that the board of Teosto will meet as soon as April, to consider the new terms of agreements. I’m hopeful this meeting may at least put in black-on-white the unofficcially accepted practises, but even better would be if Teosto really took it upon themselves to reconsider the way the whole music licensing thing should be conducted. Either way, these are promising news for game composers. Let’s hope not only empty promises.

It’s official - the 4th Audio Mostly conference will be held in Glasgow, Scotland on September 2-3 and it will be hosted by Glasgow Caledonian University. The theme this year will be "Sound and Emotions".

The call for papers can be found here.

Noticing that the deadline for chapter proposals has been pushed forward to March 16th, I post here a link to the call for chapters to a book on game sound (ed. Mark Grimshaw) http://www.igi-global.com/requests/details.asp?ID=578

Selected copy-paste:

The heart of this book is the relationship between player and game sound; how this is both effected and affected by technology and how this relationship itself impacts upon the design of computer game sound and the development of technology. It deals with both technical and theoretical aspects in a way that that stimulates ideas and broadens the potential readership base beyond computer games.

[…]

Recommended topics include, but are not limited to, the following (which may be combined in chapters):

• Hardware and software
• Sound synthesis and audio samples
• Similarity and difference in practice across game genres
• Sound stimulus
• Psychophysiology, emotion, meaning and sound
• Authenticity, realism and verisimilitude
• Audio icons and earcons in games
• Sonification
• Spatializing sound and localization
• Sound and image
• Sound and no image (audio-only games)
• Player(s) and soundscape(s)
• Sonic environment
• Engagement and flow
• Sonic interaction
• Immersion
• Virtual reality
• Relational agents
• Biofeedback

Citation info: Ekman, I. and Kajastila, R. (2009) "Localisation Cues Affect Emotional Judgements – Results from a User Study on Scary Sound", Proc. AES 35th Conference on Audio for Games, February 2009, London, UK. CD-ROM.
 
Abstract. The current paradigm for creating emotional impact in game sound is to carefully choose which sounds to play. This paper takes an alternative approach, suggesting that emotional impact of sounds can be affected by choosing how to play those sounds. We describe a novel concept for emotional sound design - emotional fine-tuning - and show how it is possible to systematically influence the emotional impact of a single sound sample. A controlled user study with 8 subjects confirmed that changing the reproduction of a sample so that source localization of the sound is challenged will increase its perceived scariness compared to the same sound with clearly detectable source. The work extends experimental research on emotion perception in sound. It has practical implications for sound design in games and other interactive media.
 
Read more…

Citation info: Ekman, I. (2008) "Psychologically Motivated Techniques for Emotional Sound in Computer Games", Proc. AudioMostly 2008, October, Piteå, Sweden, 20-26.
 
Abstract. One main function of sound in games is to create and enhance emotional impact. The expressive model for game sound has its tradition in sound design for linear audiovisual media: animation and cinema. Current theories on emotional responses to fiction are mainly concerned with linear medial, and only partly applicable to interactive systems like games. The interactivity inherent to games introduces new requirements for sound design, and suggests a break in perception compared with linear media. This work reviews work on emotional responses to fiction and applies them to the area of game sound. The synthesis is interdisciplinary, combining information and insights from a number of fields, including psychology of emotion, film sound theory, experimental research on music perception and philosophy. The paper identifies two competing frameworks for explaining fictional emotions, with specific requirements, and signature techniques for sound design. The role of sound is examined in both cases. The result is a psychologically motivated theory of sound perception capable of explaining the emotional impact of sound in film, as well as identifying the similarities and difference in emotional sound design for these two media.
 
Read more…

Yay to the guys & gals in Piteå! The proceedings from this year’s Audio Mostly are now available online, alongside publications from earlier years. This was the first peer-reviewed event of the conference series that started out in Piteå in 2006, then visited Ilmenau in southern Germany 2007 to return to cosy Sweden again this year. Be sure to check out the great papers! For a brief recap of the presentations, you may take a glimpse at my earlier posts about the event, separate for day 1 and day 2. I’ve also put up a blog-readable version of my conference paper here.