Citation info: Ekman, I. (2009) "Modelling the Emotional Listener: Making Psychological Processes Audible", Proc. AudioMostly 2009, Sepbember, Glasgow, Great Britain.
 
Abstract. There is an increasing tendency to use of procedural strategies for the creation and manipulation of sound in computer games. This development is motivating a shift in the design process; meaning is no longer tied to a specific asset or asset type, but is instead linked to the procedural manipulation of the sound material. Often the design phase for this type of game sound includes modelling sound within the virtual environment in terms of a source, a medium and a listener. This way of thinking about sound is not new; however, how it relates to emotional expression has not been investigated before. Particularly what has not been modelled is the listener as a perceptual entity, whose perception changes to reflect psychological states and processes. This study addresses the lack of research in this area. We identify four psychological processes that influence what and how sounds are heard: attention, emotion, multimodal perception and internal sound. We also provide a detailed investigation of a special case of psychological process: the perceptual distortions people suffer during extreme stress. Drawing from this empirical data, we form a listener model capable of expressing the avatar’s psychological qualities through sound manipulation. The listener model is described, along with examples on how to apply it in practise.
 
Read more…

Autumn is here, and AudioMostly is again getting near. This year, AM goes Scotland, and the event will be hosted by the Glasgow Caledonian University. I really look forward to the event, not least because of the conference theme this year: "Sound and Emotion". The program can be found here.

Looks like a nice selection of presentations, but also intimate enough to foster great conversations. And I’m really looking forward to hear Patrik Juslin’s keynote… I will also give a presentation of my own, entitled "Modelling the emotional listener: making psychological processes audible". I’ll put it up online after the event, so stay tuned!

Earlier this year, I was fortunate to be able to attend the 35th conference on Audio for Games held by the Acoustical Engineering Society. This was a very special event since it was the first ever AES conference (in a long history of events) to deal with game sound. I enjoyed the conference very much. Particularly I was delighted with the relatively even representation of both academia and industry, and the fact that the two camps really did share thoughts and views with each other. Academia and industry mixing, you don’t get that so often.

The conference was a huge success, the downside of which was that the organizers were unable to squeezen in all of those who wanted to attend. (!) Well, in addition to organizing great conferences, the AES has a fantastic habit of providing conference reports on their webpage, and now the report for the 35th is also out. So if you weren’t able to attend, you can at least browse through the topics and the whos and whens.

The report is accessible directly from here [pdf] or alternatively you can browse your way through the long list of conference reports on this page. Maybe stumble on some other interesting topics there, too…

Unfortunately, the conference papers are not provided online by the AES. However, for anyone interested in the full story on my paper (with Raine Kajastila) "Localisation Cues Affect Emotional Judgements – Results from a User Study on Scary Sound", I provide it in full length online here.

I’ve always loved flamenco - I think the style matches my temper and moods better than any other music styles I know of. I’ve never had a go at the guitar, but I’ve been dancing for a while now, enough to dare call myself a bailaora, and I’m slowly getting into singing as well. Recently, flamenco has also entered my work interests. A couple of colleagues of mine - Antti Jylhä and Cumhur Erkut - have developed a hand-clapping based interface, that will be presented at CHI2009: "A Hand Clap Interface for Sonic Interaction with the Computer". Recently, I’ve been working together with them on an extension of that work, contributing among other with insights from flamenco. The first part of this work "Exploring self-induced sonic interactions in the context of performing arts" is being presented at the Workshop on Sonic Interaction Design and its relation to Film and Theatre sound design, at the University of York, 2. April 2009.

I was actually searching for info on similar projects, but instead ran across this very well written piece by David Rubin on creating flamenco sounds for your audio project: desktop flamenco. It’s a valuable read for anyone who works on a piece in need for some authentic-sounding Flamenco, as it covers the major principles of the flamenco style, and provides reviews of a bunch of sample libraries containing flamenco effects. Nice work bringing all that info together on one page!

It’s official - the 4th Audio Mostly conference will be held in Glasgow, Scotland on September 2-3 and it will be hosted by Glasgow Caledonian University. The theme this year will be "Sound and Emotions".

The call for papers can be found here.

Citation info: Ekman, I. and Kajastila, R. (2009) "Localisation Cues Affect Emotional Judgements – Results from a User Study on Scary Sound", Proc. AES 35th Conference on Audio for Games, February 2009, London, UK. CD-ROM.
 
Abstract. The current paradigm for creating emotional impact in game sound is to carefully choose which sounds to play. This paper takes an alternative approach, suggesting that emotional impact of sounds can be affected by choosing how to play those sounds. We describe a novel concept for emotional sound design - emotional fine-tuning - and show how it is possible to systematically influence the emotional impact of a single sound sample. A controlled user study with 8 subjects confirmed that changing the reproduction of a sample so that source localization of the sound is challenged will increase its perceived scariness compared to the same sound with clearly detectable source. The work extends experimental research on emotion perception in sound. It has practical implications for sound design in games and other interactive media.
 
Read more…

Citation info: Ekman, I. (2008) "Psychologically Motivated Techniques for Emotional Sound in Computer Games", Proc. AudioMostly 2008, October, Piteå, Sweden, 20-26.
 
Abstract. One main function of sound in games is to create and enhance emotional impact. The expressive model for game sound has its tradition in sound design for linear audiovisual media: animation and cinema. Current theories on emotional responses to fiction are mainly concerned with linear medial, and only partly applicable to interactive systems like games. The interactivity inherent to games introduces new requirements for sound design, and suggests a break in perception compared with linear media. This work reviews work on emotional responses to fiction and applies them to the area of game sound. The synthesis is interdisciplinary, combining information and insights from a number of fields, including psychology of emotion, film sound theory, experimental research on music perception and philosophy. The paper identifies two competing frameworks for explaining fictional emotions, with specific requirements, and signature techniques for sound design. The role of sound is examined in both cases. The result is a psychologically motivated theory of sound perception capable of explaining the emotional impact of sound in film, as well as identifying the similarities and difference in emotional sound design for these two media.
 
Read more…

Yay to the guys & gals in Piteå! The proceedings from this year’s Audio Mostly are now available online, alongside publications from earlier years. This was the first peer-reviewed event of the conference series that started out in Piteå in 2006, then visited Ilmenau in southern Germany 2007 to return to cosy Sweden again this year. Be sure to check out the great papers! For a brief recap of the presentations, you may take a glimpse at my earlier posts about the event, separate for day 1 and day 2. I’ve also put up a blog-readable version of my conference paper here.

I notice that our paper Ekman, I. & Kajastila, R.: Spatialization Cues Affect Emotional Judgements – Results from a User Study on Scary Sound has been accepted to the AES 35th conference, running the theme Audio For Games. Looking forward to visiting London next spring.