A study on how music contributes to the experience of a video game
by Matthew Chin
Written on: 7 July 2010
This was a minor thesis (5000 words) I had to do as part of the JMC Academy course. The topic I choose was "The study of how music contributes to the experience of video game".
Abstract
This thesis will focus on music and its contribution to the experience of a video game. There are many variables that affect the way that music can contribute that will be looked into. The concept itself will be analysed through the many opinions that composers and gamers have about it. Music must be presented in a way that supports the constantly changing nature of a video game. Technology plays a role as well, as music for games didn't sound like it did today. Therefore there are going to be different ways that it can contribute, such as dynamic music. Dynamic music can be broken down in such a way that can be manipulated by the game as the player plays through the game. Gameplay also has a huge factor in selecting the appropriate music for the job. All these different techniques and situations will be discussed throughout this thesis.
Introduction
Music for video games is an interesting topic that has only recently come to a point that has sparked interest and respect in recent years. The amount of video game composers is constantly rising. Film composers are starting to compose for games, as well as live concerts that focus on game music is also starting to rise. While video game music in the past wasn't seen as important, all that has changed with technology. Technology has allowed game music to flourish into something that can compete with the film industry, as some of the game music is recorded and produced in the same way that a film would. Music of course helps us to immerse us in the experience of a game. There are also different opinions on how music is used from the composer, and the gamers' perspective, and thus there are a lot of different interpretations in how music may be used. There are also many methods, perhaps even more ways that music can contribute to a video game than a film soundtrack. Music may need to interact with the player, as most games are all about the player and their interactively with the game. These are the kind of topics that will be discussed in depth throughout this thesis. It aims to educate and inform about music and its contribution to the experience of a video game.
Literature review
Music for games has only recently bloomed to rival the quality of the film industry in recent times, but it is still constantly evolving. Music for games is slowly becoming much more recognised, which is evident in awards for games such as the "Game Developers Choice Awards". This has been growing since 2001 in which it recognises and chooses the best in a field with music composition being one of them. Video games live is another example, in which video games are played live in a concert with a real orchestra and live musicians that started out in 2005.
Therefore, written or published articles or books about video game music are uncommon at the moment. This may be due to the rapidly growing and changing nature of new video game music as it has only taken forty or so years to match the quality of film music which has been around since the beginning of the 20th century. Therefore, game music has had less time to grow and mature, and as a result there have not been very many articles or published works about this topic.
It also may be due to the technological limitations in past games, since music was made on chips and sounded very raw and rough as a result, couldn't contribute as much as a full orchestra today can; only in recent times, the game industry has been taking advantage of it, with game such as the Super Mario Galaxy series. Game music also wasn't seen as important because video game music back then was comprised of bleeps, which compared to the music that was produced for film music.
Since there are almost no published references on this topic, opinions will make up the content of this paper. These come in forms such as interviews, and reviews that cover a wide range of topics. There are a few well established sources that may be relied upon such as Gametrailers, IGN, Gamespot, and Game-OST which give opinions on their own about music in a game in a review, but also provide articles or videos on composer interviews as part of their content. Game-OST especially focuses on game soundtracks that are dedicated for composer interviews which will be very useful for this topic.
Of course, these are not the only sources that I will use, as there are other interviews and review on a variety of other sites. I noticed that homing in on a particular composer, or a game, will often discover interviews and opinions on another website from a different place. By investigating different opinions from different sources on a variety of topics related to the contribution of music in a game, this will condense all these ideas into the form of a thesis, as no one that I am aware of has done this.
Methodology
The method used to support this topic will be based on opinions from the gamers and composers. Both have views on how music contributes to the experience of a video game. Since composers are the ones that are directly involved with music in games, the bulk of the discussion will be from them. Therefore, there will be a lot of direct quotes from them as it will support certain ideas that are presented. Composers often talk in their interviews, how they approached a certain game, or any special techniques they used in order to achieve the goals they were planning of. This will greatly contribute to the topic. This doubles in a way since most of the dedicated game composers are usually gamers themselves.
Composer interviews will come from trusted commercial sources such as IGN, Gamespot, Gametrailers, Game-OST and a few others. The reason behind it is that their content is maintained to be accurate and concise as possible which is maintained by their staff members. Video interviews may be from any site, as long as the composer is being interviewed, as I can make sure that it is a valid source.
Getting opinions from gamers is a little trickier. There are game music forums that discuss game music however their opinions will not be as complex as composers can give. Therefore, the way that I am going to extract gamers' opinions, is from published interviews and game reviews from the sites mentioned before. Game-OST is a site that is dedicated to game soundtracks that has interviews and reviews on that topic. Only a proportion of gamer's opinions will be used in this thesis because, as mentioned before, they cannot give complex and in depth opinions about game music as they are not musicians.
An option of surveys was thought of, however, since music in games serves as a subconscious element, and often blending very well to the game itself, it is easy for the gamer to not notice its effect on the game, since it's not the primary focus on what the gamer is playing. Its effect only comes into play when the gamer actually pays attention to it, but of course, it depends on the game and the player. In order to obtain other people's opinion, we must make the gamer play the same scene, but have two alternate versions of music that has been turned on, and music that has been turned off. This would require getting people in, and getting them to play a series of games in which there would be not enough time to successfully complete. This is even more problematic due to the fact that some games to not allow for only the music to be turned off.
Results
During my research for reference material, I came across plenty of interviews on game composers that have a lot of opinions on the contribution to music in games. As expected though, there were few reviewers and interviewers that gave their opinions from commercial sites. The few that did didn't talk about game music in depth but will still be used in the thesis, as it is appropriate to this topic.
From the information that I have gathered, I figured that it would be best to split the discussion about this topic in five subsections. The first section will be "Game Music: The concept" where it will talk about how music in general will contribute or enhance the gaming experience. The next section will focus on film and TV music compared to game music, as they have differences and similarities between them. Following it will be technological influences on game music, as technology has had a direct role on the quality of music, and how technology has allowed game music to contribute more than it did in the past. Finally, the last section will focus on a few game examples, to demonstrate the practical applications of music, but also will also give game examples throughout the thesis.
These topics and subtopics are all based on what composers and gamers have discussed in several articles that were available.
Discussion
Game music: The concept
Music for games has almost been around as long as when video games first started. From the very beginning, music was used as an introduction to a game. One prime example was the Pac-Man arcade back in 1980. It wasn't long before music was integrated into the gameplay, with games such as the original Donkey Kong released in 1981. Music then was limited to technology and was often very short. For example, the Donkey Kong game only contained a one second repeated loop of music. Since then music for games has come a long way, and the goal has always been the same; to provide an accompaniment for the visuals that the player sees, whether it be a happy, catchy tune in Super Mario Bros, or an intense action scene in Call of Duty. "Scoring for games, film, TV, trailers is all about supporting what you're seeing." (Salta, 2010)
Music adds a very important sub-conscious element to the game that many people do not realise because the primary focus is on the game, and not the music. Sean Murray explains in an interview about the game Call of Duty: World at War. "I don't think you're going to be aware of it (the music) when you're playing because the action is so intense, and the emotions are so intense." (Murray, 2008) However, a game will not be the same without music, as it will not provide the same amount of impact.
Music plays a more important role in a game than many people realise. A lot of people don't realise or recognise "Why is this so intense?" and a lot of times, it's because of the music. So, when you apply that to games, I think that music plays at least that important of a role. (Salta, 2010).
Music has the power to even improve upon the game itself and not just merely contribute to it, even if the music is not consciously recognised. This is because of the sub-conscious effect that music has, and without it, a game can be seen worse off without the music there to contribute to the experience. "We were focus-testing, and all we did was put in music, and the scene all of a sudden got better ratings from our testers for gameplay." (Soule, 2007) Music creates a powerful way of influencing what we see. It is used as part of the experience of what makes a game enjoyable. "Music is a very powerful when it comes to creating an atmosphere for games and fuelling the rise and fall of emotions. It is a vital element for creating and enjoying games." (Kondo, 2010)
Not any music can be thrown into a game. Certain aspects of the game must be taken into effect in order for music to contribute correctly to the experience of a game. Gameplay for example has to be taken into consideration into what kind of music will most likely accompany the situation well.
For me, as a gamer, it affects the composition a great deal. It's important that the music feels like it goes hand in hand with what happens on the screen. How fast is the gameplay, how chaotic does it feel; these are very important considerations when writing music that plays throughout the game. (Kyd, 2007)
Music can also work the other way. If a score isn't used correctly, this can have the opposite effect by having the player feeling dissatisfied, and therefore it can ruin the experience instead of contributing to it. "One thing that people overlook about game music is the implementation of the game music. You can have great music, but if you over do it too little or at the wrong times, it can ruin the whole experience." (Salta, 2010)
Games also generally last a lot longer than film music, and therefore there is generally going to be a lot more music that may have to composed, or placed in. Therefore, repetition is going to be something that the composer has to deal with since it can often annoy the player in listening to the same tune over and over again. Composers can often avoid this by varying their music so it doesn't sound exactly the same.
There came a point where I was like, 'Oh, my god. This is a lot more than a movie.' Some of it needs to repeat in a clever way. And we're trying not to be redundant and boring. And sometimes there is no music. I think one of the differences with this game compared to other games, it lets some places play without music. (Zimmer, 2009)
The graphics in some games has gotten up to a point that the graphics are becoming cinematic and realistic which helps in immersing the player into this virtual world, making it seem real. However, graphics can only achieve so much, in terms of immersion. Music plays a big role in achieving that immersion as well.
The thing with ghost recon specifically, everybody knew how cinematic that was and they're really trying to bring players into a level of immersion with the gameplay and I think everyone recognises from the initial trailers that came out, to the moment you step off that chopper and you're in the game, there was a complete sense of immersion and the score had a big part in that. (E3 Interviewer, 2006)
Music can also serve as a memorable element to the game, as it is something you can take with you once a player has completed it. Great game music can often have this memorable effect instead of merely sub-consciously contributing to a game. The famous Super Mario Bros. theme is a very memorable tune beloved and remembered by the majority of gamers, and because of this tune, the original game is remembered even after all these years later. The composer of the Super Mario Bros. series Konji Kondo explains: "I think that music also plays an important role in terms of having the game stay in your memory for a long time." (Kondo, 2010)
Music has the power to change the emotions of any visual scene, and the games are no different. Music must be used correctly in order to convey what the developers wanted you to feel, through the music. As part of their job, composers must correctly interpret what the goals of the developers are in order to compose music that is effective in a video game, as music alone can change the feeling of the same scene in a video game.
You can play a completely different piece of music over the same scene in a videogame and get a completely different emotional experience from it. Imagine putting in action music like "Yeah, we're winning"; French horns are blasting away, percussion is going and you feel like "Yeah, I'm going to win this." Pick that same scene, and put this really sad, desperate melody over there that makes you feel like "I don't know if I'm going to make it through this", and it changes the whole experience. I think if they use music correctly, in a videogame, there's nothing quite like it. (Salta, 2010).
Game music compared to film and TV music
Game music, film and TV music are often compared against each other as they all have the same goal: to enhance the visuals on the screen. Therefore composers are often questioned about the comparison of game music to film and TV music. Jeremy Soule, who is a veteran game composer, commented on the nature of game and film music in an interview.
They're very similar, but the nature of interactivity offers more possibilities. A piece of music in a game may have to serve seven or eight different functions to be effective. In a film, what you see is what you get--it's a very static process once you determine what the underlying emotions need to be. (Soule, 2007)
As Jeremy Soule mentioned, games are non-linear in nature. This means they are not restricted to a timeline, like film and TV music are. Therefore, music has to be presented in a way that allows for this non-linearity so that it can accompany any situation that the player might be in. This can make composing music for video games challenging, and therefore the composer might have to think a little differently as Richard Jacques, a game composer explains.
It's not a traditional way of composing, you have to do think about it on so many levels and instead of looking at a linear piece of music like you would any TV or film, you have to think three dimensionally as well. So we have to think if we want to go from a fast cue to a slow cue, we can have a bridging section, transition, or we might have eight streams of music playing at the same time which we can fade between the two; one might be more action, one might be more calm. We had to pre-empt what the player is going to do, because in effect, they are the director, producer, and the writer. (Jacques, 2010).
Tom Salta, a game composer, also mentioned that games are constantly changing with the player. Therefore there is a lot to consider in terms of pacing, and how music will contribute to a variable timeline.
The difference of game music, is that the music has to adapt to what is going on. You watch a movie, that movie never changes. It's the same middle, beginning, and end, nothing changes. In a game, it can go a lot of different ways. It can shorten, lengthen, go here, here, here. There are a lot of technical considerations. (Salta, 2010)
Technology and game music
Technology has played a big role in how music is expressed today. In the past, music was limited to being produced by computer chips, which also gave it its characteristic sound. Now that music has progressed to being produced on sample mediums such as CDs and DVDs, this allows for a much more variety of sounds that can be used, and therefore a lot more can be expressed. For example, we now have the technology to fully utilise orchestral scores which can provide much more expressions, than computer chips can. "As the hardware changed we have been able to use lots of different timbres, and the composition tools have changed too, giving us a greater range of expression." (Kondo, 2010)
Technology has gotten to a point that it is possible for games to retain that big orchestral sound without having to actually record anything. This allows game music retain the expressiveness that an orchestra can provide, without having to bring in live musicians. The common reason why is that games may not have enough financially to afford to have real musicians come and do a live session. An example of this is the composer Doyle Donehoo who composed the hit strategy game "Dawn of War 2" using entirely virtual instruments in which his work for this game features big, heavy, orchestral sounds. "The DOW II music is mostly grand orchestral music with some non-traditional instruments mixed in for the Orcs. There were no live sessions. I did it all with my virtual orchestra and instruments." (Donehoo, 2009)
Although there is enough technology to make a virtual orchestra sound convincing, some game scores now feature real musicians playing real instruments, recorded in a studio. This gives a vibe about the music that is also used by the film industry, in terms of quality, expressiveness, and humanisation. This is also because of the fact that the commercial gaming industry, especially companies like EA, and Ubisoft can now afford it. Sean Murray explained this when he was composing the music to Call of Duty: World at War. "The orchestra gives an emotion that you don't usually get out of synth only pieces and it gives a power too, I mean there is something about having 90 musicians in same the room playing together and the ambience that builds up." (Murray. 2008)
Dynamic music in games
Since games have evolved over the past, so has the gameplay. Games are becoming increasingly much more complex and open in terms of choices and interactively. Therefore it is increasingly common for game music to adapt and react to the player's movements and choices through "dynamic music".
In the last 10 years alone, technology has evolved so much in games that we can utilise fully orchestral scores and choir, etc. But the main point about it is that we now can create scores that dynamically and reactively to the player's movements and choices, which is something that we couldn't really do in the past but that is what all composers are concentrating on now. (Jacques, 2010).
Dynamic music is when the music is design in such a way that allows the game engine to decide on when and how to use it. Dead Space for example is a survival horror shooter game makes use of dynamic music by splitting the music into layers that can be built upon, to target different levels of tension at a point.
The bulk of the score for Dead Space is dynamic - that's what is playing when you're exploring the ship or fighting Necromorphs. All the music was composed in four separate layers with different levels of tension assigned to each layer. The game engine reacts to the player's moves and decides which layers should be playing at any given point in the game. The variable mixes of music layers/tension allow for pretty seamless interaction between what happens on-screen and how the music sounds. (Graves, 2008)
Another game that uses this layer process is "Kane & Lynch: Dead Men" composed by Jesper Kyd who is known for the music of the "Hitman" series. It is another example of how music is becoming interactive with the player.
I wrote and produced the score, which consists of more than one hour of music. The score was divided into interactive music layers that respond to what you are doing. I would write a track that the team would approve, then I mixed in live elements, and the track was divided into layers so we could mix the music to the action onscreen. (Kyd, 2007).
Another method that achieves this is by splitting the music into sections instead of layers, which allows for the game engine to reorder or resequence the music instead of layering it. This technique was used by the composer Bear McCreary, who discussed this in his interview when composing for the game called "Dark Void".
What we're doing is that we're taking my compositions and we're cutting them up into individual little modules, say 15 to 20 seconds worth. Those modules then, during gameplay, can be reordered and resequenced. My goal is that the player can play the entire game through and never hear the exact piece of music twice (McCreary, 2009).
His theory is that the music stays fresh, and intense by having the music adaptable, to make the player always unfamiliar with the music to make it tense as "your mind can't get comfortable with the music". (McCreary, 2009)
Practical game examples
Super Mario Galaxy (2007) and its sequel Super Mario Galaxy 2 (2010), is a game known for its platforming gameplay, but also for its use of music. Since the Super Mario series has a rich history in terms of music, it is used in this game to not only accompany the game, but to also to give a sense of nostalgia to existing players who have played these games in the past. It also takes advantage of the current technology and improves upon these classic tracks. A gamer reviewer for Gamespot commented on this:
The incredible action is tied together beautifully by the visuals and music... But the music is even more impressive. Using a great combination of classics and new material, all of the music fits the action perfectly. The older songs have been remixed, and what's most interesting is how they sound even better than they did before. For instance, the haunting theme from Bowser's levels that first appeared in Super Mario 64 now has a choir singing the higher sections, and it adds an eerie note that will send shivers down your spine. (Mc Shea, 2010).
Sound also plays an important role on the actual gameplay, which in this case serves as a notification on certain actions the player need to take, and provide an accompaniment to the action as this Gametrailer review for the game commented:
The audio is a real star of the presentation. Yeah, you get some great orchestrated classic Mario tunes, but sound cues have been implemented into the gameplay to tell you when to jump in some levels, and instrumental will solo into each track when the action heats up. (Gametrailers, 2010).
Driving games are an example of how music for games can adapt to the environment. Tunnels are a prime example. A game called "Split/Second" makes use of this, and actually muffles the music whenever you drive through a tunnel to immerse you into the game. "The soundtrack is a dramatic techno symphony at times reminiscent of early burnout games. Music becomes muffled when you go through tunnels, and phases out to highlight wrecks, while the constant explosions keep your living room rocking." (Gametrailers, 2010)
The game "Modern Warfare 2" is a popular first person shooter game which its singleplayer campaign takes place at a variety of locations including places like Brazil, Russia, and America. Composer Hans Zimmer scored the music for the game in which he had to take these locations into consideration to help in immersing the experience to that region through the use of instrumentation. "It was easy to let the locales dictate the instruments. When we are in Brazil we are more Brazilian and when we are in Russia, we are more Russian." (Zimmer, 2009)
Bioshock 2, is a game in which it is set in an alternate 1968 underwater world in which has led to corruption. Therefore, the game's style is creepy, mysterious, and old fashioned. One of the ways to achieve this was by music. Garry Schyman's music, in which he composed for the game, played a big role in creating the atmosphere and environment. An interviewer, who had played the game really thought that music did contribute to the game's creepy atmosphere that the composer and developer were going for.
I had moments in the first game where Garry's pieces, frankly, would stress the shit out of me. There were climatic moments where there was a little too much to handle, where you'd get crowded with enemies, and then the music got all frantic, and my heart started beating faster, and I couldn't focus correctly. And when I died, I blamed the music for screwing me up. (North , 2010)
Another element that contributes is the old fashioned music, that wasn't composed by Schyman, but rather a choice made by the developers to establish the 1968 era. This included using music that was released between 1929 to 1952. "Of course, the other side of what makes BioShock musically is the old music." (North , 2010) "It really is part of what makes Rapture and that time period come alive. It really puts people back in that space. It's very shorthand for putting people back in that time." (Kamper, 2010)
Conclusion
Music contributes to the experience of a game. Orchestral music is now possible in games thanks to the advanced in technology, as well as contributing more to the concept of game music and its contribution. Dynamic music is becoming common which allows for a different approach to how music is used in games. Music for games is a unique topic, as it is distinctly different from its film and TV counterpart. Video game music will keep on expanding in the future as the game industry grows and becomes more and more recognised.
References
artificialmind101, 2009. Frag Dolls Interview with Composer, Tom Salta. Viewed 1 June 2010.
http://www.youtube.com/watch?v=g_fkbwz5y34
Cackowski-Schnell, B. Interview with Red Steel 2 Composer Tom Salta. Viewed 1 June 2010.
http://www.gameshark.com/features/689/p_0/Interview-with-Red-Steel-2-Composer-Tom-Salta-.htm
CCowell125, 2010. Richard Jacques Interview on BBC Radio 2. Viewed 1 June 2010.
http://www.youtube.com/watch?v=uEZTsg2WtaI
Destructoid. 2010. Destructoid interview: The music of BioShock 2. Viewed 7 April 2010
http://www.destructoid.com/destructoid-interview-the-music-of-bioshock-2-162931.phtml
Game-Ost, 2009. Interview with Doyle W. Donehoo - Warhammer 40K: Dawn of War II soundtrack composer. Viewed 29 June 2010.
http://www.game-ost.com/articles.php?id=57&action=view
Games, 2009. Dark Void - Bear McCreary Composer Interview. Viewed 29 June 2010.
http://www.youtube.com/watch?v=tWUz7077hxo
Game-Ost, 2008. Interview with Jason Graves - Dead Space soundtrack composer. Viewed 29 June 2010.
http://www.game-ost.com/articles.php?id=45&action=view
Gamespot, 2007. Q&A: Game music composer Jeremy Soule. Viewed 3 July 2010
http://au.gamespot.com/news/6172544.html
Gametrailers, 2006. E3 2K6 GT Interview 2. Viewed 1 June 2010.
http://www.gametrailers.com/video/e3-2k6-red-steel/10890
Gametrailers, 2007. PS3 Developer Diary - Music in Games. Viewed 1 June 2010.
http://www.gametrailers.com/video/ps3-developer-graw-2/23085
Gametrailers, 2010. Composer Interview. Viewed 1 June 2010.
http://www.gametrailers.com/video/composer-interview-call-of/42424
Gametrailers, 2010. Split/Second Review. Viewed 1 June 2010.
http://www.gametrailers.com/video/review-split-second/100330
Gametrailers, 2010. Super Mario Galaxy 2 Review. Viewed 1 June 2010.
http://www.gametrailers.com/video/review-super-mario/100404
Mc Shea,T. 2010. Super Mario Galaxy 2 Review. Viewed 1 June 2010.
http://au.gamespot.com/wii/action/supermariogalaxy2/video/6263329/super-mario-galaxy-2-video-review
NintenDaanNC, 2010. [NC UK] Koji Kondo Interview. Viewed 1 June 2010.
http://www.youtube.com/watch?v=5ilJCerCucA
North, D. 2010. Destructoid interview: The music of BioShock 2. Viewed 1 June 2010.
http://www.destructoid.com/destructoid-interview-the-music-of-bioshock-2-162931.phtml
Snider, M. 2009. Interview: 'Modern Warfare 2' composer Hans Zimmer. Viewed 17 May 2010
http://content.usatoday.com/communities/gamehunters/post/2009/11/qa-with-modern-warfare-2-composer-hans-zimmer/1
Square Enix Muisc, 2010. Interview with Tom Salta (April 2010). Viewed 17 May 2010
http://www.squareenixmusic.com/features/interviews/tomsalta.shtml
