Sunday, January 9, 2011

Sound and music.

When it comes to media, there's really only two senses that can be represented: sight and sound. Of these, we've largely talked about the former: making things look accurate and look plausible through things like costume design and accessories. Sound, despite being the other 50% of presentation, has been largely untouched. This is because sound design is a lot more subtle than visual design - while visual design is right there in the open to be observed, sound design often has a much lower level of active recognition from the audience. In some cases, you might not even notice it's there - at least not actively. Sound in media expresses itself in a few different ways.

Probably the most direct, overt sound influence, music can be broken up into two groups for purposes of believability: "diegetic" and "non-diegetic". This refers to the nature of the music; diegetic music exists in-universe (coming from a source within the film itself), while non-diegetic music is instead layered over the film with no source. For reasons of plausibility and connection to the characters, the difference is this: the former is part of the immersion, and the latter is part of the "audience experience".

Now let me establish this up front: non-diegetic music in movies and games can do a lot of great things. Music helps presentation in a lot of ways. It can evoke emotions and add a lot of powerful influence to a scene. What music does is reinforce the feelings that the audience are meant to be feeling, from hope to fear, or from awe to terror. The use of leitmotifs can help the audience identify a given character, group, or event. In games, music can help to reinforce the gameplay and make the experience feel more epic and important, whether that be for good or for ill. This is, overall, what non-diegetic music does: it creates a powerful, moving performance in a way that is often not justifiable in-universe.

However, it's also important to note that effects just as dramatic (if not more so) can come from in-universe music, as well. The most obvious example, in my mind, comes from the final scene of Zulu. The nature of the music is still inspiring and impressive, but it exists in a way that's justified and explained in-universe. In addition, what the audience is experiencing is also what the characters is experiencing. Music stirs emotions, yes, but in this case it stirs the emotions of the characters as well as the audience. The presence of music that exists in-universe can cause us to share those emotions with the characters.

The lack of music can also paint a very stark picture. Music's positive effects are also very manipulative, in the same way that a propaganda film is. Music takes an event and frames it in a way according to the emotional theme of the music. Compare the following two versions of a battle from Band of Brothers: with and without music. Or compare two versions of Omaha Beach, again with and without music (or even with inappropriate music). Music affects the emotional tone of a scene, and thus its presence can reinforce or distract from the default audience reaction to the scene.

Non-diegetic music is dishonest in one major way: it can add emotions to a scene that wouldn't have them otherwise. The battle scenes before, which were grim and overwhelming without music, are instead noble and emotionally moving in ways that the soldiers themselves would not have felt. This creates a divide between the audience, who feels those emotions, and the characters, who don't. It's comparable to the phenomenon of singing or reacting to songs that only you can hear (through headphones, for example). In short: music stirs emotions, but if the characters can't hear the music, there's going to be an emotional divide for the audience and characters.

Environmental Sounds
In most environments, there are a lot of sounds that exist in the background, suggesting a scale larger than the immediate scene. In urban environments, the ever-present buzz of people and vehicles creates the feeling that the scene actually takes place in a city, rather than a small set. In a rural setting, the same effect can be created with appropriate sounds like the chirping of birds or the operation of farm equipment. In addition, things like thunderstorms can have an emotional effect as well. As with rational character design, a lot of little stuff adds up: there's a source for every sound, and that source exists somewhere in-universe. This can create depth and make the world feel more "real".

The use of "stock" sound effects, on the other hand, can point out the falseness of the world. When was the last time the Wilhelm Scream contributed positively to a scene? At this so point it's so familiar, and thus so fake, that it immediately draws the audience out of the scene in exchange for, at best, a few laughs of recognition. The human voice, specifically, can create this kind of problem, because each person's voice is different in subtle ways. In Metal Gear Solid 4, the use of stock screams (over constantly occurring background deaths) was enough to become distracting, and made the world feel extremely fake.

The goal of environmental sounds should be to convince the audience that there is a real world in the background, with things going on that are actively causing these noises. This can add depth to a world that, for all intents and purposes, is a relatively small stage with a focused, central set of actors. If the sounds are either not justifiable or too repetitive, the fakeness of the world takes center stage, and the immersion bonus that the audience should feel is lost (if not worsened due to the jarring break).

One subsection of background noises is, naturally, gun noises. Like other background noises, the noises of the battlefield exist to show that there's other things going on, namely other battles. In the sense that "every sound has a source", though, combat noises can also be used to illustrate the fact that there is combat, which becomes important when it's the default assumption that there isn't. To return to Metal Gear Solid 4, in online mode there are two contrasting forms of match: "ongoing battles" and "find the enemy". In the former, gunfire is an assumed noise - there's always someone shooting someone somewhere on the map. In the latter, gunfire is an anomaly, meaning that someone's been spotted, and all players should react accordingly.

In real life, guns make fairly distinctive sounds. In combat, the difference between "the sound of friendly guns" and "the sound of enemy guns" is just as important as the color of their uniforms. This is why the practice of using an enemy weapon is frowned upon in real life - because it still sounds hostile, even if it's shot by someone who isn't. The sounds of artillery firing, and where it's firing from, can determine if someone is "safe" or whether they should take cover. Few games take advantage of these concepts. Company of Heroes is one exception - the different machine guns and rifles make different enough noises that you can tell what's going on or where they're coming from.

One thing that annoyed me about Killzone was that while each faction had their own main rifle (the M82 for the ISA, and the StA52 for the Helghast), there was no effort to actually regulate these guns in multiplayer. There was no reason for the ISA to use their gun exclusively, and vice versa. This was done for gameplay purposes (the different guns have different uses), but resulted in gun noises being basically useless. Being able to recognize gun noises would have helped determine if you were, for example, coming up on a group of friendlies. Removing this meant that the sounds of gunfire were just sort of a "background mess" - they were there, but you couldn't get any useful information from them.

Of course, guns aren't the only things that make sounds in combat. Identifying enemy locations from the sound of footsteps or vehicle noises is a time-honored tradition. In Metal Gear Solid 3, the player is equipped with a directional microphone to aid in this task - often necessary due to camouflaged enemies blending into the background. While vehicle noises exist in Company of Heroes (and can be heard through the fog), one thing I appreciated about the Eastern Front mod was the distinctive noise of Russian tanks - a loud, obvious "clacking" noise. This was interesting to me because it wasn't just a generic "there's a vehicle here" sound - you could specifically tell that it was a Russian tank, rather than a truck or something.

There's a lot more that could be said about sound, but it should be said in individual, focused updates. To sum up, though:
1) Music evokes emotion. If that emotion can be shared with the characters, then all the better.
2) Sound has a source. Using sound in the background indicates that whatever caused that sound also exists in the background. This can make the world more believable in terms of background events.
3) The importance of sound as an identifier should not be underestimated. Many games leave sound in the background as an unimportant layer of white noise, rather than having them be an important tactical element (as they are in real life).

No comments:

Post a Comment