IV. Sound Design Production

A. Emotional Response to Sound

From the beginning we have been programmed to respond to sound. A mother's voice, a church bell, or police sirens conjure an emotional response.  Sounds help us to decipher the world around us. They warn us of danger, call us to action and bring peace and tranquility to our lives.  The more expressive the sound is, the greater our emotional response to it.  Sound effects correctly placed in a game should evoke this response while defining the environment, circumstance and personas on screen.   Due to the random nature by which sounds are triggered in a game, they must effectively co-exist without losing definition or character when multiple sounds occur in close proximity to each other. Let us examine some general observations in game sound design.

B. Beware of Sonic Sludge!

There is a finite amount of sound data that the ear can properly interpret before fatigue sets in. It is the role of the sound programmer or director to prioritize which sounds are most important and at what times they are important. The sound designer on the other hand, must always create content that will be effective, regardless of the circumstances that exist at the time a sound is played. Good sound effects should work well alone and in combination with many other sounds. This is a challenging task, but careful forethought and planning will produce a rich, dynamic and satisfying interactive soundscape.

The key to preventing sonic fatigue is to create sound effects that vary in volume and frequency in relation to each other. A single sound effect that is loud and contains equal amounts of low, middle and high frequencies may be effective when played alone, but if all the sound effects are loud and contain a similar frequency spectrum, it becomes difficult to decipher one sound from the next.

In most cases, the sound designer delivers the sounds at a reasonably loud volume, to allow the audio director or programmer to appropriately mix those sounds into the game, setting the playback volume for each sound. However, it is the job of the sound designer to emphasize different frequencies according to the requirements of each sound. To do this, the designer must know which sounds are likely to be played together at any given time, then selectively decide which sounds will emphasize specific frequencies. Higher frequencies provide detail. Upper middle frequencies provide presence, while lower frequencies depict power or energy. Too much emphasis on high and upper-middle frequencies will lead to fatigue, while too many sounds containing lower or sub frequencies, will become muddy and detract from the overall detail of the sound design.  The goal is to create individual sounds that do not compete, but compliment. With this in mind, the sound designer must appropriately focus on the frequencies that will best suit each sound effect. This process essentially carves out any unnecessary sound space to allow additional room for other sound effects to be heard. When volumes and frequencies are selectively assigned, the sound effects will breathe and compliment each other regardless of when they play.

C. Pacing - Building Toward Climactic Moments

Now let's examine the sound design from the "Big Picture" perspective.  Game and level design documents will provide the structure of the game in terms of moments of emphasis. Generally, these structures take the form of peaks and valleys that convey changes in difficulty as the game progresses. Usually, the peaks represent a boss fight, though not necessarily so. When examined as a whole, the sound design should appropriately compliment these arching structures, and allow, from a sound perspective, a sense of building toward these peak moments. If the sound designer has examined the enemies and situations thoroughly, the overall sound design will naturally fall into place, appropriately following the peaks and valleys within the game. However, if for example, minions sound as powerful as bosses, some adjustment will be necessary to bring down the emphasis of these weaker and less difficult enemies. By not doing so will result in sound design that does not match the arching pattern of the game. To put it simply, there can be "too much of a good thing".  Let's now look at the specific areas of game sound design.

D. Ambience - Defining Environments Through Sound

Initially, ambient sound should effectively portray the setting, location and time frame of the game or its various levels. For instance, percussion and double reed music, a multitude of bartering voices and distant clanking iron would suggest a medieval marketplace. As the game progresses the role of the ambient sound is to support the circumstances with which the player is involved. Does the sound within the environment evoke danger or safety? Activity or inactivity? Conversely, ambience can be used to deceive the player through suggesting a false circumstance, such as creating a sense of calm before an ambush. Under all these conditions, good ambient sound should portray a living environment.

The psychological impact of ambient sounds can add much to the onscreen imagery, though not physically present in the scenery. For instance a distant, sustained cry of an infant suggests vulnerability or insecurity. A broken fence rattling in the wind of an abandoned city, suggests to the player a previous traumatic event. These are subtle examples used to arouse awareness in the player. More obvious sounds should be used to cue the player of his direct proximity to danger. Dark drones or muffled enemy vocalizations will prepare the player for fierce combat ahead. Fear, anticipation and anxiety are easily evoked by the careful placement of ambient sounds.

E. Impacts and Destruction - Breathing Death into the Non-living

Early on, comic books depicted the sound of the action scenes through the use of words that sonically mimicked the action. Over time, words like "thud" "pow" and "zap" lost their effectiveness. Comic book writers had to jog their imaginations to express sounds in more creative and exciting ways, such as "Kathwaaap', "fwuuuhmp" and so on. Similarly, the sound effects in early games experienced a renaissance as memory increased and streaming technology allowed for more and varied sounds to be launched under the animations. However, no increase in playback performance will ensure the effectiveness of the sound effects, if the sounds are not expressive.

From a sound perspective, impacts and destruction must primarily convey suffering and submission. These terms apply naturally to the vocal efforts triggered under an opponent or avatar under attack, but are more abstract when applied to inanimate objects. Since the human voice is the most expressive instrument in existence, applying human-like characteristics to the  'non-living', will help give the sounds a more life-like and expressive quality. Twisting, screeching metal, the deep thud and release of broken concrete and wood that creaks, pops and splinters convey expressive responses to the forces applied to them, in much the same way a grunt, moan and exhale expresses human injury.

Additionally, impacts and destruction sounds should proportionately depict the transference of energy between the weapon and the target.  A metallic ping with a ricochet is an effective response to a bullet on metal, in which the transfer of energy between a low-mass object at high speed can be observed. A missile explosion, on the other hand, is more powerful and slower to develop, therefore requiring an equally proportionate response. The sound of larger impacts with destruction should develop through three basic phases:  Attack, Sustain and Release.

The Attack is the first and shortest event of the three. It is important to note that the attack is not the sound of the weapon or projectile. In this case, a missile, contains it's own dry explosion sound that is launched under the animation of the missile explosion. Therefore the attack will be the impact sound based on the material composition of the target. Since the attack and the dry explosion of the missile will happen simultaneously, the attack should have a short period of 'lead-in' or silence to allow the peak, or initial part of the explosion of the missile to be heard uncompromised by the attack of the material impact.

Next is the Sustain, which introduces the debris and material breakdown created by the explosion. Over this phase, detail should be observed. The sustain should sound less dense than the attack so that the specific details of the destruction can adequately be heard.

The final phase is the Release, which is a response to the destruction that should characterize a kind of 'submission'. This phase of the destruction should contain lighter falling debris based on the materials destroyed, movement of dust and earth and perhaps steam.

When all three of these phases are exhibited, the destructions will sound more expressive and compliment the weapons by adequately portraying their explosive energy.

For "The Incredible Hulk - Ultimate Destruction" we maximized the detail and movement of large, explosive forces by dynamically altering the stereo field throughout the three phases of the destruction. The attack phase was almost entirely monophonic, while a quickly widening stereo field was applied to the sustain, finally resting on a wide and fixed stereo field for the release. The result was destruction that moved rapidly over a wide area, thereby adequately portraying the Hulk's enormous power.

F.  Weapons - Know Thyself, Know Thy Enemy!

It is a lesser-known fact that a gunshot at close range, sounds less threatening than from 40 or even 80 yards away. Since most people have never fired a gun, their expectations for the sound of gunshots as depicted by the entertainment media are very high. Therefore, even in games based on historical simulation, some amount of sonic sweetening will be necessary. In the case of a "period" war game, multiple recordings of the specific weapon should be blended together to create a satisfying gunshot. These might include mixing together the various distances recorded for the gunshot, as well as the dry trigger and shell discharge sounds for the specific firearm. Sounds created this way will be sonically interesting while retaining the historical accuracy of the weapon.

For science-fiction or fantasy games, the imagination is the sound designer's only limitation. As mentioned previously, the design documents will shed light on the abilities of the enemies and characters within the game. The weapons detailed in this document should explain the amount of damage incurred by each weapon. It is important that these sounds appropriately match the damage potential, since the player will, to some extent, be judging the amount of damage from each weapon by the sound it creates. For example, weapons that contain a charge-up sound before firing, indicates to the player that a great amount of force is forthcoming. Likewise a weapon that produces a large discharge noise would produce the same result.

From a stylistic perspective, weapons are an extension of the personalities of each character and should compliment the character's physical attributes, abilities and in some cases, their heritage or history. For instance, the sounds of swords, knives and shuriken should be as stealthy as the master ninja who wields them. The character of these sounds should compliment the physical qualities exhibited by the ninja and reflect the mastery of the ninja tradition. With this in mind you should expect the sounds to be light but fierce, focused and evoke quickness of movement.

G. Vehicles

Since vehicle sounds typically respond to controller movements, and not animations, they can be difficult to perform in a plausible manner.  Developers for racing games are likely to have robust code for manipulating vehicle sounds. Since we are focusing on sound production, and not programming, let's examine the basic elements that make up vehicle sounds.

In most cases the sound designer will provide four separate engine sounds per vehicle: an idle loop, acceleration, a steady thrust loop and a deceleration (engine decompression or braking). The idle will simply indicate that the vehicle is engaged. The acceleration and deceleration sounds should be designed to seamlessly crossfade into, and out of the steady thrust loop via programming. This formula is effective for simple vehicles with a low threshold of speed in which the vehicle will quickly reach maximum velocity until the button or trigger is released.

If the visual perspectives of the vehicle can be changed, so too should the sounds that accompany the vehicle. This will ensure a greater sense of realism. For instance, if inside and outside perspectives are available, subtle shifts in the observed engine sounds should be present to support the change in perspective. An inside perspective will result in a de-emphasis of the higher frequencies that are present within the engine sounds, giving those sounds the muffled quality one would expect when listening to the engine from inside. One way to perform this, is for the sound designer to supply separate versions of the engine sounds based on the perspective observed. If the sound designer has access to recordings from the various perspectives, this will be easy to supply.  However if these sound perspectives are not available, or if the vehicle is fictitious, separate mixes that include changes in equalization should be performed in order to support the visual perspectives.

For added realism, intermittent sounds can be supplied to add feedback based on the driving conditions or the state of the vehicle while operating. For instance, wheel-based vehicles will contain surface noises used to indicate the terrain (tarmac, gravel etc.). Metallic rattling and scraping is used to indicate the state of a vehicle that is damaged. The addition of these and other intermittent sounds add a heightened sense of realism and immersion when operating the vehicle.

H. Menus - Less is More

As games have become more sophisticated, so too have the menus.  Players can customize a variety of options as well as view or purchase an array of unlock-able content. This, of course requires more navigation. In most cases, sounds will accompany the navigation to provide greater sensory feedback. No matter how enjoyable these sounds may be, their repetition will soon become an annoyance. It is always safe to create short and subtle sonic events to accompany the menu navigation, so that the sound is felt rather than heard. This way the player can benefit from the additional sensory feedback, without being discouraged by the mere repetition of the sounds.

Music Production