Lesson 3: FMOD Control
How To Think Like A Time Lord, And Other Useful Tips For Everyday Sound Design
Welcome to my third lesson on the Audio Middleware Engine known as FMOD. If you’re new here, jump on back to week 1 by clicking here to get the basics down. This week will deal with how to further control FMOD events using the Logic tracks. It bears repeating the analogies I’ve been making (that are hopefully apt!): Everything in FMOD is an Event that details something. Parameters are sort of like adjectives. I don’t have any parts of speech up my sleeve to describe the Logic track, but if Parameter’s provides a description about an event, then the Logic control describes when it happens in time, and how often. Remember how the Timeline in FMOD is just another Parameter, as we covered in the second lesson? Did knowing that bother you a little bit last week? As a Parameter, shouldn’t we have some sort of control over it, like the rest of the Game Parameters we can create? Last week’s lesson dealt primarily with controlling events which spanned just one single scenario. For instance, explosions were the primary example, and while we were able to create a nearly infinite amount of variations of that explosion, they’re only good for whenever you have…well…an explosion happening. While useful, our game will also have events (such as music) that need to work to move fluidly back and forth between different states and levels of action. This is most tidily accomplished by skipping around the Timeline of your events, sort of like skipping back and forth between tracks on an album to suit your mood. The good news is that FMOD does allow you to control the Timeline Parameter. The bad news is that just letting you run wild by stopping, rewinding, and skipping around in time at will would create paradoxes and could literally ruin the space time continuum and tear the fabric of space and time itself…it’s just a LOT of responsibility, for even someone so well disposed as a sound designer. But you DO get some tools. And this week, I’m going to focus on explaining the concepts behind how you can utilize the timeline itself to offer some more advanced and complex control of how the game deals with events that span that can span many different kinds of scenarios (like, say, footsteps, which can happen on dirt, gravel, wood flooring, carpet, etc.) or single, constant events that need to react fluidly depending upon a scenario (for example, music tracks which react to the parameters of the game.)
So, read on to continue, and as always – if you have any questions, require further explanations, or wish to suggest further topics, email me at Hello@ChrisPrunotto.com or reach out to me on twitter @SoundGuyChris!
Concept: Using Logic Control tracks, you can use a single event for a wider array of related purposes. Just like with Parameters making the most out of samples to cover a wide range of scenarios, you can also use Logic to make the most out of single events to cover a wide range of application.
All these “Control” functions live on what is called the “Logic Track.” If you create a new event, you’ll notice not far off to the left from your Timeline Parameter Tab (remember, the Timeline is a parameter, just like any other game parameter you can create!), you’ll see a “Logic Tracks” label and an arrow to expand or retract the Logic Tracks display. Be default, it will already be expanded, but if it isn’t just give it a click to make the arrow point down. You should now notice a black band running across your Event Editor window, just below the time stamps. This black band is the Logic Track – easy to miss if you weren’t looking for it, but it’s the only way access the tools I’ll soon be describing.
Before I begin explaining the parts of the Logic Track, let’s just understand what it can do and what functionality it offers us. Imagine this rather uncommon scene from a video game: Your protagonist is entering a village for the very first time. The ambient music is playing – not much more than a few pads over a general loop, and everything is placid as can be. A villager calls over and your hero approaches. The music remains how it is, but as you approach, a few flutes work into the music. Questions and inquisitiveness arise. Suddenly, the villager exclaims, in true Admiral Ackbar fashion “IT’S A TRAP!” and you find you’re surrounded by ten armed guards. Instantly, the soundtrack explodes into a full bombastic orchestra. The pads are still at the bottom, but they’re buried under the now-pounding timpani section and the horns and everything is just absolute chaos on screen as your hero is battling for his life. He eventually becomes critically wounded, and so the game suddenly reacts. Everything becomes silent except for the pounding of the drums – just those timpani beats blasting away in your ears – and they are louder than ever before, mind you, as if they were your very heartbeat. Given a few moments you return to regular health and the music returns, but the odds are now closer to in your favor. There are only three guards left of the ten and even the captain has been killed. The music starts to slow from the frenetic pace from before and begins to take on a more victorious, triumphant feel. The melody starts to soar. It’s soon just you and one last guard. He doesn’t stand a chance and the music is fully reflecting the fact that the power lies in your hands now. He soon falls, and the music soon returns to its original ambient state.
This is the essence of the logic control track. You see it in more video games than ever before, and with good reason: It allows for the music to be just as reactive and dynamic as the sound effects. It allows for just as much control and to give the player so much more immersion than we as sound designers have ever been able to give before.
How Do We Build This System?
Naturally, with some logic tracks, and with some parameters! Now, there are two basic paradigms here with which one can utilize when composing interactive music. They both offer distinct advantages and drawbacks.
Concept: It bears repeating that it is very important to plan out your overall approach to how music and sound effects will be handled. Though there are countless ways of approaching any one problem, the best results will always come from using your tools to sculpt solutions, rather than smoothing out problems and inconsistencies.
The first is what I consider the horizontal approach to interactive music. The horizontal approach is best suited when the consequences of the parameter affecting the system being designed are well known in advance. For example, if you know that the piece being written will be used in battle, and will go from placid, to bombastic, to triumphant, and back – especially if these things generally happen in generally a predetermined, specific order, you can write the piece as one long song. Then, using the control tracks you can simply loop the given sections using the Logic Track as necessary, and then transition to the next section when appropriate. For example, when there are no enemies in range, it’s quite simple to just have the first thirty bars loop, and then jump or transition to the next section as needed. The horizontal approach is, in my opinion, at its absolute most effective when the order in which the sections of the piece are generally the same order that they will play back in. For example, it’d be hard to imagine the aforementioned battle theme needing to go from the placid theme to the triumphant one without passing through the chaotic, bombastic one. It generally has a given flow to it. The advantage to the horizontal approach is that it’s usually easier for traditional composers to write for. The music recordings will also be as close as possible to a perfectly polished and mixed piece. In other words, it’s often easier to achieve a somewhat higher standard to the music recordings when things are a little less linear. The composer is somewhat more able to add flourishes and intricate melodies, knowing that a sudden change in the game environment will not leave certain elements of the composition naked and out of context. The drawback is that, of course, the results are somewhat less deep interactivity and feedback, and can feel somewhat more reactionary. Games such as God of War and Assassins Creed are particularly notable for their inspiring and powerful use of the horizontal approach to interactive music.
The second paradigm is the vertical one. The vertical approach is best suited to situations where many parameters might influence any given part of the composition. Given that, looking at the timeline in FMOD for a project using the vertical approach to music, you might very well notice far fewer markers and transition in the Logic Track and far more audio tracks and the parameters being used to control the way the game and music sounds. For example, in our battle scene previously described, there were two things which affected the overall music the player was hearing: the first was the amount of enemies near him – from ten, down to zero. That parameter mainly influenced the mood of the music. When there were a lot of enemies, the score entered that chaotic and high-energy zone. Instruments were added in on top of the basic pattern. A few automations might have guided the levels of the mix to emphasize the disorder and fury on screen. As the amount of enemies dropped, so too did the mix change to alter the feeling of the game, eventually to reflect the power balance shifting in favor of the player. The second parameter was the health of the player character. When it drops to critical, it would cut all the instruments out of the mix while leaving just the timpani drums exposed to hammer away. The advantage to this approach is that it can offer a much more dynamic and reactive experience. One in which the player is emotionally, if not physically, always connected to the environment. It allows for some more creativity in terms of the structure of the composition and allows the more adventurous composer to often leave their comfort zone. The biggest drawback, of course, is that gamers are notoriously good at being unpredictable. Ninety-nine percent of players will make it through our imaginary battle just fine after the first three or four enemies fall, but one player might survive nine guards unscathed and then get taken out by the lone survivor. The biggest drawback then is that sometimes very specific scenarios may occur that the composer or sound designer did not entirely intend to happen…but oh yes, they must be accounted for! It’s hard for me to reconcile attempting to extol the virtues of interactive audio, while simultaneously telling those of you reading this lesson to be careful about just how much interactivity you put into a project, but the fact remains that scope creep can happen in audio too. If it gets too complex and unwieldy, you could very wind up losing the dynamic feedback that interactive audio is supposed to convey. Your final mix absolutely should work in every scenario, and so a good deal of time must be spent caring for each and every consideration. As your music system develops its layers of intricacy, one of the most important things to do is to be constantly checking if the events still work properly under every scenario.
Of course, keep in mind that while some games utilize one of these two approaches much more obviously than some others do, almost all games with interactive music and sound utilize both approaches to at least some degree. It’s also very important to realize that it is VERY much possible to create nearly the same soundtrack using exclusively one method or the other. The true choice is flexibility. There are no wrong answers, only what makes the most sense to you. One must always remember that giving the player feedback is an extremely important, if not the most important function, that sound design serves. Every creative decision should serve to underpin and strengthen the player experience. Neither the horizontal, nor the vertical, approach is limited to just the soundtrack of the game. Much like how parameters can offer the ability to create a complete palette of similar flavors of a given event, control logic allows you to choose from a complete palette of feelings and emotions within a given event.
Tools For Jumping Through Time
For such a powerful tool, there are actually pretty few parts to the Logic Track, and then there’s a few little bits that help extend its functionality further. Just as with regular modules, you can add them by right clicking on the logic track and selecting add the appropriate region or marker. From the list you have you have a few options:
- Add Marker – A Marker acts as a landmark, or street sign, and does not affect playback directly. A marker is instead used as a flag for other logic tools to designate important points to which the cursor will return to when it contacts a Transition marker or region. You can drop a marker anywhere on the timeline, and rename it to something appropriate by double-clicking.
- Add Tempo Marker – A Tempo Marker defines the musical tempo applied to the Timeline. When you place one, you will notice that the timestamps above the logic track changes from readings in minutes and seconds into bars and beats. A single Tempo Marker acts as a switch from regular time into musical time. It does not affect the rate at which the cursor moves, but simply calculates the amount of time each beat and bar takes, and allows quantization of the timeline for snapping modules into musical time. You will notice that if you lay down a second Tempo Marker, it will ignore all snapping, since it is a new tempo calculation it can be placed freely (As an aside, if one wanted to ignore quantization snapping with ANY module, simply hold the ctrl button while dragging.). In order to change the tempo, simply double click the Tempo Marker and write in a valid tempo and time signature. The time signature is written as a fraction, with the number of beats per bar first, and the note value of a single beat as the second number (eg: 120 beats per minute in three four time is written 120 3/4). The maximum value for the tempo is 999, and the maximum values for the time signature are 99 for the number of beats per bar (the first number in the fraction), and 16 as the note value for one beat. Because FMOD does not actually write music with MIDI or record audio, the tempo marker is simply for calculating quantization. In particular the second value of the time signature, in essence, defines the resolution of snapping. In other words, setting the tempo allows you to refine the accuracy of FMOD snapping your modules together in musical time, but your samples should already be cut to fit those snaps.
- Add Loop Region – Perhaps the most familiar tool in this arsenal to people who use traditional D.A.W.s is the Loop Region. As the name implies, if the cursor enters the region from the left, and makes it all the way to the right end of the marker, instead of continuing on outside of the region, it will return to the beginning and play repeatedly again until something forces it to leave the loop. Additionally, there are two logic modifiers here: You can add an optional Parameter Condition to the logic area of the region’s deck that forces it only to loop only while a certain parameter is within a given range, or have it repeat the loop only a percentage of the time using the die toggle and Probability knob.
- Add Sustain Point – A sustain point suspends the advancement of the cursor until explicitly told to move forward. Upon creating a Sustain Point a label called “Cues” and a button labeled “A” will appear near your Parameter controls. When the Cursor is paused by the sustain point, clicking that button will allow it to advance. Its usage might seem limited at first but it can be used in many different ways to reduce the amount of needless repetition in an event. For example, when creating a vehicle engine sound, one can place the audio file of a car engine starting and shutting down directly onto the event and then have a middle sound for idling
- Add Transition / Add Transition To… – These options create points on the timeline which redirect the cursor to the locations of other markers. When creating a transition, you can create select the Add Transition To… option in order to specify an already-existing Marker, or create a blank transition and then decide which marker to use later by right clicking it and setting a destination. The cursor will make the jump to the destination as soon as it comes into contact with the Transition Marker, unless a Transition Timeline is present, which will be discussed very soon.
- Add Transition Region / Add Transition Region To… – These options are very similar to the Transition Marker, but instead create an entire region. It includes all the same functionality as the Transition Marker, but is extended over many points instead of one. It’s important to note that the transition takes place as soon as the cursor comes into contact with the region. It is not possible to use a Transition Region in place of a Loop Region. The Transition will happen the moment the cursor enters or is otherwise placed inside the Transition Region, unless either the optional Parameter Condition or Quantization tools are being used (both of which will be discussed shortly!)
The first thing that comes to mind with many of these tools especially is that they would be designed for the facilitation of interactive musical works, but one of the overarching paradigms of FMOD is to blur the distinction between music and sound design, and to create one streamlined workflow for all game audio. To that end, it’s important to remember that there is never a wrong answer in FMOD so long as the solution is satisfactory. Do not be afraid to use “music” tools with your sound design and vice versa. However, note that logic track tools can only be added to the Timeline. Because Parameters have arbitrary values that can be manipulated in code, logic track markers on game parameters are unnecessary and therefore can only be used on the Timeline. These six logic functions create the foundations of the logic track and time manipulation in FMOD.
Now In Stores…The Optional “Time Lord Accessories Kit!”
While the previously discussed tools are all useful enough on their own, using them in conjunction with a few more tools will really help you make the absolute most of your events. I mentioned a few above. One of the most important of these remaining tools is the Trigger Behaviors tray that you may have noticed by now. It pops up on all of the sound modules, and a few parts of it pop up independently on some of the Logic Track tools. Much like the Modulation tray, it’s accessed by the little white arrow labeled “Trigger Behavior” on the left-hand side of the Deck.
The Trigger Behaviors tray offers four sections which can be used to help shape the way a given event plays back. From left to right, these sections are Logic, Delay & Quantization, Seek, and Polyphony:
- The Logic area of the tray defines the actual probability of a sound module being triggered. It is the first step which decides when a voice is allowed to play. By right clicking over the white “+” symbol, you can add an optional Parameter Condition. If the Parameter Condition is set, then the green ribbon which appears can be dragged to define a range at which the sound module can activate. This is different than just a parameter controlling the playback of an event because the Parameter Condition effectively enforces a pair of parameters controlling the playback of a single sound (Remember! A module on the Timeline is STILL on a parameter!). For instance, if I have a sound module on the timeline, I can add a Parameter Condition to the trigger behavior to only allow birds chirping when I am closer than 50 units away. I can define that in the ribbon like so:
So, this module is now effectively tied to two parameters. The first is on the Timeline, which, as always, will activate the module when the cursor intersects it. At that moment, the Logic section of the Trigger Behaviors tray will check if I am closer than 50 units away, according to the range I set up on the ribbon. If I am, the Sound is allowed to continue to the next step of the Logic process and through the Trigger Behaviors tray. This kind of check can be performed on any module, regardless of which Parameter timeline it lives on. It’s also extremely important to note that a fluctuating condition activation can rapidly re-trigger the sound module.
The following step in the Logic process is Probability, which can be toggled on or off by clicking the yellow die icon. When off, Probability is always 100%. Each time the sound is triggered by the module, it will continue on. But set to anything less than 100%, the sound will occasionally, sometimes, mostly or always get gated and thus, never be heard. So, say you have a Multi Sound Module on your Distance Parameter (rather than your timeline) at 3 units, tied to an Elevation parameter at -9 units. Even if both conditions are satisfied, if the Probability is set to 25%, the sound is likely only to be heard 1 out of every 4 times it is triggered.
- The next step of the Trigger Behavior processing is the Delay and Quantization step. The two buttons at the top of the Deck area control whether the tray functions as a Time Delay or Tempo Quantization tool. Selecting Time will display a ribbon which you can use to set either a precise delay, or a window of times from which FMOD will randomly generate a delay each time. Alternatively, one can use the Tempo Quantization tool instead of the Time Delay and delay a sound module to be played on the next eighth, quarter, dotted-quarter, half note, or at the start of the next bar, or two bars away. The tempo quantization is especially useful for keeping musical events in sync and not allowing them to enter the mix out of time.
- Seek: Start Offset allows you to define a pre-determined offset for your sound. This allows you to adjust how far into the file being played FMOD should skip before playing. One of the most useful functions is linearly increasing the offset as time increases so that loops remain in sync with one another. It’s measured in milliseconds, down to two decimal places, and the offset can be as large as ten seconds. It can, of course, be tied to another parameter as well to really start changing things up.
- Polyphony: This area of the tray offers one knob, labeled Max Voices. As in other areas of FMOD, Polyphony dictates how many items from a playlist within the given module are allowed to sound out at once. When the limit that you set here with this knob is hit, additional sounds will queue up for playback once the already playing sounds end. Unlike the 3D Panner module, which contains the overall settings for the event, there is no option here to steal voices. They must queue up.
But Wait, There’s More!
The Triggers and Behaviors tray however isn’t the only additional tool available to help you effectively and efficiently make use of your Time in FMOD (Ha! See what I did there?!). There are still yet a few more small, but incredibly powerful tools that should be touched upon:
For example, there is an oft-neglected and incredibly powerful feature within both types of Transitions (either Marker or Region), called the Transition Timeline. The best way to think of the Transition Timeline is as a mini-Timeline which is optionally played before making the jump to the destination. To create one, simply double-click on the green Transition Marker flag. From there, a timeline will expand out, and the Transition’s destination will be transposed to the end of the timeline. It can be resized by clicking and dragging the right edge of the green time-stamp bar which appears, and you can add any variety of Sound Modules or automations as you desire. One of the best uses of the Transition Timeline is to add intermediary sound effects (say, an engine revving up to speed after the engine turns on, but before the Game Parameter driving the RPM takes over), or to smooth out any applicable automation curves so that the jump from the Transition Marker to the Marker does not unnecessarily jolt the player. To close the Transition Timeline, simply double click on the Transition marker/region again.
The Parameter Condition was discussed earlier in the context of Triggers and Behaviors, but it can also be added to a regular Loop Region in your event. Just select the Loop Region and go to the Logic portion of the Deck and clicking in the grey “+” symbol. Identify which Parameter you wish to use and then use the ribbon which appears to create a range. When the Game Parameter is set between these values, the Loop Region will be on, and will function normally. When the Game Parameter is outside of the values set, it will turn off and the event will continue playing. Using this, you can trap the cursor in loops until the conditions of the game allow it to move on.
One final note is that Logic Track objects can be organized vertically. FMOD processes them from top to bottom. This can be useful if you have several probable changes that can happen at a given point. This allows you to formulate a priority list. For example, if you have an Enemies On Screen parameter and 4 states of music (Calm, 1v1, 1vFew, and 1vMany), you could loop the Calm section with a regular Loop region, and then create three Transition Regions of various priorities to check for the 1vMany state first, then the 1vFew, then 1v1 and finally, if none of those checks result in a true statement, then just continue looping the Calm region. Each state that it jumps to in the timeline could then have it’s own set of logic for moving around. As always, there are more than one way to do just about anything in FMOD so feel free to experiment.
Congratulations! You now have what it takes to manipulate time!
Seriously! That’s it for Lesson 3! This is bigger than getting your license to drive! This is bigger than the return of Whose Line Is It Anyway! This is…This big. Trust me. In the last lesson you learned that the Timeline is just another parameter with some special rules. Now, it’s been proven to you, and you have learned how to control and manipulate it. Through this lesson you should have learned some of the basic approaches to building music made for implementing into FMOD, and how to categorize those patterns of orchestration. You have learned the six basic tools that control the flow of the cursor on the Logic Track, as well as a few supplementary tools to help refine and ultimately perfect the usage of those logic tools, as well as learning how to link to Parameter Conditions, create Transition Timelines, and how to prioritize your Logic Track in a hierarchical system.
For practice, try a few of these exercises:
- Create an event with three tracks in it. Lay a basic drum loop on one track, and place two melodies on the other tracks. Use a single parameter to fade one in and one out. Loop the piece.
- Create an event with one long piece of music. Create three loop regions and use transition regions to transport the cursor freely from one region to another. (Hint! You will need several transition regions for each loop region)
- Use a Sustain Point to keep a module playing until you click the cue.
- Create a five second Timeline Transition, modulating any parameter and playing a sample as it switches (Note that a single sound module (a waveform!) will stop once the transition is over. A sound module such as a multi-sound module will continue playing even after the transition itself is executed.)
- Create a Parameter Condition tied to two parameters (hint! The sound module should be on one parameter’s timeline and the other parameter should be identified in the Logic area of the deck).
- Play a few of your favorite games and try to discern if the soundtrack is being influenced dynamically. Attempt to plot out the kind of system the game uses.
Tune back in next week for a rundown on some of the tools for dynamic mixing in FMOD, and be sure to check here for the rest of my FMOD lessons and tutorials!
See you at the next designated Time Marker!