Lesson 1: Welcome to FMOD
Hello! It’s been a while! A long time ago, I may or may not have promised a tutorial on FMOD, and in either case it’s been on my to-do list, so I’m gonna begin the process in striking it off here, right now. Welcome to the first post in my series on the use of FMOD. I was inspired to write this tutorial after meeting some of the FMOD/Firelight Technologies crew in March 2014 at the Game Developers Conference in San Francisco. You may have noticed, if you’ve come across my blog before, that all of my previous writings on game audio engine tutorials were written about Wwise. So why am I climbing aboard the S.S. FMOD now? Well, I’m not really doing that. I’m actually trying to just ride both horses. Why start now? Approximately five years ago, I rather disliked FMOD. The program felt kind of nebulous, it’s usage (which either I misunderstood, or it did not explain) went over my head, and the documentation for the software was notoriously under-developed and sparse. In short, the choice to use Wwise was made for me when I simply could not access the program, and so Wwise was what I stuck with. Fast forward to 2014, the FMOD Designer is now replaced with FMOD Studio and, even more excitingly, it is now absolutely free for indie designers to use in their games. Because of the flexibility that FMOD offers, there can be no excuses anymore. Any and all interactive audio designers should be expected to know FMOD, and Wwise, inside and out. These programs are no longer just for the big boys, and are no longer out of your budget or your reach. I would dare be the person to make the bold claim that you should never again be working on a game that is not using some sort of audio middleware ever again.
What gives me the authority to make such a claim? Because, as the FMOD manual so succinctly describes it: The sound file is not the sound that the game needs. A game designer, and certainly a game audio designer, does not live in the linear world. You only have so much space on your CD/DVD/hard disk for menial footstep samples. We can do so much more, with so much less. Especially in the indie games world, the lines between the sound creator and implementer are blurred and you will make your job so much more rewarding, while also increasing your value as an audio designer when you know and understand how to not only create the sounds your game needs, but how to implement them – something FMOD allows you to do quite easily. It is no longer acceptable to simply be satisfied with delivering folders full of .wav files to programmers and expecting them to put them in the game for you. You are much more than that.
Before we get down to business here, I want to stress one thing. This is not a tutorial on FMOD, per se. There are going to be few step-by-step directions tailored to fit specific scenarios. The scope of these lessons will be in attempting to teach newcomers to FMOD the basic concepts underlying some of the more important functions of the program and many of its common uses. As always, if you have any questions, need help, or want to request a specific tutorial on a subject in greater detail, feel free to email me at Hello@ChrisPrunotto(dot)com, or find me on Twitter and pop me a message @SoundGuyChris! Now, with all that said…read on to continue!
What Does FMOD Do?
FMOD is a strange beast. At first glance, it looks kind of like Ableton Live. It’s got the features of a typical D.A.W., but it doesn’t necessarily behave like one. With very little exception, in a linear D.A.W., all audio is placed on the timeline to some degree, and plays back in a very specific order. It is then bounced to a mixed down file that the end user will hear. In this linear paradigm, every region in the DAW has generally deterministic properties: A start time, duration, waveform, end time, and so forth. The complexities of linear audio come from a combination of layering and mixing sounds atop one another. The opposite of this linear paradigm is, naturally, the non-linear one. A non-linear D.A.W. allows for what is referred to as generative audio: using audio with non-deterministic properties in order to create (almost) infinitely unique and very complex constructions from much smaller components. For example, using only six or seven footstep samples – roughly about four seconds of total audio, one could have a very nice palette of dynamically generated footsteps with variable pitch, dynamics, and location. By creating a few parameters to govern the audio engine and feeding FMOD those parameters from the game, one is able to coax a wide variety of sounds to fit a wide variety of circumstances from only a relatively small pool of samples – without the necessity of hard coding anything into the game engine, and without making your programmers spend their valuable time for each and every step of the implementation process. The best argument for the effectiveness of the solutions offered by FMOD is quite simply the enormous amount of titles – many of which are probably in your own library – which make use of the middleware.
A Quick Rundown Of The Event Editor
Concept: Everything Is An Event!
The first thing to understand about FMOD is that everything in your game happens not at a point in time, but at a decision in space. Everything that happens in FMOD is an event. An Event is something that happens in the game world. Jumping is an event. A car accelerating is an event. The music of a game reacting to the amount of enemies in a scene is an event. Death, re-spawning, and picking up coins…you guessed it! They’re all events. Anything that triggers some sort of interaction with sound, for the purposes of FMOD, is an event. A typical game can have hundreds (if not thousands) of events. Now, let’s get started learning our way around the main workhorse of FMOD, the Event Editor. Remember, this all constitutes only but a basic rundown, and there will be more details will be explained as needed later. Again, as always, please send me an email if you have questions, or simply refer to the manuals!
Above, I’ve got an example of the Event Editor. It is the main window of FMOD and it’s likely the first thing you will see. If it isn’t, open it with Ctrl+1, or go to Window > Event Editor. The one I have open is populated with a few modules and an event. On the left side of the Event Editor is the Event Browser, and I’ve marked it inside of a blue box. The first tab simply shows all of your events, and the second tab displays your Banks. You won’t need to worry about banks until you’re ready to publish your sounds to your game engine, so pay no attention to it for now. The Event Browser is far more important to our immediate purposes. Events (things that happen) can be nested inside of other events, or reside in folders. There is also a search bar atop both of these for when the lists start getting long and unwieldy. As you can see, you can organize events primarily in 3 ways: Place events at the root level, place events inside folders (and nest folders into more folders!), and create sub-events inside of other events. This will all be more relevant and explained when the mixing stage is concerned. For now, simply right click in the Event Browser and create a new event. Then click the name of it to turn it yellow and bring it into focus in the rest of the editor.
As you select the new event, take notice of the large, upper middle section of the editor, which I’ve marked by the red box. It’s where the bulk of your work will take place. The controls along the top include the stop/play buttons, a time-code, loop and follow-cursor buttons, and then a pair of toggles to switch between Track and Strip view. Track view is the default you’re probably looking at right now. Strip View is similar to the mixer view of a typical D.A.W., but only reflects the routing of the selected event. We’ll get to all that soon enough. You’ll notice, in my screenshot, two additional knobs. Those knobs are parameters, and they’re linked to the two tabs directly above the timeline (Distance and Custom Parameter, in this case). These are parameters which control the way events play back and are the core of non-linear functionality. We’ll get to that later, but you’ve got to learn what knife is actually for before you cut anything with it. With a new, blank event, the editor will be populated with two tracks: an “Audio 1” track, and a Master track. Unlike a linear D.A.W., this Master is not the Master Output bus! You will be placing your modules on the Audio 1 track (and any additional tracks you create), but the Master is only an aggregate of the rest of the tracks in that event. Remember that a typical game can have many events playing simultaneously, and each event’s master track outputs into the mixer where it is then further routed. Take note of the narrow strip immediately below the editor pane with a translucent box in it. It’s the Birdseye view of the entire project and it’s great for navigating around quickly. Just click and drag to move the view, or drag the edges of the box to zoom in.
Below the Birdseye View is the next extremely important part of FMOD called the Deck. It’s in the green box. You can think of the deck as where your inserts and routing options in your linear D.A.W.s live. Everything from EQ to distortion to reverb all resides here. To add them, simply activate the track you want to add it to (for example, Audio 1) to bring it into focus on the Deck, and then click on the plus sign to either the left or right of the fader knob to add the effects, sends, or side chains either as pre or post fader effects. You can also right click on effects to remove them. The deck displays a lot of information, and it’s sensitive to whatever is selected – pay attention to that, because FMOD is, again, non-linear and therefore a nearly unlimited amount of routing possibilities await you!
Concept: The interface of FMOD is extremely minimalist. There are very few buttons and so FMOD relies heavily on context sensitivity. The Deck often changes based on whatever is selected at the given moment. One of the best ways to learn about things is to click them and see how the Deck reacts, or right clicking them and seeing what options are available to you!
In the farthest right area of the screen lie the last two panes. The one in the white box is the 3D Previewer, and it gives an approximation of where the listener (ie: you!) is, in relation to the sound event. You can click and drag on the white little arrow to move the listener and see how that affects the sound, and the built in parameters (which will be discussed later) can exert additional control over it – everything from vertical spatiality, to surround-sound fields. Please, of course, keep in mind that it’s called the 3D Previewer for a reason. Unless you’ve hard-coded distance properties into an audio event, the game (once the sounds are implemented correctly) will auto-magically determine how far away the sound is, and if it should be played from the right, left, center, or surround speakers. In other words, the editor helps you create everything, but the engine does all the heavy lifting and the math to place those creations in the game space.
The second of the two panes, in yellow, is the properties pane where you can use tags to organize events, create user properties to grab user-specified data and attach arguments to them (This is something for coders especially! In due time!), and leave notes describing functionalities and documentation for others. This is useful not only in case you forget what you were doing after coming back to an event after leaving it alone for a few weeks, it’s also helpful to have for collaboration and for when your manager asks what the heck “that thing” is for. 😉
Outside of the Event Editor window, there are two other windows to get to know right now. If you go to the Window menu at the top of the screen, you can select from the Event Editor (which was just covered), Mixer, the Audio Bin, Event Browser, Mixer Routing, and Profiler Windows. I will cover the Mixer, Mixer Routing, and Profiler in later sections. For now, glance at the Event Browser and Audio Bin: The Event Browser is simple enough. It mirrors the Event Browser on the left pane of the Event Editor window. It additionally lets you search for events by tag, which is excellent for filtering and finding related events quickly. The Audio Bin is where your audio resides and lives in your project when it’s not being used in the timeline or in the current event. The Audio Bin is especially useful for managing your audio files, previewing them, and adding them directly to the timeline from inside of FMOD, as opposed to dragging them in from the File Explorer. Add sounds to the Audio Bin by either importing them through the file menu or dragging them from a File Explorer Window/Finder Window on Windows/Mac. It’s also a good reminder that the goal of generative audio is to create as much as you can out of the basic building blocks, instead of using new samples for every single event.
So now that we’re familiar with the Editor…what actually goes into it?
If you take second look at the screenshot above, you might notice that most of the modules in the Audio 1 track of the timeline do not have waveforms in them like you might be used to with your typical D.A.W….And there’s a reason for that, frustrating as it might be to have to get used to. A single waveform on the timeline is a “Single Sound Module”, and it’s the simplest form of playback in FMOD. It’s the very first module I’ve placed on my timeline, and the only one with a waveform in it. It is an example of a Time-Locked module. A time-locked sound module is any single sound module that is not set to loop, and which resides directly on the timeline. This means that a Single Sound Module will play exactly what is on the waveform, exactly where the cursor is at (unless you’ve modified it by adding effects to the Deck). You can add a single sound module by simply dragging it onto the timeline from either an explorer/finder window, or direct from the Audio Bin. Though generally not as flexible as the other modules, it does have its place in interactive sound design.
Concept: Use pitch, volume, and playback randomization to introduce exponential variety to your audio!
The real star of the show is the cyan module next to the single sound module, in the example above. This is what’s called a Multi Sound Module. To create one, right click on the time line and select “Add Multi Sound.” That’s how you’ll make all modules, actually. The multi sound module’s primary function is to act like a playlist. It contains multiple sounds and plays them back – either in order, or at random, each time the cursor enters the module, regardless if it enters from the front, back, or snaps to the middle of the module (As an aside, unlike the single sound modules, the position of the cursor over the module itself has no bearing over the start position of audio playback. Playback will always begin at the beginning of the sample). To add sounds to a multi sound module, simply click the module to bring it into focus on the deck, and drag and drop sounds into the playlist. The die button and loop buttons assert global randomization and global looping for the module. Once you’ve added sounds, clicking them in the playlist will focus them on the right hand side of the deck, where a second loop button can assert local looping for just that sound. Bear in mind that a sound that is looping will continue to loop until the event is explicitly stopped. With high polyphony, you may experience some undesirable playback where events stack upon themselves endlessly and annoyingly. Always test your functionality both in the FMOD Designer and in your game!
The next module (green) is an Event Sound Module. An Event Module, when activated by the cursor touching the module, will trigger another event contained within it. This is not unlike simply pressing the play button on the nested event. You can create incredibly powerful events that control and automate the parameters of other events by nesting them together like this. Creating a blank module (right clicking on the timeline and adding an Event Sound Module) will essentially insert a blank, new event in the module which you can populate as desired. You can alternatively open the Event Browser and drag the event directly onto the timeline – this is known as a referenced event. It’s important to note though, however, that even if the cursor leaves the Event Module, the event contained inside the Event Module will not stop until explicitly ended.
The fourth module is the Scatterer Sound Module. It’s essentially a multi sound module with a few extra parameters to allow for continuous spawning of sounds to create truly generative audio. It’s especially useful for sounds that do not require precise spatial awareness. It works great on ambient sounds like birds singing in trees or leaves blowing in the wind. When selected, the playlist portion of the deck functions exactly as the multi-sound module. The left-hand portion is the where the master section (influencing global volume and pitch) and scatterer sections reside. The scatterer section has two ribbons and four knobs. The upper ribbon influences the (randomized) range of time between spawning of sounds. The lower ribbon is the min and max distance (again, randomized) to be applied to each sound. Because this ribbon influences 3D spatiality, setting the range to 0-0 will effectively make the sound 2D. It’s also important to remember that the units are arbitrary. Your game will decide whether “10” means 10 inches, feet, yards, miles, or light years. It’s also very important to remember though that most audio in games is represented by spheres! It’s no good to generate a sphere of ambiance around your player with birds singing in it because you might end up with them buried underground! Placement and spatial awareness is everything in game audio! Place the ambience where the ambience goes. The polyphony knob allows an adjustment for how many sounds may play at once, and the volume and pitch randomization knobs will do pretty much exactly what you expect. The Total Sounds dial will let you define exactly how many sounds you want generated. You can set it for infinity to have the Scatterer simply run forever, but if you only want a specific amount of sounds generated, you can set it from anywhere to 1-32. Once the total amount of sounds has been generated, the Scatterer will shut off.
The next two modules are the Programmer Sound Module and Audio Weather Module. A Programmer Sound Event simply allows the game to decide at run-time what sound will be played. It’s useful but an explanation of it is outside the scope of an introduction to FMOD, so I’ll discuss the programmer event at a later time. The Audio Weather module is an external plug in that is included with FMOD Studio. It allows for incredibly realistic rain and wind effects and shows just how powerful the paramaters system of FMOD really is. To use it, simply add it to the timeline, select it, and adjust the rain/wind knobs that appear in the deck, or change the character by selecting a new one from the drop-down list. While fun to use, please remember that Audio Weather is an external plug in and is not included in the “free for indies” licensing of FMOD. It costs $4,500 to license for the first platform of release, and $2,000 to license for each platform thereafter. If one were interested in finding more about the licensing for AudioWeather and other AudioGaming products, please feel free to contact firstname.lastname@example.org with any questions you may have.
Congratulations! Your journey into FMOD has begun!
As I’ve iterated and re-iterated, these are only the primary tools of FMOD. It’s what you will use to lay out and transform your audio from the linear kind to the nonlinear kind. Spend some time getting acquainted with the modules and how they work. Click some buttons that you aren’t sure about. Lay things out, turn some knobs, and keep playing them over and over and see what happens. Learn what you can do and what you can’t do. Don’t worry if everything is still linear! Creating non-linearity comes next! Add a bunch of tracks and see why things sound the way they sound. Start messing with the Deck and experiment to learn what things do. You can’t break anything, so don’t let fear stop you from clicking something if you don’t know what it does (it sounds silly to say, but I’m serious!). You’re always only an F1 keystroke away from the manual. The next lesson will cover parameters.
Things to try out:
- Add a single footstep sample to the timeline and have it loop. Introduce a few semitones of pitch randomization by clicking on the sample and alt-dragging the pitch knob on the deck.
- Add a few more footstep samples and change the pitch and volume randomization of each of them, individually.
- Add a single multi-sound module with a dozen footsteps in it and have them play back. Then introduce pitch shift randomizations as well.
- Add a music base and have a random melody play each time it loops (hint! use two tracks! A base track, and a melody track containing a multi-sound module!)
- Experiment with nesting events and getting a feel for the Event Sound Module. Learn how it works and what it does.
- Create some ambience with some looping multi-sound modules and a few scatterer sounds.
- Give AudioWeather a whirl!
Keep nonlinear being on,
— Chris Prunotto