Lesson 5: Integration Into Unity
Alright! Now it’s REALLY been a while since last time! I think it’s finally time to start wrapping up these lessons by answering the one question that everyone has: “Alright Chris, I’m now a master of FMOD thanks to you, but now I need to shove this thing into Unity and make them work so that I can be the master of Interactive Audio!”
Well first, that’s not a question, but point taken – to the folks who have e-mailed me, prodding me into finishing up, I can only offer my sincerest apologies because life has gotten way busy this past year. Thanks for the outreach. 🙂 We’re gonna do this. Right here, right now. If you’ve followed along with the set of tutorials so far, you should be able to efficiently bring your ideas into FMOD using the tools available to you. If you need a refresher, you can check out the overview of all the lessons at this link here or hop back to the very first one here. Unlike the last four lessons, this lesson will not build on previous concepts directly since this lesson will focus on integration concepts in tying FMOD into Unity. However, it is still crucial to know the inner workings of FMOD before you try tackling integration, so review if you need to. The instructions are the easy part – it’s knowing the concepts that will take you far.
Let’s get started. And as always – if you have any questions, require further explanations, or wish to suggest further topics, email me at Hello@ChrisPrunotto.com or reach out to me on twitter @SoundGuyChris!
A Few New Concepts Before We Begin:
Alright, so, firstly, it’s important to understand the concepts behind how the program and the middleware talk to one another. I’ve got to let you in on a big lie that I’ve been telling you: FMOD Studio is never going into your game. FMOD Studio is really just a fancy program whose job is just to talk to a bunch of audio script libraries and to package Soundbanks. It’s only job is to make it easy to visually see what is going into your final Soundbanks. Nothing more. That script library (the FMOD sound Engine) is what will be going into your game. The code you write will access those libraries that make up the sound engine, which will in turn access Soundbanks containing your audio, and then playing that audio. The important takeaway from this is to mentally separate FMOD Studio Designer from the FMOD Sound Engine. If you can do that, then you’re going to find yourself much more comfortable when you start working in code and don’t have the Designer open in front of you.
Now, that’s cool and all but what is a Soundbank? A Soundbank is essentially a very tightly packed set of audio files and their related metadatas. A single Soundbank is comprised of two files: a “.strings” file which functions sort of like a glorified table of contents, and a “.bank” file, which contains the physical collection of DSP settings, bus routing systems, and of course the converted audio files being used in the project. The most critical thing to know is that a Soundbank is a heap of sounds all packaged together and that they get loaded into memory all at the same time. Unlike some other audio systems, you can’t pick and choose pieces of a Soundbank to load. You load it all at once, and you unload it all at once. In more complex projects than this simple tutorial, you’ll have to strategically divide your Soundbanks into something that is large enough to not need constant loading and unloading, but small enough to be useful.
Here’s a quick example to hopefully demonstrate the concept: In a 2D fighting game like Mortal Kombat*, every character might have their own individual Soundbank accompanying them that includes mostly voice over clips, their exertion sounds, and maybe some defining sound effects (ie: the special lightning-affected impact sounds you hear when using Raiden). You’ll also probably load a “shared” Soundbank that is loaded in with every fight because every round needs those sounds. This might include common sounds such as general impacts, ambient noises, and especially user interface sound effects that don’t really change based on the choice of the fighters that are selected. Then each stage might also have its own Soundbank to accompany it, which has additional sound effects related specifically to the level being loaded, perhaps level-specific set piece interactions and the like. Then at the end of the game, when going back to the main menu, all of those “in-game” Soundbanks get neatly discarded from memory and the “menu” Soundbanks get loaded. The end result is a sound system that is, in many respects, very modular. (*by the way, This is only a guess at how a game like Mortal Kombat might function. I’ve got no actual experience with the way the game actually works, also I’m in absolutely no way affiliated with MK or NetherRealm or any of those wizards or anything, even though I’d love to be).
- The FMOD sound engine is really just a bunch of code sets that access Soundbanks to play audio.
- Soundbanks are really just packages of sound files and metadata that all get loaded into memory at once
To get from a blank Unity Scene to FMOD-powered audio, you need to follow five general steps:
- Connect FMOD to Unity.
- Create your Soundbank(s) in FMOD
- Importing your Soundbank(s) into Unity.
- Create a Listener object in your scene.
- Call your FMOD Event from code.
Are you ready?
Let’s Integrate FMOD Into Unity
First things first: Let’s get the game engine talking to audio engine.
A few things have changed since the last time I wrote a lesson on FMOD, and one of those things is that Unity 5 has been released. This means that you no longer need Unity Pro to run extensions in Unity. This includes FMOD! And man, let me tell you, the setup process is really ridiculously easy. It’s never been easier. Seriously. Head on over to the FMOD.com website and go to the download section to pick up the “fmodstudio<versionnumber>.unitypackage that corresponds to the version of FMOD you have installed (to check your version of FMOD, go to Help > About FMOD Studio in the FMOD program). Make sure that the version number in the .unitypackage is the same as the version number of FMOD that you’re designing in, other wise a few things may not work correctly. My version of FMOD is usually a few versions out of date so my version of the integration package is not the latest one available, so if you need to, make sure to check out the “older versions” link.
Importing is super easy. Just drop your .unitypackage somewhere memorable like your desktop and open up a Unity Scene. Go to “Assets > Import > Import Custom Package” and follow the prompts to import the entire package.
And just like that, 90% of the work of integration is done for you. What a time to be alive.
The Basics of Soundbanks
The next step is to create a bank and assign some events to it. This is also pretty straightforward!
Crack open FMOD and create a new sample event with a single sound module of any type (just a single, static sound effect is fine).
You should now take a second to notice one of the only sections of the user interface in the Event Editor Window that previous lessons did not deal with in detail: The Banks tab on the opposite side of the Events tab. This is where you arrange your Soundbanks. If you flip over to it you’ll notice a black pane and a default “Master Bank.” Before we talk about the black area with the “Build Settings” section, lets drag our sample event onto the Master Bank. You can either click and drag it from the Events list to the Banks tab and drop it onto the Master Bank, or you can you right-click the event and Assign to Bank. You can also assign events to banks wholesale by just selecting multiple and click-and-dragging or right-clicking them.
Once you’ve added your event to the Master Bank click on it to select it and take notice of the tab which appears, labeled “Desktop (PC/Mac/Linux)”, which contains a selector option to choose between formats as well as a Quality control knob. The nice thing here is that using that one knob and one selector option, you’ll set the build settings for all three platforms. If you right-click below the “Desktop (PC/Mac/Linux)” tab, you’ll be able to select from a context-sensitive menu to include other platforms as well (provided you have the correct license for them): Mobile (Android/iOS/Windows Phone), Xbox 360, Xbox One, Playstation 4, Playstation 3, Playstation Vita, and Wii U.
Here’s a quick rundown of some of the conversion options:
- PCM is the classic Pulse-Code Modulation filetype: It’s pretty much lossless and works pretty much every where.
- Vorbis is the, as you can imagine. Vorbis filetype: It’s efficient, it’s useful, it’s got a bit of audio compression audible but on the whole, it’s excellent.
- FADPCM is a newer format, developed by Firelight, the creators of FMOD. It’s a proprietary version of ADPCM that is reported to have twice the speed, better compression, and higher fidelity of of ADPCM.
- XMA is an Xbox 360/Xbox One native audio format
- AT9 is the proprietary Sony audio format that runs on the PlayStation Vita and PlayStation 4.
For more detailed descriptions of the advantages and disadvantages of these five conversion options, make sure to check out the FMOD manual.
In any case, just remember that each Soundbank can have any combination of platforms associated to it. This becomes super useful in regards to your release strategy. For example, if you have a small indie game being released to many platforms you may have Soundbanks set up like this: Your Android/Apple/Windows phone release version of your game might use Vorbis at 20% quality, since space might be at a premium and super-high quality effects is not the most important concern on mobile. The PlayStation Vita version might use the proprietary AT9 format and take advantage of both space and performance gains and set its quality at 50% – usable, but not perfect. The downloadable console releases for PlayStation 4 and Xbox One might compromise a bit and use AT9 and XMA respectively at 85% quality to retain high fidelity but still shed some space on the generally limited hard drives, but the flagship PC/Desktop/Apple/Linux version of the game would have no reservations against the use of luxurious PCM files at 100% quality. On top of that, lets say that your PlayStation Vita version of the game includes an exclusive level. So you might have a second “PS Vita Level” bank whose only platform is the PlayStation Vita.
Concept: Each Soundbank can have it’s own build settings. You can build each Soundbank to some or all intended release platforms to deliver content specific to each platform. You can also individually set quality settings to further prioritize Soundbanks that are less important to be more compressed in order to save disk space, or to otherwise build to the specifications that best meet YOUR needs!
Once you’ve got your event assigned to the Master Bank and you’ve selected your platforms (for the sake of this tutorial, the only one you need is Desktop (PC/Mac/Linux)), go ahead and save the .fspro file. Then go to ‘File > Build Banks’ (you need to save the FMOD session before you can build anything). It should only take a moment. Built Soundbank files go inside of the “Build” folder in your FMOD Project Directory. A lot of folks get tripped up by assuming they’re in the “Banks” directory. Banks is for un-built versions that are used in FMOD Studio Designer. The finished, game-usable banks go inside of “Build.” Each time you add or change events or routing, you must rebuild the Soundbanks so that the changes get reflected in Unity. Remember, Unity is not reading FMOD Studio Designer, it is only acting upon the data in the Soundbanks!
Next, in FMOD, go to ‘File > Export GUID’s’.
What’s a GUID, you ask? A GUID is a Globally Unique Identifier and the GUIDs file is a text file inside of your Build folder that basically is a translation between the 128-bit signed integer that identifies a given asset in FMOD and and translates it into user-friendly names (such as “Test Event”) which are the events you named in the Designer. It’s important to point out the GUIDs file because if you’re ever getting some sort of “Event Not Found” error, it may be worth re-exporting your GUID’s text file, especially if you’ve renamed any events lately.
Importing FMOD Soundbanks into Unity
Anyway, now that we’ve got our banks created, let’s pop em into Unity! It’s a simple, one-step process, but you might get a fairly common error here, so read on even if something goes haywire.
From the Unity Menu go to ‘FMOD > Import Banks’ and select the “Build” folder inside your FMOD project directory.
That’s it. You’re done! (Just remember though, you’re looking for the “Build” folder, not the “Banks” folder, nor are you looking for any of the individual Soundbank folders (ie: “Desktop”, “Xbox 360”, “PlayStation 4”))
Did that work? If it did, cool! You’re in business! Feel free to skip down to the next section!
However, if Unity gave you an error instead, and told you that “FMOD Studio Integration requires either Unity Pro 4 or Unity 5” even though you definitely should be able to use Plugins, don’t freak out! it’s a super easy fix: Navigate through your Project explorer in Unity to ‘Assets/Plugins/FMOD/Editor’ and open “FMODEditorExtension.cs” – this is the file that controls that FMOD menu up in the menu bar. You’ll notice on line 846, there is a pre-processor directive with “#if UNITY_” or something similar. Change this to be either “UNITY_5”, “UNITY_5_0” or “UNITY_5_1” depending on your version of Unity. To figure out which, go to Help > About Unity… and it should be right there on the first line (for me, UNITY_5_1 was the winner). Now go back to ‘FMOD > Import Banks’ and select the Build folder as described before.
Got it now? Solid.
Adding a Listener
So now you’ve got your Engine imported.
And you’ve got your Banks imported.
Now you’re gonna want to actually be able to hear anything that’s happening. So we’re going to concern ourselves with the Listener. This is another really simple one-step process. I discussed the Listener at length in Lesson 2 with it’s relationship to the Event Emitter. Think of the listener as where your ears in the game are located. There’s only going to be one Listener in any scene. If an event happens to the left of the Listener in the game space, then the sound will appear to be coming from the left channel of your speakers. All built-in parameters (Distance, Direction, Elevation, etc.) use the listener for mathematical functions to provide engine features such as distance-based attenuation or 3D spatialization.
All you have to do is locate the “FMOD_Listener.cs” script in ‘Assets/Plugins/FMOD/Editor’ and drag and drop it onto something that makes sense in your scene. 99.99% of the time, this will be either the main camera or the player character and I greatly prefer the main camera (it feels absolutely nauseating to hear the stereo image flip every single time your character turns around, in my opinion.) Some artsy-fartsy games will place the listener in odd spots, but for now, let’s assume you’re not trying to answer the age old question of “if a tree falls in a Unity-Terrain Prefab Forest and no FMOD_Listener.cs is around to hear it, does it make a sound?”)
Is it on your main Camera yet? Wasn’t that easy
Let’s Make a Ruckus
Only one thing left to do now! There’s actually two ways to actually call your event: There’s the C# scripting approach and the Unity Component approach (Well..I guess technically, they’re both use Components in their approach but bear with me here…). Before we can do either though, that we need to create a gameObject to attach our script or component to. So go ahead and create an empty gameObject (or a cube, or if you’re feeling pretty crafty, maybe a cylinder or something) and pick a method:
The scripting method is best suited for one-shot events, and it’s called through code. It’s very flexible and mostly relies on a single line of code. Most of the time you use an FMOD Event in your game, it’ll be done this way.
Attach a new c# script to your gameObject. You can call it whatever you want, and in the “Start” method of the basic code template, add this following line:
So what are we looking at here?
Well first, we’re adding this line inside of the Start method so that it’s called one single time, as soon as we start the game and the object is loaded.
Looking at the code directly, “FMOD_StudioSystem” is the namespace of the entire FMOD engine. That Object is the guts and nuts and bolts of the FMOD Sound Engine. Unlike other audio engines, don’t put the FMOD_StudioSystem directly onto any gameObjects anywhere in your scene! Just call events out of it through code, like we’re doing here. This is important to know because later, when you hit play, you’ll actually see the FMOD_StudioSystem object be created *automatically* in your scene.
Next, “instance” refers to the singleton instance of the FMOD_StudioSystem object when it’s in your scene. Again, you don’t really have to worry about it. “instance” refers simply to the existing instance of the FMOD_StudioSystem object.
Finally, “PlayOneShot” is a built-in FMOD function that plays an event, then kills it from memory. It requires two arguments. The first argument is the “FMOD Asset” in string format. In otherwords, the actual event being played. If you’re ever unsure of what the FMOD Asset is actually called head into FMOD and right-click your event or bus or *whatever* you’re trying to call in code and right-click it and select “Copy Path” – you can paste the result wherever it needs to go, and it will even include the correct prefix, too (ie: “event:”, “bus:”, etc.). The second argument that PlayOneShot takes is the location for the Asset in Vector3 format. In other words, where the sound is originating from. For this purpose, oftentimes ‘transform.position’ will be good enough – in other words, play the sound from the position of the object that is calling the code.
If you go ahead and save the project, then hit Play, you should see the FMOD_StudioSystem object get created and then hear your sound play almost immediately.
If you did, Congratulations! You’ve made noise in Unity using FMOD!
The Event Emitter component is best suited to long, endlessly repeating events (Do you remember how to loop stuff? Head back to Lesson 3 to get a refresher on the Timeline and Logic Tracks!) Each Emitter controls the playback of ONE and only one FMOD Event. To use it simply attach the “FMOD_StudioEventEmitter.cs” script from the ‘Assets/Plugins/FMOD’ folder to your gameObject using the Add Component dialog or drag and drop.
Now click the blue dot next to the empty Asset field to open the Asset Picker and select your desired FMOD event from the pop-up window, under the Assets tab. The Path field will get filled in automatically and Start Event on Awake will be filled in. That’s all you need to get it going out of the box.
Save the project and hit Play. You should see the FMOD_StudioSystem object get created, and then hear the sound play almost immediately.
Now, from other scripts, you can access the FMOD_StudioEventEmitter component and modify properties and parameters and the like (though you can, of course, just as easily do the same using the C# method). Though outside the scope of this lesson, the best way to use the FMOD_StudioEventEmitter is to now access the FMOD_StudioEventEmitter component from other scrips and do whatever manipulations you want from there.
Okay, by now you should have a fully-integrated FMOD Engine in Unity 5. Now it’s time to keep on going. Try a few of these quick challenges to get used to the flow of working in FMOD and Unity.
Why not try setting up a sandbox level using the built-in Unity 3D Controller and FMOD and then do a few of these quick challenges to get used to the FMOD/Unity workflow?
- Place a cube emitting a looping audio clip in your sandbox so you can see how the built-in Distance parameter functions (hint: Looping is done using the Logic Track!)
- Afterwards, why not modify the built-in Distance Parameter in FMOD so that your sound gets *louder* as you get further away? (hint: Remember to rebuild your banks after making changes or adding events!)
- Try placing several cube emitters, each playing a different loop and see how they interact as you move them and your player character around.
- Look up some code so that a one-shot Event is fired each time the space bar is pressed. Make it a random sound effect from a Multi-Sound pool of five effects.
- Try placing the Listener on something that’s NOT the main character or main camera and notice how the sonic perspective of the game has changed.
You might notice I’ve intentionally not asked you to create any custom parameters. Those require a bit more involvement because while the Built-In Parameters can take care of the math functions to measure distance and vector between Listener and Emitter. To use Custom Parameters, you’ll have to code those measurements in yourself. I’ll (hopefully soon) have the time to get started on opening THAT particular can of worms. Until then…
— Chris Prunotto