In working on a current project, the Twitter-fueled HTML5 -powered game Squirrel Sqript (Which is almost ready to launch, by the way!), I’ve learned a lot about cross-functionality. I acted as a programmer on this team, and as all programmers must do, I had to overcome certain unique problems presented by the platform and the project. Because the game is HTML5, our team encountered an issue in that browser-based games (particularly mobile browser-based games, and especially mobile browser based games) don’t necessarily support audio in the way you want them to. And no single codec is accepted by every browser. AND the performance hits are dramatic for even some of the simplest of audio related functions. AND the list of quirks goes on. It’s maddening! Not even sites specifically built for audio like SoundCloud offer great usability on mobile because putting your phone to sleep not only kills playback, but also the player itself in many instances on awake, forcing a refresh of the entire page. The logic is that most mobile users pay for data per gigabyte/kilobyte, and overage gets expensive, so the browser will take any chance it gets to kill your audio. That’s where audio sprites come in.
There are a myriad of solutions to browser game audio, and the audio sprite is one of the more popular. It’s not unlike the classic sprite sheet, which is still prevalent even in modern games. A sprite sheet has, as the name implies, all of the individual frames of an animation laid out next to one another, and the program then parses the images, sets them into an array, and flips through them to create a simple (or even complex) animation. There’s a pretty cool examples on the net such as this fan-created sheet of Super Metroid’s Samus Aran, hosted at SpriteDatabase.net. Like sprites, a sound designer can create a single concatenated file containing all of the sound effects to be used in the game. On the first button press of the game (say, “Start”), the audio sprite manager loads the file and loops a section of silence on the sprite file. Then, at key points in the game you can jump to other points in the file – through code and without user interaction (!) – to play sound effects as you need them. It’s quite beautiful in how simply it handles this, and the best part is that most mobile platforms are pretty good about holding entire files in RAM for you to jump around in.
There are a few drawbacks, however, as with anything. A huge sprite file takes up a lot of precious ram – audio in .wav format at 44.1kHz is the same size whether it’s silent or white noise, if the file sizes are the same length. So it’s best to make them short. If you have many sound effects, you may want to group them up by level and load them one by one (Just remember, you need an initial button press to load them!). It’s also important to be mindful of performance impacts – unlike desktops, mobile does not typically have designated VRAM and RAM. It all goes into the same pile, so if you’re displaying cool 3D graphics on mobile, make sure you test and test again, because you will find performance chokes quite often. And of course, browsers all have their own quirks to them.
Now unfortunately, the code we used for this game was originally published here on the Zynga Engineering and Code Blog, and has since been taken down for whatever reason (I’ll leave the link here for posterity, and in case it ever comes back!). So I have but only a short example and description to give, because I do not feel comfortable explicitly posting their hundreds of lines of work to paste-bin. All I can offer is a “This is how it works” and leave it for you to maybe engineer something better. 🙂
If you know of a great jukebox system like the one described, please link it! 🙂