Adding Sound to Skelly using Web Audio API
Let there be sound!
The ability to play music and sound effects is so important to designing a fun game, this core feature needed to be added in my game framework, Skelly. My goal for Skelly, The Bare Bones Game Framework, is to be an easy starting point to develop a game at a low level in C and have it run everywhere. Because the web and WASM is the most restrictive platform I want to support, I am starting with this platform. The goal is to support making web games without using Emscripten or WebGL for maximum compatibility and simplicity in the game frame work (It is "bare bones" after all). I have already argued in previous posts why I don't like Emscripten or the state of modern GPU graphics APIs, so I won't get into why these design decisions were made here.
Because of these design decisions, the basic architecture of the game is already determined. All of the game code is a WASM program which does not depend on the usual C standard libraries. Almost all of the Skelly framework code must be JS. So, once per frame drawn, audio samples need to be generated in WASM in a known place of the WASM memory buffer. Then after each frame, that data needs to be sent off to be played somehow. Implementing this half was easy because I basically already learned how to do this when making the screen pixel buffer for the graphics, which are handled in a similar way.
The hard part, was learning how the Web Audio API works and getting sound to play. Playing pre-generated samples requires making a custom AudioWorkletProcessor which runs on a separate thread than the main JS thread. The browser has the worklet run the process function whenever it feels like generating 128 more samples. at 44100 samples per second and the game running at 60 fps, this process function needs to be called several times a frame.
My first attempt to handle this was to make a circular audio buffer in WASM, set the wasm memory buffer to be a SharedArrayBuffer so the 2 threads can look into it, so this way the worklet has no overhead with transferring data, it should just have access to it. All of my research shown this is the ideal performant way to implement this, but I had a lot of problems getting my server to actually support the SharedArrayBuffer. Turns out the server needs to be set up for Cross-Origin Isolation, which my local python3 -m http.server was not doing. I want any game made with Skelly to be easy to get working on a web page without much hassle, so I decided to abandon this approach.
So instead I removed the circular buffer from the game side of the code. Now each frame you generate the new data from the start of the buffer going until the size needed to generate. Then I use a port message to send this data to the worklet at the end of every frame. On receiving the buffer of samples, the worklet puts the buffer into a queue of buffers. The worklet just keeps playing the samples in the front buffer until it reaches the end, and it simply dequeues the buffer and starts at the 0th position of the new front buffer. Each time the worklet finishes it's process function, it sends a port message saying how many samples are queued. The game tries to only generate enough samples to be 2 frames ahead. In case of a minor lag spike, there shouldn't be glitchy sounds but the audio won't be too far advanced so sound effects should give the player tight feedback to the action they have just performed.
Currently at time of writing, I do not have the ability to just load an audio file, so to test out my audio playback I have just generated 3 square waves to make a C chord. You can see the current state of this project here.
After days of no progress, and one day of glitchy audio, when I finally got it working I felt so great, like I can code anything!
