“ProofOfConcept” is my first virtual reality scene. I made it to prove out the tech stack I chose for making real-time music-driven VR experiences for web browsers.
It works in the Oculus Quest 2 browser and most desktop browsers.
It takes a little time for the music get going so don’t turn up the volume too soon. Also note that the closer you get to the objects the music is coming from, the louder the music will get.
When the scene first loads you might see a button in the top left to unmute the sound. Click it to start the music.
If a VR device is available you’ll see a button in the bottom right for entering VR mode.
To look around on desktop and mobile devices, click and drag the mouse or drag your finger on the screen.
On desktop devices, press the
WASD or arrow keys to move. Hold down the
Space key to move faster, or use
On VR devices, use the thumbstick to teleport.
The audio is generated in realtime using the WASM version of Csound. For the realtime graphics people out there, Csound is kind of like shaders for audio. In a browser it runs on the CPU in a separate AudioWorklet thread.
For the positional audio I reimplemented Google’s Omnitone library in Csound. It works pretty well and allows me to use a lot more sound sources than I can with other methods.
My composing workflow went like this:
- Sequence music in Reaper DAW with plugins generated by Cabbage from my Csound scripts.
- Bounce DAW settings and plugins to a monolithic Csound file using the steps listed here.
- Copy/paste the monolithic Csound file and its corresponding JSON file into the Babylon.js project.