Making Phunky

Wednesday, 25 February 2009

Bug Labs recently came out with their BUGsound module. I made a fun little application for it using the BUGmotion’s accelerometer stream to pick from a set of samples. The accelerometer is a 3-axis, so I assigned each axis 3 samples to choose from; the application then chooses one of those 3 samples based on that axis’ current reading and mixes the chosen samples for all the axes together. I picked the sample changeover point for each axis based on experimentation by moving the BUG around while printing out the accelerometer readings.

Figuring out how to get sample loops to mix on the fly at 44.1kHz was a little tricky, but it ended up not being much code (you can download it from http://buglabs.net/applications/Phunky if you’re interested).

Implementation

When the application starts it creates an instance of the SampleStream class, which extends InputStream, and starts playing it via IModuleAudioPlayer.play(); as far as IModuleAudioPlayer is aware, SampleStream is just a stream of data from a Wave file.

The main loop in PhunkyApplication.run() then runs forever, grabbing samples from the AccelerometerSampleStream as fast as it can, and passing the X, Y and Z values over to the SampleStream instance.

Since SampleStream extends InputStream, IModuleAudioPlayer is constantly grabbing sample data from it. SampleStream includes a faked Wave file RIFF header. It outputs that header first, and then outputs the mixed sample data from then on.

SampleStream owns nine instances of the Sample class, which also extends InputStream. Each instance of Sample wraps a raw sample data file that comes with the application (more on them later).

The nine Sample instances are assigned three-to-an-axis. Thus whenever SampleStream is asked for data to be played (after the fake Wave header has been output), it looks at the X, Y and Z values it was last given by the main application loop, uses those values to choose a Sample instance for each axis, then reads a sample from each and mixes them together.

Because the reading of accelerometer data is decoupled from the generation of the mixed sample data, there’s no concern about stuttering: SampleStream will simply continue mixing the same 3 Sample streams together until it gets updated accelerometer data.

That sounds like a lot of code, but it really isn’t: here’s the entire SampleStream.read() method (apologies for the formatting; I need to change the width of this column I really do…):

public int read() throws IOException {
int sample = 0;

if (tearDownRequested) {
System.out.println("Returning -1 to make audio stream appear done");
sample = -1;
}
else if (byteCount < riffHeader.length) {
// return the next byte of our canned header
sample = riffHeader[byteCount];
}
else {
// return the most recently computed sample based on the update() data
// pick which samples to combine to create out output sample
// based on the accelerometer values we have been given

if ((byteCount % 4) == 0) {
// we return sample data 8 bits at a time, but the samples
// themselves are 16 bits each, and in stereo so there are
// two samples that go together, so we mustn't switch streams
// in the middle of a sample or we'll be out of sync (returning
// half of one sample and half of the next). We can use the same
// byteCount as we used for the header because the header has an
// even number of bytes
xIn = chooseInput(x, xLowThreshold, xHighThreshold, x1, x2, x3);
yIn = chooseInput(y, yLowThreshold, yHighThreshold, y1, y2, y3);
zIn = chooseInput(z, zLowThreshold, zHighThreshold, z1, z2, z3);
}

try {
sample = xIn.read() + yIn.read() + zIn.read();
}
catch (IOException e) {
// ignore it as it shouldn't happen with our InputStreams anyway
}
}

++byteCount;
return sample;
}

Because IModuleAudioPlayer.play() sees SampleStream as a simple InputStream, and SampleStream mixes together some InputStreams (the Sample instances) the code can be tweaked very easily to do different things: for example you could mix in another InputStream from the microphone, replace an axis with streams chosen by buttons, or any number of variations, without much of the code needing to be modified.

Generating the Samples

At first I tried using simple samples that I generated using the tones command-line utility, but they weren’t all that exciting. So I played around with the excellent Hydrogen drum machine application instead. I made loops from existing samples in Hygrogen (claves, cowbell, floor tom, etc.) and exported them as raw 44.1kHz 16-bit stereo, little-endian, sample files. These are a lot more fun (at least for a few minutes) and sound pretty darn good through the BUGsound.

More Cowbell!

Here’s a video showing Phunky in action:

Implementation

Generating the Samples

More Cowbell!

UPDATE: