Video: YouTube: Drancing accelerometer music with Wiimotes: 3D variable frequency oscillators + amplitude variation + triggered "air drum" samples

Play video (then use fullscreen !)
ERROR: You are missing some Flash content that should appear here! Perhaps your browser cannot display it, or maybe it did not initialise correctly.
This error usually only occurs if embedding remote content fails; Please try playing the original video on the original site using the following link.

Demonstrates combined oscillator frequency variation, amplitude variation, and triggered Drum sample ("Drumming by Dancing") modes along with Drancel RGB monitor visuals projected onto the "Drancer" performer.

For this demonstration two hand-held WiiTM Remotes on a MacBook Pro were used. (The original Drancing accelerometer suit (since 1997) used 5 triaxial accelerometers in a "body star" pattern").



Each channel of each Drancel can drive a Variable Frequency Oscillator (VFO) - in this case a simple sine oscillator - with the option of locking onto discrete MIDI note frequencies, as well as a frequency scaling, a frequency offset, and an output gain.



Variable Frequency Oscillators (VFOs) are grouped in a triad corresponding to the (X,Y,Z) accelerometer channels of a Drancel.

This is the most immediately powerful Drancing mode as far as aural biofeedback is concerned, and it promotes proprioception. It sounds quite sci-fi like and eerie, and reminds one of the Theremin, except there are (in the mode shown here) 3 independent frequencies, which can be made "rub" together to create beat frequencies.

A composite JSyn software component that performs real-time audio synthesis with AM or VFO

A composite JSyn software component that performs real-time audio synthesis with AM or VFO

So now the acceleration signals are acquired and conditioned let's use them to modulate some synthesised sound with JSyn audio:

To really understand this audio synthesis one should study the JSyn API and tutorial. A brief explanation follows:

  • A slight smoothing lag is required in the modulating acceleration signal otherwise one can get sharp "clicks" caused by a jump in the modulation, since the data acquisition loop is fast enough for bumps to be heard as audio spikes !
  • The modulating signal is distributed (fan-out) and conditioned separately (scaled and offset) for VFO and AM. Of course one can play many other games like trimming and such, too, however scale and offset is sufficient for proof-of-concept.
  • One can choose between "fixed" amplitudes and frequencies or modulated amplitudes and frequences, to achieve: pure AM, AM + VFO, or pure VFO. The idea of this prototype is to demonstrate these pure modes, however of course many other syntheses are possible. The "fixed" amplitudes and frequencies can actually be varied by the user as system parameters using UI controls, so they are named here "varAmplitude" and "varFrequency". They are effectively "fixed" on the timescale of the synthesis engine compared with the slow GUI cycle.
  • The modulating or "fixed" frequency and amplitude are then fed to an oscillator (in this case a simple sine oscillator).
  • A further gain is applied, which achieves the mix into the final bus.
  • The modulated oscillation is panned to stereo bus writers for combination with other stereo syntheses in the final output mix.

With NACC=5 triaxial accelerometers (15 channels) the effect is to synthesis body movements, postures, and gestures into a rich composite sound that promotes strong aural biofeedback !

Groups of three user parameters can be master-slaved to a single UI control to impose "triaxial accelerometer" structure for the user experience. Or one can group all X, Y, or Z channels and act on those, or one can group all channels for global UI control.

Video: Drancing suit 2002: variable frequency + variable amplitude mode

Play video (then use fullscreen !)

The (X,Y,Z) channels of five 3D accelerometers vary the amplitudes and the frequencies of 5 x 3 = 15 pure sinusoidal oscillators.

One can hear "beat" frequencies generated by similar frequencies corresponding to different accelerometer channels, such as when the arms are raised to similar levels, which effect can be exploited for musical nuance and for movement therapy to measure symmetry of posture.

Syndicate content