2005-09-21:
Thanks to ongoing development of UML2.0TM support by MagicDraw UMLTM a degree of port-based systems engineering
with components, ports, and parts is now available in their MD 10 EAP Beta2 release.
|
WARNING:
I recommend that most UML users wait for the next stable MagicDraw UMLTM release MD10,
as the early release beta versions demand much tweaking and many workarounds !
|
I present here a simplified
model of the hardware and JSyn synthesis circuits of Drancing
using UML2.0TM structure diagrams:
Composite Structure Diagrams
Unfortunately MD10EAPBeta2 does not permit easy copying of parts within structures
between diagrams, so this presentation required digital extraction of images
of subcomponents from a single
very large exported Composite Structure Diagram.
Let's start at the top level composite component with component parts and drill down:
Note that there are 4 main domains within the synthesis with (arbitrarily chosen)
port colours:
-
The (continuous) analogue electrical "hardware" domain with ports in CYAN,
from which data is acquired.
-
The (discrete) digital data acquisition domain with ports in BLACK,
which conditioned signals are used to modulate the audio synthesis parameters and
to trigger samples. This cycle is slower than the JSyn audio synthesis cycle.
-
The JSyn audio synthesis cycle in MAGENTA, which
performs the audio synthesis of modulated sine oscillations and triggered samples. This
cycle is by default set at 44.1 kHz.
-
The user interface control cycle in GRAY. Selected parameters of the system may be tuned
using a GUI control layer that is not shown. Such ports are NOT exported to the component boundary, I prefer to think
of these as being driven from a layer connected "orthogonal" to the main synthesis cycles. Imagine controls
"out of the page" hovering above the component, or the front control panel of a hifi component with knobs
and switches wired up to the GRAY ports !
Okay, let's look inside a component "part" to see how it is modelled
(white box approach):
I have actually combined here two related diagrams in a composite component hierarchy,
which is a very useful trick when one is designing everything in one large design diagram. Within the
structure compartment of each component I have BOTH an "implementation diagram" of component classifiers
AND the component parts network !
-
The subcomponent "implementation diagram" with component classifiers
defines the ports (and also optionally the possible connections between components).
The ports are NOT connected (exported to) the boundary ports.
-
The component part network (structure diagram) defines the actual network that
each outer component contains, and input/output/parameter ports are
exported to the outer component boundary.
Note how the components thus nest nicely in a definition hierarchy; this works
well for certain types of systems like this one; it does not work if you are sharing
components throughout the system. I recommend that you try it this
way first; if you need the inner component elsewhere later THEN remove
its structure definition from the outer component (by simply dragging it out), in which
case you will also be taking its quarantined deep inner structure
definitions with it. Sweeeet !
When playing the Drancing gestural synthesis instrument
I usually use NACC=5 triaxial accelerometers worn on the body in a "star" shape: right hand,
left hand, right foot, left foot, and body centre (belt around the waist), which
gives 5x3=15 acceleration signals. Let's see
how those trixaxial accelerometers can be grouped ready for combined data acquisition:
Now we have 3 levels of component structure and port definitions in a hierarchy, which
is about as much as I visually recommend if you are using this nesting trick; by
4 levels components should usually be siphoned off into dedicated structure diagrams.
Signals from the accelerometers are "transmitted" to a data acquisition card where
A/D conversion is performed (let us assume a simple analogue cable bundle to a computer containing
the DAQ card, rather than digital data telemetry):
Once again the component that will become one or more parts is defined inside
the component that uses it (since it will not be needed elsewhere). Also,
the structure compartment has been used to indicate other model elements
(classes, generalizations, and notes) relevant to the component that can be elaborated later or elsewhere. The main
thing is that the external ports and parts for this component are defined.
Note how the ports of the components (classifiers) are connected to each other
throughout these examples to indicate how component parts might be connected !
However, the components thus connected are never "exported" (connected to
external ports); that may only be done with ports from component parts.
This strategy is very useful for organising the system in one place.
The acquired data is polled in C via the COMEDLIB data acquisition
routines, and some calibration and conditioning is applied. Note how the
component (classifier) network (defined on the right), and the
component parts network (defined on the left) is in this case nearly identical
since there is only one part of each defined component. Sweeet as !
So now the acceleration signals are acquired and conditioned let's use them
to modulate some synthesised sound with JSyn audio:
To really understand this audio synthesis one should visit
the JSyn site and study the API and tutorial. A
brief explanation follows:
- A slight smoothing lag is required in the modulating acceleration
signal otherwise one can get sharp "clicks" caused by a jump in the modulation,
since the data acquisition loop is fast enough for bumps to be heard
as audio spikes !
-
The modulating signal is distributed (fan-out) and conditioned separately (scaled and offset) for
FM and AM. Of course one can play many other games like trimming and such, too, however
scale and offset is sufficient for proof-of-concept.
-
One can choose between "fixed" amplitudes and frequencies or modulated
amplitudes and frequences, to achieve: pure AM, AM + FM, or pure FM.
The idea of this prototype is to demonstrate these pure modes,
however of course many other syntheses are possible.
The "fixed" amplitudes
and frequencies can actually be varied by the user as system parameters using UI controls, so they
are strangely named "varAmplitude" and "varFrequency". They are
effectively "fixed" on the timescale of the synthesis engine compared
with the slow GUI cycle.
-
The modulating or "fixed" frequency and amplitude are then fed
to an oscillator (in this case a simple sine oscillator).
-
A further gain is applied, which achieves the mix into the final bus.
-
The modulated oscillation is panned to stereo bus writers for combination
with other stereo syntheses in the final output mix.
With NACC=5 triaxial accelerometers (15 channels) the effect is to synthesis
body movements, postures, and gestures into a rich composite sound that
elicits strong aural biofeedback !
Groups of three user parameters can be master-slaved to a single
UI control to impose "triaxial accelerometer" structure
for the user experience. Or one can group all X, Y, or Z channels
and act on those, or one can group all channels for global UI control.
Lets have a look at triggering samples with the acquired acceleration signals.
Although one can associate one sample with each 1D acceleration channel, it is easier
to have just one sample per accelerometer (so that each accelerometeracts like an
"air drum") by using an RMS of the vector signal:
The samples (which can be drum sounds or any other sample) are preloaded.
The processing sequence is:
- Vector acceleration signals are "averaged" as a single RMS signal
(many other possibilities offer themselves well to the task of triggering)
- The RMS acceleration is used to (potentially) trigger
a sample according to the parameters of a Schmidt trigger,
and the final pulse will be proportional
to the triggering input signal.
-
A pre-loaded sample with pulse as amplitude is queued immediately to play next JSyn synthesis step
-
The sample is panned according to the user setting to a stereo output bus writer.
Of course the pans could also be modulated by the acceleration signals !
And there you have it, "Drumming by Dancing", or Drancing as I call it.
Just wave your hands in the air to play any sampled
sound you like !
You can even hear the sound of one hand clapping !
The basic RMS-to-Schmidt-Trigger system is enough for a basic demonstration of
Drancing. In practise a little more conditioning of the triggering signal
is required, including some phase change tricks to account for the nature
of acceleration.
One can also trigger synthetic drums - synthesised in real-time - using additional
JSyn circuits.
Now let us put it all back together. I recommend projecting the image
below onto a large wall or screen with a data projector:
With MagicDraw UMLTM one can navigate a hierarchy of Composite Structure Diagrams
vias hyperlink to diagrams assigned to Component presentation elements, which is
a natural model navigation paradigm for systems engineering with components.
On this UML2.0 modelling and the future of model-driven system
development
What I have presented is NOT yet model-driven development of port-based
systems engineering for JavaTM with UML2.0TM ! Rather,
it is an attempt - largely manual - to reverse-engineer an existing port-based
JavaTM system into a UML2.0TM representation using Composite Structure Diagrams.
It still takes a lot of patience with the MagicDraw UMLTM tool, however - as you
can see - one can do it, and it is a very useful model. It does demonstrate
port-based systems engineering design for JavaTM with UML2.0TM.
My challenge now is to use these sorts of models to generate
functioning JSyn audio synthesis circuits with minimal Java coding.
I am developing a plug-in for MagicDraw UMLTM that enables just this,
and likewise enables generation of animatable structures for Java 3DTM, to
give a complete Java animation, sonfication, and simulation environment controlled
directly from MagicDraw UMLTM, which I will also use for my scientific
modelling projects (like this radiotelescope synthesis model).
If you just want to compose audio synthesis circuits for
JSyn
without any relationship to other
JavaTM components or a UML
analysis and design model, there is a standalone
Graphical Modular Sound Editor (patch editor)
Wire for JSyn.
My (growing) wishlist for component modelling in Magicdraw UML
To make this kind of modelling easier I suggest the following improvements
(w.r.t. MD10 EAP Beta2):
-
Automatic creation of a Composite Structure Diagram assigned to a component
as a hyperlink to the diagram, to enable common-sense navigation of the
component structure hierarchy. This is the universal paradigm for systems
engineering tools (like Matlab/SimuLink, and the Houdini animation tool).
It is VERY easy to implement, and would bring save a lot of my time.
-
No default naming of typed ports with things like "unnamed1"; if a port is the
only one of its Type within its environment the role is well indicated by
the name of its Type and the name can be derived for forward engineering
from as the lower case version of the Type name. In the case where this rule
fails, or where a different name as been reverse-engineerd, an explicit name
can THEN be assigned (because only then is it needed).
There is no extra information in
trigger:Trigger, synthBusPort:SynthBusPort,
it takes time to manage, and clutters diagrams !
Tip:
Throughout your UML modelling: name Types well and let the Type do the hard work
for instances, parts, and ports whenever possible!
-
The option to display the Type of a port without the port's name (for the reasons explained
above)
-
Ability to copy and paste or even cut and paste between Composite Structure Diagrams.
-
The ability to generate a "connected parts network" from a "selected classifier network".
This would make it much easier to draw parts within Composite Structure Diagrams. At
the moment one must tediously mimic with connections the associations that one
(might have) has reverse or forward engineered. This would help a lot.
-
Or even better: the ability to forward/reverse between Composite structures
and "connected parts" networks classes that represent the parts as
attributes. That would enable some degree of model-driven development with UML2.0
in MagicDraw UMLTM. (This is what I am working on for the specific cases of
Java3D scenes and JSyn audio synthesis networks via a plug-in.)
Thanks for help
I would like to thank the following people for assisting me in this ongoing project:
-
To Gary Duncanson and the MagicDraw UMLTM sales teams: thanks for giving me
extended access to the MD10EAPBeta2 early release versions.
-
To the MagicDraw UMLTM development support teams: thanks for your ongoing improvements
to MagicDraw UMLTM, and for getting the port and part support of Composite Structure
Diagrams working.
-
Thanks to Nerijus Jankevicius, Senior Programmer and System Analyst at MagicDraw UMLTM,
for detailed feedback on UML2.0TM issues.
-
Thanks to Bran Selic (IBM) for clarifications on UML2.0TM specifications.
-
Thanks to Paolo Cantoni for frequent discussions on component modelling.
