MacMix:Mixing Music with a Mouse
©1986, 1996 Adrian Freed. All Rights Reserved.
This is a slightly revised version of a paper presented at the ICMC
in the Hague in 1986.
MacMix is a tool for browsing, viewing, mixing and editing sounds. Its
strengths are the result of an unsurprising user interface, implemented
on an Apple Macintosh computer, and Robert Gross's efficient sound file
system and mixing program.
Architecture
IRCAM, like many large computer music facilities, has chosen to build
a centralised store for sound files. This is the most economical way of
providing access to the Gigabytes of file storage required for hours of
high-quality sound. MacMix gives a musician a familiar personal work-station
environment for processing sounds shared amongst a community of users on
a large time-sharing host computer.
The Macintosh computer is connected to the host using a serial communication
line. This architecture was inspired by the Blit [Pike, Locanthi and Reiser,
1985].
A custom-designed, robust, stateless communication protocol
is used between host and workstation.
User Interface
Three types of window are provided.
Browser. The browser is used to explore the contents of the sound
file system. A window of file names in the current sound file directory
may be scrolled. Descriptive written information is available using the
Get Info. button. The Play button is used to listen to the
selected file. The samples in a file may be viewed using the the Open
button.
View WIndows. View windows contain a visual representation of
a sound. Most of the time, this representation is an amplitude envelope.
If sufficient resolution is available, the samples themselves are displayed.
Great care is taken to visually represent sound in an unambiguous way.
Lines are not drawn between sample values as this would introduces
visual information with no counterpart in the signal. Each pixel in the
amplitude envelope display represents the largest absolute sample value
in an integer number of samples. This definition of envelope has the advantages
of being easy to interpret, cheap to compute and it facilitates identification
of clicks or unusually large amplitude excursions.
View windows provide a simple editing facility. A selected portion of
a sound file can be extracted and written to a new file. An amplitude envelope
can be sketched on top of a selected sound file fragment and will be superimposed
on the signal during the extraction process.
Selected parts of sound files are sent to mix windows to form the ingredients
of a mix.
Mix Windows. The mix window is a graphical cue-sheet editor. Slices
of sound are stacked vertically like lines of text. A gain slider is available
for each slice. The position of ingredient sounds in the final mix is specified
by dragging the polygonal icons along the horizontal time axis. Fade-in
and fade-out times are changed by dragging the upper corners of the polygonal
icons. Slices can be edited with the cut, paste and copy operations, much
like lines of text are manipulated in a mouse-based text editor.
Multiple views of a single object are maintained and several mixes can
be worked on simultaneously. It is thus possible to work on a mix in small
manageable pieces in one window, whilst maintaining a global view in another
window.
Mixing operations can be applied to anything from a few samples to many
hours of sound. This means that MacMix may be applied to large final mixes
or to micro-surgery of sound.
Gestures
The user interface described above is not like the gestural controls
of a standard mixing console or the standard keyboard and screen of text
based mixing programs. Little is lost when taking away the typing and syntax
problems of text based mixing programs. However, the loss of the tight coupling
between audition and gesture, provided by standard mixing consoles, is too
great.
The problem of synchronising the audition of sound on a time-sharing
system with gestures on the Macintosh is not as hard to solve as it first
may appear. The play and record programs at IRCAM can be invoked
with a request that conversion be delayed until an external synchronisation
cue is provided. Any source of sound may be used to provide this cue. The
main application of this scheme is to synchronise transfers between computer
and tape recorder using click tracks. By using the sound output of the Macintosh
computer as a synchronisation source, clicks of the mouse button can be
accurately synchronised to sounds played on the host computer.
These gestures are logged on the bottom line of the view and mix windows.
They may then be used as reference points for future mixing.
Extensions
Uncoupling the user interface from the signal processing and sound storage
is extra work for the implementor, but this extra work pays off in many
ways.
Implementation and Testing Once the protocol between the Macintosh
and host computer was defined and documented, it was possible for the two
components of the system to be built simultaneously and independently. The
user interface was tested by simulating host responses on the Macintosh.
The host programs were tested by typing commands on a conventional terminal,
simulating those expected from the Macintosh.
New Workstations. At IRCAM, Dan Timis has developed a complete
reimplementation of the MacMix user interface for the Sun Workstation called
SunMix. The host software is exactly the same for both the Sun and the Macintosh
user interfaces.
It is interesting to compare the two user interfaces. The most striking
difference is that SunMix takes advantage of the large screen of the Sun
by using tiled windows and permanent menus rather than overlapping windows
and pull-down menus.
Rather than building a single user interface and porting it to very different
graphics systems, a new user interface is built using the tools and techniques
most easily available on each system. People already familiar with Macintosh
software find it very easy to use MacMix because the user interface follows
many of the conventions of popular word-processing, painting and drafting
programs.
New Hosts. The speed with which mixes can be described and changed
with MacMix highlights the need for real-time mixing. Robert Gross at Sprocket
Systems, a division of Lucasfilm Ltd., has implemented the host software
experimentally on a small signal processing and sound file storage system.
Reliability. Small workstations are noticably more reliable than
large time-sharing computer systems. For this reason, the MacMix protocol
is based on the idea that all critical information about a Mix is stored
in the workstation, not in the host. The only data the host is responsible
for is the sound file system. The protocol is set up so that if the host
computer malfunctions, users just wait for it to be repaired and can continue
from exactly where they were before the malfunction.
Extensibility. New commands have been added to the MacMix protocol
which extend the system beyond mixing, editing and browsing files. Dan Timis
has added commands to access IRCAM's Chant-Formes system, catalysing the
Pacha user interface developed by Fabrice Guedy. Robert Rowe has added a
command to transfer FFT data to provide spectral views.
Future Work
Reverberation... Many signal processing programs would benefit
from graphical user interfaces and integration in an environment such as
MacMix. Reverberation is a good example. It would be convenient to describe
the shape, size and characteristics of a hall to be simulated by drawing
sketches.
Perceptual Controls. The gain sliders and envelope functions control
signal amplitude. We would really like to control perceived loudness.
Spatial Mixing. Mixing, as commonly done, is a 2-dimensional process.
We would like to be able to mix sounds by placing and moving them in a simulated
space.
Acknowledgements
- Marie-Dominique Baudot
- Michele Dell-Prane
- Robert Gross
- Michael Hawley
- John Lassiter
- David Wessel
- Bennet Smith
- Dan Timis
References
PIKE, R. & LOCANTHI, B. & REISER, J. (1985), Hardware/software
trade-offs for bitmap graphics on the blit. Software-Practice and Experience
, 15, 131-152 (1985).
|