| Title | MacMix: Mixing Music with a Mouse |
| Publication Type | Conference Proceedings |
| Year of Publication | 1985 |
| Authors | Freed, Adrian |
| Conference Name | Proceedings of the USENIX Association Second Computer Graphics Workshop |
| Series Title | USENIX Association Second Computer Graphics Workshop, Monterey 1985. Proceedings |
| Pagination | 23-37 |
| Date Published | 1986 |
| Publisher | USENIX Assoc |
| Conference Location | Monterey, CA, USA |
| Abstract | MacMix is a user interface running on an Apple Macintosh computer, which communicates to processes on a UNIX machine that do the work of mixing and playing files of sound. In many ways the architecture of MacMix resembles that of airline reservation systems or bank networks, where a local terminal initiates transactions on a remote database. MacMix differs from these in that the remote database is physically close, and that the graphics user interface requires rapid response in bursts from the network. A video tape is available which shows MacMix in action. This medium is much better than the written word for communicating the details of a graphical user interface. This paper focuses on the unanticipated design and implementation problems of the project, most of which are related to networking |
| Full Text | IntroductionMacMix was developed at IRCAM, a contemporary music institute in Paris. Musicians are interested in using computers to synthesise, analyse and modify sounds. The process of mixing or blending separate sounds together is fundamental and very common in musical production of all sorts. Most of the difficulties we face with using computers to manipulate sound stem from a simple fact: we do not have a very compact digital representation of sound. We need roughly 1Mbyte of space to store 10 seconds of hi-fi quality sound. IRCAM has several Gigabytes of on-line disk storage and a large library of sound on disk packs and magnetic tape. Most installations with this amount of data to store and process have found that a few, very large, centralised systems are more cost effective than a network of distributed workstations. This may change in the next few years, but musicians seek tools today and they are seduced by convenience and efficacity of mouse-graphic based workstations. MacMix attempts to give a musician the control they seek over sound stored on a centralised system shared by other musicians. The architecture chosen was to use a popular mouse-graphics computer for the user interface and have it talk over a network to IRCAM's large sound file host computer. There are many such small computers to choose from. We chose the Macintosh because it is physically small, reliable, fairly cheap and has no noisy fan. Fan noise is inconvenient in recording studios. It would have perhaps been easier to use a UNIX workstation, since sounds at IRCAM are stored on large UNIX systems and off-the-shelf solutions exist for UNIX to UNIX communication problems. However, we will see that the network requirements of MacMix are not very cost-effectively served by such solutions. The User is in ControlUsers should feel that they are in control of MacMix and that services requested from the host result from their gestures. This requirement led to the single most important feature of the communication protocol between Macintosh and host: the Macintosh is always the master, the host is a slave. The Macintosh sends requests for service to the host and waits for one of three kinds of responses: request serviced, request not serviced, and no response at all. The slave host computer only sends messages to the Macintosh in response to a request. It cannot request service from the Macintosh. Stateless ServerThe second requirement of the protocol is derived from the observation that the host is not to be trusted. The host should not be trusted because it is less reliable than the Macintosh and it is not under the control of the user. It need not even be in the same building or country as the Macintosh. If the host can not be trusted, MacMix has to behave coherently when requests are not serviced and when the host fails to respond. It should also not be too sensitive to errors resulting from a host claiming it performed a service when it actually did not. These observations lead to the use of a stateless server on the host. If the host cannot be trusted, we should not assume that it can remember anything of early requests. This means that each request from the Macintosh has to include all the information needed for it to be completed by the host. An example of state stored in the Macintosh which could have been stored on the host is the current directory. All file names sent from the Macintosh are also sent with a directory path. Directories are managed by the Macintosh even though their contents exist on the host. PerformanceIn the long term MacMix uses very little of the available network bandwidth. This is because much of what the user is doing is provided for on the Macintosh. Every few seconds MacMix will issue a request for the contents of a directory or a few hundred samples in a sound file. The performance problem is that the user will be twiddling their thumbs as soon as the request is sent and will not stop twiddling until something changes on the screen in function of the hosts reply to the request. In other words latency is far more important than bandwidth in this application. This can be contrasted with networks used for moving files and mail from system to system. In these applications delays of minutes to hours may be acceptable, but there may be a large amount of data involved. MacMix requests are rarely longer than a 100 characters and replies rarely more than a few thousand characters. ImplementationMacMix has been successfully used with a very simple communication protocol running on RS232 communication lines at 19,200 Baud. The protocol is simple because we made the seemingly rash assumption that our communication lines were error free. The protocol has no error-correction. It has poor error detection facilities, but error recovery is very good since the Macintosh is so suspicious of the host. .P The assumption of error free lines is a good one, if the lines are installed correctly and operate over short distances. We have only observed one kind of error resulting from use of these lines: total failure. These errors occur for silly reasons: the cable is pulled from back of the Macintosh, someone accidentally unpatches the cable at the host, or rats eat through the cable. Whatever the cause, no amount of clever error-correcting will repair the cable. What if we wanted to run MacMix remotely? In France it is cheaper to use a national packet-switching network to access IRCAM than to use modems. This network provides error-free transmission. In the USA it would be necessary to use error-correcting modems. We do not feel that this is a serious limitation as conventional 1200 baud modems are too slow for this application. ImprovementsSeveral tools would have made implementation easier and more powerful. It is surprising that UNIX is not already supplied with a protocol similar to the one we devised. Many sites use UNIX to down or up load data to other systems. UNIX System V comes with the xt driver, which is used to communicate with software in the DMD terminal. Their protocol appears to be useful in applications like MacMix, but has not been published. Its main feature is that it multiplexes several channels on a single RS232 communication line. If this was available MacMix could control several host computers simultaneously, or users could interact with several UNIX processes at once on a single Macintosh. We use XON/XOFF flow control in both directions to avoid overflowing input buffers in the Macintosh and UNIX host. The UNIX host we are using, like many available, finds it buffers overflowing far more often than necessary. These systems cope with data rates from disks at least 10 times than those from serial ports. The latest generation of UNIX machines do not appear to have this problem. The MacMix server process is structured much like a UNIX shell. It parses a request. If the request makes sense it checks to see if another process is required to perform the request. If there is it handles communication with that process. If not it fulfills the request directly. Why not use the shell? Why did we create our own server process? The problem with the shell is that there is no easy way a program can parse its output and the output of the programs it executes. This output was designed for human consumption. There is a also a performance problem, as the shell spawns new processes for each user request. Our implementation spawns a single process for each service required and communicates directly to these processes. There is no overhead for process creation and death and these processes are able to cache open files. ConclusionWe believe that many interesting applications will be structured the way MacMix is. We have discussed some of the difficulties raised by such a structure. The rewards of overcoming them are hopefully demonstrated in the video tape of MacMix in action. AcknowledgementThe UNIX host software for MacMix was implemented by Robert Gross. |
| Full Text | View Full Text (HTML) |
| Full Text |
MacMix:Mixing Music with a Mouse©1986, 1996 Adrian Freed. All Rights Reserved.This is a slightly revised version of a paper presented at the ICMC in the Hague in 1986.MacMix is a tool for browsing, viewing, mixing and editing sounds. Its strengths are the result of an unsurprising user interface, implemented on an Apple Macintosh computer, and Robert Gross's efficient sound file system and mixing program. ArchitectureIRCAM, like many large computer music facilities, has chosen to build a centralised store for sound files. This is the most economical way of providing access to the Gigabytes of file storage required for hours of high-quality sound. MacMix gives a musician a familiar personal work-station environment for processing sounds shared amongst a community of users on a large time-sharing host computer.
The Macintosh computer is connected to the host using a serial communication line. This architecture was inspired by the Blit [Pike, Locanthi and Reiser, 1985]. A custom-designed, robust, stateless communication protocol is used between host and workstation. User InterfaceThree types of window are provided. Browser. The browser is used to explore the contents of the sound file system. A window of file names in the current sound file directory may be scrolled. Descriptive written information is available using the Get Info. button. The Play button is used to listen to the selected file. The samples in a file may be viewed using the the Open button.
View WIndows. View windows contain a visual representation of a sound. Most of the time, this representation is an amplitude envelope. If sufficient resolution is available, the samples themselves are displayed. Great care is taken to visually represent sound in an unambiguous way. Lines are not drawn between sample values as this would introduces visual information with no counterpart in the signal. Each pixel in the amplitude envelope display represents the largest absolute sample value in an integer number of samples. This definition of envelope has the advantages of being easy to interpret, cheap to compute and it facilitates identification of clicks or unusually large amplitude excursions. View windows provide a simple editing facility. A selected portion of a sound file can be extracted and written to a new file. An amplitude envelope can be sketched on top of a selected sound file fragment and will be superimposed on the signal during the extraction process. Selected parts of sound files are sent to mix windows to form the ingredients of a mix. Mix Windows. The mix window is a graphical cue-sheet editor. Slices of sound are stacked vertically like lines of text. A gain slider is available for each slice. The position of ingredient sounds in the final mix is specified by dragging the polygonal icons along the horizontal time axis. Fade-in and fade-out times are changed by dragging the upper corners of the polygonal icons. Slices can be edited with the cut, paste and copy operations, much like lines of text are manipulated in a mouse-based text editor.
Multiple views of a single object are maintained and several mixes can be worked on simultaneously. It is thus possible to work on a mix in small manageable pieces in one window, whilst maintaining a global view in another window. Mixing operations can be applied to anything from a few samples to many hours of sound. This means that MacMix may be applied to large final mixes or to micro-surgery of sound. GesturesThe user interface described above is not like the gestural controls of a standard mixing console or the standard keyboard and screen of text based mixing programs. Little is lost when taking away the typing and syntax problems of text based mixing programs. However, the loss of the tight coupling between audition and gesture, provided by standard mixing consoles, is too great. The problem of synchronising the audition of sound on a time-sharing system with gestures on the Macintosh is not as hard to solve as it first may appear. The play and record programs at IRCAM can be invoked with a request that conversion be delayed until an external synchronisation cue is provided. Any source of sound may be used to provide this cue. The main application of this scheme is to synchronise transfers between computer and tape recorder using click tracks. By using the sound output of the Macintosh computer as a synchronisation source, clicks of the mouse button can be accurately synchronised to sounds played on the host computer. These gestures are logged on the bottom line of the view and mix windows. They may then be used as reference points for future mixing. ExtensionsUncoupling the user interface from the signal processing and sound storage is extra work for the implementor, but this extra work pays off in many ways. Implementation and Testing Once the protocol between the Macintosh and host computer was defined and documented, it was possible for the two components of the system to be built simultaneously and independently. The user interface was tested by simulating host responses on the Macintosh. The host programs were tested by typing commands on a conventional terminal, simulating those expected from the Macintosh. New Workstations. At IRCAM, Dan Timis has developed a complete reimplementation of the MacMix user interface for the Sun Workstation called SunMix. The host software is exactly the same for both the Sun and the Macintosh user interfaces. It is interesting to compare the two user interfaces. The most striking difference is that SunMix takes advantage of the large screen of the Sun by using tiled windows and permenant menus rather than overlapping windows and pull-down menus. Rather than building a single user interface and porting it to very different graphics systems, a new user interface is built using the tools and techniques most easily available on each system. People already familiar with Macintosh software find it very easy to use MacMix because the user interface follows many of the conventions of popular word-processing, painting and drafting programs. New Hosts. The speed with which mixes can be described and changed with MacMix highlights the need for real-time mixing. Robert Gross at Sprocket Systems, a division of Lucasfilm Ltd., has implemented the host software experimentally on a small signal processing and sound file storage system. Reliability. Small workstations are noticably more reliable than large time-sharing computer systems. For this reason, the MacMix protocol is based on the idea that all critical information about a Mix is stored in the workstation, not in the host. The only data the host is responsible for is the sound file system. The protocol is set up so that if the host computer malfunctions, users just wait for it to be repaired and can continue from exactly where they were before the malfunction. Extensibility. New commands have been added to the MacMix protocol which extend the system beyond mixing, editing and browsing files. Dan Timis has added commands to access IRCAM's Chant-Formes system, catalysing the Pacha user interface developed by Fabrice Guedy. Robert Rowe has added a command to transfer FFT data to provide spectral views. Future WorkReverberation... Many signal processing programs would benefit from graphical user interfaces and integration in an environment such as MacMix. Reverberation is a good example. It would be convenient to describe the shape, size and characteristics of a hall to be simulated by drawing sketches. Perceptual Controls. The gain sliders and envelope functions control signal amplitude. We would really like to control perceived loudness. Spatial Mixing. Mixing, as commonly done, is a 2-dimensional process. We would like to be able to mix sounds by placing and moving them in a simulated space. Acknowledgements
ReferencesPIKE, R. & LOCANTHI, B. & REISER, J. (1985), Hardware/software trade-offs for bitmap graphics on the blit. Software-Practice and Experience , 15, 131-152 (1985). |