David Wessel used the metaphor of a stool to explain the importance of balancing the 3 legs of our activities at CNMAT: Research, Teaching and Musical Performance. My contribution to this balance was leading what I term here high-fidelity-prototype driven research. The qualifier is doing a lot of work. High fidelity for CNMAT meant working reliably and effectively enough for repeated live interactive performances, throughout semesters of teaching and in some cases for the decade or so it often takes for research programs. When David Wessel, Richard Felciano, John Wawrzynek and I started CNMAT in 1989, we wrote that combination of research ambition, teaching excellence and outstanding performance commitment into the design of the place. The studio was acoustically isolated, configured for ensembles, the control rooms were configured for both signal-flow experimentation and concert mixing, and the equipment we built ourselves was always cabled to be performable in any of those rooms within minutes. Our earliest CNMAT publication, the 1989 studio report, already names this stance plainly. It is a manifesto disguised as a facilities description.

A lot of research prototyping in HCI and engineering is, by choice, low-fidelity. A sketch, a Wizard-of-Oz simulation, a paper-tape interaction, a slide deck with screenshots are all reasonable tools for early exploration and elaboration of ideas. They help tease out significant avenues for future work but they have well-known limitations: they are too brittle for musicians to rehearse and perform with, and they cannot enter the practice life of anyone outside the lab. We chose, deliberately, to make prototypes that could be performed with in the evening, demonstrated in a class the next day and used for Ph. D. length research cycles. This is a little slower than sketching or building ephemeral prototypes but after a few years perfecting the base tooling and building communities of interest in industry, we were able to do it well enough and reliably enough that our apparatus instruments and software became a strong attraction to the place for great students collaborators and industry partners.

This orientation predates CNMAT emerging from my work and Miller Puckette's work with David Wessel at IRCAM in the mid-1980s. One prototype that exemplifies this was MacMix, the precursor to the digital audio workstation (DAW) David used it from the moment I got it going to create rich layered sonic textures - digitally composing, mixing and editing an album with Ushio Torikai - something rare do to digitally at that time. Without its combination of high audio fidelity, GUI quality (the first timeline editor) and facilitation of experimentation, MacMix wouldn't have been able to influence many commercial programs that followed including, Protools and the AudioFrame. I provided a high quality hardware prototype to stream audio from hard drive to audio codecs and a distribution license for MacMix to IMS enabling them to build the earliest commercially viable DAW, the Studer Editech Dyaxis system. During this period in the late 80s Miller Puckette refined his musical patching language prototype which became adapted as the now industry-gold-standard approach to creative real-time music synthesis programming environment Max/MSP and its open source cousin PD.

The first fifty years of computer music was marked by a chronic lack of sufficiently fast computers. The first significant high-fidelity prototype I architected for this was Reson8, a multi-DSP system engineered by Marie-Dominique Baudot. That project benefited from very useful feedback from the first active customers of it - now legendary composers and musical experimenters: Pauline Oliveros, Kaija Sahariaho and David Wessel. This architecture was so influential Marie-Dominique Baudot designed many variants for Studer Editech, Frog Design and others.

Computing performance isn't just about faster hardware - good algorithms that match the machines is an important strand of prototype-driven work such as: the FFT-inverse additive synthesis method developed with Xavier Rodet and Philippe Depalle (1993), the CAST additive synthesis system (1995), the fixed-point recursive oscillator work with Todd Hodes (1999), and the IIR filter externals for Max/MSP — resonators~, biquadbank~, peqbank~ — built with Tristan Jehan and Richard Dudas in 1999. These were not just papers with code attached. They were Max/MSP objects that performers used immediately. Resonators~ in particular remains in continuous active use across the Max/MSP community a quarter-century later. This longevity is an important aspect of fidelity to me. It requires a structuring of an idea in ways that it can serve as a platform - something I learned from the success of developing an access point for 3rd parties to the DAW in the form of the first audio plugins.

Visualization grew up alongside the engines because the engines needed editors. OpenSoundEdit (OSE, 1998–1999) with Amar Chaudhary, Sami Khoury and David Wessel was a unified three-dimensional direct-manipulation editor across spectral, time-domain and resonance representations, with a live synthesis server that played back the result as the parameters moved. It was a research artifact in the sense that it appeared at ICMC and AES, and it was a tool in the sense that composers used it to shape pieces. The same dual life characterized the Volumetric Acoustic Modeler that Sami Khoury and I built for the CNMAT Sound Spatialization Theatre. That paper crossed over to IEEE Visualization 1998 — a crossover that simply does not happen unless the underlying artifact is robust enough to be evaluated by a community outside its origin field.

CNMAT's Spatialization Theatre itself was both a laboratory and performance space to host important prototype immersive and spatial audio equipment. Many influential new techniques were developed such as the VBAP, and our unique 120-driver spherical speaker array. It was a room reconfigured by what we were learning, in concerts each week and in published research year by yearive conditions. It hosted dozens of composers and hundreds of concerts. Composers wrote pieces specifically for it. Visiting scholars in critical organology and sociology of music wrote about working there. The room was a high-fidelity instrument and it produced a community of practice just as our open software repos did.

The Connectivity Processor (the "Rimasbox") that Rimas Avizienis, Takahiko Suzuki, David Wessel and I created in 2000 deserves its own paragraph because it is a clean example of how fidelity propagates through a research community. We needed something better than the cabling chaos of the late-1990s laptop concert. The result was an FPGA-based bridge carrying ten channels of 24-bit audio, sixty-four channels of synchronized gesture data and four precisely-timestamped MIDI streams over 100BaseT Ethernet at sub-seven-millisecond round-trip latency into Max/MSP. Those numbers are not research aspirations; they are measurements taken under load while the box was being used in concerts. The Rimasbox is also where the augmented-instrument projects with Frances-Marie Uitti and the Slabs work with David Wessel started to become practical at performance fidelity.

The Slabs are the clearest single example I have of what a decade of high-fidelity prototyping with a key collaborator yielded. Slabs began in 2007 as a twenty-four pad pressure-sensitive force array. David Wessel made Slabs his primary performance instrument for the rest of his life. Each iteration of the hardware and the gesture-processing software was tested against his performance needs and refined. By 2016 the Slabs were not a prototype in any usual sense; they were a finished instrument with a repertoire, recordings, and constant touring schedule. David's death and the slow disappearance of the components used in the original build became the occasion for a different kind of paper — a case study in preventative digital musical instrument conservation that received best paper status at SMC 2016. That paper would not exist if the Slabs had been a low-fidelity prototype. It had to be canonical instrument first for the question of conservation to even arise.

Other augmented-instrument projects followed these values. The Augmented Cello (2006) was codeveloped with Frances-Marie Uitti, a virtuoso whose technique already pushed the cello into unusual territories with two bows and other extensions. Our job was not to design an instrument for an imagined performer but to support and extend a highly motivated collaborator. The novel rotary bow sensor, the mechanical tuning device and the polyphonic DSP effects in that project were all responses to particular musical moments Frances-Marie had been trying to realize. The Augmented Guitar work followed the same pattern with John Schott, Ahm Lee, Matthew Wright, Michael Zbyszynski and others. The published papers report and shared what we learned. The performances are where we validated it.

A parallel thread ran through the speaker work. The 120-Independent-Element Spherical Loudspeaker Array — the Icosaspeaker — that Rimas Avizienis, Andy Schmide, Matt Wright, David Wessel and I built in 2006 was, again, a physical artifact that people visited and recorded with. Acoustician colleagues in Europe and the United States carried out experiments that required physically working arrays with programmable radiation patterns, not simulations. Franz Zotter joined that work from Graz as we paired it with spherical microphone array.

The OSC story is about prototype fidelity. The 1997 paper Matthew Wright and I wrote did not invent OSC out of theoretical concerns. It crystallized a protocol that had been built because Roberto Morales, during his first visit to CNMAT, needed a way to communicate between his Max patch on a Macintosh and the Prolog interpreter that we could only run on a Sun workstation. His compositional strategy is still very interesting: he translated his musical phrases in real-time into Prolog propositions and synthesized a sonic response from the interpreter's attempts to prove or disprove the proposition. He needed it to work in front of an audience. We made it work and generalized it - recognizing that whatever computer you have in hand will benefit from talking to a remote network of faster ones. That became OSC. The whole embedded microprocessor applications of OSC with Andy Schmeder, Yotam Mann and Sebastian Madgwick that ran from 2008 through 2015 was developed in the same context - real performances raising grounded questions with instrumented prototypes to be addressed under load. The later odot, o.OM and o.io middleware work with John MacCallum, Jean Bresson, Rama Gottfried and Ilya Rostovtsev extended that line into dynamic, embedded computation using the OSC data design as its core data structure.

The piezoresistive-fabric multi-touch surfaces (2008-2011), the colocated sound surfaces with Jess Rowland (2012-2013), the conductive paper FingerPhone (2012), the haptic-and-capacitive platform with David Wessel at TEI 2015, and finally the Flex-Ability project with Lara Grant and Anna Blumenkranz (2017–2018) followed the same productive recipe: working physical artifact, refined by use, that supports actual playing or actual assistive interaction to exemplify rather than just illustrate robust ideas. Flex-Ability extended this disposition into accessibility — an e-textile interface kit for persons with limited mobility, EU WEAR Sustain-funded, exhibited at Ars Electronica 2018 and presented as a poster at CHI 2022 Workshop W36 (Toolkits & Wearables). The kit's value to its consultants in Munich and San Francisco came from the same source as Resonators~'s value to its Max users: it had to be reliable, repairable, and good enough to live with.

What did high-fidelity prototyping produce, in the end?

1) It produced artifacts and concepts that people still use decades later. Resonators~ and the Max/MSP filter externals are in active use a quarter-century later. The audio plug-in architecture spread industry-wide. OSC became the de facto interoperability protocol for networked creative tools. The Sound Spatialization Theatre and the Icosaspeaker hosted concerts and experiments for decades. The Augmented Cello and the Slabs entered the repertoire of the performers they were built for.

2) It produced a legacy of hundreds of peer-reviewed publications documenting undwerwritten by durable artifacts and apparatus whose latency can be measured, whose code can be downloaded, or whose physical presence can be visited. The publications below are a representative sample organized around the prototypes they document.

3) The most significant result of this approach is of course the education of composers, performers, engineers and scientists who passed through CNMAT, used the prototypes, contributed to them, and in many cases carry this working practice into their ongoing professional lives.

This list is a small sample of folk I have recently engaged with:

• Rimas Avizienis (Meyer Sound )
• Amanda Chaudhary (research engineer)
• Philippe Depalle (McGill)
• Richard Dudas (Hanyang University)
• Aaron Einbond (City University of London)
• Tristan Jehan (Echo Nest, Spotify)
• Peter Kassakian (Twitter)
• Sami Khoury (research engineer)
• Ahm Lee
• Sebastian Madgwick (x-io Technologies)
• John MacCallum (composer-researcher)
• Yotam Mann (Tone.js, Google)
• Roberto Morales (Universidad de Guanajuato)
• Jess Rowland (artist-researcher)
• Andy Schmeder (LBL)
• John Schott (composer-performer)
• Takahiko Suzuki (IAMAS)
• Rafael Valle (Nvidia, Meta)
• Matt Wright (Stanford CCRMA)
• David Zicarelli (Cycling74)
• Franz Zotter (IEM Graz)

A representative bibliography, organized roughly chronologically and grouped by the prototype the work was driven by:

Freed, A. MacMix Processing and Synthesis UserHooks (now known as Audio Plug-in). 1989. The audio plug-in architecture.

Wessel, D., Felciano, R., Freed, A., Wawrzynek, J. The Center for New Music and Audio Technologies: Studio Report 1989. ICMC 1989. The founding manifesto disguised as a facilities description.

Barrière, J-B., Baisnée, P-F., Freed, A., Baudot, M-D. A Digital Signal Multiprocessor and its Musical Application. 15th International Computer Music Conference, Ohio State University, 1989. The Reson8 architecture.

Freed, A., Gordon, K. DSP driver software for performance-oriented music synthesis systems. ICMC 1990.

Freed, A., Rodet, X., Depalle, P. Synthesis and control of hundreds of sinusoidal partials on a desktop computer without custom hardware. SPAT 1993; Performance, Synthesis and Control of Additive Synthesis on a Desktop Computer Using FFT-1, ICMC 1993. The FFT-inverse additive synthesis engine.

Freed, A. Codevelopment of user interface, control and digital signal processing with the HTM environment. SPAT 1994. The HTM environment.

Freed, A. Bring Your Own Control Additive Synthesis. ICMC 1995. Real-time control architecture for thousand-partial additive synthesis on a single desktop.

Wright, M., Freed, A. Open Sound Control: A New Protocol for Communicating with Sound Synthesizers. ICMC 1997. The protocol that grew out of Roberto Morales' visit.

Chaudhary, A., Freed, A., Khoury, S., Wessel, D. A 3D Graphical User Interface for Resonance Modeling. ICMC 1998; Chaudhary, A., Freed, A., Rowe, L. OpenSoundEdit: An Interactive Visualization and Editing Framework for Timbral Resources. ICMC 1998. The OSE editor.

Khoury, S., Freed, A., Wessel, D. Volumetric Modeling of Acoustic Fields in CNMAT's Sound Spatialization Theatre. AES 1998 / IEEE Visualization 1998. The Spatialization Theatre as a research instrument.

Hodes, T., Hauser, J., Freed, A., Wawrzynek, J., Wessel, D. A Fixed-Point Recursive Digital Oscillator for Additive Synthesis of Audio. ICASSP 1999. Patented method.

Jehan, T., Freed, A., Dudas, R. Musical Applications of New Filter Extensions to Max/MSP. ICMC 1999. Resonators~, biquadbank~, peqbank~ — externals still in active use 25 years later.

Chaudhary, A., Freed, A., Wright, M. An Open Architecture for Real-time Music Software. ICMC 2000. OSW.

Avizienis, R., Freed, A., Suzuki, T., Wessel, D. Scalable Connectivity Processor for Computer Music Performance Systems. ICMC 2000. The Rimasbox.

Freed, A., Isvan, O. Musical Applications of New, Multi-axis Guitar String Sensors. ICMC 2000.

Avizienis, R., Freed, A., Kassakian, P., Wessel, D. A Compact 120 Independent Element Spherical Loudspeaker Array with Programmable Radiation Patterns. AES 120, 2006. The Icosaspeaker.

Freed, A., Uitti, F-M., Wessel, D., Zbyszynski, M. Augmenting the Cello. NIME 2006; Freed, A., Lee, A., Schott, J., Uitti, F-M., Wright, M., Zbyszynski, M. Comparing Musical Control Structures and Signal Processing Strategies for the Augmented Cello and Guitar. ICMC 2006.

Berdahl, E., Smith III, J., Freed, A. Active Damping of a Vibrating String. 6th International Symposium on Active Noise and Vibration Control, 2006.

Wessel, D., Avizienis, R., Freed, A., Wright, M. A Force Sensitive Multi-touch Array Supporting Multiple 2-D Musical Control Structures. NIME 2007. The Slabs.

Schmeder, A., Freed, A. uOSC: The Open Sound Control Reference Platform for Embedded Devices. NIME 2008; Schmeder, A., Freed, A. A Low-level Embedded Service Architecture for Rapid DIY Design of Real-time Musical Instruments. NIME 2009; Schmeder, A., Freed, A. Implementation and Applications of Open Sound Control Timestamps. ICMC 2008.

Freed, A. Application of New Fiber and Malleable Materials for Agile Development of Augmented Instruments and Controllers. NIME 2008.

Freed, A. Novel and Forgotten Current-steering Techniques for Resistive Multitouch, Duotouch, and Polytouch Position Sensing with Pressure. NIME 2009.

Roh, J-S., Freed, A., Mann, Y., Wessel, D. Robust and Reliable Fabric, Piezoresistive Multitouch Sensing Surfaces for Musical Controllers. NIME 2011.

Freed, A., MacCallum, J., Schmeder, A. Dynamic, Instance-based, Object-Oriented Programming (OOP) in Max/MSP using Open Sound Control (OSC) Message Delegation. ICMC 2011.

Rowland, J., Freed, A. Flexible Surfaces for Interactive Audio. ACM ITS 2012; Colocated Surface Sound Interaction. SIGCHI 2013.

Freed, A. The Paper FingerPhone: A Case Study of Musical Instrument Redesign for Sustainability. NIME 2012.

Freed, A., Uitti, F-M., Mansfield, S., MacCallum, J. “Old” is the New “New”: a Fingerboard Case Study in Recrudescence as a NIME Development Strategy. NIME 2013.

MacCallum, J., Gottfried, R., Rostovtsev, I., Bresson, J., Freed, A. Dynamic Message-Oriented Middleware with Open Sound Control and Odot. ICMC 2015; Bresson, J., MacCallum, J., Freed, A. o.OM: Structured-Functional Communication between Computer Music Systems using OSC and Odot. FARM 2016.

Madgwick, S., Mitchell, T., Barreto, C., Freed, A. Simple Synchronization for Open Sound Control. ICMC 2015.

Freed, A., Wessel, D. An Accessible Platform for Exploring Haptic Interactions with Co-located Capacitive and Piezoresistive Sensors. TEI 2015.

Freed, A. David Wessel's Slabs: A Case Study in Preventative Digital Musical Instrument Conservation. SMC 2016. The closing of a decade-long collaborator-driven instrument-building arc.

Grant, L., Freed, A., Blumenkranz, A. Flexability: An E-textile Toolkit for Persons with Limited Mobility. CHI 2022 Workshop W36 (Toolkits & Wearables), April 30, 2022. The accessibility-focused continuation of the e-textile prototype lineage. Project documentation at flex-ability.org.