Dynamic Message-Oriented Middleware with Open Sound Control and Odot, John MacCallum, Rama Gottfried, Ilya Rostovtsev Jean Bresson Adrian Freed , International Computer Music Conference, 25/09/2015, University of North Texas, Denton, TX, USA, (2015)

o.OM: Structured-Functional Communication between Computer Music Systems using OSC and Odot, Bresson, Jean, MacCallum John, and Freed Adrian , FARM2016, Nara, Japan, (2016)

The Open Sound Control Specification

Version 1.0, March 26 2002, Matt Wright

April 19 2012, Adrian Freed

Introduction

Open Sound Control (OSC) is a open, transport-independent, message-based encoding developed for communication among agents, process, threads, computers, sound synthesizers, and other multimedia devices.

Best Practices for Open Sound Control, Schmeder, Andrew, Freed Adrian, and Wessel David , Linux Audio Conference, Utrecht, NL, (2010)

This is the third musical instrument controller I have prepared for Maker Faire. This one combines lessons learned from the previous two: the tablo e-textile controller and the large koto/guitar hybrid from last year.

This simple controller is inspired by the south African kalimba. The kalimba lends itself to rapid assembly because of its use of a single central bar held down by two screws to trap the array of tines between two pivot points. Wooden tines are used in this prototype because they are faster to shape than the traditional metal and this controller doesn’t require the tines to be tuned. The flexibility of copper tape is exploited as strips follow the contour of the flat base around the curve of a half-round pressure pivot.

This controller with no direct antecedents is made by draping conductive stretchable fabric over a cereal bowl creating a curve similar to the famous one called the witch [sic] of Agnesi. Resistive strips around the bowl are shorted out by the fabric and the varying resistances are measured and sent as OSC messages via USB control software on a host computer - typically resonance model sounds in Max/MSP.

The fabric is stretched in an embroidery hoop and draped over an inverted circular bowl. A piece of conductive plastic cut in a special shape forms a corolla on the surface of the bowl. The tips of each petal are folded inside the bowl and taped with conducting-adhesive copper tape or fabric tape. The microcontroller board measures the electrical resistances of these petals from their tip to a common center established with a gold-plated fabric ring at the flat of the bowl. As the conductive stretchable fabric (the “calyx” to complete the flower analogy) is displaced towards the bowl it shorts out different lengths of each conductive plastic petal.

The result is a circular array of nearly mass-less displacement sensors. The gesture-to-displacement relationship changes according to distance from the center of the bowl (variable “gearing)”. This allows for several different playing styles.

One style, similar to hand drum technique, involves tapping the fabric surface directly onto the bowl with the fingers of one hand and leaning towards the other side of the bowl with the palm. Another style involves both hands interacting from the outer hoop towards and around the base of the bowl.

You may also put it on the ground and use it as multiple foot “pedals”.

A second version increases the number of fabric pressure sensors on the base board from 2 to 16.

An important point about the Tablo is that the drape sensing allows both very delicate light touch to be sensed along with very muscular or fast gestures when the fabric touches the piezoelectric strips. It is very hard with stacked position and pressure sensors to achieve this dynamic range so essential for compelling hand drum gestures.

It has been designed so that all the important sensing nodes are surrounded by grounded fabric to keep the noise down.

The hoop is easily removed so that the only moving part (the stretch fabric) can be replaced or washed.

A single pressure sensor is snuck into the top of the bowl.

There is lots of interesting potential math behind the shape of the bowl: spherical doesn’t work well, a catenary may yield a nice linear relationship between the finger position and sensed value, the “found object” bowl used is very good and almost linear.

You can scale this design up to support multiple people playing it if you can afford the fabric.

You can also build it in a linear/keyboard style instead of "in the round".

This example uses the recommended OSC SLIP wrapping for OSC serial messages. Attached is an example Max/MSP patch to display the OSC messages which represent the pin states, temperature and power supply voltages. You may need slipOSC from the CNMAT downloads page to unwrap the USB serial OSC. This is a simple introduction to building OSC messages. Yotam Mann built a complete library for Arduino that does the full OSC encoding available on github CNMAT.

/*
 * OSCuino Rough
 * Copyright 2009 Adrian Freed. All Rights Reserved
 * Read and send the state of the Arduino pins to a host
 * using the recommended Open Sound Control (OSC)  serial wrapping: slip
 *
 * version 0.95
   tested on teensy 2.0, teensy++ 1.0, lilypad, basic 168 Arduino
   lilypad.
   The Arduino Mega works if you build with the latest release
   candidate Arduino.
 *
 */
#include <avr/pgmspace.h>

// The following OSC API is temporary. Although it generates valid OSC packets
// it does not use OSC best practices or provide complete OSC features
// In particular there are no time tags (no standard clock on Arduino)
// or bundles and this api only provides for vectors of the same OSC type
// these issues will be addressed in a C++ library

const byte eot = 0300;
const byte slipesc = 0333;
const byte slipescend = 0334;
const byte slipescesc = 0335;
void oscoutbyte(byte b)
{
  if(b==eot){ 
    Serial.print(slipesc);
    Serial.print(slipescend); 
  }
  else if(b==slipesc) {  
    Serial.print(slipesc);
    Serial.print(slipescesc); 
  }
  else  Serial.print(b, BYTE);

}
void oscout10b(unsigned int b)
{
  Serial.print(0, BYTE);
  Serial.print(0, BYTE);

  oscoutbyte((b>>2));
  oscoutbyte((b<<6));
}
void oscoutword( int b)
{
  Serial.print(0, BYTE);
  Serial.print(0, BYTE);
  // Serial.print(2);
  oscoutbyte((b>>8));
  oscoutbyte((b>>0));

}
void oscoutlong(unsigned long b)
{
  oscoutbyte((b>>24));
  oscoutbyte((b>>16));
  oscoutbyte((b>>8));
  oscoutbyte((b>>0));

}
void oscoutfloat(float f)
{
  union
  {
    float f;
    unsigned long l;
  } 
  x;
  x.f = f;
  oscoutlong(x.l);
}
void sendOSCempty(char *name, char *s)
{
  Serial.print(eot,BYTE);
  Serial.print(name);

  int n =0;
  while( *name++)
    ++n;
  n %= 4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);
  Serial.print(",");
  n = 1;
  while(*s)
    Serial.print(*s++), ++n;

  n %= 4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);

  Serial.print(eot,BYTE);
}    


void sendOSCints(char *name,  int *integers, int length)
{
  Serial.print(eot,BYTE);
  Serial.print(name);

  int n =0;
  while( *name++)
    ++n;
  n %= 4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);
  Serial.print(",");
  for(int i=0;i<length;++i)
  {
    Serial.print("i");
  }
  n = (length+1 )%4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);

  for(int i=0;i<length;++i)

    oscoutword(integers[i]);
  Serial.print(eot,BYTE);

} 
void sendOSCfloats(char *name,  float *floats, int length)
{
    sendOSCthings( name, 'f', (uint32_t *) floats, length);
}

void sendOSCthings(char *name, char type, uint32_t *data, int length)
{
  Serial.print(eot,BYTE);
  Serial.print(name);

  int n =0;
  while( *name++)
    ++n;
  n %= 4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);
  Serial.print(",");
  for(int i=0;i<length;++i)
  {
    Serial.print(type);
  }
  n = (length+1 )%4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);

  for(int i=0;i<length;++i)
    oscoutlong(data[i]);
  Serial.print(eot,BYTE);
}    
 static inline  void sendOSClongs(char *name,  long *longs, int length)
{
     sendOSCthings(name,'i', (uint32_t *)longs,  length);
}
void sendOSCbits(char *name, boolean *bits, int length)
{
  Serial.print(eot,BYTE);
  Serial.print(name);

  int n =0;
  while( *name++)
    ++n;
  n %= 4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);
  Serial.print(",");
  n = 1;
  for(int i=0;i<length;++i)
    Serial.print(bits[i]?'T':'F'),++n;
 
  n %= 4;
  n = 4-n;
  //Serial.print(n);
  while(n--)
    Serial.print(0,BYTE);

  Serial.print(eot,BYTE);
}    

static PROGMEM float floattable[1024] = {
0.000000, 0.000978, 0.001955, 0.002933, 0.003910, 0.004888, 0.005865, 0.006843, 0.007820, 0.008798, 0.009775, 
0.010753, 0.011730, 0.012708, 0.013685, 0.014663, 0.015640, 0.016618, 0.017595, 0.018573, 0.019550, 
0.020528, 0.021505, 0.022483, 0.023460, 0.024438, 0.025415, 0.026393, 0.027370, 0.028348, 0.029326, 
0.030303, 0.031281, 0.032258, 0.033236, 0.034213, 0.035191, 0.036168, 0.037146, 0.038123, 0.039101, 
0.040078, 0.041056, 0.042033, 0.043011, 0.043988, 0.044966, 0.045943, 0.046921, 0.047898, 0.048876, 
0.049853, 0.050831, 0.051808, 0.052786, 0.053763, 0.054741, 0.055718, 0.056696, 0.057674, 0.058651, 
0.059629, 0.060606, 0.061584, 0.062561, 0.063539, 0.064516, 0.065494, 0.066471, 0.067449, 0.068426, 
0.069404, 0.070381, 0.071359, 0.072336, 0.073314, 0.074291, 0.075269, 0.076246, 0.077224, 0.078201, 
0.079179, 0.080156, 0.081134, 0.082111, 0.083089, 0.084066, 0.085044, 0.086022, 0.086999, 0.087977, 
0.088954, 0.089932, 0.090909, 0.091887, 0.092864, 0.093842, 0.094819, 0.095797, 0.096774, 0.097752, 
0.098729, 0.099707, 0.100684, 0.101662, 0.102639, 0.103617, 0.104594, 0.105572, 0.106549, 0.107527, 
0.108504, 0.109482, 0.110459, 0.111437, 0.112414, 0.113392, 0.114370, 0.115347, 0.116325, 0.117302, 
0.118280, 0.119257, 0.120235, 0.121212, 0.122190, 0.123167, 0.124145, 0.125122, 0.126100, 0.127077, 
0.128055, 0.129032, 0.130010, 0.130987, 0.131965, 0.132942, 0.133920, 0.134897, 0.135875, 0.136852, 
0.137830, 0.138807, 0.139785, 0.140762, 0.141740, 0.142717, 0.143695, 0.144673, 0.145650, 0.146628, 
0.147605, 0.148583, 0.149560, 0.150538, 0.151515, 0.152493, 0.153470, 0.154448, 0.155425, 0.156403, 
0.157380, 0.158358, 0.159335, 0.160313, 0.161290, 0.162268, 0.163245, 0.164223, 0.165200, 0.166178, 
0.167155, 0.168133, 0.169110, 0.170088, 0.171065, 0.172043, 0.173021, 0.173998, 0.174976, 0.175953, 
0.176931, 0.177908, 0.178886, 0.179863, 0.180841, 0.181818, 0.182796, 0.183773, 0.184751, 0.185728, 
0.186706, 0.187683, 0.188661, 0.189638, 0.190616, 0.191593, 0.192571, 0.193548, 0.194526, 0.195503, 
0.196481, 0.197458, 0.198436, 0.199413, 0.200391, 0.201369, 0.202346, 0.203324, 0.204301, 0.205279, 
0.206256, 0.207234, 0.208211, 0.209189, 0.210166, 0.211144, 0.212121, 0.213099, 0.214076, 0.215054, 
0.216031, 0.217009, 0.217986, 0.218964, 0.219941, 0.220919, 0.221896, 0.222874, 0.223851, 0.224829, 
0.225806, 0.226784, 0.227761, 0.228739, 0.229717, 0.230694, 0.231672, 0.232649, 0.233627, 0.234604, 
0.235582, 0.236559, 0.237537, 0.238514, 0.239492, 0.240469, 0.241447, 0.242424, 0.243402, 0.244379, 
0.245357, 0.246334, 0.247312, 0.248289, 0.249267, 0.250244, 0.251222, 0.252199, 0.253177, 0.254154, 
0.255132, 0.256109, 0.257087, 0.258065, 0.259042, 0.260020, 0.260997, 0.261975, 0.262952, 0.263930, 
0.264907, 0.265885, 0.266862, 0.267840, 0.268817, 0.269795, 0.270772, 0.271750, 0.272727, 0.273705, 
0.274682, 0.275660, 0.276637, 0.277615, 0.278592, 0.279570, 0.280547, 0.281525, 0.282502, 0.283480, 
0.284457, 0.285435, 0.286413, 0.287390, 0.288368, 0.289345, 0.290323, 0.291300, 0.292278, 0.293255, 
0.294233, 0.295210, 0.296188, 0.297165, 0.298143, 0.299120, 0.300098, 0.301075, 0.302053, 0.303030, 
0.304008, 0.304985, 0.305963, 0.306940, 0.307918, 0.308895, 0.309873, 0.310850, 0.311828, 0.312805, 
0.313783, 0.314761, 0.315738, 0.316716, 0.317693, 0.318671, 0.319648, 0.320626, 0.321603, 0.322581, 
0.323558, 0.324536, 0.325513, 0.326491, 0.327468, 0.328446, 0.329423, 0.330401, 0.331378, 0.332356, 
0.333333, 0.334311, 0.335288, 0.336266, 0.337243, 0.338221, 0.339198, 0.340176, 0.341153, 0.342131, 
0.343109, 0.344086, 0.345064, 0.346041, 0.347019, 0.347996, 0.348974, 0.349951, 0.350929, 0.351906, 
0.352884, 0.353861, 0.354839, 0.355816, 0.356794, 0.357771, 0.358749, 0.359726, 0.360704, 0.361681, 
0.362659, 0.363636, 0.364614, 0.365591, 0.366569, 0.367546, 0.368524, 0.369501, 0.370479, 0.371457, 
0.372434, 0.373412, 0.374389, 0.375367, 0.376344, 0.377322, 0.378299, 0.379277, 0.380254, 0.381232, 
0.382209, 0.383187, 0.384164, 0.385142, 0.386119, 0.387097, 0.388074, 0.389052, 0.390029, 0.391007, 
0.391984, 0.392962, 0.393939, 0.394917, 0.395894, 0.396872, 0.397849, 0.398827, 0.399804, 0.400782, 
0.401760, 0.402737, 0.403715, 0.404692, 0.405670, 0.406647, 0.407625, 0.408602, 0.409580, 0.410557, 
0.411535, 0.412512, 0.413490, 0.414467, 0.415445, 0.416422, 0.417400, 0.418377, 0.419355, 0.420332, 
0.421310, 0.422287, 0.423265, 0.424242, 0.425220, 0.426197, 0.427175, 0.428152, 0.429130, 0.430108, 
0.431085, 0.432063, 0.433040, 0.434018, 0.434995, 0.435973, 0.436950, 0.437928, 0.438905, 0.439883, 
0.440860, 0.441838, 0.442815, 0.443793, 0.444770, 0.445748, 0.446725, 0.447703, 0.448680, 0.449658, 
0.450635, 0.451613, 0.452590, 0.453568, 0.454545, 0.455523, 0.456500, 0.457478, 0.458456, 0.459433, 
0.460411, 0.461388, 0.462366, 0.463343, 0.464321, 0.465298, 0.466276, 0.467253, 0.468231, 0.469208, 
0.470186, 0.471163, 0.472141, 0.473118, 0.474096, 0.475073, 0.476051, 0.477028, 0.478006, 0.478983, 
0.479961, 0.480938, 0.481916, 0.482893, 0.483871, 0.484848, 0.485826, 0.486804, 0.487781, 0.488759, 
0.489736, 0.490714, 0.491691, 0.492669, 0.493646, 0.494624, 0.495601, 0.496579, 0.497556, 0.498534, 
0.499511, 0.500489, 0.501466, 0.502444, 0.503421, 0.504399, 0.505376, 0.506354, 0.507331, 0.508309, 
0.509286, 0.510264, 0.511241, 0.512219, 0.513196, 0.514174, 0.515152, 0.516129, 0.517107, 0.518084, 
0.519062, 0.520039, 0.521017, 0.521994, 0.522972, 0.523949, 0.524927, 0.525904, 0.526882, 0.527859, 
0.528837, 0.529814, 0.530792, 0.531769, 0.532747, 0.533724, 0.534702, 0.535679, 0.536657, 0.537634, 
0.538612, 0.539589, 0.540567, 0.541544, 0.542522, 0.543500, 0.544477, 0.545455, 0.546432, 0.547410, 
0.548387, 0.549365, 0.550342, 0.551320, 0.552297, 0.553275, 0.554252, 0.555230, 0.556207, 0.557185, 
0.558162, 0.559140, 0.560117, 0.561095, 0.562072, 0.563050, 0.564027, 0.565005, 0.565982, 0.566960, 
0.567937, 0.568915, 0.569892, 0.570870, 0.571848, 0.572825, 0.573803, 0.574780, 0.575758, 0.576735, 
0.577713, 0.578690, 0.579668, 0.580645, 0.581623, 0.582600, 0.583578, 0.584555, 0.585533, 0.586510, 
0.587488, 0.588465, 0.589443, 0.590420, 0.591398, 0.592375, 0.593353, 0.594330, 0.595308, 0.596285, 
0.597263, 0.598240, 0.599218, 0.600196, 0.601173, 0.602151, 0.603128, 0.604106, 0.605083, 0.606061, 
0.607038, 0.608016, 0.608993, 0.609971, 0.610948, 0.611926, 0.612903, 0.613881, 0.614858, 0.615836, 
0.616813, 0.617791, 0.618768, 0.619746, 0.620723, 0.621701, 0.622678, 0.623656, 0.624633, 0.625611, 
0.626588, 0.627566, 0.628543, 0.629521, 0.630499, 0.631476, 0.632454, 0.633431, 0.634409, 0.635386, 
0.636364, 0.637341, 0.638319, 0.639296, 0.640274, 0.641251, 0.642229, 0.643206, 0.644184, 0.645161, 
0.646139, 0.647116, 0.648094, 0.649071, 0.650049, 0.651026, 0.652004, 0.652981, 0.653959, 0.654936, 
0.655914, 0.656891, 0.657869, 0.658847, 0.659824, 0.660802, 0.661779, 0.662757, 0.663734, 0.664712, 
0.665689, 0.666667, 0.667644, 0.668622, 0.669599, 0.670577, 0.671554, 0.672532, 0.673509, 0.674487, 
0.675464, 0.676442, 0.677419, 0.678397, 0.679374, 0.680352, 0.681329, 0.682307, 0.683284, 0.684262, 
0.685239, 0.686217, 0.687195, 0.688172, 0.689150, 0.690127, 0.691105, 0.692082, 0.693060, 0.694037, 
0.695015, 0.695992, 0.696970, 0.697947, 0.698925, 0.699902, 0.700880, 0.701857, 0.702835, 0.703812, 
0.704790, 0.705767, 0.706745, 0.707722, 0.708700, 0.709677, 0.710655, 0.711632, 0.712610, 0.713587, 
0.714565, 0.715543, 0.716520, 0.717498, 0.718475, 0.719453, 0.720430, 0.721408, 0.722385, 0.723363, 
0.724340, 0.725318, 0.726295, 0.727273, 0.728250, 0.729228, 0.730205, 0.731183, 0.732160, 0.733138, 
0.734115, 0.735093, 0.736070, 0.737048, 0.738025, 0.739003, 0.739980, 0.740958, 0.741935, 0.742913, 
0.743891, 0.744868, 0.745846, 0.746823, 0.747801, 0.748778, 0.749756, 0.750733, 0.751711, 0.752688, 
0.753666, 0.754643, 0.755621, 0.756598, 0.757576, 0.758553, 0.759531, 0.760508, 0.761486, 0.762463, 
0.763441, 0.764418, 0.765396, 0.766373, 0.767351, 0.768328, 0.769306, 0.770283, 0.771261, 0.772239, 
0.773216, 0.774194, 0.775171, 0.776149, 0.777126, 0.778104, 0.779081, 0.780059, 0.781036, 0.782014, 
0.782991, 0.783969, 0.784946, 0.785924, 0.786901, 0.787879, 0.788856, 0.789834, 0.790811, 0.791789, 
0.792766, 0.793744, 0.794721, 0.795699, 0.796676, 0.797654, 0.798631, 0.799609, 0.800587, 0.801564, 
0.802542, 0.803519, 0.804497, 0.805474, 0.806452, 0.807429, 0.808407, 0.809384, 0.810362, 0.811339, 
0.812317, 0.813294, 0.814272, 0.815249, 0.816227, 0.817204, 0.818182, 0.819159, 0.820137, 0.821114, 
0.822092, 0.823069, 0.824047, 0.825024, 0.826002, 0.826979, 0.827957, 0.828935, 0.829912, 0.830890, 
0.831867, 0.832845, 0.833822, 0.834800, 0.835777, 0.836755, 0.837732, 0.838710, 0.839687, 0.840665, 
0.841642, 0.842620, 0.843597, 0.844575, 0.845552, 0.846530, 0.847507, 0.848485, 0.849462, 0.850440, 
0.851417, 0.852395, 0.853372, 0.854350, 0.855327, 0.856305, 0.857283, 0.858260, 0.859238, 0.860215, 
0.861193, 0.862170, 0.863148, 0.864125, 0.865103, 0.866080, 0.867058, 0.868035, 0.869013, 0.869990, 
0.870968, 0.871945, 0.872923, 0.873900, 0.874878, 0.875855, 0.876833, 0.877810, 0.878788, 0.879765, 
0.880743, 0.881720, 0.882698, 0.883675, 0.884653, 0.885630, 0.886608, 0.887586, 0.888563, 0.889541, 
0.890518, 0.891496, 0.892473, 0.893451, 0.894428, 0.895406, 0.896383, 0.897361, 0.898338, 0.899316, 
0.900293, 0.901271, 0.902248, 0.903226, 0.904203, 0.905181, 0.906158, 0.907136, 0.908113, 0.909091, 
0.910068, 0.911046, 0.912023, 0.913001, 0.913978, 0.914956, 0.915934, 0.916911, 0.917889, 0.918866, 
0.919844, 0.920821, 0.921799, 0.922776, 0.923754, 0.924731, 0.925709, 0.926686, 0.927664, 0.928641, 
0.929619, 0.930596, 0.931574, 0.932551, 0.933529, 0.934506, 0.935484, 0.936461, 0.937439, 0.938416, 
0.939394, 0.940371, 0.941349, 0.942326, 0.943304, 0.944282, 0.945259, 0.946237, 0.947214, 0.948192, 
0.949169, 0.950147, 0.951124, 0.952102, 0.953079, 0.954057, 0.955034, 0.956012, 0.956989, 0.957967, 
0.958944, 0.959922, 0.960899, 0.961877, 0.962854, 0.963832, 0.964809, 0.965787, 0.966764, 0.967742, 
0.968719, 0.969697, 0.970674, 0.971652, 0.972630, 0.973607, 0.974585, 0.975562, 0.976540, 0.977517, 
0.978495, 0.979472, 0.980450, 0.981427, 0.982405, 0.983382, 0.984360, 0.985337, 0.986315, 0.987292, 
0.988270, 0.989247, 0.990225, 0.991202, 0.992180, 0.993157, 0.994135, 0.995112, 0.996090, 0.997067, 
0.998045, 0.999022, 1.000000, 
};


static inline float floatfrom10bitint(int n)
{
  return pgm_read_float(floattable+constrain(n,0,1023));
}

inline void digitalPullup(byte pin, boolean b) { 
  pinMode(pin, INPUT); 
  digitalWrite(pin, b?HIGH:LOW); 
}

#if defined(__AVR_ATmega1280__)
#define MAXADC 16
const int numberOfArduinoDigitalPins=54;
inline void analogPullup(byte pin, boolean b) { 
  digitalPullup(pin+54,b); 
}
inline void analogOutWrite(byte pin, byte b) {
  pinMode(pin+54, OUTPUT);  
  digitalWrite(pin+54,b);
}

#else
// teensy++
#if defined(__AVR_AT90USB646__) || defined(__AVR_AT90USB1286__)

#define MAXADC 8
const int numberOfArduinoDigitalPins=38;
inline void analogPullup(byte pin, boolean b) { 
   pinMode(38+pin,b?INPUT_PULLUP: INPUT); 
}
inline void analogOutWrite(byte pin, byte b) {
  pinMode(38+pin, OUTPUT);  // change directions of an analog pin
  digitalWrite(38+pin,b); 
}
#else
//AT90USB1286 teensy++
#if defined(__AVR_ATmega32U4__)
//teensy 2.0
#define MAXADC 12
const int numberOfArduinoDigitalPins=11; // actually the teensy 2.0 has two more non contiguously addressed (22 and 23)
inline void analogPullup(byte pin, boolean b) { 
 if(pin==11)
    pin = -1; // ouch read the diagram carefully! http://www.pjrc.com/teensy/pinout.html
   pinMode(21-pin,b?INPUT_PULLUP: INPUT); 
}
typedef enum {
  endOfpinlist=-1, a0=14,a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15 } 
pinnames;
inline void analogOutWrite(byte pin, byte b) {
  if(pin==11)
    pin = -1; // ouch
  pinMode(21-pin, OUTPUT);  // change directions of an analog pin
  digitalWrite(21-pin,b); 
}
#else
//6 or 8 depending
#define MAXADC 8
const int numberOfArduinoDigitalPins=13;
inline void analogPullup(byte pin, boolean b) { 
  digitalPullup(pin+14,b?HIGH:LOW); 
}

inline void analogOutWrite(byte pin, byte b) {
 
  pinMode(pin+14, OUTPUT);  // change directions of an analog pin
  digitalWrite(pin+14,b); 
}
#endif
#endif
#endif
const boolean debugging = false; // send the processed data over serial console for debugging instead of OSC
const boolean fulldebugging = false; // turn this on to get all the raw data in a form you can read on the serial console


void setup() {
  Serial.begin(debugging?9600:57600);
}
const int dtime=0;
#define BANDGAPREF 14   // special indicator that we want to measure the bandgap

void loop(){
 // analog inputs
  int adcs[MAXADC];
  float fbuf[MAXADC];
  int i;
  
  for(i=0;i<MAXADC;++i)
    analogPullup(i,false);
  for(i=0;i<MAXADC;++i)
    adcs[i] = analogRead(i); 
  for(i=0;i<MAXADC;++i)
    fbuf[i] = floatfrom10bitint(adcs[i]);
  sendOSCfloats("/analog", fbuf, MAXADC);  
 
  for(i=0;i<MAXADC;++i)
    analogPullup(i,true);
  for(i=0;i<MAXADC;++i)
    adcs[i] = analogRead(i); 
  for(i=0;i<MAXADC;++i)
    fbuf[i] = floatfrom10bitint(adcs[i]);
  sendOSCfloats("/up/analog", fbuf, MAXADC);  

   // temperature
  // power supply voltage
  // (clock rate)
  //digital pins
  boolean arduinodigitalpins[numberOfArduinoDigitalPins];
  for(i=0;i<numberOfArduinoDigitalPins;++i)
    digitalPullup(i,false);
  for(i=0;i<numberOfArduinoDigitalPins;++i)
    arduinodigitalpins[i] = digitalRead(i);
 sendOSCbits("/digital", arduinodigitalpins, numberOfArduinoDigitalPins);
  
  for(i=0;i<numberOfArduinoDigitalPins;++i)
    digitalPullup(i,true);
  for(i=0;i<numberOfArduinoDigitalPins;++i)
    arduinodigitalpins[i] = digitalRead(i);
 sendOSCbits("/up/digital", arduinodigitalpins, numberOfArduinoDigitalPins);
 
 #if defined(__AVR_ATmega32U4__)
Serial.send_now(); 
#endif
 #if !defined(__AVR_ATmega8__)

// temperature and power supply measurement on some Arduinos 
    // powersupply
    int result;
  // Read 1.1V reference against AVcc
#if defined(__AVR_ATmega32U4__)
  ADMUX = 0x40 | _BV(MUX4)| _BV(MUX3) | _BV(MUX2) | _BV(MUX1);
  ADCSRB =  0;
  //  ADCSRB = DEFAULT_ADCSRB | (1<<MUX5);
#elif  defined(__AVR_AT90USB646__) || defined(__AVR_AT90USB1286__)    || defined(__AVR_ATmega1280__) 
  ADMUX = 0x40| _BV(MUX4) | _BV(MUX3) | _BV(MUX2) | _BV(MUX1) ;
    ADCSRB =  0;
#else
  ADMUX = _BV(REFS0) | _BV(MUX3) | _BV(MUX2) | _BV(MUX1);
#endif
  delayMicroseconds(300); // wait for Vref to settle
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
   result = ADCL;
  result |= ADCH<<8;
#endif
//    analogReference(EXTERNAL);

  float supplyvoltage = 1.1264 *1023 / result;
  sendOSCfloats("/power", &supplyvoltage, 1);


#if defined(__AVR_ATmega32U4__) ||    (!defined(__AVR_ATmega1280__) && !defined(__AVR_ATmega8__) && !defined(__AVR_AT90USB646__) && !defined(__AVR_AT90USB1286__))
 // temperature

#if defined(__AVR_ATmega32U4__)
  ADMUX =  _BV(REFS1) | _BV(REFS0) | _BV(MUX2) | _BV(MUX1) | _BV(MUX0);
   ADCSRB =  _BV(MUX5);
#else
  ADMUX = _BV(REFS1) | _BV(REFS0) | _BV(MUX3);
#endif
  

  delayMicroseconds(200); // wait for Vref to settle
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
  result = ADCL;
  result |= ADCH<<8;

   float temp= result/1023.0;
     sendOSCfloats("/temperature", &temp, 1);
   analogReference(DEFAULT);
 #endif
 
  long l=micros();
  sendOSClongs("/microseconds",&l,1);
#if defined(__AVR_ATmega32U4__)
Serial.send_now(); 
#endif
}

1 Introduction

A better integration of computers, controllers and sound synthesizers will lead to lower costs, increased reliability, greater user convenience, and more reactive musical control. The prevailing technologies to interconnect these elements are bus (motherboard or PCI), operating system interface (software synthesis), or serial LAN (Firewire, USB, ethernet, fast ethernet). It is easy to adapt MIDI streams to this new communication substrate [11]. To do so needlessly discards new potential and perpetuates MIDI's well-documented flaws. Instead we have designed a new protocol optimized for modern transport technologies.

Open SoundControl is an open, efficient, transport-independent, message-based protocol developed for communication among computers, sound synthesizers, and other multimedia devices. Open SoundControl is a machine and operating system neutral protocol and readily implementable on constrained, embedded systems.

We begin by examining various networking technologies suitable for carrying Open SoundControl data and discuss shared features of these technologies that impact the design of our protocol. Next we discuss the encoding and formatting of data. We describe our novel addressing scheme based on a URL-style symbolic syntax. In our last section we specify a number of query messages that request information from an Open SoundControl system.

2 Transport Layer Assumptions

Open SoundControl is a transport-independent protocol, meaning that it is a format for data that can be carried across a variety of networking technologies. Networking technologies currently becoming widely available and economical include high speed busses such as PCI [8] and medium speed serial LANs such as USB [10], IEEE-1394 ("Firewire") [3], Ethernet, and Fast Ethernet [4]. Although Open SoundControl was not designed with a particular transport layer in mind, our design reflects features shared by modern networking technologies.

We assume that Open SoundControl will be transmitted on a system with a bandwidth in the 10+ megabit/sec range. MIDI's bandwidth, by contrast, is only 31.25 kilobit/sec, roughly 300 times slower. Therefore, our design is not preoccupied with squeezing musical information into the minimum number of bytes. We encode numeric data in 32-bit or 64-bit quantities, provide symbolic addressing, time-tag messages, and in general are much more liberal about using bandwidth for important features.

We assume that data will be delivered in packets (a.k.a. "datagrams") rather than as a stream of data traveling along an established connection. (Although many LANs transmit data serially, they deliver the data as a single block.) This leads to a protocol that is as stateless as possible: rather than assuming that the receiver holds some state from previous communications (e.g., MIDI's running status), we send information in larger, self-contained chunks that include all the relevant data in one place. This packet-based delivery model provides a mechanism for synchronicity: messages in the same packet (e.g., messages to start each of the notes in a chord) can be specified to have their effects occur at the same time as each other. We assume that the network services will tell us the length of each packet that we receive.

All modern networking technologies have the notion of multiple devices connected together in a LAN, with each device able to send a packet to any other device. So we assume that any number of clients might send Open SoundControl messages to a particular device. We also assume that the transport layer provides a return address mechanism that allows a device to send a response back to the device that sends it a message.

3 Data Representation

All Open SoundControl data is aligned on 4-byte boundaries. Numeric data are encoded using native machine representations of 32-bit or 64-bit big-endian twos-complement integers and IEEE floating point numbers. Strings are represented as a sequence of non-null ASCII characters followed by a null, padded with enough extra null characters to make the total length be a multiple of 4 bytes. These representations facilitate real-time performance by eliminating the need to reformat received data. (Except for the unavoidable big-endian/small-endian conversion. We note that most small-endian machines provide special instructions for this conversion.)

The basic unit of Open SoundControl data is a message, which consists of the following:

• A symbolic address and message name (a string whose meaning is described below)
• Any amount of binary data up to the end of the message, which represent the arguments to the message.

An Open SoundControl packet can contain either a single message or a bundle. A bundle consists of the following:

• The special string "#bundle" (which is illegal as a message address)
• A 64 bit fixed point time tag
• Any number of messages or bundles, each preceded by a 4-byte integer byte count

Note that bundles are recursively defined; bundles can contain other bundles.

Messages in the same bundle are atomic; their effects should be implemented concurrently by the receiver. This provides the important service in multimedia applications of specifying values that have to be set simultaneously.

Time tags allow receiving synthesizers to eliminate jitter introduced during packet transport by resynchronizing with bounded latency. They specify the desired time at which the messages in the bundle should take effect, a powerful mechanism for precise specification of rhythm and scheduling of events in the future. To implement time tags properly, message recipients need a real-time scheduler like those described by Dannenberg [1].

Time tags are represented by a 64 bit fixed point number. The first 32 bits specify the number of seconds since midnight on January 1, 1900, and the last 32 bits specify fractional parts of a second to a precision of about 200 picoseconds. This is the representation used by Internet NTP timestamps [6]. The Open SoundControl protocol does not provide a mechanism for clock synchronization; we assume either that the underlying network will provide a synchronization service or that the two systems will take advantage of a protocol such as NTP or SNTP [7].

4 Addressing Scheme

The model for Open SoundControl message addressing is a hierarchical set of dynamic objects that include, for example, synthesis voices, output channels, filters, and a memory manager. Messages are addressed to a feature of a particular object or set of objects through a hierarchical namespace similar to URL notation, e.g., /voices/drone-b/resonators/3/set-Q. This protocol does not proscribe anything about the objects that should be in this hierarchy or how they should be organized; each system that can be controlled by Open SoundControl will define its own address hierarchy. This open-ended mechanism avoids the addressing limitations inherent in protocols such as MIDI and ZIPI [5] that rely on short fixed length bit fields.

To allow efficient addressing of multiple destination objects for parameter updates, we define a pattern-matching syntax similar to regular expressions. When a message's address is a pattern, the receiving device expands the pattern into a list of all the addresses in the current hierarchy that match the pattern, similar to the way UNIX shell "globbing" interprets special characters in filenames. Each address in the list then receives a message with the given arguments.

We reserve the following special characters for pattern matching and forbid their use in object addresses: ?, *, [, ], {, and }. The # character is also forbidden in object address names to allow the bundle mechanism. Here are the rules for pattern matching:

• ? matches any single character except /
• * matches zero or more characters except /
• A string of characters in square brackets (e.g., [string]) matches any character in the string. Inside square brackets, the minus sign (-) and exclamation point (!) have special meanings: two characters separated by a minus sign indicate the range of characters between the given two in ASCII collating sequence. (A minus sign at the end of the string has no special meaning.) An exclamation point at the beginning of a bracketed string negates the sense of the list, meaning that the list matches any character not in the list. (An exclamation point anywhere besides the first character after the open bracket has no special meaning.)
• A comma-separated list of strings enclosed in curly braces (e.g., {foo,bar}) matches any of the strings in the list.

Our experience is that with modern transport technologies and careful programming, this addressing scheme incurs no significant performance penalty either in network bandwidth utilization or in message processing.

5 Requests for Information

Some Open SoundControl messages are requests for information; the receiving device (which we'll call the Responder in this section) constructs a response to the request and sends it back to the requesting device (which we'll call the Questioner). We assume that the underlying transport layer will provide a mechanism for sending a reply to the sender of a message.

Return messages are formatted according to this same Open SoundControl protocol. There is always enough information in a return message to unambiguously identify what question is being answered; this allows Questioners to send multiple queries without waiting for a response to each before sending the next. The time tag in a return message indicates the time that the Responder processed the message, i.e., the time at which the information in the response was true.

Exploring the Address Space

Any message address that ends with a trailing slash is a query asking for the list of addresses underneath the given node. These messages do not take arguments. This kind of query allows the user of an Open SoundControl system to map out the entire address space of possible messages as the system is running.

Message Type Signatures

Because Open SoundControl messages do not have any explicit type tags in their arguments, it is necessary for the sender of a message to know how to lay out the argument values so that they will be interpreted correctly.

An address that ends with the reserved name "/type-signature" is a query for the "type signature" of a message, i.e., the list of types of the arguments it expects. We have a simple syntax for representing type signatures as strings; see our WWW site for details.

Requests For Documentation

An address that ends with the reserved name "/documentation" is a request for human-readable documentation about the object or feature specified by the prefix of the address. The return message will have the same address as the request, and a single string argument. The argument will either be the URL of a WWW page documenting the given object or feature, or a human-readable string giving the documentation directly. This allows for "hot-plugging" extensible synthesis resources and internetworked multimedia applications.

Parameter Value Queries

An address that ends with the reserved name "/current-value" is a query of the current value of the parameter specified by the prefix of the address. Presumably this parameter could be set to its current value by a message with the appropriate arguments; these arguments are returned as the arguments to the message from the Responder. The address of the return message should be the same as the address of the request.

6 Conclusion

We have had successful early results transmitting this protocol over UDP and Ethernet to control real-time sound synthesis on SGI workstations from MAX [9] programs running on Macintosh computers. Composers appreciate being freed from the limited addressing model and limited numeric precision of MIDI, and find it much easier to work with symbolic names of objects than to remember an arbitrary mapping involving channel numbers, program change numbers, and controller numbers. This protocol affords satisfying reactive real-time performance.

7 References

[1] Dannenberg, R, 1989. "Real-Time Scheduling and Computer Accompaniment," in M. Mathews and J. Pierce, Editors, Current Directions in Computer Music Research, Cambridge, Massachusetts: MIT Press, pp. 225-261.

[2] Freed, A. 1996. "Firewires, LANs and Buses," from SIGGRAPH Course Notes Creating and Manipulating Sound to Enhance Computer Graphics, New Orleans, Louisiana, pp. 84-87.

[3] IEEE 1995. "1394-1995: IEEE Standard for a High Performance Serial Bus," New York: The Institute of Electronical and Electronic Engineers.

[4] IEEE 1995b. "802.3u-1995 IEEE Standards for Local and Metropolitan Area Networks: Supplement to Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications: Media Access Control (MAC) Parameters, Physical Layer, Medium Attachment Units, and Repeater for 100 Mb/s Operation," New York: The Institute of Electronical and Electronic Engineers.

[5] McMillen, K, D. Wessel, and M. Wright, 1994. "The ZIPI Music Parameter Description Language," Computer Music Journal, Volume 18, Number 4, pp. 52-73.

[6] Mills, D., 1992. "Network Time Protocol (Version 3) Specification, Implementation, and Analysis," Internet RFC 1305. (http://sunsite.auc.dk/RFC/rfc1305.html)

[7] Mills, D., 1996. "Simple Network Time Protocol (SNTP) Version 4 for Ipv4, Ipv6 and OSI," Internet RFC 2030. (http://sunsite.auc.dk/RFC/rfc2030.html)

[8] PCI 1993. PCI Local Bus Specification, Revision 2.0, Hillsboro, Oregon: PCI Special Interest Group.

[9] Puckette, M., 1991. "Combining Event and Signal Processing in the MAX Graphical Programming Environment," Computer Music Journal, Volume 15, Number 3, pp. 58-67.

[10] The USB web site is www.usb.org

[11] Yamaha 1996. "Preliminary Proposal for Audio and Music Protocol, Draft Version 0.32," Tokyo, Japan: Yamaha Corporation.

Open Sound Control: A New Protocol for Communicating with Sound Synthesizers, Wright, Matthew, and Freed Adrian , International Computer Music Conference (ICMC), 1997, Thessaloniki, Hellas, p.101-104, (1997)

1 Introduction

Throughout history, people have adapted objects in their environment into musical instruments. The computer industry has developed the digitizing tablet (a.k.a. "artist's tablet") primarily for the purpose of drawing shapes in computer graphics illustration programs. These tablets are broadly available and low cost, and they sense many control dimensions such as pen or pointing device position, pressure, and tilt. We find that these controllers can be used for musical control in a variety of interesting and musically expressive ways.

2 Characteristics of the Wacom Tablet

We use a Wacom [9] ArtZ II 1212 digitizing tablet, a 12 inch by 12 inch model with both a stylus (pen) and a puck (mouse-like device) that allows two-handed input [7]. The stylus and puck are cordless, batteryless, and light weight. The tablet has a single DB-9 connector that carries both power and control information; on the other side of the cable is a mini-din connector suitable for the serial port on a Macintosh, PC, or SGI workstation. We've had no trouble running the cable at distances of about 30 feet.

The Wacom tablet accurately outputs the absolute X and Y dimensions of both the puck and stylus as integers in the range 0 to 32480. The stylus has a pressure-sensitive tip (and a pressure-sensitive "eraser" tip on the other end) that produces pressure readings in the range 0 to 255. When only the stylus is used, the Wacom tablet also outputs tilt values in two dimensions, in the range -60 to 60 degrees. Position and tilt are reported whenever the stylus or puck is in "proximity" of the tablet, within about a centimeter of the surface.

In addition to these continuous control variables, there are many ways to trigger events. The puck has 4 buttons, and the stylus has two buttons as well. Each tip of the stylus can be considered a button; pressing the stylus into the tablet will cause the same kind of button event. Another kind of event is generated when either device enters or leaves the proximity of the tablet. For the stylus, this event indicates which side (stylus tip side or eraser side) enters or leaves proximity.

It is important to evaluate the temporal behavior of computer systems for music [6]. We performed timing experiments on SGI machines using their fast UST clock, measuring the elapsed time between events reporting values of position, tilt, and pressure while continually moving the stylus so as to change each parameter constantly. The good news is that about 75% of parameter updates came within 1 ms of the previous update. The bad news is that the elapsed times greater than 1 ms averaged about 28 ms and were distributed more or less in a bell curve around 28 ms. The source of this erratic temporal performance seems to be context switches between the X server and our application program, a consequence of Wacom's decision to make tablet data available on Unix using X11 extension device valuator events. We expect that direct access to the serial stream from the tablet will address this difficulty.

3 A Model for Mapping Tablet Data to Musical Control

We have developed a model for mapping tablet data into musical control information (either MIDI or Open SoundControl [11]) that allows performers to simply customize the musical behavior of the tablet. In our model, the two dimensional surface of the tablet is populated by any number of arbitrarily-shaped polygonal regions. These regions can overlap, and they have a vertical stacking order that determines which of two overlapping regions is above the other. Each region has a user-assigned symbolic name.

The following events may occur in a region:

· Puck or stylus enters (or leaves) the proximity of the tablet in the region

· Either end of the stylus makes (or breaks) contact with the tablet

· Puck or stylus moves into (or out of) the region

· Button press (or release) while pen or stylus is in the region

For each region the user can define a list of actions to take when any of these events occur, e.g., the stylus tip touching the tablet in a region might cause a pair of MIDI note-on events.

There are also several continuous parameters that are updated constantly by the tablet:

· X and Y coordinates for puck and stylus

· X and Y axis tilt for stylus (when not using puck)

· Stylus pressure

Each of these continuous parameters can also be mapped to a list of actions, e.g., the X axis tilt might correspond to pitch bend.

What makes our model dynamic is that actions can be added to and deleted from these lists in response to events that occur in a region. For example, pressing a button while the puck is in a region could add a mapping from puck Y position to overall volume, and having the puck enter a different region could remove this mapping. This features facilitates complex musical behaviors in response to various gestures.

To allow the mapping of data values from one range to another we provide two primitives. The first is a linear scaling operator that implements functions of the form f(x)=ax+b where a and b are constant. The desired output range is specified; the system knows the range of possible input values and can compute a and b automatically. The second is stored function evaluation. We compute user-specified memoryless nonlinear functions with an abstraction that subsumes table lookup, expression evaluation, neural network forward-pass, and other techniques. Of course the tablet itself is an excellent tool for drawing function curves.

One important feature of the tablet is that its position sensing is absolute rather than relative. This means that the performer does not need any visual feedback from a computer monitor to see which region the stylus or puck is in. The Wacom tablet has a clear plastic overlay under which paper and small objects may be placed. We can print a "map" of the user's defined regions to the scale of the tablet and place it under the overlay; this lets the user see exactly where each region is on the tablet. Removable adhesive strips for drafting may be used to create surface irregularities offereing tactile feedback.

4 Examples

4.1 Digital Tambura Interface

Our first use of the digitizing tablet for musical control was to create an interface for a drone instrument similar in function to the Indian tambura. A real tambura has 4 strings with no fretboard or fingerboard; the player plucks each open string and lets it ring. Tambura players typically pluck the 4 strings one at a time and then start over at the first string, providing a continuous drone.

A tablet interface was designed that retains features of the playing style of a real tambura. We defined six regions on the tablet corresponding to six virtual strings. Each of these virtual strings had a corresponding monophonic synthesizer voice set to a particular drone timbre and pitch. The action of touching the stylus to the tablet in a particular region represented touching a finger to a real string: it caused the currently sounding note on that string to decay fairly rapidly. Releasing the stylus from the tablet represented the second half of the pluck, where the finger releases the string and sets it vibrating. This caused the voice to start a new note.

The loudness of each note was determined by the total amount of horizontal distance traveled between the time the stylus touched the region and the time the stylus left the tablet. Timbral control came from a mapping from the X axis position at the time the stylus left the tablet to the relative balance of even and odd harmonics in the synthesizer [5].

4.2 Strumming

We use the tablet to emulate the gesture of strumming a stringed instrument. We define four to ten thin rectangular horizontal regions which represent virtual strings much as in the tambura example. Again, each of these regions has a corresponding synthesizer voice which is virtually plucked in two steps by the stylus entering and then leaving the region. The speed of the pluck (i.e., the reciprocal of the time between when the stylus enters and leaves the region) combines with the pen pressure to determine the loudness of each note.

When the upper stylus button is depressed the stylus' function changes from exciting the virtual strings to muting them. In this case, the note for each string still decays when the tablet enters the region, but a new note does not start when the string leaves the region. The middle button corresponds to a "half mute" mode where the synthesized sound is more inharmonic and each note decays faster, corresponding to the playing technique on real strummed instruments of partially muting strings with the palm.

On real stringed instruments, the position of the picking (or plucking or bowing) along the axis of the string is an important determinant of timbre. We emulate this control by mapping the X position at the time the stylus leaves the string region to a timbral control such as brightness.

We have several techniques for controlling pitch with the puck in the left hand. The first is modeled loosely on the autoharp. There are regions of the tablet that correspond to different chord roots, and each of the 4 puck buttons corresponds to a different chord quality (e.g., major, minor, dominant seventh, and diminished). Pressing a button with the puck in a particular region determines a chord, which in turn determines the pitches of each virtual string. The layout of chord regions and chord qualities, and the voicings of each chord are all easily configurable for different styles, pieces, or performers.

Another technique for pitch control is modeled on the slide guitar, with the puck taking the role of the slide. In this case the pitches of the virtual strings are in fixed intervals relative to each other, and the puck's horizontal position determines a continuous pitch offset that affects all the strings. When the puck is not in proximity of the tablet the strings are at their lowest pitches, corresponding to strumming open strings on a slide guitar. The buttons on the puck select from among a set of different tunings.

4.3 Timbre Space Navigation

CNMAT's additive synthesis system provides timbral interpolation via the timbre space model [10]. The tablet is a natural way to control position in a two-dimensional timbre space.

The simplest application uses the tablet in conjunction with another controller like a MIDI keyboard. In this case the job of the tablet is just to determine position in the timbre space, so we map the two position dimensions to the two timbre space dimensions and use the other controller to determine pitch, loudness, and articulation.

Another approach is to use both hands on the tablet. The stylus position determines timbre space position. When the stylus touches the tablet a note is articulated, and when it leaves the tablet the note ends. Stylus pressure maps to volume. The puck, held in the left hand, determines pitch via a variety of possible mappings.

A third approach is to implement a process that continually produces a stream of notes according to parameterized models of rhythm, harmony, and melody. The left hand can manipulate the parameters of this model with the puck while the right hand navigates in timbre space with the stylus.

5 Future Work

We would like to be able to recognize various kinds of strokes and gestures made with the stylus. Previous work on stroke recognition in the context of computer conducting [2] seems applicable, as does work on the integration of segmentation, recognition, and quantitative evaluation of expressive cursive gestures [8].

6 Conclusion

The tablet interface provides a musically potent and general two-dimensional interface for the control of sound synthesis and compositional algorithms. With the addition of stylus pressure and tilt, two additional dimensions are available for each hand. Our experiments have demonstrated that irregularities can be added to the surface providing tactile reference. The tablet interface is basically a spatial coordinate sensor system like the Mathews Radio Drum [1], the Buchla Lightning [3], the Theremin [4], joy sticks, and other sensor systems like ultra sound ranging. But, unlike some of these systems it offers the possibility of tactile reference, wide availability, flexible adaptation, precision, and reliability. The tablet interface and its spatial coordinate sensor cousins offer the possibility of long-lived alternative musical control structures that use mathematical abstractions in a reliable manner as the basis for a live-performance musical repertoire.

7 Acknowledgments

CNMAT gratefully acknowledges the support of Silicon Graphics, Inc., and Gibson Guitar Corporation for generous contributions that supported this work.

References

[1] Boie, B, M. Mathews, and A. Schloss. 1989. "The Radio Drum as a Synthesizer Controller," Proc. ICMC, Columbus, Ohio, pp. 42-45.

[2] Brecht, B., and G. Garnett. 1995. "Conductor Follower," Proc ICMC, Banff, Canada, pp. 185-186.

[3] The Buchla and Associates web site describes Lightning II: http://www.buchla.com/

[4] Chadabe, J. 1997. Electric Sound: The Past and Promise of Electronic Music, Englewood Cliffs, New Jersey: Prentice Hall.

[5] Freed, A. 1995. "Bring Your Own Control to Additive Synthesis," Proc ICMC, Banff, Canada, pp 303-306.

[6] Freed, A. 1997. "Operating Systems Latency Measurement and Analysis for Sound Synthesis and Processing Applications," Proc. ICMC, Thessaloniki.

[7] Kabbash, P, W. Buxton, and A. Sellen. 1995. "Two-handed Input in a Compound Task," Proc. SigCHI, Boston, Massachusetts, pp. 417-423. http://www.dgp.utoronto.ca/OTP/papers/bill.buxton/tg.html

[8] Keeler, J., D. Rumelhart, and W. Loew. 1991. "Integrated segmentation and recognition of hand-printed numerals," In R. Lippmann, J. Moody, and D. Touretzky, Editors, Neural Information Processing Systems Volume 3, San Mateo, CA: Morgan Kaufmann.

[9] The Wacom WWW site is www.wacom.com

[10] Wessel, D. 1979. "Timbre Space as a Musical Control Structure," Computer Music Journal, Volume 3, Number 2, pp 45-52.

[11] Wright, M., and A. Freed. 1997. "Open SoundControl Protocol," Proc ICMC, Thessaloniki. New Musical Control Structures from Standard Gestural Controllers, Wright, Matthew, Wessel David, and Freed Adrian , International Computer Music Conference, 1997, Thessaloniki, Hellas, p.387-390, (1997)