:title: Grains - we are sounds in the Universe :data-transition-duration: 750 :css: css/grains.css ---- :id: title Grains ====== We are sounds in the universe ----------------------------- * Dragica Kahlina (@gluggergames) * Jens-Christian Fischer (@jcfischer) ---- :data-x: r0 :data-y: r1000 What & Why ========== How far can we take the *Web Programming*, *Synthesis* and *Free Improvisation* ---- Feedback ======== The Feedback between performer and audience is key to a musical performance. What happens when you as the audience really participate? ---- But first --------- a bit of theory =============== ---- Sound Synthesis =============== ---- Additive ======== * 1822 Joseph Fourier showed that some functions can be written as a sum of sines * Using harmonics we can build up a variety of timbres * Theoretically we could use arbitrary many sines to reconstruct almost all sounds ---- :data-x: r0 :data-y: r500 :data-scale: 0.1 Saw Wave ======== .. image:: img/sawtooth.png :height: 350px :align: right ---- Envelopes ========= * eternal sound vs defined notes * most common ist ADSR (attack - decay - sustain - release) .. image:: img/envelope.svg :height: 400px :align: right ---- Subtractive =========== * Source with a broad spectrum -> noise * Filter too thin out spectrum * More "natural" sound * Modular analog synthesizer * Needs good filters ---- Some other Methods ================== * Frequency Modulation * Amplitude Modulation * Physical Modelling ---- Granular Synthesis ================== * A physical model of sound * Denis Gabor 1947 : Acoustical quanta and the theory of hearing * Based on grains or wave packages * Analog to photons for light, proposed quanta for sound * Psychoaccoustic : ear like the eye has resolution limits, granular synthesis plays with this ---- A grain ======= .. image:: img/truax.gif :height: 200px :align: right Has a source wave (with its own frequency), wraped in an envelope. Both have various parameter that can be changed and influence the timbre. ---- Heaps of grains =============== There is seldomly one grain, but mostly lots of grains. How dense, homogenous, etc. this grain clouds are changes the timbre and the way we perceive the sound. ---- Spectrum ======== .. image:: img/grains_spectrum.png :height: 600px :align: right ---- Pioneers of Granular Synthesis ============================== * Curtis Roads : Microsound * Iannis Xenakis * Trevor Whishart * B Truax * Brandtsegg, Saue, Johansen : Partikkel Opcode in CSound ---- CSound ====== ---- History ======= * 1957 first computer sound synthesis at Bell Labs by Max Mathews -> MUSIC I * Until 1970s MUSIC 11 by Barry Vercoe * Around 1986 Barry Vercoe wrote the first CSound based on C * CSound constantly incorporates new ideas ---- Example Program =============== .. image:: img/csound.png :height: 600px ---- CSound Alternatives =================== Freeware -------- * Sonic Pi * Pure Data * Supercollider Commercial ---------- * Max/MSP ---- :data-scale: 2 :data-x: r0 :data-y: r2000 What can we do with "run of the mill" technology? ================================================= Constraints: ------------ * No specific network technologies * No specific networking hardware * No low level networking software ---- Experimental Setup ================== Ingredients ----------- * Elixir_ / Phoenix_ / Erlang_ VM * WebSockets_ * Processing_ / P5_ * Open Sound Control (OSC_) * CSound_ * A certain amount of crazyness * You and your mobile phone .. _Elixir: http://elixir-lang.org .. _Phoenix: http://www.phoenixframework.org .. _Erlang: http://www.erlang.org .. _WebSockets: https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API .. _OSC: https://en.wikipedia.org/wiki/Open_Sound_Control .. _CSound: http://csound.github.io .. _Processing: http://processing.org .. _P5: http://p5js.org ---- Overview ======== .. image:: img/overview.png ---- Server ====== * Ubuntu Server running on SWITCHengines in Zurich * Elixir / Phoenix software * Receiving data from mobile clients (30 msgs / second / client) via WebSockets * Keeping state * Sending data to Laptop on Stage ---- Visualisation Client -------------------- * JavaScript receives position data from server * Calculates visualisation (flocking behaviour) with Processing * Generates OpenSoundControl (OSC) messages, sends them via Websocket * Node.js Program translates OSC messages from Websocket to UDP * CSound Software listens to OSC messages and generates grains * Audio is sent via USB to Sound Laptop ---- Sound Laptop ------------ * Various MIDI Controllers connected * MainStage software creates virtual mixing desk * Software Synths / Digital Effects * Mixes Audio from CSound with Effects * Sends Audio to Conference Hall PA ---- iPad Pro -------- * Played by Dragica in Lucerne, Switzerland who (hopefully) sees & hears what we are doing here * Runs various musical software * Sound via Video-Conference, patched to Conference Hall PA ---- OpenSourced =========== All code is available on http://gitlab.switch.ch/grains (good luck in getting it working though ;) ) ---- What could possibly go wrong? ============================= ---- Audience Participation ====================== Go To http://grains.switch.ch ----------------------------- ---- Dragica Kahlina =============== * Theoretical Physicist Uni Basel * Lecturer Digital Ideation HSLU * Sound Artist @gluggergames ---- Jens-Christian Fischer ====================== * MscIT University Liverpool * Lecturer Web ZHAW * Product Manager SWITCHengines @jcfischer