Programming Music and Video with Ruby

By Dev Bootcamp Author • June 10, 2015

As part of our speakers and workshop series at Dev Bootcamp New York City we hosted a talk by Ari Russo, a musician, video artist, Senior Developer at Paperless Post, and co-organizer of the Monthly Music Hackathon. During his talk Ari shared the open source programs he created using the Ruby programming language in order to facilitate creating music and interfacing with MIDI instruments and performing music and video live. Ari's work is a great example of how programmers can bring together creative ideas and practical needs, so I interviewed him about the programs he wrote and why the Ruby language and object-oriented programming serve him as an artist and musician.

What motivated you to write these programs and gems? What was the problem or challenge you were trying to solve?

My goal was to create music and video based around homemade software instruments. I wanted to use software that was as unobtrusive to the creative process as possible, neutral enough not to cramp my personal style, and within reach in terms of time and effort.

Why is the Ruby language a good fit for the programs you wrote?

My concern about the flow of the creative process is what led me to using Ruby. I experimented with different languages, but chose Ruby because of a combination of qualities: It's widely used, general purpose, interpreted, purely object oriented, and can interface with system libraries dynamically

The fact that Ruby is widely used and general purpose has a lot of benefits, from good documentation to a large community of people working to improve the language itself. I like the idea that these programs which are designed for artistic expression can reap the benefits like stability and performance that come from all the corporate involvement in this community.

There are also already a huge selection of other tools available for purposes not related to music. Using those tools as part of music-making was a novel idea at the time I started these projects and I don't think the possibilities are anywhere near exhausted yet.

Object orientation pairs well with music, both in terms of music theory and the way electronic music instruments are controlled by sending messages to each other. I believe that it was because of these parallels I was able to make the programs as modular as they are give them interfaces that are familiar to anyone who's used electronic music instruments.

The fact that Ruby is an interpreted language makes it possible to live-code without any additional consideration. After installing Ruby, one can launch Pry or another Ruby REPL and control their instruments that way

Finally, the ability to dynamically interface system libraries allows a user to install these programs without compiling or even configuring anything. They're just ordinary Ruby Gems.

Each of these points is an attempt at capturing a concept not directly pertinent to artistic expression and make dealing with that concept optional for the user. I believe that anyone can make good music if they're able to focus deeply on the source of their inspiration instead of rote memorization or technical hurdles.

Describe the programs and gems that you wrote - what does each of them do?

These are the lower level tools that I started with:

unimidi: A universal way to access MIDI hardware for realtime input and output in Ruby. Unimidi has the same interface across platforms (such as OSC, Linux, Windows) and with different types of Ruby (MRI, JRuby).

midi-message: Provides Ruby classes representing MIDI messages

nibbler: A message parser that can instantiate midi-message objects using raw MIDI data or streams

midi-eye: MIDI input event handling

topaz: A tempo clock that can synchronize to and from MIDI devices

micromidi: Ruby DSL for MIDI

micro-osc: Ruby DSL for OSC

After that, relying on those tools heavily, I created these programs:

diamond: A MIDI/OSC arpeggiator. An arpeggiator is a type of electronic music instrument for generating melodies from a set of user-inputted notes

mmplayer: Mplayer is a very stripped down audio/video player for unix-like systems and this gem adds MIDI control to it

patch: A universal hub for controller messages. Translates and routes OSC, MIDI and other control messages on the fly between formats

vz (not open source)- WebGL based video processor and audio visualization framework. It allows me to convert analog video into 3d meshes, do chroma keying, and pixel sorting. It's controlled via MIDI and OSC

I've also written other programs which are not as directly tied into those such as analog, a Ruby gem for easily scaling numbers and an HTML5/D3 spectrometer.

How do you use these programs in either recording or performance?

I'm currently very focused on further developing my live performance and am gearing up for several performances in the next few months. My setup is still evolving rapidly, but I'm currently using these programs to:

You can check out more of Ari's work on his website and upcoming talks and events at Dev Bootcamp NYC, San Francisco, and Chicago on our Meetup pages.


Similar Articles