How I Built TinyMusic.js (part 2)

In yesterday’s post, I wrote about the Note class in TinyMusic, and covered some of the basic music theory behind it. Today, I’ll cover the Sequence class and discuss some Web Audio API basics like audio contexts, oscillators, and scheduling.

If you didn’t read part 1, you really should go back and do that, because this post references a lot of stuff from it.

The Sequence Class

In TinyMusic, Note instances are strung together and have their playback controlled by a Sequence. Each Sequence has a tempo, volume control, and some basic EQ.

There are a couple different ways to instantiate a Sequence:

// create a Web Audio API context
var ac = new AudioContext();  
// beats per minute
var tempo = 110;  
var sequence = new Sequence( ac, tempo, [  
  new Note('G3 q'),
  new Note('E4 q'),
  new Note('C4 h')
]);

For convenience, you can also pass an array of strings, and they’ll automatically get turned into Note instances.

// create a Web Audio API context
var ac = new AudioContext();  
// beats per minute
var tempo = 110;  
var sequence = new Sequence( ac, tempo, [  
  'G3 q',
  'E4 q',
  'C4 h'
]);

So now we’ve got a Sequence with an AudioContext, a tempo, and a bunch of Notes. What’s next?

Playing A Sequence

Playing a Sequence consists of a few steps. I’ll explain each one in a bit more detail later on, but I want to start with a higher-level overview.

First, I create a new oscillator.

Next, I loop through each Note in the sequence and schedule changes to the oscillator’s frequency.

Finally, I schedule a stop() on the oscillator and, when applicable, enable looping.

Here’s the play() method in its entirety:

// run through all notes in the sequence and schedule them
Sequence.prototype.play = function( when ) {  
  when = typeof when === ‘number’ ? when : this.ac.currentTime;
  this.createOscillator(); 
  this.osc.start( when );
  this.notes.forEach(function( note, i ) { 
    when = this.scheduleNote( i, when ); 
  }.bind( this )); this.osc.stop( when );
  this.osc.onended = this.loop ? 
    this.play.bind( this, when ) : null; 
  return this;
};

The when parameter is a Web Audio timestamp, which is essentially a high-precision value (in seconds) that can be used for accurate scheduling.

Everything else will be explained below.

Creating an oscillator

This part is pretty straight-forward.

There’s a bit of extra code to support custom wave types (a bit beyond the scope of this post, and probably not of much interest to most readers) — but essentially, all we really need to do is this:

this.osc = this.ac.createOscillator();  
this.osc.type = this.waveType || ‘square’;

Possible values for waveType are sine, square, triangle, and saw.

Scheduling Frequency Changes

The entire Sequence is actually played back using a single oscillator.

All I do is loop through each note and schedule changes to the oscillator’s frequency at the appropriate time.

In other words, when you call Sequence#play, I do all of the scheduling immediately, and the Web Audio API kind of takes over from there. All of my work is done up front.

Each note is passed to Sequence#scheduleNote with its index and a timestamp for when it should begin.

Here’s the scheduleNote method:

// schedules this.notes[ index ] to play at the given time
// returns an AudioContext timestamp of when the note will *end*
Sequence.prototype.scheduleNote = function( index, when ) {  
  var duration = 60 / this.tempo * this.notes[ index ].duration,
    cutoff = duration * ( 1 - ( this.staccato || 0 ) );

  this.setFrequency( this.notes[ index ].frequency, when );

  if ( this.smoothing && this.notes[ index ].frequency ) {
    this.slide( index, when, cutoff );
  }

  this.setFrequency( 0, when + cutoff );
  return when + duration;
};

I think this might be hard to explain by going line-by-line, so I’m going to give sort of a conceptual overview instead.

When we loop through all of the Notes in the play() method, we have a starting time value called when.

Starting with the first note, we call scheduleNote(), passing when as well as the note’s index within the sequence (for the first Note, this would be 0).

From there, scheduleNote tells the oscillator “change to x frequency at y time”, and then it returns a new value for when (the previous value plus the duration of the latest Note).

Then play() moves on to the next note, and calls scheduleNote() using the new when value. Rinse and repeat for each additional Note in the sequence.

The actual oscillator scheduling happens in a method called setFrequency(), which looks like this:

// set frequency at time
Sequence.prototype.setFrequency = function( freq, when ) {  
  this.osc.frequency.setValueAtTime( freq, when );
  return this;
};

Pretty simple, right? We just use the oscillator’s setValueAtTime method, which does exactly what it looks like.

Staccato and Smoothing

TinyMusic supports two settings called staccato and smoothing.

Staccato refers to “choppiness”, or how long/short each note will be compared to its intended duration. Lots of staccato means the note is harshly truncated, and no staccato means it will play in its entirety.

Smoothing is for portamento, which is sliding from one note to the next. Higher smoothing values result in a slower slide, and no smoothing means that notes don’t slide at all.

These settings aren’t really integral to the basic functioning of TinyMusic, and I think they’re pretty easy to understand from looking at the source, so I’m going to skip over them in this post.

If you have questions about either, just leave a comment and I’ll answer to the best of my ability.

EQ and Volume

EQ and volume nodes are created as soon as a Sequence is initialized with a method called createFXNodes().

There are three nodes for EQ: bass, mid, and treble. Each is a Web Audio BiQuadFilterNode. This allows for some basic 3-band equalization.

Setting EQ values looks like this:

var sequence = new Sequence( ac, tempo, noteArray );  
// 80 Hz
sequence.bass.frequency.value = 80;  
// +4dB
sequence.bass.gain.value = 4;

Stopping Playback

Stopping playback is pretty simple, but there are a few important things that need to happen:

if ( this.osc ) {  
    this.osc.onended = null;
    this.osc.stop( 0 );
    this.osc.frequency.cancelScheduledValues( 0 );
    this.osc = null;
  }
  return this;
};

First, we just do a quick safety check to make sure we actually have an active oscillator.

Next, we null out the oscillator’s onended callback, which effectively cancels looping.

Then, we actually stop the oscillator and cancel any frequency automation that may have been scheduled.

Finally, we null out the oscillator.

Wrapping Up

There are a few things in TinyMusic that I decided not to cover here.

Some of them are “advanced” features that likely wouldn’t be of interest to most readers, and others are trivial little helper functions that you can pick up by reading the source.

Nevertheless, I hope you learned a little bit about working with music in code — and maybe got a small taste for what it’s like to build apps with the Web Audio API.

Like I said earlier, please don’t hesitate to ask questions in the comments. If I left something out, it was likely an oversight.

— -

Follow me on Twitter or Medium for more posts. I’m trying to write once a day for the next 30 days.

And if you’re in the Boston area and want to come work on crazy, top-secret things with me at Project Decibel, shoot me an email. I’m hiring.