If rhythm is the element of music that makes us want to dance—or at least tap our foot— melody is the part we hum along to, or whistle for days on end. Here are a few examples:

In this chapter, we will learn to program melodies and explore the creative possibilities this opens up. We will start by representing a melody in code, then play it back, visualize it, transform it, and make it interactive. Next, we’ll generate our own melodies—first by using a machine learning model trained on existing music, then by generating melodies from scratch at random and gradually shaping them with constraints. As we tweak parameters and listen closely, we’ll explore the elements of melody, focusing on pitch sets, range, motion/contour, and rhythm. The chapter ends with an invitation to create an interactive study on melody, drawing inspiration from historical and contemporary projects that experiment with tuning systems, generative processes, visual representations, and musical interaction.

Listening: What Makes a Melody?

A melody is a sequence of tones that we, as listeners, perceive as a single entity. How do the tones in the sequence differ from one another? Let’s listen more closely.

Tones have a duration, too ––they can be shorter or longer. Let’s draw the tones in our melody as they occur over time:

Each tone has a position in time: the first plays at 0.02s, the second at 0.72s, and so on—click here to draw these time markers. The order in which tones are played gives the melody a particular shape, called contour—for example, this melody moves in small steps. It starts by repeating a few downward motions, then drops to lower pitches that slowly make their way back up.

Another dimension we hear is timbre: the melody examples at the beginning each have their own distinctive voice color. This is the focus of the next chapter.

Machines can also listen for tones and their pitch, volume, duration, position, and timbre. In the first two interactive sketches above, an algorithm analyzes the audio and estimates the tone’s pitch or volume in real time. We’ll look at this more closely in the Machine Listening chapter.

The dimensions we named and listened to here don’t capture everything we perceive—especially performance details that can make a melody feel alive, like how each tone begins, ends, and connects to the next. Still, these dimensions give us a starting point. From here, we can build systems that represent, play back, transform, and generate melodies.

Encoding: from Sound to Score

For centuries, musicians have written down their melodies as scores—notations that specify which pitches to play, when, how long, how loudly—so they can be sung or played later. Scores have always been abstractions: they make some elements explicit and leave others to interpretation, reflecting the priorities of the musical culture they were designed for.

To work with melodies in code, we can translate them into a digital score format. In this chapter, we’ll use a simple representation: a list of tones, each described by values for the dimensions we’ve named —pitch, velocity, position in time, and duration. Here is our example melody, encoded into a JSON file, which is both human-readable and compatible with Tone:

// The sequence of tones is represented by an array
[
  // Each tone is represented by a Javascript object
  {
    "pitch": 1046.5022612023945,     // in Hertz
    "velocity": 0.6299212598425197,  // a value between 0 and 1
    "time": 0.020833333333333332,    // in seconds 
    "duration": 0.4791666666666667   // in seconds 
  },
  
  //...more tone objects
  
  { 
    "pitch": 932.3275230361799,
    "velocity": 0.7244094488188977,
    "time": 0.5416666666666666,
    "duration": 0.5156250000000001
  }
 ]

Exercise: Encode a short melody (yours or someone else’s) to use in the code examples that follow. The suggestions below are intended for more experienced readers ––if they feel unfamiliar, feel free to skip ahead and keep using the example melody for now.

• MIDI: record a performance or find an existing melody track. Then, convert it with Tone’s MIDI to JSON tool. If your melody is not within the standard Western system (more on this later), see [section on MPE].
• Audio: analyze a recording with Machine Listening tools (results may vary). We’ll discuss this approach in the Machine Learning chapter.

Playback: Interpret the Score

With a machine-readable representation of our example melody, we can now play it back. In the code example below, we use Tone.Part to take our score (an array of tone objects, each including a time value) and schedule those tones as events on Tone’s Transport timeline. As the Transport plays, each event triggers playTone, passing in the the corresponding tone object; our synth then uses that tone’s pitch, duration, and velocity to play it.

async function setup() {
  // Load our score: an array of tones, each with a "time", "pitch", "duration", and "velocity"
  sequence = await loadJSON("sequence.json");
  
  // Tone.Part takes our score and schedules each tone as an event on the timeline.
  // When it's time to play a tone, part calls playTone() and passes it that tone object.
  // The only property Part itself requires is "time" (when to play it).
  // The rest ("pitch", "duration", "velocity") are up to us — as long as playTone() uses the same names.  
  part = new Tone.Part(playTone, sequence);

  // Schedule the part to start at the beginning (time 0) of the Transport timeline
  part.start(0);
  
}

// part calls this function every time a tone is due to play.
// "time" is when to play it, "tone" is the object from our sequence.
function playTone(time, value) {
  // Play the note using its pitch, duration, and velocity (volume)
  synth.triggerAttackRelease(value.pitch/2, value.duration, time, value.velocity);
}

Open the code example above, run it in the p5.js editor, and listen to the result. Because our score doesn’t specify all details—like how each tone connects to the next—it will likely feel significantly less expressive than the original whistled recording.

Exercise. Modify playTone to interpret the score, exploring how playback choices can shape what the score leaves open. Try one or more of the ideas below:

• Shorten/lengthen tone durations, adding tiny gaps/overlaps between tones that make the tones feel more separated (staccato) or more connected (legato). Tip: Right before a tone is triggered, modify the value of its duration.
• Slide from one pitch to the next instead of jumping (a tone-to-tone slide is often called portamento; a bigger sweep across many pitches is a glissando) Tip: use synth.portamento (try 0.0-0.2 seconds)

Beyond playback and interpretation, having a coded representation opens up rich possibilities—let’s begin exploring them.

Visualization: See the Melody

An encoded melody can also be drawn —here it is, visualized in three ways:

Exercise: On paper, then procedurally, draw the same score in three different ways. How do you map time and pitch? How do you represent duration and velocity? Do you highlight the tone that is currently playing? Think about shape, hue, saturation, brightness, position, scale, and texture.

Note: going forward, you’ll see interactive illustrations paired with code examples. The illustrations make the concepts visible and playable. The code examples stay focused on the audio logic, so they are easier to follow, learn from, and reuse as you build your own projects.

Interaction: Put the Listener in Control

We can also make an encoded melody interactive—for example, by letting the listener control one or more of a tone’s properties. In the interactive sketch below, each tone in the sequence plays only when the mouse is clicked, putting the listener in control of its position in time.

// Click to play each tone in a sequence

// Starting at 0, track our position in the sequence
let i = 0;

function mouseClicked(){
	// Look up the tone at the current index
	let tone = sequence[i];
	// Play it right now using its pitch, duration, and velocity (volume)
	synth.triggerAttackRelease(tone.pitch, tone.duration, Tone.now(), tone.velocity);
	// Advance to the next tone, wrapping back to 0 after the last one
	i = (i + 1) % sequence.length;
}

Exercise: Give the listener controls to affect the sequence during playback. Here’s a few ideas to try:

• Dynamics—map an input like mouseY to the velocity of each tone (softer/louder). Tip: decide how loud the next tone should be at the moment it’s triggered.
• Navigation—let the listener move through the sequence intentionally (step backward/forward, or loop a small region). Tip: treat i as the “playhead” that the listener can move around.
• Timbre—swap the default synth for another instrument (like a Sampler) or synthesizer preset (browse options here: Tone presets), and let the user change timbre parameters (We’ll explore shaping timbre in more depth in the next chapter)

If you want to go further, try swapping the input method (QWERTY keyboard, camera-tracked gesture, sensor/data stream) while keeping the same musical controls.

Transformation: Variation Machines

With melody encoded as an array of events, we can generate variations with simple operations: reverse (play backwards), invert (play upside down), transpose (multiply all pitches by the same number), shift (start at a random position), or rotate the pitches while keeping rhythm intact. These techniques, historically common in Western composition, can be cumbersome to apply by hand. In code, they become fast and playful. Try these transformations in the interactive sketch below—one at a time, then combining them together:

Exercise: the code example below reverses a melody. Read it, then try to implement inversion, transposition, shifting, and pitch rotation in pseudocode. Pick one of these operations and program it by hand.

function reverseSequence() {

  // Get the time range of the sequence
  const times = transformedSequence.map(note => note.time);
  const minTime = Math.min(...times);
  const maxTime = Math.max(...times);
  
  // Reverse the time values so notes are played in reverse order
  for (let i = 0; i < transformedSequence.length; i++) {
    const originalTime = transformedSequence[i].time;
    const reversedTime = maxTime - (originalTime - minTime);
    transformedSequence[i].time = reversedTime;
  }
}

Generation: Make New Melodies

So far, we’ve been making variations by transforming a melody we already have. Next, we’ll use the same score format to generate entirely new melodies. But how do we choose and sequence the tones?

Machine learning

One approach is to delegate this task to machines: having algorithms learn melody patterns from existing music and then use them to generate new melodies. This is the realm of machine learning (ML), which we will explore in more depth in Chapter 8. The interactive sketch below feeds a short tone sequence to an ML model trained on thousands of melody examples. Each time we click Generate, the model produces a plausible continuation of the input melody.

As effective as their predictions can be, ML models are still black boxes: they don’t tell us much about how melodies work or how we perceive them. To investigate this, we’ll take a different approach: writing code that generates melodies from scratch. Rather than generating “good” melodies, the point is to explore how we perceive the building blocks of melody. These are the elements musicians internalize, music theorists describe, and —crucially— music technologists inscribe into instruments, scores, and musical software and hardware. Making these choices explicit allows us to rethink them and explore alternatives of our own.

Random generation

If a melody is a sequence of tones, can we generate one by simply picking tones at random? Let’s try it. In the interactive sketch below, we create an array of tone objects with random values for pitch, velocity, position (time), and duration. Try generating a few sequences, then adjust the ranges of the random functions and listen to how each change affects the output.

const sequence = [];
const nTones = 20;
const minDuration = 0.0625; // Minimum duration (sixteenth note at 120bpm)
const maxDuration = 2.0;   // Maximum duration (whole note at 120bpm)

// Create a random sequence of tones. Each tone gets a random start time
// (anywhere in the first 8 seconds), pitch, velocity, and duration.
// They don't need to be in order: Tone.Part will sort them by time before playing.
for(let i = 0; i < nTones; i++){
  const tone = {
    time: Math.random() * 8, // Random time between 0-8 seconds
    pitch: Math.random() * 900 + 100, // Random frequency between 100-1000 Hz
    velocity: Math.random(),
    duration: Math.random() * (maxDuration - minDuration) + minDuration
  };
  sequence.push(tone);
}

Do these random tone sequences feel like melodies? This will depend, to some degree, on the ranges, your musical background and taste, and sheer luck (the tones are random, after all). It may also depend on exposure. As we saw in the Rhythm chapter, repetition is key in music: it helps establish patterns and expectations. Listen to any sequence of tones long enough, and it may start to “make sense.” In the interactive sketch above, try generating a few sequences, then listen to each one at least three times. You may find some you grow fond of—perhaps conveying ambiguity, freshness, or an uneasy feeling that resonates.

At the same time, many melodies we encounter day-to-day are shaped by constraints that make them easier to follow, sing, and remember. In the next section, we’ll program constraints into the random generator and listen for what changes.

Elements of Melody: Shape Randomness

Another way to define melody is as rhythm plus pitch. We’ll begin by constraining pitch—choosing a pitch set, a range, and a way of moving from note to note. Then, just like we did in the previous chapter, we will constrain position, duration, and volume constraints to shape the melody’s rhythm.

In the previous code example, we generate a random tone sequence upfront. To explore the constraints more fluidly—and hear changes immediately—we’ll now generate tones in real time. As we explore pitch, the sequence’s rhythm will be up to the listener: they will determine when each tone is played, for how long, and how loudly.

{
    // Determined by system 
    "pitch":    // Random within a range
    
    // Determined by listener 
    "time":     // When the listener presses the mouse
    "duration": // Until the listener releases the mouse
    "velocity": // Map to mouseY 
}

Let’s try it:

function mousePressed() {
  // Pick a random frequency between 100 and 1000Hz
  let frequency = random(100, 1000);

  // Trigger (play) a tone of that frequency
  synth.triggerAttack(frequency);
}

function mouseReleased(){
  // Release (stop playing) the tone
  synth.triggerRelease();
}

Sets of pitches

Most melodies don’t use an endless number of pitches—imagine trying to remember or sing a sequence created by the interactive sketch above. Instead, they’re made from a limited set. Let’s introduce this constraint to our random sequence. In the interactive sketch below, we pick pitches from a limited (but random) set within an octave [footnote pointing to Instrument chapter]. Click the “pick random pitches” button a few times and listen to the resulting sequences. Do you prefer some pitch sets over others? Are some pitches so close together that you can’t tell them apart?

function mousePressed() {
  // An array of 5 hand-picked frequencies (the values are arbitrary)
	const fs = [200.32, 350.55, 480, 670, 800]; 
  
  // Pick a random frequency from the array
  let pos = int(random(0, fs.length));
	let frequency = fs[pos];

  // Trigger (play) a tone of that frequency
  synth.triggerAttack(frequency);
}

How might we choose a set of pitches? Let’s return to our example melody. It uses the pitches highlighted in the interactive sketch below. Press keys a-g to play them. Can you play the melody? (tip: start from g)

// Initialize an array with the
// 5 frequencies in the Kill bill whistle
const pitchSet = [164.81, 185.00, 196.00, 220.00, 246.94];
	
function keyPressed() {
  // If a number key was pressed, play the respective position in the pitch set array
  let pos = int(key) % n - 1;  
  synth.triggerAttack(pos);
}

Now, where does this pitch set come from? At first glance, there’s no obvious relationship between their frequencies, but they do follow a pattern. In fact, it’s a five-note pattern that different cultures around the world arrived at independently. In this video (“Notes & Neurons: In Search of the Common Chorus” at the World Science Festival), Bobby McFerrin playfully demonstrates how instinctively familiar this pattern is to the audience. In the Tuning Systems annex, we look at these underlying patterns in detail. In the interactive sketch below, you can play with a range of pitch sets that cultures around the world have chosen to make their melodies with:

Pitch subsets are often associated with different moods—some may feel more joyful, more melancholy, or more serene. In the interactive above, try A minor vs C major. Do you sense a difference in mood? In some cultures, subsets are also associated with different times (Greece, India, China) or seasons (India). Try 22 Srutti System’s Shadja Grama. Then, try Sa on 2/1. Do these subsets feel distinct from each other as well?

Range

Now, let’s broaden (or limit) the range of the pitch set. As you know from Chapter 1, we perceive frequencies in cycles, and each cycle is called an octave. In the interactive sketch above, use the Tuning System parameters to change the starting octave and number of octaves the melody should span. Does limiting the sequence to a certain frequency range affect whether you perceive the tones as a single entity? Is there a limit to how large the range can be before the sequence starts to feel disjointed?

Motion: interval and direction

It can be useful to think of pitch subsets as ladders—in fact, in the West, the word for pitch set is scale, which means “ladder” in Italian. We can create a melody by starting somewhere on the ladder we picked (say, on the 1st step), then leaping up and down its rungs by a number of steps, or an interval. This suggests another useful constraint for our random melodies: deciding how far we jump to the next pitch, and in which direction.

Over time, these steps and leaps trace a shape, or contour: as it moves through pitch space, how far, how fast, and in what direction does your melody move? Do tones rise to higher pitches, fall to lower ones, or stay the same? Do they move by step to adjacent tones or leap by larger intervals? Strong melodies often balance leaps and steps, and ascending and descending motion.

Can you create a melody that, over several leaps, draws an arch? A wave? A jagged zig-zag? Can you automate a certain kind of contour in code?

Rhythm

Up to this point, we have been focusing on how melodies move through pitch space. For each tone in our sequences, the code chooses its pitch. Listeners decide when to play each tone (its position), for how long (its duration), and how loudly (its volume): they determine the melodies’ rhythm.

{
    // Determined by system 
    "pitch":    // random within a pitch grid 
    
    // Determined by listener
    "time":     // When the listener presses the mouse
    "velocity": // Map to listener's mouseY position
    "duration": // Until the listener releases the mouse

}

In the Rhythm chapter, we saw how position in time and volume organize sounds into rhythms: repeating patterns create a virtual time grid in listeners’ minds and set expectations for when new sound events will happen. This is also true for melodies —clap along the rhythm of a melody, leaving pitches out, and you might still be able to recognize it.

In the next interactive sketch, the code determines both the melody’s pitch and its rhythm. As in the Rhythm chapter, we define a time grid by setting a tempo, meter, and subdivisions. Then we place tones on it: positions and durations align to grid lines, and velocities reflect the meter.

// Determined by system
{
    "pitch":    // Random position on pitch grid
    "time":     // Random position on time grid (after the previous note ends)
    "duration": // Random grid-based value from a set (i.e. 4n, 32n) 
    "velocity": // Determined based on grid position (if accents are on)
}

To vary density, we include a probability of silence. To add a touch of unpredictability, a swing variable nudges tones off the grid lines by small, random amounts. In the interactive sketch below, click Play to hear a sequence generated in real time. Then, adjust these parameters and listen to how they shape its rhythm:

Melody study

Create an interactive exploration of melody. Let your own curiosity guide the direction of the work. You might begin with a particular melody or musical concept you want to understand, an interaction or interface you want to explore, or simply an intuition you want to follow. Your study may draw on existing melodies or generate entirely new ones.

As you develop the piece, think about the melodic elements discussed in this chapter: tuning systems, pitch sets or scales (and their moods), range, motion and contour, and rhythm. Which elements are fixed, and which are variable? Within what ranges do they vary? Do they respond to participant input, change on their own, or incorporate a degree of randomness? (Participant means the person interacting with your study—someone else, you, or both.)

As in previous creative coding projects, also consider:

• Controls. What are the participant inputs, or system inputs? Are they continuous or discrete? Inputs might come from a keyboard, track pad, touch screen, camera-tracked gestures, sensors, images, audio, or data streams.
• Mapping. How do controls map to melody elements? One-to-one? One-to-many? Many-to-one? Is the relationship direct, inverse, linear, or exponential?
• Visual representation and feedback. Which elements have a visual representation? How are shape, position, scale, rotation, hue, saturation, brightness, and texture used? What are their ranges of variation?

Spend some time playing with your study. How does it negotiate character and flexibility, predictability and surprise, and ease of use and expressivity?

Example projects:

• Apotome by Khyam Allami and Counterpoint (2021). A probabilistic, transcultural browser-based generative music environment that uses microtonal tuning systems and their subsets (scales/modes).
• Pixel Synth by Olivia Jack (2016). A browser-based synthesizer that creates sound from images and drawings. Inspired by the ANS synthesizer created by Evgeny Murzin in 1937.
• Suite for Solo Voice & Electronics by Pamela Z (2017). A gesture-controlled system that captures, layers, and transforms live vocal phrases.
• Bloom by Brian Eno (2008). A generative music application that lets listeners create ambient soundscapes through simple touchscreen interactions.
• Sonic Wire Sculptor by Amit Pitaru (2003). A tablet-based system for composing looping scores. Melodies are drawn as lines that wrap around a cylindrical 3D surface.
• Oramics Machine by Daphne Oram (1962-69). An opto-electronic musical interface that read and interpreted hand-drawn waveforms (timbres) and sequences of control information for musical pitch and dynamics.