Let's Build a Synthesizer in Python!

No description has been provided for this image

In this notebook we will have fun with sound in Python: we will synthesize sounds (sine, square, triangle, noise), play chords and melodies, apply sound effects (e.g., fade-in/out, echo, vibrato, low/high-pass filter, stereo panning), and read/write audio files in Python.

Note: The goal is to learn the fundamentals of sound processing by implementing everything from scratch, but in a real project you will want to use specialized libraries such as librosa.

Sounds Waves¶

Let's start by generating a sine wave of frequency 440 Hz. That's the standard A4 (A on the 4th octave on a piano) which is often used to tune instruments.

In [1]:

import numpy as np

sample_rate = 44_100

def sine_wave(frequency, amplitude, duration):
    time = np.linspace(0, duration, round(sample_rate * duration))
    return amplitude * np.sin(2 * np.pi * frequency * time)

Let's plot the first 500 samples:

In [2]:

import matplotlib.pyplot as plt

wave_A4 = sine_wave(frequency=440, amplitude=1, duration=1)

def plot_wave(wave, title):
    plt.figure(figsize=(5, 2))
    plt.plot(wave)
    plt.xlabel("Time (samples)")
    plt.ylabel("Amplitude")
    plt.title(title)
    plt.grid()
    plt.show()

plot_wave(wave_A4[:500], "Sine wave")

Nice! Now, let's play this note using IPython's builtin Audio class which can play a given waveform at the given sample rate:

In [3]:

from IPython.display import Audio

def play_wave(wave):
    return display(Audio(wave, rate=sample_rate))

WARNING: this sound may be loud, make sure to adjust the volume so you don't damage your ears or speakers (or wake up your neighbors).

In [4]:

play_wave(wave_A4)

We made an A4 note! That's a start. Now let's generate other notes. Going up one octave corresponds to doubling the frequency. Similarly, going up by one semi-tone corresponds to multiplying the frequency by $r = \sqrt[12]{2}$ (see this cool video for more details). Indeed, there are 12 semi-tones per octave, and $r^{12} = 2$. This is called equal temperament.

In [5]:

def get_frequency(note, octave):
    return 2 ** (octave - 4 + note / 12) * 440

In [6]:

frequencies = [
    get_frequency(note, octave)
    for octave in range(2, 6)
    for note in range(12)
]
all_notes = np.concatenate([
    sine_wave(freq, amplitude=0.1, duration=0.15)
    for freq in frequencies
])
play_wave(all_notes)

This worked but what's this clicking sound between each note? Well that's because the sine_wave() function generates a wave that starts and ends brutally, and depending on the frequency it may even end with a non-zero amplitude. This is what the transition between two notes looks like (notice the brutal bounce):

In [7]:

plot_wave(all_notes[5800:7400], "Bad transition")

There are various solutions (e.g., we could continue each wave where the previous one left off, or implement a cross-fade effect), but let's simply fade in and fade out each note (for 10ms in and 10ms out):

In [8]:

def fade(wave, fade_duration=0.010):
    num_fade_samples = round(sample_rate * fade_duration)
    fade_in = np.linspace(0, 1, num_fade_samples)
    new_wave = wave.copy()
    new_wave[:num_fade_samples] *= fade_in  # fade in
    new_wave[-num_fade_samples:] *= 1 - fade_in  # fade out
    return new_wave

In [9]:

all_notes_with_fadeout = np.concatenate([
    fade(sine_wave(freq, amplitude=0.1, duration=0.15))
    for freq in frequencies
])
play_wave(all_notes_with_fadeout)

This sounds much better! Now if we look at the note transition, is nice and smooth:

In [10]:

plot_wave(all_notes_with_fadeout[5800:7400], "Smooth transition")

Chords¶

Ok, now let's play our first chord: we just need to sum the waves of all the notes, that's all. Let's play an A major chord (A + C# + E).

In [11]:

wave_A3 = sine_wave(get_frequency(0, 3), 0.5, 1)
wave_Cs3 = sine_wave(get_frequency(4, 3), 0.5, 1) # Cs = C sharp
wave_E3 = sine_wave(get_frequency(7, 3), 0.5, 1)
wave_A4 = sine_wave(get_frequency(0, 4), 0.5, 1)
chord = wave_A3 + wave_Cs3 + wave_E3 + wave_A4
play_wave(fade(chord))

The pattern looks much more complex now:

In [12]:

plot_wave(chord[:3000], "Chord")

Melodies¶

We're ready to play a melody! To enter it, it's more convenient to use the names of the notes, so let's write a helper function for that:

In [13]:

def get_frequency_by_name(name):
    if name == ".":
        return 0  # silence
    if "b" in name:
        note = "A Bb B C Db D Eb E F Gb G Ab".split().index(name[:-1])
    else:
        note = "A A# B C C# D D# E F F# G G#".split().index(name[:-1])
    octave = int(name[-1:])
    return get_frequency(note, octave)

In [14]:

def get_melody_wave(melody, waveform=sine_wave, tempo=120):
  frequencies = [get_frequency_by_name(note) for note in melody.split()]
  duration = 60 / tempo / 4
  return np.concatenate([
    fade(waveform(freq, 1.0, duration))
    for freq in frequencies
  ])

Now let's play the melody:

In [15]:

melody = (
    "C3 E3 G3 C4 E4 G3 C4 E4 "
    "C3 E3 G3 C4 E4 G3 C4 E4 "
    "C3 D3 A4 D4 F4 A4 D4 F4 "
    "C3 D3 A4 D4 F4 A4 D4 F4 "
    "B3 D3 G3 D4 F4 G3 D4 F4 "
    "B3 D3 G3 D4 F4 G3 D4 F4 "
    "C3 E3 G3 C4 E4 G3 C4 E4 "
    "C3 E3 G3 C4 E4 G3 C4 E4 "
)

melody_wave = get_melody_wave(melody)
play_wave(melody_wave)

In case you're wondering, this is Bach's Well-Tempered Clavier, Book 1: Prelude No. 1 in C Major, BWV 846.

Bass Sound With Square Waves¶

We can also generate different types of sounds, for example a square wave:

In [16]:

def square_wave(freq, amplitude, duration):
  num_samples = round(sample_rate * duration)
  if freq == 0:  # in case we just want silence
    return np.zeros(num_samples)
  t = np.linspace(0, duration, num_samples)
  phase = freq * t
  is_positive_half = (phase - np.floor(phase)) < 0.5
  return amplitude * (2 * is_positive_half - 1)

In [17]:

square_wave_A2 = fade(square_wave(get_frequency(0, 2), 0.2, 1.0))
play_wave(square_wave_A2)

Here's what the square wave looks like:

In [18]:

plot_wave(square_wave_A2[500:2500], "Square wave")

Let's use it to define a funky little bass:

In [19]:

bass = (
    "C2 . C2 C2 C1 . . C1 "
    "C2 . C2 C2 C1 . . C1 "
    "D2 . D2 D2 D1 . . D1 "
    "D2 . D2 D2 D1 . . D1 "
    "G1 . G1 G1 . G1 . G1 "
    "G1 . G1 G1 . G1 . G1 "
    "C2 . C2 C2 C1 . . C1 "
    "C2 . C2 C2 C1 . C1 C1 "
)

bass_wave = get_melody_wave(bass, waveform=square_wave)
play_wave(bass_wave)

In [20]:

plot_wave(bass_wave[:80_000], "Funky Bass Line")

Sound Effect — Echo¶

How about we add some echo?

In [21]:

from math import ceil, log

def apply_echo(wave, freq, decay, min_amplitude=0.01):
  repeats = ceil(log(min_amplitude) / log(decay))
  echo_interval_samples = round(sample_rate / freq)
  samples_with_echo = len(wave) + repeats * echo_interval_samples
  output = np.zeros(samples_with_echo)
  amplitude = 1.0
  for i in range(0, samples_with_echo - len(wave), echo_interval_samples):
    output[i:i+len(wave)] += wave * amplitude
    amplitude *= decay

  return output

In [22]:

bass_with_echo = apply_echo(bass_wave, 1.333, 0.5)
play_wave(bass_with_echo)

We can see the echo added on top of the bass:

In [23]:

plot_wave(bass_with_echo[:80_000], "Bass with echo")

Multiple Tracks¶

Hey, why don't we play the melody and bass together?

In [24]:

melody_and_bass = bass_with_echo.copy()
melody_and_bass[:len(melody_wave)] += melody_wave
play_wave(melody_and_bass)

That was pretty nice! Now we can apply some sound effects.

You can try other types of sounds if you want, such as triangle waves:

In [25]:

def triangle_wave(freq, amplitude, duration):
    t = np.linspace(0, duration, round(sample_rate * duration))
    phase = freq * t
    phi_frac = phase - np.floor(phase)
    phi_scaled = phi_frac * 2
    abs_dev = np.abs(phi_scaled - 1)
    wave = 2 * abs_dev - 1
    return amplitude * wave

triangle_bass_wave = get_melody_wave(bass, triangle_wave)
play_wave(triangle_bass_wave)

Sound Effect — Vibrato¶

There's an endless world of sound effects we can implement. For example, let's implement a vibrato effect:

In [26]:

def apply_vibrato(wave, freq, depth):
    duration = len(wave) / sample_rate
    amplitude = sine_wave(freq, 1, duration)
    return wave * (1 + depth * amplitude)

for depth in [0.2, 0.4, 0.6, 0.8, 1.0]:
  chord_with_vibrato = apply_vibrato(chord, 6, depth)
  play_wave(chord_with_vibrato)

Texture & Noise¶

We can give our sounds a bit of texture by adding some noise:

In [27]:

def noise(duration):
    return np.random.uniform(-1, 1, round(sample_rate * duration))

play_wave(wave_A4 + 0.2 * noise(1))

Sound Effect — Low-Pass Filter¶

Apply a low-pass filter using a Fast Fourier Transform (FFT) to eliminate the high-frequencies (if you don't know about the Fourier Transform, please watch this great video by 3blue1brown):

In [28]:

music_clip = melody_and_bass[:2 * sample_rate]
for cutoff_frequency in (4000, 2000, 1000, 500):
  frequencies = np.fft.rfft(music_clip)  # real domain to complex frequency domain
  frequencies[cutoff_frequency:] = 0
  low_pass_filtered = np.fft.irfft(frequencies)  # back to real domain
  play_wave(low_pass_filtered)

You can easily convert this code to a high-pass filter by zeroing out the low frequencies instead of the high frequencies (i.e., replace frequencies[cutoff_frequency:] = 0 with frequencies[:cutoff_frequency] = 0). Try it!

What if we want to change the filter over time? Well, one option is to chop the wave into small segments and apply an FFT-based low-pass filter to each segment, using a different cutoff frequency for each segment.

Alternatively, we can work directly in the time domain (without using any FFT) by applying an exponential moving average, which dampens the high frequencies:

$\text{output}(t) = \alpha \times \text{output}(t - 1) + (1 - \alpha) \text{input}(t)$

This is called a 1-pole low-pass filter. $\alpha$ controls the cutoff frequency:

$\alpha = e^{-2\pi \dfrac{\text{cutoff frequency}}{\text{sample rate}}}$

If you want a high-pass filter instead, just subtract the low-pass filtered wave from the original wave.

Let's apply a high-pass filter with a sliding cutoff frequency:

In [29]:

music_clip = melody_and_bass[:5 * sample_rate]
t = np.linspace(0, 1, len(music_clip))
cutoff_frequency = np.linspace(0.1, 1, len(music_clip)) * 1000
low_pass_filtered = np.zeros_like(music_clip)
alpha = np.exp(-2 * np.pi * cutoff_frequency / sample_rate)
for i in range(1,len(music_clip)):
    low_pass_filtered[i] = (alpha[i] * low_pass_filtered[i-1]
                                  + (1 - alpha[i]) * music_clip[i])

high_pass_filtered = music_clip - low_pass_filtered
play_wave(high_pass_filtered)

Stereo & Panning¶

If you stack two waveforms, you get a stereo waveform. For example, let's pan the sound from left to right:

In [30]:

left_wave = high_pass_filtered.copy()
right_wave = high_pass_filtered.copy()
pan = np.linspace(0, 1, len(left_wave))
left_wave *= 1 - pan
right_wave *= pan

stereo = np.stack([left_wave, right_wave])
play_wave(stereo)

Let's update our plot function to support stereo:

In [31]:

def plot_sound(stereo, title):
    if stereo.ndim == 1:
        stereo = stereo.reshape(1, -1)
    plt.figure(figsize=(5, 2))
    alpha = 1
    for channel in stereo:
      plt.plot(channel, alpha=alpha)
      alpha = 0.7
    plt.xlabel("Time (samples)")
    plt.ylabel("Amplitude")
    plt.title(title)
    plt.grid()
    plt.show()

And let's try it out on the panning sound:

In [32]:

plot_sound(stereo, "Stereo Sound")

Read/Write Audio Files¶

If you want to save your art, you can use the soundfile library:

In [33]:

import soundfile as sf

sf.write("stereo.wav", stereo.T, sample_rate)

In [34]:

loaded_wave, sample_rate = sf.read("stereo.wav")
play_wave(loaded_wave.T)

Notes:

The channel dimension is expected to be last, which is why we transpose the array (.T) before saving and after loading.
You can replace .wav with .mp3 (or other formats supported by soundfile).
Once you have saved a file, you can download it to your computer: click the folder icon in the left sidebar, then right-click on your file and select Download.

Now it's your turn to play around with sound effects. Here are a few ideas for you:

Make retro videogame sounds:
- Laser zap: descending frequency sweep + high-pass + fast decay
- Jump: upward pitch sweep + square wave
- Coin pickup: very fast arpeggio (e.g., C6 E6 G6)
- Explosion: filtered noise + exponential decay
- Footsteps: short bursts of filtered noise with alternating filters
Generate natural sounds:
- Wind: white noise + low-pass + slow cutoff oscillation
- Ocean: noise + band-pass sweep + random amplitude modulation
- Rain: lots of very short clicks at random intervals
- Fire crackle: noise + random gain spikes
Build instruments
- Kick drum: sine at 80Hz + pitch drop + exponential decay
- Snare: noise + high-pass + short decay
- Hi-hat: noise + band-pass at 8kHz
- Clap: bursts of filtered noise + slight delays
- Bell: carrier 440Hz, modulator 660Hz
Play around with FFT-domain manipulations
- Randomly reorder FFT bins
- Amplify or reduce magnitudes
- Low-pass the phase instead of the magnitude
- Zero out every other FFT bin
Try various envelopes (beyond simple fade-in/out)
Record sample sounds (e.g., your voice) and modify them in interesting ways.
Use all that to generate some transe music similar to Switch Angel

Have fun! 🎵😊