Effects in frequency domain

Stefania SerafinMusic InformaticsFall 2004

Overview

Time versus frequency domain effects The phase vocoder The STFT Time stretching Pitch shifting Cross synthesis

Overview

Until now we examined effects based on delays and working in time domain.

What happens if we want to work in the frequency domain?

In this case we talk about SPECTRAL MODELS and SPECTRAL TRANSFORMATIONS

The phase vocoder

A phase vocoder represents an ensemble of techniques which take a sound in the time domain,calculate the Fourier transform, perform some manipulations in frequency domain and then reconstruct the sound in time domain.

Analysis/synthesis steps

ANALYSIS

TRANSFORMATIONS

SYNTHESIS

INPUT

OUTPUT

Analysis synthesis steps

Analysis: STFT

STFT

Time-domain audio signal is separated into successive short frames.

Each frame is multiplied by a window function to smooth transitions.

Resulting windows are passed on to FFT module. The FFT process then extracts spectral information on the signal.

Spectral transformations

Pitch shifting Time stretching Morphing (Cross-synthesis)

Pitch shifting

How would you perform a pitch shift in time domain?

What are the drawbacks of doing it?

Solution: move to frequency domain In frequency domain 1) Calculate the spectrum 2) Shift it upwards or downwards 3) Go back to time domain

Time stretching

Time-scaling is an application that allows signal length to be stretched or shortened without affecting the frequencies of its components.

1) Calculate spectrum 2)Reduce/increase time information 3) Go back to time domain

Morphing

It is the combination of two signals

In frequency domain:

1) Calculate spectrum of signal A

2) Calculate spectrum of signal B

3) C = A B

STFT resynthesis

STFT resynthesis is the final stage of the phase vocoder process. It is based on recreating a signal in time domain from the modified spectral components of the input signal.

The third stage implements an exact inverse of the process that was used in STFT analysis of the signal. Modified spectral components of the input signal are passed through the Inverse Fourier Transform to recreate a set of time-domain frames.

Application

Farinelli

Application (ctn.)

Karaoke

In Max/MSP

Check the examples Folder. Convolution and Phase vocoder example.