GPU Technology conference| September 30th, 2009

Rudy Sarzo and Ian Williams

Have You Heard What a GPU Can Do?

- A Revolution in Audio Processing

Have You Heard What a GPU Can Do?A Revolution in Audio Processing• State of the Art for Digital Audio Processing - Ian

– Background

– Isn’t Digital Audio a solved problem?

– How the GPU can help?

• Integrating the GPU into the Professional Audio Workflow - Rudy

– Professional Digital Audio tools and the workflow

– Key problems with Digital Audio

– How the GPU can revolutionize Digital Audio Processing

Digital Audio Processing is Ubiquitous

• Digital Audio is integrated into all areas of life

– iPods, Media Players etc

– Phones, Cellphones etc

– Home Theater Systems, Digital TVs etc

– PCs – Media Players, Audio Clean up software etc

• Audio processors cover spectrum of size and form factors

– Dedicated ASICs, Engines, FPGAs, CPUs

• A wide range of Audio Editing Software

– DAW software, Plugins etc.

• Current Digital Audio processing mostly based on Linear

Convolution

– Finite Impulse Response (FIR) filter

– Samples of input waveform multiplied by samples of impulse response

and summed

– Stable – settles to zero after N+1 input samples

– Doesn’t require feedback – errors don’t propagate

• Works well in a wide range of situations

Linear Convolution

Why use a GPU for Digital Audio?

What would Marvin say if he were a GPU:

“I have a Brain the size of a planet and

you want me to convolve two buffers?”

Source: Hitch Hikers Guide to The Galaxy

Realism & Quality Drive Up Computational Complexity• Linear assumptions have key limitations:

– Can frequently sound “dry” & “unnatural”

• Some desirable characteristics occur because of non-linearities

– e.g. harmonic distortion

• Many effects are also time variant, e.g. flanging, Phasing etc.

– modeling these significantly increase computational complexity

• Multi-channel

– Allows greater control over sound placement & mix => better experience

Co-Processing for Digital Audio

• An Orchestra needs a great conductor

• However….the conductor shouldn’t play each

instrument!

• Historically professional Digital Audio Editing

has required dedicated Audio DSPs

• The GPU is a great Audio DSP

– and its already in the system!

Image courtesy of Digidesign

Emulating Non-Linear Characteristics in Digital Audio• Impulse Response (IR) Switching

– IR measured for different amplitudes of signal

– IR selected based on input amplitude

• Focusrite – Liquid Technology using“Dynamic Convolution”

Focusrite LIQUID 4 Pre Preamp

Focusrite LIQUID 4 Pre

Emulating Non-Linear Characteristics in Digital Audio (cont.)

• Volterra Series

– Volterra Series similar to Taylor Series

– Taylors series output depends strictly on the input at a particular time

– Volterra series output depends on the input to the system at all other times

Emulating Non-Linear Characteristics in Digital Audio (cont.)

• Volterra Series

– Volterra Series similar to Taylor Series

– Taylors series output depends strictly on the input at a particular time

– Volterra series output depends on the input to the system at all other times

• Diagonal Volterra Kernels

– Convolve 1st (linear) order IR, 2nd order IR, 3rd order IR, etc separately then sum the result

– Each order IR is convolved with the input signal raised to the appropriate power:

Source: “Realtime auralization employing a not-linear, not-time-invariant Convolver”, Angelo Farina 1, Adriano Farina 2, AES 2007

State of the Art Digital Audio Processing

ACUSTICA Audio – Nebula 3– VST based plug-in implementing Vectorial Volterra Kernel Technology

– Multiple simultaneous, non-Linear, time varying, level dependent effects

ACUSTICA Audio - Nebula 3• Kernel Engine

– Processes large number of kernels simultaneously

– Controlled by the Vectorial Engine

• Vectorial Engine

– Operates at defined rate (PROG RATE)

– Many IR samples continually swapped in and out:

• Multiple IRs per effect

• Multiple simultaneous effects

• Time dependent IRs

• Settings within effects

• Accurate Modeling of Complex Sophisticated Effects

– Dynamic harmonic distortion – valve preamps & compressors

ACUSTICA Audio – Nebula Sampler

• Comes with Nebula 3

• Automates measurement of IRs

– Including level and time and dependent Irs

• Easy to create new libraries of complex sophisticated effects

http://www.alessandroboschi.eu/html/en/alexb.htm

How Nebula 3 Uses NVIDIA CUDA

• Vectorial Engine currently operates on the CPU

• Kernel Engine implemented entirely on the GPU

– CUDA Streams

– Efficient algorithm leaves partial results for each harmonic on the GPU, only final results

transferred off GPU:

– Uses CU FFT library as well as hand tuned Vasily Volkov FFT algorithms

Rudy Sarzo

Digital Audio – “The Process”

Shrinking the recording studio into a computer

Source: www.vai.com

Sony Acid Pro 7

Studio Tour

How does a Professional Musician use Digital Audio?• Recording

– Highest quality very important - bit depth, sample rate etc

– Balancing what’s practical for editing

– No Clicks!!

• Editing

– Retaining high quality is still crucial

– Need to be able to work with many effects

– Evaluate alternative mixes – stereo->mono, low bit rate mp3

• Playback

– Create multiple delivery formats/mixes – MP3, AC3, 2.0, 5.1 etc.

Key Problems in Digital Audio

• Overloading / Insufficient headroom during recording

– Clicks (relying to heavily on CPU - cycle overload)

• Poor quality effects/algorithms/HW

– Just sound bad

– Captures clicks (again relying to heavily on CPU - cycle overload)

• Hard limits on the number of simultaneous effects

– Not enough CPU cycles for processing

– Mixing takes longer

– Workflow changes depending on platform (laptop, deskside DAW)

• ………

The Benefits the GPU brings to Digital Audio

Sony Acid Pro + Nebula3 VST Plug-in built on CUDA Technology

Demonstration of CUDA Processing for Audio

A Revolution in Audio Processing

• Digital Audio is here to stay

• The GPU is the perfect co-processor

– And its already in the system

• The CUDA Architecture is the catalyst

• Fermi Architecture introduces new key features to benefit audio

Thanks!

• Questions?