Processing audio in the browser!

The modern web is an amazing place. You can do almost anything in a browser that you can do in native code. Including processing audio in realtime. In javascript.

Nuts? Probably. Worth it? Definitely.

The Web Audio API lets you generate or modify audio on the fly. For example, Google Chrome Labs has an example that generates white noise by filling a buffer with random floats. MDN has an example that displays a simulated oscilloscope of the microphone input (and it’s extremely lightweight!). It’s really quite elegant.

The way that it works is that it gives us a way to programmatically modify the audio graph by inserting your own filters and processors. The specification includes this hilariously simply illustration:


…but essentially:

  • The AudioContext interface represents the entire audio system (the “audio graph”)
  • The audio graph contains many AudioNodes, which connect the audio “source” (microphone, video, etc…) to the “output” (usually your speakers)
  • About 14 kinds of nodes (AnalyzerNode, AudioBufferSourceNode, AudioBufferDestinationNode, etc…) that we mostly don’t care about right now
  • Also contains AudioWorklet, AudioWorkletProcessor, interfaces and an AudioWorkletNode.

The last few are the ones I’m interested in. I want to build CinemaGazer(1)(2)(3 from ACM) for the web. This part of the API should, in theory, enable me to process the audio in realtime (using some Voice Activity Detection), and fast forward through the silent parts of a video.

It’s also not that had to set up. First, you write a little tiny processor class in a separate file (and register it):

class CinemaGazerProcessor extends AudioWorkletProcessor {
    process (inputs, outputs, parameters) {
        return true;

registerProcessor('cinema-gazer-processor', CinemaGazerProcessor);

Then, you need to grab the AudioContext:

let audioContext = new AudioContext();
Load your module (with a promise):


Then all you need to do is to connect it up:
.then(function() {
        // debugger;
        const cinemaGazerNode = new AudioWorkletNode(audioContext, 'cinema-gazer-processor')
        const videoElement = document.getElementById("video-1");
        let audioSource = audioContext.createMediaElementSource(videoElement);
        // let audioFilter = audioContext.create
You’re basically all set then! You can write your processing code in the process method.
In the future, I hope to elaborate on CinemaGazer.

~ by Alexander Riccio on July 18, 2019.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Shelly L. Miller

I am an environmental engineer. I teach and research urban air pollution.

Lucky's Notes

Notes on math, coding, and other stuff


Abandoned places and history in the five boroughs

Open Mind

KIDS' LIVES MATTER so let's stop climate change

I learned it. I share it.

A software engineering blog by György Balássy

Kitware Inc

Delivering Innovation

The Electric Chronicles: Power in Flux

If someone ever tells you that you don't need more power, walk away. You don't need that kind of negativity in your life.

Ted's Energy Tips

Practical tips for making your home more comfortable, efficient and safe

love n grace

feel happy, be happy

Recognition, Evaluation, Control

News and views from Diamond Environmental Ltd.

greg tinkers

Sharing the successes and disasters.

Sam Thursfield

Software and technology from Galicia, Spain

Cranraspberry Blog

Sharing the things I love


Advances in biological systems.

The Embedded Code

Designing From Scratch

Sean Heelan's Blog

Verification, Program Analysis and Security


Connecting Research, Policy, and Practice in Education


A Group Complaint about Law, Liberty, and Leisure

Home & Kitchen Appliance Blog

Bad Science Debunked

Debunking dangerous junk science found on the Internet. Non-scientist friendly!

4 gravitons

The trials and tribulations of four gravitons and a physicist

Strange Quark In London

A blog about physics, citylive and much procastination

The Lumber Room

"Consign them to dust and damp by way of preserving them"

In the Dark

A blog about the Universe, and all that surrounds it

andrea elizabeth

passionate - vibrant - ambitious

Probably Dance

I can program and like games

a totally unnecessary blog

paolo severini's waste of bandwidth

Musing Mortoray

Programming and Life

PJ Naughter's space

Musings on Native mode development on Windows using C++

  Bartosz Milewski's Programming Cafe

Category Theory, Haskell, Concurrency, C++

Brandon's Thoughts

Thoughts on programming

David Crocker's Verification Blog

Formal verification of C/C++ code for critical systems

10 Minute Astronomy

Stargazing for people who think they don't have time for stargazing.

One Dev Job

notes of an interactive developer

Chief Cloud Architect & DevSecOps SME, Enterprise Architect, Agile Coach, Digital Transformation Leader, Presales & Tech Evangelist, Development Manager, Agilist, Mentor, Speaker and Author

TOGAF Certified Enterprise Architect • AWS Cloud Certified Solutions Architect • Azure Cloud Certified Solutions Architect • Scrum Alliance: Certified Scrum Professional (CSP), Certified Agile Leadership I (CAL 1), CSM, ACSM • Kanban Management Professional (KMP I & KMP II), Certified Enterprise Agility Coach (CEAC) • SAFe: Certified SAFe Architect, SAFe DevOps, Release Train Engineer (RTE), SAFe Consultant (SPC) • Certified Less Practitioner (CLP), Six Sigma (Greenbelt), Training from the Back of the Room (TBR) Trainer • Certified Agile Coach & Facilitator: ICP-ACF & ICP-ACC

The Angry Technician

No, the Internet is not broken.

Kenny Kerr

Creator of C++/WinRT and the Windows crate for Rust • Engineer on the Windows team at Microsoft • Romans 1:16

IT affinity!

The Ultimate Question of Life, the Universe, and Everything is answered somewhere else. This is just about IT.


The ramblings of a crazed mind

Molecular Musings

Development blog of the Molecule Engine

%d bloggers like this: