Category Archives: Sound

TypeScript declarations for the Web Audio API

After having done some work on porting Patchwork to Javascript last year using the Web Audio API, a project I departed because of some unsolvable problems (mainly, but not limited to, the custom AudioNodes adding a delay in the audio chain), I recently picked up a small experiment again using said API. Working with TypeScript this time, a declarations file for the Web Audio API was needed and after some searching without any result, I decided to make one myself.

There are some minor issues with it, although they don’t prevent regular use afaik:

  • I coulnd’t get readonly attributes in there using getters without Visual Studio complaining about my syntax. For now, I’ve just made them into regular properties and marked them with a comment so I (or someone else) can change them later on. (Using them as setters will be allowed by Visual Studio, but will logically give an error during runtime).
  • I don’t know how to implement constants (there are two sets of uppercase consts in there), so I also made them into regular attributes.
  • The MediaStreamAudioDestinationNode has an attribute of type MediaStream, which isn’t a part of the Web Audio API (it appears to be defined here). It doesn’t really belong in this file, but since I am not planning on using it, I just added an empty interface to prevent any errors. You should remove it if you have a declaration for the MediaStream API.

Apart from that, it works fine so if you want to do some Web Audio API programming using TypeScript yourself, you may find it useful and you can download the declarations file here.

A quick preview of the new Patchwork

Even though my last post was several months ago, I’ve been doing a lot of work on the new version of Patchwork (the modular synthesizer that I wrote in Flash) since then. I will spend more words when I release a more polished version, for now I’ll keep it very short with a link to this beta version.

It’s work in progress, comments are still appreciated though.

Patchwork 0.2: some updates

I did some updates on Patchwork, the modular sound synthesizer in Flash that I released a few weeks ago. You can check the new version here or read on to learn about what is new.

First of all, a little insight into what has happened since the initial release, because Patchwork has gotten quite some attention since then. After I put that very first version online, I started off by posting it in the programming section of Reddit (best site ever btw) as a ‘hey look what i made!’ thing, which I usually do when I’ve made something new. If Reddit likes it (and they did), you can expect (depending on the section) thousands of visitors so that’s always a nice kick-off.

After that, I sent an email to Andre Michelle, notorious guru on the subject of sound in Flash, and within 15 minutes he twittered about it, an act that already made my day. With him having a lot of interested followers on this specific subject, things really started rolling from there. People retweeted it, shared the link on Facebook and blogs all over the world wrote about Patchwork. Sweet! The best thing was Richard Devine twittering about it, and for those who don’t know him: he’s a guy who really knows how to patch some modules together.

Since then about 7500 visitors have visited my site, a number I consider quite a success, and a lot of people were having fun with Patchwork. Which made me wonder where this could and should go (apart from expanding the application itself). One thing that bothered me was reading about people creating patches without being able to see and hear what they made. So, it was quickly decided that there should be a website: user profiles, lots of patches, categories, ratings, sharing, embedding, etc. I convinced some nice people from work to help me out with that, and you can expect something in a few months. Thoughts and ideas on that, and on the current version Patchwork itself as well, are always welcome.

Then, finally… about those updates:

  • There’s a sample player in the generators section now. You can load either a local or an external wav-file and play it with a trigger signal. Remember though that samples loaded from your computer will not be saved with the patch (external ones will). For external files to be loaded, the server has to give permission for that through a crossdomain.xml file, which unfortunately isn’t always present. Although stereo samples are supported, the module has only one output at the moment. Will fix that soon.
  • The sequencer modules have a trigger output that will fire whenever they jump from the end of the sequence back to the beginning. You can use this in combination with the new reset input to synchronize multiple sequencers. Just connect the Sequence End Trigger output with the Reset Trigger input of another sequencer, and they should run synchronized.
  • The trigger sequencer now also has a continuous output which will send out its current state (either 1 or 0) continuously. This behavior is different from the trigger output, which generates only a single value of 1 (and after that 0′s again) when a selected sequencer item is encountered and can be used as a trigger signal. (Note: both outputs can in theory be used as a trigger-signal (which in Patchwork is a value of 1 and a previous value of 0), but the difference lies in this: if you’d have 2 sequential sequencer items selected, the trigger output would generate 2 triggers and the continuous output only one, since the latter would go from 0 to 1 on the first item, and stay 1 all the way through the second item.)
  • I fixed quite a huge bug in saving a patch. I messed up some ids which could corrupt a patch forever.
  • The textfield underneath a knob can be used to manually enter a value by clicking on it.
  • When turning a knob, you can hold shift to enter precision-mode. During this, the values will be rounded as well.
  • Saving a file now includes the viewport location so you will have the correct view after loading a patch.
  • I made a unified messaging system that allows me to give the user feedback on whatever is happening.
  • Patchwork now starts and loads an example automatically. This is more inviting than entering the page and staring at an empty screen.
  • The LFO (still just a simple sine, sorry) has a reset trigger input.
  • There are two types of comment modules, allowing you to include information in your patch. One is just text, the other one allows to to point something out.

I know there aren’t that much new audio modules (sawtooth oscillator, more LFOs, envelopes, effects, filters, etc) but that will come (if you’d like to help with that, let me know). Last but not least: if you make any interesting patches, I’d be happy to include them in the examples.

Modular sound synthesis in Flash: Patchwork

Realtime sound generation in Flash. Is there anything more to prove since AudioTool? Their team made an amazing application that allows you to hook up different synthesizers, effects and drumcomputers, and wrapped it all up in a very sparkling interface. So yes, realtime sound synthesis can be done and it’s actually quite easy to get started. It takes just a few lines of code to be able to write data directly to your soundcard and when you can do that, you have access to a whole lot of fun.

Having a background in audio, I’ve used quite a few music and sound applications and the modular ones always had my special interest: Max/MSP by Cycling ’74, Native Instrument’s Reaktor or the software for my Nord Modular. Yes, AudioTool is modular but I usually have more fun when I have control over the most basic soundmodules: oscillators, lfo’s, filters, stepsequencers, envelopes, etc.

So when I saw the ease of direct access to the soundcard’s output with Actionscript, I decided that a modular system of those elementary blocks should be my next project. That was about a month ago, and here is a very first result. I proudly present: Patchwork (you will need Flash Player 10 to run this, and you might want to turn your volume down when you start). Please note: the application is far from finished. Although you can create some interesting things already, there are a lot of modules and options missing to do the real good stuff. Consider this current version a proof of concept.

Patchwork: Sound Synthesis in Flash

What is it?
Patchwork is a Flash application for realtime modular sound synthesis. You can connect basic sound generation or modification modules to create either music, effects or utter noisy crap. Although at the moment it has an interface for creating a so-called patch, you will in the future be able to run your creations standalone, which means: in your own site or game.

Why did you make it?
It seemed like a fun challenge.

Dude, WTF! There’s only one module in the effects-category, which “does absolutely nothing”.
Yeah, I know. Patchwork has a severe lack of modules at the moment. Working on that.

Future plans?
Yes, a lot. As I mentioned before, Patchwork is far from finished. Currently, I have a nice codebase to build and expand on. Patchwork’s core works good for the time being so I won’t be doing much programming on that in the near future, apart from some interface-related stuff. For now I want to focus on adding new modules and expanding the capabilities of existing ones. Apart from that, I want to add the aforementioned standalone functionality, so you can add interactive music or effects to your own Flash applications.

Will you release the code?
Patchwork will probably be open sourced.

Can you tell something about its workings?
Sure can:

  • First of all, a disclaimer: I have never ever created a realtime modular sound system before. Suggestions, comments, criticism and thoughts on every aspect of Patchwork are much appreciated.
  • Patchwork runs by creating a Sound object, adding a listener for the SampleDataEvent.SAMPLE_DATA to it and execute the play() method. An event will be dispatched, stating that the buffer is empty, after the Main SoundOut module is asked for a new buffer. This module will ask whatever module is connected to its inputs for a new buffer, and this chained request will go all the way up the node-tree until a buffer is found.
  • This means that modules that are not connected to the Sound Output will not run. At all.
  • Every module-output has a cache. If any module-output gets a request for a buffer, it will calculate it and store a copy of the results in its cache. If another request is made in the same iteration, the module will not recalculate a new buffer but instead create a copy from its cache.
  • To achieve this caching, each request that’s started at the end-module (main sound out) passes along a requestId with it as well that increases with every call. This id is used by modules to decide whether an incoming request is new (and a new buffer should be calculated) or if the buffer for this was already calculated and stored in its cache.
  • One type of trouble that i didn’t prevent yet is feedback. Typing this, I realize that i havent tried it myself, but i’m quite sure that Patchwork will surrender and collapse if you create a loop somewhere. I’m thinking about wrapping the buffers in a class that also holds a list of nodes that the buffer went through. This way, i can detect if a buffer is entering a node that it already passed and act appropriately. Any comments on this subject are very welcome.
  • Every connection runs at the same speed: 44.100Hz. For audio this is the way to go, but for other signals this is a waste of CPU usage. An LFO that runs at 10Hz can do with much less precision (in time) without notable effect. I plan to create different buffers, so that for example the LFO writes only a fraction (1/2, 1/4, 1/8 etc) of the normal buffersize (thus using less CPU). Other modules will go through these buffers at a different speed; a half buffer will be read by reading every value twice. Again, suggestions on this are very welcome.
  • Patchwork was built using the Temple Library for AS3.