Realtime sound generation in Flash. Is there anything more to prove since AudioTool? Their team made an amazing application that allows you to hook up different synthesizers, effects and drumcomputers, and wrapped it all up in a very sparkling interface. So yes, realtime sound synthesis can be done and it’s actually quite easy to get started. It takes just a few lines of code to be able to write data directly to your soundcard and when you can do that, you have access to a whole lot of fun.
Having a background in audio, I’ve used quite a few music and sound applications and the modular ones always had my special interest: Max/MSP by Cycling ’74, Native Instrument’s Reaktor or the software for my Nord Modular. Yes, AudioTool is modular but I usually have more fun when I have control over the most basic soundmodules: oscillators, lfo’s, filters, stepsequencers, envelopes, etc.
So when I saw the ease of direct access to the soundcard’s output with Actionscript, I decided that a modular system of those elementary blocks should be my next project. That was about a month ago, and here is a very first result. I proudly present: Patchwork (you will need Flash Player 10 to run this, and you might want to turn your volume down when you start). Please note: the application is far from finished. Although you can create some interesting things already, there are a lot of modules and options missing to do the real good stuff. Consider this current version a proof of concept.
What is it?
Patchwork is a Flash application for realtime modular sound synthesis. You can connect basic sound generation or modification modules to create either music, effects or utter noisy crap. Although at the moment it has an interface for creating a so-called patch, you will in the future be able to run your creations standalone, which means: in your own site or game.
Why did you make it?
It seemed like a fun challenge.
Dude, WTF! There’s only one module in the effects-category, which “does absolutely nothing”.
Yeah, I know. Patchwork has a severe lack of modules at the moment. Working on that.
Yes, a lot. As I mentioned before, Patchwork is far from finished. Currently, I have a nice codebase to build and expand on. Patchwork’s core works good for the time being so I won’t be doing much programming on that in the near future, apart from some interface-related stuff. For now I want to focus on adding new modules and expanding the capabilities of existing ones. Apart from that, I want to add the aforementioned standalone functionality, so you can add interactive music or effects to your own Flash applications.
Will you release the code?
Patchwork will probably be open sourced.
Can you tell something about its workings?
- First of all, a disclaimer: I have never ever created a realtime modular sound system before. Suggestions, comments, criticism and thoughts on every aspect of Patchwork are much appreciated.
- Patchwork runs by creating a Sound object, adding a listener for the SampleDataEvent.SAMPLE_DATA to it and execute the play() method. An event will be dispatched, stating that the buffer is empty, after the Main SoundOut module is asked for a new buffer. This module will ask whatever module is connected to its inputs for a new buffer, and this chained request will go all the way up the node-tree until a buffer is found.
- This means that modules that are not connected to the Sound Output will not run. At all.
- Every module-output has a cache. If any module-output gets a request for a buffer, it will calculate it and store a copy of the results in its cache. If another request is made in the same iteration, the module will not recalculate a new buffer but instead create a copy from its cache.
- To achieve this caching, each request that’s started at the end-module (main sound out) passes along a requestId with it as well that increases with every call. This id is used by modules to decide whether an incoming request is new (and a new buffer should be calculated) or if the buffer for this was already calculated and stored in its cache.
- One type of trouble that i didn’t prevent yet is feedback. Typing this, I realize that i havent tried it myself, but i’m quite sure that Patchwork will surrender and collapse if you create a loop somewhere. I’m thinking about wrapping the buffers in a class that also holds a list of nodes that the buffer went through. This way, i can detect if a buffer is entering a node that it already passed and act appropriately. Any comments on this subject are very welcome.
- Every connection runs at the same speed: 44.100Hz. For audio this is the way to go, but for other signals this is a waste of CPU usage. An LFO that runs at 10Hz can do with much less precision (in time) without notable effect. I plan to create different buffers, so that for example the LFO writes only a fraction (1/2, 1/4, 1/8 etc) of the normal buffersize (thus using less CPU). Other modules will go through these buffers at a different speed; a half buffer will be read by reading every value twice. Again, suggestions on this are very welcome.
- Patchwork was built using the Temple Library for AS3.