Music Visualization

I'm interested in learning a little about music visualization. Google turned up nothing of any use on the subject as far as I could tell.

Anyway, what I'm talking about is a graphical display of audio that you can get in winamp, or windows media player, something like this: http://www.microsoft.com/windows/win...mpleinFull.gif is what I'm talking about.

My question is, where can I learn more about it? Or how to compose a program to do a simpler version of the above? books, websites, anything you've got.

On second thought, I should probably work on getting a program to play an mp3 or something first. (if you have any information on libraries and such related to this task, I'd love to hear about them.)

you have a datastream.

Run some algorithm over the data received in the last X seconds from stream that yields some sort of graph.
Display graph.
Repeat until end of stream.

That's all there is to it. Of course the actual results would depend on the algorithm used and the way you represent the results.
But there is no magic involved.

The above graph I believe is the channels, without knowing too much about channels in general (I ll get updated right away). You can also involve the volume or anything else you want.

In any case a sound is a wave and a wave has a form. So you just modify the basic wave visual to have another visualization. There are a lot of ways to do so. I recommend learning more about how sound is encoded digitally. Then how you can retrieve those informations from popular formats. Start from uncompressed sound formats. Then you can do it programmatically, after you have good knowledge, as described above. And you will know what you can do so you can be more original/creative

something like this: http://www.microsoft.com/windows/win...mpleinFull.gif is what I'm talking about.

That particular visualization looks like a Spectrum Analyzer which shows the frequency content over a given period of time. This is normally done with a Fast Fourier Transform which converts the audio data from the time domain to the frequency domain. (A "regular" waveform like you see with an audio editor, or with an oscilloscope, is a representation in the time domain.) You can find FFT libraries, so you don't have to write that part of the code yourself. (It could also be done using a bandpass filter for each frequency band, instead of FFT.)

Before you start trying to do any kind of graphic animation like this, you should know how to do some simple graphics. (As you may know, there are no graphics in standard C++, so it depends on your platform and what compiler & libraries you are using.)

If you don't know anything about digital audio, there is a good (but very basic) introduction on the Audacity website. Uncompressed digital audio is just a series (an array) of samples. It shouldn't be too hard to create a visual effect from the way the amplitude varies over time (i.e. you should be able to pick-out the beat to some extent.)

On second thought, I should probably work on getting a program to play an mp3 or something first.

If you are programming for Windows, here is an example program that uses DirectSound to play a WAV file. Windows comes with an MP3 decoder (but not an encoder), so you should be able to find information on how to play MP3 files (or other compressed formats).

Winamp offers a Visualization SDK (Software Development Kit) so you can create your own visual plug-ins.

You can probably find some good example code (for both graphics and audio) on Sourceforge, although these projects can get rather "big".

For Windows programming there is a ton on information on MSDN. (The problem with MSDN is finding what you are looking for.) Here is something I found on MSDN about playing MP3 files.


In summary, you need to know:

1. How to do simple amamation.
2. How digital audio works, and some audio basics such as frequency and amplitude.
3. How to use a CODEC to decode (and play) compressed audio. (You don't have to write your own CODEC).