Sound experiement help

Hey guys, I'm a digital media artist working on a piece and I was hoping for a little bit of help. To give you a quick background on my knowledge level, I'm new to C++ but I know Actionscript 2.0 fairly extensively from Flash and enough PHP to get by. The system I'm developing on is a Mac running OS X.4 and I've gotten pretty good at compiling little Linux apps here and there for projects I've worked out and I've compiled a couple of my own rudimentary C++ apps. So here's what I'm hoping to do:

I'd like to write a small command line app that will first take a grayscale image and read every pixel as a level of black (0-100 or 0-255) and place that in an array. Then the app would take a wav file (an existing song), place a start tone at the beginning of the song and from there on out adjust the overall volume (on a scale of 0-100 or 0-255) in 1 sec chunks to match up with the array from the image. The idea is to then write a companion app that could listen to a wav file, wait for that beginning tone, and then feed the sound levels back in as pixels to recreate an image (I realize the image size parameters would need to be predetermined). The application here is to etch the resulting sounds onto vinyl records and allow people to record them onto their computers and see an image. Tagging a beginning tone onto the song helps with the fact that when people record the song off of the vinyl they'll all start recording at arbitrary points at the beginning.

I know it's impractical but thats the fun of art! Anyway, I could probably cobble most of this together if I knew what image and sound libraries to look at and what methods to use. Any help would be greatly appreciated. Thanks for the patience with a fish out of water.

Wow... that's quite advanced. I doubt you could do this without any prior knowledge of C++.

Both graphics and sound are platform specific.* I don't know much about Mac or Linux, so I can't recommend any particular libraries. If you are planning on using Linux, there is a linux-specific forum here a cprogramming.com, and you can post your question there. (I assume the "typical" linux compiler includes some graphics and sound libraries.) If you are going to write an OS-X application, the compiler will include graphics and sound libraries.

WAV files are fairly simple. A (standard) WAV file is simply an array of individual samples that each represent the wave height at an instant in time. ...You "connect the dots" to re-construct the wave/sound. Just about any sound library for your choosen platform should allow you to read a WAV file into an array. And for example, if you cut all of the sample-values in the array in half, the volume will be cut in half (-6dB).

Compressed formats (MP3, etc.) require the use of a codec. You would need to decode the file in order to convert it into an array. Your sound library may include some codecs, but there is some extra overhead code involved in using one. (I've never actually programed with a codec, but I've done some preliminary research.)

Image files are similar. A bitmap is just a pixel array, and any graphics library will allow you to read/write/manipulate pixels in a bitmap. However, if you want to use a JPEG or other compressed format, you will need a library-tool to do that.



* There are no graphics, color, sound, mouse, or networking in ANSI/ISO Standard C++. Sentral is right that your project is going to be very difficult for a beginner. Most programmers learn ANSI/ISO C++ before moving on to anything platform-specific. If you were taking programing in college, you would probably take one or two full semesters of Standard C++ before taking a GUI/Graphics/sound class. (Some other languages are more graphics based, and you can do this stuff sooner.)

Doug,

This is exactly the information I'm looking for, I appreciate the help. With Actionscript 3.0 Adobe introduced some image processing libraries that I'm pretty confident I could use to read pixels so I'm not completely lost conceptually but I'd rather do it all in one language since Flash ends about there - no way to output sound or image files.

Any additional help would be wonderful

You could use a cross-platform library called SDL, for pretty much whatever you need. The standard library comes with bitmap loading only, but there is an extension called SDL_image. It allows for pretty much any image format. It also has support for sound, networking, keyboard input, etc..

Some more on bitmaps, they are basically 1's and 0's. 1's being the stuff you see, and 0's being transparent.

00000000
00001000
00001000
00001000
00001000
00001000
00001000
00001111

^This is an 'L' shape.

I have no clue what to do about that vinyl. You'd have to make your own drivers... I don't even know where to start with that. It would take years to get to that point.

If you plan on adjusting the volume 1 sec for each pixel a 5 min song would only be 300 pixels.

How would you separate your volume adjustments (containing your encoded image) from the constantly changing volume of the track you're encoding it onto ?

ManofSteel - you're right. I knew 1 sec wouldn't be all that big of a picture but I thought that once I wrote the program I could play around with the duration in order to see how quickly I could vary the volume and still get a picture out of it.

Salem - As long as you pick a song with very little variation (a few Metallica songs come to mind) you can master it in such a way that the waveform clips and essentially flattens out even though you're able to hear ebbs and flows. It's all in the song you pick. The experiment is less about audio fidelity and more about making people think about the transmission of media from a different perspective.