Some questions on Sound Systems & Audio Processing

[jff_f]

Hi all,

I want to make a sound engine. I'm still in the plannings and try to find way to do some things.

I'd like to be able to get information about the music like the tempo... and others... I'd like to be able to process the audio and add filters... But I'm not sure about this... I'm already familiar with FFT and convolutions and some other theories behind signal processing... This shouldn't be hard to apply to sound since I already did Image processing at school (we study Imagery and Numerical media... but my course in Audio Processing and Synthesis got cancelled :S).

My questions are :

-Because this is meant to be used with realtime rendering... Is it possible to do a lot of processing on the audio before playing it ? Is the FFT really a good way to do it in realtime, maybe there is a quicker way to do it by sacrificing some information (like when you don't care about the direction of the edge in a edge finder...) Is there anything that I could preprocess?

-And how do I do it in realtime ?... I mean is there any libraries that could accept the data in the memory or do I have to write the data to files on the disk and play it from there?

-Do you know of good tutorials to apply "effects" on audio and/or to get information on the audio? I have seen "http://users.iafrica.com/k/ku/kurient/dsp/algorithms.html" and I've seen the article that is on Gamedev about Beat detection (didn't got time right now to read it), but I would like to know if there was something more complete on audio processing. Maybe I didn't search enough on google or maybe I don't know what I'm really searching for (since english is a second language for me maybe just not searching for the right thing)....

-Do you know where I could find a feature comparison between DirectSound and OpenAL since they are the two I'm considering the most? or do you have other LIB/API to suggest?


After reading this I should say that, if you didn't already noticed, I didn't try to build anything yet... so I didn't try to process any audio... yet... Mostly because I don't know how I to play the processed data from memory (except maybe with the WaveOut). Last week I finally read through the OpenAL tutorials on this site and liked the way everything works. But after that OpenAL seemed to be some kind of wrapper around the libraries that are available on the architecture like for example DirectSound on the Windows, am I right ?


So many questions, so little time :wink:

JFF

[TheNut]

I’m not an audio expert (I just program with DirectSound), but I’ll try to throw in some tips.

DirectSound offers basic hardware accelerated sound adjustments such as velocity and Doppler, roll-off, and 3D orientation. So these are already done for you with that API. MS also provides a utility to import WAV files into memory for you. It’s not a straightforward API to setup and use. It takes time to write up your interface for the API and more so if you plan on utilizing streaming audio. It is however fast and correct when it comes to 3D positional audio compared to other sound libraries I’ve heard from.

Creative Labs has an SDK that allows you to play directly with EAX technology, which allows you to utilize hardware accelerated reverberation with Sound Blaster technology. If you own one of these cards, bonus. The only downfall is that you need to register for the SDK and it can be a tedious process.

FMOD is another one I’m aware of (though I never used) that is free for personal use. You might want to take a look at them to see what they offer.

Sound filters that are done via software such as time dilation/contraction and conversions of format can be performed in real time with a performance penalty to anything else running (it’s not that significant, but it is there). I wouldn’t recommend doing a lot of audio filtering if your end goal is to have a running game. You should also design your software filters to stream with the audio and not pre-process it.

[jff_f]

Well... in fact... I'm not really thinking about using the whole waveout thing because I don't really want to work with the 3d stuff and the problem I could face with multiple speakers systems (I mean like 4.1, 5.1, etc).

And after posting I thought that maybe creating some kind of format that I could use to wrap audio files and write preprocessed data there and a pointer to the start of the "real" sound file. Then I could write a tool that preprocess the audio and exports the data... I mean static things like the BPM, EQ settings and others depending on what I could do in a realtime situation and get it like :

CSound *snd = CSound::FromFile("someaudiofile.MyFormat");
int bmp = snd->GetBPM();

then the integer BMP would have the value... Any other way would be to have a timer in my class and when I calculate there should be a beat use the callback function provided so the application can react to it.


The fact is... I'd like the programs that use this library to be able to react to things like beats... peaks on some specific frequencies... or I don't know... all the things that could be used in the renderer and/or sound system. Like the BPM in the sound system could be used to sync with other samples or if playing music sync the beats to mix two songs and apply a transition script (to achieve something like the DJs are doing but scripted).

For the effects part of my questions... I think I'll stay with the ones the library I choose as to offer me. Does anyone know if the OpenAL library has access to all the effects the platform can offer? (I mean like... can is it able to work with the DirectX effects? or does it use somekind of software emulation?... or do I have to work this out myself?)


Thanx again ;)


JFF