Sound lags in multi-thread version

[Bubba]

Tonight I multi-threaded my sound library and it works very well except when there is a ton of stuff happening in the game.

The first game to use the new library is my asteroids clone. It works great for normal circumstances up to about 800 to 1000 asteroids on screen. At this point the explosion sounds begin to lag behind the actual graphical presentation. The main loop is going as fast as possible with a Sleep() thrown in for CPU usage issues. The game uses around 6 to 9 percent CPU with over 1500 asteroids on screen. Nice but the sound lag is a bit annoying.
Now for this game the lag won't happen during normal gameplay since even 600 asteroids on screen is nearly impossible to survive even with all the ship upgrades.

But moving on to something like my space game (StarX for now) this may be a problem. Do you think I should synchronize the sound with the on-screen presentation using Windows events and so forth or should the sound engine just be cruising along as fast as possible? During normal gameplay the sound engine works beautifully. I'm not even sure StarX will be pushing as many sounds per second as asteroids does so this may be a moot point. Perhaps gameplay testing will reveal more.

Ideas? For those of you that have a multi-threaded sound system do you synchronize your sounds with the render?

My system:

1. Game calls PlaySound(soundID)
2. Sound engine adds ID to queue
3. Main thread loop in sound engine always plays topmost sound ID in queue and then pops off immediately. If nothing is in the queue the main loop essentially does nothing but Sleep() for a bit.
4. Internal play functions then use an internal vector of sounds to send the sound sample data to the API so it can play the sound.

[Bubba]

Ok guess not many have multi-threaded their sound engines. After asking a few co-workers I've decided to keep it as is.

[Codeplug]

>> ... should the sound engine just be cruising along as fast as possible?
>> If nothing is in the queue the main loop essentially does nothing but Sleep() for a bit.
You can make it as fast as possible without using Sleep() and without "spinning". With just a single consumer, you can use a single manual-reset event. The event is signaled whenever data is added to the Q, and reset whenever the last item in the Q is removed. The consumer thread can then efficiently wait for, and process data using WaitForObject functions.

gg

[Sang-drax]

Originally Posted by Bubba

1. Game calls PlaySound(soundID)
2. Sound engine adds ID to queue
3. Main thread loop in sound engine always plays topmost sound ID in queue and then pops off immediately. If nothing is in the queue the main loop essentially does nothing but Sleep() for a bit.
4. Internal play functions then use an internal vector of sounds to send the sound sample data to the API so it can play the sound.

Do you pop every sound from the queue or just the first sound and then Sleep()? Because if you do that it will result in a lag if more than one sound are added to the queue during the sleep. Just remember to pop every sound and the lag will go away.

[Bubba]

The moment a sound ID is sent off to be played it is popped. Sample duration and sample state do not affect when the sound ID is popped off the queue.

[Sang-drax]

OK, I thought the sound loop was something like this:

while (true) {
  if (!soundQ.isEmpty()) {
    soundQ.pop();
    ...  
  }
  Sleep(1);
}

which will result in a lag if many sound are started at the same time.

[abachler]

PlaySound() is the problem, you should look into using DirectSound, which will give you much better performance. For a temporary fix, try using the SND_ASYNC flag, which shoudl help some. Ultimately though you want to learn DirectSound.

[Codeplug]

>> 1. Game calls PlaySound(soundID)
Given Bubba's DirectX experience (and the custom signature for "PlaySound") - I don't think that's the problem

gg

[Bubba]

The sound is, as Codeplug states, using DirectSound via DirectMusic via a lightweight multi-threaded wrapper. The wrapper can send off as many sounds as it wants to DirectMusic using the requested number of pchannels with unlimited effects for as many performances as are needed/requested by the application. Sound latency can be dialed down as low as 2 ms with no glitching using direct driver calls provided the sound card supports extremely low latencies. Multiple audiopaths each with their own effects, volume, etc., is also supported.

Music and sound effects currently run on two separate performances so groove levels and other sound pipeline objects or DMOs do not conflict with one another and also avoids possible pchannel collisions.

The core problem is that regardless whether or not the sound is multi-threaded there does come a breaking point where the sound will lag behind simply because the update/render loop starts to take priority over the sound thread probably due to the way Windows task switches. As the number of objects in the update/render loop increase the problem begins to surface. However it does require a great number of sprites on screen and a great number of sounds being played simultaneously to see the sound lag. I would estimate that with 1600 asteroids on the screen and firing with max upgrades turned on there would be close to 50 to 100 explosions per second being sent to the sound card along with nearly 40 laser sounds. Theoretically according to the docs there is no limit to how many sound samples can be played on one audiopath however experience has shown that there is a point of diminishing gains and a point where sounds are either cut off or not played to completion. Most modern games seem to set a theoretical max and when the game reaches this, new sounds are given priority and old one simply do not play or fire off.

So far I have done nothing to fix the issue since it does seem to be a fringe case. When StarX is fully functional perhaps a large space battle may bring the issue to the forefront and become less of a fringe case. To duplicate the problem in StarX I would need around 1600 vessels all continuously firing and destroying one another. That's a huge space battle and at around 5000 triangles per vessel I really don't think my engine could handle it. So far it seems to only handle about 100 ships before it starts to chug badly.

[CornedBee]

Raise the priority of the sound thread over that of the graphics thread. The ear is much more sensitive to delays than the eye - nobody notices if your frame rate drops for half a second, but most people notice if your sound lags a tenth of a second.

Then, use a proper producer-consumer queue implementation for submitting sounds. This will obviate the need for sleeping and polling, which needlessly eats performance and, worse, means that your sound thread may be caught sleeping when there's an urgent need to play a sound.
Boost.Interprocess has a message queue.

[Sang-drax]

Yeah, Sleep() is a pretty crude way of having a queue wait for a sound.

[Bubba]

Yeah, Sleep() is a pretty crude way of having a queue wait for a sound.

At no point is the system sleeping to wait for a sound. It sleeps to give cycles back to the CPU so the OS doesn't slow down. Without the sleep the tight loop slows down the rest of the system. It's a simple Sleep(10) which MS says is the lowest sleep value you can pass into the function. The only purpose is again to give back some cycles. There isn't enough going on in my main thread loop to waste cycles so I have to purposely waste a few. Had this same problem on a Core 2 Duo at work. With tight loops that do nothing and that do not get 'signaled' to proceed (IE: hold at one of the WaitFor... functions) it's necessary to waste some cycles. I could have the sound system WaitFor and then play and combine this with the message queue and semaphores but again I'm not sure I would gain anything.

Then, use a proper producer-consumer queue implementation for submitting sounds.

There are several free libs on the internet that do this but I think a message queue with semaphores and associated data is a bit overkill for this.

[abachler]

What I mean is, you need to do just in time mixing of the sound buffers immediately before they are submitted. Run the sound thread at a higher priority if you have to, since it won't be running 99% of the time. Use your list of the sounds that are playing to generate the next buffer, and update the play list with the buffer length, then remove any sounds that have completed playing.

Looking a little more at what you have said, it seems you are not using DirectSound directly (no pun intended) but through an API call, this is what I mean when i say PlaySound is the problem. You should be mixing new buffers directly as a response to the events generated by IDirectSoundNotify(). If your sound threads message loop is written using blocking calls, then there is no need whatsoever for Sleep().

[Bubba]

The whole purpose of DirectMusic is you don't have to explicitly mix the sounds yourself. The default mixing scheme works well enough that I do not need to mess with it. I can write my own mixer but that is sort of going against the design of the API. DirectMusic (even though now deprecated) actually hides the nitty gritty of DirectSound. You can still get to DirectSound through DirectMusic and do your own mixing but I really have no need. Any mixing I might do will probably be as good as or worse than that already being done. Mixing is a cakewalk but effects and so forth are not and require significant amount of DSP knowledge to do correctly which I do not have. If I was going to mix on a low level I would not use DirectSound at all and would instead code a raw mixer based on the Windows multimedia libraries. Aside from the driver level this is about as low level as you can get and I did not think writing something like this was necessary.

The sound thread is running 99% of the time. It is constantly looking at the queue and when the size is > 0 it then sends the ID off to be played. If I use blocking calls then I would need to account for when the queue has more than one sound in it and thus fire off or set the event to unblock. To me this seems less efficient than my current design.

For now blocking calls are a definite 'not going to happen' due to the current design.

I could not attach DXSoundEmitter.h so here it is:

#pragma once

#include <queue>
#include <vector>
#include "DXAudio.h"
#include "Thread.h"

class DXSoundEmitter:public Thread
{
    public:
        typedef std::queue<unsigned int> SoundQueue;
        typedef std::vector<DXSoundSegment *> SoundVector;
        typedef SoundVector::iterator SoundVectorIter;

        DXSoundEmitter(void);
        virtual ~DXSoundEmitter(void);
        static void soundProc(void *obj);
        void create(IDirectMusicLoader8 *_Loader,IDirectMusicPerformance8 *_Performance);
        void playSound(unsigned int soundID);
        unsigned int loadSound(WCHAR *Filename);
        bool isPlaying(unsigned int ID);
        void setLooping(unsigned int _ID,bool _cond);
        void setVolume(unsigned int _ID,int iVolume);
        void stopSound(unsigned int ID);
        
        virtual void Loop();

        bool allSoundsDone();
        
    private:
        SoundQueue m_PlayList;
        SoundVector m_Sounds;

        IDirectMusicLoader8       *Loader;
        IDirectMusicPerformance8  *Performance;

        void Play(unsigned int ID,unsigned int _mode = CDX_AUDIO_SECONDARYSEG);
        void PlayWithCheck(unsigned int ID,unsigned int _mode = CDX_AUDIO_SECONDARYSEG);

        unsigned int m_IDCounter;
        CRITICAL_SECTION m_CS;
        unsigned int m_PlayingID;

};

The reason for the commented code in DXSoundEmitter::setVolume() is that volume applies to each audiopath and I have not yet designed how I'm going to separate out volumes for various sound objects. I will probably need multiple audiopaths.

[abachler]

Well, if you want low latency playback I submit for your consideration that perhaps you need to embrace higher performance interfaces. DirectMusic is fien for siple playback but it really isnt intended as a low latency interface.