Threading and graphics.

Hi all this is my first post here. I am writing a program and would like some advice about how to structure it.

Here is the deal: the program takes image data from a PCI card and then displays it in an OpenGL window. I am using the Qt libraries for my GUI and when I am in the process of taking the data, converting it to what OpenGL needs, and then displaying it, the GUI hangs. I realize I need a separate thread for this.

The problem I have with this is that I also have to have a contrast slider and I have no idea where to put that. The idea is that when someone adjusts the slider, the look-up table for the image values gets update and the image is re-rendered.

How do I have this operation happen without any conflicts from the separate thread? For example, what if I try to adjust the contrast right as the image is already being rendered by the other thread? This data will come in at around 30fps, so the likelihood of this happening is pretty high.

My solution at the moment would be to only have the thread take the data and run it through the look-up table, give the signal that it is done to the main thread, and then have the main thread just render the image onto the screen. I don't know if that would work, though. I don't want to get any sort of stuttering in the GUI because the rendering happens in the main thread.

Any help would be appreciated.

This is a basic question about thread synchronization. Threads are synchronized via locks/mutexes or some form of IPC.

I have not used them, but I presume you are using the thread functions built into Qt. In that case, you might want to start here:

Qt 4.7: Synchronizing Threads

Do not make the mistake many people make and believe that you can come up with your own impromptu method of doing this. It is NOT possible -- special assembly level instructions ARE REQUIRED. If you think you have a way around this, you are wrong. Post your theory here and someone will explain why.

I admit I have absolutely no idea what I'm doing when it comes to threads.

Hey, that's a lot better than being convinced you know what you are doing when you don't.

My advice: don't try and figure it out in the context of your main project. Come up with something that parallels what you want to do and write something as simple as possible to experiment and learn with.

Yeah, that's what I usually do. Though I've been on this project (doing other parts) now for a long time, so my boss is getting impatient. To be honest, I didn't know that threading would be this complicated of an issue and didn't anticipate investing too much time in it. But there's no way around that, it seems.

You can keep two tables around. Also keep two pointers to the tables. Access the table through the pointer instead of directly. When you want to update, update the inactive table then atomically swap the pointers

Ok to start off, you said the GUI hangs when "in the process of taking the data, converting it to what OpenGL needs, and then displaying it,". I am going to guess that (very simplified) the process looks something like this:

Code:

getData(&dataContainer);
convertData(&dataContainer, &openglData);
renderData(&openglData);

Now you first need to figure out where you are spending time. I am not sure OpenGL even supports multithreaded rendering in any way and how that works with Qt and so on (can you render from a different thread than the main window thread, or what about opengl calls from different threads? I doubt it but I don't know, only one of the issues you have if time is mostly spent rendering).

Now if the time spent is mostly in the getting data and converting data parts of the process its a completely different beast altogether. This can hopefully be done in a different thread and then using Qt signalling to signal that new data is ready (converted and all) which is then copied by the main GUI thread and then used in any further rendering. Of course you have synchronize stuff aswell but by using brewbucks 2-pointer-approach you should be able to minimize the amount of actual copying needed to be done.

But most importantly, before using concurrency to solve your issues you need to know exactly what issues you want to solve (that is, exactly what is taking up time in the code).

I've used threads with Qt and OpenGL before. The first thing you need to know is that OpenGL rendering is by far easiest to do from within the "main" thread (keep reading for some ideas here). And unfortunately, Qt GUI code can only be executed from the "main" thread as well: multithreading - How to make Qt work when main thread is busy? - Stack Overflow

So you should first try to put non-display related computations in other threads if you can. Get another thread to generate the data to be displayed, or format it correctly or whatever. Hopefully you can get away with one thread to handle both Qt events and OpenGL rendering. If not:

• You could try making the rendering code re-entrant (I've had some success with this before). For example, if you have tons of vertices to display, start by making an OpenGL display list and beginning the render. After a certain amount of time has passed, exit the rendering function to allow Qt to respond to GUI events. Next time the render function is called, start with a new display list at the point you left off.
  
  Code:

  QTime timer;
  timer.start();
  
  while(...) {
      // generate data
      if((++loops % CHECK_RATE) == 0) {
          if(timer.elapsed() > milliseconds) break;
      }
  }

• If that's difficult to do you may be able to call Qt functions to process events directly from within the rendering code. I'd be very careful of infinite recursion in that case, though.

Now, if you want to do OpenGL work from multiple threads, you may be able to. I've never actually tried this, but with Qt and OpenGL there is the concept of a current OpenGL context. You can change the context in order to e.g. have several OpenGL widgets being displayed at once. And I think you can change the context to have other threads *generate* OpenGL objects (e.g. display lists), even though the main thread still needs to display the objects.
The Hacks of Life: Creating OpenGL Objects in a Second Thread - Mac, Linux, Windows
Qt OpenGL and threads | Qt DevNet forums | Qt Developer Network

I've never tried this method, but I wish you luck if you want to! Like I said: I have had some success with the re-entrant method, but there are definitely better ways. You'll have to decide what suits your application.

One other point to keep in mind: in Qt 3 this wasn't the case, but in Qt 4, signals and slots are automatically thread-safe. If one thread sends a signal to another thread, the other thread will receive the signal when it calls its Qt event loop. (In Qt 3 signals/slots always involved a direct call. You can make Qt 4 behave this way as well but in general the new behaviour is much more useful.) This may make your life easier, communication-wise.

Originally Posted by Shakti

But most importantly, before using concurrency to solve your issues you need to know exactly what issues you want to solve (that is, exactly what is taking up time in the code).

My code looks something like this:

• Take Data
• Convert through LUT
• Display data

Taking data is by far the most time-consuming, as the PCI card I take the data from needs to sync to an external camera with some given framerate. If I tell it to get a frame, it first waits for the sync signal and then takes the data. If I fire at 7.5fps, it will have to wait for the next pulse. That would be best suited for a different thread I believe.

The weird thing though is if I tell it to do something like this:

Code:

for (int i = 0; i < max_frames; i++)
{
    //take data function
    //convert through LUT function
    //display through OpenGL function
}

It will only display the OpenGL frame once the loop is finished. This has to be some issue with Qt, because running this code:

Code:

for (int i = 0; i < 10; i++)
{
    Sleep(1000);
    textbrowser.append("random text");
}

It will hang until the loop is done, and then display whatever my string was 10 times all at once. So this is going to be way more complicated than I originally thought.

Must GUI frameworks or libraries use some form of buffer or cache. So, if you do several things to a window, the effects will not be visible until the framework/library decides to display them. This is done for performance reasons. Each framework or library usually provides some method of flushing the buffer or cache (for example, if the user needs to see a particular bit of text immediately, rather than waiting until the library happens to display it) but these things need to be used sparingly.

It is generally considered a very bad idea to have multiple threads writing to a common display unless they are synchronised. The screen (or window) is a shared resource - so if two threads display things on it, the result is unpredictable. Similarly, any buffer or cache is also a shared resource (typically in some form of memory), so the results are unpredictable if two threads are attempting to write to that buffer, while the main GUI thread is attempting to clear (and display) data from it.