Thread: Modeless MessageBox() Internals

Hey all, this should be a fairly quick one!

I'm currently rolling my own MessageBox function (and maybe whacking it in a library for future use,) to allow for advanced text formatting and inheritance of various parent window properties. Modality has got me kind of stumped though.

So, in order to implement a modal "dialogue", we can call EnableWindow(hParent, FALSE) and force the user to interact with the messagebox window first. Then, as the messagebox routine completes, we can reactivate the parent window with EnableWindow(hParent, TRUE). Conversely, if we need to implement task modality, we could make the same calls whilst enumerating through the current thread's window handles. That part should be easy enough.

However, it is modeless MessageBox() functionality that is confusing me. Obviously, a modeless dialogue cannot block the main thread because otherwise execution would not flow between the message loop and the windows procedure (and this would surely mean that the user cannot interact with other windows on the same thread whilst the messagebox is open.) So I considered that the MessageBox function might create a new thread for itself and run it's own message loop and procedure separate from other windows to allow execution to continue in the main thread and avoid this blocking problem.

However, if this is the case, I'm at a loss as to how Win32 applications can seem to manage the return value of MessageBox() whilst seeming to jump over any return-handling code and continuing the flow of execution within/into the message loop.

I'm sorry, that's not a very good description at all. I think a code example would describe my confusion better; please consider:

Code:

#include <windows.h>
#include <tchar.h>

LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
    switch (msg)
    {
        case WM_LBUTTONDOWN:
            MessageBox(hwnd, _T("WM_LBUTTONDOWN event handled."), _T("WndProc"), MB_OK);
            break;

        case WM_CLOSE:
            DestroyWindow(hwnd);
            break;

        case WM_DESTROY:
            PostQuitMessage(0);
            break;

        default:
            return DefWindowProc(hwnd, msg, wParam, lParam);
    }

    return 0;
}

int WINAPI WinMain(HINSTANCE hInstance,
                   HINSTANCE hPrevInstance,
                   LPSTR lpCmdLine,
                   int nCmdShow)
{
    const TCHAR g_szClassName[] = _T("WindowClass");

    WNDCLASSEX wc;
    HWND hWnd;
    MSG Msg;

    wc.cbSize = sizeof(WNDCLASSEX);
    wc.style = 0;
    wc.lpfnWndProc = WndProc;
    wc.cbClsExtra = 0;
    wc.cbWndExtra = 0;
    wc.hInstance = hInstance;
    wc.hIcon = LoadIcon(NULL, IDI_APPLICATION);
    wc.hCursor = LoadCursor(NULL, IDC_ARROW);
    wc.hbrBackground = (HBRUSH) (COLOR_WINDOW);
    wc.lpszMenuName = NULL;
    wc.lpszClassName = g_szClassName;
    wc.hIconSm = LoadIcon(NULL, IDI_APPLICATION);

    if (!RegisterClassEx(&wc))
    {
        MessageBox(NULL, _T("Window Registration Failed!"), _T("Error!"), MB_ICONEXCLAMATION | MB_OK);
        return 0;
    }

    hWnd = CreateWindowEx(WS_EX_CLIENTEDGE,
                          g_szClassName,
                          _T("Window"),
                          WS_OVERLAPPEDWINDOW,
                          CW_USEDEFAULT, CW_USEDEFAULT, 500, 500,
                          NULL, NULL, hInstance, NULL);

    if (hWnd == NULL)
    {
        MessageBox(NULL, _T("Window Creation Failed!"), _T("Error!"), MB_ICONEXCLAMATION | MB_OK);
        return 0;
    }

    ShowWindow(hWnd, nCmdShow);
    UpdateWindow(hWnd);

    // Confusing modeless messagebox behaviour :(
    int mbresult = 0;
    mbresult = MessageBox(NULL, _T("First messagebox."), _T("WinMain"), MB_OK);

    if (mbresult == IDOK)
        MessageBox(NULL, _T("Second messagebox.\nExecution has continued."), _T("WinMain"), MB_OK);

    // The Message Loop
    while (GetMessage(&Msg, NULL, 0, 0) > 0)
    {
        TranslateMessage(&Msg);
        DispatchMessage(&Msg);
    }

    return (int) Msg.wParam;
}

When the above program is run, the main window is created and the first messagebox is displayed. However, if we ignore this messagebox and click in the client area of the main window, we can trigger the WM_LBUTTONDOWN messagebox. This supports the idea that the messagebox might be running in it's own thread; execution seems to jump over the if statement following the first messagebox (as we would expect since MessageBox hasn't returned yet and thus mbresult does not yet equal IDOK.)

So we know that execution has reached the message loop and is bouncing around between the loop and the windows procedure. The main window is fully interactive at this point, as we'd expect once code has reached the message loop. However, even if execution has reached the message loop and then we click 'Ok' on the first messagebox, the second messagebox is triggered; that is, point of execution within the source code seems to jump "upwards" out of the message loop and into the if statement.

Hopefully that explains my confusion a little better. And there was me thinking this would be a short post If anyone could please possibly explain what is going on here and, by extension, how such logic can be achieved in a hand-crafted messagebox function, that would be hugely appreciated!

Many thanks for your time,
Abyssion