Need help developing an application

Hi all!
Here is the overview of the application I am trying to develop:
Platform: Windows XP, Visual C++ 6
Application type: MFC
Features: Read a file sector by sector ( 2048 bytes) and display it in a tree view.

Here are the problems I am facing:
1. If I try to read a file which size is bigger than 1 GB, the program uses 100% of the CPU memory and then crashes

2. When the program is reading a file, I want to display a Progress Bar in a new dialog. Using CDialog->DoModal, the dialog does pop up, but the progress bar doesnt work.

Possible problem: When reading a file, I use a loop. Maybe the program stays in that loop too long leading it to crash.
Any ideas, how I should make this work.
First I want to keep the loop for reading a file, makes the program exit the loop anytimes a window message comes, and then resume the loop.
Thanks in advance!
Afrinux

I originally posted this thread in the C++ section. I was told that there is a windows section, and I have moved it here. Thanks for understanding

Quote:

---

Originally Posted by Sebastiani

>> Read a file sector by sector ( 2048 bytes) and display it in a tree view.

display *what* in the treeview - the entire file, statistics, analytic information?

---

actually it is an mpeg file. I display the mpeg headers informations for every sector.

Quote:

---

Originally Posted by Sebastiani

>> If I try to read a file which size is bigger than 1 GB, the program uses 100% of the CPU memory and then crashes

chances are you won't be able to request that much memory using standard calls (new, malloc) - Windows has a function VirtualAlloc that may be helpful to you.

---

I am not using memory allocation. I use a buffer ( unsigned char buffer[2048] )

Code:

---

while( 1 ){
  fread( buffer, sizeof(unsigned char), 2048, fp );
  // here I take out the headers informations and display them in the tree view.

  offset += 2048;
  if( offset >= file_size )  break;
}

---

Quote:

---

Originally Posted by Sebastiani

>> When the program is reading a file, I want to display a Progress Bar in a new dialog. Using CDialog->DoModal, the dialog does pop up, but the progress bar doesnt work.

try getting it to work on it's own first before incorporating it into your program - makes debugging that much easier. ;)

---

Thank you for your suggestion. Independently, the ProgressBar does work.

Quote:

---

Originally Posted by Sebastiani

>> Maybe the program stays in that loop too long leading it to crash.

why would you think that a loop could crash a program (by itself)? a better guess might be stack overflow due to recursive or large stack-framed functions being used inside the loop.

---

I will look into it.

Quote:

---

Originally Posted by Sebastiani

by the way, we have a Windows forum here that would probably be a better place for this thread.

---

Thank you very much. I thought the C++ section were the appropriate section.
Afrinux

>> I am not using memory allocation. I use a buffer ( unsigned char buffer[2048] )

Windows set's a limit on the amount of memory available to a control. from what I remember there is a way to resize the controls internal buffer and/or have it use your own. you might look into SetWindowLong to start with:

Quote:

---

Reserve extra window memory by specifying a nonzero value in the cbWndExtra member of the WNDCLASS structure used with the RegisterClass function.

---

>> actually it is an mpeg file. I display the mpeg headers informations for every sector.

perhaps you could dynamically load the part of the file being displayed and unload it when that particular sector isn't being viewed in the control?

Quote:

---

2. When the program is reading a file, I want to display a Progress Bar in a new dialog. Using CDialog->DoModal, the dialog does pop up, but the progress bar doesnt work.

---

How are you communicating the progress information to the progress dialog? Are you reading the file read through another thread?

Quote:

---

Possible problem: When reading a file, I use a loop. Maybe the program stays in that loop too long leading it to crash.

---

Windows XP will white out the window if you stay in the loop too long without updating the GUI

Quote:

---

First I want to keep the loop for reading a file, makes the program exit the loop anytimes a window message comes, and then resume the loop.

---

You could read from your file for 300ms or so, then return from the function. Wait 50ms or so, during which you handle other events. (You have to use the SetTimer method. The times won't be exact) Then resume, after the 200ms, the file read. Another way might be to use a worker thread.

Quote:

---

Originally Posted by Sebastiani

>> I am not using memory allocation. I use a buffer ( unsigned char buffer[2048] )

Windows set's a limit on the amount of memory available to a control. from what I remember there is a way to resize the controls internal buffer and/or have it use your own. you might look into SetWindowLong to start with:

---

I have read somewhere, if an application is in a loop the memory usage is near to 100%. I used the Sleep() function in the loop to reduce the memory usage to 4%, but the application takes more time to execute one operation.
I will look into SetWindowLong.

Quote:

---

>> actually it is an mpeg file. I display the mpeg headers informations for every sector.

perhaps you could dynamically load the part of the file being displayed and unload it when that particular sector isn't being viewed in the control?

---

Sorry for my ignorance, but what is dynamically loading a file? How to dynamically load a section of a file?

Quote:

---

Originally Posted by okinrus

How are you communicating the progress information to the progress dialog? Are you reading the file read through another thread?
Windows XP will white out the window if you stay in the loop too long without updating the GUI

---

Threads never came in my mind. I made a single thread application.
I think the whole problem might be laying there.
I have never made a multithreaded windows application. I will search the net, for some tutorials regarding Threads. If someone happens to know a good site on Threads please let me know.

Quote:

---

You could read from your file for 300ms or so, then return from the function. Wait 50ms or so, during which you handle other events. (You have to use the SetTimer method. The times won't be exact) Then resume, after the 200ms, the file read. Another way might be to use a worker thread.

---

Since it is a single threaded application, using a Timer seems to be a good idea.
Will a Timer be necessary if I switch my application to a multithreaded application?

Thanks again, Sebastiani, okinrus , for your time.
Afrinux

Quote:

---

Threads never came in my mind. I made a single thread application.
I think the whole problem might be laying there.

---

Threads will make things more complicated. I thought maybe your interthread messages might be getting lost because of your modal message box.

Quote:

---

I have never made a multithreaded windows application. I will search the net, for some tutorials regarding Threads. If someone happens to know a good site on Threads please let me know.

---

For MFC specific stuff, this page is pretty good. I'm sort of reading up on this stuff myself.
http://www.flounder.com/mvp_tips.htm#hreads and Processes series

Quote:

---

Since it is a single threaded application, using a Timer seems to be a good idea.
Will a Timer be necessary if I switch my application to a multithreaded application?

---

For the multithreaded approach, you'd want to update the progress bar at certain intervals and so using GetTickCount after every sector read will tell you when. You could probably get by without using SetTime. For the singlethreaded approach, you'd want to break things up, I think. Wen your program is reading the file, your application is blocked from processing the paint update messages. This article from the link I posted goes in more indepth on worker threads. http://www.flounder.com/workerthreads.htm

Look at ReadFileEx() as it allows you to specify a callback function to handle the return from a file read. In this function you could update the progress bar, handle errors or check for msgs.

ReadFileEx() creates a thread to read the file, doing quite a lot of the work for you.


>>I thought maybe your interthread messages might be getting lost because of your modal message box.

You need a modeless dialog. Modal will not allow you to send updates of the progress bar position. This is because modal dlgs do not return control (communicate with app) to the app until the dlg has been dismissed (OKed or CANCELed).

Create a class derived from CDialog based on you progress bar dlg.
Declare a member in you main window class of this class.
When you need the progress bar check the dlg member variable for a 'safe' HWND. If no safe HWND create the dlg, if a safe hwnd ShowWindow().

>>For the multithreaded approach, you'd want to update the progress bar at certain intervals

Create a worker thread to process the file.
When you load the file, find the file size / read buffer size. Set the progress range to this value.
Each loop send an update to the progress bar.

Quote:

---

You need a modeless dialog. Modal will not allow you to send updates of the progress bar position. This is because modal dlgs do not return control (communicate with app) to the app until the dlg has been dismissed (OKed or CANCELed).

---

From what I understand, the modal dialog operates on its own message loop. But by using a PostMessage to the modal dialog and the progress info may be propagated. Maybe you could clarify? According to the msdn documentation, another way is to use hooks. Never tried, though.

Quote:

---

Look at ReadFileEx() as it allows you to specify a callback function to handle the return from a file read. In this function you could update the progress bar, handle errors or check for msgs.

ReadFileEx() creates a thread to read the file, doing quite a lot of the work for you.

---

That should work, too. He could call this function multiple times and update the progress information back to the GUI.

>>But by using a PostMessage to the modal dialog and the progress info may be propagated.

To send the modal dlg msgs you need control/processing in your app, which you won't have until the modal dlg is cancelled/OKed.

A modeless dlg is much easier than hooks / threads.

//call modal dlg
CProgDlg ProgDlg;

ProgDlg.DoModal();
//app/thread stops here until ProgDlg is closed.

Quote:

---

To send the modal dlg msgs you need control/processing in your app, which you won't have until the modal dlg is cancelled/OKed.

---

The view and the dialog must still be processing some types of messages during DoModal. Overlapped areas won't get drawn othewise. I'll have to try and see what happens sometime.

Thanks okinrus, novacain!

I was learning threads. I have managed to create a worker thread that read the file.
Because of the nature of threads( still fuzzy in my head ), I left the idea of popping up a dialog, instead put the ProgressBar in the main Dialog.
Things are working fine, but I still want to pop up a Dialog to show the progress on it.
Second problem is, because I have to use Sleep() in the file read loop, reading a 1 GB file takes more than 20 minutes.
Third problem is, whenever I put delete MyThread, my application always crashes. Any idea?
I tried also:

Code:

---

GetExitCodeThread( pStreamReadTask->m_hThread, &exit_code);
if( exit_code == STILL_ACTIVE) ){
    TerminateThread(pStreamReadTask->m_hThread,0);
}

---

but exit_code is always active. Now I have an application with a task I can't kill. :o

I will look at ReadFileEx() and see if I can use it.
Thanks again both of you for your precious time.
P.S. okinrus, that's a good link you gave me there. :cool:

Derive a class from your ProgressBar dialog (Class Wiz in MSVC v6)

ie CProgressBarDlg

In the header for the Progresbar's parent's class (the window that will call the Progressbar)

//add the ProgressBar dialogs header
#include "CProgressBarDlg.h"

//add a member variable
CProgressBarDlg m_ProgressBarDlg;

When you need to show the progressbar dialog

//This calls the progressbar dialog

Code:

---

//check to see if the Progressbar Dlg has already been created
if(m_ProgressBarDlg.GetSafeHwnd())
        m_ProgressBarDlg.ShowWindow(SW_SHOW);
else
        m_ProgressBarDlg.Create(IDD_PROGRESSBAR_DLG,this);
//make sure visible
m_ProgressBarDlg.ShowWindow(SW_SHOW);

---

Quote:

---

Originally Posted by novacain

Derive a class from your ProgressBar dialog (Class Wiz in MSVC v6)

ie CProgressBarDlg

In the header for the Progresbar's parent's class (the window that will call the Progressbar)

//add the ProgressBar dialogs header
#include "CProgressBarDlg.h"

//add a member variable
CProgressBarDlg m_ProgressBarDlg;

When you need to show the progressbar dialog

//This calls the progressbar dialog

Code:

---

//check to see if the Progressbar Dlg has already been created
if(m_ProgressBarDlg.GetSafeHwnd())
        m_ProgressBarDlg.ShowWindow(SW_SHOW);
else
        m_ProgressBarDlg.Create(IDD_PROGRESSBAR_DLG,this);
//make sure visible
m_ProgressBarDlg.ShowWindow(SW_SHOW);

---

---

Hi novacain. Thanks for the reply.
I am having a compile error here m_ProgressBarDlg.Create(IDD_PROGRESSBAR_DLG,this);
After checking the function Create, it needs 4 parameters instead of 2 as follow: ::Create( DWORD, RECT, CWnd, ID )

Quote:

---

Second problem is, because I have to use Sleep() in the file read loop, reading a 1 GB file takes more than 20 minutes.

---

I don't think sleep is needed.

Quote:

---

Third problem is, whenever I put delete MyThread, my application always crashes. Any idea?

---

You need to set m_bAutoDelete to false and make sure the thread isn't running. I do

Code:

---

    CWinThread* thread = AfxBeginThread(run, 0, THREAD_PRIORITY_NORMAL,
        0, CREATE_SUSPENDED);

    thread->m_bAutoDelete = false;

    thread->ResumeThread();

---

The reason I first create the thread suspended is that the thread could conceivably complete before m_bAutoDelete is asigned. This way the thread is suspended when the m_bAutoDelete is assigned. The Website link I gave to you before goes into more depth. When I make sure to wait for the thread to exit. I do

Code:

---

    WaitForSingleObject(readWorkerThread_->m_hThread, INFINITE);
    delete readWorkerThread_;
    readWorkerThread_ = 0;

---

I know when this code is run the thread is about to exit because this code is executed when the view receives UWM_FINISHED_READ(which is posted to the view when the thread is about to exit). Before this code is executed, the static stop variable is set inside the thread so that the thread proc will eventually exit.

The thread callback itself might look something like

Code:

---

UINT ReadWorker::run(LPVOID paramPtr)
{
    int lineno = 0;
   
    ReadWorker::waitForInitializeProgressDialog();

    std::ifstream in(filename_.c_str());

    while(!stop_ && !in.eof())
    {
        std::string line;
        std::getline(in, line);

        lineno++;

        if (in)
        {
              ProgressInfo* info = new ProgressInfo(lineno, line);

              WPARAM wp = reinterpret_cast<WPARAM>(info);
              PostMessage(view_, UWM_PROGRESS_UPDATE, wp, 0);
              Sleep(1000);
        }
    }

    bool isCancelation = stop_;
    ReadFinishInfo* finfo = new ReadFinishInfo(isCancelation);   
   
    WPARAM wpInfo = reinterpret_cast<WPARAM>(finfo);

    PostMessage(view_, UWM_FINISHED_READ, wpInfo, 0);
   
    return 0;
}

---

The stop_ variable is declared volatile because it is changed and read from two different threads. The windows that receive the messages are responsible for deleting their parameters.

Hi onkirus! Thanks for your tips. Now the thread can be deleted without the application crashing.
Here is the code:
Starting the Thread:

Code:

---

        pStreamReadTask = new CDVDStreamReadTask();
        pStreamReadTask->CreateThread( CREATE_SUSPENDED, 0 );
        pStreamReadTask->m_bAutoDelete = FALSE;
        pStreamReadTask->ResumeThread();

---

Killing the Thread:

Code:

---

        WaitForSingleObject( pStreamReadTask->m_hThread, 1000 );
        delete pStreamReadTask;
        pStreamReadTask = NULL;

---

There is one problem, I have here. If I replace 1000 by INFINITE in WaitForSingleObject() like this --> WaitForSingleObject( pStreamReadTask->m_hThread, INFINITE ); the application stops there.
Any idea what might cause the problem.

Quote:

---

I don't think sleep is needed.

---

When I take out sleep(), the cpu usage reaches 100% and can't cancel the process.

Thanks for your time.
Afrinux

>>I am having a compile error here m_ProgressBarDlg.Create(IDD_PROGRESSBAR_DLG,this);

Looks like you are just trying to create a progressbar control.

In the resource editor create a dialog with your progressbar on it. Derive a class from the dialog (CDialog).

Quote:

---

There is one problem, I have here. If I replace 1000 by INFINITE in WaitForSingleObject() like this --> WaitForSingleObject( pStreamReadTask->m_hThread, INFINITE ); the application stops there.

---

The wait is there to prevent the application thread exiting while the worker thread still exists. If the wait times out, maybe your worker thread never exits from its callback function. You need someway to signal the worker thread to exit. For instance, I have

Code:

---

volatile bool ReadWorker::stop_ = true;    // static variable in ReadWorker

// Creates a thread using the run method as a callback.
// Runs in the callee's thread. 
CWinThread* ReadWorker::startThread()
{
    stop_ = false;
    CWinThread* thread = AfxBeginThread(run, 0, THREAD_PRIORITY_NORMAL,
        0, CREATE_SUSPENDED);

    thread->m_bAutoDelete = false;

    thread->ResumeThread();

    return thread;
}


// Causes the thread to eventually exit
void ReadWorker::stop()
{
    stop_ = true;
}

---

And then in the thread's callback, the thread looks every so often at the stop_ variable. The thread's callback will eventually read changed value of stop_ and therefore exit its execution loop. When this happens, thread exit event is signaled.

(Note: Microsoft doesn't recommend WaitForSignleObject INFINITE here: "A thread that uses a wait function with no time-out interval may cause the system to become deadlocked. Two examples of code that indirectly creates windows are DDE and COM CoInitialize. Therefore, if you have a thread that creates windows, use MsgWaitForMultipleObjects or MsgWaitForMultipleObjectsEx, rather than WaitForSingleObject." But I think the problem is more likely that the worker thread never exits its callback. I suspect the problems with WaitForSignelObject creating deadlocks either only occur at specific periods of time or occurs with a different processor arrangement. I haven't been able to get my test program to deadlock.)

Quote:

---

When I take out sleep(), the cpu usage reaches 100% and can't cancel the process.

---

I don't think that should occur. The worker thread is mainly doing fileio and won't be doing any extensive computation.

The code itself is kind of ugly with no error checking or exception handling. But if you want, PM me your email address and I'll send an email to you with the test program attached. The test program has a modal progress dialog, but uses a CEdit instead of a CProgressCtrl, and the files themselves are text files so it reads line by line and updates the dialog and view each time a line is read.

Quote:

---

Originally Posted by okinrus

The wait is there to prevent the application thread exiting while the worker thread still exists. If the wait times out, maybe your worker thread never exits from its callback function. You need someway to signal the worker thread to exit.

---

I made a silly mistake. I didnt try to finish to the thread. With the following code, it works. Sorry for wasting your time.

Code:

---

        TerminateThread( pStreamReadTask->m_hThread, NULL );
        WaitForSingleObject( pStreamReadTask->m_hThread, INFINITE );
        delete pStreamReadTask;
        pStreamReadTask = NULL;

---

Quote:

---

I don't think that should occur. The worker thread is mainly doing fileio and won't be doing any extensive computation.

The code itself is kind of ugly with no error checking or exception handling. But if you want, PM me your email address and I'll send an email to you with the test program attached. The test program has a modal progress dialog, but uses a CEdit instead of a CProgressCtrl, and the files themselves are text files so it reads line by line and updates the dialog and view each time a line is read.

---

Then, I am probably doing something wrong. Without the sleep() function, the user thread doesnt seem to be working.
I will p.m you to give you my email addy. I really appreciate what you are doing for me.

Quote:

---

Originally Posted by novacain

>>I am having a compile error here m_ProgressBarDlg.Create(IDD_PROGRESSBAR_DLG,this);

Looks like you are just trying to create a progressbar control.

In the resource editor create a dialog with your progressbar on it. Derive a class from the dialog (CDialog).

---

Thank you. I will try to do it. I was stuck there.

Quote:

---

Originally Posted by novacain

>>I am having a compile error here m_ProgressBarDlg.Create(IDD_PROGRESSBAR_DLG,this);

Looks like you are just trying to create a progressbar control.

In the resource editor create a dialog with your progressbar on it. Derive a class from the dialog (CDialog).

---

I have tried to follow your instructions, but I still have the same problem:

1. I created a dialog( enum { IDD = IDD_PROGRESS_BAR }; )
2. I put a progress bar on it
3. I created a class like this:

Code:

---

class CProgressBar : public CDialog
{
public:
...
        enum { IDD = IDD_PROGRESS_BAR };
        CProgressCtrl        m_ProgressBar;
.....
private:
};

---

4. I created a function to show the Dialog

Code:

---

BOOL CProgressBar::showPgrBarDlg( void )
{
        //check to see if the Progressbar Dlg has already been created
        if( m_ProgressBar.GetSafeHwnd())
                m_ProgressBar.ShowWindow(SW_SHOW);
        else
                m_ProgressBar.Create( IDD_PROGRESS_BAR, this ); 
        //make sure visible
        m_ProgressBar.ShowWindow(SW_SHOW);
}

---

Unfortunately, the same compile error keeps coming out.
Any idea what may be the causes?
Thanks in advance.
Afrinux

Ahh...
In general, you want to create a (member) dlg, using your derived class, as a child of another dialog/window.

You need to add a member variable of your class 'CProgressBar' to the window that wants to show/use the progressbar dlg (not in the ProgressBar dlg class itself).
ie add a member of CProgressBar to the main window.

Then call my 'show' code in the window that wants to show/use the progressbar dlg (not in the CProgressBar class).

Thats why you are getting the error, it is trying to create a CProgressBarCtrl (which has 4 params)

>>CProgressBar
not the best name as it might confilct with a named class.

Hi novacain! Thanks for your help. Now it works.

Code:

---

BOOL CDVDVideoSolutionDlg::showPrgBarDlg( void )
{
        clPrgBar = new CProgressBar();
        if( clPrgBar->GetSafeHwnd())
                clPrgBar->ShowWindow(SW_SHOW);
        else
                clPrgBar->Create( IDD_PROGRESS_BAR, this ); 

#if 1// center the dialog box
        CRect rectMain;
        GetClientRect(&rectMain);
        int xMain = rectMain.Width();
        int yMain = rectMain.Height();
    // Get dimensions of the dialog
        CRect rectView;
        clPrgBar->GetWindowRect(&rectView);
        int xView = rectView.Width();
        int yView = rectView.Height();
        // Calculate the (x,y) coordinate for Progress bar dialog.
        int x = (xMain - xView) / 2;
        int y = (yMain - yView) / 2;

        // Move the dialog.
        clPrgBar->MoveWindow(x, y, xView, yView, TRUE);
#endif

        //make sure visible
        clPrgBar->ShowWindow(SW_SHOW);
       
        return TRUE;
}

---

Now, I have only 2 problems:
1. Center the Progress bar dialog; the code above was supposed to do it, somehow the dialog is not centered.

2. Get the sleep() function off the code

Quote:

---

>>CProgressBar
not the best name as it might confilct with a named class.

---

Thank you for the warning, I will take it into consideration from now on.
Afrinux

Try adding the style, DS_CENTER to the progressbar dialog.

Quote:

---

Originally Posted by novacain

Try adding the style, DS_CENTER to the progressbar dialog.

---

I got it working with this:
clPrgBar->CenterWindow( this );
Thanks. By the way, are a Man Utd fan?

>>By the way, are a Man Utd fan?

No but got to admire a man who knows what is important in this life.

Quote:

---

Originally Posted by novacain

>>By the way, are a Man Utd fan?

No but got to admire a man who knows what is important in this life.

---

I second that. Thanks again for every thing.