program quits unexpected

The problem I'm experiencing is that my program is stopping sometimes (not always!!) without any sign of an error. I've tried many different things but nothing really helped. Could someone do some suggestions about what the problem could be?? When I know the problem I can look for solutions myself...

I will give a short explanation of my program because the code is too long to post here and like I said, I have no idea where the problem could be:
I' ve wrote a simulation program which have to do a lot of calculation and which contains a large number of "giant" loops (standard: 1000 iterations). The program has to read in a file and the output of the program is printed in a file (this has to be done 300 or 900 times in a row), so the entire programme (with the large loops) is again in a loop of 300 or 900 iterations. The problem is now that when I run the program with the input of 900 files and the number of iterations set between 500 and 1000, the program stops suddenly and always at exactly the same place. (this file can't be the problem because when I input the file individually, there's no problem. My only idea is that something (variable, filestream, something else) is limitted and receives too many information from my program at a certain point.

But if somebody has another idea, please tell me!

Unless you are producing enormous files (many gigabytes), I'd expect that filestreams are fine. I've written a 8GB file with normal file operations, no problem.

Most likely, the error that happens is that you go outside of some array variable and overwrite some other vital information that causes the crash.

--
Mats

It doesn't sound like you are always getting a crash (as you say

Quote:

---

...my program is stopping sometimes (not always!!) without any sign of an error...

---

), so therefore, I suspect the program could be exiting as designed. How many exit points have you defined? You could put in some printf() statements that tell you where the program is exiting to help with your debugging.

A prudent thing to do, if this is a commercial application, is to add trace logic that can be triggered from outside the code, and/or is included as a compile option/switch. Then, when these things happen, you can quite easily and quickly hone in on the area with the issue.

Quote:

---

Originally Posted by skelesp

my program is stopping sometimes (not always!!) without any sign of an error.

---

That sounds more like it's actually doing what is supposed to do. Remember that just because it doesn't print anything out, doesn't mean it hasn't encountered an error.

QuantumPete

"stops" as in you're back at the command prompt with no explanation?
Or "stops" with some error message like segmentation fault?

I'd say that with such large numbers that you're silently running out of memory. This could be down to either
- your dataset is just too large
- you're not freeing memory when you're done, and massive memory leaks are killing you.
- you're accessing memory AFTER you free it, and that just causes chaos to ensue.

Or perhaps you're running of file handles or file descriptors. The number of files you're allowed to have open at any one time is usually pretty small (very small on some systems).

thanks for the answers! Very nice and quick forum you got here.

In answer to your posts:
*The program stops = goes back to command prompt with no explanation.
*I've tried to see where the program stop, by creating a logfile, but the strange thing here was that there was always another function that was printed last after the error. And sometimes it wasn't printed completely (for example instead of normal_distribution, it printed "normal_di"). Therefore I thought the filestreams could be the problem.
*I also free the most used memory with memset at the end of the program.

@ Dino: how can I add this trace logic?

With all these answers in mind, I will check my program again tomorrow.

EDIT: sorry, this was double post

Most output (and certainly file output) is buffered, meaning just because you write it out, doesn't mean it will get written; the system will wait until you accumulate "enough" before writing to disk. To force the issue, you need to flush the buffer (look into fflush).

Also, memset most assuredly does not free any memory.

Hey, skelesp.
Maybe core dump is a choice. Your program stops without any sign, so I think it's crashed nor stopped. If you are using linux system, you can try these steps below:
1. $ ulimit -c unlimited
2. $ run your program and wait for stopping or crashing.
3. check if there is a core.xxxx file in the directory where your program locates.
4. if there is a core.xxxx file, then you can use gdb to check where your program crashed.
$ gdb -c core.xxxx

of course, you must compile your program with -g option.

Or just run it straight from a debugger... perhaps with some breakpoints here and there.

Quote:

---

Originally Posted by skelesp

@ Dino: how can I add this trace logic?

---

Code:

---


#define DEBUG 1
// to turn off, #define DEBUG 0 

#if DEBUG
printf("logic got here in such-and-such routine.....\n") ;
#endif

---

thanks to the Dino's trace logic, I was able to discover where the problem occured. (thanks Dino)

But the "error"/problem seems a bit strange to me.

I'll try to describe it as brief as possible:
I discovered that a certain variable (int act) suddenly changed to 0 during my program. This resulted in some for-loops with 0 loops (for(i=0;i<act;i++)). Like I told before, only some files were causing problems. I found the 5 (out of 900) files which were causing troubles. I narrowed down my search and focussed on those 5 files (but still inputted the whole 900 files, otherwise the problem does not occur).
I also told you before that my program performs 1000 times the same actions on every files. The first strange thing I discovered was that the problem didn't occur in the first iteration of the "problem-files". Mostly the problem was somewhere between the 500th en 900th iteration (but again not always).
Performing trace logic further and further in the code, I discovered the place where act is turned into 0. But in the "problem line" (CP_count[CP_nr] = 1;) the variable 'act' doesn't occur. How can act be changed during the execution of this line???.


my thoughts (perhaps totally wrong :)):
* It can't be a real error in the code, because for most files, it works perfectly. Even for the problem files, it works for many iteration of the loop. So, the input data isn't really the problem imo.
* The 5 problem files always turn act into 0, but this doesn't always causes an unexpected stop. But if the program stops unexpected, it always stops during calculations with one of the 5 problem files. So, the variable act becoming 0, seems the main cause of the program stopping, but probably it has to be in combination with something else.
* Because act is not in the problem line, I keep thinking how it is possible for act to suddenly change here... Anybody has a reasonable answer to this?

Does anybody have an idea about what is going wrong here, or how I should look further into my program to find the "real" problem. Please tell me!
I hope you will still want to help me with this strange problem... If you need extra information or want to see some code, tell me here or in pm.

Thanks!

Perhaps CP_nr is out of range and act just happens to be the victim?

Hmm perhaps, I will look into that option.
But can you explain me how it is possible that act is the victim of CP_nr being out of range? (I like to understand the error completely so I can learn about it for the future).

Again thanks for the very fast answer.

What actually happens if you have an index that is out of range is very unpredictable. What Dino is referring to is the fact that act may be overwritten when CP_nr goes out of range.

--
Mats

Let's say that you defined this:

Code:

---

int CP_count[5]  = 0 ;
int act = 0 ;

---

As you can see visually, act follows CP_count.
If the compiler happens to place act immediately following CP_count in the stack, and if you coded

Code:

---

CP_count[0] = 0 ;
CP_count[1] = 1 ;
CP_count[2] = 2 ;
CP_count[3] = 3 ;
CP_count[4] = 4 ;
CP_count[5] = 5 ;

---

then because 5 is out of range for the array, then the variable act would receive the value. This is what would get saved:

Code:

---

CP_count[0] would be 0
CP_count[1] would be 1
CP_count[2] would be 2
CP_count[3] would be 3
CP_count[4] would be 4
act would be 5

---

The same situation could also occur if both CP_count and int were malloced, as they could be adjacent in storage.

This is a very simple example, and overlays can be much more complex than this.

So you're saying that the range of CP_count is too small and

Code:

---

CP_count[CP_nr] = 1

---

then changes the next thing in stack (probably). This seems a logical explanation, but shouldn't it be more logical that act is changed in 1 instead of 0 (because that piece of code is writing "1".
(I will now try to run the program with larger CP_count...)

Quote:

---

Originally Posted by skelesp

So you're saying that the range of CP_count is too small and

Code:

---

CP_count[CP_nr] = 1

---

then changes the next thing in stack (probably). This seems a logical explanation, but shouldn't it be more logical that act is changed in 1 instead of 0 (because that piece of code is writing "1".
(I will now try to run the program with larger CP_count...)

---

What type of data is act, and what type of data is CP_count?

If act is a short or char data type, and CP_count is an int or long, then you may write zeros to the act.

--
Mats

Quote:

---

Originally Posted by skelesp

(I will now try to run the program with larger CP_count...)

---

That's a TERRIBLE way to fix a problem - you are possibly masking the error, or just moving it somewhere else. And, it might not even be that CP_count is the issue, like I said above, overlays can be much more complex than my example. People who code with this mentality end up getting fired.

You can determine if CP_count and act are adjacent in storage by printing out their addresses.

You can also add an error checking function to see if CP_nr is out of range for the maximum allowed value.

Indeed, CP_count is a double and act is an integer. So, 1 will probably become zero in this conversion.

I have increased the range of CP_count, and act does not change to 0 anymore!! Thanks for this solution. Unfortunately, my program still stops during one of the 5 files. But I'm happy, this strange error has been solved. Now I can look for other variables being overwritten by others :)

Thanks for the help guys!!

Quote:

---

Originally Posted by Dino

That's a TERRIBLE way to fix a problem - you are possibly masking the error, or just moving it somewhere else. And, it might not even be that CP_count is the issue, like I said above, overlays can be much more complex than my example. People who code with this mentality end up getting fired.

You can determine if CP_count and act are adjacent in storage by printing out their addresses.

You can also add an error checking function to see if CP_nr is out of range for the maximum allowed value.

---

Ok, you're probably very right, because I still have an error in my program :)

Can you explain some more what you just said?

How can I print the addresses of variables? And which error checking function can I add to check whether CP_nr is out of range?

For a routine, you could:

Code:

---

#define MAX_CP_nr = 100    ( or whatever it is )
..

int check_CP_nr(int value) {
  if (value > MAX_CP_nr) { printf("CP_nr out of range!! value = %d", value) ; exit(-1) ; }
  return 1 ;  // value is OK
}
...
...
...
if (check_CP_nr(CP_nr)) CP_count[CP_nr] = 1 ;  // as an example

---

For printing addresses,

Code:

---

printf("act is at address %p\n", &act) ;

---

I have implemented the error function and printed the addresses of act and CP_count. Result:
CP_nr out of range!! value = 101
act: 0058A200 CP_count: 00589EE0

So, one of the problems certainly is that CP_nr is going out of range (The max range of CP_count was in fact 100, so CP_nr couldn't be more than 100). I tried using

Code:

---

memset((void*)&CP_nr, 0, sizeof(int)*MAXACT);

---

at the end of each iteration of a file (I don't exactly know why :) but it helped me several times before). Perhaps this is again totally wrong?! Because after I did this, another variable was changed unexpectedly, causing the program to crash.

Dino, what is the best way to debug my program, knowing that CP_nr is causing some trouble somewhere... ?

There is actually 800 bytes (200 integers) between cp_count and act, so if it's an int, you'd be overwriting a lot more than just the cp_count. Is there something else in between?

--
Mats

In between what? I declare all my global variables in one headerfile (I know this is probably not the best way to do this, but it was the easiest for me when I started this program) Perhaps all the declarations between act and CP_count are in those 800 bytes??

Otherwise, how can I debug my program in the most efficient/easy way to discover where CP_count overwrite things...

1) Implement the same style of bounds checking for your other arrays.

2) If you are constructing or reading in any strings, you can add a strlen() test when finished to see if your string is longer than the character array that houses it. (Really, the same as the above test.)

C is notoriously susceptible to both of the above conditions.

If you are using MSVC, perhaps you can build in debug mode and that will tell you if you are corrupting the stack at runtime. Do you use the "const" attribute for all variables and function declarations it would apply to? This might have the affect (depending on your compiler) to put these values in read-only memory and thus, removing these values from the equation.

What is different about these 5 (or now 1) files? Much larger? Smaller? Longer than usual strings? Binary data in what is expected to be a text file? Lines endings different?

Quote:

---

Originally Posted by Dino

What is different about these 5 (or now 1) files? Much larger? Smaller? Longer than usual strings? Binary data in what is expected to be a text file? Lines endings different?

---

Well, actually there are only little differences between the 900 files. They are all written in the same format and contain only numbers.
I don't know whether this will be clear, but each file describes a (project)network with 30 activities (which all have different durations)+2 dummy activities (=32 activities in total).
File 1 describes a highly serial network (with lots of activities happening after each other), file 900 describes a highly parallel network (with lots of activities happening at the same time). The files in between are a combination of parallel and serial activities. So, the files themself are all the same size, have the same content, etc...

Tomorrow, I will try to find more array which cause some trouble. Now I know already more what too look for...

PS: 1 extra question (for now): when is it advisable to use memset? Because I'm using it to clear some variables at the end of my program (once got this advise from another forum), but I'm starting to thinking I shouldn't do this.

Quote:

---

Originally Posted by matsp

There is actually 800 bytes (200 integers) between cp_count and act, so if it's an int, you'd be overwriting a lot more than just the cp_count. Is there something else in between?

---

Is there a way to find out which variables or values are on those addresses between act and CP_count?

If they are global and in the same source-file, then they are in memory in the order they are declared.
It's harder if they come from different sources or local to functions.
- Local to functions is in the call order (last call at the lowest address), and there will be "stuffing" inbetween variables (return address, frame-pointer, saved registers).
- Globals in different files is PROBABLY in the order they are linked in - but there is no guarantee of that. You'd have to print addresses of the first variable in each source file.

--
Mats

skelesp, I think it's time you invested some of your time in learning an interactive debugger.

What do you exactly mean with "interactive debugger"?

What editor or IDE are you using to write your code?

Microsoft Visual C++ 2008 Express Edition

I'm on my mac right now, but built into MSVC++ 2008 Express (I'm fairly certain) is in interactive debugger. Perhaps someone has a link to a tutorial, or you can search it out yourself.

In a nutshell, an interactive debugger allows you to step through your code a line at a time (if you wish) or set breakpoints in your code so you can dig around and look at things in memory while the program is running.

Ok, I will have a look at it myself first. When it isn't clear I will come back here.