strange problem with printf !!!

[astroboy2000ir]

Hi everyone, I am writing a code to do a bootstrapping test on a set of data I have to analyse for my research, here is the code:

#include <stdlib.h>
#include <time.h>
#define ACCU 1000000 //Desired accuracy; orders of 10; over 100!!!

void bootstrapping (int h, int w, double barray[h][w], int col1, int col2) {

  
  int random;
  long aa, bb, b;
  float bootresults[1000], a, cc;
  double bootarray[h][2], *r=NULL;
  FILE *bootfp;

  r=malloc(ACCU*sizeof(double));  
  bootfp = fopen("bootstrap.results", "w");

  for(aa=0;aa<ACCU;aa++) r[aa]=0.0;
  for(aa=0;aa<1000;aa++) bootresults[aa]=0.0;
       
  srand((unsigned) time(NULL));

  for(bb=0;bb<ACCU;bb++){
    for(aa=0;aa<h;aa++){ //Fill in a random selection of sets to an equal height array
      random=rand()%h+1;
      bootarray[aa][0]=barray[random][col1];
      bootarray[aa][1]=barray[random][col2];
    } //aa
    r[bb]=pearson_product_moment_coefficient(h, 2, bootarray, 0, 1);
    printf("%f\n", r[bb]);  //  <======================This line!!!!
  } //bb

  for(aa=0;aa<ACCU;aa++) {
      b=r[aa]*1000;
      bootresults[b]++;
  } //aa

  for(aa=0; aa<1000; aa++){ //print the final result to the file
	  a=(aa*0.1)/(1000*0.1);
          fprintf(bootfp, " %-5.3f  %-8.0f\n", a, bootresults[aa]);
  } // ii

  fclose(bootfp);

}

The problem is that I added the printf you see in the indicated line to see if the analysis is correct when I was writing the program, but now that I have finished, when I remove (comment out) the line, I get a

Segmentation fault (core dumped)

error! Can any one please guide me to the origin of this strange problem? prinft is just for printing out the data isn't it? How does it change the course of the program?

I also had another question, I sometimes have to repeat the bootstrapping more than 1million times (by changing ACCU), even 1billion times based on the data if I have to, but I can't get the array "r" to get that large, I have tried the malloc function, but it doesn't work, I would really appreciate any constructive ideas for solving these two problems.

[quzah]

This isn't the code you are compiling.

  for(aa=0;aa<ACCU;aa++) r[aa]=0.0;

You don't have an array called r.

      bootarray[aa][0]=barray[random][col1];
      bootarray[aa][1]=barray[random][col2];

How do we know that col1 and col2 hold values 0 and 1?

  for(aa=0;aa<ACCU;aa++) {
      b=r[aa]*1000;
      bootresults[b]++;
  } //aa

You just blew past your array in all likelihood.


Quzah.

[astroboy2000ir]

This is actually a function in my more general code, basically there is a very very large array of like 100 columns and around 5000 rows, for various properties (~100) of the subjects I am studying (~5000). So that array is sent to this function and I choose the columns I want to do my analysis on by setting col1 and col2 when I call this function. As I said, everything (except the limit to the number of times), works perfectly and correctly when that printf is there!!!!

About the "r" array:
First I had defined it classically, but then when I turned to larger values of ACCU and saw an error I decided to move to a dynamic size setting using "malloc" as you can see. Is there any problem with that?

I don't understand what you mean by "You just blew past your array in all likelihood."!!!! "b" is a long integer and as you see the "bootresults" array is different from the "r" array.

[quzah]

I don't understand what you mean by "You just blew past your array in all likelihood."!!!! "b" is a long integer and as you see the "bootresults" array is different from the "r" array.

I mean:

  for(aa=0;aa<ACCU;aa++) {
      b=r[aa]*1000;
      bootresults[b]++;
  } //aa

Is the same thing as:

  for(aa=0;aa<ACCU;aa++) {
      bootresults[ 1000 * r[aa] ]++;
  } //aa

So unless r[aa] has a value of 0.001 or so, you are going to end up accessing bootresults[ 1000 * X ], and you only get bootresults[ 0 ] throug bootresults[ 999 ].

So that array is sent to this function and I choose the columns I want to do my analysis on by setting col1 and col2 when I call this function.

That's not really what you are doing though. You are just accessing:

      bootarray[aa][0]=barray[random][col1];
      bootarray[aa][1]=barray[random][col2];

Those two spots, assuming they are either a 0 or a 1 (actually I guess it's using w there, so you should check values from 0 to w-1). You don't make any checks in this function to see if those two values are in range (and you wonder why it's crashing.)

As I said, everything (except the limit to the number of times), works perfectly and correctly when that printf is there!!!!

Luck. You don't have to have well defined behavior for undefined behavior. That's why we call it undefined. Running merrily off the ends of your arrays is undefined.


Quzah.

[astroboy2000ir]

Well, that is exactly what I want; you see, the value of r is between 0 to 1 and I only need three digits of it, You are right; the way you have written it is simpler and makes b unnecessary, I will make this change, thanks ;-) . But still, this is a minor decoration difference, my main problem is that I don't want a list of ~100,000 values of r passing my screen every time I run this program, but when I remove the noted printf the program doesn't run!!!! Do you know why this could be?

[tabstop]

Are you using standard Pearson's r? If so, what do you want to do when r turns out negative?

[astroboy2000ir]

Yeah, I am using the Pearson product-moment correlation coefficient; as here, as you can see I have calculated in another function (which is working correctly; I have checked). Actually this data set I am using to test my program is highly correlated (r~0.8) and I have never got a negative value for r yet!!! but after I get this program working correctly I intend to generalize it to include those cases too!

[quzah]

I see your allocation of r which I missed earlier. I notice however that in addition to never bothering to check to see if anything is in range, that you also don't bother checking to see if your file opens successfully, or if your memory allocation was successful. You also don't seem to free what you have allocated either.


Quzah.

[astroboy2000ir]

You are right about the file checking, but since the output file is there and has the values inside it I considered it merely a decoration for now!!!

How can I check the memory allocation?

About the col1 and col2 that you mentioned earlier, I have to say that as I said barray is something like this; barray[5000][100], with the 100 columns being different properties of my subjects. I can assure you that "col1" and "col2" are both less than 100 here! in

    for(aa=0;aa<h;aa++){ //Fill in a random selection of sets to an equal height array
      random=rand()%h+1;
      bootarray[aa][0]=barray[random][col1];
      bootarray[aa][1]=barray[random][col2];
    } //aa

I am just assigning random selections of those data and placing them in a second array which only has the two columns I need so I can send it to the pearson_product_moment_coefficient function.

[astroboy2000ir]

By the way, I had forgot to free the allocated memory of "r", thanks for reminding me. I have to admit that I am not yet a professional C programmer yet, I am an astronomer and it is only recently that I have turned to C for my data analysis, so please excuse me for such funny mistakes!!!

[quzah]

I don't know why I didn't see this earlier:

      random=rand()%h+1;
      bootarray[aa][0]=barray[random][col1];
      bootarray[aa][1]=barray[random][col2];

That's why your program crashes.

edit - when in doubt, range check:

void bootstrapping (int h, int w, double barray[h][w], int col1, int col2)
{
    if( h > 0 && w > 0 && col1 < w && col1 > -1 && col2 < w && col2 > -1 )
    {
        int random = 0;
        long aa = 0, bb = 0, b = 0; /* your variable names leave a lot to be desired */
        float bootresults[1000] = {0}, a = 0.0, cc = 0.0;
        double bootarray[h][2] = { { 0.0, 0.0 } }, *r = NULL;
        FILE *bootfp = NULL;

        r=malloc( ACCU * sizeof *r );
        if( r == NULL )
            return; /* do something, you have no memory */

        bootfp = fopen("bootstrap.results", "w");
        if( bootfp == NULL )
            return; /* do something, fopen failed */

        for(aa=0;aa<ACCU;aa++) r[aa]=0.0;
        for(aa=0;aa<1000;aa++) bootresults[aa]=0.0;

        srand((unsigned) time(NULL)); /* do this in main() */

        for(bb=0;bb<ACCU;bb++){
            for(aa=0;aa<h;aa++){ //Fill in a random selection of sets to an equal height array
                random=rand()%h;
                bootarray[aa][0]=barray[random][col1];
                bootarray[aa][1]=barray[random][col2];
            } //aa
            r[bb]=pearson_product_moment_coefficient(h, 2, bootarray, 0, 1);
            printf("%f\n", r[bb]);  //  <======================This line!!!!
        } //bb

        for(aa=0;aa<ACCU;aa++) {
            b=r[aa]*1000;
            if( b < 1000 && b > -1 )
                bootresults[b]++;
            else
            {
                /* handle your error */
            }
        } //aa

        for(aa=0; aa<1000; aa++){ //print the final result to the file
            a=(aa*0.1)/(1000*0.1);
            fprintf(bootfp, " %-5.3f  %-8.0f\n", a, bootresults[aa]);
        } // ii

        fclose(bootfp);
        free( r );
    }
}

Quzah.

[astroboy2000ir]

I removed the +1 you mentioned but the problem is still the same!!!

[quzah]

I removed the +1 you mentioned but the problem is still the same!!!

Well then it's probably your pearson_product_moment_coefficient(h, 2, bootarray, 0, 1) function that is crashing you.

Arrays run from 0 to size -1. So if you do:

#define SIZE 4
int array[ SIZE ];

You get access to array[ 0 ] through array[ 3 ]. Four total elements. You can't access array[ 4 ] because that's past the end of what you get to use.


Quzah.

[astroboy2000ir]

I changed the designated printf to:

printf("%ld %f\n", bb, r[bb]);

to see how many times it runs over "bb", it is interesting that even when I set ACCU to 1,000,000, it runs the whole way and calculates "r" from pearson_product_moment_coefficient(h, 2, bootarray, 0, 1) one million times, but then it crashes!!! It seems that with the

free(r);

command I added I fixed my second problem, thanks for the reminder again Quzah ;-).... I hope we can fix this first problem too ;-)

[quzah]

Time to learn how to use a debugger.


Quzah.