computer crashing when running larger data, memory problem?

This is a discussion on computer crashing when running larger data, memory problem? within the C Programming forums, part of the General Programming Boards category; I am using lapack/blas implementation for my matrix operations (that operates in fortran). The rest of my code is just ...

  1. #1
    Registered User
    Join Date
    Apr 2009
    Posts
    16

    computer crashing when running larger data, memory problem?

    I am using lapack/blas implementation for my matrix operations (that operates in fortran). The rest of my code is just allocating and de-allocating memory blocks for the matrices. I need to loop through my data sets many iterations to compute my final product. So far it works for 200 runs but the computer freezes for 500 runs. Would anyone know what is the problem or where I can read up about this? People I have talked to think it's a memory-type problem but I've been careful to always free up the memory I use within a function/ loop. So is it my code or because of the lapack library? Hopefully someone can point me in the right direction. Thanks

  2. #2
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    Could be that allocating and freeing blocks of data is not a symmetrical operation. Internally the memory blocks that are allocated for the user are likely increasing in their locations... until the end of the user-memory space is reached. Then some clean up is attempted so that subsequent allocation requests start at the base again. By that time additional fragmentation may have occurred.

    I've run into problems when attempting to do rapid malloc / free cycles.

    Can you allocate just once and reuse the block each time? Perhaps allocate enough memory to accommodate the largest anticipated matrix.

  3. #3
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,057
    Post the problematic code snippet here.

  4. #4
    Registered User
    Join Date
    Apr 2009
    Posts
    16
    ok i think here is the problem, for a function that i call a couple times, the variable declarations are:

    Code:
    double **ddmatrix(int nrl,int nrh,int ncl,int nch)  //allocate/reallocate for matrix
    {
    	int i;
    	double **m;
    	m=(double **) malloc((unsigned) (nrh-nrl+1)*sizeof(double*));
    	if (!m) fprintf(stderr, "allocation failure 1 in dmatrix()");
    	m -= nrl;
    	for(i=nrl;i<=nrh;i++) {
    		m[i]=(double *) malloc((unsigned) (nch-ncl+1)*sizeof(double));
    		if (!m[i]) fprintf(stderr, "allocation failure 2 in dmatrix()");
    		m[i] -= ncl;
    	}
    	return m;
    }
    void funccall()
           double **sig;
           sig=ddmatrix(0,M-1,0,numpa*numpa-1);
           .......
    }
    i allocate with the ddmatrix function. Since it's a double array I need to free all the sig[i] from 0 to M-1 as well right? I'll try that then
    Last edited by cuizy; 09-21-2009 at 01:22 PM.

  5. #5
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    m -= mrl? Are you sure? Why do you even think that lands in memory you own.

  6. #6
    Registered User
    Join Date
    Apr 2009
    Posts
    16
    i have no idea what that line does. I actually used the allocation part from somewhere else and it seemed to work fine when i tested it. Should i delete that then?

    if I free with this function it should work right?
    Code:
    void ddmfree(int nrl,int nrh,double **m)  //free allocated matrix
    {
    	int i;
    	for(i=nrl;i<=nrh;i++) {
                    free(m[i]);
                    m[i]=NULL;
    	}
    	free (m);
            m=NULL;
    }
    Last edited by cuizy; 09-21-2009 at 01:33 PM.

  7. #7
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Well, I mean, I see what it does: if you say your array goes from 8 to 10, then you allocate three rows, and then move the pointer back so that when you say "give me the eighth row" it ends up in the right place. I just don't know that you can guarantee that it works; and if you somehow lose track of your minimums and maximums it for sure won't work.

    And when you free, you should be sure to add that nrl and ncl back beforehand.

    One other thing: Are your matrices the same size for each of these 500 runs? If your matrices go from 200x200 to 500x500, that six-and-a-quarter times as big....

  8. #8
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    I concur with tabstop. Freeing will need to add the offsets back in so that the correct address is referenced.

    Also, any failure to allocate memory is not trapped properly. Yes you've generated an error but the code continues as if everything worked. Very dangerous.

    Your ddmfree function should free the column's elements by looping ncl to <= nch. But you don't pass those arguments. Whoops.

    Then m + mrl should be freed instead of plain m.

    There is no need to assign NULL but you can if you want.

  9. #9
    Registered User
    Join Date
    Apr 2009
    Posts
    16
    oooh thanks for the help
    i guess when i'm running my new code it's not crashing since my ncl and nrl are always 0 anyways.

    Some of the matrices do change sizes for the 500 iterations. like from 3x1 to 3x10. They're not very big. And the ones that stay constant are like 3x 3000. it's probably better declaring those ones just once huh, but i can't declare them as global variables because of other restrictions.

  10. #10
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    Well, try to make all the corrections first. See if allocating and freeing hundreds if times causes any other issues...

    Whenever I think I have malloc issues I write some encapsulating functions around malloc() and free() which print out the pointers passed to them... I go through that and make sure the address match, and any nested frees are performed in the reverse order as mallocs - just in case memory fragmentation can be reduced that way.

  11. #11
    Registered User
    Join Date
    Apr 2009
    Posts
    16
    yes it works fine now. Even when i run larger data like 2000 iterations.......it just takes a while. I was wondering if the memory assignments would take up much time? Or it's probably my other operations.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Question regarding Memory Leak
    By clegs in forum C++ Programming
    Replies: 29
    Last Post: 12-07-2007, 01:57 AM
  2. Bitmasking Problem
    By mike_g in forum C++ Programming
    Replies: 13
    Last Post: 11-08-2007, 12:24 AM
  3. How to Parallel reading flat file into C ?
    By anwar_pat in forum C Programming
    Replies: 11
    Last Post: 09-16-2006, 10:44 PM
  4. Program Crashing
    By Pressure in forum C Programming
    Replies: 3
    Last Post: 04-18-2005, 11:28 PM
  5. Is it necessary to write a specific memory manager ?
    By Morglum in forum Game Programming
    Replies: 18
    Last Post: 07-01-2002, 02:41 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21