calloc() strange behaviour

[Gambalix]

Hi guys. I am using a string made up of small letters, capital letters and numbers. This is a 26+26+10 = 62 length string:

char string[] =    {'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9'};

By making all combinations possible, joining 3 characters at a time, I have 62*62*62 = 238328 combinations possible, starting in 'AAA' and ending up in '999' (following the order that appears on the string above). Each combination of 3 characters is 3 bytes long, so this would occupy 3 * 238328 bytes = 714984 bytes

I want to save these generated combinations in a file, but I dont want to generate one and save and so on! What I want is to first generate them all in memory, and then save them all at a time.
For that, I allocate memory like this:

char* ptr;
ptr = (char *) calloc(714984, 1);

and then,

if (ptr == NULL) {
        printf("\nError allocating dynamic memory! Exiting...\n");
        return EXIT_FAILURE;
    }

    ptr_cpy = ptr;

    for (i = 0; i < 62; i++) {
            for (j = 0; j < 62; j++) {
                   for (k = 0; k < 62; k++) {
                              *ptr = string[i];
                              ptr++;
                             *ptr = string[j];
                              ptr++;
                             *ptr = string[k];
                              ptr++;
                   }
    }
    }

If I try to printf the last elements of ptr on screen, all goes well:

printf("Size of PTR: %d bytes\n\n", strlen(ptr));  -> (gives 714984)
    printf("String: %c\n", ptr[714978]); -> shows 9
    printf("String: %c\n", ptr[714979]); -> shows 9
    printf("String: %c\n", ptr[714980]); -> shows 8
    printf("String: %c\n", ptr[714981]); -> shows 9
    printf("String: %c\n", ptr[714982]); -> shows 9
    printf("String: %c\n", ptr[714983]); -> shows 9

AND NOW THE FUNNY PART (and my question):

If i allocate less memory, say 712662 :
ptr = (char *) calloc(712662, 1);

it works perfectly too and also shows all perfectly generated combinations!!!!!!
If I allocate one less byte , 712661, it crashes! The threshold seems to be 712662! I dont understand why, because i thought it must be 714984.
Can you help me to understand what is going on ?
Thank you.

[Click_here]

When you say "crashes", does it give you a reason why?

Are you changing your "for" loop bounds when you change the memeory allocation?

Because if you don't, the results are undefined.

[Gambalix]

It does not give me a reason why it crashes! I am using windows 7 64bit! The typical window pops up saying "program X stops responding....bla bla bla"! My "FOR" cycle bounds are always those shown above (62). I thought that if I allocate more memory than used by the cycle would not be a problem, but if i allocate less it would be. What is happening is that i am allocating less memory than the size of the pointer ptr, and it still works perfectly. That's what I dont understand!

[jwroblewski44]

I'm assuming that this is working despite writing outside the bounds of the array. You allocate 712662 bytes, but copy in 714984 bytes. This means you are writing over memory that hasn't been assigned to that array. While you are able to access the memory addresses using the indexes that are past the array bounds, you are accessing memory that is not assigned to the array and results in undefined behavior.

That is my guess.

Also, why would you want to allocate less space than needed and copy past your bounds?

[Salem]

> printf("Size of PTR: %d bytes\n\n", strlen(ptr)); -> (gives 714984)
1. ptr is the thing that got moved to the end of the allocated memory, not ptr_copy.
2. you can't do strlen() on anything here, since you didn't explicitly append a \0.
3. If you want a \0 at the end, you need to add 1 to the total amount of memory allocated.

[Gambalix]

Thank you for your responses.
Of course I dont need to allocate less space than needed! The point here is that the system ALLOWS me to do that, and still dont give me an error! (up to the threshold I mencioned before).
The pointer ptr_cpy is there on the code only to save the original position of pointer ptr, before incrementing it! It doesnt play any role on this subject discussed here.
So my question would be: is it possible to use memory past the allocated one without a warning or an error, even knowing that there is no control of the results ? To me it seems it is, but I dont know for sure since I dont fully understand what is happening!
Any thoughts about this ?

[laserlight]

is it possible to use memory past the allocated one without a warning or an error, even knowing that there is no control of the results ?

Yes, and in fact your own experimentation has demonstrated that to you. It is possible, because with undefined behaviour, pretty much any result is valid, including the result that you would expect if the code was correct. Nonetheless, the code is not correct.

[grumpy]

Thank you for your responses.
Of course I dont need to allocate less space than needed! The point here is that the system ALLOWS me to do that, and still dont give me an error! (up to the threshold I mencioned before).
The pointer ptr_cpy is there on the code only to save the original position of pointer ptr, before incrementing it! It doesnt play any role on this subject discussed here.
So my question would be: is it possible to use memory past the allocated one without a warning or an error, even knowing that there is no control of the results ? To me it seems it is, but I dont know for sure since I dont fully understand what is happening!
Any thoughts about this ?

The C standard characterise the behaviour of your code as "undefined". The practical meaning of the term "undefined" is that, if your code does something undefined, anything is allowed to happen. It can crash your program. It can crash or power down your computer. It can even produce results that mislead you into believing there is nothing wrong.


In practice, malloc() and calloc() may allocate more memory than asked for, under the assumption that a subsequent call of realloc() may request a greater size (it is more efficient to return the same pointer than it is to allocate a new buffer, copy the contents of the old buffer to it, release the old buffer). If a greater size is allocated than asked for, it can therefore appear that your program is allowed to use more memory that it has requested. The problem is, by running past the requested length, your code is in error. With such an implementation (aka compiler and associated standard library) it can sometimes appear there is nothing wrong with calloc()ing n characters, and then accessing character n+3 in the returned buffer. The problem is, that is not guaranteed. The code can (and probably will) crash when built with a different compiler/library. It can even crash if your compiler is updated (eg the vendor releases a bug fix) and your code is rebuilt.

As to why the value of 712662 is somehow magical for your compiler/library .... I don't know and I don't care. Your code is incorrect, since it is doing things it shouldn't. The code is controlled by you, so you shouldn't permit that.