Questions to understand malloc better

[ardavirus]

Hello again, I am back with more stupid and outrageous questions. This time I am trying to understand how malloc really works. Experimenting a bit, I came to some conclusions, that I would like you, to either confirm them, or bludgeon the newbie out of me. Let's start with something very basic:

malloc(sizeof(int));

This statement should allocate 4 bytes in the heap for our program (provided that an int is 4 bytes long) and it is equivalent with the statement:

malloc(4); // bad practice I know but bear with me

Is the above conclusion right?

I am asking the confirmation to that because the following program compile and run fine when it shouldn't

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
   char *cp;
   cp = malloc(sizeof(int)); /* Using sizeof(int) is deliberate, although
                                         bad practice. */
   if (cp == NULL) {
      printf("Error with malloc!");
      exit(EXIT_FAILURE);
}
   strcpy(cp, "abcdef"); /* Only abcd can fit into the allocated block
                                    e, f, and \0 are out. */
      printf("string: %s", cp);

   free(cp);
   cp = NULL;

   return 0;
}

Output is string: abcdef. I am sure that I have 98% right (and not caring about the rest 3%) when I am saying that the above code, results in undefined behavior, since I write outside the block allocated by malloc. The only problem is that for every string I tried, I always got the correct output, and the program never crashed or produced inconsistencies. Can you please confirm me that the code above is undefined behavior and I am just being "lucky" ?

If I hadn't free the allocated memory, after the program was finished what would have happened to the block ?

Another set of questions after the following code

#include <stdio.h>
#include <stdlib.h>

int main(void) {
   struct test {
      int a;
      int b;
   };
   struct test *sp;

   sp = malloc(sizeof(*sp));
   sp -> a = 10;
   sp -> b = 20;

   printf("a = %d, b = %d", (*sp).a, (*sp).b);

   free(sp);
   sp = NULL;

   return 0;
}

Is malloc(sizeof(*sp)) the same as malloc(sizeof(struct test)) ?
If yes which one is best practice ?

Is it possible to have some pointer, of type int, pointing at the block we allocated with malloc above?
I did try to make it happen but I wasn't able. Can you please give me a hint, to produce the correct code, in case such practice is possible?

I believe that when we wrote sp = malloc(sizeof(*sp)); we didn't create any object of type "struct test" in the allocated block. We merely reserved 8 bytes for our program in the heap, and the fact that we can read and write in that block, as if it were a structure of type test, is due to the properties of the pointer that malloc returned.
If (in theory) could have a second pointer of type int pointing at the allocated block, we could manipulate the data in there. See my theoretic code below

 .
 .
struct test {
   int a;
   int b;
};
struct test *sp;
int *ip;

sp = malloc(sizeof(struct test));
ip = I don't know how but we make this pointer point at the same block;

*ip = 10;
*(ip + 1) = 20;
printf("a = %d, b = %d", sp -> a, sp -> b);
 .
 .

Is the above (theory) correct at some (any) level ?

Is it possible to directly acquire the size of the allocated memory in the heap, let's say through some hidden byte in the block that contains such info? Or the bytes of the block are exactly the number inside malloc(number) ?

If we move the pointer in the middle or the last byte of the allocated memory can we still free it with free(pointer_moved) ?

If we assign a secondary pointer at the allocated memory can we free the block through that pointer free(secondary_pointer) ?

I thank you very much for your time to read through this.

[AndrewHunter]

Can you please confirm me that the code above is undefined behavior and I am just being "lucky" ?

Ok. Confirmed.

If I hadn't free the allocated memory, after the program was finished what would have happened to the block ?

That's called a memory leak. If your program remained running and you kept doing stuff like this, you would run out of space on your heap. After the program finishes though, it doesn't matter. The OS reclaims the entire memory block used by the program.

EDIT: I am sure someone will finish up those questions.

[quzah]

Is the above conclusion right?

Yes.

I am asking the confirmation to that because the following program compile and run fine when it shouldn't[/code]Why shouldn't it? Provide your reasoning.

   strcpy(cp, "abcdef"); /* Only abcd can fit into the allocated block
                                    e, f, and \0 are out. */
      printf("string: %s", cp);

There is nothing guaranteeing the behavior of writing to memory you shouldn't be. There's nothing that says the program shouldn't compile (dynamic allocation isn't compile time, it doesn't check to make sure you have your sizes right).

Can you please confirm me that the code above is undefined behavior and I am just being "lucky" ?

It is, and you are.

If I hadn't free the allocated memory, after the program was finished what would have happened to the block ?

The OS will probably free it. It doesn't have to, but it probably will.

Is malloc(sizeof(*sp)) the same as malloc(sizeof(struct test)) ?
If yes which one is best practice ?

Yes, it's the same. The former is generally better, requiring less change if you decide that you wanted an int instead of a struct later on. All you do is change the type of sp and the malloc line is fine. Other than that, it doesn't matter.

Is it possible to have some pointer, of type int, pointing at the block we allocated with malloc above?
I did try to make it happen but I wasn't able. Can you please give me a hint, to produce the correct code, in case such practice is possible?

I believe that when we wrote sp = malloc(sizeof(*sp)); we didn't create any object of type "struct test" in the allocated block. We merely reserved 8 bytes for our program in the heap, and the fact that we can read and write in that block, as if it were a structure of type test, is due to the properties of the pointer that malloc returned.

malloc returns a void pointer, which can be assigned to anything you want.

If (in theory) could have a second pointer of type int pointing at the allocated block, we could manipulate the data in there. See my theoretic code below

 .
 .
struct test {
   int a;
   int b;
};
struct test *sp;
int *ip;

sp = malloc(sizeof(struct test));
ip = I don't know how but we make this pointer point at the same block;

*ip = 10;
*(ip + 1) = 20;
printf("a = %d, b = %d", sp -> a, sp -> b);
 .
 .

Is the above (theory) correct at some (any) level ?

Sure. Just like a union allows you to have different types of data occupying the same memory space (one at a time). Just like typecasting allows you to say one thing is something else.

Is it possible to directly acquire the size of the allocated memory in the heap, let's say through some hidden byte in the block that contains such info? Or the bytes of the block are exactly the number inside malloc(number) ?

No. You cannot find the size of a block of memory allocated. You have to keep track of that yourself. Or rather, if there were any such information, it would not be standard C, but rather a compiler extension or library which managed it for you.

If we move the pointer in the middle or the last byte of the allocated memory can we still free it with free(pointer_moved) ?

You shouldn't.

If we assign a secondary pointer at the allocated memory can we free the block through that pointer free(secondary_pointer) ?

A pointer just stores a value. The value represents a memory address, and free takes the value (a memory address) and frees it, so yes. But you will want to make sure you are treating the pointers (all of them) as invalid after you free them.


Quzah.

[anduril462]

Can you please confirm me that the code above is undefined behavior and I am just being "lucky" ?

Confirmed. Nothing stops you from writing off the end of the block of memory you allocated, and it is indeed undefined behavior. If you want to see it crash, try copying beyond the page/segment boundary where you allocated memory. This would likely be thousands, if not millions, of bytes. Also, on a 64-bit machine, an int will be 64 bits, or 8 bytes instead of 4 bytes.

If I hadn't free the allocated memory, after the program was finished what would have happened to the block ?

The OS would clean up after you. But it's still good practice to free everything you use programatically.

Another set of questions after the following code

#include <stdio.h>
#include <stdlib.h>

int main(void) {
   struct test {
      int a;
      int b;
   };
   struct test *sp;

   sp = malloc(sizeof(*sp));
   sp -> a = 10;
   sp -> b = 20;

   printf("a = %d, b = %d", (*sp).a, (*sp).b);

   free(sp);
   sp = NULL;

   return 0;
}

Is malloc(sizeof(*sp)) the same as malloc(sizeof(struct test)) ?
If yes which one is best practice ?

Yes, in that example they are equivalent. The first one is better, since if you change the type of sp, you still have a valid malloc call.

Is it possible to have some pointer, of type int, pointing at the block we allocated with malloc above?
I did try to make it happen but I wasn't able. Can you please give me a hint, to produce the correct code, in case such practice is possible?

You mean something like:

#include <stdlib.h>
#define MAX
int main(void)
{
    int *arr = malloc(10 * sizeof(*arr));  // allocate an array of 10 ints
    // do something with arr
    free(arr);
    return 0;
}

I believe that when we wrote sp = malloc(sizeof(*sp)); we didn't create any object of type "struct test" in the allocated block. We merely reserved 8 bytes for our program in the heap, and the fact that we can read and write in that block, as if it were a structure of type test, is due to the properties of the pointer that malloc returned.
If (in theory) could have a second pointer of type int pointing at the allocated block, we could manipulate the data in there. See my theoretic code below

 .
 .
struct test {
   int a;
   int b;
};
struct test *sp;
int *ip;

sp = malloc(sizeof(struct test));
ip = I don't know how but we make this pointer point at the same block;

*ip = 10;
*(ip + 1) = 20;
printf("a = %d, b = %d", sp -> a, sp -> b);
 .
 .

Is the above (theory) correct at some (any) level ?

Yep, you could totally trash the data that sp points to by assigning another pointer to the same memory and writing there. To do the assignment, you would just do ip = (int *) sp. You would need the cast to keep the compiler from complaining, though most of the time, the compiler knows better, so casting to "shut it up" is a bad idea.

Is it possible to directly acquire the size of the allocated memory in the heap, let's say through some hidden byte in the block that contains such info? Or the bytes of the block are exactly the number inside malloc(number) ?

Nope, you have to track that yourself. Certain implementations of malloc may have such a feature, but I'm not aware of any, and besides, it would be non-standard and highly unportable.

If we move the pointer in the middle or the last byte of the allocated memory can we still free it with free(pointer_moved) ?

Nope, free needs the pointer given out by malloc. It's that address that the allocation system uses to identify a block and free it.

If we assign a secondary pointer at the allocated memory can we free the block through that pointer free(secondary_pointer) ?

As long as it points to the address given out by malloc and not some other address in the middle of the block, yes. All you are doing is passing an address into free. The type of data that address supposedly points to is irrelevant. The allocation system doesn't care about type

I thank you very much for your time to read through this.

You're welcome. Hope I answered it all clearly enough.

EDIT: Too slow!

[ardavirus]

@AndrewHunter
@quzah
@anduril462

Thank you so much for your time to help me with my questions. I am trying to learn by self-teaching and I treat your answers like treasure. Thank you again. I hope someday to be able to contribute back.

P.S:

ip = (int *) sp

I was trying for hours to figure out the correct syntax for that, but sometimes I just can't see the simplest way. Cheers.

[CommonTater]

Hello again, I am back with more stupid and outrageous questions. This time I am trying to understand how malloc really works.

Ok, real simple explaination...

Calling malloc() is the same as buying a bushel basket. You get a container of a given size, what you put in it is entirely up to you. Nothing stops you from overflowing it and dropping stuff. Nothing stops you from abusing it or breaking it. Nothing stops you from leaving it sit empty. You bought a basket... now it's up to you.

[quzah]

I'd liken it to a shopping cart. Take Tater's example, and add "but you have to leave it at the store when you are done".

Edit - Removed disparaging remark about hobos.


Quzah.

[ardavirus]

Calling malloc() is the same as buying a bushel basket.

This is really a fine example

Take Tater's example, and add "but you have to leave it at the store when you are done"

yes, l must not forget to free(bushel basket);

I am asking the confirmation to that because the following program compile and run fine when it shouldn't

Why shouldn't it? Provide your reasoning.

You are right I expressed that one wrong, it should compile fine, plus I am always expecting something to go wrong in undefined behavior.

[CommonTater]

plus I am always expecting something to go wrong in undefined behavior.

By definition "Undefined Behaviour" is a function that works perfectly in every test, until you are either A) Handing it in to be graded, B) Showing it to your boss or C) installing it on your customer's computer.


One of the hardest lessons in programming, as in life is:
"Just because you can does not mean you should"

[ardavirus]

By definition "Undefined Behaviour" is a function that works perfectly in every test, until you are either A) Handing it in to be graded, B) Showing it to your boss or C) installing it on your customer's computer.

With my luck worse things could happen!

[anthonyd]

Hello again, I am back with more stupid and outrageous questions. This time I am trying to understand how malloc really works. Experimenting a bit, I came to some conclusions, that I would like you, to either confirm them, or bludgeon the newbie out of me. Let's start with something very basic:

malloc(sizeof(int));

This statement should allocate 4 bytes in the heap for our program (provided that an int is 4 bytes long) and it is equivalent with the statement:

malloc(4); // bad practice I know but bear with me

Is the above conclusion right?

Yes it is, but that has nothing to do with malloc(). The C compiler recognizes sizeof() and statically (at compile time) replaces it with an integer.

I am asking the confirmation to that because the following program compile and run fine when it shouldn't

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
   char *cp;
   cp = malloc(sizeof(int)); /* Using sizeof(int) is deliberate, although
                                         bad practice. */
   if (cp == NULL) {
      printf("Error with malloc!");
      exit(EXIT_FAILURE);
}
   strcpy(cp, "abcdef"); /* Only abcd can fit into the allocated block
                                    e, f, and \0 are out. */
      printf("string: %s", cp);

   free(cp);
   cp = NULL;

   return 0;
}

Output is string: abcdef. I am sure that I have 98% right (and not caring about the rest 3%) when I am saying that the above code, results in undefined behavior, since I write outside the block allocated by malloc. The only problem is that for every string I tried, I always got the correct output, and the program never crashed or produced inconsistencies. Can you please confirm me that the code above is undefined behavior and I am just being "lucky" ?

Here is the surprise: your program works because you do _not_ write outside the block allocated by malloc()!. The people that write libc (the library that includes malloc), are smart enough to know that if they allocate more space than what you ask for (usually one, or more pages) then when you call malloc() again in your program, they can just give you the ... "leftovers" and save some time. Also, if you really want the dirty details, when you ask the OS to give you some memory it is going to give you whole pages, not a few bytes at a time. malloc() is the library call that micromanages these pages for you, so you have the illusion of allocating a few bytes at a time. Internally malloc() will call some system call (like mmap(), or brk()) to deal with the OS in whole pages.

If I hadn't free the allocated memory, after the program was finished what would have happened to the block ?

First of all, beyond your malloc()ed buffer, your program occupies a lot of space in memory. Think of the "stack" for example. When you call a recursive function, you don't stop to ask for more stack every once in a while. Also, if your recursion goes crazy, eventually you'll get a segfault. That means that somebody (the compiler) has made sure that your program will allocate "enough" memory upon start-up to satisfy a reasonable stack behavior. Why am I saying all this? To make you worry less about your little buffer causing trouble when you exit without free()ing it.
However, you should _always_ make sure that for every malloc() there is a free() that will execute in as many execution paths as the malloc() will. Leaving memory behind for the OS to clean up is a bad practice that leads to bad programs.

Another set of questions after the following code

#include <stdio.h>
#include <stdlib.h>

int main(void) {
   struct test {
      int a;
      int b;
   };
   struct test *sp;

   sp = malloc(sizeof(*sp));
   sp -> a = 10;
   sp -> b = 20;

   printf("a = %d, b = %d", (*sp).a, (*sp).b);

   free(sp);
   sp = NULL;

   return 0;
}

Is malloc(sizeof(*sp)) the same as malloc(sizeof(struct test)) ?
If yes which one is best practice ?

Yes they are the same, but I would argue for the latter malloc(sizeof(struct test)), because by looking at it you know what's going on. If your malloc() is in the middle of a large function and the declaration of "sp" is in the beginning, the code is not going to be very readable if you use the former. The argument about changing the type of sp and your code still working is weak. Sure, the sizeof() will work, but nothing else that is using sp assuming it is of type struct test will work.

Is it possible to have some pointer, of type int, pointing at the block we allocated with malloc above?
I did try to make it happen but I wasn't able. Can you please give me a hint, to produce the correct code, in case such practice is possible?

Yes it is, because malloc() does _not_ understand, or care for, C types. That's why you should _always_ cast the return value of malloc() to the type of the pointer you are working with.
To make an analogy similar to the bushels, but closer to reality, think of malloc() as your buddy with a notebook. Every time you need to remember some stuff you ask him for some "memory" and he picks a page in the notebook and gives you the page number (actually a pointer holds just a number that you can print with printf("%p",ptr) if you want to play further with this). Then you read, or write into this memory by saying I want to write 'a' in pagenumber+0bytes, 'b' in pagenumber+1byte and so on.
The type of the pointer has nothing to do with malloc(). Types are only used by the compiler and here is why:
If you have "int *ptr" and (after malloc) you say somewhere in your program ptr[0]+ptr[1] the compiler needs to generate code that accesses two values that are sizeof(int) bytes apart. If you had "double *ptr" and you wrote the same code the compiler would need to generate code that accesses values sizeof(double) bytes apart. malloc() gives you a chunk of memory and then your program can put, or get, whatever you want in it, or from it.
Look at this code and try to understand what it does and how it does it:

#include <stdio.h>
#include <stdlib.h>

int main(int c, char **v){
    int *p1, i;
    char *p2;
    int value;

    p1 = (int *)malloc(sizeof(int));
    p1[0] = 1 + 3*256 + 5*256*256 + 7*256*256*256;
    p2 = (char *)p1;
    for(i=0; i<4; i++){
        value = (int)p2[i];
        printf("%d\n",value);
    }
    free(p1);
    return 0;
}

I believe that when we wrote sp = malloc(sizeof(*sp)); we didn't create any object of type "struct test" in the allocated block. We merely reserved 8 bytes for our program in the heap,

up to here it's correct.

and the fact that we can read and write in that block, as if it were a structure of type test, is due to the properties of the pointer that malloc returned.

No, the pointer malloc() returned has no properties, it's just a number. The beginning of the allocated space in the address space of your program to be exact.

If (in theory) could have a second pointer of type int pointing at the allocated block, we could manipulate the data in there. See my theoretic code below

 .
 .
struct test {
   int a;
   int b;
};
struct test *sp;
int *ip;

sp = malloc(sizeof(struct test));
ip = I don't know how but we make this pointer point at the same block;

*ip = 10;
*(ip + 1) = 20;
printf("a = %d, b = %d", sp -> a, sp -> b);
 .
 .

Is the above (theory) correct at some (any) level ?

Yes, as per my code example above.

Is it possible to directly acquire the size of the allocated memory in the heap, let's say through some hidden byte in the block that contains such info? Or the bytes of the block are exactly the number inside malloc(number) ?

Yes you can! but you should never do it!!
malloc() stores information about the allocated buffer as well as things like the next available chunk of memory ... somewhere. Some implementations of malloc() store it in the bytes just before the pointer (so things like ptr[-1] might give it to you), some others store it ... somewhere else. The key here is that malloc() _has_ to store this information somewhere, so that free() can figure our what it's freeing. However, there is no standard behavior, so every implementation of malloc() is free (excuse the pun) to store it anywhere the implementor saw fit.

If we move the pointer in the middle or the last byte of the allocated memory can we still free it with free(pointer_moved) ?

If we assign a secondary pointer at the allocated memory can we free the block through that pointer free(secondary_pointer) ?

No and Yes, for the reason I explained above ... that is, the free has to be able to find the "meta-data" of the pointer, so it has to be the same pointer malloc() returned (but it doesn't have to be stored in the original variable you stored it).

[AndrewHunter]

That's why you should _always_ cast the return value of malloc() to the type of the pointer you are working with.

No, you shouldn't. Please read this FAQ

[quzah]

for the reason I explained above

What makes you think that he's going to read your explanation if he's ignored ours?

"But he didn't ignore yours!"

Then what makes you think you needed to explain it a fourth time to him?


Quzah.

[ardavirus]

@antonyd
Thank you for the "dirty details", they are above my level of course, but it is good to know about them, for I will know what to search, in case I might need such "details" in a far future, when (hopefully) I would have evolved as a programmer. After this post I will try to figure out your program and I will come back with an answer

No, you shouldn't. Please read this FAQ

Thank you for the disagreement and the FAQ. I will surely adapt the FAQ's way of "Casting malloc" but it surely is good to know other practices (now considered bad) and why they are used. Experienced programmers are still humans (I think they are) and for all humans is valid that, "old habits die hard".

What makes you think that he's going to read your explanation if he's ignored ours?
"But he didn't ignore yours!"
Then what makes you think you needed to explain it a fourth time to him?

I have never ignored any explanation given to me in this forum, I consider you guys my tutors and I always treasure your efforts to answer my questions. I am trying thanking you individually and doing my best to show you, that I have learned something more, because of your answer.
The more times an explanation is given, the better the chances, to enter this thick skull of mine.

[Adak]

A void pointer, which malloc returns, is like water - it can take any shape. You can tell your pitcher of water that you're about to pour it into a glass or bowl shaped container, as much as you like. Or any shape at all.

See how much good it does. < LOL >