memory allocation

[SCRIPT_KITTEH]

Hi folks,

I read this sentence in a book I'm reading about C, and I don't fully understand:

To declare an array called averages, that contains 200 floating-point elements, the declaration

float averages[200]

is used. This declaration causes enough space inside the computer's memory to be reserved to contain 200 floating-point numbers.

So my first question is what exactly does it mean by "computers memory"? The part of my hard drive that contains the directory where I compile the C program containing said array? my RAM?

My second question is this, for each possible element in the array, how does the computer know exactly how much space to allocate? Or does it allow for the maximum value your 64 bit or 32 bit or whatever computer can hold for each index in the array?

[anduril462]

> So my first question is what exactly does it mean by "computers memory"? The part of my hard drive that contains the directory where I compile the C program containing said array? my RAM?
The simple answer is RAM. Certainly, from the point of view of your program, those floats exist in RAM, and when your program needs to access them, they will be in RAM. Though in reality, it's a bit more complicated than that. Read up on virtual memory if you want to know more.

> My second question is this, for each possible element in the array, how does the computer know exactly how much space to allocate? Or does it allow for the maximum value your 64 bit or 32 bit or whatever computer can hold for each index in the array?
By definition, a variable of type float (single-precision floating point nubmer) occupies 32 bits, or 4 bytes¹. The double type (double-precision floating point number) is 64 bits by definition. A char, by definition is always 1 byte. The other integer types are guaranteed by the standard to be at least a certain size, but may be bigger, depending on the implementation. But since the implementation (which includes the compiler) defines the sizes, the compiler knows how much memory to reserve for each.

¹ There is a constant defined by the implementation (compiler, libraries, OS, etc) called CHAR_BIT that determines how many bits in a byte. It could theoretically be 16, making 32 bits = 2 bytes, however every system I am aware of has CHAR_BIT defined as 8.

[jim mcnamara]

Every datatype has it's own storage requirement based on system architecture.

The size of a float is probably 32 bits, an integer 32 bits. Why probably?

There are standards about how to perform floating point arithmetic operations that say an IEEE-754 compliant single precision variable (a float) is supposed to be 32 bits. So by no coincidence, floats are 32 bit.

Most computers are either32 bit or 64 bit nowadays. 32bit usually means the word size of the system is 32bits == size of integer, 64bit may mean integers are 64 bits. There is no integer size in bits written in stone. Note: And there are odd implementations, maybe in a realtime embedded system, where those values are different, and do not follow standards. That is the reason for the waffly answer.

The compiler knows the memory size requirements of variables just for your system, based on datatype. The compiler developers program it with that data. The compiler creates a compiled image that, when executed, has the storage that is needed for the float - stored usually in RAM. Why usually? There is virtual memory - memory created and managed by the OS that parks that memory on disk. For small programs, the whole program starts out in RAM. For programs that allocate and use lots of memory the OS may "decide" to keep some of the process memory on disk until it is needed later on. Maybe never. Virtual memory.

A long conditional answer for a simple question.

https://en.wikipedia.org/wiki/C_data_types

https://en.wikipedia.org/wiki/Virtual_memory

[SCRIPT_KITTEH]

Thank you guys for the replies! I had no idea about the size standards for different data types, that's pretty interesting. I guess this whole subject in general is a lot more complex than I thought!

[rcgldr]

When you declare an array, the space for that array is reserved at compile time. If the array is declared outside of any functions, it will reside in the global data portion of a program. If the array is declared within a function, it will normally reside on the stack, with the stack adjusted at function entry to compensate for the space used by the array. If the array is declared as static either outside or within a function, it will normally reside in the global data portion of a program; where the standard states that static variables have only one instance, from the start to the end of a program.

[King Mir]

All data in a C program is conceptually in "memory". Each C type, T, needs a fixed amount of memory equal to sizeof(T), which gives the size in units of sizeof(char). Every "int" has the same size in memory. Every float has the same size. Every char has a size that's always 1, which is almost always equal to 1 byte or 8 bits. 200 floats probably have need 800 bytes of memory, and indeed it you did sizeof(averages), that's probably what you'd get.

So data takes up a specific block of memory, with a starting and ending location, with the difference between them equal to the size of the type. Getting the memory location is generally done via the & operator, though this is not always allowed. From the perspective of understanding the behavior of C code, all data is in memory.



That describes what memory is defined by the C language standard. But if you're curious about how memory is actually modeled by your PC, or if you're interested in making your code execute fast, there's more to know. C memory was designed to model the behavior of RAM, "Random Access Memory" on ancient computers. But that's not what memory really is. There are several reasons for this.

• Firstly, compilers may remove objects that you don't need for your program to work, so your compiled program may not have the same data as you explicitly write out. For example, if you write a long equation that contains only constants, chances are the compiler will just compute the result of that, and use it as a single object.
• Second, In addition to RAM, data can also be stored in CPU registers. These behave just like RAM, except there are very few of them. They are also very fast to access. In modern processors, registers are a piece of data with a small number attached. Compilers may chose to store any C object in registers, instead of the part of the computer that behaves like ancient RAM. Some CPUs require objects to be placed in registers before doing any computation on them. This also means that objects may be moved between memory and RAM. Non primitive types are often only partially moved into registers, as needed.
• Third, even objects that aren't eliminated by the compiler or put into registers aren't necessarily put onto RAM. They are instead part of what's called "virtual memory". "virtual" means that it behaves like memory, referring to the RAM on computers long ago. But modern RAM isn't used like that, it can be too small. To allow programs to use more memory, operating system controlled memory "pages" are used, which are stored onto the hard dive when they don't fit into your physical RAM, aka physical memory. Memory that's allocated but not written to may also not be loaded to physical memory.
• Fourth, modern memory, even when it's on a page that's loaded in physical memory, doesn't behave like ancient RAM in another way: it's cached. Because accessing RAM is relatively slow, your CPU will the data into the CPU before using it, and try to do as much as it can with the data on the CPU. There may be up to three levels of cache on the CPU, with smaller caches providing faster access to bigger caches, and the biggest cache providing access to memory.

Note this is in contradiction with the people above saying that C memory == virtual memory or C memory == RAM. Both of those are simplifications of what's really going on in your PC.

[SCRIPT_KITTEH]

When you declare an array, the space for that array is reserved at compile time. If the array is declared outside of any functions, it will reside in the global data portion of a program.

Including main () ?

What is "stack", by the way?

All data in a C program is conceptually in "memory". Each C type, T, needs a fixed amount of memory equal to sizeof(T), which gives the size in units of sizeof(char). Every "int" has the same size in memory. Every float has the same size. Every char has a size that's always 1, which is almost always equal to 1 byte or 8 bits. 200 floats probably have need 800 bytes of memory, and indeed it you did sizeof(averages), that's probably what you'd get.

So data takes up a specific block of memory, with a starting and ending location, with the difference between them equal to the size of the type. Getting the memory location is generally done via the & operator, though this is not always allowed. From the perspective of understanding the behavior of C code, all data is in memory.



That describes what memory is defined by the C language standard. But if you're curious about how memory is actually modeled by your PC, or if you're interested in making your code execute fast, there's more to know. C memory was designed to model the behavior of RAM, "Random Access Memory" on ancient computers. But that's not what memory really is. There are several reasons for this.

• Firstly, compilers may remove objects that you don't need for your program to work, so your compiled program may not have the same data as you explicitly write out. For example, if you write a long equation that contains only constants, chances are the compiler will just compute the result of that, and use it as a single object.
• Second, In addition to RAM, data can also be stored in CPU registers. These behave just like RAM, except there are very few of them. They are also very fast to access. In modern processors, registers are a piece of data with a small number attached. Compilers may chose to store any C object in registers, instead of the part of the computer that behaves like ancient RAM. Some CPUs require objects to be placed in registers before doing any computation on them. This also means that objects may be moved between memory and RAM. Non primitive types are often only partially moved into registers, as needed.
• Third, even objects that aren't eliminated by the compiler or put into registers aren't necessarily put onto RAM. They are instead part of what's called "virtual memory". "virtual" means that it behaves like memory, referring to the RAM on computers long ago. But modern RAM isn't used like that, it can be too small. To allow programs to use more memory, operating system controlled memory "pages" are used, which are stored onto the hard dive when they don't fit into your physical RAM, aka physical memory. Memory that's allocated but not written to may also not be loaded to physical memory.
• Fourth, modern memory, even when it's on a page that's loaded in physical memory, doesn't behave like ancient RAM in another way: it's cached. Because accessing RAM is relatively slow, your CPU will the data into the CPU before using it, and try to do as much as it can with the data on the CPU. There may be up to three levels of cache on the CPU, with smaller caches providing faster access to bigger caches, and the biggest cache providing access to memory.

Note this is in contradiction with the people above saying that C memory == virtual memory or C memory == RAM. Both of those are simplifications of what's really going on in your PC.

*head explodes*

The bits I think I followed are pretty neat though. I had no idea about compilers simplifying things like that, or about computer registers.

If I may ask, what exactly *is* a compiler? I know what it does is make my C code understandable to the computer, but is the compiler itself essentially a bunch of complex code stored somewhere on my computer? What language would a compiler be written in?

[rcgldr]

When you declare an array, the space for that array is reserved at compile time ...

Including main () ?

Yes, main() is just another function, although it's the first function in your code to be called.

What is "stack"?

Wiki article:

Stack (abstract data type) - Wikipedia, the free encyclopedia

On almost all computers, a push decrements a stack pointer, then stores data using the stack pointer, and a pop gets data using the stack pointer, then increments the stack pointer. The decrements and increments correspond to the size of the data being pushed or popped.

If I may ask, what exactly *is* a compiler?

A compiler converts the source code into a linkable "object" file that contains data and machine code, with information on how to fill in addresses for the external functions called by the source code. The linker then converts one or more object files into a loadable program. Note, a "library" is a collection of pre-compiled object files.

What language would a compiler be written in?

Somewhere in the distant past, the first version of a compiler was written assembly language. Once that first version is working, then later versions of that compiler can be written in the language of the compiler, but the first version is usually kept in case some unexpected problem occurs. When producing a compiler for a new processor type, a compiler may be run on an existing processor to produce machine code for the new processor, called a "cross" compiler. Once that is working the "cross" compiler can be used to create a "native" compiler that runs on the new processor.

[anduril462]

To supplement what rcgldr said, the stack King Mir was referring to was probably the call stack.

[King Mir]

What is "stack", by the way?

Stack variables are function local, non static variables. There are three places where data can be stored for extended periods of time: the stack, global section, and heap. Stack variables behave like a Last-In-First-Out structure; the variables in the deepest function are the last to be created, and the first to be destroyed.

*head explodes*

The bits I think I followed are pretty neat though. I had no idea about compilers simplifying things like that, or about computer registers.

The most important part of that is the first two paragraphs, because you need to understand them for program correctness. But I, and I'm sure others, are happy to explain more on any of it.

If I may ask, what exactly *is* a compiler? I know what it does is make my C code understandable to the computer, but is the compiler itself essentially a bunch of complex code stored somewhere on my computer? What language would a compiler be written in?

A compiler is a program that translates one computer language into another, usually into machine code. Computers, without additional software, can only understand machine code, so all C programs are translated into machine code. In translating, the compiler will also generally try to make the translation execute as fast as it can, effectively improving your program before translating.

Most C compilers are coded either in C or C++. Of course the executable is machine code.

A compiler converts the source code into a linkable "object" file that contains data and machine code, with information on how to fill in addresses for the external functions called by the source code. The linker then converts one or more object files into a loadable program. Note, a "library" is a collection of pre-compiled object files.

There are two definitions of compiler. A compiler can refer to what can better be called the "build system", the collection of programs that convert multiple source files, and resources, into a single executable. And it can refer to the particular parts of that build system that do the main part of translation, as rcgldr refers to here. What I said about translating applies to both.

Knowing the parts of the build system, the compiler and linker, is actually very useful, because both can tell you about errors in your program, and you sometimes need to understand what those errors mean.

To supplement what rcgldr said, the stack King Mir was referring to was probably the call stack.

It was actually rcgldr that mentioned the stack. But that is a better link for what he's talking about.

[SCRIPT_KITTEH]

Stack variables are function local, non static variables.

So a stack variable are variables in C that exist within a function (as opposed to... libraries like <stdio.h>?) & non-static, so they don't change, so... like an int as opposed to a const int?

There are three places where data can be stored for extended periods of time: the stack, global section, and heap.

Okay so there's stack variables, and then there's a... place? called the stack, in addition to two other places called global section and heap. Where are these three things? My hard drive or my RAM?

Stack variables behave like a Last-In-First-Out structure; the variables in the deepest function are the last to be created, and the first to be destroyed.

So like a variable in a function, that's in a function, that's in main () would be the last to be "created" and first to be "destroyed"? What exactly do you mean by created and destroyed? Wait, is all this about "stack" talking about how a compiler looks at variables in a C program? It looks at the deepest ones first when it compiles or something?

The most important part of that is the first two paragraphs, because you need to understand them for program correctness. But I, and I'm sure others, are happy to explain more on any of it.

Thanks, I really appreciate it. Yeah, as you can see I still have a lot of questions but I didn't want to be annoying. So about those first two paragraphs...

All data in a C program is conceptually in "memory".

What memory? My RAM? My hard drive? Virtual Memory? (I tried to understand the wiki page on virtual memory... and yeeeah, lol, I don't even know what to ask to clarify that)

Each C type, T, needs a fixed amount of memory equal to sizeof(T), which gives the size in units of sizeof(char).

Before when I read this I had no idea what you meant with "T", but I'm thinking now it's just an abbreviation for memory type, and you're talking about here the same thing they said earlier in this thread about different variable types having different standard memory sizes.

Every "int" has the same size in memory. Every float has the same size. Every char has a size that's always 1, which is almost always equal to 1 byte or 8 bits.

Here's where I start to lack understanding on this topic. I'm going to assume you mean exactly what you're saying, and that is every single int variable in a C program is the exact same size, not every int variable being allocated the same size of memory. How can this be though? The computer sees numbers in binary, & 2 in binary is "10." A bit is 1 binary digit, ergo, 2 in binary is 2 bits. 4 in binary is "100", which is 3 binary digits, or 3 bits. So how is

int x = 2;
int y = 4;

the same size??

So data takes up a specific block of memory, with a starting and ending location, with the difference between them equal to the size of the type.

block of memory of what? >.<

From the perspective of understanding the behavior of C code, all data is in memory.

By memory do you mean... data stored in memory... somewhere?

edit: Where do you guys learn all this stuff? I am thinking about just sending myself back to school and getting another Bachelor's, in Computer Science. Self-study can feel overwhelming at times

[rcgldr]

Okay so there's stack variables, and then there's a... place? called the stack, in addition to two other places called global section and heap. Where are these three things? My hard drive or my RAM?

They are all in RAM. In the case of virtual memory, portions of the ram may be swapped to the hard drive and then back, but while your program is running an using a variable, that variable will be in RAM when that variable is used.

Where do you guys learn all this stuff?

In my case, it was because I learned assembly language for several types of processors. Learning assembly code for a PC would be enough to understand most of this stuff. Few programmers know about the internals for how operating systems and processors actually implement virtual memory, but for the most part all most programmers need to know is that each program (for a 32 bit or 64 bit) program appears to have a continuous address space from virtual memory.

[King Mir]

Okay so there's stack variables, and then there's a... place? called the stack, in addition to two other places called global section and heap. Where are these three things? My hard drive or my RAM?

block of memory of what? >.<

By memory do you mean... data stored in memory... somewhere?

What memory? My RAM? My hard drive? Virtual Memory? (I tried to understand the wiki page on virtual memory... and yeeeah, lol, I don't even know what to ask to clarify that)

You can think of memory as a large continuous list of bytes. Every byte has a position, or "address", representing it's location. All data in your program is somewhere on that list. Each object uses 1 or more consecutive bytes in memory. The number of bytes used depends on the type of the object.

Roughly, C memory is RAM.

So a stack variable are variables in C that exist within a function (as opposed to... libraries like <stdio.h>?) & non-static, so they don't change, so... like an int as opposed to a const int?

No.

int global; /*global variable*/

int main(){
  static int static_var; /*static function variable, acts like global*/
  int auto_var; /*stack or "automatic" variable*/
  int *heap=malloc(4);/*heap memory returned by malloc*/
  free(heap);
  return 0;
}

So like a variable in a function, that's in a function, that's in main () would be the last to be "created" and first to be "destroyed"? What exactly do you mean by created and destroyed?

My bad, those are C++ terms. In C, all that happens is that space is reserved for them, after which you can use them, and later freed, after which you're not allowed to refer to it any more.

Wait, is all this about "stack" talking about how a compiler looks at variables in a C program? It looks at the deepest ones first when it compiles or something?

No, it's how function variables behave. Perhaps it's enough for now to understand that function variables are called stack variables.

Before when I read this I had no idea what you meant with "T", but I'm thinking now it's just an abbreviation for memory type, and you're talking about here the same thing they said earlier in this thread about different variable types having different standard memory sizes.

T was just a name of an arbitrary type, so that I could later refer to the same type in the sentence. So the size of an int is sizeof(int). The size of "struct Foo" is sizeof(struct Foo).

Here's where I start to lack understanding on this topic. I'm going to assume you mean exactly what you're saying, and that is every single int variable in a C program is the exact same size, not every int variable being allocated the same size of memory. How can this be though? The computer sees numbers in binary, & 2 in binary is "10." A bit is 1 binary digit, ergo, 2 in binary is 2 bits. 4 in binary is "100", which is 3 binary digits, or 3 bits. So how is

int x = 2;
int y = 4;

the same size??

The computer does see numbers in binary, but it appends leading zeros to them. So two as the computer sees it is probably: 0000 0000 0000 0000 0000 0000 0000 0010.
Four is: 0000 0000 0000 0000 0000 0000 0000 0100.

In this example, the computer has 32 binary digits, or 4 Bytes, for every "int".

edit: Where do you guys learn all this stuff? I am thinking about just sending myself back to school and getting another Bachelor's, in Computer Science. Self-study can feel overwhelming at times

Yeah, most of it was from college. For self study, I'd recommend having at least a book or lecture series to follow. They are better organized than forum answers are going to be.

[SCRIPT_KITTEH]

-C memory == RAM
-var in function == stack var

got it!

That makes sense now about the 0's. Thanks for taking the time to explain all of this to me

[laserlight]

-var in function == stack var

Not quite; read post #5 again:

If the array is declared as static either outside or within a function, it will normally reside in the global data portion of a program; where the standard states that static variables have only one instance, from the start to the end of a program.

In C terminology, we say that the array has static storage duration.