memory management...

This is a discussion on memory management... within the Game Programming forums, part of the General Programming Boards category; Does anyone have a link to a very good memory manager tutorial?...

  1. #1
    Banned master5001's Avatar
    Join Date
    Aug 2001
    Location
    Visalia, CA, USA
    Posts
    3,685

    memory management...

    Does anyone have a link to a very good memory manager tutorial?

  2. #2
    Skunkmeister Stoned_Coder's Avatar
    Join Date
    Aug 2001
    Posts
    2,572
    memory management comes in all sorts of guises at all sorts of levels. Explain a little more about the sort of memory management you want to do.
    Free the weed!! Class B to class C is not good enough!!
    And the FAQ is here :- http://faq.cprogramming.com/cgi-bin/smartfaq.cgi

  3. #3
    Sayeh
    Guest
    Actually, memory management is very simple, from the O/S perspective. Memory Management has one and only one purpose--

    To fulfill memory requests and keep the heap clean.

    ---

    Let me explain a little bit about how memory managers actually work (something Microsoft has not yet implimented by the way, even in XP)...

    ---

    First, when an application runs, the O/S allocates a 'heap' of RAM of the size specified by the compiler when the executable was built. This is an adjustable field, but many times is defaulted to something like 384K.

    Then, it loads the application to that 0-base address (relative to the heap) and sets the application stack pointer to start running there. The application's stub resolves the jumptables, links, and static/global variable spaces, establishes a stackframe, sets up master pointers and then jumps to main().

    ---

    Now, once an application is up and running, a good memory manager will then fulfill any RAM requests (eg. 'malloc()', etc.), so long as

    a) there is enough free RAM available in the Application Heap, or
    b) there is enough free RAM available in the System Heap

    In either case, the memory should be consecutive. Free memory should related to 'largest free block'... Microsoft, however just builds an array of pointers to free blocks of any size. So for example, if you needed a block 100K in size, Microsoft might actually allocate 20 different blocks with 20 pointers, each block being 5K in size. In essense you would be working with a 'super-block'. To the application it thinks it's getting a 100K block- but in reality performance is slowed because the memory manager must traverse 20 different blocks transparently to the application. This creates bad overhead process hits because the O/S isn't written correctly.

    A properly written memory manager would always be looking to move blocks around in RAM (heap compaction), and keep wide-open spaces, wide-open.

    In any event, the block is allocated.

    RAM blocks usually have a 12-byte header preceding the address of the block you are handed. The memory manager then knows all it needs to know about each RAMblock because it can query this header and learn about the block. Usually the memory manager will have a master list of pointers for all blocks used by the application.

    ---

    Now, if you mean managing your own application memory, it means dynamic allocation, keeping track of and properly deallocating your pointers. Failing gracefully if sufficient RAM can't be allocated for a process, etc.

    You might try www.microsoft.com or a bookstore.

  4. #4
    Linguistic Engineer... doubleanti's Avatar
    Join Date
    Aug 2001
    Location
    CA
    Posts
    2,459
    speaking of which... under which conditions could malloc/realloc/calloc fail besides inadequate remaining free memory? i'd like to know so i can gauge free memory at the very start, and decide whether or not to bomb with an 'insufficient memory' error at the very beginning... or i could malloc it all at the very beginning, but i may/may not use all that space so it's something i frown upon... any advice? i know you know your stuff Sayeh, so help me out! thanks...
    hasafraggin shizigishin oppashigger...

  5. #5
    Sayeh
    Guest
    In all fairness, I'm not going to give a "definitive" answer on this one, doubleanti, because I can't vouch for how Windows is written. Except that it was poorly done and based on OOPs.

    There are many cases where any Xalloc() call will fail and it's because some other portion of the O/S has trashed it (such as winsock scrambling a block header...

    All this is to say, that I can't say "Here are the only X ways in which Xalloc() will fail..."

    There are just too much bizarre ways in which it can fail-- even if you've done everything right!

  6. #6
    Linguistic Engineer... doubleanti's Avatar
    Join Date
    Aug 2001
    Location
    CA
    Posts
    2,459
    yeah, that really stinks that we don't really know who's-on-first, since it could have been rewritten to begin with... that really stinks... and i think that that undermines what it means to be a programmer... to be on-first...
    hasafraggin shizigishin oppashigger...

  7. #7
    Blank
    Join Date
    Aug 2001
    Posts
    1,034

  8. #8
    Blank
    Join Date
    Aug 2001
    Posts
    1,034
    http://www.cs.jhu.edu/~yairamir/cs418/os5/
    also looks like it has some good information.

  9. #9
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,590
    I've noticed that a lot of people do not use new and delete, especially inside of Windows code. Why? All of the source files for OWL in Borland use new and delete and never use the *alloc(s). The same is true for DOS code. Many people just do not use new and delete. My books do differentiate between new and delete and the *alloc(s). Seems to me if you use C++ you would use new and delete and vice versa. I've also heard that the reason is because new and delete do not work well with dynamic memory, but this cannot be true as OWL performs flawlessly using new and delete.

    Is this the same reason why people use printf() in C++ rather than cout - just depends on what they are used to using.

    For Windows apps I usually allocate from the system using the memory management functions in the API. However, some of that becomes very unstable if a previous app has thrashed the memory by not unlocking or failing to free up a memory handle/pointer.

    Been a while since I've done some Windows code so I've forgotten the mem functions in Windows - been doing lots of DJGPP and LFB stuff. Love my DJGPP.

  10. #10
    Banned master5001's Avatar
    Join Date
    Aug 2001
    Location
    Visalia, CA, USA
    Posts
    3,685
    Thanks for the links, everyone. Every post was helpful. To quickly address Bubba's question about new and delete there are a number of reasons why they aren't used as much. For games, it is always better to make a custom memory manager. Don't forget that both new and delete are operators, therefore you can overload them.

    Since my current project is very platform dependent, I have found myself open to many options. The manager I wrote actually does not use linked lists, like any other one i have written before now. I found that just using the headers and pointer arithmetic works just fine.

  11. #11
    Linguistic Engineer... doubleanti's Avatar
    Join Date
    Aug 2001
    Location
    CA
    Posts
    2,459
    say bubba, sounds like i'm gonna switch to cout and new/delete then! can you give me any resources on the subject? like a complete listing of their usage... [i can sort of grasp them... but not as much as i could a regular function]... i have C:The Complete Reference... which doesn't help too much... [new and delete has a small section tho...] oh, and i learnt C first without caring much for the ++ parts... but now that i use them throughly and on a regular basis, i'd really think it time to switch... PM me... thank you very much...
    hasafraggin shizigishin oppashigger...

  12. #12
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,590
    The syntax for using new and delete is:

    For single C data types and user defined types
    pointer_var = new var_type;
    delete pointer_var;


    For arrays of C data types and user defined types
    pointer_var = new var_type[arraysize];
    delete [size] pointer_var;


    I'll use a simple VGA buffer to explain new and delete.

    Code:
    unsigned char *Screen=new unsigned char[64000];
    delete [64000] Screen;   //delete 64000 objects of type   
                                           //unsigned char
    
    typedef unsigned char BYTE;
    BYTE *Screen=new BYTE[64000];
    delete [64000] Screen;  //delete 64000 objects of type BYTE
    Although I rarely see - delete [size] pointer_var - used, it is correct to specify the size rather than just using delete [] pointer_var. Newer compilers may substitute in the [size] portion for you - older compilers will flag an error if you just use []. The [size] portion informs delete the size of the memory object to delete - It already knows the type so you do not have to type cast for delete. It knows the size of the object, but it does not know how many objects are in the array - This is what the [size] portion tells delete. Remember the memory footprint is numobjects*sizeof(object) not just sizeof(object).

    Remember that because of the way that memory allocation is managed with new you can only use delete to free up memory if you used new to allocate it. Very important.

    Both allocate memory from the heap, both return null on failure. But, new does a couple more things than malloc() does.
    • New automatically computes the size of the type being allocated. You do not have to use the sizeof operator. This prevents the wrong amount of memory from being accidentally allocated.
    • New automatically returns the correct pointer type so you don't have to use a type cast.
    • It is possible to initialize and object by using new. For instance - int *myint; myint=new int(1000); This will initialize myint with the value of 1000. You cannot initialize arrays with new, however.
    • It is possible to overload new and delete relative to a class.


    New calls the constructor for the object you wish to create if you are creating a class object. Delete will call the class's destructor. Very nice way to handle memory as it relates to objects and classes. However note that if you have any classes that have virtual functions or are using derived types then you will be using late binding and your program will run a bit slower than if you used early binding. From my book it does not seem to me that using new and delete is any slower than using the *alloc(s). Speed as it relates to objects is based on early and late binding and which one of them you are using. The speed hit is negligible on today's systems so using virtual functions or derived types is not a bad idea. It allows your program greater functionality and allows you to create one interface to do multiple things.

    The beauty of new and delete is that you can create arrays of objects and delete them correctly since it already knows the size and type of the object. Notice that I said the size of the object. If you have an array of objects, you should specify the array size or number of elements in the delete operator using delete [size] objectpointer.
    Code:
    struct vertex
    {
       double x;
       double y;
       double z;
       void Test(void);
    };
    
    //Array example
    vertex *MyVertexes=new vertex[10];
    delete [10] MyVertexes;    
    
    //Non-array example
    vertex *MySingleVertex=new vertex;
    delete MySingleVertex;    //not an array of vertexes
    This could have just as easily been done if vertex was a class.


    Overloading new and delete
    To overload new and delete:

    void *operator new(size_t size)
    {
    //allocate here
    return pointer_to_mem;
    }

    void *operator delete(void *mem)
    {
    //free memory pointed to by *mem
    }

    size_t is an integer type. These (new and delete) can be overloaded globally or relative to one or more classes. This can become confusing just remember that when you use new and delete with an object that is controlled by a class that has overloaded new and delete, the class's version of new and delete will be used, just like when overloading unary operators.

    About cout and other I/O functions
    C++'s I/O system is rather large so I could not explain every facet of it here. My book has an entire chapter devoted to it so I'm going to even try to explain it here. Look on some websites about C++ I/O system or get a book by Herbert Schildt or Bjarne Strousop. With C++ I/O system it is possible to stream objects to disk and restore them later - this is helpful in Windows when saving or preserving the current desktop state. It allows you to create a persistent desktop. The I/O system is very powerful and very useful.

  13. #13
    zen
    zen is offline
    of Zen Hall zen's Avatar
    Join Date
    Aug 2001
    Posts
    1,007
    Although I rarely see - delete [size] pointer_var - used, it is correct to specify the size rather than just using delete [] pointer_var. Newer compilers may substitute in the [size] portion for you - older compilers will flag an error if you just use []. The [size] portion informs delete the size of the memory object to delete - It already knows the type so you do not have to type cast for delete.
    AFAIK the C++ standard just requires the use of the [] operator for deletion of arrays and not delete [size]. The system will maintain information about the size of the memory allocated, so it knows how much to delete (as it does using free()) and the operator [] is needed to ensure multiple destructors are called.

    Also most new and delete implementations seem to call malloc() and free() and so are only really needed for the automatic calling of constructors and destructors.
    zen

  14. #14
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,590
    Yes, new does use malloc but it is eaiser to use new with objects than always allocating numobjects*sizeof(objects). Also like you said it is vital to use new and delete when it comes to classes, ctors, and dtors. If you type in delete [] in older compilers it will say operand expected in .... You must use delete [size] pointer_var or it will not work. My Borland C++ 4.52 does accept delete [] or delete [size].

  15. #15
    Linguistic Engineer... doubleanti's Avatar
    Join Date
    Aug 2001
    Location
    CA
    Posts
    2,459
    hey thanks a lot bubba... and zen too! you guys really help me out! thanks...
    hasafraggin shizigishin oppashigger...

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. tools for finding memory leaks
    By stanlvw in forum C++ Programming
    Replies: 4
    Last Post: 04-03-2009, 11:41 AM
  2. Replies: 7
    Last Post: 02-06-2009, 11:27 AM
  3. Valgrind says I suck at memory management :)
    By carrotcake1029 in forum C Programming
    Replies: 6
    Last Post: 02-01-2009, 07:10 PM
  4. pointers
    By InvariantLoop in forum C Programming
    Replies: 13
    Last Post: 02-04-2005, 08:32 AM
  5. Managing shared memory lookups
    By clancyPC in forum Linux Programming
    Replies: 0
    Last Post: 10-08-2003, 04:44 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21