Thread: memory problems with map(), need an urgent reply

  1. #1
    Registered User
    Join Date
    Apr 2010
    Posts
    3

    Unhappy memory problems with map(), need an urgent reply

    Hi,

    I'm using a map<string, bool> and populating it with random values to see its memory behaviour. The problem is that after every 10 thousand values, I call map.clear(). Since, string has a destructor, the memory should be freed. But the memory use of the program keeps on rocketing up. Here is the small code:

    Code:
    int i = 0;
    map<string, bool> myMap;
    string temp = "";
    while (true) {
    	temp += i;
    	myMap[temp] = true;
    	i++;
    	if (i > 5000) {
    		cout << "Entering" << endl;
    		myMap.clear();
    		i = 0;
    	}
    }
    The memory use keeps on climbing up, until the program is forced to terminate by the OS. I want to be able to run this forever, is it possible?

    What if I weren't using string but char * and allocating memory to it with the new operator?

    Please reply soon if you know the answer, because I want to finish this thing in a few hours, and this is my last hope. Thanks.

  2. #2
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    Well, perhaps it would be clearer if you wrote:
    Code:
    for (;;)
    {
        map<string, bool> myMap;
        string temp;
        for (int i = 0; i <= 5000; ++i)
        {
            temp.append(1, i);
            myMap[temp] = true;
        }
        cout << "Entering" << endl;
    }
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  3. #3
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    Or to put it another way, what do you want the key strings to be? (If you just want them to be "5001", "5002", etc., then you need to change how you build your string.)

  4. #4
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    If you create a new map (or clear an old one) and then add an element with an index of 5000, or 10000, or 15000, you now have a 5000/10000/15000 element list. That's what you are doing with temp.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  5. #5
    Registered User
    Join Date
    Apr 2010
    Posts
    3
    Thanks, I just realize now that was one stupid question.

  6. #6
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    Quote Originally Posted by MK27
    If you create a new map (or clear an old one) and then add an element with an index of 5000, or 10000, or 15000, you now have a 5000/10000/15000 element list.
    That is not the case.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  7. #7
    Registered User VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,607
    Only thing I can think of is that map doesn't want to let go of the memory it allocated just in case you might use it again later. It's probably using some type of lazy implementation where it only releases memory when it absolutely has to. I think it does something similar for allocation in that it will not allocate a new chunk until it has to.

    I have also seen map memory grow to enormous sizes and it never wants to let go of it. Some time back I was tasked with figuring out something very similar. My only conclusion was that it was very hard to determine how much memory a given map would require at any one time. I had very simple code much like yours and then checked the memory via API calls. I could not find a consistent behavior except in allocation. Map would allocate a sizeable chunk and once that was filled it would then allocate another sizeable chunk. The allocation scheme for the STL is predictable and constant but I could never figure out deallocation. Perhaps other more knowledgeable members here can fill in the gaps. I will check Effective STL to see if I can find anything useful.

    Because of my experiences with map I normally opt to use pointers inside of them so that on removal I can manually delete and manually erase the object from the container. Ostringstream appears to have this same type of behavior. If you do not allow ostringstream to go out of scope or manually clear it via .str("") it will continually eat memory.
    Last edited by VirtualAce; 04-24-2010 at 12:29 PM.

  8. #8
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    Quote Originally Posted by Bubba
    Only thing I can think of is that map doesn't want to let go of the memory it allocated just in case you might use it again later. It's probably using some type of lazy implementation where it only releases memory when it absolutely has to. I think it does something similar for allocation in that it will not allocate a new chunk until it has to.
    That is what I suspected at first (I was about to suggest the swap with a default constructed temporary trick), but then that should mean that on the next round, the already allocated space would have been used.

    The problem is probably just due to the unlimited appending to temp that my example fixes (or not, as tabstop noted).
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  9. #9
    Registered User
    Join Date
    Apr 2010
    Posts
    3
    Yes, laserlight is right, the map wasn't eating up the memory so fast, the string was.

    Anyway, thanks guys, now I'm trying to debug my actual problem with char *, I'll let you know if I have any further questions. Thanks

  10. #10
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by laserlight View Post
    That is not the case.
    No, I suppose not -- it's a map, not an indexed array. At first glance I assumed temp was an int, not a string.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Memory Fragmentation with Dynamic FIFO Queue
    By fguy817817 in forum Linux Programming
    Replies: 17
    Last Post: 10-31-2009, 04:17 AM
  2. tools for finding memory leaks
    By stanlvw in forum C++ Programming
    Replies: 4
    Last Post: 04-03-2009, 11:41 AM
  3. pointers
    By InvariantLoop in forum C Programming
    Replies: 13
    Last Post: 02-04-2005, 09:32 AM
  4. Replies: 15
    Last Post: 04-19-2002, 02:23 PM
  5. Ascii conversion problems with dynamic memory
    By Butters in forum C++ Programming
    Replies: 2
    Last Post: 01-29-2002, 12:22 PM

Tags for this Thread