Memory Management: Swapping in data

Hello,
Is there any way I can control what resides in the main memory?
I have two heavy data structures and only one can reside in the main memory at a given time.
is there a way I can swp-in-swap-out data on a need basis?
Thanks,
Smita

Yes, when you want to swap them, save one to a file, deallocate the memory, then allocate and load the second one.

Is mmap available for your platform? I mean, the obvious answer is to simply leave an accessed timestamp on any given chunk of memory but that implies swapping to disc, deallocating old memory, allocating new memory and loading an old (previously saved) state when you need to swap; I don't think mmap incurs all of that (well directly; indirectly yeah). If the point of the exercise is to simply show "swappiness" then the steps previously-mentioned come into play and should be simple to implement. If the data structures are the same size then that makes things a little simpler (just overwrite main memory when swapping, skipping the reallocation fun).

So you are going to swap from storage somewhere?

That means using a FILE stream. You can serialize the data and save it that way.

Since you need to swap and cannot have more than one structure in memory at a time, you will have to store each of them seperately, ie, you will need twice as much storage space as memory. When you want to swap, you write the current structure out, then you overwrite that space by loading the other, previously saved file.

mmap would probably also be handy here, as jeffcobb points out. In that case I believe you can just use storage for both structures -- you do not have to keep either of them in memory. But I haven't used mmap much.

Thank you!

This program actually maintains low precison and higher precision trees with ton of information. It did occur to me to write to file and de-allocate memory, and then load in the second tree, but this operation is done in a loop. The first tree is updated with some information, then the second, then the first again, and so on till the breaking condition is met.

I see mmap might be a good option but it might end up taking the same amount of time as file IO. I will try it out first.

Thanks for your inputs again!

Quote:

---

Originally Posted by abachler

Yes, when you want to swap them, save one to a file, deallocate the memory, then allocate and load the second one.

---

Ablacher;
The quote didn't pick up your sig but it does pose an interesting question at a time when there are a lot of people walking around hand-waving about how tragic things are. I don't agree with you about everything but I do agree with you about this...

They need food, but more critically, they need fuel. They have to run hundreds of trucks, front end loaders, ambulances, cranes, generators for emgergency power, etc.

Some of that equipment will have to be bought or leased - so more $$$.

They have lots of sources of food aid (OxFam, etc.), but not many places will send them significant $$$ to run the rescue, the relief, and help rebuild, too.

When you have a disaster, you want $$$. Try not to be cynical, although it's easy to do it with a place like Haiti.

// And now, back to the thread !

WRT to the cost of file IO: what else are you going to do in order to preserve the state of the data structure that you are swapping out? NVRAM? Depending on the hardware the costs might be the same. Put it this way: what *else* are you going to do to preserve state?

Its a good point to bring up. Since the updations are purely tree traversals starting at the root node, I don't see a need to preserve any additional state of the DS (current pointers,etc).

However, if I do have to extend the functionality and carry over some critical state, a good idea would be to include the information in the file itself. So the file ends up having the initial few bytes of DS state followed by the data.

NVRAM might be a good idea, but files help me in debugging issues.