partial serialize/deserialize a structure

This is a discussion on partial serialize/deserialize a structure within the C Programming forums, part of the General Programming Boards category; Hello everyone, I have a large number of strcture instances, the I can not have all of them in memory. ...

  1. #1
    Registered User
    Join Date
    May 2006
    Posts
    1,579

    partial serialize/deserialize a structure

    Hello everyone,


    I have a large number of strcture instances, the I can not have all of them in memory. So I have the idea of serialize/deserialize partial of them to/from disk. The operation I need on these structure instances is to enumerate elements one by one, and I want to provide applications which use this interface to enumerate elements be transparent to how and which parts of structure instances is written/read to/from disk -- and the application only will feel read from memory.

    I am wondering whether there are any samples/tutorials to make a reference?


    thanks in advance,
    George

  2. #2
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    I don't know of any samples/tutorials, but this is fairly easy to do yourself.

    In your processing loop, just keep track of the numbers that have been processed, whenever the number of processed structs reaches a good threshold, have it call your RefillStructs function, and re-fill the array of structs, from the disk.

    You want to refill from the disk in the largest chunks your program can comfortably manage, not one at a time. Think in terms of hundreds or even thousands at a time being read from HD.

    Adak

  3. #3
    Registered User
    Join Date
    May 2006
    Posts
    1,579

    Post

    Thank you Adak,


    Quote Originally Posted by Adak
    I don't know of any samples/tutorials, but this is fairly easy to do yourself.

    In your processing loop, just keep track of the numbers that have been processed, whenever the number of processed structs reaches a good threshold, have it call your RefillStructs function, and re-fill the array of structs, from the disk.

    You want to refill from the disk in the largest chunks your program can comfortably manage, not one at a time. Think in terms of hundreds or even thousands at a time being read from HD.

    Adak
    I roughly know your idea ... But I have never done some things before ... Could you recommend some samples please?


    regards,
    George

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Problem referencing structure elements by pointer
    By trillianjedi in forum C Programming
    Replies: 19
    Last Post: 06-13-2008, 05:46 PM
  2. Replies: 5
    Last Post: 02-14-2006, 08:04 AM
  3. Dikumud
    By maxorator in forum C++ Programming
    Replies: 1
    Last Post: 10-01-2005, 06:39 AM
  4. Serial Communications in C
    By ExDigit in forum Windows Programming
    Replies: 7
    Last Post: 01-09-2002, 09:52 AM
  5. C structure within structure problem, need help
    By Unregistered in forum C Programming
    Replies: 5
    Last Post: 11-30-2001, 04:48 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21