Thread: Synchronization problem, sort of producer/consumer

  1. #1
    Registered User
    Join Date
    Jan 2012
    Posts
    2

    Synchronization problem, sort of producer/consumer

    Hi,

    I have 5 sqlite tables, using the sqlite3 api.

    In one process I'm moving a record from table 1 to 2, 2 to 3, etc. I'm doing the move with a lock on the table so no one else touches it during the move.

    In the second process I'm going through the tables once per function call, trying to select the record to see which table the record is in. I try to select from table 1, if it's found I return the value 1. If not, I try table 2, etc. If it is not found in any table (the record can be moved back from table 2 to table 1 but then it gets deleted from table 1 so it would be considered deleted once it leaves table 2 and I don't have to find it in table 1 to be correct) then I return 0.

    The second process does not use any locks.

    I only want the second process to return 0 if the record is truly not there - if it's in transition between tables I don't want to return 0 by accident.

    I think that since I'm locking the tables during transitions, and since I'm only allowed to move (and select) in the forward direction (1->2->3->4->5) I should not be missing the record. Is this a simple programming 101 problem that I'm just not remembering?

    I'm wondering if anyone knows what this problem would be called (I know it's not quite producer/consumer, but some name like that) that I could look up to see it solved in a textbook?

    Any info is appreciated. Thanks.

    JP
    Last edited by jpenney; 01-25-2012 at 03:00 PM.

  2. #2
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    If your move from tables 1->2, 2->3, etc is done by selecting a record, inserting it in the new table, then deleting it from the old table, and always in the forward direction, and your search is always in the forward direction, then you shouldn't miss anything.

    Can I ask why you have 5 different tables with ostensibly the same data, and you're moving records between them? If the records are the same in each table, why not simply use one table with some sort of status flag on the record that would be equivalent to table number, and skip the whole synchronizing thing all together. No need for table locking, looksups happen in one table only, and it's cheaper and easier to "move" a record between tables by simply updating one field. Failing that, look into selecting from multiple tables. I've never used sqlite and I'm far from a SQL expert, but a quick Google search turned up SQLite Tutorial. Read the section on the attach command here. It sounds promising.

  3. #3
    Registered User
    Join Date
    Jan 2012
    Posts
    2

    Thanks for the feedback. That confirms my assumptions.

    Quote Originally Posted by anduril462 View Post

    Can I ask why you have 5 different tables with ostensibly the same data, and you're moving records between them? If the records are the same in each table, why not simply use one table with some sort of status flag on the record that would be equivalent to table number, and skip the whole synchronizing thing all together.

    .
    You're right, that would be ideal. However it's legacy code. Bleah. There are several other processes and functions that update these tables so the structure is not changing anytime soon, sadly.

    There must be something else happening so I'll start digging in.

    Thanks again.

  4. #4
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    Oh, I see you edited your post around the time I was writing my reply. Not sure what you changed, but...

    Does locking in the first process block you from reading in the second? If not, you could miss data that way. Also, you may want to consider locking the tables when you do your lookup in process 2, to prevent process 1 from changing data underneath you. Also, I found this method for selecting from multiple tables: SQL with SQLite (see the "selecting from two tables" section). Again, I'm not a SQL or sqlite expert, so I don't know just how atomic any of this stuff is.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Producer Consumer concurrency problem
    By Kurjack in forum C Programming
    Replies: 0
    Last Post: 10-18-2010, 02:21 PM
  2. consumer producer problem with threads
    By nabi in forum C Programming
    Replies: 3
    Last Post: 08-27-2010, 09:51 AM
  3. Producer/Consumer Problem
    By AvengedGoat in forum C++ Programming
    Replies: 5
    Last Post: 03-21-2010, 12:33 PM
  4. pthread producer consumer problem
    By Amit Chikorde in forum C Programming
    Replies: 1
    Last Post: 01-17-2007, 07:39 PM
  5. Producer consumer problem
    By traz in forum C Programming
    Replies: 2
    Last Post: 11-08-2002, 08:04 PM