problem with sending array of struct over shared memory

This is a discussion on problem with sending array of struct over shared memory within the C Programming forums, part of the General Programming Boards category; ahhhh, thank you VEERYYYY much, I was already loosing my mind. And I'm new to C programming so I'm still ...

  1. #16
    Registered User
    Join Date
    May 2008
    Posts
    14
    ahhhh, thank you VEERYYYY much, I was already loosing my mind. And I'm new to C programming so I'm still at war with pointers. Now it seams so logical and easy but I wouldn't never figure it out by myself. btw. now it's working nice

  2. #17
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by jet-plane View Post
    ahhhh, thank you VEERYYYY much, I was already loosing my mind. And I'm new to C programming so I'm still at war with pointers. Now it seams so logical and easy but I wouldn't never figure it out by myself. btw. now it's working nice
    Yes, it took me a bit of thinking to get what was going on in the first place - and the printouts helped me tell that the pointer was walking big steps of 0x190 (400), where you expect to move 20 bytes (0x14).

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #18
    Registered User
    Join Date
    May 2008
    Posts
    14
    I must thank you again, you made my day
    Last edited by jet-plane; 05-08-2008 at 10:52 AM. Reason: double post :)

  4. #19
    Registered User
    Join Date
    May 2008
    Posts
    14
    Eh I did it again , I nested the struct to add locking mechanism, and now it's jumping again by 0x190 instead just 20 bytes.
    Code:
    #include <stdlib.h>
    #include <stdio.h>
     
    #include <sys/types.h>
    #include <sys/ipc.h>
    #include <sys/shm.h>
     
    #define KEY (5679)
    
    typedef struct
    {
            int x, y, xo, yo, id;
    }command[20], *command_ptr;
    
    typedef struct _tcommand
    {
    	int lock;
    	command comm;
    }tcommand;
    
    main ()
    {
            int shmid, shmkeyid, i, size;
            command *tuio_cmd;
    	tcommand *tcomm;
            size = sizeof( *tcomm);
    
            shmid = shmget (KEY, size, 0666 | IPC_CREAT);
            if (shmid != -1)
            {
                    tcomm = shmat (shmid, NULL, 0);
                    if (tcomm != (void *) -1)
                    {
    			tuio_cmd = &(*tcomm).comm;
                            printf ("Memory tcomm@:&#37;X tcom->comm@: %X, size of struct:%d\n\n", (int) tcomm, (int)tcomm->comm,(int) tuio_cmd,size);
    			tuio_cmd[4]->xo = 1; //Set old X just to test if block
                                    for( i=0; i<20; i++ )
    				{
    					command_ptr p = &(*tuio_cmd)[i]; // <--- pointer p points to memory location of i'th member of tuio_cmd array
    					printf ("Memory tuio_cmd[%d]@: %p\n",i,p->x);
    // 					if (tuio_cmd[i]->xo) // Check if there is old x and y, it's enaugh to check just one
    // 						printf ("num:%d,X:%d,Y:%d,Xo:%d,Yo:%d,Id:%d\n", i,tuio_cmd[i]->x,tuio_cmd[i]->y, tuio_cmd[i]->xo, tuio_cmd[i]->yo, tuio_cmd[i]->id);
    //                                 	else // We get just one point coordinates
    // 						printf ("X:%d,Y:%d,Id:%d\n", tuio_cmd[i]->x,tuio_cmd[i]->y,tuio_cmd[i]->id);
    				}
                            if (shmdt (tcomm) == -1)
                            {
                                    fprintf (stderr, "shmdt failed\n");
                                    exit (EXIT_FAILURE);
                            }
                            if (shmctl (shmid, IPC_RMID, 0) == -1)
                            {
                                    fprintf (stderr, "shmctl(IPC_RMID) failed\n");
                                    exit (EXIT_FAILURE);
                            }
                    }
            else fprintf(stdout,"error attaching memory\n");
            }
            else fprintf(stdout,"error in allocating shm\n");
    return 0;
    }
    Output at the moment:
    Memory tcomm@:B7F81000 tcom->comm@: B7F81004,tuio: B7F81004, size of struct:404

    Memory tuio_cmd[0]@: (nil)
    Memory tuio_cmd[1]@: (nil)
    ... (repeating)

    I have tried every possible combination that crossed my mind .. from (*tuio_cmd)[i]->x to tuio_cmd[i].x with and without pointers, even as its shown here I added p as pointer to the memory. Obviously without success.
    And I see that the memory allocation for the tuio_cmd is good (start + 4 bytes for int of lock). I really don't understand how the !@$!#% is done.
    If someone is willing to give me a hand again ok. If not, it doesn't matter anymore, as I don't want to exploit the good will of the ppl.

    [edit]
    Oh finally, I think I got it ..
    Code:
    command_ptr p = &((*tuio_cmd)[i]); // <--- pointer p points to memory location of i'th member of tuio_cmd array
    Output:
    Memory tcomm@:B7F99000 tcom->comm@: B7F99004,tuio: B7F99004, size of struct:404

    Memory tuio_cmd[0]@: 0xb7f99004
    Memory tuio_cmd[1]@: 0xb7f99018
    Memory tuio_cmd[2]@: 0xb7f9902c
    Memory tuio_cmd[3]@: 0xb7f99040
    Memory tuio_cmd[4]@: 0xb7f99054
    ....
    Last edited by jet-plane; 05-08-2008 at 11:01 AM.

  5. #20
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,660
    First......take a break

    When you come back, remove that "[20]" out of your typedef - that's making *my* head hurt!

    Code:
    typedef struct command_
    {
            int x, y, xo, yo, id;
    } command;
    
    typedef struct tcommand_
    {
    	int lock;
    	command comms[20];
    } tcommand;
    Fix the rest based on that. Post yours progress here if you hit a wall.

    gg

  6. #21
    Registered User
    Join Date
    May 2008
    Posts
    14
    Eheh I know .. that's hurting my head too, but as you see I'm having really hard time with it,
    finally I made it. But now I would like to optimize a little and beautify the code. It's just too ugly this way. About that [20] eheh I tried that few times before .. when I was stuck at the first issue, I'll try again and see If I'll have any luck. If you see anything else that could be "fixed" or typed more simple plz. let me know.

  7. #22
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,660
    In my code editor, I turn off real tab characters and use nothing but spaces (editor inserts spaces when I hit tab). That will help keep your posted code looking the same as it does in your editor.

    My personal rule-of-thumb is never to "hide a pointer" inside a typedef, like "command_ptr" - which also "hides" an array-20 inside itself.......(head starting to hurt)....

    The code is a bit more "self-documenting" that way as well. Consider
    Code:
       command comms[20];
    vs
       command comm; // command is really "array-20"
    >> anything else that could be "fixed"
    Probably need to add proper memory synchronization to your shared memory accesses.
    How many producers?
    How many consumers?
    How do consumers know when something has been produced?
    How to producers know consumers are ready to consume?
    What is the policy for dealing with more "production" than the consumers can handle?

    gg

  8. #23
    Registered User
    Join Date
    May 2008
    Posts
    14
    Ok listening your suggestions, I have come to following:
    Code:
    #include <stdlib.h>
    #include <stdio.h>
     
    #include <sys/types.h>
    #include <sys/ipc.h>
    #include <sys/shm.h>
     
    #define KEY (5679)
    
    typedef struct
    {
            int x, y, xo, yo, id;
    }command; // command is really "array-20"
    
    typedef struct _tcommand
    {
    	int lock;
    	command comm[20];
    }tcommand;
    
    main ()
    {
            int shmid, shmkeyid, i, size;
            command *tuio_cmd[20],*p;
    	tcommand *tcomm;
            size = sizeof( *tcomm);
    
            shmid = shmget (KEY, size, 0666 | IPC_CREAT);
            if (shmid != -1)
            {
                    tcomm = shmat (shmid, NULL, 0);
                    if (tcomm != (void *) -1)
                    {
    			tuio_cmd[0] = &(*tcomm).comm;
    //                         printf ("Memory tcomm@:&#37;X tcom->comm@: %X,tuio: %X, size of struct:%d\n\n", (int) tcomm, (int)tcomm->comm,(int) tuio_cmd,size);
    			while(1) 
    			{
    			if(((*tcomm).lock)==1)
    			{
                                    for( i=0; i<20; i++ )
    				{
    					p = &((*tuio_cmd)[i]); // <--- pointer p points to memory location of i'th member of tuio_cmd array
    					if (p->xo) // Check if there is old x and y, it's enaugh to check just one
    						printf ("num:%d,X:%d,Y:%d,Xo:%d,Yo:%d,Id:%d\n", i,p->x,p->y, p->xo, p->yo, p->id);
                                    	else // We got just one point coordinates
    						printf ("num:%d,X:%d,Y:%d,Id:%d\n",i, p->x,p->y,p->id);
    				}
    			((*tcomm).lock)=0;
    			}
    			}
                            if (shmdt (tcomm) == -1)
                            {
                                    fprintf (stderr, "shmdt failed\n");
                                    exit (EXIT_FAILURE);
                            }
                            if (shmctl (shmid, IPC_RMID, 0) == -1)
                            {
                                    fprintf (stderr, "shmctl(IPC_RMID) failed\n");
                                    exit (EXIT_FAILURE);
                            }
                    }
            else fprintf(stdout,"error attaching memory\n");
            }
            else fprintf(stdout,"error in allocating shm\n");
    return 0;
    }
    Now I have warrning: assignment from incompatible pointer type @34 line tuio_cmd[0] = &(*tcomm).comm;

    About consumer/producers .. producer is actually python script, so I'm not afraid that it can
    send more that the C part can handle. About synchronization, I have added lock int just for that.
    I'm still not sure this is optimized code .. so if you see something else that could be "adjusted" plz share.. and thnx for assisting me.

    Ps. offcourse everything is working well now. (as far I can see for now)
    Last edited by jet-plane; 05-08-2008 at 01:45 PM.

  9. #24
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,660
    >> command *tuio_cmd[20]
    That is a fixed size array of pointers to command. Not what you want.
    Here is everything simplified, types, code, and tabs:
    Code:
    #include <stdlib.h>
    #include <stdio.h>
    #include <sys/types.h>
    #include <sys/ipc.h>
    #include <sys/shm.h>
     
    #define KEY (5679)
    
    typedef struct
    {
        int x, y, xo, yo, id;
    } command;
    
    typedef struct
    {
        int lock;
        command comm[20];
    } tcommand;
    
    int main()
    {
        int shmid, i;
        tcommand *tcomm;
        
        shmid = shmget(KEY, sizeof(*tcomm), 0666 | IPC_CREAT);
        if (shmid == -1)
        {
            fprintf(stderr,"error in allocating shm\n");
            exit(EXIT_FAILURE);
        }//if
                
        tcomm = shmat(shmid, NULL, 0);
        if (tcomm == (void*)-1)
        {
            fprintf(stderr,"error attaching memory\n");
            exit(EXIT_FAILURE);
        }//if
    
        while (1) 
        {
            // TODO: use real synchronization
            if (tcomm->lock == 1)
            {
                for (i = 0; i < 20; ++i)
                {
                    command *p = &(tcomm->comm[i]);
    
                    // Check if there is old x and y, it's enough to check just one
                    if (p->xo) 
                        printf("num:&#37;d,X:%d,Y:%d,Xo:%d,Yo:%d,Id:%d\n", 
                               i, p->x, p->y, p->xo, p->yo, p->id);
                    else 
                        printf("num:%d,X:%d,Y:%d,Id:%d\n",
                               i, p->x, p->y, p->id);
                }//for
    
                tcomm->lock = 0;
            }//if
    
            // TODO - don't use 100% CPU
            // TODO - some way to stop
        }//while
    
        if (shmdt(tcomm) == -1)
        {
            fprintf(stderr, "shmdt failed\n");
            exit(EXIT_FAILURE);
        }//if
    
        if (shmctl(shmid, IPC_RMID, 0) == -1)
        {
            fprintf(stderr, "shmctl(IPC_RMID) failed\n");
            exit(EXIT_FAILURE);
        }//if
        
        return 0;
    }//main
    So you got some TODO's there one you're comfortable with the code.

    gg

  10. #25
    Registered User
    Join Date
    May 2008
    Posts
    14
    Ok I added the semaphore locking, and exit out the loop with previous lock. Only TODO remaining is dont use 100&#37; of cpu, what do you mean by that ?
    here is the code:
    Code:
    #include <stdlib.h>
    #include <stdio.h>
    #include <sys/types.h>
    #include <sys/ipc.h>
    #include <sys/shm.h>
    
    #include <sys/sem.h>
    
    #define KEY (5679)
    
    typedef struct
    {
        int x, y, xo, yo, id;
    } command;
    
    typedef struct
    {
        int lock;
        command comm[20];
    } tcommand;
    
    void seminit()
    {
       int id;
            union semun {
                    int val;
                    struct semid_ds *buf;
                    ushort * array;
            } argument;
    
       argument.val = 0;
       id = semget(KEY, 1, 0666 | IPC_CREAT);
       if(id < 0)
       {
          fprintf(stderr, "Unable to obtain semaphore.\n");
          exit(0);
       }
       if( semctl(id, 0, SETVAL, argument) < 0)
       {
          fprintf( stderr, "Cannot set semaphore value.\n");
       }
       else
       {
          fprintf(stderr, "Semaphore %d initialized.\n", KEY);
       }
    }
    
    
    int main()
    {
        int shmid, i;
        tcommand *tcomm;
    
        shmid = shmget(KEY, sizeof(*tcomm), 0666 | IPC_CREAT);
        if (shmid == -1)
        {
            fprintf(stderr,"error in allocating shm\n");
            exit(EXIT_FAILURE);
        }//if
        tcomm = shmat(shmid, NULL, 0);
        if (tcomm == (void*)-1)
        {
            fprintf(stderr,"error attaching memory\n");
            exit(EXIT_FAILURE);
        }//if
        seminit(); //Init the semaphore
        int id;
        struct sembuf operations[1];
        int retval;
        id = semget(KEY, 1, 0666); //semaphores and shared memory have separate key spaces, so one can safely use the same key for each.
        if(id < 0)
        {
           fprintf(stderr, "Cannot find semaphore, exiting.\n");
           exit(0);
        }
        do
        {
        operations[0].sem_num = 0;
        operations[0].sem_op = -1;
        operations[0].sem_flg = 0;
        retval = semop(id, operations, 1);
                for (i = 0; i < 20; ++i)
                {
                    command *p = &(tcomm->comm[i]);
                    // Check if there is old x and y, it's enough to check just one
                    if (p->xo) 
                        printf("num:%d,X:%d,Y:%d,Xo:%d,Yo:%d,Id:%d\n", 
                               i, p->x, p->y, p->xo, p->yo, p->id);
                    else 
                        printf("num:%d,X:%d,Y:%d,Id:%d\n",
                               i, p->x, p->y, p->id);
                }//for
        }while (!tcomm->lock);
        if (shmdt(tcomm) == -1)
        {
            fprintf(stderr, "shmdt failed\n");
            exit(EXIT_FAILURE);
        }//if
        if (shmctl(shmid, IPC_RMID, 0) == -1)
        {
            fprintf(stderr, "shmctl(IPC_RMID) failed\n");
            exit(EXIT_FAILURE);
        }//if
        return 0;
    }//main
    Another question, I'm thinking on using this in threaded program, so I'm not sure if semaphore approach is
    wise, as it's blocking and waiting for signal.And one more thing, why is now sitting and waiting for second release of semaphore .. I added while after do, so it should exit as soon the condition is satisfied.
    [edit]
    I got it why it was entering in the loop for second time, it was the other side (python script) which had the glitch.. now Is working well.And about the thread .. I'm still able to release the semaphore with
    signal from main thread so that wouldn't have to be the problem I think.
    Last edited by jet-plane; 05-09-2008 at 06:27 AM.

  11. #26
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,660
    >> ... don't use 100% of cpu, what do you mean by that?
    Before, there was just a while(1) loop that did nothing but if() and printf(). The OS and CPU will gladly run that loop as fast as it can. When there's no work to be done, you should be blocked in the OS somewhere.

    >> Ok I added the semaphore
    You are on the right track!

    >> I'm not sure [of] semaphore approach
    There are still a few things to consider. Right now your loop will block at semop() until the semaphore value is non-zero. At this point, we're waiting for the python script to populate the data. Once the data is populated in shared memory, the python script increments the semaphore - which in turn allows the semop() call to to return. This works great if you only need to send one parcel of data.

    But what if we want to send multiple tcommand's? How does the python script know that it's ok to write more data? In other words, how does it know that we've completed reading everything it just sent us?

    There are several ways to tackle this depending on what you want to do. Just for an example, let's say they you wanted your producer (python script) to block until the produced data has been consumed - meaning no data should be lost, it should all be consumed.
    Code:
    semaphore s = 0;
    
    producer()
    {
       while()
       {
            wait until s == 0
            produce tcommand
            set s = 2
       }
    }
    
    consumer()
    {
        while()
        {
            s -= 1 // blocking semop
            consume tcommand
            s -= 1 // should not block since producer set s = 2 (assumes single consumer)
        }
    }
    >> I'm thinking on using this in threaded program
    What we've been talking about so far is IPC over shared memory between two processes. But in reality, it's not much different from two threads communicating over a malloc'd chunk of memory

    gg

  12. #27
    Registered User
    Join Date
    May 2008
    Posts
    14
    That worked really nice, good logic .. I must learn to think like that. I tried different approach, which I found in other producer/consumer C examples. But they weren't appropriate for my case as I'm using python, and It's hard to synchronize it. About the thread .. I didn't mean that I'm going to share the malloced memory over the multiple threads, I'm using one thread just to get and parse the commands received over shared memory. Anyway as I'm going to release the code (after I finish it) as GPL, I'm going to share here also, since there could be someone else who could benefit from this. It's a nice way to connect python and C over low latency solution
    So the C part in question looks now like this:
    Code:
        operations[0].sem_num = 0;
        operations[0].sem_op = -1;
        operations[0].sem_flg = 0;
        retval = semop(semid, operations, 1);
                for (i = 0; i < MAXBLOBS; ++i)
                {
                    command *p = &(tcomm->comm[i]);
    		// Check for blob Id, If there isn't any skip, it's void
    		if (p->id)
    		{
                    // Check if there is old x and y, it's enough to check just one
                    if (p->xo) 
                        printf("num:&#37;d,X:%d,Y:%d,Xo:%d,Yo:%d,Id:%d\n", 
                               i, p->x, p->y, p->xo, p->yo, p->id);
                    else 
                        printf("num:%d,X:%d,Y:%d,Id:%d\n",
                               i, p->x, p->y, p->id);
    		}//if
                }//for
        operations[0].sem_num = 0;
        operations[0].sem_op = -1;
        operations[0].sem_flg = 0;
        retval = semop(semid, operations, 1);
    And the python part :
    Code:
    #!/usr/bin/env python
    import shm
    import ctypes
    class tuio(ctypes.Structure):
    	_fields_ = [
    		('x',ctypes.c_int),
    		('y',ctypes.c_int),
    		('xo',ctypes.c_int),
    		('yo',ctypes.c_int),
    		('id',ctypes.c_int)
    	]
    class nested(ctypes.Structure):
    	_fields_=[
    		('lock',ctypes.c_int),
    		('comm',tuio*20)
    	]
    x=100 #some test data
    y=123 #same here
    key = 5679
    i=0 #send just array [0]
    mem = shm.memory(shm.getshmid(key))
    sem = shm.semaphore(shm.getsemid(key))
    sem.setval(0) #set the semaphore = 0
    mem.attach()
    tuio_cmd = nested()
    tuio_cmd.comm[i].x=x
    tuio_cmd.comm[i].y=y
    tuio_cmd.comm[i].id=1
    tuio_cmd.lock=1 #sets the lock to 1 and the main C program exits on this
    sem.Z() #wait for semaphore == 0
    mem.write(tuio_cmd) #write to the memory
    sem.setval(2) #set the semaphore s=2
    PS. Thnx for the hints and guide trough this voyage in C programming.
    Last edited by jet-plane; 05-10-2008 at 04:25 AM.

Page 2 of 2 FirstFirst 12
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Link List math
    By t014y in forum C Programming
    Replies: 17
    Last Post: 02-20-2009, 05:55 PM
  2. Fixing my program
    By Mcwaffle in forum C Programming
    Replies: 5
    Last Post: 11-05-2008, 02:55 AM
  3. Replies: 2
    Last Post: 07-11-2008, 07:39 AM
  4. RE: client/server shared memory problem
    By hampycalc in forum C Programming
    Replies: 0
    Last Post: 03-10-2006, 01:26 PM
  5. Binary Search Trees Part III
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 10-02-2004, 03:00 PM

Tags for this Thread


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21