BSD mmap for shared memory - am I right?

I've had to piece this together from documentation, so if you could just confirm my conclusions and correct me (if necessary), that'd be awesome. The only good tutorial I could find on the topic is at fscked.org, and it's full of "TO DO" pages.

If you'll recall from a previous thread, I have a program in which the "server" process spawns a number of "client" threads (this is for a shell - so I'm using the term server in the same sense as X-Server). The server maintains a doubly-linked list to represent all the clients, with each client having a pointer to the node that represents it. Clients can be removed, inserted, and moved around by the server (hence the double-linked list structure).

This wasn't working, and that's how my attention got drawn to the fact that malloc'ed memory isn't shared between processes, even if you fork after malloc'ing it (I was very surprized - is this correct?). So I looked at shared memory, and it appears I have two options: System V's IPC, and BSD's mmap. It's my understanding that mmap is the more popular of the two choices, so I've chosen it (thoughts on this?)

Now the actual calls to mmap and munmap are what's confusing me.

Code:

---

myPointer = (myType *)mmap(0, sizeof(myType), PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, NULL, 0);

---

Where myPointer is just like the pointer return by malloc. I'd like this to act just like malloc'ed memory, except for the fact that it is shared (meaning, I don't want to actually have files written to the disk - I just want it in RAM). Other than that, I really don't care where it is, hence the NULL and 0's. Does this call look right? Have I provided the right flags, etc...?

Then when I call munmap, I would do this:

Code:

---

myPointer = (myPointer, sizeof(myType));

---

Is THAT correct? All the man pages I found listed the first argument for both methods as being "void *start", but all the examples I found specified 0 as the first parameter for mmap. If that's not right, my question is - how does munmap know what memory to free, since I've created an arbitrary number of clients with mmap, and I just close them one by one as the user dictates.

Again - just a confirmation / correction is all I'm looking for, but if you have any thoughts on how I could go about this a better way - I'm all ears.

Is this of any help:
http://fscked.org/writings/SHM/shm-2.html

I'm guessing you missed the unmap part of the second code-snippet...

--
Mats

Quote:

---

Originally Posted by sean

If you'll recall from a previous thread, I have a program in which...

This wasn't working, and that's how my attention got drawn to the fact that malloc'ed memory isn't shared between processes, even if you fork after malloc'ing it

---

HA HA! I remember that thread and actually you wrote it as if it already worked:

Quote:

---

Originally Posted by sean, elsewhere

They also share a linked list of structs (where each node represents a child, and contains the file pointer for the FIFO to the client, the pid, a unique name, etc....). The first node is malloc'd before the first process fork's, and it's expanded as new children are created by other children processes.

---

To which I remember thinking, wow, I'll have to go back and try something like that again, because I had thought it wouldn't work last time...

Quote:

---

(I was very surprized - is this correct?)

---

Yes, that's right, it won't work.

You don't appear to have gotten very far with mmap, so you might as well look at this, which is as far as I went but it is a working example. I used an on disk file and it was accessible to another process, but I haven't made the next step (IPC using mmap at both ends), but I seem to have to believed then that it would work, so I would be interested to find out if you get it to do so. Really really.

Quote:

---

(I was very surprized - is this correct?)

---

ps. why not use pthreads? That will probably be easier and more useful than mmap...

I've considered POSIX, but it seemed so "not my style" - but it probably would be better - I'll look into it.

I've been using ncurses (including panels) for all my i/o - is there a good POSIX equivalent that I'm completely missing, or is ncurses still the best choice?

Come to think of it I had a better reason for using fork - I wanted every client to be in it's own process so that if it locked up, the server could kill it without affecting any other client (the Google Chrome strategy...) Of course, being brand new to posix, maybe there's an equivalent, but that's why I went the fork() route.

Quote:

---

Originally Posted by sean

Come to think of it I had a better reason for using fork - I wanted every client to be in it's own process so that if it locked up, the server could kill it without affecting any other client (the Google Chrome strategy...) Of course, being brand new to posix, maybe there's an equivalent, but that's why I went the fork() route.

---

fork() is definitely more straightforward, just now you have that IPC issue. That's why I'm so curious if you can get this mmap deal to work, it would be super.

For some reason I worry that it may not unless you actually use a disk file, because each process will probably have to have a seperate map of the file (you won't be able to pass the address, same as a pointer). Which might then mean the kernel or something would restrict the file such that you can't mmap simultaneously from two seperate processes. In my experiment "the other process" was just viewing the file -- so it has to work, because you could change the file as easily as you could view it from outside.

But on a big link list using a disk file is out of the question.

Quote:

---

I've considered POSIX, but it seemed so "not my style"

---

but you're already using it, fork(), mmap(), and most of the syscalls you're probably using are part of this standard as long as you take care of the options you use (which are sometimes specific to the o/s). Concerning the usage of mmap(), remember that you must have an existing object to map (referenced by the fd parameter), the old way involves the shmget/at/etc. routines, the portable posix ones are shm_create/etc. Once the segment is created, you can map it and use it as if it was the result of a malloc() call.

Quote:

---

fork(), mmap(), and most of the syscalls you're probably using are part of this standard

---

Fair enough - but were they not originally part of UNIX-like systems before POSIX adopted them as their own standard?

As for threads - I would definitely rather have each client run in it's own process, so shared memory is a must.

Quote:

---

you must have an existing object to map (referenced by the fd parameter)

---

My understanding was that you could use the MAP_ANONYMOUS option and map part of the RAM - which would be ideal, and then the fd parameter is apparently ignored. My question then is how do you reference the stored memory? And how do you free it?

At this point, I'm not even sure if I can use mmap because I would call mmap to allocate space for the struct, fork, and then both process would have a pointer to the memory. When I need a new client, again, I allocate new space, assign the "next" pointer in the existing struct to the new memory, and fork again, now the second client has a pointer to the second struct. The way munmap gets used in the examples, I don't know if you can call munmap on individual allocations (i.e. structs) or if munmap unmaps everything your program has mapped - because I have no pointer to a specific part of allocated memory.

If there was a "shared" version of malloc - it'd be perfect. I'm looking at the POSIX shm methods now...


edit: I'm looking at the "System V" methods according to the HOW-TO linked below (which I have found to be an excellent source for learning about pipes, etc...) The functions include shmget, shmat, shmdt, etc... They appear to be exactly what I'm looking for. My only concern is portability - it would've been nice to have this work on BSD. This looks strikingly similar to what I've seen present elsewhere as "THE posix way", so.... I don't know. thoughts?

http://tldp.org/LDP/lpg/node7.html

Quote:

---

Fair enough - but were they not originally part of UNIX-like systems before POSIX adopted them as their own standard?

---

that's the point: posix is the unix standard, it defines the api and features (some required, some optional, like threads, but there is a lot more) you can expect to find on a standard unix system (which means a lot of them now), this ranges from basic system functions to advanced realtime and distributed computing. This standardization work started at the time a bunch of unices were created from the original version, but with incompatible interfaces (google unix wars).

Quote:

---

My understanding was that you could use the MAP_ANONYMOUS option and map part of the RAM - which would be ideal

---

I never used this linux-specific hack. But you can achieve the same thing, with a few supplementary calls (shm*) and have a portable code, but if it is not of your concern, go ahead with MAP_ANONYMOUS.

Quote:

---

My question then is how do you reference the stored memory? And how do you free it?

---

mmap() returns the address of the 1st mapped page and you undo that with unmap() (deallocate all pages), you can let the kernel decides which location is best or force one (deprecated obviously).

Quote:

---

At this point, I'm not even sure if I can use mmap because I would call mmap to allocate space for the struct, fork, [...] allocated memory.

---

no, mmap() is not an equivalent of malloc(), it allocates pages of memory, not blocks of arbitrary sizes like malloc; those functions are not designed to do the same job. Once you have called mmap() all the pages are allocated and ready, you don't need to malloc() anything anymore, just access the whole range with pointers and store what you want in it.

so to create shared memory, you have at least 3 options mmap(MAP_ANONYMOUS)[linux only], shm_create()+mmap()/munmap()[posix], or shmget/at/dt/ctl()[old ipc]

Ah - PAGED memory - that's the concept I was missing completely. Thanks for explaining that more clearly. I was looking around and it appears that some people have problems with the ipc method because the system places a limit on how much shm you can get. So either way - I'll have problem if my linked list gets too large. If POSIX offers betters portability - I'm jumping on that train.

What I think I'll do is track the number of structs, and allocate another page is it gets necessary (unless this is a huge impossibility, but if I start from scratch and redesign the program, I think I can do it)

Shared mapping are still shared after a call to fork(). Here is the proof:

Code:

---

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <unistd.h>
#include <sys/mman.h>

int main()
{
    void *shmem = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
                      MAP_SHARED | MAP_ANONYMOUS, 0, 0);
    if(fork() == 0)
    {
        /* For demonstration purposes only -- this is not real synchronization */
        sleep(1);
        printf("%s\n", shmem);
        exit(0);
    }
    else
    {
        strcpy(shmem, "Hello, world!");
        wait(NULL);
    }
    return 0;
}

---

Yay! Shared memory between parent and child without silly shm regions or using threads.

EDIT: root4 calls this a "hack," and I suppose it is if you consider all non-portable coding to be a hack. But you can achieve it while still remaining POSIX by using an actual file to back the mapping. Small price to pay, IMHO.

The problem I see myself having, however, is storing an expandable linked list (containing mainly file descriptors, pointers, and a pid) in that kind of structure...

edit: appears I have to use offsets instead of the "next" and "prev" pointers... niice...

Yes, pointers in shared memory is not a good idea - the memory is not always at the same virtual address in all users of the shared memory, so you need some other way to determine the address of some object - either an offset, or index, or some other "relative" measure.

--
Mats

Well that should be a lot of fun! Thanks for all the responses - I guess I have my work cut out for me.

Quote:

---

Originally Posted by sean

Well that should be a lot of fun! Thanks for all the responses - I guess I have my work cut out for me.

---

You can usually control where a memory segment is mapped in your virtual address space. You can do this with mmap() or shmat(). The system should honor your request unless there is a specific reason it can't be mapped there.

The example I gave of mapping an anonymous region then forking will guarantee that child and parent both have the same mapping at the same address.

But matsp's point is valid -- pointers in shared memory regions are tricky at best.

Quote:

---

Originally Posted by brewbuck

Shared mapping are still shared after a call to fork(). Here is the proof:

---

That was great brewbuck; what I was hoping sean would eventually figure out and demonstrate because I've been too lazy to try...

Quote:

---

Originally Posted by sean

The problem I see myself having, however, is storing an expandable linked list (containing mainly file descriptors, pointers, and a pid) in that kind of structure...

edit: appears I have to use offsets instead of the "next" and "prev" pointers... niice...

---

I only noticed this now, but why would you store file descriptors in a shared memory segment? They will only be meaningful to a single process. If you fork(), the same descriptors are shared between parent and child, but this is not true in general. And any descriptors created after the fork() will not be shared, even between parent and child. (You can achieve this sharing with clone() and CLONE_FILES but that's not what fork() does)

Originally the plan was to have the client process open a log file, write data to it as actions were performed, and then close it when it was done. Of course, in the event that one of the client processes locked up and the server process had to kill it, I wanted the server to be able to fclose() the file without data being lost. Of course that was entirely "theoretical" up until this point, since I hadn't even got shared memory to work - so if files won't work between two processes, I'll have to find another work-around to that.

Here's an even crazier idea (although I can't express how hopeful I am for you; I even lit little candles today that say "sean--mmap" in cursive on them):

Start a local socket server (PF_LOCAL) and have all the children connect to it as clients. Everyone starts with a copy of the data relevant to them, then work out some signals for making sure everyone stays abreast of what they need to, via the cleverness of the parent. Once you have a few general functions worked out, "the local socket" will work as fast as the front side bus (or whatever that thing is called).

I mean, correct me if I'm wrong but there must be a gizzillion super-complex database setups that work continuously over the internet, which means nothing but what passes thru a socket.

I'm gonna have to start personally thinking of shared memory the same way as goto: if you think you need it, your program might need redesigning...

So I have an "input server", a number of "client processes", and whenever there's a client process created, there's an "output server" piped to it (with a named FIFO), which handles the logging and the outputting. So if the "client process" locks up, the input server knows the name of the FIFO, and can also send a specific message to the output server that closes the file and finishes up.

So the input server just keeps it's own linked list containing the process ID of the client and the name of the FIFO. Then I don't need shared memory - and I only have one process accessing the logs, panels and windows.

There are cases where shared memory is very much useful - a particular case is some sort of "database" where the data is rarely or never changed, that is shared between applications. A typical example would be a font-server. Font's aren't exactly changing rapidly (you may find that different sizes and typefaces are being used at different times, but the names of fonts and the ones available in a particular system tends to be pretty constant over time).

One could also consider "sharing one-writer many readers data", e.g. a multiplayer game could have a shared memory section where each player has a "slot" in a shared memory to update their own players position and actions with others. All players can read other players, but each player will only ever modify their own data (this may include data that informs the other players to update their player, e.g. transfer of ownership of items, fighting that results in damage, etc).

But for your average "I want this data to be transferred from here to there", then shared memory is probably not the right thing to do.

--
Mats

Yeah - when I took a good look at what HAD to be shared - it was WINDOW *'s and PANEL *'s for ncurses (plus that file pointer) - which if I'm not mistaken points to data that's been malloc'ed by ncurses. So that would've been a big problem..

So thanks for the responses - my apologies to MK 27 who was waiting for me to get mmap working!