BigAngryDog

Hi there,

I'm writing a class which can be used to read/write files. I want to implement file locking, and wondering how best to achieve it in Linux.

I understand that to lock files using fcntrl(), I must open them with 'open()', rather than 'fopen()'. However, fopen() implements read/write buffering, whereas open() usually does not.

As I see it, I have two options:

1. When the class opens a file, it also created a 'lock file' beside it. Presumably, the main file is opened using fopen(), whereas the lock file is created with open() and then locked. When opening an existing file, my class should therefore check for an existing lock file to determine whether the file can be opened in the required mode. I understand that this is a pretty standard approach.

Alternatively, I could implement the following approach and was wondering what people think:

2. Use the open() call to open/create a single main file only (i.e. no extra lock file), and fcntrl() to lock it. However, because read/writes are unlikely to be buffered, I would have to implement my own buffering within the workings of my class.

Is there any overriding reason why I should go with option 1 over option 2?

Thanks

bithub

open() will do buffering as long as you don't pass it the O_DIRECT flag.

__________________
bit∙hub [bit-huhb] n. A source and destination for information.

cyberfish

If you implement locking yourself, you will have to make sure checking and creating a lock file is an atomic operation.

Otherwise nasty things can still happen (if you start 2 instances at the same time, and instance A opens a file, and before A had the chance to create the lock file, instance B also checks, and successfully opens that file...)

BigAngryDog

Hey thanks for response.

>open() will do buffering as long as you don't pass it the O_DIRECT flag.

That's interesting. I admit to being a little confused on the issue. There's an interesting discussion on the topic here:

fopen() vs. open() - C / C++ answers

A couple of key quotes are:
and


OK, assuming open() gives the desired level of buffering, what's the advantage of using a separate lock file?

root4

read/write() do buffering (the kernel maintains caches and so on for files) that's why you have to sync() your file systemss before turning off your machine, however it costs you a syscall contrary to fread/fwrite() (which do the syscall periodically). i don't see what bothers you about fopen() over open(), you can still use fcntl() or even flock() while having open your file with fopen(), just get back the fd with fileno().

Ronix

If you want to use FILE objects and your platform supports them, you can use flockfile / funlockfile since they're likely to do the same thing you're trying to implement.

BigAngryDog

>i don't see what bothers you about fopen() over open(), you can still use fcntl() or even flock() while having open your file with fopen(), just get back the fd with fileno().

Because my past attempts at this failed to work. But I'm starting to realise that, perhaps, it is my understanding of file locking on Linux that is inadequate.

On Windows, it is possible to exclusively open a file (i.e. preventing other processes from opening it also). It was this I was expecting to achieve on Linux.

On Linux, however, I'm currently reading that file locking is advisory by default. Meaning, I guess, that if I lock the contents of a file, this will not prevent another process from opening that file in read-write mode. Rather, the process must specifically check for the lock status.

Is this correct?

I'm not just trying to get some code working here, rather I'm trying to understand the whole approach to file locking on Linux.

With advisory locking and the need for cooperating processes, I guess the use of separate locking files makes a little more sense. Any thoughts?

MK27

Hacking Linux Exposed

Looks like you have to mount the filesystem with a switch to indicate to the kernel to permit this.

__________________

Accuracy and integrity mean nothing if you don't make it past the censors...PYTHAGORAS

root4

that's interesting, i was not aware of the 'mand' mount option. On the other side, there is a relevant file in the Documentation/filesystems folder of the kernel source (mandatory-locking.txt), it looks a bit out of date so i don't know if the problems discussed are still valid. Some other problems are also discussed on the wikipedia 'File locking' page (unlink). As long as the kernel doesn't enforce the mandatory locking policy, it's hard to do anything; but linux is only one unix flavor and it seems others can do it (solaris, hp-ux...).

and at last this is what posix says:

MK27

I can kind of understand why. It is hard to see why a process needs to lock a file. If the process created it, why would it require a lock, since no other process will "guess" the name and start doing crazy things for no reason to a file it has nothing to do with, and if the process has a conflict with itself, that is due to sloppy programming. If it did not create the file, it should not just be allowed to lock bunches of files, even if it is allowed to delete them -- this is bound to be more of a security risk than the opposite, since other processes that use the file will probably be able to deal with one that is missing easier than one which is locked, and you could tie up tons of resources this way. If the process hangs, your lock is stuck there.

A lot can be done with owner/group permissions. Eg, you can create a new user for the process, but put it in an exising group, and use that to separate read vs write access for self vs. other processes. You can also setuid() getuid() yourself repeatedly.

Of course, a privileged process can override that, but that is the point. You don't have the freedom to do whatever anywhere with the filesystem and you shouldn't have it, either; you should not be allowed to go so far in making your app bulletproof that you jeopardize the system itself.

BigAngryDog says this is for what I presume to be a general purpose utility class. So it's a nice thought, but I'll wager no one can actually come up with a real use for this (ie, something that can't be done another way) that isn't irrationally paranoid (ie, protects against imaginary threats).

__________________

Accuracy and integrity mean nothing if you don't make it past the censors...PYTHAGORAS