Check if file is opened for reading (unix)

**IsmAvatar2** · 06-24-2009

My program is to scan a directory for new files, and then forward them along. However, I cannot forward a file until it is complete, and as far as I'm aware, the only way to tell if the file is complete is if no other process is accessing it.

I tried checking for write locks using fcntl, but either I'm using it wrong or the process doesn't put a write lock on the file.

So basically, I'm looking for something similar to fuser <file> or lsof <file>. What command/library am I looking for here? Don't say "exec", because there's no sense getting a full wheel when all you need is a spoke.

Thanks in advance,
-IsmAvatar

(If you want me to post the code I tried to check for locks, just say so, but I figured there's nothing further to be said there)

**MK27** · 06-24-2009

Originally Posted by IsmAvatar2

My program is to scan a directory for new files, and then forward them along. However, I cannot forward a file until it is complete, and as far as I'm aware, the only way to tell if the file is complete is if no other process is accessing it.

Since you do not need to open a file when scanning a directory, it is hard to tell what you mean here -- are you worried about other processes, not part of your program running on the system that could be accessing the file?

**IsmAvatar2** · 06-24-2009

Yes. My program is to scan the directory for *new*, *completed* files created by another process.

Program 1 (not in my control) opens file "out1.txt" for writing.
P1 writes some bytes to file.
P1 closes file.
P1 opens file "out2.txt" for writing.
P1 writes some bytes to file, but doesn't finish.
Program 2 (my program) opens directory, and finds "out1.txt"
P2 finds out that the file is closed and completed, so transmits its information.
P2 next finds "out2.txt"
P2 finds out that the file is still open, so it ignores it.
P2 finds no more files, closes directory, and terminates (until it's run again)
P1 finishes writing "out2.txt" and closes it.

**MK27** · 06-24-2009

Originally Posted by IsmAvatar2

Yes. My program is to scan the directory for *new*, *completed* files created by another process.

I've never had to deal with this so I won't be much help -- my first thought was to try something with fcntl(), but I see you've already done that.

Most people would consider this a task for a shell script, where you could use lsof, etc "natively". My hunch is you will have to use a system/exec/popen to access the shell anyway.

**IsmAvatar2** · 06-24-2009

My concern is that my program may not have access to lsof or fuser, as they oftentimes require root privileges. I was also hoping that I might just be able to use their lower level routines without having to use the whole program and parse the output. But of course skimming through lsof's source code can take hours.

**itCbitC** · 06-24-2009

Post the code snippet that calls fcntl(); P2 needs to open "out2.txt" for writing in exclusive mode and methinks you are trying to get a shared lock instead.

**Zlatko** · 06-24-2009

Hello.
You probably didn't see the bottom of the fuser man page. There's an fuser function in man section 2. I never used it myself.

Edit:
Sorry, there is no fuser function on linux, I was looking on my Tru64 system. On linux, check /proc/pid/fd for a list of files in use.

**IsmAvatar2** · 06-24-2009

itCbitC:

Code:

int main() {
 char *fn = "/tmp/test.txt"; //the test file
 FILE *F = fopen(fn,"w"); //this simulates another process writing to a file but not closing it yet

 //We now attempt to detect that the file is still open
 int fd = open(fn,O_RDONLY);
 struct flock fl;
  fl.l_type   = F_WRLCK;  /* F_RDLCK, F_WRLCK, F_UNLCK */
  fl.l_whence = SEEK_SET; /* SEEK_SET, SEEK_CUR, SEEK_END */
  fl.l_start  = 0; /* Offset from l_whence */
  fl.l_len    = 0; /* length, 0 = to EOF */
  fl.l_pid    = getpid();

 printf("%d %d %d %d %d\n",fl.l_type,fl.l_whence,fl.l_start,fl.l_len,fl.l_pid);
 //1 0 0 0 19865
 int ret = fcntl(fd,F_GETLK,&fl);
 printf("%d %d %d %d %d\n",fl.l_type,fl.l_whence,fl.l_start,fl.l_len,fl.l_pid);
 //2 0 0 0 19865

 sleep(3); //this gives me time to manually run fuser/lsof to observe that the file is indeed open

 //finalize
 fclose(F);
 return 0;
}

According to the fcntl documentation, if the lock is not detected, it will keep the struct the same but replace the mode with UNLCK, which is precisely what it does here.

Zlatko, are you referring to this?

fuser cannot report on any processes that it doesn’t have permission to
look at the file descriptor table for. The most common time this prob-
lem occurs is when looking for TCP or UDP sockets when running fuser as
a non-root user. In this case fuser will report no access

Because "man 2 fuser" doesn't contain anything: "No entry for fuser in section 2 of the manual"
Regardless, when I try to run it without root, I get the following:
$ fuser
bash: fuser: command not found
$

**Zlatko** · 06-24-2009

Originally Posted by IsmAvatar2

Zlatko, are you referring to this?

Because "man 2 fuser" doesn't contain anything: "No entry for fuser in section 2 of the manual"
Regardless, when I try to run it without root, I get the following:
$ fuser
bash: fuser: command not found
$

Yes, what I meant in my edit was that there appears to be no fuser C function on linux. I assume you're using linux. I find that I can look at the /proc/pid/fd directory for a process I own, but not for those of other users. If the process creating for the file is owned by the same user as the process waiting for it, the one waiting should have no trouble reading the fd directory.

If you have access to the source of the creator, then a file lock like lockf() is probably easier.

**itCbitC** · 06-24-2009

Request for exclusive lock will fail if file descriptor is not opened with write access.
Instead of trying to get info about a lock on the file try to set a lock on it.
The return value of fcntl() indicates if a lock already exists on that file.

Code:

struct flock fl;
int fd = open(fn, O_RDWR);
fl.l_type   = F_WRLCK;  /* F_RDLCK, F_WRLCK, F_UNLCK */
fl.l_whence = SEEK_SET; /* SEEK_SET, SEEK_CUR, SEEK_END */
fl.l_start  = 0; /* Offset from l_whence */
fl.l_len    = 0; /* length, 0 = to EOF */

if (fcntl(fd, F_SETLK, &fl) == -1)
    printf("%s\n", errno == EACCESS||errno == EAGAIN ? "In use by another process":"another error occurred");
else
    /* lock granted so do I/O on the file */

**IsmAvatar2** · 06-24-2009

Zlatko: I cannot edit the source of the process modifying these files. /proc/pid/fd sounds like a good idea, but how do I know which ones refer to the file that I'm interested in?

itCbitC: In order to better test your idea, I created 2 processes.
The first one opens a file for writing, waits 5 seconds, and then closes it and terminates.
The second one simply attempts to put a write lock on the same file (as per your code), and reports the return value and any changes to the struct.

I ran the first one, and during those 5 seconds, I ran the second one. The second one reports return value 0 (lock granted) and no change to the structure (mode: 1, everything else is 0).
Unless I'm mistaken, this confirms that the file may be opened for writing without having a write lock on it, which further confirms that I need another way to detect if the file is opened for writing or not.

**Zlatko** · 06-24-2009

Originally Posted by IsmAvatar2

Zlatko: I cannot edit the source of the process modifying these files. /proc/pid/fd sounds like a good idea, but how do I know which ones refer to the file that I'm interested in?

Well, you need to do some experimenting. Here is how I would start.
The /proc/pid/fd directory has links to all the files in use by the program.
Assuming you know the pid of the producer, try going through the /proc/pid/fd directory with the opendir/readdir/closedir functions. Each dirent, as returned by readdir, will have a filename. That filename might be the name of the link or it might be the name of the actual file being used. I'm not sure. However, try checking the dirent.d_ino number against the inode number of the file you're interested in and see if they match. You can get the inode of the file you're interested in with the stat function.

**brewbuck** · 06-24-2009

Originally Posted by IsmAvatar2

My program is to scan a directory for new files, and then forward them along. However, I cannot forward a file until it is complete, and as far as I'm aware, the only way to tell if the file is complete is if no other process is accessing it.

Whatever program is depositing the files in this directory is broken. The files should be created elsewhere, then atomically moved into the depository using the rename() syscall. At the moment the file becomes visible in the directory it is already complete.

This is a fairly common problem which is solved in the above manner, not by using gross hacks.

**Zlatko** · 06-24-2009

Since it is not possible to modify the producer program, you may need to resort to unsavoury methods. Here is a quick and dirty program that might help. I'll leave the error checking and variable naming up to you.

Code:

#include <sys/types.h>
#include <dirent.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>

int main(void)
{
	struct stat statbuf;
	DIR* dir = opendir("/proc/14867/fd");
	struct dirent* ent;

	// foo.txt is the file that pid 14867 is producing
	if (stat("/usr/home/foo.txt",  &statbuf) < 0)
	{
		perror("Cannot stat");
		exit(1);
	}

	while ((ent = readdir(dir)) != NULL)
	{
		char buf[128];
		struct stat statbuf2;
		sprintf(buf, "/proc/14867/fd/%s", ent->d_name);
		stat(buf, &statbuf2);

		if (statbuf2.st_ino == statbuf.st_ino)
		{
			// This means that pid 14867 still has foo.txt open
			fprintf(stderr, "YOU GOT IT\n");
			break;
		}
	}

	closedir(dir);
}

**IsmAvatar2** · 06-24-2009

brewbuck: The program that is depositing the files is ftp. If you want to say FTP is broken, be my guest.

Zlatko: Wouldn't it be more efficient to just get the d_ino from the dirent, rather than trying to stat it from the filename? Or is there a reason you chose to do it this way?

Thread: Check if file is opened for reading (unix)

Thread Tools

Search Thread

Display

Check if file is opened for writing (unix)

Similar Threads

C Delete opened file

Formatting a text file...

Need Help Fixing My C Program. Deals with File I/O

spell check in C using a dictionary file

C programming with Unix functions for manipulation of a virtual file system

Tags for this Thread