How to synchronize file texts with POSIX threads in C

[Yuri21]

Well, I'm starting on POSIX threads, and I have several doubts, one of them is how to synchronize the texts of a file, for example I want several threads to be able to read lines from the file without repeating, I thought about using the dup, to stay duplicating the descriptors, but it doesn't seem like the best way, or if it works, if I use the mutex, I would have to keep blocking the reading or blocking the opening of the file

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

/*variable fp, to share the opening of the file and variable c,
 to share the reading of the data,
 the program has a race condition problem*/


FILE *fp;
char c;

void *func(void) {  
          fp = fopen("file.txt", 'r");
          if(!fp) {
              perror("DEBUG: ");
              exit(EXIT_FAILURE);
          }

          while((c = fgetc(fp)) != EOF) 
                  putchar(c);
  
         return NULL;
}

int main(void) {

     pthread_t threads[5];

     for(int i=0; i < 5; i++)
        pthread_create(&threads[i], NULL, (void *)func, NULL);

     for(int i=0; i < 5; i++) 
         pthread_join(threads[i], NULL);

     fclose(fp);

     return 0;
}

[Salem]

Well a file is a single and sequential resource, so on some level, you're going to have to make all the threads wait their turn, while just ONE thread does what it needs to do.

[Yuri21]

Well a file is a single and sequential resource, so on some level, you're going to have to make all the threads wait their turn, while just ONE thread does what it needs to do.

I didn't really think about it, the file being sequential, but just for testing, I want each thread to read a line, and block all other threads, while reading, as soon as I finish reading that line, another thread is released to read another line and the other threads will be blocked again and so on, I thought about using pthread_cond_wait to do this, what do you think?

[Salem]

That seems like an idea worth looking into.

[Yuri21]

That seems like an idea worth looking into.

I decided to synchronize using a condition variable, up to and all right, now I don't understand why the first text of a file is being read by two threads, which generates the race conditions problem, besides that I had to put the process from the main thread to sleep, stops the time to start another thread and I still have to put a thread on hold which depends exclusively if the amount of line in the file is divisible by 2, I made a debugger with helgrind, but I couldn't find the error

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
 
/*variable fp, to share the opening of the file and variable c,
 to share the reading of the data,
 the program has a race condition problem*/
 
 
FILE *fp;

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;


void *func(void) {
      
          char c;

          fp = fopen("file.txt", "r");
          if(!fp) {
              perror("DEBUG: ");
              exit(EXIT_FAILURE);
          }
 
          pthread_mutex_lock(&mutex);
          while((c = fgetc(fp)) != EOF) {
               printf("ID THREAD = %zu CHARACTER= %c\n", pthread_self(), c);
               if(c == '\n') {
                  pthread_cond_wait(&cond, &mutex);
                } else {
                  pthread_cond_signal(&cond);
              }
          }
          pthread_mutex_unlock(&mutex);
    
          
}
 
int main(void) {
 
     pthread_t thread1, thread2;
     
     pthread_create(&thread1, NULL, (void *)func, NULL);
     sleep(1);
     pthread_create(&thread2, NULL, (void *)func, NULL);

     pthread_join(thread2, NULL);
     fclose(fp);
 
     return 0;
}

$ cat file.txt
test1
test2
test3
test4
test5
test6

[christop]

One issue I see is that you're opening the file in each thread (and then saving the file handle in a shared variable, fp). It should be opened only once, before the first thread starts.

Another issue is that you're saving the return value of fgetc() in a char. fgetc() returns an int, and it should be saved in an int.

[Yuri21]

Well, I would have to open the stream on the main thread, but the texts would not be synchronized, and regarding fgetc, it reads a char and converts it to an unsigned integer which can be implicitly converted to char again, so no problem .

[christop]

Well, I would have to open the stream on the main thread, but the texts would not be synchronized

Why would the texts not be synchronized? If you properly use a mutex in the threads to read lines from the stream, the threads will be synchronized.

This is roughly how I'd do it:

1. In the main thread, open the stream once.
2. In the main thread, start your threads.
3. In each thread, attempt to lock the mutex. If the thread obtained the mutex, read a line from the stream and then unlock the mutex. Repeat this step until EOF is reached.
4. In the main thread, join all threads.
5. In the main thread, close the stream.

(This serializes all of the threads, but that's the nature of a serial stream of data, as Salem mentioned.)

regarding fgetc, it reads a char and converts it to an unsigned integer which can be implicitly converted to char again, so no problem .

The problems are that a) if char is unsigned by default, it'll never compare equal to EOF, and b) if char is signed by default, you won't be able to handle bytes with the value 0xff (255) because that compares equal to EOF. The standard allows a compiler to treat char as either signed or unsigned by default, and either way there's a problem with saving the return value of fgetc() in a char. Why do the wrong thing when doing the right thing is easier? ("int" is only 3 keystrokes vs 4 keystrokes for "char".)

[Yuri21]

Why would the texts not be synchronized? If you properly use a mutex in the threads to read lines from the stream, the threads will be synchronized.

This is roughly how I'd do it:

1. In the main thread, open the stream once.
2. In the main thread, start your threads.
3. In each thread, attempt to lock the mutex. If the thread obtained the mutex, read a line from the stream and then unlock the mutex. Repeat this step until EOF is reached.
4. In the main thread, join all threads.
5. In the main thread, close the stream.

(This serializes all of the threads, but that's the nature of a serial stream of data, as Salem mentioned.)


The problems are that a) if char is unsigned by default, it'll never compare equal to EOF, and b) if char is signed by default, you won't be able to handle bytes with the value 0xff (255) because that compares equal to EOF. The standard allows a compiler to treat char as either signed or unsigned by default, and either way there's a problem with saving the return value of fgetc() in a char. Why do the wrong thing when doing the right thing is easier? ("int" is only 3 keystrokes vs 4 keystrokes for "char".)

I tested it the way you said, opened the fp stream in the main thread of the process, but it entered an infinite loop, I will debug more calmly to understand what happened, and regarding the char type, it can really be signed or unsigned by default, thanks for the tips.

[christop]

Post your latest code and we can help.

[Salem]

pthread_cond_signal(3) - Linux man page

Multiple Awakenings by Condition Signal

On a multi-processor, it may be impossible for an implementation of pthread_cond_signal() to avoid the unblocking of more than one thread blocked on a condition variable. For example, consider the following partial implementation of pthread_cond_wait() and pthread_cond_signal(), executed by two threads in the order given. One thread is trying to wait on the condition variable, another is concurrently executing pthread_cond_signal(), while a third thread is already waiting.

You need to spend a few days reading all the material available on the pthread API, then come up with some design that matches your requirement.

Just hacking bits of code will not make it work.

[Yuri21]

Colleague, I didn't hack any code, I got to read the documentation for some functions of the api pthread, but in my view, the program was working normally, I did some tests here and saw that the problem is in reading the stream files, but I really have to study a lot more about threads.

[Yuri21]

I managed to fix the problem, the christop tips
about opening the flow on the main thread, it really was the best solution, the first time I tested it went into an infinite loop, I don't remember why, I posted this question on the stackoverflow and they told me the same thing, I did the test again and everything went well, finally thanks to everyone who helped me.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>


pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;

void *func(void *param) {

      char c;

      FILE *fp = param;
      pthread_mutex_lock(&mutex);
      while((c = fgetc(fp)) != EOF) {
        printf("TH = %zu CHARACTER = %c\n", pthread_self(), c);
        if(c == '\n') {
              pthread_cond_wait(&cond, &mutex);
            } else {
              pthread_cond_signal(&cond);
          }
      }
      pthread_mutex_unlock(&mutex);

      pthread_exit(NULL);
 }

 int main(void) {

    pthread_t thread1, thread2;

      FILE *fp = fopen("file.txt", "r");
      if(!fp) {
          perror("DEBUG: ");
          exit(EXIT_FAILURE);
      }

    pthread_create(&thread1, NULL, func, fp);
    sleep(1);
    pthread_create(&thread2, NULL, func, fp);
    pthread_join(thread2, NULL);

    fclose(fp);

    return 0;
}

[Hodor]

Broken on my Raspberry Pi (char defaults to unsigned). Change line 12 to int c;

[Yuri21]

Oh yeah, I forgot to change it, thank you