When is a pipe writer done?
I'm trying to figure out the best way for the writer on a pipe to let the reader know that its done writing.
Background:
I have a process running that reads commands on STDIN, interprets the commands, and writes results to STDOUT. This process never exits. Furthermore, the process could take hours to execute a command. Also, the process may send zero or more lines to STDOUT when it's finished with a command. I want to send a command to this process, wait for a response, and then send another command, wait for a response, and so on. The problem is I don't know how long to wait for a response. I can use select() with a timeout value, but if I give it a timeout value that's long enough to accommodate a long-running command, select() will waste time waiting on a short-running command.
Here are some ways I've thought of to solve the problem:
1) Look for a key in the result that got written to STDOUT which indicates that the process is finished writing. Problem: the result contains arbitrary data. If the result happens to contain the key in its data, I would have to escape it somehow.
2) Send the number of bytes that will be transmitted over STDOUT before sending the result. Problem: The process I'm talking about is an existing program (it's been around since 1984, actually) and customers don't want to see byte counts in the standard output stream.
3) Send SIGUSR1 or SIGUSR2 when the process is done writing a result to STDOUT. Problem: If the process that invoked (i.e. a shell) the program in question doesn't handle SIGUSR1 or SIGUSR2, it'll get killed.
4) Use Sun RPC instead of STDOUT to send results. This could work, though I don't really like working with Sun RPC (it's difficult to integrate it into an existing C++ app).
5) Use a C++ XML-RPC implementation for IPC. Problem: While this is easier than Sun RPC, our legal department has to qualify any open source shared library we use (it takes months for them to do this). In addition, they have to re-qualify it for every new version we use (another several months).
6) Make a flow control socket that sends a message when a result is finished. So far this is the best thing I can think of.
7) Use some other for of IPC such as queues, shared memory, etc. I haven't explored this yet.
Do you guys have any advice for the best way to solve my problem?
Sometimes problems are easier to explain in code. Here's an example program that illustrates what I'm dealing with. There's a child that writes a random number of messages (with random delays in between them) to a pipe, and the parent somehow must determine when the child is done writing. Presently, it fails miserably because the select() timeout is too short.
Code:
#include <iostream>
#include <string>
#include <cstdlib>
#include <cassert>
#include <sys/wait.h>
// The maximum number of seconds between the child's messages
#define MAX_WAIT 2
using namespace std;
// Easy way to get a random number
unsigned int rand_num(unsigned int limit) {
assert(limit);
time_t t = time(NULL);
unsigned int seed = t;
unsigned int div = RAND_MAX / limit;
return (rand_r(&seed) / (div == 0 ? 1 : div));
}
int main() {
int pfds[2];
pipe(pfds);
// Fork off the child
pid_t pid = fork();
if (pid == 0)
{
// Setup the pipe
dup2(pfds[1], STDOUT_FILENO);
// The child prints a random number of messages with a random wait period between messages
cerr << "Child starting to send messages" << endl;
for (unsigned int i = 0; i < rand_num(8); ++i) {
sleep(rand_num(8)); // This is sometimes larger than MAX_WAIT!!
cout << "Here's line " << i << endl;
cerr << "The child sent line " << i << endl;
}
cerr << "Child stopped sending messages" << endl;
// Wait for another 10 seconds to simulate a long-running server process
sleep(10);
exit(0);
}
// Read the output
char buf[2048];
struct timeval tv;
fd_set read_set;
ssize_t bytes_read;
int retval;
do
{
tv.tv_sec = MAX_WAIT;
tv.tv_usec = 0;
FD_ZERO(&read_set);
FD_SET(pfds[0], &read_set);
retval = select(pfds[0] + 1, &read_set, NULL, NULL, &tv);
if (retval <= 0)
break;
if (FD_ISSET(pfds[0], &read_set))
{
// Read and print what the child sent
bytes_read = read(pfds[0], buf, sizeof(buf) - 1);
buf[bytes_read] = '\0';
cerr << buf;
}
} while (retval > 0);
// Wait for the child to exit
waitpid(pid, NULL, 0);
return 0;
}
