Thread: Lot of WRITE calls in parallel in C WRITE test code

Hi all ,

I am writing READ and WRITE test codes in C to read and write the data to a storage cluster. So basically the test code configures the cluster, creates an IO context and begins reading and writing data. Currently I can write data synchronoulsy.

Now I want to test high number of writes. Which means there should be a lot of write calls in parallel in my C WRITE test code. I am struggling with it. How can I improve the following code to have high number of WRITES ? . Should I use a for loop? PLEASE help me on this . Following is the example WRITE test code:

Code:

#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<rados/librados.h>

int main(int argc, const char *argv[])
{

  /* Declare the cluster handle and required arguments. */
  rados_t cluster;
  //char cluster_name[] = "ceph";
  char cluster_name[] = "test cluster";
  char user_name[] = "client.admin";
  uint64_t flags;

  /* Initialize the cluster handle with the "ceph" cluster name and the "client.admin" user */
  int err;
  err = rados_create2(&cluster, cluster_name, user_name, flags);

  if (err < 0) {
    fprintf(stderr, "%s: Couldn't create the cluster handle! %s\n", argv[0],
            strerror(-err));
    exit(EXIT_FAILURE);
  } else {
    printf("\nCreated a cluster handle.\n");
  }

  /* Read a configuration file to configure the cluster handle. */

  err = rados_conf_read_file(cluster, "/etc/ceph/neo-neo.conf");
  if (err < 0) {
    fprintf(stderr, "%s: cannot read config file: %s\n", argv[0],
            strerror(-err));
    exit(EXIT_FAILURE);
  } else {
    printf("\nRead the config file.\n");
  }

  /* Read command line arguments */
  err = rados_conf_parse_argv(cluster, argc, argv);
  if (err < 0) {
    fprintf(stderr, "%s: cannot parse command line arguments: %s\n", argv[0],
            strerror(-err));
    exit(EXIT_FAILURE);
  } else {
    printf("\nRead the command line arguments.\n");
  }

  /* Connect to the cluster */
  err = rados_connect(cluster);
  if (err < 0) {
    fprintf(stderr, "%s: cannot connect to cluster: %s\n", argv[0],
            strerror(-err));
    exit(EXIT_FAILURE);
  } else {
    printf("\nConnected to the cluster.\n");
  }

  /* First declare an I/O Context */

  rados_ioctx_t io;
  //char *poolname = "data";
  char *poolname = "neo";

  err = rados_ioctx_create(cluster, poolname, &io);
  if (err < 0) {
    fprintf(stderr, "%s: cannot open rados pool %s: %s\n", argv[0], poolname,
            strerror(-err));
    rados_shutdown(cluster);
    exit(EXIT_FAILURE);
  } else {
    printf("\nCreated I/O context.\n");
  }

  /* Write data to the cluster synchronously. */
  err = rados_write(io, "test", "Test Message!", 16, 0);
  if (err < 0) {
    fprintf(stderr, "%s: Cannot write object \"neo-obj\" to pool %s: %s\n",
            argv[0], poolname, strerror(-err));
    rados_ioctx_destroy(io);
    rados_shutdown(cluster);
    exit(1);
  } else {
    printf("\nWrote \"Test\" to object \"neo-obj\".\n");
  }

  char xattr[] = "en_US";
  err = rados_setxattr(io, "neo-obj", "lang", xattr, 5);
  if (err < 0) {
    fprintf(stderr, "%s: Cannot write xattr to pool %s: %s\n", argv[0],
            poolname, strerror(-err));
    rados_ioctx_destroy(io);
    rados_shutdown(cluster);
    exit(1);
  } else {
    printf("\nWrote \"en_US\" to xattr \"lang\" for object \"neo-obj\".\n");
  }

}

As a matter of courtesy, you should tell everyone all the places you post.
Parallel WRITE calls in parallel in C WR - C++ Forum

So that people don't waste their time telling you things you've already found out on some other forum.

After a bit of searching, I found Librados (C) — Ceph Documentation
Is this the thing you're using?

If it is, then scroll down to "Asynchronous IO".