Thread: Lot of WRITE calls in parallel in C WRITE test code

  1. #1
    Registered User
    Join Date
    Apr 2020
    Posts
    2

    Lot of WRITE calls in parallel in C WRITE test code

    Hi all ,

    I am writing READ and WRITE test codes in C to read and write the data to a storage cluster. So basically the test code configures the cluster, creates an IO context and begins reading and writing data. Currently I can write data synchronoulsy.

    Now I want to test high number of writes. Which means there should be a lot of write calls in parallel in my C WRITE test code. I am struggling with it. How can I improve the following code to have high number of WRITES ? . Should I use a for loop? PLEASE help me on this . Following is the example WRITE test code:

    Code:
    #include<stdio.h>
    #include<stdlib.h>
    #include<string.h>
    #include<rados/librados.h>
    
    int main(int argc, const char *argv[])
    {
    
      /* Declare the cluster handle and required arguments. */
      rados_t cluster;
      //char cluster_name[] = "ceph";
      char cluster_name[] = "test cluster";
      char user_name[] = "client.admin";
      uint64_t flags;
    
      /* Initialize the cluster handle with the "ceph" cluster name and the "client.admin" user */
      int err;
      err = rados_create2(&cluster, cluster_name, user_name, flags);
    
      if (err < 0) {
        fprintf(stderr, "%s: Couldn't create the cluster handle! %s\n", argv[0],
                strerror(-err));
        exit(EXIT_FAILURE);
      } else {
        printf("\nCreated a cluster handle.\n");
      }
    
      /* Read a configuration file to configure the cluster handle. */
    
      err = rados_conf_read_file(cluster, "/etc/ceph/neo-neo.conf");
      if (err < 0) {
        fprintf(stderr, "%s: cannot read config file: %s\n", argv[0],
                strerror(-err));
        exit(EXIT_FAILURE);
      } else {
        printf("\nRead the config file.\n");
      }
    
      /* Read command line arguments */
      err = rados_conf_parse_argv(cluster, argc, argv);
      if (err < 0) {
        fprintf(stderr, "%s: cannot parse command line arguments: %s\n", argv[0],
                strerror(-err));
        exit(EXIT_FAILURE);
      } else {
        printf("\nRead the command line arguments.\n");
      }
    
      /* Connect to the cluster */
      err = rados_connect(cluster);
      if (err < 0) {
        fprintf(stderr, "%s: cannot connect to cluster: %s\n", argv[0],
                strerror(-err));
        exit(EXIT_FAILURE);
      } else {
        printf("\nConnected to the cluster.\n");
      }
    
      /* First declare an I/O Context */
    
      rados_ioctx_t io;
      //char *poolname = "data";
      char *poolname = "neo";
    
      err = rados_ioctx_create(cluster, poolname, &io);
      if (err < 0) {
        fprintf(stderr, "%s: cannot open rados pool %s: %s\n", argv[0], poolname,
                strerror(-err));
        rados_shutdown(cluster);
        exit(EXIT_FAILURE);
      } else {
        printf("\nCreated I/O context.\n");
      }
    
      /* Write data to the cluster synchronously. */
      err = rados_write(io, "test", "Test Message!", 16, 0);
      if (err < 0) {
        fprintf(stderr, "%s: Cannot write object \"neo-obj\" to pool %s: %s\n",
                argv[0], poolname, strerror(-err));
        rados_ioctx_destroy(io);
        rados_shutdown(cluster);
        exit(1);
      } else {
        printf("\nWrote \"Test\" to object \"neo-obj\".\n");
      }
    
      char xattr[] = "en_US";
      err = rados_setxattr(io, "neo-obj", "lang", xattr, 5);
      if (err < 0) {
        fprintf(stderr, "%s: Cannot write xattr to pool %s: %s\n", argv[0],
                poolname, strerror(-err));
        rados_ioctx_destroy(io);
        rados_shutdown(cluster);
        exit(1);
      } else {
        printf("\nWrote \"en_US\" to xattr \"lang\" for object \"neo-obj\".\n");
      }
    
    }
    Last edited by Salem; 04-27-2020 at 08:07 AM. Reason: Removed crayola

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    38,165
    As a matter of courtesy, you should tell everyone all the places you post.
    Parallel WRITE calls in parallel in C WR - C++ Forum

    So that people don't waste their time telling you things you've already found out on some other forum.

    After a bit of searching, I found Librados (C) — Ceph Documentation
    Is this the thing you're using?

    If it is, then scroll down to "Asynchronous IO".
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  3. #3
    Registered User
    Join Date
    Apr 2020
    Posts
    2
    I have post in other forums because not everyone is on all forums. And if I find the right answer, I can always comment in the thread that I have got the right answer, hence no more answers needed. If you haven't found that comment by me then you are not at all wasting your time because it means I have not found my answer . But your point well taken. In future I will mention it.

    Having said that, thanks to you for pointing out "Asynchronous IO". I had not read that. Yes I am using Librados API as it is also clear from my example test code. The API mentions rados_aio_write to write data to an object asynchronously. It basically queues the write. I have a question. Lets say if I have to do ten thousand WRITES. Then how can I use rados_aio_write?

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 4
    Last Post: 06-25-2018, 11:51 PM
  2. write code mmc test using linux
    By quanghuyen in forum Linux Programming
    Replies: 3
    Last Post: 12-08-2011, 08:15 AM
  3. how can I write code to test a[100];
    By zcrself in forum C Programming
    Replies: 23
    Last Post: 12-26-2010, 07:25 PM
  4. how can I write code to test a[100];
    By zcrself in forum C Programming
    Replies: 1
    Last Post: 12-22-2010, 12:15 AM
  5. write and read system calls
    By nacho4d in forum C Programming
    Replies: 4
    Last Post: 01-28-2008, 10:59 AM

Tags for this Thread