Like Tree2Likes
  • 2 Post By anduril462

Reading data from a file and binning it

This is a discussion on Reading data from a file and binning it within the C Programming forums, part of the General Programming Boards category; Hi There I wondered if you could possibly help me I have a data file that I want to read ...

  1. #1
    Registered User
    Join Date
    Apr 2010
    Posts
    20

    Reading data from a file and binning it

    Hi There
    I wondered if you could possibly help me
    I have a data file that I want to read into an array and then put into bins of equal size so I can plot a histogram in excel. I have written the following code which seems to compile fine but does not actually work, the programme never completes! If someone could tell me where I am going wrong it would be greatly appreciated
    Code:
    #include <stdio.h>
    
            int main()
            {
            int i, time[78],j;
            float hit[1800],value,event;
            FILE *fp, *fw;
                fp = fopen("damadata.xlsx" , "r");
                fw = fopen("bindata.txt", "w");
                
                for (i=0; i<1800; i++)  /* enter loop to set array elements to zero*/
                {
                 hit[i] = 0;
                 time[i] = 0;
                }                                          /*All array elements now set to zero, exit loop*/
                for(i=0; i<1800; i++)                      /*Enter loop to read all lines of data into hit array*/
                {
                fscanf(fp,"%f\n", &event);                 /*Each line is read into a seperate array element*/
                hit[i]=event;
                }                                  
              for (i = 0; i < 1800; i++)                  /*Now cycle through array containing data elements*/
              {                                           /*Open Loop 1*/
                    value = hit[i];                       /*Value now equal to element i of array*/                             
              for (j = 0; j < 77; j++)                    /*enter loop to bin data into 77 bins of group of 60*/
              {                                           /*Open Loop 2*/
              if (value >= j*60 && value < ((j+1)*60))    /*Tests if the value lies within a particular boundary*/
              {
              time[j] = time[j] + 1;                       /*If it is add 1 element into bin j*/
              }                                  
              else
              { 
                   time[j] = time[j];                       /*If not element j remains the same*/
              }
              }                                             /*Close Loop 1*/
              }                                             /*Close Loop 2*/
              for (j = 0; j <77; j++)                       /*Loop to print the number of data points in each bin*/
              {
              fprintf(fw, "%d\t%f\n" , j , &time[j]);       /*Prints the bin number and the number of elements in this bin*/
              }
              return(0);
    }

  2. #2
    Registered User whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    7,738
    It's extremely likely that your code is failing to read the file. Forgetting all the novice mistakes, you will have problems reading this kind of format anyway. The xlsx is a proprietary, binary file format. If you want to work with Excel data in your own programs, it's a really good idea to save native Excel spreadsheets in the portable CSV format first.

  3. #3
    Registered User
    Join Date
    Apr 2010
    Posts
    20
    Thanks, I've changed the data into a .txt file but it still isn't working!

  4. #4
    Registered User whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    7,738
    When you changed it to a txt file did you open it with notepad and look at the data? Until you can read it, you won't be able to use fscanf to read it.

  5. #5
    Registered User
    Join Date
    Apr 2010
    Posts
    20
    I don't quite understand, I have manually created the new data.txt file and also changed the
    Code:
    fp = fopen("damadata.xlsx" , "r");
    line to now read
    Code:
     fp = fopen("damadata.txt" , "r");
    I thought this opened the file?Sorry if I am wrong and many thanks for your help!

  6. #6
    - - - - - - - - oogabooga's Avatar
    Join Date
    Jan 2008
    Posts
    2,808
    The question is, can you read it yourself if you open it in notepad.
    You don't understand "open it with notepad"?
    The cost of software maintenance increases with the square of the programmer's creativity. - Robert D. Bliss

  7. #7
    Registered User
    Join Date
    Apr 2010
    Posts
    20
    Sorry yes I can, it is just a list of numbers in a column

  8. #8
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,553
    • Your indentation is atrocious. If it's hard to read your code, it's going to be easy to make mistakes and hard to find them. Indent properly, one of the first 3 styles from here is good: Indent style - Wikipedia, the free encyclopedia.
    • Don't use magic numbers. #define N_HITS as 1800 and N_TIME_BINS as 78. Replace any 1800, 77 and 78 with N_HITS, N_TIME_BINS or N_TIME_BINS-1.
    • Don't call your array "time". That's the name of a common function in the C standard library. time_bins is descriptive and safe.
    • You never check to see if fopen succeeded. If you can't open the files, exit with an error code.
    • Your first for loop initializes 1800 elements of time, but there's only 78 of them. This is a massive buffer overflow, and results in undefined behavior. You can initialize the whole thing in the declaration (see below)
    • You should check the return value of fscanf. If it fails, you should probably stop processing and exit with an error code.
    • The else clause does not do anything useful, just remove it.
    • Better yet, your whole inner loop (lines 23-34) can be simplified to time[hit[i]/60]++;
    • Be a good boy and close the files when you're done with them.


    That's all I got for now from a quick visual inspection.

    Code:
    #define N_TIME_BINS 78
    #define N_HITS 1800
    int time_bins[N_TIMES] = {0};
    float hit[N_HITS] = {0};
    MK27 and stahta01 like this.

  9. #9
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by Fixxxer View Post
    Sorry yes I can, it is just a list of numbers in a column
    And does that make sense to you? Ie, do you know what changes you want to make to it?

    Just changing the file suffix will not change anything else about the file.

    Initially you said, "I have written the following code which seems to compile fine but does not actually work, the programme never completes!". If you are more specific, it's more likely someone might have some more specific advice.

    Also, you need to indent your nested loops properly. It is much easier to read and debug code that way. If that is too hard for you or you don't have time, etc, then solving your problem will also be too difficult and time-consuming, so you should give up programming now
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  10. #10
    Registered User
    Join Date
    Apr 2010
    Posts
    20
    Many Thanks for your help, I'll give it a go!

  11. #11
    Registered User
    Join Date
    Apr 2010
    Posts
    20
    Hi There
    I have now made the changes to my code so that it now reads
    Code:
    #include <stdio.h>
    
    #define N_HITS 1800
    #define N_TIME_BINS 78
       int main()
         {
         int i, time_bins[N_TIME_BINS],j;
         double hit[N_HITS],value,event;
         FILE *fp, *fw;
         fp = fopen("damadata.txt" , "r");
         fw = fopen("bindata.txt", "w");
                
     for (i=0; i<N_HITS ; i++)  /* enter loop to set array elements to zero*/
         {
           hit[i] = 0;
         }
     for (i=0; i<N_TIME_BINS ; i++)
         {
           time_bins[i] = 0;
         }
                
     if (fp != NULL)
         {                                 
            for(i=0; i<N_HITS; i++)                      /*Enter loop to read all lines of data into hit array*/
              {
              fscanf(fp,"%f\n", event);                 /*Each line is read into a seperate array element*/
              hit[i]=event; 
              }                                 
         for (i = 0; i < N_HITS; i++)                  /*Now cycle through array containing data elements*/
             {                                          /*Open Loop 1*/                          
                for (j = 0; j < N_TIME_BINS; j++)
                {
                   value = hit[i];                       /*Value now equal to element i of array*/                     /*enter loop to bin data into 77 bins of group of 60*/
                if (value >= (j*60) && value < ((j+1)*60))    /*Tests if the value lies within a particular boundary*/
                    {
                    time_bins[j] = (time_bins[j] + 1);                       /*If it is add 1 element into bin j*/
                    }                                  
                else
                    { 
                    time_bins[j] = (time_bins[j]);                       /*If not element j remains the same*/
                    }
               }                                             /*Close Loop 1*/
            }                                             /*Close Loop 2*/
        for (j = 0; j <N_TIME_BINS; j++)                       /*Loop to print the number of data points in each bin*/
              {
              fprintf(fw, "%d\t" , j);       /*Prints the bin number and the number of elements in this bin*/
    	      fprintf(fw, "%d\n" , &time_bins[j]);
              }
       }
       else {
             printf("Error: The file you specified could not be found!");
            }
              fclose(fp);
              fclose(fw);
         return(0);
    }
    However I get the following error messages when I compile and I don't know how to sort them out!
    C:\Users\Toshiba\Documents\Pelles C Projects\pellab.c(26): warning #2234: Argument 3 to 'fscanf' does not match the format string; expected 'double *' but found 'double'.
    C:\Users\Toshiba\Documents\Pelles C Projects\pellab.c(47): warning #2234: Argument 3 to 'fprintf' does not match the format string; expected 'int' but found 'int *'.
    C:\Users\Toshiba\Documents\Pelles C Projects\pellab.c(8): warning #2116: Local 'event' is used without being initialized.
    Building pellab.exe.
    I'm not sure what the difference between double* and double is! Many thanks for your time!

  12. #12
    Registered User
    Join Date
    Apr 2010
    Posts
    20
    Not to worry guys, solved it now

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 13
    Last Post: 05-31-2009, 11:30 AM
  2. reading in file data
    By sickofc in forum C Programming
    Replies: 2
    Last Post: 03-01-2006, 04:16 PM
  3. Replies: 2
    Last Post: 06-16-2005, 10:03 AM
  4. reading data from a file
    By brianptodd in forum C++ Programming
    Replies: 1
    Last Post: 11-07-2003, 05:50 PM
  5. Help reading data file...
    By Unregistered in forum C Programming
    Replies: 4
    Last Post: 02-25-2002, 11:49 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21