Thread: issues reading a file.

  1. #1
    Registered User
    Join Date
    Jul 2012
    Posts
    8

    issues reading a file.

    hello all. I am in the proccess of writing a program for my astrophysics club. to sum up the function of the program basically, the program simulates taking pictures of the universe based on its location and other details which is read from a .txt file then outputs various magnitudes (determined by clarity of the photo) based on various filters, in a table. Writing the program, everything seems to work the way it needs to, except in the .txt file, nothing is seeming to be read past the 1900th number.

    At first I thought it was because the range of my rows were too small, so I changed it from a const int to a const long long; however the program still reads to the 1900th number. I was wondering if there was a way to modify the program so that all the numbers would be read and appropriately sorted.

    I'm not sure if this helps, but I am using netbeans v7.1.2 as my ide and cygwin as my complier. Also, for the final version of the program I want to include 3 more .txt files , for a total of 5 .txt files, not just the 2 I am posting here.

    Hm... there seems to be an issue uploading directly to the forum so here's a link where you utilize the .txt files. 4shared folder - My 4shared


    the code:
    Code:
    #include <iostream>
    #include <fstream>
    #include <iomanip>
    #include <cstdlib>
    #include <string>
    #include <sstream>
    using namespace std;
    const long long ROWS=100;
    const int COLS=26;
    int linearSearch(string[],string, int);
    int main(){
        ifstream file1("test.txt");
        ifstream file2("test1.txt");
        
        double fullTable1[ROWS][COLS];
        double extractTable1[ROWS][COLS];
        double fullTable2[ROWS][COLS];
        double extractTable2[ROWS][COLS];
        double combineTable[ROWS][COLS];
        
        string st1[ROWS],st2[ROWS];
        string s1,s2;
        string line;
        
        ostringstream convert;
        
        int count1=0;
        int count2=0;
        int count3=0;
        for (int i=0;i<28;i++)      //Ignore the command line in file 1
        getline(file1, line);
        for (int i=0;i<28;i++)      //Ignore the command lines in file 2
        getline(file2, line);
        for(int row=0;row<ROWS;row++)       //Read into the two files
            for(int col=0;col<COLS;col++){
                file1>>fullTable1[row][col];
                file2>>fullTable2[row][col];
            }
        for(int row=0;row<ROWS;row++)       //Read file 1 into array 1
            if (fullTable1[row][6]<40){
            extractTable1[count1][0]=fullTable1[count1][0];
            extractTable1[count1][1]=fullTable1[row][3];
            extractTable1[count1][2]=fullTable1[row][4];
            extractTable1[count1][3]=fullTable1[row][6];
            count1++;
        }
        for(int row=0;row<ROWS;row++)       //Read file 2 into array 2
            if (fullTable2[row][6]<40){
            extractTable2[count2][0]=fullTable2[count2][0];
            extractTable2[count2][1]=fullTable2[row][3];
            extractTable2[count2][2]=fullTable2[row][4];
            extractTable2[count2][3]=fullTable2[row][6];
            count2++;
        }
        for(int row=0;row<count1;row++){
            convert<<extractTable1[row][1];
            s1=convert.str();
            convert.str("");
            convert<<extractTable1[row][2];
            s2=convert.str();
            convert.str("");
            st1[row]=s1+s2;
        }
         
        for(int row=0;row<count2;row++){
            convert<<extractTable2[row][1];
            s1=convert.str();
            convert.str("");
            convert<<extractTable2[row][2];
            s2=convert.str();
            convert.str("");
            st2[row]=s1+s2;
        }
        
        int pos;
        for(int row=0;row<count1;row++){
            //Assign the address in file 1 to two one-dimensional arrays
            s1=st1[row];
            //Search for the address in file 2
            pos=linearSearch(st2,s1,count2);
            //Read into a combined table if the address matches
            if(pos!=-1){
                combineTable[count3][0]=extractTable1[count3][0];
                combineTable[count3][1]=extractTable1[row][1];
                combineTable[count3][2]=extractTable1[row][2];
                combineTable[count3][3]=extractTable1[row][3];
                combineTable[count3][4]=extractTable2[pos][3];
                count3++;
            }
        }
        for(int row=0;row<count3;row++){
                cout<<setw(6)<<showpoint<<setprecision(0)<<combineTable[row][0]<<"\t";
                cout<<fixed<<setprecision(4);
                cout<<setw(10)<<combineTable[row][1]<<"\t";
                cout<<setw(10)<<combineTable[row][2]<<"\t";
                cout<<setw(10)<<combineTable[row][3]<<"\t";
                cout<<setw(10)<<combineTable[row][4]<<"\t";
                cout<<endl;
        }
        file1.close();
        file2.close();
        return 0;
    }
    int linearSearch(string st2[], string s, int c){
        int position=-1;
        for (int row=0;row<c;row++)
            if (st2[row]==s)
                position=row;
        return position;
    }

  2. #2
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,613
    Dude put your files in a pastebin or something everyone can read.

  3. #3
    Registered User
    Join Date
    Jul 2012
    Posts
    8
    Quote Originally Posted by whiteflags View Post
    Dude put your files in a pastebin or something everyone can read.
    Well the file is 8.37 mb, paste bin only allows 500kb for general users and the others I know of don't allow that much data storage. If you are able to, would you be able to direct me to a site, where everyone can read without downloading and has a much more generous file limit?

  4. #4
    Programming Wraith GReaper's Avatar
    Join Date
    Apr 2009
    Location
    Greece
    Posts
    2,739
    SourceForge - Download, Develop and Publish Free Open Source Software has enormous( unlimited? ) space for any project. The data are available for download though.
    Devoted my life to programming...

  5. #5
    - - - - - - - - oogabooga's Avatar
    Join Date
    Jan 2008
    Posts
    2,808
    For anyone who's interested, the data file (test.txt) looks like this:
    Code:
    #   1 NUMBER          Running object number
    
    #   2 X_IMAGE         Object position along x                         [pixel]
    
    #   3 Y_IMAGE         Object position along y                         [pixel]
    
    #   4 ALPHA_J2000     Right ascension of barycenter (J2000)           [deg]
    
    #   5 DELTA_J2000     Declination of barycenter (J2000)               [deg]
    
    #   6 ISOAREA_IMAGE   Isophotal area above Analysis threshold         [pixel**2]
    
    #   7 MAG_AUTO        Kron-like elliptical aperture magnitude         [mag]
    
    #   8 MAGERR_AUTO     RMS error for AUTO magnitude                    [mag]
    
    #   9 MAG_ISO         Isophotal magnitude                             [mag]
    
    #  10 MAGERR_ISO      RMS error for isophotal magnitude               [mag]
    
    #  11 MAG_APER        Fixed aperture magnitude vector                 [mag]
    
    #  18 MAGERR_APER     RMS error vector for fixed aperture mag.        [mag]
    
    #  25 FWHM_WORLD      FWHM assuming a gaussian core                   [deg]
    
    #  26 CLASS_STAR      S/G classifier output
             
    1  14694.082  12841.841  53.0192692 -27.7743614        50  24.4896   0.1888  24.4424   0.1847  24.6076  24.5918  24.5106  24.4891  24.5233  24.7303  24.9398   0.1990   0.1977   0.1905   0.1889   0.1925   0.2139   0.2418  0.000213601  1.00
             2   6338.234  19986.797  53.1766081 -27.6553073       167  21.2993   0.0434  21.2438   0.0423  23.0089  22.2797  21.6558  21.4358  21.3300  21.4004  21.7729   0.0953   0.0681   0.0511   0.0462   0.0440   0.0455   0.0541 0.0004282809  1.00
             3   6383.001  19977.406  53.1757658 -27.6554641        12  27.1615   0.6486  27.4791   0.7488  27.3424  27.8093  99.0000  99.0000  99.0000  99.0000  99.0000   0.7032   0.8785  99.0000  99.0000  99.0000  99.0000  99.0000 0.0001029543  0.94
             4   6423.653  19908.910  53.1750015 -27.6566060       150  25.3458   0.2880  25.4591   0.2962  27.1526  26.4144  26.0386  26.0654  25.7570  25.6199  99.0000   0.6441   0.4586   0.3864   0.3936   0.3439   0.3284  99.0000 0.0005680354  0.93
             5   6361.602  19875.633  53.1761693 -27.6571602        86  25.9357   0.3719  26.2141   0.4197  27.4007  26.8468  26.5389  26.1682  25.9270  26.2229  99.0000   0.7225   0.5605   0.4879   0.4131   0.3731   0.4443  99.0000 0.0002902908  0.67
             6   6402.465  19922.904  53.1754001 -27.6563726       376  99.0000  99.0000  24.6398   0.2033  27.9265  26.9952  26.4444  25.7744  25.3279  25.1515  99.0000   0.9223   0.6005   0.4668   0.3434   0.2807   0.2616  99.0000 0.0008087613  0.97
             7   6383.945  19905.408  53.1757487 -27.6566641       714  23.5897   0.1256  23.0612   0.0979  25.1333  24.4648  24.0854  23.7483  23.5909  23.8558  26.2520   0.2536   0.1864   0.1566   0.1341   0.1249   0.1418   0.4822 0.0003005085  0.98
             8   6423.822  19963.979  53.1749979 -27.6556882      1720  18.8034   0.0137  18.7673   0.0135  19.5258  19.0503  18.8779  18.8419  18.8124  18.7922  18.7805   0.0192   0.0154   0.0142   0.0140   0.0138   0.0137   0.0136 0.0002871867  0.97
             9   6461.808  19952.777  53.1742832 -27.6558752       117  25.9546   0.3768  25.5787   0.3128  27.4193  26.4763  26.0245  25.6954  25.8719  26.6531  99.0000   0.7288   0.4720   0.3839   0.3309   0.3634   0.5561  99.0000 0.0002008076  0.86
            10   6369.883  19973.572  53.1760127 -27.6555279         5  26.8555   0.5617  26.7846   0.5429  26.7658  27.1703  99.0000  99.0000  99.0000  99.0000  99.0000   0.5386   0.6515  99.0000  99.0000  99.0000  99.0000  99.0000 6.679268e-05  0.99
    It continues like that until the end. The whole file has 31221 lines.

    The other file is similar, but only has 38 lines. (Is that the complete file?)
    The cost of software maintenance increases with the square of the programmer's creativity. - Robert D. Bliss

  6. #6
    - - - - - - - - oogabooga's Avatar
    Join Date
    Jan 2008
    Posts
    2,808
    When you say nothing is read "past the 1900th number" do you mean that, since there are 26 numbers per data line (after you skip the first 28 lines), that it seemingly stops reading numbers in about the middle of the 73rd line? How do you know that?

    Also, you're trying to read 100 rows from each file, but test2.txt (as posted) only has about 10 rows of data. What's up with that?

    And here's a slightly better version of linearSearch:
    Code:
    int linearSearch(string st2[], string s, int c){
        for (int row = 0; row < c; row++)
            if (st2[row] == s)
                return row;
        return -1;
    }
    The cost of software maintenance increases with the square of the programmer's creativity. - Robert D. Bliss

  7. #7
    Registered User
    Join Date
    Jul 2012
    Posts
    8
    The other file is similar, but only has 38 lines. (Is that the complete file?)
    my mistake. I forgot to mention that the 2nd file is actually an excerpt of the 1st file. It is used to make sure the code works.

    When you say nothing is read "past the 1900th number" do you mean that, since there are 26 numbers per data line (after you skip the first 28 lines), that it seemingly stops reading numbers in about the middle of the 73rd line? How do you know that?
    another slip ^^;. I meant to say that is the number of lines it stops reading. for some reason, not all of the file is read. is the file simply too large?

    Also, you're trying to read 100 rows from each file, but test2.txt (as posted) only has about 10 rows of data. What's up with that?
    What I lacked to mentioned earlier was the 2nd file is a excerpt of the first to make sure the code worked.

    In terms why I chose 100, it was trial and error, mostly. I originally made the ROW = 10000, the build failed, then I tried 2000 and it failed. using 100 seems to work, but it gives me incomplete results.

  8. #8
    Registered User
    Join Date
    May 2010
    Posts
    4,633
    You may want to consider using a vector of a structure. The structure would hold the individual elements contained in each line of your input file. Part of your problem that you are having is the possibility of running out of stack space to hold your arrays.

    Jim

  9. #9
    Registered User
    Join Date
    Jul 2012
    Posts
    8
    Quote Originally Posted by jimblumberg View Post
    You may want to consider using a vector of a structure.
    I'm still fairly a novice in c++, so I don't know how what vectors are and how to proceed using one =/.

  10. #10
    - - - - - - - - oogabooga's Avatar
    Join Date
    Jan 2008
    Posts
    2,808
    Try moving the definition of your 2D arrays outside (just above) main. That way they won't be allocated on the stack. This is kind of a "quick fix" and only suitable for a small program like yours.

    It would be good to reduce the amount of storage you're using. For instance, you declare extractTable and combineTable as having 26 columns whereas they only use 4. Also, you don't need to read the entire contents of the files into the fullTable arrays. Just read in a single row into a 1D array (of 26 columns) and transfer the data to the 2D (4 column) arrays right away. Also, you could probably print the data out directly instead of temporarily storing it in combineTable.
    The cost of software maintenance increases with the square of the programmer's creativity. - Robert D. Bliss

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Having issues with a .dat file
    By aw1742 in forum C Programming
    Replies: 3
    Last Post: 03-21-2012, 10:10 AM
  2. Replies: 15
    Last Post: 04-19-2011, 07:29 AM
  3. issues reading from a file
    By sanddune008 in forum C Programming
    Replies: 3
    Last Post: 07-16-2010, 05:01 AM
  4. String Reading Issues
    By Acolyte in forum C Programming
    Replies: 8
    Last Post: 11-19-2007, 07:37 AM
  5. Issues reading text files
    By ozzy34 in forum C++ Programming
    Replies: 5
    Last Post: 06-01-2004, 08:15 AM