Thread: file reading problem

  1. #1
    Registered User
    Join Date
    May 2008
    Posts
    16

    file reading problem

    Hello guys and girls

    Ok, what I am trying to do is this:

    I have a number of csv files that need to be read into a program, they are all store in one location, then they will be manipulated in various ways so I can extract information from the data. For me to do that I will need to keep track of all the files. I can manually read in all the files by typing in all the name, but since there will be around 90 to 100 csv files at one time. It will be a right pain to type in their name one at a time.

    So my question is this, is there anyway I can specify a folder (instead of individual file name), and write a function that will read all the files in the folder and store them individually into an array/vector so I can manipulate the data using other function?

    If anyone know how to do it that will be great. I know I maybe asking a very simple thing but I am still very new to programming. So thanks in advance

    Regards

    Grasshopper Esq

  2. #2
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,057
    It's non-standard, but it can be done. There's an FAQ about it, in fact. http://faq.cprogramming.com/cgi-bin/...&id=1044780608

    [edit] A very easy way to do this is to take the files to process as command-line parameters. Then you can run the program from the command prompt like this, for example:
    Code:
    program *.csv
    [/edit]
    dwk

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell


    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ: cpwiki.sf.net

    My website: http://dwks.theprogrammingsite.com/
    Projects: codeform, xuni, atlantis, nort, etc.

  3. #3
    Registered User
    Join Date
    Jan 2005
    Posts
    7,366
    Note that the FAQ doesn't include one excellent, new option. That's boost::filesystem, which is a cross platform library that will provide a lot of that functionality.

  4. #4
    Registered User
    Join Date
    Jun 2008
    Location
    Houston, Texas
    Posts
    43
    Quote Originally Posted by dwks View Post
    It's non-standard, but it can be done.
    So what's the standard way of handling this situation?

    I ask because the situation comes up a lot in my field and I've always been meaning to figure this out. When we're running samples on the large HPLC and GC/MS machines (or almost any other analysis machine for that matter), the machine will usually output a separate file for each sample. When you have hundreds of samples, it can get pretty tedious. My old place of work wasn't really open to new ideas... so I never got to work on this (once the files are read in, manipulation would be amazingly quick). Still, I can't imagine this being that much of a rare problem.

    Was there no solution before Boost?
    Last edited by Shaun32887; 07-02-2008 at 02:51 PM.

  5. #5
    Registered User
    Join Date
    Jan 2005
    Posts
    7,366
    >> So what's the standard way of handling this situation?
    I think dwks was referring to accessing the file system. For that, there is no standard way since C++ doesn't have anything in its library to access a file system.

    The solutions before boost were the same as they are now- platform specific libraries (see the linked FAQ). The POSIX interface was supported on many platforms, and that still might be a good way to go. The situation is still tedious, the only question is which library you will use to access the folder information and how portable that library is.

  6. #6
    Registered User
    Join Date
    Jul 2003
    Posts
    110
    Quote Originally Posted by Shaun32887 View Post
    So what's the standard way of handling this situation?
    Well, that's a great question.

    Usually, my first instinct is to omit file handling and directory walking as much as I can from within my program and rely on the shell for a great deal of it, something like dwks was suggesting in an earlier post. I'm only suggesting that for your case which seems to be batch processing a variable number of consistently formatted input files. However, that's more of the UNIX philosophy than any "standard way of handling."

    But, there's something to be said for that, especially during development.

    If you just take input from cin and write to cout, your program becomes more adaptable and very friendly to pipelines so it can hook into more complex filters. Often, the utility programs provided by your operating system are much better at file and directory listing than a hand-rolled version. Also, testing is a snap, because you can start by entering simple test cases yourself as you build up the program and then redirecting the cases from test files as you move further along in development.

    Plus, what if you want to eventually look at *two* directories and not one. I'm not sure that's realistic in your case, but I'm just pointing out the scalability advantage here as well.

    Overall, I like it because I can concentrate on the problem itself and developing a solution, rather than the mechanics of file handling which isn't really the point of the program.

    It's just something to consider.

    Wouldn't C++ be even better with a glob operator? Only kidding of course, this isn't Perl.

  7. #7
    Registered User
    Join Date
    May 2008
    Posts
    16
    thanks alot guys, will give it a try. It's nice to come to a forum where people want to help, been to a few and most of the member just ignore you or talk in jargon. Really appreciate it

    Regards

    Grasshopper esq

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. File transfer- the file sometimes not full transferred
    By shu_fei86 in forum C# Programming
    Replies: 13
    Last Post: 03-13-2009, 12:44 PM
  2. Post...
    By maxorator in forum C++ Programming
    Replies: 12
    Last Post: 10-11-2005, 08:39 AM
  3. Replies: 20
    Last Post: 06-12-2005, 11:53 PM
  4. Replies: 3
    Last Post: 03-04-2005, 02:46 PM
  5. Problem reading file
    By winsonlee in forum C Programming
    Replies: 2
    Last Post: 04-23-2004, 06:52 AM