Thread: Basic file handling - again!

Originally Posted by stevendt

Hi //Tater,

Thanks for the info. I'm doing a bit more than just renaming the files though.

I am checking for files that match the given filespec (of the form YYYYMMDD.CSV) and then opening them in turn, extracting about a dozen variables, storing the variables to a database then renaming the processed files as ".txt" for archiving. Each file contains sets of timestamped values that I need to store in the DB.

To prevent duplicate writes to the database, I don't write the data if it is earlier than the last entry in the database - that's why I need the files in the correct order. They **should** be written in the correct order, and FindFirstFile will **probably** return them in the order that I want, at least on NTFS, but not necessarily on FAT32, but the intent here was to make sure that I was processing the files in the correct name order,

regards
Dave

Use my example... index *.csv ... parse the filenames as you need to... the WIN32_FIND_DATA (Click me, I'm a link!) struct gives you the file dates as well so you can easily get your timestamps directly from there. Now that you got your filename, and you've checked the timestamps ... Use a function to Open the candidate files and split out the needed data. ... then move on to the next file... and so on until the list completes. You don't need a list of files... that's slow grungy and stupid... grab one directory listing, do what you gotta do then move on to the next....

Again... no reservations... doing this the system() way is just plain silly.


Ok, how about this...

Code:

BOOL DoDirectoryStuff(char *WildCards)
{  WIN32_FIND_DATA fd;
   HANDLE fh;

   fh = FindFirstFile(WildCards,&fd);
   if (fh == INVALID_HANDLE_VALUE)
     return FALSE;
   do 
     {  

// insert several megabytes of code here.

     }
    while(FindNextFile(fh,&fd);
    FindClose(fh);
    return TRUE;
}

Originally Posted by quzah

You are still only ever doing something to one file at a time, and you aren't comparing them to each other, so all you really need to do is just use FindNextFile until you are done. I assume you have some actual notation in your act of renaming that lets you know not to process it. So just look at the name and ignore it if you are done with it.
Quzah.

What he really needs to do is click those little red thingies in our messages and see just how much information is in that struct....

Quzah,

the problem would be (and it is by design) if the files were out of order, with say "20110927.csv" preceeding "20110926.csv", the first file would be processed correctly and the data written to the database. However, when "20110926" was processed, the program would check for the last date/time data in the database (which was now from the 27th) and ignore the data from the 26th so this data would not make it to the database. I could have chosen to always write data regardless but the database does not data to be out of time sequence order, so I chose to use this logic.

regards
Dave

Oooops - sorry!

"Red Thingies" being clicked as we speak!

Off to read some more :-)

My suggestion is just my suggestion, and goes back to my DOS roots, using Turbo C.

It's simply a way to do it.

If you learned how to make it work using the Windows API, that would be great.

Originally Posted by Adak

My suggestion is just my suggestion, and goes back to my DOS roots, using Turbo C.

It's simply a way to do it.

If you learned how to make it work using the Windows API, that would be great.

LOL... and so should you Mr Flintstone

Hey now! That's "Mr. Fllintstone who's learning Go", to you, "Barney"!

Rofl!!!!

Standard issue DOS bird, of course! What else?

OK, for all you guys in the stone age, please bear with someone who's obviously from an earlier time :-)

I'm missing something here, I've read the detail of the WIN32_FIND_DATA structure and do see that it contains a lot of useful information for the found file. However, going back to the purpose of this thread and the question of whether FindFirstFile can be relied upon to return the first alphabetically named file, I don't see how the extra data in the structure helps too much?

FindFirstFile returns a file matching the filespec, I can find out it's creation date etc., but surely I need to look at all the remaining files to ensure that it really is the first one? i.e., I'd need to compare the structure elements from FindNextFile for all other files?

regards
Dave

Originally Posted by stevendt

OK, for all you guys in the stone age, please bear with someone who's obviously from an earlier time :-)

I'm missing something here, I've read the detail of the WIN32_FIND_DATA structure and do see that it contains a lot of useful information for the found file. However, going back to the purpose of this thread and the question of whether FindFirstFile can be relied upon to return the first alphabetically named file, I don't see how the extra data in the structure helps too much?

FindFirstFile returns a file matching the filespec, I can find out it's creation date etc., but surely I need to look at all the remaining files to ensure that it really is the first one? i.e., I'd need to compare the structure elements from FindNextFile for all other files?

regards
Dave

Ok... I'm going to guess you've never done anything like this before...
Assuming your database is upto date AND nobody writes to it but you...

1) get the "top" of the database ... the date of the last entry... and store it in a variable someplace.
2) open the directory listing...
3) go through each file listing
........ if the file date is newer than the top of the database... add the info.
........ if the file date is older than the top of the database... simply ignore it.
4) get the next file listing
5) loop through 3, 4, 5 until all entries are processed.

Can you understand that with a fixed top date --also called a "high water mark"-- it actually does not matter one little bit what order the directory listing comes up in... older is older, newer is newer...

On the other hand, if you're silly enough to update the high water mark after each entry, you screw yourself into having to load potentially thousands of files, scan every single struct, sort them by date, then work through a list that *absolutely has* to be in order...

It's not the order of the listing that's your problem... it's updating the high water mark when you should not.

This is the part that I thought was a bit much with using findFirst(), (which of course, WAS part of Turbo C, along with findNext(), long before they were made part of the Windows API, just for your edification).