Communication with txt files (continuing)

This is a discussion on Communication with txt files (continuing) within the Windows Programming forums, part of the Platform Specific Boards category; Hello again This thread is connected to previous thread on http://cboard.cprogramming.com/showthread.php?t=92532 . I was advised to use in my app ...

  1. #1
    Registered User
    Join Date
    Apr 2007
    Posts
    162

    Communication with txt files (continuing)

    Hello again

    This thread is connected to previous thread on Retrieve string(s) from (txt) file.

    I was advised to use in my app a ifstream function, suddenly I have no idea how to start and how to use this function in win32 application?
    I tryed to look it up but apparantly nothing worked well.

    Please can you give me some advise how to use this function or recommend some other method which might be appropriate in this case?

  2. #2
    Reverse Engineer maxorator's Avatar
    Join Date
    Aug 2005
    Location
    Estonia
    Posts
    2,318
    There are generally two choices when dealing with reading and processing file data. The first one is to read a file by pieces. It's advantage is low memory consumption.
    Code:
    bool LoadListbox(char* lpFileName,HWND hWndReceiver){
        char *lpData=NULL,*lpLine=NULL;
        bool bRtn=true;
        DWORD dwRead=0,dwCount=0;
        HANDLE hFile=CreateFile(lpFileName,GENERIC_READ,7,0,OPEN_EXISTING,0,0);
        if(hFile==INVALID_HANDLE_VALUE){ return false; }
        lpData=new char[512];
        lpLine=new char[512];
        if(!ReadFile(hFile,lpData,512,&dwRead,0)){ bRtn=false; }
        while(dwRead&&bRtn){
            for(int i=0;i<dwRead;i++){
                if(lpData[i]=='\n'){
                    lpLine[dwCount]='\0';
                    SendDlgItemMessage(hWndReceiver,IDC_LISTEN,LB_ADDSTRING,0,(LPARAM)lpLine);
                    dwCount=0;
                }
                else if(lpData[i]!='\r'&&dwCount<511){
                    lpLine[dwCount]=lpData[i];
                    dwCount++;
                }
            }
            if(!ReadFile(hFile,lpData,512,&dwRead,0)){ bRtn=false; }
        }
        if(dwCount){
            lpLine[dwCount]='\0';
            SendDlgItemMessage(hWndReceiver,IDC_LISTEN,LB_ADDSTRING,0,(LPARAM)lpLine);
        }
        delete[] lpLine;
        delete[] lpData;
        CloseHandle(hFile);
        return bRtn;
    }
    And your way:
    Code:
    bool LoadListboxBig(char* lpFileName,HWND hWndReceiver){
        DWORD dwRead=0,dwCount=0,dwSize=0;
        char *lpData=NULL,*lpLine=NULL;
        HANDLE hFile=CreateFile(lpFileName,GENERIC_READ,7,0,OPEN_EXISTING,0,0);
        if(hFile==INVALID_HANDLE_VALUE){ return false; }
        dwSize=GetFileSize(hFile,0);
        if(dwSize){
            lpData=(char*)HeapAlloc(GetProcessHeap(),0,dwSize);
            if(!lpData){
                CloseHandle(hFile);
                return false;
            }
            if(!ReadFile(hFile,lpData,dwSize,&dwRead,0)){
                CloseHandle(hFile);
                HeapFree(GetProcessHeap(),0,lpData);
                return false;
            }
            lpLine=new char[512];
            for(int i=0;i<dwRead;i++){
                if(lpData[i]=='\n'){
                    lpLine[dwCount]='\0';
                    SendDlgItemMessage(hWndReceiver,IDC_LISTEN,LB_ADDSTRING,0,(LPARAM)lpLine);
                    dwCount=0;
                }
                else if(lpData[i]!='\r'&&dwCount<511){
                    lpLine[dwCount]=lpData[i];
                    dwCount++;
                }
            }
            if(dwCount){
                lpLine[dwCount]='\0';
                SendDlgItemMessage(hWndReceiver,IDC_LISTEN,LB_ADDSTRING,0,(LPARAM)lpLine);
            }
            delete[] lpLine;
        }
        HeapFree(GetProcessHeap(),0,lpData);
        CloseHandle(hFile);
        return true;
    }
    I haven't tested them so don't mind any little mistakes - you get the idea.
    Last edited by maxorator; 08-11-2007 at 10:22 AM.
    "The Internet treats censorship as damage and routes around it." - John Gilmore

  3. #3
    Registered User
    Join Date
    Apr 2007
    Posts
    162
    Ok now its working well... Thank you so much for help

  4. #4
    Yes, my avatar is stolen anonytmouse's Avatar
    Join Date
    Dec 2002
    Posts
    2,544
    Or you can use fgets as suggested by Daved:
    Code:
    bool LoadListbox(char* sFileName, HWND hWndDlg, int controlID)
    {
        char sLine[1024];
        FILE* hFile;
    
        hFile = fopen(sFileName, "r");
    
        if (hFile)
        {
            while ( fgets(sLine, sizeof(sLine), hFile) )
            {
                int len = strlen(sLine);
                if (sLine[len - 1] == '\n') sLine[len - 1] = '\0';
    
                SendDlgItemMessage(hWndDlg, controlID, LB_ADDSTRING, 0, (LPARAM) sLine);
            }
    
            fclose(hFile);
            return true;
        }
    
        return false;
    }

  5. #5
    Reverse Engineer maxorator's Avatar
    Join Date
    Aug 2005
    Location
    Estonia
    Posts
    2,318
    Well, if I've got a choice between Win32 API and CRT I would always use Win32 API functions unless advised otherwise by Microsoft documentation. CRT functions adventually call the Win32 API functions but to that point they manage to bloat the executable and performance. They are only meant for portability, not for high-performance Windows-based applications.
    Last edited by maxorator; 08-12-2007 at 02:50 PM.
    "The Internet treats censorship as damage and routes around it." - John Gilmore

  6. #6
    train spotter
    Join Date
    Aug 2001
    Location
    near a computer
    Posts
    3,853
    I prefer to read the whole file in to a dynamically alloc'ed buffer with one read. Then close the file and process the data.

    This has the advantage of speed (as each read is very slow) but then I deal with files with 10,000s of records.

    Also the file is open for the shortest time possible, reducing shared access confilicts.

    Low memory consumption is not usually a factor in systems with Gb to spare.

    EDIT:
    I see that is what you had before.

    You just need to look for the end of line characters in the file buffer once loaded. ie a way of determining the start and end of a line.
    Then send that portion of the buffer to the list (after adding a terminator '\0')
    Last edited by novacain; 08-13-2007 at 01:31 AM.
    "Man alone suffers so excruciatingly in the world that he was compelled to invent laughter."
    Friedrich Nietzsche

    "I spent a lot of my money on booze, birds and fast cars......the rest I squandered."
    George Best

    "If you are going through hell....keep going."
    Winston Churchill

  7. #7
    int x = *((int *) NULL); Cactus_Hugger's Avatar
    Join Date
    Jul 2003
    Location
    Banks of the River Styx
    Posts
    902
    Well, if I've got a choice between Win32 API and CRT I would always use Win32 API functions unless advised otherwise by Microsoft documentation. CRT functions adventually call the Win32 API functions but to that point they manage to bloat the executable and performance.
    I do just the opposite. If I've got a choice, I go with fopen() and friends. You say they "bloat" the executable, and performance. On my system, a minimal example executable with fopen() uses <17KB, before any optimizations or stripping. Not enough bloat, if any, to worry about. (Other libraries add half a MB or more... *cough*wxWidgets*cough*)
    As for performance, that's a harder one, so I wrote a program to read a file, and time it: (I don't actually know how much faster Win32 would be, so, let's find out.)
    Code:
    CRT_Read(): 730036224 bytes read.
    CRT  : 981156 milliseconds.
    Win32_Read(): 730036224 bytes read.
    Win32: 962075 milliseconds.
    Indeed, Win32 is faster. (I should hope so.) However, it only faired 2&#37; better here. This test was conducted on an AMD (450 MHz) Win98 box. Advances in processor power should narrow the margin between fopen() and the Win32 equivalents even more. That said, I find the loss in portability from not using fopen() to outweigh any speed or size benefits. (The test program itself was 6KB, and used both methods.)
    Now, if I was working on a project that did insane amounts of file I/O and needed every last performance enhancement, I might think about it.
    Furthermore, I'd expect whatever storage media you're reading off of to be more of a bottleneck than differences CRT/Win32.
    (Edit: The moderators on the Microsoft forums seem to think fopen()...)

    The test I conducted was on the above box, on a ~700MB file. (The Knoppix CD ISO, actually.) Hopefully a large file will reduce/eliminate any caching of the file. (If Win9x does so.) I might repeat a later test on XP. Need a big file though...
    The program I used to test with is attached, if it interests anyone.
    Attached Files Attached Files
    Last edited by Cactus_Hugger; 08-13-2007 at 01:49 PM.
    long time; /* know C? */
    Unprecedented performance: Nothing ever ran this slow before.
    Any sufficiently advanced bug is indistinguishable from a feature.
    Real Programmers confuse Halloween and Christmas, because dec 25 == oct 31.
    The best way to accelerate an IBM is at 9.8 m/s/s.
    recursion (re - cur' - zhun) n. 1. (see recursion)

  8. #8
    Reverse Engineer maxorator's Avatar
    Join Date
    Aug 2005
    Location
    Estonia
    Posts
    2,318
    I guess 97&#37; percent goes to hard drive reading, 1% goes to Win32 API function code itself and 2% to the CRT.
    In the contrary, you need to test on little files.

    Test trying to open a file that doesn't exist 100 000 times.
    "The Internet treats censorship as damage and routes around it." - John Gilmore

  9. #9
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    I don't know what you are suggesting: Creating 100K files or trying to open a non-existing file 100K times? For the case of the above discussion, it makes no big difference - as I believe you don't want to create a new text file to read from(!!), nor are you interested in opening files that don't exist - or do you not know whether the file is there or not?

    And by the way, a lower level function doesn't necessarily mean more performance. It can do, but for example, a high level file operation using the C runtime library may well do more caching/buffering than the lower level function, which means that on small reads, you get less performance.

    Also, the rule is: figure out which bit of code uses 90% of your CPU (or 90% of your memory usage) and optimize that first. Then start over when you've made it significantly better... ;-)

    Presumably, you load your list-box once, and use it multiple times, so loading it is a small factor.

    --
    Mats

  10. #10
    int x = *((int *) NULL); Cactus_Hugger's Avatar
    Join Date
    Jul 2003
    Location
    Banks of the River Styx
    Posts
    902
    Test trying to open a file that doesn't exist 100 000 times.
    Alright. Win98 faired pathetically. I actually only looped it 1 000 times. (Win98 / 450 MHz)
    Code:
    Trying to open file "c:\nonexists" 1000 times.
    CRT  : 11041 milliseconds.
    Win32: 7015 milliseconds.
    I repeated the test on WinXP, (1.8 GHz):
    Code:
    Trying to open file "c:\nonexists" 100000 times.
    CRT  : 9672 milliseconds.
    Win32: 9953 milliseconds.
    They're much closer together on XP. (Not to mention oodles faster.)
    Lastly, I tried Linux. (500 MHz):
    Code:
    Trying to open file "nonexists" 100000 times.
    CRT  : 836 milliseconds.
    None of the tested operating systems seem to do actual disk I/O for the entire duration. (Which is good.)
    The CRT outperformed the Win32 API once on Win9x, but no other test did. It outperformed the Win32 API every time on XP. (I ran more than one test, the results above are just samples.)

    EDIT: The above test is probably skewed. The three directories that it is looking for "nonexists" in had varying numbers of files, which may have helped/hindered some systems.
    Last edited by Cactus_Hugger; 08-14-2007 at 07:35 AM.
    long time; /* know C? */
    Unprecedented performance: Nothing ever ran this slow before.
    Any sufficiently advanced bug is indistinguishable from a feature.
    Real Programmers confuse Halloween and Christmas, because dec 25 == oct 31.
    The best way to accelerate an IBM is at 9.8 m/s/s.
    recursion (re - cur' - zhun) n. 1. (see recursion)

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Help with loading files into rich text box
    By blueparukia in forum C# Programming
    Replies: 3
    Last Post: 10-19-2007, 12:59 AM
  2. Looking for communication lib
    By BrownB in forum C Programming
    Replies: 3
    Last Post: 04-27-2005, 10:01 AM
  3. Editing TXT files in C++
    By Nexus in forum C++ Programming
    Replies: 4
    Last Post: 03-24-2005, 04:34 AM
  4. Help...using strings and txt files
    By Juicehead in forum C Programming
    Replies: 0
    Last Post: 11-26-2002, 08:01 AM
  5. printing txt files
    By baphomet in forum A Brief History of Cprogramming.com
    Replies: 2
    Last Post: 03-10-2002, 08:53 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21