Recursive FTP directory exploration using Wininet

This is a discussion on Recursive FTP directory exploration using Wininet within the Networking/Device Communication forums, part of the General Programming Boards category; Hello, I have a problem trying to count the files of a directory and its subdirectories using a recursive function; ...

  1. #1
    Registered User
    Join Date
    Mar 2005
    Location
    Juneda
    Posts
    287

    Recursive FTP directory exploration using Wininet

    Hello, I have a problem trying to count the files of a directory and its subdirectories using a recursive function; I have done a separated test from my main program to see why I did'nt work and try to find solutions, but I'm not able to find them. That's what I have:

    Code:
    //headers and some globals
    #include <wininet.h>
    #include <windows.h>
    
    static int padding=-1;
    Code:
    void printf_error()
    {
    unsigned long dw,dq;
    char bff[512];
    LPVOID lpMsgBuf;
    
    if(GetLastError()==ERROR_INTERNET_EXTENDED_ERROR)
        {
        dq=512;
        InternetGetLastResponseInfo(&dw,bff,&dq);
        printf("(IGLRI)%s\n",bff);
        }
    else
        {
        FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER|FORMAT_MESSAGE_FROM_SYSTEM,NULL,GetLastError(),MAKELANGID(LANG_NEUTRAL,SUBLANG_DEFAULT),(LPTSTR)&lpMsgBuf,0,NULL);
        printf("(GLE)(%ld)%s\n",GetLastError(),lpMsgBuf);
        LocalFree(lpMsgBuf);
        }
    }
    
    
    void backslash(char *bff)
    {
    if(bff[strlen(bff)-1]!='/')
        {
        strcat(bff,"/");
        }
    }
    
    
    void printf_pad()
    {
    int q;
    for(q=0;q<padding;q++)
        {
        printf("  ");
        }
    }
    
    
    int conta_arxius(HINTERNET hCon,char *dir_treball)
    {
    //conta_arxius=cout files
    //dir_treball=working directory, is the current selected dir
    
    int inc,ctd=0;
    HINTERNET hf=NULL;
    WIN32_FIND_DATA fd;
    char bff[2048];
    
    padding++;
    if(!(hf=FtpFindFirstFile(hCon,NULL,&fd,0,0)))
        {
        ctd=-1;
        printf("Error FindFirstFile: ");
        printf_error();
        goto finalitza;
        }
    do
        {
        if(fd.dwFileAttributes==FILE_ATTRIBUTE_DIRECTORY)
            {
            sprintf(bff,"%s",dir_treball);
            backslash(bff);
            sprintf(bff,fd.cFileName);
            
            printf_pad();
            printf("Select %s\n",bff);
            if(!FtpSetCurrentDirectory(hCon,bff))
                {
                ctd=-1;
                printf("Error SetCurrentDirectori: ");
                printf_error();
                goto finalitza;
                }
            
            printf("Count %s\n",bff);
            if((inc=conta_arxius(hCon,dir_treball))<0)
                {
                ctd=-1;
                printf_error();
                goto finalitza;
                }
            
            printf("Return to %s\n",dir_treball);
            if(!FtpSetCurrentDirectory(hCon,dir_treball))
                {
                ctd=-1;
                printf("Error SetCurrentDirectori: ");
                printf_error();
                goto finalitza;
                }
            
            ctd+=inc;
            }
        else
            {
            printf_pad();
            printf("%s\n",fd.cFileName);
            ctd++;
            }
        } while(InternetFindNextFile(hf,&fd));
    InternetCloseHandle(hf);
    hf=NULL;
    
    finalitza:
    if(hf) {InternetCloseHandle(hf);}
    padding--;
    return ctd;
    }
    The start call to the function is done by

    Code:
    ctd=conta_arxius(hCon,folder_to_explore);
    where 'folder_to_explore' is the current directory on the 'hCon' FTP instance that I use to work.

    If I omit the recursive count (that is, if I comment the line to the recursive call to the function), the function works (obviously only counting the files for the first directory). But when I make the first recursive call, the function 'FtpFindFirstFile' fails and I get an error through 'GetLastError()' that's 317 (and I don't get any kind of error through 'InternetGetLastResponseInfo'). But when I use this error code to display a text information it only shows 'null'. That error code is ERROR_MR_MID_NOT_FOUND that means that the system isn't able to find any error code for the error.

    Does someone had the same problem? Or maybe I'm doing something wrong on the woking-idea of my code? I'm totally lost.

    Thank's in advance
    Niara

  2. #2
    Registered User
    Join Date
    Mar 2005
    Location
    Juneda
    Posts
    287
    I have make still another test:

    I have closed the FtpFindFirstFile handle before make the recursive call, and now works (the recursive part), but when it returns to the function it cannot continue explorating the directory because I have closed the exploration handle. It seems like I can have only one handle opened, is that? Then how to do it with recursion?

    More thank's in advance
    Niara

  3. #3
    CSharpener vart's Avatar
    Join Date
    Oct 2006
    Location
    Rishon LeZion, Israel
    Posts
    6,484
    Quote Originally Posted by Niara View Post
    Then how to do it with recursion?
    Finish current folder - fill list of names of subfolders

    Only after no more folders in the current folder is found - Call your function for each found subfolder
    The first 90% of a project takes 90% of the time,
    the last 10% takes the other 90% of the time.

  4. #4
    Registered User
    Join Date
    Mar 2005
    Location
    Juneda
    Posts
    287
    Hello vart, thanks for your time and your help. I'll try as you say, suppose I understand what you mean:

    -. Open a handle to explore the folder
    -. Make a list of subfolders
    -. Close the exploration handle
    -. Walk throught the list to explore each subfolder

    For the moment it looks good, let me code for a while...

    Thank's
    Niara

  5. #5
    Registered User
    Join Date
    Mar 2005
    Location
    Juneda
    Posts
    287
    Hay it works, now I can go on with my program. Here's the result:

    Code:
    int conta_arxius(HINTERNET hCon,char *dir_treball)
    {
    int inc,ctd=0,q;
    struct _ARRAY *dirs,*ptr;
    HINTERNET hf;
    WIN32_FIND_DATA fd;
    char bff[2048];
    
    padding++;
    if(!(hf=FtpFindFirstFile(hCon,NULL,&fd,0,0)))
        {
        ctd=-1;
        printf("Error FindFirstFile: ");
        printf_error();
        goto finalitza;
        }
    
    dirs=NULL;
    do
        {
        if(fd.dwFileAttributes==FILE_ATTRIBUTE_DIRECTORY)
            {
            array_add(&dirs,fd.cFileName);
            }
        else
            {
            printf_pad();
            printf("%s\n",fd.cFileName);
            ctd++;
            }
        } while(InternetFindNextFile(hf,&fd));
    InternetCloseHandle(hf);
    hf=NULL;
    
    for(q=0;q<array_length(dirs);q++)
        {
        if((ptr=array_get(dirs,q)))
            {
            sprintf(bff,"%s",dir_treball);
            backslash(bff);
            strcat(bff,ptr->data);
            
            printf("Selecciona %s\n",bff);
            if(!FtpSetCurrentDirectory(hCon,bff))
                {
                ctd=-1;
                printf("Error SetCurrentDirectori: ");
                printf_error();
                goto finalitza;
                }
            
            printf_pad();
            printf("Explora %s\n",bff);
            if((inc=conta_arxius(hCon,bff))<0)
                {
                ctd=-1;
                printf_error();
                goto finalitza;
                }
            
            printf("Retorna a %s\n",dir_treball);
            if(!FtpSetCurrentDirectory(hCon,dir_treball))
                {
                ctd=-1;
                printf("Error SetCurrentDirectori: ");
                printf_error();
                goto finalitza;
                }
    
            ctd+=inc;
            }
        }
    
    finalitza:
    array_reset(dirs);
    padding--;
    return ctd;
    }
    The list _ARRAY is a simple linked list, the other functions are the same as in my first post.

    Thank's vart for your help
    Niara
    Last edited by Niara; 04-14-2010 at 12:09 PM.

  6. #6
    Registered User
    Join Date
    Mar 2010
    Posts
    68

    Cool This might make you happy... :)

    This is a little patcher program that I made. Have fun with it. It will connect to the ftp server and recursively go through each directory and download files if the ones on the server are newer than the ones located on the clients hard drive.

    Feel free to modify any part and use it as you wish.. Good Luck

    Code:
    #include "stdafx.h"
    #include <windows.h>
    #include <wininet.h>
    #include <iostream>
    #include <list>
    #include <string>
    #include <stdlib.h> 
    #include <fstream>
    #pragma comment(lib, "wininet.lib")
    #pragma comment(lib, "user32.lib")
    
    using namespace std;
    
    void StartClient();
    void GetFiles(HINTERNET &hConnnect);
    void ListDirectory(HINTERNET &hConnnect);
    bool FindFile(HINTERNET &hConnect, HINTERNET &hFind, LPCWSTR temp, WIN32_FIND_DATA &SdirInfo);
    wchar_t relative[MAX_PATH];
    size_t relativesize;
    
    
    int _tmain(int argc, _TCHAR* argv[])
    {
    	/*
    	if(RunDirectx){
    		STARTUPINFO sj;
    		PROCESS_INFORMATION pj;
    		memset(&sj, 0, sizeof(sj));
    		sj.cb = sizeof(sj);
    		ZeroMemory( &pj, sizeof(pj) );
    		// Start the child process p1.exe. Make sure p1.exe is in the
    		// same folder as current application. Otherwise write the full path in first argument.
    		if(!CreateProcess(L".\\directx\\DXSETUP.exe", NULL, NULL, NULL, FALSE, 0, NULL, NULL, &sj, &pj)){
    			printf( "CreateProcess failed (%d)\n", GetLastError() );
    		}
    		// Wait until child processes exit.
    		WaitForSingleObject( pj.hProcess, INFINITE );
    		// Close process and thread handles.
    		CloseHandle( pj.hProcess );
    		CloseHandle( pj.hThread );
    	}
    	*/
    	HINTERNET hInternet = InternetOpen(L"Patcher", INTERNET_OPEN_TYPE_DIRECT, NULL, NULL, 0);
    	DWORD err;
    	if (hInternet == NULL){
    		err=GetLastError();
    		cout<<"Internet Connection Not Detected. If you are connected, please notify smasherprog@gmail.com with this error number:"<<err<<endl;
    		InternetCloseHandle(hInternet);
            return 0;
        }
    	cout<<"Internet Connectioned Detected..."<<endl;
    	HINTERNET hConnect = InternetConnect(hInternet, L"nolimitsdesigns.com", INTERNET_DEFAULT_FTP_PORT, L"anonymous@nolimitsdesigns.com", NULL, INTERNET_SERVICE_FTP, 0, 0);
    	if (hConnect == NULL){
    		err=GetLastError();
    		cout<<"Cannot Connect to the ftp server. please notify smasherprog@gmail.com with this error number:"<<err<<endl;
    		InternetCloseHandle(hConnect);
    		InternetCloseHandle(hInternet);
            return 0;
        }
    	cout<<"Successfully connected to the ftp server.."<<endl;
    	relativesize =GetCurrentDirectory(MAX_PATH, relative);
    	if(relativesize==0){
    		cout<<"error calling GetCurrentDirectory"<<endl;
    	} else {
    		relativesize*=sizeof(wchar_t);// i do not want the null char at the end
    		wcout<<"Current relative Client Directory set to: "<<relative<<endl<<"  Lenght of Directory is: "<<relativesize<<endl;
    	}
    	cout<<"Beginning search for files... Please wait"<<endl;
    	GetFiles(hConnect);
    	InternetCloseHandle(hInternet);
    	InternetCloseHandle(hConnect);
    	
    	cout<<"Continue to load client (Y/N)?:";
    	char k;
    	cin>>k;
    	if(k=='y' || k=='Y'){
    		StartClient();
    	}
    	return 0;
    }
    void StartClient(){
    	system("Client.exe");
    	cout<<"Starting Client..";
    }
    void GetFiles(HINTERNET &hConnect){
    	WIN32_FIND_DATA SdirInfo;
    	HINTERNET       hFind;
    	DWORD           err;
    	size_t CurDirSize(0);// the number of bytes in array.. not the string lenght
    	wchar_t CurDir[MAX_PATH], ServDir[MAX_PATH];// servDir is the servers directory, curdir is the client
    	DWORD pt(MAX_PATH);
    	if(FtpGetCurrentDirectory(hConnect, CurDir, &pt)){
    		CurDirSize=wcsnlen(CurDir, MAX_PATH)*sizeof(wchar_t);// i do not want the null char at the end
    		memcpy(ServDir, CurDir, CurDirSize*sizeof(wchar_t));// make a copy before we modify
    		for(size_t i(0); i!=CurDirSize; i++){
    			if(CurDir[i]==47){// using ascii numbers because I am cool like that!
    				CurDir[i]=92;
    			}
    		}
    		wcout<<"Current Server Directory is: "<<ServDir<<endl<<"Lenght of Directory is:"<<CurDirSize<<endl;
    		if(!SetCurrentDirectory(relative)){
    			wcout<<"Failed to SetCurrentDirectory on the client to"<<relative<<endl;
    		} else {
    			wcout<<"SetCurrentDirectory on the client to"<<relative<<endl;
    		}
    		if(CurDirSize>2){// if we should change directories.. then do it
    			wchar_t newdir[MAX_PATH];
    			memcpy(newdir, relative, relativesize);
    			size_t beg(relativesize/sizeof(wchar_t));
    			size_t end(beg + (CurDirSize/sizeof(wchar_t)));
    			for( size_t i(0); beg!=end; beg++){
    				newdir[beg]=CurDir[i++];
    			}
    			newdir[end]=0;// add the null char
    			if(!CreateDirectory(newdir, NULL)){
    					err = GetLastError( );
    					if(err==ERROR_ALREADY_EXISTS){
    						wcout<<"Directory: "<<newdir<<"   Exists, continuing search."<<endl;
    					} else{
    						wcout<<"Bad error calling CreateDirectory on: "<<newdir<<endl;
    						return;
    					}
    			} else {
    				wcout<<"Created the directory: "<<newdir<<endl;
    			}
    			if(!SetCurrentDirectory(newdir)){
    				wcout<<"Failed to SetCurrentDirectory on the client to"<<newdir<<endl;
    			}
    		}// else skip this.. its the first directory / 
    	} else {
    		err = GetLastError( );
    		cout<<"Error calling get directory... "<<err<<endl;
    		return;
    	}
    	ListDirectory(hConnect);
    	hFind = FtpFindFirstFile( hConnect, NULL, &SdirInfo, INTERNET_FLAG_DONT_CACHE, 0 );
    	if ( hFind == NULL ){
    		err = GetLastError( );
    		if( err == ERROR_NO_MORE_FILES ){
    			cout<<"End of Directory listing"<<endl;
    		} else {
    			cout<<"FtpFindFirstFile Failed "<<err<<endl;
    		}
    		return;// done listing the directory get out
    	}
    	if(SdirInfo.cFileName[0]==46){// we are working on periods!! ewwww, I need a tampon
    		InternetFindNextFile( hFind, (LPVOID) &SdirInfo );
    	}
    	for( ; InternetFindNextFile( hFind, (LPVOID) &SdirInfo ); ){
    		if( SdirInfo.dwFileAttributes==FILE_ATTRIBUTE_DIRECTORY){
    			if(wcscmp(SdirInfo.cFileName , L"incoming")==0){// skip the file
    				continue;
    			}
    			wcout<<"Attempting to change directories on the ftp server to: "<<SdirInfo.cFileName<<"    ....";
    			if(FtpSetCurrentDirectory(hConnect, SdirInfo.cFileName)){
    				cout<<"Succesful"<<endl;
    				InternetCloseHandle(hFind);
    				GetFiles(hConnect);// recursive function to search through all directories
    				wcout<<"Resuming search .. Attempting to change directories on the ftp server to: "<<ServDir<<"    ....";;
    				if(FtpSetCurrentDirectory(hConnect, ServDir)){ // resuming the search, change directories back
    					cout<<"Success"<<endl;
    					wchar_t newdir[MAX_PATH];
    					memcpy(newdir, relative, relativesize);
    					size_t beg(relativesize/sizeof(wchar_t));
    					size_t end(beg + (CurDirSize/sizeof(wchar_t)));
    					for( size_t i(0); beg!=end; beg++){
    						newdir[beg]=CurDir[i++];
    					}
    					newdir[end]=0;// add the null char
    					if(!SetCurrentDirectory(newdir)){
    						wcout<<"Failed to SetCurrentDirectory on the client to"<<newdir<<endl;
    					} else {
    						wcout<<"SetCurrentDirectory on the client to"<<newdir<<endl;
    					}
    					ListDirectory(hConnect);
    					wchar_t temp[MAX_PATH];
    					size_t q(0);
    					do{
    						temp[q]=SdirInfo.cFileName[q];
    					}while(SdirInfo.cFileName[q++]!=0);
    					temp[q]=0;// add null char
    					if(!FindFile(hConnect, hFind, temp, SdirInfo)){
    						return;// cannot find the file, something is really wrong. Get out now!
    					} else {
    						continue;
    					}
    				} else {
    					err = GetLastError( );
    					wchar_t see[MAX_PATH];
    					DWORD wer(0), msd(MAX_PATH);
    					if(InternetGetLastResponseInfo(&wer, see, &msd)){
    						wcout<<"Error "<<see<<endl;
    					}
    					cout<<"Failed "<<err;
    				}
    			} else {
    				err = GetLastError( );
    				wchar_t see[MAX_PATH];
    				DWORD wer(0), msd(MAX_PATH);
    				if(InternetGetLastResponseInfo(&wer, see, &msd)){
    					wcout<<"Error "<<see<<endl;
    				}
    				cout<<"Failed"<<err;
    				return;
    			}
    		} else {// we are deadling with files, not directories
    			if(wcscmp(SdirInfo.cFileName , L".ftpquota")==0){// skip the file
    				continue;
    			}
    			WIN32_FIND_DATA tfile;
    			HANDLE t = FindFirstFile(SdirInfo.cFileName, &tfile);// this is the file on the local drive
    			SYSTEMTIME ltime, stime;
    			FileTimeToSystemTime(&tfile.ftLastWriteTime, &ltime);
    			FileTimeToSystemTime(&SdirInfo.ftLastWriteTime, &stime); 
    			if(t!=INVALID_HANDLE_VALUE){// we have the file 
    				wcout<<"File "<<tfile.cFileName<<" found on hard drive. Checking to see if this matches the file on the server...";
    				if(tfile.nFileSizeLow == SdirInfo.nFileSizeLow && ltime.wYear >= stime.wYear){// we might have the same file... do better check
    					if(ltime.wMonth > stime.wMonth){// update-to-date file..... continue
    						cout<<" Same file found. Continuing to next."<<endl;
    						continue;
    					} else if(ltime.wMonth == stime.wMonth){// check date
    						if(ltime.wDay >= stime.wDay){
    							cout<<" Same file found. Continuing to next."<<endl;
    							continue;
    						}// else we need to update... download
    					}// else the file has a month less than the servers.... download file
    				}  
    			}
    			wcout<<"File: "<<SdirInfo.cFileName<<"  Does not exist. Downloading file from the server .."<<endl;
    			FtpGetFile(hConnect, SdirInfo.cFileName, SdirInfo.cFileName, false, FILE_ATTRIBUTE_NORMAL, FTP_TRANSFER_TYPE_BINARY, 0);
    		}
    	} 
    	cout<<"End of Directory listing"<<endl;
    	InternetCloseHandle(hFind);
    }
    void ListDirectory(HINTERNET &hConnect){
    	WIN32_FIND_DATA SdirInfo;
    	HINTERNET       hFind;
    	DWORD           err;
    	cout<<"Listing Server Directory..."<<endl;
    	hFind = FtpFindFirstFile( hConnect, NULL, &SdirInfo, INTERNET_FLAG_DONT_CACHE, 0 );
    	if ( hFind == NULL ){
    		err = GetLastError( );
    		if( err == ERROR_NO_MORE_FILES ){
    			cout<<"End of Directory listing"<<endl;
    		} else {
    			cout<<"FtpFindFirstFile Failed "<<err<<endl;
    		}
    		return;// done listing the directory get out
    	}
    	wcout<<SdirInfo.cFileName<<endl;
    	for( ; InternetFindNextFile( hFind, (LPVOID) &SdirInfo ); ){
    		wcout<<SdirInfo.cFileName<<endl;
    	}
    	InternetCloseHandle(hFind);
    }
    bool FindFile(HINTERNET &hConnect, HINTERNET &hFind, LPCWSTR temp, WIN32_FIND_DATA &SdirInfo){
    	DWORD           err;
    	wcout<<"Finding the file "<<temp<<endl;
    	hFind = FtpFindFirstFile( hConnect, NULL, &SdirInfo, INTERNET_FLAG_DONT_CACHE, 0 );
    	if ( hFind == NULL ){
    		err = GetLastError( );
    		if( err == ERROR_NO_MORE_FILES ){
    			cout<<"End of Directory listing"<<endl;
    		} else {
    			cout<<"FtpFindFirstFile Failed "<<err<<endl;
    		}
    		return false;// done listing the directory get out
    	}
    	do{
    		if(wcscmp(temp, SdirInfo.cFileName)==0){
    			wcout<<"Found "<<SdirInfo.cFileName<<endl;
    			return true;
    		}
    	}while(InternetFindNextFile( hFind, (LPVOID) &SdirInfo ));
    	wcout<<"Could Not Find the File "<<temp<<endl;
    	return false;
    }
    Last edited by smasherprog; 04-14-2010 at 04:28 PM.

  7. #7
    Registered User
    Join Date
    Mar 2005
    Location
    Juneda
    Posts
    287
    "This might make you happy... "

    Of course! I'll take a look, for the moment I didn't thought about using such feature, but maybe I'll 'steal' part of your idea

    Thank's
    Niara

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Couple errors please help :-D
    By JJJIrish05 in forum C Programming
    Replies: 9
    Last Post: 03-06-2008, 01:54 AM
  2. Improvements on Recursive Directory Listing
    By laney69er in forum C Programming
    Replies: 1
    Last Post: 03-15-2006, 12:39 AM
  3. recursive directory listing help
    By laney69er in forum C Programming
    Replies: 2
    Last Post: 03-13-2006, 12:07 PM
  4. Whats FTP
    By Stan100 in forum Tech Board
    Replies: 13
    Last Post: 01-22-2004, 04:10 PM
  5. Scanning a directory on ftp
    By pinkcheese in forum Windows Programming
    Replies: 7
    Last Post: 10-13-2002, 12:01 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21