Thread: Fast HTTP download, need a little help

  1. #1
    Registered User
    Join Date
    Oct 2008
    Posts
    5

    Question Fast HTTP download, need a little help

    Hello, I am using a Winsock DLL made specialy fro BlitzBasic.
    In the dll there is a function called tcpconnect, and it looks like this :
    Code:
    //////////////////////////////////////////////////////////////////////////////////////////////
    bool CSocket::tcpconnect(char *address, int port, int mode)
    {
    SOCKADDR_IN addr;
    LPHOSTENT hostEntry;
    sockid = socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
    u_long i = 1;
    ioctlsocket(sockid, FIONBIO,&i);
    
    if((hostEntry = gethostbyname(address))==NULL)
    {
    closesocket(sockid);
    return false;
    }
    addr.sin_family = AF_INET;
    addr.sin_addr = *((LPIN_ADDR)*hostEntry->h_addr_list);
    addr.sin_port = htons((u_short)port);
    
    connect(sockid, (LPSOCKADDR)&addr, sizeof(SOCKADDR_IN));
    
    return true;
    }
    /////////////////////////////////////////////////////////////////////////////////////////////////
    I am using this to conect to a website and download a page (html page).
    I tested this function for about 100 websites, and on most websites it takes 0.001 seconds to connect, but on other websites, it takes even 0.3 seconds to connect.

    I don't have a problem if the connection takes to long, but the problem is that by calling tcpconnect (from the DLL), if the connection is not instant, it hangs the program until it connects .

    I also have a function tcpconnected(tcpid) that checks if a tcp is connected.

    What should I modify in tcpconnect function from above, so that it automaticaly returns true, without any lags. I will check later one if it's connected.

    I made a little reasearch and I noticed that I should make the socket async.
    I am using the folowing code for tcp connect now :

    Code:
    bool CSocket::tcpconnect(char *address, int port, double h)
    	{
    	SOCKADDR_IN addr;
    	LPHOSTENT  hostEntry;
    	HWND hwin;
    	hwin=(HWND)(int)h;
    	sockid = socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
    	WSAAsyncSelect (sockid, hwin, 1045, FD_READ | FD_CONNECT | FD_CLOSE); 
    
    	hostEntry = gethostbyname(address);
    
    	addr.sin_family = AF_INET; //family
    	addr.sin_addr = *((LPIN_ADDR)*hostEntry->h_addr_list); //server ip
    	addr.sin_port = htons((u_short)port); //set port
    
    	connect(sockid, (SOCKADDR *)&addr, sizeof(addr));
    
    	return true;
    }
    I noticed that the only problem (hang) is with connect function.
    If I remove it, the function tcpconnect is executed in about 0,001 seconds.
    If I enable it, the same problem, sometimes it taes 0,02 seconds, sometimes 0,4.

    Of course, I can't connect to the server if I don't use the connect function. So is there an alternative ?
    How can I make this function be executed instantly, and as I sayed, I will check with another function if the connection is established.


    Can someone help ?
    Thank you

  2. #2
    Registered User
    Join Date
    Oct 2006
    Posts
    3,445
    Quote Originally Posted by sodevrom View Post
    How can I make this function be executed instantly, and as I sayed, I will check with another function if the connection is established.
    run connect() from within a separate thread, and set a global flag of some kind when it returns. From your main thread, periodically check for the flag to be set, and once you see it set, continue with your program.

  3. #3
    Registered User
    Join Date
    Oct 2008
    Posts
    5
    Hi, do you think you can help me with this ?
    I am not an expert with C++, I programm in a different programming language, that is not nearly as hard as C++
    I would really appreciate this.
    Thank you

  4. #4
    Registered User
    Join Date
    Oct 2008
    Posts
    5
    Hi again,
    I have done some research about threads, but unfortunatly I didn't manage to make this work. I will show you below a sample class (similar to what I am using), and I would appreciate if you could guide me a little.

    Code:
    class MC
    {
    int a,b;
    public :
    int calculate_sum(int a,int b);
    };
    
    int MC::calculate_sum(int a,int b)
        {
        this->a=a;
        this->b=b;
        return a+b; 
        }
    Of course this just just a sample, I know you don't need a thread to calculate a+b . The ideea is that calculate_sum recieves parameters a and b.

    How can I make calculate_sum run in a thread ?
    What should I add/modify ?

    Thank you
    Last edited by sodevrom; 10-14-2008 at 06:45 PM.

  5. #5
    Registered User
    Join Date
    Oct 2006
    Posts
    3,445
    As far as threading goes, I'm most familiar with the pthreads implementation, which is fully portable across dozens of platforms.

    here's a basic example;
    Code:
    #include <pthread.h>
    #include <iostream>
    
    using std::cout;
    using std::endl;
    
    // this is the function that executes when the threads are created.
    // it takes a void* parameter, which could just as easily be any type of pointer
    void* thread_proc(void* ptr)
    {
      cout << "String passed to thread was: " << (char*)ptr << endl;
      pthread_exit(NULL);
    }
    
    int main(void)
    {
      // declare two pthread_t objects to keep track of the threads
      // as far as I know, they are just integer types
      pthread_t thread1, thread2;
    
      // two strings to send to the threads.
      char* str1 = "This is thread1.";
      char* str2 = "This is thread2.";
    
      // create the threads
      pthread_create(&thread1, NULL, thread_proc, str1);
      pthread_create(&thread2, NULL, thread_proc, str2);
    
      cout << "In main thread." << endl;
    
      // wait for both threads to finish before continuing
      pthread_join(thread1, NULL);
      pthread_join(thread2, NULL);
    
      // done so return
      return 0;
    }
    if you install the pthreads libraries for windows, you can use code like this in your windows apps.

    One thing you need to remember is that a thread cannot be started with a non-static class member function. It's not a big deal though, as you can create a static member function that takes a pointer that it will use as "this".

    This example should get you started, and I know lots of people here on this board are familiar with pthreads, so finding help should be no problem if you run into trouble. For further reading, I suggest http://www.yolinux.com/TUTORIALS/Lin...ixThreads.html. it is specifically for linux, but all of the code that is not platform-specific will work on windows as well if you have pthreads installed.

    here is an example of how you might make a thread class that will spawn a new thread for each object that is created:
    Code:
    class function_object
    {
      public:
        void operator () (void* ptr)
        {
          cout "string: " << ptr << endl;
        }
    };
    
    template<class T>
    class Thread
    {
      public:
        Thread(void* param)
        {
          pthread_create(&thread, NULL, Thread::thread_proc, param);
          pthread_join(thread, NULL);
        }
    
        static void* thread_proc(void* ptr)
        {
          T* obj = new T();
          obj(ptr);
          delete obj;
          pthread_exit(NULL);
        }
      private:
        pthread_t thread;
    };
    
    // To create a new thread, you would use the following code:
    Thread<function_object>* mythread = new Thread<function_object>(some_pointer);

  6. #6
    Registered User
    Join Date
    Oct 2008
    Posts
    5
    Hi,
    Thank you for the help, but unfortunatly I can't manage to make this work.
    Can you show me on my example how can I call the function : MC::calculate_sum threaded ?
    As you see, it has 2 parameters, not only one.
    Also, I would prefer to use the function : AfxBeginThread to start the thread.

    Sorry if I am beeing a nag, but I am not very advanced in c++, and If I see how I can call calculate_sum using a thread, I will find my way.

    Thanx

  7. #7
    Registered User
    Join Date
    Oct 2006
    Posts
    3,445
    Sorry, but pthreads only lets you pass one parameter to the thread procedure. You could declare a struct or class to send it, and then you could have all the parameters you want.

  8. #8
    Registered User
    Join Date
    Oct 2008
    Posts
    5
    I am trying very hard to make this work, but I can't. I need a sample source code to make this possible.
    Please help me someone.

  9. #9
    Banned master5001's Avatar
    Join Date
    Aug 2001
    Location
    Visalia, CA, USA
    Posts
    3,685
    Sorry... I am only a C+ expert. Can you at least try to do as Elkvis just said and then ask for help when your attempts lead no where?

  10. #10
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Chappell Hill, Texas
    Posts
    2,332
    Quote Originally Posted by Elkvis View Post
    As far as threading goes, I'm most familiar with the pthreads implementation, which is fully portable across dozens of platforms.

    here's a basic example;
    Code:
    #include <pthread.h>
    #include <iostream>
    
    using std::cout;
    using std::endl;
    
    // this is the function that executes when the threads are created.
    // it takes a void* parameter, which could just as easily be any type of pointer
    void* thread_proc(void* ptr)
    {
      cout << "String passed to thread was: " << (char*)ptr << endl;
      pthread_exit(NULL);
    }
    
    int main(void)
    {
      // declare two pthread_t objects to keep track of the threads
      // as far as I know, they are just integer types
      pthread_t thread1, thread2;
    
      // two strings to send to the threads.
      char* str1 = "This is thread1.";
      char* str2 = "This is thread2.";
    
      // create the threads
      pthread_create(&thread1, NULL, thread_proc, str1);
      pthread_create(&thread2, NULL, thread_proc, str2);
    
      cout << "In main thread." << endl;
    
      // wait for both threads to finish before continuing
      pthread_join(thread1, NULL);
      pthread_join(thread2, NULL);
    
      // done so return
      return 0;
    }
    if you install the pthreads libraries for windows, you can use code like this in your windows apps.

    One thing you need to remember is that a thread cannot be started with a non-static class member function. It's not a big deal though, as you can create a static member function that takes a pointer that it will use as "this".

    This example should get you started, and I know lots of people here on this board are familiar with pthreads, so finding help should be no problem if you run into trouble. For further reading, I suggest http://www.yolinux.com/TUTORIALS/Lin...ixThreads.html. it is specifically for linux, but all of the code that is not platform-specific will work on windows as well if you have pthreads installed.

    here is an example of how you might make a thread class that will spawn a new thread for each object that is created:
    Code:
    class function_object
    {
      public:
        void operator () (void* ptr)
        {
          cout "string: " << ptr << endl;
        }
    };
    
    template<class T>
    class Thread
    {
      public:
        Thread(void* param)
        {
          pthread_create(&thread, NULL, Thread::thread_proc, param);
          pthread_join(thread, NULL);
        }
    
        static void* thread_proc(void* ptr)
        {
          T* obj = new T();
          obj(ptr);
          delete obj;
          pthread_exit(NULL);
        }
      private:
        pthread_t thread;
    };
    
    // To create a new thread, you would use the following code:
    Thread<function_object>* mythread = new Thread<function_object>(some_pointer);
    Elkvis, thanks for postiing this example. I compiled it and ran it. It appears that std::cout does not lock the output stream. I got this output:

    Code:
    String passed to thread was: In main thread.This is thread1.String passed to thread was: 
    
    This is thread2.
    (I did change this one line:
    Code:
     cout << "In main thread." << '\n' << std::endl;
    Mainframe assembler programmer by trade. C coder when I can.

  11. #11
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Chappell Hill, Texas
    Posts
    2,332
    Anyway, (to complete my thought - I was interrupted), what mechanism would be used to enqueue the output stream so messages don' get interspersed within other messages?
    Mainframe assembler programmer by trade. C coder when I can.

  12. #12
    Registered User
    Join Date
    Oct 2006
    Posts
    3,445
    in a single process (which may contain many threads), all threads have access to the same console. in fact, in windows, and in unix-like operating systems, no process may attach to more than one console. pthreads also has the capability to use something called a mutex. what a mutex does is basically lets the threading system know that the thread that is locking the mutex wants exclusive access to global objects (like the console), and will unlock the mutex when it is finished doing what it is doing. an example of this might be the following updated version of the first example I gave you:

    Code:
    #include <pthread.h>
    #include <iostream>
    
    using std::cout;
    using std::endl;
    
    pthread_mutex my_mutex = PTHREAD_MUTEX_INITIALIZER;
    
    // this is the function that executes when the threads are created.
    // it takes a void* parameter, which could just as easily be any type of pointer
    void* thread_proc(void* ptr)
    {
      // lock the mutex so that only this bit of code can access global objects
      pthread_mutex_lock(&my_mutex);
    
      cout << "String passed to thread was: " << (char*)ptr << endl;
    
      // we're done, so unlock it
      pthread_mutex_unlock(&my_mutex);
      pthread_exit(NULL);
    }
    
    int main(void)
    {
      // declare two pthread_t objects to keep track of the threads
      // as far as I know, they are just integer types
      pthread_t thread1, thread2;
    
      // two strings to send to the threads.
      char* str1 = "This is thread1.";
      char* str2 = "This is thread2.";
    
      // create the threads
      pthread_create(&thread1, NULL, thread_proc, str1);
      pthread_create(&thread2, NULL, thread_proc, str2);
    
      pthread_mutex_lock(&my_mutex);
      cout << "In main thread." << endl;
      pthread_mutex_unlock(&my_mutex);
    
      // wait for both threads to finish before continuing
      pthread_join(thread1, NULL);
      pthread_join(thread2, NULL);
    
      // done so return
      return 0;
    }

  13. #13
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Chappell Hill, Texas
    Posts
    2,332
    Thanks. Worked perfect. One small typo:

    Code:
    pthread_mutex_t my_mutex = PTHREAD_MUTEX_INITIALIZER;
    Mainframe assembler programmer by trade. C coder when I can.

  14. #14
    Registered User
    Join Date
    Oct 2006
    Posts
    3,445
    Quote Originally Posted by Dino View Post
    One small typo
    sorry.... it was really late at night

  15. #15
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Chappell Hill, Texas
    Posts
    2,332
    np - it caused me to actually lift a finger and look something up!
    Mainframe assembler programmer by trade. C coder when I can.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Download File through HTTP Proxy
    By cyberCLoWn in forum Windows Programming
    Replies: 6
    Last Post: 06-19-2007, 01:40 AM
  2. http download
    By keeper in forum C++ Programming
    Replies: 11
    Last Post: 05-24-2006, 05:11 AM
  3. Replies: 4
    Last Post: 09-30-2005, 02:51 AM
  4. IE 6 status bar
    By DavidP in forum Tech Board
    Replies: 15
    Last Post: 10-23-2002, 05:31 PM