Fast HTTP download, need a little help

Hello, I am using a Winsock DLL made specialy fro BlitzBasic.
In the dll there is a function called tcpconnect, and it looks like this :

Code:

---

//////////////////////////////////////////////////////////////////////////////////////////////
bool CSocket::tcpconnect(char *address, int port, int mode)
{
SOCKADDR_IN addr;
LPHOSTENT hostEntry;
sockid = socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
u_long i = 1;
ioctlsocket(sockid, FIONBIO,&i);

if((hostEntry = gethostbyname(address))==NULL)
{
closesocket(sockid);
return false;
}
addr.sin_family = AF_INET;
addr.sin_addr = *((LPIN_ADDR)*hostEntry->h_addr_list);
addr.sin_port = htons((u_short)port);

connect(sockid, (LPSOCKADDR)&addr, sizeof(SOCKADDR_IN));

return true;
}
/////////////////////////////////////////////////////////////////////////////////////////////////

---

I am using this to conect to a website and download a page (html page).
I tested this function for about 100 websites, and on most websites it takes 0.001 seconds to connect, but on other websites, it takes even 0.3 seconds to connect.

I don't have a problem if the connection takes to long, but the problem is that by calling tcpconnect (from the DLL), if the connection is not instant, it hangs the program until it connects .

I also have a function tcpconnected(tcpid) that checks if a tcp is connected.

What should I modify in tcpconnect function from above, so that it automaticaly returns true, without any lags. I will check later one if it's connected.

I made a little reasearch and I noticed that I should make the socket async.
I am using the folowing code for tcp connect now :

Code:

---

bool CSocket::tcpconnect(char *address, int port, double h)
        {
        SOCKADDR_IN addr;
        LPHOSTENT  hostEntry;
        HWND hwin;
        hwin=(HWND)(int)h;
        sockid = socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
        WSAAsyncSelect (sockid, hwin, 1045, FD_READ | FD_CONNECT | FD_CLOSE);

        hostEntry = gethostbyname(address);

        addr.sin_family = AF_INET; //family
        addr.sin_addr = *((LPIN_ADDR)*hostEntry->h_addr_list); //server ip
        addr.sin_port = htons((u_short)port); //set port

        connect(sockid, (SOCKADDR *)&addr, sizeof(addr));

        return true;
}

---

I noticed that the only problem (hang) is with connect function.
If I remove it, the function tcpconnect is executed in about 0,001 seconds.
If I enable it, the same problem, sometimes it taes 0,02 seconds, sometimes 0,4.

Of course, I can't connect to the server if I don't use the connect function. So is there an alternative ?
How can I make this function be executed instantly, and as I sayed, I will check with another function if the connection is established.


Can someone help ?
Thank you

Quote:

---

Originally Posted by sodevrom

How can I make this function be executed instantly, and as I sayed, I will check with another function if the connection is established.

---

run connect() from within a separate thread, and set a global flag of some kind when it returns. From your main thread, periodically check for the flag to be set, and once you see it set, continue with your program.

Hi, do you think you can help me with this ?
I am not an expert with C++, I programm in a different programming language, that is not nearly as hard as C++
I would really appreciate this.
Thank you

Hi again,
I have done some research about threads, but unfortunatly I didn't manage to make this work. I will show you below a sample class (similar to what I am using), and I would appreciate if you could guide me a little.

Code:

---

class MC
{
int a,b;
public :
int calculate_sum(int a,int b);
};

int MC::calculate_sum(int a,int b)
    {
    this->a=a;
    this->b=b;
    return a+b;
    }

---

Of course this just just a sample, I know you don't need a thread to calculate a+b :D. The ideea is that calculate_sum recieves parameters a and b.

How can I make calculate_sum run in a thread ?
What should I add/modify ?

Thank you

As far as threading goes, I'm most familiar with the pthreads implementation, which is fully portable across dozens of platforms.

here's a basic example;

Code:

---

#include <pthread.h>
#include <iostream>

using std::cout;
using std::endl;

// this is the function that executes when the threads are created.
// it takes a void* parameter, which could just as easily be any type of pointer
void* thread_proc(void* ptr)
{
  cout << "String passed to thread was: " << (char*)ptr << endl;
  pthread_exit(NULL);
}

int main(void)
{
  // declare two pthread_t objects to keep track of the threads
  // as far as I know, they are just integer types
  pthread_t thread1, thread2;

  // two strings to send to the threads.
  char* str1 = "This is thread1.";
  char* str2 = "This is thread2.";

  // create the threads
  pthread_create(&thread1, NULL, thread_proc, str1);
  pthread_create(&thread2, NULL, thread_proc, str2);

  cout << "In main thread." << endl;

  // wait for both threads to finish before continuing
  pthread_join(thread1, NULL);
  pthread_join(thread2, NULL);

  // done so return
  return 0;
}

---

if you install the pthreads libraries for windows, you can use code like this in your windows apps.

One thing you need to remember is that a thread cannot be started with a non-static class member function. It's not a big deal though, as you can create a static member function that takes a pointer that it will use as "this".

This example should get you started, and I know lots of people here on this board are familiar with pthreads, so finding help should be no problem if you run into trouble. For further reading, I suggest http://www.yolinux.com/TUTORIALS/Lin...ixThreads.html. it is specifically for linux, but all of the code that is not platform-specific will work on windows as well if you have pthreads installed.

here is an example of how you might make a thread class that will spawn a new thread for each object that is created:

Code:

---

class function_object
{
  public:
    void operator () (void* ptr)
    {
      cout "string: " << ptr << endl;
    }
};

template<class T>
class Thread
{
  public:
    Thread(void* param)
    {
      pthread_create(&thread, NULL, Thread::thread_proc, param);
      pthread_join(thread, NULL);
    }

    static void* thread_proc(void* ptr)
    {
      T* obj = new T();
      obj(ptr);
      delete obj;
      pthread_exit(NULL);
    }
  private:
    pthread_t thread;
};

// To create a new thread, you would use the following code:
Thread<function_object>* mythread = new Thread<function_object>(some_pointer);

---

Hi,
Thank you for the help, but unfortunatly I can't manage to make this work.
Can you show me on my example how can I call the function : MC::calculate_sum threaded ?
As you see, it has 2 parameters, not only one.
Also, I would prefer to use the function : AfxBeginThread to start the thread.

Sorry if I am beeing a nag, but I am not very advanced in c++, and If I see how I can call calculate_sum using a thread, I will find my way.

Thanx

Sorry, but pthreads only lets you pass one parameter to the thread procedure. You could declare a struct or class to send it, and then you could have all the parameters you want.

I am trying very hard to make this work, but I can't. I need a sample source code to make this possible.
Please help me someone.

Sorry... I am only a C+ expert. Can you at least try to do as Elkvis just said and then ask for help when your attempts lead no where?

Quote:

---

Originally Posted by Elkvis

As far as threading goes, I'm most familiar with the pthreads implementation, which is fully portable across dozens of platforms.

here's a basic example;

Code:

---

#include <pthread.h>
#include <iostream>

using std::cout;
using std::endl;

// this is the function that executes when the threads are created.
// it takes a void* parameter, which could just as easily be any type of pointer
void* thread_proc(void* ptr)
{
  cout << "String passed to thread was: " << (char*)ptr << endl;
  pthread_exit(NULL);
}

int main(void)
{
  // declare two pthread_t objects to keep track of the threads
  // as far as I know, they are just integer types
  pthread_t thread1, thread2;

  // two strings to send to the threads.
  char* str1 = "This is thread1.";
  char* str2 = "This is thread2.";

  // create the threads
  pthread_create(&thread1, NULL, thread_proc, str1);
  pthread_create(&thread2, NULL, thread_proc, str2);

  cout << "In main thread." << endl;

  // wait for both threads to finish before continuing
  pthread_join(thread1, NULL);
  pthread_join(thread2, NULL);

  // done so return
  return 0;
}

---

if you install the pthreads libraries for windows, you can use code like this in your windows apps.

One thing you need to remember is that a thread cannot be started with a non-static class member function. It's not a big deal though, as you can create a static member function that takes a pointer that it will use as "this".

This example should get you started, and I know lots of people here on this board are familiar with pthreads, so finding help should be no problem if you run into trouble. For further reading, I suggest http://www.yolinux.com/TUTORIALS/Lin...ixThreads.html. it is specifically for linux, but all of the code that is not platform-specific will work on windows as well if you have pthreads installed.

here is an example of how you might make a thread class that will spawn a new thread for each object that is created:

Code:

---

class function_object
{
  public:
    void operator () (void* ptr)
    {
      cout "string: " << ptr << endl;
    }
};

template<class T>
class Thread
{
  public:
    Thread(void* param)
    {
      pthread_create(&thread, NULL, Thread::thread_proc, param);
      pthread_join(thread, NULL);
    }

    static void* thread_proc(void* ptr)
    {
      T* obj = new T();
      obj(ptr);
      delete obj;
      pthread_exit(NULL);
    }
  private:
    pthread_t thread;
};

// To create a new thread, you would use the following code:
Thread<function_object>* mythread = new Thread<function_object>(some_pointer);

---

---

Elkvis, thanks for postiing this example. I compiled it and ran it. It appears that std::cout does not lock the output stream. I got this output:

Code:

---

String passed to thread was: In main thread.This is thread1.String passed to thread was:

This is thread2.

---

(I did change this one line:

Code:

---

cout << "In main thread." << '\n' << std::endl;

---

Anyway, (to complete my thought - I was interrupted), what mechanism would be used to enqueue the output stream so messages don' get interspersed within other messages?

in a single process (which may contain many threads), all threads have access to the same console. in fact, in windows, and in unix-like operating systems, no process may attach to more than one console. pthreads also has the capability to use something called a mutex. what a mutex does is basically lets the threading system know that the thread that is locking the mutex wants exclusive access to global objects (like the console), and will unlock the mutex when it is finished doing what it is doing. an example of this might be the following updated version of the first example I gave you:

Code:

---

#include <pthread.h>
#include <iostream>

using std::cout;
using std::endl;

pthread_mutex my_mutex = PTHREAD_MUTEX_INITIALIZER;

// this is the function that executes when the threads are created.
// it takes a void* parameter, which could just as easily be any type of pointer
void* thread_proc(void* ptr)
{
  // lock the mutex so that only this bit of code can access global objects
  pthread_mutex_lock(&my_mutex);

  cout << "String passed to thread was: " << (char*)ptr << endl;

  // we're done, so unlock it
  pthread_mutex_unlock(&my_mutex);
  pthread_exit(NULL);
}

int main(void)
{
  // declare two pthread_t objects to keep track of the threads
  // as far as I know, they are just integer types
  pthread_t thread1, thread2;

  // two strings to send to the threads.
  char* str1 = "This is thread1.";
  char* str2 = "This is thread2.";

  // create the threads
  pthread_create(&thread1, NULL, thread_proc, str1);
  pthread_create(&thread2, NULL, thread_proc, str2);

  pthread_mutex_lock(&my_mutex);
  cout << "In main thread." << endl;
  pthread_mutex_unlock(&my_mutex);

  // wait for both threads to finish before continuing
  pthread_join(thread1, NULL);
  pthread_join(thread2, NULL);

  // done so return
  return 0;
}

---

Thanks. Worked perfect. One small typo:

Code:

---

pthread_mutex_t my_mutex = PTHREAD_MUTEX_INITIALIZER;

---

Quote:

---

Originally Posted by Dino

One small typo

---

sorry.... it was really late at night :)

np - it caused me to actually lift a finger and look something up!

Quote:

---

Originally Posted by Dino

[/code]
(I did change this one line:

Code:

---

cout << "In main thread." << '\n' << std::endl;

---

---

the \n is completely unnecessary. std::endl does exactly the same thing, and is portable. on all platforms it will be the endline character(s) for the current platform. on some it's \r\n, and others it's just \n. what's interesting is that in that line, the program is making several function calls behind the scenes. it's writing "In main thread." to std::cout, and then it's writing std::endl. this means that the thread scheduler can interrupt between them, and you could get something else printed between "In main thread." and std::endl, unless you're using mutexes.

Quote:

---

Originally Posted by Elkvis

the \n is completely unnecessary. std::endl does exactly the same thing, and is portable. on all platforms it will be the endline character(s) for the current platform. on some it's \r\n, and others it's just \n. what's interesting is that in that line, the program is making several function calls behind the scenes. it's writing "In main thread." to std::cout, and then it's writing std::endl. this means that the thread scheduler can interrupt between them, and you could get something else printed between "In main thread." and std::endl, unless you're using mutexes.

---

I know that - I just was being totally up front in posting my output so that when you saw the extra line break, you didn't freak out. Adding the \n was just a little "what if" test I did when observing the behavior.

Thanks again for the tutorial! A condensed, clean up version of this thread should go in the FAQs.

Todd