Read from URL website to a .txt file

This is a discussion on Read from URL website to a .txt file within the C Programming forums, part of the General Programming Boards category; Hi everyone, new on this site, but hope someone can help me solve this problem. I need to read the ...

  1. #1
    Registered User
    Join Date
    Apr 2011
    Posts
    4

    Smile Read from URL website to a .txt file

    Hi everyone, new on this site, but hope someone can help me solve this problem.


    I need to read the text from this URL in to a text file on my computer... but can't figure out how to do it.
    URL: http://www.radar4u.com/REAB_groto/downld02.txt



    I've tried open it like a regular txt file, figured the program wouldn't notice the difference since it is a .txt file (only it is on the web)
    No problems if I download it first and the write it over to another file, but not when I'm trying directly from the URL. If it can't be solved with C code C++ works to, but prefer C.

    Please help
    Last edited by Hinken777; 04-07-2011 at 07:29 AM.

  2. #2
    Registered User claudiu's Avatar
    Join Date
    Feb 2010
    Location
    London, United Kingdom
    Posts
    2,094
    Show us what you have tried so far. This is not a homework service.
    1. Get rid of gets(). Never ever ever use it again. Replace it with fgets() and use that instead.
    2. Get rid of void main and replace it with int main(void) and return 0 at the end of the function.
    3. Get rid of conio.h and other antiquated DOS crap headers.
    4. Don't cast the return value of malloc, even if you always always always make sure that stdlib.h is included.

  3. #3
    Registered User
    Join Date
    Apr 2011
    Posts
    4
    This is one of my attempts
    Reads the file in to memory and then write it to file, if it would work...
    works fine if I change webURL to a file on my computer

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    int main () {
      FILE * pFile;
      long lSize;
      char * buffer;
      size_t result;
    
      pFile = fopen ("http://www.radar4u.com/REAB_groto/downld02.txt" , "rb" );
      if (pFile==NULL) {fputs ("File error",stderr); exit (1);}
    
      // obtain file size:
      fseek (pFile , 0 , SEEK_END);
      lSize = ftell (pFile);
      rewind (pFile);
    
      // allocate memory to contain the whole file:
      buffer = (char*) malloc (sizeof(char)*lSize);
      if (buffer == NULL) {fputs ("Memory error",stderr); exit (2);}
    
      // copy the file into the buffer:
      result = fread (buffer,1,lSize,pFile);
      if (result != lSize) {fputs ("Reading error",stderr); exit (3);}
    
      /* the whole file is now loaded in the memory buffer. */
      // Print on screen or to file...
    //  printf(": %s", buffer);
      FILE * wFile;
      wFile = fopen ("skriv.txt","w");
      fputs (buffer,wFile);
    
    
      // terminate
      fclose (wFile);
      fclose (pFile);
      free (buffer);
      system("PAUSE");
      return 0;
    }

  4. #4
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    I think you'll find that communicating on the internet is just a tad more difficult than simply opening a file....

    HTTP Made Really Easy

  5. #5
    a_capitalist_story
    Join Date
    Dec 2007
    Posts
    2,648

  6. #6
    Registered User
    Join Date
    Apr 2011
    Posts
    4
    I think you'll find that communicating on the internet is just a tad more difficult than simply opening a file....

    HTTP Made Really Easy


    __________________
    //Tater
    Read a bit from your link.
    And this may work some how... But this is HTML programing?!, is it possible to implement this in C??

    Sample HTTP Exchange
    Code:
    To retrieve the file at the URL
    
        http://www.somehost.com/path/file.html
    
    first open a socket to the host www.somehost.com, port 80 (use the default port of 80 because none is specified in the URL). Then, send something like the following through the socket:
    
        GET /path/file.html HTTP/1.0
        From: someuser@jmarshall.com
        User-Agent: HTTPTool/1.0
        [blank line here]
    
    The server should respond with something like the following, sent back through the same socket:
    
        HTTP/1.0 200 OK
        Date: Fri, 31 Dec 1999 23:59:59 GMT
        Content-Type: text/html
        Content-Length: 1354
    
        <html>
        <body>
        <h1>Happy New Millennium!</h1>
        (more file contents)
          .
          .
          .
        </body>
        </html>
    
    After sending the response, the server closes the socket.
    Last edited by Hinken777; 04-07-2011 at 11:45 AM.

  7. #7
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Yes you can do it in C ... but you'll need a working knowlege of socket (network) programming before you start.

    There are a couple of libraries that pretend to make it easier that you might try. But even then it's still some pretty advanced programming.

  8. #8
    Registered User
    Join Date
    Apr 2011
    Posts
    4
    I was afraid of this :/ but which librarys are those?? Can see how much of it I understand...

  9. #9
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. C++ std routines
    By siavoshkc in forum C++ Programming
    Replies: 33
    Last Post: 07-28-2006, 12:13 AM
  2. Dikumud
    By maxorator in forum C++ Programming
    Replies: 1
    Last Post: 10-01-2005, 06:39 AM
  3. Replies: 3
    Last Post: 03-04-2005, 01:46 PM
  4. :| file i/o read last line of .txt
    By Unregistered in forum C++ Programming
    Replies: 1
    Last Post: 03-18-2002, 11:29 AM
  5. what does this mean to you?
    By pkananen in forum C++ Programming
    Replies: 8
    Last Post: 02-04-2002, 02:58 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21