Thread: Curl Upload of files more than 2GB fails !!!

  1. #1
    Registered User
    Join Date
    Mar 2014
    Posts
    2

    Curl Upload of files more than 2GB fails !!!

    This code works fine for files less than 2GB ,but for files greater than gb It returns 200 code . But upload doesnt happen .
    The master buffer entry observed is
    (master_buffer = 0x0000000006cde170 "HTTP/1.1 200 OK Connection: close
    Cache-Control: no-cache, no-store
    task-id:OpaqueRef:ac7b2340-1d50-3830-793c-d8e56759ef1a
    content-type: application/octet-stream")

    Code:
    FILE * hd_src ;
    struct stat file_info;
    char *restoreFile = new char[file.length() + 1];
    strcpy(restoreFile,file.c_str());
    stat(restoreFile , &file_info);
    hd_src = fopen(restoreFile, "rb");
    
    curl_global_init(CURL_GLOBAL_ALL);
    CURL *curlHandle=NULL;
    curlHandle=curl_easy_init();
    
    
                
    curl_easy_setopt(curlHandle, CURLOPT_READFUNCTION, &XenServerBackup::read_callback);
    curl_easy_setopt(curlHandle, CURLOPT_NOSIGNAL, true);
    curl_easy_setopt(curlHandle, CURLOPT_NOPROGRESS, 1);
    curl_easy_setopt(curlHandle, CURLOPT_BUFFERSIZE, MAX_BUF);
    curl_easy_setopt(curlHandle, CURLOPT_SSL_VERIFYPEER, 0);            
    curl_easy_setopt(curlHandle, CURLOPT_SSL_VERIFYHOST, 0);
    curl_easy_setopt(curlHandle,CURLOPT_HTTPAUTH,CURLAUTH_BASIC;
    curl_easy_setopt(curlHandle, CURLOPT_CUSTOMREQUEST, "PUT");
    curl_easy_setopt(curlHandle, CURLOPT_URL, _urlPathName.c_str());
    curl_easy_setopt(curlHandle, CURLOPT_VERBOSE, 0);                
    curl_easy_setopt(curlHandle, CURLOPT_USERPWD, authKey.c_str());    
    curl_easy_setopt(curlHandle, CURLOPT_UPLOAD, 1L);
    curl_easy_setopt(curlHandle, CURLOPT_READDATA, hd_src);        
    curl_easy_setopt(curlHandle, CURLOPT_INFILESIZE_LARGE,(curl_off_t)file_info.st_size);
    curl_easy_setopt(curlHandle, CURLOPT_LOW_SPEED_LIMIT, 1);
    curl_easy_setopt(curlHandle, CURLOPT_LOW_SPEED_TIME, 5);
    curl_easy_setopt(curlHandle, CURLOPT_IGNORE_CONTENT_LENGTH, 1);
                
    int success = curl_easy_perform(curlHandle);    
    int error =0;
    curl_easy_getinfo(curlHandle, CURLINFO_RESPONSE_CODE, &error);
    Last edited by Salem; 03-14-2014 at 01:43 AM. Reason: Added code tags - learn to use them

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,659
    stat(2): file status - Linux man page
    Check the return result, and pay attention to the description of EOVERFLOW.

    Likewise, check curl_off_t is a 64-bit type as well.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  3. #3
    Registered User
    Join Date
    Mar 2014
    Posts
    2
    I think the problem is on the curl initialization part , Am i missing to set an curl parameter ??

  4. #4
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,659
    > I think the problem is on the curl initialization part , Am i missing to set an curl parameter ??
    And how do you arrive at that conclusion?

    Did you add some code to check for various error conditions?

    Perhaps if you posted a complete (ie, compilable) example which demonstrates the problem, we could be of further assistance.

    Just posting a snippet and then posting your guesses won't attract a great deal of interest on our part.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  5. #5
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    You could try Googling "curl 2gb limit" also. It seems to pull up a huge list of information for you to look through. Almost certainly you will find an answer to your question.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Upload and Download Files quickly
    By Jerry900 in forum C Programming
    Replies: 9
    Last Post: 10-19-2012, 09:03 AM
  2. Replies: 1
    Last Post: 01-25-2012, 11:50 AM
  3. can't open two files at once, 2nd fopen fails
    By jeffmarc in forum C Programming
    Replies: 3
    Last Post: 09-18-2009, 03:06 PM
  4. Can we upload files for future ref for others?
    By aspand in forum A Brief History of Cprogramming.com
    Replies: 4
    Last Post: 06-12-2002, 11:13 PM
  5. anyone know how to upload long sound files into a program?
    By Golden Bunny in forum C++ Programming
    Replies: 18
    Last Post: 04-11-2002, 02:21 PM

Tags for this Thread