Sockets... Sending Binary Data. Please help

This is a discussion on Sockets... Sending Binary Data. Please help within the C++ Programming forums, part of the General Programming Boards category; I am working on a socket library in windows. I am able to send a file but there are 2 ...

  1. #1
    Registered User
    Join Date
    Jan 2007
    Posts
    38

    Exclamation Sockets... Sending Binary Data. Please help

    I am working on a socket library in windows. I am able to send a file but there are 2 problems I am encountering.

    Issue 1:

    IF the file is small, say 120 bytes When i send the file from server to client the files does not send UNLESS I put in a sleep(1) after my send (server).

    Issue 2:

    The Client seems to be revieving 1 extra recieve and adding BUFF_SIZE amount of data extra to every single binary file that is sent. I have also noticed that at time the client does not recieve _BUFFSIZE bytes, the odd time (very odd) the number is less the _BUFFSIZE bytes recieved.

    NOTES:

    _BUFFSIZE = 100 for these example, I have played around w/ larger and smaller buffer sizes and I have same result.

    I am atempting to send various files all w. same results (.mp3, .wma, .txt, .doc, .avi)

    Here is my Send and Reveive from my socket Lib... I have made 100% certain my program only calls these 1 time each.

    There is some trivial code (counters) in here for my debuging purposes.

    Server
    Code:
    int ServerFileTransfer::sendFile( std::string const& strFileName )
    {
    	ifstream transferFile(strFileName.c_str(), ios::binary);
    
    	int bytes = -1;
    
    	if ( transferFile.is_open() )
    	{
    		ULONG filelength;
    		transferFile.seekg(0, ios::end);
    		filelength = transferFile.tellg();
    
    		// put back to start of file for reading
    		transferFile.seekg(0, ios::beg);
    
    		TCHAR sendBuff[ _BUFFSIZE ] = {0};
    
    		int count = 0;
    		ULONG bytesSent = 0;
    		for( bytesSent; bytesSent < filelength && !transferFile.eof(); bytesSent += _BUFFSIZE)
    		{
    			transferFile.read( sendBuff, _BUFFSIZE );
    			bytes = send( _cliSocketData, sendBuff, _BUFFSIZE * sizeof( TCHAR ), 0 );
    			++count;
    			if ( !isSocketOK( bytes ) )
    				return 0;
    		}
    
    		bytesSent -= _BUFFSIZE;
    
    		if ( filelength > bytesSent )
    		{
    			ULONG bytesRem = filelength - bytesSent;
    			transferFile.read( sendBuff, bytesRem);
    			bytes = send( _cliSocketData, sendBuff, bytesRem * sizeof( TCHAR ), 0 );
    			send
    			++count;
    			if ( !isSocketOK( bytes ) )
    				return 0;
    		}
    		transferFile.close();
    	}
    
    	return true;
    }
    Client
    Code:
    int ClientFileTransfer::receiveFile( std::string& strFileName  )
    {
    	TCHAR recvBuff[ _BUFFSIZE ];
    	int buffLen = _BUFFSIZE * sizeof( TCHAR );
    	int bytes = 0;
    
    	bool isOK = true;
    
    
    	ofstream outFile(strFileName.c_str(), ios::binary);
    	static ULONG bytesRecv = 0;
    	int count = 0;
    	while ((bytes = recv( _dataSock, recvBuff, buffLen, 0 )) > 0)
    	{
    		if ( bytes != _BUFFSIZE )
    			int x=0;
    		bytesRecv += bytes;
    		if (bytes != 100)
    			int x = 0;
    		TransmitFile()
    		outFile.write(recvBuff, bytes);
    		if( !( isOK = isSocketOK( bytes ) ))
    			break;
    		count++;
    	}
    	outFile.close();
    
    	return isOK;
    }
    Feel free to make any suggestions, I will try them all and give feedback on how I make out.

    Thanks,

  2. #2
    int x = *((int *) NULL); Cactus_Hugger's Avatar
    Join Date
    Jul 2003
    Location
    Banks of the River Styx
    Posts
    902
    EOF checking might very well be your problem here. I don't use the C++/STL I/O classes a lot, so I'm no expert there. But, here's the segment of concern:
    Code:
    transferFile.read( sendBuff, _BUFFSIZE );
    bytes = send( _cliSocketData, sendBuff, _BUFFSIZE * sizeof( TCHAR ), 0 );
    What if read() hits the end of the file, and that last chunk isn't exactly _BUFFSIZE bytes? read() fails, and the buffer is not completely filled. However, you send it as if it were full anyways. You seem to try to check for this after the for() loop, but it's too late by then.

    Here's what I'd do:
    Try to fill the buffer. Immediately after, check for eof. If you've hit one, see how many bytes you've read (is gcount() > 0?). If it ==0, break your loop. If it is >0, send, but only send gcount(). Something like:
    Code:
    do
    {
       transferFile.read(sendbuf, _BUFFSIZE);
       if(transferFile.eof() && transferFile.gcount() == 0) break;
       bytes = send(your_socket, sendbuf, transferFile.gcount(), 0);
    } while(!transferFile.eof());
    Three assumptions not to make:
    1) If the C++ eof() works like the C one (and I'm 95&#37; sure it does), then it will only return true after the EOF has been read in, NOT if the next operation will return EOF.
    2) _BUFSIZE == _BUFSIZE * sizeof(char) : You use different sizes in read() and send(). Don't. In fact, leave TCHAR out completely - use something like unsigned char (or my favourite, uint8_t) - it's a file, and you're just doing a binary transfer. TCHAR may not be 1 byte depending on your settings, and it needlessly reduces portability. (And are you really sending characters? Not for MP3, WMA, etc - raw data, better represented with unsigned char/uint8_t imho.)
    3) Don't assume send() will actually send everything. I didn't correct this in my above snippet, but send() might not actually send() everything you pass it. Check send()'s return value - if it's less than the size you passed to it, you need to call it again, starting where it left off until it actually sends the whole buffer. You may want to wrap it in a small function to do this. See the man page.

    And of course, error checking never hurts.
    For odd data being sent, try WireShark or Ethereal - they'll allow you to see what data is really being sent, and are an invaluable tool when network programs go haywire. You may consider downloading one (they're both free). Get the server to send the correct stuff first, then work on recv'ing it.
    Last edited by Cactus_Hugger; 03-28-2007 at 10:27 PM.
    long time; /* know C? */
    Unprecedented performance: Nothing ever ran this slow before.
    Any sufficiently advanced bug is indistinguishable from a feature.
    Real Programmers confuse Halloween and Christmas, because dec 25 == oct 31.
    The best way to accelerate an IBM is at 9.8 m/s/s.
    recursion (re - cur' - zhun) n. 1. (see recursion)

  3. #3
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    Thank, I ll be sure to try everything out tomorrow before dinner. its a little late now and Im going to bed. One thing I wanted to point out that I may not have made clear is that when I say that in the client receive < _BUFFSIZE this does not happen at the EOF, it happens in the middle of the transfer.

    And I think the eof is working correctly ( i may be wrong ) but if I send a file that is 30554 bytes and the _BUFFSIZE is 1000, the last 554 bytes do get sent fine. But again, for some reason I end up w/ an extrea 1000 bytes. So Cliend side the file would be 31554.

    So if anyone else has suggestion or see's something that may be causing my issues please let me know. Again i will post an update w/ my changes tomorrow if the above suggestions help out any.

    Thanks,

  4. #4
    CSharpener vart's Avatar
    Join Date
    Oct 2006
    Location
    Rishon LeZion, Israel
    Posts
    6,474
    When the last read failed - the buffer stil contains 554 bytes of previous read and EOF
    you send this buffer second time and it is appended to the file - removing the old EOF symbol and attaching the new one... So you get your strange size...

    As was said - check the number of bytes actually read from the file and send exactly this number of bytes instead of your current approach
    The first 90% of a project takes 90% of the time,
    the last 10% takes the other 90% of the time.

  5. #5
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    Thanks guys, this worked very well.

    However the other problem of small files not sending any bytes exists.

    Any idea why this could be.

  6. #6
    Lean Mean Coding Machine KONI's Avatar
    Join Date
    Mar 2007
    Location
    Luxembourg, Europe
    Posts
    444
    Just guessing, couldn't it be a problem of flushing the transfer channel ?

  7. #7
    int x = *((int *) NULL); Cactus_Hugger's Avatar
    Join Date
    Jul 2003
    Location
    Banks of the River Styx
    Posts
    902
    Now might be a good time to repost what has changed in your code, since the problem seems to have moved to a different location.
    Tools such as Wireshark/Ethereal are invaluable for seeing what's actually sent, but here's another thought: If you have Putty, you can connect to a server in Raw mode, and it'll show you what's being sent. Then just transfer a small text file, and see if it's transferring right.
    Essentially, use a 3rd party tool that works to verify that the server is sending data as it should. Once you know the server to work, then tackle the issues in the client.
    long time; /* know C? */
    Unprecedented performance: Nothing ever ran this slow before.
    Any sufficiently advanced bug is indistinguishable from a feature.
    Real Programmers confuse Halloween and Christmas, because dec 25 == oct 31.
    The best way to accelerate an IBM is at 9.8 m/s/s.
    recursion (re - cur' - zhun) n. 1. (see recursion)

  8. #8
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    Hey I tried Putty out, works great and I'm sure it will help me out w/ this project. When i transfer the file Putty does display all the contents of my small text file, so it looks like the server is sending it ok, but the client is not receiving it. Here is my updated code... I have not yet gotten away from TCHAR, ill get to that, but that shouldnt have any effect on anything.

    And again, keep in mind it sends larger files > 1MB fine, its only having a problem sending smaller files.

    Server
    Code:
    int ServerFileTransfer::sendFile( std::string const& strFileName )
    {
    	ifstream transferFile(strFileName.c_str(), ios::binary);
    
    	if ( transferFile.is_open() )
    	{
    		ULONG filelength;
    		transferFile.seekg(0, ios::end);
    		filelength = transferFile.tellg();
    
    		// put back to start of file for reading
    		transferFile.seekg(0, ios::beg);
    
    		TCHAR sendBuff[ _FILEBUFFSIZE ] = {0};
    
    		do
    		{
    			transferFile.read(sendBuff, _FILEBUFFSIZE);
    			if( transferFile.eof() && transferFile.gcount() == 0 ) break;
    			if ( !isSocketOK( send( _cliSocketData, sendBuff, transferFile.gcount(), 0 ) ) )
    				return false;
    		} while( !transferFile.eof() );
    		transferFile.close();
    	}
    
    	return true;
    }

    Client
    Code:
    int ClientFileTransfer::receiveFile( std::string& strFileName  )
    {
    	TCHAR recvBuff[ _FILEBUFFSIZE ];
    	int buffLen = _FILEBUFFSIZE * sizeof( TCHAR );
    	int bytes = 0;
    
    	bool isOK = true;
    
    	ofstream outFile(strFileName.c_str(), ios::binary);
    	static ULONG bytesRecv = 0;
    	while ((bytes = recv( _dataSock, recvBuff, buffLen, 0 )) > 0)
    	{
    		bytesRecv += bytes;
    		outFile.write(recvBuff, bytes);
    		if( !( isOK = isSocketOK( bytes )) || bytes != _FILEBUFFSIZE )
    			break;
    	}
    	outFile.close();
    
    	return isOK;
    }
    Thanks,

  9. #9
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,453
    All this use of eof is bothersome - have you read the FAQ ?

    > transferFile.read(sendBuff, _FILEBUFFSIZE);
    Doesn't this return the number of characters actually read?

    Code:
    while ( (n=transferFile.read(sendBuff, _FILEBUFFSIZE)) != 0 ) {
      // now do something with 'n' bytes (NOT _FILEBUFFSIZE bytes or any other count)
      // Eg.
      totalBytes += n;
      // for example, try all your loops with just counting how many bytes there are in the file
      // Until you get consistent counts with respect to the file sizes, nothing else matters - GIGO
    }
    if ( !transferFile.eof() ) {
      // oops - reading didn't get to the end of the file
    }
    transferFile.close();
    > send( _cliSocketData, sendBuff, transferFile.gcount(), 0 )
    Just because you asked to send 'n' bytes doesn't mean it actually did it.
    send() returns with the number of bytes actually sent - if this is less than you asked, you need to call send again with the bytes which didn't get sent.

    Ditto on the recv(), it tells you how many were received, which for a TCP connection could be any number, not necessarily related to the sizes used in send() at the other end.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  10. #10
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    If I would have listened to the 1st reply I prob wouldnt have had the problem of small files not sending. The issue was that I was using TCHAR. After changing file transfer to use plain char it started working.

    Thanks again for all who helped.

  11. #11
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    All this use of eof is bothersome - have you read the FAQ ?

    > transferFile.read(sendBuff, _FILEBUFFSIZE);
    Doesn't this return the number of characters actually read?
    That is not true. read actually returns *this.


    If I would have listened to the 1st reply I prob wouldnt have had the problem of small files not sending. The issue was that I was using TCHAR. After changing file transfer to use plain char it started working.

    Now I have disovered a new problem and it looks like its the way i receive.

    Where I say (Client)

    Code:
    if( !( isOK = isSocketOK( bytes )) || bytes != _FILEBUFFSIZE )
            break;
    This worked for smaller files so I thought all was good, but when I tried to send a file > 20MB There are times that my server does not always send _FILEBUFFSIZE bytes causing my client to break out of the loop when it should not. The reason i put bytes != _FILEBUFFSIZE into my if is because if that was not there the recv would hang after the last set of data was sent from the server.

    Is there anouther way around this to I can ensure all of my data is succesfully sent? Possibly modify the code on server side to ensure _FILEBUFFSIZE bytes are always sent unless it is infact the last round of bytes that should be sent. Or is there anouther way I can modify the client so it accepts any number of bytes up to _FILEBUFFSIZE but still exits once the recv is complete?

    I have an idea of what to do in the server and keep you posted if anythinng works, but any suggestions would be apriciated.

  12. #12
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,453
    So how do you find out the number of bytes read then?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  13. #13
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    transferFile.gcount() tells you the number of bytes read

    What I am thinking of doing is something like this:

    Code:
    		UINT bytesToSend = _FILEBUFFSIZE;
    		do
    		{
    			bytesToSend = _FILEBUFFSIZE;
    			do 
    			{
    				transferFile.read(sendBuff, bytesToSend);
    				bytesToSend -= transferFile.gcount();
    				if( transferFile.eof() && transferFile.gcount() == 0) break;
    			} while( bytesToSend != 0 );
    
    			if ( !isSocketOK( send( _cliSocketData, sendBuff, transferFile.gcount(), 0 ) ) )
    				return false;			
    		} while( !transferFile.eof() && !transferFile.gcount() == 0 );
    		transferFile.close();
    So i ensure that _BUFFSIZE is always sent unless its the last set of data. On client side i can keep the recv loop going unless _BUFFSIZE is not revc indicating that its the end of file. The sending part of this seems to be working but the revc is hanging and not receiving anything for some reason.

  14. #14
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    Dang I just realized a stupid mistake in my solution. The way i made it, it will overwrite what was previously in the sendBuff when it reads.... this will not work.

  15. #15
    Registered User
    Join Date
    Jan 2007
    Posts
    38
    Finally I think everything is resolved.

    What I did is I went back to old way as suggested to read data from the buffer until 0 bytes are read and its the end of file. I send that data (is not always _BUFFSIZE) and I receieve. In my recevece I check to make sure there will be more data that is being sent using select(...) fuction. I specify a timeout time of 1 second... if it hits that, then i break out of the receieve loop. This seems to be working on all scenario's

    Thanks again for everyone's help.

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Sending data - line by line?
    By tuckker in forum C Programming
    Replies: 0
    Last Post: 02-21-2009, 08:31 PM
  2. Changing header data in binary file
    By neruocomp in forum C Programming
    Replies: 8
    Last Post: 11-14-2008, 06:30 PM
  3. Replies: 48
    Last Post: 09-26-2008, 03:45 AM
  4. Sockets: send/receive continuous stream of data?
    By diddy02 in forum Networking/Device Communication
    Replies: 1
    Last Post: 08-09-2008, 12:52 AM
  5. Where's the EPIPE signal?
    By marc.andrysco in forum Networking/Device Communication
    Replies: 0
    Last Post: 12-23-2006, 07:04 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21