Thread: Length not quite what it should be

  1. #1
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839

    Length not quite what it should be

    I managed to clean up a large part of the code I'm planning to share via github but encountered a weird issue with length, my test string is "Testing, testing, 123" which gives me a length of 21 for the src parameter, the problem is that somehow the calculation from bytes converted results in 128, here's the problem area:
    Code:
    tok.src = mcc_iconv_tok_mem( tok.src, srcm->addr, srcm->size );
    	tok.dst = mcc_iconv_tok_mem( tok.dst, dstm->addr, dstm->size );
    	while ( (ret = mcc_iconv( &tok )) == ENOMEM ) {
    		ret = mcc_vecsize(dstv,dstm->size + MCC_BUFSIZ,sizeof(mcc_ch8_t));
    		if ( ret != EXIT_SUCCESS ) break;
    		tok.dst = mcc_iconv_tok_mem( tok.dst, dstm->addr, dstm->size );
    	}
    	mcc_getall_done:
    #if 0
    	/* Least ideal, assumes total success */
    	dstv->use += srcv->use;
    #elif 1
    	/* Slow (uses a loop with subcall) so not ideal, highest accuracy though, currently in use to verify everything else works correctly */
    	dstv->use = mcc_ch8len((mcc_ch8_t*)(dstm->addr));
    #else
    	/* Should work yet somehow reults in bigger than expected length */
    	dstv->use += (tok.dst.done / sizeof(mcc_ch8_t));
    #endif
    And just in case, the mcc_iconv* functions:
    Code:
    MCC_ICONV_TOK mcc_iconv_new_tok( iconv_t cd ) {
    	MCC_ICONV_TOK tok = {0};
    	tok.cd = cd;
    	return tok;
    }
    MCC_ICONV_MEM mcc_iconv_tok_mem(
    	MCC_ICONV_MEM prv, void *src, size_t size )
    {
    	MCC_ICONV_MEM mem = {0};
    	char *ptr = (char*)src;
    	mem.addr = &(ptr[prv.done]);
    	mem.left = size - prv.done;
    	mem.done = prv.done;
    	mem.size = size;
    	return mem;
    }
    
    int mcc_iconv( MCC_ICONV_TOK *tok ) {
    	int ret = EXIT_SUCCESS;
    	errno = ret;
    	if ( !tok->dst.addr || !tok->dst.left )
    		return ENOMEM;
    	if ( !tok->src.addr || !tok->src.left ) {
    		(void)memset( tok->dst.addr, 0, tok->dst.left );
    		tok->dst.left = 0;
    		tok->dst.done = tok->dst.size - tok->dst.left;
    		return EXIT_SUCCESS;
    	}
    	while ( iconv( tok->cd,
    		&(tok->src.addr), &(tok->src.left),
    		&(tok->dst.addr), &(tok->dst.left)) ) {
    		/* Must allow room for 21bit UTF character (4 characters)*/
    		if ( tok->dst.left < sizeof(mcc_utf_t) ) {
    			ret = ENOMEM;
    			break;
    		}
    		if ( errno != ret ) {
    			ret = errno;
    			errno = EXIT_SUCCESS;
    			break;
    		}
    	}
    	tok->src.done = tok->src.size - tok->src.left;
    	tok->dst.done = tok->dst.size - tok->dst.left;
    	return ret;
    }

  2. #2
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    I think I managed to resolve the problem, a different is still showing but that may be unrelated (although it doesn't show when I use mcc_ch8len).
    I changed the declaration of the source size to this:
    Code:
    tok.src = mcc_iconv_tok_mem( tok.src, srcm->addr, srcv->use * mcc_char_info.size );
    And added a check for 0 remaining bytes of source in mcc_iconv() since I forgot to do that, for those interested the other problem was this:
    Code:
    make word.run (in directory: /run/media/zxuiji/ZXUIJI_1TB/github/mc)
    cc -fPIC -Wall -Wno-multichar  -shared -o ./libmcc_mem.so -c mcc_mem.c
    cc -fPIC -Wall -Wno-multichar  -shared -o ./libmcc_get.so -c mcc_get.c
    cc -fPIC -Wall -Wno-multichar  -shared -o ./libmcc_tsc.so -c mcc_tsc.c
    cc -fPIC -Wall -Wno-multichar  -shared -o ./libmcc_base62.so -c mcc_base62.c
    cc -fPIC -Wall -Wno-multichar  -D OUT=word.elf -o ./word.elf word.c ./libmcc_mem.so ./libmcc_get.so ./libmcc_tsc.so ./libmcc_base62.so
    ./word.elf
    Text: '' Length: 0
    Text: 'Testing, testing, 123' Length: 0
    Line/s: 'Testing, testing, 123' Length: 0
    Line/s: 'Testing, testing, 123' Length: 0
    Word/s: 'Testing, ' 'testing, ' '123' Length: 0
    Word/s: 'Testing, ' 'testing, ' '123' Length: 0
    Char/s: Length: 0
    Char/s: 'T' 'e' 's' 't' 'i' 'n' 'g' ',' ' ' 't' 'e' 's' 't' 'i' 'n' 'g' ',' ' ' '1' '2' '3' Length: 0
    rm libmcc_get.so word.elf libmcc_tsc.so libmcc_mem.so libmcc_base62.so
    Compilation finished successfully.
    The tests where run on a UTF-8 string and a wide character string each using the same text so the results should've been the same (which it is when I use mcc_ch8len)

  3. #3
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    I finally found the cause, as I think it was Hordor mentioned in a different thread of mine it wasn't enough of the picture above to spot the cause, the cause layed in an if statement right before which skips that loop altogether, I had forgotten to actually set tok.dst.done after using memcpy()

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. get length of a stream
    By np2k in forum Linux Programming
    Replies: 4
    Last Post: 07-01-2010, 03:44 AM
  2. c++ array length?
    By corbinc in forum C++ Programming
    Replies: 2
    Last Post: 11-07-2009, 02:11 PM
  3. Measuring the length
    By kiros88 in forum C Programming
    Replies: 7
    Last Post: 09-11-2009, 04:32 PM
  4. Help with length of buffer
    By Ducky in forum C++ Programming
    Replies: 3
    Last Post: 07-01-2009, 05:39 AM
  5. wd[x].length returning 1, but it is 6!
    By Mythic Fr0st in forum C++ Programming
    Replies: 2
    Last Post: 01-17-2007, 08:06 PM

Tags for this Thread