Making this thread for others to find now that I figured out where I was going wrong, 1st some base code for such people to play with:
Now for the benefit of those who came looking for such a thread I'll state the problem I encountered when I 1st tried to use this function.Code:#include <limits.h> #include <inttypes.h> #include <uchar.h> #include <errno.h> #include <ctype.h> #include <string.h> #include <stdlib.h> #include <stdio.h> #include <iconv.h> typedef signed schar; typedef unsigned uchar; typedef long long llong; typedef unsigned long long ullong; #define bitsof(T) (sizeof(T) * CHAR_BIT) #define bitsof_by2(T) (bitsof(T) / 2) #define bitsof_by4(T) (bitsof(T) / 4) typedef struct i64 { uchar part[8]; } I64; I64 i64_parts( uint_least64_t val ) { #define by2 (bitsof(uint_least64_t) / 2) #define by4 (bitsof(uint_least64_t) / 4) #define by8 (bitsof(uint_least64_t) / 8) uint_least64_t half0 = ((val << by2) >> by2), half1 = (val >> by2); uint_least64_t max = -1, large = (max >> (by2 + by4)), small = (large >> by8); ushort high = (half0 >> by4), low = (half0 & large); I64 i64; i64.part[0] = (low & small); i64.part[1] = (low >> by8); i64.part[2] = (high & small); i64.part[3] = (high >> by8); high = (half1 >> by4); low = (half1 & large); i64.part[4] = (low & small); i64.part[5] = (low >> by8); i64.part[6] = (high & small); i64.part[7] = (high >> by8); #undef by8 #undef by4 #undef by2 return i64; } int const abcd = 'abcd'; int const kanji = 0x2ea2; char const *mizu = "\u2ea2"; int main(int argc, char *argv[]) { I64 i64; char const *LANG = getenv("LANG"); char *unic = (char*)(&kanji); char const *CSET = NULL; size_t leng = 2; char32_t _wide = 0; char *wide = (char*)(&_wide); size_t size = sizeof(char32_t); iconv_t ic; if ( !LANG ) { (void)fail( EADDRNOTAVAIL ); (void)puts("Couldn't retrieve LANG varaiable, bailing.."); return 1; } CSET = strstr( LANG, "." ); CSET++; (void)printf("Local charset detect as '%s'\n", CSET ); if ( argc == 1 ) { (void)puts("1 argument found, entering unit test mode:\n"); (void)printf( "abcd: %%c = '%c'\n", abcd ); i64 = i64_parts( abcd ); (void)printf( "abcd: %%c:%%c:%%c:%%c = '%c':'%c':'%c':'%c'\n", i64.part[3], i64.part[2], i64.part[1], i64.part[0] ); (void)printf( "mizu: %%s = '%s'\n", mizu ); (void)puts("Now attempting to convert UTF-16 to locale charset\n"); ic = iconv_open( CSET, "UTF-16" ); if ( ic < 0 ) { (void)puts("Failed to open a iconv_t instance, aborting...\n"); return 1; } (void)iconv( ic, &unic, &leng, &wide, &size ); (void)iconv_close(ic); (void)printf("wide: %%s = '%s'\n", &_wide ); (void)printf("wide: 0x%%08X = 0x%08X\n", _wide ); } return 0; }
As you'll notice when the looking at the docs for iconv_open it needs the source encoding identified in addition to the target encoding, that's fine, that makes sense, the problem was getting the local encoding in the 1st place, now I was doing this with output to my terminal to check I was getting the expected output, so the iconv_t is opened fine, the problem came after, iconv(), now having glanced at the docs expected input I naively thought I could just pass a pointer to an array, obviously the compiler complained at me for this and so I created another variable and pointed it to the array and passed that variable, so far so good, but then I ran into a new problem, I was not getting the expected output, the reason for this I found out just a few minutes before writing this post/thread, turns out that iconv() actually changes that pointer and when I used it after calling iconv() I didn't realise I needed to reset the pointer, or rather I was a dumbass and didn't look closely at the docs. So in short remember to reset this pointer if you're using it as the start of a string/character, also I forgot to squeeze this in earlier but to get the locale use getenv("LANG"), after checking you got a valid pointer (bail if you don't or look for another valid environment variable) use strstr(<VAR>,".") to filter out the langauge and get to the dot right before the encoding, then just increment the pointer, the thing to bare in mind however is that enviroment variables may not be up to date and the user could have switched encoding after booting your process, I do not know of a solution to this problem so if someone knows of one please feel free to reply to this thread with it.
