Unicode 2 byte wide characters

Hello,

I was wondering about Unicode, specifically 2 byte characters, if i have an array of 2 byte unicode characters (each byte is an element) how would i be able to print these characters.

for instance, the array looks mostly like this:

00 a3 00 8c 00 b9 etc

i.e. whenever i try to print it it thinks there is a null terminator and quits printing the string

is there a 16 byte unsigned char type?

i am really stuck, any light you could shed would be greatly appreciated

thanks

Use wchar_t
As in
wchar_t mybuffer[100];

Use printf("%S",mybuffer);
Note the upper case.

Though you'll need a modern compiler for this.

thanks for the reply, say if the bytes are locked up in a char array, is there a way to copy the bytes into this wchat_t array 1 byte at a time, or do you need to copy it two bytes at a time?

thanks
ps -- i wont use void main... :)

reading at unicode.org it says it is O.K. to truncate 2 byte unicode (16bit big endian) code sequences to 1 byte because it will preserve characters c <= 127, like ascii.

this wold only stuff up glyphs right?

also, i will be sending these strings to objective-c which as far as i know can only support UTF8 c strings.

i think i will go for the lossy conversion rather than mess around with wide characters...

thanks for your help

when i print byte for byte this sequence:

0x00 0x6C 0x00 0x6F 0x00

the letters 'l' and 'o' are printed but when i sprintf the 0x6c then 0x6f i get 'l' and nothing.

i then looked up 6f in ascii and its just nothing...

i am totally stuck now

but still, thankyou for your replies thus far

Can I ask why you are doing this and why you think the first element of a unicode character is 0x00 (because I'm pretty sure it isn't)?

Quote:

---

Originally Posted by davo666

thanks for the reply, say if the bytes are locked up in a char array, is there a way to copy the bytes into this wchat_t array 1 byte at a time, or do you need to copy it two bytes at a time?

---

You can drop the "locked up" from that sentence.
If they are in a char array, then simply cast it to a wchar_t array:

Code:

---

char ch[] = {0x00, 0x6C, 0x00, 0x6F, 0x00, 0x00, 0x00};
printf("%S", (wchar_t*)mybuffer);

---

ok, i will extrapolate:

i am getting an OBEX packet delivered over Bluetooth. one of the packet fields is 'file name' - in 2 byte unicode characters.

the problem is the array received is a void * (such that casting it to a uint16_t gets me the wrong first element of the array) so i have to cast it to uint8_t.

as you can see now, i have 2 byte unicode characters in a single byte array such that

00 6b 00 fc 00 73 00 00 <-- 2 byte null terminator

is in the middle of an array but represents characters... i need to get them into a 2 byte char array on their own so i can write the filename and contents to disk.

its a really hard task though...

thanks

i thought about the casting, but wouldnt that read 4 bytes at a time? also the bytes i need are 9 bytes in (i.e. the 8th element of the array)

Quote:

---

Originally Posted by davo666

(such that casting it to a uint16_t gets me the wrong first element of the array) so i have to cast it to uint8_t.

---

Dude, the first character cannot really be 0x00. Maybe you should look at this.

I don't think you should use an unsigned datatype in the conversion. It may seem that in hex it's all the same, but I would observe that normal unicode is signed and mostly negative.

Quote:

---

Originally Posted by davo666

ok, i will extrapolate:

i am getting an OBEX packet delivered over Bluetooth. one of the packet fields is 'file name' - in 2 byte unicode characters.

the problem is the array received is a void * (such that casting it to a uint16_t gets me the wrong first element of the array) so i have to cast it to uint8_t.

as you can see now, i have 2 byte unicode characters in a single byte array such that

00 6b 00 fc 00 73 00 00 <-- 2 byte null terminator

is in the middle of an array but represents characters... i need to get them into a 2 byte char array on their own so i can write the filename and contents to disk.

---

You mean elaborate, not extrapolate.
Are you working on a big-endian platform? If not, I believe you are casting an address that is off by one. There's nothing hard about this, it's easy, you just have to cast the right address to wchar_t*. You may have to cast it from void to uint8_t first, so that you can then add your 8 or 10 bytes etc, but then cast it to wchar_t.
If this wont give you the correct output, then please post the code you're trying to use.

the double casting worked, you are great!

i still couldnt apply that to swprintf though :s but this is what i have so far:

name_len - 3 is the number of bytes, including the null termintor that the string goes for.

Code:

---

here is a byte printout of the array, (starting at payload[0])

0x82 0x00 0xD6 0x01 0x00 0x13 0x00 0x6C 0x00 0x6F 0x00 0x6C 0x00 0x2E 0x00 0x74 0x00 0x78 0x00 0x74 0x00 0x00
                                                    -- start of filename field                                                                                        2 byte null(end)

wchar_t * file_name[200];

        for(i = 0 ; i < name_len-3 ; i++){
                wprintf(L"%C",(wchar_t *)((u8_t *)p->payload)[i+6]); //works

                swprintf(&file_name[i], name_len-3 ,L"%C",(wchar_t *)((u8_t *)p->payload)[i+6]); //dosent work...
        }

        wprintf(L"%S",file_name); //dosent work

---

thanks!

and why casting to pointer? isn't it wchar_t?

you havn't read this http://msdn.microsoft.com/en-us/libr...3w(VS.71).aspx

sorry, i got casted to (wchar_t) but that still didnt help... im using GCC on an ARM platform msdn is the last thing i would read :)

thanks

i did read the man page... from my re-reading i changed %C to %c but otherwise i dont know what to do -- arg1 is a pointer to a wide char array? arg2 is the maximum length of the string (?) arg 3 is the format and the rest are referred to in the format...

is this correct?

ps, dont get my wrong i love how you dont tell me, but rather let me learn on my own (i just need a small extra clue :) )

ok im going for the sprintf the bytes to a char array and going from there i found this piece of code does not compile

Code:

---

#include <stdio.h>
#include <wchar.h>

int main() {


        char ch[] = {0x00, 0x6C, 0x00, 0x6F, 0x00, 0x00};
        printf("%S", (wchar_t*)ch);

return 0;

}

---

at all i tried it on multiple architectures.. nothing. right now if i could print the array file_name then i would be a happy person. another thing, does creat() take UTF-16?

thanks

The following works for me in VS2005

Code:

---

#include <stdio.h>
#include <wchar.h>

int main()
{
    char ch[] = {0x6C, 0x00, 0x6F, 0x00, 0x00,0x00 };
    wchar_t* pCh = (wchar_t*)ch;
    wprintf(L"%s\n", pCh);
    printf("%S\n", pCh);
    return 0;
}

---

I have no gcc to check it as well

that interprets it as a 1 byte string, becuase it prints 'l' (ell) and quits, also if you add a trailing 0x00, (which is evident in my string) then it doesnt print anything...

this is becming an annoyance... is it really that hard to print a 16 bit character string?

(Im not paying out on you, im just surprised :s)

thanks