Thread: Reading a language file (UTF-8 to wstring)

Earlier, I asked a question about i18n. Thanks to those who answered (I could post a thanks there, but I don't want to bump the question unnecessarily). However, I have another question about the same subject.

For the program I'm making, I want to have a single file (let's say encoded in UTF-8, could be anything but I prefer UTF-8) for translations of certain sentences. Now, how can I best input that into a wchar_t?
As far as I know, I can't simply use a wfstream, since the format in which it reads isn't defined by the standard - could be unicode or UTF8. Also, I can't find a way to portably change it to UTF8. Besides, I tested something like this: "wstring s; while(wcin>>s);". It would run the while loop as long as I typed ASCII characters; the moment I typed, for instance a euro sign, it quit the loop. While my terminal does seem to support the euro sign.
I could read it in byte after byte and automatically decode the UTF-8 to unicode. Done that before, it's really easy, but then it's probably not portable to add characters to the wstring. The wstring, as I understood, can be stored as both UTF-16 (16-bit wchar_t in Windows) or UTF-32 (in Linux). Maybe even UTF-8 in other architectures.

So, what is a *portable* way to input a UTF-8 file into a wide string?


Thanks in advance,
EVOEx

Originally Posted by matsp

This looks like something that would correspond well to what you want. I don't know of any better source, but I'd be surprised if there isn't one out there.

http://www.codeproject.com/KB/string/UtfConverter.aspx

--
Mats

Thanks a lot for your reply. It doesn't look too portable, making a distinction for the size of wchar_t, but if this is as portable as it's going to get I can live with it.
I do get the impression nationalization is, in every aspect, really insufficient as of yet. The documentation I read about it is inconsistent and unportable. But I'll guess we'll have to wait for C++0x for it to get better...

Thanks,
EVOEx