hello, sorry to bother you again, ive heard sth about Unicode standard thats been being implemented in C++ standard !
so ive seen console applications that uses other languages other than English for representing data !
how is it possible ? is there any kind of class or header file that eases the use of such works in console projects ? or i should myself do sth about it !
if i am the one who has to do sth , what are those thing i should do to be able to use custom language in an app !
tanx
You just need to use unicode, wchar_t instead of char.
Also note that that every unicode string literals has a capital L in front of it. Example L"Hello World".
Unicode also has their own C library equivalent functions. For example, wcscpy instead of strcpy.
Visual Studio also provides its own "TCHAR" routines. TCHAR is a typedef for either char or wchar_t depending on settings.
Again, TCHAR has their own functions, such as tcscpy (I believe) instead of strcpy or wcscpy.
All TCHAR strings are enclosed in _T(). Example: _T("Hello World").
tanx Elysia ,Quote:
Originally Posted by Elysia
so by simply replacing type char with wchar_t its done ? !
is it implemented in Iostream header ?
wchar_t is a built-in type, so you don't need to include anything.
TCHAR typically resides in tchar.h.
If you're looking at doing real-time multilingualism on the command line, I'd look into a package called gettext. it is originally a unix program, but has been ported to windows, and while I'm not familiar with its use, I have seen it in action, and it appears to work very well.
Yes, but if you're using wchar_t strings instead of char strings, you need to use the 'w' version of everything.Quote:
Originally Posted by Masterx
Ex. std::wstring, std::wcout, std::wcin, std::wfstream...
The current C++ '03 standard library does not support Unicode output. wcout and wfstream with both perform a wchar_t to char conversion before output. This behavior can be changed using a custom codecvt facet. However, Unicode output still doesn't work on Windows with the MS CRT. For Windows, the only way to get UTF16 console Unicode output is to use WriteConsoleW(). So you could create a custom wstreambuf which uses WriteConsoleW() as well.
gg
In addition, wchar_t is a horribly muddled type that only brings portability problems. At the same time, char strings are a mess of encodings. Writing international software in C++ invariably involves stuff beyond the standard library.
C++0x will bring a minor improvement in this regard.
tanx all i was about to ask why this piece of code i provided does not work!
so im a little bit confused you know!Code:#include <iostream>
#include <tchar.h>
#include <windows.h>
//#include <string>
using namespace std;
int main()
{
int i=0;
char buf[200];
wchar_t s[] = L"چشم";//a string consisting of 3 non-english characters meaning "Eye"!
char c[] = "چشم";//equivalant to wchar_t s[] = L"eye"; and char c[] = "eye";
for (int i=0;c[i]!='\0';i++)
{
cout<<c[i];
}
CharToOemW(s, buf); cout<<endl<<buf<<endl;
return 0;
}
first of let me tell you , im planing to code for diffrent platforms ! i mean planing to code Cross-platform programs , so thats why i just need to stick to the standard C\C++ . i would be very thankful if you Dears contribute solutions in a way that serve the purpose specified . and the solutions are not limited to the specific Platforms . Thanks in advance. (by the way ive already used windows specific solution " CharToOemW()" just to see how it works! but unfortunately this didnt even work! )
and now second , wellif i am going to implement such a thing in Windows, how am i supposed to do that ? i mean how should i use this WriteConsoleW() stuff !!? (any sample code to show how it is implemented ?)Quote:
the only way to get UTF16 console Unicode output is to use WriteConsoleW(). So you could create a custom wstreambuf which uses WriteConsoleW() as well.
andand this doesnt need to add anything to the program ? such as some kinda headers or i dont now , any kind of define or typedef stuff?Quote:
Yes, but if you're using wchar_t strings instead of char strings, you need to use the 'w' version of everything.
Ex. std::wstring, std::wcout, std::wcin, std::wfstream...
(im asking this question beacuse meanwhile i can not test the code to see whether it works or not , thats why im asking you such a thing! sorry for that !)
isnt it supposed to have real time multilingualism when going through such stages ? so what kind of multilingual program my code would be after all?( after using such instructions in the program? !)Quote:
If you're looking at doing real-time multilingualism on the command line, I'd look into a package called gettext. it is originally a unix program, but has been ported to windows, and while I'm not familiar with its use, I have seen it in action, and it appears to work very well.
and whats that !? a thirdparty app i should use to gain such a feature under windows ? ! its the only way ? if so , i wonder what are those unicode stuff being implemented in C\C++ standard so far!?
and to finish this post , ive heard that if i can change the default font of console i may be able to gain multilingualism! ist it true ? and if so how to do that !
Many thanx to you all , i really appreciate your valuable comments and solutions tanx
edited !
didnt work!? did i miss any thing ? !!Quote:
Yes, but if you're using wchar_t strings instead of char strings, you need to use the 'w' version of everything.
Ex. std::wstring, std::wcout, std::wcin, std::wfstream...
Code:#include <iostream>
#include <tchar.h>
#include <windows.h>
//#include <string>
using namespace std;
using std::wcout;
int main()
{
int i=0;
char buf[200];
wchar_t s[] = L"چشم";//a string consisting of 3 non-english characters meaning "Eye"!
char c[] = "چشم";//equivalant to wchar_t s[] = L"eye"; and char c[] = "eye";
for (int i=0;c[i]!='\0';i++)
{
wcout<<c[i];
}
CharToOemW(s, buf); cout<<endl<<buf<<endl;
return 0;
}
>> how am i supposed to do that ? i mean how should i use this WriteConsoleW()
The online reference for Windows API's/SDK's is MSDN: http://msdn.microsoft.com/en-us/library/ms687401.aspx You use GetStdHandle() to get the first parameter for WriteConsoleW().
>> ive heard that if i can change the default font of console i may be able to gain multilingualism! ist it true ? and if so how to do that !
The font used by the console (cmd.exe) needs to support Unicode. "Raster Fonts" does not support Unicode, "Lucida Console" does. http://commandwindows.com/configure.htm
>> didnt work!? did i miss any thing ? !!
As I mentioned in my previous post, wcout and wfstream both perform a wchar_t to char conversion before output.
Here are some things you need to be aware of if you want to write Unicode-aware code and make cross-platform.
Unicode
Part of the Unicode standard is to provide a mapping of character glyphs to a unique integer value. At present, you need at lease a 32bit integer to contain the largest Unicode character value.
When writing source code to deal with Unicode, you need to be aware of how the Unicode characters are encoded. There are several encoding schemes including: UTF8, UTF16LE, UTF16BE, UTF32LE, UTF32BE. The LE and BE versions stand for little-endian/big-endian.
wchar_t
This is the character type used by the standard for doing wide stuff. The standard doesn't require that all implementations make wchar_t a particular size. Which also means that wide string litterals (wich map to wchar_t's) are also implementation defined.
Windows compilers use 16bit wchar_t's and encode wide string literals using UTF16LE.
GCC on *nix uses 32bit wchar_t's and encodes wide string literals using UTF32. The architecture determines if it's LE or BE.
>> wchar_t s[] = L"چشم";
There are a few problems with doing something like this. First, you need to make sure your compiler even supports the compilation of source files saved as Unicode. Most modern ones do. VC 6.0 does not. Next you have to make sure your editor saves the file in some Unicode format (that the compiler supports).
>> char c[] = "چشم";
This shouldn't even compile. If it does, then the editor saved the file as ASCII and the character values within the c array are meaningless.
So as you can see, there are many things to consider if you truly hope to write cross-platform code that handles Unicode encodings.
gg
thanks a million CodePlug:) i really dont know how to thank you. Great Stuff, i really appreciate it ,Quote:
Originally Posted by Codeplug
and about compiler. i use Codeblocks IDE and MingW Gnu gcc compiler.
well dont know if it supports the feature or not! gotta ask .
so at last ,i would be able to run an app when i get my compiler fixed! ok
ill be informing you if i face any problem ,
many many tanx again
hello , im back again .
i gave it a try and on gcc i kept getting an error stating " wcout is not a member of std!" for a simple wchar-t use~! so first of all ,is it the gcc problem that does not support such a thing!?
second , as it seems ive gotta do sth like this , implement a function in my source code , that enables me to switch between unix and windows Operating systems!
this function in complition time should find out if it is a windows platform or not! and if it finds it windows ,voila .! and now i can stick to the WriteConsoleW() , and get teh job done under windows ~ but still i have no other solution for linux (because gcc seems to have a problem !there!)
how to do this ? !
This is a library issue, not a gcc issue. wcout doesn't exist on my MinGW port, but does on my Mac.
never had a problem in producing Dutch console output, wide characters required.
Same is true for Italian, French, Spanish, German, etc. etc. if you're willing to put up with loosing a few special characters (which has long been accepted way back in the days of typewriters).
so what should i do? !!Quote:
Originally Posted by tabstop
----
and by the way , i appreciate your help on finding a way in determining the Platform in compilation time! any comments ?
sorry i dind get you !would you explain it abit more again !Quote:
Originally Posted by jwenting
tanx
I think jwenting's coment refers to the fact that most european languages can be used with A-Za-z and a few others that are available in the ANSI character set (same as ISO-8859-1, I think?). However, I doubt that Arabic, Hebrew, Hindu, Urdu, Thai, Chinese, Japanese, Greek, and several other languages do not even use A-Z as representations of their native language, and in this case you NEED a wide character set. As to what you do to achieve that, my guess would be that if you want to make your life easy, you skip console applications, as support for multilingual console apps is very limited.
--
Mats
tanx a milion dear Mats, as you ve just mentioned ,the asian langs are different and they dont use A,z etc stuff,i go along with it specialy on console application ,Quote:
Originally Posted by matsp
but the catch here is , i chose C++ because its chalenging! , and i chose console since it is again more chalenging!and it wants you to understand and make what you want in a hard wayconsidered to other programmming langs(full of experience and excitment for newbies like me).
i want the job done! and im not gonna leave it alone before i test the current sulotions!and see the outcome.
to tell you the truth, im working on the GUI version too, using wxwidgets, but first ive got to accomplish this task and then go for the other one.
again many tanx .
It seems like the MS libraries do have a wcout, however, gcc-mingw uses a fairly old version of the MS C runtime, so it doesn't have that.
Not sure what options you have here - using MS Visual Studio 2008 may be one choice.
--
Mats
and thus , only one sulotion remains(to me!)!and that is doing it in a windows way! using a windows Api!Quote:
Originally Posted by matsp
there is an issue here! so to achive this, using a windows API in a expected cross platform application; i need a function which makes it clear that the code is being compiled under windows and thus would run in windows Os , then after that ,it will execute bunch of codes that does the job here! (manipulating unicode strings )and
if not the function will skip that part and accroding to the platform( here usullay *nix platforms)in its compilition time,it would normaly act. and compiles!
as far as i know ,unicode is farily implemented such platforms (rather than windows).and i will not face any problems in linux or Mac Os .
how to write that function to determing whether is windows or not! in complition time!
using size of() will do the trick?(ive heard wchar_t is 2 byte in windows and 4 bytes on *nix platforms! is it true ?)
tanx
wcout is not a windows API. My point about using the Visual studio is that it HAS wcout, which is a standard function.
As to OTHER poratibility problems, yes, they may well exist now and in the future.
--
Mats
The new C++ standard is supposed to fix all those multi-language problems... but it is still far away.
so first ive gotta give it a try with VS 2008 compiler ( i think!?) and then continue on! solving the *nix specific problems !
tanx .
I use STLport with MinGW - which does have wide iostreams.
http://www.stlport.org/
It's been explained already that wide streams do not support Unicode. Trying to do so in a cross-platform manner is even further complicated in that wchar_t and literal wide-string encodings are implementation defined.
Having said all that - you can prevent the implicit wchar_t to char conversion in wide streams by using your own codecvt facet. This works ok for wide fstreams under most implementations, but still doesn't handle screen output for any windows compiler I have access to. However, starting with the VC++ 2008 CRT, you can use a new MS extension to enable Unicode (UTF16LE) output with wcout. Here's some code to play with:
I would interested in knowing if this works for anyone running *nix and a Unicode-enabled terminal.Code:#include <iostream>
#include <iomanip>
#include <fstream>
#include <locale>
#include <string>
//------------------------------------------------------------------------------
// null_wcodecvt is a codecvt facet that prevents the implicit conversion of
// whcar_t to char in wide streams
typedef std::codecvt<wchar_t , char , mbstate_t> null_wcodecvt_base;
class null_wcodecvt : public null_wcodecvt_base
{
public:
explicit null_wcodecvt(size_t refs = 0) : null_wcodecvt_base(refs) {}
protected:
virtual result do_out(mbstate_t&,
const wchar_t* from,
const wchar_t* from_end,
const wchar_t*& from_next,
char* to,
char* to_end,
char*& to_next) const
{
size_t len = (from_end - from) * sizeof(wchar_t);
memcpy(to, from, len);
from_next = from_end;
to_next = to + len;
return ok;
}//do_out
virtual result do_in(mbstate_t&,
const char* from,
const char* from_end,
const char*& from_next,
wchar_t* to,
wchar_t* to_end,
wchar_t*& to_next) const
{
size_t len = (from_end - from);
memcpy(to, from, len);
from_next = from_end;
to_next = to + (len / sizeof(wchar_t));
return ok;
}//do_in
virtual result do_unshift(mbstate_t&, char* to, char*,
char*& to_next) const
{
to_next = to;
return noconv;
}//do_unshift
virtual int do_length(mbstate_t&, const char* from,
const char* end, size_t max) const
{
return (int)((max < (size_t)(end - from)) ? max : (end - from));
}//do_length
virtual bool do_always_noconv() const throw()
{
return true;
}//do_always_noconv
virtual int do_encoding() const throw()
{
return sizeof(wchar_t);
}//do_encoding
virtual int do_max_length() const throw()
{
return sizeof(wchar_t);
}//do_max_length
};//null_wcodecvt
//------------------------------------------------------------------------------
std::wostream& wendl(std::wostream& out)
{
// this is needed for files opened in binary mode under Windows in order
// to retain Windows-style newline
#if defined(_WIN32)
out.put(L'\r');
#endif
out.put(L'\n');
out.flush();
return out;
}//wendl
//------------------------------------------------------------------------------
const wchar_t UTF_BOM = 0xfeff;
const wchar_t CHECK_SYM = L'\u221a';
//------------------------------------------------------------------------------
void file_test()
{
// create UTF16LE text file for windows, UTF32[BE|LE] for *nix
std::wfstream file;
null_wcodecvt wcodec(1);
std::locale wloc(std::locale::classic(), &wcodec);
file.imbue(wloc);
file.open("data.txt", std::ios::out | std::ios::binary);
if (!file)
{
std::cerr << "Failed to open data.txt for writting" << std::endl;
return;
}//if
file << UTF_BOM << L"data = " << 42 << CHECK_SYM << wendl;
file.close();
}//file_test
//------------------------------------------------------------------------------
#if defined(_MSC_VER) && (_MSC_VER >= 1500)
# define HAVE_O_U16TEXT
# include <fcntl.h>
# include <io.h>
#endif
void wcout_test()
{
#if defined(HAVE_O_U16TEXT)
// newest MS CRT supports UTF16 output using special mode
int mode = _setmode(_fileno(stdout), _O_U16TEXT);
#else
// for everything else, use our codecvt facet - I haven't seen this work
// for any windows compiler...may work for *nix but untested
null_wcodecvt wcodec(1);
std::locale wloc(std::locale::classic(), &wcodec);
std::wcout.imbue(wloc);
#endif
std::wcout << L"U+221a - [" << wchar_t(0x221a) << L"]" << std::endl;
#if defined(HAVE_O_U16TEXT)
// revert to original mode
_setmode(_fileno(stdout), mode);
#endif
std::cout << "Testing 1, 2, 3" << std::endl;
}//wcout_test
//------------------------------------------------------------------------------
int main()
{
file_test();
wcout_test();
return 0;
}//main
There's still the underlying problem that the wchar_t encoding is implementation defined - which means that using wchar_t as a cross-platform Unicode character is useless.
"Forcing" wchar_t to do something useful with Unicode is really a platform and implementation dependent exercise.
gg
many many tanx dear Codeplug, i will test it and i will tell what happens
again tanx alot .
i really appreciate your helps , all of you .
tanx
At the risk of being assaulted by some of the others here, why not just use C standard functions instead of iostreams?
Because they're really no better in this department.
I have some insite that is only going to apply to OS specific situations. I googled and found a few libraries floating around that are more apt for doing localization.