VC++ not Unicode friendly? C2001: newline in constant

PaulBlay · 06-16-2009, 02:32 AM

I've got this little bit of simple code, and it produces lots of "C2001: newline in constant" errors.

The odd thing is that the source code file is in Unicode (UTF-8) and the program is being compiled in Unicode mode (Character Set: Use Unicode Character Set). So why is it complaining?

Code:

#define ARROW_DOWN	138 /* '\x8A' */
#define ARROW_LEFT	139 /* '\x8B' */
#define ARROW_RIGHT	140 /* '\x8C' */
#define ARROW_UP	141 /* '\x8D' */

#ifdef WINDOWS
#  define ARROW_N __T('↑')
#  define ARROW_E __T('→')
#  define ARROW_S __T('↓')
#  define ARROW_W __T('←')
#  define ARROW_NE __T('↗')
#  define ARROW_SE __T('↘')
#  define ARROW_SW __T('↙')
#  define ARROW_NW __T('↖')
#  define ARROW_WAIT __T('↻')

/* Analogous to isdigit() etc in ctypes */
#  define isarrow(c)	( \
	(((int)c >= ARROW_DOWN) && ((int)c <= ARROW_UP)) \
	|| (((int)c >= ARROW_W) && ((int)c <= ARROW_S)) \
	|| (((int)c >= ARROW_NW) && ((int)c <= ARROW_SW)) \
	|| ((int)c >= ARROW_WAIT) \
	)

#else

#  define isarrow(c)	( \
	(((int)c >= ARROW_DOWN) && ((int)c <= ARROW_UP)) \
	)

#endif /* WINDOWS */

I need to have lots of Unicode text in my project or I will be put to a great deal of additional effort. If it's going to do this all the time I'll be really annoyed.

PaulBlay · 06-16-2009, 02:39 AM

Nevermind folks - I've found the problem.

For future reference these errors occurred because I was saving with

Unicode (UTF-8 without signature) - Codepage 65001

as soon as I switched to

Unicode (UTF-8 with signature) - Codepage 65001

they disappeared.

This is not nice behaviour on behalf of VC++, but at least it's liveable with (until I want to support Linux ;-).

EVOEx · 06-16-2009, 02:42 AM

To be honest, I found unicode support is lacking in general, in all of the C++ world (though C++0x is going to fix that, at least till some extends). If you only need it to run under VC++, there may be some easy method that works. However, that would completely kill all portability.

If you need it portable, I'd advise you to do this: Pick either UTF8, UTF16 or UTF32 to store all data in. UTF32 is a lot easier, while UTF8 is generally a lot smaller, and UTF16 is in the middle (and commonly the worst of both worlds, in stead of the best).
Now, either get or make your own string class that supports this. In case of UTF32 you can just use std::string<uint32_t> or something similar. You'll probably also have to write some conversion routines between UTF's (at least from/to UTF8), but if you go to the wikipedia pages the implementations of these algorithms are trivial.
Then write all the text to some format in a text file and store it using a UTF you picked (might be another as well). Read it appropriately, convert it if required, and read it into the string class.
It's even more fun if you actually want to use it, because it is completely OS dependant. Want to show it in a Windows messagebox? That's UTF16. Want to show it in a linux console? UTF8 (well, in my case it was). Some other linux window manager? Probably depends on which one.

All in all, a mess to use if you need it to be portable.

Codeplug · 06-16-2009, 08:47 AM

>> # define ARROW_N __T('↑')
Two issues:
1) There's no point is using __T(). TCHAR's are for code that needs to support SBCS, MBCS, and Unicode at the same (depending on the build target). What you want to use is the Unicode character U+2191. So you could do something like this:

Code:

const wchar_t ARROW_N = L'\u2191';

2) Don't use extended characters within your source code. There's no telling what you'll get after the compiler has translated from the "source character set" to the "execution character set" - since it's implementation defined.

gg

PaulBlay · 06-16-2009, 08:55 AM

Quote:

Originally Posted by Codeplug

>> # define ARROW_N __T('↑')
Two issues:
1) There's no point is using __T(). TCHAR's are for code that needs to support SBCS, MBCS, and Unicode at the same (depending on the build target). What you want to use is the Unicode character U+2191. So you could do something like this:

Code:

const wchar_t ARROW_N = L'\u2191';

I'm not using __T( for it's MBCS / SBCS functionality, I'm using it as a handy-to-remember macro name that I can redefine or search/replace at a later point.

Codeplug · 06-16-2009, 09:34 AM

Then I would suggest making and using your own macro. __T() has an implied purpose when used (at least when others are reading your code).

gg

Elysia · 06-16-2009, 12:35 PM

Quote:

Originally Posted by PaulBlay

Nevermind folks - I've found the problem.

For future reference these errors occurred because I was saving with

Unicode (UTF-8 without signature) - Codepage 65001

as soon as I switched to

Unicode (UTF-8 with signature) - Codepage 65001

they disappeared.

This is not nice behaviour on behalf of VC++, but at least it's liveable with (until I want to support Linux ;-).

This is not nice of you.
How is VC++ supposed to know what format the file is saved in without a signature?

PaulBlay · 06-17-2009, 02:43 AM

Quote:

Originally Posted by Elysia

This is not nice of you.
How is VC++ supposed to know what format the file is saved in without a signature?

By looking at it. Visual Studio does, after all, display the file correctly in the IDE. (As set with the "Auto-detect UTF-8 encoding without signature." option) So the left hand knows what it's doing - it didn't occur to me that the right hand hadn't been informed.

06-16-2009, 02:39 AM	#2
PaulBlay Registered User Join Date: Mar 2009 Posts: 46	Nevermind folks - I've found the problem. For future reference these errors occurred because I was saving with Unicode (UTF-8 without signature) - Codepage 65001 as soon as I switched to Unicode (UTF-8 with signature) - Codepage 65001 they disappeared. This is not nice behaviour on behalf of VC++, but at least it's liveable with (until I want to support Linux ;-).
PaulBlay is offline

06-16-2009, 02:42 AM	#3
EVOEx Registered User Join Date: Oct 2008 Posts: 452	To be honest, I found unicode support is lacking in general, in all of the C++ world (though C++0x is going to fix that, at least till some extends). If you only need it to run under VC++, there may be some easy method that works. However, that would completely kill all portability. If you need it portable, I'd advise you to do this: Pick either UTF8, UTF16 or UTF32 to store all data in. UTF32 is a lot easier, while UTF8 is generally a lot smaller, and UTF16 is in the middle (and commonly the worst of both worlds, in stead of the best). Now, either get or make your own string class that supports this. In case of UTF32 you can just use std::string<uint32_t> or something similar. You'll probably also have to write some conversion routines between UTF's (at least from/to UTF8), but if you go to the wikipedia pages the implementations of these algorithms are trivial. Then write all the text to some format in a text file and store it using a UTF you picked (might be another as well). Read it appropriately, convert it if required, and read it into the string class. It's even more fun if you actually want to use it, because it is completely OS dependant. Want to show it in a Windows messagebox? That's UTF16. Want to show it in a linux console? UTF8 (well, in my case it was). Some other linux window manager? Probably depends on which one. All in all, a mess to use if you need it to be portable.
EVOEx is offline

06-16-2009, 08:47 AM	#4
Codeplug Registered User Join Date: Mar 2003 Posts: 3,844	>> # define ARROW_N __T('↑') Two issues: 1) There's no point is using __T(). TCHAR's are for code that needs to support SBCS, MBCS, and Unicode at the same (depending on the build target). What you want to use is the Unicode character U+2191. So you could do something like this: Code: const wchar_t ARROW_N = L'\u2191'; 2) Don't use extended characters within your source code. There's no telling what you'll get after the compiler has translated from the "source character set" to the "execution character set" - since it's implementation defined. gg __________________ How to Ask Questions The Smart Way How to Report Bugs Effectively
Codeplug is offline

06-16-2009, 09:34 AM	#6
Codeplug Registered User Join Date: Mar 2003 Posts: 3,844	Then I would suggest making and using your own macro. __T() has an implied purpose when used (at least when others are reading your code). gg __________________ How to Ask Questions The Smart Way How to Report Bugs Effectively
Codeplug is offline

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Screwy Linker Error - VC2005	Tonto	C++ Programming	5	06-19-2007 02:39 PM
We Got _DEBUG Errors	Tonto	Windows Programming	5	12-22-2006 05:45 PM
load gif into program	willc0de4food	Windows Programming	14	01-11-2006 10:43 AM
Possible circular definition with singleton objects	techrolla	C++ Programming	3	12-26-2004 10:46 AM
UNICODE and GET_STATE	Registered	C++ Programming	1	07-15-2002 03:23 PM