NAQ: Everything you never wanted to know about CPP

Also known as: Making gibberish do your work for you.

Since it's snowing out and I have nothing better to do, I will share some horrible-looking 'best practices' for the C preprocessor here.

First off, for the newbies, we need to clear up a few things about what you can do with CPP.

Strings and stringizing
The C compiler takes any two strings next to each other, and makes one string of them.

The preprocessor puts quotes around macro arguments with the stringizing operator.

Code:

---

const char* sz = "This is "    "a string.";
(sz is "This is a string.")

#define MakeString( a )  # a
const char* sz2 = MakeString( Bob ) " Spork";
(sz2 is "Bob Spork")

/**
 * \brief Interpret, then convert a preprocessor directive (d) into a string 
 * \param d Token to expand and make a string from
**/
#define ppTokStr(d)                        _ppTokStr(d)
#define _ppTokStr(d)                                _ppTokStra(d)
#define _ppTokStra(d)                                        #d


#define bob chuck
const char* sz3 = "Up" MakeString( Bob ) ;
(sz2 is "Upchuck")

/**
 * It's helpful to have preprocessor tokens that do nothing.
**/
#define ppNULL

#ifdef _MSC_VER
/**
 * \brief Output a string with file and line compatible with the local IDE - naiive
 * \param txt Text to append after (or ppNULL )
 *
 * Note for MSVC you MUST turn off 'edit and continue' and 'incremental build', or __LINE__ doesn't expand right
 * You should do so anyway because MSVC will choke on complex macro expansion, which is what this whole template library is.
**/
#define ppFileLine(txt)                __FILE__ "(" ppTokStr(__LINE__) ") : " txt
#else
/**
 * \brief Output a string with file and line compatible with the local IDE - naiive
 * \param txt Text to append after (or ppNULL )
**/
#define ppFileLine(txt)                __FILE__ ":" ppTokStr(__LINE__) ": " txt
#endif

#define ppFileLine(txt)                __FILE__ "(" ppTokStr(__LINE__) ") : " txt

---

Token pasting

Token pasting allows you to take two macro arguments and glue them together. In this example, the concatentation is put off through extra levels of call to do the concatenation. This is so the higher level call can expand the arguments before token-pasting them. This lets you define and expand things before they're pasted together.

Code:

---

#define ppConcat(a,b)                _ppConcat(a,b)
#define _ppConcat(a,b)                        _ppConcata(a,b)
#define _ppConcata(a,b)                        a ## b

ppConcat(bob,cat) makes: bobcat

#define bob blah
ppConcat(bob,cat) makes: blahcat

---

Doing things several times

Code:

---

#define ppOps( op, count ) ppConcat( _ppOP, ppToken(count) )(op)
        #define _ppOP0(op)       
        #define _ppOP1(op)        op
        #define _ppOP2(op)        op op
        #define _ppOP3(op)        op op op
        #define _ppOP4(op)        op op op op
        #define _ppOP5(op)        op op op op op
        #define _ppOP6(op)        op op op op op op
        #define _ppOP7(op)        op op op op op op op
        #define _ppOP8(op)        op op op op op op op op
        #define _ppOP9(op)        op op op op op op op op op
        #define _ppOP10(op)        op op op op op op op op op op
                #define _ppOP11(op) _ppOP10(op) _ppOP1(op)
                #define _ppOP12(op) _ppOP10(op) _ppOP2(op)
                #define _ppOP13(op) _ppOP10(op) _ppOP3(op)
                #define _ppOP14(op) _ppOP10(op) _ppOP4(op)
                #define _ppOP15(op) _ppOP10(op) _ppOP5(op)
                #define _ppOP16(op) _ppOP10(op) _ppOP6(op)
                #define _ppOP17(op) _ppOP10(op) _ppOP7(op)
                #define _ppOP18(op) _ppOP10(op) _ppOP8(op)
                #define _ppOP19(op) _ppOP10(op) _ppOP9(op)
        #define _ppOP20(op) _ppOP10(op) _ppOP10(op)
            /* And on and on, however long you want to go */

/*
 * \brief Constant: Raise some constant base value, 'b' to the xth power (ppExp(10,3) = 1000)
 * \param b Base value
 * \param x Power; must be an integer constant (i.e. 3, not 'x')
 */
#define ppExponent( b, x )        ( 1 ppOps( * (b), x ) )

---

So, (1 ppOps( * 10, 3 )) is made into (1 * 10 * 10 * 10) by the preprocessor, and the C compiler makes a constant value of 1000 from that when it evaluates the constants.

Macros to macros (to macros...)

You can use the preprocessor like a primitive database, and query rows and columns based on the content of a macro that you either painstakingly typed, or had a CASE tool spew forth for you (for instance that 'ppOps' mess is made by a 50 line tool that just spews that mess out for me, so I don't have to type it).

By making lists of things like this, you can keep track of a lot of things you need to make many seperate tables out of, all in one place, and you can extract the content where you need it.

Code:

---

#define table_o_crap( def ) \
  def( Bob, "123-4567", 48,  12 ) \
  def( Ann, "456-7890", 56,  11 ) \
  def( Cal,  "867-5309", 35,  9 ) \
  def( Bob, "333-EATS", 72,  214 ) \

#define GetNameEnum( name, phone, age, etc ) name
#define GetNameCase( name, phone, age, etc ) case name:
#define GetNameString( name, phone, age, etc ) #name
#define GetPhone( name, phone, age, etc ) phone
#define GetAge( name, phone, age, etc ) age
#define GetEtc( name, phone, age, etc ) etc
#define GetString( name, phone, age, etc ) #name " Phone:" phone, " Age:" #age " Etc:" #etc "\n"
#define GetStruct( name, phone, age, etc ) { #name, phone, age, etc },

---

Getting Obscene
You can use the preprocessor to do a lot of interesting things. There are two ways to do the following, but I'm going to show you the 'hard' (to use) way first.

Code:

---


/*
 * Establish template patterns
 */

#define DEF_DECL( item )                decl_##item
#define decl_BEGIN( classname )                                typedef struct classname {
#define decl_SCALAR( type, label, value )                type label;
#define decl_COMPOUND( type, label )                        type label;
#define decl_END( classname )                                } classname;\
                                                                                        extern void classname##_init( classname* self );\
                                                                                        extern void classname##_destroy( classname* self );\


#define DEF_INIT( item )                init_##item
#define init_BEGIN( classname )                                void classname##_init( classname* self ) {
#define init_SCALAR( type, label, value )                self->label = (value);
#define init_COMPOUND( type, label )                        type##_init( &self->label );
#define init_END( classname )                                }
 
#define DEF_DESTROY( item )                destroy_##item
#define destroy_BEGIN( classname )                        void classname##_destroy( classname* self ) {
#define destroy_SCALAR( type, label, value ) 
#define destroy_COMPOUND( type, label )                        type##_destroy( &self->label );
#define destroy_END( classname )            }


/* And serialization, XML I/O, database, dispatch, encoding, etc functions based on this pattern! */

#define CLASS_DECLARE( class_decl )                class_decl( DEF_DECL )
#define CLASS_IMPLEMENT( class_decl )        \
class_decl( DEF_INIT ) \
class_decl( DEF_DESTROY ) \

#define XYZ_DECL( def ) \
def( BEGIN( XYZ ) )\
  def( SCALAR( float, X, 0.0f ) )\
  def( SCALAR( float, Y, 0.0f ) )\
  def( SCALAR( float, Z, 0.0f ) )\
def( END( XYZ ) )

#define MYSTRUCT_DECL( def ) \
def( BEGIN( MyStruct ) )\
  def( SCALAR( int, id, ++globalID ) )\
  def( COMPOUND( XYZ, location ) )\
def( END( MyStruct ) )

CLASS_DECLARE( XYZ_DECL );
CLASS_DECLARE( MYSTRUCT_DECL );


int globalID = 0;

CLASS_IMPLEMENT( XYZ_DECL );
CLASS_IMPLEMENT( MYSTRUCT_DECL );


int main( void )
{
        MyStruct mys;
        MyStruct_init( &mys );
        return 0;
}

---

Here's the preprocessor output... (piped through 'indent', else a LOT of these would be one-liner spew).

Code:

---

cl /E ppSample.cpp|indent

typedef struct XYZ
{
  float X;
  float Y;
  float Z;
} XYZ;
extern void XYZ_init (XYZ * self);
extern void XYZ_destroy (XYZ * self);;
typedef struct MyStruct
{
  int id;
  XYZ location;
} MyStruct;
extern void MyStruct_init (MyStruct * self);
extern void MyStruct_destroy (MyStruct * self);;


int globalID = 0;

void
XYZ_init (XYZ * self)
{
  self->X = (0.0f);
  self->Y = (0.0f);
  self->Z = (0.0f);
} void

XYZ_destroy (XYZ * self)
{
};
void
MyStruct_init (MyStruct * self)
{
  self->id = (++globalID);
  XYZ_init (&self->location);
} void

MyStruct_destroy (MyStruct * self)
{
  XYZ_destroy (&self->location);
};


int
main (void)
{
  MyStruct mys;
  MyStruct_init (&mys);
  return 0;
}

---

Anyway, in a large project with a lot of this auto-generated code, a whole lot of grunty work can be done by the preprocessor, so when you add a new member to a 'class', you don't need to remember to add/change pieces of code scattered all over the project. This technique works just as well for C++.

Generally, you would want more kinds of scalars (as well as thoroughly defined scalar types), such as ones that are supposed to be serialized or stored in a database, versus ones that are not. You may also want to declare various functions through the class factory, so that 'finding' them (or virtualizing them) becomes trivial.

Next post: The 'easy' way to do what I just showed you (though it takes a lot more code to implement). It makes it possible to step through the code, sort of like a C++ template, without piping it through the preprocessor and compiling the output.

As promised (though nobody in fact cares), here is the 'easy' and robust way to define records that will be constructed, destructed, serialized, compared, copied, made from/into XML or database records, etc. with the preprocessor, so you don't have to maintain all of that code.



dg_patterns.h

Code:

---

#define record_begin(label)                                typedef struct label {
#define record_scalar(label,type,val)                type label;
#define record_compound(label,type)                        type label;
#define record_end(label)                                } label;\
                                                                                /* These functions are bundled up so we can semi-trivially manufacture record declarations for extensions */\
                                                                                ctl_datagen_declare_functions(label);\

---

dg_patterns.temp.h

Code:

---

#undef record_begin
#undef record_scalar
#undef record_compound
#undef record_end

/* Make initializer */
#define record_begin(label)                                                        void ppConcat(label,_init)( label* self ) \
                                                                                                        {\
                                                                                                                assertptr( self ); /* Assert: valid inputs */
#define record_scalar(label,type,val)                                        self->label = (type)(val);
#define record_compound(label,type)                                                ppC(ppConcat(type,_init)(&self->label);) /* C++ will invoke its default constructor */
#define record_end(label)                                                        }

#include MYDATA_H

#undef record_begin
#undef record_scalar
#undef record_compound
#undef record_end

/* Make destructor */
#define record_begin(label)                                                        void ppConcat(label,_denit)( label* self ) \
                                                                                                        {\
                                                                                                                assertptr( self ); /* Assert: valid inputs */
#define record_scalar(label,type,val)
#define record_compound(label,type)                                                ppC(ppConcat(type,_denit)(&self->label);) /* C++ will invoke its default constructor */
#define record_end(label)                                                        }

#include MYDATA_H

#undef record_begin
#undef record_scalar
#undef record_compound
#undef record_end


/* Copy operator */
#define record_begin(label)                                                        void ppConcat(label,_copy)( label* dst, const label* src ) \
                                                                                                        {\
                                                                                                                assertptr(dst);\
                                                                                                                assertptr(src);
#define record_scalar(label,type,val)                                        dst->label = src->label;
#define record_compound(label,type)                                                ppConcat(type,_copy)(&dst->label,&src->label);
#define record_end(label)                                                        }

#include MYDATA_H

#undef record_begin
#undef record_scalar
#undef record_compound
#undef record_end

/* Serializers, comparisons, metrics, etc... */

#undef MYDATA_H

---

mydata.h

Code:

---

/* A structure */
record_begin(XY)
        record_scalar(        y,        float32,        0.0f )
        record_scalar(        x,        float32,        0.0f )
record_end(XY)

/* Another structure */
record_begin(XYZ)
        record_scalar(        z,        float32,        0.0f )
        record_scalar(        y,        float32,        0.0f )
        record_scalar(        x,        float32,        0.0f )
record_end(XYZ)

/* A big compound to test lots of things */
record_begin(MyClass)
        record_scalar(        mint16,                int16,                2 )
        record_scalar(        muint16,        uint16,                3 )
        record_scalar(        mint32,                int32,                4 )
        record_scalar(        muint32,        uint32,                5 )
        record_scalar(        mint64,                int64,                6 )
        record_scalar(        muint64,        uint64,                7 )
        record_scalar(        mfloat32,        float32,        8.9f )
        record_scalar(        mfloat64,        float64,        9.1 )
        record_compound(xy,                        XY        )                /* Compound types */
        record_compound(xyz,                XYZ        )                /* Compound types */
record_end(MyClass)

---

mydata.c

Code:

---


/* Import definitions */
#include "dg_patterns.h"
#include "mydata.h"

...

/* Generate code with class factory templates */
#define MYDATA_H "mydata.h"
#include "dg_patterns.temp.h"

---

Note:
When the code is generated, most debuggers step through the lines in 'mydata.h'. This makes it easy to tell it was in, say, a structure copy function of a member of a class of data. Your class declaration is not composed of a multi-line macro (though many parts of the factory are).

Why the heck would someone want to do this???

Well, consider a very large project with a hundred kinds of records to both store and send over a network. You add an 'x' to one record. If you had programmed it the conventional way, you would have to track down every kind of initialization and serialization of that record and add 'x' to that code as well. If you miss one instance of that code, you make a nasty new bug that could corrupt data belonging to paying users, and the source of that bug might be difficult to track down, being buried in long lists of very similar and repetitive serialization code.

With the preprocessor making the code for you, if the macros work (and they should be thoroughly tested ahead of time and have a LOT of unit tests composed for them), the code will work, and you can add properties on a whim, and the entire suite of initialization, copy, protocol, serialization, etc. for this objcet is updated at once, so you can concentrate on the feature you added 'x' for, instead of chasing down all the various implementations of that stuff that perhaps other team members are also trying to work on.

Even more usefully, the name of every member of your data can be derived and stuck in a table or wrapped in code, such that data (or an interpreter) that references "MyClass.xyz.x" could get/set the value of it. This means that integrating your native data into an interpreter, or writing an interpreter based on your native data becomes a trivial low-maintenance task, instead of an infuriating, high maintenance drudgery. Also, reading/writing/logging/etc. data as XML or CSV or a tagged binary format becomes trivial. You can just write a set of macros matching the particular operations you want, and the code is implemented for ALL data using this technique FOR YOU.

Of course, lots of people (merely ignorant of this technique, or just stupid) will insist that all of the code for a project be hand written and hand maintained, and make convulsive shuddering motions when you mention using a 'preprocessor'. They can either be enlightened, or not. If you have ANYTHING much more complex than a school "Hello World" project, this technique will yield you simpler, much smaller and more robust code than you could ever make by hand, and it can be maintained across platforms without external 'helper' tools, because the C preprocessor really can handle it.

Quote:

---

Of course, lots of people (merely ignorant of this technique, or just stupid) will insist that all of the code for a project be hand written and hand maintained, and make convulsive shuddering motions when you mention using a 'preprocessor'.

---

The main reason us "ignorant or stupid" coders shudder when someone mentions using a preprocessor like this, is due to the fact that at one time or another we had to track down one or more runtime bugs in heavily macro'd code (which someone else wrote).

This sort of technique can be handy in certain (rare) conditions, but the fact is that far too many people abuse the preprocesser, and end up introducing bugs which become hard to track down.

In your instance, you are arguing for the fact that this technique can help avoid introducing bugs when the layout of your data changes, and this is a valid point. The fact is that in this case most of us opt for a database solution thus making this argument moot :) I'm not knocking what you've done as a pointless excersise, it can be fun and educational to expand the preconceived uses of the preprocesser. But for production code, I would say this is a big no-no.

Wow! There is a lot of interesting stuff there, although I'll have to read it a couple more times to comprehend most of it. Thanks for the write-up.

Share and Enjoy.

As for 'no-no', actually all preprocessor code is easier to debug than C++ template code for one simple reason. It can be piped through the preprocessor and then through 'indent', and the intermediate output is perfectly readable source code, that CAN be built instead of the original and debugged. I would maintain that if you shied away from preprocessor helpers, you simply gave up too easy. Some preprocessors even have additional flags to make doing this even easier (i.e. don't expand include files, or eat comments).

The second preprocessor construct post has a couple of nice things going for it:
1. It will call out the correct source line if you have a typo.
2. If it builds it will almost certainly work right the first time.

When you talk about a library like this, arguing about being able to debug it is like complaining that you can't source level debug into OS calls or third party libraries. Once the library is tight, there is never any excuse to pry into its inner workings besides idle curiosity. When you make a blanket statement about a new paradigm like this (not really so new - it's been around for a long time), it sounds a lot like someone complaining about how their first little python program doesn't run, and they can't fix it because they can't step directly into the C source code that python was written in from a python debugger.

MyFile.c

Code:

---

#ifdef PP_OUTPUT
#include "MyFile.ppc"
#else
/* Preprocessor intensive code */
#endif

---

With this simple construct, you could make a debug-enabled build target that with -DPP_OUTPUT on the command line, and that took each C/C++ target and piped it through the preprocessor and gnu indent (or your favorite pretty-print) before compiling the preprocessed code. When stepping thtough the code, the "ppMyFile.c" is what you'll see.

Something on the order of...
cpp Myfile.c|indent>MyFile.ppc

I tend to automate a LOT of things, including writing cheesy programs to generate C code, and in some cases, adding the source and build to these cheesy programs to my build as a dependency before building more of the project. I like to come up with ways for the computer to do my work for me, especially when it's repetitious drudgery. Especially when it's very easily machine-generated code that could be very error prone to type by hand.

In any case, IF the preprocessor output is thoroughly verified before the balance of the project depends on it, there NEVER is any trouble-shooting to do of the preprocessor output. Even when there is, the definitions still make writing universal checks and double-checks for debug passes trivial (i.e. all structures/classes can have truly correct equivalency tests, so you can verify ALL kinds of serialization). The fact that you can automatically have XML I/O (or other text I/O formats) built in means that you can record/review/export/import all kinds of debug scenarios, including cooking them up by hand. You won't even try to do any of that if you have to code each scenario for each of hundreds of records in a typical medium to large scale client/server project manually.

For instance, I have arrays and containers integrated into the "real thing", and what that gives me is the capability to send/receive/write/read/compare/etc. compounds of compounds containing containers of scalars/compounds, yet I don't have to write any 'special' code to use it, or to make it more compact, or to make it more portable or proof against future versions, yet this is all extremely lightweight (even the XML parser doesn't allocate or copy any memory unless it's returning a decoded string to a variable length container - and that container allocates the memory) and takes only one pass at runtime. A serial packets' length is known before I have to serialize it, so I can allocate exactly the right amount (using an integer constant, if there's no variable length content) before sticking it onto an output queue. Save memory, save passes through realloc, save cache misses and the whole lot of issues, and since the packet code becomes so atomic, save a LOT of time tinkering with places where you already know for sure there isn't any problem.

The real question is why is this on the C board and not the C++ board?


Quzah.

Or why we really need yet another attempt to rewrite C++ using the C preprocessor :rolleyes:

I tuned out when I saw the first \ in a macro

Do you work for Microsoft? Because your code abuses the pre-processor about as much as theirs does. That is hideous.

Imagine trying to track down the bugs. The compiler throws out some weird stuff and highlights the wrong line when the error comes from a line in a macro.

Too many macros and the pre-processor is NOT for re-writing the language.

Abuse? Imagine? I wrote the code, and did 'track down the bugs'. I've also 'tracked down the bugs' in past projects that did not use similar techniques to mine: the bugs were far more numerous and time consuming. The only implementation I know of that seriously blows chunks on preprocessor macros is Microsoft's (all versions of their compiler, at least since Visual C++ 5), when you leave their 'incremental' compiler options on.

Rewrite C++? You may as well accuse me of putting babies in a microwave. I'd sooner eat the contents of a cat litter box than have anything to do with C++ again.

Why on the C board instead of C++? This is C. Apparently a lot of people seem to believe you need C++ to make object oriented code. You need C++ for OOP like you need a Cadillac to save gas.

> than have anything to do with C++ again.
So what's all this then - some kneejerk reaction to some imagined "inefficiency" of the C++ language? Sure C++ can be inefficient, just like C, but that's down entirely to programmer ineptitude rather than the language.

Or an attempt to level the playing field with your peers by introducing something which you find easier to use because they find it harder to use?

Quote:

---

Originally Posted by evildave

Why on the C board instead of C++? This is C. Apparently a lot of people seem to believe you need C++ to make object oriented code. You need C++ for OOP like you need a Cadillac to save gas.

---

If I wanted to learn "all there is to know about CPP", I'd go to a C++ forum (actually, I wouldn't, but that's a different matter). You don't learn "all there is to know about CPP" on a C forum. I hate having to explain the obvious to people all the time.


Quzah.

Quote:

---

Originally Posted by quzah

If I wanted to learn "all there is to know about CPP", I'd go to a C++ forum (actually, I wouldn't, but that's a different matter). You don't learn "all there is to know about CPP" on a C forum. I hate having to explain the obvious to people all the time.


Quzah.

---

CPP = C Pre-Processor.

Ah. Well he kept talking about C++ also in his post. :P

Here we'll play sentence structure police again. "All you ever wanted to know about the CPP." versus "All you ever wanted to know about C++." See, you can know about C++, but you have to know about the CPP.

If people could actually form complete sentences around here things would turn out much better. But at least I finally know why it's posted here instead of the C++ forum. ;)

Quzah.

I will not pass judgement on evildave's preprocessor demonstration. I wouldn't use it in those ways without very good reasons, and do not encourage others to do so, but if the code does not need to be maintained by anyone except the author, it's a matter of taste. In a project shared with other programmers, especially one where the code may be shared with a wide audience, as not everyone is as well versed in the wonders of C pre-processor macros as I am (and evildave is).

Back between 1988 and 1990, I posted a program called yacpp (for "Yet Another C PreProcessor) to alt.sources. It was intended as a pre-preprocessor, and used "@" in place of "#". (I neither have a copy of the code, nor can I find my postings of or about it in the various Usenet archives.)

It offered a similar set of facilities to those provided by Macro assemblers such as the Macro-11 (a PDP-11 macro assembler produced by Digital Equipment Corp.), and included the following features: (this is an incomplete list)

• Multi-line macros
• Macro variables with expressions
• Macro conditional and looping structures
• Run-time prompting for values
• Argument decomposition and string operations

My program was widely condemned by many people through both e-mails and Usenet News postings as being a lame attempt at redefining the C language. It was asserted by its detractors that programs written to depend on this tool would be non-portable and impossible to debug. I claimed that it made my life as a programmer easier, and that used as intended, made code maintenance easier, but that claim fell on mostly deaf ears. A few years later, I was still getting the occasional flame for the program, but added to the criticisims was that it was written in C, and that all such tools ought to be written in perl (camp 1) or GNU-emacs macros (camp 2), so my guess is that someone was using it.

My point in bringing up this little bit of ancient history is that I now see that the initial detractors were right in many, but not all, of their objections. For a person who knows C, but has never read the manpage for yacpp, encountering the following would be more than a little mystifying:

Code:

---

@def_int        EntryNum            1
@def_macro  NextEntry
(EntryNum)
@set              EntryNum            (EntryNum + 1)
@end_macro

#define  PRJ_VOL              NextEntry  /* 1 */
#define  PRJ_BRT              NextEntry  /* 2 */

int mapping[EntryNum] = {PRJ_VOL, PRJ_BRT};

---

It was, in effect, an unintentional attempt to create a new programming language on the back of C.

These days, I use C macros in very transparent ways. I use them to make code easer to read and maintain, but not to redefine language structures. I even use the inline-function like macros sometimes for portability, where on one system, a value is a simple variable, but on others it's a call through a pointer to a function where the return value isn't quite what the rest of the code expects. I only use it where I need to be sure of the scope in which the code is running, and I comment both the definition and the use (usually in a block comment at the beginning of the function or block) so that other people reading the code understand what's going on without having to remember the definition of the macro.

Quote:

---

especially one where the code may be shared with a wide audience, as not everyone is as well versed in the wonders of C pre-processor macros as I am (and evildave is).

---

OMG. We are like so your students. Please teach us while we soak up your vast knowledge. I'm trying to be as awesome as you at the pre-processor code but I know I'll never come close to your level. Your knowledge and breadth of information just amazes me. We bow before your greatness.


:o :rolleyes: