understanding c preprocessor directives...

hello everybody!
I am trying to understand some code of linux but i am not 100% able to understand preprocessor's directives and programming conventions...

here is the code i want to understand:

Code:

---

#define __define_initcall(level,fn,id) \
        static initcall_t __initcall_##fn##id __attribute_used__ \
        __attribute__((__section__(".initcall" level ".init"))) = fn

#define device_initcall(fn)                __define_initcall("6",fn,6)

#define __initcall(fn) device_initcall(fn)

#define module_init(x)        __initcall(x);

---

I can understand almost everything besides the 1st #define (all the rest are substitutions of symbols, isn't it?):
the symbol

Code:

---

__define_initcall(level,fn,id)

---

will be replaced with

Code:

---

static initcall_t __initcall_##fn##id __attribute_used__  __attribute__((__section__(".initcall" level ".init"))) = fn

---

the ##fn## has to be replaced with fn and the symbols at the left and right have to be concatenated, eg:

Code:

---

__define_initcall("6",fn,6)

---

will become

Code:

---

static initcall_t __initcall_fnid __attribute_used__  __attribute__((__section__(".initcall" level ".init"))) = fn

---

is this a correct interpretation? what is the convention behing the double underscore (__) before the name of the macro?

thanks for your help! Bye

Well, but level will get replaced with "6" and id with 6 as well.

Quote:

---

Originally Posted by tabstop

Well, but level will get replaced with "6" and id with 6 as well.

---

so we can say the answer is:

Code:

---

static initcall_t __initcall_fn6 __attribute_used__  __attribute__((__section__(".initcall6.init"))) = fn

---

?
in general, when is the use of "__" advised?
thanks again! bye

Actually, double underscore symbols are reserved for the compiler implementation, so techncially, that is "undefined behaviour" and can cause breakages - but I doubt that gcc will ever use those symbol names, and and since gcc is the only compiler used to compile the Linux kernel (which my guess is that these macros come from, more specifically the kernel loadable module section). Of course, the likelyhood of that actually happening is probably low.

And to explain a bit further:
__atrribute_used__ tells the compiler that it should not warn about unused variables or functions, even if the compiler doesn't think it is use.

The __attribute__((__section__(".initcall6.init"))) part tells the compiler to put this symbol in a special section. In particular ".initcallX.init" is a special set of sections that are processed by the system loader. The entire content of initcallX.init is assumed to be function pointers, and the system loader will call all those functions in the order they appear, starting with the lowest X.

--
Mats

Quote:

---

Originally Posted by matsp

Actually, double underscore symbols are reserved for the compiler implementation, so techncially, that is "undefined behaviour" and can cause breakages - but I doubt that gcc will ever use those symbol names, and and since gcc is the only compiler used to compile the Linux kernel (which my guess is that these macros come from, more specifically the kernel loadable module section). Of course, the likelyhood of that actually happening is probably low.

And to explain a bit further:
__atrribute_used__ tells the compiler that it should not warn about unused variables or functions, even if the compiler doesn't think it is use.

The __attribute__((__section__(".initcall6.init"))) part tells the compiler to put this symbol in a special section. In particular ".initcallX.init" is a special set of sections that are processed by the system loader. The entire content of initcallX.init is assumed to be function pointers, and the system loader will call all those functions in the order they appear, starting with the lowest X.

--
Mats

---

thanks a lot Mats!
of course your guess is correct: I am studying modules right now! :)

Quote:

---

Originally Posted by smoking81

thanks a lot Mats!
of course your guess is correct: I am studying modules right now! :)

---

I did think so. Now you can impress your teacher by explaining exactly how a module is initialized ;).

--
Mats

We should add that these are gcc extensions. Avoid like the plague if possible.
The entire linux kernel is unportable and can't even be compiled with other compilers, plus it's full of macro hacks, from what I can tell.

Quote:

---

Originally Posted by Elysia

We should add that these are gcc extensions. Avoid like the plague if possible.
The entire linux kernel is unportable and can't even be compiled with other compilers, plus it's full of macro hacks, from what I can tell.

---

And the Windows kernel, of course, is compilable by a handful of compilers, right? No, not so.

Unless the OS is SPECIFICALLY written to support many compilers, it usually doesn't (which is HARD - the OS I work with supports about 4 different compilers, and quite frequently, something goes wrong with one of the less commonly used compilers, because some developer added/changed some code in a way that causes the build on one of the compilers that developer DIDN'T test to fail).

And there are plenty of macro "hacks" in our OS too, since we have to do things like:

Code:

---

#if COMPILER1
#define A abc
#elsif COMPILER2
#define A bca
#else
#define A foo
#endif

---

It is not easy to write an OS. It is a lot harder to write an OS that works on multiple compilers.

Those macros are perfectly valid for the purpose they have: To support automatic calling of initialization functions for kernel modules. That is not something you would want to compiler with any other compiler anyways, because it integrates directly to the kernel.

There is NO standard way to define what the section name a particular variable should go in, so how can we do this in a portable way? You'd have to use #if/#elsif... coding to sort that out - and often the syntax is such that you can't just replace one thing with another, because one compiler uses a #pragma, the other an attribute or other compiler extension.

--
Mats

Quote:

---

Originally Posted by matsp

And the Windows kernel, of course, is compilable by a handful of compilers, right? No, not so.

---

I thought it wouldn't.
However, that doesn't mean they should use extensions, especially not without specific #ifdef/#ifndef for specific compilers.
What if they need to switch compiler at some point?

Quote:

---

And there are plenty of macro "hacks" in our OS too, since we have to do things like:

Code:

---

#if COMPILER1
#define A abc
#elsif COMPILER2
#define A bca
#else
#define A foo
#endif

---

---

I'd much rather want to do away with hacks. Of course, if I can't see them, I really can't say if there's a solution to those hacks using another language...

Quote:

---

It is not easy to write an OS. It is a lot harder to write an OS that works on multiple compilers.

---

I just want to add that I'm not defending any OS, or that they should make it work with many compilers.
But I don't like hacks or extensions.

Quote:

---

There is NO standard way to define what the section name a particular variable should go in, so how can we do this in a portable way? You'd have to use #if/#elsif... coding to sort that out - and often the syntax is such that you can't just replace one thing with another, because one compiler uses a #pragma, the other an attribute or other compiler extension.

---

I do understand that there is need for unportable code, but I'm just adding that it is unportable and that you should avoid them like the plague. I'm not implying you should always stay clear - clearly if the need is greater than the added cost of not doing so, it should/can be done.

What I'd like to see is a kernel / OS that's written or works in C++ and not C.
I don't think it's impossible, but Windows kernel and Linux kernel are just too old (too much written in and for C in mind) for that to work reliably...

Quote:

---

Originally Posted by Elysia

However, that doesn't mean they should use extensions, especially not without specific #ifdef/#ifndef for specific compilers.
What if they need to switch compiler at some point?

---

Why would they need to switch compiler? They can always fork GCC if necessary, after all.

Quote:

---

Originally Posted by Elysia

What I'd like to see is a kernel / OS that's written or works in C++ and not C.

---

I recall matsp saying that the OS he works on is written in C++. matsp, is this true?

The OS I work on is almost entirely written in C++, with the exception of:
1. A few bits of assembler, because it's not possible / suitable to write those funcitons in C++.
2. Some libraries that come from third parties, that are written in C rather than C++ - such as OpenGL libraries that by defintion is a C library, not a C++ library [although one could of course implement it as C++ - but that severely limits the number of suppliers of OpenGL that one could license the code from]. I'm pretty sure there are some other libraries that come from third parties that aren't available as C++ code, and we do not want to or have the resources to rewrite those as C++.

I'm not going to discuss in public what company I work for.

--
Mats

Quote:

---

Originally Posted by laserlight

Why would they need to switch compiler? They can always fork GCC if necessary, after all.

---

Well, I don't know, I'm just listing the worst case if scenario.
The kernel is slow and they realize there's a much faster compiler out there that has recently been released?

Quote:

---

Originally Posted by matsp

The OS I work on is almost entirely written in C++, with the exception of:

---

That is awesome, but I do wonder how advanced functionality, such as templates, it uses, and if it's heavily OOP or uses typical C++ paradigms and such or if it's just pretty much C written as C++?

Quote:

---

Originally Posted by Elysia

I do understand that there is need for unportable code, but I'm just adding that it is unportable and that you should avoid them like the plague. I'm not implying you should always stay clear - clearly if the need is greater than the added cost of not doing so, it should/can be done.

---

I do agree that you should not frivolously (sp?) use these constructs. And they aren't used frequently in kernel modules in Linux - kernel modules or kernel drivers are unportable to a large extent in and of themselves, so I don't really see the problem with that.

I'm pretty sure no-one has suggested that we should add __attribute__(__segment("...")) type things to code arbitrarily without further understanding of the consequences.

I also fail to see what C++ has to do with this - we want to stuff the address of a function into a specific segment [like text/code, data, bss, etc] so that the function can be executed as part of the loading of a kernel module. How would you do that in standard C++? You wouldn't because there is no standard solution for it. And without changing how the kernel works [and how else would you suggest that we store the initialization functions in for a kernel module?], we can not do anything else.

Linux in itself, despite relying on gcc, is very portable, shown by the number of processor architectures that it works on - far more than Windows or any other OS that I know of aside from maybe one or two embedded OS's or such.

--
Mats

Quote:

---

Originally Posted by matsp

I do agree that you should not frivolously (sp?) use these constructs. And they aren't used frequently in kernel modules in Linux - kernel modules or kernel drivers are unportable to a large extent in and of themselves, so I don't really see the problem with that.

---

Well, at least it's good to hear, but when I see a snippet from a kernel with worst case scenario code and haven't actually looked at the entire source, I draw conclusions that it may be littered with bad practices kind of code.

Quote:

---

I'm pretty sure no-one has suggested that we should add __attribute__(__segment("...")) type things to code arbitrarily without further understanding of the consequences.

---

You never know with some people, especially when they read kernel code ;)

Quote:

---

I also fail to see what C++ has to do with this - we want to stuff the address of a function into a specific segment [like text/code, data, bss, etc] so that the function can be executed as part of the loading of a kernel module. How would you do that in standard C++? You wouldn't because there is no standard solution for it. And without changing how the kernel works [and how else would you suggest that we store the initialization functions in for a kernel module?], we can not do anything else.

---

Oh, I was just wishing to see a kernel written in C++ instead of that horrible, gibberish C code that makes me tick faster (and that's bad, of course).

Quote:

---

Linux in itself, despite relying on gcc, is very portable, shown by the number of processor architectures that it works on - far more than Windows or any other OS that I know of aside from maybe one or two embedded OS's or such.

---

Windows is bad™ ;)

Quote:

---

Originally Posted by Elysia

Well, I don't know, I'm just listing the worst case if scenario.
The kernel is slow and they realize there's a much faster compiler out there that has recently been released?

---

Usually, only a small portion of the kernel is actually being exercised frequently, so it is probably easier to rewrite those particular functions in assembler (inline or out-of-line). Yes, that's even less portable, but it's still easier than replacing the compiler, because that affects more of the code. We have recently had one of our software engineers spend about 5 weeks trying to compile one component with a new compiler. That is not the whole OS, but ONE graphics component [admittedly a fairly large one].

Quote:

---

That is awesome, but I do wonder how advanced functionality, such as templates, it uses, and if it's heavily OOP or uses typical C++ paradigms and such or if it's just pretty much C written as C++?

---

Not a lot of templating, but some. It is entirely OOP, not C written as C++. For example, you create a thread object to create a thread, and then call a begin method to make the thread go active. Likewise to create a process, or a semaphore, etc, etc.

--
Mats

Quote:

---

Originally Posted by Elysia

The kernel is slow and they realize there's a much faster compiler out there that has recently been released?

---

Yeah, I know that you're just giving a worst case scenario... but I think that it really is unlikely. If there was a compiler that was much better at optimisation, they would probably help GCC implement the ground breaking optimisation techniques involved, perhaps even by merging code from that compiler, if possible. If that compiler was not Free/Open Source Software, it would not even be considered anyway, no matter how good it was (especially after the affair with Bitkeeper, a proprietary version control system).

Of course, GCC is prohibited from introducing particular forms of register allocation, due to commercial companies holding patents in that area. I'm sure some other register allocation can be used instead, and with identical effect.

There is also a dramatical performance difference between gcc in x86 and some other gcc implementations (when I compared gcc 2.x with a commercial alternative on AMD 29K, gcc 2.x generated roughly 40-50% more instructions - and in 29K, 40-50% more instructions mean 40-50% longer execution, since all instructions are "equal").

--
Mats

Quote:

---

Originally Posted by matsp

Actually, double underscore symbols are reserved for the compiler implementation, so techncially, that is "undefined behaviour" and can cause breakages - but I doubt that gcc will ever use those symbol names, and and since gcc is the only compiler used to compile the Linux kernel (which my guess is that these macros come from, more specifically the kernel loadable module section). Of course, the likelyhood of that actually happening is probably low.

And to explain a bit further:
__atrribute_used__ tells the compiler that it should not warn about unused variables or functions, even if the compiler doesn't think it is use.

The __attribute__((__section__(".initcall6.init"))) part tells the compiler to put this symbol in a special section. In particular ".initcallX.init" is a special set of sections that are processed by the system loader. The entire content of initcallX.init is assumed to be function pointers, and the system loader will call all those functions in the order they appear, starting with the lowest X.

--
Mats

---

Hi again!
Reading again and trying to understand better your answer I pointed my browser to http://gcc.gnu.org/onlinedocs/gcc/
I didn't find __attribute_used__ there but your explaination is clear so it doesn't matter..
About the section attribute, I just wonder if by "section" it is meant one of the sections of the executable file which is generated and which (in the case this executable is in ELF format, and this is the case in Linux) can be read with the help of readelf and objdump? In fact, the gcc doc says that the attribute "section" forces the compiler to place the corresponding code in a specific "section" which is different by the default "text"... Did I understand correctly?

Another question: what about the other names starting by "__".. (eg __initcall...)? Where could I find further documentation to understand what is the logic behind this "__"?

thanks again! :) bye

To start from the back: __ is just a couple of "letters" in an identifier. Identifiers can start with A-Za-z_ and then continue with an arbitrary number of the same or digits. One could choose XX instead of __. And as I stated earlier, __ is actually reserved to be used by the compiler implementation for implementation specific symbols. There is no "logic behind" this other than "if we start with two rather unusual 'letters', then the symbol is less likely to have the same name as something else".

Yes, sections are what you find in an ELF file, where ".text" is the common section for code, ".data" the common section for variables.

--
Mats