Thread: CAMP - new open-source reflection library for C++

  1. #1
    Registered User
    Join Date
    Jun 2010
    Posts
    10

    CAMP - new open-source reflection library for C++

    Hi

    I'd like to introduce a new (and very exciting) library to the C++ community: CAMP.

    CAMP is an open-source library (under the LGPL license) that allows to emulate reflection features in C++, like you can find in Java, .Net, etc. In short, reflection allows to manipulate classes, functions, properties and objects dynamically without having to know about (at compile time) their existence, their name or even their prototype.

    So what can you do with CAMP? Well, for those who know Qt and its QObjects with abstract signals, slots and properties, you can basically do the same thing with CAMP. But it's not limited to that, we can also imagine a variety of applications based on meta-information: automatic serialization in XML or other format, a generic object editor, writing bindings to script languages in a few lines of code, ...

    Another example is boost.python; in fact CAMP is highly inspired from this library (and luabind, which is also inspired from boost.python). So you can declare your classes, functions, properties, enums, etc. in a way very similar to these two libraries, except that the resulting meta-information is available directly as building blocks for something else, you're not limited to using it for the X scripting language.

    We recently published a brand new website for the community, with a forum, a wiki, a bug tracker, a file repository, etc. :
    CAMP Developer Zone

    Don't hesitate to give your feedback, or ask questions if some parts of CAMP are not clear to you

  2. #2
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    O_o

    That's some nice spam.

    *shrug*

    At least it is open source...

    Soma

  3. #3
    Registered User
    Join Date
    Jun 2010
    Posts
    10
    Sorry if it looks like spam.

    I read the forum rules before posting, and there's nothing against posting to talk about an open-source project that may be interesting for the C++ community. And I think it really is interesting. I'm really looking for relevant feedback from experienced C++ developers, I'm not just throwing a link there and never coming back. So please let's start discussing about what you think of this library

    Of course if I'm really breaking one of the the forum rules, just delete this topic and sorry for the inconvenience.

  4. #4
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by Laurent Gomila View Post
    Of course if I'm really breaking one of the the forum rules, just delete this topic and sorry for the inconvenience.
    People announce projects here all the time, it's not against the rules, just that the actual forum should have been this one:

    Projects and Job Recruitment

    One of the mods will probably move it.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  5. #5
    Registered User
    Join Date
    Jun 2010
    Posts
    10
    I see. But the "Projects and Job Recruitment" looks empty, it's like nobody ever goes there (368 topics vs 42,000 for the C++ forum).

    Anyway, I let the moderators do their job.

  6. #6
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by Laurent Gomila View Post
    I see. But the "Projects and Job Recruitment" looks empty, it's like nobody ever goes there (368 topics vs 42,000 for the C++ forum).
    That means your post will stay near the top of the forum for weeks instead of being buried and forgotten here by the end of the day Pretty sure most of the regs here do browse the project forum (without posting). If not they are probably disinterested in such things anyway.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  7. #7
    Registered User VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,607
    If not they are probably disinterested in such things anyway.
    If by that you mean there is no need for reflection in C++ then, yes, I would agree.

  8. #8
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446
    I mean, maybe for inspecting/enumerating type members. But I can't think of a reason I'd want to do that. Besides that's one heavy load because all objects metadata needs to be stored as code. Just ugh! And there's no library or framework that will change that.

    For anything else templates, boost::type_traits, RTTI, even boost::any will suffice.

    And the comparison with Qt is just plain wrong. Qt does what it does because it uses a meta compiler; The QT Meta Object Compiler (moc). Not a library or framework.
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

  9. #9
    Registered User
    Join Date
    Jun 2010
    Posts
    10
    CAMP is not made for binding C++ stuff to other C++ stuff. It's made to expose your C++ data types to "the outside", where you can only find things dynamically by their name and manipulate them through string values -- in other words, places where the compiled C++ information is not available.

    For example:
    - Binding your classes to script. Luabind and boost.python are great, but what if you want to support both Python and Lua? Will you write the exact same binding twice?
    - Enabling automatic import and export of your object and their properties to XML or other text formats. Doing so only requires to manipulate class and properties names and their values as strings, you don't care about the static C++ type information there.
    - Writing a generic GUI object editor. Again, all you need is the properties names, their type and a generic way to get and set their value from a string data.

    This project was not written by a passionate C++ developer who wanted to write useless templated code, it's written by an industrial company with specific needs.

    For example, in the game studio where I worked before, we used intensively a similar library of our own in 3 main areas:
    - binding the game entities to script (we used GameMonkey)
    - automatic import/export to XML
    - editing game objects in the level editor
    It was really powerful because all that we, the developers, had to write, is the abstract interface of the new classes. Then the game designers would automatically see these new classes in their scripts, in the level editor and in the XML configuration files. And all these three things use the same, unified interface (class, functions and properties names).
    It was really a huge time saver!

    And the comparison with Qt is just plain wrong. Qt does what it does because it uses a meta compiler; The QT Meta Object Compiler (moc). Not a library or framework.
    They use MOC and the QObject base class because they need meta information. They need to be able to find and use signals, slots and properties by their name, and call them with abstract QVariant values. This is exactly what CAMP does, using different techniques than Qt.

    And look what they do with their meta-object system:
    - QtDesigner
    - QtScript
    - QML
    - QtStateMachine
    - Qt animation framework
    - and a lot of other stuff that I don't know about (I'm sure they use it intensively in web stuff)

    I wouldn't say that it is useless.
    Last edited by Laurent Gomila; 06-16-2010 at 01:04 AM.

  10. #10
    &TH of undefined behavior Fordy's Avatar
    Join Date
    Aug 2001
    Posts
    5,793
    Quote Originally Posted by Laurent Gomila View Post
    I see. But the "Projects and Job Recruitment" looks empty, it's like nobody ever goes there (368 topics vs 42,000 for the C++ forum).

    Anyway, I let the moderators do their job.
    Moved, but I left a permanent redirect on the tech board.

  11. #11
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    This project was not written by a passionate C++ developer who wanted to write useless templated code, it's written by an industrial company with specific needs.
    This could be said of much of the worst code in history.

    I wouldn't say that it is useless.
    You shouldn't bill your library as providing reflection. From what I can tell from the documentation, it is a long way from such a goal.

    That said, a generalized API for binding C++ constructs to and from "dumb strings" isn't useless. I just don't think the protocol you've designed is useful or practical. Of course, the documentation is severely lacking; maybe you should come back when it is better.

    Soma

  12. #12
    Registered User
    Join Date
    Jun 2010
    Posts
    10
    This could be said of much of the worst code in history.
    Of course. But that doesn't mean that my code is bad. This doesn't mean anything in fact, except that the library is not useless.

    You shouldn't bill your library as providing reflection. From what I can tell from the documentation, it is a long way from such a goal.
    I know, that's why most people are confused about CAMP when they first read about it.
    But it truly is reflection: you can ask to the system which classes exist, what there name is, what properties they declare, you can manipulate them by their name, get/set them with abstract values, etc. What is it if not reflection?

    That said, a generalized API for binding C++ constructs to and from "dumb strings" isn't useless. I just don't think the protocol you've designed is useful or practical.
    Can you explain your point of view? Why isn't it practical, and what, in your opinion, would be more useful?

    Of course, the documentation is severely lacking; maybe you should come back when it is better.
    Can you tell me what's missing - except more examples/explanations for people who don't understand CAMP, and that we'll soon add?

  13. #13
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446
    Quote Originally Posted by Laurent Gomila View Post
    Can you explain your point of view? Why isn't it practical, and what, in your opinion, would be more useful?
    The question is not direct at me, but I'll give here my quick opinion.

    The major problem you have there is metadata storing at compile time. Your bind method, declare<T> is one huge impediment. It's not that you could do this in any other way with a library. You really couldn't, as I mentioned before. But in any mid size project the number of types you create is very large. Having to store their metadata during code is both slow and prone to errors. Even though it's assumed the user doesn't need to bind each and every type they create, it's also assumed the number of types they need to bind is still quite large on any project where they would need this functionality.

    Then you have one other problem; Templates. I didn't inspect your code. But I have a sneaky suspicion you didn't provide binding for templates, because you soon realized you cannot store metadata on a template basis. You would have to store it on an instantiation basis. And this mean you would be forcing a programmer using templates to bind hundreds of instantiations, most of them which would never be used at runtime. You would be forcing your user to grow their code exponentially. And to deviate from a C++ very basic principle: You don't pay for what you don't use.

    Then there's a final problem. Code optimizations. You cannot guarantee to the user of your library that any given member of a type is actually there. Member functions, operators, even entire classes, enumerators, functions and variables can be optimized away by the compiler. In order to succeed, I suspect you are forcing the compiler to not optimize them away, by having your library exposing them itself. You completely killed one of the most powerful optimizations the compiler can do for the user.

    ...
    EDIT: To be clear, I see a point to what you are trying to do. There indeed can be some obscure, rarely seen, uses for a reflection-like system in C++. I can't think of one. But I suspect there is. But a library/framework is never the way to do this. It's too heavy on the user. It forces them to create a huge amount of added code just to support reflection at runtime and it will never be able to satisfactorily deal with template programming. You asked me before if I would prefer instead to create two separate bindings if I needed to support two scripting languages. My obvious answer is yes! I definitely would that to having to do what you propose.

    It's even more scary when you think about code maintenance. When you need to do any sort of refactoring on your previous code, you are creating a maintenance nightmare since you force the user to maintain also the metadata.

    The only viable alternative is a metadata compiler, like Qt uses. Never, ever, a library.
    Last edited by Mario F.; 06-16-2010 at 06:26 AM.
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

  14. #14
    Registered User
    Join Date
    Jun 2010
    Posts
    10
    The major problem you have there is metadata storing at compile time. Your bind method, declare<T> is one huge impediment. It's not that you could do this in any other way with a library. You really couldn't, as I mentioned before. But in any mid size project the number of types you create is very large. Having to store their metadata during code is both slow and prone to errors. Even though it's assumed the user doesn't need to bind each and every type they create, it's also assumed the number of types they need to bind is still quite large on any project where they would need this functionality.
    Not using CAMP to achieve the same goals would be even more prone to errors and verbose.
    How would to write better code doing the same things without CAMP?

    Then you have one other problem; Templates. I didn't inspect your code. But I have a sneaky suspicion you didn't provide binding for templates, because you soon realized you cannot store metadata on a template basis. You would have to store it on an instantiation basis. And this mean you would be forcing a programmer using templates to bind hundreds of instantiations, most of them which would never be used at runtime. You would be forcing your user to grow their code exponentially. And to deviate from a C++ very basic principle: You don't pay for what you don't use.
    Of course it is technically impossible. But the concept of templates vanishes as soon as you leave the C++ world anyway. I can't see any usage for a template binding in a Lua, XML or GUI module for example.
    And if you want to bind several specializations of the same template you can still do it with CAMP (using a template function for binding it so that you don't have to duplicate code). So what's the problem? I mean, you pay for something, at the end you have a generic metadata system that you can use to do plenty of things automatically. CAMP is not made to save the world, just to provide an extra level of abstraction to communicate with C++ types for users that need it.

    Then there's a final problem. Code optimizations. You cannot guarantee to the user of your library that any given member of a type is actually there. Member functions, operators, even entire classes, enumerators, functions and variables can be optimized away by the compiler. In order to succeed, I suspect you are forcing the compiler to not optimize them away, by having your library exposing them itself. You completely killed one of the most powerful optimizations the compiler can do for the user.
    I don't get this point, what do you mean?

    To be clear, I see a point to what you are trying to do. There indeed can be some obscure, rarely seen, uses for a reflection-like system in C++.
    I noticed that you didn't answer to my previous post, where I talk about all the Qt modules that are based on their meta-object system. Would you say that those modules implement "obscure, rarely seen" features? You seem to know and like Qt, so you should be aware of all the benefits of a meta-object system.

    The only viable alternative is a metadata compiler, like Qt uses. Never, ever, a library.
    Precompilers generate C++ code, and this code is designed to use a particular (often internal and hidden) framework. How is that different from CAMP? I mean, nothing prevents you from using a precompiler to generate your CAMP declarations.
    In fact, I think that it would be a great addition for CAMP, and we'll probably work with open-source generic C++ parser to create our own CAMP-based precompiler. What do you think about it? It solves all your problems, right?

  15. #15
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    This doesn't mean anything in fact, except that the library is not useless.
    If you know this, don't use such a statement as an argument for your work.

    I know, that's why most people are confused about CAMP when they first read about it.
    Again. If you know this, why not be clear about what your work is and is not capable of?

    Why isn't it practical, and what, in your opinion, would be more useful?
    The protocol you've developed is not practical because the additional burden on the developer. You need to alleviate this burden if you want this to be widely acknowledged. As much as I hate implementation macros, I feel this protocol could be improved with additional macro facilities to reduce repetition and the initial burden.

    And this mean you would be forcing a programmer using templates to bind hundreds of instantiations, most of them which would never be used at runtime. You would be forcing your user to grow their code exponentially.
    This doesn't make any sense. If the instantiation doesn't provide a necessary facility, why would the developer take the time to bind it? If the instantiation does provide a necessary facility (even rarely used), the code for the instantiation would already be required by the binary.

    You asked me before if I would prefer instead to create two separate bindings if I needed to support two scripting languages. My obvious answer is yes! I definitely would that to having to do what you propose.
    *shrug*

    I'm not even sure how this library alleviates the issue. As far as I can tell, it only supports a few trivial primitives ("dumb string"). You'd still need to code the necessary compatibility layer the API for the embedded language interpreter expects. I suppose, in the fullness of time, new primitives for popular projects may be available.

    How would to write better code doing the same things without CAMP?
    As it stands, as far as I can tell, the library only provides the introspection and invocation parts of reflection primarily as a means of binding C++ constructs to "dumb strings" or other interpretations/layers. As a general purpose library layer providing such facilities you could do only a little better in the implementation. (The protocol and API layer could be much better.) A library designed to be used during the development of constructs specifically designed to be exported by such facilities could be far superior.

    Of course it is technically impossible. But the concept of templates vanishes as soon as you leave the C++ world anyway. I can't see any usage for a template binding in a Lua, XML or GUI module for example.
    And if you want to bind several specializations of the same template you can still do it with CAMP (using a template function for binding it so that you don't have to duplicate code). So what's the problem? I mean, you pay for something, at the end you have a generic metadata system that you can use to do plenty of things automatically. CAMP is not made to save the world, just to provide an extra level of abstraction to communicate with C++ types for users that need it.
    I imagine that also applies to other facilities that "vanish" like operators, function overloads, and variable argument functions?

    Soma

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Why Open Source?
    By sarah22 in forum General Discussions
    Replies: 17
    Last Post: 03-22-2010, 03:48 PM
  2. Joining an open source project
    By Poincare in forum C Programming
    Replies: 5
    Last Post: 11-27-2009, 03:44 PM
  3. iterate through types in a library (reflection)
    By bling in forum C++ Programming
    Replies: 1
    Last Post: 10-29-2008, 05:15 PM
  4. Source code of the standard library functions...
    By Nutshell in forum C Programming
    Replies: 2
    Last Post: 01-21-2002, 12:35 PM
  5. OpenGL Open Source
    By gnu-ehacks in forum Game Programming
    Replies: 2
    Last Post: 11-29-2001, 12:50 AM

Tags for this Thread