Thread: Custom Calling Conventions w/ GCC?

  1. #1
    Password:
    Join Date
    Dec 2009
    Location
    NC
    Posts
    587

    Custom Calling Conventions w/ GCC?

    Avoiding patching, how could I configure GCC to use a custom calling convention? Say I wanted to specify ints should be passed in registers a, c, b, d in that order, how could I do that?

  2. #2
    Password:
    Join Date
    Dec 2009
    Location
    NC
    Posts
    587
    I guess I should clarify, by a, c, b, d, I mean either the 32 bit or 64 bit (e/r)*x registers.

  3. #3
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,656
    So how would you distinquish between calling your functions (with some params in regs) with other library functions (which expect everything on the stack)?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  4. #4
    Registered User
    Join Date
    Dec 2008
    Location
    Black River
    Posts
    128
    You can use gcc's regparm(n) function attribute to specify arguments passed via registers. As far as I can tell, you can only use it for x86, which means no 64bit registers.
    Stick close to your desks and never program a thing,
    And you all may sit in the standards commitee!

  5. #5
    Password:
    Join Date
    Dec 2009
    Location
    NC
    Posts
    587
    > So how would you distinquish between calling your functions (with some params in regs) with other library functions (which expect everything on the stack)?
    I wouldn't need to. Everything in my OS would use my calling convention. It will be as widely used as Windows' stdcall. I want to use a stack based calling convention in my 64bit OS, but for x86-64, GCC defaults to a really confusing register based convention. I want my OS to be easy to program for in both C and asm. Using a register based convention makes asm a real pain.

    regparm is similar, but I actually want to avoid register based conventions. My example was just to show how specific I wanted to be, not what I really wanted to do.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Profiler Valgrind
    By afflictedd2 in forum C++ Programming
    Replies: 4
    Last Post: 07-18-2008, 09:38 AM
  2. Run time differences in Linux gcc versions
    By circuitbreaker in forum C++ Programming
    Replies: 7
    Last Post: 02-14-2008, 11:09 PM
  3. gcc or g++ Question
    By TK in forum Linux Programming
    Replies: 3
    Last Post: 07-17-2002, 10:36 AM
  4. gcc 3.1
    By itld in forum C++ Programming
    Replies: 1
    Last Post: 05-18-2002, 04:00 AM
  5. GCC compile error coz of file pointer
    By Shadow in forum C Programming
    Replies: 5
    Last Post: 04-20-2002, 12:50 PM