MSI custom action dll function question

This is a discussion on MSI custom action dll function question within the Windows Programming forums, part of the Platform Specific Boards category; I am currently developing a MSI package and after doing some research on custom action dll, I found the following ...

  1. #1
    Registered User
    Join Date
    Jul 2008
    Posts
    19

    MSI custom action dll function question

    I am currently developing a MSI package and after doing some research on custom action dll, I found the following link.

    http://msdn.microsoft.com/en-us/libr...38(VS.85).aspx

    It says :

    Note that any called functions, including custom actions in DLLs, must specify the __stdcall calling convention. For example, to call CustomAction use the following.

    Code:
    #include <windows.h>
    #include <msi.h>
    #include <Msiquery.h>
    
    UINT __stdcall CustomAction(MSIHANDLE hInstall)
    So I understand that I need UINT as my function return type and __stdcall as a calling convention. For those of you who have worked with MSI before, do you know if putting the argument "MSIHANDLE hInstall" is required as well?

    I've tried running my MSI with the argument removed from the function, and it ran well, but I was afraid if might unexpectedly crash one day because of that.

  2. #2
    a_capitalist_story
    Join Date
    Dec 2007
    Posts
    2,650
    If you're not using the handle, i.e. to get properties or some other data out of the installation process itself, then I imagine it's not a big issue.

  3. #3
    'Allo, 'Allo, Allo
    Join Date
    Apr 2008
    Posts
    611
    Remember that with stdcall the callee cleans the stack. If you change the number of arguments that the function takes you'll end up unbalancing it for the calling function.

  4. #4
    Registered User
    Join Date
    Jul 2008
    Posts
    19
    Quote Originally Posted by adeyblue View Post
    Remember that with stdcall the callee cleans the stack. If you change the number of arguments that the function takes you'll end up unbalancing it for the calling function.
    Thank you guys for the comments. adeyblue, so were you saying that if the callee (the custom action dll function) doesn't specify the argument, then there's no one there to free the msi handle, and the stack trace would get corrupted? But the caller of the function (msi engine) wouldn't push the handle to the stack in the first place if the callee doesn't take any argument (the handle in this case), would it?
    Last edited by donglee; 02-12-2009 at 03:46 PM.

  5. #5
    'Allo, 'Allo, Allo
    Join Date
    Apr 2008
    Posts
    611
    The code that calls the MSI wouldn't know that your function didn't take the handle. MS declared a prototype to follow and, since their code has no knowledge of yours beyond that it exists, it will always call it the same way pushing the handle and expecting the custom function to pop it off and clean the stack.

    If you've got a recent-ish version of Visual Studio, you can compile the following with the stack frames run time check enabled (Solution Properties-> Configuration Properties->C/C++->Code Generation->Basic Runtime Checks), and that should throw up an error. Alternatively, compile without and check the stackpointer value in the last cout with the rest.

    Code:
    #include <windows.h>
    #include <iostream>
    #include <msi.h>
    
    // the function pointer the function should use
    typedef UINT (__stdcall*CustomFunc)(MSIHANDLE);
    
    // test functions
    UINT __stdcall CustomAction(MSIHANDLE arg)
    {
        return 0;
    }
    
    UINT __stdcall CustomActionNoArg()
    {
        return 0;
    }
    
    int main()
    {
        void* stackptr = 0;
    
        MSIHANDLE mHandle = NULL;
        CustomFunc cust = CustomAction;
        _asm mov stackptr, esp; // save the stack pointer
    
        std::cout << "Stack pointer before good call: " << stackptr << '\n';
        cust(mHandle);
        _asm mov stackptr, esp;
    
        std::cout << "Stack pointer after good call: " << stackptr << '\n';
    
        // this is the equivalent of what happens in the msi code
        // only the result will be returned from GetProcAddress
        cust = (CustomFunc)CustomActionNoArg;
    
        _asm mov stackptr, esp;
    
        std::cout << "Stack pointer before bad call: " << stackptr << '\n';
        cust(mHandle);
        _asm mov stackptr, esp;
    
        std::cout << "Stack pointer after bad call: " << stackptr << '\n';
    }
    I get the following
    Stack pointer before good call: 0012FF70
    Stack pointer after good call: 0012FF70
    Stack pointer before bad call: 0012FF70
    Stack pointer after bad call: 0012FF6C
    I don't know which code calls CustomAction so I can't tell if it will cause any damage or negative effects in the installer but safe to say, it's not good practice.

  6. #6
    Registered User
    Join Date
    Mar 2005
    Location
    Mountaintop, Pa
    Posts
    1,059
    Quote Originally Posted by donglee View Post
    I am currently developing a MSI package and after doing some research on custom action dll, I found the following link.

    http://msdn.microsoft.com/en-us/libr...38(VS.85).aspx

    It says :



    So I understand that I need UINT as my function return type and __stdcall as a calling convention. For those of you who have worked with MSI before, do you know if putting the argument "MSIHANDLE hInstall" is required as well?

    I've tried running my MSI with the argument removed from the function, and it ran well, but I was afraid if might unexpectedly crash one day because of that.

    Post your MSI concerns on the MS MSI (installer) discussion newsgroup. I'm sure Alex31 will agree.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 4
    Last Post: 05-13-2011, 08:28 AM
  2. dllimport function not allowed
    By steve1_rm in forum C++ Programming
    Replies: 5
    Last Post: 03-11-2008, 03:33 AM
  3. DLL function by ordinal
    By George2 in forum Tech Board
    Replies: 2
    Last Post: 02-07-2008, 12:08 AM
  4. Including lib in a lib
    By bibiteinfo in forum C++ Programming
    Replies: 0
    Last Post: 02-07-2006, 01:28 PM
  5. Request for comments
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 15
    Last Post: 01-02-2004, 09:33 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21