Thread: Exe & data block generation

  1. #1
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446

    Exe & data block generation

    For Windows.

    I need to generate an executable and merge a data block to it, all at runtime. That executable needs to know where this data block is and perform any necessary operations on it. To illustrate, I need similar functionality to the generation of a setup executable.

    However this is new ground for me and I'd appreciate if you have any advice on where I should start educating myself.
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

  2. #2
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    What platform is it for?

  3. #3
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446
    Read the OP slowly
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

  4. #4
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Appending arbitrary data to the end of an EXE file does not damage the file. You could put some uniquely identifiable string at the beginning of your data block, then append it to the file. To find the block, the EXE opens itself and scans for the marker string.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  5. #5
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Quote Originally Posted by Mario F. View Post
    Read the OP slowly
    Heheh. Distracted...as usual.

    Are you taking pieces from an existing exe, or simply generating raw opcodes, etc?

  6. #6
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446
    Not sure yet how I should approach it.

    I suppose I could create the executable on a different project and then serialize it on my main project where I plan to assign the datablock(s) and generate the final executable during runtime.

    But how do I instruct my executable to read attached datablocks as brewbuck suggests? That's my first doubt.

    I guess I need an idiots guide to the PE format. But I'm having trouble finding such a thing.
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

  7. #7
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Quote Originally Posted by Mario F. View Post
    Not sure yet how I should approach it.

    I suppose I could create the executable on a different project and then serialize it on my main project where I plan to assign the datablock(s) and generate the final executable during runtime.

    But how do I instruct my executable to read attached datablocks as brewbuck suggests? That's my first doubt.

    I guess I need an idiots guide to the PE format. But I'm having trouble finding such a thing.
    Well, it's a huge spec, and quite fickle. I would go with an approach similar to what Brewbuck suggested: Declare a global array in the program with a size equal to (or at least large enough for) the section you'll be adding to it, with some predefined pattern to be searched for. Generate the EXE, search for the pattern, patch, execute.

  8. #8
    Registered User
    Join Date
    Dec 2006
    Location
    Canada
    Posts
    3,229
    Doesn't it have some kind of a checksum?

  9. #9
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by cyberfish View Post
    Doesn't it have some kind of a checksum?
    It does, but Windows doesn't enforce it.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  10. #10
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Quote Originally Posted by brewbuck View Post
    It does, but Windows doesn't enforce it.
    One exception to that is with registered device drivers, though.

  11. #11
    Registered User C_ntua's Avatar
    Join Date
    Jun 2008
    Posts
    1,853
    If checksum was a problem, you could convert the opcode to assembly and insert the data as a sting. Assign a variable to the string so your exe file can use the data. Then compile to opcode again having a valid exe. Not the simpliest way to go...

  12. #12
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Well, I was bored.

    Here's a simple proof-of-concept using the global array modification approach:

    Code:
    
    /*
        demo.cpp
    */
    
    #include <iostream>
    #include <fstream>
    #include <stdexcept>
    #include <vector>
    #include <string>
    #include <iterator>
    #include <cstring>
    #include <algorithm>
    
    using namespace std;
    
    typedef vector< unsigned char >
        buffer_type;
    
    inline void error( string const& message = string( ) )
    {
        throw runtime_error( message );
    }    
        
    void read_contents( string const& file, buffer_type& buffer )
    {
        ifstream
            in( file.data( ), ios::binary );
        if( !in )
            error( string( "cannot open input file '" ) + file + "'" );
        buffer.clear( );
        for( ;; )
        {
            int
                ch = in.get( );
            if( in.bad( ) )
                error( "file system read failure" );
            if( in.eof( ) )
                break;
            buffer.push_back( ch );    
        }
    } 
    
    int main( int argc, char** argv )
    {
        char const*
            program = argv[ 0 ];
        cerr << "File Patch Utility / File to Array Declaration Translator" << endl;
        try
        {
        /*
            Find and patch a pattern in a file
        */
            if( argc == 4 )
            {
                char const
                    * pattern_file = argv[ 1 ], 
                    * patch_file = argv[ 2 ], 
                    * target_file = argv[ 3 ];
                buffer_type
                    pattern_buffer, 
                    patch_buffer, 
                    target_buffer;
                read_contents( pattern_file, pattern_buffer );
                read_contents( patch_file, patch_buffer );
                read_contents( target_file, target_buffer );
            /*
                For some reason, the ios::in flag is needed to prevent file truncation
            */
                ofstream
                    out( target_file, ios::binary | ios::in );
                if( !out )
                    error( string( "cannot open target file '" ) + target_file + "'" );
                buffer_type::iterator
                    found = search
                ( 
                    target_buffer.begin( ), 
                    target_buffer.end( ), 
                    pattern_buffer.begin( ), 
                    pattern_buffer.end( ) 
                );
                if( found == target_buffer.end( ) )
                    error
                    ( 
                        string( "pattern in '" ) + pattern_file + 
                        "' not found in target '" + target_file + "'"                
                    );
                size_t
                    position = distance( target_buffer.begin( ), found );
                out.seekp( position, ios::beg );
                copy
                ( 
                    patch_buffer.begin( ), 
                    patch_buffer.end( ), 
                    ostream_iterator< buffer_type::value_type >( out ) 
                );
                if( out.bad( ) )
                    error( "file system write failure" );
            }
        /*
            Translate a file to array declaration source-code
        */
            else if( argc == 3 )
            {
                char const
                    * source_file = argv[ 1 ], 
                    * target_file = argv[ 2 ];
                buffer_type
                    source_buffer;
                read_contents( source_file, source_buffer );
                ofstream
                    out( target_file, ios::trunc );
                if( !out )
                    error( string( "cannot open output file '" ) + target_file + "'" );
                out << "{";
                for( size_t index = 0, size = source_buffer.size( ), width = 15; index < size; ++index )
                {
                    if( index != 0 )
                        out << ", ";
                    if( index % width == 0 )
                        out << endl << "\t";
                    out << ( void* )source_buffer[ index ];
                }
                out << endl << "};" << endl;
                if( out.bad( ) )
                    error( "file system write failure" );
            }
            else
                error( );
        }
        catch( exception const& error )
        {
            if( strlen( error.what( ) ) != 0 )
                cerr << "Error: " << error.what( ) << endl;
            cerr << "Usage: " << program << " <pattern_file> <patch_file> <target_file>" << endl;    
            cerr << " ...OR..." << endl;
            cerr << "Usage: " << program << " <source_file> <target_file>" << endl;    
            return 1;
        }
        return 0;
    }
    So let's say we have the file 'pattern.txt' that contains the search pattern:

    "-> This is the data to be searched for. Ideally it should be longer to prevent false positives. <-"

    First we translate it to a source-code compatible array initializer, then copy that to our target program:

    C:\> demo pattern.txt declaration.txt

    Code:
    
    /*
        test.cpp
    */
    
    #include <iostream>
    #include <algorithm>
    #include <iterator>
    #include <cstring>
    
    #define LARGEST_BUFFER_SIZE_NEEDED 8192
    
    char buffer[ LARGEST_BUFFER_SIZE_NEEDED ] = 
    {
        0x2d, 0x3e, 0x20, 0x54, 0x68, 0x69, 0x73, 0x20, 0x69, 0x73, 0x20, 0x74, 0x68, 0x65, 0x20, 
        0x64, 0x61, 0x74, 0x61, 0x20, 0x74, 0x6f, 0x20, 0x62, 0x65, 0x20, 0x73, 0x65, 0x61, 0x72, 
        0x63, 0x68, 0x65, 0x64, 0x20, 0x66, 0x6f, 0x72, 0x2e, 0x20, 0x49, 0x64, 0x65, 0x61, 0x6c, 
        0x6c, 0x79, 0x20, 0x69, 0x74, 0x20, 0x73, 0x68, 0x6f, 0x75, 0x6c, 0x64, 0x20, 0x62, 0x65, 
        0x20, 0x6c, 0x6f, 0x6e, 0x67, 0x65, 0x72, 0x20, 0x74, 0x6f, 0x20, 0x70, 0x72, 0x65, 0x76, 
        0x65, 0x6e, 0x74, 0x20, 0x66, 0x61, 0x6c, 0x73, 0x65, 0x20, 0x70, 0x6f, 0x73, 0x69, 0x74, 
        0x69, 0x76, 0x65, 0x73, 0x2e, 0x20, 0x3c, 0x2d
    };
    
    int main( void )
    {
        std::copy( buffer, buffer + strlen( buffer ), std::ostream_iterator< char >( std::cout ) );
        return 0;
    }
    Now let's say we have the file 'patch.txt' that contains the text:

    "'I wonder if I've been changed in the night? Let me think. Was I the same when I got up this morning? I almost think I can remember feeling a little different. But if I'm not the same, the next question is "Who in the world am I?" Ah, that's the great puzzle!'
    -- Alice in Wonderland"

    Patch the file, and then verify that it overwrote the correct slot:

    C:\> demo pattern.txt patch.txt test.exe
    C:\> test

    The only real limitation is the fixed block-size constraint. If that isn't practical, then what Brewbuck suggested would probably be the best route to go.

  13. #13
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446
    Not bad at all, sir.

    The fixed block is not a biggie. I could always pad my output. However this goes to say how really bad it is to try and solve a problem at 4am. Had I refrain from posting until morning, and I probably would have found the answer which was right in front of my eyes: I can always have the executable read itself from disk to search for the datablock header.

    Thanks folks. And sorry for this...
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

  14. #14
    'Allo, 'Allo, Allo
    Join Date
    Apr 2008
    Posts
    639
    Resources allow this without restriction on size. BeginUpdateResource / UpdateResource / EndUpdateResource will add data to an exe, and the usual FindResource / LoadResource / LockResource / SizeofResource give you access to them. Resource functions

    In fact, that's exactly the method my DynSxS tool uses to check whether the relevant VS runtimes are installed before extracting and running the real program of interest from its' resources. It's the second link in my sig if you want some sample code. It doesn't generate an exe at runtime but if you can do that it should be easy to insert a few extra pushes and calls to get the pointer to the data.

  15. #15
    (?<!re)tired Mario F.'s Avatar
    Join Date
    May 2006
    Location
    Ireland
    Posts
    8,446
    Quote Originally Posted by adeyblue View Post
    Resources allow this without restriction on size. BeginUpdateResource / UpdateResource / EndUpdateResource will add data to an exe, and the usual FindResource / LoadResource / LockResource / SizeofResource give you access to them. Resource functions
    Nice approach indeed. Wouldn't think of that, if you hadn't tell me.

    I do not have any porting requirements, so I could use that. However, I don't want to use windows headers on this one. Still, thank you adeyblue.
    Originally Posted by brewbuck:
    Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Lame null append cause buffer to crash
    By cmoo in forum C Programming
    Replies: 8
    Last Post: 12-29-2008, 03:27 AM
  2. Replies: 3
    Last Post: 04-18-2008, 10:06 AM
  3. question about a working linked list
    By cold_dog in forum C++ Programming
    Replies: 23
    Last Post: 09-13-2006, 01:00 AM
  4. All u wanted to know about data types&more
    By SAMSAM in forum Windows Programming
    Replies: 6
    Last Post: 03-11-2003, 03:22 PM
  5. C Programming Question
    By TK in forum A Brief History of Cprogramming.com
    Replies: 13
    Last Post: 07-04-2002, 07:11 PM