Widnows 7 DEP

[VirtualAce]

I just found today that the reason some of my older games do not run on Vista / Windows 7 is due to Data Execution Prevention. My theory is based on my experiences with Olly Debugger and game executables. Most game executables are encrypted which means when you run the exe a small stub runs that loads the encrypted data into memory and converts it to opcodes and then jumps to the start address and the game begins. Well this is exactly what DEP is trying to prevent. It prevents programs from running code from the data area which is a usually a sign the program is malicious.

So if you have any games that do not run in Windows 7 or Vista and setting them to compatibility mode does not work I would turn off DEP for those programs.

Note that this can happen with applications too if they run stubs or code that triggers DEP to shut down the program.

I've been trying to solve some of my game launch issues for a very long time and I finally have almost every old game from XP running in Windows 7 64 bit. Even modern games like the Sims 3 suffer from this. I would imagine there is a way for programmers to temporarily disable DEP or ask for permission to run code from / in the data area so that every encrypted exe out there does not flag DEP and shut down the program. Many programs seem to be incompatible with DEP.

Has anyone experienced DEP shutting down your legitimate program either at work or at home?

[Yarin]

Use FOSS </smartass>

[Fordy]

Your system should have a policy for DEP and you should be able to exclude DEP for specific apps in the control panel (at least you could in XP, though DEP is better with later windows and so might work a little differently).

By the way, there are API functions like VirtualProtectEx that allow you to mark allocated memory as executable, but I guess before DEP nobody cared!

[SMurf]

This is just another case of programmers who should know better... okay, perhaps not the ones who work for EA... going for the path of least resistance, when the correct solution has been documented for almost 10 years.

Every time that this happens, Jesus cries, kittens die and MS tries to sell another convoluted sling to developer corporations with the promise that it'll make their programs better.

[brewbuck]

If a VM region is marked "executable" then DEP will not prevent code from executing there even if that region is writable (Windows doesn't implement W^X protection). It's still possible to implement self-modifying code on Windows, but older games that assume that writable == executable may break. Worst case, even if Windows is changed to a W^X model, you can always twiddle protection flags via VirtualProtect() and related functions.

DEP is just a band-aid to prevent some of the more naive buffer overflow exploits. It doesn't offer much in the way of serious software protection.

[phantomotap]

Every time that this happens, Jesus cries, kittens die and MS tries to sell another convoluted sling to developer corporations with the promise that it'll make their programs better.

I get what this is, but I have to wonder anyway how "DEP" would make other vendor programs better?

[Edit]
*in before paan ban*
[/Edit]

Soma

[VirtualAce]

So perhaps the issue is that the stubs in the older executables simply assume writable == executable since they were written before DEP and you could get away with it. Newer stubs and encryption most likely use some of the methods described in this post so they do not get shut down by DEP. Encrypted EXEs are quite common when it comes to video games. Most if not all of my games are encrypted and all my Steam, Impulse and Gamefly games are definitely encrypted. This is why game patches for the retail version of the game or 3rd party patch utilities do not work on Steam, Impulse or Gamefly versions of the executables. Those utilities are looking for specific addresses and specific hex values at those addresses and when they cannot find them they will not run. Most of the utils I use are to alter the screen resolution to 1600x900 or change the FOV, etc. You would think developers would put this in an ini file or xml config file somewhere instead of hard-coding values into the executable.

[Fordy]

Bubs, you've piqued my interest on this. How do the apps allocate the memory for the stubs? I wonder if there's a way of patching, proxying or some other way of getting the app to mark the memory as executable before usage?

[brewbuck]

Every packer/wrapper works differently, but MOST of them work directly in the image VM space, i.e. they rely on the section tables of the image to provide the correct protection flags (if not, they use VirtualProtect() to alter the flags to allow them to perform their operations).

In the case of a game that (incorrectly) assumes W==X, you MAY be able to fix it by altering the section tables and setting execute permission on each section that also has write permission. If you actually want to try doing this, I will PM some pseudocode of how to alter the PE headers in such a way.

However, old games should still be working -- the PE header contains a flag that indicates that the EXE/DLL is "DEP-aware." If this flag is not set, the OS is supposed to alter its behavior to allow for the faulty assumption. But this does not work in all circumstances. You may need to mess with Windows compatibility modes to get things working right.

[VirtualAce]

Most of the games are now working. The only game so far that absolutely refuses to run is Thief 3. I believe it is b/c of the way the original copy protection worked from the original CD. Also not sure why places like GoG and Steam sell the game if it does not work on Windows 7 64-bit. There are reports on forums of people getting the GoG version to work on Windows 7 64-bit without setting any compatibility settings. Those forums are not much help in tracking down how to achieve this and most of the suggestions do not work. Thief 3 is a bit weird b/c it is executed by a parent process which I assume is also decrypting the executable. I have a book authored by a developer who worked on Thief 3 so perhaps I could email him for some information on how I might be able to get it to work.

I cannot get Olly debugger to look at the executable b/c it is encrypted. Also b/c it does not run I cannot attach later when the executable is decrypted in memory so I'm not sure what to do. I wonder if I could figure out which type of encryption it is using and decrypt it myself. Since it is a game and must be decrypted rather fast I'm sure it is not that strong. What really stinks is that I own the entire Thief collection from years ago and it also does not work. So I have two copies of Thief 3 and neither of them work.

[brewbuck]

Also b/c it does not run I cannot attach later when the executable is decrypted in memory so I'm not sure what to do.

A W^X dumper may be able to do this for you. Basically, you hook VirtualAlloc() and VirtualProtect(). You intercept any request to set a region writable or executable (or both). Any request to set a region executable is silently ignored by your hook. Eventually after decryption the loader will transfer control to the region. Because you masked out the execute bit, this will produce a SEH exception, which you catch with a global handler. From there, you dump the page, turn the execute bit on, and let the thing continue.

At the end you splice all the page dumps together to create the full image.