Enforcing Machine Code Restrictions?

Hello,

I'm currently looking at creating a sandboxed environment for running code, but I don't necessarily wanna go down the whole VM route. Cross-platform usage is not a concern at this point. What I want to do is create a situation where you can compile the usual (in this case, x86) code and run it via a sort of loader which completely separates what you do from the operating system.

"You've described a VM you ignorant (insert bizarre foreign colloquialism here)!", you've no doubt said to yourself. Ah, but i want to be cleverererer than that! ;)

If we look at the Win32 environment specifically; let's say that I have a loader that effectively just reads your code from a file, shoves into an executable page in memory and starts a thread at its entry point. Your code isn't in PE format, you can't statically link to any Win32 functions.

So you're running in my loader's address space... but there's a problem. Although you can't import any Win32 functions, because your code is just x86 you can attempt to look at a location in the address space that you think contains KERNEL32.DLL (because every Windows program has this in its address space). From there, you can walk the export table and effectively link to the functions you need to break out of this dump and do some REAL stuff...

Now, Address Space Layout Randomization (which was introduced in Vista) mitigates this to some extent, but what I would like to do is pre-empt code from getting this far.

The best way I can think of doing this is by scanning the code prior to execution for particular instructions (like CALL for example) and analyzing the context to see if badness is being attempted. This would work swimmingly for an immediate operand, but would be harder to do with registers or indirect memory locations without effectively interpreting the code (which I want to avoid!).

One way I could perhaps alleviate this is to enforce (via software) separation of code and data segments, so that no instruction can read or write memory in the code segment. I suppose I could extend this to "locking down" the segment registers so that if instructions like "POP CS" are detected then the program can be stopped before execution. The only thing is, is there any difference between the CS and DS registers in most Win32 programs?

I know I'm in way over my head here, so I'll keep the question simple: can I do what I want to do without resorting to interpretation? Am I wasting my time even thinking about this?

I'd say no can do. You cannot scan the code before execution, because it's easy enough to get around such things by being tricky enough, e.g. xor-masking the relevant code and unmasking it at runtime. Not to mention the incredible pattern recognition engine you'd need to realize that any given code is malicious, even if you can scan it.
Since you run in the same address space, you can't do page protection things, because you'd be killing your own loader.

There are two things that could work here.
One is going to the kernel level and implementing your own subsystem. Windows already has the Win32, OS/2 and POSIX subsystems. Incidently, only applications running in the Win32 subsystem need kernel32.dll and friends. You could - in theory - write your own loader and subsystem at the kernel level and have processes that can use only your API for doing stuff. The problem with this approach is that I have no idea if MS has ever released documentation on how to write a subsystem, or if they reserve that right for themselves.
The other option is to use virtualization support to kind of let the client program run inside such an environment. The problem with this is the huge programming effort (you'd be implementing your own little OS, even if it is very small and specialized).

I would say that preventing code from touching the code segment (or pages if segments aren't really relevant anymore) would go most of the way to stopping it from hiding things.

One other idea I've had overnight is to somehow replace long JMPs and CALLs with a call to a sort of gate function which checks the context at that point and decides whether to allow or raise an exception. As any calculations would have already been done then there's no guesswork involved, just checking that the destination address is within bounds. No other instructions would need to be looked at. Sound feasible?

Quote:

---

Originally Posted by SMurf

One other idea I've had overnight is to somehow replace long JMPs and CALLs with a call to a sort of gate function which checks the context at that point and decides whether to allow or raise an exception. As any calculations would have already been done then there's no guesswork involved, just checking that the destination address is within bounds. No other instructions would need to be looked at. Sound feasible?

---

How exactly are you planning to replace 6 bytes of code (original jmp/call) with, for example, 12 bytes (gateway call)?

Yes, you're right, that did cross my mind... ;)

Only way I can think of is to require that those particular instructions in the compiled code include NOPs for padding. Not ideal.

Quote:

---

Originally Posted by maxorator

How exactly are you planning to replace 6 bytes of code (original jmp/call) with, for example, 12 bytes (gateway call)?

---

The same way everybody else does that -- a trampoline.

There is no way to prevent malicious code from making a total ass of itself except through avirtual machine, thats what virtual machines are for. Now you could write your own OS or subsystem, and limit what teh code in ring 3 has access to insofar as callgates, but ultimately it can always do somethign malicious with those as well, if they for example allow file writing.

Just bite the bullet and run Bochs, thent he code xcan do whatever it wants and you dont have to be bothered with the specifics of the VM< just write plain old code and import it through a cd or hard drive image.

abachler, I know a VM would solve this problem easy, but the idea is to see how far I could control a more conventional situation, given the opportunity to review the the entire code prior to execution.

Another idea I've come up with is to scan for long JMPs and CALLs, make a note of their locations and then replace the opcode in question with an invalid one. This would generate an illegal instruction exception which I could then handle, check the faulting address against the table that I made earlier and check the access bounds that way (or leave it unhandled if I didn't cause it). Would there be any drawbacks to that approach?

As CornedBee pointed out, the program could easily hide JMP's/CALL's, making scanning pointless. It seems the only way to do this would be a VM...

Surely marking code pages as read only would nix that?

True. In that case, it might be doable. But you'd probably need to use a different invalid instruction for each type of branch, to keep track of things. In each case, you could just overwrite the first byte of the branch instruction. Of course, if it's a CALL, you'd need to push the return address onto the stack manually...

Quote:

---

Originally Posted by SMurf

Surely marking code pages as read only would nix that?

---

Yes, but you also have to mark all non-code pages as NX.

Can I be pedantic and point out that POP CS is not a legal x86 instruction. The fact is that the bit pattern that if you follow other PUSH/POP pairs, POP CS is 0x0F, which is used as a prefix for x86 instructions.

Next, as you say, CS and DS cover the same area in memory.

Scanning for jmp/call instructions will not prevent this:

Code:

---

    push a
    push b
    ret
a:
....

b:
  ....
  ret

---

This will jump to b, and return to a.

And if you think build something that scans and detects that, I can almost certainly make something up that is harder to discover, using offsets and the return address of the current function or some such.

Also consider jump tables (such as you'd use with switch-statements for example).

--
Mats

Quote:

---

Originally Posted by matsp

Can I be pedantic and point out that POP CS is not a legal x86 instruction. The fact is that the bit pattern that if you follow other PUSH/POP pairs, POP CS is 0x0F, which is used as a prefix for x86 instructions.

Next, as you say, CS and DS cover the same area in memory.

Scanning for jmp/call instructions will not prevent this:

Code:

---

    push a
    push b
    ret
a:
....

b:
  ....
  ret

---

This will jump to b, and return to a.

And if you think build something that scans and detects that, I can almost certainly make something up that is harder to discover, using offsets and the return address of the current function or some such.

Also consider jump tables (such as you'd use with switch-statements for example).

--
Mats

---

Things aren't looking so well. :D

I just thought of another thing, too. Since raw data can be mixed with the instructions, the scanner would have to follow all possible paths to make sure it doesn't inadvertantly overwrite a chunk of data.

CS and DS do not necessarily point to the same area of memory, and in fact may not be ABLE to point to the same area for some systems where the status pins are used as address pins to impliment a harvard architecture.

matsp: Yes, yes, I know. I was using POP CS as an ideological thing (IIRC I wanted to use that particular instruction when I was playing about with assembler a few months ago and was outraged to find that I couldn't! :().

I've looked at the RET thing and could feasibly track those as well, although this would start to get to the point where I'd be generating exceptions every few instructions, hmm :o

Sebastiani: Aha, but seeing as this is my own executable format, what say I specify that the code pages cannot contain any data? I expect to be able to identify each and every byte as an instruction or part of an instruction and if I can't... well, the code won't be run. ;)

This avoids the whole halting problem as far as the scanner is concerned. All it needs to know is the size of all instructions.

I suppose that my idea is effectively a VM with the processing for free (the code itself is already native, but the resources involved aren't).

It s possible to POP CS, although not specifically with that mnemonic. Simply PUSH the value onto the stack and execute an IRET, although this also pops IP and causes execution to continue at the target address.

Quote:

---

Originally Posted by abachler

It s possible to POP CS, although not specifically with that mnemonic. Simply PUSH the value onto the stack and execute an IRET, although this also pops IP and causes execution to continue at the target address.

---

You do not need to use IRET (in fact, that would probably not WORK), but a "far ret" would work fine, and "pop" CS. But the point was more that there is no instruction in x86 that restored CS on it's own from the stack (or for that matter "loads" CS from memory, or some such). You have to use one of the JUMP, CALL or RET instructions that take a CS:rIP pair - or use task-switching instructions).

As to DS and CS being the same, I was referring to x86. Most other processors do not have DS and CS register. Of course, there are at least some processors that have a harvard or pseudo-harvard architecture. A good way to avoid accidentally executing data!

Note also that my simple push, push, ret example is just ONE of many different ways that we could come up with a "stack-frame that points to a function". You most likely would have to completely follow the whole of the instruciton flow to know what's going on. A more complex example:

Code:

---

    mov ebp, esp
    push ebp
    mov eax, 0xFFFFFFFF
    xor  eax, ebp
    not eax
    sub eax, 4
    mov [eax], a
    sub eax, 4
    mov [eax], b
    xchg esp, eax
    ret
a:
    leave
    ret

...
b:
...

---

[I don't guarantee that the above code is actually CORRECT - but something along those lines WILL be able to execute correctly].

By the time you can follow such things, you have pretty much build a complete x86 instructin simulator. Which is a MAJOR task.

--
Mats

I'm still trying to wrap my head around that one! But you know, it also just goes to show that there really are a lot of paradigms that higher level languages could employ in implementing certain features. You just don't think of that much unless you've worked a lot with assembly, I guess.

tbh I've always doubted that I could make this, because if we can just watch a couple instructions to get safe full-speed execution without coming up with our own instruction set, someone would deffo have done it before me. I'm not the sharpest, unfortunately. ;)

However there only so many instructions that are used for flow control and specifically for changing CS and/or EIP, you could get to a point where you could at least protect against "accidental" bad addresses, even if the system could be defeated in a particular manner.

This question was effectively me asking myself (referring to you) "why" I am making the VM that I am making when I could to some extents cheat. The answer is that it's the only way to be sure (you can't mess with a VM when the native code only exists at runtime)

Sure, it comes down to "what are you preventing" - if you are trying to prevent "single diigt age" kids from eating the biscuits, or are you trying to prevent "professional" criminals from robbbing a bank, or professional spies from stealing military secrets.

The solution to the first is not a solution to the second or third threat (although the other way around probably is!)

And a common problem in secure computing, by the way - who can you trust, and what can you trust them with.

--
Mats

Yes, I was refering to the status opins ont eh x86, which cna be used to restrict memory accesses based on teh segment register used for the address. This effectively gives access to increased memory addressability, although afaik it was only used in some industrial computers back in teh 8086 days to extend memory to 4MB and to give multiprocessors access to a shared memory region through the ES register. This of course requires a processors without cache or paging enabled.