Virtual memory in user space (Operting Systems)

[Elysia]

Question:
I was wondering if it's actually possible to put all the memory management of an OS into the user space?

Thoughts:
I suppose that all traps must go through the kernel, but nothing would stop the kernel from calling up to user space, right?
The question popped up in my head when reading about a user-mode paging system which, according to the book, wouldn't have access to the R and M (referenced and modified). But I can't quite grasp the why of it.
I mean, sure, if the paging table is in the kernel space, then it wouldn't be able to access it, but if it were in user space, there would be no problems. And as far as I know, there is no hardware that can keep track of all the R and M bits of all pages.

Discussions are welcome!
Operating systems are fascinating.

[phantomotap]

I was wondering if it's actually possible to put all the memory management of an OS into the user space?

Yes, it could be done, but it would be a bad idea. (If only because a simple bug in user level code could trash the maps and everything else.)

*shrug*

You could always setup a multiple stage process though where... oh, wait, we already do that pretty much everywhere.

Soma

[Elysia]

A bad idea?
The alternative, kernel space, could potentially grind the OS to a halt, anyway, so I don't see why it would be a bad idea?

[phantomotap]

The alternative, kernel space, could potentially grind the OS to a halt, anyway, so I don't see why it would be a bad idea?

What?

Soma

[Elysia]

Rephrasing:
Meaning that a bug in the kernel space can cause a BSOD.
A bug in user space can crash the process, but not cause a BSOD.
So why is it put to put it in user space?

[phantomotap]

A bug in user space can crash the process, but not cause a BSOD.

Well, as we've talked about before, it absolutely can in any event because user code can misuse and abuse kernel routines, but that's not strictly relevant. The relevant bit here is that you are talking about putting crucial operating system machinery into user space meaning that user space problems can cause problems directly--without even needing to go the abusing a kernel space routine route.

Soma

[Elysia]

I see. Well, I suppose that since virtual memory is the lowest gear in the OS, it does not really benefit from user space as much as other things might.
Is it really such a bad idea, though? I can certainly not see much benefit, but not much disadvantage, either. What is the worst that could happen if it were in user space instead of kernel space?

[bithub]

Rephrasing:
Meaning that a bug in the kernel space can cause a BSOD.
A bug in user space can crash the process, but not cause a BSOD.
So why is it put to put it in user space?

The reason that a bug in user space only crashes the process is because of the fact that memory management is done in kernel space. If all memory management was done in user space, a single bad application could trash the entire system.

Other reasons memory management is done in kernel space?
- Page sharing. Two processes that load the same library can map in the same pages which saves on total memory usage.
- Kernel memory. The kernel needs memory too, but it doesn't run in user space.
- Security. If any process can read the memory contents of another process, this presents security concerns.
- Disk caching.

[Elysia]

The reason that a bug in user space only crashes the process is because of the fact that memory management is done in kernel space. If all memory management was done in user space, a single bad application could trash the entire system.

Well, I'm not sure I buy this.
We agree that the memory manage must not contain bugs. If it crashes, it brings down the system with it, whether in user space or kernel space.

- Page sharing. Two processes that load the same library can map in the same pages which saves on total memory usage.

This can be done in user space too, AFAIK. I don't see why not?

- Kernel memory. The kernel needs memory too, but it doesn't run in user space.

This sounds like the biggest problem. Unless some upcalls are possible, but they're typically considered a bad idea... hmmm.

- Security. If any process can read the memory contents of another process, this presents security concerns.

The OS controls the processes, so it could deny all read and writes to any portion of the memory manager's memory.
Besides, all other processes use virtual memory while the memory manage deals with physical memory.
I don't see the security issue there, really.

- Disk caching.

Why is this not possible in user mode?

Meh. Maybe I should be asking if there is any good reason for doing memory management fully in user space.
At least some of it can be done in user space.

[whiteflags]

Why is this not possible in user mode?

In order to actually do disk caching, you would need to write what you want there and restore it to RAM later. I'm pretty sure it isn't much more complicated than that but it's all done by a device driver. How or why would you want to run that in userland, too?

[zacs7]

Perhaps you should have just asked "possible to get rid of the 'lands'?"... Seriously, why?

Oh yes, we want to be doing that, it was such a good idea 20 years ago.

[grumpy]

Perhaps you should have just asked "possible to get rid of the 'lands'?"... Seriously, why?

The usual justifications are either performance (minimising overhead) or making life easier for the programmer (enabling simpler methods rather than complex APIs with their pesky error checking). Problem is, those benefits usually come at the expense of reliability - programmers have demonstrated convincingly that they cannot be trusted to ensure system integrity if they have access to system resources from userland code.

Oh yes, we want to be doing that, it was such a good idea 20 years ago.

Indeed. In the days of MS-DOS and early versions of windows that ran on top of DOS (up to windows version 3.11 IIRC) all memory on the machine was accessible from user-mode code. One notable characteristic of those systems is that they were unable to protect themselves from actions of running programs. Quite a few programs were written, accidentally or deliberately, that could easily crash the system, overwrite memory being used by other programs, and also do things like reformat hard drives without confirmation.

Protected mode processors, memory management units, kernel mode, etc were designed to prevent user-land processes from having unfettered access to system resources that might be used by other processes (whether physical or virtual). One of the arts of modern operating system design is selecting what actions are permitted in "user mode" versus those that are permitted in "kernel mode".

So, in answer to the original question, yes it would be possible to design an operating system that places system memory management into user space. That would run completely at odds with the way things have developed over the last 20 years or so (the trends have been to increasingly prevent user programs from managing system memory). It would probably be easier to tailor your own operating system, rather than attempting to modify a modern operating system to allow it, as most modern operating systems are specifically designed to do the opposite.

[Elysia]

Don't misunderstand my question here.
Virtual memory is a critical part of the operating system that should not be removed.
However, the question is really: is it possible to put the code managing the virtual memory for all processes (and possible the OS, too) in user space?
A paging service that determines which pages to evict from memory and possibly which ones to page back in might run in user space, for example.
Drivers can run in user space, too. Threads can also run in user space, all the while not compromising OS security.
The applications know nothing of how threads and processes are scheduled, and how memory is managed, nor should they.

But can the integral OS parts run in user space? If they can, usually there are good reasons for doing so (a mix of performance and safety).
For example, if a driver crashes in kernel space, it brings down the OS. It if crashes in user space, it can just be restarted and the OS continues to run normally. And if the drivers do not need to trap to the kernel, it will also be faster.
(There is a reason why Microsoft put the graphics driver in user space for Vista.)

[whiteflags]

Drivers can run in user space, too.

But can the integral OS parts run in user space?

For example, if a driver crashes in kernel space, it brings down the OS. It if crashes in user space, it can just be restarted and the OS continues to run normally. And if the drivers do not need to trap to the kernel, it will also be faster.
(There is a reason why Microsoft put the graphics driver in user space for Vista.)

Relating this to my post, you're saying a disk driver could and should run in userland. That means that the computer can't do anything with the drive until the OS boots. You'd have to write an OS that can't have anything to do at start up and shutdown with said disk. We do have operating systems that don't use the hard drive at those times (see LiveCDs) but I don't think you can bootstrap an OS without something being accessible to the kernel first. Your boot loader needs to be able to access storage device foo to start OS bar. Conversely, you can do bootstrapping without graphics working.

[Elysia]

Obviously I haven't been tasked with writing an OS, so I don't know real world implications of things such as the boot.
I have seen a design where basically the pager is in user space and the kernel simply notifies the pager process to do disk I/O for it.
It could simply map some page into the pager memory and then unmap if when the disk I/O is complete.
Wouldn't a similar approach for a disk driver be possible?