I am working on a program that will be compiling user-sent C/C++ source code and running the resulting executable on a server. The source code is expected to solve an exact task, but obviously I can't control what the user sends, so this is where my security concerns come in.
First of all, I can just remove the <windows.h> <psapi.h> etc headers so the user will just get a compile error if he sends something that tries to spawn processes or delete all files and stuff like that. But, is there a way I can stop the system() function? How about just opening files for writing with fopen(filepath, "w");? This still makes it possible for the user to delete files, albeit only if he knows the actual path (with fopen anyway, system would still be a problem).
Would running as a limited user stop programs that try to run delete commands on system files, for example? What about important non-system files? Can I somehow run the user exe in a secure environment where he can only open the files I allow him to and he's not allowed to run any system commands? What do you think would be the best course of action security-wise?