The new operator allocates objects on the heap (1) which exists as long as you don't call delete, while a normal object is allocated on the stack (2), which is destructed on scope exit.
Code:
int* foo() //(2)
{
int x = 100;
return &x; //Undefined behaviour, as x does not exist after the function exits
}
int* foo()//(1)
{
int * x = new int(100);
return x;//Okay, but you *must* call delete after you are done with the return value.
}
Also, stack space is very limited on normal computers, while the heap has all (not really, read up on virtual memory if you want details) the free memory in your RAM.