Because this is fundamental dynamic object creation. Without 'new' and 'placement new' STL could not do its job. C++ gives you a wide variety of tools and it is your responsibility to use them properly. STL is just an "additional" implementation, which can be replaced with a different one if you want.
For example, in Java, you cannot implement your own String class, which would work in the same way. Why? Because, for example, you cannot overload the + operator. Java creators "take care" of the programmers disallowing them to overload operators, because from their point of view operator overloading can be overused (and lead to bad designs or something), but they do it themselves - they CHEAT.
In C++ you can create your std::string, your own vectors, your own STL. C++ goes even farther - it enables you to overload the new operator.
If it cannot be copied it will not compile (if you attempt to copy it). If copies are expensive... they are expensive, what do you want to know more? You can use smart pointer and copy only the pointer, which is another thing that can be added to the language by 3rd party, unlike in java where you must stick to its references. And yes, smart pointers also make use of new/delete.Passing by value means you make a copy of the object. What if it is of a type that can not be copied or copies are expensive?