The Java VM could dynamically optimize the heavely used portions. For example, the Java VM could optimize a section of code to perform fast with respect to one program configuation, then reoptimize the code. I don't see any reason why a C++ compiler with priofiling information couldn't do this, but the Java VM ought to be able to outperform most C++ compilers.
Java VMs have a JIT compiler which converts java bytecode to machine code. The machine code for frequently used parts of the application are saved so that it doesn't have to be re-interpreted. So if you have a java program in which 95% of the CPU usage is done in a small loop, that loop wont be re-interpreted each iteration. In that case, the java program will run just as fast as its C/C++ equivalent.