You must use delete [] instead of delete to get rid of the allocated memory since you're using new []. Come to think of that, Why not just use the vector since you've already stored all the data?
Code:
unsigned int value = *(reinterpret_cast<unsigned int*>(&fullVVuffer[0]));
Eliminates the copying-
And why one byte at a time? If you know how much it's going to be, why not read X bytes at a time? It may be a little trickier, but it will improve performance. If you don't know the size, read in chunks until you have no more to read.