Just because I don't own my neighbor's house doesn't mean I can't go inside it. I just run the risk of getting caught and thrown in jail. But 9 times out of 10, I'd probably get lucky.

Same deal. You're accessing memory you shouldn't. The sheriff might come along any second now and toss you in the clink. Or not.