The best time to start optimizing your code is when you no longer plan to use it.
Make no mistake. And I'm getting ready for a very harsh week because of that. There's really no level of optimization that we should apply beyond that which we do at the Design phase. After that, upgrading processors or increasing RAM will always become a cheaper alternatives to compromising your code maintainability. Don't do it. Remember, code maintainability is your only goal in computer programming. It's the only thing that makes it possible for you to code for reusability, or scalability, or easy of use, or whatever else are your requirements at the time.
If this bothers you... next time, Design better.