Lesshardtofind, you are talking like a hardware designer! (Not a bad thing)
Memory usage and performance are inextricably linked through the CPU cache. The more data you can fit in cache, the fewer cache misses per datum will be incurred. However, the processor might be faster when working with some data types rather than others. The tradeoff may be different from processor to processor. And the data access pattern matters as well -- the CPU/cache may be efficient at streaming access but worse at random access.
Let actual requirements drive your decisions, not some vague need to be "more efficient." Decide how much memory you need the thing to use and how fast it has to work, then figure out how to satisfy those requirements.
