When you're working on lots of them in memory (e.g. heightmap) then the bottleneck will be the memory IO, so the smaller size is faster.
....

Internal bus data widths have long surpassed the available default data type data widths. Like in um...the 80286.