A battery keeps a potential difference via the electromotive force (EMF) which is usually just some sort of chemical that maintains the voltage across the terminals such that a current can flow. The actual potential is:
EMF - Ir where r is the bit of resistance innate to the battery. I asked a physics professor what actually happens when a battery dies, and he said that it is due to the internal resistance eventually skyrocketing, but he couldn't explain further. Does anybody know any more about what's actually going on with the internal resistance when a battery dies?
