Printable View
I've attached my poor, poor layman's excuse for a tokeniser. I've got no idea where this heap problem it coming from, except that visual studio pops up a heap corruption-type dialog box when I step into "delete t;" (in main.cpp).
After looking at the output window (for more information) I've worked out that I'm overwriting the source buffer somewhere (the displayed size grows and shrinks as I add / remove from test.txt).
NOTE: Attachment is a .zip masquerading as a .txt.
Well there's an awful lot of assuming that there is a \0 at the end of your char* array, and there isn't.
You need to allocate n+1 bytes to store n chars and a \0
Many things... First, you compare in some places again '0' and '9', yet when you compare for letters you use raw integer codes, I assume ASCII. Why not use the standard functions for this, isalpha(), isdigit(), isspace(), etc?
Your check for decimal point is nonfunctional, because it is inside an if-condition that could only be true if it was a DIGIT, not a decimal point. Same goes for minus sign. It's as good as saying if(0)...
The entire condition on the immediately following else is pointless, since it is just the negation of the first one.
You never check for null bytes in any of the token recognizers. So you go way off into neverland most of the time.
You modify the string which is being parsed. with that replace_xxxx() function. Since we've already established that you are going off into bad memory, this means you are trashing the heap.
The biggest issue is the checking of null bytes. You only ever do this in the main tokenizer loop. You need to be checking EVERYWHERE you loop. Including after the whitespace-skipping loop inside the main tokenizer loop. And don't modify the input if you can help it -- why are you doing that anyway?
Whoa. I really, really need to brush up on C string processing, eh?
That replace_XXXX function was a quick knock-up because the debugger was reporting some mess at the end of the string. Looking at your comments I imagine this has something to do with my forgetting a terminating NULL, or not checking for it / assuming it's there.
Thanks for the comments guys - I'll be re-tutoring myself on strings over the weekend :)
EDIT: Oh and I think it's worth me going over conditionals.