I'm attempting to write some code that will extract certain tokens from a string of variable length. The code I have now is long and messy (loaded with switch statements, if/thens, etc), and I'm sure there's a more elegant solution. For the sake of brevity, say this is the list of valid tokens:

Code:
A
AZ
@
:...:        //String between colons to be read as a string and not tokenized
{0-9}    //Any single-digit integer)
So, valid strings could be:

Code:
A@5AAZ
@01:Hello, world!:@A
AAAZ:AZ:A1
@7@7AZ
For the most part, the strings will be >255 characters in length, and read from a plain text file. The number of valid tokens is more likeley to be several dozen. Some tokens will be followed by a string literal. The code also needs to be able to ignore anything that isn't a valid token. Order counts.

Are there any tricks to make this easier? Or do I really need to brute force my way through the string? As it stands now, my code is pages of this:

Code:
switch(input_char)
{
case 'A':
    if(next_char == 'Z')
    {
        //do stuff
    }
    else
    {
        //do some other stuff
    }
    break;
//etc.
}
It's functional, but it's pretty ugly. And anytime I need to add/change a token, it's not easy. Any help would be greatly appreciated.