I didn't look at all the code. But this:
Code:
str = malloc(strlen(enc) - 2);
strncpy(str, (enc + 2), (strlen(enc) - 2));
for (i=0; i<strlen(str); i++)
is wrong.
strncpy() will not necessarily create a string. If there is no null byte in the first n bytes of the source, where n is the third argument, no null byte will be placed in the target. Without a null byte, you haven't got a string. Thus calling strlen(str) is invalid.
You'll want to allocate one extra byte, and then manually place the null character.
Incidentally, string literals already contain a null character, so unless you want two at the end of your strings, you needn't add one yourself.