Thanks for your reply Adak. I am now getting the idea. I changed loops to this.
Code:
for (p=2;p<100;p++)
{
for (q=2;q<100;q++)
{
n=p*q;
inv=(p-1)*(q-1);
for (e=2;e<100;e++)
{
do
{
s=(d*e)%inv;
d++;
}while(s!=1);
d=d-1;
for(j=0;j<strlen(m);j++)
{
M=1;
for(i=0;i<d;i++)
M=M*C[j]%n;
M=M%n;
M=M+96;
h[j]=M;
}
if((strcmp(h, words)) == 0) //words is the 2d array you mentioned.
printf("p is %d, q is %d, e is %d",p,q,e);
}
}
}
But still it doesn't work. After i enter a number to decrypt it just hangs there, nothing happens. any idea why? And also could you help me about using 2d arrays? How should i store words to it and check? Also performance isn't important right now, since if don't get this working until tomorrow i'll have serious problems with professor.