Search:

Your matrix multiplication code doesn't appear correct.
Notice that the multiplication of a matrix and its transpose is a square matrix.

Nice going with the multiplication.
Care to show your code?

The right part of the code to include a call to the exit() function is inside the input loop.
You need an if there too.

The function exit() has its prototype in <stdlib.h>. You need to include that system header.

And you might want to review how to multiply matrixes (Wikipedia has a nice article) rather than...