First, please use the [.code] and [./code] tags. It makes everyone's life much easier.
Secondly, I'm going to re-write your program. See if you can make heads or tails of it.
Third, I'm going to ask exactly what you're trying to achieve here.
int main ()
int m, n, c; //variables for Pythagorean Theorem
std::cout << "Please enter a number.\n";
std::cin >> m; //enter the larger number
std::cout << "Now enter a number smaller than the first.\n";
std::cin >> n; //enter the smaller number
if(n >= m) //Check for correct input.
std::cout << "You entered incorrect input.\n";
c = (m * m) + (n * n); //find the solution
std::cout << "Your answer is " << c << '\n';
std::cout << "Would you like to perform another problem? (y, n) ";
std::cin >> answer;
tolower(answer); // Convert your answer to lower-case.
if(answer == 'n') return(0);