First, please use the [.code] and [./code] tags. It makes everyone's life much easier.
Secondly, I'm going to re-write your program. See if you can make heads or tails of it.
Code:
#include <iostream>
int main ()
{
char answer;
int m, n, c; //variables for Pythagorean Theorem
std::cout << "Please enter a number.\n";
std::cin >> m; //enter the larger number
std::cout << "Now enter a number smaller than the first.\n";
std::cin >> n; //enter the smaller number
if(n >= m) //Check for correct input.
{
std::cout << "You entered incorrect input.\n";
return(0);
}
c = (m * m) + (n * n); //find the solution
std::cout << "Your answer is " << c << '\n';
std::cout << "Would you like to perform another problem? (y, n) ";
std::cin >> answer;
tolower(answer); // Convert your answer to lower-case.
if(answer == 'n') return(0);
else return(0);
}
Third, I'm going to ask exactly what you're trying to achieve here.