Like Tree2Likes

Functional testing

This is a discussion on Functional testing within the Tech Board forums, part of the Community Boards category; Hi guys, just a quick question. How do you usually make a functional testing, ie. checking a function to work ...

  1. #1
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476

    Functional testing

    Hi guys, just a quick question. How do you usually make a functional testing, ie. checking a function to work as expected? What are the steps? Thanks.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  2. #2
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,239
    How would you tell if a car works?
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  3. #3
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by brewbuck View Post
    How would you tell if a car works?
    Well, I guess I would try driving it around. If I manage to get somewhere and home again, it obviously works fine, right?

    @g4j31a5: Come up with various inputs (ie, the function parameters) and do your best to decide what the function should output for these, without actually using the function. Then feed the inputs to the function and check if the output is what you think it should be. Each set of inputs, expected output, and actual output is a "test case". The mechanism you use to feed these to the function and record the results is a "test harness". Depending on the nature of the function, it may be easy to come up with a list of hundreds of test cases; if not, try to use at least 5-10. These should mostly be valid input (and cover the range of what "valid" input would be), but 20% or so should be invalid inputs, so you can test if the function can handle erroneous data nicely -- eg, it doesn't just seg fault, it reports an error, and the error is correct. Note that "report an error" does not mean a printf/cout statement. It is better to use the return value, or to throw an exception, etc. This way, the caller can decide how to deal with the problem.

    Testing of that sort is more akin to getting your car tuned and checked in a shop . You may want to keep your test cases around so that if you change something inside the function, you can easily double check that you didn't break anything.
    Last edited by MK27; 03-19-2012 at 11:27 AM.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  4. #4
    Registered User
    Join Date
    Sep 2007
    Posts
    131
    It may depend on what the function does. For most functions, it's fairly obvious what the results should be. For some, it's not.

    For my geometry utilities, I used pristine test cases (geometries easily figured by hand) to compare results. Then moved on to concave, but still easily calculated geometries, such as pipes. My 3D modeler can calculate the area of a any number of primitives, so so for the pipe, I would find the area of an end cap and multiply it by the length. Simple to compare, but wasn't so simple to fix.

    Then there is error reporting. Situations that I know should not happen or can happen and would screw up the findings. I dump numbers to a file for viewing later on to see what case was.

  5. #5
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    21,711
    Quote Originally Posted by g4j31a5
    How do you usually make a functional testing, ie. checking a function to work as expected?
    Quote Originally Posted by MK27
    Come up with various inputs (ie, the function parameters) and do your best to decide what the function should output for these, without actually using the function.
    Quote Originally Posted by Cynic
    It may depend on what the function does. For most functions, it's fairly obvious what the results should be.
    Heh, I had the impression that "functional testing" refers to testing of functionality (whether the program functions as designed), whereas testing of individual functions (as units of code) is "unit testing".
    C + C++ Compiler: MinGW port of GCC
    Version Control System: Bazaar

    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  6. #6
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    4,246
    Maybe I'm reading too much, or little, into what brewbuck said, but I think he has the right of it.

    For most code you'll never prove that it is correct... only that it seems to behave correctly.

    Soma

  7. #7
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    21,711
    Quote Originally Posted by phantomotap
    For most code you'll never prove that it is correct... only that it seems to behave correctly.
    This reminds me of the quote attributed to Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it."
    C + C++ Compiler: MinGW port of GCC
    Version Control System: Bazaar

    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  8. #8
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by phantomotap View Post
    For most code you'll never prove that it is correct... only that it seems to behave correctly.
    True, but testing is not about correctness proving, it is about checking to see if something seems to behave correctly .

    Also worth noting: proving something correct does not actually mean it will then seem to behave correctly, if you have misconceived the application of what you have proven correct.

    I don't test what I write as much as I should (because I enjoy debugging later so much) but I did read somewhere that "studies have shown" that the number of bugs found in say 10 poorly tested units over the next 6 months will be much higher than those found in 10 similar but well tested units, even if the exact same people wrote all the code -- regardless of whether anything is "proven correct" in the process.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  9. #9
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    4,246
    True, but testing is not about correctness proving, it is about checking to see if something seems to behave correctly.
    *shrug*

    I view this as being the differences between "Correct" and "Accurate".

    Granted, they are my definitions, but that really is almost exactly how I use "Correct" and "Accurate".

    And just in case it comes up: "Consistent" being "If it fails, for pities sake make it fail the same way every time.".

    Also worth noting: proving something correct does not actually mean it will then seem to behave correctly, if you have misconceived the application of what you have proven correct.
    I really don't think your mind could conceive of a truer statement. ^_~

    Soma

  10. #10
    Registered User
    Join Date
    Jun 2005
    Posts
    6,284
    Quote Originally Posted by MK27 View Post
    Also worth noting: proving something correct does not actually mean it will then seem to behave correctly, if you have misconceived the application of what you have proven correct.
    True. Generally speaking, testing can only prove lack of correctness and cannot prove correctness. An error is detected in testing is positive proof of the existence of an error. However, the converse is not true: the absence of an error in test can (and, in practice, usually does) just mean the test cases were incomplete and did not exercise faulty code.

    Proving completeness of a set of test cases is challenging, particularly for non-trivial code.
    memcpy likes this.
    Right 98% of the time, and don't care about the other 3%.

  11. #11
    Registered User
    Join Date
    Dec 2011
    Posts
    795
    I would have to agree with Grumpy.

    On a similarly related note, does anyone know of a better way to test an assembly-language function I wrote? It is designed to be a "left-trim string" function, taking a string and an offset, and trimming the string from the left that many characters, then copying it into an output string.

    For example, if the offset was 3 and the input was
    "asdf asdf asdf"
    "af4tr3cf sf gffff"

    it would produce:
    "f asdf asdf"
    "tr3cf sf gffff"


    I did the following tests, and they all worked:

    • Zero offset (just copied the string)
    • Null string (made output = null)
    • One line (trimmed normally)
    • Offset longer than line length (works, any lines shorter than offset were removed during trimming)
    • Offset equal to line length (leaves lines, works correctly)
    • And a couple more tests of standard strings


    Does anyone know of any more "complete" tests I could do?

  12. #12
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,480
    @memcpy
    I don't see where "lines" come into this, it's a string trimming/copy function. What is your "one line" test? What are your "couple more tests of standard strings"?

    Most of the tests you did are corner/edge cases. Those usually check bizarre stuff that might break your code outright, i.e. how it performs under abnormal usage, like a caller passing in NULL pointers. The only other possible corner case I can think of would be an empty string, "". It probably falls under the zero offset or offset longer than line length cases, but it is the type of input that is commonly missed in designing and/or testing the algo, and the kind of input that could easily break an algo.

    Also, you want to test some non-corner/edge cases. Perhaps these are the one line and standard strings tests, but I'm not sure. They should be run to verify that your code works as expected under normal operation, such as the examples you gave.

    Lastly, since it's assembly, how are you treating the offset parameter? If it's signed, you need to check what happens if you pass in negative values or you could redefine it to take an unsigned number and use unsigned operations when computing offset.
    memcpy likes this.

  13. #13
    In the Land of Diddly-Doo g4j31a5's Avatar
    Join Date
    Jul 2006
    Posts
    476
    Wow, thanks for all the replies. And yeah, my bad. laserlight was right, it should've been called unit testing instead of functional testing. So in a way, testing should consists of these:
    1. Test cases for inputs.
    2. Predetermined values to be compared with the output which are calculated manually by the tester.
    3. The error reporting.

    Did I miss anything else?

    I actually got a few tests which required me to make a program that checks that a particular function works as intended. For example:
    Write a program to check that std::ldexp(x, exp) == x * 2^exp across ldexp() input domain.

    Considering that question, what I should do is this:
    1. Wrote a list of test cases (eg. 1: x = 1, exp = 2; 2: x=-25, exp = -3; 3: x = 11, exp = 0; 4: x = -25, exp = 0; etc...)
    2. Do an "assert(std::ldexp(x, exp) == (x * (2^exp)));" in a loop for each of the test cases's values.

    Is this right? Also, another question. How do you make a list of good test cases?

    Thanks.
    ERROR: Brain not found. Please insert a new brain!

    “Do nothing which is of no use.” - Miyamoto Musashi.

  14. #14
    Registered User
    Join Date
    Sep 2007
    Posts
    131
    Since x is a floating point number, you may want to test against values 0 < x < 1 as well. In fact, you may want to expand that to -1 < x < 1.

  15. #15
    Registered User
    Join Date
    Sep 2007
    Posts
    131
    Quote Originally Posted by laserlight View Post
    Heh, I had the impression that "functional testing" refers to testing of functionality (whether the program functions as designed), whereas testing of individual functions (as units of code) is "unit testing".
    I'm not so hardline.

    ie. checking a function to work as expected?
    I read this and knew what he meant, even if his vernacular was off.

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. My functional syllogism program
    By jeremy duncan in forum C Programming
    Replies: 3
    Last Post: 12-15-2011, 12:23 PM
  2. functional derivation better?
    By CodeMonkey in forum C++ Programming
    Replies: 3
    Last Post: 08-16-2007, 12:57 AM
  3. Functional programming languages... r they really functional?
    By code_mutant in forum C++ Programming
    Replies: 10
    Last Post: 02-25-2004, 04:29 AM
  4. functional dependencies
    By DMaxJ in forum C++ Programming
    Replies: 10
    Last Post: 10-23-2002, 07:07 AM
  5. Problem with non-functional window
    By Magos in forum Windows Programming
    Replies: 5
    Last Post: 04-14-2002, 11:32 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21