Thread: Unite Testing in c

I want to try versatile Criterion testing library. We have three files test.c and add.c and add.h . Compile with:

Code:

clang -lcriterion -Wall -Wextra -pedantic -std=c11 test.c add.c -o add

First begin by writing some tests:

test.c

Code:

#include <criterion/criterion.h>
#include <criterion/new/assert.h>
#include <limits.h>
#include "add.h"


TestSuite(suite1);


Test(suite1, low)
{
    int r1 = add(0, 0);
    cr_expect_eq(r1, 0);
    
    int r2 = add(0, 1);
    cr_expect_eq(r2, 1);
    
    int r3 = add(-1, 1);
    cr_expect_eq(r3, 0);
}


Test(suite1, random)
{
    int r1 = add(1429, 15317);
    cr_expect_eq(r1, 16746);
    
    int r2 = add(4760214, 8989342);
    cr_expect_eq(r2, 13749556);
}


Test(suite1, high)
{
    int r3 = add(INT_MAX, INT_MAX);
    cr_expect_eq(r3, 2 * INT_MAX);
}

add.c

Code:

#include "add.h"


int add(int x, int y)
{
    return -1000;
}

Good news is , all the tests are failing:

Code:

[----] test.c:32: Assertion Failed
[----]   
[----]   The expression (r3) == (2 * 2147483647) is false.
[----]   
[----] test.c:11: Assertion Failed
[----]   
[----]   The expression (r1) == (0) is false.
[----]   
[----] test.c:14: Assertion Failed
[----]   
[----]   The expression (r2) == (1) is false.
[----]   
[FAIL] suite1::high: (0.00s)
[----] test.c:17: Assertion Failed
[----]   
[----]   The expression (r3) == (0) is false.
[----]   
[----] test.c:23: Assertion Failed
[----]   
[----]   The expression (r1) == (16746) is false.
[----]   
[----] test.c:26: Assertion Failed
[----]   
[----]   The expression (r2) == (13749556) is false.
[----]   
[FAIL] suite1::low: (0.00s)
[FAIL] suite1::random: (0.00s)
[====] Synthesis: Tested: 3 | Passing: 0 | Failing: 3 | Crashing: 0

Now try to pass the tests by writing better add function:

add.c

Code:

#include "add.h"


int add(int x, int y)
{
    int r = x + y;
    return r;
}

Results:

Code:

[====] Synthesis: Tested: 3 | Passing: 3 | Failing: 0 | Crashing: 0

There seems to be a warning from compiler:

Code:

test.c:32:24: warning: overflow in expression; result is -2 with type 'int' [-Winteger-overflow]

You may try:
add.c

Code:

#include <stdint.h>
#include "add.h"


int64_t add(int32_t x, int32_t y)
{
    int64_t r = x + y;
    return r;
}

And modify rest of the code accordingly.

The notion of "unit testing" is about writing tests that apply specifically to a small unit.

As much as possible, your tests should be simple tests of functionality and should demonstrate how the function behaves in response to a particular condition. You should be able to describe the "particular condition" in a single short sentence: "Test how ADD works when ... one of the parameters is zero"; or "Test how ADD works when ... one of the parameters is negative".

You can be more specific if needed. For example, a DIVIDE function would behave differently when the numerator was zero than when the denominator was zero. So you might need two tests.

Unit tests are encouraged to be "white box" tests -- that is, you are encouraged to use knowledge of the implementation to design your tests. So if you know that the ADD function behaves differently based on the sign of the parameters, then you should write tests that explicitly target parameters with different signs. If you know that the ADD function has a special case for INT_MIN and INT_MAX, then you should write tests that supply those values as parameters. And so on.

Unit tests can be written to support very specific cases. In fact, you might find it helpful to write unit tests for a static function that is not visible outside the file. The right way to handle this scenario is to #include the C source file into the test file, so that the static function becomes part of the same translation unit and so is visible to your test functions.

Unit tests generally have three phases: setup, test, evaluate. During the setup phase, you do whatever is necessary to prepare for your test. For example, if your unit test was "... after a division-by-zero error occurs" then you would set up by forcing a divide-by-zero error, then do your test step.

The test phase is generally just calling your function, or calling some other function which you know will call your function. If you have an iterator that you are testing, you might test it by calling a "search the container" function that you know uses the iterator. Or you might construct an iterator, test it, and continue. It depends how complex the iterator is, and if the iterator exists for any purpose other than searching.

The evaluate phase can be as simple as comparing a result against an expected value: r = ADD(1, 2); assert(r == 3); or it might require checking a data structure. If you are testing an insertion-sort, you will want to assert that the container is in sorted order after a series of insertions. That can be a somewhat complex operation (iterate over the container, asserting that each element is greater or lesser than the prior one).

You should start by writing a single test case: test_ADD_happy_path. If you have different files for each function or subsystem, then you might remove "ADD" from the name since the test case is located in add.test.c (or test_add.c, but I like to use extensions since make knows how to parse them). The happy path is the "1 + 2 = 3" case, it shows how things work in a simple, no-tricks, no-errors scenario. That's the only happy case you write -- everything else is "special" relative to the happy path: there is some specific detail you are checking, like I mentioned above.

When you find bugs in your code after releasing it, add a special test case for that specific error. It is nice if the test case can be a unit test, but the really tricky bugs might have to be system tests or integration tests, if multiple components are involved.

Code:

void
test_ADD_works_after_borrow_issue_1728()
{
    // Per issue #1728, on Sperry Unisys model 3 computers,
    // when ADD is run after a SUBTRACT that set the borrow
    // flag, it can add-with-carry and be off by one.

    // setup
    int r1 = SUBTRACT(1, 2); // expect -1, set borrow flag

    // test
    int r2 = ADD(1, 2); // expect 3, add-no-carry

    // evaluate
    assert(r1 == -1, "subtract should borrow");
    assert(r2 == 3, "add should not carry-in");
}

If you are contributing to someone else's project, providing a test case to demonstrate the problem is way better than just filing a bug report. In general, you want to:

• Search the bugs database in case your issue has been found already.
• File a bug report with details on your system and steps to reproduce.
• Submit a test case that reproduces the bug.
• Submit a code change that makes the test case stop failing.

You will be more loved the further you go along that series. Thrashing the disk on the server while searching the bugs tells the project owner that someone is looking at their project. Submitting a bug report tells them someone is using it. Submitting a test case tells them that someone is invested in using it, and they are not alone. Submitting a fix tells them to sit back and have a beer, other people are helping.

You will find the same thing applies to you, when others have trouble with your software.

Originally Posted by Salem

First you would separate functions out so they are testable to begin with.
A ball-of-mud main.c with everything in it doesn't lend itself to being tested.

In this tiny example, you compile the proper code with
gcc main.c add.c

And you compile and run the tests with
gcc add_test.c add.c && ./a.out

If you get no messages, your tests were successful.

I ran your code and got error because you are using main keyword in two source file

Code:

 C:\Users\Kittu\Desktop\C Example>gcc -o hello main.c add.c add_test.cC:\Users\Kittu\AppData\Local\Temp\cciAVqxV.o:add_test.c:(.text+0x0): multiple definition of `main'
C:\Users\Kittu\AppData\Local\Temp\ccymyc0i.o:main.c:(.text+0x0): first defined here
collect2.exe: error: ld returned 1 exit status

In a what case i should get fail message because i already change main name and set different value b = 3 I get result 13

Originally Posted by Salem

First you would separate functions out so

In this tiny example, you compile the proper code with

gcc main.c add.c

When I ran the command gcc main.c add.c I received an error message

Code:

gcc: error: main.c: No such file or directory

even i tried gcc -o main.c add.c

Code:

 c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../libmingw32.a(main.o):(.text.startup+0xa0): undefined reference to `WinMain@16'
collect2.exe: error: ld returned 1 exit status

Originally Posted by Salem

And you compile and run the tests with
gcc add_test.c add.c && ./a.out

If you get no messages, your tests were successful.

When I ran the command gcc -o hello add_test.c add.c I don't get any message.

Originally Posted by Salem

> even i tried gcc -o main.c add.c
Nice, you just trashed your source file with a mis-guided -o option.

> When I ran the command gcc main.c add.c I received an error message
Right, and I'm supposed to believe you managed to read my post and create the 4 necessary files?

I apologize for the confusion and the errors I encountered. Thank you for your patience


I use the following commands to compile and run the main code:


gcc -o main main.c add.c
main


I use the following commands to compile and run the test code:
gcc -o add_test add_test.c add.c
add_test


After running the tests, I check if there are any error messages. If there are none, it indicates that the tests passed successfully.


To intentionally introduce an error into the code provided, I modify the ADD() function in add.c to return an incorrect result.

Code:

// add.c
#include "add.h"
int ADD ( int x, int y)
{
    int r = x - y;
    return r;
}

I got result Test ADD() failed: Incorrect result when i compile and ran code