Hello and happy holidays!
It has been a few weeks since my last post, but rest assured, I have been busy (I have been focused on a lot of coding, but have been too lazy to write a new blog post).
The next path on this journey leads us to the: unit testing infrastructure.
As I previously mentioned, I decided to use the testing framework Ceedling.
Throughout this post, I will explain what unit testing is, the benefits of unit testing as well as how its implemented in STORfs.
What is unit testing?
The following quote from this article (which serves as a solid introduction to unit testing) explains the concept of unit testing:
A unit test is a block of code that verifies the accuracy of a smaller, isolated block of application code, typically a function or method. The unit test is designed to check that the block of code runs as expected, according to the developer’s theoretical logic behind it. The unit test is only capable of interacting with the block of code via inputs and captured asserted (true or false) output.
This testing allows precise coverage of a module (file/class) to provide greater confidence in the code being integrated into the project, also called the SUT or System Under Test.
When testing a module it is important to validate a few metrics:
- Functional output, does the unit of code perform as expected
- Code coverage
- Branching conditions
With an emphasis on functional output, which is a qualitative metric. Ensuring the behavior of the function actually performs up to the expectations of the developer is the main reason to test! But many developers fall in the trap of trying to chase the other two metrics.
Code coverage and branching conditions are easy to quantify as percentages:
- Code coverage equates to the lines of code that have been tested
- Branching conditions determines if all logic control statements (
if, else if, else) has been exercised
if (x == 10 || y == 11) has 3 branching conditions that can make this control statement true or false:
- If x is 10, this is true, y is not considered
- If x is not 10 but y is 11, this is true
- if x is not 10 and y is not 11, this is false
Usually the more branching conditionals a function has the more complex it is.
Reducing that function’s complexity makes the code easier to test. This can be done by extracting some of the inner logic into smaller functions (i.e. an extraction refactor).
How is unit testing performed?
Usually a framework such as Ceedling makes unit testing easier to enroll into a code base.
Testing is done to validate the output of a function in accordance to provided inputs.
As a simple example consider the following C function:
int sum(int x, int y) {
return x + y;
}
A unit test for this function would look as so:
void test_sum(void) {
assert(sum(2, 2) == 4);
assert(sum(5, 10) != 200);
}
Code can also be mocked, stubbed or faked in a unit test (there is also spies and dummies but I believe keeping unit testing simple with the three aforementioned methods are the way to go).
Mocking involves setting expectations for a function to be invoked in the SUT, and returning a value from that function.
This validates the behavior of the SUT.
The following C example shows how a mock is performed:
int system_self_test(void) {
if (check_sensors() == false) {
return 1;
}
if (check_actuators() == false) {
return 1;
}
return 0;
}
The SUT, system_self_test invokes two functions:
check_sensorscheck_actuators
The unit test with mocked functions would look like the following:
void test_system_self_test(void) {
mock_expect_and_return_check_sensors(true);
mock_expect_and_return_check_actuators(false);
assert(system_self_test() == 1);
}
The mock function calls tell the unit testing framework to expect a function to be invoked within the test and returns a value, true or false in this example.
Stubbing is similar to mocking but does not hold the functions invoked in the SUT to any expectations. This provides simply a canned response. So the SUT in the previous example could look as so:
void test_system_self_test(void) {
stub_return_check_sensors(false);
stub_return_check_actuators(false); // Don't care if this is called
assert(system_self_test() == 1);
}
Because check_sensors returns false in the SUT, check_actuators will not be invoked, but the unit test will still pass because no expectation is passed onto check_actuators. If this were the previous mock example where check_actuators is held to an expectation of being invoked, the unit test would fail!
Fakes are a used to implement a light weight version of a function that is implemented in the SUT. This gives the developer tighter control over what the function might return and is generally useful for controlling state driven decisions. An example of of a fake for the example we have been working with might look something as so:
bool check_sensors(void) {
return true;
}
bool check_actuators(void) {
return false;
}
void test_system_self_test(void) {
assert(system_self_test() == 1);
}
Here the values of the fake functions are static and it would be more simple to mock or stub. But, with more complex functions which are state driven or need to perform some logic on a set of data, fakes can be incredibly useful.
Why unit tests for STORfs?
The refactor I completed in my last post broke storfs.c into smaller modules.
This modularization is perfect for unit testing because:
- Each file has a focused responsibility
- Functions can be tested in isolation
- Edge cases (corruption, full disk) are easier to simulate
How are unit tests implemented on STORfs?
As previously mentioned, the Ceedling framework is being used for unit testing this library.
I decided on Ceedling because:
- It targets the C language
- It is easy to setup
- Testing API is not incredibly complex
- There are awesome plugins
- Not much is necessary when adding new files to the project
- Originally, ceedling was targeted towards embedded systems (although now just used as a generic C unit testing framework)
- Easy to fake flash operations
In order to setup Ceedling and ensure consistency between multiple developer machines, I created an Ubuntu based Docker container to host all of the packages/tooling necessary.
If further interested in the contents of the Dockerfile, it is located here.
I also create a Makefile used to create the docker image as well as to run the unit tests. This setup makes it pretty easy to run the unit tests. The only tool requirements for a develop to have installed for this setup are Docker and GNU Make. In order to run the unit tests only two commands are necessary to be evoked from the command line:
make init only has to be invoked once to set up the container and whenever an update is made to the Dockerfile, then make is used following that to invoke the unit tests!
Within the test directory of the project, files prepended with fake_ will be used for fakes, files prepended with tests_ are used to test specific files in the src directory, mocks and stubs are auto-generated by Ceedling when running the tests!
For the work covered in this post, I have only created a fake_flash file which mimics data storage from the filesystem in the local machines RAM.
The following PR#6 implements the work described above!
Until next time!
.
).
