−Table of ContentsUnit-testing (embedded) C applications with CeedlingJust like a lot of other embedded software engineers, I used to ship my embedded applications to production without testing them properly. Only some manual tests were done. I was under the impression that there's no real way to test them: you know, embedded applications run in a custom hardware and interact with this specific hardware heavily, which makes them not so easy to test automatically.
But the more I worked in this field, the more I thought that there should be some way to make my applications more reliable. Of course, I strive to develop right application design and write good implementation, but mistakes (sometimes very silly ones) happen. And one day, I've come across the awesome book: Test Driven Development for Embedded C by James W. Grenning. It is a very good book, and it explains the topic very thoroughly; highly recommended. Although this article is based heavily on what I've learned from this book, it is not just a very short version of it: while trying to put my knowledge to practice, I found some new tools that simplify the process even more. I hope that the article will help you to start quickly. What we will testEmbedded development differs from other software engineering fields in that embedded application has to interact with the hardware, and the hardware might be very different from application to application. Unfortunately, I've seen a great deal of code which performs some application logic while interacting with the hardware directly. It is sometimes done in the name of efficiency (on the very low-end chips, even function calls might be considered as expensive), but eventually it might lead to one's habit, which, of course, is to be avoided. The main idea (which is very good in itself, not only for unit testing) is to separate the hardware interaction and the application logic as much as possible. Then, we end up with a bunch of separate modules, which can be tested outside of the hardware. And when we write our application with the testability in mind, we literally have to separate things. So, one “side effect” of writing a testable code is the modularity, which is a good thing to have. Testable code is all good! So, in this article, we will not test the hardware layer. It has to be as thin as possible, and I still test it manually. What we will test is everything else: the modules that use hardware layer, and the modules that use other modules. While there are techniques to run tests on the target hardware (and the book by James W. Grenning touches upon them, among other things), this article focuses on testing on the host machine instead. And it's probably worth mentioning that unit tests are not the silver bullet that will magically turn your projects in completely bug-free ones. This is not true even for desktop programming, where we use the same compiler for tests and for production; and in the embedded world, it's even worse, since:
Still, from my personal experience, the final outcome is a way better on carefully tested projects than on untested ones. At least, well-written tests will save you from your own silly mistakes. And if you're like me, you definitely want that. Tools that we'll be usingThe aforementioned book takes advantage of several awesome tools:
These tools are surely life-changing for me, but there is something that is still missing: the test build system. Probably it didn't yet exist when the book was written, so it isn't mentioned in the book at all. Here it is:
Armed with these great tools, let's move on. Sample projectNo surprise, we'll be learning how to write unit tests by testing example project. Let it be rather simple device, which I have actually developed firmware for: the indicator for battery charger that runs on 8-bit PIC16 MCU with 16 kB flash and 1 kB RAM. As you see, resources are quite limited. The repository of this example project can be found here: https://github.com/dimonomid/test_ceedling_example. What does the project look like
In order to bring the project to its initial state, checkout to the tag $ git checkout v0.02 This device should merely measure some voltages (which are converted to an integer via ADC), and display them properly. It should also be available to communicate with the checker device, which will calibrate the electric circuits. Initial project tree looks as follows: . └── src ├── appl │ ├── appl_adc_channels.h │ └── appl.c ├── bsp │ ├── bsp_adc.c │ └── bsp_adc.h └── util ├── adc_handler.c ├── adc_handler.h ├── itoae.c ├── itoae.h └── util_macros.h There are three directories:
Since the resources are very limited, we don't use a RTOS: just a super-loop. We also can't use any
So, the int main(void) { //TODO: MCU-specific init //-- init MCU-specific ADC stuff bsp_adc__init(); //-- enter endless loop for (;;){ bsp_adc__proceed(); } }
This is a quite common scheme for non-RTOS solutions: in the super-loop, we just call each module's
The /** * Type that is used for ADC raw counts. */ typedef uint16_t T_BspAdcCounts; /** * Perform module initialization, including the hardware initialization */ void bsp_adc__init(void); /** * Returns raw ADC counts for the specified channel */ T_BspAdcCounts bsp_adc__value__get(enum E_ApplAdcChannel channel_num); /** * To be repeatedly called from the application's super loop: stores current * measurement when it's ready and switches between different channels as * appropriate */ void bsp_adc__proceed(void);
As you see, it's very easy. When we call
There is an
For each instance of ADC handler ( T_ADCHandler_Res adc_handler__ctor( T_ADCHandler *me, const T_ADCHandler_CtorParams *p_params ); Passing a pointer to parameters: /** * Constructor params for ADC handler */ typedef struct S_ADCHandler_CtorParams { /** * Maximum value that ADC could return. Say, for 10-bit ADC it should be * 0x3ff. */ T_ADCHandler_CountsValue max_counts; /** * The board-dependent maximum voltage that could be measured, it * corresponds to the max_counts. * * It is only needed for calculation of nominal multiplier. */ T_ADCHandler_Voltage bsp_max_voltage; /** * Calibration data: summand and multiplier. * * Set just all zeros to use nominal. Nominal mul will be calculated by * max_counts and bsp_max_voltage. add will be copied from * nominal_add (see below) */ T_ADCHandler_Clb clb; /** * Nominal summand, in volts */ T_ADCHandler_Voltage nominal_add_volts; } T_ADCHandler_CtorParams;
And then, when we want to convert some raw ADC value T_ADCHandler_Voltage my_voltage = adc_handler__voltage__get_by_counts_value( &my_instance, my_raw_adc_value ); As you see, the ADC handler module is completely self-contained: it doesn't have any dependencies. Modules like this are the easiest ones to test, so, let's start our test journey from ADC handler. Install testing toolsBefore we can go any further, we need to install the aforementioned Ceedling and all accompanying tools. Thanks to guys from ThrowTheSwitch.org, installation process is very simple. You need ruby for this. Once you have ruby installed, install Ceedling by typing in your terminal: $ gem install ceedling
And that's it! Now you have all tools ready to work, and among other things, there is a Adding test stuff to project
The
Let's move on: $ ceedling new test_ceedling
This will create the new directory
The heart of the test build system is the project file: :paths: :test: - +:test/** - -:test/support :source: - src/** :support: - test/support
As you might have already guessed, we need to change :source: - ../src/appl - ../src/bsp - ../src/util
Now, in order to avoid confusion, let's delete the auto-created $ touch test_ceedling/build/.gitkeep $ touch test_ceedling/test/support/.gitkeep Add files to the repository and commit: $ git add . $ git commit
Note: you can get everything done by using the prepared repository. Type there:
Actually, our test build system, though empty, is ready to run! Try it: make sure you're in the $ rake test:all You should see the following output: -------------------- OVERALL TEST SUMMARY -------------------- No tests executed. It works, and it predictably reports that we have no tests. So, let's add some meat to the bones! Testing standalone modulesWriting test for ADC handlerAs mentioned above, we'll start by writing tests for our ADC handler, since it is one of the easiest things to test: it has no application-specific dependencies. The job of ADC handler is to convert from raw ADC counts to voltage, and vice versa. We'll test this functionality. First of all, let's create blank test file. I have a template for this: /******************************************************************************* * INCLUDED FILES ******************************************************************************/ //-- unity: unit test framework #include "unity.h" //-- module being tested // TODO /******************************************************************************* * DEFINITIONS ******************************************************************************/ /******************************************************************************* * PRIVATE TYPES ******************************************************************************/ /******************************************************************************* * PRIVATE DATA ******************************************************************************/ /******************************************************************************* * PRIVATE FUNCTIONS ******************************************************************************/ /******************************************************************************* * SETUP, TEARDOWN ******************************************************************************/ void setUp(void) { } void tearDown(void) { } /******************************************************************************* * TESTS ******************************************************************************/ void test_first(void) { //TODO }
We can use this template whenever we need to write new test. Our tests are just functions with names that start with
We also have two special functions: We begin by adding the header of the module being tested: //-- module being tested: ADC handler #include "adc_handler.h" Then, add an instance that we'll run tests against, together with the result code returned from constructor: /******************************************************************************* * PRIVATE DATA ******************************************************************************/ static T_ADCHandler _adc_handler; static T_ADCHandler_Res _ctor_result;
And then, construct/destruct it in void setUp(void) { T_ADCHandler_CtorParams params = {}; //-- 10-bit ADC params.max_counts = 0x3ff; //-- board-dependent maximum measured voltage: 10 Volts params.bsp_max_voltage = 10/*V*/ * ADC_HANDLER__SCALE_FACTOR__U; //-- the offset is 0 Volts params.nominal_add_volts = 0/*V*/; //-- construct the ADC handler, saving the result to _ctor_result _ctor_result = adc_handler__ctor(&_adc_handler, ¶ms); } void tearDown(void) { adc_handler__dtor(&_adc_handler); }
Now, the easiest test we can come up with is to check that constructor has returned successful status, i.e. void test_ctor_ok(void) { //-- check that constructor returned OK TEST_ASSERT_EQUAL_INT(ADC_HANDLER_RES__OK, _ctor_result); } We're ready to run our first test! $ rake test:all Test 'test_adc_handler.c' ------------------------- Generating runner for test_adc_handler.c... Compiling test_adc_handler_runner.c... Compiling test_adc_handler.c... Compiling unity.c... Compiling adc_handler.c... Compiling cmock.c... Linking test_adc_handler.out... Running test_adc_handler.out... ----------- TEST OUTPUT ----------- [test_adc_handler.c] - "" -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 1 PASSED: 1 FAILED: 0 IGNORED: 0 It works, and our test has passed. Good!
Among other things, you see that Ceedling has figured out that it needs to build
If you used to apply TDD practices, you know that it's good to make sure our tests fail if code behaves in wrong way. We don't do TDD here, since we already have some code before we write tests, but we still can make sure that our tests can fail. If we change ----------- TEST OUTPUT ----------- [test_adc_handler.c] - "" ------------------- FAILED TEST SUMMARY ------------------- [test_adc_handler.c] Test: test_ctor_ok At line (72): "Expected 1 Was 6" -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 1 PASSED: 0 FAILED: 1 IGNORED: 0 --------------------- BUILD FAILURE SUMMARY --------------------- Unit test failures.
That's it. Let's change
We definitely don't want to include build output in the repository, so, add the test_ceedling/build And then, commit changes: $ git add . $ git commit -m"added test_adc_handler"
Note: you can get everything done by using the prepared repository. Type there:
Now let's check that ADC handler is actually able to convert from ADC counts to voltage. We'll test the function Here we go: void test_counts_to_voltage(void) { T_ADCHandler_Voltage voltage; //------------------------------------------------------------------ voltage = adc_handler__voltage__get_by_counts_value( &_adc_handler, 0 ); TEST_ASSERT_EQUAL_INT(0/*V*/ * ADC_HANDLER__SCALE_FACTOR__U, voltage); //------------------------------------------------------------------ voltage = adc_handler__voltage__get_by_counts_value( &_adc_handler, 0x3ff ); TEST_ASSERT_EQUAL_INT(10/*V*/ * ADC_HANDLER__SCALE_FACTOR__U, voltage); //------------------------------------------------------------------ voltage = adc_handler__voltage__get_by_counts_value( &_adc_handler, 0x3ff / 3 ); TEST_ASSERT_EQUAL_INT((3.33/*V*/ * ADC_HANDLER__SCALE_FACTOR__U), voltage); } If we run this test, it should pass: -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 2 PASSED: 2 FAILED: 0 IGNORED: 0 So, we can be sure now that ADC handler performs its basic job.
Note: you can get everything done by using the prepared repository. Type there: Writing test for integer-to-string conversion
As was mentioned before, we can't use floats since they are too expensive for this cheap MCU, so, we store voltage as integer in Volts, multiplied by the factor Its API looks like this: /** * Itoa a bit extended: allows to set minimal length of the string * (effectively allowing us to align text by right edge), * and allows to put decimal point at some fixed position. * * @param p_buf * where to save string data * @param value * value to convert to string * @param dpp * decimal point position. If 0, then no decimal point is put. * if 1, then it is put one digit from the right side, etc. * @param min_len * minimum length of the string. If actual string is shorter than * specified length, then leftmost characters are filled with * fill_char. * @param fill_char * character to fill "extra" space */ void itoae(uint8_t *p_buf, int value, int dpp, int min_len, uint8_t fill_char);
You can find the source in the file
As you see, this function is very straightforward to test as well. Let's create new file //-- module being tested #include "itoae.h" We're going to have a buffer for generated string: #define _BUF_LEN 20 /** * Buffer to store generated string data */ static uint8_t _buf[ _BUF_LEN ];
As well as the function that wills the buffer with void _fill_with_0xff(void) { int i; for (i = 0; i < sizeof(_buf); i++){ _buf[i] = 0xff; } } We will call this function before each assert, so that the buffer is reinitialized every time. And a couple of simple tests: void test_basic( void ) { _fill_with_0xff(); itoae(_buf, 123, 0, 0, '0'); TEST_ASSERT_EQUAL_STRING("123", _buf); _fill_with_0xff(); itoae(_buf, -123, 0, 0, '0'); TEST_ASSERT_EQUAL_STRING("-123", _buf); } void test_dpp( void ) { _fill_with_0xff(); itoae(_buf, 123, 1, 0, '0'); TEST_ASSERT_EQUAL_STRING("12.3", _buf); _fill_with_0xff(); itoae(_buf, 123, 2, 0, '0'); TEST_ASSERT_EQUAL_STRING("1.23", _buf); _fill_with_0xff(); itoae(_buf, 123, 3, 0, '0'); TEST_ASSERT_EQUAL_STRING("0.123", _buf); _fill_with_0xff(); itoae(_buf, 123, 4, 0, '0'); TEST_ASSERT_EQUAL_STRING("0.0123", _buf); } Run the tests: ------------------- FAILED TEST SUMMARY ------------------- [test_itoae.c] Test: test_dpp At line (127): "Expected '12.3' Was '12.3\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF\0xFF'" -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 4 PASSED: 3 FAILED: 1 IGNORED: 0
Oops! Something went wrong. It seems that although the dot was inserted to the string correctly, the terminating After examining the source, I see that here's the offending piece of code: int i; for (i = 0; i < (dpp + 1/*null-terminate*/); i++){ p_buf[len - i] = p_buf[len - i - 1]; } p_buf[len - dpp] = '.'; Although I provided the handling of terminating null char, there is a traditional off-by-one error here. The correct code looks as follows: int i; for (i = 0; i < (dpp + 1/*null-terminate*/); i++){ p_buf[len - i + 1] = p_buf[len - i]; } p_buf[len - dpp] = '.'; Save, switch to the terminal, run the tests again: -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 4 PASSED: 4 FAILED: 0 IGNORED: 0
Cool! These very simple tests already saved me from the very silly mistake. Believe it or not, this mistake with
Note: not all tests code is included in the article, since it is very repetitive and straightforward. You can get everything done by using the prepared repository. Type there: Testing modules with dependenciesThe application ADC module
Our application needs for some resourceful way to get current voltage on some particular channel, in Volts. It would be unwise to use our
Now, let's add the module
The header //-- for T_ADCHandler_Voltage #include "adc_handler.h" //-- for enum E_ApplAdcChannel #include "appl_adc_channels.h" /** * Initialize module */ void appl_adc__init(void); /** * Get current voltage of the given channel. */ T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num);
This function should call
For now, let's just fake the implementation ( #include "appl_adc.h" void appl_adc__init(void) { //TODO } T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num) { //TODO return 0; }
And get to tests: create new file //-- module being tested #include "appl_adc.h"
Our void setUp(void) { //-- before each test, re-initialize appl_adc module appl_adc__init(); } void tearDown(void) { //-- nothing to do here }
And get to the test for void test_voltage_get(void) { // ..... }
As was mentioned before,
The answer is - with CMock. And Ceedling will help us here as well: if we need to “mock” some module, all we need to do is to include the header of the module to mock, with the Go on then: add the following include directive: //-- mocked modules #include "mock_bsp_adc.h"
Now, the module being tested will use mocked versions of all functions from void test_voltage_get(void) { //-- We expect bsp_adc__value__get() to be called: bsp_adc__value__get_ExpectAndReturn( //-- the argument that is expected to be given to // bsp_adc__value__get() APPL_ADC_CH__I_SETT, //-- and the value that bsp_adc__value__get() should return (0x3ff / 2) ); //-- actually call the function being tested, that should perform // all pending expected calls T_ADCHandler_Voltage voltage = appl_adc__voltage__get( APPL_ADC_CH__I_SETT ); //-- check the voltage returned (we assume that adc_handler is initialized // with the same params, where 0x3ff is the maximum ADC value, and // it corresponds to the value (10 * ADC_HANDLER__SCALE_FACTOR__U)) TEST_ASSERT_EQUAL_INT((5 * ADC_HANDLER__SCALE_FACTOR__U), voltage); } If we run the tests, we get the following result: ------------------- FAILED TEST SUMMARY ------------------- [test_appl_adc.c] Test: test_voltage_get At line (77): "Expected 500 Was 0"
So it complains that returned value was wrong. Okay, let's fake our dummy T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num) { //TODO return 500; } And run tests again: ------------------- FAILED TEST SUMMARY ------------------- [test_appl_adc.c] Test: test_voltage_get At line (56): "Function 'bsp_adc__value__get' called less times than expected."
Oh, cool! It reports that the function
Well, it's time to implement
Ok, it seems, everything is right. Try to run tests: Linking test_appl_adc.out... build/test/out/appl_adc.o: In function 0097ppl_adc__init': /home/dimon/projects/indicator_git/test_ceedling/../src/appl/appl_adc.c:70: undefined reference to 0097dc_handler__ctor' build/test/out/appl_adc.o: In function 0097ppl_adc__voltage__get': /home/dimon/projects/indicator_git/test_ceedling/../src/appl/appl_adc.c:76: undefined reference to 0097dc_handler__voltage__get_by_counts_value' collect2: error: ld returned 1 exit status .... NOTICE: If the linker reports missing symbols, the following may be to blame: 1. Test lacks #include statements corresponding to needed source files. 2. Project search paths do not contain source files corresponding to #include statements in the test. 3. Test does not #include needed mocks.
Oh, dear. Linker complains about undefined reference to ADC handler functions. And Ceedling is being very kind here by providing us with useful notice: as it suggests, one of the possible reasons is that test lacks //-- other modules that need to be compiled #include "adc_handler.h" Now, the tests should finally work: -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 7 PASSED: 7 FAILED: 0 IGNORED: 0
Note: you can get everything done by using the prepared repository. Type there: More isolated test of the application ADC module
As you might have noticed, in the previous section we in fact ended up testing two modules at once:
You might have already guessed that we're going to mock ADC handler as well, instead of using real code. So, first of all, let's remove //-- mocked modules #include "mock_bsp_adc.h" #include "mock_adc_handler.h" And now, we in fact have several options. Ignore arguments given to ADC handler
The easiest option is to ignore arguments given to void test_voltage_get(void) { //-- We expect bsp_adc__value__get() to be called: bsp_adc__value__get_ExpectAndReturn( //-- the argument that is expected to be given to // bsp_adc__value__get() APPL_ADC_CH__I_SETT, //-- and the value that bsp_adc__value__get() should return 123 ); //-- Expect call to adc_handler__voltage__get_by_counts_value(), // ignoring arguments. The mocked version just returns 456. adc_handler__voltage__get_by_counts_value_IgnoreAndReturn(456); //-- actually call the function being tested, that should perform // all pending expected calls T_ADCHandler_Voltage voltage = appl_adc__voltage__get( APPL_ADC_CH__I_SETT ); //-- check the voltage returned (it should be 456 from the mock above) TEST_ASSERT_EQUAL_INT(456, voltage); } Run tests: ------------------- FAILED TEST SUMMARY ------------------- [test_appl_adc.c] Test: test_voltage_get At line (57): "Function 'adc_handler__ctor' called more times than expected."
Oh yes, we forgot that now we have to mock not only void setUp(void) { //-- ADC handler constructor is going to be called for each channel. // Mocked constructors all return ADC_HANDLER_RES__OK. enum E_ApplAdcChannel channel; for (channel = 0; channel < APPL_ADC_CH_CNT; channel++){ adc_handler__ctor_IgnoreAndReturn(ADC_HANDLER_RES__OK); } //-- before each test, re-initialize appl_adc module appl_adc__init(); } Now tests pass.
Note: you can get everything done by using the prepared repository. Type there: Check arguments given to ADC handler
The arguments given to
So, if we want to check arguments, we need to access /** * For usage in tests only! */ T_ADCHandler *_appl_adc__adc_handler__get(enum E_ApplAdcChannel channel) { return &_adc_handlers[channel]; }
We won't include this function prototype in the header file /******************************************************************************* * EXTERNAL FUNCTION PROTOTYPES ******************************************************************************/ extern T_ADCHandler *_appl_adc__adc_handler__get(enum E_ApplAdcChannel channel);
And now, instead of ignoring arguments given to //-- Expect call to adc_handler__voltage__get_by_counts_value(), // ignoring arguments. The mocked version just returns 456. adc_handler__voltage__get_by_counts_value_ExpectAndReturn( //-- pointer to the appropriate ADC handler instance _appl_adc__adc_handler__get(APPL_ADC_CH__I_SETT), //-- value returned from bsp_adc__value__get() 123, //-- returned value in Volts 456 );
Notice that the value returned from mocked
Note: you can get everything done by using the prepared repository. Type there: Use mock with callback
Apart from easy-to-use helpers, CMock provides us with the very flexible callback helper. The callback should have the same signature as the mocked function, but it takes one additional argument:
In the callback, we might check whatever we want, and if something goes wrong, we can call Unity macro Let's implement such a callback: /******************************************************************************* * PRIVATE FUNCTIONS ******************************************************************************/ static T_ADCHandler_Voltage _get_by_counts_value_Callback( T_ADCHandler *me, T_ADCHandler_CountsValue counts_value, int num_calls ) { T_ADCHandler_Voltage ret = 0; switch (num_calls){ case 0: if (counts_value != 123){ //-- We can check whatever we want here. For example, we may // check the data pointed to by "me", but NOTE that currently // it is just zeros, since we have mocked adc_handler__ctor() // as well, so the original constructor isn't called, and // instances are left unitialized. TEST_FAIL_MESSAGE( "adc_handler__voltage__get_by_counts_value() was called " "with wrong counts_value" ); } ret = 456; break; default: TEST_FAIL_MESSAGE( "adc_handler__voltage__get_by_counts_value() was called " "too many times" ); break; } return ret; }
And in our //-- Expect call to adc_handler__voltage__get_by_counts_value() adc_handler__voltage__get_by_counts_value_StubWithCallback( _get_by_counts_value_Callback ); Although callbacks like this don't look quite elegant, and for this particular example it is an unnecessary overkill, they are extremely flexible. So, keep it in your toolbox, and use when appropriate.
Note: you can get everything done by using the prepared repository. Type there: Dealing with compiler-specific stuff
Compilers often have some useful non-standard built-in things. For example, the XC8 Microchip compiler has the function
I often use it for some conditions that should never happen. For example, our #include "xc.h" // ..... T_ADCHandler_Voltage appl_adc__voltage__get(enum E_ApplAdcChannel channel_num) { T_ADCHandler_Voltage ret = 0; if (channel_num < APPL_ADC_CH_CNT){ ret = adc_handler__voltage__get_by_counts_value( &_adc_handlers[ channel_num ], bsp_adc__value__get(channel_num) ); } else { //-- illegal channel_num given: should never be here __builtin_software_breakpoint(); } return ret; } Checks like this are a must have in any application, but if we try to run tests, we'll end up with the following error: Compiling appl_adc.c... ../src/appl/appl_adc.c:15:16: fatal error: xc.h: No such file or directory #include "xc.h"
Obviously, GCC (which is used for tests by default) have neither such a built-in function, nor the
We can address this problem by using the Ceedling “support” directory, which is located by default at
If we run tests now, we'll have different error: Linking test_appl_adc.out... build/test/out/appl_adc.o: In function 0097ppl_adc__voltage__get': /home/dimon/projects/indicator_git/test_ceedling/../src/appl/appl_adc.c:101: undefined reference to 0095_builtin_software_breakpoint' collect2: error: ld returned 1 exit status
Nice: at least, our #include "mock_xc.h" Now, run tests, and they pass! -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 7 PASSED: 7 FAILED: 0 IGNORED: 0
And we can write one more test: let's check that void test_voltage_get_wrong_channel_number(void) { //-- we expect __builtin_software_breakpoint() to be called ... __builtin_software_breakpoint_Expect(); //-- ... when we call appl_adc__voltage__get() with illegal // channel number. appl_adc__voltage__get( APPL_ADC_CH_CNT ); } And tests pass again: -------------------- OVERALL TEST SUMMARY -------------------- TESTED: 8 PASSED: 8 FAILED: 0 IGNORED: 0
You're encouraged to verify that if we remove a call to
Note: you can get everything done by using the prepared repository. Type there: Some more notes about testing on the host machineTesting on host machine is quite convenient: running tests is just a matter of a few keystrokes, tests run fast, and we get results almost immediately. But we should take some care, since the architectures are different. As already discussed above, different compilers have some built-in functions. Apart from this, the memory alignment is often different: at 8-bit MCUs, the alignment is 1 byte, but on your host machine it's usually 4 or 8 bytes (depending on your architecture). So, we have to multi-target our applications.
I often find myself creating a file like #if defined(__XC8__) //-- no need for "packed" attr on 8-bit MCU # define __MY_CROSS_ATTR_PACKED #elif defined(__GNUC__) # define __MY_CROSS_ATTR_PACKED __attribute__((packed)) #endif
And in the application code, I use the macro This way, we can write code that works both on target MCU as well as on the host machine. Of course, it takes additional effort and time, but so do tests in general. I spend a lot of time writing tests these days. It pays off very well. Other testing ideasWriting tests code is often considered as a tedious process, and I can't entirely disagree. However, I always encourage myself to find some new ways to test things, instead of repeatedly test this and that.
As an example, consider the EEPROM module (Electrically Erasable Programmable Read-Only Memory). We will most likely end up with MCU- or board-specific module
In addition to int16_t appl_eeprom__adc_clb_mul__get(enum E_ApplAdcChannel channel); void appl_eeprom__adc_clb_mul__set(enum E_ApplAdcChannel channel, int16_t mul); If the application is rather large, there may be tons of functions like that. It is very tedious to test them all separately.
Instead, we may think about the most easy mistake to make. For things like the
So, I often use the technique like this: define stub callbacks for
Then, perform “write” test: I call every “write” function from And, of course, exactly the same test should be done for “read” functions. It is much more fun (and fast) than test each and every function separately, and in the end we'll have tests that are reliable enough. ConclusionThe tools by guys from ThrowTheSwitch.org allow us to test our C code almost painlessly. Thank you, guys! I hope this article helps you to get somewhat big picture about Ceedling and its companions, and I encourage you to examine the documentation, which is quite concise. The easiest way to get documentation of all components in one place is to create new ceedling project by executing: $ ceedling new my_project
And navigate to And again, if you feel serious about investing a great deal of time into testing your embedded designs (which is probably a good idea), consider reading the book Test Driven Development for Embedded C by James W. Grenning, which explains various testing methods, approaches and tools very thoroughly. Let's write C code that doesn't suck! You might as well be interested in:
DiscussionFor me a little bit complex way to start. In our projects we using very simple approarch based on asserts ('ASSERT(clause)', 'ASSERT_EQL(el1, el2)'…) and integrate them directly with embedded code, which is called in Debug after start-up procedures in Microcontroller automatically. For example a module with very simple switch-logic - https:///0IZVXQ. Asserts like this and unit tests that I'm talking about in the article are entirely different things, and they are not mutually exclusive. Of course, I use asserts, and it doesn't prevent me from unit-testing my code as well. Dmitry, code on your page https:///0IZVXQ needs header file for those ASSERTS definition - where can I find them ? Hey! This is a great tutorial. Just one thing that came to my mind while reading it. Don't you think that in itoae tests you should have used setUp to call _fill_with_0xff(). Repetition is a bad thing here. My eyes hurt. Thanks for the comment. Well, I was thinking about it, but then, we'd end up with lots of 2-line test functions, and each one should be named. This is even worse I believe, so I implemented it as it is now. Nice article, Dmitry. But I agree with Marek, repetion smells bad. Therefore, I'd recommend the following: static void test_itoae_dpp_expect(int dpp, const char *expected) { _fill_with_0xff(); itoae(_buf, 123, dpp, 0, '0'); TEST_ASSERT_EQUAL_STRING(expected, _buf); } void test_dpp( void ) { test_itoae_dpp_expect(1, "12.3"); test_itoae_dpp_expect(2, "1.23"); test_itoae_dpp_expect(3, "0.123"); test_itoae_dpp_expect(4, "0.0123"); } For _fill_with_0xff() I'd just use memset(_buf, 0xff, sizeof(_buf)); (from string.h) One more (important) remark: leading underscores followed by uppercase letter are reserved for the compiler (_BUF), the same applies for all symbols with two underscores (appl_adc__init) http:///questions/228783/what-are-the-rules-about-using-an-underscore-in-a-c-identifier Best regards, Andre Hi Andre, Thank you very much for your quite valuble remark about the underscores and uppercase letters! Learned something new today. As to the avoiding repetition, yes, I admit that your solution is much better than mine. I'll try to find time and fix the article. Thanks again! may I propose that you get rid of all the above and create an SIMPLE (!!!) code with say one (2 the most!) SIMPLE functions in just a FEW source / header files. The way it is now is too complex to illustrate how that works - I quit ~ 15% down the track
I'm afraid I won't get rid of all the above. I'm not sure I understand what's so hard in testing, for example, And btw I've read some of “simple” tutorials before, and I was actually disappointed since I wanted more than just the plain basics. So eventually I figured it myself, and have written my own tutorial. If you want some tutorial that only touches basics, just get another one.
Matt Chernosky, 2015/12/15 05:09
Hi Dmitry, Nice to see such a thorough tutorial! I've used Ceedling too, and I think it's great for testing embedded software (not that there are a whole lot of options). The documentation isn't always very helpful though. I'm still not seeing unit testing used a lot in the embedded world. It's work like this that is going to help get embedded software out of the dark ages. Matt I tried it and the amount of time needed to setup and design tests is greater than the savings in later testing. All is very complex and often confusing. And because of that there is a high chance that the 'tests themselves' need to be verified/tested as well. So I would stay with the old way of testing.
Hmm, we must have different notions of what is large amount of time, especially about setup. Setup basically is: $ gem install ceedling $ ceedling new test_ceedling
And in Large amount of time, huh? :) Anyway, it's not at all a surprise that the people that got used to the “old way of testing” find learning formal tests an unnecessary effort. Up to you, of course.
Matt Chernosky, 2016/02/29 05:00
Ceedling makes it so easy to add new tests! Just add functions starting with test_ to the module test file… and the tests are automatically discovered and run. And, auto-generate mocks from .h files with a single #include “mock_ “! If it's difficult to write the tests, I'd recommend trying the test-driven (TDD) approach. It's a different mindset, but it has so many advantages. It really forces you to think about how each independent unit is going to be used. Hi Dmitry, Thanks for the great tutorial. do you have any idea how to setup the project with eclipse. I tried to build it in eclipse but i've got the following error: Cannot run program “rake”: Launching failed Hi, thanks for the comment.
I never tried to run it from Eclipse, so I'm afraid I won't be able to help. However, the error message suggests that you don't have
hmijail, 2016/04/12 19:38
I agree that some streamlining would be good, but this is the only acceptable “docs” I have found for Ceedling & friends. The disaster that is the official documentation has managed to send me a couple of times to reassess CMocka and Cpputest… … which is a pity, given how powerful the whole set looks like once one manages to get the full picture. I started with Unity and was successful in covering the basics. Now when i jumped onto cmock i had a load of promlems on my door. 1. Everytime i mocked my header file(Func.h) with default settings… i was able to generate mock files. With only Func_Expectandreturn(int, int). In case to generate other function with :Cexception, :Ignore, :Array, etc i was unable to do so. i didn't much relevant data in the internet as well. 2. from the cmock doc folder, i tried doing some thing like this: ruby cmock.rb –mock_path=“[myPath]\mocks” –plugin=:Cexception,:Ignore Func.h This command in the prompt generated mocks but only with default configuration. i.e all i had was Func_Expectandreturn(int, int) thats it. Not Func_ExpectAndThrow. 3. where have i missed, am i suppose to include Cexception/src .h file somewhere? if yes, then where. 4. Also tried rakefile: — load 'cmock.rb' cmock = CMock.new(:plugins ⇒ [:cexception, :ignore], :mock_path ⇒ '[PATH]\mocks') — this also didn't work. ended up with default mocking as above. Please Help me. Nitin Sinha India You need to add the plugins for cmock to generate the other mocks.
Look at the CMock documentation; if you're using Ceedling, look at the
An array of which plugins to enable. ':expect' is always active. Also available currently:
Thanks Dmitry, this post has helped me numerous times - I keep returning to it after my Unity unit test knowledge has advanced further and it always helps fill in a gap or two! T Hi. The Test-Driven Development for Embedded C ebook by James Grenning can be bought DRM free(!) from The Pragmatic Bookshelf website: https:///book/jgade/test-driven-development-for-embedded-c. They also send updates to the books into your Dropbox and/or your email account. Thanks for this great article. Is there a way to do data driven testing with this framework. And after comparing this whole setup with Cunit I found ceedling very nice. The modular structure of this framework is awesome and something unheard of in the crass and crude embedded domain. Kudos!!! Hello Dmitry, Thanks for writing your tutorial. I found you from the site (http://www./articles). Anyway, when I followed your tutorial and issued this step: ceedling new test_ceedling it did not create a rakefile.rb in my test_ceedling directory. Also, this command: rake test:all will generate an error since there is no rakefile.rb. I checked back at the http://www./ceedling page and it appears that they are saying to use this command in place of “rake test:all”: ceedling test:all Anyway, I am new to ceedling, so I am just providing you this information in case I am doing something wrong or if something has changed in ceedling itself since the time you wrote your tutorial. Thanks, Mike Hello Mike! Turns out, there is a change in a newer version of ceedling; I'll need to check it out and update the tutorial accordingly. Thanks for reporting! Hi Mike, Dmitry, Thanks for the great tutorial. In the newer version of ceedling, you only have to cd to the test_ceedling directory and run “ceedling”. This will automatically run the rakefile, which has been moved into the vendor folder. I did have to update the ceedling installation included in the git repo. Run ceedling new test_ceedling, and it will prompt about a bunch of files to overwrite. If I remember correctly, you should only need to keep the project.yml file and replace all the others. Thanks! Hi Dimitry Thank you for the most helpful tutorial. I have module A which uses module B. For most tests of module A I would like to call the real functions of module B. But there a few tests which I want to use the mocked functions of module B (mostly when I want to generate error code returns of module B) Is there a way to use both mocked and real Module B ? Alternatively is it possible to create two tests suits for Module A. Thanks, Shaul Hi Shaul,
Thanks for the comment. Unfortunately, I don't think it's possible to use both mocked and real functions of the module B in the same file, but yes, the workaround would be to have two test files: in the first one you'll include the real module B header like Hi Dimitry, Thank you for the prompt response. If understand correctly, there is no way to use ceedling module:create[Filename] command for creating two test files for the same module. The second test file should be created manually in the test folder. Is that right ? Thanks, Shaul Hi Dmitry, I'm deciding if I use Unity or CppuTest to apply TDD in my designs, what do you think about Cpputest? I will apply TDD to designs with PIC too. Thanks, wishing you the best! Hi Ezequiel, Honestly for embedded stuff I only used tools described in this article. In particular, I never used Cpputest, so I'm afraid I can't be helpful here. Hi Dmitry, Is there any way that I can run my unit test cases on a real MCU/Simulator? Have you done something like that? Thanks Hi Raju, I haven't done myself (running on PC is very fast and good for me), but running on target or simulator should be no problem. The tests are pure C code called from a main, just like any embedded SW. The biggest problem might be the limited resources (flash) in your target, so you _may_ need to split your tests in multiple programs and upload/run them separately. Regards Hi Dmitry, This is an excellent tutorial. It would be great if you can update it with updated Ceedling. Also, it would be a great help if you can add side requirements and installing procedures, (installing gcc/MinGW etc). I can then directly share this with my students. Hello , I am using Unity to test one function which starts with passing a ptr, But when i try to test the same i am getting segmentation fault. Can you help me out in this?
I'd use the debugger and check the value of the pointer. Chances are high that the pointer is invalid or NULL and the use within your function let's the program crash. Regards |
|