5. ユニットテスト

2007年11月の時点で、私たちはマスターに入るすべての新機能は、ユニットテストを伴うことを要求しています。当初は私たちはこの要件はqgis_coreに制限してきましたが、皆様が次のセクションで説明するユニットテストのための手続きに精通した後は、この要件をコードベースの他の部分に拡張するつもりです。

5.1. QGISのテストフレームワーク - 概要

Unit testing is carried out using a combination of QTestLib (the Qt testing library) and CTest (a framework for compiling and running tests as part of the CMake build process). Lets take an overview of the process before we delve into the details:

1. There is some code you want to test, e.g. a class or function. Extreme programming advocates suggest that the code should not even be written yet when you start building your tests, and then as you implement your code you can immediately validate each new functional part you add with your test. In practice you will probably need to write tests for pre-existing code in QGIS since we are starting with a testing framework well after much application logic has already been implemented.

2. ユニットテストを作成します。これはcore libの場合 <QGIS Source Dir>/tests/src/core の下で起こります。テストは、基本的には、クラスのインスタンスを作成し、そのクラスにいくつかのメソッドを呼び出すクライアントです。それは各メソッドからの戻りをチェックし、期待値と一致することを確認します。呼び出しのいずれかに障害が発生した場合、そのユニットは失敗します。

3. テストクラスにQtTestLibマクロを含めます。このマクロは、Qtのメタオブジェクトコンパイラ（MOC）によって処理され、実行可能なアプリケーションにテストクラスを拡張します。

4. テストを構築するtestsディレクトリにCMakeLists.txtにセクションを追加します。

5. ENABLE_TESTING がccmake/cmakesetupで有効になっていることを確認します。これは、makeと入力すると、テストが実際にコンパイルされることを保証します。

6. You optionally add test data to <QGIS Source Dir>/tests/testdata if your test is data driven (e.g. needs to load a shapefile). These test data should be as small as possible and wherever possible you should use the existing datasets already there. Your tests should never modify this data in situ, but rather make a temporary copy somewhere if needed.

7. ソースをコンパイルしてインストールします。これは通常の make && (sudo) make install 手順を使用して実行します。

8. You run your tests. This is normally done simply by doing make test after the make install step, though we will explain other approaches that offer more fine grained control over running tests.

Right with that overview in mind, we will delve into a bit of detail. We've already done much of the configuration for you in CMake and other places in the source tree so all you need to do are the easy bits - writing unit tests!

5.2. ユニットテストの作成

Creating a unit test is easy - typically you will do this by just creating a single .cpp file (no .h file is used) and implement all your test methods as public methods that return void. We'll use a simple test class for QgsRasterLayer throughout the section that follows to illustrate. By convention we will name our test with the same name as the class they are testing but prefixed with 'Test'. So our test implementation goes in a file called testqgsrasterlayer.cpp and the class itself will be TestQgsRasterLayer. First we add our standard copyright banner:

/***************************************************************************
 testqgsvectorfilewriter.cpp
 --------------------------------------
  Date : Friday, Jan 27, 2015
  Copyright: (C) 2015 by Tim Sutton
  Email: tim@kartoza.com
 ***************************************************************************
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 ***************************************************************************/

Next we start our includes needed for the tests we plan to run. There is one special include all tests should have:

#include <QtTest/QtTest>

その他は、必要となる可能性があるヘッダは何でも引っ張って、通常通りあなたのクラスを実装し続けるだけです：

//Qt includes...
#include <QObject>
#include <QString>
#include <QObject>
#include <QApplication>
#include <QFileInfo>
#include <QDir>

//qgis includes...
#include <qgsrasterlayer.h>
#include <qgsrasterbandstats.h>
#include <qgsapplication.h>

ここではクラス宣言と実装の両方を単一のファイルに組み合わせているので、クラス宣言が次に来ます。doxygenのドキュメントで始めます。すべてのテストケースは適切に文書化されなければなりません。すべてのユニットテストが生成されたDoxygenのドキュメントのモジュールとして表示されるように、doxygenのingroupディレクティブを使用します。その後にユニットテストの短い説明が来て、このクラスはQObjectを継承し、Q_OBJECTマクロをインクルードする必要があります。

/** \ingroup UnitTests
 * This is a unit test for the QgsRasterLayer class.
 */

class TestQgsRasterLayer: public QObject
{
    Q_OBJECT

All our test methods are implemented as private slots. The QtTest framework will sequentially call each private slot method in the test class. There are four 'special' methods which if implemented will be called at the start of the unit test (initTestCase), at the end of the unit test (cleanupTestCase). Before each test method is called, the init() method will be called and after each test method is called the cleanup() method is called. These methods are handy in that they allow you to allocate and cleanup resources prior to running each test, and the test unit as a whole.

private slots:
  // will be called before the first testfunction is executed.
  void initTestCase();
  // will be called after the last testfunction was executed.
  void cleanupTestCase(){};
  // will be called before each testfunction is executed.
  void init(){};
  // will be called after every testfunction.
  void cleanup();

Then come your test methods, all of which should take no parameters and should return void. The methods will be called in order of declaration. We are implementing two methods here which illustrate two types of testing.

In the first case we want to generally test if the various parts of the class are working, We can use a functional testing approach. Once again, extreme programmers would advocate writing these tests before implementing the class. Then as you work your way through your class implementation you iteratively run your unit tests. More and more test functions should complete successfully as your class implementation work progresses, and when the whole unit test passes, your new class is done and is now complete with a repeatable way to validate it.

Typically your unit tests would only cover the public API of your class, and normally you do not need to write tests for accessors and mutators. If it should happen that an accessor or mutator is not working as expected you would normally implement a regression test to check for this.

//
// Functional Testing
//

/** Check if a raster is valid. */
void isValid();

// more functional tests here ...

5.2.1. Implementing a regression test

Next we implement our regression tests. Regression tests should be implemented to replicate the conditions of a particular bug. For example:

1. We received a report by email that the cell count by rasters was off by 1, throwing off all the statistics for the raster bands.

2. We opened a bug report (ticket #832)

3. We created a regression test that replicated the bug using a small test dataset (a 10x10 raster).

4. We ran the test, verifying that it did indeed fail (the cell count was 99 instead of 100).

5. Then we went to fix the bug and reran the unit test and the regression test passed. We committed the regression test along with the bug fix. Now if anybody breakes this in the source code again in the future, we can immediately identify that the code has regressed.

   Better yet, before committing any changes in the future, running our tests will ensure our changes don't have unexpected side effects - like breaking existing functionality.

回帰テストへの1つの以上の利点があります - それらは時間を節約できます。ソースに変更を加えてからアプリケーションを実行して問題を再現するために複雑な一連のステップを実行することに関わるバグを修正したことがあれば、バグを修正する前に回帰テストを実装するだけで、効率的な方法でバグの解決のためのテストを自動化できることがすぐに明らかになるでしょう。

To implement your regression test, you should follow the naming convention of regression<TicketID> for your test functions. If no ticket exists for the regression, you should create one first. Using this approach allows the person running a failed regression test easily go and find out more information.

//
// Regression Testing
//

/** This is our second test case...to check if a raster
 *  reports its dimensions properly. It is a regression test
 *  for ticket #832 which was fixed with change r7650.
 */
void regression832();

// more regression tests go here ...

Finally in your test class declaration you can declare privately any data members and helper methods your unit test may need. In our case we will declare a QgsRasterLayer * which can be used by any of our test methods. The raster layer will be created in the initTestCase() function which is run before any other tests, and then destroyed using cleanupTestCase() which is run after all tests. By declaring helper methods (which may be called by various test functions) privately, you can ensure that they won't be automatically run by the QTest executable that is created when we compile our test.

  private:
    // Here we have any data structures that may need to
    // be used in many test cases.
    QgsRasterLayer * mpLayer;
};

これで私たちのクラス宣言を終了します。実装は単に同じファイル中で下方にインライン化されます。まず、私たちのinitおよびcleanup関数：

void TestQgsRasterLayer::initTestCase()
{
  // init QGIS's paths - true means that all path will be inited from prefix
  QString qgisPath = QCoreApplication::applicationDirPath ();
  QgsApplication::setPrefixPath(qgisPath, TRUE);
#ifdef Q_OS_LINUX
  QgsApplication::setPkgDataPath(qgisPath + "/../share/qgis");
#endif
  //create some objects that will be used in all tests...

  std::cout << "PrefixPATH: " << QgsApplication::prefixPath().toLocal8Bit().data() << std::endl;
  std::cout << "PluginPATH: " << QgsApplication::pluginPath().toLocal8Bit().data() << std::endl;
  std::cout << "PkgData PATH: " << QgsApplication::pkgDataPath().toLocal8Bit().data() << std::endl;
  std::cout << "User DB PATH: " << QgsApplication::qgisUserDbFilePath().toLocal8Bit().data() << std::endl;

  //create a raster layer that will be used in all tests...
  QString myFileName (TEST_DATA_DIR); //defined in CmakeLists.txt
  myFileName = myFileName + QDir::separator() + "tenbytenraster.asc";
  QFileInfo myRasterFileInfo ( myFileName );
  mpLayer = new QgsRasterLayer ( myRasterFileInfo.filePath(),
  myRasterFileInfo.completeBaseName() );
}

void TestQgsRasterLayer::cleanupTestCase()
{
  delete mpLayer;
}

上記のinit関数はいくつかの興味深い物事を示しています。

1. We needed to manually set the QGIS application data path so that resources such as srs.db can be found properly.

2. Secondly, this is a data driven test so we needed to provide a way to generically locate the tenbytenraster.asc file. This was achieved by using the compiler define TEST_DATA_PATH. The define is created in the CMakeLists.txt configuration file under <QGIS Source Root>/tests/CMakeLists.txt and is available to all QGIS unit tests. If you need test data for your test, commit it under <QGIS Source Root>/tests/testdata. You should only commit very small datasets here. If your test needs to modify the test data, it should make a copy of it first.

Qtではまた、テストをデータ駆動するためのいくつかの他の興味深いメカニズムを提供していますので、この話題に関してもっと知りたいと興味をお持ちの方は、Qtのドキュメントを参照してください。

Next lets look at our functional test. The isValid() test simply checks the raster layer was correctly loaded in the initTestCase. QVERIFY is a Qt macro that you can use to evaluate a test condition. There are a few other use macros Qt provide for use in your tests including:

• QCOMPARE ( actual, expected )

• QEXPECT_FAIL ( dataIndex, comment, mode )

• QFAIL ( message )

• QFETCH ( type, name )

• QSKIP ( description, mode )

• QTEST ( actual, testElement )

• QTEST_APPLESS_MAIN ( TestClass )

• QTEST_MAIN ( TestClass )

• QTEST_NOOP_MAIN ()

• QVERIFY2 ( condition, message )

• QVERIFY ( condition )

• QWARN ( message )

これらのマクロの一部は（詳細はQtのドキュメントを参照してください）、テスト駆動型のデータのためのQtフレームワークを使用している場合にのみ便利です。

void TestQgsRasterLayer::isValid()
{
  QVERIFY ( mpLayer->isValid() );
}

通常、あなたの機能テストが可能な場合、あなたのクラスのパブリックAPIのすべての機能範囲をカバーするでしょう。私たちの機能テストは out the way、回帰テストの例を見ることができます。

バグ＃832での問題はセル数の誤った報告なので、私たちのテストを書くことは、単純にセル数が期待値を満たしていることをQVERIFYを使用して確認する問題です。

void TestQgsRasterLayer::regression832()
{
  QVERIFY ( mpLayer->getRasterXDim() == 10 );
  QVERIFY ( mpLayer->getRasterYDim() == 10 );
  // regression check for ticket #832
  // note getRasterBandStats call is base 1
  QVERIFY ( mpLayer->getRasterBandStats(1).elementCountInt == 100 );
}

With all the unit test functions implemented, there's one final thing we need to add to our test class:

QTEST_MAIN(TestQgsRasterLayer)
#include "testqgsrasterlayer.moc"

The purpose of these two lines is to signal to Qt's moc that this is a QtTest (it will generate a main method that in turn calls each test function. The last line is the include for the MOC generated sources. You should replace testqgsrasterlayer with the name of your class in lower case.

5.3. Comparing images for rendering tests

Rendering images on different environments can produce subtle differences due to platform-specific implementations (e.g. different font rendering and antialiasing algorithms), to the fonts available on the system and for other obscure reasons.

When a rendering test runs on Travis and fails, look for the dash link at the very bottom of the Travis log. This link will take you to a cdash page where you can see the rendered vs expected images, along with a "difference" image which highlights in red any pixels which did not match the reference image.

The QGIS unit test system has support for adding "mask" images, which are used to indicate when a rendered image may differ from the reference image. A mask image is an image (with the same name as the reference image, but including a _mask.png suffix), and should be the same dimensions as the reference image. In a mask image the pixel values indicate how much that individual pixel can differ from the reference image, so a black pixel indicates that the pixel in the rendered image must exactly match the same pixel in the reference image. A pixel with RGB 2, 2, 2 means that the rendered image can vary by up to 2 in its RGB values from the reference image, and a fully white pixel (255, 255, 255) means that the pixel is effectively ignored when comparing the expected and rendered images.

A utility script to generate mask images is available as scripts/generate_test_mask_image.py. This script is used by passing it the path of a reference image (e.g. tests/testdata/control_images/annotations/expected_annotation_fillstyle/expected_annotation_fillstyle.png) and the path to your rendered image.

E.g.

scripts/generate_test_mask_image.py tests/testdata/control_images/annotations/expected_annotation_fillstyle/expected_annotation_fillstyle.png /tmp/path_to_rendered_image.png

You can shortcut the path to the reference image by passing a partial part of the test name instead, e.g.

scripts/generate_test_mask_image.py annotation_fillstyle /tmp/path_to_rendered_image.png

(This shortcut only works if a single matching reference image is found. If multiple matches are found you will need to provide the full path to the reference image.)

The script also accepts http urls for the rendered image, so you can directly copy a rendered image url from the cdash results page and pass it to the script.

Be careful when generating mask images - you should always view the generated mask image and review any white areas in the image. Since these pixels are ignored, make sure that these white images do not cover any important portions of the reference image -- otherwise your unit test will be meaningless!

Similarly, you can manually "white out" portions of the mask if you deliberately want to exclude them from the test. This can be useful e.g. for tests which mix symbol and text rendering (such as legend tests), where the unit test is not designed to test the rendered text and you don't want the test to be subject to cross-platform text rendering differences.

To compare images in QGIS unit tests you should use the class QgsMultiRenderChecker or one of its subclasses.

To improve tests robustness here are few tips:

1. Disable antialiasing if you can, as this minimizes cross-platform rendering differences.

2. Make sure your reference images are "chunky"... i.e. don't have 1 px wide lines or other fine features, and use large, bold fonts (14 points or more is recommended).

3. Sometimes tests generate slightly different sized images (e.g. legend rendering tests, where the image size is dependent on font rendering size - which is subject to cross-platform differences). To account for this, use QgsMultiRenderChecker::setSizeTolerance() and specify the maximum number of pixels that the rendered image width and height differ from the reference image.

4. Don't use transparent backgrounds in reference images (CDash does not support them). Instead, use QgsMultiRenderChecker::drawBackground() to draw a checkboard pattern for the reference image background.

5. When fonts are required, use the font specified in QgsFontUtils::standardTestFontFamily() ("QGIS Vera Sans").

If travis reports errors for new images (for instance due to antialiasing or font differences), the script parse_dash_results.py can help you when you are updating the local test masks.

5.4. CMakeLists.txtにユニットテストを追加する

Adding your unit test to the build system is simply a matter of editing the CMakeLists.txt in the test directory, cloning one of the existing test blocks, and then replacing your test class name into it. For example:

# QgsRasterLayer test
ADD_QGIS_TEST(rasterlayertest testqgsrasterlayer.cpp)

5.4.1. 説明されたADD_QGIS_TESTマクロ

We'll run through these lines briefly to explain what they do, but if you are not interested, just do the step explained in the above section.

MACRO (ADD_QGIS_TEST testname testsrc)
SET(qgis_${testname}_SRCS ${testsrc} ${util_SRCS})
SET(qgis_${testname}_MOC_CPPS ${testsrc})
QT4_WRAP_CPP(qgis_${testname}_MOC_SRCS ${qgis_${testname}_MOC_CPPS})
ADD_CUSTOM_TARGET(qgis_${testname}moc ALL DEPENDS ${qgis_${testname}_MOC_SRCS})
ADD_EXECUTABLE(qgis_${testname} ${qgis_${testname}_SRCS})
ADD_DEPENDENCIES(qgis_${testname} qgis_${testname}moc)
TARGET_LINK_LIBRARIES(qgis_${testname} ${QT_LIBRARIES} qgis_core)
SET_TARGET_PROPERTIES(qgis_${testname}
PROPERTIES
# skip the full RPATH for the build tree
SKIP_BUILD_RPATHTRUE
# when building, use the install RPATH already
# (so it doesn't need to relink when installing)
BUILD_WITH_INSTALL_RPATH TRUE
# the RPATH to be used when installing
INSTALL_RPATH ${QGIS_LIB_DIR}
# add the automatically determined parts of the RPATH
# which point to directories outside the build tree to the install RPATH
INSTALL_RPATH_USE_LINK_PATH true)
IF (APPLE)
# For Mac OS X, the executable must be at the root of the bundle's executable folder
INSTALL(TARGETS qgis_${testname} RUNTIME DESTINATION ${CMAKE_INSTALL_PREFIX})
ADD_TEST(qgis_${testname} ${CMAKE_INSTALL_PREFIX}/qgis_${testname})
ELSE (APPLE)
INSTALL(TARGETS qgis_${testname} RUNTIME DESTINATION ${CMAKE_INSTALL_PREFIX}/bin)
ADD_TEST(qgis_${testname} ${CMAKE_INSTALL_PREFIX}/bin/qgis_${testname})
ENDIF (APPLE)
ENDMACRO (ADD_QGIS_TEST)

Let's look a little more in detail at the individual lines. First we define the list of sources for our test. Since we have only one source file (following the methodology described above where class declaration and definition are in the same file) its a simple statement:

SET(qgis_${testname}_SRCS ${testsrc} ${util_SRCS})

我々のテストクラスは、Qtのメタオブジェクトコンパイラ（MOC）を介して実行する必要がありますので、それが実現するために何行かも提供する必要があります：

SET(qgis_${testname}_MOC_CPPS ${testsrc})
QT4_WRAP_CPP(qgis_${testname}_MOC_SRCS ${qgis_${testname}_MOC_CPPS})
ADD_CUSTOM_TARGET(qgis_${testname}moc ALL DEPENDS ${qgis_${testname}_MOC_SRCS})

Next we tell cmake that it must make an executable from the test class. Remember in the previous section on the last line of the class implementation we included the moc outputs directly into our test class, so that will give it (among other things) a main method so the class can be compiled as an executable:

ADD_EXECUTABLE(qgis_${testname} ${qgis_${testname}_SRCS})
ADD_DEPENDENCIES(qgis_${testname} qgis_${testname}moc)

Next we need to specify any library dependencies. At the moment, classes have been implemented with a catch-all QT_LIBRARIES dependency, but we will be working to replace that with the specific Qt libraries that each class needs only. Of course you also need to link to the relevant qgis libraries as required by your unit test.

TARGET_LINK_LIBRARIES(qgis_${testname} ${QT_LIBRARIES} qgis_core)

Next we tell cmake to install the tests to the same place as the qgis binaries itself. This is something we plan to remove in the future so that the tests can run directly from inside the source tree.

SET_TARGET_PROPERTIES(qgis_${testname}
PROPERTIES
# skip the full RPATH for the build tree
SKIP_BUILD_RPATHTRUE
# when building, use the install RPATH already
# (so it doesn't need to relink when installing)
BUILD_WITH_INSTALL_RPATH TRUE
# the RPATH to be used when installing
INSTALL_RPATH ${QGIS_LIB_DIR}
# add the automatically determined parts of the RPATH
# which point to directories outside the build tree to the install RPATH
INSTALL_RPATH_USE_LINK_PATH true)
IF (APPLE)
# For Mac OS X, the executable must be at the root of the bundle's executable folder
INSTALL(TARGETS qgis_${testname} RUNTIME DESTINATION ${CMAKE_INSTALL_PREFIX})
ADD_TEST(qgis_${testname} ${CMAKE_INSTALL_PREFIX}/qgis_${testname})
ELSE (APPLE)
INSTALL(TARGETS qgis_${testname} RUNTIME DESTINATION ${CMAKE_INSTALL_PREFIX}/bin)
ADD_TEST(qgis_${testname} ${CMAKE_INSTALL_PREFIX}/bin/qgis_${testname})
ENDIF (APPLE)

Finally the above uses ADD_TEST to register the test with cmake / ctest. Here is where the best magic happens - we register the class with ctest. If you recall in the overview we gave in the beginning of this section, we are using both QtTest and CTest together. To recap, QtTest adds a main method to your test unit and handles calling your test methods within the class. It also provides some macros like QVERIFY that you can use as to test for failure of the tests using conditions. The output from a QtTest unit test is an executable which you can run from the command line. However when you have a suite of tests and you want to run each executable in turn, and better yet integrate running tests into the build process, the CTest is what we use.

5.5. ユニットテストをビルドする

ユニットテストをビルドするには、cmakeの設定で ENABLE_TESTS=true を確認するだけが必要です。これを行うには2つの方法があります：

1. ccmake.. （またはウィンドウズの下では cmakesetup.. ）を実行し、対話的に ENABLE_TESTS フラグを ON に設定します。

2. cmakeのためのコマンドラインフラグを追加します、例えば cmake -DENABLE_TESTS=true ..

それ以外は、通常通りQGISをビルドするだけであり、またテストもビルドする必要があります。

5.6. テストを実行

テストを実行する最も簡単な方法は、通常のビルドプロセスの一部としてです：

make && make install && make test

The make test command will invoke CTest which will run each test that was registered using the ADD_TEST CMake directive described above. Typical output from make test will look like this:

Running tests...
Start processing tests
Test project /Users/tim/dev/cpp/qgis/build
## 13 Testing qgis_applicationtest***Exception: Other
## 23 Testing qgis_filewritertest *** Passed
## 33 Testing qgis_rasterlayertest*** Passed

## 0 tests passed, 3 tests failed out of 3

The following tests FAILED:
## 1- qgis_applicationtest (OTHER_FAULT)
Errors while running CTest
make: *** [test] Error 8

テストが失敗した場合は、それが失敗した理由をより密接に検討するctestコマンドを使用できます。実行したいテストするために正規表現を指定するためには -R オプションを、冗長な出力を得るためには -V オプションを使用してください：

$ ctest -R appl -V

Start processing tests
Test project /Users/tim/dev/cpp/qgis/build
Constructing a list of tests
Done constructing a list of tests
Changing directory into /Users/tim/dev/cpp/qgis/build/tests/src/core
## 13 Testing qgis_applicationtest
Test command: /Users/tim/dev/cpp/qgis/build/tests/src/core/qgis_applicationtest
********* Start testing of TestQgsApplication *********
Config: Using QTest library 4.3.0, Qt 4.3.0
PASS : TestQgsApplication::initTestCase()
PrefixPATH: /Users/tim/dev/cpp/qgis/build/tests/src/core/../
PluginPATH: /Users/tim/dev/cpp/qgis/build/tests/src/core/..//lib/qgis
PkgData PATH: /Users/tim/dev/cpp/qgis/build/tests/src/core/..//share/qgis
User DB PATH: /Users/tim/.qgis/qgis.db
PASS : TestQgsApplication::getPaths()
PrefixPATH: /Users/tim/dev/cpp/qgis/build/tests/src/core/../
PluginPATH: /Users/tim/dev/cpp/qgis/build/tests/src/core/..//lib/qgis
PkgData PATH: /Users/tim/dev/cpp/qgis/build/tests/src/core/..//share/qgis
User DB PATH: /Users/tim/.qgis/qgis.db
QDEBUG : TestQgsApplication::checkTheme() Checking if a theme icon exists:
QDEBUG : TestQgsApplication::checkTheme()
/Users/tim/dev/cpp/qgis/build/tests/src/core/..//share/qgis/themes/default//mIconProjectionDisabled.png
FAIL!: TestQgsApplication::checkTheme() '!myPixmap.isNull()' returned FALSE. ()
Loc: [/Users/tim/dev/cpp/qgis/tests/src/core/testqgsapplication.cpp(59)]
PASS : TestQgsApplication::cleanupTestCase()
Totals: 3 passed, 1 failed, 0 skipped
********* Finished testing of TestQgsApplication *********
-- Process completed
***Failed

## 0 tests passed, 1 tests failed out of 1

The following tests FAILED:
## 1- qgis_applicationtest (Failed)
Errors while running CTest

5.6.1. Running individual tests

C++ tests are ordinary applications. You can run them from the build folder like any executable.

$ ./output/bin/qgis_dxfexporttest

********* Start testing of TestQgsDxfExport *********
Config: Using QtTest library 5.12.5, Qt 5.12.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 9.2.1 20190827 (Red Hat 9.2.1-1))
PASS   : TestQgsDxfExport::initTestCase()
PASS   : TestQgsDxfExport::testPoints()
PASS   : TestQgsDxfExport::testLines()
...
Totals: 19 passed, 4 failed, 0 skipped, 0 blacklisted, 612ms
********* Finished testing of TestQgsDxfExport *********

These tests also take command line arguments. This makes it possible to run a specific subset of tests:

$ ./output/bin/qgis_dxfexporttest testPoints
********* Start testing of TestQgsDxfExport *********
Config: Using QtTest library 5.12.5, Qt 5.12.5 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 9.2.1 20190827 (Red Hat 9.2.1-1))
PASS   : TestQgsDxfExport::initTestCase()
PASS   : TestQgsDxfExport::testPoints()
PASS   : TestQgsDxfExport::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted, 272ms
********* Finished testing of TestQgsDxfExport *********

5.6.2. Debugging unit tests

C++ Tests

For C++ unit tests, QtCreator automatically adds run targets, so you can start them from the debugger.

If you go to Projects and there to the Build & Run --> Desktop Run tab, you can also specify command line parameters that will allow a subset of the tests to be run inside a .cpp file in the debugger.

Python Tests

It's also possible to start Python unit tests from QtCreator with GDB. For this, you need to go to Projects and choose Run under Build & Run. Then add a new Run configuration with the executable /usr/bin/python3 and the Command line arguments set to the path of the unit test python file, e.g. /home/user/dev/qgis/QGIS/tests/src/python/test_qgsattributeformeditorwidget.py.

Now also change the Run Environment and add 3 new variables:

Variable	Value
PYTHONPATH	[build]/output/python/:[build]/output/python/plugins:[source]/tests/src/python
QGIS_PREFIX_PATH	[build]/output
LD_LIBRARY_PATH	[build]/output/lib

Replace [build] with your build directory and [source] with your source directory.

5.6.3. Have fun

Well that concludes this section on writing unit tests in QGIS. We hope you will get into the habit of writing test to test new functionality and to check for regressions. Some aspects of the test system (in particular the CMakeLists.txt parts) are still being worked on so that the testing framework works in a truly platform independent way.