Test events - strategies to avoid repetition and lengthy test scripts

Overview

• Motivation
• Sense method calls to environment only where needed - Option::oSILENT
• Check only relevant recorded events - ClearAllStoredEvents
• Use external text files - CompareTextFiles
• Export text files from TestEvents - TransferEventsToTextFile
• Import text files to TestEvents for selective checks - ReadEventsFromTextFile

Motivation

Assuming you have a complex interaction of your test object with several objects within its environment. A good test will observe that all objects of the environment are called with correct methods and parameter values and in the correct call sequence.

But when you have verified within a test case that all these calls to the environment are correct you may have many additional test cases each focussing on some special (error) behaviour concerning only a single aspect/component within the environment. In those test cases it is no longer needed to again check in detail the interaction to all other components of the environment.

The following sections give advice to reduce the amount of repeated checks via a series of calls to TTB_EXP.

Sense method calls to environment only where needed - Option::oSILENT

Instrument the fake objects within your test environment only for those test cases where you really need to watch the calls to them.

If your test objects derive from TTB::SimulatedObjectBase you have the generic possibility to switch verbosity of each method on and off by setting Option::oSILENT. For more details see http://www.gerald-fahrnholz.eu/sw/display_contents.php?file_name=simulated_objects.htm#headLine_9.

Check only relevant recorded events - ClearAllStoredEvents

You can still record all interactions with environment objects. Within test case you simply verify only those interactions which are relevant. After the required checks delete all other recorded interactions via TheTestEvents()->ClearAllStoredEvents().

Example:

    TTB_INFO("Check all events with correct order");
    SimulateGenerationOfSomeTestEvents();
    TTB_EXP("Some event occured");
    TTB_EXP("Another event happened");
    TTB_EXP("This is the last event");

    TTB_INFO("Check only some events without check of order");
    SimulateGenerationOfSomeTestEvents();
    TTB_EXP_VAR("This is the last event");
    TTB_EXP_VAR("Some event occured");
    TTB::TheTestEvents()->ClearAllStoredEvents();
    // Remark: Not checked events would be detected by the
    // framework at end of test case

Tip
The recorded events are still visible within test protocol for analysis of error situations.

Use external text files - CompareTextFiles

Assume you can collect relevant test events somehow within a text file, then you can check even lengthy contents within a single line within your test script by calling method CompareTextFiles:

    TTB_INFO_S("\nComparing files, expecting identical contents and return value 'true'");
    TTB_CHECK_COND(CompareTextFiles(fullPathToTextFile_1, fullPathToTextFile_2));

    TTB_INFO_S("\nComparing different files, expecting error events and return value 'false'");
    TTB_CHECK_EQUAL(CompareTextFiles(fullPathToTextFile_1, fullPathToTextFile_3), false);
    TTB_EXP("CompareTextFiles: Difference detected in line 2");
    TTB_EXP("           fileA: >Line 2<");
    TTB_EXP("           fileB: >Line number 2<");
    TTB_EXP("                        ^    ^   ");

Export text files from TestEvents - TransferEventsToTextFile

Instead of checking recorded TestEvents with multiple calls to TTB_EXP you can export all not yet checked events to an external text file by calling method TransferEventsToTextFile. The generated text file can be checked by comparing it with a predefined pattern file:

    TTB_CMD(SimulateGenerationOfSomeTestEvents);
    TTB::TheTestEvents()->TransferEventsToTextFile("RecordedEvents.txt");
    
    TTB_INFO_S("\nNow check the file contents");
    TTB_CHECK_COND(CompareTextFiles(fullPathToPatternFile, TTB::GetRecordedFilePath("RecordedEvents.txt")));

Import text files to TestEvents for selective checks - ReadEventsFromTextFile

You can import a text file with collected test data into TestEvents. With use of macros TTB_EXP or TTB_EXP_VAR you can verify only the interesting lines and forget about the rest of the file contents:

    TTB_INFO_S("\nCheck all file contents");
    TTB::TheTestEvents()->ReadEventsFromTextFile(fullPathToTextFile_1);
    TTB_EXP("First line of text file");
    TTB_EXP("Line 2");
    TTB_EXP("Line 3");
    TTB_EXP("Last Line of text file");

    TTB_INFO_S("\nCheck only some file contents");
    TTB::TheTestEvents()->ReadEventsFromTextFile(fullPathToTextFile_1);
    TTB_EXP_VAR("Line 2");
    TTB_EXP_VAR("Line 3");
    TTB::TheTestEvents()->ClearAllStoredEvents();
    // Remark: Not checked events would be detected by the
    // framework at end of test case

It is even possible to treat XML files as simple text files to perform selective checking:

    TTB_INFO_S("\nYou can check a (well formed) XML file as a simple text file");
    TTB::TheTestEvents()->ReadEventsFromTextFile(xmlFilePath);
    TTB_EXP_VAR("    <Interior>");
    TTB_EXP_VAR("      <Something>Some text</Something>");
    TTB_EXP_VAR("      <date>28.09.09</date>");
    TTB_EXP_VAR("    </Interior>");
    TTB::TheTestEvents()->ClearAllStoredEvents();

For some cases this may be a simpler method to check XML contents than described in Comparing an xml tree with a pattern file.