Testing test suite frameworks

Now that I am starting to add more features I was also beginning to feel the need to test the framework itself, but how do you test test suite frameworks?

I did so by adding a new layer on top of the framework: this layer runs test suites and check that their results are what is expected.

And because the vocabulary  to describe the XForms Unit suites is very experimental and likely to change a lot in the near feature I have kept the format used by this high level layer minimal and distinct from the description of the suites.

In fact the description of these tests is limited to a reference to the test suite to run and a list of expected results:

<test>
    <title>Sample suite</title>
    <suite href="../../suites/hello-world/suite.xml"/>
    <expected>
        <case>
            <assert expected="true"/>
        </case>
    </expected>
</test>

Extract from the tests for m:assertEqual assertions

These tests can then be run. The output format is currently pretty raw:

Screenshot of test results

Despite its crudeness, this tool is really powerful and very helpful for changes such as the update in the suite vocabulary committed a few minutes ago.

My next steps is to add assertions on other control properties (read/write status, validity and optionality).

Leave a Reply

Your email address will not be published. Required fields are marked *