A vocabulary to describe test suites

That leads on to the question, at what level should we test?

Stephen Cameron, 2013-08-22 06:54

Indeed and we have to remember that XForms is following MVC principles.

In an XForms form the model is embedded in an xf:model element, the view is elsewhere in document’s body and the controller is spread out within the entire document using separate elements names (so called actions) and the risk of confusion between these three components is limited.

In test cases on the contrary the pattern is to invoke actions from the controller to simulate real world situations and to perform checks within either the view or the controller and I think that the risk of confusion is much higher and that it will be useful to clearly separate these levels.

This separation cannot easily be done adding containers for the model, view or controller since a test suite will typically involved a number of individual actions and tests on each of these layers and I think that this is a very good use case for namespaces.

My proposal for the vocabulary that will describe the test suites is thus to use four different namespaces:

  • http://xformsunit.org/namespaces/suite/ for the suite itself
  • http://xformsunit.org/namespaces/model/ for the model
  • http://xformsunit.org/namespaces/view/ for the view
  • http://xformsunit.org/namespaces/controller/ for the controller

The simple suite described in the blog post which has introduced the project would then become:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="../../suite.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="../../suite.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<suite xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms"
    xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://xformsunit.org/namespaces/suite/"
    xmlns:m="http://xformsunit.org/namespaces/model/" xmlns:v="http://xformsunit.org/namespaces/view/"
    xmlns:c="http://xformsunit.org/namespaces/controller/">
    <form src="hello-world.xhtml"/>

    <!-- The test cases -->
    <case id="test-greetings">
        <title>Test that greetings are correctly set</title>
        <c:setvalue ref="instance('instance')/PersonGivenName">Eric</c:setvalue>
        <m:assertEqual>
            <m:actual ref="instance('instance')/Greetings"/>
            <m:expected>Hello Eric. We hope you like XForms!</m:expected>
            <m:message>The greetings should be the concatenation of "Hello ", the given name and ". We hope you like
                XForms!".</m:message>
        </m:assertEqual>
    </case>
</suite>

suite.xml

This one uses the controller to set a value and performs a test on the model level. And yes, you’ve see the <?xml-model?> right, there is a schema for this markup. The normative schema is written in Examplotron:

<?xml version="1.0" encoding="UTF-8"?>
<suite xmlns="http://xformsunit.org/namespaces/suite/" xmlns:m="http://xformsunit.org/namespaces/model/"
    xmlns:v="http://xformsunit.org/namespaces/view/" xmlns:c="http://xformsunit.org/namespaces/controller/"
    xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms"
    xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:eg="http://examplotron.org/0/">
    <!-- Reference to the form to test -->
    <form src="hello-world.xhtml"/>

    <!-- The test cases -->
    <case id="test-greetings" eg:occurs="*" eg:content="eg:interleave">
        <!-- Title -->
        <title>Test that the greetings are displayed and enabled.</title>
        <!-- Set a value (similar to XForms' setvalue action) -->
        <c:setvalue ref="instance('instance')/PersonGivenName" eg:occurs="*">Eric</c:setvalue>
        <!-- Check that an instance node is equal to its expected value -->
        <m:assertEqual eg:occurs="*">
            <!-- Actual value -->
            <m:actual ref="instance('instance')/Greetings"/>
            <!-- Expected -->
            <m:expected>Hello Eric. We hope you like XForms!</m:expected>
            <!-- Message to display when the test fails -->
            <m:message>The greetings should be the concatenation of "Hello ", the given name and ". We hope you like
                XForms!".</m:message>
        </m:assertEqual>
        <!-- Check if a control is enabled -->
        <v:assertEnabled eg:occurs="*">
            <!-- Identification of the control. Being an element leaves more flexibility to use other means than id/idref -->
            <v:control idref="greetings-control"/>
            <!-- Expected value (boolean) -->
            <v:expected>true</v:expected>
            <!-- Message to display if the test fails -->
            <v:message>The greetings control should be enabled at that point.</v:message>
        </v:assertEnabled>
    </case>
</suite>

suite.eg

Examplotron schemas are also document samples and this one performs an additional test on the view to check the status of a control.

This schema can be compiled into RELAX NG using either the XML or the compact syntax.

And now I need to implement the <v:assertEnabled/> which, given my recent interoperability misadventures, won’t be that easy for betterFORM and XSLTForms!

As always, your feedback is most welcome and needed.

10 thoughts on “A vocabulary to describe test suites

  1. In general namespaces are most useful to identify an entire format, so I’d lean to using a single namespace with distinct names, which would be just as clear but easier to process.

    1. Hi Liam,

      I must say I have mixed feelings about this usage of namespaces here.

      I am not a big fan of namespaces and I first thought I would use no namespace at all for this vocabulary since they would brought no obvious added value.

      Then there was this comment from Stephen Cameron wondering at which level should the tests be performed making me think we should clearly differentiate the elements dealing with the model, the view and the controller. And how do you differentiate categories of elements more clearly than putting them into different namespaces?

      Syntactically speaking, don’t you think we differentiate assertions dealing with the model or the view more clearly using <m:assertEqual> and <v:assertEqual> than <m-assertEqual>/<v-assertEqual> or <m.assertEqual>/<v.assertEqual>?

      Then, implementing that in XSLT, I ended up writing a template matching “c:*” and I have seen that as a legitimization of my choice: match=”c:*” seems like a much more readable way of matching all the elements dealing with the controller than match=”*[starts-with(local-name(), ‘c-‘)]”. Beside that this kind of proves that these elements have a different nature and deserve to be identified by their own namespace.

      That being said, I am still wondering if this worth the burden of namespaces…

  2. What is the burden of namespaces? There is a lot of talk about it right now but I have never understood it from an XML-application developers view. Please enlight me 🙂

    1. Johan,

      As an XML application developer I tend to avoid using namespaces when I don’t really need them because they add complexity for most of the common tasks (by order of importance):

      * They need to be properly declared in XSLT, XQuery, Schematron, RELAX NG, …
      * They are a source of common errors when writing schemas for XML Schema even for W3C members ;), see http://lists.w3.org/Archives/Public/public-forms-testsuite/2013Aug/0001.html !
      * They are kind of messy in SAX and DOM.
      * They are a nightmare for XPointer.

      That being said, the major issue is when you need to explain them to beginners. As a trainer I can say that it takes more time an efforts to explain them than it should!

      Eric

    1. Liam,

      I don’t think we should sacrifice neither learnability nor elegance, but that would be a long debate 😉 …

      More concretely, in that case would you consider <v.assertEqual> more “learnable” than <v:assertEqual>? Or should we be more verbose and write <view.assertEqual>?

      Eric

    1. For my part I think the no-namespace version is fine, and probably easier to work with. I do see your point about being able to use view:* rather than starts-with(local-name(), “view.”) although in both XQuery and XSLT, and in XPath 3, you could define a function isview($e) and hide the difference.

Leave a Reply

Your email address will not be published. Required fields are marked *