Test generation is an experimental feature for a subset of the language.

The test cases generator allows to generate testcases from a CommaSuite interface. For interfaces with input parameters, a parameters file is required. To enable test cases generation add the following to the .prj file, e.g. using content assist (ctrl + space):

import "Camera.interface"

Project Camera {
  Generate Test {
      // Test_IRetroCamera is the name of this task (can be anything).
      Test_IRetroCamera for interface ICamera {
          // Template to use, can be "java" or a path (from project root) to a custom Jinja template
          template: java
          // template: "my_own_template.j2"
		  output-file: 'Test.java'
	      max-depth: 1000
          // Path to parameters file if
		  params: 'interface/Camera.params'
      }
  }
}

To generate the tests, run the .prj (right click on the .prj file → Run As → Run generators). The output can be found in the src-gen/testcases directory.

Built-in Java template

The java template generates test cases based on JUnit. An example is included in CommaSuite and can be imported by clicking File (left-top) → New → Example → Test Generation Example → Next → Finish. Running the generator (right click on the .prj file → Run As → Run generators) will generate the test cases under src-gen/testcases.camera/TestCases.java (which can be executed by right click on TestCases.java → Run As → JUnit Test):

package test.ICamera;

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Assertions;

@SuppressWarnings("unused")
public class TestCases {
    private static final Adapter adapter = new Adapter();

    @BeforeAll
    public static void beforeAll() {
        adapter.beforeAll();
    }

    @BeforeEach
    public void beforeEach() {
        adapter.beforeEach();
    }

    @AfterEach
    public void afterEach() {
        adapter.afterEach();
    }

    @AfterAll
    public static void afterAll() {
        adapter.afterAll();
    }

    @Test
    public void testScenario1() throws InterruptedException {
        /*
         * Call command PowerOn()
         *
         * When calling command TakePicture() we should get:
         * - Return value: Ok
         *
         * Call command PowerOff()
         *
         * Call command PowerOn()
         *
         * When calling command TakePicture() we should get:
         * - Return value: Ok
         *
         * Call command PowerOff()
         *
         * Call command PowerOn()
         *
         * When calling command TakePicture() we should get:
         * - Return value: OutOfSpace
         *
         * When calling command SetZoom(3) we should get:
         * - Notification ZoomChanged with value 3
         * - Return value: Ok
         */

        var result0 = adapter.call("PowerOn", null);

        var result2 = adapter.call("TakePicture", "Ok");
        Assertions.assertEquals("Ok", result2.returnValue);

        var result4 = adapter.call("PowerOff", null);

        var result6 = adapter.call("PowerOn", null);

        var result8 = adapter.call("TakePicture", "Ok");
        Assertions.assertEquals("Ok", result8.returnValue);

        var result10 = adapter.call("PowerOff", null);

        var result12 = adapter.call("PowerOn", null);

        var result14 = adapter.call("TakePicture", "OutOfSpace");
        Assertions.assertEquals("OutOfSpace", result14.returnValue);

        var result16 = adapter.call("SetZoom", "Ok", 3);
        var result17 = adapter.waitForNotification("ZoomChanged", 3);
        Assertions.assertTrue(result17);
        Assertions.assertEquals("Ok", result16.returnValue);
    }

    ...
}

The adapter is responsible for doing the communication with the system under test (SUT). The (before|after)(All|Each) methods allow the adapter to do any setup/teardown of the SUT.

The testScenario* methods contain the test cases. Each test case starts with a human readable description followed by the test code. As can be seen above, methods are called and the return value (if any) is checked. Also any expected notifications are checked.

The adapter code contains system specific logic and is therefore manually written. The example can be found under src/test.interfaceCamera/Adapter.java. This adapter contains:

  • Classes to handle and check events generated by the SUT (AutoResetEvent, Event, EventHandler)

  • The call method to execute a command/signal on the SUT with a given set of parameters.

  • Before and after each call the beforeEvent and afterEvent methods are called. These method can be used to trigger any non-standard behavior on the interface (e.g. an error situation which would normally not occur).

The Test Generation Example also contains an example for components. The testcases can be found under src-gen/test.componentCamera/TestCases.java.

In general, the Java test generator assumes a file Adapter.java is available in a package with the same name as the generated package that contains TestCases.java, i.e., “test.[name of component or interface]”.

Moreover, the execution of generated JUnit tests requires a few preparation steps (that have already been performed for the Test Generation Example):

  • Ensure JUnit 5 is available :

    Right-click on project, select Build Path > Configure Build Path;
    select tab Libraries > Add Library > select JUnit5
  • Define src-gen as source folder:

    Right-click on project, select Build Path > Configure Build Path;
    select tab Source > Add Folder > select src-gen