Ever more functionality is demanded by end-customers from the software-intensive systems they use. At the
same time their expectations with respect to the correct operation, safety and security of these systems
have become higher than ever. Severe system failures can lead to significant damage or even loss of life,
while successful cyber intrusions can destroy their reputation. As such, breaches in both safety and
security can have a significant adverse impact on business and reputation. Due to the dramatic increase in
the complexity of the software itself, the intricate interaction modes between the software and the external
world, and the sheer magnitude of the customisability of the software, these systems have become
increasingly difficult to develop and verify by traditional development processes and testing methods. The
ATAC project took on the gauntlet to resolve such challenges by researching, evaluating and rolling out a
number of methodologies, associated processes and tools to efficiently and automatically verify complex and
highly configurable software-intensive systems.
ATAC’s overarching goal was always to push the functional, safety and security requirements coverage envelope
while minimising the number of test cases required to achieve the necessary degree of coverage and
confidence in the software, hence greatly reducing the required testing effort for both the fully automated
and remaining manual test cases. To ensure that the researched solutions and tools solve real industry needs
and to maximise their potential application in other settings, all R&I activities were anchored around a
jointly defined set of industrial case studies in different application domains. During the project, a
flexible and heterogeneous portfolio of different tools and methodologies was developed to cover the needs
of these different application domains as it was soon recognised that one-size-fits all approach would be
less than optimal.
During ATAC, Barco’s Collaborative Media Team developed a framework that provided better support for tracing
system requirements down to detailed requirements, linking them with corresponding test-cases on all levels
of the V-model. This framework reduced the software verification and validation effort by 20% and has since
been rolled out company-wide thanks to this success.
Bittium Wireless developed an automated VoIP network test tool in ATAC and introduced a production testing
platform for a Bittium product family, both of which Bittium has continued to develop after the project’s
end in 2014. Today, the automated VoIP network test tool is used in Bittium as a standard testing framework
in all VoIP based products and the company automated more than 90% of its functional testing for VoIP based
products resulting in significant verification cost savings.
Bombardier’s prototype tool developed in ATAC minimised the number of test vectors required to demonstrate
sufficient code coverage, massively decreasing the time needed by testers to demonstrate these coverage
objectives. In addition, the tool gave a figure on the achieved code coverage by testing when the
automatically tool-generated test vectors were used. Maximatecc collaborated closely with Bombardier and
ATAC has made it possible to develop their products and services for the simulation of embedded systems and
applications, especially their SimTecc simulation platform. The flexibility of SimTecc is that it can be
applied both at component level for module testing and at application level for testing complete machines
and vehicles, e.g. a complete train.
The ATAC results have enabled Valmet Automation to run over 1500 automated tests in less than one hour,
where manual testing would take days and be much more error-prone. In ATAC, Ericsson automated millions of
test cases that are executed daily in different test systems, and test code is now delivered together with
software code and executed with every change to the system.
ATAC has been a key ingredient in the formation and establishment of the Software Testing Laboratory (STL)
research group at Mälardalen University, one of the largest of its kind in Sweden. Kaunas University of
Technology founded one of the largest mobile application testing laboratories in the Baltic States.