1 Introduction
2 Overview
3 Installation and Start of Program
4 User Interface and Functionality
5 Files
6 Tutorials
7 Appendix

2 Overview

SOTA supports static program analysis and dynamic program testing.

During the static program analysis the source code of the program is analyzed and as a result SOTA determines ten software metrics like the Cyclomatic complexity or the number of modified boundary-interior paths. Before the analysis SOTA needs to parse the source code.

The dynamic program test (from now on referred to as program test) is a structure-oriented and control flow-related program test forming the main part of SOTA. Depending on the specific test case SOTA determines nine code coverage metrics during program execution, such as branch coverage or multiple-condition coverage and provides a graphical representation of the results. Before the analysis, it is necessary to instrumentate the program that will be tested.

There are three basic ways to employ SOTA in a program test: the manual program test, in combination with an external testing system or integrated into an automatic testing system.

Durng the manual program test, the program is tested manually, i.e. started by hand, functions are executed, etc. This can be done by using a development environment like Eclipse in addition to SOTA, or with SOTA itself by integrating an Ant build file and a startup script.

The only difference when working with an external testing system is the method of testing. As with the manual test, SOTA controls pre- and postprocessing. However, for the program test itself a separate testing system is used, such as ATOSj. This results in a program sequence SOTA, ATOSj, SOTA - without any internal linking.

When working with an automatic test system SOTA can be embedded as a library (SOTA-ATM) allowing the system to use the non-GUI functionality of SOTA. Core functions of SOTA can either be called through command line parameters or a SotaATM instance based on a class included in the library.

The work with SOTA is divided into the preparatory, testing and evaluation phases.

Basically, the preparatory phase for the program test consists of reading the source code, determining the type of instrumentation and instrumentating the source files. For a manual program test or the use of an external testing system this is done via the graphical user interface. When using SOTA-ATM with an automatic testing system the desired behaviour is evoked by calling the appropriate library routines or starting SOTA-ATM via command line parameters.

During the testing phase the compilation, start of program and program test are executed. Compiling the instrumented source files is outside the remit of SOTA. However, the compilation can be initiated by integrating a corresponding Ant build script. If a fitting batch file is available, it is even possible to start the program with SOTA. Given that these two files are included in SOTA the manual program test can be carried out without using an external development environment. During the program test, the instrumentations that were added to the source code produce a log file containing the data necessary for a complete reconstruction of the program.

Finally, during the evaluation phase the original source code is restored, the log files are read and the coverage metrics calculated. SOTA enables the user to evaluate the results visually and export them as an HTML report.

Program Test with SOTA

phases

tasks

exploitation method

manual program test

with external testing system (ATOSj)

integrated into automatic testing system

preparatory phase

reading and parsing source code

SOTA

SOTA

SOTA-ATM

configuration of instrumentation

SOTA

SOTA

SOTA-ATM / configuration file

instrumentation of source code

SOTA

SOTA

SOTA-ATM

testing phase

compilation of source code

external / with SOTA using a script

external

external

start of program

external / with SOTA using script

external

external

program test

manual

external

external

evaluation phase

restoration of original source code

SOTA

SOTA

SOTA-ATM

reading of log files and calculation of coverage metrics

SOTA

SOTA

SOTA-ATM

visualization of results

SOTA

SOTA

none / SOTA (after the tests)

export of report

SOTA

SOTA

SOTA-ATM