SOTA User's Guide - Version 1.0 |
SOTA supports static program analysis and dynamic program testing.
During the static program analysis the source code of the program is analyzed and as a result SOTA determines ten software metrics like the Cyclomatic complexity or the number of modified boundary-interior paths. Before the analysis SOTA needs to parse the source code.
The dynamic program test (from now on referred to as program test) is a structure-oriented and control flow-related program test forming the main part of SOTA. Depending on the specific test case SOTA determines nine code coverage metrics during program execution, such as branch coverage or multiple-condition coverage and provides a graphical representation of the results. Before the analysis, it is necessary to instrumentate the program that will be tested.
There are three basic ways to employ SOTA in a program test: the manual program test, in combination with an external testing system or integrated into an automatic testing system.
Durng the manual program test, the program is tested manually, i.e. started by hand, functions are executed, etc. This can be done by using a development environment like Eclipse in addition to SOTA, or with SOTA itself by integrating an Ant build file and a startup script.
The only difference when working with an external testing system is the method of testing. As with the manual test, SOTA controls pre- and postprocessing. However, for the program test itself a separate testing system is used, such as ATOSj. This results in a program sequence SOTA, ATOSj, SOTA - without any internal linking.
When working with an automatic test system SOTA can be embedded as a library (SOTA-ATM) allowing the system to use the non-GUI functionality of SOTA. Core functions of SOTA can either be called through command line parameters or a SotaATM instance based on a class included in the library.
The work with SOTA is divided into the preparatory, testing and evaluation phases.
Basically, the preparatory phase for the program test consists of reading the source code, determining the type of instrumentation and instrumentating the source files. For a manual program test or the use of an external testing system this is done via the graphical user interface. When using SOTA-ATM with an automatic testing system the desired behaviour is evoked by calling the appropriate library routines or starting SOTA-ATM via command line parameters.
During the testing phase the compilation, start of program and program test are executed. Compiling the instrumented source files is outside the remit of SOTA. However, the compilation can be initiated by integrating a corresponding Ant build script. If a fitting batch file is available, it is even possible to start the program with SOTA. Given that these two files are included in SOTA the manual program test can be carried out without using an external development environment. During the program test, the instrumentations that were added to the source code produce a log file containing the data necessary for a complete reconstruction of the program.
Finally, during the evaluation phase the original source code is restored, the log files are read and the coverage metrics calculated. SOTA enables the user to evaluate the results visually and export them as an HTML report.
phases |
tasks |
exploitation method |
||
manual program test |
with external testing system (ATOSj) |
integrated into automatic testing system |
||
preparatory phase |
reading and parsing source code |
SOTA |
SOTA |
SOTA-ATM |
configuration of instrumentation |
SOTA |
SOTA |
SOTA-ATM / configuration file |
|
instrumentation of source code |
SOTA |
SOTA |
SOTA-ATM |
|
testing phase |
compilation of source code |
external / with SOTA using a script |
external |
external |
start of program |
external / with SOTA using script |
external |
external |
|
program test |
manual |
external |
external |
|
evaluation phase |
restoration of original source code |
SOTA |
SOTA |
SOTA-ATM |
reading of log files and calculation of coverage metrics |
SOTA |
SOTA |
SOTA-ATM |
|
visualization of results |
SOTA |
SOTA |
none / SOTA (after the tests) |
|
export of report |
SOTA |
SOTA |
SOTA-ATM |