SOTA User's Guide - Version 1.0 |
The SOTA root directory contains the SOTA system, as e.g. the boot file SOTA.exe and the Eclipse-Rich-Client- Platform installed with SOTA, as well as further special SOTA files and the project files.
sota.log, sota_<date>_<index>.log |
Here SOTA logs all its activities. |
language.spec |
The specification file for all supported languages. |
ASCLogger.jar |
The logging component for Java test programs must be included in the test programm and demands an initalization file named ASCLogger.ini (see below) during the test. |
<projectname>.project |
For each project the general project data are noted here. |
SOTA-ATM.jar |
The automatic test module of SOTA enabling the testing funcionality to be applicable without GUI. SOTA-ATM.jar is an executable jar file which can also be imported as a program library. This allows access via command line (scripts) or software. See the tutorial for more information. |
The root directory of the project contains (possibly in a subdirectory) the sources of the test program and at the same location their backups that are generated by SOTA on creating the project. SOTA puts all automatically generated report files at this location unless the user activated an inquiry via data selection dialog in the preferences.
(\src\) <xyz>.java |
The source files of the test projects are (partially) instrumented after starting the test. |
(\src\) <xyz>.java.backup |
The backup of the original source files is created before starting the test for all source files that are not instrumented. |
(\bin\)<xyz>.class |
The compiled class files which may be instrumented depending on the state of the source files. |
report.html, report_<datum>_<index>.html |
The report files generated by SOTA. |
<antbuildfilename>.xml |
An possibly existant Ant buildfile allowing the compilation of the test program in SOTA. For Eclipse projects it can be exported via File -> Export -> Ant Buildfile. |
<runscriptname>.bat |
A batch file for booting the test program which allows, in case it is included in SOTA, the manual program test in SOTA in correspondance with the Ant build script. |
The test program is booted in the execution directory. In most cases this directory corresponds to the root directory of the program. An exception may e.g. be the testing of an Eclipse-RCP application in Eclipse, since then the execution direction is the root directory of the Rich-Client-Platform, i.e. in general: ..\eclipse\ .
ASCLogger.ini |
On starting the test this initialization file for the logging component is copied into this directory where it is accessed by the class ASCLogger.jar in order to create the log file. |
<testname>.log |
The log files created by ASCLogger.ini. |
To use SOTA it is not necessary to adjust the project files. However, should the user wish to adopt SOTA-ATM as an automatic test module, it may be beneficial to create or change the project files manually or with a script in order to gain comprehensive control over the test.
The project files used by SOTA are simple XML files which contain the project specific information as values of individual entities. Their format is specified by scheme definition project.dtd.
The project file defines a project which is at least defined by the following values:
Name |
The name of the project which must be identical with the filename without ending. |
Language |
The language of the project which must be listed in the language specification. |
Prefix |
The prefix which enables marking variables introduced by SOTA during the instrumentation process, this avoids name collisions. |
BackupExtension |
The ending used for backup files generated by SOTA. |
ProjectDir |
The root directory of the project. |
ExecDir |
The execution directory of the project. |
SourceFiles |
A list of source files (as SourceFile) belonging to the project. |
The following values may be used optionally to enable special features of the program:
AntLocation |
The path to the Apache-Ant installation. This is necessary to compile the project automatically. |
AntBuildFile |
The Ant buildfile, which enables the compilation of the buildfile. |
RunScript |
The script used for booting the project. |
ISchemes |
A list of instrumentation schemes (as IScheme) which enable the variable instrumentation of the project. An IScheme consists of a name and a mapping of each structure of the project (data, class, function) to an instrumentation level, and, optionally, a description of the IScheme. |
The following code is an exemplary project file for the project Digit. It contains the defintion for an IScheme which instruments the only source code file according to level 1 and the method 'evalutateDigitSequence' according to level 2.
<Project>
<Name>Digit</Name>
<Language>Java</Language>
<Prefix>asc</Prefix>
<BackupExtension>backup</BackupExtension>
<ProjectDir>D:\Development\workspace\Digit</ProjectDir>
<ExecDir>D:\Development\workspace\Digit</ExecDir>
<SourceFiles>
<File>D:\Development\workspace\Ziffer\src\Digit.java</File>
</SourceFiles>
<ISchemes>
<IScheme>
<Name>Scheme F</Name>
<Description>Digit.java Lvl1, evaluateDigitSeqeuence Lvl2</Description>
<Level1>
<Item>Digit.java</Item>
</Level1>
<Level2>
<Item>Digit.java:Digit::evaluateDigitSequence(String)</Item>
</Level2>
</IScheme>
</ISchemes>
</Project>
The following report is a sample report for the project Digit for the input values '..', '.2', '1', '1.1' and without input value. (for more information on the program see 6.1.1.)
Each report file begins with the name of the project and the date of creation. Following that is a list of all coverage metrics including the values for this project obtained during the test, as well as individual static metrics. The other tables of the report file will only be created, if the user selected the corresponding options in the preferences. The standard settings cause the output of all tables.
If Show Testlogs is selected, a list with all test files used for this report follows. The corresponding ISchemes, as well as their desciptions, are also included. Activating Use all tests for report causes all imported test logs to be used for the report and to be listed here.
The option Show Classes generated a table which lists all classes of the project in addition to the overall project and their individual coverage metrics which may be highlighted in color according to their values (cf. Preferences).
Show Functions causes a table for each class to appear after the item Detailed Coverage. The tables contain the coverage metrics of the classes themselves and all their functions. A link from each class in the table Coverage of Classes refers to the corresponding list of their functions in the section Detailed Coverage.
A comprehensive sample report for the project HUSemOrg is included in the user documentation.
SOTA Coverage Report |
created: 2009-03-23 13:54:35
Function Entry-Exit Coverage (FEEC) |
100,00% |
# Files |
1 |
|
Statement Coverage (C0) |
100,00% |
# Classes (TopLevel- + inner Classes) |
1 (1 + 0) |
|
Decision Coverage (C1) |
100,00% |
# Functions |
2 | |
Condition Coverage (C2) |
95,00% |
# Lines |
35 | |
Minimal Multiple Decision Coverage (MMDC) |
93,75% |
# Statements |
19 | |
Modified Condition Decision Coverage (MCDC) |
50,00% |
# Conditions |
16 |
|
Multiple Condition Coverage (C3) |
46,43% |
|
| |
Modified Boundary-Interior Path Coverage (ModBI) |
26,09% |
|
|
|
Boundary-Interior Path Coverage (BI) |
26,09% |
|
|
test .. |
|
Level 2 instrumentation |
|
test .2 |
|
Level 2 instrumentation |
|
test 1 |
|
Level 2 instrumentation |
|
test 1.1 |
|
Level 2 instrumentation |
|
test empty |
|
Level 2 instrumentation |
FEEC |
C0 |
C1 |
C2 |
MMDC |
MCDC |
C3 |
ModBI |
BI |
|
Project Digit | 100,00 |
100,00 |
100,00 |
95,00 |
93,75 |
50,00 |
46,43 |
26,09 |
13,95 |
100,00 |
100,00 |
100,00 |
95,00 |
93,75 |
50,00 |
46,43 |
26,09 |
13,95 |
FEEC |
C0 |
C1 |
C2 |
MMDC |
MCDC |
C3 |
ModBI |
BI |
|
100,00 |
100,00 |
100,00 |
95,00 |
93,75 |
50,00 |
46,43 |
26,09 |
13,95 |
|
- main (String[]) | 100,00 |
100,00 |
--- |
--- |
--- |
--- |
--- |
100,00 |
100,00 |
- evaluateDigitSequence (String) | 100,00 |
100,00 |
100,00 |
95,00 |
93,75 |
50,00 |
46,43 |
22,73 |
11,90 |