1 Introduction
2 Overview
3 Installation and Start of Program
4 User Interface and Functionality
5 Files
6.1 Overview
6.2 Project File
6.3 Report Files
6 Tutorials
7 Appendix

5 Files

5.1 Overview

SOTA root directory

The SOTA root directory contains the SOTA system, as e.g. the boot file SOTA.exe and the Eclipse-Rich-Client- Platform installed with SOTA, as well as further special SOTA files and the project files.

sota.log, sota_<date>_<index>.log

Here SOTA logs all its activities.

language.spec

The specification file for all supported languages.

ASCLogger.jar

The logging component for Java test programs must be included in the test programm and demands an initalization file named ASCLogger.ini (see below) during the test.

<projectname>.project

For each project the general project data are noted here.

SOTA-ATM.jar

The automatic test module of SOTA enabling the testing funcionality to be applicable without GUI. SOTA-ATM.jar is an executable jar file which can also be imported as a program library. This allows access via command line (scripts) or software. See the tutorial for more information.

Root directory of the test program (Project directory)

The root directory of the project contains (possibly in a subdirectory) the sources of the test program and at the same location their backups that are generated by SOTA on creating the project. SOTA puts all automatically generated report files at this location unless the user activated an inquiry via data selection dialog in the preferences.

(\src\) <xyz>.java

The source files of the test projects are (partially) instrumented after starting the test.

(\src\) <xyz>.java.backup

The backup of the original source files is created before starting the test for all source files that are not instrumented.

(\bin\)<xyz>.class

The compiled class files which may be instrumented depending on the state of the source files.

report.html, report_<datum>_<index>.html

The report files generated by SOTA.

<antbuildfilename>.xml

An possibly existant Ant buildfile allowing the compilation of the test program in SOTA. For Eclipse projects it can be exported via File -> Export -> Ant Buildfile.

<runscriptname>.bat

A batch file for booting the test program which allows, in case it is included in SOTA, the manual program test in SOTA in correspondance with the Ant build script.

Execution directory of the test program (Execution directory)

The test program is booted in the execution directory. In most cases this directory corresponds to the root directory of the program. An exception may e.g. be the testing of an Eclipse-RCP application in Eclipse, since then the execution direction is the root directory of the Rich-Client-Platform, i.e. in general: ..\eclipse\ .

ASCLogger.ini

On starting the test this initialization file for the logging component is copied into this directory where it is accessed by the class ASCLogger.jar in order to create the log file.

<testname>.log

The log files created by ASCLogger.ini.

5.2 Project File

To use SOTA it is not necessary to adjust the project files. However, should the user wish to adopt SOTA-ATM as an automatic test module, it may be beneficial to create or change the project files manually or with a script in order to gain comprehensive control over the test.

The project files used by SOTA are simple XML files which contain the project specific information as values of individual entities. Their format is specified by scheme definition project.dtd.

The project file defines a project which is at least defined by the following values:

Name

The name of the project which must be identical with the filename without ending.

Language

The language of the project which must be listed in the language specification.

Prefix

The prefix which enables marking variables introduced by SOTA during the instrumentation process, this avoids name collisions.

BackupExtension

The ending used for backup files generated by SOTA.

ProjectDir

The root directory of the project.

ExecDir

The execution directory of the project.

SourceFiles

A list of source files (as SourceFile) belonging to the project.

The following values may be used optionally to enable special features of the program:

AntLocation

The path to the Apache-Ant installation. This is necessary to compile the project automatically.

AntBuildFile

The Ant buildfile, which enables the compilation of the buildfile.

RunScript

The script used for booting the project.

ISchemes

A list of instrumentation schemes (as IScheme) which enable the variable instrumentation of the project. An IScheme consists of a name and a mapping of each structure of the project (data, class, function) to an instrumentation level, and, optionally, a description of the IScheme.

The following code is an exemplary project file for the project Digit. It contains the defintion for an IScheme which instruments the only source code file according to level 1 and the method 'evalutateDigitSequence' according to level 2.


         <Project>
            <Name>Digit</Name>
            <Language>Java</Language>
            <Prefix>asc</Prefix>
            <BackupExtension>backup</BackupExtension>
            <ProjectDir>D:\Development\workspace\Digit</ProjectDir>
            <ExecDir>D:\Development\workspace\Digit</ExecDir>
            <SourceFiles>
               <File>D:\Development\workspace\Ziffer\src\Digit.java</File>
            </SourceFiles>
            <ISchemes>
               <IScheme>
                  <Name>Scheme F</Name>
                  <Description>Digit.java Lvl1, evaluateDigitSeqeuence Lvl2</Description>
                  <Level1>
                     <Item>Digit.java</Item>
                  </Level1>
                  <Level2>
                     <Item>Digit.java:Digit::evaluateDigitSequence(String)</Item>
                  </Level2>
               </IScheme>
            </ISchemes>
         </Project>

5.3 Report File

The following report is a sample report for the project Digit for the input values '..', '.2', '1', '1.1' and without input value. (for more information on the program see 6.1.1.)

Each report file begins with the name of the project and the date of creation. Following that is a list of all coverage metrics including the values for this project obtained during the test, as well as individual static metrics. The other tables of the report file will only be created, if the user selected the corresponding options in the preferences. The standard settings cause the output of all tables.

If Show Testlogs is selected, a list with all test files used for this report follows. The corresponding ISchemes, as well as their desciptions, are also included. Activating Use all tests for report causes all imported test logs to be used for the report and to be listed here.

The option Show Classes generated a table which lists all classes of the project in addition to the overall project and their individual coverage metrics which may be highlighted in color according to their values (cf. Preferences).

Show Functions causes a table for each class to appear after the item Detailed Coverage. The tables contain the coverage metrics of the classes themselves and all their functions. A link from each class in the table Coverage of Classes refers to the corresponding list of their functions in the section Detailed Coverage.

A comprehensive sample report for the project HUSemOrg is included in the user documentation.

SOTA Coverage Report


Project: Digit

created: 2009-03-23 13:54:35

Function Entry-Exit Coverage (FEEC)

100,00%

# Files

1

Statement Coverage (C0)

100,00%

# Classes (TopLevel- + inner Classes)

1 (1 + 0)

Decision Coverage (C1)

100,00%

# Functions

2

Condition Coverage (C2)

95,00%

# Lines

35

Minimal Multiple Decision Coverage (MMDC)

93,75%

# Statements

19

Modified Condition Decision Coverage (MCDC)

50,00%

# Conditions

16

Multiple Condition Coverage (C3)

46,43%

Modified Boundary-Interior Path Coverage (ModBI)

26,09%

Boundary-Interior Path Coverage (BI)

26,09%


Tests

test ..

Level 2 instrumentation

test .2

Level 2 instrumentation

test 1

Level 2 instrumentation

test 1.1

Level 2 instrumentation

test empty

Level 2 instrumentation


Coverage of Classes


top

FEEC

C0

C1

C2

MMDC

MCDC

C3

ModBI

BI

Project Digit

100,00

100,00

100,00

95,00

93,75

50,00

46,43

26,09

13,95

Class Digit

100,00

100,00

100,00

95,00

93,75

50,00

46,43

26,09

13,95


Detailed Coverage


top

FEEC

C0

C1

C2

MMDC

MCDC

C3

ModBI

BI

Class Digit

100,00

100,00

100,00

95,00

93,75

50,00

46,43

26,09

13,95

- main (String[])

100,00

100,00

---

---

---

---

---

100,00

100,00

- evaluateDigitSequence (String)

100,00

100,00

100,00

95,00

93,75

50,00

46,43

22,73

11,90