Tutorial: Using our Tools for Model Driven Language Development

written by Andreas Blunk, June 2007

This tutorial shows how our language modelling tools can be used together to create the structure and behaviour of a computer language. The demonstration is based on an example language for modelling and executing composite state automatons. Before we start, you should download the complete language definition from here or check it out from SVN (project hub.sam.stateautomaton). It contains an Eclipse project that includes all the necessary resources we are using in this tutorial. A working Eclipse 3.2 installation is required (download and install from here). When you have downloaded the archive, run Eclipse, create a new general project and import the archive into that project. There are other tools and Eclipse plugins needed during the tutorial, but you can install them later. When you have finished setting up the project, you may proceed with the tutorial. Don't worry when you have build problems at this point, we will solve them later on.

We begin with an introduction into our language modelling method and then construct the example language by using the tools that we have build for this method.

Introduction

Computer languages consist of different aspects like structure, static constraints, semantics and notation. Like any other piece of software, they can be modelled. Language modelling reduces language development to modelling each language aspect, so that tools and specifications can automatically be derived.


the different language aspects		  

Structure models of the language concepts are the basis for all other language aspects. They define the language's abstract syntax in a language meta-model (they define the models the user can create). All other language aspects are based on the language meta-model. They provide meta-model elements with additional information that is needed to describe them, e.g. notation and semantic.

For modelling structure, we use the CMOF language as a meta-meta-model. It allows object-oriented modelling and provides enhanced features like subsetting and redefinition that allow a flexible design and embrace reuse of common language concepts.

But structure is just one aspect of a language. Many languages are more than just static structure. They describe dynamic processes. If we want to execute, interpret or simulate models in these languages, we have to define their execution semantics. Our approach is to define the language syntax (basic concepts) first. Then extend it with runtime structures that define additional data structures where runtime information is saved. This information allows execution of different runtime instances based on the same syntax model. We then augment the model with operations and apply behaviour descriptions to them that describe state changes in a model instance. Different possibilities exist to describe behaviour, e.g. pure Java code or activity diagrams from UML.

In order to understand this tutorial, you should be familiar with our CMOF-based programming framework A MOF 2.0 for Java. Further knowledge of our operational approach to define execution semantics may be helpful. Please refer to the following articles and papers for detailed information:

Developing an Example Language

We decided to create a language for modelling and executing composite state automatons. The reason for this choice is that composite state automatons are small enough to present them in a tutorial and yet complicated enough to explain most of the features in our tools. We first describe the process for creating the language structure and then continue with the definition of the execution semantics.

The basic concepts of the language are described in a syntax meta-model. These are: automaton, transition and state with a special initial state and final state. The next figure shows how a model in the language could look like.

syntax

Based on the language concepts, the syntax model should reflect the following rules: An automaton has a name. It may have many states and many transitions. There has to be one intial state and one or more final states. Each state has a name and outgoing as well as incoming transitions. Transitions describe state changes by specifying on which input token a transition can fire. A transition must have one source and one target state. Later, we execute an automaton on a sequence of input tokens, but this is part of the semantics.

Furthermore a state may contain a sub-automaton. This is a language feature that can be compared to types in computer languages. We define an automaton (the type) and use it as a sub-automaton (instances of the type) in many different states (in this case: composite states). It must be possible for each sub-automaton to reside in a different state at runtime. Thus extra runtime information is needed to execute the language. This language feature can be compared to an object as an instance of class. Each object has its own identity and its own state. Whenever we have such a type concept in a language, we need runtime structures that deal with runtime information. Composite state automatons are a good example to show this language aspect here. But lets concentrate on building the syntax model first and then go over to runtime structures.

Modelling Structure

We can model all parts of the language structure with the UML modelling tool MagicDraw UML. We support loading and saving MagicDraw models within our modelling tools. This allows us to use MagicDraw UML as the basis for modelling structure. But we are not dependend on it, you can use any UML tool that can load and save a model in XMI format version 2.0. However, further explanation will refer to MagicDraw UML, so you should get it now and then proceed.

Syntax

We first model the language syntax with respect to the basic concepts we identified before. As modelling with MagicDraw UML is straightforward, we instantly present the syntax model in the figure below. You may open the model in MagicDraw UML. It is named StateAutomaton.syntax.mdxml and can be found in the resources-directory. But be aware: you will see a much more complex model in this file. We are going to evolve the model below to the more complex one during the tutorial.

Figure: Syntax Meta-model in MagicDraw

syntax

Static Constraints

The syntax meta-model defines the structure of models. But in some situations this information is not restrictive enough to prevent incorrect models. E.g. we would say that a composite state automaton model is incorrect if its initial state has no outgoing transition. From a structural view this would be perfectly correct because a state is not forced to have an outgoing transition, but from a semantic view such a model is wrong. To prevent such models, we can further restrict them by applying static constraints to meta-model classes. These constraints may be checked at some later point, e.g. when a model is loaded from an XMI file or before it is executed. This language aspect is known as the static analysis of languages. For the specification of static constraints, we attach OCL invariants to meta-model classes.

In the state automaton example we added the following OCL invariant (named automaton-invariant) to the meta-model class Automaton:

     self.state->size() > 0 and initialState->size() = 1 and finalState->size() > 0 and initialState.outgoing->size() = 1

You may add more static constraints with MagicDraw UML. You just have to open a class specification and then create a new constraint as shown in the figure below. You have to choose "OCL 2.0" as the constraint's language. We will show you later how to check these constraints.

Runtime Structure

Model execution requires additional runtime information (e.g. the current state of an automaton). Such runtime information could be stored in the model itself. But this approach has several flaws. One problem is that we will lose the model's initial state if we execute it. An even stronger argument is that during model execution we often need several runtime representations of the same model element. E.g. in our language, each automaton has to remember its current state. When we reuse automatons as sub-automatons in composite states, each runtime instance of a sub-automaton can reside in a different state.

Therefore we need a runtime element for automatons in our example language. We create a class AutomatonRuntime and model the relationship to class Automaton as a runtime-representation-of relationship with an UML realisation arrow. Instances of AutomatonRuntime represent running automatons with distinct runtime information. We then establish an association from class AutomatonRuntime to class State with the association end named currentState. This allows us to remember the current state in each automaton runtime instance.

However, saving just the current state is not enough runtime information for composite state automatons. We also have to save runtime instances of sub-automatons for each composite state of the owning automaton. So when we enter a composite state A, then there is a runtime instance of the corresponding sub-automaton of A that is in some inner state. At some point, we will leave the state A and when we come back to A, the same runtime instance auf A's sub-automaton must still be present. A itself is part of an owning automaton whose runtime instance represents the current state of execution. Besides the current state information, we now also save instances of AutomatonRuntime for every composite state. This is a reflexive association in class AutomatonRuntime that can be modelled with a qualified access to one association end named compositeState. The qualifier is a composite state of type State.

Figure: Relations between syntax and runtime classes and the meta layers. M3 is CMOF, M2 is the language meta-model and M1 is a user model. Unlabeled UML realisation arrows represent instance-of relationships between meta layers.

But how are runtime classes instantiated and who manages the connection to syntax classes? Well, there has to be some kind of modelling framework that takes responsibility. We present our modelling framework in the next section. When we load a model with runtime-representation-of relationships and instruct the framework to look for them, then some magic happens and additional operations are added to syntax and runtime classes. E.g., the framework will generate an operation metaCreateAutomatonRuntime for class Automaton and an operation metaDelete for class AutomatonRuntime. Implementations of these operations are supplied by the modelling framework. So in a semantic description we can invoke these operations and the modelling framework takes care of the connection between syntax and runtime instances.

Figure: representation of syntax and runtime classes in the modelling framework

Figure: syntax meta-model extended with runtime information

complete structure model

Programming with CMOF-based Models

Our language meta-model is based on CMOF. In order to work with CMOF-based models, we developed a programming framework: A MOF 2.0 for Java (abbreviated AMOF2). It allows storing and managing meta-models and models. A binary version of AMOF2 is already supplied with the example language (located in resources/lib/aMOF2.0forJava.jar). Our language meta-model has to be in a format that AMOF2 can understand. It currently understands XMI version 2.0 and file formats of several proprietary UML modelling tools. One of them is MagicDraw UML which we have already used to model the language structure. Programming with the model in Java requires the generation of a model repository with AMOF2.

Repository Generation

The generation of repositories is a general task that has to be repeated everytime meta-models change. Therefore we supply several Ant tasks with AMOF2. We can use them in an Ant file to generate the repository code.

Figure: repository-build.xml

    1 <project name="StateAutomaton" basedir="." default="generate-repository">
    2 
    3     <property name="src-dir" value="${basedir}/src"/>
    4     <property name="gen-src-dir" value="${basedir}/generated-src"/>
    5     <property name="bin-dir" value="${basedir}/bin"/>
    6 
    7     <path id="classpath">
    8         <pathelement path="${bin-dir}"/>
    9         <fileset dir="${basedir}/resources/lib">
   10             <include name="**/*.jar"/>
   11         </fileset>
   12     </path>
   13 
   14     <target name="init">
   15         <mkdir dir="${bin-dir}"/>
   16         <typedef name="package" classname="hub.sam.mof.ant.Package" classpathref="classpath"/>
   17         <taskdef name="generatecode" classname="hub.sam.mof.ant.GenerateCode" classpathref="classpath"/>

Here we declare new Ant tasks from AMOF2 which we use in subsequent targets.

   18     </target>
   19 
   20     <target name="clean">
   21         <delete dir="${bin-dir}"/>
   22         <delete includeemptydirs="true">
   23           <fileset dir="${gen-src-dir}" includes="**/*"/>
   24         </delete>
   25     </target>
   26 
   27     <target name="generate-repository" depends="clean,init">
   28         <generatecode src="resources/StateAutomaton.syntax.mdxml" md="true" destDir="./generated-src" instances="true">

This task generates the repository code. The parameters have the following meaning. 

   29             <package name="model" javaPackagePrefix="hub.sam.stateautomaton"/>

The meta-model is located in the CMOF package "model". We add a prefix to the model's package so that code will be generated in Java package "hub.sam.stateautomaton.model" instead of just "model".

   30         </generatecode>
   31     </target>
   32 
   33 </project>

Now its time for you to become active! You should now generate the state automaton repository by executing the generate-repository target in the file repository-build.xml.

Creating a User Model

At this point we are able to create a model in the state automaton language (as instance of the language meta-model). This can be compared to a program in traditional computer languages. For demonstration purposes the model has to use composite states. Therefore we create a model that consists of an automaton X with composite states A and B and both states contain a sub-automaton Y. The automaton Y has two (simple) states C and D. Additionally all automatons have initial and final states and transitions between them (see figure below).

Figure: example state automaton model

We created this model by programming Java code (located in class hub.sam.stateautomaton.StateAutomaton, method createLargeTestModel). If we really wanted to use our language, we would have to build a graphical editor. Another possibility would be to extend the language meta-model with additional information and then generate a graphical editor. But we do not have this technology at the moment :)

The next section explains how the automaton X can be executed for a sequence of input tokens like "dbdecacf".

Execution Semantics

Introduction

The first question is: What does it mean to execute a model written in our language?
The answer is different from language to language. In this case, executing a model means: consume a sequence of input tokens in a dedicated automaton by transitioning between the automaton's states. Each firing of a transition changes the automaton's current state. If there is no transition in the current state and the current state is a composite state, then consumation is forwared to the sub-automaton. The token is disgarded, if it cannot be consumed at all. This explanation is referred to as the execution semantic of this language.

The next question is: How do we model execution semantics?
Our approch is to use operational semantics. We augment a meta-model with operations and then add semantic descriptions to them. We can describe semantics by programming Java code, modelling behaviour with UML like activities or writing query expressions with OCL.

For our example language we add the following operations:

Figure: complete structure meta-model

Now we present examples for the different implementations of the meta-model's operations: OCL queries, Java and Activities.

OCL Queries

The operation getEnabledTransition from our example language is a query operation. But how do we attach an OCL expression to an operation with MagicDraw UML?

First, we have to open the specification of the operation and then check that we see all properties.

In the properties page the operation has to be marked as "isQuery".

We then look for the body condition property and expand it on the right.

A dialog opens where we can enter the OCL expression. We have to select "OCL 2.0" as the language.

The OCL query is then added as an owned rule to this operations properties.

Java Code

AMOF2 lets us define custom implementations of a model's operations in a Java class named <meta-model-class>Custom, placed in the same package as the meta-model class. Java implementations for all the operations above are supplied with the Eclipse project. We present two of them here and you may look at the others yourself. The two operations are Automaton::run and AutomatonRuntime::initialise.

Figure: source code AutomatonCustom.java

    1 package hub.sam.stateautomaton.model;
    2 
    3 public class AutomatonCustom extends AutomatonDlg {
    4 
    5     @Override
    6     public void run(java.lang.String input) {
    7         AutomatonRuntime runtime = self.metaCreateAutomatonRuntime();
    8         runtime.initialise();
    9 
   10         for (int i=0; i < input.length(); i++) {
   11             java.lang.String token = input.substring(i, i+1);
   12             boolean consumed = runtime.consume(token);
   13             if (!consumed) {
   14                 // ignore token
   15                 System.out.println("ignoring token '" + token + "'");
   16             }
   17         }
   18 
   19         runtime.destroy();
   20     }
   21 
   22 }

Figure: source code AutomatonRuntimeCustom.java

    1 package hub.sam.stateautomaton.model;
    2 
    3 public class AutomatonRuntimeCustom extends AutomatonRuntimeDlg {
    4 
    5     @Override
    6     public void initialise() {
    7         setCurrentState(getMetaClassifierAutomaton().getInitialState());
    8         Transition initialTransition = getCurrentState().getOutgoing().iterator().next();
    9         initialTransition.fire(self);
   10     }
   11 
   12     // ...
   13 
   14 }

After we wrote custom implementations for all operations, we can execute our test model as follows:
(source code extracted from class hub.sam.stateautomaton.StateAutomaton)

    1 Automaton automaton = createLargeTestModel(testFactory);
    2 automatonRuntime.run("dbdecacf");

Activities

The problem with implementations in Java is that they are not model based and so tools cannot be derived (e.g. a debugger). What we need is a model based specification of the language semantics. Our approach is to use an activity language that is similar to UML activities. Actually it is a subset of UML activities and its semantics is based on the well-known petrinet formalism, so there is a solid foundation. It is designed to describe semantics in CMOF-based models, and so we named it MOF Action Semantics (abbreviated MAS). The language defines a set of atomic actions that can be used to describe operational semantics. Activities are composed of these actions plus some more elements for control and object flow. Each activity models the behaviour of one operation. If the model is executed, the corresponding activities will be interpreted in the context of a model instance. Without going into the details of the activity language, we now present our tools for modelling them.

We developed a graphical editor for really using the MAS language to develop behaviour models. The editor is known as MASE (MAS Editor) and it is based on Eclipse's GEF. MASE lets us model the behaviour of one operation in an activity diagram. So for every activity there will be one opened editor instance.

There is another plugin, named MOF 2.0 Browser Plugin, which can be used to navigate through CMOF-based models (you may know the plugin from the AMOF Tutorial page).

We use this plugin to navigate to an operation in the syntax meta-model of a language and then open a context-menu at that operation to create or edit its behaviour. This opens a new MASE instance where we can model the activity.

At this point, we need to install the required plugins. We then show you how to use them.

Now we are going to use the installed plugins and edit the activities of some operations that we supplied with the example language. While operations are part of a syntax meta-model that is based on the CMOF meta-meta-model, activities are part of a semantic model that is based on the MAS meta-model. We hold these two models in seperate XMI files and then use a MAS context file to connect them. This file is a simple Java properties file. It defines the properties syntax and semantic that refer to the two model files. The next figure shows the MAS context file for our example language that is located in resources/StateAutomaton.masctx.

     syntax = StateAutomaton.syntax.mdxml
     semantic = StateAutomaton.semantic.xml

MAS context files can be loaded with the MOF 2.0 Browser Plugin. We provide a special Eclipse perspective named "Mof Action Semantics Editor" for modelling activities. When you open the perspective, you will find the MOF 2.0 Browser Plugin at the left, optional MAS Editors at the right and a properties view at the bottom.

You can now add a new MAS context to the repository in the Model View at the left and then open the MAS context file in resources/StateAutomaton.masctx.

The syntax and semantic model are loaded into two seperate extents in the AMOF2 repository. We can browse the syntax model tree and apply behaviour in the form of MAS activities to operations.

This opens a new MAS editor for the operation model.Transition.fire.

The MAS language itself is not explained in this tutorial. If you want to learn how to model activities with MAS, you have to read our paper Human Comprehensible and Machine Processable Specifications of Operational Semantics.

Executing the User Model

The last step is to actually execute a model in the language. The next paragraph explains the required Java code.

    1 package hub.sam.stateautomaton;
    2 
    3 import hub.sam.stateautomaton.model.Automaton;
    4 import hub.sam.stateautomaton.model.FinalState;
    5 import hub.sam.stateautomaton.model.InitialState;
    6 import hub.sam.stateautomaton.model.State;
    7 import hub.sam.stateautomaton.model.Transition;
    8 import hub.sam.stateautomaton.model.modelFactory;
    9 import hub.sam.mas.execution.MasExecutionHelper;
   10 import hub.sam.mas.management.MasContext;
   11 import hub.sam.mas.management.MasModelContainer;
   12 import hub.sam.mas.management.MasRepository;
   13 import hub.sam.mas.management.MasXmiFiles;
   14 import hub.sam.mas.management.SimpleMasXmiFiles;
   15 import hub.sam.mof.Repository;
   16 import hub.sam.mof.management.MofModel;
   17 import hub.sam.mof.management.MofModelManager;
   18 import hub.sam.mof.ocl.OclEnvironment;
   19 import hub.sam.mof.ocl.OclObjectEnvironment;
   20 
   21 public class StateAutomaton {
   22 
   23     public void execute() throws Exception {
   24         Repository repository = Repository.getLocalRepository();
   25         Repository.getConfiguration().setWarnAboutForeignExtentObjectUsage(false);
   26         Repository.getConfiguration().setGenerousXMI(true);
   27 
   28         MasXmiFiles xmiFiles = new SimpleMasXmiFiles("resources/", "StateAutomaton.masctx");

Loads XMI file locations for a syntax and a semantic model from a MAS context file. The path of the model files is always relative to the MAS context file.

   29 
   30         MasModelContainer masModelContainer = new MasModelContainer(repository);

Creates a new container object for the syntax and the semantic model.

   31 
   32         masModelContainer.loadMasModel(xmiFiles.getMasFile());

Loads the semantic model from the XMI file location into the container.

   33 
   34         masModelContainer.loadSyntaxModelForExecution(xmiFiles.getSyntaxFile(), "Package:model");

Loads the syntax model for execution. This form of loading the syntax model creates implicit elements for runtime-representation-of relationships. In the execution case we need these elements but we do not want to see them when editing the model, and so there is also a loadSyntaxModelForEditing method.

   35         masModelContainer.getSyntaxModel().addJavaPackagePrefix("hub.sam.stateautomaton");

Adds the Java package prefix to the model again. We already added this prefix when the repository was generated, but MagicDraw UML does not allow us to save this information with its XMI files.

   36 
   37         MasContext masContext = MasRepository.getInstance().createMasContext(masModelContainer);

Creates a MAS context object for the models in the container. The context object manages the connection between operations and activities. Basically it allows the MAS implementations manager to find activities for operations and the editing environment to create and delete links.

   38 
   39         MofModelManager testManager = new MofModelManager(repository);
   40         testManager.setM2Model(masModelContainer.getSyntaxModel());
   41         MofModel testModel = testManager.createM1Model("test");
   42         modelFactory testFactory = (modelFactory) testModel.getFactory();

Creates an environment for the test model by using our new model management interface. The interface will be described in detail in the AMOF2 tutorial as soon as possible. For now, we give the following explanation: We first create a test model manager that can manage models of different meta-layers. We then set the syntax model from the model container as the M2 model and create our test model as the M1 model. We retrieve a factory for the test model, which we use later to create model elements. In the past, there was no such structure like a model manager that could keep all the important information together in one place.

   43 
   44         MasExecutionHelper.prepareRun(repository, masContext, testModel);

Prepares the test model for execution within the MAS context and installs implementations managers for activities, ocl queries and Java code (in order).

   45 
   46         Automaton automaton = createLargeTestModel(testFactory);
   47 
   48         boolean invariantCheckResult = testModel.getExtent().getAdaptor(OclEnvironment.class).checkAllInvariantsOnAllObjects();
   49         if (!invariantCheckResult) {
   50             System.out.println("Warning: Meta-model invariants evaluated to false.");
   51         }

Checks the static constraints that we defined in the beginning of the tutorial. All OCL invariants defined in the meta-model are checked against the test model.

   52 
   53         automaton.run("dbdecacf");

Finally: executes the test model.

   54     }
   55 
   56     public static void main(String[] args) throws Exception {
   57         new StateAutomaton().execute();
   58     }
   59 
   60     private Automaton createLargeTestModel(modelFactory factory) {
   61         // ...
   62     }
   63 
   64 }

Appendix

Background information: AMOF2 Implementations Managers

When an operation is invoked on an object in the AMOF2 repository, AMOF2 looks for installed implementations managers and uses them if they can supply an implementation. Different implementations managers exist. Lets take a look at the source code of method MasExecutionHelper.prepareRun which we have used above.

    1 public class MasExecutionHelper {
    2 
    3     public static void prepareRun(Repository repository, MasContext masContext, MofModel m1Model) throws Exception {
    4         ExecutionEnvironment env = new ExecutionEnvironment(m1Model.getExtent(),
    5                 masContext.getMasModel().getMetaModel().getExtent(), repository);
    6 
    7         Extent semanticExtent = masContext.getMasModel().getExtent();
    8         ((ExtentImpl) semanticExtent).setCustomImplementationsManager(new ImplementationsManagerContainer(
    9                 new ImplementationsManager[] {
   10                         new MultiLevelImplementationsManager(masContext.getMasModel().getFactory()),
   11                         new ImplementationsManagerImpl() }
   12         ));

At this point, things become complicated :). We install implementations managers for activities in our language's semantics extent. We have to do this because MAS itself is a language and we use this language for modelling the behaviour of other languages (e.g. the state automatons). It consists of a syntax meta-model and execution semantics with an implementation in Java code. You can think of its execution semantics as the engine that interprets a given activity of your language. We install a MultiLevelImplementationsManager first. It handles implementations for runtime operations (e.g. metaCreateAutomatonRuntime, metaDelete). Next, we install an ImplementationsManagerImpl which handles implementations in Java code. Below you can see two more implementations managers. At the model extent we additionally install managers for activities (MASImplementationsManager) and OCL queries (OclImplementationsManager). The order in which we install implementations managers matters. The first manager is checked first for an implementation, then the second and so on. This way we could model just some of the operations with activities and others with Java code.

   13 
   14         // MultiLevelImplementationsManager creates runtime instances
   15         ((ExtentImpl) m1Model.getExtent()).setCustomImplementationsManager(new ImplementationsManagerContainer(
   16                 new ImplementationsManager[] {
   17                         new MASImplementationsManager(masContext, env),
   18                         new MultiLevelImplementationsManager(m1Model.getFactory()),
   19                         new OclImplementationsManager(),
   20                         new ImplementationsManagerImpl() }
   21         ));
   22     }
   23 
   24 }