eclipse edt - egl inner workings - june 2011

www.eclipse.org/edt

Project members:

The Inner Workings of EGL:a Solution to the Complexities of Modern Application Development

Tim W WilsonSTSM & EGL Chief Architect, [email protected]

EGL Development Tools Project (EDT) - www.eclipse.org/edt

2

Agenda

Motivation for an Open EGL

Structure of EGL language

Extending the EGL language Adding abstract concepts

Extending Code Generation

Demo Extended Java code generation for native Android application


3

Evolution of Language Abstraction

Hardware

OS

Runtime Platform

Assembler

C, COBOL

Java

EGL

One

One

One

One

One

One

One

Many

Many

Many

One

Single Machine

Multiple Machine

FutureOne

Many

Evolution of Execution Environment

Evolution of Programming Languages

Range of Deployment Options


4

Model Driven Development (MDD) and Platform Independence

MDD uses UML to define Platform Independent Models (PIMs)

Model Compilers read these models and ‘compile’ them to Platform Specific Models (PSMs) Abstract concepts mapped to concrete implementations for a given platform

If all programming can stay at the PIM level then one can say that the programming language is Platform Independent

UML is just another programming language when used to program an entire application without ever dropping down into the generated code Why should it be the only programming language that does this?


5

EGL Compiler as Model Compiler EGL is a programming language mixed together with a modeling mechanism and

extensible model compiler Designed with the intention of expressing precise execution semantics

What is an Extensible Model Compiler? New concepts added to programming model

e.g. Entity, UI programming model, new Datatypes (i.e. ZonedUnicodeDecimal)

Existing compiler extended to map instances of new concepts used in the code to appropriate runtime

Very high fidelity interleaving of new extended code generation into existing code generation

Can override/extend any aspect down to low level Expressions

The term ‘Extension’ in the context of EGL means anything added in the programming model that requires the code generation be extended to implement the specified semantics of the extensions


6

Motivation for Open EGL: EGL Development Tools on Eclipse (EDT)

EGL can target diverse platforms for the same code because it models everything in a platform independent way Platform specifics are stated as input to model compiler

IBM has been the only one able to define new abstractions and transformations

General interest in this approach can only occur if others can do the same thing

The purpose of EDT is to provide others the ability to: create domain specific extensions, i.e. UI programming, DB IO, etc

create new or extend existing transforms to make concrete the new extensions

have development environment (IDE) able to use these extended compilers

i.e. debug at EGL source code level for java and javascript target environments


7

Agenda







8

EGL Basic Concepts

Various component types that programmers can define Data Item – Creates a Type alias from extended data declaration

Record - aggregate of typed fields into new type

Program - Main program starts run unit – used for batch processing

Library – Static set of data and functions – local to run-unit

Service – Remotable set of functions

Interface – Abstract definition of a Service

Handler – Object-based component containing instance data and functions

ExternalType – Abstract type definition of a type implementation that exists outside of EGL

Annotations and Stereotypes Meta data applied to program elements

Drives transformations as well as tooling

Abstract Statements meant to be extended get, add, delete, replace, close, call, etc.


9

EGL Syntactic Structure

All type definitions are in a package

Imports of types outside current package

Declaration of a program

Data and Function declarations

Statements and Expressions

Extensible IO Statement

Function invocation


10

Extending EGL

Extension means: Any addition that requires the compiler be extended to map the extension to an

appropriate runtime

Categories of Extension Native Types

Type definitions that are abstract only and depend on the compiler to map them to appropriate runtime type

Simple Types - types that have no internal structure

Is a special kind of native type

Stereotypes, Annotations - metadata applied to types and members

Extensible Action Statements

Statements sensitive to the Stereotype metadata associated with its operands


11

EGL Extensions: Simple Types

There are no built-in ‘primitive types’ in EGL

All types have an explicit definition

Simple types highlighted as keywords are aliases for some explicit type definition

Types aliases are configurable

All valid operators are defined on each type

Valid conversions between types defined explicitly

decimal egl.lang.AnyDecimalstring egl.lang.AnyString


12

Simple Type Example: egl.lang.AnyDecimal

All Simple Types are externaltype declarations External types denote the implementation is outside of EGL

The type declaration signifies to the compiler how the type can be used

Code generation will map the use of these types to appropriate runtime

Previous code example


13

EGL Extensions: Native Types

Native types are abstract definitions that represent some native implementation

The compiler is responsible for mapping uses of a given type to an implementation

SysLib is a Native Type

The function writeStdOut() is mapped to platform specific implementation

Java: System.out.println() COBOL: DISPLAY…

Native types allow modeling of platform independent concepts

Extensible code generation allows the use of these types to be folded into an existing compiler


14

Native Type Example: egl.lang.SysLib

Completely abstract API – concrete implementations dependent on target platform


15

EGL Extensions: Stereotypes and Annotations Stereotypes allow typed metadata to be applied to any type

Annotations allow typed metadata to be applied to any member (field, function…)


16

EGL Extensions: Extensible Action Statements

EGL contains a set of abstract ‘Action’ statements add, close, delete, get, open, replace, etc

Action statements have an abstract execution specification that is made concrete by the metadata associated with the given operands get – read an object from a data source

add – add an object to a data source

This example shows the use of a variable in which its type has the ‘Entity’ stereotype applied

The abstract semantic of GET plus the supplied metadata of ‘Entity’ allows a concrete specification

Code generator extended to implement the GET as an SQL SELECT statement Entity metadata supplied the necessary mapping to the

database tables


17

Java

EDT – Code compilation process

EGL Source Code

EGL Compiler Model

Extensible Code Generation Framework

Java JavaScript C/C++ .Net …

Pluggable Generators

Eclipse

JEE Dojo

Pluggable Extension Examples

Possible

Android

COBOL

CICS/IMS


18

EGL Structure: The EGL Model


19

EGL Model

So – how are all these classes defined?


20

EGL Program Definition: Detail

All EGL Model Classes are a subclass of Element

All EGL Model Classes are instances of EClass as denoted by the <<EClass>> stereotyping in the diagram

What is an EClass?


21

The MOF (Meta Object Framework) Model


22

EDT MOF Framework All values are instances of EObject – including EClass

All EObject values have a reference to their EClass instance

Dynamic interface for manipulating EObjects e.g. myEObject.eGet(“field1”)

Serialization to Binary or XML form Abstracted such that other serializations possible

Replaces Intermediate Representation (IR) files seen in RBD today

Java Class generator for creating specific java classes for each EClass High performance

Easier programming

Transformation framework Initial java class implementations of each EClass generated using this framework

All EGL code generators based off same framework


23

The EDT Modeling Framework: org.eclipse.edt.mof MOF (Meta Object Framework) Model

Meta model used to create other models

Used to create the EGL Model

EGL Model Meta model used by compiler to create EGL

elements

Program, Record, Expression, etc

EGL syntax can be used to define both MOF and EGL model elements EGL Compiler can create instances of either

model

EGL can therefore be used to define arbitrary models

MOF Model

EGL Model


24

Using EGL to define a Model EGL Meta Model defined by EGL itself

Special Stereotype MofClass tells compiler to create EClass instances instead of ExternalType instances

Compiler creates EClass instance

because of MofClass

Stereotype

Compiler creates normal

Externaltype instance


25

XML Serialization of EClass - StructPart

This object is an instance of org.eclipse.edt.mof.EClass

Instances of org.eclipse.edt.mof.egl.Field


26

XML Serialization of ExternalType - Customer

This object is an instance of org.eclipse.edt.mof.egl.ExternalType

Instances of org.eclipse.edt.mof.egl.Field


27

Agenda







28

Module Structure of EDT: org.eclipse.edt

org.eclipse.edt.ide

org.eclipse.edt.core

org.eclipse.edt.gen


29

Extensible Model Compilers: org.eclipse.edt.gen

Core Model Transformation framework Model to Model, i.e. EDT MOF model to EMF model

Model to Text, i.e. EGL Model model to Java

All EDT code generation based upon this framework

Simple and lightweight and highly extensible

Extensibility comes from configurable ‘Templates’ A Template is simply a java Class

Each model element has a Template associated with it

Each Template implements methods that perform some transformation on its given model element

All template method invocation follows the inheritance hierarchy of the given model

If a given template method does not exist in the specific template of the model element method lookup will go up the super type chain

Extending an existing transformation is simply a matter of adding and replacing templates with new ones that:

Add new

Replace the original

Extend the original


30

Content of a Generator

The purpose of a ‘Generator’ is to transform instances of some model into either text or instances of some other model

A ‘Generator’ is composed of a set of ‘Templates’ Each Template is a simple Java class

Each model class has an associated Template

All transformations handled in the templates

Validation

Model to Model transformation (M2M)

Model to Text (M2T)

Extending a Generator means: Adding new templates

Overriding existing templates

Complete override

Pre/post processing with invocation of super class methods


31

The EDT Code Generation Framework: org.eclipse.edt.mof.codegen.api

Framework composed of two basic classes

org.eclipse.edt.mof.codegen.api.Template Transformation/validation implementation Usually one Template class per Model class that needs transformation

org.eclipse.edt.mof.codegen.api.Context Contains the global state of a transformation Implements dynamic template method Invocation

Context implements dynamic invocation of methods on Templates

Object invoke(String methodName, Object object, Object…args) Object invokeSuper(String methodName, Object object, Object…args)

Dynamic invocation means:

Look up the template associated with Class of the second parameter - Object object Look up the method with the given method name methodName and signature based on the args Object…args Invoke method on the template passing object and args


32

EGL Model to Template Associations

ProgramTemplate

ClassTemplate

StructTemplate

ClassifierTemplate

NamedElementTemplate

ElementTemplate Each Model Class has associated Template

Each Template implements sets of methods for various purposes Analysis, Validation, Code generation

Methods in each set implement a particular API clients invoke for particular situations

Templates themselves do NOT follow inheritance of the model Dynamic template method invocation follows the

inheritance chain of the model

Allows for easy overriding of Template classes Replacing Template classes does not interfere

with any Template hierarchy


33

Mapping File of EDT Core Java Generator

EGL Model type signature=Template java class type signature


34

Dynamic Template Method Invocation Example: Java Generation of Program

Appropriate Template looked up based on Class of part argument

Program ProgramTemplate

genPart() method lookup in ProgramTemplate If not found go up the super type chain of Program

and look for the genPart() method in the associated Template for that super type EGLClass ClassTemplate

If not found continue up the chain until found

Invoke genPart(part, context, out)

This approach allows one to think about transformation implementation as if the methods were actually implemented in the model classes themselves


35

Dynamic Invocation Example: Field Template

FieldTemplate

genDeclaration()

MemberTemplate

genDeclaration()

NamedElementTemplate

genName()

ElementTemplate

Lookup starts at Field but moves up to NamedElement


36

EDT Core Generators

EDT will provide ‘core’ generators for Java and JavaScript

A ‘core’ generator is one that implements a transformation to a target runtime technology that adheres to the semantics of the ‘Core’ EGL Language

EGL Core will be the EGL specification containing: Syntax and semantics for declaring EGL definitions

Execution semantics for statements and expressions

Semantic descriptions for extensible IO statements

GET, ADD, etc. with no description for specifics like SQL access

EGL Core will not specify any metadata specifications except for those necessary to define other metadata Annotation and Stereotype are types themselves known to the compiler

Core generators


37

EDT Extended Generators An Extended Generator contains:

Specific set of metadata types, i.e. SQLRecord, TextProgram, etc

Extension to a core generator with new templates built for new set of metadata types

Runtime support libraries if necessary

EDT will also provide a set of extended generators to support common platforms

JEE Generator extends core Java generator Support of Web Services, JPA, etc.

Core JavaScript Generator extended to handle current RUI programming model

Extended generators

www.eclipse.org/edt

Project members:

Android Generator: an example


39

UI Programming Model Example: Define Native Types

Window

View

ViewGroup

TableLayout

1 contentView

* children

Abstract concepts derived from concepts common across UI frameworks Window – deployable UI unit

Android – Activity

Web – HTML Page

View – Super type of all UI controls

Android – View

SWT – Widget

ViewGroup – contains View children

Android – ViewGroup

Dojo – Container

TableLayout – kind of ViewGroup for Tables


40

UI Programming Model Example: Define Native Types in EGL

Abstract concepts defined using Externaltypewith AbstractType stereotype

Event handlers modeled as Delegates

Inheritance modeled directly

Externaltype declarations allow for full Class modeling


41

UI Programming Model Example: Define Concepts in EGL

Core UI Concepts

Stereotypes for implementation

Concrete Widgets


42

UI Programming Model Example: Define Stereotypes for Implementation

Native Types by definition are abstract and have no specific EGL implementation

Stereotypes allow EGL implementation definitions to be part of an object model introduced with the native types.

defaultSuperType setting means that any handler stereotyped by WindowHandler will subclass the Window native type

Allows abstract API of Window to be accessed anywhere within a handler stereotyped by WindowHandler

Extended Generator will fold in appropriate implementation for the abstract API anywhere in the code it is accessed


43

Example Usage: Mortgage Calculator

Event handler declaration

Stereotype used to implement a WindowSimilar conceptually to RUIHandler

Invocation of the Amortize Service

Declaration of Widgets and properties


44

Mapping to Target Platform: Android and Dojo

Abstract concepts need concrete mapping to intended runtime platform

Native Android is a Java framework Android generator would be an extension of the core EDT Java generator

Dojo is a JavaScript framework Dojo generator would be an extension of the core EDT JavaScript generator

This example shows a simple mapping to Android platform Use of widgets and layout Calling a web service


45

Mapping File of EDT Core Java Generator

EGL Model type signature=Template java class type signature


46

Map Android Model to Android Templates

Mapping of model classes to their templates

Overrides of EGL Core Java Templates


47

Create Necessary Templates

Override existing method from Template from Base Java Generator

Template method invocation


48

Android Mortgage Calculator Example


49

Android Mortgage Calculator Detail

Native type abstract API access


50

Mapping Native Types to Runtime Types

EGL native type type signature = Runtime java class type signature


51

Native Type Template Example: TextViewTemplate

All native type API access handled in the template associated with particular native type

Native type templates understand the mapping of the model to the runtime

Will use the runtime java class API

Delegation will occur for the following Expression types:

Assignment Name, MemberAccess FunctionInvocation

aTextView.text = “123”; aTextView.setText(“123”);

EGL Generated Java

aStr = aTextView.text; aStr = aTextView.getText();


52

Create the AndroidGenerator Command Line Tool

Android generator extends core Java generator

Adds command line parameters as necessary

Define where the configuration is for adding and overriding code generation templates

Define where the configuration is for mapping native types to concrete classes in the runtime


53

Plugging Android Generator into the IDE

Implement a few simple classes in an IDE plug-in GeneratorProvider TabProvider Extension of the command line generator to be invoked by EGL builder

Example Android IDE plug-in classes acme.generator.android.ide.AndroidGeneratorProvider acme.generator.android.ide.AndroidTabProvider acme.generator.android.ide.EclipseAndroidGenerator


54

Workspace Preferences: EGL Compiler

Multiple EGL Compilers in the IDE

Each compiler can have multiple generators associated with it

Workspace preference defines default settings for EGL Compiler for a new EGL Project

Project specific settings can override these defaults for each EGL Project


55

For more information visit our project page

Project schedules Including detailed 0.7.0

iteration/milestone plans

Downloadable builds and source

Committers and status

New and noteworthy

Documentation

Presentations

Access to bugs and requirements

http://www.eclipse.org/edt/

EDT 0.7.0 M1 now

available!

www.eclipse.org/edt

Project members:

Thank You

eclipse edt - egl inner workings - june 2011

Technology