NED version 2.0 DRAFT ===================== Topology description language for OMNeT++ Author: Andras Varga $Revision: 1.1 $ This document contains the DRAFT of NED-2, a major version of the NED language. As the document converges, nedtool and the NEDXML library will cover a growing subset of this specification until full support. The old nedc compiler is going to be maintained until nedtool is mature enough to completely take over its responsiblities. The NED-2 DTD (ned-2-draft.dtd) is an unseparable part of this document. 1. Requirements Fundamental requirements (in arbitrary order): o must support message types (maybe other components too) o channels must be made more flexible (with parameters, etc.) o components must be self-describing as far as possible (e.g. documentation) o object-oriented concepts must be supported o more flexible gate, parameter, channel and connection handling o must support generation of extensible code (via C++ base classes) o must be readable by grapical editors (ie. must not be or contain a programming language) o syntactical differences from NED 1.0 are allowed, but automatic conversion should be possible o functionality/semantics must be backwards compatible with NED 1.0 o format must be highly readable (more than XML is!) o familiar, consistent syntax (e.g. more conventional use of "," and ";" characters o extensibility o must support XML data interchange o better presentation support for components, incl. supporting switching between presentations according to component state The items below explain, refine and concretize these requirements. Components - from NED 1.0: simple module, compound module, channel, network - new component types: message, packet, struct -- nedc generates C++ classes from them, see "Code generation" requirements - message classes will be subclassed from cMessage, packet classes will be subclassed from cPacket - NED structs will generate simple C++ structs (classes?) which can be used in internal data structures of the model Inheritance for NED components - (single) inheritance for module, channel, network, message etc. types - inheritance may add new parameters - inheritance may set parameters to fixed values (thereby hiding them from derived and embedding components) - inheritance of compound modules may add new gates and submodules and connections Better parameter handling - support string operation (substring, concatenation, etc.) - support parameter arrays - submodule parameters must be able to reference other parameters of the same module - distinguish between numeric types (i.e. integer vs double) - parameters to module, channel etc. components can be immediately set to fixed values at the place of their declarations. Fixed values cannot be overridden in enclosing components. Enums - symbolic names with associated values (integers) - nedc must generate "reflection" code - parameter type may be an enum - is it needed to add values to an existing enum? ("inheritance") Better gate handling - support vector gates with fixed sizes (which don't need 'gatesizes:') - support specifying acceptable message types for gates (BONeS-like feature) Better channel support - must be able to extend channels with new parameters - inheritance for channels Better connections - pass parameters to channels when used in connections - single "if" clause for a several connections (e.g. grouped by {..}) - possibility to create and use temp variables within loops (?) - possibility to omit gate index (just say "next free gate"); useful for random topologies Properties - in addition to parameters, components should also have "properties". Properties will contain "external" or "meta" info which isn't really part of the model. - properties are stored per type, not per instance (except display string???) - example properties: display string, description, package - should be possible to add extra properties everywhere Self-documenting components - possibility to add description to top-level components (module, channel, etc.) - possibility to add description to parameters - possibility to add description to gates Packages - must be possible to logically group components into packages - packages must be hierarchical (like Java packages) Presentation - default presentation (default icon, default line color, etc.) for components (modules, channels, messages) - support for changing component presentation at runtime (e.g. to display component's internal state) - presentation details should NOT need to be known to simulation code in order to be able to switch to a different presentation (-->symbolic names should be used) - presentation for messages: either dependent on connection (channel) or dependent on message type or message parameter Data interchange - support XML as interchange format (NED-XML mapping needed) - the XML format must be able to fully capture the NED file, including formatting (indentation, comments, line breaks, etc.) - in the XML format, the NED formatting information should be kept separated from significant content, and be made optional - the XML format should be specified with an XML Schema (DTDs are obsolete) - the XML format should allow expressions to be stored either parsed or unparsed. Parsed form is needed e.g. by nedc, while gned is entirely happy with the unparsed one (gned doesn't need to understand expressions) - when defining the format of parsed expressions, MathML should be regarded Code generation for messages, packets and structs - nedc should generate C++ classes, where parameters are added as class members with the appropriate get/set methods - implication: declarations of message, packet and struct parameters must name the C/C++ types of the parameters - compiler must generate "reflection" code (in the Java sense), ie. fields must be available internally via a generic way (because Tkenv must be able to display the fields in objects). [One way to do this is to generate a separate descriptor object; another is to add reflection code directly to the classes] - must support optional use of the "Generation Gap" pattern (i.e. genera- ting a C++ base class explicitly intended for subclassing by the user) - class generation implies generating _n.h files Code generation for modules, channels and networks - parameters will remain represented as cPars - nedc should generate convenience functions to access individual parameters/gates by direct method call (e.g. parameter "foo" can be accessed both via conventional par("foo") and generated getFoo() method; gate "Bar" can be accessed via gate("bar") and gateBar() methods) - must support optional use of the "Generation Gap" pattern (i.e. genera- ting a C++ base class explicitly intended for subclassing by the user) - class generation implies generating _n.h files 2. Design considerations Possible syntax alternatives - conventional NED vs. C++-like syntax simple A...endsimple --or-- simple A {...} - Pascal vs. C/C++-like parameter declarations string s --or-- s: string --> must use C/C++ style, Pascal style doesn't allow immediate initialization - documentation: with language constructs vs. in special comments (like Javadoc) The proposed syntax is the following: - C++-like syntax, C++-like parameter and gate declarations - descriptions map to comments (like Javadoc) 3. Specification 3.1 Illustrative code examples Enum syntax: enum ProtocolCode { PROT_IP = 1; PROT_ICMP = 2; PROT_ARP; // numbering goes on like in C++: ARP=3, RARP=4, TCP=5 PROT_RARP; PROT_TCP; }; Simple and compound module syntax: package "generic.queueing"; // default package for following components /** <-- denotes documentation comment * Simple FIFO module with a msg/sec processing rate and a finite buffer * capacity. Messages which arrive when the queue is full are dropped. */ simple Queue { properties: display="i=queue"; // default presentation is a queue icon package="generic.queueing"; // override default package parameters: /// processing rate <-- another style of documentation comment numeric msgPerSec; /// max length of queue numeric maxLength; gates: input messagesIn[] accepts(Packet); // input gates output messagesOut; // output gate }; /// /// Generic router module with variable number of ports. /// module GenericRouter { properties: display="i=bluebox"; baseclass=true; // use "Generation gap" pattern parameters: string brandAndModel; numeric numPorts, msgPerSec; // 2 parameters declared on one line numeric portDataRates[numPorts]; // parameter vector numeric k=0; // parameter declared and set at once gates: input portsIn[numPorts]; output portsOut[numPorts], extraPort; // 2 gates declared on one line submodules: rte: Rte { properties: display="i=router"; parameters: processingMode=MSG_PER_SEC; // comes from an enum numPorts=numPorts; processingRate=msgPerSec; }; ports: Port[numPorts] { parameters: datarate=portDataRates[index]; }; connections: for i=0..numPorts-1 { rte.out[i] --> ports[i].in; rte.in[i] <-- ports[i].out; } // condition syntax ???? for i=0..numPorts-1 { rte.out[i] --> ports[i].in if i/2==floor(i/2); if i/2==floor(i/2) { // no arbitrary nesting! only one level of "if" inside a "for" allowed rte.out[i] --> ports[i].in; rte.in[i] <-- ports[i].out; } } // connection syntax and display strings ???? rte.out[i] --> ports[i].in display "o=red,2"; // ^-- drawback: "display" is both an identifier (property name) and // a reserved keyword this way :-( rte.out[i] --> ports[i].in { properties: display = "o=red,2"; // other properties may follow... }; rte.out[i] --> ports[i].in {properties: display "o=red,2";}; // parameter syntax to channel instances ???? rte.out[i] --> FooChannel --> ports[i].in; rte.out[i] --> FooChannel {delay=5;} --> ports[i].in; rte.out[i] --> FooChannel {parameters: delay=5;} --> ports[i].in; // "channel" keyword is optional below: rte.out[i] --> channel {parameters: delay=5;} --> ports[i].in; rte.out[i] --> channel {delay=5;} --> ports[i].in; rte.out[i] --> {parameters: delay=5;} --> ports[i].in; rte.out[i] --> {delay=5;} --> ports[i].in; }; /// /// A specific router type. /// module MyRouter extends GenericRouter { // inheritance parameters: string brandAndModel="Sixco AL-400C"; // parameter value is fixed numeric numPorts=6; // parameter value is fixed string maintainer; // new parameter // properties, gates and submodules unchanged from Router }; Channel syntax: channel FooChannel { properties: baseclass=true; parameters: numeric error = 0.01; numeric whatever; // extra parameters possible }; Message/packet syntax: message FooMessage { properties: baseclass=true; fields: int version=2; int protocolId enum(ProtocolCode); int length; }; message SpecialFooMessage extends FooMessage { properties: baseclass=true; fields: int protocolId=PROT_ARP; // PROT_ARP comes from an enum; the field // will be set to PROT_ARP in the C++ constructor }; Struct syntax: message FooMessage { properties: baseclass=true; fields: int version=2; int protocolId enum(ProtocolCode); int length; }; 3.2 Components Component types: simple module, compound module, channel, network, message, packet, struct. 3.2.1 Modules, channels, networks Essentially, like in NED 1.0. Inheritance, more flexible parameters,... Better channel support - must be able to extend channels with new parameters - inheritance for channels 3.2.2 Messages, packets, structs Messages, packets and structs are new to NED. The purpose of introducing messages and packets is to increase type-safety and efficiency of the simulation code. Formerly, cPars were the only way to add data to message object. cPars are fairly complex objects themselves so they made the simulation slow(er than it needs to be), and they were also error-prone because the cPar object had to be added individually to each message object. The solution is to move message/packet descriptions into NED, and have nedc generate subclasses of cMessage and cPacket. nedc also adds the necessary data members (those in the "fields" section of the NED packet/message) and their get/set methods. nedc also generates the necessary reflection code which makes it possible to inspect the messages/packets together with their fields in Tkenv. Message classes will be subclassed from cMessage, packet classes will be subclassed from cPacket. NED structs are a way to add data structures to the simulation which can be inspected in Tkenv. nedc translates NED structs to the corresponding C++ structs (or classes?) and also generates the necessary reflection code which makes it possible to inspect them in Tkenv without any custom programming. 3.2.3 Packages - must be possible to logically group components into packages - packages must be hierarchical (like Java packages) Component names (without package name) must be unique. (C++ linker conflicts! -- C++ packages?) 3.2.4 Inheritance NED supports (single) inheritance for the following component types: Simple Compound Subm Chan Net Msg Pkt Struct x x - x x x x x Syntax: message extends {...} Inheritance may: - add or override properties - add new parameters or fields - set parameters to fixed values (thereby hiding them from derived and embedding components) - for simple and compound modules, it may add new gates and define gate sizes for vector gates of undefined size - for compound modules, it may also add submodules and connections 3.2.5 Enums - symbolic names with associated values (integers) - nedc must generate "reflection" code - parameter type may be an enum - is it needed to add values to an existing enum? ("inheritance") 3.3 Sections 3.3.1 Components support table The following table shows which sections may be present within different component declarations: Simple Compound Subm Chan Net Msg Pkt Struct properties: x x x x x x x x parameters: x x x x x - - - fields: - - - - - x x x gates: x x - - - - - - submodules: - x - - - - - - gatesizes: - - x - - - - - connections: - x - - - - - - on: x x x - x - - - The order of sections within the component is insignificant, only the relative order of conditional sections of the same type are important. Duplicate sections are also permitted (non-conditional ones only) and will be merged internally by nedc. conditional sections? like in 1.0 ??? try eliminate gatesizes...? it's ugly... 3.3.2 Properties Properties contain "external" or "meta" info which isn't really part of the model. - properties are stored per type, not per instance (except display string???) - example properties: display string, description, package - should be possible to add extra properties everywhere types of properties??? (string only? or cPar?) The following properties exist: baseclass: value=yes/no; per-class property display string: value=string; per-class AND per-object property description: value=string; per-class property; implicitly comes from the doc comments package: value=string; per-class property; default value comes from last 'package' directive The following properties are supported with the different component types: Simple Compound Subm Chan Net Msg Pkt Struct baseclass x x - x x x x x display string x x x x x x x - description x x x x x x x x package x x x x x x x x 3.3.3 Parameters (2 types of sections: param decls + param assignments) parameters to module, channel etc. components can be immediately set to fixed values at the place of their declarations. Fixed values cannot be overridden in enclosing components. Types: distinguish between numeric types (i.e. integer vs double) Specifying enums for integers: int protocolId enum(ProtocolCode); Parameter arrays: declaration, usage... - paramsizes??? - initialization: index ranges? 3.3.4 Fields The following data types will be supported for fields: - primitive types: bool, short, int, long, unsigned short, unsigned int, unsigned long, double - fixed-size arrays of the above primitive types - variable-sized arrays of the above primitive types (stored as a ptr to a dynamically allocated array plus an int for array size; generated class will have an allocate function for each variable-sized array) - char - fixed size arrays of chars (handled as a single string) - a dynamically allocated string (stored as char*) - struct/class (?); fixed and variable-sized arrays of them (?) (--> how to declare them to NED? what about cObject ownership stuff?) Enums can be assigned to variables of integral types. 3.3.5 Gates - support vector gates with fixed sizes (which don't need 'gatesizes:') - support specifying acceptable message types for gates (BONeS-like feature) 3.3.6 Submodules Like in NED 1.0... Contains sections, etc... 3.3.7 Gatesizes Like in NED 1.0. Size of fixed-size gate vectors cannot be overridden. 3.3.8 Connections - pass parameters to channels when used in connections - single "if" clause for a several connections (e.g. grouped by {..}) - possibility to create and use temp variables within loops (?) 3.3.9 On 3.4 Better expression handling 3.4.1 String operations + sign concatenation = equals < > <= >= ordering length(s) substring(s,pos,len) ... 3.4.2 Parameter references Expressions can reference other parameters of the same module: addr - param of the enclosing module (or channel) gen[4].addr - param of a (another) submodule (only within a compound module) this.addr - another param of this submodule (only within a submodule! in a network???) (where exactly?? check all places where expressions may occur!!!) How do we handle the special expression "index"...? "index" and "this.index" here too? what with parameter index (with param vector)? Maybe ".index"? 3.4.3 Integer vs double expressions Distinguish between integer vs double type expressions and arithmetic! 3.5 Comments and documentation - possibility to add description to top-level components (module, channel, etc.) - possibility to add description to parameters - possibility to add description to gates Doc comments: /** */ and /// placement: like in gned? 3.6 Presentation Several presentations in a display string: one default and several others, tagged with a desciptive label, e.g.: "idle" --> gray rectangle "busy" --> red rectangle Syntax in display strings is like: labelname{display-string-tags}, e.g: "p=100,100;idle{i=graybox};busy{i=redbox}" Simple module C++ code could refer to the label if it wants to change the presentation, e.g. by calling a future setAppearance(const char *label) function: if (!isBusy) setAppearance("idle"); else setAppearance("busy"); Message display strings... - presentation for messages: either dependent on connection (channel) or dependent on message type or message parameter Expressions (values of parameters or fields in a display string?)... 3.7 Code generation 3.7.1 Generated headers nedc generates _n.h files which need to be included into the corresponding simple module sources. This is because nedc generates a number of C++ classes, and the user will write code that uses these classes. (For example, see classes generated from NED messages and packets.) 3.7.2 Messages, packets and structs For messages, packets and structs, nedc generates C++ classes. The base class is the one given in NED with the 'extends' clause, or the following if there's no 'extends': message --> cMessage packet --> cPacket struct --> sStruct (contains the getDescriptor() method only, see later) The fields ('fields' section) are added as protected class members (of long, double, etc. type) with the appropriate get/set methods. An example: packet MyPacket { fields: long length; double weight=0.5; } The generated C++ code: class MyPacket : public cPacket { protected: long length; double weight; public: MyPacket(const char *name) : cPacket(name), weight(0.5) {} // ...other generated functions... long getLength() {return length;} void setLength(long length) {this->length=length;} double getWeight() {return weight;} void setWeight(double weight) {this->weight=weight;} }; TBD: Work this out for all possible field types! 3.7.3. Reflection code for messages, packets and structs nedc generates "reflection" code (in the Java sense) for the message, packet and struct classes. The reflection code enables enumerating, getting and setting the fields of the classes in a generic way, without compile-time knowledge about the specific class. (This enables Tkenv to display the fields in objects). The way to do this is to generate a separate descriptor object. An example: class MyPacketDescriptor : public cObject { public: //...some generated methods... int getFieldCount() {return 2;} // "length" and "weight" const char *getFieldName(int index) { switch(index) { case 0: return "length"; case 1: return "weigth"; default: return NULL; // bad index } } const char *getFieldType(int index) { switch(index) { case 0: return "long"; case 1: return "double"; default: return NULL; // bad index } } long getLongField(void *object, int index) { MyPacket *p = object; switch(index) { case 0: return p->getLength(); default: return 0; // ERROR: bad index or not a long field } } double getDoubleField(void *object, int index) { MyPacket *p = object; switch(index) { case 1: return p->getWeigth(); default: return 0; // ERROR: bad index or not a double field } } void setLongField(void *object, int index, long value) { //... } void setDoubleField(void *object, int index, double value) { //... } }; Given a MyPacketDescriptor object, it's possible to write code (e.g. as part of Tkenv) which displays and modifies the contents of a MyPacket, without knowing the MyPacket class at compile time. 3.7.4 Generation gap pattern nedc supports optional use of the "Generation Gap" pattern (i.e. genera- ting a C++ base class explicitly intended for subclassing by the user) - class generation implies generating _n.h files Code generation: the C++ base classes - base classes ("Generation Gap") - must support optional use of the "Generation Gap" pattern (i.e. genera- ting a C++ base class explicitly intended for subclassing by the user) 3.7.5 Parameters and gates Parameters and gates continue to be represented as cPars and can be accessed via the generic functions (e.g.: par(const char *paramname), gate(gatename, index)). However, in addition, nedc generates convenience functions to access individual parameters/gates by direct method call (e.g. parameter "foo" can be accessed both via conventional par("foo") and generated getFoo() method; gate "Bar" can be accessed via gate("bar") and gateBar() methods). 3.7.6 Enums Need a class to store NED enum types (a set of (symbolic name, integer value) pairs). Parameter or field types may take values from an enum, and we have to store this info in the cPar or the descriptor object. 3.7.7 Generating expressions The original version of the nedc compiler generated "dynamic" expressions which were evaluated at runtime, by the cPar class. The new nedc generates static (compiled) expressions wherever possible. Static expressions can be a lot more effective at runtime, and potentially easier to generate. Example: the expression 3*p1+p3 (where p1 is long and p3 is cPar) can be compiled into the following C++ class: class Expr12 : public LongExpression { private: long p1; cPar& p2; public: Expr12(long ap1, cPar& ap2) : p1(ap1), p2(ap2) {} virtual long evaluate() {return 3*p1+p2;} }; where LongExpression is a generic base class: class LongExpression { public: virtual long evaluate() = 0; }; An Expr12 instance can be stored and evaluated by a LongExpression pointer. In addition to LongExpression, we need the DoubleExpression and StringExpression base classes too. Note: expression-valued cPars could also work in the future with compiled expressions. Dynamic expressions can further be supported using the special LongDynamicExpression, DoubleDynamicExpression and StringDynamicExpression classes (which would rely by composition on a single DynamicExpression class). 3.7.8 Display strings Must add a setAppearance(const char *label) method (or the like) to cModule, cGate (cMessage?) to support switching between appearances at runtime. 4. NED and XML 4.1 Architecture of NED tools must support XML as interchange format (NED-XML mapping needed) see appendix for XML schema use data classes instead of DOM??? (JAXB) NED-related tools (e.g. nedc and gned) should use DOM, the programming interface of XML as their internal data structure. This will make it possible to easily convert between a variety of network description representations (NED,XML,database tables,C++ code,etc), and to manipulate them programmatically. (The DOM trees must of course conform to the NED XML binding.) ---> C++ (*_n.cc) XML ---> ---> live network NED ---> DOM ---> XML database ---> ---> NED ---> database Programming languages: Java and C++. For C++, the IBM/Apache XML parser and DOM library is strongly recommended. Java parsers and DOM libraries are more standardized, so it's less important which implementation is used (Sun, IBM, Oracle, etc.) 4.2 XML format 4.2.1 General The XML support in the OMNeT++ 2.0 GNED can be used as a starting point. 4.2.2 NED formatting info The following info should be stored: - whitespaces (line breaks, indentation, in-line whitespace) - comments (text and position of comment) - formatting and position of doc comments (since the text of the doc comment should be available separately) - choice of syntax alternatives (i.e. "numeric a; numeric b;" or "numeric a, b;") - other...? Maybe a separate element (?) like: chars-after=" // bla%0d%0a" chars-middle1="" alt-syntax="merge-with-next" /> This element would be inserted into all XML elements (gates, parameters, etc.) Difficulty: for GNED, it won't be so easy to offer comments for editing as before... 4.2.3 Parsed expressions MathML!!! (semantic syntax) A. Appendices A.1 Grammar A.2 XML schema A.3 Extensions needed to simulation kernel The following features must be supported by the simulation kernel to make a NED-2 compiler possible: - enum, message, packet, struct types - inheritance for components - storing properties in all component descriptors - storing properties in module, gate, parameter instances (?) - setAppearance(const char *tagname) function - separate long/double handling in cPar - string arithmetics in cPar - more...? A.5 References XML reference DOM reference URL to IBM/Apache XML parser URL to Sun, IBM, Oracle XML parsers ------------------------------------------------------------------------- Random thoughts. - all parameters should be "const" unless explicitly declared "volatile"! - make gatesizes optional -- vector size can grow as needed - with connections, support "use next free gate index" construct, e.g: mod1.out[next] --> mod2.in[next] - current gate and parameter declaration syntax is confusing! gates: : ; parameters: : , ... note that a,b,c: string; means a:numeric, b:numeric, c:string; !!! - submodule syntax: submodule a : A { parameters: ....; } -