Roadmap

As a volunteer-run Open Source effort, we cannot commit to a timeline. We’re actively looking for corporate sponsors, with some prospects in sight.

In progress for after version 2024.1

Delta protocol

As described in our reference architecture, via delta protocol a client receives an initial snapshot similar to the bulk case. Afterwards, the client continues to receive changes (aka deltas) from the repo; it also writes back changes continuously.[1]

Derived models

As described in our reference architecture, derived models are calculated from other (original or derived) models without direct human interaction. They are usually some form of analysis result, such as one related to a type system. Nodes in derived models are typically associated with an original node – e.g., the type computed for an AST node. The repository manages this association. Derived models may be persisted or be recalculated on the fly.[2]

Envisioned

We highly encourage any participation in LionWeb. So if you’d like to work on some of these features, or even only have an urgent need for them: let us know! This will influence the feature’s priority.

The list below is in no particular order.

Standard derived models

We intend to support several common language (workbench) functionalities via derived models. For each functionality, LionWeb would define a base language. Any implementation might use these base languages directly, or define its own extension languages.

As an example, LionWeb’s validation base language might define concepts for Error, Warning, and Info. My validator implementation would create instances of Error and Info, but also include a custom compiler-based validation language that defines GeneratorWarning and CompilerWarning as specializations of Warning. A client that only knows about LionWeb’s validation base language can process both of the specialized warnings in a generic fashion. Another, compiler-aware client might display GeneratorWarnings differently than CompilerWarnings.

Base language candidates are typically common language functionalities that might be expensive to calculate. This includes:

Validation

Checking a model for consistency and other errors is one of the most common operations on models. It might also be expensive: if we wanted to assure globally unique names, we had to check all named things in our model on every name change. The basic structure of validation is very similar in most systems: any issue has a severity level, refers to a node, might relate to a feature, and provides an error message.[3]

Scoping

Calculating the scope of visible targets at any given place is necessary for both parser-based clients (for name binding / linking) and projectional clients / IDEs (for auto-completion). Similar to validation, the basic structure of scoping is very similar in most systems; this base language would condense the common concepts.[4]

Typesystem

We need to calculate the type of a node in several contexts, including validation (e.g. "do the types of these assigned nodes match?") and scoping (e.g. "which members does the thing on the left side of the dot have?").[5]

Access control

Different users might have different level of access to the whole repository, or parts of it. Some users might have read-only access (e.g. via public web), whereas others can write to the repository.[6]

Versioning / Live collaboration

A repository might support version control and/or live collaboration of users.

The repository might have a version control backend like git, or support live collaboration like Google Docs.[7]

Note
These two features are lumped into one because from an implementation point of view, both need a history of all edit actions to actually work.

Locking

A repository might support locking the whole repository, partitions, or subtrees. The lock might be established with optimistic or pessimistic locking strategies.[8]

Federation

A repository A might access a different repository B, and provide B's contents to A's users. The two repositories might be synchronized on a regular schedule, or continuously.[9]

Global symbol index

A global index of (fully qualified) names can be very useful for any kind of name lookup, for example in parser-based name resolution, scoping, or auto-completion.[10]

Identifier mapping API

Different clients might refer to the same node by different ids. The repository might support them by automatically mapping between the repository’s internal id, and the id known to each client.[11]

Language migration API

Support a common way how to handle language migrations, e.g. by specific messages like "I’m about to migrate this partition", or "You must not use language X in a version older than abc".[12]

Store invalid text typed at arbitrary place

Given a valid input with invalid text at few places, both tolerant parsers and projectional editors can provide a "mostly correct" AST representation. Internally, they have some way of keeping the invalid text around. We could support a standard way to store such invalid text in the repository.[13]

Define a standard way to represent model diffs

Calculating the difference between two sets of nodes is common, complex, and potentially expensive task. It might be useful to have a standard format to exchange the difference, and the Delta protocol could provide a solid base.[14]

Complex query API

More complex filters for [bulk-protocol] and/or Delta protocol might help saving communication overhead. For example, I might query via bulk only for Dog concepts, or only receive delta updates for Dogs with furColor == red.[15]

Commented nodes

In source code it’s very common to comment out a part of the contents. We might come up with a standard way to support similar functionality to comment out some nodes.[16]

Support custom DataTypes

A user might define their own DataTypes and support their (de-)serialization.[17]

Shared annotations in built-ins language

We might add some Annotations to the built-ins language, e.g. range constraints on integers, implementation per platform, @Deprecated, etc.[18] Another useful set of annotations would be review comments: For any kind of collaboration, it’s very helpful for reviewers to attach their remarks directly to their context — think of the blue speech bubbles in the right margin of a Word document.[19]

For some use cases (e.g. merging), it might be useful to know that the order of features does not matter.[20]

Interpretation API

In most advanced DSLs, we want to evaluate or "execute" models directly, i.e. without intermediate generation step. Interpreters can be implemented in various ways, but usually have a very standard interface.[21]

File generation API

Most modeling systems want to create some artifacts from the model contents, for example source code, documents, or images. This interface can be standardized to receive some input nodes, maybe some parameters, and result in one or more files.[22]

Note
This functionality concerns model-to-text / model-to-binary generators. Model-to-model generators would be implemented as Processors that run on-demand.

Versions

2024.1

  • Specified Bulk Repository Access API It is used to store and retrieve nodes in batches at the moment of invocation.[23] It is intended for CRUD operations on (larger) sets of nodes, and enables integration with "simple" editor (i.e. no real-time collaboration). Additionally, this API allows to list the available Partitions, get Languages known to the repository, and request unused node ids.

  • Removed built-in JSON primitive datatype[24]

  • Added StructuredDataType[25]

  • Removed pre-defined node ids and introduced pre-defined resolveInfos[26]

  • Disallow mixing containments and annotations[27]

2023.1

Stable serialization format (including JSON schema) and meta-metamodel describing Languages, Concepts, Interfaces, Annotations, Enumerations, PrimitiveTypes, Properties, Containments, and References. The standard library contains classifiers Node and INamed, and primitive types Boolean, String, Integer, and JSON.

History

2023

November

  • Initial release 2023.1

  • Presentation at LangDevCon 2023

May

  • First public presentation at MPS Community Meetup 2023

2022

October

  • Start of regular LionWeb core group meetings

September

  • Founding Meeting at LangDevCon 2022

  • Set up LionWeb-io GitHub organization


1. Repo API: Change-based event notifications #28
2. issues on derived models
3. Define language for validations as derived model #176
4. Define language for scoping as derived model #177
5. Define language for Typesystem as derived model #178
6. Repo API: Access control #27
7. Repo API: Versioning / Collaboration #26
8. Repo API: Locking #30
9. Repository federation #23
10. Principle: "everything is a model" / "everything is a node" #144
11. Provide id mapping API #94
12. Do we specify how language evolution works? #72
13. How to store invalid text typed at arbitrary places? #62
14. Define a standard way to represent model diffs #40
15. Additional API: Complex queries #39
16. Support commentable nodes #15
17. Support custom DataTypes? #10
18. Annotation instances shipped by default #12
19. Define language for review comment annotations #175
20. If and how to represent ordered / unordered containments in M3 #4
21. Define Interpretation API #179
22. Define file generation API #180
23. Repo API: Bulk read/write #25
24. Remove JSON standard Primitive Datatype #277
25. Value Types #265
26. Make LionCore M3 and builtins language definitions have version-dependent IDs #279
27. We disallow concept nodes in annoations / annotation instances in containments #320