Running KEITH

Table of Contents

maxLevel	2

Setting up your

...

Eclipse

For everything not mentioned here refer to Getting Eclipse guide. Choose the 2021-06 Eclipse version and I personally recommend Eclipse for RCP & RAP developers since the Plug-In Development perspective is the default one. Another helpful perspective might be the Git perspective.

Use the installer go to advanced mode, add the KIELER url

...

. If you plan to develop for the semantic language server (e.g. for the compiler) or to work with the SCCharts language, you should select KIELER semantics; for diagram only KIELER pragmatics. In any case select the keith stream in semantics or the master stream in pragmatics.

If you plan to develop in elk at the same time first select the Eclipse Layout Kernel setup and after that either the semantics or pragmatics setup.

Wait till everything installs and the setup tasks finish. If you have any problems in this stage refer to the Getting Eclipse guide

...

or ask your advisors.

Make sure that you have necessary forks of ELK/KLighD set up.

If you have problems in the workspace that are still there after a clean build do the following:

Disable Project>Build automatically
Select all KLighD and pragmatics plugins and do Project>Clean>Only selected and build only selected
Do the same for the semantics projects
Enable Project>Build automatically

Java Application

If you have used the semantics setup, there is a pre-configured run configuration that you can use to execute the KIELER Language Server. For that, go into the Run>[Run|Debug] Configurations>Java Application>LanguageServer with KLighD in Workspace and execute that.

Described below is how you can set this configuration up for yourself for any manual configuration within the language server:

To run the language server go to Run Configurations create a new

...

Image Removed

You have to edit the arguments too. The Vm arguments host and port are added to connect the LS via socket.

...

Java Application run configuration.

Select the Projectde.cau.cs.kieler.language.server or de.cau.cs.kieler.pragmatics.language.server and the Main class de.cau.cs.kieler.language.server.LanguageServer

...

or de.cau.cs.kieler.pragmatics.language.server.PragmaticsLanguageServer.

Image Added

In the next step all projects that you want to include in your language server have to be added to the classpath.

Go to Classpath, select User Entries, click Add Projects..., and select all required projects (if you are unsure just add all of them).

Click on Advanced>Add Folders add select the project folders of all projects you added earlier.

In the Arguments tab make sure to add -Dport=5007 to the VM arguments.

The default port to which KEITH tries to connect is 5007. You can of course change this for the language server but be aware that this has to be changed in KEITH too.

Setting up a KEITH developer setup...

General requirements:

node (and additional dependencies see Theia developer guide)
npm (whatever node installs)
yarn (latest version)
Python (2.7.X)
gcc, g++, and make (for native dependencies of some npm packages)
Visual Studio Code (latest version)
a cloned keith repository

... on linux:

(Theia has a guide for extension development that might be helpful)

install node

...

(for the version we refer to the Theia developer guide):

Code Block
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.5/install.sh \| bash nvm install

...

Install python if you haven't (remember: Python 2: , Python 3: ).

Install yarn (a package manager build on the package manager npm):

Code Block
npm install -g yarn

... on windows:

Install node

...

for windows. I personally used the .msi. For the version refer to the Theia developer guide.

Use that to install windows-build-tools by executing the command in an administrative powershell.

Code Block
npm install -g windows-build-tools

This installs make, gcc, g++, python and all this. Somehow this does not really terminate. If nothing happens anymore it may be finished, just kill the process if it does not terminate.

All the installed executables are not in the path and that is okay. This is not needed since yarn/npm knows how to call them when needed.

Yarn can be downloaded and installed from here.

Known Problems in this step

If python3 was already installed this may cause some problems.

... on mac:

Get a package manager, something like brew.

Use brew to install all necessary stuff.

Apparently there is an issue with xcode-select: Theia developers recommend the following:

Code Block
xcode-select --install

After doing this for your OS all that is missing is running KEITH (in developer setup) and setting up your

...

Eclipse for language server development).

Stuff that may help

Running the already build LS

Go to the latest Bamboo build and go to Artifacts.

Image Removed

Select Language Server Zip and download the LS and unpack it somewhere.

Locate the kieler.ini file. Depending on the OS it has a different location (linux; toplevel, windows, toplevel, mac: Content/Eclipse/kieler.ini)

Paste the following at the beginning of the ini-file:

Code Block

language	bash

-application 
de.cau.cs.kieler.language.server.LanguageServer 
-noSplash

Since an eclipse application is built, this is needed to start the LS without a splashscreen.

If you want to connect that LS via socket to your Theia application (KEITH) add the following to the vmargs:

Code Block
-Dport=5007

5007 is the standard port KEITH is currently connecting to in socket mode. You can find this port in your Theia application at the following location:

Assume you are in the keith repository. Go to keith-app, you should see something like this:

Image Removed

Open the package.json. In the package.json are several scripts defined.

Image Removed

The LSP_PORT option is used to activate the connection via socket. It is also possible to specify a relative location to a LS via LS_PATH=<path to LS>.

How to run KEITH in developer setup (socket)

Run the following to build and run KEITH in its developer setup (in socket mode, so the LS has to be started separately)

Running KEITH in the browser

Code Block

language	bash

yarn && cd keith-app && yarn run socket

yarn builds all the stuff. yarn run socket in keith-app starts the application. After an initial build via yarn you can run yarn

...

watch to watch the changes in your repository. In another console you run yarn run socket in keith-app. Now refreshing your browser is enough to apply the changes.

Per default the KEITH opens on localhost:3000.

It is required to restart the language server if KEITH is restarted, since the diagram view has a problem (since theia-sprotty is used) to reconnect after that.

Known issues for windows:

nsfw.code not found: In the top level package.json exists a script called postinstall. Remove this on windows, delete the node_modules folder and rebuilt the application. This is a known issue of electron-builder.

Known issues on mac:

Since SWT is still used as part of the diagram synthesis (but is not relevant anymore). Since it is not called on the main thread this causes a deadlock. Therefore mac just does not work.

Known issues:

Refreshing the browser is not enough for the diagram to work. If the diagram is needed the language server has to be restarted before the browser is refreshed. This is a known issue in theia-sprotty.

Developing for KEITH

We use java ServiceLoader to register stuff. Here is a small example how a LanguageServerExtension is registered via a ServiceLoader and how it is used:

Register LanguageServerExtensions (ServiceLoader Example)

This is a LanguageServerExtension. It has to be used in the de.cau.cs.kieler.language.server plugin. Since the language-server-plugin should not have dependencies to all plugins that define a language server extension dependency inversion is used to prevent that. A ServiceLoader does exactly that.

Here is such an example extension, the KiCoolLanguageServerExtension:

Code Block

package de.cau.cs.kieler.kicool.ide.language.server


/**
 * @author really fancy name
 *
 */
@Singleton
class KiCoolLanguageServerExtension implements ILanguageServerExtension, CommandExtension, ILanguageClientProvider {
	// fancy extension stuff

	var KeithLanguageClient client
	// A language server extension must implement the initialize method,
	// it is however only called if the extension is registered via a language.
	// This should never be the case, so this is never called.
    override initialize(ILanguageServerAccess access) {
        this.languageServerAccess = access
    }
    
	// implement ILanguageClientProvider
    override setLanguageClient(LanguageClient client) {
        this.client = client as KeithLanguageClient
    }
    
	// implement ILanguageClientProvider
    override getLanguageClient() {
        return this.client
    }

}

The CommandExtension defines all commands (requests or notifications) that are send from client to server. An example how this looks like can be seen in the code snippet Example CommandExtension is an example how to define a server side extension interface.

The ILanguageClientProvider should be implemented by an extension that plans to send messages from the server to the client.

This language server extension is provided by a corresponding contribution, which is later used to access it:

Code Block

package de.cau.cs.kieler.kicool.ide.language.server

import com.google.inject.Injector
import de.cau.cs.kieler.language.server.ILanguageServerContribution

/**
 * @author really fancy name
 *
 */
class KiCoolLanguageServerContribution implements ILanguageServerContribution {
    
    override getLanguageServerExtension(Injector injector) {
        return injector.getInstance(KiCoolLanguageServerExtension)
    }
}

Create a file called de.cau.cs.kieler.language.server.ILanguageServerContribution in <plugin>/META-INF/services/ (in this example this is de.cau.cs.kieler.kicool.ide). The name of the file refers to the contribution interface that should be used to provide the contribution. The content of the file is the following:

Code Block
de.cau.cs.kieler.kicool.ide.language.server.KiCoolLanguageServerContribution

This is the fully qualified name of the contribution written earlier.

The language server uses all LanguageServerExtensions like this:

Code Block

var iLanguageServerExtensions = <Object>newArrayList(languageServer) // list of all language server extensions
for (lse : KielerServiceLoader.load(ILanguageServerContribution)) { // load all contributions
	iLanguageServerExtensions.add(lse.getLanguageServerExtension(injector))
}

The resulting list of implementions is used to add the extensions to the language server.

Register an extension (on server side)

See example above for ServiceLoader and initial stuff.

What is still missing are the contents of the CommandExtension implemented by the KiCoolLanguageServerExtension. This is an interface defining all additional commands. The CommandExtension looks like this.

Code Block

title	Example CommandExtension

package de.cau.cs.kieler.kicool.ide.language.server

import java.util.concurrent.CompletableFuture
import org.eclipse.lsp4j.jsonrpc.services.JsonRequest
import org.eclipse.lsp4j.jsonrpc.services.JsonSegment

/**
 * Interface to the LSP extension commands
 * 
 * @author really fancy name
 *
 */
@JsonSegment('keith/kicool')
interface CommandExtension {
    
    /**
     * Compiles file given by uri with compilationsystem given by command.
     */
    @JsonRequest('compile')
    def CompletableFuture<CompilationResults> compile(String uri, String clientId, String command, boolean inplace);
    
    /**
     * Build diagram for snapshot with id index for file given by uri. Only works, if the file was already compiled.
     */
    @JsonRequest('show')
    def CompletableFuture<String> show(String uri, String clientId, int index)
    
    /**
     * Returns all compilation systems which are applicable for the file at given uri.
     * 
     * @param uri URI as string to get compilation systems for
     * @param filter boolean indicating whether compilation systems should be filtered
     */
    @JsonRequest('get-systems')
    def CompletableFuture<Object> getSystems(String uri, boolean filterSystems)
}

This defines three json-rpc commands: "keith/kicool/compile", "keith/kicool/show", "keith/kicool/get-systems". These are implemented in KiCoolLanguageServerExtension.

Server Client communication interface

Not only messages from client to server but rather mesages from server client might be needed

Register and calling an extension (on client side)

Language server extension do not have to be registered on the client side. It is just called.

You can send a request or a notification to the language server like this:

Code Block

const lclient = await this.client.languageClient
const snapshotsDescriptions: CodeContainer = await lclient.sendRequest("keith/kicool/compile", [uri, KeithDiagramManager.DIAGRAM_TYPE + '_sprotty', command,
	this.compilerWidget.compileInplace]) as CodeContainer
// or via a thenable
client.languageClient.then(lClient => {
lClient.sendRequest("keith/kicool/compile").then((snapshotsDescriptions: CodeContainer) => {
	// very important stuff
}
// await is preferred, since it is shorter.

In this example client is an instance of a language client. It is usually injected like this:

Code Block
constructor( @inject(KeithLanguageClientContribution) public readonly client: KeithLanguageClientContribution // other injected classes ) { // constructor stuff }

How to make a new package for KEITH

Clone the KEITH repository.

Open the keith folder in VSCode. You are know in the keith directory in VSCode.

You see something like this: TODO picture

Create a new folder called keith-<your extension name>.

Copy a package.json, a tslint.json, a tsconfig.json, and a src folder into the folder.

Add keith-<your extension name> to workspaces in the top level package.json.

Add "keith-<your extension name>": "0.1.0" to the dependencies in the top level package.json and the product package.json files (e.g. the package.json in keith-app).

What is in the src directory?

The source directory has three optional subfolders.

node: Holds all backend related classes. This does currently only exist in the keith-language package.
common: Holds general helper methods, string constants and some data classes
browser: Holds all widgets, contribution for commands, menus, and widgets, and the frontend-extension.

The frontend-extension

This binds all necessary classes. Look at existing frontend extension in KEITH or Theia to see how this is done.

More examples for stuff

See Theia examples.

How to write a widget

There are different kinds of widgets that are commonly used in KEITH or in existing Theia packages.

BaseWidget: Very basic
ReactWidget: A render method has to be implemented that redraws the widget on demand. Additionally several on* event methods can beimplemented.
TreeWidget: Extends the ReactWidget and draws the contents of the widget in a tree view.

If a widget has a state it should implement the StatefulWidget interface, which allows to imlement a store and restore method.

Look at examples in KEITH or Theia to see how this is done.

How to make a new module for sprotty (see actionModule, ...)

Info
If you previously build keith electron, you have to execute `yarn run rebuild:browser`

Run Launch in Chrome via VSCode to open a chrome browser on localhost:3000

This is necessary to be able to debug in VSCode.

Running KEITH as (unbundled) electron app

Code Block

language	bash

yarn && yarn run rebuild:electron && cd keith-app-electron && yarn run socket

yarn builds all the stuff. yarn run socket in keith-app-electron starts the application. After an initial build via yarn you can run yarn watch to watch the changes in your repository. In another console you run yarn run socket in keith-app-electron. Now refreshing your browser is enough to apply the changes.

Info
If you previously build keith electron, you have to execute `yarn run rebuild:electron`

Running the already build LS

Note
In the current builds, there seems to be a problem with the packaging of the jar file and executing will cause a ClassNotFoundException for org/eclipse/ui/IStorageEditorInput when initializing the LS. We are looking into this issue. For now, setup your Eclipse as described above.

Go to the latest Bamboo build and go to Artifacts.

Select the language server for your OS (this will be a jar file) and run it via:

Code Block
java -Dport=5007 -jar <name-of-the-jar-file>

5007 is the standard port KEITH is currently connecting to in socket mode. You can find this port in your Theia application at the following location:

Assume you are in the keith repository. Go to keith-app, you should see something like this:

Image Added

Open the package.json. In the package.json are several scripts defined.

Code Block

language	js

"scripts": {
        "prepare": "yarn run clean && yarn build",
        "clean": "theia clean",
        "build": "theia build --mode development",
        "start": "theia start --root-dir=../workspace",
        "socket": "node ./src-gen/backend/main.js --root-dir=../workspace --LSP_PORT=5007 --port=3000 --loglevel=debug",
        "watch": "theia build --watch --mode development"
    },

The LSP_PORT option is used to activate the connection via socket. It is also possible to specify a relative location to a LS via LS_PATH=<path to LS>.

Known Issues

Known issues for windows:

nsfw.code not found: In the top level package.json exists a script called postinstall. Remove this on windows, delete the node_modules folder and rebuilt the application. This is a known issue of electron-builder.

Known issues on mac:

(this might already be resolved, has not been tested yet though)

Since SWT is still used as part of the diagram synthesis (but is not relevant anymore). Since it is not called on the main thread this causes a deadlock. Therefore mac just does not work.

Known issues:

KEITH works in the browser/electron app, but not in the electron app/browser with the following error message:
symbol lookup error: ... symbol lookup error: .../keith/node_modules/nsfw/build/Release/nsfw.node: undefined symbol: _ZN2v816FunctionTemplate3NewEPNS_7IsolateEPFvRKNS_20FunctionCallbackInfoINS_5ValueEEEENS_5LocalIS4_EENSA_INS_9SignatureEEEiNS_19ConstructorBehaviorENS_14SideEffectTypeE Done in 0.90s.
- run yarn run rebuild:electron/browser after yarn to fix this. If it does not work, delete the node_modules folder and try again (for browser version rebuild browser is not needed, since yarn already builds the correct sources).

Page tree

Versions Compared

Old Version 20

New Version Current

Key

Running KEITH

Setting up your

Eclipse

Java Application

Image Removed

Setting up a KEITH developer setup...

... on linux:

... on windows:

Known Problems in this step

... on mac:

Stuff that may help

Running the already build LS

How to run KEITH in developer setup (socket)

Running KEITH in the browser

Known issues for windows:

Known issues on mac:

Known issues:

Developing for KEITH

Register LanguageServerExtensions (ServiceLoader Example)

Register an extension (on server side)

Server Client communication interface

Register and calling an extension (on client side)

How to make a new package for KEITH

What is in the src directory?

The frontend-extension

More examples for stuff

How to write a widget

How to make a new module for sprotty (see actionModule, ...)

Running KEITH as (unbundled) electron app

Running the already build LS

Known Issues

Known issues for windows:

Known issues on mac:

Known issues:

Page tree

Page History

Versions Compared

Old Version 20

New Version Current

Key

Running KEITH

Setting up your

Eclipse

Java Application

Image Removed

Setting up a KEITH developer setup...

... on linux:

... on windows:

Known Problems in this step

... on mac:

Stuff that may help

Running the already build LS

How to run KEITH in developer setup (socket)

Running KEITH in the browser

Known issues for windows:

Known issues on mac:

Known issues:

Developing for KEITH

Register LanguageServerExtensions (ServiceLoader Example)

Register an extension (on server side)

Server Client communication interface

Register and calling an extension (on client side)

How to make a new package for KEITH

What is in the src directory?

The frontend-extension

More examples for stuff

How to write a widget

How to make a new module for sprotty (see actionModule, ...)

Running KEITH as (unbundled) electron app

Running the already build LS

Known Issues

Known issues for windows:

Known issues on mac:

Known issues: