Migrating a project¶

A kibo 1.2 project generates with a script: a generate.py of its own, or dsm_util.py create_python_package / create_node_package. A kibo 2 project states its generation in a kibo.toml, and kibo-project runs it. This page turns the first into the second.

The generated code changes too: code written against it moves as Migrating application code says. A project that renders templates of its own moves them as Migrating a template pack says.

Before you start¶

Python 3.11 or later with dsviper (on Python 3.10, also tomli), and Java 17.
kibo_project.py, the kibo 2 jar and the kibo-template-viper 2 pack. In the DevKit they are tools/kibo_project.py, kibo-2/tools/kibo-2.*.jar and kibo-2/templates/, where kibo-project finds them on its own. Elsewhere, set KIBO_JAR and KIBO_TEMPLATES (Where the generator comes from).
If the project has templates of its own, render their before now, while the kibo 1.2 jar, pack and .dsm.json are still in place (the method). Until those templates are migrated, a target that renders them fails its validation: leave their features out of the target’s features at first, and add them once migrated.
Then delete, or move aside, everything kibo 1.2 generated. A 1.2 module left in place still imports and hides a call not yet migrated.

What the script did, and where it goes¶

A kibo 1.2 script does the same few things, whatever its length. Each has a line in kibo.toml, or disappears.

The script	In kibo 2
`DSMBuilder.assemble(<path>)`, then `parse()` and a report check	`[project] definitions = "<path>"`: a `.dsm` file, a directory of them, or a list of either. Errors stop the generation.
writes `<name>.dsm.json`	Nothing: kibo-project writes it beside `kibo.toml`, named after `[project] infrastructure` whatever the targets set. Ignore it in version control.
`-n <name>`	`[project] infrastructure = "<name>"`. A target whose name differed (`Features` for C++, `features` for Python) sets its own `infrastructure`.
finds the jar and checks the pack’s line	`[generator] templates = "2"`. `KIBO_JAR` and `KIBO_TEMPLATES` still override where they are found.
a list of template directories, one `java -jar` call each	`features = [...]` on a target: see the table below.
`-c cpp` / `-c python`	the target’s name, `[target.cpp]`, `[target.python]`, `[target.typescript]`, or `language = "..."`. TypeScript was rendered with `-c python`; it is now its own target.
`-o <dir>`	`output = "<dir>"`, with the changes below.
writes `*_Resources.hpp`, `resources.py` or `resources.ts`	Nothing: the pack writes the embedded definitions, in the encoding it declares. Delete that code.
a second call for `wheel/pyproject.toml.stg` or `typescript/project`	the `Wheel` (Python) or `Package` (TypeScript) feature. The `pyproject.toml` it writes keeps the name, the version, the packages and the `dsviper` dependency; authors, maintainers, a readme, classifiers and keywords are the packager’s to add.
`--cpp`, `--python`, `--typescript` flags	`kibo_project.py generate --target <name>`; with no `--target`, every target.

Where each output lands changed:

Target	kibo 1.2 `-o`	kibo 2 `output`
C++	a directory of `<Name>_<Template>.hpp/.cpp`	a directory, flat: `<infrastructure>_<unit>_<template>.hpp/.cpp`, one set per DSM namespace, and `<infrastructure>_resources.hpp`
Python	the package directory itself (`python/features`)	the directory above the package: the package is `<output>/<infrastructure>/`, `pyproject.toml` (feature `Wheel`) lands at `<output>`
TypeScript	two calls: sources to `<root>/src`, `package.json` to `<root>`	the package root, once: sources go to `<output>/src`, `package.json` and `tsconfig.json` (feature `Package`) to `<output>`

Setting a Python target’s output to the old package directory nests the package one level too deep (python/features/features/). The package directory is the infrastructure exactly as written, not snake-cased: when the 1.2 package directory was not the -n name (-n MetaProject into .../metaproject), set the target’s infrastructure to the directory’s name, the name the package is imported by.

From templates to features¶

kibo 1.2 rendered template directories; kibo 2 selects features, and a feature brings the features it requires (Attachments brings Paths, which brings Fields). Some 1.2 directories merged, others are gone because the runtime now covers them.

kibo 1.2 template directory	kibo 2 feature
C++ `Model`, `Data`, `ValueType`	`Base` (the field names and paths of `Model` are `Fields` and `Paths`)
C++ `Attachments`	`Attachments`
C++ `FunctionPool`, `AttachmentFunctionPool`	`Pool` (renders nothing for a model that declares no pool)
C++ `FunctionPoolRemote`, `AttachmentFunctionPoolRemote`	`PoolRemote`
C++ `Stream`, `ValueCodec`	none: the runtime’s static layer, through the codec `Base` generates
C++ `Json`, `ValueHasher`	none: JSON, XML and a hexdigest are one runtime call on what `codec::encode` returns
C++ `Database`	none: `Viper::Database`; the attachments take a database as well as a state
C++ `AttachmentFunctionPool_Attachments`	none: the generated attachments, or `Definitions.inject()` with an `AttachmentMutating`
C++ `Python`	none: `Definitions.inject()` computes the same constants at run time
C++ `Test`, `TestApp`	none: they tested the generator. A project that rendered them keeps them as its own templates (below)
Python `package`	`Base` and `Attachments`; `Pool` if the model declares pools
Python `wheel/pyproject.toml.stg`	`Wheel`
TypeScript `typescript`	`Base` and `Attachments`; `Pool` if the model declares pools
TypeScript `typescript/project`	`Package`

The 1.2 Python and TypeScript packages held the attachments and pools of every model; in kibo 2 they are features, so a project that used them names them. The full list of features and what each generates is in kibo-template-viper features.

Templates of your own¶

A template the pack does not ship — a project’s test programme, a report, a binding of its own — is declared in a manifest of the project, in the pack’s format. Its templates sit beside it, in a directory per target:

templates/
├── features.json
└── cpp/
    ├── test.hpp.stg
    ├── test.cpp.stg
    └── test_app.cpp.stg

{
  "cpp": {
    "Test": {
      "doc": "What it takes to put a unit to the test.",
      "templates": ["test.cpp.stg", "test.hpp.stg"],
      "requires": ["Base", "Attachments"]
    },
    "TestApp": {
      "doc": "The programme that runs the test over the whole model.",
      "templates": ["test_app.cpp.stg"],
      "requires": ["Test", "Pool"]
    }
  }
}

kibo.toml names the manifest in [generator] manifests, and a target selects its features as it selects the pack’s. A feature name the pack already declares is refused. requires names the features the template’s output needs beside it, the pack’s as well as the manifest’s; kibo-project renders them too.

A template of your own renders where the pack’s do: for C++ into output, for Python into the package, <output>/<infrastructure>/, for TypeScript into <output>/src. A module it adds to a Python or TypeScript package takes a name a DSM namespace or pool could also take; reserve it, so that such a namespace stops the generation instead of overwriting the module:

{
  "python": {
    "Report": {
      "doc": "A report of the model's types, as a module of the package.",
      "templates": ["report.py.stg"],
      "requires": ["Base"]
    }
  },
  "reserved": {
    "python": {"namespace": ["report"], "pool": ["report"]}
  }
}

A template written for kibo 1.2 reads Template Model 1: it renders, under kibo 2, only once migrated (Migrating a template pack). kibo 2 also renders a template once per scope it declares — main(m) keeps rendering it once for the whole model, as in 1.2.

A template that rendered to another directory than the rest (TestApp into .) is a target of its own, with with_requirements = false: it renders only the features it names, and another target of the same language and infrastructure renders what they require.

[target.app]
language = "cpp"
features = ["TestApp"]
with_requirements = false
output = "."

A complete example¶

A kibo 1.2 generate.py (shortened):

NAMESPACE = 'Features'
DSM_SOURCE = arguments.definitions        # all.dsm

# C++, into Features/
for template in ['Model', 'Data', 'Stream', 'Json', 'Database',
                 'Attachments', 'AttachmentFunctionPool_Attachments',
                 'ValueType', 'ValueCodec', 'ValueHasher', 'Test']:
    generate(namespace=NAMESPACE, template=template, output='Features')
generate_resource(definitions=DEFINITIONS, output='Features/Features_Resources.hpp')
generate(namespace=NAMESPACE, template='TestApp', output='.')

# Python, into python/features, then resources.py
generate_package(name='features', output='python/features')

# TypeScript: -t typescript into typescript/features/src, -t typescript/project into
# typescript/features, then resources.ts
generate_typescript(name='features', package_root='typescript/features')

The same project in kibo 2, with Test and TestApp moved to the project’s templates/features.json and migrated to Template Model 2:

[project]
definitions = "all.dsm"
infrastructure = "features"

[generator]
templates = "2"
manifests = ["templates/features.json"]

[target.cpp]
infrastructure = "Features"
features = ["Base", "Attachments", "Pool", "Test"]
output = "Features"

[target.app]
language = "cpp"
infrastructure = "Features"
features = ["TestApp"]
with_requirements = false
output = "."

[target.python]
features = ["Base", "Attachments", "Pool", "Wheel"]
output = "python"

[target.typescript]
features = ["Base", "Attachments", "Pool", "Package"]
output = "typescript/features"

Stream, Json, Database, ValueCodec, ValueHasher and AttachmentFunctionPool_Attachments have no line: the runtime covers them. [target.cpp] renders Pool and Test, which TestApp requires: a target with with_requirements = false is refused unless another target of the same language and infrastructure renders them.

For dsm_util.py create_python_package model.dsm --wheel, run in a directory: the package was ./model/, pyproject.toml at .:

[project]
definitions = "model.dsm"
infrastructure = "model"

[generator]
templates = "2"

[target.python]
features = ["Base", "Attachments", "Wheel"]
output = "."

For dsm_util.py create_node_package model.dsm: the package root was ./model/:

[target.typescript]
features = ["Base", "Attachments", "Package"]
output = "model"

Check the result¶

python3 kibo_project.py plan kibo.toml — the jar and the pack it found, and per target its output directory, the features and every template, marked sources or root. Nothing is written. Check each output directory and feature list against what the project’s build expects.
python3 kibo_project.py generate kibo.toml — generates every target, then validates the Python (imported, every structure built, mypy --strict) and the TypeScript (tsc).
Build the project. The C++ file names and namespaces changed, and so did the Python and TypeScript modules: the compiler and the type checkers list what the code still names the 1.2 way, which Migrating application code moves.
python3 kibo_project.py check kibo.toml — once the project builds, exits 0 when the generated files in place are those kibo.toml generates. Run it in the project’s checks.