npm Packages

Making the generated TypeScript package buildable and installable — the counterpart of Python wheels, over the @digitalsubstrate/dsviper Node binding.

The Package feature

The TypeScript Base feature renders the sources of an ES module package into src/. The Package feature adds, at the package root — the target’s output — the two files that make it a package:

  • package.json — the name, the exports, the runtime range, the build script;

  • tsconfig.json — the compiler settings the sources are written for.

Both are rendered by kibo-project from the project’s kibo.toml:

[project]
definitions = "definitions"
infrastructure = "features"

[generator]
templates = "2"

[target.typescript]
features = ["DatabaseAttachments", "CommitAttachments", "Package"]
output = "typescript"
python3 kibo_project.py generate kibo.toml

infrastructure names the package (features here). Name the attachment usages the project needs — DatabaseAttachments over a Database, CommitAttachments over a Commit state — and add "PoolRemote" or "AttachmentPoolRemote" when the model declares pools a client calls.

The generated files

package.json, as rendered (JSON has no comments, so it carries no header; its peerDependencies name the runtime and its range):

{
    "name": "features",
    "version": "1.0.0",
    "description": "TypeScript bindings generated by kibo for the features model",
    "type": "module",
    "exports": {
        ".": {
            "types": "./dist/index.d.ts",
            "import": "./dist/index.js"
        },
        "./pools": {
            "types": "./dist/pools.d.ts",
            "import": "./dist/pools.js"
        },
        "./attachment_pools": {
            "types": "./dist/attachment_pools.d.ts",
            "import": "./dist/attachment_pools.js"
        },
        "./*/database_attachments": {
            "types": "./dist/*/database_attachments.d.ts",
            "import": "./dist/*/database_attachments.js"
        },
        "./*/commit_attachments": {
            "types": "./dist/*/commit_attachments.d.ts",
            "import": "./dist/*/commit_attachments.js"
        },
        "./*/fields": {
            "types": "./dist/*/fields.d.ts",
            "import": "./dist/*/fields.js"
        },
        "./*/paths": {
            "types": "./dist/*/paths.d.ts",
            "import": "./dist/*/paths.js"
        },
        "./*": {
            "types": "./dist/*/index.d.ts",
            "import": "./dist/*/index.js"
        }
    },
    "files": ["dist", "src"],
    "scripts": {
        "build": "tsc -p tsconfig.json"
    },
    "peerDependencies": {
        "@digitalsubstrate/dsviper": ">=1.2.16 <1.3.0"
    },
    "devDependencies": {
        "@digitalsubstrate/dsviper": ">=1.2.16 <1.3.0",
        "@types/node": ">=20",
        "typescript": ">=5"
    },
    "license": "UNLICENSED"
}

tsconfig.json, as rendered:

// Generated from features.dsm.json by kibo-2.1.0.jar. Do not edit by hand.
// Templates: kibo-template-viper 2.1.0 (MIT), Template Model 2.

{
    "compilerOptions": {
        "target": "ES2022",
        "module": "NodeNext",
        "moduleResolution": "NodeNext",
        "declaration": true,
        "strict": true,
        "esModuleInterop": true,
        "skipLibCheck": true,
        "outDir": "dist",
        "rootDir": "src",
        "types": ["node"]
    },
    "include": ["src/**/*.ts"]
}

Adjust version and license for your release; keep the @digitalsubstrate/dsviper range: it is the pack’s (see Version management).

The runtime is a peer dependency: the package uses it, the project that installs the package provides it. npm installs one copy, the project’s, and refuses a project whose version lies outside the range, rather than installing a second copy beside it. The same range sits in devDependencies, so that npm install in the package root has a runtime to build against.

Build

In the package root:

npm install
npm run build        # tsc -p tsconfig.json

or npx tsc -p .. The compiler turns src/ into dist/: a .js and a .d.ts per module, which is what the exports point to.

Exports

Export

Resolves to

Holds

features (.)

dist/index.js

definitions(), AnyConceptKey, AnyValue, Key, AttachmentBase, every container class, and each namespace as a namespace (demo)

features/pools (./pools)

dist/pools.js

every function pool, one namespace each (tools), holding its NAME, UUID and Remote; always declared, it resolves once the package is rendered with PoolRemote (without it, importing features/pools fails)

features/attachment_pools (./attachment_pools)

dist/attachment_pools.js

every attachment function pool, the same way; resolves with AttachmentPoolRemote

features/<unit>/database_attachments (./*/database_attachments)

dist/<unit>/database_attachments.js

a namespace’s attachments over a Database, one class per concept (DatabaseAttachments)

features/<unit>/commit_attachments (./*/commit_attachments)

dist/<unit>/commit_attachments.js

a namespace’s attachments over a Commit state (CommitAttachments)

features/<unit>/fields, features/<unit>/paths

dist/<unit>/fields.js, dist/<unit>/paths.js

a namespace’s field names and paths (Fields, Paths)

features/<dir> (./*)

dist/<dir>/index.js

one namespace (features/demo) or one pool (features/tools) on its own

A pool is reached through features/pools, features/attachment_pools or its own directory, a namespace’s attachments through the module of a usage: the entry module and a namespace’s index.ts belong to Base and cannot import what only the pool and attachment features render.

Using it from a project

The package imports @digitalsubstrate/dsviper, and so does any code that opens a database or builds a runtime value. Both must reach one installation of the runtime: make @digitalsubstrate/dsviper a dependency of the project, in the range the package’s package.json declares as its peer, so that a single copy serves both. Two copies of the native binding cannot share a process — the second one to load stops with a message naming both, and npm ls @digitalsubstrate/dsviper finds them.

import dsviper from "@digitalsubstrate/dsviper";
import { definitions, demo, Vector_of_uint8 } from "features";
import * as database_attachments from "features/demo/database_attachments";

const s = new demo.StructureS({ f_float: 1.5, f_string: "hello" });
const v = new Vector_of_uint8([1, 2, 3]);
const key = demo.ConceptAKey.create();

const db = dsviper.Database.createInMemory();
db.extendDefinitions(definitions());
db.beginTransaction();
database_attachments.ConceptA.properties.set(db, key, new demo.StructureV({ f_string: "stored" }));
db.commit();
console.log(database_attachments.ConceptA.properties.get(db, key).unwrap().f_string);   // stored

const same = demo.StructureS.wrapValue(s.unwrapValue());   // the same value, not a copy
same.f_string = "changed";
console.log(s.f_string, v.length);                          // changed 3

A namespace can also be imported on its own, and the pools from their subpath:

import { StructureS } from "features/demo";
import { tools } from "features/pools";      // rendered with PoolRemote

console.log(new StructureS({ f_string: "from features/demo" }).f_string);
console.log(tools.NAME);                       // Tools

The package’s entry module documents itself — its header names the units, the containers, definitions() and the model’s pools, and shows how a document is stored; the Node page of the generated SDK covers the API.

Type-checking

The sources are written for the settings of the generated tsconfig.json: ES modules resolved the Node way, and default imports of the CommonJS binding. A project that compiles against the package uses the same:

{
    "compilerOptions": {
        "target": "ES2022",
        "module": "NodeNext",
        "moduleResolution": "NodeNext",
        "strict": true,
        "esModuleInterop": true,
        "skipLibCheck": true,
        "outDir": "dist",
        "rootDir": "src",
        "types": ["node"]
    },
    "include": ["src/**/*.ts"]
}

with "type": "module" in its package.json. A bare tsc --strict file.ts uses other defaults and fails on the binding’s import.

Package structure

A model named features with one namespace Demo and one function pool Tools, rendered with DatabaseAttachments, CommitAttachments, PoolRemote and Package:

typescript/
├── package.json             # Package
├── tsconfig.json            # Package
└── src/
    ├── index.ts             # the entry: definitions(), AnyConceptKey, AnyValue, Key, AttachmentBase, containers, demo
    ├── definitions.ts       # definitions(), decoded once from resources.ts
    ├── containers.ts        # one class per container shape the model uses: Vector_of_uint8, …
    ├── resources.ts         # the embedded definitions
    ├── demo/                # one directory per DSM namespace: Demo is demo
    │   ├── index.ts         # re-exports data
    │   ├── data.ts          # structures, enumerations, keys, and the runtime ids
    │   ├── fields.ts        # Fields (required by CommitAttachments): field names
    │   ├── paths.ts         # Paths (required by CommitAttachments): field paths
    │   ├── database_attachments.ts  # DatabaseAttachments: one class per concept, one accessor per attachment
    │   └── commit_attachments.ts    # CommitAttachments: the same, over a Commit state
    ├── pools.ts             # PoolRemote: every function pool, one namespace each
    ├── tools/               # PoolRemote: one directory per function pool
    │   ├── index.ts
    │   └── remote.ts        # NAME, UUID and Remote
    └── _codegen/            # the runtime every generated package carries
        ├── attachment.ts
        ├── container.ts
        ├── proxy.ts
        ├── registry.ts
        └── value.ts

The build adds dist/. TypeScript emits the client side of a pool only, Remote: no host embeds Node, and the host side, Pool, is Python’s and C++’s (see Python and TypeScript).

Distribution

Pack the package into a tarball:

npm pack

This gives features-1.0.0.tgz, with dist, src and package.json (the files entry). Publish it to a private registry, or install the tarball in the consuming project beside the runtime, which the project depends on directly — the package names it as a peer and does not install it:

npm install @digitalsubstrate/dsviper ./features-1.0.0.tgz

Version management

Two numbers meet in package.json:

  • The package’s own version, "version": "1.0.0" as rendered — the version of your model’s surface. Bump it for each release, following semver against what the model declares; the file is regenerated with the project, so set it where the release is made.

  • The runtime range, >=1.2.16 <1.3.0. It is the pack’s: the floor is what the generated code uses (here, the binding whose remote attachment call accepts an AttachmentGetting, which decodes a written xarray from the list of its elements, and which names a write’s interface CommitAttachmentMutating), the ceiling the runtime’s compatibility contract. The header of every generated module names the same range.

Embedded definitions

kibo-project writes the model’s definitions into src/resources.ts, as a base64 string:

// Generated from features.dsm.json by kibo-project 0.1.0. Do not edit by hand.

export const B64_DEFINITIONS = "BBAAAAAAAAAAAR4N7cE1H6dPIDbxcOGmdXC5Cg1/…";

definitions.ts decodes it once, with the binding’s default codec, into the DefinitionsConst that definitions() returns. The string is not compressed — the Python package’s resources.py compresses with zlib; the encoding of each target is declared by the pack’s layout (see Templated Features).